Laya for Rust
Rust has several independent Laya runtimes. Some load the upstream safetensors directly with Hugging Face candle, others run ONNX exports or ggmlc GGUF files, and a few wrap the model in a Jev-compatible HTTP server.
77 projects
keel
codejunkie99
A macOS coding workspace uses local Laya or hosted Jev to select task routes
sys1
alvarobartt
Serves Laya decision models through a Rust System One API with CPU, CUDA, and Metal support
ollaya
ollaya-dev
Pulls and serves local decision models, including Laya, through a TypeSafe-compatible API
laya-goish
centillex-labs
Runs Laya GGUF models in a Rust HTTP server for structured text decisions
laya-candle
Trystan-SA
Runs Laya checkpoints locally from Rust using the Candle inference framework
laya-rs
redwolf2019
Provides a Rust HTTP inference runtime and server for multilingual Laya models
laya-rs
apiplant
Reimplements Laya inference and training in Rust using the Candle framework
laya-rust
aovestdipaperino
Implements pure-Rust inference for Laya using the Candle framework
kevala
bvolpato
A Rust WebAssembly engine runs Laya and other decision models locally in browsers
kime
tamnd
A Rust inference engine and HTTP server runs Laya checkpoints across CPU, CUDA, and Apple GPUs
laya-codex
pilotspace
Ranks repository code for Claude Code using the Laya re-ranker, hooks, and an MCP server
laya-pong
aovestdipaperino
Demonstrates Laya-driven paddle decisions in a browser Pong game
laya-candle
b0xtch
Runs Laya safetensors natively in Rust with Candle and CPU, Metal, or CUDA backends
layaRustparser
guptchar
A Rust framework normalizes firewall logs and uses Laya for bounded triage
duckdb-jev-spatial
mraad
Adds DuckDB Jev-style SQL functions backed by a local Laya model for spatial examples
laya-rs
Fanaperana
Reimplements Laya in Rust for zero-shot text classification with verified numerical parity
laya-candle
mannlohchab
Runs English Laya decision-model inference using the Candle framework
laya-gguf
ZeroDegress
A GGUF conversion of Laya for inference with the laya-rust engine
laya-rust
zerodegress
Implements a Rust inference engine for Laya with CUDA, CPU, and MLX backends
laya-ko-decision-onnx
2nugu
Fine-tunes Laya for Korean decisions and provides PyTorch training and ONNX export scripts
laya-rk3588-turingpi-rk1
elepedus
Runs Laya inference on RK3588 Turing Pi RK1 modules through Mali GPU and Rockchip NPU backends
sys1rust
krishhgg
Runs Laya decision models on Apple Silicon with a Rust and MLX server compatible with the upstream API
POK-Agent
Acekorneya
Provides a Windows computer-use agent with JEV or local Laya as its fast action model
RLaya
DaragonTech
A Rust binding runs Laya decisions in-process through LibLayaX
Libraries
laya-rust is a pure-Rust port on candle with no Python, torch or ONNX export step. It runs on CPU, Metal or CUDA:
cargo build --release # CPU, library + laya CLI
cargo build --release --features metal # Apple GPU
cargo build --release --features cuda # NVIDIA
let agent = Agent::from_dir("models/laya-base", Options::default())?;
let response = agent.system_one(&json!("Charged twice for order #4417."), &questions)?;
It also includes a local web console (--features serve) and an exfil_triage example. The README reports it catches 10 of 11 credential-exfiltration commands on a thirty-command held-out set at a 0.5 threshold.
laya-candle is another candle port with a Router, loading checkpoints from the Hub (Rust 1.88+):
[dependencies]
laya-candle = "0.1"
serde_json = "1"
Servers
sys1 is a System One compatible API server on axum and candle, with token-based batching:
cargo install sys1 --features cpu
sys1 --model-id convaiinnovations/laya --dtype auto
It answers at /v1/systemone, and its README reports about 14 ms per query on an NVIDIA RTX Pro 6000.
Also: ollaya, an Ollama-style daemon written in Rust; laya-rs, a Linux CPU server on ONNX Runtime; and laya-goish, which serves ggmlc GGUF files.
Apps
laya-pong is a browser Pong game with the physics in WebAssembly and the decision from native Laya over HTTP, at 18.7 ms p50 on Metal according to its README. keel is a Rust/GPUI coding workspace with a local Laya selector.
Caveats
laya-rust builds f32 only on CPU, because candle has no f16 CPU kernels. That doubles the weights to 1.69 GB resident. laya-rs uses POST /v1/system-one (with a hyphen), so it is not a drop-in Jev endpoint. Its README says so directly.
More ways to use Laya