Laya for Rust
Rust has several independent Laya runtimes. Some load the upstream safetensors directly with Hugging Face candle, others run ONNX exports or ggmlc GGUF files, and a few wrap the model in a Jev-compatible HTTP server.
0 projects
Libraries
laya-rust is a pure-Rust port on candle with no Python, torch or ONNX export step. It runs on CPU, Metal or CUDA:
cargo build --release # CPU, library + laya CLI
cargo build --release --features metal # Apple GPU
cargo build --release --features cuda # NVIDIA
let agent = Agent::from_dir("models/laya-base", Options::default())?;
let response = agent.system_one(&json!("Charged twice for order #4417."), &questions)?;
It also includes a local web console (--features serve) and an exfil_triage example. The README reports it catches 10 of 11 credential-exfiltration commands on a thirty-command held-out set at a 0.5 threshold.
laya-candle is another candle port with a Router, loading checkpoints from the Hub (Rust 1.88+):
[dependencies]
laya-candle = "0.1"
serde_json = "1"
Servers
sys1 is a System One compatible API server on axum and candle, with token-based batching:
cargo install sys1 --features cpu
sys1 --model-id convaiinnovations/laya --dtype auto
It answers at /v1/systemone, and its README reports about 14 ms per query on an NVIDIA RTX Pro 6000.
Also: ollaya, an Ollama-style daemon written in Rust; laya-rs, a Linux CPU server on ONNX Runtime; and laya-goish, which serves ggmlc GGUF files.
Apps
laya-pong is a browser Pong game with the physics in WebAssembly and the decision from native Laya over HTTP, at 18.7 ms p50 on Metal according to its README. keel is a Rust/GPUI coding workspace with a local Laya selector.
Caveats
laya-rust builds f32 only on CPU, because candle has no f16 CPU kernels. That doubles the weights to 1.69 GB resident. laya-rs uses POST /v1/system-one (with a hyphen), so it is not a drop-in Jev endpoint. Its README says so directly.
More ways to use Laya