Laya on ONNX Runtime
ONNX is how most non-Python Laya runtimes work. An exported graph plus the tokenizer and calibration config is enough to answer typed questions with ONNX Runtime, and several projects publish ready-made exports so you can skip the PyTorch conversion step.
211 projects
laya
mes1234
Provides a Python implementation of the multilingual Laya typed-decision model
Laya-Render
gtang3646
Deploys a quantized Laya decision model as an ONNX-powered HTTP API on Render
jev_flappy_bird
iamdgarcia
Controls Flappy Bird by using Laya ONNX inference to choose whether to flap
LayaSample
altafmehmood
Analyzes hotel guest feedback with the Laya decision model through ONNX Runtime
trace-sec
vk.amogh
Security checks for coding agents with SecureBERT and a fine-tuned Laya ONNX router
laya-local-benchmark
saksham-personal
Benchmarks and packages offline CPU Laya decision workflows for Windows VDI environments
laya-spike
naraaruna6302
Provides a multilingual typed-decision model with Python, serving, MCP, and ONNX options
layalike
mage-uwu
Provides a multilingual System 1 decision engine with typed outputs and integrations for serving and agent frameworks
Kirk-laya
Benjamacs1
Runs an ONNX fine-tune of Laya locally to classify Minecraft chat lines
Laya
muhammadbilalcp
Provides a multilingual, non-autoregressive decision engine with Python, server, and integration packages
Laya-
muhammadbilalcp
Provides a multilingual, non-autoregressive decision engine with Python, server, and integration packages
laya-utils
yazeedallabadi24
Runs the Laya decision model as an INT8 ONNX CLI and Docker utility
megathon_arihant
arihant1602
Builds an on-device voice incident assistant with a fine-tuned Laya classifier and speech models
sysone-cpu
TayHCode
Runs Laya typed-decision checkpoints on CPU with PyTorch or optional ONNX Runtime
local-ai-agent
DanielGodriNeto
Uses a local Laya classifier to choose presets in a voice-controlled desktop agent
laya
jetmiyoung
Provides a multilingual typed-decision engine with Python and integrations for serving and agent frameworks
laya-service
FSSCoding
Serves GPU-accelerated Laya inference over HTTP with caching and a CLI
laya-rs
ParkSnoopy
Provides a Rust/Burn runtime for Laya inference with Python fine-tuning and ONNX export
burn-laya
ParkSnoopy
Runs typed local inference for Laya on Rust/Burn and provides fine-tuning and ONNX export tools
Laya
areeb26
Provides a multilingual typed-decision model with Python, server, MCP, and framework integrations
laya-siro
Cihaimasuiro
Runs Laya decisions in a low-memory Windows automation engine using Rust and ONNX Runtime
terraforge-packs
thomassanders3888-svg
Publishes ONNX NPC model packs derived from the Laya base model
Decision
l0d0v1c
Runs a French decision oracle locally in the browser using a quantized ONNX Laya model
laya-mcp
Hypokoto
Runs the Laya ONNX decision model locally and exposes policy and code-analysis tools through MCP
Use a pre-converted export
inferenceprince/laya-onnx is an fp16 export of the English checkpoint. From its model card:
pip install onnxruntime huggingface_hub tokenizers numpy
hf download inferenceprince/laya-onnx --local-dir laya-onnx
The card includes a full Python example that builds the prompt, runs the graph and applies the fitted temperature. It reports load-to-first-answer of 3 to 5 s against 25 to 35 s for PyTorch on an Intel i5-14400F. The official package also has a laya[onnx] extra with an ONNXAgent class in laya.onnx_agent.
INT8 on CPU
edgejev converts Laya to ONNX, quantizes it to INT8 and serves it:
uv tool install "edgejev[build]"
edgejev build --backend laya --out ./jev-int8
Its README reports 15.6 ms per question for the INT8 multilingual model on a 4 vCPU Xeon (Cascade Lake) and 32.1 ms in fp32, with the model shrinking from 1290 MB to 324 MB. The same table shows AG News accuracy dropping from 92.8% to 91.2%.
ONNX in other languages
- Node.js: @receptron/laya, using receptron/laya-onnx.
- Java: Laya4j.
- Go: laya-onnx (MstyAI).
- Ruby: ruby-laya and laya (EduardoGHdez).
- Windows GPUs through DirectML: laya-windows.
- Local server: ollaya runs ONNX Runtime on CPU and CUDA.
- Browser: layaForWeb with ONNX Runtime Web.
Caveats
The graph returns raw logits. Divide by the fitted temperature from rl_agent_config.json before the softmax, or the probabilities mean nothing. Quantized builds drift from PyTorch: the layaForWeb README reports 97.9% top-answer agreement for its int8 build, and it notes that its confidence values are approximate because calibration was fitted on the full-precision model.
More ways to use Laya