Laya on ONNX Runtime
ONNX is how most non-Python Laya runtimes work. An exported graph plus the tokenizer and calibration config is enough to answer typed questions with ONNX Runtime, and several projects publish ready-made exports so you can skip the PyTorch conversion step.
76 projects
laya-multilingual-onnx
soyelmismo
Provides CPU-optimized, quantized ONNX checkpoints for multilingual Laya
layaForWeb
VishalMysore
Provides quantized ONNX checkpoints and a demo for running Laya in browsers
laya-onnx
rarha
Provides an ONNX export of the Laya model
laya-onnx
mariojcr
Exports Laya checkpoints to ONNX for inference with ONNX Runtime
cut-laya-onnx
harshpreet931
An 8-bit ONNX conversion runs Laya text classification in browser-based editors
kevin
gauravsaini
Runs a browser automation agent that uses Laya-ONNX to rank candidate actions
system-one
asynq-io
Offers a provider-neutral SDK for typed decisions through hosted and local backends
laya-micro
osamabinIaggin
Prunes and quantizes Laya for low-memory single-board computer deployment
laya-ko-decision-onnx
2nugu
Fine-tunes Laya for Korean decisions and provides PyTorch training and ONNX export scripts
code-oracle
wahyuzero
Verifies code changes with AST checks and Laya-based residual-drift evaluation
jevtpp
wiatrM
A C++20 library for typed model-backed decisions with optional ONNX Runtime and native Laya backends
reflex-hooks-oss
lia210350
A Node provider runs the Laya decision model locally through ONNX for reflex-hooks
laya-onnx-bench
gdelatournelle
Provides a reproducible evaluation set for Laya's ONNX decision backend
meldecision
meldltd
Serves Laya decision models through a GoFiber API using ONNX Runtime
laya-web
nvkudva
Runs a quantized Laya decision model entirely in the browser using ONNX Runtime Web
laya-web-poc
alexander-voronkov
A browser prototype runs a quantized Laya model locally for typed question answering
laya-bot-det
koteitan
Classifies Nostr authors as bots with Laya running in the browser
laya-int8
koteitan
Provides an int8-quantized Laya Multilingual ONNX model split into downloadable parts
laya-claude-plugin
noorforeverone
Provides a Claude Code plugin with local Laya decision tools and an MCP server
laya-onnx-runtime
zenryokukikai
Ports the Laya multilingual decision model to CPU-only ONNX Runtime
rsdecider-inference
House-of-Imaginations
Runs Laya decision models in a Rust server using ONNX Runtime
laya
waqas-duck
Runs the Laya decision model from Node.js and TypeScript using ONNX Runtime
laya-onnx
Geoking2104
Runs Laya decision models with ONNX Runtime and provides export, inference, and benchmark tools
laya-fastapi
remominor
Serves Laya typed decisions through a local FastAPI and ONNX Runtime deployment
Use a pre-converted export
inferenceprince/laya-onnx is an fp16 export of the English checkpoint. From its model card:
pip install onnxruntime huggingface_hub tokenizers numpy
hf download inferenceprince/laya-onnx --local-dir laya-onnx
The card includes a full Python example that builds the prompt, runs the graph and applies the fitted temperature. It reports load-to-first-answer of 3 to 5 s against 25 to 35 s for PyTorch on an Intel i5-14400F. The official package also has a laya[onnx] extra with an ONNXAgent class in laya.onnx_agent.
INT8 on CPU
edgejev converts Laya to ONNX, quantizes it to INT8 and serves it:
uv tool install "edgejev[build]"
edgejev build --backend laya --out ./jev-int8
Its README reports 15.6 ms per question for the INT8 multilingual model on a 4 vCPU Xeon (Cascade Lake) and 32.1 ms in fp32, with the model shrinking from 1290 MB to 324 MB. The same table shows AG News accuracy dropping from 92.8% to 91.2%.
ONNX in other languages
- Node.js: @receptron/laya, using receptron/laya-onnx.
- Java: Laya4j.
- Go: laya-onnx (MstyAI).
- Ruby: ruby-laya and laya (EduardoGHdez).
- Windows GPUs through DirectML: laya-windows.
- Local server: ollaya runs ONNX Runtime on CPU and CUDA.
- Browser: layaForWeb with ONNX Runtime Web.
Caveats
The graph returns raw logits. Divide by the fitted temperature from rl_agent_config.json before the softmax, or the probabilities mean nothing. Quantized builds drift from PyTorch: the layaForWeb README reports 97.9% top-answer agreement for its int8 build, and it notes that its confidence values are approximate because calibration was fitted on the full-precision model.
More ways to use Laya