Laya on ONNX Runtime
ONNX is how most non-Python Laya runtimes work. An exported graph plus the tokenizer and calibration config is enough to answer typed questions with ONNX Runtime, and several projects publish ready-made exports so you can skip the PyTorch conversion step.
211 projects
laya-scam-detector
LiChace
Detects Chinese and English scam messages with a fine-tuned Laya model in the browser
QLaya
saipy10
An experiment distills and quantizes Laya decision models for faster on-device inference
laya-js
desplega-ai
A TypeScript runtime and Hono server load ONNX Laya checkpoints and answer typed decision questions
laya-browser-guard
pateti-hub
A privacy-first browser extension analyzes website security using optional local Laya ONNX decisions
qa-triage-demo
lucad87
Uses a fine-tuned Laya decision model to triage Playwright failures and guide verified fixes or bug reports
codepilot
GorbunovS
Indexes code and provides MCP search tools with Laya-powered reranking
gmr_golpe_pix
GumerJr
Fine-tunes Laya to detect Pix payment scams and evaluates local CPU inference
laya-mcp
kennedyshead
Serves local Laya routing decisions to coding agents through an MCP server
Laya_try
kameusagiyahoo
Tests Laya in browsers and on iPhone, including ONNX inference and a runnable Google ADK workflow
test-laya
tom-scott-dev
Runs a 2D town game where NPC behavior is driven by local Laya decisions
laya-test
supermy
Runs quantized Laya inference locally on Android and Termux using native ONNX Runtime
arka-laya-model
xansa
Provides ONNX files for a Laya decision model fine-tuned for a chatbot knowledge base
laya-webgpu
game-design-projects
Runs Laya typed decisions in the browser using WebGPU and ONNX Runtime Web
laya-demo
mkhante
Provides a standalone Go server and client that embed native Laya inference
sima-laya
dotimothy
Compiles and runs Laya decision models on SiMa.ai Modalix hardware
laya-webgpu
TakedaShinken
Runs Laya typed-decision models in browsers with WebGPU and ONNX Runtime Web
laya-on-rk3588-jetson
suharvest
Converts and benchmarks Laya for deployment on Rockchip RK3588 and NVIDIA Jetson Orin
@web4kit/decider-laya
shadyfast8
Runs the open Laya System One model offline in-process using ONNX Runtime
@heryox/ped
heryox
Runs Laya System 1 decision models from Node.js and TypeScript using ONNX Runtime
laya
mpjivdc
Provides a multilingual, non-autoregressive model for typed text decisions with Python and TypeScript interfaces
jev-laya-dsh
alexdoandev
Adds a local Laya decision layer for routing prompts and gating risky agent actions
SparkX_EA
mukhesh88
Uses Laya with ONNX Runtime to score market conditions and support automated trading decisions
sysone
inovacc
A Rust binary runs Laya through ONNX Runtime and serves the System One HTTP protocol
sentrygate
mnjkshrm
A local Node.js guard that uses Laya to detect prompt injections and redact PII
Use a pre-converted export
inferenceprince/laya-onnx is an fp16 export of the English checkpoint. From its model card:
pip install onnxruntime huggingface_hub tokenizers numpy
hf download inferenceprince/laya-onnx --local-dir laya-onnx
The card includes a full Python example that builds the prompt, runs the graph and applies the fitted temperature. It reports load-to-first-answer of 3 to 5 s against 25 to 35 s for PyTorch on an Intel i5-14400F. The official package also has a laya[onnx] extra with an ONNXAgent class in laya.onnx_agent.
INT8 on CPU
edgejev converts Laya to ONNX, quantizes it to INT8 and serves it:
uv tool install "edgejev[build]"
edgejev build --backend laya --out ./jev-int8
Its README reports 15.6 ms per question for the INT8 multilingual model on a 4 vCPU Xeon (Cascade Lake) and 32.1 ms in fp32, with the model shrinking from 1290 MB to 324 MB. The same table shows AG News accuracy dropping from 92.8% to 91.2%.
ONNX in other languages
- Node.js: @receptron/laya, using receptron/laya-onnx.
- Java: Laya4j.
- Go: laya-onnx (MstyAI).
- Ruby: ruby-laya and laya (EduardoGHdez).
- Windows GPUs through DirectML: laya-windows.
- Local server: ollaya runs ONNX Runtime on CPU and CUDA.
- Browser: layaForWeb with ONNX Runtime Web.
Caveats
The graph returns raw logits. Divide by the fitted temperature from rl_agent_config.json before the softmax, or the probabilities mean nothing. Quantized builds drift from PyTorch: the layaForWeb README reports 97.9% top-answer agreement for its int8 build, and it notes that its confidence values are approximate because calibration was fitted on the full-precision model.
More ways to use Laya