Laya on ONNX Runtime
ONNX is how most non-Python Laya runtimes work. An exported graph plus the tokenizer and calibration config is enough to answer typed questions with ONNX Runtime, and several projects publish ready-made exports so you can skip the PyTorch conversion step.
211 projects
laya-flow-builder
myrest
A visual decision-tree editor and HTTP API run multilingual Laya ONNX models
laya-directml-reshape-fix
xKOKOKOx
Reproduces and verifies an ONNX Runtime DirectML Reshape workaround on a Laya multilingual export
system-one-bench
DarkStark9000
Measures Laya inference latency, execution phases, GPU kernels and answer parity across backends
laya-strix
alytaphoenix
Runs fine-tuned Laya decision inference on an AMD XDNA2 NPU and serves it through the Jev protocol
laya-bangla-playground
algolyzer
Runs Bangla and Banglish Laya decisions locally in a browser
oracle-laya-system-one
loiclefevre
Runs the Laya System One ONNX model in Oracle AI Database using PL/SQL
Snapshot-Laya
Shisetsu-Code
Provides a local interface for reviewing Laya classifications of game controls using ONNX and DirectML
laya-showcase-android
Josh-Reimer
Runs the Laya typed-decision model locally in an Android showcase app
playwright-system1-triage
kint4
Classifies Playwright failures and benchmarks calibrated CI decisions using local Laya and the Jev API
runlaya
Khip01
Serves Laya ONNX models through an API for choice, score, and yes-or-no predictions
laya-weights
SemanticaBook
Hosts quantized ONNX weights for in-browser inference with the multilingual Laya model
alvaress
GitHub Actions
Installs an OpenCode agent harness that makes local decisions with Laya and validates code changes
semanticabook
eduardorrs
Converts PDFs to EPUB and Markdown, using optional local Laya classification to recover document structure
laya-gomoku-k3
TechyLX
Runs Laya on a SpacemiT K3 board to power a browser-based Gomoku game
laya-notes
itsdula
Tags local Markdown notes by running the Laya ONNX model on the user's computer
owl_laya
wndgur2
Fine-tunes and evaluates Laya for on-device classification of Owl timeline segments
laya-zcode-stack
Ydjin1984
Runs a local Laya decision engine and ZCode plugin with MCP tools for coding-agent workflows
layaForWeb
realizer-1966
Runs a quantized Laya decision model in the browser with ONNX Runtime Web
laya-web-ko
realizer-1966
Runs a quantized Laya decision model in the browser with ONNX Runtime Web
Use a pre-converted export
inferenceprince/laya-onnx is an fp16 export of the English checkpoint. From its model card:
pip install onnxruntime huggingface_hub tokenizers numpy
hf download inferenceprince/laya-onnx --local-dir laya-onnx
The card includes a full Python example that builds the prompt, runs the graph and applies the fitted temperature. It reports load-to-first-answer of 3 to 5 s against 25 to 35 s for PyTorch on an Intel i5-14400F. The official package also has a laya[onnx] extra with an ONNXAgent class in laya.onnx_agent.
INT8 on CPU
edgejev converts Laya to ONNX, quantizes it to INT8 and serves it:
uv tool install "edgejev[build]"
edgejev build --backend laya --out ./jev-int8
Its README reports 15.6 ms per question for the INT8 multilingual model on a 4 vCPU Xeon (Cascade Lake) and 32.1 ms in fp32, with the model shrinking from 1290 MB to 324 MB. The same table shows AG News accuracy dropping from 92.8% to 91.2%.
ONNX in other languages
- Node.js: @receptron/laya, using receptron/laya-onnx.
- Java: Laya4j.
- Go: laya-onnx (MstyAI).
- Ruby: ruby-laya and laya (EduardoGHdez).
- Windows GPUs through DirectML: laya-windows.
- Local server: ollaya runs ONNX Runtime on CPU and CUDA.
- Browser: layaForWeb with ONNX Runtime Web.
Caveats
The graph returns raw logits. Divide by the fitted temperature from rl_agent_config.json before the softmax, or the probabilities mean nothing. Quantized builds drift from PyTorch: the layaForWeb README reports 97.9% top-answer agreement for its int8 build, and it notes that its confidence values are approximate because calibration was fitted on the full-precision model.
More ways to use Laya