Laya on ONNX Runtime

ONNX is how most non-Python Laya runtimes work. An exported graph plus the tokenizer and calibration config is enough to answer typed questions with ONNX Runtime, and several projects publish ready-made exports so you can skip the PyTorch conversion step.

211 projects

laya-mini-cpu-research

VynoDePal

Shares experimental CPU student-model checkpoints and controlled evaluations using Laya as the teacher

1·HF model

ONNXFine-tunes & weights

laya

nishanth-kj

Provides a multilingual typed-decision model and Python package with optional serving and integration extras

1·HF model

PythonONNX

layaForWebTrained

VishalMysore

Provides a quantized ONNX version of Laya typed decisions for browser inference

1·HF model

ONNXBrowser

daylens-laya-onnx

aaronalexS

Provides an ONNX conversion of Laya for local activity labeling in the Daylens desktop app

1·HF model

ONNXNode.js

laya-web-q8

go-vertika

Provides an 8-bit ONNX quantization of Laya for in-browser decision inference

1·HF model

ONNXBrowser

laya-en-fp16

marrviin

FP16 ONNX export of the English Laya checkpoint (no model card)

1·HF model

ONNXFine-tunes & weights

laya-onnx

duration465

Exports Laya’s multilingual checkpoint to ONNX for use with the @receptron/laya Node.js package

1·HF model

ONNXNode.js

sunday-laya-router

rithulkamesh

Fine-tunes Laya for Sunday agent intent routing and distributes the model as an ONNX export

1·HF model

ONNXAgents

laya-mmbert-npu-s25u

Edgsj

Deploys a Laya mmBERT model on the Galaxy S25 Ultra Hexagon NPU

1·HF model

ONNXAndroid

laya-typed-decisions-onnx

ningsuhen

Converts Laya typed decisions to fp16 ONNX for browser inference with WebGPU

1·HF model

ONNXBrowser

system-one

asynq-io

Offers a provider-neutral SDK for typed decisions through hosted and local backends

1·GitHub repo

PythonONNX

laya-micro

osamabinIaggin

Prunes and quantizes Laya for low-memory single-board computer deployment

1·GitHub repo

PythonONNX

laya-ko-decision-onnx

2nugu

Fine-tunes Laya for Korean decisions and provides PyTorch training and ONNX export scripts

1·GitHub repo

PythonONNX

code-oracle

wahyuzero

Verifies code changes with AST checks and Laya-based residual-drift evaluation

1·GitHub repo

PythonONNX

laya-web-demo

mizchi

Runs Laya typed decisions in browser games using ONNX Runtime Web and WebGPU

1·HF Space

ONNXBrowser

layaMOE

vishalmysore

Routes Laya requests to domain-specific expert heads, with a browser demo using ONNX Runtime Web

1·GitHub repo

PythonONNX

laya-pi

trinacria-tech

Runs Laya decision APIs and demos on a Raspberry Pi 4 using ONNX Runtime or PyTorch

1·GitHub repo

PythonONNX

LayaOnnx

dot182

Wraps exported Laya ONNX checkpoints in a C# interface using ONNX Runtime

1·GitHub repo

ONNXRuntimes & ports

laya-skill

StevenLi-phoenix

A Claude Code skill runs, fine-tunes, evaluates, and exports Laya typed-decision models

1·GitHub repo

PythonMLX

Laya-Go-Launcher

Ling0727-ai

Loads Laya models and launches inference services through TensorRT or ONNX Runtime

1·GitHub repo

GoONNX

POK-Agent

Acekorneya

Provides a Windows computer-use agent with JEV or local Laya as its fast action model

1·GitHub repo

RustONNX

SafeSphere.ai

akshayvibe

Detects harassment and stalking locally in social and email services using Laya via ONNX

1·GitHub repo

Node.jsONNX

browser-laya

wexare-ai

Runs the Laya decision model in browsers using WebGPU or ONNX Runtime Web

1·GitHub repo

BrowserONNX

laya-ort

Fruank4

Runs Laya typed decisions on Linux x86_64 using ONNX Runtime

1·GitHub repo

PythonONNX

Use a pre-converted export

inferenceprince/laya-onnx is an fp16 export of the English checkpoint. From its model card:

pip install onnxruntime huggingface_hub tokenizers numpy
hf download inferenceprince/laya-onnx --local-dir laya-onnx

The card includes a full Python example that builds the prompt, runs the graph and applies the fitted temperature. It reports load-to-first-answer of 3 to 5 s against 25 to 35 s for PyTorch on an Intel i5-14400F. The official package also has a laya[onnx] extra with an ONNXAgent class in laya.onnx_agent.

INT8 on CPU

edgejev converts Laya to ONNX, quantizes it to INT8 and serves it:

uv tool install "edgejev[build]"
edgejev build --backend laya --out ./jev-int8

Its README reports 15.6 ms per question for the INT8 multilingual model on a 4 vCPU Xeon (Cascade Lake) and 32.1 ms in fp32, with the model shrinking from 1290 MB to 324 MB. The same table shows AG News accuracy dropping from 92.8% to 91.2%.

ONNX in other languages

Node.js: @receptron/laya, using receptron/laya-onnx.
Java: Laya4j.
Go: laya-onnx (MstyAI).
Ruby: ruby-laya and laya (EduardoGHdez).
Windows GPUs through DirectML: laya-windows.
Local server: ollaya runs ONNX Runtime on CPU and CUDA.
Browser: layaForWeb with ONNX Runtime Web.

Caveats

The graph returns raw logits. Divide by the fitted temperature from rl_agent_config.json before the softmax, or the probabilities mean nothing. Quantized builds drift from PyTorch: the layaForWeb README reports 97.9% top-answer agreement for its int8 build, and it notes that its confidence values are approximate because calibration was fitted on the full-precision model.