Hosted Laya: $1 free, no card
laya.tools
Submit

Laya on ONNX Runtime

ONNX is how most non-Python Laya runtimes work. An exported graph plus the tokenizer and calibration config is enough to answer typed questions with ONNX Runtime, and several projects publish ready-made exports so you can skip the PyTorch conversion step.

211 projects

T

laya-typed-decisions-onnx

ti3x-m

Exports the Laya typed-decisions checkpoint to ONNX for CPU and browser inference

247·HF model
K

open-jev-laya-multilingual-onnx

killkli

A browser-ready ONNX export of Laya multilingual for typed decisions

124·HF model
T

laya-cybersec

TextCortex

A multilingual Laya fine-tune detects prompt injection and data exfiltration, with an included ONNX build

59·HF model
S

laya-multilingual-onnx

soyelmismo

Provides CPU-optimized, quantized ONNX checkpoints for multilingual Laya

2·HF model
M

laya-onnx

mariojcr

Exports Laya checkpoints to ONNX for inference with ONNX Runtime

2·HF model
E

laya-multilingual-onnx

Emerald7664

CPU-optimized ONNX checkpoints for multilingual Laya decision inference

2·HF model
2

laya-ko

2nugu

Fine-tunes Laya for Korean and evaluates it on classification, relation extraction, inference, and similarity tasks

2·HF model
D

laya-onnx

desplega

Exports three Laya decision checkpoints to ONNX for use from Node or Bun

2·HF model
N

laya-web-q8

nvkudva

Runs an 8-bit ONNX conversion of Laya locally in web browsers

2·HF model
S

recmeets-laya-words

somniusx

A Greek- and English-language Laya fine-tune identifies likely speech-recognition word errors

2·HF model
N

laya-onnx-q8

nvkudva

Runs an 8-bit ONNX export of Laya in the browser for local text classification and decisions

2·HF model
N

laya-home-fine-tune-en

nohaah

Fine-tunes Laya to map English smart-home commands to device actions for an offline assistant

2·HF model

laya-portable

MatteoGauthier

Exports Laya to ONNX and provides JavaScript runtimes for Node.js and browser inference

2·GitHub repo

laya-snake-k3

JiongLab

Runs an offline Laya-powered Snake game on SpaceMIT K3 using ONNX and AI Core

2·GitHub repo

laya.axera

AXERA-TECH

Exports and calibrates Laya models for AXERA NPU deployment

2·GitHub repo

laya-decide

woaiys3

Runs a quantized Laya decision model offline on Android to score and rank user-provided options

2·GitHub repo
I

laya-onnx-int8

inferenceprince

Provides an int8 weight-only ONNX export of Laya for ONNX Runtime

199·HF model
T

laya-onnx

tozp

Hosts ONNX-converted English Laya model weights in FP32, FP16, and INT8 formats

34·HF model
D

laya

Dariusdup

Provides a multilingual non-autoregressive model for calibrated typed decisions over text

18·HF model
C

laya-mobile

charioteer

Converts the Laya model to Core ML and ONNX for on-device use on iOS and Android

5·HF model
V

layaForWeb

VishalMysore

Provides quantized ONNX checkpoints and a demo for running Laya in browsers

1·HF model
R

laya-onnx

rarha

Provides an ONNX export of the Laya model

1·HF model
H

cut-laya-onnx

harshpreet931

An 8-bit ONNX conversion runs Laya text classification in browser-based editors

1·HF model
S

laya-multilingual-onnx-int8

Sharjeelbaig

Provides an INT8 ONNX quantization of Laya multilingual for CPU inference in JavaScript and TypeScript

1·HF model

Use a pre-converted export

inferenceprince/laya-onnx is an fp16 export of the English checkpoint. From its model card:

pip install onnxruntime huggingface_hub tokenizers numpy
hf download inferenceprince/laya-onnx --local-dir laya-onnx

The card includes a full Python example that builds the prompt, runs the graph and applies the fitted temperature. It reports load-to-first-answer of 3 to 5 s against 25 to 35 s for PyTorch on an Intel i5-14400F. The official package also has a laya[onnx] extra with an ONNXAgent class in laya.onnx_agent.

INT8 on CPU

edgejev converts Laya to ONNX, quantizes it to INT8 and serves it:

uv tool install "edgejev[build]"
edgejev build --backend laya --out ./jev-int8

Its README reports 15.6 ms per question for the INT8 multilingual model on a 4 vCPU Xeon (Cascade Lake) and 32.1 ms in fp32, with the model shrinking from 1290 MB to 324 MB. The same table shows AG News accuracy dropping from 92.8% to 91.2%.

ONNX in other languages

Caveats

The graph returns raw logits. Divide by the fitted temperature from rl_agent_config.json before the softmax, or the probabilities mean nothing. Quantized builds drift from PyTorch: the layaForWeb README reports 97.9% top-answer agreement for its int8 build, and it notes that its confidence values are approximate because calibration was fitted on the full-precision model.

More ways to use Laya

Run Laya on your MacLaya on MLXLaya on Core MLUse Laya from Node.jsLaya for PythonLaya on OpenVINOLaya for RustLaya in C++ and GGUFRun Laya in the browserLaya for Go, Java, Elixir, Ruby and ZigContent moderation with LayaText classification with LayaRouting with LayaLaya for agentsLaya in coding toolsLaya for search and RAGGames played by LayaLaya benchmarksSelf-host a Laya API serverModernBERT and mmBERT in LayaZero-shot classification with LayaLaya alternativesHosted Laya API