Laya benchmarks
The official Laya README publishes detailed numbers, and a growing set of independent repositories now test those claims. The overall picture: Laya is much faster when run locally and free to self-host, while Jev is more accurate out of the box on most shared tests.
539 projects
laya-model-research
RoyalMilkteaMaster
Research code evaluates Laya as a control model for multi-hop question-answering loops
jev-laya-kev-winnow-qwen-comparison
srvkataria
Compares Qwen, Jev, Laya, and Kev on support-ticket routing
pi0-laya-experiments
must-I-jpg
Compares Laya action selection with baselines in reproducible π0 simulations on ALOHA and LIBERO-Long
laya-demo
vighnesh-chavan
Demonstrates Laya typed decisions in Doom and a custom pygame shooting game
FaradayLayaBenchmark
kurtfi
Benchmarks local models for MCP and agent security with a fail-fast evaluation pipeline
sys1-bench
ugly-custard
Benchmarks Laya and Von decision encoders and provides a practical Laya usage guide
btc-decision-map
Carolyn-ai
A BTC research dashboard compares local Laya decisions with a rule-enhanced Laya path and optional Jev
jev-vs-laya-vs-vega
janagarajsn
Benchmarks Jev, Laya, and Vega on synthetic customer-support ticket triage
sysone-cpu
TayHCode
Runs Laya typed-decision checkpoints on CPU with PyTorch or optional ONNX Runtime
DecideLint
Eileenes
Tests, calibrates and sets abstention thresholds for typed-decision models including Laya
verdict
ashiksharonm
Fine-tunes Laya for multi-check LLM guardrails and evaluates it with open benchmarks
jey_pro_v2
buibui2026
Builds and evaluates a Laya fine-tuning baseline for legal RAG judging
system-one-harness
mev0lent
Compares Laya and other decision models as gates for HTTP response triage
laya-benchmark-poc
BanHun28
Independent benchmark of Laya 0.3.23 on intent, typed-decision and support-ticket tasks
laya-pharma-surveillance
ericmjl
Uses runnable notebooks to evaluate Laya on pharmaceutical surveillance workloads
benchmark-llm-iot-laya-jev
alinembs
Compares local Laya and hosted Jev decisions for controlling MQTT-connected lights and a fan
Laya-LLM-Fine-tuning-and-benchmark-evaluation
sudhi-hub
Fine-tunes Laya for typed decision tasks and evaluates it with benchmark datasets
decision-model-arena
dante01yoon
Compares four decision models, including local Laya, across games and image-based decisions
d1-ane
nathanrchn
Converts and benchmarks decision models on Apple Neural Engine, with a Tetris demo
laya-comparison
felipegoodman
Compares Laya with a lightweight local language model
decision_api_benchmark
guming3d
Benchmarks decision engines, including Laya, on model routing accuracy, latency and cost
layatrade2
fearlessfly786-png
Fine-tunes and evaluates Laya System One on BTCUSDT trading decisions
layatrade3
fearlessfly786-png
Trains and evaluates a two-head Laya trading-decision model
rlcd-tool-selection
VectorSophie
Compares Laya and Jev tool selection with native and decomposed LLM tool-calling approaches
Official numbers
The core README and its BENCHMARKS.md report latency on a Tesla T4 (32.8 ms for one question with laya-multilingual, 39.5 ms with laya) and accuracy across 51 languages. Its Jev figures are third-party published, not measured by the Laya authors.
Independent head-to-heads
- sysone-bench: byte-identical inputs across 751 states. Jev leads on triage, guardrails, moderation, Banking77 and multilingual intent. Laya leads on AG News and MNLI.
- jev-laya-benchmark: 1,470 synthetic items. Jev 92.9% vs Laya 65.3% of judgments correct. For a single question, Laya was 42 ms locally on an M3 Pro and Jev was 136 ms. Jev was faster above 3 to 4 questions per request.
- laya-jev-lab: 40 Chinese support tickets. Jev 78% at 588 ms, Laya 57% at 7.6 ms on an M4 Max. A cascade that escalates below 0.60 confidence matched Jev's accuracy at 1.8x its speed.
- JevBench: a composite board. In v1.4.2, Laya (English, CPU) ranks 41st with a score of 30.25 and Jev 1.13.0 ranks 2nd with 63.29.
- laya-mcp eval: key accuracy 32.5% for Laya vs 98.8% for Jev, with median latency of 26 ms vs 138 ms.
CPU latency
laya-cpu-benchmark measured laya-multilingual in steady state on an Intel i9-14900HX. One question took 50 ms for a 71-token state, 307 ms for 261 tokens, and 1,315 ms for 927 tokens. Batching gave little gain on CPU. Its conclusion: a realistic CPU call is 0.3 to 2 seconds, not 33 ms.
Run your own
git clone https://github.com/glukicov/laya_router && cd laya_router
uv sync --all-extras
uv run laya-router eval run --backend laya --device mps
Or use jevbench (dhruvmehra) to compare Jev, Laya, LLMs and BERT models on public datasets.
Reading these numbers
Most of these suites are small (40 to 1,500 items), synthetic or in one domain, and their authors say so. Latency depends on hardware, state length and question count. The core README's own advice: treat Laya as a fast base to specialise, and fine-tune for accuracy.
More ways to use Laya