Laya benchmarks
The official Laya README publishes detailed numbers, and a growing set of independent repositories now test those claims. The overall picture: Laya is much faster when run locally and free to self-host, while Jev is more accurate out of the box on most shared tests.
228 projects
laya-lab
Parswanadh
Coordinates long-context decision-engine research for the open Laya model
laya-indic-bench
jay123anta
Evaluates Laya decision models on Hindi and Assamese
jev-laya-tetris
HarryReidx
Benchmarks TypeSafe Jev against local Laya in a competitive Tetris duel
jev-n-laya
jcezardasilva
Compares Jev, Laya, Von, and Gemma on typed-decision tasks
laya-lk-bench
mithilyr
Stress-tests Laya decision-model performance on Sinhala and Tamil offensive-language data
laya-vs-jev
alilibx
Benchmarks open-weight Laya models against TypeSafe Jev on labelled datasets
laya-cuda-bench
bhushankinge
Benchmarks Laya inference throughput, latency and cost across NVIDIA GPUs and serving backends
laya-cn-study
Adkid-Zephyr
Collects Chinese-first post-training experiments, datasets, and evaluations for Laya
laya-vs-jev-traffic
ameeetgaikwad
Compares local Laya and cloud Jev in a real-time traffic-control simulation
jev-laya-classification-bench
bhushankinge
Benchmarks Jev, Laya, and Qwen on classification of federal IT solicitations
von-laya-jev-paint-compare
zhangyunting123
Creates side-by-side paintings from typed decisions by VON, Laya, and Jev
laya-coreml-vs-jev-benchmark
sallout
Compares Laya and Jev on zero-shot intent classification benchmarks
laya-v2-agent-routing
mdad-elec
Fine-tunes a Laya System-One router to select the price tier for each agent turn
cbjev
tomek7667
Implements a Jev-compatible decision model and benchmarks it directly against Laya
korean-decision-benchmark
jkf87
Benchmarks SemIf, Decider, Laya, and Jev on Korean hate-speech classification
synthetic-ehr-decision-evals
MadCodeTX
Benchmarks decision models on synthetic EHR administrative-routing tasks
rlcd_x_1b-4b_instruct
mrgonzales-dev
Tests RLCD models including Laya and explores their use with 2B–4B models
gyra
Gowtham-R-2002
Fine-tunes Laya for fast coding-agent decisions and evaluates it against labeled tests
jev-vs-llms
ivanviragine
Compares Jev, Laya, other open models, and chat LLMs on loaded questions in two languages
wizden-moonlander
WizdenOrg
A lunar lander game compares Laya and Jev as typed-decision flight controllers
small-decision-model-cross-domain-degradation
SnackTerminator
A controlled study evaluates cross-domain and cross-lingual transfer in Jev-class decision models
decision-models-under-pressure
gazelle93
A benchmark compares seven decision models as candidate lists grow, options reorder and distractors become harder
ai-experiments
nadeem4
A collection of documented AI experiments reports protocols and results for model evaluations
laya-playground
shwetankg07
Three browser games measure Laya’s decisions against exact ground truth
Official numbers
The core README and its BENCHMARKS.md report latency on a Tesla T4 (32.8 ms for one question with laya-multilingual, 39.5 ms with laya) and accuracy across 51 languages. Its Jev figures are third-party published, not measured by the Laya authors.
Independent head-to-heads
- sysone-bench: byte-identical inputs across 751 states. Jev leads on triage, guardrails, moderation, Banking77 and multilingual intent. Laya leads on AG News and MNLI.
- jev-laya-benchmark: 1,470 synthetic items. Jev 92.9% vs Laya 65.3% of judgments correct. For a single question, Laya was 42 ms locally on an M3 Pro and Jev was 136 ms. Jev was faster above 3 to 4 questions per request.
- laya-jev-lab: 40 Chinese support tickets. Jev 78% at 588 ms, Laya 57% at 7.6 ms on an M4 Max. A cascade that escalates below 0.60 confidence matched Jev's accuracy at 1.8x its speed.
- JevBench: a composite board. In v1.4.2, Laya (English, CPU) ranks 41st with a score of 30.25 and Jev 1.13.0 ranks 2nd with 63.29.
- laya-mcp eval: key accuracy 32.5% for Laya vs 98.8% for Jev, with median latency of 26 ms vs 138 ms.
CPU latency
laya-cpu-benchmark measured laya-multilingual in steady state on an Intel i9-14900HX. One question took 50 ms for a 71-token state, 307 ms for 261 tokens, and 1,315 ms for 927 tokens. Batching gave little gain on CPU. Its conclusion: a realistic CPU call is 0.3 to 2 seconds, not 33 ms.
Run your own
git clone https://github.com/glukicov/laya_router && cd laya_router
uv sync --all-extras
uv run laya-router eval run --backend laya --device mps
Or use jevbench (dhruvmehra) to compare Jev, Laya, LLMs and BERT models on public datasets.
Reading these numbers
Most of these suites are small (40 to 1,500 items), synthetic or in one domain, and their authors say so. Latency depends on hardware, state length and question count. The core README's own advice: treat Laya as a fast base to specialise, and fine-tune for accuracy.
More ways to use Laya