Laya benchmarks
The official Laya README publishes detailed numbers, and a growing set of independent repositories now test those claims. The overall picture: Laya is much faster when run locally and free to self-host, while Jev is more accurate out of the box on most shared tests.
539 projects
laya-snake
zxrneu
Builds an autonomous Snake game agent using Laya decisions and BFS pathfinding
jev-vs-laya
DDnim
Benchmarks Jev and Laya on typed decisions for reviewing SQL statements
doom-war
rythmn1111
Compares local MLX Laya and hosted Jev as decision-makers in a Doom deathmatch
laya-vs-jev
zaferayan
Benchmarks multilingual Laya against hosted Jev across 900 cases, three tasks, and six languages
laya-cuda
Alexw1111
Provides a lightweight CUDA inference library for Laya
laya-cpu-benchmark
taiheqi718-art
Measures steady-state CPU latency for the multilingual Laya model
decision-models-under-pressure
gazelle93
A benchmark compares seven decision models as candidate lists grow, options reorder and distractors become harder
laya-mlx-zh
ZLHAOOO
Provides Chinese fine-tuned Laya weights, training data, and evaluations for Apple Silicon MLX
pacman-arena
hakan-gecili
Compares Laya with other local decision models playing Pac-Man under identical conditions
systemone-lab
baldellimtt
Demonstrates typed System One decisions with local Laya, a playground, routing examples, and evaluation tests
fin-laya-mlx-6-rounds
ya9919
A 0.42B-parameter financial decision model trained on CHFS, CGSS, CFPS and CLDS data
laya-typed-decisions-421m-coreml-ane
NMTZ
Provides fixed-shape Core ML artifacts converted from the 421M Laya typed-decisions checkpoint
laya-typed-decisions
akshatbindal
Fine-tunes Laya on the typed-decisions benchmark and compares results with Jev
laya-mini-cpu-research
VynoDePal
Shares experimental CPU student-model checkpoints and controlled evaluations using Laya as the teacher
laya-typed-decisions-turkish-mmlu-10k
canbingol
A Laya checkpoint fine-tuned on 10,000 Turkish MMLU typed-decision examples
laya-browser-agent
knpatil
Fine-tunes a ModernBERT-based Laya model for autonomous browser control
laya-typed-decisions
yashchouhann
Fine-tunes Laya on a typed-decisions benchmark and reports comparative results
laya-fullft-typed-decisions
jjshin
Fine-tunes Laya with pure PyTorch RLCD for typed decisions and jailbreak detection
laya-typed-decisions
huangxn
A fine-tuned Laya model evaluated on the Typed-Decisions benchmark
fin-laya-mlx-3-rounds
ya9919
A 0.42B-parameter financial decision model trained on CHFS and CGSS data
decision-model-playground
tedliou
A local web playground comparing Laya and Jev for recommending articles
jev-vs-open-decision-models
elcronos
Benchmarks Jev against Laya and PrismNLI on zero-shot text classification tasks
JevCoreML
GodModeAI2025
Provides Core ML decision models, a Swift package, an HTTP server, and a demo app
laya
xosi
Runs Laya typed-decision inference locally on Apple Silicon with MLX
Official numbers
The core README and its BENCHMARKS.md report latency on a Tesla T4 (32.8 ms for one question with laya-multilingual, 39.5 ms with laya) and accuracy across 51 languages. Its Jev figures are third-party published, not measured by the Laya authors.
Independent head-to-heads
- sysone-bench: byte-identical inputs across 751 states. Jev leads on triage, guardrails, moderation, Banking77 and multilingual intent. Laya leads on AG News and MNLI.
- jev-laya-benchmark: 1,470 synthetic items. Jev 92.9% vs Laya 65.3% of judgments correct. For a single question, Laya was 42 ms locally on an M3 Pro and Jev was 136 ms. Jev was faster above 3 to 4 questions per request.
- laya-jev-lab: 40 Chinese support tickets. Jev 78% at 588 ms, Laya 57% at 7.6 ms on an M4 Max. A cascade that escalates below 0.60 confidence matched Jev's accuracy at 1.8x its speed.
- JevBench: a composite board. In v1.4.2, Laya (English, CPU) ranks 41st with a score of 30.25 and Jev 1.13.0 ranks 2nd with 63.29.
- laya-mcp eval: key accuracy 32.5% for Laya vs 98.8% for Jev, with median latency of 26 ms vs 138 ms.
CPU latency
laya-cpu-benchmark measured laya-multilingual in steady state on an Intel i9-14900HX. One question took 50 ms for a 71-token state, 307 ms for 261 tokens, and 1,315 ms for 927 tokens. Batching gave little gain on CPU. Its conclusion: a realistic CPU call is 0.3 to 2 seconds, not 33 ms.
Run your own
git clone https://github.com/glukicov/laya_router && cd laya_router
uv sync --all-extras
uv run laya-router eval run --backend laya --device mps
Or use jevbench (dhruvmehra) to compare Jev, Laya, LLMs and BERT models on public datasets.
Reading these numbers
Most of these suites are small (40 to 1,500 items), synthetic or in one domain, and their authors say so. Latency depends on hardware, state length and question count. The core README's own advice: treat Laya as a fast base to specialise, and fine-tune for accuracy.
More ways to use Laya