Laya benchmarks
The official Laya README publishes detailed numbers, and a growing set of independent repositories now test those claims. The overall picture: Laya is much faster when run locally and free to self-host, while Jev is more accurate out of the box on most shared tests.
539 projects
laya-mlx-mirror-227
Shijiuwei
Runs Laya inference locally on Apple Silicon using MLX, with a terminal Snake demo
clasificacion-tipada
tux22
Reproduces benchmarks comparing Jev, Laya, and local LLM classifiers on public datasets
laya-reasoning-demo
Prady0909
Demonstrates a recurrent reasoning extension of Laya with adaptive halting
Laya
mayataji3-star
Evaluates Laya on synthetic support tickets and preserves labels, predictions, code, and results
email_Jev_Laya
APRESIA
Benchmarks Laya, Jev and Qwen on email classification, including a Laya-to-Qwen cascade
layax
ohboyftw
Adds token-level option scoring and an abstention gate to Laya decision models
laya-swe
Mukul-svg
Uses Laya as a fast memory controller for autonomous coding agents and evaluates it on software tasks
laya-fintiment
shanaka95
Fine-tunes Laya for three-class financial sentiment and provides the training code, evaluations, and logs
laya-onnx
D-Ajay-Kumar
Converts Laya weights to ONNX and benchmarks the converted model
laya-chess
Srimanvarma001
A local chess app compares Laya-suggested moves with a one-ply material check
laya-local-setup
jigs10
Provides commands and examples for running Laya locally and comparing its decisions with Jev
laya-client-example
angelxmoreno
Uses a Laya HTTP client to triage GitHub issues and compare category predictions with repository labels
jev-vs-laya
soummyaanon
A web-based arena compares Jev and Laya classifiers on shared tasks and reports performance metrics
laya_demo_test
j75689
A Snake demo compares Python and Rust Laya runners with ONNX Runtime and Core ML backends
app-intents-laya-study
junshanxu
Studies macOS App Intents and CLI agents with and without a local Laya next-action gate
jev-laya-cube-arena
189-sketch
Compares Jev and local Laya on Rubik’s cube next-move decisions
laya-alcohol-visit-classifier
Aurumdev952
Fine-tunes and benchmarks Laya for alcohol attribution in hospital visit notes
pi-context-gc
Lausi-h2ai
Archives old tool outputs for Pi coding agents and evaluates retention policies with local judges
jev-decomposition-tr
betulsimsek
Tests question decomposition for typed decisions by comparing Jev, Laya, and a local decoder
impulse
roby-avo
Creates a physics arena to play against decision models and inspect their choices
system1-judge-eval
wanqili857-byte
A reproducible evaluation of Laya and other System-1 models on out-of-domain agent decisions
system-one-playground
goodboybeau
A macOS playground comparing Laya, Decider, Kev and Jev on accuracy, calibration and performance
layaUI-HK
whksoup
Runs a local Laya playground, games, benchmark, and agent skill
jev-spaceshooter-demo-laya
jstdlee
Steers a retro space shooter with Laya decision models through a local API
Official numbers
The core README and its BENCHMARKS.md report latency on a Tesla T4 (32.8 ms for one question with laya-multilingual, 39.5 ms with laya) and accuracy across 51 languages. Its Jev figures are third-party published, not measured by the Laya authors.
Independent head-to-heads
- sysone-bench: byte-identical inputs across 751 states. Jev leads on triage, guardrails, moderation, Banking77 and multilingual intent. Laya leads on AG News and MNLI.
- jev-laya-benchmark: 1,470 synthetic items. Jev 92.9% vs Laya 65.3% of judgments correct. For a single question, Laya was 42 ms locally on an M3 Pro and Jev was 136 ms. Jev was faster above 3 to 4 questions per request.
- laya-jev-lab: 40 Chinese support tickets. Jev 78% at 588 ms, Laya 57% at 7.6 ms on an M4 Max. A cascade that escalates below 0.60 confidence matched Jev's accuracy at 1.8x its speed.
- JevBench: a composite board. In v1.4.2, Laya (English, CPU) ranks 41st with a score of 30.25 and Jev 1.13.0 ranks 2nd with 63.29.
- laya-mcp eval: key accuracy 32.5% for Laya vs 98.8% for Jev, with median latency of 26 ms vs 138 ms.
CPU latency
laya-cpu-benchmark measured laya-multilingual in steady state on an Intel i9-14900HX. One question took 50 ms for a 71-token state, 307 ms for 261 tokens, and 1,315 ms for 927 tokens. Batching gave little gain on CPU. Its conclusion: a realistic CPU call is 0.3 to 2 seconds, not 33 ms.
Run your own
git clone https://github.com/glukicov/laya_router && cd laya_router
uv sync --all-extras
uv run laya-router eval run --backend laya --device mps
Or use jevbench (dhruvmehra) to compare Jev, Laya, LLMs and BERT models on public datasets.
Reading these numbers
Most of these suites are small (40 to 1,500 items), synthetic or in one domain, and their authors say so. Latency depends on hardware, state length and question count. The core README's own advice: treat Laya as a fast base to specialise, and fine-tune for accuracy.
More ways to use Laya