Laya benchmarks
The official Laya README publishes detailed numbers, and a growing set of independent repositories now test those claims. The overall picture: Laya is much faster when run locally and free to self-host, while Jev is more accurate out of the box on most shared tests.
228 projects
jev-arena
Samge0
Compares Jev, NanoJev, and Laya decisions in Tetris and 1024 with recorded gameplay replays
computer-use-testing
yoaugust
Tests Jev, Laya, and CUA for computer-use tasks and plans further reinforcement-learning fine-tuning
laya
DarkWanderer
Runs experiments with the open-source Laya System-1 decision model
jev-laya
wuzhiping
Probes the multilingual Laya model on Apple Silicon and records machine-readable inference results
laya-demo
tjpajala
Benchmarks Laya and Open-Jev on JevBench and PubMedQA with a Dockerized comparison interface
laya-ft
Alexander-Ollman
A reproducible study benchmarks Jev against fine-tuned Laya on workflows and chat moderation
laya-doom
cohenom
Displays Laya's live tactical votes alongside heuristic-driven Doom combat
laya-chess
buzzo123
Provides chess games against a computer that selects moves using Laya
Laya-test
LouisMoretti
Runs Laya locally for message classification and benchmarks its server and prediction performance
laya-mlx
MohammadAsadi-7
Runs Laya typed-decision models natively on Apple Silicon using MLX
laya-demo
cyyeh
Provides a Gradio playground, CLI scenarios, and device benchmarks for Laya
laya-test
petrixh
Runs Laya as the decision-making pilot in a browser game with reproducible evaluations
Laya-2048
DjTaNg-404
Uses local Laya weights to play 2048 with browser, terminal, and batch-testing interfaces
laya-snake-cuda
kuchris
Compares Laya and SemIf for local real-time Snake decisions with a CUDA dashboard
lunar-mpc-laya
mraad
Pairs Laya with adaptive model-predictive control in a lunar lander decision game
laya-mlx-demo
knishika62
Uses laya-mlx to classify Japanese virtual streamer personas and compare results with an LLM
jev-vs-laya
mouadse
Benchmarks hosted Jev, Kev-9B, and Laya on Moroccan Darija sentiment classification
laya-onnx-runtime
zenryokukikai
Ports the Laya multilingual decision model to CPU-only ONNX Runtime
laya-snake-arena
sathwikkuncham
Compares local Laya and Jev decision engines in a configurable Snake arena
jev-laya-chess-bench
abe17124
Compares TypeSafe Jev and Laya in head-to-head chess games using legal moves
jev-laya-japanese-business-benchmark
snsk
Compares Jev and Laya on a Japanese business decision benchmark
decisionmakertest
vittoriobrehautduran
Benchmarks Jev and Laya on typed decisions, labeled cases, and a small dungeon game
ai-decision-lab
uibuckets
Provides a local playground and benchmark harness for Laya, with optional Jev comparisons
IOCArena
hc-nolan
Compares Jev, Von, and Laya decision models on VirusTotal data
Official numbers
The core README and its BENCHMARKS.md report latency on a Tesla T4 (32.8 ms for one question with laya-multilingual, 39.5 ms with laya) and accuracy across 51 languages. Its Jev figures are third-party published, not measured by the Laya authors.
Independent head-to-heads
- sysone-bench: byte-identical inputs across 751 states. Jev leads on triage, guardrails, moderation, Banking77 and multilingual intent. Laya leads on AG News and MNLI.
- jev-laya-benchmark: 1,470 synthetic items. Jev 92.9% vs Laya 65.3% of judgments correct. For a single question, Laya was 42 ms locally on an M3 Pro and Jev was 136 ms. Jev was faster above 3 to 4 questions per request.
- laya-jev-lab: 40 Chinese support tickets. Jev 78% at 588 ms, Laya 57% at 7.6 ms on an M4 Max. A cascade that escalates below 0.60 confidence matched Jev's accuracy at 1.8x its speed.
- JevBench: a composite board. In v1.4.2, Laya (English, CPU) ranks 41st with a score of 30.25 and Jev 1.13.0 ranks 2nd with 63.29.
- laya-mcp eval: key accuracy 32.5% for Laya vs 98.8% for Jev, with median latency of 26 ms vs 138 ms.
CPU latency
laya-cpu-benchmark measured laya-multilingual in steady state on an Intel i9-14900HX. One question took 50 ms for a 71-token state, 307 ms for 261 tokens, and 1,315 ms for 927 tokens. Batching gave little gain on CPU. Its conclusion: a realistic CPU call is 0.3 to 2 seconds, not 33 ms.
Run your own
git clone https://github.com/glukicov/laya_router && cd laya_router
uv sync --all-extras
uv run laya-router eval run --backend laya --device mps
Or use jevbench (dhruvmehra) to compare Jev, Laya, LLMs and BERT models on public datasets.
Reading these numbers
Most of these suites are small (40 to 1,500 items), synthetic or in one domain, and their authors say so. Latency depends on hardware, state length and question count. The core README's own advice: treat Laya as a fast base to specialise, and fine-tune for accuracy.
More ways to use Laya