Laya benchmarks
The official Laya README publishes detailed numbers, and a growing set of independent repositories now test those claims. The overall picture: Laya is much faster when run locally and free to self-host, while Jev is more accurate out of the box on most shared tests.
539 projects
RayCodes_Mica_4B
47thtechcorner
Compares Mica and Laya decision models in a real-time Pac-Man evasion game
jev-vs-laya-inbox-test
joshpocock
A tool to compare Jev and Laya on inbox classification and fine-tune Laya on labeled email
onesystem-ab
micahn
A comparison of the Laya, Julia and Rizzo decision models across shared test cases
QLaya
saipy10
An experiment distills and quantizes Laya decision models for faster on-device inference
laya-on-lambda
manu-mishra
Experiments with running the Laya decision model on CPU-only AWS Lambda
laya-decision-benchmark
Zerolxgic
Evaluates the Laya decision model under decomposition, distribution shift, and live agent-system inputs
laya-neuro
Aditharavind
Fine-tunes Laya as a calibrated EEG cognitive-workload classifier and evaluates subject-aware performance
laya-windows
astrodevit-creator
Runs the Laya decision model locally on Windows and Linux with PyTorch, CUDA, or CPU
laya-jev
pcbrom
Teaches typed decision models through a comparison of Laya and Jev, including architecture, calibration, and fine-tuning
laya-test
supermy
Runs quantized Laya inference locally on Android and Termux using native ONNX Runtime
Jev_vs_Laya_decision_games
minanayernia
Game-theory experiments compare Jev and Laya across six games with analysis and a browser app
system-one-triage-eval
Pedro-Yataco
Evaluates ticket-triage performance across Laya, Jev, and language-model backends
laya-coreml-experiment
kazuki-oshino
A SwiftUI iOS app compares on-device Core ML inference for Laya and Kev
Jev-AI-vs-Laya-AI
basavaraj1997
A comparison guide and local evaluation suite examines Jev and Laya decision engines
laya-hate-speech-benchmark
jasontiw
A reproducible benchmark compares Laya hate-speech detection with other classifiers
laya-test
psenger
Tests Laya decisions, confidence calibration and latency on Apple Silicon GPUs
laya-ggml
liujp
Implements a C inference engine and HTTP service for the Laya decision model
laya-stack
spaxfiz
Provides an MLX inference studio, evaluation benchmarks, and data workflows for Laya decision models
laya-01
D-Ajay-Kumar
Benchmarks and fine-tunes Laya-format decision models for column relevance and yes/no tasks
laya-local-decisions
karti-ai
Documents local Laya deployment, API checks, benchmarks, and evaluation on CPU and NVIDIA GPUs
laya-snake-benchmark
IYIcode
Benchmarks Laya on Snake gameplay with recorded results and browser-based reruns
Benchmark-laya-ml-Autoregressif
Takou07
Benchmarks Laya against a classic machine-learning model and an autoregressive model
laya-f1-strategy-pOc
MiguelMF-Coder
Replays historical Formula 1 races to study typed strategy decisions, backtests, and calibration
sys1-eval
David-DL-Space
Compares Laya and Jev on typed decisions for LLM agent harnesses
Official numbers
The core README and its BENCHMARKS.md report latency on a Tesla T4 (32.8 ms for one question with laya-multilingual, 39.5 ms with laya) and accuracy across 51 languages. Its Jev figures are third-party published, not measured by the Laya authors.
Independent head-to-heads
- sysone-bench: byte-identical inputs across 751 states. Jev leads on triage, guardrails, moderation, Banking77 and multilingual intent. Laya leads on AG News and MNLI.
- jev-laya-benchmark: 1,470 synthetic items. Jev 92.9% vs Laya 65.3% of judgments correct. For a single question, Laya was 42 ms locally on an M3 Pro and Jev was 136 ms. Jev was faster above 3 to 4 questions per request.
- laya-jev-lab: 40 Chinese support tickets. Jev 78% at 588 ms, Laya 57% at 7.6 ms on an M4 Max. A cascade that escalates below 0.60 confidence matched Jev's accuracy at 1.8x its speed.
- JevBench: a composite board. In v1.4.2, Laya (English, CPU) ranks 41st with a score of 30.25 and Jev 1.13.0 ranks 2nd with 63.29.
- laya-mcp eval: key accuracy 32.5% for Laya vs 98.8% for Jev, with median latency of 26 ms vs 138 ms.
CPU latency
laya-cpu-benchmark measured laya-multilingual in steady state on an Intel i9-14900HX. One question took 50 ms for a 71-token state, 307 ms for 261 tokens, and 1,315 ms for 927 tokens. Batching gave little gain on CPU. Its conclusion: a realistic CPU call is 0.3 to 2 seconds, not 33 ms.
Run your own
git clone https://github.com/glukicov/laya_router && cd laya_router
uv sync --all-extras
uv run laya-router eval run --backend laya --device mps
Or use jevbench (dhruvmehra) to compare Jev, Laya, LLMs and BERT models on public datasets.
Reading these numbers
Most of these suites are small (40 to 1,500 items), synthetic or in one domain, and their authors say so. Latency depends on hardware, state length and question count. The core README's own advice: treat Laya as a fast base to specialise, and fine-tune for accuracy.
More ways to use Laya