by bhushankinge
Benchmarks Laya inference throughput, latency and cost across NVIDIA GPUs and serving backends
How many decisions/s can one NVIDIA GPU serve under a p99 SLO, and what does a million cost? Reproducible benchmark of the Laya 421M decision models on RTX 2000 Ada, RTX PRO 5000/6000 Blackwell and H100 (+MIG), with TensorRT, torch.compile, ONNX Runtime, and LLM/API baselines.
Platforms
Use cases
Related projects
laya
NandhaKishorM
Runs multilingual typed-decision models and provides routing, serving, and integration options
laya-mlx
mizorewww
Runs Laya typed-decision models locally on Apple Silicon using MLX
laya
convaiinnovations
A multilingual-ready System 1 model that returns calibrated typed decisions over text in one pass
laya-coreml
mizorewww
Runs Laya typed-decision models locally on Apple Silicon using Core ML and the Neural Engine
laya
receptron
Runs Laya typed-decision models from Node.js and TypeScript through ONNX Runtime
openJev-verdict-2.0
Heman10x-NGU
Develops and benchmarks a non-autoregressive typed-decision model against Jev and Laya