Routing with Laya
A router runs before every request, so its own latency and cost land on the critical path. Laya answers a routing question in one forward pass with no generated tokens, which makes it a natural fit for choosing an LLM tier, a support queue or a tool.
114 projects
laya
convaiinnovations
A multilingual-ready System 1 model that returns calibrated typed decisions over text in one pass
laya-multilingual
convaiinnovations
A multilingual System 1 model that returns calibrated typed decisions over text in one pass
laya-demo
convaiinnovations
A Gradio demo for Laya typed decisions, multilingual routing, moderation, and RAG filtering
keel
codejunkie99
A macOS coding workspace uses local Laya or hosted Jev to select task routes
hermes-nerve
keeltrace
Adds typed decision oversight and tool gating to Hermes agents using Jev or self-hosted Laya
ComfyUI-Laya
jtydhr88
Routes ComfyUI workflows using typed decisions from the Laya model
laya_router
glukicov
A Python model router compares local Laya decisions with a GPT-5 nano routing model
jev-chat-windows-laya
ZJemYoung
Adds an optional local Laya backend to a Windows WeChat chat assistant
laya-grounded
Luni
Fine-tunes Laya to improve grounding, contradiction handling, and calibration
layaRustparser
guptchar
A Rust framework normalizes firewall logs and uses Laya for bounded triage
home-assistant-laya
allenporter
Runs Laya as a local conversation agent for Home Assistant Assist
pi-laya-router
AdelysAlberto
Routes Pi coding-agent prompts using local Laya classification on Apple Silicon
auto-mode-for-paseo
obetomuniz
Routes Paseo messages to personas using TypeSafe Jev or a local Laya model
openzl-laya
madeye
Adds optional on-device Laya routing to OpenZL compression on Linux and macOS
pi-pignon
siiick
A Pi coding-agent extension routes prompts by difficulty using local Laya or hosted Jev
laya-thalamus
lcbkmm
Middleware uses Laya for agent tool routing with confidence-based LLM fallback and evaluation
openpave-jev
cnrai
Provides typed-decision commands for PAVE and Claude with local Laya and hosted Jev providers
laya-multilingual-onnx
soyelmismo
Provides CPU-optimized, quantized ONNX checkpoints for multilingual Laya
hermes-skill-router
cdepuy
Routes tasks to relevant Hermes skills using a local Laya model or SemIf
decide
gopaljigaur
Provides a Python client that chains Laya with hosted and other decision-model backends
laya-omni-agent
yashrastogi069-dev
An autonomous agent combines local Laya or Jev routing with cloud reasoning and browser, OS, and data tools
Tern
AmRitJain0442
A model router uses local Laya-MLX inference to select between configurable provider tiers
moex-wall-trader
antonBy77
Researches MOEX order-book signals with a Laya- or Jev-compatible AI trading gate
system-one-router
mmornati
A Go gateway routes prompts to language models using Jev or a locally running Laya decision model
Built-in router preset
The core package ships a question set for small-versus-frontier model routing:
import laya
agent = laya.load("convaiinnovations/laya")
routing = agent.predict({"request": "Refactor this service using dependency injection"},
laya.router_questions())
For LangGraph there is a conditional-edge router with a confidence fallback:
from laya.integrations.langchain import LayaRouter
router = LayaRouter(
criteria={"billing": "invoices, charges", "tech": "bugs, outages"},
confidence_threshold=0.80,
fallback="human_agent",
)
workflow.add_conditional_edges("triage", router)
Note that laya.Router is a different thing: it picks which Laya checkpoint answers, based on script and language.
Measured: Laya vs a reasoning model
laya_router put Laya and GPT-5 nano behind one endpoint and tested 180 labelled requests. Both scored 0.600 route accuracy, at 184 ms p50 for Laya and 6,415 ms for GPT-5 nano. The README also found that rewording the tier descriptions moved accuracy by 21 points.
git clone https://github.com/glukicov/laya_router && cd laya_router
uv sync --all-extras
uv run laya-router serve --backend laya
Projects
- system-one-router: an OpenAI- and Anthropic-compatible Go gateway where Laya or Jev answers topic, complexity, risk and private-data questions, then picks the cheapest adequate model.
- pi-pignon: a pi coding-agent extension that picks a model tier per prompt from a local
laya-serve(about 75 ms on Apple Silicon, per its README) or Jev. - laya-mcp (NVentimiglia): a
route_taskMCP tool for light vs frontier routing.
Caveats
In laya_router, Laya got 97% of small requests right but only 7 of 56 medium ones, so check each tier separately, not just overall accuracy. Write tier descriptions with concrete examples, and send low-confidence answers to a fallback.
More ways to use Laya