Routing with Laya
A router runs before every request, so its own latency and cost land on the critical path. Laya answers a routing question in one forward pass with no generated tokens, which makes it a natural fit for choosing an LLM tier, a support queue or a tool.
114 projects
laya-idjvsuen
muhfalihr
Fine-tuned Laya models for classification and decisions across Indonesian, Javanese, Sundanese, and English
laya-service
manul-audio
Wraps Laya predictions in a FastAPI service for Paperclip agent decisions
laya-gateway
aios-b-612
Routes coding-agent tool choices through Laya in a local LLM gateway
omp-laya
agusbyna
An omp wrapper adds Laya decisions and prompt pre-screening to agent workflows
laya-decide
Luminousyyh
A DeepSeek Harness skill gates file and command actions using a local decision model
pi-laya-router
rabi
Adds Laya decision tools and prompt-based model routing to the pi agent
gatelaya
Diwas2055
Adds Laya-powered multilingual firewall and routing guardrails to LiteLLM Proxy
omarchy-agent
pranav6266
Runs an offline Linux voice assistant that uses Laya to route user requests
@ejstembler/pi-classifier-router
ejstembler
Routes Pi and Oh My Pi prompts to models using typed decisions from Jev or Laya
laya-demo-0921
2045max
Demonstrates Laya decisions for routing, triage, moderation, RAG filtering, and other tasks
laya-demo
nile1801
Demonstrates typed decisions, calibrated probabilities, and multilingual routing with Laya checkpoints
laya-triage
Mezahir2025
Provides a Laya-based message triage API for Make.com
laya-school-email-triage
nccyber
Demonstrates Laya for routing and assessing urgency in synthetic school emails
laya_api
Anuragggggggg
Serves Laya typed-decision endpoints for generic decisions and candidate routing
laya
henrybit
Demonstrates Laya typed decisions with English and multilingual checkpoints
laya-demo
aired
Demonstrates Laya typed decisions for triage, guardrails, filtering, moderation, and routing
laya-demo
nickfury6023
Demonstrates Laya typed decisions for triage, guardrails, filtering, moderation, and routing
laya-demo
dungdq1
Demonstrates Laya typed decisions for routing, moderation, guardrails, and retrieval filtering
Built-in router preset
The core package ships a question set for small-versus-frontier model routing:
import laya
agent = laya.load("convaiinnovations/laya")
routing = agent.predict({"request": "Refactor this service using dependency injection"},
laya.router_questions())
For LangGraph there is a conditional-edge router with a confidence fallback:
from laya.integrations.langchain import LayaRouter
router = LayaRouter(
criteria={"billing": "invoices, charges", "tech": "bugs, outages"},
confidence_threshold=0.80,
fallback="human_agent",
)
workflow.add_conditional_edges("triage", router)
Note that laya.Router is a different thing: it picks which Laya checkpoint answers, based on script and language.
Measured: Laya vs a reasoning model
laya_router put Laya and GPT-5 nano behind one endpoint and tested 180 labelled requests. Both scored 0.600 route accuracy, at 184 ms p50 for Laya and 6,415 ms for GPT-5 nano. The README also found that rewording the tier descriptions moved accuracy by 21 points.
git clone https://github.com/glukicov/laya_router && cd laya_router
uv sync --all-extras
uv run laya-router serve --backend laya
Projects
- system-one-router: an OpenAI- and Anthropic-compatible Go gateway where Laya or Jev answers topic, complexity, risk and private-data questions, then picks the cheapest adequate model.
- pi-pignon: a pi coding-agent extension that picks a model tier per prompt from a local
laya-serve(about 75 ms on Apple Silicon, per its README) or Jev. - laya-mcp (NVentimiglia): a
route_taskMCP tool for light vs frontier routing.
Caveats
In laya_router, Laya got 97% of small requests right but only 7 of 56 medium ones, so check each tier separately, not just overall accuracy. Write tier descriptions with concrete examples, and send low-confidence answers to a fallback.
More ways to use Laya