Run Laya on your Mac
Apple Silicon is the best-covered platform in the Laya ecosystem. Independent ports run the upstream checkpoints on the MLX GPU, through Core ML on the Neural Engine, or inside native Swift apps. Each port publishes its own speed and fidelity measurements, all taken on the author's own Mac.
308 projects
laya-mlx-http
mjrussell91
Serves the Laya MLX question-answering model through a local FastAPI endpoint
laya_ai_model_demo
lalithdabilpuram01
Tests Laya on labelled support emails and measures inference latency across local devices
ai-sec-lab
shsingh
A Mac-based Kubernetes lab demonstrates prompt-security defenses using Laya and NOVA
local-jev-bench
tak-bro
Benchmarks local Jev-style decision models, including Laya, on Apple Silicon
laya-sentiment-demo
Ramg77
Demonstrates multilingual sentiment classification with calibrated on-device predictions
laya-mcp
kennedyshead
Serves local Laya routing decisions to coding agents through an MCP server
laya-api
gugaucb
Serves Laya and Laya-MLX models through OpenAI- and Anthropic-compatible APIs
laya-coreml-hello-world
tuk-solita
A local web page demonstrates Laya choice, score, and yes/no predictions using a Core ML model
sys1rust
kgarg2468
A Rust and MLX service runs Laya decision models locally and exposes the System 1 API
notchvoice
Starlord-999
A hands-free Mac voice assistant that uses Laya-MLX for on-device decisions
laya-coreml-experiment
kazuki-oshino
A SwiftUI iOS app compares on-device Core ML inference for Laya and Kev
laya-test
psenger
Tests Laya decisions, confidence calibration and latency on Apple Silicon GPUs
laya-hybrid-harness
Ra77Data
Runs a local Laya decision model and delegates uncertain cases to a cloud service
laya-stack
spaxfiz
Provides an MLX inference studio, evaluation benchmarks, and data workflows for Laya decision models
laya-service
FnSGit
Serves local Laya inference through a FastAPI endpoint compatible with the Jev protocol
laya-01
D-Ajay-Kumar
Benchmarks and fine-tunes Laya-format decision models for column relevance and yes/no tasks
LAYA_finetune
ameerhmz
Fine-tunes Laya into a multilingual, non-autoregressive decision model and provides deployment formats
laya-service-desk-demo
choas
Demonstrates a local Laya-powered customer service bot using prepared replies and human handoffs
local-systemone
axiomantic
Provides a self-hosted System 1 decision daemon and MCP server with a native Laya backend
local-settings-slm
AkhileshManda
Uses Laya to generate Android settings decisions for distilling into an on-device Qwen model
LocalFirstGmailIntelligence
karan19
Sets up local Gmail intelligence models on Apple Silicon, including Laya-MLX
laya_server
smarty-kiki
Serves a local Laya decision model through a Jev-compatible API on Apple Silicon
laya-auto-mode
tuncer-byte
Adds Laya-powered automatic model routing to the GitHub Copilot Chat model picker
AgentSmith
nsh1999
Standalone demos explore local Laya inference, typed decisions, and LLM agent workflows
Pick a runtime
MLX (Python). laya-mlx is the most widely used port. It needs Apple Silicon, Python 3.11+ and macOS 14+:
pip install laya-mlx
import laya_mlx as laya
agent = laya.load("aac6fef/laya-mlx")
result = agent.predict(
"I was billed twice. Please refund the duplicate.",
{"department": {"type": "choice",
"instructions": "Who should handle this?",
"criteria": ["billing", "technical", "sales"]}},
)
print(result["answers"]["department"])
Its README reports 13.42 ms P50 for one short question with the 421M English checkpoint and 7.39 ms with the 322M multilingual one, FP16 on an M3 Max.
Core ML and the Neural Engine. laya-coreml needs macOS 15+ and Python 3.11 to 3.13 (pip install laya-coreml). Its README reports 4.98 ms P50 for one short multilingual decision on the M3 Max Neural Engine. laya-apple runs the MLX GPU and the Neural Engine together and serves a Jev-compatible API:
pip install 'laya-apple[serve,ane]'
laya-apple serve # http://127.0.0.1:8642
Swift. laya-mlx-swift embeds Laya in a macOS app with MLX (about 804 MiB download). LayaKit is a Swift package over the laya-coreml bundles, and FluidUse ships a LayaManager for Core ML.
PyTorch MPS. The official laya package also runs on the Mac GPU; laya-mps wraps it in a local server with low-memory modes.
Caveats
- All of these are independent ports, not official Convai Innovations releases. Most publish fidelity checks against upstream answers; read them.
- The Neural Engine bundle in laya-coreml has a 96-token total limit covering question, options and state. Use the 1024-token general bundle for longer input.
- Latency numbers are single-machine measurements. Your Mac, input length and question count will change them.
More ways to use Laya