Laya on Core ML
Core ML lets Laya run on the Apple Neural Engine as well as the GPU and CPU. Several projects convert the upstream checkpoints and publish ready bundles. They use the same choice, score and noul question types as the Python package.
34 projects
laya-ANE
vipuldivyanshu92
Ports Laya inference to Core ML for Apple devices, with a SwiftUI iPhone demonstration app
laya-tictactoe
Kasa-Harendra
A terminal Tic-Tac-Toe demo runs the Laya model locally through Core ML on Apple Silicon
laya-duum
Hadlock
Uses the Laya Core ML decision model to play the Duum game from semantic state
faceid-bench
mazenDDr
A face-unlock benchmark measures detection and recognition pipelines and tests Laya for yes/no decisions
laya
StevenJPx2
Provides a Swift and Core ML runtime, daemon, CLI, and distillation tools for Laya
homebrew-tap
madeye
Provides Homebrew formulae, including one for OpenZL-Laya local integer routing
laya-decision-web
LeeeeeeM
Runs a local browser demo for Laya decisions in Snake and side-scrolling game scenarios
jev-systemone-local
katya4oyu
Plans a local Jev-compatible System One server with MLX, Core ML, and ONNX Laya backends
laya-universal
abusuraihsakhri
Provides cross-platform inference utilities for Laya checkpoints using ONNX Runtime and MLX
laya-coreml-vs-jev-benchmark
sallout
Compares Laya and Jev on zero-shot intent classification benchmarks
laya-coreml
laya-coreml is the reference Core ML port. Requirements: Apple Silicon, macOS 15+, Python 3.11 to 3.13.
pip install laya-coreml
import laya_coreml as laya
agent = laya.load("aac6fef/laya-multilingual-coreml-ane")
result = agent.predict(
"The customer requests a refund of a duplicate payment.",
{"refund": {"type": "noul",
"instructions": "Does the customer request a refund?"}},
)
print(result["answers"]["refund"])
Its README reports 4.98 ms P50 and 5.31 ms P95 for one short multilingual decision on an M3 Max Neural Engine in FP16, and 2.78x better whole-system energy per decision than compiled MLX FP16. No PyTorch, Transformers or MLX is needed for inference.
Published bundles include aac6fef/laya-coreml (English, 512 tokens), aac6fef/laya-multilingual-coreml (1024 tokens) and aac6fef/laya-multilingual-coreml-ane.
Other Core ML projects
- LayaKit: a Swift package and
laya-cliover the laya-coreml bundles. - FluidUse: a Swift
LayaManagerthat runs the FluidInference/laya-coreml buckets. - laya.cpp: native C++ with a Core ML backend, shipped as a macOS arm64 binary.
- laya-apple and laya-fast: split work between the MLX GPU and the Neural Engine.
- keel: a macOS coding workspace that runs its Laya selector through Core ML.
Caveats
- The laya-coreml Neural Engine bundle accepts 96 tokens in total (question, options and state). Longer requests raise a capacity error.
- The laya-coreml README notes that a 1024-token request on the Neural Engine took about 91.7 ms, so the short-input speed does not carry over to long context.
- laya-apple warns that an ordinary Core ML export can run on the Neural Engine without error and still change answers. Prefer ports that publish parity checks.
More ways to use Laya