Laya on Core ML

Core ML lets Laya run on the Apple Neural Engine as well as the GPU and CPU. Several projects convert the upstream checkpoints and publish ready bundles. They use the same choice, score and noul question types as the Python package.

52 projects

laya-coreml-experiment

kazuki-oshino

A SwiftUI iOS app compares on-device Core ML inference for Laya and Kev

0·GitHub repo

SwiftCore ML

laya-hybrid-harness

Ra77Data

Runs a local Laya decision model and delegates uncertain cases to a cloud service

0·GitHub repo

PythonCore ML

Laya_Jev_opensource_guide

gotree94

Explains how to self-host Laya and Jev, including setup, fine-tuning, and benchmarks

0·GitHub repo

PythonMLX

d1-ane

nathanrchn

Converts and benchmarks decision models on Apple Neural Engine, with a Tetris demo

0·GitHub repo

PythonCore ML

laya-coreml

laya-coreml is the reference Core ML port. Requirements: Apple Silicon, macOS 15+, Python 3.11 to 3.13.

pip install laya-coreml

import laya_coreml as laya

agent = laya.load("aac6fef/laya-multilingual-coreml-ane")
result = agent.predict(
    "The customer requests a refund of a duplicate payment.",
    {"refund": {"type": "noul",
                "instructions": "Does the customer request a refund?"}},
)
print(result["answers"]["refund"])

Its README reports 4.98 ms P50 and 5.31 ms P95 for one short multilingual decision on an M3 Max Neural Engine in FP16, and 2.78x better whole-system energy per decision than compiled MLX FP16. No PyTorch, Transformers or MLX is needed for inference.

Published bundles include aac6fef/laya-coreml (English, 512 tokens), aac6fef/laya-multilingual-coreml (1024 tokens) and aac6fef/laya-multilingual-coreml-ane.

Other Core ML projects

LayaKit: a Swift package and laya-cli over the laya-coreml bundles.
FluidUse: a Swift LayaManager that runs the FluidInference/laya-coreml buckets.
laya.cpp: native C++ with a Core ML backend, shipped as a macOS arm64 binary.
laya-apple and laya-fast: split work between the MLX GPU and the Neural Engine.
keel: a macOS coding workspace that runs its Laya selector through Core ML.

Caveats

The laya-coreml Neural Engine bundle accepts 96 tokens in total (question, options and state). Longer requests raise a capacity error.
The laya-coreml README notes that a 1024-token request on the Neural Engine took about 91.7 ms, so the short-input speed does not carry over to long context.
laya-apple warns that an ordinary Core ML export can run on the Neural Engine without error and still change answers. Prefer ports that publish parity checks.