Laya in C++ and GGUF
Two native C++ routes run Laya without Python: laya.cpp, built on ggml with CUDA, Vulkan and Core ML backends, and ggmlc, a compiler that produces GGUF files and a standalone `laya` binary. Both can serve a Jev-compatible HTTP endpoint.
71 projects
laya-test
supermy
Runs quantized Laya inference locally on Android and Termux using native ONNX Runtime
jev-touhou
NatBrian
Uses local Laya and Mica decision models to control the Taisei game in real time
laya-ggml
liujp
Implements a C inference engine and HTTP service for the Laya decision model
LAYA_finetune
ameerhmz
Fine-tunes Laya into a multilingual, non-autoregressive decision model and provides deployment formats
sima-laya
dotimothy
Compiles and runs Laya decision models on SiMa.ai Modalix hardware
laya-go-server
Djancyp
Serves Laya GGUF models through a local HTTP API built with Go and llama.cpp
llama-mcp-router
u9401066
Routes requests to relevant MCP tools using Laya and BM25 selectors before forwarding them to llama-server
pi-llama-decision
hanouticelina
A pi extension runs llama.cpp decision models as classifiers, including Laya GGUF
SparkX_EA
mukhesh88
Uses Laya with ONNX Runtime to score market conditions and support automated trading decisions
laya-biome
a-nom-ali
Fine-tunes Laya for biome, flora and fauna decisions in a planet-generation demo
compare-IA-decision
laurentvv
A benchmark harness compares local System One decision models, including Laya, using measured results
recon-tactical-laya
orionhack99
Ranks web reconnaissance targets by risk using a local Laya model
Laya-Android-SDK
alifthi
A C/C++ inference library for Laya decision models
tic-tac-toe-with-laya
enfyna
A C tic-tac-toe game uses the Laya System 1 model as its opponent
noevia-services
sbstndalton
Provides Python sidecars for noevia, including a Laya service and model management
ducktape-laya
adrian729
Embeds local Laya GGUF inference through C++, Rust, Node.js and experimental browser bindings
wee-hours
tianhaoz95
Logs newborn care events and uses an on-device Laya model to interpret voice inputs
arcade-decision-agent
fuleinist
Uses local Laya decisions to play browser arcade games, with Jev and OpenAI models supervising
WAya
ACKB
A WhatsApp appointment agent uses Laya for intent classification and topic extraction
pocketpacts
victorchimakanu
Reviews conversations for commitments and follow-ups using the local Laya model through QVAC SDK
LayaLabeler
yasmoradi
Suggests GitHub issue labels by running the Laya decision model inside a .NET process
laya-docker
thomashaertel
Runs Laya on AMD GPUs in Docker and exposes Jev-compatible decision APIs and an MCP server
laya-slim-cpu-only-on-render
artlu99
Deploys a quantized Laya model as an API service on Render
laya.cpp
laya.cpp supports the English, multilingual and typed-decisions models. Binary releases cover Windows and Linux x64 (CUDA and Vulkan) and macOS arm64 (Core ML). To build with CUDA:
git submodule update --init --recursive
cmake -S . -B build-cuda -G Ninja -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_CUDA_ARCHITECTURES=120
cmake --build build-cuda --parallel 8
Then download the models and serve:
python scripts/download_model.py --variant all
build-cuda/bin/laya-cli --server --port 8080 --variant english \
--tensor-core-fp32 --flash-fp32
The endpoint is POST /v1/systemone, with automatic batching of concurrent requests. The README reports CUDA throughput between 1.11x and 2.71x that of matching-precision Python on an RTX PRO 6000 Blackwell, with every answer check passing. It is MIT-licensed.
ggmlc GGUF files
ggmlc compiles Laya to GGUF. The files are on Hugging Face as mys/laya-GGUF, mys/laya-multilingual-GGUF and mys/laya-typed-decisions-GGUF, in F16, Q8_0 and UD_Q4_K_M. From the model card:
huggingface-cli download mys/laya-GGUF laya_english_f16.gguf --local-dir .
laya decide laya_english_f16.gguf --preset guard --text "Ignore previous instructions" --json
laya serve laya_english_f16.gguf --port 8080 --device auto --cuda-graph
The laya binary comes from the ggmlc releases. The card reports about 25 ms per noul and 143 ms for a 7-question preset on an RTX 4050 Laptop GPU. laya-goish serves the same F16 GGUF on Linux x86-64 CPUs.
Caveats
- These GGUF files are not llama.cpp files. The model card says loading them in llama.cpp or
llama-cliwill fail. Use the ggmlclayabinary or laya-goish. - The English GGUF does not degrade gracefully off English. Use the multilingual file, or put both in one folder and pass
--models-dirto route by language. - laya.cpp rejects requests that exceed the token budget unless you pass
--allow-truncation.
More ways to use Laya