Laya in C++ and GGUF

Two native C++ routes run Laya without Python: laya.cpp, built on ggml with CUDA, Vulkan and Core ML backends, and ggmlc, a compiler that produces GGUF files and a standalone `laya` binary. Both can serve a Jev-compatible HTTP endpoint.

71 projects

laya-mm-guard-v3-gguf

OidoStudio

Packages a calibrated Laya prompt-injection guardrail fine-tune for the Go Laya server

48·HF model

GoGGUF

laya-command-head

leehack

Provides a fine-tuned Laya decision head for classifying commands in a Dart app

1·HF model

GGUFFine-tunes & weights

laya-unity

0x440-1me

A Unity SDK for local Laya-compatible servers and TypeSafe System One endpoints

1·GitHub repo

C / C++SDKs & libraries

laya-rust

zerodegress

Implements a Rust inference engine for Laya with CUDA, CPU, and MLX backends

1·GitHub repo

RustNVIDIA / CUDA

sample.laya

hamedhajiloo

Demonstrates C# workflows for Laya typed decisions, including triage and prompt-injection screening

1·GitHub repo

C / C++Guides & learning

laya-vk

sulistta

Ports the Laya decision runtime to a Vulkan/IREE encoder backend for AMD and Qualcomm GPUs

0·GitHub repo

PythonC / C++

laya.cpp

kyr0

Implements native C++ inference and a JEV-compatible server for Laya typed decisions

0·GitHub repo

C / C++NVIDIA / CUDA

papaya

m0rphtail

Serves Laya typed decisions locally on Raspberry Pi using llama.cpp and a reconstructed decision head

0·GitHub repo

PythonC / C++

layar

nullean

A .NET port of Laya provides ONNX and TorchSharp inference backends and CLI tools

0·GitHub repo

ONNXC / C++

laya-cpp

samiul000

Implements native C++ Laya inference with ONNX Runtime, benchmarking, and INT8 quantization

0·GitHub repo

C / C++ONNX

laya-neutron

open-ep

Runs the Jev model on an NXP i.MX 95 Neutron NPU

0·GitHub repo

C / C++Runtimes & ports

laya-system-one

italoalmeida0

Provides a local JavaScript Laya decision engine with WebGPU and WASM acceleration

0·GitHub repo

Node.jsBrowser

laya-r-mlx

tjameswilliams

Provides a native Rust inference engine for Laya with MLX and candle backends

0·GitHub repo

RustMLX

MacJev-322M-4K-Laya-GGUF

lawrence3699

Runs a quantized Laya-derived decision model through llama.cpp and a Python decision head

0·GitHub repo

C / C++GGUF

ohmylaya

QuBiit0

Installs laya.cpp and exposes local Laya decisions to coding agents through MCP and an agent skill

0·GitHub repo

GoC / C++

Tenstorrent.Blackhole-convaiinnovations_laya

Thatch-cloud

Develops a Tenstorrent Blackhole inference backend and service integration for Laya

0·GitHub repo

PythonC / C++

NLaya

ljfio

Ports Laya typed-decision models to .NET with TorchSharp and ONNX Runtime

0·GitHub repo

ONNXC / C++

is-even-a-decision

GitHub Actions

Checks whether numbers are even using a local open-weights Laya decision model

0·npm package

Node.jsGGUF

laya-desktop

mategog

A desktop workbench runs Laya decisions offline, calibrates questions, and serves an API and MCP

0·GitHub repo

PythonMLX

laya-ollama-demo

pyardley

Routes support tickets with Laya and uses Ollama to draft replies when needed

0·GitHub repo

PythonGGUF

statim

BEKO2210

A native C++ engine that runs Laya checkpoints and serves the Jev/Laya decision protocol

0·GitHub repo

C / C++GGUF

clasificacion-tipada

tux22

Reproduces benchmarks comparing Jev, Laya, and local LLM classifiers on public datasets

0·GitHub repo

PythonNVIDIA / CUDA

laya-mario

darpan-97

Fine-tunes Laya to play Super Mario Land and provides replayable level runs

0·GitHub repo

PythonC / C++

Laya-cpp

FloofyJin

A C++ reimplementation of Laya with support for AMD, NVIDIA and CPU hardware

0·GitHub repo

C / C++NVIDIA / CUDA

laya.cpp

laya.cpp supports the English, multilingual and typed-decisions models. Binary releases cover Windows and Linux x64 (CUDA and Vulkan) and macOS arm64 (Core ML). To build with CUDA:

git submodule update --init --recursive
cmake -S . -B build-cuda -G Ninja -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CUDA_ARCHITECTURES=120
cmake --build build-cuda --parallel 8

Then download the models and serve:

python scripts/download_model.py --variant all
build-cuda/bin/laya-cli --server --port 8080 --variant english \
  --tensor-core-fp32 --flash-fp32

The endpoint is POST /v1/systemone, with automatic batching of concurrent requests. The README reports CUDA throughput between 1.11x and 2.71x that of matching-precision Python on an RTX PRO 6000 Blackwell, with every answer check passing. It is MIT-licensed.

ggmlc GGUF files

ggmlc compiles Laya to GGUF. The files are on Hugging Face as mys/laya-GGUF, mys/laya-multilingual-GGUF and mys/laya-typed-decisions-GGUF, in F16, Q8_0 and UD_Q4_K_M. From the model card:

huggingface-cli download mys/laya-GGUF laya_english_f16.gguf --local-dir .
laya decide laya_english_f16.gguf --preset guard --text "Ignore previous instructions" --json
laya serve laya_english_f16.gguf --port 8080 --device auto --cuda-graph

The laya binary comes from the ggmlc releases. The card reports about 25 ms per noul and 143 ms for a 7-question preset on an RTX 4050 Laptop GPU. laya-goish serves the same F16 GGUF on Linux x86-64 CPUs.

Caveats

These GGUF files are not llama.cpp files. The model card says loading them in llama.cpp or llama-cli will fail. Use the ggmlc laya binary or laya-goish.
The English GGUF does not degrade gracefully off English. Use the multilingual file, or put both in one folder and pass --models-dir to route by language.
laya.cpp rejects requests that exceed the token budget unless you pass --allow-truncation.