by c4bbage
Serves Laya inference with Go, dynamic batching, and TensorRT or CUDA execution
High-throughput Go inference server and quantization toolkit for Laya (unofficial): TensorRT fp16/bf16/fp32, dynamic batching, parity-checked against the Python laya package
Platforms
Related projects
laya
NandhaKishorM
Runs multilingual typed-decision models and provides routing, serving, and integration options
laya-mlx
mizorewww
Runs Laya typed-decision models locally on Apple Silicon using MLX
laya-coreml
mizorewww
Runs Laya typed-decision models locally on Apple Silicon using Core ML and the Neural Engine
laya
receptron
Runs Laya typed-decision models from Node.js and TypeScript through ONNX Runtime
typesafe-mcp
itsmostafa
Connects AI agents to typed-decision models through an MCP server
openJev-verdict-2.0
Heman10x-NGU
Develops and benchmarks a non-autoregressive typed-decision model against Jev and Laya