Laya-CoreML: 4.98 ms decisions on the Neural Engine, 2.78x better energy than MLX

# Laya-CoreML

**Open-weight typed decisions on Apple Silicon: Core ML, Neural Engine, zero generated tokens.**

[中文](https://github.com/mizorewww/laya-coreml/blob/main/README.zh-CN.md) · [Usage reference](https://github.com/mizorewww/laya-coreml/blob/main/docs/USAGE.md) · [Benchmarks](https://github.com/mizorewww/laya-coreml/blob/main/docs/ANE_BENCHMARKS.md) · [PyPI](https://pypi.org/project/laya-coreml/) · [Hugging Face weights](https://huggingface.co/aac6fef/laya-multilingual-coreml-adaptive-fp16) · [Changelog](https://github.com/mizorewww/laya-coreml/blob/main/CHANGELOG.md)

Laya answers constrained questions (pick an option, score against a rubric, judge true or
false) with calibrated probabilities from one bidirectional forward pass. It generates no tokens
and returns no JSON to parse. This package runs the original [Laya](https://github.com/NandhaKishorM/laya)
weights as Core ML programs on the CPU, GPU or Neural Engine. Inference needs no PyTorch,
Transformers or MLX.

**4.98 ms** median for one short multilingual decision on the M3 Max Neural Engine, using
**2.78× less system energy** than compiled MLX.



## Quick start

Requires Apple Silicon, macOS 15+ and Python 3.11–3.13.

```bash
pip install laya-coreml
```

```python
import laya_coreml as laya

agent = laya.load("aac6fef/laya-multilingual-coreml-adaptive-fp16")
result = agent.predict(
    "I was billed twice. Please refund the duplicate.",
    {
        "department": {
            "type": "choice",
            "instructions": "Which department should handle this request?",
            "criteria": {
                "billing": "Payments, invoices, refunds, and duplicate charges.",
                "technical": "Broken features, errors, and product troubleshooting.",
                "sales": "Pricing, upgrad