laya.cpp: standalone C++ inference with CUDA and Vulkan backends

# laya.cpp

Native C++ inference for Laya typed decisions, powered by ggml with CUDA and
Vulkan and Apple Core ML backends. Tokenization, inference and JSON output run without Python.
Supports the `english`, `multilingual` and `typed-decisions` models, plus a
JEV-compatible HTTP server with automatic request batching.

[Binary releases](https://github.com/lkarlslund/laya.cpp/releases) provide Windows
and Linux x64 CUDA/Vulkan executables plus a macOS arm64 Core ML executable. See
[runtime requirements](docs/releases.md) to choose CUDA 12 or 13, check drivers, Windows CUDA DLLs and
compiled Core ML model buckets.

## Performance

Latest paired comparisons against matching-precision Python, using 250 fixed
questions across all three models at batch sizes 1, 2, 4 and 8. Higher is better;
**1× means equal throughput**. NVIDIA measurements use an **RTX PRO 6000
Blackwell (96 GB), capped at 450 W**; AMD measurements use a **Radeon 8060S**.

| GPU | Backend / mode | Throughput relative to Python |
|---|---|---:|
| NVIDIA | CUDA optimized FP32 | 1.35–2.48× |
| NVIDIA | CUDA BF16 | 1.11–2.71× |
| NVIDIA | Vulkan plain FP32 | 0.44–0.73× |
| NVIDIA | Vulkan compensated FP32 | 0.60–1.14× |
| NVIDIA | Vulkan FP16 / BF16 | 0.64–1.02× |
| AMD | Vulkan plain FP32 | 0.61–1.82× |
| AMD | Vulkan compensated FP32 | 1.26–2.45× |
| AMD | Vulkan FP16 | 0.72–1.25× |
| AMD | Vulkan BF16 | 0.49–0.92× |

All measured answer checks pass: exact categories and numeric absolute error
at most **0.0001**. Timings include preprocessing, inference and formatting,
excluding model loading and JSON transport.

See [latest measurements](docs/performance.md) for per-model throughput,
Vulkan FP32 results, direct CUDA/Vulkan comparisons and measurement identities.

## Build

Requires a C++20 compiler, CMake 3.2