Laya MPS: typed decisions on a Mac GPU via PyTorch Metal, ~32 ms median

# Laya MPS

**Run Jev-style typed decisions locally on your Mac with low RAM usage and fast responses.**

Laya delivers typed decisions with ~32 ms median latency using ~2.1 GiB RAM on
M5 Pro, with a slower ~0.74 GiB mode for lower memory use.



MPS stands for [Metal Performance Shaders](https://developer.apple.com/metal/pytorch/),
which PyTorch uses to run Laya on your Mac's GPU.

[Laya typed-decisions](https://huggingface.co/convaiinnovations/laya-typed-decisions)
chooses options, scores inputs, and estimates whether statements are true. The
English model specializes in customer service, invoices, security incidents,
and agent traces. It is not a general-purpose language model.
[Model comparison and benchmarks](benchmarks/README.md).

## Requirements

- Apple Silicon Mac (M1 or newer), macOS 14+.
- Git and [uv](https://docs.astral.sh/uv/getting-started/installation/). uv installs Python 3.12 if needed.
- Allow 4 GB of free disk space for the runtime, download cache, and ~843 MB model.

## Get started

Clone the repository and start the server:

```bash
git clone https://github.com/afshinm/laya-mps.git
cd laya-mps
./scripts/serve.sh
```

The first run installs dependencies and downloads the model. Open
**[the demo](http://127.0.0.1:8000/demo/)** and click **Play** or **Run Benchmark**.
The benchmark runs for 60 seconds and plots response latency.

The server runs on `127.0.0.1:8000`. Later runs use local files and inference
works offline. Stop it with **Ctrl+C**. To update a clone, stop the server,
run `git pull`, then run the script again.

## Memory settings

All settings run the same complete model in FP32. Measured on an **M5 Pro,
24 GiB RAM**, macOS 26.4:

| Setting | Peak process RAM | Median decision latency |
| --- | ---: | ---: |
| `minimal` | **0.74 GiB** |