laya-jev-lab: a local-first cascade that matches Jev's accuracy at 1.8x the speed

# laya-jev-lab

Measured comparisons of typed-decision models — **Jev** (TypeSafe, closed API) vs **Laya** (Convai, open weights) — plus a working **cascade** that matches Jev's accuracy at ~1.8× the speed.

Everything here was run on an Apple M4 Max (macOS 27, Python 3.14, MLX 0.32.2). Every number in this README comes from a script in this repo, and the raw output is in [`results/`](results/).

## Why this repo exists

Jev and Laya are the same idea from two directions: **don't generate text — answer a typed question and return a probability.** Jev is a closed API; Laya is open-weight and runs locally in milliseconds.

Almost all published comparisons are vendor-supplied. This repo is an independent attempt to answer three practical questions:

1. **Which primitives are actually reliable?** (`choice` / `score` / `noul`)
2. **Can confidence be trusted to gate automation?**
3. **Does a local-first cascade buy you anything over just calling the API?**

Short answers: only some primitives, only sometimes, and yes — if your traffic is mostly clear-cut.

## Headline results

### Jev vs Laya, 40 Chinese support-ticket classifications

| | accuracy | mean confidence | latency |
|---|---|---|---|
| Jev (API) | **31/40 = 78%** | 0.88 | 588 ms |
| Laya (local MLX) | 23/40 = 57% | 0.71 | **7.6 ms** |

By difficulty:

| tier | Jev | Laya |
|---|---|---|
| clear (single intent), n=20 | **100%** | 75% |
| ambiguous (multi-intent), n=10 | 70% | 60% |
| boundary ("你好", "???", "test"), n=10 | 40% | 20% |

Jev is perfect on clear inputs, frequently with confidence exactly 1.00. Both models struggle on boundary inputs — those messages genuinely have no correct label, which is a task-design problem, not a model problem.

### The cascade

```
input → Laya local (7.6 ms)
          ├─ confid