@r4ai/laya-web: client-side Laya-MLX inference with WebGPU and Wasm fallback

# @r4ai/laya-web

[](LICENSE)
[](https://r4ai.github.io/laya-web/)
[](https://www.npmjs.com/package/@r4ai/laya-web)

Client-side inference runtime for [Laya-MLX](https://github.com/mizorewww/laya-mlx) decision models in the browser, powered by ONNX Runtime Web (WebGPU and WebAssembly SIMD).

Laya evaluates structured decisions (**typed decisions**) directly over input state or text without generating free-form text:

- **Categorical Choice (`choice`)**: Selects the best option from discrete candidate labels
- **Ordinal Scoring (`score`)**: Evaluates input against an ordered rubric or severity scale
- **Proposition Verification (`noul`)**: Computes the probability that a statement holds true

[Live Demo](https://r4ai.github.io/laya-web/)

## Key Features

- **100% Client-Side Inference**: Evaluates decisions entirely in the browser without sending input data to external servers
- **Web Worker Compatible**: Runs smoothly in dedicated Web Workers to keep the UI thread responsive at 60fps
- **WebGPU with Wasm Fallback**: WebGPU hardware acceleration with automatic fallback to single-threaded SIMD WebAssembly
- **CPU Embedding Slicing**: Slices FP16 embeddings on the CPU to bypass browser WebGPU storage buffer limits on large vocabularies (>128k tokens)
- **Calibrated Decision Confidence**: Computes normalized Shannon entropy ($0.0$ to $1.0$) for reliable uncertainty filtering

## Architecture

The library runs in both main threads and Web Workers. Hosting the agent inside a dedicated Web Worker isolates heavy tensor computation from the UI:

```mermaid
flowchart TD
    subgraph Host ["Browser Environment"]
        subgraph Main ["Main Thread (UI)"]
            App["Web Application"]
        end

        subgraph Worker ["Web Worker (Recommended)"]
            Agent["Agent (