madewithlaya

Catalog / Documents & data

0087GitHub

chunklaya: why Laya stops reading after ~200 tokens, and a chunking harness that fixes it

Measures the decision head's attention drop-off on long inputs across 400 documents, then chunks, screens and aggregates, or locates the one passage that matters.

Open source ↗ github.comcostfree, no extra cost over a larger context windowtime-
myxamediyar/chunklayaREADME ↗
# chunklaya — unbounded-context System 1

**A decision model with no context limit. Tested at 1,000,000 tokens.**

Long-input harness for [Laya](https://github.com/NandhaKishorM/laya), the open-weights System 1
decision model: typed questions — `noul`, `choice`, `score` — answered over one piece of text in a
single forward pass, no autoregression. It's the open answer to TypeSafe's closed Jev.

Nothing in the harness scales with document length: chunk, locate the passage the question is about,
and spend exactly one forward pass on it. On the public Jev needle benchmark that holds from 24k
tokens to 1M — **AUC 0.993 at every size, unchanged to three decimals**, one passage scored, 0.9 s for
a million-token document and 15 ms per question after it's indexed. Jev itself stops at 32k.

The honest boundary, measured in the same runs: that's a *lookup* — a question naming something you
can search for. A *categorical* question ("is there any sports story in here?") has nothing to rank
on, so every passage gets scored, and the answer is then only as good as the detector's
false-positive rate times the number of passages — good to a few hundred passages, decaying after
(0.985 at 70, 0.785 at 5600). Both curves are below, with the things that didn't fix the second one.

Laya's checkpoints are configured for 512–1024 tokens, and Jev reads 32k. So what happens if you just
feed Laya more text? Not what you'd guess. The encoder handles long input fine — it's pretrained to
8192 tokens. The problem is the decision head sitting on top of it, and it's a weirder failure than
"runs out of context": past roughly the first 200 tokens, it stops paying attention to anything you
give it. A fact buried in the middle of a long document is effectively invisible to it.

This repo measures that fail

Also filed under Documents & data

  1. 0104

    zero-shot-ie-bench: GLiNER 2.5, GLiFormer, GLiClass, Laya, Von and Jev in one web UI

    Seven zero-shot extraction and classification families demoed, benchmarked and cross-compared, with sizes, licences and costs in one table.

    @umstek · Documents & data · free for the local rows

  2. 0103

    Zero-shot benchmark: Laya's advertised win over Jev does not reproduce on four datasets

    Frozen protocol, raw predictions, calibration and latency for Jev 1.13, PrismNLI-0.4B and Laya. Laya ties Jev on dair-ai/emotion and trails by 10–33 points elsewhere.

    @elcronos · Documents & data

  3. 0102

    jevbench: Jev vs GPT-5-mini, Claude Sonnet 5, fine-tuned DistilBERT, BART NLI and Laya

    Six classifiers over SST-2, AG News and more, reporting accuracy, macro-F1, ECE, p50/p95 latency, throughput and cost side by side. Laya runs locally on MPS or CPU.

    @dhruvmehra · Documents & data · $0 for the local rows

  4. 0098

    Agentic GraphRAG over S&P 10-Ks with Laya as a shadow decision engine beside Jev

    Anthony Alcaraz's talk demo routes typed decisions through Jev and shows Laya's answers beside them. On the TSMC run Laya rejected every true claim; that's the point.

    @AnthonyAlcaraz · Documents & data · free (Laya shadow); Bedrock for reasoning