SubQ open-sources Decision LCR, a 72K–115K-token benchmark for decision models
ryanallred · x · 2026-10-16
Subquadratic open-sourced Decision LCR (subq-ai/decision-lcr on Hugging Face), a benchmark for evaluating decision models on realistic long-context tasks:
- Adapted from Artificial Analysis' AA-LCR: 100 human-written questions over real 72K–115K-token document sets, converted into 118 decisions, each run with three option orderings to control for position bias (Apache-2.0).
- Built because long-context benchmarks for decision models barely existed; the team argues it mimics real-world tasks better than many LLM long-context benchmarks.
- Their SubQ Snap model handles the >32K-token "danger zone" well — 85% of tokens it processes come from requests above 32K — and scores 74% on Decision LCR vs 58% for Jevons, with SSA keeping long context fast and cheap.
More from Research
- Google's Willow chip cuts quantum errors 40x, raising Q-Day concerns for Bitcoin — jamestagg · 2026-10-16
- A Cookbook for Machine Learning: compiling ML goals into tractable optimization — goyal__pramod · 2026-10-16
- Transluce vets: a year of neuron autointerp work still can't explain complex behaviors — aryaman2020 · 2026-10-16
- Transformer trained on flying bat recordings yields hippocampus 'digital twin' revealing hidden structure — patrickmineault · 2026-10-16
- One year of Recursive Language Models: GPT-5-mini beats GPT-5 by 110% on 132k-token tasks, cheaper too — a1zhang · 2026-10-16
- MIT PhD Sharut Gupta joins DeepMind; Isola hails her representational alignment defense talk — phillip_isola · 2026-10-16