Instruction Capacity paper quantifies rule-following limits: GPT-4o mini holds ~37 rules at 95% accuracy
Hennadii Kutomanov · hf · 2026-10-30
Independent researcher Hennadii Kutomanov published a paper proposing a statistical instrument to measure LLM instruction capacity under controlled load. Atomic replacement rules (single-token word swaps with computable answers) turn each application into Bernoulli trials, letting the model of replacement probability yield a capacity number with uncertainty. GPT-4o mini holds about 37–38 rules at 95% accuracy. The work gives a quantified way to express how instruction-following degrades as load grows.
More from Models
- ML Commons launches 'benchmarks as a service' to kill benchmaxxing — RexDouglass · 2026-10-30
- Gemini 4 Carbon takes #1 on Artificial Analysis, Anthropic S-1 in 60+ charts — altryne · 2026-10-30
- Leaker shows OpenAI shipped his ChatGPT texting feature ideas two days later — flowersslop · 2026-10-30
- First true omni model runs fully in-browser: 280 tok/s with WebGPU — victormustar · 2026-10-30
- Hidden ChatGPT checkout text reveals shared usage limits across ChatGPT and Codex starting December — btibor91 · 2026-10-30
- Grok 4.7 ranks least sycophantic model in AA-PersonalityBench, Slop score falls from 68 to 6 — XFreeze · 2026-10-30