StateBench's four difficulty levels, from remembering a birthday to tracking swapped cups
allen_ai · x · 2026-11-04
StateBench splits state tracking into 4 levels—r-trivial, aperiodic, soluble, and general finite—ranging from remembering fixed facts like a birth date to tracking a ball as cups are repeatedly swapped, giving a graded view of where models fail.
More from Research
- Team burns 10B tokens in 2.5 days to build Omniscience, a city-scale world model — Kuprel · 2026-11-04
- Berkeley researchers use RL to improve robot data collection with Data Inoculation — berkeley_ai · 2026-11-04
- Map2Video uses maps and street view imagery to direct AI video of real locations — ryosuzk · 2026-11-04
- Xudong Sun wins honorable mention for SIGOPS Dennis M. Ritchie Dissertation Award — LingmingZhang · 2026-11-04
- First in-human gene correction fixes one-letter DNA typo in patient's liver — theimposingshadow · 2026-11-04
- 15-year-old's attention kernel survives 32k tokens where standard PyTorch OOMs, 11x faster — vankownlaw · 2026-11-04