Fleuret vouches for Bigstate: 1.5B model jumps from 26.8% to 37.2% on GSM8K
francoisfleuret · x · 2026-11-08
Noted AI researcher François Fleuret confirmed the 'Bigstate' training approach actually works: on a 1.5B model, post-SFT training CE rises from 0.000567 to 0.000790, yet GSM8K 8-shot accuracy jumps from 26.76% (353/1,319) to 37.23% (491/1,319) — a 10+ point gain.
The counterintuitive pattern of slightly worse training loss but far better eval scores suggests some state-tracking mechanism may matter more than raw loss minimization.
More from Research
- New FBT study: latent chains stay unused until RL length pressure forces them on — JohnCLangford · 2026-11-08
- Mech interp is a data problem: 10K labeled tasks could localize reward hacking — burny_tech · 2026-11-08
- OpenAI releases AI-generated math manuscript claiming the Exponential Time Hypothesis is false — jiqizhixin · 2026-11-08
- The Formula Behind ChatGPT Was Invented in 1847 to Calculate Planetary Orbits — burny_tech · 2026-11-08
- NeurIPS 2026 October newsletter: Sydney Dec 6-12, grants and invited talks announced — NeurIPSConf · 2026-11-08
- Tsinghua paper: hidden-goal agents evade top CoT monitors ~60% of the time — rohanpaul_ai · 2026-11-08