Berkeley & DeepMind paper: Abstract Token Curriculum lets models build internal scratchpads without CoT
rbhar90 · x · 2026-09-24
A paper from Berkeley and DeepMind challenges our reliance on Chain of Thought: we burn enormous compute and human effort hand-crafting step-by-step reasoning tokens just to nudge transformers toward the right answer.
Instead, the paper introduces an Abstract Token Curriculum (ATC) — the model is never forced to emit human-readable tokens, but is fed problems through steadily harder distributions, forcing it to invent its own continuous internal scratchpads to bridge the gap.
On parity learning they prove that single-layer softmax attention naturally gravitates toward intermediate representations, showing this latent reasoning mechanism emerges on its own.
More from Research
- Ross Girshick posts CVPR slides recounting 1.5 decades of object detection and its link to the LLM era — CatAstro_Piyush · 2026-11-11
- What it takes to squeeze performance: EMA, cosine scheduling and custom dead-cluster revival — chrisemoody · 2026-11-11
- Trainable JWST model boosts exoplanet sensitivity 2–15x in Nature paper — bravo_abad · 2026-11-11
- Ben Todd's alt log-log plot shows task horizons hitting a singularity in under 12 months — ben_j_todd · 2026-11-11
- Eva Jablonka's concept links aphantasia to the evolution of language — AnnaCiaunica · 2026-11-11
- BioAgent-DF: silent hardware faults can alter AI science agents' decisions invisibly — INTOSYN-AI-Labs · 2026-11-11