Paper: Transformers converge to invariant algorithmic cores across training runs
pickover · x · 2026-11-20
A new paper by Joshua S. Schiffman argues training selects for behavior, not circuitry — many weight configurations implement the same function, so studying a single trained network risks describing accidents of one run. It introduces Algorithmic Core Extraction (ACE) to isolate compact subspaces necessary and sufficient for a task, recurring across independently trained models, with causal validation:
- Markov-chain transformers embed 3D cores in nearly orthogonal subspaces yet recover identical transition spectra
- Modular-addition transformers form compact cyclic cores at grokking that later inflate under continued regularization; this functional redundancy speeds memorization-to-generalization, yielding an inverse scaling law for grokking time
- Across six LMs spanning two orders of magnitude (GPT-2 S/M/L, LLaMA-3.1, Gemma-2, Qwen2.5), subject-verb agreement is governed by a single steerable axis aligned across architectures; flipping it inverts grammatical number in open-ended generation
Related event: Paper: Transformers Converge to Invariant Algorithmic Cores Across Models(2 posts)→
More from Research
- Precise 3D localization of real-world captures unlocks change detection, AR/VR and robotics — bilawalsidhu · 2026-11-20
- MRPD replays expert RL logs to consolidate agents at ~1/7 the cost of MOPD — illinois · 2026-11-20
- World Snapshot Models unify 1,000+ visual tasks as generation — World-Snapshot · 2026-11-20
- Maze Bench: best LLMs score at most 16% on complex 3D puzzle game — patience_cave · 2026-11-20
- Multi-agent RL recipe turns pass@k into pass@1 on hard chess puzzles — shizhediao · 2026-11-20
- Steering a Local LLM's Attention Directly: Sliders That Shift Activations, Not Prompts — round · 2026-11-20