Speculation: OpenAI may internalize reasoning via synthetic CoT rewriting and FiM
cephaloform · x · 2026-09-04
@ueaj speculates OpenAI likely isn't doing looped transformers (MoEUT possible), since RL signal isn't dense enough to saturate frontier math with minimal reasoning; more plausible is synthetic rewriting of CoTs—terse rewrites or dropping segments for next-gen pretraining to fill-in-the-middle, internalizing reasoning into weights. cephaloform confirms using a similar technique for a 2B GAN's reasoning discriminator: mass attempts, FiM-train the CoTs, fill in reasoning for gold answers, then SFT on synthetic chains and RL. Unverified speculation.
More from Models
- LMArena launches Image Edit leaderboard to rank models — arena · 2026-09-05
- Knowledge worker seeks cheaper Claude alternatives with strong instruction-following and long context — SemiMagnum · 2026-09-05
- Every to host live camp comparing Anthropic Fable 5.1 and OpenAI GPT-6 Astra — danshipper · 2026-09-05
- Real-world testing suggests Artificial Analysis Index is gamed and unrepresentative — PerformanceRound7913 · 2026-09-05
- Microsoft's MAI-Image-2.6-Flash hits #3 in image editing, jumping 34 Elo over last Flash — ArtificialAnlys · 2026-09-05
- User claims Artificial Analysis Index is easy to game, doesn't match real-world performance — PerformanceRound7913 · 2026-09-05