Llama 405B Base Model Natively Mimics o1's Chain-of-Thought from 128 Pages of Context

Sauers_ · x · 2026-11-15

Sauers points out that base models aren't just role-playing pretraining corpora — they natively support out-of-distribution in-context learning (OOD ICL). In an experiment, Llama 405B base, given 128 pages of o1's chain-of-thought as context, successfully simulated o1's CoT reasoning style, showing base models can imitate unseen reasoning patterns without fine-tuning.

Original post →

Llama 405B Base Model Natively Mimics o1's Chain-of-Thought from 128 Pages of Context

More from Models