Anthropic's 2021 research roadmap surfaces: 1T models, alignment scaling laws already on the list
srush_nlp · x · 2026-10-14
Anthropic's internal 2021 research roadmap, made public earlier this year due to copyright litigation, was dug up by @smitop and shared by srushnlp.
The document covers:
- Pretraining and Data: building the best language+code models and datasets (natural language, code, diffs, PRs, debugging); finetuning on diff/Wikidiff data to connect human intentionality to model behavior; studying training of 1T+ parameter models, very large contexts (or equivalents for packing memory/experience), and data quality.
- Human Feedback and Alignment: training models to be generally helpful via human feedback; identifying and classifying alignment failures; explicit "Scaling Laws for Alignment" research, including robustness of Reward Models on on-policy feedback and GAN-discriminator datasets, and resource requirements for alignment goals.
It offers a rare look at Anthropic's early priorities, which map closely onto the later Claude/RLHF trajectory.
More from Companies & People
- Coinbase fine-tuned Qwen3.5-9B for fraud prevention — beat frontier models, cheaper — lmoroney · 2026-10-14
- Great ideas per hour: the Progress community is booming across mags, talks and student groups — Afinetheorem · 2026-10-14
- OpenAI bugging case likely amateur, not state spy op — planted devices may remain — beffjezos · 2026-10-14
- Character.AI has 12M MAU spending 120 min/day — and VC logic can't price it — signulll · 2026-10-14
- 81% of after-hours calls to US medical practices go unanswered, Prosper AI finds — HealthcareAIGuy · 2026-10-14
- Meta CFO warned Zuckerberg the superintelligence bet could end 'very, very bad' — robleclerc · 2026-10-14