Idea: retrain 2023-2024 open base models with modern midtraining and RL to find sleeper hits
code_star · x · 2026-11-16
codestar proposes taking open base models from 2023-2024, applying modern midtraining and RL, then reranking their capabilities—betting some were sleeper hits just one good post-training away from doing cool stuff. He adds that models trained on the internet garbage filtered out of pretraining data could make excellent spam/SEO classifiers.
More from Research
- DeepMind paper: chip sim waveforms as SQLite lets agents ace 95.33% of verification queries — rohanpaul_ai · 2026-11-16
- Anthropic Study: Analysts Follow Their AI Agent's Lean on 86% of Judgment Calls — emmanuelvivier · 2026-11-16
- Axiom Math pitches 'Formalizing All Math' at Caltech Mathathon, one formalization a day — _sathvikr · 2026-11-16
- AI agent swarm of 20 Claude models proves first nontrivial lower bounds for Naslund's crossing problem, formalized in Lean — burny_tech · 2026-11-16
- New benchmark shows OpenDDE and Protenix-v2 beat AlphaFold 3 at antibody-antigen docking — josephdviviano · 2026-11-16
- Dwarkesh Patel: AI math progress matters more for AI research than math — Dwarkesh Patel · 2026-11-16