Idea: retrain 2023-2024 open base models with modern midtraining and RL to find sleeper hits

code_star · x · 2026-11-16

codestar proposes taking open base models from 2023-2024, applying modern midtraining and RL, then reranking their capabilities—betting some were sleeper hits just one good post-training away from doing cool stuff. He adds that models trained on the internet garbage filtered out of pretraining data could make excellent spam/SEO classifiers.

Original post →

More from Research

Research channel →