Open scaling-law questions: NTP on a code repo vs RL on its tasks — which is more compute-efficient?
brianryhuang · x · 2026-10-15
Brian Ryhuang poses a set of PhD-worthy code data scaling-law questions: for a given repo, is next-token prediction (optimizing repo perplexity) or RL on repo tasks more compute-efficient? Do answers change if tasks must be synthesized vs already exist? Can one repo serve both NTP and RL without overfitting or interference — and when? Do repo properties or model capability level shift the answers? No answers given, but a sharp research agenda for code-model training.
More from Research
- Stanford's CausalSmith automates causal inference research with Lean-verified proofs — StanfordAILab · 2026-10-15
- Meta taps 5M health records to let doctors write AI-assisted research — shuchaobi · 2026-10-15
- Cellular Intelligence wins Best Poster at scverse 2026 for perturbation platform — arjunrajlab · 2026-10-15
- BARRACUDA microfluidic chip traps a million single cells to map gene circuit design space at scale — NikoMcCarty · 2026-10-15
- Stanford's DexAgent turns one human egocentric video into dexterous robot training data — StanfordAILab · 2026-10-15
- Johns Hopkins wins HCOMP 2026 Best Paper for reader-centered summarization evaluation — mdredze · 2026-10-15