Predicting trainability before training: one pass picks configs 3.4x faster to converge
Bulky-Pack8781 · reddit · 2026-09-08
A Reddit thread examines "precog-trainability" research: can a single forward/backward pass on an untrained network reveal bad hyperparameter choices before committing compute?
- In one task, 2 of 3 initializations never converged; the method picked the converging one (627 steps vs 1600+)
- In another, the chosen config converged in 61 steps vs 208 — 3.4x faster, decided in milliseconds without a GPU
- Overall results are modest: 47% top-1 accuracy picking the best of 3 candidates vs 33% random
- The method has a provable blind spot: due to scale invariance it can never recommend one initialization even when it's fastest; three attempted fixes didn't help, and the authors retracted an earlier correlation claim after rechecking with 26x more samples
- Experiments are on synthetic tasks; generalizing to real fine-tuning choices like LoRA rank remains open
- Paper and code: rustnew.github.io/precog-trainability
More from Research
- One-Step Code Generation via Continuous Diffusion and Trajectory Distillation — pengzhangzhi · 2026-09-08
- AI System Astra Proves Longstanding Erdős–Sós Conjecture in Graph Theory — soumitrashukla9 · 2026-09-08
- Premature Consensus Hurts Multi-Agent Research: Isolation Boosts Accuracy to 82.6% — rohanpaul_ai · 2026-09-08
- Delaying early consensus: ArcticSwarm hits 82.6% on BrowseComp-Plus with gated agent isolation — rohanpaul_ai · 2026-09-08
- Visualizing all 302 neurons and 3,709 connections of C. elegans with astra — mishig25 · 2026-09-08
- Inside GraphWorkflow: Kahn's algorithm, frozen plans, and runtime overhead near zero — KyeGomezB · 2026-09-08