'Idea Basins': independent runs of the same task plateau at different scores compute can't close
menhguin · x · 2026-10-18
jackcai1206 observes that independent runs of the same task plateau at substantially different scores, with gaps persisting even after considerable additional compute. Embedding each run's solutions shows every trajectory stuck in its own small region of solution space — he calls these "Idea Basins."
Quoting, menhguin suggests plotting score vs compute (compute being matchable in theory), noting the framing is increasingly useful since cheap tokens mean experiments are no longer the bottleneck — compute and time are.
More from Research
- DeepSeek ships V4.1-Flash-1017: faster, shorter outputs, a claimed Pareto improvement — SonglinYang4 · 2026-10-18
- PyNNDescent 0.7 benchmark: ~6x faster kNN graph builds on 64-core Xeon — leland_mcinnes · 2026-10-18
- Terence Tao: OpenAI's quasi-Riemann result is the biggest advance in 50 years — jracine22 · 2026-10-18
- What is wrong with AI writing - ICLR reviewing edition — bobrodsky · 2026-10-18
- Meta, MIT and Yale propose ReHarness: backward revision for agentic continual learning — HanGuo97 · 2026-10-18
- Fields Medalist Kontsevich: AI Agent Suggested a Better Math Definition — luislamb · 2026-10-18