willcb: most modern LM ideas are testable on 3B-active models at 32K context
willcb · x · 2026-11-17
Adding to his argument that research ideas should be compressed into smaller experiments, willcb states that fairly few ideas in modern language modeling can't be tested on 3B-active models at 32K context — a concrete budget guideline for compute-limited research teams.
More from Research
- After MazeBench gets benchmaxxed, developers turn to secretive AI benchmarks — amplifiedamp · 2026-11-17
- InclusionMed launches open medical AI benchmark atlas after auditing 388 candidates — The InclusionMed Collaboration · 2026-11-17
- Tencent Hunyuan's Accelerate Bench tests 16 coding agents on 32 CPU optimization tasks — Tencent-Hunyuan · 2026-11-17
- SWE-CoWork: coding agents pass 63% of tests but complete fewer than 8% of tasks — Tokenwave-AI · 2026-11-17
- Centered Ablation: a closed-form fix that boosts refusal-direction ablation by up to 46 points — knoveleng · 2026-11-17
- DominoBench: VLMs can't reliably predict physical outcomes from still images — UniversityofSouthernCalifornia · 2026-11-17