AutoBenchmark wraps up: human direction helps when autoresearch loops stall, plus Rebuttal Bench
jaseweston · x · 2026-09-30
Final part of the AutoBenchmark thread (with full Meta AI blog post): when the autoresearch loop stalls, human direction at that point helps. The team also introduces Rebuttal Bench (assessment) — testing whether a paper rebuttal actually resolves reviewer weaknesses. The blog details the method: autoresearch agents iteratively create benchmarks with feedback from solvers and external verifiers (human + AI), studying the scope and necessity of human contribution in open-ended tasks.
Related event: Meta AI introduces AutoBenchmark for automated benchmark creation(3 posts)→
More from Research
- Hand-editing 0.2% of weights plus 100 CPU-minutes turns an English TTS model into an 18MB Sanskrit one — kastnerkyle · 2026-11-16
- TotalSegmentator Measures Aortic Dilation on Chest CTs in 56s per Scan — MaziyarPanahi · 2026-11-16
- LatamGPT Releases 178M-Document Corpus with 290B Tokens on Hugging Face — OmarUFlorez · 2026-11-16
- Why chirality matters for AI for Science: enantiomers behave differently in biology — madeofAjala · 2026-11-16
- RSS 2027 submissions open with FAQ clarifying two-stage review and negative-result bar — GeorgiaChal · 2026-11-16
- Mobile AI Factory: agents auto-build on-device SLMs with 19x less GPU time — pmttyji · 2026-11-16