Meta FAIR paper: research agents work best with humans, hack detection still weak
armand_ruiz · x · 2026-11-03
Meta FAIR, with Johns Hopkins, CMU and UW, released Aligning Autoresearch, studying long-horizon end-to-end research agents. Core finding: agents working with humans beat fully autonomous research.
Key results:
- AutoBenchmark: unaided agent-built benchmarks saturate once solver scores exceed 80; detailed human direction halves solver scores — humans remain pivotal, especially in ideation.
- HackBench: 169 real autoresearch trajectories (median >1M chars, 264 commands), testing whether agents can locate a hidden hacked command.
- Hack localization is still hard: best monitor Claude Opus 5 reaches only 64.88 F1, Muse Wonder 61.47, all others below 53.
- AutoInteract: RL with simulated human users lifts multi-turn Pass@1 on LiveCodeBench-Pro from 23.0→43.2 (4B) and 37.2→50.6 (27B).
- A rubric-aligned RL method also improves agent-to-human text communication.
The paper argues human-agent co-improvement is both safer and mutually reinforcing.
More from Research
- LLMs are getting good at directly writing neural network weights—what does that mean for gradient descent? — RichmanRonald · 2026-11-03
- UCSB awards IEE seed funding to three ECE projects on energy-efficient AI and quantum hardware — ninamiolane · 2026-11-03
- OpenAI deploys activation classifiers (probes) on frontier models starting from GPT-5.6 — bobabowen · 2026-11-03
- Google's ScientistTwo runs the full research loop autonomously, out-rating human papers — udmrzn · 2026-11-03
- ML4 tech report drops, detailing pretraining architecture and agentic performance interplay — Fluke_Ellington · 2026-11-03
- TuneJury: a model trained on human judgments predicts music preferences — mittu1204 · 2026-11-03