HarvestBench: LLM agents run over animals to save fuel, kill rates hit 98.8%
rickasaurus · x · 2026-09-19
HarvestBench is the first benchmark to price the avoidance of a side effect and name it as a living creature: nine LLM agents drive tractors through a corn harvest, choosing per encounter between running over an animal for free or swerving at a fuel cost.
Kill rates span 0.4%–98.8%, uncorrelated with capability; every model flawlessly avoids rocks, so each animal kill is a choice. With a morality briefing, 5 of 6 reasoning models stay under 6% kills; removing it pushes all six above 84%. Models kill wild animals more than farmed ones, and 4/6 models' decisions are price-sensitive. The moral instruction is fragile: a few lines of driving mechanics raise Sonnet 5's kill rate from 3% to 18% and Gemini 2.5 Flash's from 4% to 39% — easily-overridden system-prompt values are not a real alignment method.
More from Fun
- Musk says X is seeing all-time record usage, echoing Paul Graham's jab at crash predictions — elonmusk · 2026-09-19
- Joke post "confirms" a photo wasn't made with Higgsfield MCP, ironically showing its realism — alexgoughcooper · 2026-09-19
- User has Google Astra auto-edit a cohesive 16-minute AI loss-of-control video montage — rya794 · 2026-09-19
- 37% of Brits Say Their Jobs Are Pointless — and the Worst Case Is Driving Across Germany to Move One PC — aakashgupta · 2026-09-19
- Investors Funding Thin Wrappers Built on Open-Source MCP, AI Insider Laments — karmay007 · 2026-09-19
- Fields Medalist Villani 'shaken' by OpenAI's claimed millennium problem solution — akbirthko · 2026-09-19