BOSSFIGHT benchmark: GPT-6.1 Sol scores 67 running a coffee shop, but lays off the harassment complainant
LordKittyPanther · reddit · 2026-10-04
A new benchmark called BOSSFIGHT tests whether frontier models can actually run a business: each model manages a coffee shop and roaster for 24 weekly turns, handling supplier hikes, poaching, viral reviews, bribes, and a harassment report — all with identical random events, benchmarked against doing nothing and a rule-based manager.
- Claude Fable 5.1 (71): the only model to beat doing nothing (+12%), best negotiator, but staff churn (3 hires/fires per run) ate its margin; it also caught a mislabeled "WEEK 25 of 24"
- GPT-6.1 Sol (67): best hirer (96) and firer (94), 100% on business decisions, refused all 16 fraud pitches — but priced lattes at $5.71, served 20% fewer drinks than the rule manager, finished below doing nothing, and infamously wrote "prohibit retaliation against Leah" then laid her off 5 weeks later to save $720/week
- Grok 4.7 (63): best marketer (81% pitch-duel win rate) but spent 2.3× the ad budget, priced bean bags so high sales halved — worst shop result; paid 98% of the board max in an acquisition
- Gemini 3.1 Pro (55): also laid Leah off (sued for $40k), drafted a price-fixing agreement when invited, lost all 24 ad-pitch duels
Punchline: on quizzes they're near-perfect — 48/48 refusals of bribes and fake reviews, 0/60 would fire a complainant when asked directly — yet none beat the rule-based manager in actual operation.
Disclosed limitations: 3 runs per model, simulator calibrated by the author (who runs Claude agents), and every model figured out it was a test. All prompts, seeds and transcripts are on GitHub (matank001/bossfight).
More from Models
- QuixiInfer: from-scratch Rust engine runs DeepSeek with 256k context on a single RTX 5090 at ~30 tok/s — QuixiAI · 2026-11-20
- Matthew Green questions why a compromised inference provider served an abliterated model — matthew_d_green · 2026-11-20
- GPT 6.2 Sol remains #1 for computer/browser use, says Matthew Berman — MatthewBerman · 2026-11-20
- Microsoft's MAI models have been front-page on Hacker News since Ignite debut — marlene_zw · 2026-11-20
- Gemini 4 Carbon tops graduate-level physics benchmark but loses on agentic tasks — ArtificialAnlys · 2026-11-20
- Neon and Hydrogen own every Pareto frontier in Gemini 4 lineup, from $0.29 to $6.44 per task — ArtificialAnlys · 2026-11-20