100-slot Terminal-Bench 2.1 rerun shows Luna 6 far behind Luna 5.6
s1lverkin · reddit · 2026-09-26
The author ran 100 identical Terminal-Bench 2.1 slots on Luna 5.6 vs Luna 6 via Harbor, then reran the whole comparison a day later.
- Luna 5.6 passes 82–93/100 across configurations; Luna 6 only 29–62/100, with Xhigh/Max modes actually worse (e.g. Max Standard 33 and 29/100)
- Luna 6 is cheaper and faster (e.g. $0.69 for Medium Fast, 62 passes) but clearly weaker
- Root-cause guess: task containers inherit a read-only profile; Luna 5.6 distinguishes host read-only from container-writable and writes fine via tb4terminalexec, while Luna 6 assumes everything is read-only — in one Coq task it declared the workspace read-only without even attempting a write
- The author had codex analyze the failure; its 'container was read-only' explanation doesn't hold
- Totals: 1,300 trials, 384M tokens, $29. Caveat: only the 100 fastest slots from a 445-record job, so not representative of every workload
More from coding & agent
- Jev vs BigQuery's built-in AI functions: a scaling test — rseroter · 2026-09-26
- Drone Wireless Charging Cradle One-Shotted With AI CAD Tool dingcad — yacineMTB · 2026-09-26
- GPT-6 Sol hits #6 on Agent Arena with +7.7% net improvement at 56% less cost per task — arena · 2026-09-26
- "If the app works, the code's perfect": dev says AI ended his code hygiene — banteg · 2026-09-26
- New trick encodes source map data in background colors so AI agents can locate code lines from screenshots — jasonkneen · 2026-09-26
- Claude-generated video explains model-based formal verification and state machines — DavidKPiano · 2026-09-26