100-slot Terminal-Bench 2.1 rerun shows Luna 6 far behind Luna 5.6

s1lverkin · reddit · 2026-09-26

The author ran 100 identical Terminal-Bench 2.1 slots on Luna 5.6 vs Luna 6 via Harbor, then reran the whole comparison a day later.

Luna 5.6 passes 82–93/100 across configurations; Luna 6 only 29–62/100, with Xhigh/Max modes actually worse (e.g. Max Standard 33 and 29/100)
Luna 6 is cheaper and faster (e.g. $0.69 for Medium Fast, 62 passes) but clearly weaker
Root-cause guess: task containers inherit a read-only profile; Luna 5.6 distinguishes host read-only from container-writable and writes fine via tb4terminalexec, while Luna 6 assumes everything is read-only — in one Coq task it declared the workspace read-only without even attempting a write
The author had codex analyze the failure; its 'container was read-only' explanation doesn't hold
Totals: 1,300 trials, 384M tokens, $29. Caveat: only the 100 fastest slots from a 445-record job, so not representative of every workload

More from coding & agent