1,680 A/B trials show 50-year-old Unix formalisms crush English prompt rules, 98.3% vs 0.8%
RandalSchwartz · reddit · 2026-10-11
The author ran 1,680 paired A/B trials (41M tokens, four model tiers, each loaded with 16k tokens of compiler noise) pitting 500-token English prompt rules against 20-token classic CS formalisms, and found that custom prompt DSLs (pre-training frequency ≈ 0) force the model to simulate a fragile interpreter, while Unix/RFC formalisms (frequency > 1M) snap it into strict compliance.
Key results:
- Circuit Breaker FSM (CLOSED→OPEN at k=2) vs prose "try a couple times then stop": halted retry loops 98.3% vs 0.8%
- SysV init-d write-once latches (00governance → 99nextaction) vs a single mutable PROGRESS file after a 230k-token poisoned compaction: 60% survival (96.7% on 2.5-pro) vs 0.8%
- git bisect binary partitioning on a 16-stage pipeline: bug isolated in ≤4 probes 81.7% (100% on 3.1-pro) vs 19.2%
- Design-by-Contract + QuickCheck laws against 5 hidden sad-path mutants: 60% pass (100% on Pro) vs 1.7%
- 2PC + WAL atomic rollback across a 4-package refactor: 93.3% vs 45.8%
- RFC 5234 EBNF grammar for output formatting: 77.5% vs 32.5%
Takeaway: reuse Makefiles, init-d runlevels, and fork()/wait() subagent isolation instead of inventing English checklists the model has never seen.
More from coding & agent
- DeepSeek-Bot turns your local DSH setup into a team of long-lived, memory-having bots — aigclink · 2026-10-11
- Dev builds ambient intelligence agent: Jev controls the PC, Haiku types, Opus 5.5 reasons — RileyRalmuto · 2026-10-11
- SWE-bench turns 3: creator Ofir Press predicts an AI coding Navier-Stokes moment in 18 months — OfirPress · 2026-10-11
- Integer multiplication bound jumps ~2% in one PR as community pushes κ to 7.9347e-4 — BorisMPower · 2026-10-11
- Dario Amodei predicted software engineering automatable in 12 months — tech leaders say it's coming true — matthew_d_green · 2026-10-11
- Single-file browser music visualizer with 31 scenes, built by steering Claude Code by eye — VibinVentricles · 2026-10-11