Weight Space Programming: Claude Opus 5.5 beats GPT-6 Astra at hand-written 18-weight parity transformer
RobertTLange · x · 2026-11-08
Robert Tjarko Lange had Claude Opus 5.5 and GPT-6 Astra (Codex) program weights of a looped transformer block initialized to zero, using a Python sandbox and up to 200 forward-pass feedbacks, aiming for 100% test accuracy with as few nonzero weights as possible:
- 4-digit multiplication: Opus 273 weights, beating the 355-weight network that inspired the experiment; Astra 657
- Parity up to 60 bits (a classic task trained transformers fail at): Opus 18 weights vs Astra 24; the trick is attention as mean-pooling over bits, generalizing to longer strings than practiced
- Bracket matching (Dyck-2): 55 vs 262 weights; attention maps show each bracket looking back at its partner across 3 loop rounds
- Rule 110 cellular automaton: 323 vs 630
All four networks are exact on unseen sequences. Astra found correct networks within minutes but plateaued early; Opus thinks longer per step and kept finding sparser constructions, winning all four tasks.
More from Models
- LightOn releases LightOnOCR-3: 0.8B-4B open-source OCR models under Apache 2.0 — IgorCarron · 2026-11-08
- Abliterated GLM's "I am compressed humanity" self-description goes viral — RileyRalmuto · 2026-11-08
- antirez: DwarfStar now runs Kimi K3.1 PRO inference from safetensors via SSD streaming — antirez · 2026-11-08
- teortaxes: Plugin creation in DeepSeek Hub feels impressively natural — teortaxesTex · 2026-11-08
- Tencent's Terminal-RSI: a 27B model that rewrites and retrains itself with zero humans — Scobleizer · 2026-11-08
- Benchmaxxed again? Dev says Qwen 3.8 2.4T flopped in real use vs smaller GLM 5.3 — serious_mehta · 2026-11-08