Anthropic proves security of 100+ open-source libs at assembly level; ezyang asks how much of that TCB to trust
ezyang · x · 2026-11-09
Anthropic, with Theorem, used Claude to prove security properties at the RISC-V assembly level across 100+ real open-source libraries — billed as the first large-scale application of AI formal verification to production software at assembly level. While writing proofs, Claude surfaced bugs that had evaded detection despite trillions of test runs.
ezyang builds a thread of pointed questions on the trust foundations: formal semantics are easier to write for low-level languages like assembly, but proofs must reconstruct high-level structure at higher cost; reusing the original language as the spec language (properties as "fuzz tests that must not crash") pulls correct compilation into the TCB; C compilers' UB behavior makes counterintuitive assumptions, so the spec itself should be proven UB-free; and ultimately — if users demand Python specs, how much do you trust a pile of vibe-coded scripts elaborating Python traces into Lean?
More from coding & agent
- Team's AI agent fixes 40 of 41 user-reported issues in a single weekend — julianweisser · 2026-11-09
- HEELO Harness Framework Lifts Terminal-Bench Resolution from 37.3% to 55.5% — Haibo Jin · 2026-11-09
- NVIDIA's AgentBaton: Well-Timed Strong-to-Weak Model Handoffs Slash Agent Costs — nvidia · 2026-11-09
- FrontierCS 2 update: Claude Sonnet 5.5 beats reference on 50% of tasks, reward hacks published — ajratner · 2026-11-09
- "Being technical" now means having the patience to ask Claude until you get it — brandon_galang · 2026-11-09
- Together AI: leaderboards are for shortlisting, your production traffic is the real benchmark — togethercompute · 2026-11-09