AI-generated code passes all tests but is riddled with opaque contracts, finds Stanford professor's review
RobbWiller · x · 2026-11-17
Russ Poldrack reviewed code Astra & Fable generated for his boldtailor project: all 257 tests pass, but he found terrible internal naming (a write() that writes nothing), widespread unannounced in-place mutation, overused custom context managers with hidden closeout requirements, and a god-like mutable State object with implicit step dependencies. A follow-up AI audit produced a 218-line survey of these opaque contracts, published on GitHub.
More from coding & agent
- Claude Code demolished Microsoft's decade-long VS Code strategy in under 6 months, says Gergely Orosz — rickasaurus · 2026-11-17
- Personal agent checks email every 5 minutes on DeepSeek V4.1 Flash at near-zero cost — alexcovo_eth · 2026-11-17
- Riley Brown: After 50+ DMs, Nothing Beats Claude Code Projects Right Now — angrypenguinPNG · 2026-11-17
- MCP server turns your saved reels and posts into searchable notes for Claude — PaRtHuS · 2026-11-17
- Stardock CEO shows agents building games across PCs and Macs, with hard-won engineering lessons — draginol · 2026-11-17
- sandbox-mcp: Hanselman's MCP server gives AI agents disposable real Windows 11 machines — unixterminal · 2026-11-17