jessicata: Coding AI Took 2 Days for a Simple Eval Harness, No Real Speedup
jessi_cata · x · 2026-11-15
AI safety researcher jessicata pushes back on claims that coding AI replaces programmer units, with a firsthand datapoint: building a seemingly simple eval harness to spec with Astra + Codex (Sol 6.1) took her 2 days, mostly waiting on models — she could have coded it herself in similar time. She suspects Anthropic employees are extreme power users inflating their perceived uplift, and cites studies (e.g. Jellyfish) showing low software-engineering gains. David Manheim counters that 3+ orders of magnitude cheaper matters regardless; jessicata proposes a testable crux: fully automatically writing large informally specified applications.
More from AGI Musings
- Insurance researcher pushes back: coding AI can't yet replace whole programmer teams — davidmanheim · 2026-11-15
- Business Owner With No Tech Background Nets $41,500 Deal 30 Days After Wiring Claude Into His Workflow — aryanXmahajan · 2026-11-15
- Musk Boosts Claim That 'AGI Has Been Achieved' via Multi-Model Stack — elonmusk · 2026-11-15
- AI researcher: employment collapse is 'baked in' to today's available tech — davidmanheim · 2026-11-15
- If intelligence creates power, why assume it creates the right to use it? — PierceLilholt · 2026-11-15
- Debate: AI doesn't need to replace every task, just whole business units — davidmanheim · 2026-11-15