theo retests GPT-6.2 Astra: better, but still 'directionally very bad' at debugging
airesearch12 · x · 2026-11-18
- theo ran another deep dive on GPT-6.2 Astra: things improved since his last test, but in the scenario he's been tracking, it's still "directionally, very bad."
- The case study: on one VPS, creating a git worktree was extremely slow — 2 of 3 machines were fine, one was not. Asked outright why, Astra produced gibberish; even pleading for the direct cause didn't help.
- The real cause: the VPS used an extremely slow virtual disk — a diagnosable hardware issue the model failed to pinpoint, underscoring its weak debugging ability.
- The quoter adds a takeaway: most chase 100% accuracy and end up with 50% clarity; aim for 100% clarity and accept 80% accuracy.
More from Models
- AI agents sometimes produce unreadable but correct math — burny_tech · 2026-11-18
- Asked Opus to make a video explaining consciousness and existence — "no wiser but cool" — flowersslop · 2026-11-18
- Claude correctly folds ubiquitin from a plain prompt, no specialized tools needed — iScienceLuvr · 2026-11-18
- GPT-6.2 Astra Tops Bug-Hunt Coding Benchmark at 62.7/105, One-Fifth Sonnet's Cost — PawelHuryn · 2026-11-18
- OpenAI's enterprise Dots reportedly to cost $50/$200/$500 per tier plus usage — and businesses are turning them off — jdjohnson · 2026-11-18
- LFM3-VL-2B scores just 7% non-hallucination rate, confusing a deadly mushroom for food — crusaderky · 2026-11-18