Codebase Understanding Test: Big Differences Across Models
emax · x · 2026-07-18
The author reports that on their current codebase, only fable high/xhigh and sol-high/xhigh/max still give fairly accurate codebase reading results.
In contrast, opus 4.8 and gpt-5.5 perform noticeably worse on this codebase, often drawing incorrect conclusions. Overall, sharing practical usability differences in code understanding tasks.
More from coding & agent
- Alchemy adds effect/Workflow support across Durable Objects, Workflows and Lambda Durable Functions — samgoodwin89 · 2026-10-27
- Alchemy's Lambda Web Functions run plain Node web servers on Lambda, no OpenNext needed — samgoodwin89 · 2026-10-27
- Alchemy Domain brings domain-driven design and story testing to Effect backends — samgoodwin89 · 2026-10-27
- Alchemy makes all ~2,850 AWS bindings work from any cloud, not just S3/Lambda/DynamoDB — samgoodwin89 · 2026-10-27
- Production AI agent scored 95% robustness at baseline, 37% under adversarial attack — njyx · 2026-10-27
- Chaining Grok Bot and Codex to detect and mass-block AI reply accounts — vista8 · 2026-10-27