Gradle's eval harness pits Fable 5.1 vs Astra 6 on one bug—Astra 6 faster and cheaper
Party_Till_I_Die · reddit · 2026-09-10
A Gradle engineer shares an in-house eval harness built to test whether a skill or code/doc change actually helps coding agents. Using it to compare Fable 5.1 and Astra 6 on the same Gradle GitHub issue, both fixed the bug, but Astra 6 did it faster and cheaper. Full diffs, costs and token counts in the blog post.
More from coding & agent
- Open-Source Ref2V Workflow Auto-Transcribes Media and Formats Prompts for H3 Video Generation — bstr3k · 2026-09-10
- K3 hit 77% speedup optimizing mjwarp kernels; GPT-6 could only add 0.38% — YouJiacheng · 2026-09-10
- K3 got 77% speedup on mjwarp kernels; GPT-6 Astra added just 0.38% — YouJiacheng · 2026-09-10
- Same async program, four different outputs: paper maps the async/await design space across 7 runtimes — IanArawjo · 2026-09-10
- Dev says AI-written PRs keep getting smarter, and reviewing them is getting harder — smlpth · 2026-09-10
- Automattic launches here.now, instant web hosting for agents with 500k sites published — iannuttall · 2026-09-10