Astral Founder Defends Benchmarks Amid "Useless" Backlash

After critics mocked the industry's shift from hyping benchmarks to calling them useless, Astral founder Charlie Marsh (creator of uv and Ruff) responded that building good benchmarks is extremely hard yet worthwhile, and existing ones still hold value.

2026-09-27 ~ 2026-09-27 · 2 related posts

• Ruff author defends benchmarks: flawed but still the fairest way to compare AI models — EricBuess · 2026-09-27
• Astral founder defends benchmarks as useful but limited, admits better ones are hard — charliermarsh · 2026-09-27