Auditing HumanEval with MCP + Codex: fixing deep dataset issues
code_star · x · 2026-08-22
Using Codex via the MCP server, the author launched 47 concurrent sessions to audit every sample in MBPP and HumanEval variants within 2 hours. This process fixed deep issues in prompts, test cases, formatting, and even upstream bugs.
Related event: Developer Uses Codex MCP to Audit HumanEval and MBPP Benchmarks(2 posts)→
More from coding & agent
- Devin Oncall tops SREGym with 100% end-to-end success, auto-resolving production incidents — tianyin_xu · 2026-10-16
- Running a multi-model UI design arena: Paseo orchestrates four design drafts per model — vista8 · 2026-10-16
- Cortex A/B-tests coding-agent skills and rules by replaying developer corrections before keeping them — Samuele Giampieri · 2026-10-16
- Tool calls are the hidden bottleneck killing AI coding speed — Crabbox and 5 fixes — steipete · 2026-10-16
- Agentic sandbox WorldBox doubles GPT-6 Astra's MazeBench score, 23 to 61 gems — ZhitingHu · 2026-10-16
- Two prompts: Gemini 4 Argon turns a single-player game into full multiplayer — chongdashu · 2026-10-16