Dev Open-Sources XCEval: Deterministic Evaluation Tools for AI Agents in Xcode 27
rudrank · x · 2026-07-31
Developer @rudrank shared updates to his AI coding agent evaluation tools for Xcode 27 Beta 4:
- Decoupled Modules: He exposed XCEvalCore and XCEvalFormat as library products, so other tools no longer need to copy the decoder.
- Testing Capabilities: The test lab covers persistence, model judges, tool calls, synthetic datasets, and xcresult export across four Apple platforms.
- Architecture: The agent stays outside the CLI. xceval provides deterministic tools for targets, timeouts, logs, sample selection, and more for agents like Codex, returning machine-readable errors. Ultimately, the developer's Swift code decides what a correct answer is.
Related event: XCEval Open-Sourced for Xcode 27 AI Agent Evaluation(2 posts)→
More from coding & agent
- Cloning Mystery Games for $7: Kimi K3 + MCP Demo — mervenoyann · 2026-07-31
- Testing Kimi K3 Agent: Automates Game Dev with MCP and Flux — mervenoyann · 2026-07-31
- DeepSeek demonstrates autonomous subagent orchestration without prompts — teortaxesTex · 2026-07-31
- Struggling to find MCP tools? Developer seeks best practices for AI coding agents — kampak212 · 2026-07-31
- Read/write is not enough: A four-dimensional risk model for AI agent permissions — Harshit-24 · 2026-07-31
- Generating synced SFX for videos while preserving voiceover using SFX MCP — Soggy-Weather169 · 2026-07-31