Oqoqo: hillclimb your MCP server with evals across 11 coding agents
pegasus013 · reddit · 2026-10-29
A co-founder of Oqoqo shared the tool for iterating on MCP servers: run the same tasks across different agents, see where they get stuck, tweak tool descriptions or implementation, and rerun — effectively hillclimbing your MCP without manually retrying prompts across clients.
Key points:
- Supports 11 harnesses including Claude Code, Codex, Cursor, Copilot, OpenCode, Pi and Hermes;
- Provides full trajectories plus success rates, tool calls, tokens and cost;
- Ablation mode runs the same tasks without your MCP to check whether it actually helps;
- Ships as a plugin so your coding agent can run the whole experiment — e.g. "create an eval set for this MCP and test it across Cursor, Claude Code and Codex."
More from coding & agent
- Pydantic's Code Mode report: 51% faster long tasks across 8 models and 5,276 runs — samuelcolvin · 2026-10-29
- Decagon details a self-improving autoresearch loop for inference stack optimization — kimberlywtan · 2026-10-29
- Dev uses Claude-built library to ship Obsidian plugin that plans music practice with spaced review — dSebastien · 2026-10-29
- Sitzmann open-sources texwerk, a self-hosted collaborative LaTeX editor with native agent support — vincesitzmann · 2026-10-29
- Professor runs DeckWerk's headless server mode for real-time lab slide collaboration — vincesitzmann · 2026-10-29
- Rime launches Prism, a speech-to-speech model that kills dead air in voice agent calls — testingcatalog · 2026-10-29