Ox Alpha tests average in bug fixing, trailing Grok 4.6 on benchmark
PawelHuryn · x · 2026-08-26
PawelHuryn shared hands-on test results of the stealth model Ox Alpha (GLM family) on OpenRouter, finding it disappointing.
Test Context: Based on the Bug Hunt Bench, measuring the ability to fix 105 real planted bugs across two repositories.
Results:
- Ox Alpha: Fixed 16/105 bugs, showing average performance.
- Grok 4.6: Fixed 27/105 bugs, performing significantly better.
Although Ox Alpha is currently free and has no hit limits, its capabilities did not meet the hype. The author anticipates that the release of Grok 4.7 could significantly alter the rankings. Live benchmark data is available online.
Related event: Stealth Model Ox Alpha Proves Mediocre at Coding, Trails Grok 4.6(3 posts)→
More from coding & agent
- Self Labs launches Agent Vault to let AI agents log in and pay with credit cards — EXM7777 · 2026-10-16
- Plane Agents: triggers, not prompts, are what turn chatbots into workflow agents — JosephJacks_ · 2026-10-16
- Claude eng lead Felix Rieseberg publishes tutorial on being more ambitious with AI — danshipper · 2026-10-16
- Qwen Code Desktop v0.25.2 ships managed-agent fixes and MCP config repair — github-actions[bot] · 2026-10-16
- NVIDIA makes 7 AI agent courses free, from agent basics to securing agents — ifioknkem · 2026-10-16
- Claude runs 9-hour task, then stalls waiting for Drive permission — juliey4 · 2026-10-16