Grok Build tops HF Harness Leaderboard in all four configs but fails on text-only models

TheZachMueller · x · 2026-11-18

Joel Niklaus added xAI's coding agent Grok Build to the Hugging Face Harness Leaderboard: same models, same 198 coding tasks (Terminal-Bench 2.1 and DeepSWE), three attempts each, only the harness varies. On GLM-5.3-Flash and DeepSeek-V4.1-Flash at low and high reasoning effort it places 3rd, 1st, 2nd and 1st — the only harness of 11 to make the top 3 in all four, though statistically tied with 6-7 others each time. On text-only Ripple-65B-Preview a habit costs it: its file tool tells the model "You see image files as images," so the model believes it is multimodal, sends an image, the server refuses, and Grok Build ends the attempt on the spot — happening in 38 cases.

Original post →

More from coding & agent

coding & agent channel →