Benchmarking Kimi K3, GLM 5.2, and DeepSeek V4 Pro in Agent Workflows
Teknium · x · 2026-07-31
GMI Cloud evaluated Kimi K3, GLM 5.2, and DeepSeek V4 Pro on the tinyMMLU dataset using Hermes Agent and OpenCode frameworks.
- Kimi K3: Scored 96 with Hermes in just 40s/110K tokens, and 93 with OpenCode.
- GLM 5.2: Scored 94 on both, but Hermes consumed significantly fewer tokens than OpenCode.
- DeepSeek V4 Pro: Achieved the highest score of 97 with Hermes, while OpenCode was faster and extremely cheap ($0.06).
The test reveals that pairing models with the right agent framework drastically impacts speed, token efficiency, and accuracy.
Related event: Agent Framework Benchmarks: Kimi K3 Shines in Lightweight Efficiency(2 posts)→
More from coding & agent
- 'LLM code is sloppy' rebutted with real ffmpeg code: identical if/else branches in a top library — kingslyj · 2026-10-16
- Agent actions need checks outside the model: NVIDIA demos lean on guardrails amid wave of real agent incidents — Creamy-And-Crowded · 2026-10-16
- Vitalik builds "paranoid mode" agent wrapper with Tor, zkAPI and local prompt rewriting — DavideCrapis · 2026-10-16
- Agent fine-tunes Qwen3.5-0.8B with synthetic data: 4.7%→76.3% accuracy, 11.9× faster — ben_burtenshaw · 2026-10-16
- Beam-501B agent demo: solo beats 5-agent team on ProgramBench rebuild task — ben_burtenshaw · 2026-10-16
- Beam agent builds BeamSpeak, a local Parakeet dictation app for macOS — ben_burtenshaw · 2026-10-16