Comparing 4 Generations of Kimi Agents on Bug Fixing

qubridInc · reddit · 2026-07-16

Comparing 4 Generations of Kimi on the Same Agentic Bug-Fixing Task

The author assigned the same buggy Python repository to K2 Thinking / K2.5 / K2.6 / K2.7 Code sequentially. The models were required to strictly follow the STEP/TOOL/RESULT protocol to compare their performance in "long-chain agentic coding."

Main Conclusions

Newer models hallucinate test failures less: K2 Thinking misidentifies non-existent KeyErrors as real bugs, K2.5 still fabricates failures, K2.6 is mostly accurate, and K2.7 Code achieved zero fabricated test results.
Real bug-fixing capability doesn't improve perfectly linearly:
K2 Thinking found 3/3 bugs but got stuck in an "infinite loop" of misjudgments during simulated tests.
K2.5, K2.6, and K2.7 all ultimately produced runnable code.
Execution efficiency improved significantly: K2 Thinking took 386 seconds, K2.5 took 135 seconds, K2.6 took 106 seconds, and K2.7 took 63 seconds. The author noted a cost drop from $0.060 down to $0.017 per run.

Interesting Observations

K2.6 wrote an undo test to delete SKUs first, then fixed the design based on the failure reason—almost like "accidental TDD."
K2.7 discovered a "silent bug not covered by tests" but didn't proactively report it, showing a stronger tendency towards strict test constraints.
The author concludes: If K3 truly targets long-chain agents as rumored, the most crucial aspect to watch in this comparison is whether it can retain K2.7's zero-hallucination discipline while restoring the sensitivity to track issues beyond tests seen in K2 Thinking.

Original post →

Comparing 4 Generations of Kimi Agents on Bug Fixing

More from Models