KAT-Coder-ProV2.5 Engineering Capabilities Tested
新智元 · wechat · 2026-07-14
KAT-Coder-ProV2.5 Tested: An Engineering-Grade Coding Model
Xin Zhiyuan conducted multiple rounds of testing on Kuaishou's KAT-Coder-ProV2.5. The focus wasn't on "completing a few lines of code," but rather on having it execute complete engineering tasks: building playable mini-games, generating complex interactive web pages, fixing real open-source repository issues, and adding features like chunked resumable uploads to an upload system. The article concludes that its performance on long-horizon engineering tasks is exceptionally strong, closely approaching the first tier.
Test Highlights
1. Complex Frontend & Interaction Generation
- Generated a playable penalty shootout mini-game with nearly 600 lines of prompts in 34 minutes.
- Generated a single-file HTML demo similar to Minecraft using 400+ lines of prompts, including blocks, mining, particles, procedural textures, and sunset effects.
- Also generated a Canvas2D-based solar system simulation requiring complex constraints like true orbital elements, a unified timeline, front/back occlusion of rings, and pause/rewind for rotation.
2. Real Repository Issue Fixing
- On a real bug in the open-source library humanize, it located the root cause after reading a few files: the complement storage of timedelta and taking absolute values separately caused negative durations to be incorrectly translated.
- The article notes the closed-loop time was about 1 minute and 20 seconds, and the diagnosis matched the official fix direction.
3. Engineering-Grade Feature Completion
- In a chunked upload system, it implemented a whole set of engineering details including resumable uploads, restart recovery, atomic metadata writes, and duplicate chunk validation.
- The final output was about 1400 lines of code, 5 new APIs, and 26 upload tests, with npm test fully passing.
Benchmark Scores
The article lists its performance across multiple evaluations:
- PinchBench: 94.2
- SWE-BenchPro: 65.2, second only to Opus4.8's 69.2
- KATCodeBench: 53.1
- KATClawBench: 85.5
Methodology & Training System
The article also breaks down the underlying engineering approach:
- AutoBuilder: Automatically sets up runnable and verifiable repository environments, boosting environment setup success rates from 16.5% to 57.2%.
- KwaiClawEnv: Converts real business workflows into trainable tasks, averaging about 15 tool calls per trajectory, with the longest exceeding 100 steps.
- Large-scale RL: Utilizes multi-framework scaffolding, long-horizon credit assignment, and a three-tier reward mechanism to improve exploration, error correction, and verification capabilities in complex tasks.
- MOPD: Integrates capabilities like long-horizon engineering, general Agent skills, terminal operations, frontend aesthetics, and general knowledge into a single model via multi-teacher online policy distillation.
Conclusion
The core takeaway is that the next phase of coding models relies less on parameter scale and more on environments, training trajectories, RL stability, and engineering infrastructure. For developers, these models are starting to possess the ability to "directly take on complete issues and workflows," rather than just filling in code snippets.
More from coding & agent
- A roundup of AI agents and MCP resources, including how to evaluate agents — _jaydeepkarale · 2026-07-21
- A full course shows how to build and deploy an AI agent with OpenAI and LangChain — _jaydeepkarale · 2026-07-21
- A beginner guide to AI agents points readers to a Stanford webinar — _jaydeepkarale · 2026-07-21
- A practical guide on how to evaluate AI agents — _jaydeepkarale · 2026-07-21
- MCP is headed toward easier scale, event-driven extensions, and workable file uploads — EricBuess · 2026-07-21
- Developers debate the missing composition model for AI agents — threepointone · 2026-07-21