Salesforce: Fine-tuning a weak model to copy Gemini drops success 15%; correcting its own failures works
rohanpaul_ai · x · 2026-09-22
Salesforce AI research finds that once an agent's prompts, tools, and workflow are tuned around a weaker model (Qwen3-Coder-30B-A3B), fine-tuning it to copy a stronger Gemini model makes things worse.
- Across 7 enterprise tasks, tuning the agent setup around Qwen raised average success from 29.2% to 78.0%; Gemini reached 93.6% in the same setup.
- Fine-tuning Qwen on complete Gemini runs dropped success to 63.1%, falling on all 7 tasks — Qwen copied Gemini's planning style, which no longer fit the setup built around its original behavior.
- Keeping Qwen's failed runs and letting Gemini correct only the wrong step pushed success to 79.7% without losing earlier gains.
Lesson: tune the model without breaking the agent setup already working around it.
Related event: Salesforce: Imitating Stronger Models' Trajectories Hurts Agent Performance(2 posts)→
More from coding & agent
- Exa MCP hits 5,000 GitHub stars as AI agents flock to its search integration — TheIshanGoswami · 2026-09-22
- 670,000 agent skills, no trust layer: bot scan finds 69% never reliably fire — markjeffrey · 2026-09-22
- Training on production traces: single-trajectory RL may unlock continual learning — rhythmrg · 2026-09-22
- Anthropic's Swiss cheese model explains why passing evals isn't enough for agents — hugobowne · 2026-09-22
- OpenAI's artists are now all using Codex in their workflow — andrew_n_carr · 2026-09-22
- TinyTorch: PyTorch's free curriculum to build an ML framework from scratch in 20 modules — PyTorch · 2026-09-22