Salesforce paper: fine-tuning on evolved harnesses makes weak models worse on all 7 tasks
omarsar0 · x · 2026-09-10
A new Salesforce paper co-evolves agent harnesses and models: a harness evolved with a weak model across 7 enterprise tasks, followed by fine-tuning on a stronger expert's trajectories under that harness, dropped performance by 4-30 points on all tasks for Qwen3-Coder and Gemma 4. The same fine-tuning helps under the unevolved harness, pointing to model-harness fit: imitation transfers planning strategies the weak model can't execute. The proposed fix uses a meta-level agent to target failing turns instead of copying whole trajectories.
Related event: Salesforce: Co-evolving Harnesses and Models Shapes Fine-tuning Outcomes(2 posts)→
More from coding & agent
- Resin lets an agent build standalone LLM inference engines; vLLM and SGLang optimize DeepSeek kernels differently — AccBalanced · 2026-10-27
- How I let my cloud Grok Bot code on my Mac mini via Tailscale, no public ports — talkaboutdesign · 2026-10-27
- Context layout is a cost decision: one memory edit can bust your entire agent prompt cache — dscape · 2026-10-27
- Alchemy adds effect/Workflow support across Durable Objects, Workflows and Lambda Durable Functions — samgoodwin89 · 2026-10-27
- Alchemy's Lambda Web Functions run plain Node web servers on Lambda, no OpenNext needed — samgoodwin89 · 2026-10-27
- Alchemy Domain brings domain-driven design and story testing to Effect backends — samgoodwin89 · 2026-10-27