Qwen 3.8 27B pi adapter tested: 4x fewer tool calls, 54% faster, but no harder-task gains
Fabulous_Pollution10 · reddit · 2026-11-23
Testing the pi LoRA adapter on Qwen 3.8 27B across 10 hard SWE/ML tasks (3 runs each): the adapter teaches the model to use pi's native tools instead of falling back to bash — tool calls drop 244→62 (4x), agent time 39→18 min (-54%), pass@1 13.3%→23.3% with thinking off. With thinking on (xhigh, 81,920 output cap), turns drop 97→74 but reasoning tokens rise 57k→68k with no time savings; pass@1 33.3%→36.7%. Takeaway: the adapter improves harness efficiency, not problem-solving ceiling. Also: default 16k/32k output caps are too small — models can spend the whole response thinking and return nothing.
More from coding & agent
- Layout Studio for MOSS-Image v1.1.1 ships ComfyUI node and PNG prompt import — Rex-Racer9 · 2026-11-23
- Stanford's free RAG lecture goes beyond chunking: BM25, HNSW, and agents — stanfordnlp · 2026-11-23
- Stanford puts its AI agent-building course CS329Z on YouTube for free — stanfordnlp · 2026-11-23
- Don't trust your agent's 'verified' claim: 5 practical rules for AI self-verification — dotey · 2026-11-23
- LoRA Dataset Studio adds MCP so Claude or Cursor can run your LoRA training pipeline — Ill-Ant-9489 · 2026-11-23
- 44-Minute 3D Scene: Blender MCP + ThreeJs + Self-Built Skill Codex Workflow — OpenAIDevs · 2026-11-23