LoRA + decision head on Qwen3.5-4B: full recipe for a 12-task agent router
MaziyarPanahi · x · 2026-11-07
MaziyarPanahi shares a complete training recipe for a small agent-task router: 1,500 rows from each of 12 agent tasks (18k total), source labels only, capped so no task dominates. LoRA r=16 plus a decision head on Qwen3.5-4B, lr 2e-4, batch 8x4. A time budget stopped training at 0.67 of an epoch, then 400 held-out rows calibrated the confidence. A cheap, reproducible pattern for small-model task routing.
More from coding & agent
- Testing shows coding harnesses can be harmful or hugely helpful depending on the model — JoshPurtell · 2026-11-07
- Tip: Run /prototype on Small UI Changes to Get Several Options to Pick From — mattpocockuk · 2026-11-07
- Databricks VP of Engineering and Neon Co-founder on Building Databases for AI Agents — matei_zaharia · 2026-11-07
- Multi-model coding apps are being acquired by labs — the survivors are open source — petergyang · 2026-11-07
- One folder that Claude reads first: the underrated Claude setup for solo operators — PrajwalTomar_ · 2026-11-07
- Linear's AI code review tool Reviews now free on all plans, no Linear tracking required — julianlehr · 2026-11-07