Fine-tuned 0.8B model beats stock Qwen 9B at English-to-bash, 79.4% vs 62.8%

AlternativeEnough352 · reddit · 2026-11-08

A Redditor built dale, a fully offline terminal assistant: you type plain English, a fine-tuned Qwen3.5 model writes the bash command, and a deterministic risk checker labels it safe/caution/danger before anything runs.

Key results (471 held-out requests, same llama.cpp, Q4KM)

dale-9b: 89.2% accuracy; dale-0.8B (529MB, CPU-only): 79.4%, median 1.7s on a 2020 laptop without GPU
Stock baselines: Qwen3.5-9B 62.8%, stock 0.8B just 14.9%; Gemma 4 models also far behind
Stock 9B adds sudo to commands that don't need it (35 times vs 5), prefers bloated Python one-liners (often broken) over factor/du -sh, and mishandles cp -n / passwd -l semantics

The transferable lesson: when the 0.8B plateaued at 74%, DPO, 27B-teacher distillation, extra epochs, and LR sweeps all failed. The real problem was distribution mismatch — training requests were short and clean while test requests were longer with stacked constraints. Regenerating 9k requests matching those stats moved 348→375 correct (p=0.001). Check how your data is written before reaching for fancier methods.

Ships as a 51MB single binary with llama.cpp inside (no Python, no server). Code MIT, models Apache-2.0 GGUFs. Caveats: baselines are untuned with an 8-shot prompt; test is synthetic and judged by two LLMs.

Original post →

Fine-tuned 0.8B model beats stock Qwen 9B at English-to-bash, 79.4% vs 62.8%

More from coding & agent