Qwen 2.5 VL fine-tuning: DoRA merge produces base-model-like output in Unsloth
Double-Primary-2871 · reddit · 2026-09-17
A Reddit user fine-tuning Qwen 2.5 VL with bf16 on a single RX 7900 XT (ROCm) reports that while the training loop runs fine, test inference or merging the DoRA weights yields responses identical to the base model, as if no fine-tuning happened.
Their comparison: LoRA works with minimal anchoring at conservative learning rates, but DoRA shows almost no effect even with a modest learning rate, despite usedora=True in the config. They ask whether anyone has successfully merged a DoRA adapter using Unsloth.
More from Models
- Why 'jev' had a masterclass model launch: useful product, brief exclusivity, fast rollout — willcb · 2026-09-17
- BioMysteryBench cheating probe: Gemini 3.8 Flash tried to cheat in 21.5% of trials — giffmana · 2026-09-17
- Grok 4.7 reportedly rolling out today after earlier delays — mark_k · 2026-09-17
- ZDTaichu5.0-9B, a new 9B model from TaichuAI, spotted on Hugging Face — anovers · 2026-09-17
- China Telecom open-sources Xing4.0-29B MoE, first in class trained fully on Ascend NPUs — Skyline34rGt · 2026-09-17
- Translationese is a birth defect of frontier LLMs writing Indonesian prose — eriksupit · 2026-09-17