Same LoRA on every Qwen3.5 size: 9B beats 27B, and nearly all the gain is learning to say no

spilldahill · reddit · 2026-10-23

The author fine-tuned five Qwen3.5 dense sizes (0.8B to 27B) with an identical LoRA recipe on BioRED biomedical relation extraction, where 88% of the 10,042 test entity pairs have no relation.

Key results

Off-the-shelf models scored 2-8% F1; after LoRA all sizes hit 48-55%
9B was best (55.3%); 27B was significantly worse (paired bootstrap -5.65), likely because the trainer halved its learning rate
Nearly all the gain came from learning to output an empty list: false labels on negatives dropped 11-21x, precision rose from 1-5% to 43-48%
Reference anchor: BioRED's PubMedBERT baseline scored 47.7 F1; human annotator agreement was 78%

Advice for extraction fine-tuning

Count empty answers first — if most candidates have no answer, the base model's urge to say something caps precision, and a bigger base won't fix it (the stock 27B still labeled 70% of negatives)
Make "nothing" an explicit output ({"relations": []}) and train on it; 1:2 positive-to-negative sampling sufficed despite a 12% positive test rate
Score malformed output as wrong

Original post →

Same LoRA on every Qwen3.5 size: 9B beats 27B, and nearly all the gain is learning to say no

More from coding & agent