Merging LoRAs via rank concatenation: orthogonal adapters combine losslessly

yb2698 · x · 2026-11-20

yb2698 explains the mechanism behind the "super-adapter" for Qwen3.8-27B: the three harness adapters' weight updates are nearly orthogonal (cosine 0.01), so they can be concatenated along the rank dimension at weight ½ each, yielding a single rank-96 LoRA that applies all three updates exactly.

Caveat: with a single harness and thinking off, that harness's own specialist adapter is still the better pick. The merged adapter is also far more efficient — 53–73% fewer tool calls with thinking off and 19–48% fewer with thinking on, on tasks both solve.

Related event: yb2698 merges a week of LoRA adapters into one super-adapter, boosting Qwen coding scores by up to 20 points(9 posts)→

Original post →

More from coding & agent

coding & agent channel →