Merging LoRAs via rank concatenation: orthogonal adapters combine losslessly
yb2698 · x · 2026-11-20
yb2698 explains the mechanism behind the "super-adapter" for Qwen3.8-27B: the three harness adapters' weight updates are nearly orthogonal (cosine 0.01), so they can be concatenated along the rank dimension at weight ½ each, yielding a single rank-96 LoRA that applies all three updates exactly.
Caveat: with a single harness and thinking off, that harness's own specialist adapter is still the better pick. The merged adapter is also far more efficient — 53–73% fewer tool calls with thinking off and 19–48% fewer with thinking on, on tasks both solve.
More from coding & agent
- DeepMind's Antigravity Robotics Plugin: 2 Prompts, ~20 Minutes to Command a Real FR3 Duo Arm — jackyliang42 · 2026-11-21
- Ruff author warns: setting MYPY speeds up type checking on Pydantic but collapses types to Mapping[str, Any] — charliermarsh · 2026-11-21
- Asking Astra 6.2 to Review Code Written by Opus 5.5 Yields Almost Nothing — rickasaurus · 2026-11-21
- Security & Web3 Firewall MCP Server Hits Official Registry: Detects Tool-Poisoning and Rug-Pull Drift — ezelech · 2026-11-21
- Artificial Analysis Launches Search API Benchmark Covering 26 Products Across 14 Providers — ArtificialAnlys · 2026-11-21
- Pydantic hacks Mypy detection: why type checkers crawl and how to fake your way to speed — charliermarsh · 2026-11-21