GCMP update: LoRA-quality at 2.9M params, serves 256 adapters on one A40 with zero throughput drop

textclf · reddit · 2026-11-21

Author shares full benchmarks for GCMP, a LoRA replacement training just 2.9M params (11 MB) on Qwen3.8-27B.

Quality: statistically tied with LoRA r=8, r=32 and VeRA across 9 benchmarks (paired McNemar tests, all p ≥ 0.15). Since r=8 matches r=32, GCMP is 14x smaller than LoRA r=8 (not 55x as earlier claimed). VeRA is the same size and quality and trains 2x faster, but vLLM can't serve it.

Serving many adapters (one A40 48GB, vLLM 0.27.1, CQ 4-bit base, 32 concurrent requests, random adapter each):

CQ 4-bit base matches BF16 on all 9 benchmarks while NF4 loses 12 points on GSM8K. Main limitation: GCMP trains 2x slower per step; next up is faster CQ decode kernels.

Original post →

More from Infra

Infra channel →