GCMP matches LoRA r=32 on 27B model with 55x fewer adapter parameters (11MB vs 320MB)
textclf · reddit · 2026-11-17
The author benchmarks GCMP, a LoRA replacement: on Qwen3.8-27B it trains 2.9M params (11MB) vs 159M (320MB) for LoRA r=32 on the same 7 projections — 55x smaller.
- Accuracy: identical training (Alpaca, 500 steps) and lm-evaluation-harness scoring with McNemar tests. On BF16 base, no significant difference on any benchmark (p≥0.2); on NF4, GCMP wins GSM8K by +4.0 (p=0.04) and ties elsewhere, with slightly lower answer loss.
- Why size matters: multi-tenant vLLM serving (mixed-adapter batches in one kernel call, no merging/swapping), resident GPU memory (11MB/adapter plus fixed overhead), storage (1,000 adapters ≈ 11GB vs 320GB), and edge/small-GPU fine-tunes.
- Caveats: 2x LoRA step time; generous r=32 baseline (r=4/8/16 comparisons pending); vLLM serving currently needs a CQ base; one model/dataset/run, 5-point detection granularity.
- CQ update: CQ base matches BF16 (NF4 loses 12 GSM8K points), but after fine-tuning CQ's GSM8K lead vanishes (83.0 vs 83.3 vs NF4+GCMP), and both 4-bit stacks trail BF16+GCMP by 6 points.
More from Research
- NVIDIA launches Nemotron for Science: RL post-training adds 17% on drug-metabolism prediction — NVIDIAAI · 2026-11-17
- NVIDIA launches Nemotron for Science: fully open models, datasets and agent recipes — NVIDIAAI · 2026-11-17
- Top researchers are quietly stopping paper submissions, observer notes — avt_im · 2026-11-17
- MIT CSAIL's 0.016% weight-space edit lifts Qwen3.5-0.8B MATH-500 6x without retraining — RandalSchwartz · 2026-11-17
- 130-page paper introduces "bottleneck pressure": work AI can't do gains value as the rest speeds up — soumitrashukla9 · 2026-11-17
- Goodfire co-founder Tom McGrath on interpretability and steering model training — adityaag · 2026-11-17