NVIDIA's LoGRA cuts RL training memory by 45.7%, using less than LoRA
omarsar0 · x · 2026-10-13
NVIDIA published LoGRA, a memory-efficient RL post-training method for LLMs. It compresses weight gradients with rank-256 random projections redrawn each step (never building full gradients), plus predicted-KL step control to keep updates within a KL budget.
Key results:
- Updates all model weights with less memory than LoRA: at 1.5B, LoGRA averages 7.2 GiB vs 13.2 GiB for rank-256 LoRA
- Pass@4 essentially unchanged (81.8% vs 81.4%), Pass@1 about 2 points lower
- Up to 45.7% average memory savings; enables stable training of a 27B model for 1,100+ steps on a single 8-GPU node where dense Adam OOMs
Code is open-sourced in the Molt library.
Related event: NVIDIA's LoGRA Cuts RL Training Memory by 45.7%(2 posts)→
More from Infra
- Running local GLM 5.3 Flash that calls Claude or Codex — fans reveal the handoff — HankYeomans · 2026-10-13
- AI demand sends Nanya's DRAM margins to 85.7%, Q3 profit tops past 7 years combined — BenBajarin · 2026-10-13
- Engineer Shares a Blog on Scaling AI Inference Stacks for Many Customers — abhijithneil · 2026-10-13
- Intel's Crescent Island GPU lands in D-Series lineup with up to 480GB of LPDDR5X — Monad_Maya · 2026-10-13
- NVIDIA chief scientist headlines silicon-to-systems panel at Applied Materials Epic launch — BenBajarin · 2026-10-13
- Zuckerberg: Owning data centers is a financial edge as depreciated H100s gain value — BrentM_BuildGuy · 2026-10-13