MEM v3 Open-Sourced: Dynamic Batch Scaling Against CUDA OOM, Survives +10GB VRAM Shocks
uBazzyZ- · reddit · 2026-09-27
The author open-sourced MEM v3, a "memory governor" for PyTorch that tackles CUDA out-of-memory crashes during long training and fine-tuning runs by watching VRAM and throughput in real time and dynamically adjusting batch size and gradient accumulation without stopping the process.
Key features:
- Dynamic lane switching: scales batch size up or down in milliseconds based on actual GPU memory pressure, instead of keeping batch sizes overly small out of caution
- Chaos resistance: tested against sudden +10GB VRAM allocation shocks without crashing
- Crash-proof checkpoints: atomic file replacement with SHA-256 checks across rotating slots, so power outages won't corrupt saved weights
- Live telemetry: built-in local web dashboard tracking loss, throughput, and lane switches
A zero-setup Colab demo is available; repo: nobazzy/mem-llm-orchestrator.
More from Infra
- Harvard Puts Full ML Systems Curriculum CS249r Online for Free — techNmak · 2026-09-27
- SpaceX's vertical empire: from rockets to Colossus AI compute powering Grok — XFreeze · 2026-09-27
- Token-efficient reasoning model Swift-1.5-Qwen3.8-27B trends on Hugging Face — ukisai · 2026-09-27
- Hacker uses NVIDIA DGX Station to run local models for CAD, invites SF folks to join — hudzah · 2026-09-27
- Yacine goes 'absurdly bullish' on TensorTorrent in rare endorsement — yacineMTB · 2026-09-27
- DeepSeek V5 leak: 2T parameters, reportedly trained fully on Huawei Ascend chips — teortaxesTex · 2026-09-27