NVIDIA Scales Matrix Factorization to 1M×1M, Doubling Single-GPU Capacity
marc_stampfli · x · 2026-07-30
NVIDIA researchers published a study on accelerated Symmetric Non-negative Matrix Factorization (SymNMF), successfully scaling computations to 1M×1M matrices across 64 GB200 GPUs.
Technical Breakthroughs
- Memory Optimization: A trace-identity reformulation eliminates all $n \times n$ intermediates, roughly doubling single-GPU capacity to handle $n \approx 10^5$.
- Algorithm Evaluation: The team systematically tested 7 algorithm families (over 30 configurations). At the 1M scale, 5 AdaGrad-family methods still successfully converge.
- Performance: Block-SVRG AdaptGrow wins on the ill-conditioned tail-dependence spectrum due to lower per-iteration cost, while full-batch AdaGrad performs best on the dominant-low-rank correlation spectrum.
More from Infra
- China's AI boom drives up to 660% profit surges for domestic chip firms; GPU maker MetaX turns profitable — pstAsiatech · 2026-10-16
- Qdrant tests Perplexity's visual PDF embeddings: 21x smaller vectors, 99.9% after rescoring — jrxna · 2026-10-16
- When Does an Agent Actually Need an API Gateway? Reddit Weighs Direct Calls vs. Unified Endpoints — TurbulentStiffness · 2026-10-16
- Taiwan to build $4B AI manufacturing complex in Texas to back Nvidia and TSMC — Polymarket · 2026-10-16
- When the KV Cache Is Slower Than the GPU: Remote KV Tiers Backfire in Production — MultiverseComputingCAI · 2026-10-16
- GitHub rewrote the 800k-line Copilot runtime in Rust using AI agents, cutting CPU and memory by over 90% — bibryam · 2026-10-16