Student-built ApexW optimizer speeds up SD 1.5 LoRA training 6.3% on a free T4
Ready_Parking_5435 · reddit · 2026-11-08
A student with only a free NVIDIA T4 (16GB) released ApexW, a custom optimizer that reduces CUDA kernel launch overhead for many small parameter updates during PyTorch training:
- Rank 16 LoRA benchmark (SD 1.5, 1000 steps): AdamW 1.92 it/s / 516.2s vs ApexW 2.04 it/s / 485.6s — 6.3% higher throughput with no extra VRAM
- At rank 32 the gain shrinks to 0.5% — optimizer overhead matters less as compute shifts to forward/backward passes
- Loss curves and sample generations match AdamW closely; no obvious quality trade-off observed
- Code on GitHub with a Colab for reproduction; seeking independent benchmarks on better hardware
More from Research
- GPT-5.2 Pro solves COLT 2022 open problem on accelerated L1-regularized PageRank — kfountou · 2026-11-08
- LDLM opens up the frozen encoder: text latent diffusion that trains its own latent space at NeurIPS 2026 — kastnerkyle · 2026-11-08
- Professor slams ICLR reviews from non-experts, calls for qualification stage at top AI conferences — xbresson · 2026-11-08
- Google AI Detects Hypertension From a Regular Smartwatch, Catching 65.8% vs 55.3% for Cuff — emmanuelvivier · 2026-11-08
- Impagliazzo's five worlds table resurfaces: which complexity universe do we live in? — matthew_d_green · 2026-11-08
- Lilly and Novo use aging clocks as trial readouts, hearts up to 4 years younger — omarabudayyeh · 2026-11-08