New ApexW optimizer claims lower loss than AdamW with half the VRAM of bitsandbytes

omendUUde · reddit · 2026-11-18

A Redditor released ApexW, a from-scratch training optimizer claiming to beat both AdamW on convergence and bitsandbytes on memory:

The author admits these are only 60-100 step sanity checks; full multi-epoch fine-tunes and downstream evals are next. Promising early numbers, but unproven over long training runs.

Original post →

More from Infra

Infra channel →