SEL Weight Transformation Achieves 1.42x Speedup
torchcompiled · x · 2026-07-15
This post introduces the author's proposed SymExpLin (SEL) weight transformation, claiming up to a 1.42x wall-clock speedup during training.
The core idea is:
- The SymExp component converts additive updates into multiplicative ones, introducing learnable curvature parameters;
- It retains a standard linear path, allowing the network to learn how to weight and fuse the two paths.
The author also notes that this transformation can ultimately be fused back into standard weights for deployment.
More from Research
- AI veteran Thomas Dietterich: stop using cs.AI as your primary arXiv category — tdietterich · 2026-10-27
- arXiv advice from T. Dietterich: wrong primary category will almost surely get your paper on hold — tdietterich · 2026-10-27
- arXiv has 4,000+ papers on hold amid submission surge; moderator shares how to avoid it — tdietterich · 2026-10-27
- Community-trained MOSS Sharpness Slider LoRA: 72 photos, 500 steps, +/-1 focus control — aoleg77 · 2026-10-27
- CoreWeave, You.com and NVIDIA lift BrowseComp 22.9% in 3-week RL post-training sprint — RichardSocher · 2026-10-27
- Open-source decision models sold their own base in Red Alert 2 — then learned until no human could beat them — heathercmiller · 2026-10-27