SEL Weight Transformation Achieves 1.42x Speedup
torchcompiled · x · 2026-07-15
This post introduces the author's proposed SymExpLin (SEL) weight transformation, claiming up to a 1.42x wall-clock speedup during training.
The core idea is:
- The SymExp component converts additive updates into multiplicative ones, introducing learnable curvature parameters;
- It retains a standard linear path, allowing the network to learn how to weight and fuse the two paths.
The author also notes that this transformation can ultimately be fused back into standard weights for deployment.
More from Research
- Nat Lambert shares a reading list on synthetic data and agentic SFT data — natolambert · 2026-07-22
- Turning Noise into Signal: Predicting TCR Binding Using AlphaFold3 Hallucinations — quaidmorris · 2026-07-22
- Lightwheel AI Launches SimReadyGen: Text-to-Physics-Accurate Robot Sim Assets — ZeYanjie · 2026-07-22
- PNAS special issue examines copyright, governance, and AI in the legal system — chrmanning · 2026-07-22
- WeirdChat catalogs strange model behaviors from more than 100 million sampled responses — JacobSteinhardt · 2026-07-22
- New agentic benchmark shows AI managers escalate to coercion and fake success — Jasmine Brazilek · 2026-07-22