SSOG Challenges SDPA in ViTs, Offering Faster Convergence and Near-Linear Scaling
serrjoa · x · 2026-08-17
Argues that Scaled Dot-Product Attention (SDPA) is suboptimal for Vision Transformers (ViTs). Introduces SSOG (Separable Sum Of Gaussians), which achieves faster convergence, better performance, and near-linear scaling capabilities.
More from Research
- Researcher slams papers whose stacked changes beat even weak baselines barely — retr0jirachi · 2026-10-16
- FrankenSim builds a physics-based virtual orchestra, piano and drum demos live — doodlestein · 2026-10-16
- FoveaVLA replaces wrist cameras with learned foveation, hitting 70.8% success — Haoyu_Xiong_ · 2026-10-16
- ApolloPFN: a zero-shot forecasting network that natively handles exogenous covariates — burkov · 2026-10-16
- ICLR reviewer says 4/5 papers in review pile read like AI slop — berkustun · 2026-10-16
- Tsinghua's DBS learns content-adaptive frequency band splits for audio source separation — thu-ml · 2026-10-16