ViT can express any conv layer mathematically, but CNN priors still win on small edge devices
burny_tech · x · 2026-11-19
A Reddit comment argues a ViT with enough attention heads can mathematically express any convolutional layer — CNNs just fix the prior ahead of time, and with enough data that prior may be wrong. Large ViTs trained on enormous data are why they're popular.
But the tradeoff flips on small edge devices: a small ViT may lack the parameters to express what you need, so a CNN's built-in prior still holds value there.
More from Infra
- India's Sarvam AI denies using subsidised GPUs to host Chinese open models — HimanshiET · 2026-11-19
- FT: Anthropic needs ~$602B yearly revenue to earn 20% on $1T compute commitments — rohanpaul_ai · 2026-11-19
- FT: Anthropic needs ~$602B yearly revenue to earn 20% on ~$1T compute commitments — rohanpaul_ai · 2026-11-19
- Linux Mint website goes dark, showing hosting placeholder pages — kingslyj · 2026-11-19
- Tencent Hunyuan's Hy4-Flash engine PR reveals tensor parallel, MTP drafts, CPU expert lane — TencentHunyuan · 2026-11-19
- Commentator calls Jensen Huang's China compute claim bullshit: won't outpace rest of world — teortaxesTex · 2026-11-19