ViT can express any conv layer mathematically, but CNN priors still win on small edge devices

burny_tech · x · 2026-11-19

A Reddit comment argues a ViT with enough attention heads can mathematically express any convolutional layer — CNNs just fix the prior ahead of time, and with enough data that prior may be wrong. Large ViTs trained on enormous data are why they're popular.

But the tradeoff flips on small edge devices: a small ViT may lack the parameters to express what you need, so a CNN's built-in prior still holds value there.

Original post →

More from Infra

Infra channel →