Transformer effective receptive field far smaller than theoretical, echoing 2017 CNN finding
CSProfKGD · x · 2026-10-22
A CS professor notes that new work showing the effective receptive field in transformers is significantly smaller than the theoretical one mirrors an earlier finding in convolutional networks. He cites Luo et al.'s classic paper (arXiv:1701.04128), which showed that effective receptive fields in deep CNNs follow a Gaussian distribution and occupy only a fraction of the full theoretical field, analyzed the effects of activations, dropout, sub-sampling and skip connections, and suggested ways to enlarge them.
Related event: Study Finds DiT Effective Receptive Field Far Smaller Than Theoretical(3 posts)→
More from Research
- Yoav Goldberg shares review stack: 4 of 5 submissions rejected as slop — yoavgo · 2026-10-22
- AI Benchmarks Are Climbing the Wrong Hills: Contrary Research's Deep Dive on Benchmaxxing — rseroter · 2026-10-22
- ICLR reviewer avoids LLM-sounding titles when bidding: no bidders may signal a bad paper — gsarti_ · 2026-10-22
- Ataraxis Breast: AI test estimates breast cancer recurrence risk from pathology slides — kjgeras · 2026-10-22
- Agent rewrites optimizer source code mid-training to build a reactive optimizer — giffmana · 2026-10-22
- Recursia hits 28-53% zero-shot on LIBERO-Pro Long, far ahead of ASPIRE and π0.5 — jiqizhixin · 2026-10-22