Feedback connections only boost robust vision when paired with dropout, study finds
maier_ak · x · 2026-10-23
Andreas Maier reviews a Patterns paper from the Hertie Institute, Tübingen, Rome, and Karl Friston's group: biological visual cortex is full of feedback, yet adding it to artificial nets has yielded mixed results. This study offers a specific answer — top-down feedback alone does little for robustness; it needs noise in the network to pay off.
- Setup: three-layer conv recurrent networks trained on CIFAR-10/100, viewing each image over 10 recurrent steps; architectures with vs. without top-down projections, with matched parameter counts and multiple instances per class.
- Dropout (silencing half the units during training) models intrinsic neuronal variability; only the feedback + dropout combination improves robustness.
- Robustness checks: Gaussian activation noise replaces dropout to rule out feedback advantages from training signals; limitations include small CIFAR models and weak transfer attacks. Code and weights are open-sourced.
Related event: Top-down feedback needs noise to boost visual robustness, study finds(2 posts)→
More from Research
- JEPA may keep unpredictable features: new paper gives set-level characterization — hisspikeness · 2026-10-23
- GAFU Uses Generation-Side Representations to Boost Fine-Grained Visual Understanding in Unified Multimodal Models — Tengju Ru · 2026-10-23
- NTU Study: Learned Unmasking Policies Improve Speed-Accuracy Trade-offs in Diffusion Language Models — nanyang-technological-university-singapore · 2026-10-23
- NBER releases 'The Economics of Transformative AI' edited by Agrawal and Brynjolfsson — JMateosGarcia · 2026-10-23
- Swapping only the eval harness moves results 30%: designing agent evals like a scientist — AI Engineer · 2026-10-23
- Jane Street and NVIDIA speed up Mamba-2 with custom cuTile kernels — blelbach · 2026-10-23