Recurrent network dynamics explain the time course of perceptual grouping in natural scenes
Sami Mollard, Alekh K. Ashok, Lore Goetschalckx, Drew Linsley, Thomas Serre, Sander M. Bohte, Pieter R. Roelfsema
q-bio.NC
2026-10-05
A four-layer GammaNet on a frozen COCO FPN learns cued segmentation by spreading enhanced activity and explains 19.7% of human grouping-RT variance, against a 19.8% noise ceiling.
People take hundreds of milliseconds to bind pixels in a natural scene into objects, much longer than the first feedforward sweep through visual cortex. In curve tracing, reaction time grows roughly linearly with curve length. In monkey V1, enhanced firing arrives later in receptive fields farther along the target, and later again past a bottleneck where target and distractor nearly meet. The psychological counterpart is object-based attention spreading over a surface.
The Growth-Cone model reads that delay off receptive-field scale. Farther from a distractor, larger fields can take part and the spread speeds up. With outlines given in advance, the model fits human reaction times. Texture, internal edges, or an object that is easy to recognize, and the fit falls off. Natural images do not supply a clean contour. Older recurrent grouping models stop at those contours. Affinity methods put a clock on top of static features instead of letting network activity generate the time course.
GammaNet is a recurrent convnet with horizontal links and top-down feedback, trained here for cued segmentation. Every pixel of the cued object receives that object's category; all other pixels are background. A category target converges more readily than a binary same-or-different bit, and one mask supervises many pairwise judgments together.
Input comes from a frozen Panoptic R101-FPN pretrained on COCO. Its weights stay fixed. Clustering the features into eight groups separates parts such as heads and torsos, and a linear readout recovers outer contours and internal boundaries. Two instances of the same category still leave the cue's object unspecified. A learned projection cuts the dimension and passes the map to GammaNet.
GammaNet has four layers, coarser at each step up. Each layer is an hGRU on a retinotopic grid, with feedforward and feedback units kept separate. Horizontal connections carry long-range interactions. Inhibition produces surround suppression; excitation lets activity move across space. A linear readout on the lowest layer yields a class at every pixel.
Learning uses C-RBP. Gradients are computed from equilibrium activity, so the full trajectory does not have to be stored, and the credit signal for a weight update can sit at the synapse. Five networks trained for 21,000 steps each. The loss sees only the final labels. The images are the COCO segmentation subset that already has human same-different reaction times.
Mean IoU was 0.74±0.01 over five runs. SAM reached 0.81±0.02 on the same images. GammaNet has about 8 million parameters to SAM's 632 million, and about 5 million image-passes to 23 million once FPN pretraining is included.
Enhancement appeared near the cue and spread across the cued object, while activity on the distractor dropped. On the full test set the gap between cued and distractor conditions grew monotonically. The loss never asked for that schedule.
Deleting internal edges barely moved the final mask. On ten edited images IoU went from 0.86 to 0.88 (paired t-test, t(4)=4.47, p=0.011). Arrival time moved much more, and only when the cue sat near the edge. In the example, the edge added 4.0±1.18 timesteps up close and 0.6±0.24 timesteps when the cue was far. Across the ten images the near-cue delay was 2.3±0.33 timesteps (t(4)=6.8, p=0.0024) and the far-cue delay was 0.02±0.64 (t(4)=0.03, p=0.98), with an edge-by-distance interaction (F(1,4)=7.9, p=0.048). Near the cue the signal first traveled along the edge. Far away it crossed at once.
Driving the top hGRU layer until a linear classifier called the person an umbrella, without editing lower layers directly, stopped the crossing. Perturbing the whole object cut coverage past the edge (t(4)=64, p<0.001, Bonferroni-corrected) and did not reliably cut coverage before it (t(4)=3.23, p=0.22). Perturbing only the far side was enough (t(4)=14.67, p<0.001). Perturbing only the near side did nothing measurable (t(4)=0.97, p=1.0). Getting across an internal edge depends on category activity on the far side.
Model reaction time is the timestep when the segmentation signal reaches the second cue. The network saw only the fixation cue, and arrival times were averaged over the five runs. Held-out accuracy was 78%, against 90% for humans. The model is scored with a stricter segmentation rule, so those two numbers are not a shared leaderboard.
| Model | RT variance explained | vs 4-layer GammaNet |
| Noise ceiling | 19.8% (18.2-21.4%) | p=0.99 |
| GammaNet, 4 layers | 19.7% (12-27%) | reference |
| GammaNet, 3 layers | 4.2% (0-8.3%) | p=8.4e-7 |
| Growth-Cone, contours only | 16.8% (9.4-24%) | p=0.45 |
| Mollard et al., contours only | 15.7% (8.5-23%) | p=0.29 |
| Goetschalckx et al., contours only | 6.1% (1.1-11.1%) | p=3.9e-5 |
| Adeli et al. 2023, affinity | 7.3% (2-12%) | p=2.8e-4 |
| Adeli et al. 2026, fit to RTs | 20.5% (13-28%) | p=0.82 |
For models trained more than once, the table shows the best run. Three layers explain 4.2%. Growth-Cone and Mollard et al. were scored on contour drawings with color, texture, and internal structure removed. Adeli et al. 2023 explain 7.3% from feature affinity on natural images. Their 2026 readout, trained on the reaction times, reaches 20.5%, not reliably different from four-layer GammaNet. The GammaNet loss never included reaction time.
SAM and Mask2Former are built to emit several object masks at once. Grouping of an extended stimulus in biological vision handles one object at a time, and GammaNet spreads enhancement across one object only. The IoU gap is 0.07. The point estimate of explained grouping-time variance sits on the noise ceiling, with no reaction-time labels in the loss.
Two predictions are ready for experiment. An internal edge should slow grouping near the cue and barely matter far from it. Category information about the cued object should be decodable in early visual cortex, including V1, first near the cue and then across the object's retinotopic footprint. About 8 million parameters, and credit assignment that does not run back through the whole trajectory, put the recurrent stage closer to a rule cortex could implement. The frozen FPN in front of it is still an off-the-shelf net trained with ordinary backpropagation.
The paper states two limits. Semantic features come from an FPN that was pretrained apart and then frozen, so recurrent grouping was never learned jointly with its front end. Border ownership is absent: the model does not explicitly mark which side of a contour belongs to the object.
The noise-ceiling match is a match of point estimates, 19.7% against 19.8%. The intervals are not that tight: 12-27% for GammaNet, 18.2-21.4% for the ceiling. Reporting the best of several runs makes the comparison look stronger. The internal-edge test uses ten images, and the interaction is p=0.048. Relabeling a person as an umbrella is an adversarial perturbation, far from cortical feedback. Timesteps are not converted to milliseconds, so the link to human reaction times is about variance, not clock time.
Before any recurrent step, the FPN already supports readouts of contours, internal edges, and part clusters. GammaNet shows that spread across those edges, and the reaction-time fit, can be learned on that representation. Whether an equally local rule can learn the representation itself is untested.