What masking geometry works best for EEG foundation models?
Pierre Guetschel, Bruno Aristimunha, Yassine El Ouahidi, Arnaud Delorme, Thomas Moreau, Michael Tangermann
cs.LG
2026-09-27
With everything else fixed, a sweep of 58 EEG models finds MAE and JEPA agree the best mask is 2s×9cm; at that setting a 12.7M model matches REVE-Base for about 14 H100-hours.
EEG foundation models have multiplied: REVE, LaBraM, BIOT, BrainBERT, Neuro-GPT, nearly all pre-trained with masked prediction, hiding part of the input and training the network to recover it from the visible rest. The mask defines what gets predicted and from which context, which makes it a central design choice. Yet every new model ships a new backbone, corpus, objective, and mask at once, so nobody can attribute downstream gains to any single component. Vision SSL treated mask shape as a first-class ablation axis years ago (MAE swept ratios, I-JEPA and V-JEPA ablated block shapes). EEG had no equivalent. This paper runs the missing controlled study instead of proposing another model.
Fix everything, vary two axes. Masking strategies from the literature are unified into three parameters: temporal block length L, spatial radius r, and mask ratio ρ. The radius r is a spherical cap on the scalp around an electrode; L is how long the hidden time segment is. Extremes recover known strategies: (L=1, r="one") is random patch masking, r="all" gives temporal blocks, L=33 masks whole channels. A block count from an inclusion-exclusion formula keeps the expected ratio at 0.55, taken from REVE.
The grid sweeps L∈{1,2,4,8,16,33} and r∈{"one",6,9,12,"all"} cm, drops the degenerate full-mask cell for 29 configurations, and replicates everything under two frameworks: MAE (input-space reconstruction) and JEPA (latent prediction against an EMA teacher). That yields 58 models sharing the REVE-Small backbone (12.7M parameters), the 4.4 TB open subset of the REVE corpus, and the full training protocol, so any downstream difference traces to the mask or the framework. Evaluation runs on the 12 OpenEEGBench datasets with a linear probe (frozen features, a linear head only), 5 seeds, and a two-level bootstrap for significance.
The interpretation anchor is EEG's redundancy structure: within-channel autocorrelation falls below R²≈0.2 within about 2 s, while between-channel correlation decays slowly across the scalp due to volume conduction. Longer L drains temporal context; larger r drains spatial context, forcing the network to integrate along the other axis.
| Comparison | Metric | Result |
| MAE vs JEPA | Optimal mask | Both converge on (L=2 s, r=9 cm) |
| Best config vs REVE-Base (69M params, 6 TB) | OpenEEGBench linear probe | Matches from epoch 4 onward |
| Best-minus-worst within framework | Normalized score | 0.32 (MAE) vs 0.77 (JEPA), a 2.4× gap |
| Statistically tied with the winner | Bootstrap test | 11 MAE and 9 JEPA configurations |
| Pre-training cost | Compute | 14 H100-hours per run vs 260 A100-hours for REVE-Base |
Failure modes are shared: r="one" (the masked patch is trivially predictable from same-channel neighbors, too little learning signal), r="all", and long blocks L∈{8,16} at moderate radii. L=33 is the exception: masking a whole channel turns the task into pure cross-channel prediction, hard but tractable, and MAE lands three L=33 cells in the statistically tied cluster. Outside the failure modes the mask choice matters surprisingly little; the tied cluster spans r∈{6,9,12} cm and L∈{1,2,4}.
JEPA has one failure at r="all" that MAE structurally cannot have, named bias-inflation collapse: from epoch 3, more pre-training makes downstream performance worse. Because JEPA's targets come from an EMA teacher, the predictor can match them from positional embeddings alone, via a channel-position lookup table, ignoring the EEG content. The per-channel feature mean inflates 2-3× while the input-driven residual shrinks 5-10×. Standard collapse detectors (variance, EMA divergence, loss thresholds) all miss it, and VICReg- or C-JEPA-style regularizers do not rescue it.
Practitioners get an actionable default, (L=2 s, r=9 cm), with no gain from further tuning inside r∈{6,9,12} cm × L∈{1,2,4}, and two regimes to avoid. JEPA users need more care: a bad mask costs up to 0.77 normalized score, 2.4× the MAE penalty.
The general principle travels: calibrate the mask to the signal's redundancy structure and mask along the axis where information is shared but not trivially accessible. EEG is spatially redundant and temporally less so, which is exactly where the optimum lands.
The 14 H100-hour budget, open code, and 58 checkpoints lower the entry bar for EEG foundation-model work and invite scaling-law studies. Bias-inflation collapse is a broader warning for JEPA on multichannel signals: in-training metrics cannot validate pre-training quality here; only a downstream probe exposes the failure.
From the authors: the core grid uses one backbone and one corpus dominated by clinical EEG and BCI paradigms, so transfer to other corpora, hyperparameters, and consumer-grade EEG is unverified; ρ is fixed at 0.55 and interacts with (L, r), with no joint sweep; one of the 12 downstream datasets (PhysioNet-MI, 48.5 of 34,000 hours) sits inside the pre-training corpus, though REVE-Base additionally saw TUAB and TUEV and still scores higher on all three, so the comparison is not inflated; OpenEEGBench overlaps with other EEG models' corpora, so collective benchmark overfitting cannot be ruled out.
Two more caveats. The compute comparison is not scale-matched (12.7M vs 69M parameters, 4.4 vs 6 TB); the savings come from the whole pipeline, not the mask alone, as the authors state. Every conclusion rests on linear probing, and whether fine-tuning tells the same story is not shown in the main text. OpenEEGBench is also co-developed by authors of this paper, disclosed in a footnote, but worth knowing when weighing the benchmark choice.