Decoding silent reading from non-invasive EEG
Ingo Marquardt, Anthilia Alchanat, Priyanka Jain
cs.LG, q-bio.NC
2026-08-20
A CLIP decoder on ~49 h of 19-channel dry EEG recovers open-vocab silent-reading words: within-run top-10 gain averages 6.7 pp and still scales log-linearly.
Invasive BCIs now decode inner speech from motor cortex at conversational rates over a 125,000-word vocabulary. Scalp EEG has not caught up. The bottleneck is labels: a timestamped transcript of someone's spontaneous inner monologue cannot be collected. Cued repetition is slow, boring, and unauditable. Retrospective reports smear events across hundreds of milliseconds, which is fatal for a signal whose structure lives on a tens-of-milliseconds scale.
nubrain treats silent reading as the proxy that can actually scale. One participant, 19 dry electrodes, rapid serial visual presentation of continuous narrative: about 49 hours and 240,000 word presentations. The question is how much lexical and semantic information a contrastive decoder can pull from that EEG.
The text is Sherlock Holmes prose, about 600 words per run, with the story continuing across runs. Font, size, colour, and letter spacing are redrawn on every trial so word identity is not tied to a fixed visual template. Each word stays on screen for 500–900 ms, scaled by character count, plus up to 100 ms of jitter. The first 171 runs used a 100 ms blank between words; later runs used 0 ms.
Preprocessing is deliberately light: linear detrend, 50 Hz notch, 1–80 Hz band-pass, drop any trial with a sample above 100 μV. The analysis window has a fixed length so the model cannot read word length off the epoch duration.
A dual-path convolutional encoder produces a 256-d feature per trial: a global path collapsed by attention pooling, and a local path that keeps six coarse time bins. An optional four-layer causal transformer then looks only at earlier trials in the same run. Targets are Llama-3.1-8B hidden states. Layer 0 is the input embedding (word identity, no narrative). Layer 20 is a mid-depth state that carries preceding context. Both sides are projected into a shared 256-d L2-normalised space and trained with a symmetric CLIP loss.
Scoring is word-grouped top-10 retrieval inside 512-trial pools, minus an empirical permutation baseline, so chance is 0 percentage points. The primary metric is within-run gain: every candidate comes from the same passage, so cross-run topic tracking cannot help. A run-mate swap of predictions measures the passage-identity component. Masking the current trial's EEG with a learned [MASK] token isolates the transformer's positional embedding, the only trial-varying non-neural input.
Across 576 Sweep-1 fits, mean within-run gain was 6.7±2.7 pp, and every fit was positive; all 192 Sweep-3 fits were positive too. Pre-training noise on this metric sits around ±0.2 pp. The observed minimum of 1.4 pp is an order of magnitude above that.
| Setup | Within-run (pp) | Overall (pp) | Context-tracking (pp) |
| L0, no transformer | 7.5±2.0 | 7.9±1.9 | 1.0±0.5 |
| L0 + transformer | 6.2±2.2 | 7.9±2.3 | 1.9±0.8 |
| L20, no transformer | 5.5±2.5 | 11.0±3.2 | 4.6±1.6 |
| L20 + transformer | 7.8±3.3 | 19.8±6.1 | 5.9±1.5 |
With non-contextual targets and no sequence model, almost all of the gain is word-level: context tracking is 1.0 pp. L20 plus a transformer posts the largest overall gain, but about 30% of 19.8 pp (5.9 pp) survives a run-mate swap and is therefore passage identity. Position probes cut further: of 7.8 pp within-run, 2.7 pp is the positional prior, 1.9 pp is preceding-word EEG, and 3.1 pp is the current trial. After subtracting position, lexical decoding sits at 5.1 pp, which does not beat the 7.5 pp of the L0, no-transformer cell.
The signal is not confined to frequent function words. Sweep 1, L0 no transformer: rare / mid / frequent = 3.2 / 3.2 / 8.1 pp. L20 plus transformer: 6.0 / 6.0 / 8.1 pp.
A scaling sweep held the validation set fixed and grew the training set from 10% to 100% of runs. L20 plus transformer: within-run gain 6.2 → 15.0 pp, slope +8.7 pp per decade of data, R²=0.98. L0, no transformer: 5.0 → 10.1 pp, slope +4.8. At 49 h the curves are still rising.
Removing O1, O2, T5, and T6 dropped within-run gain from 9.2 to 6.3 pp (−32%) while context tracking barely moved (−4%). About two thirds of the word-level gain survives. EEG spatial resolution is poor, so this is not a clean non-visual proof. A 0.7 s window beat 0.5 s (8.3 vs 6.9 pp); 0.9 s added nothing. Training-time window jitter of ±100 ms cut the gain by 39%.
This is a data-scaling step toward a non-invasive inner-speech BCI, not inner-speech decoding. Invasive work already suggests shared representations across reading, listening, speaking, and inner speech. Whether that sharing survives scalp EEG SNR is untested here. What is established: open-vocabulary word-level information is recoverable from dry-electrode EEG during silent reading of narrative prose, in one densely sampled person, at a data volume that is still on the rising part of a log-linear curve.
The practical recipe is CLIP alignment to pretrained LLM states, plus an evaluation that splits topic tracking and positional priors from lexical decoding. Quote the 19.8 pp overall number as word-level accuracy and the paper is being misread. Hardware is a Wearable Sensing DSI-24: 19 dry channels at 600 Hz, chosen for long comfortable sessions rather than wet-electrode density.
Every number comes from one person. About 60 other participants have one or two sessions each; that cross-subject analysis is not in this report. Checkpoints and configurations were picked on the validation set, so absolute magnitudes are upper bounds.
RSVP pins words at screen centre and removes saccades and parafoveal preview. Timing is clean; the task is not natural reading. The decoder retrieves from a supplied candidate set; it does not generate text. The visual-form confound is still open. Randomised typography kills template matching, not the systematic link between letter string and word identity. Signal that survives occipital-channel removal can still be volume-conducted visual activity. The authors treat a future listening condition as the decisive test. It is not in this paper.
Inner-speech transfer remains a later stage: show that reading-and-listening training extracts a modality-general language component, then fine-tune on a smaller inner-speech set.