Removing Timing Shortcuts Improves Non-Invasive Brain-to-Text
Dulhan Jayalath, Oiwi Parker Jones
cs.LG, q-bio.NC
2026-10-01
Overlapping word windows leak word duration: joint decoding gains reproduce without brain data; independent decoding plus aggregation and an LLM prior cuts WER to 36.6%.
Non-invasive speech BCIs try to turn MEG/EEG signals recorded outside the skull back into words. The current paradigm came from d'Ascoli et al. (Nature Communications, 2025): cut a three-second window of brain activity at each word's onset, encode all windows in a sentence jointly with a transformer, and predict every word at once. Joint decoding lifted word classification by 50% on average, and the recipe has since spread through a dozen-odd follow-up papers, evaluations, and benchmarks.
Most of that lift does not come from the brain. Windows are longer than words, so adjacent windows overlap, and the same samples sit at different offsets in neighbouring inputs. Those offsets reveal the interval between word onsets, which tracks word duration at r = 0.90. Duration is informative about identity (“the” is short, “supercalifragilisticexpialidocious” is not), so the network can narrow the candidates from timing alone. Textbook shortcut learning.
Diagnosis first, then a one-line fix.
The diagnosis is a synthetic control. Replace the MEG with a synthetic continuous signal unrelated to the stimulus, but keep cutting windows at the true word onsets. The overlap structure survives; the brain information is gone. Joint decoding scores 22.3% balanced word accuracy on real MEG, 22.0% on the synthetic signal, and 22.9% when the model receives only the encoded interval between words. Make neighbouring windows share no samples and accuracy falls to 5.8%, below the 9.5% of isolated decoding on real MEG. In this dataset, 99.9996% of adjacent word pairs have overlapping windows sharing 90.6% of their samples on average. The effect replicates across 32 more subjects and two more datasets. Running the same control on Brain2Qwerty, the typed-sentence decoder, leaves synthetic inputs at 6.8% versus 34.9% with real MEG, so that line does not ride the shortcut.
The fix: decode each window independently, so the network never sees its sentence neighbours. The encoder shrinks to a small CNN with temporal pooling plus a residual MLP, about 20M parameters versus roughly 200M for the original. Training keeps the contrastive objective that pulls each window embedding toward the T5-large embedding of its word; at test time, cosine similarities against a candidate vocabulary become probabilities through a temperature.
With the shortcut gone, two established techniques start to pay off. Aggregation: when a word has k recorded responses, average the predicted embeddings and let their agreement sharpen the score. Language prior: add λ times the log likelihood of a frozen Qwen3-8B-Base to the neural scores and beam-search over a 92-word vocabulary, with a fixed prompt describing a patient making a short request to hospital staff. The full recipe is SimpleB2T.
Diagnosis (subject 0 of LibriBrain100, 50-word vocabulary, balanced top-1 accuracy, chance about 2%):
| Input | Decoding | Accuracy |
| Real MEG | joint | 22.3% |
| Synthetic, no brain info | joint | 22.0% |
| Inter-word interval only | joint | 22.9% |
| Real MEG | isolated | 9.5% |
| Synthetic, no overlap | isolated | 5.8% |
Clinical communication benchmark (200 sentences, 92-word vocabulary, five observations per word, Core subset):
| Method | WER | Sentence match rate |
| LM only | 73.8% | 2.0% |
| d'Ascoli + LM | 79.4% | 2.4% |
| SimpleB2T (k=1) | 65.6% | 6.4% |
| SimpleB2T (k=5) | 36.6% | 26.0% |
Brain alone reaches 74.3% WER and the LM alone 73.8%; combined, 36.6%. The complementarity is example-specific: noise matched to MEG statistics worsens WER to 97.1%, and shuffling which embedding belongs to which word gives 88.3%. Across parts of speech, neural evidence carries verbs while the LM carries function words. The 36.6% approaches the 25.6% median WER of the 2021 invasive implant result from Moses et al., under different conditions: a paralysed patient attempting speech, a 50-word vocabulary, 50 test sentences.
Half of the paper is a measurement warning aimed at a whole line of work. Of the nine datasets analysed by d'Ascoli et al., only the two reading paradigms that give every word equal presentation time are immune, and those are exactly the two where joint decoding brought no gain in the original paper. Any pipeline that jointly models aligned, overlapping windows carries the same risk, and the synthetic control that exposes it costs almost nothing to run.
The other half is a cheap recipe. Once words are decoded independently, repetition and an LLM prior, both standard elsewhere, deliver large gains from five observations per word, against the 100 observations per phoneme averaged in the 2025 PNPL competition. The neural model is 20M parameters next to an off-the-shelf LM, and code plus benchmark are public.
The authors state the main ones themselves: the setting is word-aligned, so word onsets and sentence lengths are given; the task decodes perceived speech, not imagined speech, and they do not claim a practical BCI; the benchmark has no clinical validation.
Context the numbers need: 36.6% WER was earned inside a 92-word, formulaic clinical domain where the linguistic prior is strong. Swapping the clinical prompt for a generic one moves WER from 36.6% to 55.2%, so the headline figure is sensitive to how the prior is set. Better sentences are often already inside the beam; oracle selection among candidates would reach 16.4%, so a large share of the remaining error sits in ranking rather than neural decoding. The benchmark sentences were drafted with GPT-6 assistance (disclosed and manually reviewed), and each sentence position is assembled from separately recorded word occurrences rather than continuous speech. Five observations per word also means five repetitions per word in practice, so the recording burden has not disappeared.