Decoding speech perception from non-invasive brain recordings
A. Défossez, C. Caucheteux, J. Rapin, O. Kabeli, J.R. King
For: researcher
The problem
Decoding perceived speech from non-invasive brain recordings (EEG/MEG) has remained limited in accuracy because of low signal-to-noise ratio and high inter-subject variability.
Method
A contrastive-learning model was trained to align self-supervised speech representations with MEG and EEG recordings from 175 healthy volunteers listening to stories and isolated sentences, combining four public datasets.
Result
The model reached up to 70.7% top-10 segment-identification accuracy with MEG (41.3% top-1), versus only 17.7–25.7% with EEG, with the best individual participants exceeding 80% top-1 accuracy.
Why it matters
Shows that deep language models can extract rich linguistic structure from non-invasive recordings without any surgical electrode implantation.
Limitations
EEG performance remains far below MEG, MEG hardware is large and non-portable, and the model needs long per-participant recording sessions.