Brain-computer interfaces
Decoding intent from brain and muscle signals, from EEG and implanted arrays to wrist EMG, to control robots and computers.
Start here: seminal papers
Reach and grasp by people with tetraplegia using a neurally controlled robotic arm★ Must read
Nature, May 2012
Two people paralyzed by brainstem stroke reached for and grasped foam balls with a robot arm, driving its 3D velocity and hand closing with signals from a 96-channel array in motor cortex. Grasp success reached 62.2% of trials for one participant, and the other drank coffee from a bottle in 4 of 6 attempts.
EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces★ Must read
November 2016
EEGNet is a small convolutional network for EEG that learns temporal filters, then frequency-specific spatial filters, using depthwise and separable convolutions. With about 800 to 2,300 parameters, two orders of magnitude fewer than DeepConvNet, it matches or beats reference methods across four BCI paradigms when training data is limited.
Deep learning with convolutional neural networks for EEG decoding and visualizationSeminal
March 2017
Convolutional networks trained end-to-end on raw EEG decode imagined and executed movements as well as filter bank common spatial patterns (FBCSP), the standard hand-engineered method, with mean accuracy of 84.0% against 82.1%. New visualizations show the networks learn to use alpha, beta and high-gamma band power.
BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG dataSeminal
January 2021
BENDR adapts wav2vec 2.0 to EEG, pretraining a convolutional encoder and a transformer with a masked contrastive task on about 1.5 TB of unlabelled clinical recordings. The single pretrained model transfers to five downstream EEG tasks recorded with other hardware and subjects, reaching 86.7 balanced accuracy on left/right motor imagery.
Representation learning for neural population activity with Neural Data TransformersSeminal
August 2021
The Neural Data Transformer (NDT) replaces the recurrent network in LFADS-style models of neural spiking with a BERT-style Transformer encoder trained by masking timesteps. It matches AutoLFADS on monkey reaching data (0.918 vs 0.915 R2 for hand-velocity decoding) while inferring in 3.9 ms, 6.7x faster.
Decoding speech perception from non-invasive brain recordingsSeminal
August 2022
A convolutional network trained with a contrastive loss matches 3 s of MEG or EEG to wav2vec 2.0 features of the speech a person is hearing. Across 175 volunteers it picks the right segment out of more than 1,000 with up to 41% top-1 accuracy for MEG.
BrainBERT: Self-supervised representation learning for intracranial recordingsSeminal
February 2023
A Transformer pretrained by masked spectrogram reconstruction on 43.7 hours of unannotated intracranial recordings from 10 subjects. A linear decoder on its fine-tuned embeddings reaches 0.83 average ROC-AUC on four audio and speech tasks, against 0.59 for a linear decoder on the raw signal.
A high-performance speech neuroprosthesisSeminal
Nature, August 2023
An RNN reads spiking activity from microelectrode arrays in ventral premotor cortex and, with a language model, turns attempted speech into text for a participant with ALS who can no longer speak intelligibly. She reached 62 words per minute, with a 23.8% word error rate on a 125,000-word vocabulary and 9.1% on a 50-word one.
Brain decoding: toward real-time reconstruction of visual perceptionSeminal
October 2023
Decodes the images a person is seeing from MEG recordings by aligning brain activity to pretrained image embeddings and feeding the predictions to a diffusion model. The deep MEG decoder retrieves images 7X better than linear decoders, reaching about 70% top-5 accuracy on a 200-image test set.
A Unified, Scalable Framework for Neural Population Decoding★ Must read
October 2023
POYO turns every spike into a token with a learned unit embedding and a timestamp, so one PerceiverIO-style transformer can train on recordings with different neurons. Trained on 27,373 units from seven monkeys, it adapts to unseen sessions and animals by learning only new unit embeddings or by finetuning, reaching R2 0.9482 on NLB-Maze against 0.9062 for the best prior baseline.
NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities★ Must read
November 2023
NOIR lets a person command a robot arm or mobile manipulator through non-invasive EEG by decoding what object to act on, how, and where, then running a parameterized primitive skill. Three users completed 20 long-horizon household tasks in 1.83 attempts on average.
Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCISeminal
May 2024
LaBraM is an EEG foundation model, pre-trained by masked prediction of discrete neural tokens on about 2,500 hours of EEG from around 20 datasets with differing electrode layouts. Fine-tuned, it beats all compared baselines on abnormal detection, event classification, emotion recognition and gait prediction, e.g. 0.64 vs 0.53 balanced accuracy on TUEV with the 5.8M-parameter Base model.
emg2qwerty: A Large Dataset with Baselines for Touch Typing using Surface ElectromyographySeminal
October 2024
A public dataset of wrist sEMG recorded while 108 people touch type, 346 hours in 1,135 sessions, with keylogger ground truth. A speech-recognition-style baseline decodes keystrokes at 6.95% character error rate once personalized, but over 50% on unseen users.
Brain-to-Text Decoding: A Non-invasive Approach via TypingSeminal
February 2025
Brain2Qwerty decodes the sentences people type from non-invasive brain recordings, using a convolutional module, a transformer and a character-level language model. With MEG it reaches a 32% character error rate on average (19% for the best participant), against 67% with EEG.
A generic non-invasive neuromotor interface for human-computer interactionSeminal
Nature, September 2025
A dry-electrode wristband reads muscle signals (sEMG), and networks trained on data from thousands of people decode wrist movement, finger gestures and handwriting for new users without calibration. Naive users reach 0.88 gesture detections per second and handwrite at 20.9 words per minute; 20 min of a user's own data cuts handwriting errors a further 16%.
Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.