EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces

Vernon J. Lawhern, Amelia J. Solon, Nicholas R. Waytowich, Stephen M. Gordon, Chou P. Hung, Brent J. Lance

★ Must read November 2016

TL;DR EEGNet is a small convolutional network for EEG that learns temporal filters, then frequency-specific spatial filters, using depthwise and separable convolutions. With about 800 to 2,300 parameters, two orders of magnitude fewer than DeepConvNet, it matches or beats reference methods across four BCI paradigms when training data is limited.

What they do

EEG-based BCIs usually rely on feature extractors and classifiers hand-tailored to one signal type. EEGNet is a single compact CNN meant to work across paradigms, train on limited data, and keep its learned features interpretable. It is tested on four datasets: P300, error-related negativity (ERN), movement-related cortical potentials (MRCP) and sensory motor rhythms (SMR), both within-subject and cross-subject.

How it works

Block 1. F1 temporal convolutions of length 64 samples (half the 128 Hz sampling rate) act as band-pass filters. A depthwise convolution across all C channels then learns D spatial filters per temporal filter, inspired in part by filter-bank common spatial patterns (FBCSP).
Block 2. A separable convolution first summarizes each feature map over 500 ms, then mixes the maps with pointwise convolutions. The output goes straight to a softmax, with no dense layer.
Training. Adam, cross-entropy loss, batch normalization, ELU, dropout (0.5 within-subject, 0.25 cross-subject), and class weights for imbalanced data.
Interpretation. Three views of a trained model: time-frequency summaries of spatial-filter outputs, direct plots of kernel weights, and single-trial relevance with DeepLIFT.

Results

EEGNet-4,2 and EEGNet-8,2 have 796 to 2,322 trainable parameters, against 152,219 to 175,727 for DeepConvNet and 40,644 to 104,722 for ShallowConvNet.
Within-subject: both EEGNet models significantly outperform DeepConvNet, ShallowConvNet and xDAWN+RG on MRCP; on P300 all CNNs beat xDAWN+RG; on SMR, EEGNet-8,2 is similar to FBCSP.
Cross-subject: EEGNet and DeepConvNet perform similarly, and there is no significant difference among the CNNs on SMR.
Spatial filters learned near 12 Hz correlate with FBCSP’s 8-12 Hz filters (correlations of 0.93, 0.83 and -0.93). In a P300 ablation, a model keeping only Filter 4 still reaches 0.8637 AUC, against 0.9054 for the full model.

Limitations

On cross-subject ERN, the traditional xDAWN+RG pipeline outperforms every CNN, EEGNet included. The larger EEGNet-8,2 gives no statistically significant gain over EEGNet-4,2. On SMR, EEGNet matches FBCSP and ShallowConvNet without exceeding them. The severe underperformance of ShallowConvNet on within-subject MRCP is left unexplained.

Figures

From Figure 1: Overall visualization of the EEGNet architecture. The network starts with a temporal convolution (seco... — From Figure 1: Overall visualization of the EEGNet architecture. The network starts with a temporal convolution (second column) to learn frequency filters, then uses a depthwise convolution (middle column), connected to each feature map individually, to learn frequency-specific spatial filters. The separable convolution (fourth column) learns a temporal summary for each feature map individually, then a pointwise convolution learns how to mix the feature maps together.

From Figure 2: 4-fold within-subject classification performance for the P300, ERN and MRCP datasets for each model, a... — From Figure 2: 4-fold within-subject classification performance for the P300, ERN and MRCP datasets for each model, averaged over all folds and all subjects. There is minimal difference between the CNN models for the P300 dataset, while in the MRCP dataset both EEGNet models outperform all other models.

From Figure 4: Cross-subject classification performance for the P300, ERN and MRCP datasets for each model, averaged ... — From Figure 4: Cross-subject classification performance for the P300, ERN and MRCP datasets for each model, averaged for 30 folds. For the P300 and MRCP datasets there is minimal difference between the DeepConvNet and the EEGNet models, with both models outperforming ShallowConvNet. For the ERN dataset the reference algorithm (xDAWN + RG) significantly outperforms all other models.

From Figure 7: Features derived from a within-subject trained EEGNet-8,2 model for Subject 3 of the SMR dataset. Each... — From Figure 7: Features derived from a within-subject trained EEGNet-8,2 model for Subject 3 of the SMR dataset. Each of the 8 columns shows the learned temporal kernel for a 0.25 second window (top) with its two associated spatial filters (bottom two).

Abstract

Brain computer interfaces (BCI) enable direct communication with a computer, using neural activity as the control signal. This neural signal is generally chosen from a variety of well-studied electroencephalogram (EEG) signals. For a given BCI paradigm, feature extractors and classifiers are tailored to the distinct characteristics of its expected EEG control signal, limiting its application to that specific signal. Convolutional Neural Networks (CNNs), which have been used in computer vision and speech recognition, have successfully been applied to EEG-based BCIs; however, they have mainly been applied to single BCI paradigms and thus it remains unclear how these architectures generalize to other paradigms. Here, we ask if we can design a single CNN architecture to accurately classify EEG signals from different BCI paradigms, while simultaneously being as compact as possible. In this work we introduce EEGNet, a compact convolutional network for EEG-based BCIs. We introduce the use of depthwise and separable convolutions to construct an EEG-specific model which encapsulates well-known EEG feature extraction concepts for BCI. We compare EEGNet to current state-of-the-art approaches across four BCI paradigms: P300 visual-evoked potentials, error-related negativity responses (ERN), movement-related cortical potentials (MRCP), and sensory motor rhythms (SMR). We show that EEGNet generalizes across paradigms better than the reference algorithms when only limited training data is available. We demonstrate three different approaches to visualize the contents of a trained EEGNet model to enable interpretation of the learned features. Our results suggest that EEGNet is robust enough to learn a wide variety of interpretable features over a range of BCI tasks, suggesting that the observed performances were not due to artifact or noise sources in the data.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong