The prediction of cognitive tasks from electroencephalography (EEG) signals have allowed to discriminate the cognitive states emitted by the subjects and to carry out robust monitoring of cognition; a fact that is associated with the attention and performance of an individual's behavior, allowing greater control in the experiments. The objective of this work is to perform the prediction of tasks in the function of the auditory steady-state response (ASSR). Twenty-two subjects underwent three types of tasks: counting, reading and rest, accompanied by a constant stimulus. Images were obtained from the Inter Trial phase coherence (ITPC) to train classification algorithms based on convolutional neural networks (CNN) in order to separate the tasks performed by the subjects. Performance evaluation of the classification algorithm shows very good separation between count, read and rest with an AUROC of 0.95. This is significantly better than a feedforward neural network and a pre-trained convolutional deep neural network.
Task detection from electroencephalography (EEG) signal allows us to discriminate between specific
cognitive states and so monitor cognition. This is associated with the attention and performance of
an individual’s behavior
It is important to highlight that discrimination of mental tasks has the purpose of monitoring the
behavior of an individual, rather than establishing a correlation between EEG measurements and
the final result of the tasks. In particular, there are patterns that might be able to detect cognitive
states between different users.
Part of the behavior is associated with the individual’s auditory quality, which can be monitored
through evoked potentials that estimate hearing sensitivity. In fact, specifically the Auditory Steady
State Responses (ASSRs) are used to measure the ability of local cortical networks to generate
activity and thus be able to differentiate individuals with normal hearing sensitivity from those with
varying degrees of auditory sensorineural loss.
It should be noted that ASSRs are obtained when an auditory stimulus that is periodically
presented produces an electroencephalographic response. Although there are investigations that
address the study of tasks and use artificial neural networks for the classification of waveforms of
Event-related Potential (ERP)
On the other hand, deep neural networks have been applied successfully in different fields
and with performances that outperforms conventional machine learning techniques. In particular,
convolutional neural networks are a special type of deep networks that have proved very effective
in classifying images because they have the ability to extract relevant characteristics, which they
use with a non-linear classifier
The objective of the present work is to make the prediction of the mental task performed by a subject from the EEG signal when it is under a steady state response to an auditory stimulus.
We used a dataset from
The channels that were selected to extract the information of the response to the stimulus were F3, F1, Fz, F2, F4, FC3, FC1, FCz, FC2, FC4, C3, C1, Cz, C2, C4. The sampling frequency was set at 1024Hz, while the number of trials per subject varies between 100 and 120.
The coherent averaging of N trials of EEG signals consists in obtaining the average in each instant of time of segments of signals of equal size and that are started with the stimulus applied. The objective of applying this technique is to reduce noise and random activations, but on the other hand it is expected to highlight the evoked potentials in response to the stimulus. In this work, coherent averages were applied every 15 trials of EGG signals of the same task.
The Inter Trial Phase Coherence (ITPC) is a computational technique that averages the complex
representation of a unit vector, obtained from the phase angle of a trial at a given time, represented
using Euler’s formula. The method was introduced by
IT P Ctf = óóóóón*1 Ér=1 eiktfr óóóóó (1) where n represents the number of trials and eiktfr is the complex polar representation of a phase angle k on trial r, at time-frequency point tf .
The result of the ITPC is an image whose pixels have values that are in the range of 0 to 1,
where 0 indicates evenly distributed phase angles and 1 indicates completely identical phase angles
corresponding to complete coherence
In this work, 3 types of classifiers will be evaluated, which are explained below:
1. Feedforward or Fully Connected Neural Network: It is a type of artificial neural network
consisting of 1 layer of input neurons, 1 or more layers of hidden neurons and 1 output layer.
All neurons in one layer are connected to all neurons in the next layer. This type of network
has no recurrence, lateral connections, nor connections to layers farther than the consecutive
ones. The learning algorithm used is Backpropagation. These networks have the property of
being universal approximators. (More details of these networks see
It should be noted that a CNN can consist of several Convolution layers and several Pooling layers. In this work, the CNN network architecture is composed of two stages, in the first stage there are the 4 layers of convolution with kernel (3x3), followed by layers of average pooling and max pooling and 4 layers of batch normalization, which they are responsible for the extraction of features and dimensionality reduction respectively, in the second stage there are 2 fully-connected layers, responsible for classification. The learning algorithm used is called RMSprop.
In this work, an ad-hoc CNN model was developed for the available data, where the resulting
architecture tries to preserve the parsimony. Because in addition to the CNN model ITPC
images are incorporated, the model will be called Hybrid-CNN
3. VGG16: The VGG16 model is a convolutional neural network with specific architecture and
has been applied in different contexts (see
In this paper, a framework for the classification of tasks from ASSR signals is proposed. The scheme used is shown in figure 2. The proposed scheme consists of the following stages: 1. Data Acquisition: The EEG signals were recorded using a ANT device of 50mV/V and 64
WaveGuard EEG channels. 2. Channel selection and coherent trial averaging: From the EGG signals, the 15 channels closest to Cz were selected where the greatest response to the stimulus is visualized (see section Acquisition of EEG Signals). A coherent averaging of 15 trials for each channel is performed. 3. ITPC: The ITPC method explained in the Inter Trial Phase Coherence section is applied to generate the spectral images. The resulting images have dimensions 21 717. An image bank with 7155 samples was obtained. 4. Feature Extraction: This stage consists of a series of filters that are convolved with the input signals. Afterwards, an activation functions of max-pooling type are applied which generates a downsampling and, at the same time, they work as detectors of relevant characteristics. 5. Classification: At this stage, fully connected non-linear activation neuron layers were used.
From the activated filters obtained from the previous stage a characteristic vector is generated that is processed by the neuronal classifier. The result is the final classification in one of the following tags: Read, Count and Rest.
This section presents the results of the different types of classifiers that were implemented to predict the tasks from the ASSR data, the best results for each model are reported. In the figure 3 a distribution of the confusion matrix of each of the classifiers can be visualized (a) Feedforward Artificial Neural Network
As can be seen in the ROC curves (figure 4), the VGG16 and hybrid-CNN models are above the non-discrimination line, otherwise the FANN model is very close, tending the but performance of the three models, the VGG16 have a good performance with respect to the classification of the tasks but it is not an optimal model, on the contrary the hybrid-CNN algorithm shows the best performance in all the tasks obtaining the best results of area under the curve. (a) Feed forward
We have proposed the application of a convolutional neural network to analyze and classify EEG signals of steady-state auditory responses. To improve the performance of the convolutional neural network, coherent averaging of 15 trials was performed and images were then obtained by applying the ITPC method. The results show that with the pipeline of the proposed model for the prediction of cognitive tasks can be made with an an AUROC of 0.95, corresponding to a sensitivity (recall) score of 0.85.
Future work is required in order to increase the number of subjects in the study, specially considering people with some alteration or that presents cognitive difficulties.
The authors acknowledge the support of the grant REDI170367 from CONICYT.