This study presents a comprehensive exploration of EEG-based motor imagery classification using advanced deep learning architectures. Focusing on six distinct motor imagery classes, we investigate the performance of convolutional neural networks (CNN), CNN with Long Short-Term Memory (CNN-LSTM), and CNN with Bidirectional LSTM (CNN-BILSTM) models. The CNN architecture excels with a remarkable accuracy of 99.86%, while the CNN-LSTM and CNN-BILSTM models achieve 98.39% and 99.27%, respectively, showcasing their efectiveness in decoding EEG signals associated with imagined movements.The results underscore the potential applications of this research in fields such as assistive robotics and automation, showcasing the ability to translate cognitive intent into robotic actions. This study ofers valuable insights into the realm of deep learning for EEG analysis, setting the stage for advancements in brain-computer interfaces and human-machine interaction.
representations from EEG data [
Various advanced deep-learning techniques were uti- sification of MI-EEG signals associated with right-hand
lized to improve Motor Imagery’s (MI) precision. As an ifnger movements[22].
example, in this study [
In Ref [
In addition, several research has investigated difer- Computer Interface (BCI) research.
ent methods, such as Long Short-Term Memory (LSTM), Obtained from over 1500 EEG recordings with the
parwhich have shown good results in motor imagery clas- ticipation of 109 volunteers, the dataset ofers a
comsification using EEG. In light of promising findings, one prehensive data collection. The experiments, facilitated
study [
Many research utilize BiLSTM as an excellent case involve imagined movements associated with both the study. This research [21] introduces a novel approach right and left fists, as well as periods of relaxation. On the for decoding imagined finger motions using MI-EEG other hand, tasks 6, 10, and 14 involve imagined movedata. The approach efectively tackles noise challenges ments of both fists and both feet. in small, noisy signals using Empirical Mode Decompo(2) (3)
Tanh() = − − + − ReLU() = max(0, )
role in handling classification tasks. These layers, known as fully connected layers, establish connections between every neuron within a layer and those from its preceding layer. The ultimate layer of fully connected layers serves as the output layer, functioning as the classifier in the CNN architecture.
lem in traditional RNNs, introduces memory cells with gating mechanisms, including input, forget, and output gates, to control the flow of information. It comprises a cell state representing long-term memory and a hidden state representing short-term memory or output [26]. CNNs are very good at classifying images because they use neural layers to learn hierarchically organized features. People are now interested in making CNNs use data that isn’t pictures, like time-series data. CNNs are a great way to get features from EEG data and recognize patterns. This is because they can show how electrodes are spread out in space and how brain activity changes over time [24, 25]. Convolutional layers, pooling layers, activation functions, and fully connected layers are the main parts that make up a CNN. To make output feature Figure 2: The architecture of a LSTM model [27]. maps, convolutional layers use convolutional kernels to run convolutions. At the same time, pooling layers subsample feature maps while keeping the most essential Bidirectional Long Short-Term Memory (Bi-LSTM) is characteristics. Adding activation functions like Sigmoid, an extension of the traditional LSTM, a type of recurTanh, and ReLU to the network creates non-linearity, a rent neural network (RNN). LSTMs are adept at capturvital part of matching inputs to outputs correctly. ing and retaining long-term dependencies in sequential Sigmoid function: data, making them suitable for applications like natural language processing, time series prediction, and speech Sigmoid() = 1 (1) recognition [28].
1 + − The term "bidirectional" in Bi-LSTM refers to the fact
that it processes input sequences in both forward and backward directions. This bidirectional processing helps the network capture information from both the past and the future of a given time step, allowing it to better understand the context and dependencies in the sequence.
The Bi-LSTM architecture consists of two LSTM layers, one that processes the input sequence in the forward direction and another in the backward direction. The results from both directions are often combined before passing them on to the next layer or used for the final prediction[28, 30].
and Dense layers. This hybrid approach synergizes the strengths of CNN for spatial features and Bidirectional LSTM for temporal dependencies, resulting in a robust classification model.
A five-fold stratified k-fold cross-validation is implemented using the StratifiedKFold function from scikitlearn. The dataset is divided into training and testing sets for each fold, and each model is compiled with categorical cross-entropy, Adam optimizer, and accuracy as the metric. The training spans 100 epochs with a batch size of 32, facilitating a thorough assessment of the model’s performance across diverse data subsets.
ployed key scoring metrics, including Accuracy, Recall, Precision, and F1-Score. These metrics provide a comprehensive evaluation of the models’ efectiveness in handling classification tasks. Each sign gives a diferent view of diferent parts of a model’s efectiveness, and when mixed, they provide a complete visualization: Precision =
Recall =
TP TP + FP
TP TP + FN
includes applying an 8–30 Hz filter and a notch filter, followed by sampling at a frequency of 125 Hz. This es- Precision · Recall sential preprocessing step refines the raw EEG signals, ef- F1-score = 2 · Precision + Recall fectively eliminating undesirable frequency components and ensuring the data’s suitability for further analysis. TP + TN The 8–30 Hz filter is instrumental in concentrating on per- Accuracy = tinent frequency bands linked to neural activity, while TP + TN + FP + FN the notch filter serves to eliminate specific unwanted where TP are the true positives, FP the false positives, frequencies, such as those associated with power line TN the true negatives, and FN the false negatives. interference.
The CNN architecture, comprising Conv1D, Batch Nor- Table 1 malization, MaxPooling1D, Dropout, Flatten, and Dense Comparison of Architectures. layers, demonstrates efectiveness in classifying EEG data across 6 classes. The CNN with Long Short-Term Architecture Precision Recall F1 Accuracy Memory (CNN-LSTM) model seamlessly integrates CNN CNN 1.00 1.00 1.00 99.86% and LSTM layers to capture spatial and temporal fea- CNN-LSTM 0.98 0.98 0.98 98.39% tures. This architecture includes CNN, Batch Normaliza- CNN-BILSTM 1.00 0.99 0.99 99.27% tion, MaxPooling1D, Dropout, LSTM, Flatten, and Dense layers, showcasing commendable accuracy in EEG data The CNN architecture showcased exceptional perforclassification by skillfully combining spatial and tempo- mance, achieving perfect precision, recall, and F1-Score, ral aspects. The CNN with Bidirectional LSTM (CNN- leading to an impressive overall accuracy of 99.86%. BiLSTM) architecture enhances EEG data classification by This underscores the model’s efectiveness in accurately combining CNN and Bidirectional LSTM networks. The classifying EEG data. The CNN-LSTM model, although model incorporates CNN, Batch Normalization, MaxPool- slightly less accurate, still demonstrated commendable ing1D, Dropout, Bidirectional LSTM, Dropout, Flatten, results, with precision, recall, and F1-Score values at 0.98 and an overall accuracy of 98.39%. This model efectively combines spatial and temporal features for EEG classification, emphasizing a balance between complexity and accuracy. The CNN-BILSTM architecture displayed a well-rounded performance, with precision, recall, and F1-Score all reaching 1, 0.99, and 0.99, respectively.
Combining CNN for spatial features and Bidirectional LSTM for temporal dependencies, this hybrid approach achieved an accuracy of 99.27%, highlighting its eficacy in accurate EEG data classification.
The evaluation results of three distinct architectures, namely CNN, CNN-LSTM, and CNN-BILSTM, shed light on their respective performances in classifying EEG data with 6 classes. The CNN model exhibits exceptional performance across all metrics. It achieves precision, recall, and an F1-score of 1.00 for most classes, highlighting its ability to classify each class accurately. The overall accuracy of 99.86% underscores the model’s efectiveness in capturing intricate patterns within the EEG data. The precision-recall curves for each class demonstrate the model’s robustness and reliability. The CNN-LSTM model, incorporating both convolutional and long shortterm memory layers, displays a commendable performance but with a slight decrease in precision, recall, and F1-score compared to the pure CNN model. This suggests a potential trade-of between model complexity and overall performance. The accuracy of 98.39% indicates reliable classification, though less than the CNN architecture. The CNN-BILSTM model, combining the strengths of CNN and Bidirectional LSTM, provides an excellent balance between precision, recall, and F1-score. With precision above 1 for most classes and an accuracy of 99.27%, it demonstrates the model’s ability to capture both spatial and temporal dependencies in the EEG data. The bidirectional processing contributes to understanding the context and dependencies in the sequence.
The outcomes obtained from the implemented models indicate their proficiency in recognizing patterns and extracting features from EEG data, resulting in successful classification. This underscores the appropriateness and efectiveness of the selected models for the specific EEG
cation, evaluating CNN, CNN-LSTM, and CNN-BILSTM models for six selected classes. Results demonstrate exceptional accuracy (CNN: 99.86%, CNN-LSTM: 98.39%, CNN-BILSTM: 99.27%). The discussion introduces a compelling scenario, envisioning the practical application of the six selected classes in robot navigation through brain-computer interface technology. This scenario exemplifies the potential real-world impact of motor imagery classification, providing a seamless link between cognitive intent and robotic actions. Despite successes, challenges in real-time processing and model robustness persist. The study encourages further refinement and addresses practical considerations for broader implementation. The findings contribute to the field, shaping the future of human-machine interaction, particularly in assistive robotics and intelligent automation.
Real or Imagined Movement Closing/opening left hand Closing/opening right hand Opening/closing both fists Opening/closing both feet Corresponding Commands Turning left Turning right Stopping Going forward
As the user engages in motor imagery actions, the EEG signals associated with the specific classes are decoded in real-time by the implemented classification models. The system translates these decoded signals into corresponding robot commands, enabling the user to navigate the robot efortlessly. For instance, by simply imagining the closure of the left hand, the robot seamlessly executes a left turn, ofering an intuitive and natural interaction mechanism.
However, challenges and considerations arise in the implementation of such a scenario. Real-time processing, user adaptation to the BCI system, and robustness in various environmental conditions are factors that require careful attention. The user’s cognitive load, comfort, and the need for continuous improvement in the classification models become focal points for refinement.
Despite these challenges, the envisioned scenario highlights the potential transformative impact of motor imagery classification in robot navigation. The seamless fusion of cognitive intent and robotic action could revolutionize human-robot interaction, paving the way for intuitive and accessible control mechanisms in diverse applications, ranging from assistive robotics to smart