The article presents the application of the EEG neuroheadset as a component of intelligent learning systems. Traditionally, intelligent learning systems include three knowledge representation models: an expert model, a mentor model, and a student model. The student model reflects the level of his knowledge, as well as how the task is perceived, how the student reacts to the warning and help of the mentor. The inclusion into the student model of the indicators of his reaction to the task based on the analysis of physiological signals seems to be an extremely promising direction of research aimed at improving learning outcomes and developing individual computer-based teaching methods based on feedback. One of the important indicators that can be monitored using the EEG is cognitive load. The paper analyzes methods for measuring cognitive load based on EEG data. An approach to organizing the monitoring of cognitive load using Python libraries is proposed. The MNE library was used to process the EEG data, and the PyEEG library was used to extract the features from the EEG.
Currently, one of paradigms for constructing intelligent learning systems is presentation of the educational process as a student knowledge management process. Intelligent learning systems are based on the idea of selecting an individual training scenario, taking into account the cognitive abilities of the student. It is known from educational psychology that teaching is most effective when the educational process is organized in such a way that to prevent cognitive overload of the student which results in dulling physiological attention. Learning performance can be improved by adapting learning tasks to human information processing capabilities.
Currently,
within
pedagogical
neuroscience, innovative methodologies are being developed for analyzing cognitive load of students
on the basis of measuring physiological indicators. The electroencephalogram (EEG) is considered as
a highly
sensitive
tool for
adapting
the
process
of
human-computer interaction
to
the
psychophysiological student state. The availability of devices based on non-invasive EEG allows to
improve understanding of basic mechanisms of perception of teaching materials [
The development of methods for measuring cognitive load from the EEG data is an area of active
research. Thus, the work [
2020 Copyright for this paper by its authors.
independence from the assessment method. The disadvantages include the need of using expensive
equipment, long analysis time, and the necessity for expert knowledge while interpretation. The work
[
The work [
This paper analyzes possibilities of using the EEG data for intelligent learning systems. The object of the research is the organization of monitoring of cognitive load based on EEG data. The paper is organized as follows. Section 1 substantiates the relevance of the research topic and provides an overview of publications on the research topic. Section 2 analyzes the main ideas of using the EEG neuroheadset as a component of intelligent learning systems. Section 3 discusses the issues of measuring cognitive load based on EEG data and proposes an approach to organizing monitoring of cognitive load using Python libraries. Section 4 summarizes the work done.
Development of neurotechnologies has led to the emergence of consumer-grade neurointerfaces
that allow recording EEG signals. In this regard, the possibilities of using EEG neuroheadsets in the
educational process, in particular, for monitoring the cognitive load of a student, began to be actively
investigated. The Emotiv Epoc and OpenBCI neuroheadsets are among the market leaders between
portable neurotechnical research devices [
Compliance with the standard 10–20 (10–10) system is also carried out when creating various consumer-grade neurodevices. However, if in professional electroencephalographs the number of electrodes involves the installation of at least 21 electrodes, then the number of electrodes in consumer-grade neurodevices is much less. Figure 1 shows a comparison between the international 10-10 electrode placement system and the Emotiv Epoc neuroheadset. The latter includes 14 electrodes. The location of electrodes is fixed.
OpenBCI is an open platform based on ton the ADS1299 Texas Instrument micro-controller to provide users with an access to the raw data on brain wave activity without rigid restrictions on the number and placement of electrodes. OpenBCI offers a variety of data acquisition boards (4, 8 and 16 electrodes) that connect to PCs, laptops, smartphones, and any Bluetooth enabled device. OpenBCI board electrodes can be used with a 3D printed headset, an OpenBCI EEG Electrode Cap, and any traditional system of cup electrode attachment. Figure 2 shows a comparison of the international 1010 electrode placement system and a typical electrode placement on an OpenBCI EEG 3D printed headset.
Traditionally, five frequency ranges are distinguished in the EEG: δ delta (less than 4 Hz), θ theta
(from 4 to less than 8 Hz), α alpha (from 8 to less than 13 Hz), β beta (from 13 to less than 30 Hz), γ
gamma (more than 30 Hz) [
Intelligent learning systems are aimed at optimizing the learning process for each user of the
system. To this end, they create and maintain domain knowledge models, learning prosess model,
student model, and an interface model. A domain knowledge model (or an expert's model) defines a
set of elementary knowledge that a student must assimilate in the process of studying an academic
discipline. The learning process model (or mentor model) formalizes pedagogical strategies and
provides a decision-making process on the tactics of providing educational information to the user,
based on the student's knowledge model. The student model includes facts and relationships regarding
the student's knowledge and skills in the field of knowledge being studied. The student's model is a
formatted description of the learner in the educational process: how he perceives the task, what he
does, how he answers questions, how he asks, how he reacts to the warning and help of the mentor.
The interface model controls the interaction between the user and the intelligent learning system [
The first intelligent learning environments appeared in the 1970s. Since then, many learning
systems of this class have been developed. The best known among them are SHERLOCK,
SQLTutor, ActiveMath, Guru [
The architecture of an intelligent learning system with the use of an EEG neuroheadset assumes that the intelligent learning system receives, as additional data, the results of the assessment and / or the predicted state of the student according to the EEG data. Figure 3 shows a generalized architecture of an intelligent learning system using an EEG neuroheadset.
EEG data collection and analysis unit EEG streaming software
User interface
EEG records
Intelligent learning system
Expert model Mentor model
Student model
EEG data preprocessing and analysis modules
The student wears an EEG neuroheadset when interacting with the intelligent learning system through the user interface. The EEG Analysis module processes physiological EEG signals and transmits the results for inclusion into the student's model. The main limitations of using EEG neuroheadsets include the following: cost, the need of training the system to interpret EEG data for a specific user, response delay, since the data is analyzed over a certain period of time, discomfort from wearing the headset. In addition, the differences in the number of electrodes challenge the developers of EEG data processing and analysis modules for intelligent learning systems with the task of comparing the electrodes arrangement on neuroheadsets and applying feature extraction methods taking into account a specific set of electrodes on the neuroheadset. 2.3.
Cognitive load is a quantitative measure of the mental effort required to complete a task [
Currently, there is no unambiguous way to assess cognitive load using EEG data. The most commonly used methods include [16, 17]: 1) event-related desynchronization and synchronization (ERD / ERS index), 2) theta-alpha ratio (TAR index), 3) methods based on machine learning.
The first method reflects the percentage decrease (event-related desynchronization, ERD) or increase (event-related synchronization; ERS) in band power during the activation interval compared to the baseline interval. The baseline (or reference) interval usually reflects the period of time before the stimulus without any requirement for the task. The activation interval refers to the time period while working on a task. To assess the cognitive load, the ERD / ERS index is used, which can be calculated using the formula [16]: % = BaseLineIntBandPower−TaskIntBandPower × 100,
BaseLineIntBandPower where BaseLineIntBandPower is baseline interval band power, TaskIntBandPower is task interval band power.
Measurement of the ERD / ERS index is done for each electrode and task and then averaged over the tasks to improve reliability.
The second method is based on the calculation of the theta-alpha ratio, which in recent years has been considered as a potentially important indicator of the study of cognitive abilities. It is assumed that the theta-alpha ratio differs depending on age and cognitive abilities, and therefore this indicator can be used both to measure cognitive load and to identify cognitive impairments in the elderly [18]. The TAR index can be calculated using the following formula [17]: = thetaF3+ thetaF4 alphaP7+ alphaP8 , where thetaF3 and thetaF4 are spectral powers of theta band in the electrodes F3 and F4 (frontal region in both hemispheres of the brain), alphaP7 и alphaP8 are the spectral powers of the alpha band in the electrodes P7 and P8 (parietal region of both hemispheres of the brain). Spectral power for frequency bands can be obtained using Fast Fourier Transform.
The third group of methods presupposes the presence of a labeled training sample for training the classifier, the creation of which is extremely problematic in the context of the problem under consideration due to the lengthy process of data collection and labeling. (1) (2)
As part of the experimental part of the work, the possibilities of measuring cognitive load from EEG data using Python libraries were investigated. The TAR index was used as a method for measuring cognitive load. A dataset from [19] was used as a data source. In the case of using a neuroheadset to obtain data in real time, it is necessary to use the appropriate technologies for streaming EEG data and libraries for recording EEG data into a file in *.edf format. For example, for the Emotiv Epoc neuroheadset, it is EMOTIV Cortex technology and cortex.py library.
The dataset under consideration contains records of three minutes of rest state and the first minute (out of four minutes of the task) of performing arithmetic tasks in the mind. The dataset contains records for 36 students. Since files with rest state recordings are actually about 3 minutes long (for example, for Subject03 the total duration is 2 minutes 50 seconds), and there are also several seconds with zero amplitude signals at the end of the recording, a fragment was selected from each recording to calculate the TAR index from the 20th second. The fragment was one minute long. For records of the arithmetic tasks execution, a section from 0 to 50 seconds is selected. The record segmentation scheme is shown in Figure 4. The size of the computation window (segment) is 6 seconds, the window overlap is 50%. The average TAR values for 36 participants are shown in Figure 5. As can be seen from Figure 5, for most of the participants, the average TAR index was greater in the first minute of calculations compared to the minute of rest. Perhaps participation in the experiment could be perceived by some participants as a stressful situation in itself, waiting for the assignment to be given could be accompanied by significant anxiety, and not all participants were able to really adapt three minutes before the start of rest state recording.
V u , e d u t i l p m A
At the same time, considering the TAR index values by segment, several situations can be distinguished: 1) the average TAR index at rest state is lower than during arithmetic tasks, the fluctuation of the index at rest is insignificant, at the beginning of the solvering of arithmetic tasks the TAR index rises and fluctuates slightly; 2) the average TAR index at rest state is lower than during arithmetic tasks, the fluctuation of the index at rest is insignificant, at the beginning of calculations the TAR index rises and continues to grow; 3) the average TAR index at rest state is higher than during computations, at the beginning of arithmetic tasks computations the TAR index decreases. The change in the TAR index by segments for participant No. 7 is shown in Figure 6. 1,5 2 1 0,5
0 0,58 0,56 0,54 0,52
0,5 0,48 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36
1 2 3 4 5 6 7 8 9 1011121314151617
For processing raw EEG data, a Python program was developed. The MNE Python library was used for loading and preprocessing the EEG data. After loading the data from the EDF file, the channel names are matched to the standard names from the 10-10 system. Next, the channels required for calculating the TAR index are selected. A 2-15 Hz bandpass filter is used to remove frequencies that are not used in the analysis. Then the data is segmented. The theta band spectral power and the alpha band spectral power are extracted for each segment, and then the TAR index is calculated for each segment. The pyEEG library was used to extract features. The average value of the TAR index is calculated from the array of obtained TAR indexes, the results are written to the file.
The conducted research allows us to draw the following conclusions. Monitoring the TAR index within intelligent learning systems can provide objective data on changes in cognitive load. However, its use will require the development of mechanisms that allow you to determine whether the growth of the index is due to the complexity of the task.
The need to choose hardware and software inevitably arises when developing intelligent learning systems involving the use of EEG neuroheadsets. These hardware and software should ensure processing of the EEG signals and the transmission of indicators for their inclusion into the model of knowledge about the student. The most suitable option for solving this problem seems to be the development of the software component that will be used to receive and process raw EEG data. This software component should accept information on the types of electrodes as input data, make it possible to choose a suitable method for measuring cognitive load from the knowledge base, and process the data of EEG signals from the electrodes of the neuroheadset. The software component must send to the output the results of the EEG data analysis for inclusion into the model of knowledge about the student. In present article the issues of measuring cognitive load according to the EEG data were investigated, a software module in Python for performing this task was developed which can be used as a component for collecting and analyzing the EEG data.
The research is supported by Ministry of Science and Higher Education of Russian Federation (project No. FSUN-2020-0009).
[16] P.Antonenko, F. Paas, R. Grabner, et al. Using Electroencephalography to Measure Cognitive
Load. Educ Psychol Rev, 22:425–438, 2010. DOI: 10.1007/s10648-010-9130-y. [17] L.Cabañero, R.Hervás, I.González, J.Fontecha, T.Mondéjar, J. Bravo. Analysis of Cognitive
Load Using EEG when Interacting with Mobile Devices. Proceedings, 31:70, 2019. [18] J.P. Trammell, P.G .MacRae, G. Davis, D. Bergstedt, A. E. Anderson. The Relationship of Cognitive Performance and the Theta-Alpha Power Ratio Is Age-Dependent: An EEG Study of Short Term Memory and Reasoning during Task and Resting-State in Healthy Young and Old Adults. Front Aging Neurosci, 9:364, Nov 2017. DOI: 10.3389/fnagi.2017.00364. [19] I. Zyma, S. Tukaev, I. Seleznov, K. Kiyono, A .Popov, M. Chernykh, O. Shpenkov, Electroencephalograms during Mental Arithmetic Task Performance. Data, 4:14, 2019.