In this paper, some questions regarding design of intellectual systems using biological feedback for user mental state management are considered. An example implementation of such system based on the Psychology Experiment Building Language (PEBL) and EMOTIV EPOC+ neurointerface is shown. The results of the comparison of the EMOTIV EPOC+ interface and the Encephalan-EEGR-19/26 mini-encephalograph, which indicate the sufficient accuracy of the EPOC+ interface for use in neurobiofeedback systems, are presented.
When working with biofeedback systems, a large number of parameters should be taken into account. For their evaluation, and also for choice of the overall behavior management strategy, the use of intelligent systems appears extremely promising. Intellectual system is a software system capable of solving complex and compound tasks, associated with a certain problem domain or field of knowledge, information about which are stored in the memory of such intellectual system. The structure of the intellectual system consists of three main blocks - knowledge base, decision-making mechanism and intelligent interface. This system is capable of reasoning about effect method by means of biological feedback.
The purpose of this work is to develop an adaptive user state management system using biofeedback. A key feature of such system should be not only the usage of EEG data for user’s state assessment and correction, but also the usage of user's head movement. Also, in order to verify the possibility of using Emotiv EPOC+ headset in such a system, a comparison of EPOC+ headset and the professional medical encephalograph "Encephalan-EEGR-19/26" mini was conducted. 2
Neurofeedback therapy is widely used in treatments of patients with attention deficit disorder, hyperactivity and learning difficulties. During the therapy course, some type of feedback is used to teach patient how to perform self-regulation and reach the overall state of well-being. Feedback can be visual, audial or even tactile. It can be implemented as color change on the screen, increase / decrease of virtual object’s size or proportions, various sound effects, etc. The main role of biofeedback is, while providing all necessary information, to help patient to achieve a balanced state by means of selfregulation and self-control.
The scheme of the adaptive training process using biofeedback is presented below.
Thus, the user receives information about his state. For example, in the study [5], user looked at the monitor screen, the degree of coloration of which depended on the power of the alpha rhythm. By means of neurofeedback, user was able to control his mental state.
There are many ways to adjust user’s state. The main difference between them is the communication method used to inform the user about his condition at a given time. These methods can be separated in three main categories: • Audial effects; • Visual effects; • Tactile effects.
After communication method is determined, the next step is to select a correction effect and calculate parameters necessary for effective changing of user’s state. Usually some kind of physical or mental conscious activity is used as a corrective effect. For example, in the 2005 year study [6], subjects experiencing chronic pain, while watching animation of the flame on a display, were asked to try to reduce painful sensations by mentally extinguishing the flame and reducing its size. In the paper [5] the study is described, in which subject looked at the screen with a red circle drawn on it, and the color saturation of this circle depended on the level of alpha rhythm of the subject, thus encouraging him to maintain the relaxed state and keep circle as bright as possible. Also, the corrective effect may be focusing on some object or sensation (i.e. meditation), or performing various respiratory exercises.
To describe the state of the user, system may use different metrics. Below is a list of metrics used in this paper. METRICS 1. Electroencephalograms (EEG) are used to monitor brain activity in real time.
2. One of the most effective methods for analyzing the motor activity of a person is the use of accelerometer data to evaluate the accelerations experienced by a person or one of the parts of his body.
3. Indicators of the user's attention allow us to evaluate the state of his cognitive functions directly during the operation of the system.
During the performance of cognitive loads, the most informative indication of patient’s functional state is the electrical activity of the brain - the EEG. Estimation of EEG data can be performed in a variety of ways, and in this paper we have concentrated on the evaluation of several power band features.
Assessment of the movement of the head during the execution of tasks allows us to identify the state of the subject and assess the concentration on the task. Also, the data of a head flickering make it much easier to identify artifacts associated with movement in the EEG record.
As a device that allows to collect information of both brain electrical activity and head movement, a wireless multichannel headset with a high resolution Emotiv EPOC+ was chosen. The characteristics of the headset are shown in Table 1.
up to 12 hours
Impedance measurement Real-time contact quality using patented system
In addition to EEG sensors, Emotiv EPOC+ is equipped with a three-axis accelerometer, which allows synchronous recording of EEG and acceleration of the human head. Head movement data can also be used for analysis of the physical and psychoemotional state of a person.
One of the most effective methods for analyzing human motor activity is the use of accelerometer data to evaluate the accelerations experienced by a person or some part of his body. The three-axis accelerometer provides information on the magnitude of the active accelerations along three axes, respectively. The signal measured by the accelerometer is a linear sum of three components: • Acceleration caused body movement; • Acceleration caused by gravity; • Noise inherent to the measuring system.
Thus, to assess the motion of the subject, it is necessary to separate signal components characterizing the motion in
space. According to the study [7], the acceleration frequency caused by human movement is in the range from 0 to 20 Hz.
The gravitational component is in the range from 0 to 0.3 Hz. A component containing instrumental noises is usually in
the range above 20 Hz. To isolate the motion component from the signal, according to [
To assess the state of the user, in addition to the data of physiological activity, a widely accepted test for the variability of attention, implemented in the programming language PEBL, was used. PEBL is a simple programming language designed to create and conduct many standard experiments. This free software is licensed under GPL, its compiled executables and source code available without charge. PEBL is designed to be easily used on multiple computing platforms. Its current implementation uses SDL as its implementation platform, which is also a cross-platform library that is compiled for Win32, Linux and Macintosh. PEBL is implemented primarily in C ++ (you do not need to know C ++ to use PEBL), it also uses flex and bison (GNU versions of lex and yacc).
• Free software for creating psychological experiments; • Allows you to create your own experiments or use ready-made experiments; • Allows you to freely share experiments without a license or commitment.
The PEBL language is used to implement various specific tests designed for psychological experiments.
Test of Variables of Attention (TOVA) is such continuous performance test that examines the function of attention and is frequently used in the diagnosis of attention deficit hyperactivity disorder (ADHD). This test offers both visual and auditory measures of attention using targets that the subject is instructed to respond to, as well as non-targeted, to which individuals are instructed to withhold their responses. During the TOVA, the individual responds to the presence of the target and refrains from responding during the appearance of an untargeted signal. The first half of TOVA consists of a relatively low ratio of target and non-target presentations (one target for 3.5 non-target), which is a challenge for those with low attention span; the second half consists of a relatively high ratio of target and non-target representations (3.5 goals per 1 non-target object), which is difficult for patients with impulsive and hyperactive behavior. Figure 3 shows one of the stages of the TOVA test. [9] TOVA tests, expressed as z scores: first half response time, total response time variability, and second half d′ score (ZResponse Time Half 1 + Zd' Half 2 × -1 + ZResponse Time Variability Total). The d′ Measure is derived from the Theory of Signal Detection and indicates the sensitivity of the response. TOVA provides a summary of the quarters for each of the variables listed above, which is very useful for tracking potential changes in performance.
To verify the capabilities of the EMOTIV EPOC+ headset, a series of experiments was conducted with a general type of research. The study was conducted on a group consisting of 7 healthy subjects, with average age about 23 ± 3. Each of the subjects agreed for the gathered data processing. Stages of research: • Functional rest (FR) - 5 minutes; • The first TOVA test (T1) - 3 minutes; • Hyperventilation load (HL) - 3 minutes; • The second TOVA test (T2) - 3 minutes; • Aftereffect (AE) - 5 minutes.
For the first time, the subjects underwent a study using the Emotiv EPOC+ headset, the second time using the "Encephalan-EEGR-19/26" mini medical encephalograph. As a verification parameter, the ratio of alpha-rhythm to thetarhythm and beta-rhythm to theta-rhythm was used. 3
As a result of the work, a conceptual model of intellectual system was developed that realizes adaptive training process based on neurobiofeedback with Emotiv EPOC+ headset and the test of variability attention TOVA usage.
The intelligent system consists of two blocks - the data acquisition unit and the biofeedback loop unit. The data acquisition unit includes the Emotiv EPOC + headset and the data receiving unit. The block of the biofeedback loop contains a system for determining the user's mental state, a system for selecting the corrective action and a transmission interface for the control command.
The system based on the algorithm described above, after receiving the data on the human condition, sends this data to the computer via wireless Bluetooth interface. After this, the primary data filtering is performed using a bandpass filter with 4Hz - 45Hz limits for EEG data and 0.3Hz - 20Hz for motion data. An important feature of the system is the usage of accelerometers in mitigation of EEG artifacts associated with the movement of the head. The next step is to highlight the key features for each of the signals. After this, a structure containing data about all parameters describing the current state of the user is sent to the queue.
In the next step, the intelligent processing unit receives the user state structure from the queue and the test results obtained in the TOVA test. Using all the information, the block determines the user's state. If the user is in the target state, a supporting feedback system is selected that stabilizes the user's state. In the case of a state mismatch, block determines the parameters of the corrective feedback, which is aimed at transferring the user to the desired state. This feedback loop continues throughout the entire session.
As a corrective effect, the breathing control technique is used. This technique assesses the state of the user after performing the first test, then calculates the breathing parameters necessary to transfer the user to the target state.
To verify the Emotiv EPOC+ headset, Bland-Altman method was used. Graphs with comparisons of the alpha-rhythm power ratio to theta rhythm and beta-rhythm power to the theta rhythm are shown below in Figure 6.
As seen in the graphs given above, the values of the power ratios are quite close to each other, which makes it possible to talk about the possibility of using the Emotiv EPOC+ interface with an accuracy on a par with the professional medical encephalograph "Encephalan-EEGR-19/26" mini. 4
As result of the work, an intelligent system of adaptive training process based on neurobiofeedback with Emotiv EPOC+ neuroheadset usage was developed. A special feature of this implementation is the usage of accelerometer and TOVA test to assess the level of cognitive functions and to determine the parameters of the corrective effect of biofeedback. Verification makes the usage of an Emotiv headset in systems of similar type more plausible and advisable. In the future, the area of interest represents an increase in the number of modalities, an expansion of the range of possible corrective actions, functional and performance improvements.