Attention and vigilance detection based on Electroencephalography - A summary of a literature review

Abstract

This document introduces a systematic literature review on attention and vigilance detection using Electroencephalography (EEG). The purpose of this study is to investigate the state of the art in the field, find gaps, suggest future work and find answers to the research questions of the study. The review consists of an introduction that describes prerequisites for writing current study and general terms, research methods with a description of research methodology, results section which answers research questions, discussion, and conclusion sections.

Introduction

The use of biophysical signals in the analysis of human physiological state and well-being got broad popularity in different areas of science. Medical experts use those signals to study the processes in our bodies and how external factors affect those processes. Computer science researches have acknowledged the role of attention and other mental states in the well being of a software individuals, teams, and organizations [ 46, 47, 13, 14, 39, 45, 8, 21, 20, 32 ], and use biosignals to build systems that will help people monitor their state and develop new analysis techniques that will help understand the meaning of the collected signals better.

One of the methods for reading brain activity is Electroencephalography (EEG). It studies the functional state of the brain by recording its bioelectric activity. This method provides a wide scope for experiments because it allows us to interpret data online, conduct experiments during various activities since it is portable, non-invasive and does not require the help of doctors.

As was said previously, Computer Science (CS) researchers use EEG in combination with other bio-signals to build Brain-Computer Interfaces (BCI) – systems that can measure the activity of the brain and the central nervous system, analyze it and convert it into artificial output. Using different CS algorithms and techniques BCIs can clean, enhance or improve the natural output of the brain. Moreover, many data analysis techniques can be applied in BCIs to predict human behavior or state. Such systems can be used in different fields of research such as e-learning, driving, performance at work and analysis of programmers’ behavior.

Copyright c 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

The goal of the research is to perform a preliminary review the current status of the use of bio-signals in the studies of the human physiological state and how EEG was used in the field. Also, it aims to collect a list of EEG analysis techniques that can be referenced in further studies.

Section 2 describes methods, states the research questions, defines the search process and queries used and lists inclusion and exclusion criteria. Section 3 gives answers to the research questions stated in the previous section. Sections 4 and 5 are dedicated to discussion and conclusion, respectively. 2

Research method

This section describes the research process and steps performed during the Systematic Literature Review (SLR). First, the formulation of the research questions and their importance is given. Then inclusion and exclusion criteria were given as well as the data collection process is described. 2.1

Research Questions

To identify the primary studies that address the topic of our SLR we formulated three research questions (RQ). Our study aims to answer the following ones:

RQ1: How biophysical signals were used in determining the conditions of human activity or work?

RQ2: In the framework of RQ1, how EEG was used?

RQ3: What kind of EEG analysis techniques were used?

2.2

Search Process

Our search process was a manual search in the two largest digital libraries available: ACM Digital Library and IEEE Xplore Digital Library. For each RQ the keywords were extracted and proper search queries were defined using those keywords. All the results from all 3 search queries were exported to a Rayyan QCRI [ 38 ] – a web-application for collaboration on systematic literature reviews. 2.3

Inclusion and Exclusion Criteria

During the review process the studies were checked to satisfy the following inclusion criteria:

Availability online to ensure paper accessibility

Focus on biophysical signals and especially brain activity

Focus on measuring the level of attention or stress to ensure its compliance with the study

Format of the research paper (papers, books, thesis, posts, videos, etc.)

Methods description and approaches of brain activity and biophysical signal analysis

Focus on studies of work environment

Written in English

The studies of the following topics were excluded from further processing:

Any paper that does not satisfy any of the inclusion criteria.

Papers written by the same authors describing the same factors. 2

2.4

Data Collection

The data extracted from the reviewed materials were:

The main area of the research

The research question/questions of the study

The authors of the research

The summary of the research

The gaps in the research and the areas of further studies

3 3.1

Results

RQ1: How biophysical signals were used in determining the conditions of human activity or work

Literature analysis shows that conditions of human activity or work can be slipped into three parts: measuring attention, stress detection, and tracking programmers’ activity. 3.1.1

Measuring attention A study by [ 30 ] proposes to use EEG signals for Attention Recognition (AR) and extends previous research that used eye-gaze, face-detection, head pose and distance from the monitor to track user’s attention. AR is a promising field that can be applied in many areas such as e-learning, driving, and most relevant - in measuring awareness during video conferences. In [ 17 ] EEG was used to determine the attention level, while the subject was performing a learning task. In [ 5 ] EEG was used to estimate alertness in real-time. In [ 31 ] presented a single channel wireless EEG device which can detect driver’s fatigue level in real-time on a mobile device such as smartphone or tablet. Measuring attention is very important in many fields, such as detecting drivers’ drowsiness and workers’ fatigue.

Analyzing all the previously mentioned studies we gathered the techniques for attention measurement into a list, which states that attention could be measured via: heart rate variability [ 43 ] galvanic skin response [ 43 ] pupil diameter, eye blink frequency [ 43 ] brain activity measurement (EEG, MEG (Magnetoencephalogram), fNIRS (functional near-infrared spectroscopy), ECoG (electrocorticogram), fMRI (functional magnetic resonance imaging), etc.) [ 43 ], positron emission tomography (PET), transcranial magnetic stimulation (TMS), near-infrared spectroscopy (NIRS). [ 25 ] Conner’s Continuous Performance Test (CPT) is a simple examination technique: the test subject has to react in case a rare signal appears. The method is described in the studies [ 11 ] and [ 25 ].

The test of variables attention (T. O. V. A.) is an objective neuropsychological assessment of attention. It is a very simple computer game in which the response of the test subjects to a visual or auditory stimulus is measured. [ 25 ] [ 5 ], [ 12 ] [ 5 ] 3.1.2

Stress detection Several studies presented designs of the systems that monitor the human physical and mental state in the working environment. Using different biophysical signals and environmental measures they detect stress levels of employees.

A new aparatus [ 4 ] was designed to assess the stress levels of call-center operators. The study uses two types of sensors to monitor the working environment: environmental and physiological. The evaluation of stress relies more on the latter signals. The goal of the authors was to design the system to improve the well-being of the employees with the application of multi-sensor analysis.

The portable system described in [2] measures biophysical signals in real-time and notifies unwanted mental behavior. The notifications are sent in case the following conditions in the worker are detected: 1) absent-minded/inattentive, 2) stressed, 3) extreme fear, 4) anger, 5) stun/daze, 6) overloaded with work, 7) drowsiness, and 8) dizziness. The author focuses on neuroergonomics as a primary field of study. As well as the previous study, this one aimed to design a system to predict human mental and physical state and increase productivity and well being at work. However, the range of biological signals collected was significantly broader than in [ 4 ] and brain and muscle activity analysis was used.

The device proposed in [ 3 ] determines the relaxation level of the user. It consists of the Virtual reality headset and the olfactory necklace. The necklace changes the intensity of aroma, depending on the subjects’ EEG datagrams.

In [ 42 ] the mental stress was measured while solving arithmetic tasks. The [ 9 ] detected the difficulty of program comprehension tasks among the students.

The [ 48 ] describes a method to determine the drivers’ vigilance level.

In the context of the studies mentioned above the following biophysical signals were used: heart rate [2] [ 4 ]; body temperature [2]; galvanic skin resistance [ 4 ] – showed that increasing skin conductance indicates the rise of stress level; blood pressure [2] - a sensor is placed in the temple part of the head or in the upper part of the shoulder depending on the type of device; EEG [2] [ 3 ] [ 9 ] [ 9 ] [ 18 ] [ 19 ] [ 34 ] [ 15 ] [ 29 ] [ 26 ] [ 36 ] [ 48 ];

EMG [2]; 3.1.3

Assessing programmers activity A study [ 6 ] shows how EEG can be applied to understand the mental activities of programmers during pair programming. Here, a portable multichannel EEG device was used to understand if there is any difference in the mental processes of the minds of developers when they use different development approaches. The data were collected during several pair programming sessions where two developers played the role of a ”driver” and ”navigator” consecutively. The goal then was to determine whether those activities induce a higher level of concentration.

Another study [ 28 ] in this field compares the cognitive activities of novice and expert developers and assesses their programming language comprehension. By conducting an EEG experiment they showed that indeed there is a clear difference in how these two groups understand programming languages. There was a higher brain activation in certain electrodes, expert programmers showed better short-term memory and comprehension abilities in general.

Analyzing the results, it can be said that the approach of using EEG to analyze the brain activity of developers is rather effective and practical, as it can be used in the normal programmers surrounding and show good results in distinguishing between different brain activity patterns.

It was observed that EEG is one of the most popular and easy ways to measure people’s attention and stress, because of its ease of use and relatively accurate results. 3.2 The study conducted by [2] relies on the concept of neuroergonomics design, and especially aspects like stress, attention, drowsiness, and others to design efficient systems to be used by humans. To measure these metrics several methods are used in neuroergonomics, but one of the most relevant is neuroimaging. The authors of the study designed a system that keeps live feed about human’s psychophysiological information and used EEG as their primary method to measure brain activity.

They designed a simple BCI’s where one dry EEG electrode sensor is placed on the forehead. The authors use a high-pass filter and a low-pass filter to clean the noise at low/high frequencies and a notch filter to filter specific bands of the signal. After passing all filters and amplification the resulting signal is then converted to digital format. The study shows how collecting EEG data can help in creating effective and comprehensive BCI systems to monitor behavior and well-being at work.

Another research from [ 30 ] investigates how EEG can extend the techniques for AR. Previously EEG was used mainly for emotion recognition. This study mostly focuses on the methods of EEG data processing, feature extraction, and further attention classification. The EEG data were collected while subjects were reading or watching random content. After finishing each subject filled a self-assessment form. Later, based on the gained results, the data were divided into five classes and preprocessed. The classification algorithms were then applied to the acquired data. By doing so authors propose to use EEG data for AR and probably supplement the techniques used previously in these kinds of studies.

In [ 5 ] study, EEG signals are used to estimate the alertness level by recording the response time for the Test Of Variables Of Attention (TOVA) and the EEG signals in parallel. The correlation between those two measures was then studied. The results of the experiment show that EEG can be used in real-time systems that estimate human alertness.

In [ 3 ] researchers conducted the experiments: 5 min control experiment and 5 min with VR headset and olfactory necklace (with lavender aroma), where the 360 degrees beach was shown to test subject. EEG was recorded using commercial Muse headband. It provides four flexible electrodes located at 10-20 positions TP9, AF7, AF8 and TP10 with reference for Fpz After the experiment, a test subject filled in a questionnaire. The authors showed, that there is 25% boost of actual relaxation and 26.1% percent boost with questionnaire study.

In [ 42 ] the test subject filled the demographic form PSS. Then 10 seconds of a calm picture was shown in the beginning and at the end of the experiment. After that subject was asked to solve 10 arithmetic questions. After the experiment test subject was asked to determine highly stress stages, namely before, during or after the mental induced task. EEG was used to record data from the test subject. The Mindset 24 Topographic Neuro Mapping Instrument by Nolan Computer Systems LLC was used. The 10-20 recording system was used, namely: Fp1, Fp2, F7, F3, Fz, F4, F8, T3, C3, Cz, C4, T4, T5, P3, Pz, P4, T6, O1 and O2. Sampling frequency: 256 Hz, impedance <5 kOm, cutoff range: -80 to 80

In [ 9 ] a relax screen was shown to test subjects, then they solved 3 practice questions and 9 ordinary questions using TPS. Between each question, the relaxing screen was showed to take a break. EEG was used to record the programming activity. Emotiv Epoc device with 16 channels: 14+2 reference, following the 10-20 international system. Sampling frequency: 128 Hz.

In [ 48 ] the subject completed the 90 minutes simulated driving. The subjects required to lie on a bed if they feel sleepy. The EEG was recorded with Neuroscan with 64 electrodes, 2 of them were EOG.

In [1] the impact of in-vehicle secondary tasks on driver cognitive state during driving was measured. This was done by capturing the changes in EEG dynamics. Authors employed a wearable data acquisition platform to collect wireless EEG data from six subjects during a naturalistic driving session and investigated six potentially distracting stimuli.

In [16] shown the evolution of mental fatigue in a Stroop task using electroencephalography (EEG) with an independent component analysis (ICA) method. Specifically, two aspects of mental fatigue, i.e., mental effort and mental engagement, were tracked by the ongoing oscillatory dynamics from frontal independent component (IC) related to cognitive control and posterior ICs related to attention. 3.3

RQ3: What kind of analysis of EEG results was used?

This subsection describes the main techniques to analyze EEG data. It starts with Machine Learning Techniques and proceeds with others. A lot of the papers used Machine Learning techniques for analysis of EEG data. They can be divided into three groups: Neural Networks, Classification Algorithms, and Other techniques. 3.3.1

Machine Learning Techniques Nowadays, Neural Networks is a widely used machine learning approach. In [ 35 ] study, authors train an MLP NN to learn characteristics of EEG that define attention state. The main goal of this study was to investigate the hidden nature of attention mechanisms while recording the subject’s EEG data. The novelty of the proposed method is in using four levels of attention instead of 2 as was used in previous studies. Also, the authors emphasize the fact that identifying an attentive state is easier than inattentive because of more noise and irrelevant information recorded during the inattentive state. The [24] research focuses on the continuous detection of changes in human alertness and EEG power spectrum on a minute time scale. Authors emphasize the variability of EEG dynamics and say that group statistics used in previous studies cannot be used effectively. So information collected from each operator is then fed to a neural network to adapt to individual differences in EEG dynamics. The results are then compared to linear models. A novel approach was described in [ 15 ]. The authors used Convolutional Neural Network as a feature extractor. The data was preprocessed, first, by statistical indicators, to remove points with the subject’s score standard deviation of more than 2 times the mean. Then two feature vectors were built: linear by Pearson’s coefficients and nonlinear by SL matrix. Then, linear and non-linear features are fused by the CNN framework.

The more classical approach is K-nearest neighbours. In [ 30 ] the authors of the study introduce EEG measures to track emotions and attention. The project applies classification algorithms to the EEG signals and k-NN was one of them. After extracting 13 important features a k-NN classifier was used to divide the data into both 3 and 5 attention classes. The [22] shows how to detect driving fatigue based on k-NN and the correlation coefficient of the subject’s Attention and Meditation. Naive Bayes classification is also a widely used machine learning algorithm. In [ 9 ] authors measured task difficulty. EEG data were first normalized by computing the mean on all channels and subtracting it from each channel for each subject. Then filtering was done on 1-second segments by Elliptic Infinite Impulse Response filtered described by Manoilov. Then four types of features were extracted: Energy, Event-Related Desynchronization, Frequency ratio, and Asymmetry ratio. After that, the Naive Bayes classifier was used to classify the data from each feature vector. 3.4

Other methods

There are several techniques worth mentioning, for example, P300. P300 (also called P3) wave [ 10 ] is an endogenous potential that surfaces itself as a positive deflection in the voltage with an average latency of roughly 250 to 550 ms depending upon the task [ 33 ]. It is generally elicited during the process of decision making and is usually elicited after 300 ms of the occurrence of the stimulus. It has been shown that the amplitude of the P300 peak decreases by a significant amount due to the presence of fatigue[ 43 ]. Another commonly used approach is the Independent Component Analysis (ICA) [ 37 ]. ICA is a widely used method for decomposing multi-channel data into components that are statistically independent (ICs). In the context of EEG data analysis, some components should represent brain activity, while others should represent noise resulting from eye and muscle movements. 4

Discussion

This section represents different findings of this Systematic Literature review. The Systematic Literature review aimed to identify current progress in biophysical signal usage in IT and cross fields. From 317 publications 40 publications have answered the Research questions. The result of the review showed that in recent years, from 2015 till 2019, there is a high interest in developing systems with the help of biophysical signals. This study mostly focused on describing methods for attention, emotion recognition and experimental procedures.

With the recent increased interest in Machine Learning and availability of such data sets as DEAP [ 27 ] and AMIGOS [23], a high number of studies described Machine Learning methods for attention.

Some studies focused on the feature extraction of data for future usage in classification algorithms. The development of such methods is a good indicator of interest in biophysical signal-based systems.

A little number of studies used the programmers as the main experiment subjects. Thus, giving research opportunity to investigate new methods based on biophysical signals. Such research should help to develop a system of attention recognition for IT developers, giving industrial companies sufficient information about the performance of their employees.

Also, the primary data collection method was EEG, several studies used other biophysical signals such as EMG, heart rate, blood pressure. The potential combined usage of different biophysical signals is an open question. 5

Conclusion

The Systematic literature review clearly shows the high interest in using EEG based systems for attention and emotion recognition. Mostly, all studies are developing new techniques in Machine Learning signal processing.

Analysis and processing techniques were separated into different groups according to the ML method used. Based on the review of the techniques Section 3.3 gives sufficient information regarding data preprocessing and classification methods used. As there is a lack of study on programmers’ performance, future research should be more focused on this topic, focusing also on metrics and open systems [ 44, 40, 41 ] and understanding the dynamics of the collaboration between people [ 7 ]. The question: “How can we help programmers perform better using the biophysical system?” remains open. Researchers may try to answer this question with the help of methods described in this Systematic Literature Review. [1] Vahid Alizadeh1 and Omid Dehzangi. The impact of secondary tasks on drivers during naturalistic driving:

Analysis of eeg dynamics. 2016. [2] Awad M Aljuaid. Theoretical design of eeg-based neuroergonomics integrated portable system, applying direct psychophysiological indicators. In 2019 Industrial & Systems Engineering Conference (ISEC), pages 1–6. IEEE, 2019. [16] Member IEEE Guofa Shou and Lei Ding. Ongoung eeg oscillatory dynamics suggesting evolution of mental fatigue in a color-word matching stroop task. 2013. [22] Zhijiang Wan Chen Hu Jian He, Dongdong Liu. A noninvasive real-time driving fatigue detection technology based on left prefrontal attention and meditation eeg. [23] Nicu Sebe Ioannis Patras Juan Abdon Miranda-Correa, Mojtaba Khomami Abadi. Amigos: A dataset for affect, personality and mood research on individuals and groups. 2017. [24] Tzyy-Ping Jung, Scott Makeig, Magnus Stensmo, and Terrence J Sejnowski. Estimating alertness from the eeg power spectrum. IEEE transactions on biomedical engineering, 44(1):60–69, 1997.

References

[3] J. Amores , R. Richer , N. Zhao , P. Maes , and B. M. Eskofier . Promoting relaxation using virtual reality, olfactory interfaces and wearable eeg . 2018 IEEE 15th International Conference on Wearable and Implantable Body Sensor Networks (BSN) , pages 98 - 101 , 2018 .

[4] M Andersson , A Avamini , A Colosimo , A D 'Amico , F Davide , C Di Natale , S Ganci , M Gutknecht , M Holmberg , E Mazzone , et al. Angelo evaluation: application of a multisensor system for psycho-physiological stress detection in working environments . In ICECS 2001. 8th IEEE International Conference on Electronics, Circuits and Systems (Cat. No. 01EX483) , volume 3 , pages 1509 - 1512 . IEEE, 2001 .

[5] Luzheng Bi , Ran Zhang, and Zhilong Chen. Study on real-time detection of alertness based on eeg . In 2007 IEEE/ICME International Conference on Complex Medical Engineering , pages 1490 - 1493 . IEEE, 2007 .

[6] Sara Busechian , Vladimir Ivanov, Alan Rogers, Ilyas Sirazitdinov, Giancarlo Succi, Alexander Tormasov, and Jooyong Yi . Understanding the impact of pair programming on the minds of developers . In 2018 IEEE/ACM 40th International Conference on Software Engineering: New Ideas and Emerging Technologies Results (ICSENIER) , pages 85 - 88 . IEEE, 2018 .

[7] Irina D Coman , Pierre N Robillard, Alberto Sillitti , and Giancarlo Succi . Cooperation, collaboration and pairprogramming: Field studies on backup behavior . Journal of Systems and Software , 91 : 124 - 134 , 2014 .

[8] Enrico Di Bella , Alberto Sillitti, and Giancarlo Succi . A multivariate classification of open source developers . Information Sciences , 221 : 72 - 83 , 2013 .

[9] A. Duraisingam , R. Palaniappan , and S. Andrews . Cognitive task difficulty analysis using eeg and data mining . 2017 Conference on Emerging Devices and Smart Systems (ICEDSS) , pages 52 - 57 , 2017 .

[10] R. Wijesinghe E. Donchin , K. M. Spencer . The mental prosthesis: assessing the speed of a p300-based braincomputer interface . IEEE transactions on rehabilitation engineering , vol. 8 , no. 2 , 2000 .

[11] C. Keith Conners et al. Continuous performance test performance in a normative epidemiological sample . Journal of Abnormal Child Psychology , 2003 .

[12] Gordon B. Forbes. Clinical utility of the test of variables of attention (tova) in the diagnosis of attentiondeficit/hyperactivity disordere . Journal Of Clinical Psychology , 1998 .

[13] Ilenia Fronza , Alberto Sillitti, and Giancarlo Succi . An Interpretation of the Results of the Analysis of Pair Programming During Novices Integration in a Team . In Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement , ESEM '09 , pages 225 - 235 . IEEE Computer Society, 2009 .

[14] Ilenia Fronza , Alberto Sillitti, Giancarlo Succi, and Jelena Vlasenko . Understanding How Novices Are Integrated in a Team Analysing Their Tool Usage . In Proceedings of the 2011 International Conference on Software and Systems Process, ICSSP '11 , pages 204 - 207 . ACM, 2011 .

[15] K. Guo , H. Mei , X. Xie , and X. Xu . A convolutional neural network feature fusion framework with ensemble learning for eeg-based emotion classification . 2019 IEEE MTT-S International Microwave Biomedical Conference (IMBioC) , 1 : 1 - 4 , 2019 .

[17] B. Hu , X. Li , S. Sun , and M. Ratcliffe . Attention recognition in eeg-based affective learning research using cfs+knn algorithm . IEEE/ACM Transactions on Computational Biology and Bioinformatics , 15 ( 1 ): 38 - 45 , 2018 .

[18] Shin ichi Ito, Yasue Mitsukura, Minoru Fukumi, and Jianting Cao . Detecting method of music to match the user's mood in prefrontal cortex eeg activity using the ga . 2007 International Conference on Control, Automation and Systems , pages 2142 - 2145 , 2007 .

[19] S. Ito , M. Ito , and M. Fukumi . Method to detect impression evaluation patterns on music listened to using eeg analysis technique . 2015 10th Asian Control Conference (ASCC) , pages 1 - 6 , 2015 .

[20] Vladimir Ivanov , Manuel Mazzara, Witold Pedrycz, Alberto Sillitti, and Giancarlo Succi . Assessing the Process of an Eastern European Software SME Using Systemic Analysis, GQM, and Reliability Growth Models: A Case Study . In Proceedings of the 38th International Conference on Software Engineering Companion (ICSE 2016 ), pages 251 - 259 , Austin, Texas, May 2016 . ACM.

[21] Andrea Janes and Giancarlo Succi . Lean Software Development in Action . Springer, Heidelberg, Germany, 2014 .

[25] J. Katona . Examination and comparison of the eeg based attention test with cpt and t .o.v.a. 2014 .

[26] P. Keelawat , N. Thammasan , B. Kijsirikul , and M. Numao . Subject-independent emotion recognition during music listening based on eeg using deep convolutional neural networks . 2019 IEEE 15th International Colloquium on Signal Processing Its Applications (CSPA) , pages 21 - 26 , 2019 .

[27] S. Koelstra , C. Muhl , M. Soleymani , J. Lee , A. Yazdani , T. Ebrahimi , T. Pun , A. Nijholt , and I. Patras. Deap: A database for emotion analysis ;using physiological signals . IEEE Transactions on Affective Computing , 3 ( 1 ): 18 - 31 , Jan 2012 .

[28] SeolHwa Lee , Andrew Matteson, Danial Hooshyar, SongHyun Kim, JaeBum Jung, GiChun Nam, and HeuiSeok Lim . Comparing programming language comprehension between novice and expert programmers using eeg analysis . In 2016 IEEE 16th International Conference on Bioinformatics and Bioengineering (BIBE) , pages 350 - 355 . IEEE, 2016 .

[29] K. Li , X. Li , Y. Zhang , and A. Zhang. Affective state recognition from eeg with deep belief networks . 2013 IEEE International Conference on Bioinformatics and Biomedicine , pages 305 - 310 , 2013 .

[30] Xiaowei Li , Bin Hu , Qunxi Dong, William Campbell, Philip Moore, and Hong Peng . Eeg-based attention recognition . In 2011 6th International Conference on Pervasive Computing and Applications , pages 196 - 201 . IEEE, 2011 .

[31] Wei-Gang Liang Chun-Hsiang Chuang Shao-Wei Lu Yi-Chen Lu Tien-Yang Hsiung Hsu-Hsuan Wu ChinTeng Lin Li-Wei Ko , Wei-Kai Lai . Single channel wireless eeg device for real-time fatigue level detection . 2015 .

[32] Stanislav Livinov , Marat Mingazov, Vladislav Myachikov, Vladimir Ivanov, Yuliya Palamarchuk, Pavel Sozonov, and Giancarlo Succi . A tool for visualizing the execution of programs and stack traces especially suited for novice programmers . In Proceedings of the 12th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE 2017 ), Porto, Portugal, April 2017 .

[33] A. Steed M. Donnerer . Using a p300 brain-computer interface in an immersive virtual environment . Presence: Teleoperators and Virtual Environments , vol. 19 , no. 1 , 2010 .

[34] J. A. Miranda-Correa and I. Patras . A multi-task cascaded network for prediction of affect, personality, mood and social context using eeg signals . 2018 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018 ), pages 373 - 380 , 2018 .

[35] Mostafa Mohammadpour and Saeed Mozaffari . Classification of eeg-based attention for brain computer interface . In 2017 3rd Iranian Conference on Intelligent Systems and Signal Processing (ICSPIS) , pages 34 - 37 . IEEE, 2017 .

[36] T. Ogawa , S. Karungaru , Y. Mitsukura , M. Fukumi , and N. Akamatsu . Feature extraction in listen'ng to music using statistical analystis of the eeg . 2006 SICE-ICASE International Joint Conference , pages 5120 - 5123 , 2006 .

[37] James A O'Sullivan , Richard B Reilly, and Edmund C Lalor. Improved decoding of attentional selection in a cocktail party environment with eeg via automatic selection of relevant independent components . In 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) , pages 5740 - 5743 . IEEE, 2015 .

[38] Mourad Ouzzani , Hossam Hammady, Zbys Fedorowicz, and Ahmed Elmagarmid . Rayyan-a web and mobile app for systematic reviews . Systematic Reviews , 5 ( 1 ): 210 , 2016 .

[39] Witold Pedrycz , Barbara Russo, and Giancarlo Succi . A model of job satisfaction for collaborative development processes . Journal of Systems and Software , 84 ( 5 ): 739 - 752 , 2011 .

[40] Etiel Petrinja , Alberto Sillitti, and Giancarlo Succi . Comparing OpenBRR, QSOS, and OMM assessment models . In Open Source Software: New Horizons - Proceedings of the 6th International IFIP WG 2.13 Conference on Open Source Systems, OSS 2010 , pages 224 - 238 , Notre Dame , IN, USA, May 2010 . Springer, Heidelberg.

[41] Bruno Rossi , Barbara Russo, and Giancarlo Succi . Modelling Failures Occurrences of Open Source Software with Reliability Growth . In Open Source Software: New Horizons - Proceedings of the 6th International IFIP WG 2.13 Conference on Open Source Systems, OSS 2010 , pages 268 - 280 , Notre Dame , IN, USA, May 2010 . Springer, Heidelberg.

[42] A. Saidatul , M. P. Paulraj , S. Yaacob , and M. A. Yusnita . Analysis of eeg signals during relaxation and mental stress condition using ar modeling techniques . 2011 IEEE International Conference on Control System, Computing and Engineering , pages 477 - 481 , 2011 .

[43] Debasis Samanta Shabnam Samima , Monalisa Sarma . Correlation of p300 erps with visual stimuli and its application to vigilance detection . 2017 .

[44] Alberto Sillitti , Andrea Janes, Giancarlo Succi, and Tullio Vernazza . Measures for mobile users: an architecture . Journal of Systems Architecture , 50 ( 7 ): 393 - 405 , 2004 .

[45] Alberto Sillitti , Giancarlo Succi, and Jelena Vlasenko . Understanding the Impact of Pair Programming on Developers Attention: A Case Study on a Large Industrial Experimentation . In Proceedings of the 34th International Conference on Software Engineering , ICSE ' 12 , pages 1094 - 1101 , Piscataway, NJ, USA, June 2012 . IEEE Press.

[46] Giancarlo Succi , Luigi Benedicenti, and Tullio Vernazza . Analysis of the effects of software reuse on customer satisfaction in an RPG environment . IEEE Transactions on Software Engineering , 27 ( 5 ): 473 - 479 , 2001 .

[47] Giancarlo Succi , Witold Pedrycz, Michele Marchesi, and Laurie Williams . Preliminary analysis of the effects of pair programming on job satisfaction . In Proceedings of the 3rd International Conference on Extreme Programming (XP) , pages 212 - 215 , May 2002 .

[48] H. Sun , L. Bi , X. Lu , B. Fan , and Y. Guo . Vigilance analysis based on eeg band power using support vector machine . 2015 8th International Congress on Image and Signal Processing (CISP) , pages 1090 - 1094 , 2015 .