=Paper= {{Paper |id=Vol-3070/short03 |storemode=property |title=Research Proposal: Integration of EEG and Oculometry Methods for External Cognitive Load Assessment in Graphical User Interfaces |pdfUrl=https://ceur-ws.org/Vol-3070/short03.pdf |volume=Vol-3070 |authors=Paula Andrea Cesarino Vargas,Luis Eduardo Bautista Rojas,María Fernanda Maradei García }} ==Research Proposal: Integration of EEG and Oculometry Methods for External Cognitive Load Assessment in Graphical User Interfaces== https://ceur-ws.org/Vol-3070/short03.pdf

Research proposal: integration of EEG and oculometry
methods for external cognitive load assessment in graphical
user interfaces

Paula Andrea Cesarino Vargas1, Luis Eduardo Bautista Rojas2, María Fernanda Maradei García3
1
Industrial University of Santander
paulacesarino@hotmail.com
2
Industrial University of Santander
luis.bautista@correo.uis.edu.co
3
Industrial University of Santander
mafemar@correo.uis.edu.co
Summary.Within the framework of the master's degree project in innovation and design, a
research proposal is proposed that presents the integration of objective techniques to evaluate the
cognitive load generated by the graphical user interface in an augmented reality environment, based
on an experimental design unifactorial, cross-sectional, retrospective, at an explanatory level. This
proposal is relevant as it will provide empirical evidence for the graphical user interface design
process. It is expected, at the end of this project, to have a procedure manual for the evaluation of
graphical user interfaces based on the level of external cognitive load.

Keywords: GUI, Graphical user interface, External cognitive load, EEG, eye tracking, Augmente
reality.

1. Introduction
With the fourth industrial revolution, technologies such as augmented reality (AR) have been
making headway in areas such as learning and skills training, given their ability to merge real and
virtual objects in practice environments and the ability to provide information in time. real to
apprentice (Akçayır & Akçayır, 2017). Several studies have shown that cognitive overload can be
an important aspect of usability(Adams, 2007) and augmented environments are likely to be
particularly sensitive to its effects.

When a person is making use of an augmented environment, a series of cognitive processes are
triggered directly linked to that activity. The level of cognitive load is an indicator that could be used
to evaluate increased user-environment interaction. In the nineties, Sweller(Sweller, 1994)proposed
the cognitive load theory (CLT), an instructional theory based on the knowledge of human cognitive
architecture and focused directly on the limitations of working memory and the automation of long-
term memory schemas. However, despite the fact that CLT is widely known from its scientific
theory, its measurement for instructional materials (especially in multimedia teaching) is mainly
based on indirect, subjective or both methods.(Brunken et al., 2010).
This proposal focuses on the process of designing graphical user interfaces for augmented reality
environments, specifically in the evaluation phase of design alternatives. Focused on defining a
framework that allows the designer to objectively evaluate his interface from the level of extrinsic
cognitive load generated by the visual stimulus. The foregoing, through a cross-technique approach
of electroencephalography (EEG) and eye tracking, providing the designer with pertinent
information for the redefinition of the interfaces.

1.1 Formulation of the problem

The Cognitive Load (CC) can be defined as the total amount of mental activity processed,
consciously, at the moment in which the subject completes a task (Paas et al., 2003). However, not
all CC is the same type; There are three classes: a) intrinsic, b) extrinsic, and c) Germanic. Intrinsic
QC is related to the complexity of the task itself, depending on the conceptual difficulty of the
material and the skill of the learner to develop it; the extrinsic CC is responsible for contaminating

Copyright © 2021 for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 1
or saturating the information that is presented, thus affecting working memory; generally it relates
to multimedia materials or interfaces. Finally, Germanic CC is directly related to learning and the
processes of abstraction and information processing towards long-term memory.(Sweller, 1994).
The measurement of external cognitive load continues to be a challenge, since this construct
cannot be measured directly, but rather requires the evaluation of dimensions related to it, which
are: mental load, mental effort and performance(Andrade-Lotero, 2012). Many augmented reality
applications involve tasks that require the use of a large amount of cognitive resources and since
users have limited them, if an evaluation is not carried out to verify that the graphical interface
design does not impose an external cognitive load on the user high, technology could add more
complexity and influence the learning process. However, conducting this assessment during
augmented reality experiences is especially complex, since learners are exposed to a large amount
of interaction.(Akçayir et al., 2016). An essential question in this study is whether there are valid,
reliable and practical methods that can measure external cognitive load in augmented reality
environments.
In the literature, two different approaches are generally found for such evaluation: a) requesting
users to subjectively rate their perceived cognitive load or b) using objective measures, commonly
physiological measures. In this proposal, an objective magnitude that is robust, continuously
measurable and sensitive enough to solve the bad signal-to-noise ratio is sought.

1.2 Justification

As human-computer interaction (HCI) systems are becoming ubiquitous and used to perform critical
tasks in different domains, the need to measure the cognitive load caused by a purpose-designed
HCI system is becoming increasingly common. increasingly important in stages prior to
implementation (Neville et al., 2005). Although HCI evaluation methods have also advanced with
the evolution of systems and techniques, such as verbal reports, observations of task fulfillment,
concurrent verbalization, questionnaires, etc., they are being used to evaluate interactive systems for
their effectiveness and efficiency, however, there is still a shortage of methods that can directly
measure the cognitive load caused by the system.
Current tools have allowed us to measure cognitive load from a weighted analysis; An interesting
observation is that although researchers are continually trying to find or develop secondary task and
physiological measures, the internal consistency of these measures requires further study. Especially
to determine the extrinsic cognitive load, referring to when the learner is interacting with a material
or interface whose design or execution has irrelevant elements that hinder the processes of both
construction and automation of schemes, which will allow the learning of complex concepts.

2 Methodological framework
To carry out the development of the project, a methodology based on the methodological approach
of(Blessing & Chakrabarti, 2009)one of the popular models for design research; This method
belongs to the field of research in design engineering and consists of 4 stages: definition of criteria,
descriptive study I, prescriptive study and descriptive study II. Each of the objectives of the Design
Throught Research (DTR) phase

Table 1. Methodology for the fulfillment of the objectives related to the DTR methodology

Target Research level Activities Method Subject
Identify the variables that Typical descriptive Theoretical project Context analysis Cognitive load in
determine the level of framework augmented reality
cognitive activity Experimental EEG
through EEG and Evaluation metrics analysis Oculometry
oculometry analysis
Assess cognitive load Descriptive Determine the level of Experimental EEG
variables with respect to correlational reliability of the objective Oculometry
brain waves and eye cognitive load External cognitive
movement measurement with respect load scale
to the subjective one
Evaluate the integration Explanatory To determine the level of Experimental EEG
of the two techniques as Descriptive and reliability of the cognitive Oculometry
a tool for the objective inferential statistical load measurement with a Cross technique
measurement of external analysis from the multimodal approach
cognitive load. comparison of means. compared to the unimodal
one

2
It is expected, at the end of the development of this research work, to have a procedure manual
that describes the activities related to the evaluation of external cognitive load of graphical user
interfaces applying crossed techniques; This manual will have a chronological narrative that
systematically requires data capture, a guide for the treatment of electroencephalographic signals
and a list of metrics that differentiate the levels of external cognitive load resulting from its interface,
in this way, the designer will be able to evaluate objectively its design alternatives for augmented
reality environments and obtain the necessary information for its redefinition, if required.
The above, based on the case study for the evaluation of external cognitive load of a prototype
for learning and training activities in an augmented environment, which allows to demonstrate the
significant improvement of the evaluation of cognitive load of each of the techniques separately.
compared to the application of integrated techniques.

2.1 Research design

This work is framed as an investigation through design, experimental, retrospective and cross-
sectional at an explanatory level. The hypothesis proposed by the researcher affirms that multimodal
approaches that combine data from different sensors improve the specific evaluation of external
cognitive load caused by the stimulus in graphical user interfaces.
To confirm the previous hypothesis, the development of an experiment with a unifactorial design
is proposed, in order to verify the values for the estimation of cognitive load from the frequency,
voltage and amplitude of the waves. This information will be acquired with the EMOTIV EPOC +
device, which allows us to recognize the report of performance metrics of stress, commitment and
concentration. On the other hand, for the analysis of the percentage of pupil dilation, the SMI eye
tracking device (SMI Eye Tracking Glasses SMI ETG) will be used; with the aim of verifying the
information obtained in the literature and determining the level of reliability of the evaluation
methods with respect to a subjective scale.

Table 2. Scheme of the experiment

Independent variable Dependent variable Controlled variables
Augmented interface design Extrinsic cognitive load Training activity

Number of treatments: 2 Range O1 - Related to EEG illumination
Frequency, voltage and
Treatment 1 (T1) amplitude of brain waves
Low level of interactivity
O2 Range - Related to eye
Treatment 2 (T2) tracking
High level of interactivity Number of blinks
Percentage of pupil dilation

The test will be carried out in person in a closed room with controlled lighting. The protocol
begins with the explanation of the experiment, and the signing of the informed consent. The user is
directed to a hair washing area in order to meet biosafety parameters and at the same time to control
the humidity of the scalp that will facilitate subsequent data collection. Once the washing is done,
the subject is positioned in front of the guide marks to start with the calibration of the SMI glasses,
the user must follow the marks presented; Afterwards, the EPOC + electroencephalograph will be
positioned and through the EMOTIV APP application, a quality of contact of the electrodes will be
established not lower than 95%, to finalize the assembly of equipment, the Hololens augmented
reality glasses will be positioned.
For the analysis of results, the ordinal categorical variables will comply with a statistical analysis
based on contingency tables and frequency diagrams. Likewise, a comparison of samples will be
carried out to determine the significant differences between them, using the Chi-square test statistic
with a p-value <0.05 to reject the null hypothesis of equality.
In the case of continuous data, once the data has been obtained and filtered, it will be analyzed
descriptively through the SPSS statistical software where the internal behavior of the data is
analyzed from measures of central tendency and position, such as box plots and whiskers to establish
consistency in the data.

3
Finally, the inferential analysis will be carried out, in this case, if the data are parametric, a mean
comparison study will be carried out using ANOVA as a test statistic and it will be considered that
there are significant differences for p-value <0.05. For non-parametric data: The Kruskal-Wallis test
would be used initially to determine if there are differences between the groups and later a Wilcoxon
signed rank test to compare the two related samples.
With the analysis of the data, it is expected to accept the established hypothesis that it supposes
an improvement in the specific evaluation of external cognitive load caused by the stimulus in
graphical user interfaces from the use of multimodal approaches.

References
1. Adams, R. (2007). Decision and stress: cognition and e-accessibility in the information workplace. Univ
Access Inf Soc, 5, 363–379. https://doi.org/10.1007/s10209-006-0061-9
2. Akçayir, M., Akçayir, G., Pektaş, HM, & Ocak, MA (2016). Augmented reality in science laboratories: The
effects of augmented reality on university students' laboratory skills and attitudes toward science
laboratories. Computers in Human Behavior, 57, 334–342. https://doi.org/10.1016/j.chb.2015.12.054
3. Akçayır, M., & Akçayır, G. (2017). Advantages and challenges associated with augmented reality for
education: A systematic review of the literature. Educational Research Review, 20, 1–11.
https://doi.org/10.1016/j.edurev.2016.11.002
4. Andrade-Lotero, LA (2012). Cognitive load theory, design and multimedia learning: A state of the art. Magis:
International Journal of Research in Education, 5 (10), 75–92.
5. Blessing, & Chakrabarti. (2009). DRM: A Design Research Methodology. In DRM, a Design Research
Methodology (pp. 11–42). Springer US. https://doi.org/https://doi.org/10.1007/978-1-84882-587-1_2
6. Brunken, R., Plass, JL, Leutner, D., & Brünken, R. (2010). Direct Measurement of Cognitive Load in
Multimedia Learning Direct Measurement of Cognitive Load in Multimedia Learning. Educational
Psychologist, 38 (1), 53–61. https://doi.org/https://doi.org/10.1207/S15326985EP3801_7
7. Neville, AS, Salmon, PM, Walker, GH, Baber, C., & Jenkins, DP (2005). Human Factors Methods: A
Practical Guide for Engineering. In Ashgate Publishing (Ed.), Angewandte Chemie International Edition
(First Edit, Vol. 6, Issue 11, pp. 77–115). Taylor & Francis.
8. Paas, F., Tuovinen, JE, Tabbers, H., & Van Gerven, PWM (2003). Cognitive load measurement as a means
to advance cognitive load theory. Educational Psychologist, 38 (1), 63–71.
https://doi.org/10.1207/S15326985EP3801_8
9. Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and
Instruction, 4 (4), 295–312. https://doi.org/10.1016/0959-4752(94)90003-5