=Paper=
{{Paper
|id=Vol-2610/paper10
|storemode=property
|title=Endowing Head-Mounted Displays with Physiological Sensing for Augmenting Human Learning and Cognition
|pdfUrl=https://ceur-ws.org/Vol-2610/paper10.pdf
|volume=Vol-2610
|authors=Evangelos Niforatos,Athanasios Vourvopoulos ,Michail Giannakos
|dblpUrl=https://dblp.org/rec/conf/lak/NiforatosVG20
}}
==Endowing Head-Mounted Displays with Physiological Sensing for Augmenting Human Learning and Cognition==
https://ceur-ws.org/Vol-2610/paper10.pdf
th
Companion Proceedings 10 International Conference on Learning Analytics & Knowledge (LAK20)
Endowing Head-Mounted Displays with Physiological Sensing for
Augmenting Human Learning and Cognition
Evangelos Niforatos
Norwegian University of Science and Technology (NTNU), Trondheim, Norway
evangelos.niforatos@ntnu.no
Athanasios Vourvopoulos
Instituto Superior Técnico, University of Lisbon
athanasios.vourvopoulos@tecnico.ulisboa.pt
Michail Giannakos
Norwegian University of Science and Technology (NTNU), Trondheim, Norway
michailg@ntnu.no
ABSTRACT: The EEGlass prototype is a merger between a Head-Mounted Display (HMD) and
a brain-sensing platform with a set of electroencephalography (EEG) electrodes at the
contact points with the skull. EEGlass measures unobtrusively the activity of the human brain
facilitating the interaction with HMDs for augmenting human cognition. Among others,
EEGlass is intended for collection of context-aware EEG measurements, supporting learning
and cognitive experiments outside the laboratory environment. Thus, we expect EEGlass will
promote the implementation and application of ecologically valid research methods (studies
in the user's natural context).
Keywords: Head-Mounted Displays, Electroencephalography, Brain-Computer Interfaces,
Neuroadaptive Systems.
1 INTRODUCTION
Human cognition is typically a composite notion we use for describing the states of the cognitive
processes that underpin it. Namely, attention; memory recall; learning; decision-making; and
problem-solving. Augmenting human cognition boils down to gauging the states of the underlying
cognitive processes and deciding on an intervention. On one hand, electroencephalography (EEG),
and other measures of physiological responses, have been extensively utilized for measuring
attention, monitoring cognitive workload, assessing learning experience, and even evaluating
software usability. On the other hand, contemporary HMDs, such as Augmented Reality (AR) smart
glasses, are progressively becoming socially acceptable and ubiquitous by approaching the size and
design of normal eyewear (Niforatos & Vidal, 2019). Thus, a merger between HMDs and EEG appears
to be promising. HMDs bear a significant potential in hosting an array of physiological sensors in
contact with the human skull, while situated in front of our most highly-esteemed perceptive organ:
our eyes. In this work, we draw on the HMD form factor for designing, developing, and evaluating
EEGlass, an EEG-Eyeware prototype for ubiquitous brain-computer interaction (Vourvopoulos et al.,
2019).
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 1
th
Companion Proceedings 10 International Conference on Learning Analytics & Knowledge (LAK20)
2 EEGLASS PROTOTYPE
The latest version of the EEGlass prototype (see Figure 1) is comprised of a Vuzix Blades1 (Vuzix,
Rochester, USA) HMD fitted with EEG electrodes that connect to a Cyton Biosensing Board by
OpenBCI2 (OpenBCI, NY, USA). Vuzix Blades is a pair of AR smart glasses that features a monocular
and transparent waveguide display, with a 19-degrees field of view, and a resolution of 480 x 853
pixels. Vuzix Blades is equipped with an 8MP camera, Bluetooth and Wi-Fi connectivity modules, a
range of sensors (e.g., inertial measurement unit, microphones, etc.), and runs Android OS. OpenBCI
is a popular and relatevely low-cost open hardware and software platform for the collection and
analysis of biosignals such as EEG, EMG (Electromyography), and ECG (Electrocardiography), inspired
by the grassroots movement of DIY (“Do It Yourself”). The Cyton board encompasses 8 biopotential
input channels (for hosting up to 8 electrodes), a 3-axes accelerometer, local storage, Bluetooth
connectivity module, while being fully programmable and Arduino compatible. Evidently, the
EEGlass electrode topology is restricted by the form factor of Vuzix Blades and at the contact points
with the skull. Thus, EEGlass utilizes 3 electrodes (plus 2 for reference and ground) based on the
standard 10-20 EEG system (see Figure 1) for measuring brain activity: 1 electrode placed inwards at
the top of the eyewear bridge touching the skull at glabella, and 2 more electrodes at the inner side
of the eyewear temples, touching the left and right mastoids, behind the left and right ears,
respectively. Both the Cyton Board and Vuzix Blades are connected to an external power source for
enabling and prolonging mobile usage.
3 CURRENT STATE AND NEXT STEPS
Our first aim is to investigate how reliably EEGlass can capture brain activity, particularly when
featuring an electrode topology imposed by the form factor of an HMD. For this, we compare brain
activity captured via EEGlass with that captured via a standard EEG system as baseline. So far, we
have tested a previous version of the EEGlass prototype, implemented with eyewear frames. Limited
trials with 1 participant indicated that the EEGlass is capable of capturing brain activity manifested in
two modes of resting state: (a) eyes open and focused on a target, and (b) eyes closed. Brain activity
recorded during resting state with EEGlass demonstrated similar variations in frequency and
amplitude to when recorded with an established EEG system. Recorded brain activity linked to upper
limb motor-action displayed significant differences when compared to that captured with an
established EEG system due to the fundamentally different electrode topology of EEGlass.
Nevertheless, EEGlass managed to capture upper limb motor-action relying on signal propagation
over the skull through volume conduction (van den Broek et al., 1998). EEGlass also detected subtle
eye movements in 4 basic directions, displaying an eye-tracking potential particularly useful for
navigating in HMD interfaces.
Low sample size (N=1) and stationary experimental settings are significant limitations that we will
address over the next studies. However, human skull and brain anatomy is almost homogeneous,
1 https://www.vuzix.com/products/blade-smart-glasses
2 https://openbci.com/
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 2
th
Companion Proceedings 10 International Conference on Learning Analytics & Knowledge (LAK20)
and the HMD form factor ensures a rather stable electrode contact with the skull, only somewhat
influenced by movement. In future iterations, we will utilize machine learning for training algorithms
to match input from EEGlass to that of established EEG systems. We believe a merger between EEG
and HMDs bears an unprecedented potential to “close the loop” by increasing the communication
bandwidth between human and machine and paving the way for cognition-aware systems (Niforatos
et al., 2017).
(a) (b)
TP9
TP10
Cyton Board by
Power source OpenBCI
Nz
Ref. GND
Figure 1: (a) The EEGlass prototype comprised of a Vuzix Blades HMD fitted with 5 EEG electrodes,
based on the 10-20 system, connecting to a Cyton OpenBCI board and a mobile power supply. (b)
A user wearing EEGlass.
4 APPLICATIONS FOR LEARNING
Besides the promising application areas in augmenting human cognition in general, we believe
EEGlass also bears significant potential in facilitating learning. For example, after investigating
EEGlass in reliably measuring cognitive activity in the wild, we will introduce it to the classroom.
Although EEG can capture the subtle cognitive processes associated with learning (e.g., attention
and concentration levels), performing EEG experiments in a classroom with the typical EEG headsets
is deemed cumbersome and often inappropriate. Thus, we expect that EEGlass can be a viable
alternative in collecting unobtrusively the brain activity of students related to learning. Moreover,
the HMD component of EEGlass can be utilized for projecting information about the learning
content in pre, post or during learning stage, and even on the go. We expect that by presenting our
prototype to the CrossMMLA workshop, we will spark ideation and generate discussions about
different applications and user scenarios for EEGlass about enhancing learning and the entire
spectrum of human cognition.
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 3
th
Companion Proceedings 10 International Conference on Learning Analytics & Knowledge (LAK20)
ACKNOWLEDGEMENTS
This project is a product of a research collaboration between Norwegian University of Science and
Technology (NTNU) and Institute for Systems and Robotics (ISR-Lisbon), receiving funding from Swiss
National Science Foundation (SNSF) and the ERCIM “Alain Bensoussan” Fellowship.
REFERENCES
van den Broek, S. P., Reinders, F., Donderwinkel, M., & Peters, M. J. (1998). Volume conduction
effects in EEG and MEG. Electroencephalography and clinical neurophysiology, 106(6), 522-
534.
Niforatos, E., Vourvopoulos, A., & Langheinrich, M. (2017, September). Amplifying human cognition:
bridging the cognitive gap between human and machine. In Proceedings of the 2017 acm
international joint conference on pervasive and ubiquitous computing and proceedings of the
2017 acm international symposium on wearable computers (pp. 673-680). ACM
Niforatos, E., & Vidal, M. (2019, March). Effects of a Monocular Laser-Based Head-Mounted Display
on Human Night Vision. Retrieved from https://dl.acm.org/citation.cfm?id=3311858
Vourvopoulos, A., Niforatos, E., & Giannakos, M. (2019, September). EEGlass: An EEG-eyeware
prototype for ubiquitous brain-computer interaction. In Proceedings of the 2019 ACM
International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of
the 2019 ACM International Symposium on Wearable Computers (pp. 647-652). ACM
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 4