On the assessment of user experience in VR traffic environments, a method to evaluate the mental state of subjects is required to measure the effects of dynamic events in the traffic environments. EEG is one of the promising devices for measuring the subject mental status. We propose to analyze EEG responses against traffic scenes, including traffic accidents, for the evaluation of the surprise feelings of subjects in VR traffic environments by a set of HMD and an EEG device. Throughout the experiments, we examined whether EEG can evaluate subjects' emotional states in response to the events in VR space. Three traffic scenes and 10 subjects were prepared for the experiment. The EEG analysis was made on beta frequency band power related to surprise response.
State-of-the-art VR technology could provide
an immersive experience of the real world [
In conventional studies, a subjective
evaluation questionnaire is used for the
assessment [
The use of physiological signals could solve
this problem. Among physiological signals, we
focused on EEG. It has been reported that EEG
can estimate the emotional state of subjects [
Studies on VR and EEG have also been
conducted [
In this paper, we propose to analyze EEG responses against traffic scenes, including traffic accidents, for the evaluation of the surprise feelings of subjects in VR traffic environments. We chose traffic accidents as a dynamic event.
Subjects in the experiment will have a
surprising experience through an accident caused
by a momentary collision of vehicles reproduced
in a VR space. The beta frequency band in the
frontal lobe EEG is used for EEG analysis. It has
been reported as an indicator of vigilance and
concentration [
The VR space is constructed using Unity. We prepared ten subjects for the experiment. The subjects will watch vehicle collision scenes a total of two times.
Studies have been conducted using VR and
EEG [
The study by Dorota Kaminska et al. classified
stress levels in a VR environment [
The study by Valasileios Aspiotis et al.
investigated whether the experience of heights in
a VR space affects stress using EEG [
Based on these previous studies, it is possible to estimate consciousness in the state of wearing VR and present an experimental environment similar to the natural environment. 2.2.
Studies using EEG have been applied various
fields, including physiological research such as
cognition, medicine and education [
Wei-Lang et al. reported that frontal lobe EEG
is effective in emotion estimation [
Saira-Bano et al. reported that subjects' fatigue
affects the beta frequency band [
Subjects wear the VR device while wearing the
electroencephalograph. A previous study
reported that wearing the VR device and
electroencephalograph simultaneously did not
affect the data [
We adopt only FP1 and FP2 at the frontal lobe for the EEG sensing as these two are thought to play a dominant role in finding surprising feelings.
When dealing with EEG, the effects of noise
must be considered [
For the EEG data obtained from the experiment, the EEG was converted from the time domain to the frequency domain by FFT. The time average of the beta frequency band was then calculated, and the beta frequency band power was calculated. In the following, the beta frequency band power is called BP.
In this study, a road space was constructed in a VR space. The VR space consists of 155m x 160m. Figure 2 shows the actual road space constructed.
The VR situation in the experiment consists of three turns: Turn1 to familiarize the subject with VR, Turn-2 with no accidents, and Turn-3 with accidents. Turn-1 is a video to familiarize the subject with the VR space and includes Turn1-1 with no vehicles present (Figure 3) and Turn-1-2 with vehicles present (Figure 4). Turn-2 is where no vehicle collisions occur (Figure 5). Turn2 is provided for comparison with Turn-3 (Figure 6).
Subjects are positioned in the VR space at the roadside. Figure 7 shows the subject's placement position and viewing range. Subjects watch the scene in VR within the range shown in Figure 7.
The surprise experience of this study is presented in Turn-3. In this turn, a vehicle crash scene is recreated, and the subject watches the vehicle crash scene in front of the subjects. All VR turns consist of 30 seconds. The vehicle collision, in Turn-3, occurs 20 seconds after the start of the turn.
This study used HTC's VIVE Pro Eye (VIVE) as the VR device and Emotive's EMOTIV EPOC Flex as the EEG measurement device.
The VR space was constructed using Unity. The VIVE Pro Eye has a display resolution of 1440 x 1600 pixels per eye, a frame rate of 90 Hz, and a viewing angle of 110 degrees.
The time window to calculate the frequency of EEG is set to 2.0 seconds at the sampling frequency of 128Hz. The window is shifted at 0.125 seconds intervals.
Before starting the experiment, subjects were instructed not to move their heads. This instruction was to minimize noise generation due to head rotation. Because the subject's visual range is limited, the vehicle collision video was set to occur in front of the subject's eyes.
The experiment consists of turns for EEG acquisition during closed-eye rest and open-eye rest, Turn-1, Turn-2, and Turn-3. Turn-2 and Turn-3 are played twice. In the following, the first occurrence of Turn-2 is referred to as Turn-2-I, and the second occurrence of Turn-2 is referred to as Turn-2-II. The same convention applies to Turn-3, with Turn-3-I and Turn-3-II.
Figure 8 shows an overview of the experiment flow. After the experiment, a subjective evaluation questionnaire was conducted using a 5point rating scale. The question is “How much are you surprised by the scene you watched?” It was conducted using rating items where 5 indicated “surprised” and 1 indicated “not surprised.” Figure 9 shows the questionnaire sheet.
The EEG was analyzed during the resting state with closed eyes, resting state with open eyes, Turn-2, and Turn-3. The analysis time for the resting state with closed eyes, resting state with open eyes and Turn-2 was 30 seconds, while the time used to analyze Turn-3 was 3 seconds after the vehicle collision.
We report the results of an experiment. 10 people took part in the experiment. All subjects were interviewed prior to the experiment, and it was known that all of them had VR experience. All subjects were in their 20s (9 males and 1 female).
Figure 11 shows the changes in BP during open-eye rest, Turn-2-I, and Turn-3-I relative to the baseline values of BP during closed-eye rest. The horizontal axis is the subject label, and the vertical axis is the BP value. Turning to the bar graphs for each subject, the analysis results for open eye rest, Turn-2, and Turn-3 are displayed.
As a result, BP was highest during the traffic accident in 6 out of 10 people.
Figure 12 shows the increase and decrease in BP for Turn-3-II when Turn-3-I was used as the baseline. Both the horizontal and vertical axes are the same as in Figure 5. As a result, in 8 out of 10 people, Turn-3-I had higher BP values than Turn3-II.
Based on the subjective evaluation questionnaire results, this study provided the participants with a sufficient surprise experience. The lower values in the subjective evaluation questionnaire for Turn-3-II than for Turn-3-I were due to familiarity with the repeated occurrence of the event.
The results of the EEG analysis showed that BP values were maximal for more than half of the subjects. This result suggests that it is highly likely that the BP was obtained in the EEG when the subjects were surprised. Comparison with the EEG data obtained during the repetitions showed higher BP values for Turn-3-I than for Turn-3-II in most subjects. This EEG result could be consistent with the results of the subjective evaluation questionnaire and could be due to the same cause.
There are individual differences in EEG. The fact that no increase in BP was observed for all subjects in this study is mainly due to individual differences. In order to consider that BP is associated with surprise, a statistical study needs to be conducted with a more significant number of people. In this study, the age range of the subjects was limited to those in their 20s. It is necessary to expand the scope of the survey in order to consider the more general application.
We proposed to analyze EEG responses against traffic scenes, including traffic accidents, for the evaluation of the surprise feelings of subjects in VR traffic environments. We chose traffic accidents as a dynamic event. We focused on surprise feeling as an emotional state and the analysis was conducted on the beta frequency band.
Three traffic scenes and 10 subjects were prepared for the experiment. We conducted a traffic experiment and EEG analysis focusing on the emotional state of surprise to examine whether EEG is effective in estimating emotion for momentary events in VR experiments.
The relationship between surprise and EEG was investigated using EEG data obtained when subjects were surprised and a subjective evaluation questionnaire administered after the experiment.
The experiment results showed that BP values increased the most during the surprise experience in 6 out of 10 people. As a result of the repetition of the surprise experience, in 8 out of 10 people, the BP value decreased during the second surprise experience compared to the first surprise experience. This could be consistent with the results of the subjective evaluation questionnaire.
Part of this research was supported by JSPS Kaken 21H03476. 3D scene of the traffic environment is provided by Professor Hiroaki Yano at University of Tsukuba.