This study seeks to evaluate a virtual reality system and games available on the PlayStation 4 VR system. Participants completed two questionnaires during the experiment. The first questionnaire was developed to evaluate simulator sickness in high-fidelity simulators [3]. The second questionnaire was developed by our lab to assess a number of characteristics of the VR environment (e.g., image quality, auditory surround, orientation, pointing, lag, and control). We have collected a sample of 85 participants. Participants were asked to complete surveys about demographics and current video game use and then choose a game from a list of ten different games to play on the VR system. The seven games were Demo Disk, Batman, Dangerball, Eve Valkyrie, Headmaster, London Heist, Rush of Blood, and VR Luge. Each participant played the game for about 30 minutes. Finally, they completed the simulator sickness and the VR system questionnaires. Simulator sickness was low for 58% of the participants, mild for 33%, and strong for 9%. Scores varied among the games. Ratings for sound location and image lag were good but calibration, head gear, some image blurriness were less favorable. These results can be used by game producers to develop more satisfactory VR game environments.
Miscellaneous.
INTRODUCTION
Virtual reality systems have been in research and
development for decades since the early days of Sutherland
and his students [
Virtual reality environments create a number of challenges for the user which will be discussed in the next two sections.
GHItaly18: 2nd Workshop on Games-Human Interaction, May 29th, 2018, Castiglione della Pescaia, Grosseto (Italy) Copyright © 2018 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
Simulator Sickness
Simulator sickness was first observed when persons were
being trained using flight simulators and other training
environments particularly in the military. The common
symptoms are general discomfort, headache, nausea,
vomiting, pallor, sweating, fatigue, drowsiness, and
disorientation [
After testing out a number of VR games and considering
the literature on the human factors in VR, a questionnaire
was designed to assess a number of these factors on a
9point scale. The virtual reality system questionnaire
(VRSQ) is shown in Figure 3. This study will be used to
evaluate this questionnaire and improve it for future
research.
Usability and Play Testing
In order to assess levels of simulator sickness and evaluate
games and systems, participants were asked to engage in a
form of play testing [
Study In order to assess the prevalence of virtual reality sickness and to assess human factors issues, a popular commercial system was selected and a variety of different games were selected for testing. Participants played the games and completed the questionnaires.
METHODOLOGY System and Games The PlayStation 4 VR system was used with its headgear, PlayStation Eye, and either a standard controller or two PlayStation Move controllers. The system was set up in a user testing room that was 10 x 18 feet. The participant was usually seated in a swivel chair or for some games standing in a fixed position. An assistant set up the game, helped the participant with the headgear and controller(s), and in some instances helped to instruct the participant as to what to do during the game.
The seven games were selected to cover a range of genre and interaction styles. The games were as follows: Demo Disk: Selection of demo games. Head position, PS controller, or PS Move controllers can be used to select demo games. Demo games use a variety of interaction styles.
Batman: Arkham VR: Action adventure mystery. Move controllers are used to grab and throw objects and for menu selection. Movement is from one discrete location to another via teleportation. Virtual hands are displayed. Danger Ball: VR Worlds: Sports action similar to dodgeball. Head, hands, and feet are used to dodge and hit balls at targets. No controller.
EVE: Valkyrie: Action shooter. PS controller is used to steer a spaceship and to fire weapons at enemies. Player is seated in the moving spaceship.
London Heist: Action adventure first person shooter. PS controller or PS Move controllers are used to shoot and grab objects. Player is in fixed positions. Virtual hands and guns are displayed.
Rush of Blood: First person shooter. PS Move controllers appear as guns in the player’s hands. Virtual legs and feet are also displayed. Player is seated in a roller coaster car that moves on its own.
VR Luge: Sports racing. Steering of the sled is done by moving the head to left or right. No controller is used. Virtual body is shown on the sled.
Participants Undergraduates were recruited from psychology courses at the University of Maryland and received partial credit for participation. The sample was composed of 36 females and 49 males and ranged in age from 18 to 45. The mean age was 19.87, the median 19, and the mode 18. Participants varied widely in video game experience, time and frequency of play per day, and types of games played. None of the participants had played a VR game before. Procedure Participants were greeted and told what was expected of them in the study and signed an informed consent form. First they completed a demographic question and a survey of their video game interests and experience. They were then asked to select a game to play from a list giving the title of the game, its ESRB rating, cover art for the game, and a short explanation of what the game was about. A lab assistant set up the game, helped with the headgear and controllers, and started the game. Participants played the game for 30 minutes unless they asked to stop due to sickness. Eight of the 85 participants asked to stop due to sickness.
After playing the game, participants were asked to fill out the SSQ and VRSQ questionnaires.
RESULTS VR Sickness Figure 4 shows the order of occurrence of the 16 symptoms, with headache, nausea, sweating, fatigue, general discomfort, “fullness of head” and eyestrain topping the list.
Participants varied greatly in their susceptibility to VR sickness. Clearly some participants experienced VR sickness in some games. Eight participants requested to stop the game due to sickness. Figure 5 shows the distribution of SSQ scores for the 85 participants. The majority of participants fell in the low range (58%), a third fell in the mid range (33%), and 9% fell in the high range.When clustered by game as shown in Figure 6, it is clear that some games led to higher levels of VR sickness than others. Eve: Valkyrie was the worst, probably due to the disorientation of spinning around in the spaceship. The Demo Disk was second. In this case, participants went from a central VR space into demo games and back. This resulted in a change in orientation and mode of controlling the interaction which could lead to disorientation. The VR Luge was third worst. In this game, the player is whipping down a winding road at high speed trying to avoid collisions with moving vehicles and stationary objects.
Games in which the player was in a stationary or seated position (The London Heist, Rush of Blood, and Batman) led to the least VR sickness.
Human Factors Figure 7 shows the mean rating for each item on VRSQ. All of the ratings were above midpoint of 5 except for “Some of the image was blurred.” The lab assistants and participants found that perspiration often fogged up the headgear and blurred part of the field of view. “Trying to see what is behind” was difficult either turning the head or swiveling the chair. Headgear was also a problem. Participants were not initially familiar with how to adjust the helmet and the viewing screen for maximum comfort. The lab assistant usually had to help them. Methods of calibration varied from game to game and again the lab had to help. On the positive side, there were few auditory problems and sounds were easy to locate. The image did not skip or lag, it appeared in 360 surround, and it was easy to look to the left or the right.
When sorting these ratings out by game as shown in Figure 8, the largest differences were for “Trying to see what is behind.” Batman faired the best probably because the game action was always directly in front of the player. Eve: Valkyrie was the worst due to the configuration of the spaceship. Large differences also occurred for “Locating buttons on the PS controller.” This appeared to be due to the extent of use of the PS controller in the game. The same was true for “Aiming at targets using the head” again probably due to the extent of use of this method in the game. Other differences also occurred due to the game’s reliance one that feature. “Fast image lag” was a problem for VR Luge due to the high rate of speed of the player moving through the streets; whereas in Rush of Blood, the VR space and targets within it did not move as fast. In games such as Eve: Valkyrie that requires the player to look all around for objects, “All of the image blurred” was a problem whereas in London Heist and Batman, the player really only needs to focus on one spot.
CONCLUSION
The use of virtual reality in video games is in its infancy.
Considerable research and development is needed to
integrate game play into the VR environment. Game
producers need to play-test their games to evaluate them
across a number of factors for playability [
In order to control the effects of the VR environment, developers need to evaluate games along a number of human factors. The VRSQ was designed in this study to assess these factors. The visual quality of the environment depends on the clarity of the image of game focus as well as the 360 surround for a number of games. The image lag is particularly relevant for fast moving environments in the game play. Many different methods for menu selection, pointing, aiming, shooting, and grasping are being explored. Movement through the VR space whether continuous or discrete point-to-point must be worked out. Each factor and interaction style needs to be evaluated by players as the game is developed and in final play testing. The VRSQ is meant to assist in game development to minimize virtual reality sickness and maximize the player’s VR game experience.
ACKNOWLEDGMENTS Thanks to Emily Maranto, Kaitlin Cossentino, Karsyn Wagner, Meredith Reid, Joey Barnet, and Reza Mahmud for running participants and assisting in the data analysis.