Energetic music has the potential to enhance immersion but can also negatively impact task performance. This study measured the efects of music on a video game suited for energetic music using both subjective (questionnaires) and objective (facial and physiological analysis) measures. Participants played under two conditions: with and without music. While the music condition led to higher reports of immersion, no significant diference between the two conditions was found. Performance scores were better without music, suggesting that energetic music, while increasing immersion, may disrupt performance by intensifying the task. These findings emphasize the importance of aligning immersion with task relevance, as the efects of music on performance are context-dependent. Further investigation into the balance between engagement, task perception, and performance is needed.
Music has been shown to have a significant negative efect on cognitive performance,
especially on high-intensity tasks [
A study by [
Another study [
Given these findings, this study aims to explore the efect of energetic music on task immersion and perception while playing a vertically scrolling video game or vertical scroller type of a video game1. Specifically, we will investigate how varying levels of immersion impact task performance, and if higher immersion is indeed related to higher task performance. By understanding these dynamics, the research seeks to clarify the relationship between music, immersion, and cognitive performance in this specific context of a vertical scroller video game.
We have formulated the following research questions to guide our study: RQ1 “How does energetic music during gameplay of a scrolling dodge video game afect the level of immersion experienced by the players?”, RQ2 “How does energetic music during gameplay afect game perception (workload, pace, arousal)? ” and RQ3 “How does game immersion and perception afect overall task performance? ” To address these research questions, the study aims to achieve the following objectives: • Develop a vertical scroller video game designed to measure player immersion under diferent music conditions. • Conduct a user study involving participants who will play the game with and without music. • Collect data on player immersion and performance using a combination of questionnaires, eye tracking, facial expression, skin conductivity, and game scores. • Evaluate the impact of energetic music on immersion, and how immersion is related to overall task performance.
The relationship between video game immersion and music is already well documented. It is
known that music and sound efects play a large role in video game immersion, but studies into
this topic show inconclusive objective results, suggesting a more nuanced relationship between
the music provided in the video game and video game immersion. One study [
This lack of objective conclusion presents an opportunity for this study to utilize objective measures of immersion, such as pupil size, electrodermal activity, in order to find a more objective determiner of immersion.
Research has already extensively documented the efects of music on cognitive performance,
showing varied results based on the type of music and task. One study [
Interestingly, one article [
Several studies [
Consequently, our study aims to address the issue of musical compatibility and nuance by creating an environment where highly energetic music is appropriately matched to the task intensity, such as in a vertical scroller video game. In this type of game the player views the action from a top-down perspective, with the background scrolling from the top to the bottom of the screen, setting a rhythm for gameplay, and requiring the player to respond rapidly to the scrolling environment. By selecting music that aligns with the nature of the task, we tried to ensure that the music enhanced rather than detracted from the task experience. As a result, to efectively explore the impact of music on task perception and performance, we have developed a video game specifically for this user study. This game, serves as the experimental environment for our study.
The game developed for this user study is titled Dance Dance Mania, a Dance Dance Revolution inspired vertical scrolling and timing-based game designed with a colorful 8-bit retro aesthetic to maintain player engagement. We developed our own game to avoid the familiarity with the existing games which could benefit some uswers, and to better control the game exectution (same scenario, the increasing of the pace, time of the game run, etc.). With existing games this was not possible and too many other variables would be introduced.
In the game developped players need to press the correct arrow keys in sync with arrows falling down the screen when these overlap with the gray arrows on the bottom of the screen as shown in Figure 1. The game starts slowly, with arrows taking 4.5 seconds to reach the bottom and appearing every 2 seconds, allowing players to adjust. As the game progresses, the speed increases, with arrows moving 6 times faster (0.75 seconds to the bottom) and generated at 2 arrows per second, creating a more challenging experience within the 120-second (2 minutes) round.
The scoring system is simple: a “Perfect” hit (correct timing of pressing the arrow key when the falling arrow completely overlaps with the gray arrow) adds 2 points to the score, a “Good” (slightly mistimed pressing of the arrow key when the falling arrow slightly overlaps with the gray arrow) adds 1 point, while a “Wrong” key press or a “Missed” arrow deducts 1 point. At the end of each round, the player’s score and performance details (number of Perfects, Goods, Wrongs, and Misses) are displayed. Both the game’s music and sound efects follow the retro theme, creating a cohesive audiovisual experience.
A pre-determined sequence of arrow generation ensures all participants face the same dificulty level. Participants play four rounds of the game, experiencing either music in the first and third rounds or the second and fourth rounds, based on their assigned starting condition.
Participants were recruited through university boards and word-of-mouth. A total of 10 participants, aged between 20 and 23, signed up for the study. There was no criteria for participant selection, and no incentives were ofered for participation. All participants were university students. They were informed that the study involved playing a video game under difering conditions, but information regarding the study’s expectations, hypothesis, and research questions was not disclosed.
The study was conducted in a controlled environment to minimize external variables. The setup involved the use of iMotions software to collect facial expressions, eye tracking data, and galvanic skin response together with heartbeat. Participants were randomly assigned to one of two conditions, while ensuring that each had an equal number of participants.
Upon arrival, participants were briefed on the study’s procedure. They were then asked to sign informed consent forms, which explained their rights, the voluntary nature of their participation, and the confidentiality of their data.
Next, the experimenter showcased a 10 second demo of the game, to help participants understand the controls, the mechanics, and what diferent sound efects refer to. This demonstration did not involve the use of music. Following that, the experimenter explained diferent hardware that would be used to monitor the participants immersion level. The Shimmer 3 GSR Unit was applied to the participant’s non-dominant hand to avoid interference with gameplay controls.
Next, participants played four rounds of Dance Dance Mania each lasting 2 minutes. In one condition the participants experienced music and sound efects in the first and third round of the game and only sound efects in the second and fourth round. In the other condition participants experienced music and sound efects in the second and fourth round of the game and only sound efects in the first and third round. After the second round, participants were given a brief break to relax.
We opted for four rounds of the game to make sure the participants got familiar with the game during the first two rounds and knew what to expect in the music condition and no-music condition. We then only took the results of the third and fourth round. This ensured that the majority of the learning happened during the first two rounds and the learning efect is smallest between round 3 and 4. Following the third round, participants completed the NASA-TLX and the Immersive Experience Questionnaire (IEQ), documenting their experience specific to that round. After the fourth round, they completed the same questionnaires, this time focusing on their experience in the final round. Finally, participants filled out a custom-made questionnaire about their music and gaming history, as well as demographic information including age and gender. At the end of the study, the participant’s game scores were saved and labeled accordingly. For a quick summary of the study procedure, refer to Table 1.
4.3.1. Questionnaires
The questionnaires completed after round 3 and round 4 are the aforementioned NASA-TLX
task (work)load index2 and the Immersive Experience Questionnaire (IEQ) developed by [
The IEQ was slightly changed by omitting questions not pertinent to this particular game, such as questions about in-game events and story. Most questions were rated on a 7-point Likert scale, while others required participants to select discrete answers such as “Yes”, “No”, or “Maybe”.
The questionnaire related to the participants’ music and game history included items designed to identify potential outliers and provide a foundation for future research.
In order to measure Physiological response, the following iMotions products were used:
• Afectiva AFFDEX [
For each timestamp, Afectiva provides a set of facial expression and a value ranging from 0 to 100 for each facial expression (a higher value means that particular emotion is more expressed or noticeable), For some emotions that can be either negative or positive (i.e Valence), a value ranging from -100 to 100 is provided instead (less than 0 means negative valence, more than 0 means positive valence) 2NASA-TLX https://en.wikipedia.org/wiki/NASA-TLX • Tobii Pro Spectrum screen for eye tracking purposes. At each timestamp, we are provided with multiple values related to eye movement. For the purpose of this study, we will use pupil size parameter. • a Shimmer 3 GSR Unit attached to the participants non-dominant hand. The Shimmer measures electrodermal reactions and heart rate. The main electrodermal reaction used in this study is GSR Conductance, which reflects changes in skin conductance related to physiological arousal.
For each participant, data was collected for the number of “Perfects”, “Goods”, “Misses”, and “Wrongs” within each round of gameplay, as well as the final score for that round. Additionally, the round number and whether or not the round included music were recorded.
NASA-TLX was score was derived using a weighted sum of the participants’ responses, with the
weights provided by a manual on how to calculate NASA-TLX scores [
We will utilize the following metrics provided by the aforementioned iMotions tools: • Engagement: Reflects the level of involvement with the task. • Valence: Indicates the positive or negative emotional tone. • Attention: Measures how attentively a participant is focused. • Brow Furrow: Associated with concentration or confusion. • Eye Widen: Often a sign of surprise or heightened attention. • Lip Press: Can indicate stress or concentration. • Lip Suck: May be a sign of nervousness or deep thought. • Mouth Open: Could signal surprise or high arousal.
• Smile: Reflects positive engagement or enjoyment.
Higher values in these metrics generally suggest greater immersion.
TA larger pupil size captured by Tobii Pro Spectrum eye tracker is typically associated with higher levels of arousal, which, in the context of our study, correlates with increased immersion. Elevated skin conductance and heart rate captured by Shimmer 3 GSR Unit are also often associated with higher levels of arousal, which may indicate increased either immersion or nervousness in the task.
To efectively utilize these values in our data analysis, we used the mean of each measurement across the entire duration of each round. By averaging these values, we can provide a clearer comparison of emotion levels both between participants and across diferent conditions, such as rounds with music and rounds without music.
As by the study procedure described in subsection 4.2 we used a within-subject design. The influence of music on task performance is measured in the same way as immersion. Prior to data analysis, Shapiro-Wilk tests and Q-Q plots will be used to determine whether the diferent results follow a normal distribution across all participants. If a given immersion or performance result follows a normal distribution, we will use a paired t-test. On the other hand, if a performance and immersion result does not follow normal distribution, we will use a Wilcoxon test.
Prior to data analysis, Shapiro-Wilk tests and Q-Q plots confirmed a normal distribution of the questionnaire results.
For the within-subject design, NASA-TLX workload scores and IEQ immersion scores were compared between the same participants, for rounds 3 and 4. The mean NASA-TLX score when participants experienced music condition was slightly higher ( = 979, = 367.1) compared to when they experienced no-music condition ( = 976, = 313.7). Despite this small diference, the paired t-test results ((9) = 0.088, = 0.9312 = 0.008) showed no significant diference in the NASA-TLX score between the two conditions. Conversely, the mean Immersion score when participants experienced music condition was larger ( = 50.8, = 7.6) compared to participants who experienced no-music condition ( = 45.8, = 6.8). The paired t-test results ((9) = 1.811, = 0.10 = 0.691) suggested a marginally significant diference between the two conditions, with a large efect size.
Prior to data analysis, Shapiro-Wilk tests and Q-Q plots confirmed a normal distribution of the aggregated data captured by iMotions. We observed no discernible diferences in the mean values of emotions between the two conditions. While some emotions exhibited higher mean values under the music condition, others showed higher mean values under the no-music condition. Utilizing paired t-tests, we found no significant diferences between the two conditions for any of the analyzed emotions, additionally, the efect sizes were very small. The most notable result was observed for the “Lip Press” expression. Here, the mean value for the expression when participants were exposed to music condition was slightly higher ( = 4.27, = 7.46) compared to when they were exposed to no-music condition ( = 2.65, = 4.61). The paired t-test yielded: (9) = 1.72, = 0.12, Cohen’s efect size = 0.08. Even then, the efect size is minimal. For detailed statistical results of all emotions refer, to Table 2
Utilizing the game scores, we can determine whether music has a significant impact on video game performance.
Prior to data analysis, Shapiro-Wilk tests and Q-Q plots confirmed a normal distribution of the aggregated data. A within-subject design was used to analyze gameplay performance between the two conditions: As mentioned, in order to minimize the learning curve efect, we opted to only use rounds 3 and 4. We will now refer to rounds 3 and 4 as such: the rounds where people experienced music as the music condition and the rounds where people did not experience music as the no-music condition.
• Score: The mean scores in the music condition was actually lower ( = 248.5, = 54.05) than in the no-music condition ( = 256.4, = 48.7). Despite this, the paired t-test results ((9) = − 0.94, = 0.37, Cohen’s efect size = 0.20) showed no significant diference between the scores. • The number of “Goods”: the mean amount of “Goods” was actually higher in the music condition ( = 46.8, = 19.42) compared to no-music condition ( = 39.3, = 14.06). The paired t-tests showed a borderline significant diference between the two conditions, and a small to medium efect size ( (9) = 1.78, = 0.10, Cohen’s efect size = 0.45). • The number of “Perfects”: The mean amount was smaller in the music condition ( = 110.9, = 24.50) compared to the no-music condition ( = 119.1, = 21.24). The paired t-test results showed no significant diference ( (9) = − 1.72, = 0.12, Cohen’s efect size = 0.35. • The number of “Misses”: The mean amount was slightly higher in the music condition ( = 9.6, = 10.00) than in the no-music condition ( = 10.01, = 8.44). Alongside this, the paired t-test showed no significant diference( (9) = 0.26, = 0.80, Cohen’s efect size = 0.06). • The number of “Wrongs”: The mean amount was slightly higher in the music condition ( = 13.2, = 14.80) than in the no-music condition ( = 12.1, = 12.12). The paired t-test showed no significant diference( (9) = 0.34, = 0.74, Cohen’s efect size = 0.08).
Even though the mean NASA-TLX workload score was slightly higher in music condition, the diference in workload scores was statistically insignificant, and the efect size was negligible. Thus, it is reasonable to assume that music has no efect on the perceived workload. On the other hand, the mean values of the immersion scores were notably higher in music condition. Despite the non-significant diference, the lower p-value and larger efect size suggest that music might enhance the immersive experience of the game. With that in mind, there is a possibility that a Type II error occurred (i.e a false negative) due to the small sample size, indicating that the study might not have had suficient power to detect a significant diference.
When analyzing the physiological data, the diferent emotions that would indicate immersion, did not follow a discernible trend or pattern. Engagement, attention, valence, and smile counts were higher in the no-music condition, while the number of joy expressions, lip presses, and brow furrows were higher in the music condition. Due to the non-significant diferences and small efect sizes, it can be assumed that music either has no significant efect on immersion or that the participants did not express immersion through facial expressions.
Additionally, the higher GSR conductance in the music condition, although insignificant, might indicate heightened emotional responses such as fear, excitement, anxiety, or stress. This supports the idea that energetic music, especially unfamiliar music, might increase nervousness. This could either indicate heightened immersion levels or a break in immersion. Based on self-reported immersion scores, GSR conductance could be a supporting factor for music’s impact on immersion.
The game scores were higher in no-music condition, and while these diferences are also non-significant, The fact that 9 out of 10 participants played better when there was no music involved, as well as obtained the smaller amount of “Misses”, and “Wrongs” and the larger amount of “Perfects” strongly supports the idea that music, especially unfamiliar music is distracting and can lead to more errors, even if it matches the taks at hand. There is also a possibility that a Type II error occurred in the significance testing, due to the small amount of participants.
Research question 1: How does energetic music afect the level of immersion experienced by the player • The diference in immersion scores in the self reported immersion questionnaires, as well as the diference in GSR conductance, is a strong indicator that music positively influences the level of immersion experienced by the player.
Research question 2: How does energetic music during gameplay afect game perception (workload, pace, arousal)? • Due to the conflicting results, namely, the NASA-TLX workload score yielding no significant diference, supporting the idea that energetic music has little to no efect on game perception, while the GSR Conductivity indicated higher levels of arousal, which could mean that the participants were more stressed, anxious, or in panic due to the game being perceived as faster, or more dificult, we cannot come to a definite answer. In a subsequent questionnaire, participants were asked if the music changed their perception of the pace or intensity of the game, to which 7 answered that the pace of the game felt faster, 2 answered that the pace of the game felt slower, while 1 answered that it felt the same. While this result does not provide statistical significance, it does provide insight into the relation between energetic music and task perception, but further research is needed. Research question 3: How does immersion and game perception afect game performance? • While it is clear that the presence of music negatively impacted game performance, the exact reasoning behind this efect can be explored in various ways. One possible interpretation is that participants perceived the game as more dificult or fast-paced when music was present, leading to a higher propensity for mistakes. This notion is slightly supported by the findings related to the second research question. A third interpretation is that participants did indeed experience higher immersion levels, but this immersion was related more to the overall game experience rather than the specific gameplay itself. The presence of music might have led participants to feel that they should enjoy the game fully, rather than focus on performing the task to the best of their abilities. This suggests that music might have influenced their emotional engagement with the game environment rather than their cognitive engagement with the gameplay mechanics. The emotional engagement, driven by energetic music, could have made the participants feel more connected to the game setting, yet distracted them from optimal gameplay performance. Additionally, energetic music is often described as distracting, thus leading to a divided attention scenario. Participants might have found themselves paying more attention to the music, which created an immersive auditory environment, but at the expense of their concentration on the game’s mechanics and objectives.
Our findings closely align with some of the previous studies. Namely, research suggesting that
music negatively impacts game performance, and that energetic, unfamiliar music is seen as
distracting [
Our findings also align with research suggesting that music helps with the immersive
experience [
Thus, it appears that immersion only supports task performance when it is closely tied to the task itself. In cases where the immersive element, such as high-tempo music, is not task-related, it may not contribute positively to performance and can even hinder it.
Additionally, task immersion is shown to be most impactful to performance when the task relies heavily on working memory, spatial reasoning, and the ability to mentally manipulate and store information (e.g trying to memorize a pattern, or an image) [14, 15]. Moreover, studies have shown that immersion is least significant when the task performed is highly reliant on motor skills, reaction time and the ability to synchronize physical movements with visual inputs [16], such as the video game in our study.
First, the results suggest that simply increasing immersion through elements like high-tempo music does not inherently enhance performance. Instead, the efectiveness of immersion is highly context-dependent. So, designers of immersive experiences should be mindful of how immersion elements relate to the specific task.
Second, the impact of diferent types of music on task performance empathizes the need for careful selection of audio stimuli. While calming music has been shown to improve performance, energetic and intense music might increase stress levels and lower eficiency. This information can guide the selection of background music in various settings, including workplaces, educational environments, and therapeutic settings, to optimize user experience and performance.
Regarding stress management, the negative impact of high-tempo music on performance underlines the importance of managing stress in immersive environments. Designers should be aware of how immersive elements, such as music, afect stress levels and overall well-being. providing options for users to select their preferred auditory environment could mitigate potential negative efects.
In practice, understanding that immersion must be relevant to the task can improve the design of virtual environments and other immersive applications. By coordinating immersive elements with the specific goals of the task, practitioners can create more efective and engaging environments that enhance user performance and satisfaction.
There were a few limitations of the study. The most impactful limitation is the small sample size. This likely hindered statistical power, increasing the risk of Type II errors, and incorporated a large amount of participant variability, which could be a confounding variable. This additional variability might have influenced the results, making it dificult to isolate the efects of music. An additional limitation would be the hardware used. During the user study, the Shimmer unit failed to detect a heart rate, resulting in the heart rate values being unusable, which would have otherwise been a useful statistic. The study focuses on a single type of game and one specific type of music (energetic, high-tempo). While this focus allows for controlled testing, it limits the generalizability of the findings to other game genres or music types. Future research could include diferent genres of music or diferent types of games (e.g., strategy, puzzle, adventure games).
This study aimed to analyze the impact of energetic music on game immersion, game perception and performance. and whether there is a reasonable connection between them. We collected a combination of subjective questionnaires (NASA-TLX, IEQ) and objective measures (eye tracking, skin conductivity, facial expressions) as indicators of immersion. Additionally, the efect of music on game performance was assessed through in-game scores. The statistical analysis found no statistically significant diferences in mean values for most emotional and physiological measures between the music and no-music conditions. Additionally, the analysis found no statistically significant diferences in game scores. Despite this lack of statistical significance, several key observations emerged: the mean values of immersion in the immersion questionnaire were higher in the music condition than in the no-music condition. This suggests that participants felt more immersed when music was present. However, this increased sense of immersion did not translate into improved performance, as the no-music condition had higher performance scores on average.
These findings lead to a deeper examination of the relationship between immersion and performance. The observed discrepancy, where increased immersion was associated with lower performance, suggests that the energetic music might have been detrimental to performance. This prompted a hypothesis that immersion, to positively impact performance, needs to be directly related to the task at hand. Previous research supports the notion that immersion can enhance performance when it is closely aligned with task requirements. In contrast, when the immersive element, such as high-tempo music, is not relevant to the task, it may not contribute positively and could even impair performance.
The relationship between immersion, perception and performance needs to be further investigated by exploring how diferent types of immersion, particularly those tailored to align with the nature of the task, influence performance. Studies could investigate various immersive elements and their direct relevance to specific tasks to better understand how immersion can be optimized for diferent activities. For instance, examining how task-specific immersion afects performance could reveal more about the conditions under which immersion enhances task outcomes.
Exploring individual diferences, such as personal music preferences, baseline stress levels, or familiarity with the task, could ofer insights into why some individuals experience varying efects compared to others. Expanding research to include diferent contexts beyond video games, such as educational environments, workplaces, or therapeutic settings, would help understand how immersive elements like music impact performance in a variety of real-world scenarios.
This research was funded by the Slovenian Research Agency, grant number P1-0383, P5-0433, IO-0035, J5-50155 and J7-50096. This work has also been supported by the research program CogniCom (0013103) at the University of Primorska. [14] J. Gisler, C. Hift, A. Kunz, V. Holzwarth, Designing virtual training environments: Does immersion increase task performance?, in: 2020 International Conference on Cyberworlds (CW), IEEE, 2020, pp. 125–128. doi:10.1109/CW49994.2020.00026. [15] M. Slater, V. Linakis, M. Usoh, R. Kooper, Immersion, presence, and performance in virtual environments: An experiment with tri-dimensional chess, ACM Virtual Reality Software and Technology (VRST) (1999). doi:10.1145/3304181.3304216. [16] T. Rose, K. B. Chen, Efect of levels of immersion on performance and presence in virtual occupational tasks, in: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, volume 62, SAGE Publications Sage CA: Los Angeles, CA, 2018, pp. 2079–2083. doi:10.1177/1541931218621469. [17] N. Perham, J. Vizard, Can preference for background music mediate the irrelevant sound efect?, Applied Cognitive Psychology 25 (2011) 625 – 631. doi: 10.1002/acp.1731. [18] D. Sanseverino, A. Caputo, C. G. Cortese, C. Ghislieri, “don’t stop the music,” please: The relationship between music use at work, satisfaction, and performance, Behavioral Sciences 13 (2022) 15. doi:10.3390/bs13010015. [19] M. C. Muhmenthaler, B. Meier, Diferent impact of task switching and response-category conflict on subsequent memory, Psychological research 85 (2021) 679–696. doi: 0.1007/ s00426-019-01274-3.