This preliminary study investigates unexplored immersive approaches of music as a therapeutic tool, focusing on its potential to promote wellness and support the healing process, engaging human psychophysiological responses. The project's core is an interdisciplinary approach that combines music therapy theories with art and immersive technology, aiming to amplify music's therapeutic efects through interactive sound environments. To investigate these efects, physiological parameters such as heart rate and electrodermal activity, collected in a previous study, were used to inspire and generate art. The integration of artistic and scientific techniques, through the ”Soundscapes of the soul” project, has allowed us to translate these physiological responses into visual representations, ofering a multisensorial experience. These initial researches open new perspectives on the application of music therapy and immersive technologies in improving psychological and physical well-being.
CEUR ceur-ws.org
The idea behind this research stems from the growing interest in music therapy and psychology about the transformative power of music and its therapeutic potential. In an age marked by rising psychological disorders and an intensified focus on mental well-being, music may be considered not only as an art form or source of entertainment but also as a potential tool for enhancing mental and physical health.
In recent decades, music therapy has gained increasing importance in psychology and medicine, with numerous studies confirming its efectiveness in treating various psychological and physiological disorders. However, the link between music, emotions, and physiological responses remains an open field of study, with many areas still to be explored.
In literature, there is a broad consensus that music could be employed to influence emotions
and moods. Music therapy theories, such as those developed by Nordof-Robbins [
At the same time, neuroscientific studies have shown how music can stimulate diferent
areas of the brain[
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). electrodermal activity has only been partially explored, and further research is needed to fully understand the underlying mechanisms.
This research aims to investigate more deeply the interaction between music and emotions by examining the measurable physiological efects elicited through music listening, utilizing an artistic project that translates physiological data into generative art. This study states the following hypothesis: (1) music has the capacity to evoke measurable psycho-physiological responses, influencing parameters such as heart rate and electrodermal activity, whose behaviour could be visually translated into dynamic art forms; (2) familiar melodies, compared to unfamiliar ones, are expected to resonate more profoundly, fostering positive emotions and states of comfort that manifest in art reflecting more relaxed patterns; (3) immersive technologies can intensify the emotional and physiological experience of music, creating a layered, multi-sensory engagement that draws the body and mind into deeper resonance, captured vividly in the generative artwork produced from participants’ physiological data.
The concept of this project is to create a bridge between art and science, using art as a way to promote well-being. Exploring how diferent types of melodies impact physiological responses highlights the potential for art to intersect with scientific research, influencing both body and mind. This study involves physiological parameters such as heart rate and skin conductance, with future plans to include brain activity. This approach utilizes technology not only to deepen the understanding of music’s efects but also to pave new pathways for therapeutic applications rooted in art.
The aim of this work is twofold: on the one hand, to collect empirical data that will contribute to the scientific literature on music therapy, exploring how music could be an efective therapeutic tool; on the other hand, to artistically translate these physiological responses into visual works that reflect the emotional and physical impact of music on the subjects involved.
The project addresses three main objectives: scientific, artistic, and educational. From a scientific perspective, this research aims to contribute to the exploration of music therapy, based on the hypothesis that music can be a powerful tool for enhancing psychophysiological well-being. Currently, the study provides a conceptual framework and preliminary insights into how music may influence human physiological and emotional states. On the artistic front, the project seeks to create a bridge between science and art, using scientific data as inspiration for visual works. Lastly, from an educational perspective, the goal is to foster greater awareness of the therapeutic potential of music among the general public and within the scientific community.
The project is divided into three main phases. The first phase focuses on data collection from previous studies, where physiological data such as heart rate and skin conductance are gathered to observe the psychophysiological responses to diferent types of music. In the second phase, the collected data is used to create generative art through coding, translating the physiological responses into visual representations. The final phase involves 3D modeling of the exhibition space, where the generative art is integrated into the environment, providing an immersive, spatial experience that allows visitors to engage with the efects of music on the body and mind.
A collaboration with the MultiMedia Signal Processing (MMSP) Laboratory of the Department of
Informatics, Systems and Communication at the University of Milano-Bicocca, was fundamental
to obtain physiological data to be employed in the artistic project. Forty participants were
involved in a previous experiment. During the experiment, the subjects listened to six songs, two
of which were specially chosen by them to evoke their memory and emotions. During the whole
experiment, the heartbeat of each participant was collected through Photopletysmography
(PPG) using the Shimmer3 GSR+ Unit [
The second phase of the project focuses on the development of a generative art code, where
the collected data is transformed into dynamic visual art, creating a direct representation of
the physiological responses to music. Generative art is a form of digital art that leverages
algorithms and mathematical models to create evolving, dynamic artworks. Unlike traditional
static pieces, generative art unfolds in real-time, allowing it to exist as a continuous process
that evolves in space and time[
This approach was selected because it reflects the fluid nature of human emotions and physiological responses, making it an ideal medium for visualizing the impact of music on inner states. By employing generative art, we established a bridge between science and art, transforming real-time physiological responses to music into visually dynamic experiences. This approach allowed the creation of art that evolves based on participants’ unique psychophysiological data, such as heart rate and skin conductance. This data-driven generative system provided a scientific basis for the artwork, enabling a personalized, immersive experience that reflects the emotional and physical impact of music on each individual. In this way, the project showcases how artistic expression can be informed by empirical data, ofering a novel perspective on the interplay between art, music, and human physiology. Figure 2 is a frame of the final opera, where the speed of particles was mapped directly to the subjects’ heartbeat, while skin conductance data modulated the saturation of the visuals. Whenever there was a significant fluctuation in heart rate, it would control the speed of the particles, making them stop and accelerate, creating a pulsing efect to evoke the sensation of a heartbeat. Similarly, for skin conductance, the higher the value, the more saturated the colors became, visually reflecting the intensity of the participant’s emotional arousal.
This algorithm was designed to produce evolving, unique works of art that not only embodied the physiological responses to music but also visually captured the personal emotional journey of each participant. The result was an artistic expression of raw biological data, blending science with art in an interactive, immersive way.
However, the static image alone here reported cannot fully convey the dynamic nature and profound impact of this artistic translation. The subtle interplay between the participants’ emotional states and the music becomes evident through the real-time motion of the particles.In those preliminary representations, familiar melodies seem to evoke a sense of calm, causing the particles to move in a smooth, harmonious flow.
In contrast, unfamiliar music show more pronounced fluctuations in heart rate, resulting in dramatic particle movements that alternated between complete stillness and sudden, rapid bursts of motion, frame by frame. This remarkable visual contrast highlighted the unique ways individuals respond to diferent auditory stimuli, suggesting deep connection between music, emotion, and physiology. Further studies could deepen these aspects to validate them as common patterns, increasing the subjects involved.
To further bridge the gap between science and the public, in the third phase of the project an interactive exhibition space is designed to make scientific concepts more accessible through the medium of art 4 3. This exhibition would not only invite visitors to engage with the potential benefits of music but also demonstrate how art and science can be intricately intertwined, ofering a deeper understanding of both disciplines. This immersive environment also would allow us to monitor participants’ physiological responses, such as heart rate and skin conductance while listening to diferent melodies.
Visually representing contextually those data let the audience experience the music’s impact on the body, promoting greater awareness of its therapeutic potential and creating a bridge between science, art, and personal experience. Technology plays a central role in the project, helping to create an immersive and interactive experience for visitors. Sound-isolating headphones 6 allow one to enjoy the melodies optimally, improving concentration and promoting a deep perception of sound stimuli.
Physiological parameters such as heart rate, body temperature, and brain activity will be measured by specially placed external sensors to which participants connect before the start of the experience. Advanced holographic technologies 5 would project the generative artworks, creating a dynamic three-dimensional space in which visitors can move freely, surrounded by floating particles responding to their physiological data. Up to four people can experience this simultaneously, with their projections fused together and responding to the same musical stimuli.
In addition, isolated cabins are available, as shown in 6, ofering a controlled environment for an even deeper and more personalized immersion, allowing participants to explore the interaction between music and body intensively and individually. Programmable LEDs, placed around the room, change colour and intensity in harmony with music and generative artwork, creating an immersive atmosphere that maintains low brightness to enhance the holographic projections.
The visitor’s experience is conceived as a narrative path that begins with an introduction to the communicative power of music 7, through informative panels and explanatory videos, and culminates in the interactive experience itself in the following space. At the end of the exhibition, visitors will find a dedicated reflection area designed to enhance their post-experience contemplation 8. This space features ergonomic seating and sound-absorbing walls, creating a quiet environment for introspection. Visitors are encouraged to reflect on their experience and share their thoughts by providing feedback through tablets placed in this zone.
Soundscapes of the soul is not just an exhibition space, but represents a comprehensive experience that explores the deep connection between music and emotions, and in this case, how both of them can be linked to generative art. This project ofers visitors a unique journey into the world of interactive generative art, combining technology, science, and creativity to raise awareness about the importance of music for our psycho-physical well-being, highlighting their relationship through the visual representation of our body reaction to music listening.
This research contributes to the hypothesis that music may impact significantly on the human physiological responses and suggests that familiarity with a musical piece significantly modulates these responses. Visually interpreting the participants’ physiological responses to music allowed us to translate those scientific data into more comprehensible language for a wider audience, using art as a medium. This artistic representation of the data provided a unique lens through which to observe the physiological impacts of music, ofering insights into the calming or stimulating efects diferent tunes can elicit.
The reduction in heart rate and skin conductance while listening to familiar tunes suggests that musical familiarity could activate mechanisms of comfort and safety, which are reflected in stress reduction. This efect may be particularly useful in clinical settings, where music could be used as a tool to reduce anxiety and promote relaxation, especially in patients with stress-related disorders or chronic anxiety. The ability of music to modulate the physiological parameters of patients ofers interesting prospects for music therapy as a non-invasive and accessible intervention.
In addition, based on the interpretation of the artwork, it is hypothesized that listening to unfamiliar melodies may elicit greater emotional and physiological activation than familiar ones, suggesting a more intense cognitive response to new stimuli. This assumption opens new avenues for exploring the potential role of music in cognitive rehabilitation, though further empirical investigation is required to substantiate these ideas. It may be of interest to explore whether exposure to unfamiliar melodies can promote cognitive recovery in patients with brain injuries or cognitive deficits, such as those resulting from trauma or neurodegenerative disorders. The hypothesis is that music can not only facilitate relaxation but also stimulate more complex cognitive processes, promoting neuronal plasticity and contributing to the functional reorganisation of the brain.
Research suggests that immersive technologies could be efective tools for amplifying the
efects of music, as the integration of visual and sound stimuli may further enhance emotional
and physiological responses. The use of immersive environments, combined with music therapy,
could ofer new therapeutic solutions for patients with neuropsychological disorders, helping
them to recover cognitive and motor functions through a multisensory experience. The
integration of virtual reality and musical biofeedback would help with the treatment of conditions
such as depression, anxiety disorders and post-traumatic stress disorders (PTSD)[
Another promising application area is physical rehabilitation. The relaxing and activating efects of music, combined with immersive technologies, could facilitate the rehabilitation of patients with motor dificulties or paralysis, stimulating muscle activation and reducing emotional stress associated with physical rehabilitation.
In the future, it would be interesting to include the impact of music on brain responses in the project, using techniques such as functional magnetic resonance imaging or positron emission tomography. These tools could provide a more detailed view of the areas of the brain involved in emotional regulation during musical listening, contributing to a deeper understanding of the underlying neuropsychological mechanisms, allowing us to more eficiently use immersive environments as a therapeutic tool.
This study aims to raise awareness of the innovative use of technology in exploring the influence of music on human physiological responses, addressing an ongoing issue in a novel way. By examining the potential impact of familiar and unfamiliar melodies on emotional and physiological activation, the project emphasizes how music can be leveraged not only as an art form but also as a powerful tool for understanding psycho-physical well-being. These insights suggest that familiar melodies may foster relaxation, while unfamiliar ones could induce heightened emotional responses, opening new opportunities for research into the therapeutic potential of music.
The exploration of immersive technologies ofers a new frontier for enhancing the efectiveness of music therapy. The study highlights the potential for multisensory environments that combine music with interactive visual stimuli to amplify therapeutic efects, particularly in patients with neuropsychological or motor disorders.
Although the study does not ofer conclusive evidence, it sets the stage for further investigation, proposing innovative pathways to improve the application of music therapy and expand its use in medical and rehabilitation contexts.
I would like to extend my deepest gratitude to professor Simone Lombardi from Accademia of Fine Arts Santa Giulia of Brescia, whose dedication and expertise have been essential guiding this journey. His patience and encouragement pushed to explore new frontiers and enriched the development of this project.
We are also sincerely thankful to Professor Francesca Gasparini and Aurora Saibene from MMSPlab (https://mmsp.unimib.it/) of Milano-Bicocca University, for their continuous scientific support, significant contributions, and invaluable advice. Lastly, also Milano-Bicocca University itself should be surely thanked for allowing us to use their laboratory facilities, which made this study possible.