The Internet of Musical Things (IoMusT) and ubiquitous music (ubimus) are interrelated research fields concerned with the design and development of advanced technology to support novel musical and artistic experiences. IoMusT primarily motivates the deployment of embedded computing devices and platforms for musically oriented individual or collaborative activities. Ubimus schemes are musically driven while keeping the participating agents and entities (such as tablets or computers), interfaces (hardware, or software), interaction metaphors (visual, tactile, or gestural), and instruments (traditional or novel) not necessarily complex. It also encourages the design of new musical interactions for participants with little know-how of the musical domain. Within the context of initiatives embracing the internet of things (IoT) for healthcare, alongside remodeling IoT devices for IoMusT interactions, this paper considers the area of ubiquitous musical interaction design for music therapies, ofering an example application. This convergence, with the appropriate technologies, could enable new opportunities for the promotion of wellbeing, both individually and collectively, in-person and remotely.
Musical Interactions for
The Internet of Musical Things is a confluence of cross disciplinary fields including music
technology, the internet of things, human–computer interaction, and artificial intelligence
applied to musical contexts. As a technological perspective, the IoMusT ecosystem is composed
of three core components: i) musical things, ii) connectivity and iii) applications and services.
In the IoMusT network, musical things are computing devices of any form such as wearables,
computers, and tablets dedicated to the production and/or reception of musical content. Musical
information is data detected and processed by a musical thing and is sent to a human or
another musical thing across a network [
Wellbeing is broader than just the absence of a disease in human body. It brings psychological,
social, and spiritual health along with physical health. Music has been widely used for therapies
improving resilience and building confidence to deal with various scenarios of anxiety and
stress. Recently [
Ubimus believes that music is universal, and everyone has the potential to create sounds that
are musical. It started in 2007 and afords new metaphors for creative actions (e.g., soundsphere
[
IoMusT uses Wireless Sensor Networks (WSNs), IoT, and Tactile Internet (TI) as background
communication technologies. These technologies enable the development of SMIs. SMIs cross
boundaries of traditional music settings and venues such as studios and stages due to embedded
technologies for sound production and processing [
Figure 1 highlights where IoT and IoMusT can share the same devices and computing resources for healthcare and music making together. To let cross modal devices and sensors communicate for such applications, protocols and communication standards for wireless data collection, processing and transmission are required as a common language. For example, electromyography sensor (EMG) measures small electrical signals generated by human muscles as they are moved. This includes lifting an arm up, clenching a fist, or gestures like moving a finger.
With IoT-based monitoring devices designed in form of bands or headsets that observe
physiological data, the next stage of IoMusT is to give meaning to this data. Thus, the data is
analyzed to detect an artistic intention that reflects some emotion. This data is sent to a musical
interface that configures the smart instrument presets or music sound repositories based on
the users’ requirement. The physiological data can be mapped to various music parameters
such as pitch, timbre, or volume. In active music therapy, users create melodies or improvise,
and this process is continuous. Designing interfaces for such interactions is a challenging task.
The prediction of human musical activity can also inform the interface configurations to ofer
more customized resources, but it is also dificult. To accommodate an “active” role of music,
the user would perform gestures that are simpler as the interaction begins. In the therapeutic
scenario, the configuration of movement-musical events would partially depend on the user but
the system would take responsibility for constraining the user to a selection of musical objects
on the interface that are specifically designed for IoMusT. If the initial interaction is based on
the triggering of musical events when two arms are moved, the user would have the control to
add more such mappings between their movements and the music objects as the interaction
evolves. It might be too optimistic to say at this stage, but use of artificial intelligence for
human-centered interfaces presented in [
In our ongoing project we are working on interaction modalities for musical interaction including
color-based metaphors for amateurs and gesture-based musical interactions using computer
vision techniques. The aim is to design and develop low-cost, easy-to-use, and available
technologies for music making and defining new applications based on these systems as an
application to the field of IoMusT. Currently, sensing of the heart rate and galvanic skin response
as shown in figure 2 (b) are used for a low fidelity prototype design of a wrist band (see figure
2 (a)) that triggers music based on the Solfeggio note frequencies for body and mind health.
These Solfeggio tones are known as sound healing modalities for various mental, emotional, and
physical ailments [
The short paper has introduced a music-based rhythmic vibration device that responds to a users’ stress level. It embraces the concepts and ideas of using IoT (the sensors), ubimus (the ease of use), and IoMusT (the musical responsiveness) together in the context of health and wellbeing. This proposed framework is a work in progress and the mapping of movements to music, inclusion of interaction over the network and user testing are the next tasks.