Technology based on networks has long ago become state of the art for music recording, concert venues and even as a medium for collaborative performances over the Internet. However, there is a drawback attached to this technology. Latency is a permanent issue associated to any network technology. On the other hand, musicianship relies on time perception. The performance of music and playing an instrument are activities highly affected by latency. The present paper describes a methodology in order to measure how latency and the performed western musical instruments are related. In addition, the latency tolerance range (LTR) is presented as a comparison measurement between the different musical instrument groups. The results are based on the exploratory data analysis of the gathered data.
Research on the issue of latency and musicianship is focused on timing and
the improvement of network technical conditions, in order to enable
collaborative performances [
A total of 31 test subjects playing 17 different western musical instruments took
part in a listening test and answered a questionnaire. Every musician played
only his or her main musical instrument. The different musical instruments were
divided into the groups of aerophones, chordophones, membranophones and
idiophones based on the Hornbostel-Sachs taxonomy [
The setup of the listening test is described in [
Every one of the 31 musicians played the music score five different times according to the tempi 90 BPM, 120 BPM, 150 BPM, 180 BPM and 210 BPM, using a different metronome each time. The three metronomes were regulated using the MIDI protocol and have not been delayed. At the end of the listening test, 15 different latency values were gathered (5 tempi and 3 metronomes).
It is possible to define the total latency of the system used in the listening test by means of the following mathematical expression:
Lt = La + Lc + Ld (1)
Lt is the total latency, which is usually measured in milliseconds. The total latency is the sum of the latency due to the sound transmission La, the latency produced by the analog (sound waves) to digital (information in the DAW) to analog (sound waves to the headphones) conversion Lc and Ld which is the network latency simulated in the listening test.
The latencies La and Lc can be considered constants. The value Lc was measured and was the same for every listening test (12.208ms). On the other hand, the value of La is directly related to the distance of the microphone to the instrument as presented in Table 1. Prior to the beginning of each listening test, the distance microphone to instrument and the sound pressure level (SPL) of the musical instrument were measured, in order to guarantee equality of conditions. In addition to those measurements, the level gain in dB of the microphone and the headphones, as well as the reverberation time (RT60 in seconds) and noise level of the different rooms were measured. The information presented in Table 1 summarizes the range, median, mean, variance and standard deviation for the 31 test subjects (nbr.val).
Based on the information presented in Table 1 it is clear that all of the instruments were recorded under similar conditions. Furthermore, the results of the first column have a very low variance, which means that the distance between microphone and instrument was almost the same. In other words, La, which is the latency produced due to sound transmission, can be assumed as a constant. 4
The value Ld, which is the simulated network latency, is the numerical outcome of the listening test. As stated before, musicians played the score while listening to their own delayed3 signal (latency was introduced). The score consisted of two bars and was played in a loop. Every second repetition (4 bars), the delay was increased by 10ms. The latency range Ld variated from 0 to 300ms. At 0ms only La and Lc were present and imperceptible for the majority of musicians. The latency value Ld was obtained by a performance disruption, when the musician interrupted the performance the value Ld was notated. 3
After having collected the different latency values Ld, in addition to further numerical information of the questionnaire such as age, hours of instrument practice with and without metronome and years of experience on the instrument, it is possible to visualize the information in a scatter plot as presented in Figure 2. 3 The concepts of latency and delay are the same for this work.
In Figure 2, every numerical variable is plotted against each other. The variables are written on the diagonal and all relationships above and under the diagonal are mirrored. The numerical scale for age ranges from 0 to 50 years, however mainly young European musicians took part in the research. The scale for the years of experience ranges from 0 to 25 years, practice hours with metronome from 0 to 8 hours and without (practice hours) from 0 to 30 hours. Finally all latency values (Ld) distributed according to the five different tempi (90 BPM to 210 BPM) have the range 0 to 300 milliseconds.
The linear trends in Figure 2 are easy to identify for columns five to nine, which are related to the latency values Ld. Additionally, another linear relationship is present between age and years of experience (column 3, row 1) which is obvious. The older a musician is, the more years of experience he or she has. On the other hand, it is interesting that no visual relationships exists between the years of experience, musical practice with and without metronome and the latency values Ld for the different tempi. It may indicate that those variables have no meaningful role at all, regarding the ability to cope with latency.
A further procedure is to visually analyse the shape of the distribution. Figure 3 presents the explorative data analysis (EDA) for the distribution of the gathered data. 6
Medina Victoria, Jorge
From the raw data plot in the upper left of Figure 3, two details are clear. Firstly, the data has not a random distribution and secondly, the data shows a decaying behaviour. The next plot on the right is a histogram, additional to the observation regarding a non-normal distribution. It is also possible to see the outliers (data in the 300 bar). Finally, the lower right plot (Q-Q plot) confirms that the data is non-normally distributed. The confidence intervals (red dashed lines) are clearly overpassed.
Table 2 lists all musical instruments that have been analysed. The selection
of test subjects was not completely random. This purposive sample represents a
virtual population and constitute a random sample [
In Figure 4, data from the three different metronomes (aural, visual and
aural-visual) is presented and data from similar instruments, e.g. piano (1),
piano (2) and piano upright (11) have been averaged. Figure 4 represents the
latency value (Ld) vs. tempo in BPM. This relationship is known from
Barbosa [
It can be noticed that there are very few differences between the results of the different metronomes (except for instruments such as marimba, bassoon and alto saxophone), therefore, it may be possible to average the latency results Ld of the three different metronomes. Observing Figure 4, it is clear that choosing a western musical instrument has an impact regarding the ability to cope with latency. For example, a musician playing the snare drum tends to interrupt the performance earlier (lower latency value Ld) compared to a cello player.
The data of all musical instruments, according to the different five tempi of
the experiment could be presented using the kernel density estimation (KDE).
The KDE is a smoothed version of the histogram [
Medina Victoria, Jorge
It is important to state that the listening test delivered only five discrete latency values Ld, according to every tempi and to every metronome, as seen in Figure 4. For the Kernel density estimation (KDE) in Figure 5, the values of every metronome (aural, visual and aural visual) were averaged.
Figure 5 clearly shows the peaks for the latency values Ld. The higher the tempo (210 BPM and 180 BPM) the narrower the peak. It indicates that by playing at higher tempi, all musicians stop the performance at approximately the same latency value Ld. For the tempi 90 BPM, 120 BPM and 150 BPM there is no observable narrow peak. On the contrary, for some tempi (90 BPM) there are more than one peak and different values of Ld present. It might suggest that different musical instruments have dissimilar latency values Ld, where the performance is interrupted. The lower peaks at 300ms (right side of Figure 5) are produced by those musicians who were able to play even at the higher latency values (Ld) up to 300ms (e.g. trumpet in B). Those values of Ld can be assumed as outliers. 3.1
Latency Tolerance Range The measure of the latency tolerance range (LTR) is the estimation of a range in milliseconds where latency is tolerable according to the different musical instrument groups. The (LTR) can be defined as the difference between the third and first quartile of the gathered data and is better suited than the mean or median to describe non-normal distributions.
The latency tolerance range (LTR) is measured in milliseconds and its mathematical expression is: (2)
The latency tolerance range is dependant on the amount of data gathered. However, the tendency is clear. Playing a membranophone may be a difficult task regarding latency. Aerophones, chordophones and idiophones may perform music as an ensemble within similar latency ranges, without any inconvenience. 4
The present paper presented the exploratory data analysis of a controlled listening test. The research design and test characteristics such as the use of a 10
Medina Victoria, Jorge metronome and a score enabled comparisons of the results. Based only on the exploratory data analysis of the gathered data, it is possible to claim that the ability to cope with latency is affected by the musical instrument played. Performing on different musical instruments may have an effect on the ability to cope with latency. Furthermore, the years of experience playing an instrument, the hours of practice and even the use of a metronome while practicing may not affect this ability at all.
The unbalanced design regarding the number of musical instruments tested per instruments type and the lower number of test subjects or musicians might diminish the external validity of the outcomes. However, there are clear tendencies and patterns and the effect is far from random.
The influence of the musical instrument, regarding the latency issue, decreases while performing at faster tempi above 180 BPM. On the contrary, this influence increases by playing a musical instrument at lower tempi under 150 BPM. In addition, the latency tolerance range (LTR) enable the comparison of results regarding the issue of latency between the different musical instrument groups.
The results and the methodology presented, may constitute a further step in the modelling of musical instruments for technologies such as virtual reality. Moreover, based on the results presented, software requirements for artistic and educational projects, such as network musical live performances and distance learning programs can be further developed to fit better the requirements regarding musical instrument groups.
Ability of Western Musicians to Cope with Latency 11