The spreadsheet provides an intuitive paradigm for the expression of musical scores. Musical scores can be expressed as data tables, with each record corresponding to a place in time and each column corresponding to a note or instrument. Sheetmusic is a plugin for Gnumeric that provides music sequencing spreadsheet functions. Tools like Sheetmusic provide intuitive music composition interfaces for people who are used to data analysis software. Moreover, they help us plot data with the sense of sound.
In my view, we separate data soni cation from data visualization only because of technological constraints; there isn't a fundamental di erence between these two processes.
Combining the visual and auditory senses, we can plot data in the form of music videos. One example of this is the FMS Symphony ( gure 1). In the FMS Symphony, each beat of music corresponds to a business day during the period between 2005 and 2013, the pitch of one instrument corresponds to the United States interest rate, the pitch of another instrument corresponds to the distance to the United States debt ceiling, and the activation of certain ourishes corresponds to changes in the balance of the United States treasury. These data are also represented visually, through the combination of an animated line plot and a Cherno face [1]. 1.3 Food Why stop at just vision and hearing? We can plot data as food and use all ve senses. One example of this is Census Spices, a set of spices that represent di erent neighborhoods based on demographics collected by the United States Census [5].
We at csv soundsystem have been exploring multisensory data plotting methods, including music videos and food. In our production of data-driven music videos, we have recognized a need for data analysis software and music software to be more strongly integrated. We wanted a more seamless transition between modeling and music, and we wanted it to be easier for data analysts to work with music. We have developed tools like Sheetmusic to bridge this gap. To use the language of the Grammar of Graphics, [8] we have abstract data and concrete plot elements, and we de ne aesthetics that provide mappings between the abstract data and the concrete elements. The primitive plot elements that we use for music are things like key, rhythm, pitch, and interval.
We've found that the tabular representation of data aligns very well with typical representations of music. Our data music tools work by mapping these two concepts to each other.
We can think of data tables as collections of similar things, with the same sorts of information being collected about each thing. In tidy data tables [7] each row corresponds to an observation (a thing), and each column corresponds to a variable. We add more rows to the table as we observe more things, and we add more columns to the table as we collect more information about each thing.
We can think of music as a composition of many di erent sounds over time, with sounds coming from many di erent instruments. In musical scores we represent time as movement from left to right, and we represent di erent notes played at the same time by di erent dots on a sta , The sta becomes wider as the song gets longer, (They are often spread across multiple pages.) and we add more dots as we add more notes ( gure 2).
Rather than composing music as traditional sheet music, we can use a table-editing program of our choice to compose this sort of table. Our data music software simply adds musical functions to table containers in various data analysis tools. Sheetmusic is our o ering for spreadsheets, but we also have libraries for R data frames [3] and Pandas data frames [2].
Let's divide Sheetmusic's features into two groups. The rst group is spreadsheet functions for music synthesis|these are functions like CHORD_PROGRESSION that take spreadsheet cells or values as input and return values to other spreadsheet cells. The second group is functions for rendering the music to external devices, including MIDI and sheetmusic.
Sheetmusic expects that the spreadsheet be organized as follows. Each column corresponds to a musical track, and different tracks can have di erent music instruments. Row corresponds to a beat (of time). Each cell contains the frequency of sound to be played, represented in scienti c notation (C4, D4, &c.). 3.2
The data analyst can use conventional spreadsheet approaches for composing music. For example, the following function can be used to produce a major scale in a spreadsheet column. =IONIAN_SCALE("A4") Once you have a major scale in one column, you can easily make chords with a spreadsheet functions like this. =MAJOR_THIRD_INTERVAL(B1) If you put this in cell B1, A1 and B1 will form a major third interval.
Once we have composed our piece, we can select the appropriate cells, specify the key and time signatures of the piece, and export it as MIDI or sheetmusic.
It is possible, of course, to convert to any number of music formats, just as we can convert spreadsheets to any number of data table formats. Only MIDI and sheetmusic are implemented at present, but you can indirectly convert to many formats by rst saving as MIDI and then converting from MIDI to your output format of choice.
I've discussed how we can use Sheetmusic for conventional music composition. To use it as a plotting tool, we simply have to map our abstract data to musical notes. Sheetmusic provides the FROMWHOLENUMBER function to enable this. If we imagine an in nitely wide piano with the C0 as the left-most note, FROMWHOLENUMBER starts at C0 and walks i keys to the right, where i is the argument passed to FROMWHOLENUMBER. After using ordinary spreadsheet modeling functions to manipulate data, a user may scale and round the data appropriately and then run FROMWHOLENUMBER to convert them into notes.
I hope that I've shown how data can be plotted in the form of music. I would be remiss not to discuss the merits of this plotting method.
When we plot data as music, we e ectively let data compose music for us. We still have to choose datasets that will produce interesting music and map the data to the music appropriately, but the randomness of the data can provide the various subtleties of music that we would otherwise have to design ourselves.
When we start using data analysis software for other things, we blur the line between data analysis and other things. Data analysis seems very magical to many people. When we represent data as familiar things like music, people seem to be a bit less scared of data analysis.
The use of multiple senses may also allow for the expression of high-dimensional datasets. Tufte advocates for the production of visuals that express the multivariate nature of the world.[6] I think that the use of multiple senses has the potential to facilitate the expression of more easily express many variables at once, and this may aid in the identi cation of high-dimensional relationships.
[1] B. Abelson, J. Bialar, B. DeWilde, M. Keller, T. Levine, and C. Podkul. FMS Symphony, 2013. [2] csv soundsystem. Data music for big data analysis, 2013. [3] csv soundsystem. ddr: Data-driven Rhythms in R, 2013. [4] S. Ferguson, W. Martens, and D. Cabrera. Statistical
Soni cation for Exploratory Data Analysis. [5] H. Kang-Brown. Making Census Data Taste Like New
York City, 2013. [6] E. R. Tufte. The Visual Display of Quantitative
Information. Graphics Press, Cheshire, CT, USA, 1986. [7] H. Wickham. Tidy data.
http://vita.had.co.nz/papers/tidy-data.pdf. [8] L. Wilkinson. The Grammar of Graphics.
Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2005.