Process mining is applied to a wide variety of use cases, most typically for processes like order-tocash and purchase-to-pay. Respective algorithms are able to discover bottlenecks, identify recurring patterns and deal with concept drifts. So far, use cases in cultural heritage scenarios are scarce. Music scores provide an ideal opportunity for process mining algorithms; they are structured into notes, measures, repetitions, parts, instruments, etc. In addition, there are compelling characteristics, like varying dynamics, transformations through modulations, and phase delays as in a fugue. In this demo article, we present a tool that is able to do a process-style analysis of music scores. Most notably, it transforms scores into an event log and performs basic process discovery. We see potential not only for music theorists but also as a pedagogical tool to illustrate process mining concepts and as a means to produce event logs for advancing process mining algorithms and visualizations.
Process mining is used in a wide variety of use cases for all kinds of industrial and societal
processes [
Use cases in the field of cultural heritage are rare so far. For instance, process mining
techniques have been compared to the methodological tool chaîne opératoire common in
archaeology [
In this demo paper, we present a proof-of-concept tool that is able to transform musical scores in the popular MusicXML notation to event logs. This enables new interpretations of musical art, for instance, by analyzing rhythm and repetitions in a process-oriented visualization. Specifically, we encode a piano piece of a recent number one hit single as event log. The visualized results are surprisingly clear. Therefore, we are optimistic that this type of analysis can help advance not only music theory but also process mining by making the large body of music scores available as event logs. Our tool can also be used as a pedagogical instrument to vividly explain business process management concepts.
Regarding music and data mining, there are not many related work applying standard data
mining methods to music; a notable exception is the book Music Data Mining [
Especially in orchestral pieces, there is a lot of concurrency by instruments played simultaneously; throughout history, there have been many genres with similar patterns, like repetitions and variations. We postulate the following research question: Can music scores be processed so that existing process mining tools can be used to analyze musical pieces?
Dynamics: The volume of (a sequence of) notes; could represent the amount of resources consumed.
Modulation: Refers to a key change, if a melody is replayed on a diferent base note; corresponds to concept drifts.
Note: Basic unit of a score, besides pauses; can be considered the most fine-grained activity. Repetition: Looped parts, often explicitly denoted variations; stands for re-executions. Rhythm: A sequence of beats forming recognizable patterns; could represent the timing of events.
Typically, these features are combined in infinite possible variations. Throughout history, particular styles have formed, such as the fugue, where a musical theme is repeated in various pitches and through diferent instruments. It is trivial to see further parallels to process mining techniques such as frequent pattern mining and local process models.
Our general approach is as follows. First, we parse a digital music score file. Currently, the entire piece of music is considered as one case. In a preprocessing step, we either select a single part (e.g., the left or right hand in a piano piece or a specific instrument in a classic orchestra piece) or merge all notes played in parallel. We then step through the score, following repetitions, and create one activity per unique measure, c.f. Figure 1. As the activity notion, we serialize the measure and reuse existing activities if the same musical pattern has already been discovered. Based on the annotated tempo, we calculate a timestamp for each measure.
Our implementation is based on a Python script using the music21 [
As an example piece, we chose a relatively simple piano arrangement of the worldwide number-one hit “As It Was” by Harry Styles as a MusicXML file. It is a suitable example, as the characteristic sequence shown in Figure 1 fits in one measure and is repeated several times. The resulting process model is shown in Figure 2 as a directly-follows graph (DFG). It is very easy to see that the melody known from Figure 1 (where the left hand has a break) forms the first and the last measure of the song. In addition, several repetitive patterns can be recognized, including a large one, as well as the repeated succession of the characteristic sequence {5, 6, 7, 8}. On the web frontend, the activities in the process model can be clicked to replay the corresponding notes as audio.
In this paper, we positioned process mining as a technique to analyze music. This is possible since music scores share many characteristics with business processes. As a proof-of-concept, we presented a tool that transforms musical notation into event logs so that hundreds of process mining techniques can be applied directly. Therefore, we can answer the research question from Section 1; it is possible to repurpose process mining tools for music scores by performing a number of preprocessing steps. In the demo, we analyzed the 2022 hit “As It Was” by Harry Styles by transforming a piano arrangement as an event log and creating a DFG.
The current algorithm that turns music scores into process models follows a very simple scheme by taking measures as input and assigning them labels sequentially. As a next step, we plan to realize diferent activity notions beyond measures, such as complete melodies, and thereby validate event abstraction techniques. Similarly, we experiment with our case notion to infer the process behavior of entire musical epochs possibly. With object-centric event logs, we could model the concurrent interplay of multiple instruments and the resolution of measures to notes. We plan to discuss our tool with music theorists to evaluate its applicability. It could give a new perspective for analyzing extensive music collections, e.g., the whole Köchel catalog, a collection of compositions by Wolfgang Amadeus Mozart. The current approach focuses on the modern Western notation of music, neglecting other notation systems of the world. Similar formal notations exist for dance notation that could be similarly incorporated into our framework.
Process models as preprocessed scores could also serve as intermediary inputs to machine learning algorithms that output creative compositions or be used to explain them. Also, measure embeddings could be used instead of simple look-up tables, similar to word2vec. Given the large body of available music scores, our tool could help advance process mining techniques by providing countless (acoustically replayable and explainable) examples for training or be used in teaching to illustrate BPM concepts. Likewise, acoustically playable process models could facilitate the accessibility of process mining tools for visually impaired people.
Besides music scores, our approach is transferable to streaming settings by capturing live music. This would make the parallels to techniques such as conformance checking even more apparent, as slight variations like length and hitting a wrong note are recognizable as deviations.