Musical patterns are important for many musicological tasks, such as genre classification, identifying common origins of pieces, and measuring similarities between compositions. Our previous research has defined several types of patterns in Irish traditional music and developed tools for extracting them from databases of musical scores. We now wish to enable flexible and eficient querying, open access, preservation, and integration with multiple data sources. To address these needs we use semantic web technologies. We present a music pattern ontology based on the Music Annotation Pattern (an ontology design pattern [1]) which formalizes key concepts in musical annotations. We developed a pipeline (Patterns2KG) to process existing data through the ontology, resulting in the production of Knowledge Graph triples. In this research, we processed a total of 40,152 compositions from The Session dataset to create the knowledge graph, comprising approximately 45 million (44,979,281) statements. We show sample SPARQL queries to document model usage. In assessing the efectiveness of our work, we engaged with musicologists to gain insights into their specific needs, which we translated into Competency Questions (CQs). These CQs serve to evaluate the model.
In musicology, musical analysis enhances our understanding of the content of the music. It provides insights into musical style and genre, assessing the compositional techniques used by the composer, tracking the musical development and evolution of diferent styles and genres over time, classifying music, recognizing genres, and even generating music. In oral traditions, where pieces are transmitted from generation to generation without being written down, finding musical relationships among pieces is particularly important because they can give insight into the evolution of, and connections between, musical traditions.
Musical analysis may focus on one or more “layers” of the music, such as musical structure, harmony, rhythm, dynamics, and texture. This work follows the most-common approach in studies of Western folk music, focusing on melody, which is defined simply as a sequence of notes, with durations and rests.
Patterns within or between such sequences are the focus of our research. Using multiple definitions of patterns, and tools for extracting patterns from databases of musical scores in the Irish folk tradition, we have previously created a large database of patterns. In the current paper, we describe the process of formalising this data as Linked Open Data (LOD), using an ontology and knowledge graph (KG).
Pattern is a central concept in many fields. Mathematics has been called “the science of
patterns” [
Pattern analysis plays a significant role in the rich understanding of music. For example,
Schenker [
Creating a complete analytical taxonomy of pattern types and definitions has not been achieved in previous work, and we do not attempt this. To be concrete, we have focused on a limited taxonomy of pattern types, all based on -grams of scale degree values. In Western music, a pitch can be defined simply as an integer (e.g., in the range [0, 127] in the common chromatic MIDI encoding). For example, the “middle C” note on the piano is MIDI note 60. Several other integer representations for pitch are also common but discussion is out of scope here. We also use the concept of accented notes: an accented note is a note which occurs on a beat. With these definitions, we have a simple taxonomy. In our input corpus we count all -grams of consecutive accented notes (represented as scale degree values) for = 4 … 10 . In our earlier work, we developed the FoNN (FOlk N-gram aNalysis) tools1 which exhaustively and deterministically extract patterns from compositions. Briefly, all subsequences of length are extracted, and any which occur more than once in the corpus are taken as patterns. Although we focus on Irish traditional music, the representation and all methods in the paper are suitable for other forms of European folk music and to some extent for other forms of music when represented monophonically.
The focus of this research is to present and preserve patterns within a robust structure, enabling efective music analysis. We propose to utilize ontologies and KGs to represent, organize, and structure musical pattern information, thus facilitating easier access, manipulation, and data analysis. Additionally, ontologies and KGs establish a shared vocabulary and standardized method for representing musical information. This, in turn, facilitates the integration of data from various corpora, cross-referencing, and linking related compositions. Furthermore, it provides a platform for discovering relationships among diferent pattern definitions as well.
Therefore, firstly, we developed an ontology for musical patterns. It includes data such as the
musical composition where the pattern is found, frequency and locations of pattern occurrences,
the duration of the composition, the pattern contents, the pattern type (i.e. the specific definition
of the pattern in use), and the complete composition contents. The latter is to facilitate pattern
tracing within a composition. Additionally, the ontology includes metadata, such as composition
name, family (applicable to “tune families” which occur in traditional music), composition type
(e.g. jig or reel, again applicable to traditional music), key signature, and the name of the corpus
from which the composition is drawn. We will describe the ontology and KG in Section 2.
Further, we also developed a custom software pipeline, based on previous work by project
partners [
In this section, we present the proposed ontology and relevant details related to the requirements for modeling patterns, as outlined in the previous sections. We provide information about the software components that were developed to populate the knowledge graph.
Our work follows an existing workflow, the Smashub 2 workflow. Smashub uses an ontology for
annotations in a musical score or recording, and this ontology is part of the Polifonia Ontology
Network (PON)3. The PON is a set of OWL ontology modules that describe the content and
context of tangible and intangible musical cultural heritage assets across Europe [
To adapt the Smashub workflow to the case of melodic patterns, we created a custom JAMS pattern schema and SPARQL query, which will be described in Section 3
Several other ontologies focusing on music have been described in the literature, but none focussing on musical patterns.
The purpose of the MAP-ODP is to model diferent types of musical annotation such as chords,
structure, and patterns [
jams:JAMSAnnotation jams:JAMSIncludesObservation
jams:JAMSObservation rdfs:subClassOf jams:includesObservation owl:allValuesFrom jams:JAMSScoreObservation rdfs:subClassOf jams:JAMSScoreObservation rdfs:subClassOf jcmmaofcmr::se:: hhhhttttttttpppp:sss/::///:w///3ww/i33wdii3.ddo..iroodgrr./ggop//ropploogill/fiipoffonoolinnaiii/aafo//onOOntNNio//lmmaouu/gssOyiiN/cc/jaaallcm--osfcr/eoeam/tpuorsei/tion/
There are two classes defined, named as jams:JAMSObservation and jams:JAMSAnnotation. The jams:JAMSAnnotation class includes multiple observations. The main concept that expresses patterns is the jams:ScorePatternOccurrence class. It specifies the occurrence of a pattern at a specific time in a specific composition. Its properties, such as jams:hasLocation and jams:hasPatternType, and jams:hasPatternComplexity are defined to represent the location, type, and complexity of a pattern respectively. The jams:hasPatternComplexity gives a simple numerical representation of complexity. It is the fraction of unique pitch values in a pattern. This can be used for various tasks such as filtering common patterns, interesting and unique patterns, etc.
The actual content (i.e., scale degree values) is described as a separate class, i.e., jams:ofPattern and it creates links between various patterns across the corpus. This will be used later to connect various composition in a corpus or diferent corpora. The jams:hasPatternComplexity represents how complex a pattern is which is the fraction of unique pitch values in a pattern and the length of the pitch. The current version of the ontology only shows the concepts and properties associated with patterns. In addition to this, various metadata information of the musical composition is also modeled, including jams:key, jams:timeSignature, jams:transcriber, jams:tuneContent, and jams:tuneFamily, etc.
Figure 2 shows an illustrative extract from the KG, with important concepts and relationships. As shown, the KG contains musical compositions with metadata such as titles. Each composition may have annotations, in particular annotations of pattern occurrences. A pattern occurrence is a location in time in a particular composition where a pattern occurs. A pattern has contents. mc:MusicalComposition JAMSAnnotation jams:isJAMSAnnotationOf
jams:includesObservation mc:title jams:includesObservation "1stAugust935"^j^axmds::sStcroirnegPatternOccurrence jams:ScorePatternOccurrence jams:ofPattern jams:hasPatternType ja"m1s0:"h^a^sxLsodc:aitniton jams:hasPatternType jams:ofPatjtaemrsn":3h5a"s^L^oxcsadtp:iiiotnncth-class-values^^xsd:string j(a4m,s2:,P4a,t1t,e4r)n pitch-class-values^^xsd:stringj(a7m,s0:,P7a,t0t,e7r)n jams:ofPattern "5"^^xpsidt:cihn-tclass-values^^xsd:string "48"^^xsd:int jams:hasLocation jams:hasPatternType jams:ofPattern pitch-class-values^^xsd:string
jams:hasLocation "1stAugust14126"^^xd:string jams:hasPatternType mc:title jams:ScorePatternOccurrence jams:ScorePatternOccurrence
jams:includesObservation jams:includesObservation jams:isJAMSAnnotationOf mc:MusicalComposition JAMSAnnotation
It illustrates also that pattern observations are recorded along with additional details such as pattern content (jams:ofPattern), types (jams:hasPatternType), and locations (jams:hasLocation). As described, many types of pattern are possible, but in our KG only the “pitch-class-values” type is used, so the pattern contents is the list of integer pitch values.
The Patterns2KG4 pipeline consists of several stages. First, pattern sequences are extracted
from the dataset using FoNN tools. Details are discussed in Section 3.2. The dataset includes
composition metadata, and details can be found in Section 3.1. Next, the Patterns2KG pipeline
processes the pattern sequences to generate a .jams file for each composition. P2KG uses a
specialized schema to represent patterns, which is described in Section 3.3. Next, P2KG loads
each .jams file and converts it to RDF. This is accomplished through a SPARQL query that
automatically loads the JAMS file and uses SPARQL Anything [
We begin by specifying the dataset we used. In our previous work [
The FoNN tools take input in ABC format and, for each composition, create sequences
representing pitch, onset, duration, and velocity values for each note using the music21 Python
library [
The MIR community uses JAMS: A JSON Annotated Music Specification for Reproducible MIR
Research [
The JAMS annotation ofers a default pattern schema named pattern_jku; however, the default schema is inadequate to model our requirements. For instance, it consists of fields such as pattern_id, midi_pitch, occurrence_id, and staff whereas we require to define types of patterns, lengths of patterns, and pattern contents so that we can preserve them in KG in the later stages. Therefore, a custom schema was designed, suitable for patterns extracted by FoNN tools, as shown on the left side of Figure 3. This schema is named pattern_fonn, with components including the pattern contents and location.
Using custom text-processing Python scripts, the pattern database was converted to .jams ifles which follow this schema. A snippet of the output is shown on the right side of Figure 3. The content is organised in two main sections, one for pattern information and the other for file metadata. Each composition leads to one .jams file, containing a list of annotations under data. Each annotation contains generic JAMS fields (suitable for any JAMS annotation), such as time and duration; and also contains pattern_fonn-specific fields under value. In the file_metadata section the identifiers link to a composition file on the The Session website.
3.4. Knowledge Graph – RDF generation using SPARQL Anything In the next stage, each .jams file is converted into a KG in the form of an RDF file. This is done using a custom SPARQL query7 This is a dynamically generated SPARQL construct query that transforms the JAMS file into RDF graph statements.
Two details of the SPARQL query are worth noting, as follows.
First, the unique URI for representing a jams:JAMSAnnotation is hashed with SHA1 to ensure interoperability with the other Polifonia Ontology Network (PON) modules.
Second, each jams:Pattern is also represented by a hashed URI. This URI means that multiple occurrences of the same pattern point to the same pattern object, as shown in Figure 2. It also shows how patterns of the same content are linked together to create an aggregated view of the whole knowledge graph. It is also worth mentioning that a composition can belong to any corpus and thus compositions of diferent corpora can be conveniently linked together, and thus it ofers an opportunity to perform inter-corpus pattern analysis. Furthermore, it is scalable in the sense that further corpora can be automatically added to enrich the knowledge graph without making any changes to the existing model. It is also highly flexible: third parties can introduce novel pattern types (e.g., not based on -grams) using jams:hasPatternType.
A total of approximately 45 million (44,979,281) statements were generated. The process takes time linear in the number of compositions – several hours for our complete KG. The resulting KG is publicly available via Blazegraph8 with a SPARQL endpoint9.
We used the eXtreme Design methodology, a well-known method for the evaluation of ontologies.
According to this, an Ontology shall address a set of competency questions (CQs). Later the
CQs are translated to SPARQL queries for extracting the modeled knowledge to ensure the
retrieved result serves its purpose. The CQs play a frame of reference role for evaluating
ontologies [
In order to address these CQs, we formulated SPARQL queries as shown below. They demonstrate the suitability of our proposed model in capturing pattern-related information of use to musicologists.
Later, a comprehensive analysis of each question was conducted, and we now present our responses and actions below. Essential namespace prefixes are given in Listing 1 for use in later Listings. These queries can be executed using the SPARQL endpoint already mentioned.
In Listing 4, the ?observation URI is constructed using pattern content, allowing the same URI to be identified in other compositions representing the same pattern. Consequently, in the subsequent statement, ?anotherTuneJamsAnnotation jams:includesObservation ?observation provides access to another composition’s name. Finally, a filter is applied to ensure that both annotations are not identical, to avoid self-retrieval.
Listing 2: Finding metadata information on a composition or corpus: query and results. Notice that providing a title will retrieve multiple variants, which have the same title but distinct IDs. SELECT distinct (concat(?tuneId, "-" ,?tuneTitle) as ?TuneInfo) ?TuneType ?TuneSignature WHERE { VALUES ?Pattern {<5_1_6_2_4_1>} ?observation jams:ofPattern ?pattern . ?tuneFile jams:includesObservation ?observation . ?tuneFile jams:isJAMSAnnotationOf ?tune . ?tune mc:title ?tuneTitle . ?tune jams:tuneId ?tuneId . ?tune jams:tuneType ?TuneType . ?tune jams:timeSignature ?TuneSignature . } LIMIT 4
10963-A Day In Sligo jig 6_8 11909-A Jig For Bernie jig 6_8 12265-All Alive jig 6_8 13055-Apples In Winter jig 6_8
Listing 3: Finding compositions where a given pattern was found: query and results. GivenTuneInfo MatchedTuneInfo SharedPattern Bucks Of Oranmore, The For The Sake Of Old Decency 2_5_2_5_2_5 Bucks Of Oranmore, The Tom Keane’s 2_5_2_5_2_5 Bucks Of Oranmore, The Hedgehog, The 2_5_2_5_2_5 Bucks Of Oranmore, The Furze In Bloom, The 2_5_2_5_2_5 Bucks Of Oranmore, The Lady Madelina Sinclair 2_5_2_5_2_5 Listing 4: Given a composition, find a ranked list of similar compositions (based on pattern similarity): query and results.
The Knowledge Graph (KG) serves as a semantic network that connects entities of diverse natures, allowing for easy integration of multiple data sources and scaling up the encoded information. Additionally, it facilitates the execution of complex queries that reveal relationships among participating entities. In this study, a pattern ontology based on Musical Annotation patterns was proposed, and a JAMS pipeline was developed to create a KG of the music patterns (n-gram pitch-class-values). We processed a total of 40,152 compositions of The Session dataset to create the KG, which comprises approximately 45 million (44,979,281) statements. The KG was evaluated using competency questions. By describing these CQs we have demonstrated some of the typical analyses which might be carried out by musicologists. The current modeling provides a solid foundation to undertake a deeper analysis of musical compositions and also has SELECT (concat(?tuneId,"-",?tuneName) as ?TuneInfo) ?Pattern WHERE { VALUES ?givenPattern {'5_2_5_2'} ?observation jams:ofPattern ?Pattern .
FILTER regex(str(?Pattern), ?givenPattern). ?tuneFile jams:includesObservation ?observation . ?tuneFile jams:isJAMSAnnotationOf ?musicalComposition . ?musicalComposition mc:title ?tuneName . ?musicalComposition jams:tuneId ?tuneId . } LIMIT 5
2069-Moving Bog, The 2_5_2_5_2_5 2069-Moving Bog, The 5_2_5_2_5_2 2069-Moving Bog, The 5_2_5_2_5_2 2069-Moving Bog, The 5_2_5_2_1 10268-White Houses Of Shieldaig 7_3_5_2_5_2 Listing 5: Given a pattern, find all compositions when it or a pattern that contains that pattern occurs: query and results. We store -gram patterns for multiple values of . Consequently, we can use this query to identify compositions where the specified pattern is a part of another pattern. This represents a containment relationship. the potential to answer typical musicologist requirements. In the future, we aim to integrate multiple corpora for cross-corpora analysis. We are also working on a user interface which uses our KG as a back-end, allowing non-technical musicologists to ask and answer research questions based on our KG. Finally, we would like to engage a wider community to ensure we can answer more complex types of user queries.
This work is supported by the Polifonia project. The Polifonia project has received funding from the EU Horizon 2020 research and innovation program under grant agreement No 101004746. The primary author completed this work while with the University of Galway.