Symbolic representations of music are emerging as an important data domain both for the music industry, and for computer science research, aiding in the organization of large collections of music and facilitating the development of creative and interactive AI. An aspect of symbolic representations of music, which di erentiates it from audio representations, is their suitability to be linked with notions from music theory which have been developed over the centuries. One core such notion is that of functional harmony, which involves analyzing progressions of chords. This paper proposes a description of the theory of functional harmony within the OWL 2 RL pro le and experimentally demonstrates its practical use.
Music in general is complex and can be analyzed in many di erent ways, across di erent levels of abstraction, and regarding di erent aspects of it. Algorithmic analysis of music is increasingly determining how music is consumed {for instance through recommender systems of various streaming platforms, and how music is produced {for instance using plugins which implement various music analysis algorithms within digital audio workstations (DAWs). Music information retrieval (MIR) and automatic music analysis on a large scale, mostly rely on signal processing and machine learning (ML) {in the audio domain{ and do not make use of symbolic representations. However, there exists an abundance of symbolic music which is openly available on the web and could be utilized, in conjunction with knowledge representation technologies, for analyzing music.
A particular aspect of music which may be analyzed based on symbolic
representations is that of harmony. In music, harmony relates to the way humans
perceive superpositions of sounds {for instance chords{ and how sequences of
Copyright © 2021 for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
such superpositions {such as chord progressions{ may be used within a
composition in order to serve the underlying music and the emotions it conveys. There
are multiple di erent theories of harmony, across di erent cultures, which
however share some common features. Many approaches to harmonic analysis for
instance focus on the ideas of building tension and resolution by the appropriate
use of chords or pitches within a piece of music. One of the most widely used
such frameworks for analyzing harmony, especially in western culture, is that of
tonal harmony [
Algorithms for automatic functional harmonic analysis have been proposed
in the literature and have successfully been applied in a practical setting. An
important example of such work is HarmTrace [
In the proposed approach, we represent the theory of tonal harmony, modal
harmony and functional harmonic analysis using Description Logic (DL)
languages. DL languages enrich knowledge representation systems, in terms of the
language expressivity for describing the knowledge base, the reasoning
techniques and the e ectiveness of the interaction between the user and the system
[
By representing music theory as a TBox we are able to perform analysis of chord progressions via reasoning. We have identi ed many bene ts of this approach when compared to others, namely ease of use {through the utilization of standardized technologies such as the web ontology language OWL 2, extendibility of the framework by either adding axioms to the TBox or by describing different aspects of music theory with description logics, and their ability to be interlinked with other sources of data {since these technologies are intended for use on the semantic web.
The rest of the paper is structured as follows: Section 2 presents a
comprehensive list of ontologies and data models which are aimed at representing
information related to music. Section 3 describes in detail the translation of tonal
and modal harmony into a fragment of description logics which is compatible
with the OWL 2 RL [
Symbolic representations of music vary in form, information content and
application domain. One of the most widely used such representations is the Musical
Instrument Digital Interface (MIDI) [
Alvaro et al. [
chords.com
2 http://motools.sf.net/timeline/timeline.html
3 http://motools.sf.net/event/event.html
http://www.guitartabs.cc,
http://www.eknowledge representation. Jones et al. [
Sabbir et al. [
In this section we describe the terminology used from music theory, and we
present axioms which may be used to infer the function of a chord in the context
of a chord progression [
Let V = hCN; RN; INi be a vocabulary, with CN, RN, IN mutually disjoint nite sets of concept, role and individual names, respectively. Let also T and A be a terminology (TBox) and an assertional database (ABox), respectively, over V using a Description Logics (DL) dialect L, i.e. a set of axioms and assertions that use elements of V and constructors of L. The pair hV; Li is a DL-language, and K = hT ; Ai is a knowledge base (KB) over this language.
The Web Ontology Language (OWL) is a family of knowledge representation languages for authoring ontologies. OWL classes correspond to description logic (DL) concepts, OWL properties to DL roles, while individuals are called the same way in both the OWL and the DL terminology. 3.2
In Western culture, music typically uses twelve pitch classes across any number
of octaves, the musical notes (mto:Note). Harmony concerns itself with sets of
the twelve pitch classes in a given context. The context is ambiguous {it relates
to both local and global characteristics of a piece of music, however in every case
it involves a speci c pitch class which is called the tonic note. Humans tend to
perceive superpositions of sounds by identifying a fundamental frequency, which
in most cases we assume to be the tonic note, and all other frequencies are
perceived in relation to the fundamental frequency [
Scales and Modes In music theory, a scale (mto:Key ) is any set of musical notes ordered by pitch. The root of the scale is its tonic note. In Western music, scales are typically formed by seven di erent pitches (heptatonic scales). A di erence in pitch between two notes is called an interval (mto:Interval ). The smallest of these intervals is a semitone (mto:Semitone). A tone (mto:Tone) is an interval composed by two semitones. A diatonic scale is an heptatonic scale (mto:HeptatonicScale) structured using ve tone (T) intervals and two semitone (S) intervals. The two semitone intervals should be separated by two and three tones. The most commonly used diatonic scale is the major scale (mto:MajorKey ). The sequence of its intervals is (T-T-S-T-T-T-S). In Western music, there are seven di erent diatonic scales, which are built by taking a di erent position of the major scale as their root. They are most commonly known as the Modes of major scale (Ionian/Major, Dorian, Phrygian, Lydian, Mixolydian, Aeolian/Natural Minor, and Locrian). The most commonly used non-diatonic scales are the harmonic minor (mto:HarmonicMinorScale) and the melodic minor scale (mto:MelodicMinorScale). Modes that are structured using the same seven pitches are called relative modes. Modes that are structured using the same root are called parallel modes. In heptatonic scales, the position each different note holds is called scale degree (mto:Degree), usually starting with rst for tonic.
Tertian Chords In music theory, a chord (mto:Chord ) is any set of three or more musical notes that are heard as if sounding simultaneously. In Western music, chords are typically based on thirds known as Tertian Chords. The most frequently used tertian chords are triads (mto:Triad ). Triads consist of three distinct notes: the root, an interval of a third above the root and an interval of a fth above the root. Tertian Chords with four or more notes are called extended chords. Extended chords include seventh chords (mto:SeventhTetrad {a triad with an interval of a seventh above the root), ninth chords (mto:NinthPentad {a seventh chord with an interval of a ninth above the root), eleventh chords (a ninth chord with an interval of an eleventh above the root) and a thirteenth chord (an eleventh chord with an interval of a thirteenth chord above the root). Other Chords The most commonly used non Tertian chords are the suspended chords (mto:SuspendedChord ) and the added ninth chords. A suspended chord is a tertian chord whose third interval above the root has been replaced by a second or a fourth interval above the root. An added ninth chord is a triad with an interval of a ninth above the root.
Chord Progressions In musical composition, a series of chords is called a chord
progression (mto:HarmonicProgression ). In Western music, chord progressions
are the foundation of harmony. All popular western music genres use chord
progressions as the base on which melody and rhythm are built.
Tonal Harmony As we mentioned, the tonic note is the root of any mode.
It is also its tonic center, meaning that every other pitch is perceived in
relation to it. In tonal harmony, a chord can serve three di erent functions: a
tonic function, a dominant function or a subdominant function. A tonic chord
gives us the acoustic impression that we are in the tonic center. A dominant
chord creates an acoustic instability that most frequently leads to a tonic chord
(resolution). A subdominant chord has an abstract function. Whilst it is not
the tonic center, it is not as unstable as a dominant chord. It can lead to a
tonic chord or prepare a dominant chord. The rules of tonal harmony apply
to the major scale (Ionian Mode) and the minor scales (Harmonic, Melodic
and Natural Minor Mode). In the major scale, the tonic chord is the chord
of rst degree (fho:IonianTonic) and the relative tonic is the chord of sixth
degree (fho:IonianRelativeTonic), dominant chords are the chords of fth and
seventh degree (fho:IonianDominant,fho:IonianSeventhDominant ) and
subdominant chords are the chords of fourth and second degree (fho:IonianSubdominant,
fho:IonianRelativeSubdominant ). Chords of third degree can serve as a
relative tonic or a relative dominant (fho:IonianRelativeDominant ). In the minor
scales, the tonic chord is the chord of rst degree (fho:AeolianTonic) and the
relative tonic is the chord of third degree (fho:AeolianRelativeTonic),
dominant chords are the chords of fth and seventh degree (fho:AeolianDominant,
fho:AeolianRelativeDominant ) and subdominant chords are the chords of fourth
and sixth degree (fho:AeolianSubdominant, fho:AeolianRelativeSubdominant ). We
extend these rules to all modes depending on the type of their rst degree chord.
Modal Harmony Modal Harmony [
Chords At the top level of our hierarchy is the fho:Chord mto:Chord class . There are direct subclasses for di erent types of tertian and suspended chords: AugChord t DimChord t MajChord t MinChord t SusChord v Chord and for different note cardinalities of chords: Triad t Tetrad t ::: v Chord. There is also a direct subclass for chords which belong to a mode ModalChord v Chord, where the class ModalChord has as direct subclasses, all major and minor scales:
CMajorScaleChord t CsMajorScaleChord t ::: v ModalChord and each major scale chord class has as direct subclasses each of the seven corresponding modes:
CIonianChord t DDorianChord t ::: v CMajorScaleChord In addition, each of these classes which correspond to the chords of a speci c mode has as direct subclasses the scale degrees of the mode:
CIonianFirst t CIonianSecond t ::: v CIonianChord Finally, each scale degree has as direct subclasses, all tertian and suspended chords which belong to the speci c scale degree of the speci c mode:
C t Csus4 t Cmaj7 t ::: v CIonianFirst The type of each chord (Major, Minor, Diminished, Augmented, Suspended), in addition to the number of notes it consists of are also asserted in the ontology:
Cmaj9 v Pentad u MajorChord Tonalities At the top level of our hierarchy is also the Tonality class. Its subclasses are all the di erent modes:
CIonianTonality t CsIonianTonality t DDorianTonality t ::: v Tonality Each Tonality has its Tonic chord. To describe this concept, we de ne the object property hasTonality with a domain of Chord and range of Tonality:
CIonianFirst u 9hasTonality:CIonianTonality v CIonianTonic All Tonic chords are subclasses of the class Tonic.
Chord Functions At the top level of our hierarchy is also the FunctionChord class. Among its subclasses are the three harmonic functions:
TonicChord t DominantChord t SubdominantChord v FunctionChord Each of these classes has nine subclasses, one for each mode:
IonianDominant t DorianDominant t LocrianDominant t ::: v DominantChord and each of these classes has twelve subclasses, one for each pitch class: AIonianDominant t AsIonianDominant t BIonianDominant t ::: v IonianDominant Chord Progressions and Axioms For representing chord progressions we de ne the object properties hasNext hasPrevious with a domain and range of Chord. Now we can add axioms to each FunctionChord, based on the functions of the scale degree of each mode as we have described, for example:
CIonianFifth u 9hasNext:CIonianFirst v CIonianDominant
Parallel Functions In order to use concepts of the Modal Harmony, we rst introduce the ParallelTonic class fho:ParallelTonic. Its direct subclasses are the Tonic chords of every di erent pitch:
CParallelTonic t CsParallelTonic t DParallelTonic t ::: v ParallelTonic Each of these subclasses has its direct subclasses:
CIonianTonic t CDorianTonic t CPhrygianTonic t ::: v CParallelTonic Now we can extend the chord functions by adding axioms based on the Modal harmony, for example:
CIonianFifth u 9hasNext:CParallelTonic v CParallelDominant Local Functions In a music piece, the tonic center can change. Any chord can serve as a tonic center depending on its relationship with its adjacent chords. For this reason, we introduce the classes LocalTonic, LocalDominant and LocalSubdominant, where:
Tonic u 9hasPrevious:ParallelDominant v LocalTonic
DominantChord u 9hasNext:LocalTonic v LocalDominant
DominantChord u 9hasNext:LocalDominant v LocalSubdominant 4 4.1
For setting up our experiments we rst gathered a set of commonly used datasets of music annotated with chords, and represented them them in an RDF knowledge graph. We fed chord progressions for which the key was known through HarmTrace and compared the resulting analyses with those resulting from reasoning on our ontology. We setup a semantic repository where we loaded the ontology (described in section 3) and data (described in section 4.1.2), and then get the answers for a set of SPARQL queries which are described in detail in section 4.2. All experiments were performed on the same Intel Core i7-7500U machine running at 2.90 GHz with 8 GB of installed RAM and a stable internet connection of 4.84 Mbps average upload speed.
Datasets For the evaluation procedure we gathered four datasets from di erent
sources. These are:
{ Isophonics Beatles Reference Annotations [
Comparison with HarmTrace HarmTrace performs an automatic functional harmonic analysis based on the rules of Tonal Harmony. Taking as input the chord progression and the key of a musical piece, it generates a parse tree to visualize the function of each chord. It performs deletions and insertions of chords, so every chord depicted on the tree has a function based on the rules of Tonal Harmony. HarmTrace uses only tonic and dominant function to characterize chords or groups of chords. We used HarmTrace to analyze the hooktheory Dataset, which is the only Dataset with known song keys. The analysis took
8 https://www.hooktheory.com/ 9 https://graphdb.ontotext.com/ 1253 seconds (1.68s per chord progression). HarmTrace performed 3832 deletions (5.25 per chord progression) and 4768 insertions (6.41 per chord progression) in a total of 16883 chords. In this given dataset, HarmTrace found 3264 chords or groups of chords which serve a tonic function and 7604 chords or groups of chords which serve a dominant function. In order to compare our system with HarmTrace, we enriched our ontology with the concepts of SongTonic, SongRelativeTonic, SongDominant and SongRelativeDominant using the object property hasSongTonality, considering that the given dataset provides the tonality of every chord progression. Our system found 2767 chords that serve a tonic function and 7529 chords that serve a dominant function. In Figure 1 we show an example chord progression, along with the analysis generated by HarmTrace and in Table 1 we show the analysis from reasoning on our ontology for the same progression.
Even though our ontology cannot be directly compared with HarmTrace, mainly due to them being based on di erent theoretical frameworks (tonal vs modal functional harmony) and due to the fact that HarmTrace inserts and deletes chords, the resulting analyses are in most cases identical. A bene t of HarmTrace is the hierarchical representation of analyses which could provide a musician with more insight when compared to our approach. On the other hand, our approach is theoretically more thorough, since tonal harmony is part of modal harmony, in addition to being empirically more usable. We base this claim rstly on the fact that insertions and deletions of chords performed by HarmTrace can be confusing for a user who has not extensively studied the grammar which HarmTrace is based on, and secondly we argue that using description logics along with standardized technologies such as OWL 2, provides more usability and extendability when compared to Haskell. Dataset Comparison For this set of experiments we performed SPARQL Queries in order to nd di erences based on the harmonic structure between
Inferred Types LocalSubdominant D PhrygianParallelDominant
LocalDominant G IonianParallelDominant G AeolianSongTonic C AeolianParallelDominant LocalSubdominant F MixolydianParallelDominant
LocalDominant B ParallelDominant G AeolianRelativeTonic E LydianParallelDominant
LocalTonic A LocrianParallelDominant LocalSubdominant D ParallelDominant
LocalDominant G ParallelDominant G AeolianSongTonic C MixolydianDominant
LocalDominant F ParallelDominant
LocalTonic B[ AeolianDominant LocalSubdominant E[ ParallelDominant
LocalDominant A LocrianDominant
LocalTonic
LocalDominant G ParallelDominant
G AeolianSongTonic the Jazz Dataset, and the Billboard and Beatles Dataset, which can both be labeled as pop-rock genre. On these datasets, the tonality of each song is not known, so we are not able to de ne the tonic center. We will focus on the dominant and subdominant functions instead. As we mentioned, a chord with a dominant function is used to build tension, whereas a chord with a subdominant function has a much more peaceful sound when it resolves to its tonic. We can also assume that a chord with a local tonic function, which does not serve a dominant function, o ers a brief acoustic stability. The results of our queries are shown in Table 2.
As we can see, in the Jazz genre, over the half of the chords are used to build tension. In contrast, in the pop-rock genre, the dominant and subdominant functions are more equally distributed. This demonstrates that even with simple SPARQL Queries, we are able to nd distinct di erences between dissimilar music genres based only on the harmonic structure.
Expressing notions from music theory with description logics provides a practical way for semantically enriching datasets of symbolically represented music. This process could signi cantly aid in the organization of large collections of music, in addition to potentially enhancing performance for various music information retrieval tasks such as genre or mood classi cation . We plan to explore the impact of utilizing these additional features for machine learning pipelines in future work.
Furthermore, formally representing music theory with description logics could be useful for music education. Learning abstract concepts such as harmony can be challenging for music students. By using knowledge representation we are able to provide countless real world examples of chord patterns, in addition to providing explanations for entailments. One of our future research directions involves working with music educators with the goal of developing a tool, by utilizing music theory knowledge representation, to be used by music students in an educational setting.
Additionally, RDF triples provide a versatile way to represent music information. A direction of future research involves converting MIDI and MusicXML les into RDF using concepts introduced by our ontology in order to perform functional harmonic analysis. Considering that MIDI and MusicXML are the most common tools used for symbolic music representation, this will provide accessibility to a much larger variety of music information as well as make the procedure of functional harmonic analysis easier to the average user.
Finally, as mentioned in the introduction, there are multiple music-theoretical frameworks for describing harmony, especially when taking under consideration di erent cultures. In addition there are other aspects of music besides harmony, such as rhythm, melody and structure, which have strong theoretical foundations. Thus another direction of our future research involves representing additional notions from music theory with Description Logics. 6
This research is carried out / funded in the context of the project \Music Synthesis based on Knowledge Representation, Automated Reasoning and Machine Learning" (MIS 5049188) under the call for proposals \Researchers' support with an emphasis on young researchers-2nd Cycle". The project is co- nanced by Greece and the European Union (European Social Fund- ESF) by the Operational Programme Human Resources Development, Education and Lifelong Learning 2014-2020."