This paper describes a text-based Question-Answering (QA) system for content-based music information retrieval (MIR) according to the C@merata task description [12,13].
Query parsing and music content operations are kept separate and the only connection between them is through an intermediate layer.
There are three major components of this approach are: • • •
The script language
A music information workflow interpreter
The query interpreter resolves the query text into a script that describes a music information workflow. This is a layered process that required hard-coded knowledge about valid query terms and types (see 2.2).
The script language consists of “information request” statements that are formed by the clauses: “select”, “from” and “where” having similar functionality as that described by the Structured Query Language (SQL) (see 2.3).
The music information workflow interpreter connects the script with a set of music-related functions that are built on top of the music21 framework (see 2.4). discuss the development of the question types over the past three years and in particular focus on the more sophisticated methods adopted for question generation this year. We will then present the participating systems for this year and discuss the results which they obtained.
The query interpreter is a class that is initalised with a “language” file that contains information about valid terms, their types, composite types and, composite type relations. Composite types are music concepts and will be referred to as entities. This file stores generic terms, but some values, e.g. names of the parts are extracted from music data.
The terms of a query phrase can be: • • • values, music concept/entity keywords, E, music concept/entity relation keywords, R.
For example, “dotted quarter note dominant 7th” is a chord entity. Entities are further categorized into “content” and “context” types. Although in the question set we tested, context entities are the parts and measures and content entities are note, rest, chord and simultaneity, it is the relation keywords that define what is the search context and what is the target content. Relations enable the transformation of the query into a structured request by defining the context-content relation. The conditions are just the entity attributes.
Some of the relation types that were identified in the tested question set are shown below (the “<>” symbol means any type of entity): < > (" ") < > , <(duration, pitch, note, chord)> < > ("followed by") < >, <(duration, pitch, note, chord)> < > ("in", "in the") < > contextual and conditional < > ("of", "of a") < > < > ("parallel") < > ("repeated") <> ("time","times") < > ("between”, “between the") < > ("and") < > < > ("against", "only against") < > …
The terms of the query phrase are processed in layers starting by identifying the type of each one. Next, composite types and words are grouped into entities. After all the types are matched, the entity relations are identified. Last, the query is converted into an information request using “select-from-where” statements.
Parse the query First pass: terms to types Second pass: type groups and relations Third pass: Content and Context identification Fourth pass: Make information request Run information request script with music framework seven-note chord in the harpsichord context : parts, condition: instrument get type : chord condition : cardinality value
After the query phrase analysis a script that contains a structured information request is generated by converting the identified entities and their relations into a sequence of “selectfrom-where” statements.
By ordering and nesting such statements, all the queries that were tested were successfully converted into this workflow representation.
The use of a “language” file is a way to pass knowledge to the system about how to parse phrases. It contains: • • • value collections grouped in primary types ◦ e.g. 15-18 is type range.int primitive types grouped in music concepts/entities ◦ “dotted quarter” is a duration entity ◦ “first inversion of a triad” is a chord entity Relation definitions ◦ groups of entities
The structured information request that was described in the previous section is parsed from a music information retrieval interpreter that compiles an executable music21 script using music21 functions such as “getElementByClass()” and a plethora of features for music21.elements to compare with. Operating within the music21 ontology, we can perform conditional part 1. 2. 2.1 2.2 2.3 2.4 3 selection, measure selection based on range, and get attribute values for basic elements such as note, rest and chord type.
One way to avoid over-analyzing the query into complicated
information requests is to use more complex representations, such
as note-sequences (VIS) [
These are preliminary results and the approach is under development. In the rest of this section we discuss how queries resolve into information requests and the difficulties in the process. # 3 octave leap in violin I context : part, instrument type and number get type : melodic interval, keyword "leap" condition : interval value # 5
Bb3, A3, G3, F3, E3 note,con:seq:comma, note, con:seq:comma, note, con:seq:comma, note, con:seq:comma, note context : complete piece ? separate parts ? get type : pitch sequence # 9
parallel thirds in measures 15-18 con:relation, interval_type, con:where:in, key, int:comp:range context : measures get type : chords:condition:thirds condition : parallel #10
authentic cadence in measures 14-18 cadence_type, key, con:where:in, key, int:comp:range context : measures get type : cadence condition : cadence type # 18 consecutive sixths between the Altos and Basses in measures 73-80 con:temp_relation, num:position, con:selection:between_the, term, con:and, term, con:where:in, key, num:comp:range context : measures, int-range relation : between X and Y X type : part Y type : part content : melodic sequence condition : interval type #22
flute dotted half note only against strings term, duration:exp, duration, key, ? , con:temp_relation:against, ? (find the string parts?) general_polyphony, pitch, on:where: in_the, term, con:where:in, key, num:int, rule:direction context : parts, instrument type : duration, composite relation : only_against term : part group conditions > not empty ? #29
flute, oboe and bassoon in unison in measures 1-56 term, term, con:and, term. conection:where-condition:in, interval_type, context:where:in, int:comp,range context : measures context : parts, the instruments type : notes condition : same notes #33
semibreve tied to a minim in the Bass clef duration, con:notation:tied:tied_to_a, duration, con:where:in_the, term, key=type context : parts ? or measures ? relation : <a> tied_to <b> a type : duration b type : duration # 44 four eighth notes in the bottom part context : part, relative position relation : sequence
: number <durations,pitches,notes> type : note, conditions: duration # 63 C D E F D E C in semiquavers repeated after a semiquaver context : all relation : X repeated after Y X type : sequence, type: pitch-class X cond : duration Y type : duration # 77 harmonic octave in the bass clef context : measures, clef: type : harmonic interval Notice the assumption in defining the context that bass clef can appear anywhere in the score and it does mean a complete part. # 86 whole-note unison E2 E3 E4 context : all parts type : chord, from notes in all parts condition : pitch content condition : duration #94 crotchet tied to crotchet context : single parts relation : X "tied to" Y X type : duration Y type : duration # 186 whole-note chord context : single part ? all parts ? type : chord
The C@merata task became very demanding this year; however, this approach seems promising. The use of the intermediate information level created space for interpretations and generally allowed operations aimed at language understanding. Natural language was avoided but this approach seems to resemble natural language query patterns. Even if the query language stays in a limited dictionary and syntax, as long as it serves its purpose as an interface for information retrieval, it is worth attention.
The “segmentation ontology” (Fields et al., 2011) is an interesting idea. This work addresses large parts of the current approach’s need for an ontology, it provides implementations in RDF-OWL language for knowledge representations.