The aim of this tutorial is first to provide in-depth explanations of DOREMUS, a model for describing music metadata. We will demonstrate how real data coming from musical libraries can be converted to this model by presenting the whole DOREMUS tools chain. We will illustrate how the DOREMUS data can be used for query answering and consumed through various applications including an exploratory search engine and music recommender systems.
• Information systems → Ontologies; Recommender systems; Semantic web description languages; Music retrieval;
Music information can be very complex. Describing a classical masterpiece in all its form (the composition, the score, the various publications, a performance, a recording, the derivative works, etc.) is a complex activity. An even more challenging task consists in describing jazz and ethnic music for which the performance plays a central role, the music is generally not written and the authorship is not well defined. In the context of the DOREMUS research project 1, we develop tools and methods to manage music catalogues on the web using semantic web technologies.
In this tutorial, we show strategies and tools for managing music knowledge. In the Section 2, we present the DOREMUS model for describing music, together with music specific controlled vocabularies. In the Section 3, we present tools for converting music datasets, taking as example the ones coming from the rich musical archives of three leading cultural institutions in France – the Bibliothèque Nationale de France (BnF), the Philharmonie de Paris (PP) and Radio France (RF) – describing musical works, publications, performances and concerts. We demonstrate the expressiveness of the model showing how complex music-specific queries can be answered. Finally, we describe strategies for data visualisation and recommendation in the Section 4.
Among the music ontologies, the most known example is the
Music Ontology [
The DOREMUS model2 is an extension of FRBRoo, for describing
cultural objects [
On top of the FRBRoo original classes and properties, specific
ones have been added in order to describe aspects of a work that
are specifically related to music, such as the musical key, the genre,
the tempo, the medium of performance (MoP), etc. [
Each triplet contains an information that, at the same time, can live autonomously and be linked to the other entities. Thinking about a classic work, we will have a triplet for the composition, one for any performance event, one for every manifestation (i.e. the score), etc., all connected in the graph. A jazz improvisation that consists in an extemporaneous creation of a new work, will have only the triplet for the Performance Work, Performance Expression and Performance Creation, in absence of the moment of composition and writing of the score that are almost mandatory for classical music and without the need to be attached to any other entity. It is considered a work per se. All the Work entities of each triplet are then connected to a Complex Work, a class that has the objective of collecting together all the representations — both the conceptual and sensory ones (manifestation) — of the same creative idea.
The result is a model that, if on one side is quite complex and hard to adopt, on the other hand has a very detailed expressiveness. The graph depicted inFigure 1 shows a real example from our data: Beethoven’s Sonata for piano and cello n.13. 2.2
A large number of properties that are involved in the music
description are supposed to contain values that are shared among
diferent entities: diferent composition can have as genre “sonata”,
diferent performer can play a “bassoon”, diferent authors can have
as function “composer” or “lyricist”. These labels can be expressed
in multiple languages or in alternative forms (i.e. “sax” and
“saxophone”, or the French keys "Do majeur" and "Ut majeur"), making
2http://data.doremus.org/ontology/
3http://data.doremus.org/expression/614925f2-1da7-39c1-8fb7-4866b1d39fc7
reconciliation hard. Our choice is to use controlled vocabularies
for those common concepts. A controlled vocabulary is a thematic
thesaurus of entities, each one being again identified with a URI. We
are using SKOS [
Diferent kinds of vocabularies are required for describing music. Some of them are already available on the web: this is the case of MIMO4 for describing musical instruments, or RAMEAU5 for musical genres, ethnic groups, etc. Some others are not published in a suitable format for the Web of Data, or the version published is not as complete as other formats that are available to libraries or in online sources: this happens with the vocabularies published by the International Association of Music Libraries (IAML), 6 that have been published after the start of the project and for which we sometimes provide more details (labels, languages, etc.). Finally, there is also the case of vocabularies that do not exist at all and that we generate on the base of real data coming from the partners, enriched by an editorial process that involved also librarians. As 4http://www.mimo-db.eu/ 5http://rameau.bnf.fr/ 6http://iflastandards.info/ns/unimarc/ a result, we collected, implemented and published 15 controlled vocabularies belonging to 6 diferent categories 7. 3
Both the French National Library (BnF) and Philharmonie of Paris
make use of the MARC format for representing the music metadata.
The flat structure of MARC, which consists in a succession of fields
and subfields (Figure 2), reflects the purpose of converting printed
or handwritten records in a computer form. Although MARC is a
standard, its adoption is restricted to the library world, making its
serialization to other formats (usually XML) a need for an actual
use. MARC fields are also not labeled explicitly, but encoded with
numbers, with the consequence of having to use a manual for
deciphering the content. The semantics of these fields and subfields is
not trivial: a subfield can change its meaning depending on the field,
under which it is found, and on the particular variant of MARC
(UNIMARC and INTERMARC). A field or subfield can contain
information about diferent entities, like the first performance and the
ifrst publication combined in the same field of the notes, without a
clear separation. Often, the information is represented in the form
of a free text [
The benefits of moving from MARC to an RDF-based solution
consist in the interoperability and the integration among libraries
and with third party actors, with the possibility of realizing smart
federated search [
For the conversion task, we rely on marc2rdf, 8 an open source
prototype we developed for the automatic conversion of MARC
bibliographic records to RDF using the DOREMUS ontology [
The converter is composed of diferent modules, that works in succession. First, a file parser reads the MARC file and makes the content accessible by field and subfield number. We implemented a converting module for both the INTERMARC and UNIMARC variants. Then, it builds the RDF graph reading the fields and assigning their content to the DOREMUS property suggested in the transfer rules.
Then the free-text interpreter extracts further information from the plain text fields, that includes editorial notes.This amounts to do a knowledge-aware parsing, since we search in the string exactly the information we want to instantiate from the model (i.e. the MoP from the casting notes, or the date and the publisher from the first publication note). The parsing is realized through empirically defined regular expression, that are going to be supported by Named Entity Recognition techniques as a future work. Finally, the string2vocabulary component performs an automatic mapping of string literals to URIs coming from controlled vocabularies. All variants for a concept label are considered in order to deal with potential diferences in naming terms. As additional feature, this component is able to recognise and correct some noise that is present in the source MARC file: this is the case of musical keys declared as genre, or fields for the opus number that contain actually a catalog number and vice-versa. These cases and other typos and mistakes have been identified thanks to the conversion process and the visualization of the converted data, supporting the source institution in they work of updating and correcting constantly their data. 3.2
Apart from MARC, we are converting other source bases (in XML), that are too specific to be handled by a single converter. Therefore, we developed ad hoc software that have a generic workflow: parse the input file and collect the required information, create the graph structure in RDF, run the string2vocabulary module described previously. This procedure creates diferent graphs, one for each source. Those source databases are complementary but also contain overlaps (e.g. two databases that describe the same work or the same performance with complementary metadata). We have started to automatically interlink the datasets, so that the resulting knowledge graph provides a richer description of each work. 8https://github.com/DOREMUS-ANR/marc2rdf Before the beginning of the project, a list of questions have been collected from experts of the partner institutions9. These questions reflect real needs of the institutions and reveal problems that they face daily in the task of selecting information from the database (e.g. concert organisation or broadcast programming) or for supporting librarian and musicologist studies. They can be related to practical use cases (the search of all the scores that suit a particular formation), to musicologist topics (the music of a certain region in a particular historical period), to interesting stats (the works usually performed or published together), or to curious connections between works, performances or artists. Most of the questions are very specific and complex, so that it is very hard to find their answer by simply querying the search engines currently available on the web. We have grouped these questions in categories, according to the DOREMUS classes involved in the question.
Table 1 provides an overview of how many queries we can currently write for each category. The implementation of recordings, scores, performance that is still work in progress – along with the interconnection to the LOD repositories – is one important reason for which some questions have not yet been translated into SPARQL and other ones have not results. 9https://github.com/DOREMUS-ANR/knowledge-base/tree/master/query-examples
We consider exploration and recommendation as two sides of the same medal. With the first one, we let the user browse the datasets, discover connections on his own, understand how we build the knowledge. Through recommendation, we remove this responsibility to the user with the purpose of presenting what he needs in a particular moment.
We developed the first version of Overture, a web prototype of an exploratory search engine for DOREMUS data. The application makes requests directly to our SPARQL endpoint10 and provides the information in a nice user interface.
At the top of the user interface, the navigation bar allows the user to navigate between the main concepts of the DOREMUS model: expression, performance, score, recording, artist. The challenge is in giving to the final user a complete vision on the data of each class and letting him/her understand how they are connected to each other. We keep as example Beethoven’s Sonata for piano and cello n.1 11. Aside from the diferent versions of the title, the composer and a textual description, the page provides details on the information we have about the work, like the musical key, the genres, the intended MoP, the opus number. When these values come from a controlled vocabulary, a link is presented in order to search for expressions that share the same value (for example, the same genre or the same musical key). A timeline shows the most important events related to the work (the composition, the premiere, the first publication). Other performances and publications can be represented below. The background is a portrait of the composer that comes from DBpedia. It is retrieved thanks to the presence in the DOREMUS database of owl:sameAs links. These links comes in part from the International Standard Name Identifier (ISNI) service 12, in part thanks to an interlinking realised by matching the artist name, birth and death date in the diferent datasets.
What should we suggest to a user listening Beethoven? Similar
musicians should share with the German composer some features: the
period, similar properties on the compositions (genre, key, casting)
or similar instrument played (the piano itself, or also the
harpsichord that is in the same family). But how to define a similarity
measure that take into acount these concepts? We propose a
solution based on graph embeddings generated at diferent levels:
(1) For simple features (e.g. genre, key, instrument), we
compute for each term an embedding applying node2vec [
Using graph embeddings reduces the similarity problem as the
reverse of an euclidean distance. if some properties are missing, we
apply a penalisation computed as percentage of missing feature in
the target vector with respect to the seed one [
The biggest advantage of this method is that the embeddings computation is required only for the simple features: each embedding is re-used in diferent combination. Because diferent weights can be assigned to each property in order to tune up the recommendation, we plan to experiment with neural networks in order to discover the best weighting strategy.
This work has been partially supported by the French National Research Agency (ANR) within the DOREMUS Project, under grant number ANR-14-CE24-0020.