<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Musical Meetups: a Knowledge Graph approach for Historical Social Network Analysis</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Alba Morales T</string-name>
          <email>alba.morales-tirado@open.ac.uk</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Jason Carvalho</string-name>
          <email>jason.carvalho@open.ac.uk</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Paul Mulholland</string-name>
          <email>paul.mulholland@open.ac.uk</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Enrico Daga</string-name>
          <email>enrico.daga@open.ac.uk</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Knowledge Media Institute, The Open University</institution>
          ,
          <addr-line>Milton Keynes</addr-line>
          ,
          <country country="UK">United Kingdom</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>The large-scale analysis of historical events data makes it possible to trace key points of cultural and social exchange in history. There has been research focused on facilitating the integration and interpretation of events from heterogeneous sources (such as memoirs, books, and biographies) mainly considering events as a sequence of spatiotemporal objects. However, exploring and discovering new connections (e.g., collaborations, interactions) between people does require characterising those events with dimensions that are relevant to the scholarly enquiry such as the actual participants and nature of the event. This paper describes the concept of historical meetup to represent the encounters (for instance, collaborations, exchanges, links) between personalities of European history and formalise its constituent parts as an ontology. Furthermore, we report on preliminary work undertaken to generate a Knowledge Graph of historical meetups extracted from encyclopedic sources, i.e. biographies collected from Wikipedia. We discuss our results and illustrate the challenges of extracting such type of knowledge from biographical sources. The current experimental setting explores historical meetups in the European musical culture between 1800 and 1945. Our work sketches the basis for applying event knowledge graphs to cultural and social history research, providing support for the analysis, and exchange of ideas and practices.</p>
      </abstract>
      <kwd-group>
        <kwd>Analysis</kwd>
        <kwd>Historical meetups</kwd>
        <kwd>historical social network</kwd>
        <kwd>Knowledge Graph</kwd>
        <kwd>Polifonia</kwd>
        <kwd>MEETUPS Ontology</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        The scholarly study of cultural and social heritage is a dynamic flow of experiences, leaving
heterogeneous traces dificult to capture, connect, access, interpret, and valorise. Social network
analysis (SNA) has gathered the attention of researchers across sociology and computer science
and can be a useful tool for cultural heritage research. By mapping out the connections, facts,
and interactions between individuals, SNA can support the exploration and understanding of
experiences, collaboration patterns and relationships, casting new light on aspects of cultural and
social exchange [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. For instance, social network analysis can be used to study the relationships
between musicians, producers, and composers in the musical world and how these relationships
influence the production and creation of musical pieces.
      </p>
      <p>
        However, the interpretation and analysis of historical encounters using documentary sources
can be a complex task. For example, sources are usually heterogeneous and complex datasets
nEvelop-O
[
        <xref ref-type="bibr" rid="ref1 ref2">1, 2</xref>
        ], such as books, memoirs, and biographies; the extraction of knowledge involves the
implementation of various techniques and methods that typically requires technical expertise
and skills that might not be common among scholars in the cultural heritage domain. Crucially,
the main challenge researchers face is precisely the discovery of meaningful collaborations and
influences among people. Beyond human participants, social interactions involve other entities
such as organisations (e.g., record companies), places (location of the encounter), temporal
relations (when and for how long) and types of events (e.g., festivals or concerts), including
multiple and overlapping historical encounters [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. Clearly, the analysis cannot be dedicated
only to one entity (e.g., people) but extended to cover and complete the overall picture of such
interactions.
      </p>
      <p>In this paper, we introduce the concept of historical meetups to describe historical encounters,
for instance, collaborations and exchanges between personalities in European history. The
MEETUPS Ontology models the constituent elements of a historical meetup, including entities
such as people, places, temporal relations and types of events. The ontology aims to support
researchers in performing a thorough analysis of social networks and discovering potential new
links, therefore, addressing challenges in terms of interpretation and analysis complexity. The
design rationale of the ontology was motivated by considering the knowledge requirements of
scholars exploring social-historical networks and formalised as Competency Questions (CQs)
in Section 4. Furthermore, MEETUPS Ontology is used as the framework to create a Knowledge
Graph (KG) of historical meetups extracted from encyclopedic sources, for instance, Wikipedia
biographies. The experimental setting explores social interactions in the European music world
between 1800 and 1945. We develop our work in the context of the MEETUPS Pilot1 part of the
Polifonia project. Our main contributions are:
• The MEETUPS Ontology2 for representing historical encounters between personalities in</p>
      <p>European history.
• The preliminary work that implements a knowledge extraction pipeline3 for building a</p>
      <p>KG of historical meetups from biographic sources, specifically Wikipedia.
• The first version of the MEETUPS Knowledge Graph 4.</p>
      <p>• A set of SPARQL queries for validation of the MEETUPS Ontology.</p>
      <p>The rest of the paper is organised as follows. Section 2, presents a review of relevant research
on the construction of KG for cultural heritage and SNA of historical meetups crafted from
biographies. Section 3 details the methodology used to build the MEETUPS Ontology. Section 4
gives an account of the knowledge requirements and the development process for the ontology.
Section 5 presents the evaluation of the ontology. Section 6 describes preliminary work to build
a KG of historical meetups, and Section 7 summarises the challenges found in building the KG.
Finally, Section 8 presents the concluding remarks and future work.</p>
      <p>1MEETUPS Pilot: https://polifonia-project.eu/pilots/meetups/
2MEETUPS Ontology repository: https://github.com/polifonia-project/meetups-ontology
3Knowledge extraction pipeline repository: https://github.com/polifonia-project/meetups_pilot
4MEETUPS Knowledge Graph repository: https://github.com/polifonia-project/meetups-knowledge-graph</p>
    </sec>
    <sec id="sec-2">
      <title>2. State of the art</title>
      <p>
        The collection of data about historical encounters makes it possible to trace key points of
cultural exchange, dissemination and interesting connections between people, organisations
and other entities. However, documentary sources (e.g., books, biographies, memoirs) are often
unstructured and dificult to analyse, making it challenging for researchers to gain insights into
the lives of individuals and the historical contexts in which they collaborated. The construction
of knowledge graphs to represent structured data has the potential to shed light on the vast
documentary resources (physical and digital) and provide an overview of events that is both
comprehensive and easy to understand for scholars in the cultural heritage domain [
        <xref ref-type="bibr" rid="ref1 ref4">1, 4</xref>
        ],
importantly supporting social network analysis methods.
      </p>
      <p>
        Initial work in cultural heritage and social network analysis focused on using Linked Data
(LD) and Knowledge Graphs (KG) to study general aspects of artist interaction. For instance, in
the musical domain, research explored how certain musical pieces influenced musical genre
development and aspects of music composition [
        <xref ref-type="bibr" rid="ref5 ref6">5, 6</xref>
        ]. However, the discovery of meaningful
collaborations and cultural exchange among artists requires an analysis that should be expanded
beyond general aspects and incorporate events as an essential component of cultural heritage
research which brings together other entities such as participants, locations, and temporal
relations [
        <xref ref-type="bibr" rid="ref1 ref2 ref3 ref7">7, 3, 1, 2</xref>
        ].
      </p>
      <p>
        By drawing on the concept of historical events, current research expands on the development
of ontological models and KGs for implementing event-centric approaches. For instance, work
by [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] extracts events from a temporal perspective. In contrast, work by [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] prioritise the
analysis of events from the perspective of social interactions by linking biographies of historical
personalities, both studying specific elements of an event. The work presented by [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] aims
to provide users with tools that facilitate the exploration of personalities’ biographies. In
their work, [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] follow a knowledge-based approach to connect other people’s biographies and
places, supporting knowledge discovery. However, their approach is limited to single document
analysis, meaning events only gather one biographee’s events from Finish personalities.
      </p>
      <p>
        In contrast to these studies, our approach focuses on modelling events from the perspective of
musical exchanges, including various entities, to bring a comprehensive view of meetings. Our
approach aims to design a whole process to extract data and build a knowledge base of historical
meetups. Unlike [
        <xref ref-type="bibr" rid="ref8 ref9">8, 9</xref>
        ] that expand their work from existing databases, we use biographies
collected from Wikipedia. Furthermore, we expand the analysis of personalities across Europe
from the 1800s to 1945 instead of being specific to a location, such as the [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] interested in Finish
personalities. Importantly, we list encounters’ categories built in collaboration with musical
scholars, facilitating the study of knowledge exchanges and providing means for discovering
interesting links between personalities.
      </p>
    </sec>
    <sec id="sec-3">
      <title>3. The Historical MEETUPS Ontology</title>
      <p>
        In what follows, we describe the methodology used to develop the MEETUPS Ontology and
present the model for representing historical meetups. We follow ontology engineering good
practices and methodologies [
        <xref ref-type="bibr" rid="ref10 ref11 ref12">10, 11, 12</xref>
        ], and devise the following steps:
• Abstracting the scenario and identifying knowledge requirements. In the first step,
we identify the scope and purpose of the ontology. As described in Section 1, in this paper,
we introduce the concept of historical meetups to represent encounters and collaborations
that influenced personalities of European history. In this work, our experimental setting
explores historical meetups in the musical culture of Europe between 1800 and 1945 and
uses this scenario as a guide for identifying key ontology concepts and relationships
that define our ontology. The knowledge requirements of scholars looking to uncover
such encounters are identified and stated in the form of Competency Questions (CQs).
The CQs will guide the ontology development process, providing the main framework to
evaluate the expressiveness of the ontology.
• Ontology design and construction - implementation. The second step consists of a)
identifying the concepts and relations constituting the ontology, b) using definitions to
express the ontology in a formal language, and c) determining the reuse of established
ontologies and how to integrate them into the MEETUPS Ontology. We built the ontology
using the Protege ontology editor and the Web Ontology Language (OWL).
• Knowledge Graph population. The third step is dedicated to building the MEETUPS
Knowledge Graph (KG). We developed a knowledge extraction pipeline that determines
the entities part of a historical meetup and performs Natural Processing Language (NLP)
tasks to extract and link such entities. We use SPARQL Anything [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ] to generate RDF
data and store the KG for evaluation in the next step.
• Ontology validation. The last step consists of the evaluation of the ontology. The main
aspect to consider is the fulfilment of the CQs. We code the CQs into SPARQL queries and
use them to interrogate the KG built in the previous step. By comparing the query results
and the expected values from the CQs, we expect to obtain validation if the ontology
fulfils the knowledge requirements.
      </p>
      <p>A detailed account of the construction process for the MEETUPS Ontology is given in the
following section.</p>
    </sec>
    <sec id="sec-4">
      <title>4. Implementation of the MEETUPS Ontology</title>
      <p>This section is divided into two parts. The first part introduces the process for identifying
and capturing knowledge requirements. The second part is dedicated to defining the classes,
relationships and ontology construction.</p>
      <sec id="sec-4-1">
        <title>4.1. Knowledge requirements and CQs</title>
        <p>First, we capture the knowledge requirements regarding support for historians and teachers
in the exploration and visualisation of encounters between personalities in European musical
history. The knowledge requirements were collected as part of collaborative work between
scholars and researchers from the music department at The Open University and the Knowledge
Media Institute (KMi); this work is part of the Polifonia project and the MEETUPS pilot.</p>
        <p>Three Persona stories5 were developed to exemplify the knowledge requirements from the
5Persona stories: https://github.com/polifonia-project/stories
perspective of target communities and scholarly users. The stories describe a series of scenarios
regarding users’ scholarly activities, their data needs and interest in terms of music and cultural
heritage, for instance, their requirements to analyse interactions between music personalities
and the discovery of patterns for social exchange. The predominant Persona story for our
experimental works is Ortenz, selected by music scholars. In what follows, we give a brief
description of the Personas.</p>
        <p>• Ortenz: She is a research fellow at the Music Department. Her background is in literature
and art history. Ortenz would like to have a system for visualising events (meetings of
composers and musicians) in time and space in order to track musicians’ careers, their
overlap and intersections, gather trends in time and space and make emerging patterns
of knowledge transmission.
• David: He is a professor and music historian working in the music department of a
university. David is interested in understanding the social history of music, e.g. who
were the musicians, who was the audience, and how a particular musical environment
relate to the wider musical environment.
• Sophia: She is a musicologist and a practising musician. Sophia is interested in
understanding the social-historical reasons behind how music was created and how it sounds.</p>
        <p>We provide typical sentences from the historical domain that exemplify the information
about personalities and events of interest for scholars. In this case, we take textual excerpts
from Edward Elgar’s6 biography:</p>
        <p>These examples are interesting for several reasons: (a) they contain details of events that
influenced Elgard’s work: he attended other personalities’ performances (e.g., Saint-Saëns , he
was inspired by contemporaneous personalities (e.g., Mary Lygon); (2) it contains explicit time
references such as 1902 and 1880 which serve as guidance to search for other musicians that
were working at the same time; (3) direct connections to other music personalities with whom
he interacted; (4) and places he visited. For example, it is well accepted that European musicians
influenced Elgard’s work; however, questions such as was he influenced by their surroundings
(e.g., places he lived in, people he met, and shows he attended, what type of events)? Was his
work part of emerging trends or patterns, when this happened? who were the contemporaneous
emerging personalities? are of interest to music scholars.</p>
        <p>Based on the description of the Personas’ stories, a list of CQs was generated. The list of
relevant CQs is enumerated below:</p>
        <p>6Edward Elgar biography, source Wikipedia: https://en.wikipedia.org/wiki/Edward_Elgar</p>
        <sec id="sec-4-1-1">
          <title>David</title>
        </sec>
        <sec id="sec-4-1-2">
          <title>Sophia</title>
          <p>Sophia
Ortenz
Ortenz
Ortenz
Ortenz
Ortenz
Ortenz
Ortenz
Ortenz
Ortenz
Ortenz</p>
        </sec>
        <sec id="sec-4-1-3">
          <title>Ortenz Table 1: List of Competency Questions (CQs).</title>
          <p>From this collaborative requirements acquisition exercise, it is clear that researchers expect
to explore encounters that could potentially reveal unexpected connections and relationships.
There is also a need for tools that support the analysis of social interactions from the perspective
of ”events”, including layers such as time, places and participants. Therefore, the concept
of historical meetups was introduced to describe the study of intersections and links among
personalities from the perspective of the event and taking into account other dimensions, such
as participants, place and time expressions.</p>
          <p>The meetup type describes the reason/purpose for an encounter. According to scholars
and researchers of the Music Department at The Open University, meetups can be grouped
as business and career (e.g., contract, retirement), personal life (e.g., born, divorce, family),
coincidence (e.g., meet, find, discover), education (e.g., learn, teach, conservatoire), public
celebration or music-making (e.g., play, produce). Other dimensions of analysis include the
participants involved, for instance, the person that is the subject of interest and the people
interacting in the encounter, the place where it took place (e.g., city, country, venue), and the
date when it took place. There could also be events that do not contain all the entities described
previously.</p>
        </sec>
      </sec>
      <sec id="sec-4-2">
        <title>4.2. Ontology construction</title>
        <p>From this analysis, we identify the vocabulary, relationships, and properties of the entities of
interest. Therefore, this section is dedicated to describing the core components of the MEETUPS</p>
        <p>
          Ontology. Furthermore, we explain how we incorporated established ontologies such as Time
Ontology [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ], PROV Ontology [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ] and SEM (Simple Event Model) [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ] in our design.
        </p>
        <p>The preferred prefix for MEETUPS Ontology is meetups: and the namespace for all MEETUPS
terms is https://w3id.org/polifonia/ontology/meetups-ontology#. The core class of
MEETUPS ontology is meetups:Meetup (see Figure 1), which is the representation of a historical
meetup between people in the musical world. The main classes of the ontology are Purpose,
Participant, Place and TimeExpression.</p>
        <p>A historical meetup is defined by the purpose of the encounter, a meetups:Purpose class,
which could be classified into one of the types of encounters: :BusinessCareer, :PersonalLife,
:Coincidence, :Education, :PublicCelebration or :MusicMaking. A meetup involves at
least one person (a meetups:Subject) and one or more participants represented by the class
meetups: Participant that could take a specific role ( meetups:hasRole) in the musical
encounter (meetups:Role). Historical meetups typically include a reference to the place
(meetups:Place) where people or the activity developed and the date (meetups:TimeExpression)
that indicates when this encounter took place.</p>
        <p>Moreover, the ontology design includes classes for recording the provenance of the sources
(see Figure 2). As described previously, typically data can be collected from encyclopedic
resources such as biographies, and open public web pages; these sources are represented with
the class meetups:Sources. At the same time, a source can hasProvider, represented by
meetups:Provider class; for example, in our experiments, we use Wikipedia.</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>5. Evaluation of the ontology</title>
      <p>As described in Section 1, this work aims to represent historical encounters between
personalities in European history. We built the MEETUPS Ontology following the list of knowledge
requirements collected in Section 4. We use a sample of randomly selected biographies to build
the KG and provide data for evaluation.</p>
      <p>We used Protégé and the Hermit reasoner to check the formal consistency of the ontology.
In order to evaluate the expressivity of the ontology, we grouped the Competency Questions
(CQs) listed in Section 4 according to the most relevant type of entity it questions about and
then encoded them into SPARQL queries7.</p>
      <p>CQs (1 to 5) focus on place entities. These CQs inquire particularly about the locations
where people met or visited. We take the example of the English composer Edward Elgar and
build two queries (a) one that retrieves all the places he visited during his life, (b) one that
retrieves all the places where he had an encounter with Adolf Pollitzer. As shown in Figures 3a
and 3b, the ontology support such queries.</p>
      <p>(a) Places visited by a given artist (b) Places linked to two artist encounters
Figure 3: SPARQL queries for CQs focused on place entities</p>
      <p>CQs (6 and 7) focus on participant entities. These competency questions refer to finding
other contemporary artists and what were their social networks. We follow the example of
Edward Elgar and built two queries (a) one that retrieves all contemporaneous artists given a
specific date (see Figure 4a) and (b) a query that retrieves all people the artist met during their
life and their role (see Figure 4b).</p>
      <p>CQs (8 to 9) focus on time expression entities. These competency questions inquire
about the time when artists met. We exemplify how the ontology answers this question by
7All available in the repository.</p>
      <p>(a) People working at the same time
Figure 4: SPARQL queries for CQs focus on participant
(b) People met during their life
building a query (see Figure 5a) that retrieves the date specific personalities met.</p>
      <p>CQs (10 to 18) focus on purpose/events entities. Finally, these CQs inquired about
what was the main purpose of the meeting. For instance, CQ 11 asks for the places where
an artist performed (e.g., the meetup type is :MusicMaking), or CQ 12 questions where they
live (PersonalLife), questions that can be answered thanks to the list of categories of events
gathered as part of the knowledge requirements task (see Section 4). In the example (see
Figure 5b), we query the list of meetups and their type.</p>
      <p>(a) Date at which personalities met
Figure 5: SPARQL queries for CQs 10 to 18
(b) List of events by type</p>
    </sec>
    <sec id="sec-6">
      <title>6. Preliminary extraction pipeline and Knowledge Graph generation</title>
      <p>This section describes the knowledge extraction pipeline developed to build the MEETUPS
Knowledge Graph (see Figure 6). As described in Section 1 our work focuses on facilitating the
discovery of connections between personalities of European history. Therefore, our approach
is focused on identifying the entities that are part of a historical meetup: people, places, time
expressions and meetup type. The entity recognition and disambiguation process is performed at
the sentence level. We start by collecting data that provides an overview of music personalities’
lives, such as biographies in text format. Next, we process the corpus to link entities such as
places and people, extracting time expressions and classifying them according to meetup types.
The output of this step is a bag of entities at the sentence level, grouped by paragraphs and
sections. Once the entities are extracted, we run the process to find encounters within the
text, at sentence level or on a larger chunk of text using coreference resolution. The last step
is dedicated to building the knowledge graph of historical meetups using as a framework the
MEETUPS Ontology and the data output from the previous step. In what follows, we describe
in detail the steps taken towards constructing the KG.</p>
      <p>Data collection. The first step of the pipeline is dedicated to the collection of data that
supports music historians’ exploration of encounters between people in the musical world
in Europe between the 1800s and 1945. Therefore, we rely on documentary sources such as
biographies and select an open-access database such as Wikipedia as the primary data input.
To obtain the list of targeted personalities, we built and queried the DBpedia SPARQL endpoint.
We filtered biographies by date of birth in order to reflect our targeted sample and gathered a
total of 33,309 biographies. Each biography was collected in text format from Wikipedia, using
the identifier from DBpedia. In order to make experiments manageable, we randomly sampled
a total of 1000 biographies that will be inspected in the following steps.</p>
      <p>Cleaning &amp; Segmentation. In the previous step, the biographies were collected in text
format; therefore, the second step is dedicated to preparing the dataset to execute the
NamedEntity Recognition and Disambiguation (NERD) tasks. First, we clean the text by removing line
breaks, special characters and empty spaces. In order to facilitate the location and classification
of the entities, the text is organised in sentences, grouped by paragraphs and sections. The
ifnal output is a corpus of biographies, each one containing a set of sentences identifiable by its
position within paragraphs and sections.</p>
      <p>
        Entity recognition. As explained previously, the main objective is to explore encounters by
characterising them with dimensions that are relevant to scholars, specifically: people, places,
temporal expressions and types of encounters. Therefore, we focus on locating and identifying
the constituent parts of a historical meetup within sentences. In what follows, we detail the
tasks developed to recognise each type of entity:
• Named-entity recognition and disambiguation (NERD) for people and place entities. People
and places are two main elements that characterise a historical meetup. As listed in Table 1,
knowing who was involved and where encounters happened is required. Therefore, this
task is focused on determining people or places mentioned in the corpus and specifying
them uniquely using a targeted knowledge base. We make use of DBpedia Spotlight8
tool to automatically annotate mentions of people and places linking them to DBpedia
resources.
8DBpedia-Spotlight: https://www.dbpedia-spotlight.org/
• Identification of temporal expressions. Another important requirement has to do with
the date or moment in time when a particular encounter happened. To extract this
information, this task uses a rule-based Time Expression recognition tagger based on
research by [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ] and SynTime software 9. The authors implement a three-layer system
that recognises time expressions using syntactic token types and general heuristic rules.
Unlike SynTime, our software uses NLTK Toolkit and was developed using Python; also,
the heuristic rules were revised and expanded for our experiments. Furthermore, this
component classifies time expressions into three types of expressions: time range (e.g.,
from XX to XX, from XX, to XX, until XX), time point (e.g., exact date, 23/03/1294), and
time reference (e.g., usually incomplete dates (19 April), two weeks, later this year).
      </p>
      <p>
        Identification of meetup themes. The main element of a historical meetup is the theme, in
other words, the topic/subject of the encounter. This task uses Machine Learning techniques and
performs a semi-supervised classification process to annotate the meetup type (see Section 4.1)
of each sentence in the corpus. In what follows, we describe the steps of the theme identification
process in detail:
• Building the training dataset. In order to create the training dataset, we use a distant
supervision approach. First, we built a curated list of keywords for each one of the meetup
types; the list was developed with the advice of domain experts. Next, we randomly
select a sample of sentences and annotate them automatically with one of the classes
(meetup types) if it includes one of the keywords in the list. The final output is a dataset
of sentences (a total of 16140) annotated according to one of the meetup types (see Table 2
for statistics).
• Classification task - training and testing of Machine Learning (ML) algorithms. In this step,
we use the dataset built previously and implement a supervised classification approach.
As shown in Table 2, the distribution of sentences by class is unequal; therefore, the
ifrst task is building a balanced dataset. We take as reference the lowest number of
sentences per group, in this case, the class Education with 629 sentences. For each class,
we select randomly 629 annotated sentences to ensure each class has the same number of
annotations. The new training dataset has a total of 3774 annotated sentences. Then, we
prepare the dataset for the training process; we use SBERT to capture the context of the
sentences [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ] and as input for the ML training. Next, we randomly divide the dataset
into training and testing sets (80-20), both with the same proportion of classes. We train
a Multilayer Perceptron (MLP) classifier (sklearn Python library), commonly used for
natural language processing tasks, as it handles a large number of input features. We
manually check a sample of 50 sentences and confirm results are suficiently accurate (we
leave a formal evaluation to future work). Finally, we apply the classifier to the entire
corpus; the final output has a total of 73800 sentences annotated according to the meetup
type.
      </p>
      <p>The final output is a sentence-level bag of semantic entities: meetup types, people, places and
temporal expressions. We aim to use this new corpus to build a knowledge graph of historical
meetups.</p>
      <p>9SynTime software: https://github.com/zhongxiaoshi/syntime</p>
      <p>Knowledge Graph construction. In this last step, we focus our eforts on building a
knowledge graph of historical MEETUPS, using the entities found at a sentence level. Crucially,
we aim to identify missing elements at the sentence level and annotate them for resolution in the
next version of the KG, for instance, implementing a coreference resolution task to determine if
missing elements are located in previous sentences.</p>
      <p>
        We built the KG following the MEETUPS Ontology model. The entities found at the sentence
level represent a meetup candidate. The KG also stores provenance data (e.g., source, type of
source, location identifier in the text). The KG was constructed from CSV files resulting from
the process described so far, using SPARQL Anything [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ] and is currently published in Turtle
RDF format.
      </p>
      <p>The MEETUPS KG contains data from 1000 artists’ biographies. As summarised in Table 3,
around 74445 historical meetups, there are 51425 mentions of people involved in diferent
encounters. So far, the historical meetups gather around 5595 places and 79838 time expressions.</p>
    </sec>
    <sec id="sec-7">
      <title>7. Challenges of KG generation and discussion</title>
      <p>The evaluation of the MEETUPS knowledge graph has been made incrementally to ensure the
data extraction process complies with the knowledge requirements (see Section 4). Here we
present the input data validation using the first version of the KG. The data covers people, place,
time expressions and meetup types. All these entities were extracted at the sentence level. We
designed and executed a short survey as follows:
• Select randomly the sentences from musicians’ biographies for evaluation.
– The evaluation dataset was built from a random sample of 12 biographies from the
total analysed in the first iteration (1000 biographies).
– Due to time and resource constraints regarding the annotation and evaluation of
the sentences, we selected one paragraph and its corresponding sentences from
each biography. Some paragraphs contained only one sentence, others up to seven
sentences, reaching a total of 40 sentences. Each sentence listed the total entities
resulting from the knowledge extraction pipeline.
• For the data collection process:
– For each sentence, we asked participants to analyse the correctness of the
automatically identified entities in terms of the number of sentences identified as meetups,
and the number of people, places, time expressions and meetup types correctly
extracted.
– For each sentence, we asked participants to indicate the number of missing entities
not identified by the automatic annotation process.</p>
      <p>From the forty sentences, 34 (85%) were identified as potential meetups/encounters. Table 4
displays in detail the number of sentences that mention a type of entity. Of the 34 sentences
identified as a meetup, nine sentences (26%) provide information about all four types of entities.
Twenty-five sentences (74%) lack one or two of any of the types of entities.</p>
      <p>We move now to analysing the entities extracted at the sentence level. First, regarding the
completeness of sentences to represent a historical meetup:
• Results show that 26% of sentences (1 out of 4 sentences) are comprehensive (including
all four types of entities), and 74% mention at least one out of the four types of entities.
Therefore, it is possible that a historical meetup is not fully identifiable from one sentence.
For example, the sentence: “Born Al Albertini in Chester, Pennsylvania, United States, he
went to South Philadelphia High School.” does not include a time entity but still provides
data to answer the CQs. Future work regarding the completeness of historical meetups
should explore cases of missing entities, such as searching for information in preceding
sentences at paragraph and section levels.</p>
      <p>Second, regarding the quality of the extracted entities:
• As mentioned in the previous point, sentences representing historical meetups include
one or all four types of entities. Of the total of sentences analysed, 62% of them mention
people and 38% of sentences do not (see Table 5). From the sentences that refer to people,
the knowledge extraction process correctly identifies them in 62% of the sentences and
missed people entities in 19% of sentences. People were incorrectly and partially identified
in 19% of the sentences.</p>
      <p>– The cases in which people were incorrectly identified mainly pertain to homonyms
names. For example, people’s names are used to name prizes, and festivals, among
other cases. Further validation includes querying DBpedia/Wikidata to determine
if the entity is a person (for example, has a date of birth) and the entity type (for
example, if it refers to a person or musician).
– In cases where people entities were partially identified, we found most cases referred
to entities named in a previous sentence or at the beginning of the paragraph. The
next steps include exploring neighbouring sentences to identify such entities and
implementing a coreference resolution task.
• From sentences that mention places, the knowledge extraction process correctly identifies
place entities in 87% of the sentences, and only 9% of sentences were missed. Place entities
were partially and incorrectly identified in 4% of the sentences. For the few cases in which
places were missed, incorrectly and partially identified, the analysis shows the definition
of places should be expanded. For instance, ’Conservatoire de Paris’ can be identified as
being in ’Paris’; this entity is of type dbo:Campus not dbo:Place. Future work should
consider the level of detail required to recognise place entities.
• In both cases, people and place entities, a few instances were missing in DBpedia; for
these cases it is essential to consider other databases such as Wikidata to link resources.
• From sentences that include references to time, the knowledge extraction process correctly
identifies them in 42% of the sentences and 58% of sentences were missing. The analysis
shows that in most cases, time expressions such as years (e.g., in 1923, 1899-1900), relative
expressions (e.g., mid-eighties, late 1960) or indirect references (e.g., two months later, last
year) are missing. The time expression identification process should expand the heuristic
rules in the next iteration.
• Regarding the meetup type identification, all sentences are annotated with one of the
classes identified in Section 4.1. The knowledge extraction process correctly identifies the
correct type in 74% of the sentences. The analysis shows that in most cases, the meetup
type is partially correct or incorrect when one sentence refers to more than one event
in the same sentence. The time expression identification process should consider these
cases in the next iteration.</p>
    </sec>
    <sec id="sec-8">
      <title>8. Conclusions and Future work</title>
      <p>In this paper, we presented the MEETUPS Ontology, a model that formally characterises the
encounters between personalities of European history. We introduced the concept of the
historical meetup to describe such encounters, for instance, collaborations and knowledge
exchanges between people. The ontology describes the elements of a meetup, including entities
such as participants, locations, time expressions and meetup types. We presented our approach
as an essential tool to address the knowledge requirements of musical scholars and cultural
heritage researchers who face challenges when analysing artists’ social links and influences.
Importantly, in collaboration with experts, we compiled a list of knowledge requirements
expressed as Competency Questions (CQs). These CQs later served to validate the expressivity
of the ontology.</p>
      <p>Furthermore, we presented the preliminary work on developing the Knowledge Extraction
pipeline for identifying the elements of historical meetups. The experiments were developed in
the music cultural heritage domain context and used as data source biographies collected from
Wikipedia. The execution of the pipeline produced the first version of the MEETUPS Knowledge
Graph, crucially providing an overview of the main challenges to tackle to extract and represent
the constituent elements of a historical meetup. Therefore, future work includes addressing the
challenges summarised in Section 7 and the data quality evaluation of the knowledge graph.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>J.</given-names>
            <surname>Tuominen</surname>
          </string-name>
          , E. Hyvönen,
          <string-name>
            <given-names>P.</given-names>
            <surname>Leskinen</surname>
          </string-name>
          ,
          <string-name>
            <surname>Bio</surname>
            <given-names>CRM</given-names>
          </string-name>
          :
          <article-title>A Data Model for Representing Biographical Data for Prosopographical Research (</article-title>
          <year>2018</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>E.</given-names>
            <surname>Hyvönen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Leskinen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Tamper</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Rantala</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Ikkala</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Tuominen</surname>
          </string-name>
          , K. Keravuori, BiographySampo - Publishing
          <source>and Enriching Biographies on the Semantic Web for Digital Humanities Research, Lecture Notes in Computer Science</source>
          ,
          <year>2019</year>
          , p.
          <fpage>574</fpage>
          -
          <lpage>589</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>S.</given-names>
            <surname>Gottschalk</surname>
          </string-name>
          , E. Demidova, EventKG:
          <string-name>
            <given-names>A Multilingual</given-names>
            <surname>Event-Centric Temporal Knowledge Graph</surname>
          </string-name>
          (
          <year>2018</year>
          ). URL: http://arxiv.org/abs/
          <year>1804</year>
          .04526, arXiv:
          <year>1804</year>
          .04526 [cs].
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>E.</given-names>
            <surname>Hyvönen</surname>
          </string-name>
          ,
          <article-title>Using the semantic web in digital humanities: Shift from data publishing to data-analysis and serendipitous knowledge discovery, Semantic Web (</article-title>
          <year>2020</year>
          )
          <fpage>187</fpage>
          -
          <lpage>193</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>N. J.</given-names>
            <surname>Bryan</surname>
          </string-name>
          ,
          <string-name>
            <surname>G.</surname>
          </string-name>
          <article-title>Wang, Musical Influence Network Analysis and Rank of Sample-Based Music</article-title>
          , in: ISMIR,
          <year>2011</year>
          , pp.
          <fpage>329</fpage>
          -
          <lpage>334</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>M.</given-names>
            <surname>Kopel</surname>
          </string-name>
          ,
          <source>Analyzing Music Metadata on Artist Influence, in: Intelligent Information and Database Systems</source>
          , Springer International Publishing,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>A.</given-names>
            <surname>Fokkens</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Legêne</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Schreiber</surname>
          </string-name>
          , V. de Boer,
          <article-title>Biographynet: Extracting relations between people and events (</article-title>
          <year>2018</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>S.</given-names>
            <surname>Menini</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Sprugnoli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Moretti</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Bignotti</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Tonelli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Lepri</surname>
          </string-name>
          , RAMBLE ON:
          <article-title>Tracing Movements of Popular Historical Figures</article-title>
          ,
          <source>in: Proceedings of the 15th Conference of the European Association for Computational Linguistics</source>
          ,
          <string-name>
            <surname>ACL</surname>
          </string-name>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>P.</given-names>
            <surname>Beytía</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Schobin</surname>
          </string-name>
          , Networked Pantheon:
          <article-title>a Relational Database of Globally Famous People (</article-title>
          <year>2018</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>M.</given-names>
            <surname>Uschold</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Gruninger</surname>
          </string-name>
          ,
          <article-title>Ontologies: principles, methods and applications</article-title>
          ,
          <source>The Knowledge Engineering Review</source>
          <volume>11</volume>
          (
          <year>1996</year>
          )
          <fpage>93</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>V.</given-names>
            <surname>Presutti</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Daga</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Gangemi</surname>
          </string-name>
          , E. Blomqvist,
          <article-title>eXtreme Design with Content Ontology Design Patterns</article-title>
          ,
          <source>Proceedings Workshop on Ontology Patterns</source>
          (
          <year>2009</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>M. C.</given-names>
            <surname>Suárez-Figueroa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Gómez-Pérez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Fernández-López</surname>
          </string-name>
          ,
          <article-title>The NeOn Methodology for Ontology Engineering</article-title>
          , Springer Berlin Heidelberg, Berlin, Heidelberg,
          <year>2012</year>
          , p.
          <fpage>9</fpage>
          -
          <lpage>34</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>E.</given-names>
            <surname>Daga</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Asprino</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Mulholland</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Gangemi</surname>
          </string-name>
          ,
          <string-name>
            <surname>Facade-X</surname>
          </string-name>
          :
          <article-title>An Opinionated Approach to SPARQL Anything</article-title>
          , IOS Press,
          <year>2021</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>J. R.</given-names>
            <surname>Hobbs</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Feng</surname>
          </string-name>
          , Time ontology in OWL,
          <year>2006</year>
          . URL: https://www.w3.org/TR/owl-time/.
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>T.</given-names>
            <surname>Lebo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Sahoo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>McGuinness</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Belhajjame</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Cheney</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Corsar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Garijo</surname>
          </string-name>
          , S. SoilandReyes, S. Zednik,
          <string-name>
            <given-names>J.</given-names>
            <surname>Zhao</surname>
          </string-name>
          ,
          <string-name>
            <surname>PROV-O:The</surname>
            <given-names>PROV</given-names>
          </string-name>
          ontology (
          <year>2013</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <surname>W. R. van Hage</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          <string-name>
            <surname>Malaisé</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Segers</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          <string-name>
            <surname>Hollink</surname>
          </string-name>
          , G. Schreiber,
          <article-title>Design and use of the Simple Event Model (SEM)</article-title>
          ,
          <source>Journal of Web Semantics</source>
          <volume>9</volume>
          (
          <year>2011</year>
          )
          <fpage>128</fpage>
          -
          <lpage>136</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>X.</given-names>
            <surname>Zhong</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Sun</surname>
          </string-name>
          , E. Cambria,
          <article-title>Time Expression Analysis and Recognition Using Syntactic Token Types</article-title>
          and
          <article-title>General Heuristic Rules</article-title>
          ,
          <source>in: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics</source>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>N.</given-names>
            <surname>Reimers</surname>
          </string-name>
          ,
          <string-name>
            <surname>I. Gurevych</surname>
          </string-name>
          , Sentence-BERT:
          <article-title>Sentence Embeddings using Siamese BERTNetworks,</article-title>
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>