Wikipedia is a rich data source for knowledge from all domains. As part of this knowledge, historical and daily events (news) are collected for different languages on special pages and in event portals. As only a small amount of events is available in structured form in DBpedia, we extract these events with a rule-based approach from Wikipedia pages. In this paper we focus on three aspects: (1) extending our prior method for extracting events for a daily granularity, (2) the automatic classification of events and (3) finding relationships between events. As a result, we have extracted a data set of about 170,000 events covering different languages and granularities. On the basis of one language set, we have automatically built categories for about 70% of the events of another language set. For nearly every event, we have been able to find related events.
Wikipedia is an extensive resource for different types of events like historical events
or news that are user-contributed and quality-proven. Although there is plenty of
information on historical events in Wikipedia, only a small fraction of these events is
available in a structured form in DBpedia. In prior work we have focused on
extracting and publishing these events for the use in the semantic web and other
applications [
The remainder of this paper is as follows: Section 2 presents related work. In Section 3, we address the question on how events can be detected, extracted, processed and presented in different forms for the semantic web (Workshop questions 1, 2 and 3). In Section 4 we present an approach on how events can be automatically classified with categories (Question 1). In Section 5 we show how relationships between events from different languages and granularities can be found (Question 1). 2
There is a range of systems specialized for the extraction of events and temporal
relations from free text. The TARSQI toolkit [
Some work has been done for the extraction of events from Wikipedia articles with
machine learning or rule-based approaches and the presentation for the end user in
user interfaces with timelines and maps. The approach of Bhole [
Other work concentrates on the extension of knowledge bases like DBpedia [
There is a collection of ontologies for the modeling of events in RDF like
EVENT1, LODE [
However, most related work in this field is about the extraction of events from free text or knowledge bases like Wikipedia or YAGO and the enrichment of entities from text or knowledge bases with temporal information. Not much work has been done on 1 http://motools.sourceforge.net/event/event.html 2 http://www.iptc.org/EventsML/ the further enrichment of event datasets such as adding relations or additional information like categorizations. 3
Wikipedia is a rich data source for events of different topics, languages and
granularity. Most research focuses on the extraction of events from the full text of
Wikipedia articles and on relating it to the appropriate entities. Major historical events
have their own article, or events are collected in articles for a special topic. Events are
also collected in time units of different granularity (i.e. years or months) available for
different languages. These articles contain lists with events, whose structure is
relatively stable. In prior work we have focused on the extraction of events from
yearbased articles, which include information on individual years for different languages
[
The resulting data set is then further processed. For the automatic classification see Section 4, for the finding of relationships between events see Section 5. We also crawl the Wikipedia API to add an individual image to each event for the use in the timeline.
We provide access to the extracted events via the Web-API, SPARQL endpoint,
Linked Data Interface and in a timeline. The Web-API3 gives lightweight and fast
access to the events. Events can be queried by several URL parameters like
begin_date, end_date, lang, query, format, html, links, limit, order, category,
granularity and related. Users can query for keywords or time periods, and results are
returned in XML or JSON format. The Linked Data Interface4 holds a representation
of the yearly English dataset in the LODE ontology [
Event Parsing & Link Extraction +Adding Images +Adding Categories +Adding Relationships Web API
1F.1rontendE
Fig. 1. Processing, extraction and provision pipeline. v
3.21.2 ExtevnEraction of daily events 1.4 evnE 1.5 evnE 1.3 evnE 1.6 tsevnE
t t t t e
In addition to the extraction of yearly eventss presented ins[
Analyzing the data set The overall data set has been analyzed as a prerequisite to the automatic classification and the search for relationships between events. The number of extracted events and extraction quotients for different languages and granularity are shown in Table 1. The categories in German events are created from subheadings on the corresponding Wikipedia page. Yearly German events are categorized with one or two categories by headings of rank 2 or 3, which can be used for the automatic classification of events. Table 2 shows the ten most used categorizations for German events. In English or other languages categorizations are rarely used. The number of links and entities per event can be seen in Table 3. In the German and English dataset most events have between one and four links. To provide a useful semantic description of events, it is necessary to have types attached to these events. Possible types could be "Political Event", "Sports Event", etc. In the crawled datasets, some events already have types extracted from the Wikipedia pages, while others do not. Therefore, we use machine learning to add the types where they are not present.
The datasets we have crawled already contain links to Wikipedia articles. In order
to generate useful machine learning features, we have transformed these links to
DBpedia entities. For inferring event types, we have enhanced our datasets consisting
of events and their descriptions by more features: the direct types (rdf:type) and the
categories (dcterms:subject) of the entities linked to an event, both including their
transitive closures (regarding rdfs:subClassOf and skos:broader, respectively). For
enhancing the datasets, we have used our framework FeGeLOD [
As discussed above, the majority of events in our datasets comprises between one and four links to entities. Therefore, we have concentrated on such events in our analysis. We have conducted two experiments: first, we have inferred the event types on events from the German dataset, using cross validation for evaluation. Second, we have learned models on the German datasets and used these models to classify events from the English dataset, where types are not present. In the second experiment, we have evaluated the results manually on random subsets of the English dataset.
Figure 2 depicts the classification accuracy achieved in the first experiment, using 10-fold cross validation on the German dataset. We have used four random subsets of 1,000 events which we have processed by adding features and classifying them with three different commonly used machine learning algorithms: i.e., Naïve Bayes, Ripper (in the JRip implementation), and Support Vector Machines (using the Weka SMO implementation, treating the multi-class problem by using 1 vs. 1 classification with voting). As a baseline, we have predicted the largest class of the sample. It can be observed that the categories of related entities are more discriminative than the direct types. The best results (around 80% accuracy) are achieved with Support Vector Machines.
100 90 80 70 60 50 ) 40 (rycau% 321000 ccA 0 yspT trsegeo e i a
C One Entity s e i r o g e t a C + s e p y T yspT trseoge e i a C s e i r o g e t a C + s e p y
T
yspT trsgeeo e i a C s e i r o g e t a C + s e p y T
sypT trsoege e i a C
SMO s e i r o g e t a C + s e p y T
Since Support Vector Machines have yielded the best results in the first experiment, we have trained four SVMs for the second experiment, one for each number of related entities (one through four), using the subsets of 1,000 events. We have then used these models to classify four subsets of the English dataset, consisting of 50 events each. The results of that classification have been evaluated manually.
The results are shown in Figure 3. First, we have tested the best performing combination of the first experiment, using both categories and direct types of the related entities. Since the results were not satisfying, we have conducted a second evaluation using only direct types, which yielded better results. The most likely reason why categories work less well as features than classes is that the German and the English DBpedia use the same set of classes (i.e., DBpedia and YAGO ontology classes, among others), but different categories. In our experiments, we have observed that only a subset of the categories used in the German DBpedia have a corresponding category in the English DBpedia. Thus, categories, despite their discriminative power in a single-language scenario, are less suitable for training cross-language models.
In summary, we have been able to achieve a classification accuracy of around 70% for the English dataset, using a model trained on the German dataset. The results of both experiments show that machine learning with features from DBpedia is a feasible way to achieve an automatic classification of the extracted events. 5
With a dataset of events for different languages and granularities it is interesting to know which relations between these events exist. To find relationships, different features of the events could be used: (1) time, (2) categories, (3) topic/content or (4) links. Time as a single criterion is not by far enough. The category is too simplistic and there are only a few categories. Relationships based on the topic/content of the event are not easy to find as the events only include micro-text with a few words or sentences. Taking links as a criterion, we have to consider which links to take and how many links. In our approach we use a combination of the features time and links for extracting relationships between events.
As described in Section 3.3, we have extracted 178,502 events in total. From these, 172,189 events include links. As a preprocessing step, we transform every nonEnglish link to the English equivalent by querying the inter-language link from the Wikipedia API. As a result, every event from different languages contains links to English Wikipedia/DBpedia entities.
In the following, we analyze this set of events. As first step we vary the number of links that two events have to share and count the events that share this number of links with at least one other event (see Table 4). In detail, we consider two events to share a link if these events contain a link to the same DBpedia entity. From our analysis results it can be seen that 95.8 % of the events (that include links) share at least one link with at least one other event. As we are dealing with a multi-lingual set of events, it is interesting to know how many events share one link with at least one event of a different language. In our set of events, 155,769 events share at least one link with at least one other event of a different language, which is 90.5 % of the events in the set. 75.7% of the events include a link to another granularity, i.e. from year to month or vice versa. So far, we have looked for events that share one link in the overall database. In the following, we vary the time interval in which we search for these events (see Table 5). In detail, if we look at an event at time x, an interval of one month means that we search for events in the time interval [x-15 days : x + 15 days]. For the time-based analysis, we can only consider events where the date includes information on the day (and not only on the month and year). In our set these are 109,510 events.
Based on this analysis we have been able to define the relatedness between two events A and B with the time interval minimal and the number of shared links maximal between these events. Whereby we have found that in our dataset, a large part has at least one link in common (95.8%) within a time interval of a year (82.4%) and we can also find links to other languages (90.5%) and granularities (75.7%).We have implemented the relatedness feature in the Web-API. To compute related events for an individual event, we query for events that have at least one link in common within a time interval of plus/minus ten years and then sort results first by number of shared links and then by time distance to the original event.
For example, the query for Arab Spring5 finds eleven events from the yearly English dataset and related events from other languages and granularities. For example, the event of 2011/01/14: “Arab Spring: The Tunisian government falls after a month of increasingly violent protests President Zine El Abidine Ben Ali flees to Saudi Arabia after 23 years in power.” lists equivalent events from different languages, i.e. Italian: “In Tunisia, dopo violente proteste…”, Spanish: “en Túnez el presidente Zine El Abidine Ben…”, German: “Tunis/Tunesien: Nach den schweren Unruhen der Vortage verhängt Präsident Zine el-Abidine…” and from a month/news view: “Thousands of people protest across the country demanding the resignation of President Zine El Abidine Ben Ali. [Link] (BBC)”
As a final step we have compiled an evaluation set with 100 events and 5 related events for each and analyzed them manually. We have found that the perceived relatedness between two events (1) depends on the time interval between events and (2) depends on the count (1 vs. 4), type (general types like Consul vs. finer types like Julius Caesar) and position (at the beginning or the end of the description) of shared links.
In summary, we have been able to find a related event for nearly every event in the dataset, also for events from other languages and granularities. 6
We have extracted an event dataset from Wikipedia with about 170,000 events for different languages and granularities. A part of these events includes categories which can be used to automatically build categories for about 70% of another language set on the basis of links to other Wikipedia/DBpedia entities. The same linking base is used together with a time interval to extract related events for nearly every event, also for different languages and granularities.
At the moment, we only use Wikipedia/DBpedia links that are already included in
the events' descriptive texts. However, those links are not always complete or
available in other data sets. Using automatic tools such as DBpedia spotlight [
At the end of Section 5 we have shown that the perceived quality of events depends also on the abstractness of links. The analysis on how the abstractness of links can be modeled and used as an additional feature for the ranking of related events remains to future work. 5 http://www.vizgr.org/historical-events/search.php?query=arab%20spring&related=true