Recent research and development have created the necessary ingredients for a major push in web-scale language understanding: large repositories of structured knowledge (DBpedia, the Google knowledge graph, Freebase, YAGO) progress in language processing (parsing, information extraction, computational semantics), linguistic knowledge resources (Treebanks, WordNet, BabelNet, UWN) and new powerful techniques for machine learning. A major goal is the automatic aggregation of knowledge from textual data. A central component of this endeavor is relation extraction (RE). In this paper, we will outline a new approach to connecting repositories of world knowledge with linguistic knowledge (syntactic and lexical semantics) via web-scale relation extraction technologies.
The powerful vision of a semantic web has already started materializing through
large repositories of structured knowledge extracted from Wikipedia and other
sources of texts or data bases. The existence of these knowledge resources and
further progress in linking open data collections has nurtured the demand for
even more extensive sources of structured knowledge. Although some (linked)
open data collections can feed into the stock of digital knowledge, the most
effective growth is expected from automatic extraction of information and knowledge
out of texts. Knowledge repositories such as the DBpedia [
In our own research [
However, so far the learned statistical models or rule/pattern sets are not
freely usable. For any given relation (including events and facts) in DBpedia,
Freebase or Yago, we have many instances but we do not have a knowledge
resource that tells us for the covered types of relations and facts which patterns
or lexical concepts a language owns to represent and describe instances of these
relations. If we had such a resource, it would be comparatively easy to build
extraction engines for any of the relations. We are happy to share our extraction
rules but they come in a special format suited for our relation extraction system
DARE [
Instead of trying to promote our rule set and format, we would like to propose in this paper a new knowledge resource that for each covered target relation contains dependency structures of linguistic constructions representing the target relation itself and semantically associated relations, which also indicate an instance of the target relation. We call this new resource type a SAR-graph, a dependency graph of Semantically Associated Relations. We could also term such a graph a language graph, because it represents the linguistic patterns for a relation in a knowledge graph. A language graph can be thought of as a bridge between the language and the knowledge graph, a bridge that characterizes the ways in which a language can express instances of a relation, and thus a mapping from strings to things. First samples of such language graphs have been built by means of the rule learning facility of the DARE relation extraction system. But we will also propose a simple and straightforward instrument for populating these language graphs by annotated examples. These are sentences containing a mention of a relation instance. 2
DARE can handle target relations of varying arity through a compositional and
recursive rule representation and a bottom-up rule discovery strategy. A DARE
rule for an n-ary relation can be composed of rules for its projections, namely,
rules that extract a subset of the n arguments. Furthermore, it defines explicitly
the semantic roles of linguistic arguments for the target relation. The following
examples illustrate the DARE rule and its extraction strategy. Example 1. is
a relation instance of the target relation from [
Example 2. Mohamed ElBaradei, won the 2005 Nobel Prize for Peace on Friday.
Given Example 1. as a seed, Example 1. matches with the sentence in Example 2. and DARE assigns the semantic roles known in the seed to the matched linguistic arguments in Example 2. Fig. 1. is a simplified dependency tree of Example 2. with named entity annotations and corresponding semantic role labelling after the match with the seed. DARE utilizes a bottom-up rule discovery strategy to extract rules from such semantic role labelled dependency trees.
sutjbjjjejctjjj “win” UUUUobUjUecUtU*
Year: Year rddddddldexd-dmoddddddPddridlzedex:-P“mPordrizizee_” NXaXmXXeXXmoXdXXXXXXX+ “for”
pcomp-n
2pos verb 3 64lmexo-dfoerm “awctiinv”e75 66head 6 6 6 66daughters < subject hhead 6 6 6 6 6 6 4 7 7 7 7 7 1 Personi , 77 7 2 2rule year_prize_area_1 :: 33 777 64object 46 <3P4riYzeea_rA,r2ePar>ize_Name,7575>775 3 Output :: < 1 Winner, 2 Prize_Name, 3 Prize_Area, 4 Year >
From the tree in Fig. 1., DARE learns three rules in a bottom-up manner, each step with a one tree depth. The first rule is extracted from the subtree dominated by the preposition “for”, extracting the argument Prize_Area (Area), while the second rule makes use of the subtree dominated by the noun “Prize”, extracting the arguments Year (Year) and Prize_Name (Prize), and calling the first rule for the argument Prize_Area (Area). The third rule “winner_prize_area_year_1” is depicted in Figure 2. The value of Rule body is extracted from the dependency tree. In “winner_prize_area_year_1”, the subject value Person fills the semantic role Winner. The object value calls internally the second rule called “year_prize_area_1”, which handles the other arguments
The Web-DARE system is a further development of the DARE system [
We have started to use our learned extraction patterns as start content for building an open resource that bridges the gap between the world knowledge as encoded in knowledge repositories on the one hand and the representation of facts and events in human language texts on the other. For each considered target relation represented in the knowledge graphs, such a resource will consist of merged dependency graphs for all relevant patterns learned from mentions of relation instances.
We will employ lexical knowledge bases such as BabelNet, WordNet,
VerbNet, UWN to extend the content words in the dependency relations by
semantically related words (synonyms, hyponyms, hyperonyms). This will allow further
merging of subgraphs and will also make the sar-graphs more general. Such
a sar-graph will help to identify and compose mentions of argument entities
and projections of an n-ary relation. In Figure 3, we depict linkings between
knowledge graphs, relation-specific dependency grammar based relation
extraction rules and relation-specific lexical semantic graphs. Given the facts in the
knowledge graphs such as Freebase and the free texts provided on WWW,
relation extraction systems such as our DARE and Web-DARE systems can learn
grammar-based relation extraction rules for each relation type available in the
knowledge graphs. Given the relation extraction rules, sentence mentions from
which the rules are learned and general lexical semantic network such as
BabelNet, we can learn and extract relation-specific lexical semantic graphs as we
have done in [
A sar-graph can be built for every n-ary relation Rha1; :::; ani such as marriage RhP erson_1; P erson_2; CeremonyLoc; F romDate; T oDatei and every language l. A sar-graph of a relation R is a directed graph with labeled edges and vertices. The relation R we call the target relation. The function of the sar-graph is to represent the linguistic constructions the language l provides for reporting instances of R or for just referring to such instances. The linguistic constructions are represented as dependency structures that only include words belonging to the construction and slots for the arguments. Thus a sar-graph is composed of syntactic dependency graphs. Their edges denote dependency relations. Each edge is labeled with the tag the parser has assigned to the dependency. Vertices come in two flavors: One type of vertices denotes a regular node in a dependency structure, thus it is labeled with a word. Vertices of the second type represent the slots for the arguments of the target relation, instead of a word, they are labeled by the name of the argument, e.g. P erson_1.
If some given language l had only one single construction to express an instance of R then the dependency structure of this construction would be the entire sar-graph. But if the language offered alternatives to this construction, i.e. paraphrases, their dependency structures would also be entered into the sar-graph. They would be connected in such a way that all vertices labeled by the same argument name are merged.
Our rules do not just detect constructions that denote the entire relation but also many constructions referring to aspects or parts of the relation instance. As long as these constructions indicate an instance of the target relation, they are needed for high-recall relation extraction. One type of constructions that are not true paraphrases of the constructions exactly expressing the n-ary relation, denote instances of projections of R. A sar-graph for the following two English constructions would look as presented in Figure 4.
Constructions that refer to some part or aspect of the relation would normally be seen as sufficient evidence of an instance even if there could be contexts in which this implication is canceled.
Other constructions refer to relations that entail the target relations without being part of it.
And finally there are constructions referring to semantically connected relations that by themselves might not be used for safely detecting instances of R but that could be employed for recall-optimized applications or for the probabilistic detection process that combines several pieces of evidence.
Some entirely probabilistic entailments are caused by social conventions or behavioral preferences.
home.
In a next step, we extend the sar-graphs by lexical semantic knowledge.
During the semantic filtering of our rules [
The following artificially constrained example in Figure 5 may serve to illustrate the structure and contents of a sar-graph. The target relation is marriage again. We include only five constructions extracted from the five listed sentences. After each sentence we list the dependency relations from the full parse that belong to the construction. For better readability, we omit the reading numbers in this example.
Fig. 5: Example of sar-graph including lexical semantic information
The most obvious application of our approach is information extraction. As
the sar-graphs we have already built contain all the patterns we had learned
and tested for n relations, we know that the information in a sar-graph can be
successfully applied to regular relation extraction, i.e. the detection of instances
in sentences. Without being able to already prove it, we also believe that
sargraphs are especially suited for the extraction of n-ary relations across sentence
boundaries. When we worked on relation extraction from sequences of sentences
[
is a urologist from Dallas. I attended their wedding event in 2011.
Here several associated relations from the sar-graph are applied including the synonym hubby for husband. The sar-graph also tells us which roles of the target relation are filled by the associated relations.
We expect that sar-graphs could also become useful for summarization, since they permit to find constructions that express all or most arguments of a relation in one sentence. Finally, they might be employed for the generation of sentences from database facts, e.g., in reports. Because of the range of paraphrases, generation could produce stylistic variation as extensively used in reports written by human authors. 6
As described above, the first sar-graphs were simply built by combining the dependency structures in automatically acquired relation extraction rules. But this is not the only way to populate sar-graphs. Just as the DARE rule-learning formalism can extract a rule from a mention, the extraction could also happen from a sample sentence. All that is needed is some annotation marking the arguments of the target relation. The sentence will then be parsed. The dependency structure of the construction will be determined as the minimal spanning tree containing the arguments. The instantiated arguments will be substituted by the names of the argument places given in the markup. The content words of the extracted structure will be disambiguated utilizing BabelNet. If the content words are not already contained in the sar-graph, semantically related content words will also be added as exemplified in Figure 5. 7
We have shown how to combine the automatically accumulated knowledge about the means a language provides for speaking about a certain relation into one connected graph. We have also described, how such a graph could be built or extended from annotated examples.
A network that combines several semantic relations describing different parts or aspects of a fragment of the world is somewhat reminiscent of so-called semantic nets. As such the semantic combination of multiple relations seems to belong into the semantic knowledge base, such as the knowledge graph or some specialized ontology. So why do we not simply build a language-independent semantic network first and then look for the linguistic constructions that different languages use to express the relations. Such a strategy would throw us back to a research paradigm in which knowledge engineering precedes any attempt of language understanding. From experience we have learned that there could be numerous different ontologies just for the thematic area marriage. Lawyers, event managers, relationship counselors, vital statisticians may come up with completely different ways to select and structure the relevant knowledge pieces. How could we decide on the best ontology for relation extraction? Would any of such intellectually created ontologies contain a relation for exchanging the vows and one for tying the knot? How would the vows and the knot be represented? The great advantage of the bottom up empirical approach we are taking is that our sar-graphs are determined by the way people refer to a relation (event type, process, etc.). This makes them suited for semantic text analytics including information extraction.
Another important advantage is the association of a graph to a specific language. A Greek report on a wedding may refer to the wedding crowns for bride and groom, in an English sar-graph for the marriage relation, such crowns would not show up. In a Greek wedding the betrothal can be a part of the entire ceremony in other cultures it must have taken place a certain period before the wedding. In some cultures, exchanging the rings means getting married in others there is no such concept.
We are convinced that we need the interaction of two strategies to build up a growing stock of structured knowledge in the spirit of a semantic web. One strategy that starts from structuring growing portions of textual knowledge sources such as the Wikipedia and extends this knowledge by structured data such as linked open data, and another strategy that uses and extends the resulting repositories of structured knowledge by extracting from all sorts of texts much more knowledge, especially contingent knowledge. The novel type of repository we have proposed will on the one hand facilitate the latter process and on the other hand maintain the link of the accumulated domain-sorted linguistic knowledge with DBpedia, knowledge graph or similar knowledge resources.
Acknowledgements. This research was partially supported by the German Federal Ministry of Education and Research (BMBF) through the projects Deependance (contract 01IW11003), by Google through a Faculty Research Award granted in July 2012 and a Focused Research Award granted in July 2013.