Information Extraction is the art of extracting structured information from natural language text, and it has come a long way in recent years. Many systems focus on binary relationships between two entities - a subject and an object. However, most natural language text contains complex information such as beliefs, causality, anteriority, or relationships that span several sentences. In this paper, we survey existing approaches at this frontier, and outline promising directions of future work.
systems can deal with such information. In this survey article, we focus on 5 systems that have particular proInformation extraction (IE) is the process of extracting visions for dealing with more than triples: FRED [4], machine-readable, structured information from natural K-Parser [5], ClausIE [6], MinIE [7], and OpenIE [8]. language text. For example, given the sentence “An- We systematically analyze their ability to extract difgelina Jolie stars in the superhero film The Eternals”, ferent types of complex information. We study the dian IE system can extract the fact ⟨Angelina Jolie, stars, mensions that have been identified as particularly chalThe Eternals⟩. IE finds numerous applications, be it in lenging [9]: beliefs, negation, causality, anteriority, search engines, science, or the digital humanities [1]. ary relations, cross-sentence references, and contrast. In journalism, for example, IE techniques have been Finally, we outline the open challenges in the area. Our used for fact checking [2], to examine the “Panama Pa- survey is structured as follows: In Section 2, we dispers” [3], or to extract semantic information from web cuss the diferent dimensions of knowledge that we are pages (as in Reuter’s Calais service). interested in, and our test set of sentences. Section 3
Most of these IE systems extract triples, i.e., facts presents the systems that we study, and their results that consist of a subject, a predicate, and an object. In on our test sentences. Section 4 summarizes our findour example ⟨Angelina Jolie, stars, The Eternals⟩, the ings, before Section 5 concludes. subject is Angelina Jolie, the predicate is stars, and the object is The Eternals. However, much of the information that we care about is not of this form. Con- 2. Dimensions of complex sider for example the following sentence (taken from knowledge the Wikipedia article about Angelina Jolie): “Jolie applied for adoption as a single parent, because Vietnam’s adoption regulations do not allow unmarried couples to co-adopt”. This sentence does not talk about a simple triple. Instead, it contains a negation, a modifier (“as a single parent”), and a causal relationship. A cursory reading of any Wikipedia article, blog, journalistic piece of text, or even just the present paper suggests that the majority of sentences is not concerned with simple triples, but with more complex information.
The question thus arises to what degree current IE
Numerous surveys discuss information extraction systems (e.g., [10, 11, 12, 13]). While [13] also studies interproposition relationships, to the best of our knowledge, no survey has yet systematically compared the performance of IE systems for complex knowledge that goes beyond triples. In this survey, we focus on the following dimensions (loosely based on [9]): Belief. We are interested in sentences where a subject expresses a belief in a hypothetical statement, as in “Researchers believe that the Corona virus will disappear”. An IE system shall extract that researchers have a certain belief, but the system shall not extract that the Corona virus will de facto disappear (since this is just a hypothetical scenario).
Negations are sentences that contain a negation particle, such as “Brad Pitt and Angelina Jolie are no longer
together”. The IE system should explicitly pinpoint the negative polarity of the sentence.
Causality expresses a causal relationship between two We focus on IE systems that aspire to extract more than events, as in “The SpaceX rocket launch has been post- simple triples. We found the following state-of-the-art poned because of the bad weather”. We want the IE systems: ClausIE, MinIE, and OpenIE, K-Parser, and system to extract the two events, and to link them by a FRED. These systems fall into two categories: Semantic causal relation. parsers extract a structure from the input sentence that Anteriority expresses that one event happened before mirrors semantic relationships such as “agent of”. Repanother one, as in “Once the votes are closed, the new resentatives of these systems are FRED and K-Parser. president will be known”, or in “Monica took her driver’s Then there are Information Extraction Systems in the license before buying a car”. Again, we want the IE sys- proper sense of the word, which extract facts in a quasitem to extract two events and a link between them. logical representation. We study the systems ClausIE, Contrast is expressed by conjunctions such as MinIE, and OpenIE. “although”, and indicates that one event happened de- Another distinction is between open and closed IE spite another one – as in “Even though the weather was systems. Closed IE systems aim to extract facts whose bad, they enjoyed the concert”. This constellation car- components are from a predefined catalog of entities ries a slightly diferent meaning from the sentence that and relations. For example, from “Jolie will appear in links the two propositions by a simple “and”, and we The Eternals”, they aim to extract ⟨Angelina_Jolie, stars, want the system to mirror this. The_Eternals_(2020_movie)⟩ – where “Jolie” has been N-ary relations are relations with more than two par- mapped to the entity Angelina_Jolie, the “will appear” ticipants, as in “Emma bought a book from FNAC for is mapped to the predefined relation stars, and the movie $12 as a mother’s day gift”. Here, the goal is to extract is identified unambiguously. This is what the FRED the agent Emma, the patient (the book), a purpose, the system does. Open systems, in contrast, use spans of price, and the provenance of the book – in the spirit of the input sentence as subject, predicate, and object of frames [14]. the triple – as in ⟨Jolie, will appear in, The Eternals⟩. Anaphoras are words (such as “she”) that refer to an This is what ClausIE, MinIE, K-Parser, and the eponymic entity that has already been mentioned, as in “Walter OpenIE systems do. There are near-philosophical deElias Disney was passionate about drawing since he bates about which of the two paradigms is better suited was young”. Here, we want the system to link “he” to for IE. Often, the outputs of a closed IE system are betDisney. ter for reasoning and querying (because diferent surInter-sentence relations are relations between events face forms of the same entity have been canonicalized), that are expressed in diferent sentences. For example, but open IE systems can extract information from a we can say “Covid-19 spread around the world. For this wider variety of sentences (because they are not limreason, many countries went into lockdown”. Here, we ited to the predefined relations). Here, we just note that expect the system to extract a causal relationship be- the choice of open versus closed is a dimension that is tween the two sentences. The same principle can be orthogonal to the issues that we study in this survey. applied to causality, anteriority, -ary relations, con- We shall now present each system in detail, comtrast, and anaphoras. ment on its usability, and investigate how it deals with
Note that we do not expect the output of the sys- our test sentences. tem to be of a certain shape; we just want the aspect of knowledge to be mirrored in some way in the output 3.1. ClausIE of the system. To systematically analyse the diferent IE systems, we prepared 3-5 test sentences for each of our dimensions of complex knowledge. Our sentences vary the order, the topic, and the syntactic devices. For example, for causality, we have sentences that use “because” followed by a noun phrase, or by a subordinate clause. Table 1 shows some example sentences. Our full set of test sentences is available at our project Webpage https://github.com/michka2/Complex-IE.
ClausIE (Clause-Based Open Information Extraction [6]) is an open information extraction system. Each extracted fact contains a subject, a relation, and optionally one or several arguments. The system uses dependency parsing to identify the clauses in the sentence, i.e., the parts that express a coherent piece of information. Each clause is then transformed into a fact. The system can be found online1, and is easy to run. Let us now study how ClausIE performs on our test set.
Beliefs. ClausIE deals with this kind of sentences by generating two independent propositions. For example, from “Aristotle thought that the Sun moves around the Earth”, ClausIE will extract: (“Aristotle”, “thought”,“that the Sun moves around
the Earth”) (“the Sun”, “moves”, “around the Earth”)
Negation. ClausIE extracts the negative verb as the relation between the two arguments, as shown here: (“Ross and Rachel”, “were not”, “on a break”) (“Scientists”, “ignored”, ”E. at first thinking he lost his mind”) (“Scientists”, “ignored”, ”Einstein at first) (“Scientists”, “be thinking”, ”he lost his mind”) (“he”, “lost”, ”his mind”) (“his”, “has”, ”mind”)
a single sentence at a time. Thus, it is unable to see connections between two sentences, be it causality, anteriority, or anything else.
Summary. ClausIE identifies clauses, i.e., pieces of a sentence that express a coherent piece of information. Its results project away the aspects of anteriority, causality, contrast, and belief. Anaphoras are not resolved. However, ClausIE can efectively deal with
The SpaceX rocket launch has been postponed because of bad weather.
Once the votes are closed, the new president will be known.
It is not necessary to sanitize your groceries.
Angelina Jolie has been named Hollywood’s best paid actress.
Walter Disney was passionate about drawing since he was young.
Even though the weather was bad, they enjoyed the concert.
Lady Diana died in a car crash. Since that night, many people believe that her death was ordered by the Royal Family because of her relationship with Dodi Fayed.
Causality. ClausIE does not create a particular link be- -ary relations. tween two causally related propositions. We show here the result of “Trump wants to ban Twitter because of 3.2. MinIE the criticism he is receiving”:
N-ary relations are a setting that ClausIE handles explicitly. It extracts one proposition for each contributing entity: MinIE (Minimizing Facts in Open Information Extraction [7]) is an open information extraction system based (“Trump”, “wants”,“to ban Twitter because of on ClausIE. It augments the output by information on the criticism he is receiving”) polarity, modality, attribution, and quantities with se(“he”, “is receiving”, “criticism”) mantic annotations. In return it removes parts that are considered overly specific. The system is available online2, and is easy to run.
We used the “safe mode” of MinIE for our experiments, because it omits only very few pieces of information. On our test sentences, MinIE performs as fol(“Emma”, “bought”,“a pride and prejudice book from lows:
fnac for $12”) Belief. MinIE deals particularly well with this kind of (“Emma”, “bought”,“a pride and prejudice book as sentence. It detects verbs, adverbs and adjectives that a mother’s day gift”) express possibility and certainty, and it annotates the (“a mother”, “has”, “day gift”) triples accordingly. For “Aristotle thought that the Sun moves around the Earth”, we obtain
mother owned a “day gift”. However, the first line of the output correctly shows that there is one main phrase, where the parts with “from FNAC” and “As mother’s day gift” are optional (shown by the question mark).
Anaphoras are left as is by ClausIE, as shown in the parsing of “Scientists ignored Einstein at first, thinking he lost his mind”: Triple: “Sun”, “moves around”, “Earth”
Factuality: (+, CT), Attribution: (Aristotle, (+, PS))
and that it is a certainty (“CT”). The triple itself is also positive, attributed to Aristotle, and a possibility (as indicated by the nested “PS”). If we change the verb
from “thought” to “confirms”, then the nested factual- and other links between two parts of a sentence are also ity changes to “CT” (for “certainty”). not visible in MinIE’s output.
Negation is also a challenge where MinIE shines: It extracts the verb without negation, and changes the po- 3.3. OpenIE larity from “+” to “-”.
Causality is not explicitly treated in MinIE. For exam- The paradigm of Open Information extraction has been ple, “The SpaceX rocket launch has been postponed be- pushed forward by a series of systems from the Univercause of the bad weather” yields: sity of Washington:
Textrunner [15], ReVerb [16], OLLIE [8], and the latTriple: “SpaceX rocket launch”, est system, OpenIE 5 [8]. We focus on OpenIE 5, which “has been postponed because of”, is available online3. The tool is not easy to run: We “bad weather” have to download the build tool SBT4, a Language Model Factuality: (+, CT), Attribution: none (from a Google drive), and the actual OpenIE code from Anteriority, likewise, is not mirrored explicitly in MinIE.the git repository. These items have to be in specific When we test the sentence "Once the votes are closed, folders, and the actual system runs as a Scala program the new president will be known", MinIE does not de- by help of the Java runtime environment. Let us now tect the link between the two facts (the factualities are see how OpenIE 5 works on our sample sentences: all (+,CT), with no attribution): Belief. OpenIE 5 can correctly identify nested sentences. For our example sentence “Aristotle thought that the Sun moves around the Earth”, it extracts: Triple: “Votes”, “are closed”, “Once” Triple: “Votes”, “are closed”
Triple: “new president”, “be known”
N-ary relations are handled well by MinIE, since it is based on ClausIE. For our example sentence with the book from FNAC, we obtain:
Triple: “Emma”, “bought a book from”, “FNAC” Triple: “Emma”,“bought a book from FNAC for”, “QUANT_0_1” Triple: “Emma”,“bought a book from FNAC as”, “mother’s day gift”
Triple: “mother”, “has ”, “day gift”
Inter-sentence relations. Much like ClausIE, MinIE can deal with only a single sentence at a time. Summary. MinIE is based on ClausIE, and thus shines natively on -ary relations. Furthermore, it explicitly extracts polarity and attribution, two of our desiderata. Anaphoras, in contrast, are left unresolved. The causal
Context(Aristotle thought; List[0,17)):
(earth; moved; around the sun) (Aristotle; thought; earth moved around the sun)
pothesis, and attributes it to Aristotle. This retains that the hypothesis is not asserted. The expression “List [0,17)” represents the location of the context in the input sentence.
Negation. OpenIE 5 deals with negative sentences by extracting the verb in its negative form as the relation part:
Angeline Jolie; is not; from Tunisia
argument:
Anteriority is represented by a special time-indicator in Open IE 5. For “Monica took her driver’s license before buying a car”, we obtain:
arguments for the output fact. Our example sentence with the book yields:
can get confused. It either omits relevant information or generates wrong tuples. For example, for “Michelle Obama is not only known as Obama’s wife, but also as an advocate for poverty awareness”, we obtain only:
Covid-19; cannot be compared; to previous FRED is available online as a demonstration5. It can pandemics; because medical research has also be used programmatically through an API (with a advanced a lot. key from the developers). medical research; has advanced; a lot. Belief. In FRED, a belief sentence gives rise to one main event (the act of believing), which has as object another event, the belief itself. Hence, “Aristotle thought that the Earth moved around the Sun” yields the parse shown in Figure 1. Here, “Think” is the main event, which has Aristotle as an Agent and “Move” as a theme.
This theme is again an event, which has a theme and a Monica; took; her driver’s licence;T: before buying location. All predicates are standard relations from the a car RDF, OWL, and VerbNet (“VN”) vocabulary. Furthermore, all entities have been mapped to unambiguous DBpedia entities.
Negation is expressed by an additional relation “boxing: hasTruthValue” that leads to a node labeled with “boxing:False”. (All parsing trees are available on our Emma; bought; a book; from FNAC; for $12 as a gift Web page.)
Causality between two events in a single sentence is expressed, quite naturally, by a relation “fred:because” between the main event and the event that is the reason for the main event. The same technique is used for anteriority and contrast.
N-ary relations are handled very well by FRED: Each Michelle Obama; is not known; as Obama’s wife participant of the event is linked by a dedicated relation to the main verb.
Anaphoras are not linked to their referent by FRED.
The system creates a node for the pronoun, and labels it with “male” or “female”, but does not establish the we; arrived; T: on time link to the referent. the bus; was; late Inter-sentence relations. FRED does not detect the link between two events across two sentences. It deals with each event separately. FRED is also not able to perform anaphora resolution across two sentences.
Summary. FRED can detect negation, and represent beliefs, causality, anteriority, and contrasting sentences.
It also deals well with n-ary relations. However, it does not resolve anaphoras, and it does not detect relationships across sentences.
5. For “Despite the fact that the bus was late, we arrived on time”, we obtain:
They just remain as unlinked pronouns in the triples. Inter-sentence relations do not receive special treatment in OpenIE 5. The system just proceeds sentence by sentence.
Summary. OpenIE 5 shines with temporal expressions and belief sentences, for which it outputs special types of triples. For anaphoras, causality, contrast, and -ary relations, Open IE 5 resorts basically to a chunking of the input sentence.
3.4. FRED natural language text into a machine-readable representation. It generates a graph similar to the one we FRED [4] is a semantic parsing system that is particu- have already seen for FRED. K-parser identifies eventlarly tailored to the Semantic Web. In this spirit, it pro- event relations like causality or anteriority, event-entity duces a machine-readable RDF/OWL representation of relations such as semantic roles, and instance-of relathe sentence, categorizes each entity into a set of pre- tions between entities and classes. The K-parser demondefined classes, disambiguates the mentioned entities stration is no longer available online, but the code can to existing entities in the DBpedia knowledge base, and be downloaded6, and can be run by carefully following uses standard Semantic Web vocabulary wherever pos- the instructions. Let us now study the performance of sible. For this purpose, the system combines a wide ar- this system on our set of test sentences: ray of NLP and Semantic Web tools.
6https://github.com/arpit7123/K-Parser-JAR 3.5. Knowledge Parser Belief. For belief sentences, K-parser creates one event for each proposition – without detecting the link between them. For the sentence “Aristotle thought that the Earth moved around the Sun”, K-parser yields: has(Thought, Agent, Aristotle) has(Aristotle, semantic_role, thinker) has(moved, agent, Earth) has(Earth, semantic_role, mover) has(think, is_subclass_of, cognition) has(move, is_subclass_of, motion) Negation is specifically identified by K-Parser. It links the negated verb and the negation particle by the relation “negative”. For example, for “Joe Tribuani doesn’t speak French”, we obtain: has(He, is_subclass_of, person) has(He, semantic_role, candidate) has(Barack Obama-1, trait, American-4)
logical order of the events in an input sentence and extracts relations such as next_event and previous_event. However, whether this works or not depends on the time conjunction. For example, for “Once the votes are closed, the new president will be known”, we obtained two separate events. For “Monica took her driver’s license before buying a car”, we obtain the correct chaining of events (as shown by this excerpt): has(took, agent, Monica) has(took, next_event, buying) has(speak, negative, not) has(speak, is_subclass_of, communication) has(not, is_subclass_of, all) has(speak, is_subclass_of, communication) has(does, instance_of, do) has(Joe_Tribuani, is_subclass_of, person) has(Joe_Tribuani-1, semantic_role, talker) has(French, semantic_role, language) has(do, is_subclass_of, social) has(Speak, agent, Joe_Tribuani) has(speak, recipient, French)
for contrasting events. It treats each fact independently.
N-ary relations. As a semantic parser, K-parser can easily attach additional participants to an event. Furthermore, it enriches the graph with additional relations based on its ontology, identifying, e.g., instances of person, location, place etc.
Inter-sentence relations. Much like FRED, K-parser has dificulties with longer sentences. It also cannot link facts across diferent sentences.
Summary. K-Parser is a semantic parser that provides Causality. According to [5], K-parser deals with causal semantic annotations in addition to extracting the facts relationships by detecting discourse markers such as from the input sentence. It explicitly flags negated verbs, “because” and then extracting the relation caused_by. but it does not resolve anaphoras, it does not make the However, we were not able to reproduce this behav- distinction between a fact and a belief, and it cannot ior. Let us consider the sentence “Trump wants to ban deal with relationships across sentences. Twitter because of the criticism he is receiving”. Kparser considered the causal part separately, without making the connection between the two facts. Nonethe- 4. Analysis less, it linked the “wants” and “ban” events with the “objective” relationship. K-parser adds this binding when As we have seen, modern IE systems can cover much we have an event as an argument of another event. more than simple triples. Still, some desiderata remain open: has(Trump, is_subclass_of, person) Belief. With this kind of sentences, we need to rehas(Trump, semantic_role, lawgiver) tain the context in order not to afirm the belief statehas(Trump, semantic_role, wanter) ment as a fact. Open IE 5 handles this category of senhas(wants, agent, Trump) tences very well, by explicitly creating a context obhas(ban, agent, Trump) ject. MinIE, too, deals very well with this kind of senhas(ban, objective, wants) tences, by explicitly attributing the hypothesis to the has(ban, is_subclass_of, social) subject. FRED, too can easily create nested statements has(ban, recipient, twitter) in its graph output. The other systems erase the dishas(receive, is_subclass_of, possession) tinction between a belief and a fact. This is, of course, has(receiving, supporting_verb, is) problematic for downstream applications. has(criticism, is_subclass_of, communication) Negation. ClausIE and Open IE 5 just use the negated verb as is. MinIE, K-Parser, and FRED, in contrast, are able to specifically identify the negative polarity of the sentence. This is useful, e.g., for querying: The user can ask for all negative statements, or for all statements with a certain predicate (finding also the negative ones).
is an American politician. He was elected as the 44th president of the United States”. The system returns a long list of instance-of and subclass-of facts, where the crucial ones leave the pronoun untouched: We can also imagine applications for reasoning (where Inter-sentence relations. None of the systems we negative statements become counter-evidence for a hy- have seen can establish links across sentences. This pothesis), or for machine learning (where negative sen- is an important shortcoming, since complex informatences can serve as negative training examples). tion is often split across several statements in order to Causality and contrast are more challenging. Only avoid lengthy sentences. For example, not all events in FRED can see such a link between two events – and a biography are necessarily narrated in chronological only by copying the conjunction verbatim into the graph. order (Wikipedia biographies, e.g., usually first discuss
What we would want instead is an explicit causal re- the public life of a person and then the personal life). lationship. For example, for the sentence “Trump wants Here, anteriority markers such as “After that” provide to ban Twitter because of the criticism he is receiving”, an important clue. However, the IE systems we have we would expect: studied would completely lose the temporal order of events. The same goes for more complex chains of reasoning, where one sentence gives the reason for the preceding one. These links, likewise, are lost.
Such a representation would allow querying for the 5. Conclusion causes of an event, or for identifying transitive chains of causality. Possible knowledge representation for- In this survey, we have studied how state-of-the-art IE malisms for this type of facts are studied in [9]. systems deal with complex information that does not Anteriority could be treated similarly to causality. Cur- fit neatly into a single triple. While most systems can rently, only Open IE 5 and FRED can make such a re- easily deal with -ary relationships, none of them can lationship between two events explicit: Open IE 5 by perform anaphora resolution or see relationships beadding a temporal marker, and FRED by a link between yond single sentences. two event nodes. For future work, this survey can be extended by study
Again, in the ideal case, we would see something like ing more information extraction systems: the following (for our example sentence “Monica took her driver’s license before buying a car”):
F1: “Monica”, “took”, “driver’s license” F2: “Monica”, “bought”, “a car” F3: F1, “before”, F2
reasoning, establishing, e.g., transitive anteriority between two events that are not directly linked.
N-ary relations. All the systems we have studied can deal with -ary relations. Graphene [20] is a semantic parser that focuses on Anaphoras. In the ideal case, a system would replace n-ary predicate-argument structures. the pronoun by its referent. For example, consider “Barack Obama is an American politician. He was elected Our survey can also be extended to more dimensions as the 44th president of the United States”. We would of complex information: expect:
“Barack Obama ”, “is”, “ American politician” “Barack Obama”, “was elected as 44th president of”, Sentiments attribute a personal valuation to an event “ US” or an object.
ment is true.
However, none of the systems we have studied links a personal pronoun to its referent. This is a pity, because anaphoras are quite common in everyday written language (since they avoid repeating the subject of a sentence). When the anaphoras are not linked, the resulting triples are useless for applications such as querying or reasoning. If they could be linked, an important additional source of information could be tapped.
by the grant ANR-20-CHIA-0012-01 (“NoRDF”). propositions in open information extraction, in: EMNLP, 2016. [1] G. Weikum, J. Hofart, F. M. Suchanek, Knowl- [19] F. Corcoglioniti, M. Rospocher, A. Palmero Aproedge harvesting: Achievements and challenges, sio, Extracting knowledge from text with pikes, in: LNCS, 2019. in: ISWC, 2015. [2] H. Rashkin, E. Choi, J. Y. Jang, S. Volkova, Y. Choi, [20] M. Cetto, C. Niklaus, A. Freitas, S. Handschuh, Truth of varying shades: Analyzing language in Graphene: Semantically-linked propositions in fake news and political fact-checking, in: EMNLP, open information extraction, in: ACL, 2018. 2017. [3] T. Plattner, D. Orel, O. Steiner, Flexible data scraping, multi-language indexing, entity extraction and taxonomies: Tadam, a swiss tool to deal with huge amounts of unstructured data, in: Computational Journalism Symposium, 2016. [4] A. Gangemi, V. Presutti, D. Reforgiato Recupero,
A. G. Nuzzolese, F. Draicchio, M. Mongiovì, Semantic web machine reading with fred, Semantic
Web 8 (2017). [5] A. Sharma, N. H. Vo, S. Aditya, C. Baral, Towards addressing the winograd schema challenge—building and using a semantic parser and a knowledge hunting module, in: IJCAI, 2015. [6] L. Del Corro, R. Gemulla, Clausie: clause-based
open information extraction, in: WWW, 2013. [7] K. Gashteovski, R. Gemulla, L. d. Corro, Minie: minimizing facts in open information extraction, in: EMNLP, 2017. [8] M. Schmitz, S. Soderland, R. Bart, O. Etzioni, et al.,
Open language learning for information extraction, in: EMNLP, 2012. [9] F. M. Suchanek, The Need to Move Beyond
Triples, in: Text2Story workshop, 2020. [10] C.-H. Chang, M. Kayed, M. R. Girgis, K. F. Shaalan,
A survey of web information extraction systems,
TKDE 18 (2006). [11] J. L. Martinez-Rodriguez, A. Hogan, I. Lopez
Arevalo, Information extraction meets the semantic web: a survey, Semantic Web (2020). [12] S. Sarawagi, Information extraction, Now Pub
lishers Inc, 2008. [13] C. Niklaus, M. Cetto, A. Freitas, S. Handschuh,
A survey on open information extraction, arXiv preprint arXiv:1806.05599 (2018). [14] C. F. Baker, C. J. Fillmore, J. B. Lowe, The berkeley
framenet project, in: ACL, 1998. [15] O. Etzioni, M. Banko, S. Soderland, D. S. Weld,
Open information extraction from the web,
Comm. ACM 51 (2008). [16] A. Fader, S. Soderland, O. Etzioni, Identifying relations for open information extraction, in:
EMNLP, 2011. [17] R. E. Prasojo, M. Kacimi, W. Nutt, Stufie: Semantic tagging of unlabeled facets using fine-grained information extraction, in: CIKM, 2018. [18] N. Bhutani, H. Jagadish, D. Radev, Nested