This paper presents a method for the automatic conversion of constituency trees into deep dependency graphs consisting of primary, secondary, and semantic relations. Our work is distinguished from previous work concerning the generation of shallow dependency trees such that it generates dependency graphs incorporating deep structures in which relations stay consistent regardless of their surface positions, and derives relations between out-of-domain arguments, caused by syntactic variations such as open clause, relative clause, or coordination, and their predicates so the complete argument structures are represented for both verbal and non-verbal predicates. Our deep dependency graph conversion recovers important argument relations that would be missed by dependency tree conversion, and merges syntactic and semantic relations into one unified representation, which can reduce the bundle of developing another layer of annotation dedicated for predicate argument structures. Our graph conversion method is applied to six corpora in English and generated over 4.6M dependency graphs covering 20 different genres.1
Several approaches have been proposed for the automatic conversion of constituency
trees into dependency trees in English [
John gave Mary a book vs. vs.
Furthermore, since dependency trees are bounded by tree properties, single-root,
connected, single-head, and acyclic, they cannot represent any argument structure
that would break these properties [25]. Such argument structures occur often, where
an argument is shared by multiple predicates (e.g., open clauses, coordination) or it
becomes the head of its predicate by the syntax (e.g., relative clauses). Preserving the
tree properties allows the development of efficient parsing models [
This paper presents a method that converts the Penn Treebank style constituency
trees [27] into deep dependency graphs. Our dependency graphs are motivated by
deep structures [
Nivre [
There exist several corpora consisting of deep dependency graphs. Kromann
[24] introduced the Danish Dependency Treebank containing dependency graphs
with long-distance dependencies, gapping relations, and anaphoric reference links.
Al-Raheb et al. [
Although not directly related, it is worth mentioning the existing corpora
consisting of predicate argument structures. Baker et al. [
Our deep dependency graphs (DDG) preserve only two out of the four tree properties:
single-root and connected. Two types of dependencies are used to represent DDG.
The primary dependencies, represented by the top arcs in figures, form dependency
trees similar to the ones introduced by the Universal Dependencies (UD) [
Copula Non-verbal predicates are mostly constructed by copulas. DDG considers both the prototypical copula (e.g., be) as well as semi-copulas (e.g., become, remain). Non-verbal predicates with copulas can be easily identified by checking the function tag PRD (secondary predicate) in constituency trees (Figure 1a). Unlike UD, the preposition becomes the head of a preposition phrase when it is a predicate in DDG (Figure 1b). This is to avoid multiple subjects per predicate, which would be caused by making a clause as the head of a prepositional phrase (Figure 1c). Light verb construction Non-verbal predicates can also be constructed by lightverbs, which are not annotated in constituency trees but they are in PropBank [21]. A set of light verbs L = {make, take, have, do, give, keep}, a set of 2,474 eventive nouns N = {call, development, violation, : : :}, and a map M 2 jLj jNj ! jPj = {(give, call) ! to, (make, development) ! of, . . . } of prepositions indicating the objects of the nominal predicates are collected from PropBank. Given a verb v 2 L with the direct object n 2 N, v is considered a light verb and the preposition phrase that immediately follows n and contains the preposition p M(v; n) is considered the object of n in DDG (Figure 2b). This lexicon-based approach yields about 2.5 times more light verb constructions than PropBank annotation; further assessment of this pseudo annotation should be performed, which we will explore in the future. 3.2
Dative Indirect objects as well as preposition phrases whose semantic roles are the same as the indirect objects are considered datives. A nominal phrase is identified as an indirect object if it is followed by another nominal phrase representing the direct object (Figure 3a). A preposition phrase is considered a dative if it has either the function tag DTV (dative; Figure 3b) or BNF (benefactive; Figure 3c). Whether or not all benefactives should be considered datives is opened to a discussion; we plan to analyze this by using large unstructured data such as Wikipedia to measure the likelihood of dative constructions for each verb. 2All figures are provided together at the end of this paper. Expletive Both the existential there and the extrapositional it in the subject position are considered expletives. The existential there can be identified by checking the part-of-speech tag EX. The extrapositional it is indicated by the empty category *EXP*-d in constituency trees, where d is the index to the referent clause (Figure 4c). When there exists an expletive, DDG labels the referent as the subject of the main predicate (Figures 4a and 4d) such that it is consistently represented regardless of the syntactic alternations, whereas it is not the case in UD (Figures 4b and 4e). Passive construction Arguments in passive constructions are recognized as they would be in active constructions. The NP-movement for a passive construction is indicated by the empty category *-d in the constituency tree, where d is the index to the antecedent (Figures 5a and 5b). However, the NP-movement for a reduced passive construction is indicated by the empty category * with no index provided for the antecedent (Figure 5c). To find the antecedents in reduced passive constructions, we use the heuristic provided by NLP4J, an open source NLP toolkit, which gives over 99% agreement to the manual annotation of this kind in PropBank.3 In Figure 5, John, Mary, and book, are the subject (nsbj), the dative (dat), and the object (obj) of the predicate give, regardless of their syntactic variations in the active, passive, and reduced passive constructions, which can be achieved by deriving dependency relations from the empty categories. Note that the object relation in Figure 5c would cause a cyclic relation among primary dependencies such that it is represented by the secondary dependency in DDG.
Small clause A small clause is a declarative clause that consists of a subject and a secondary predicate, identified by the function tags SBJ and PRD, respectively. There are two kinds of small clauses found in constituency trees, one with an internal subject and the other with an external subject. Figure 6 shows examples of small clauses with internal subjects. In this case, John is consistently recognized as the subject of the adjectival predicate smart in the declarative clause (Figure 6a), the small clause (Figure 6b), and the small clause in the passive construction (Figure 6c). The subject relation in Figure 6c causes the non-projective dependency, which adds another complexity to DDG; nonetheless, making John as the subject of consider instead of smart as in UD would yield different relations between active (Figure 7a) and passive (Figure 7b) constructions, which is against the main objective of DDG.
Unlike the case of a small clause with the internal subject, a small clause with the external subject contains the empty category *PRO*-d where d is the index to the external subject. In this case, the external subject takes two separate semantic roles, one from its matrix verb and the other from the secondary predicate in the small clause. In Figure 8, John is consistently recognized as the object of the verbal predicate call and the subject of the nominal predicate baptist for both the active (Figure 8a) and the passive (Figure 8b) constructions in DDG, whereas it is not the case in UD. The subject relation between John and baptist is preserved by the secondary dependency to avoid multiple heads among the primary dependencies. 3This heuristic is currently used to pseudo annotate these links in PropBank, labeled as LINK-PSV. Open clause An open clause is a clause with the external subject indicated by the empty category *PRO*-d (see the description above). Figure 9 shows examples of open clauses. The external subjects are represented by the secondary dependencies to avoid multiple heads. Notice that the head of the open clause, teach, in Figure 9b is assigned with the semantic role prp (purpose) extracted from the function tag PRP, which gives a more fine-grained relation to this type (Section 3.4). Relative clause The NP-movement for the relativizer in a relative clause is noted by the empty category *T*-d in the constituency tree. Each relativizer is assigned with the dependency relation before its NP-movement and labeled as r-*, indicating that there exists a referent to this relativizer that should be assigned with the same relation. In Figure 10a, the relativizer who becomes the subject of the predicate smart so it is labeled as r-nsbj, implying that there exists the referent John that should be considered the real subject of smart. Similarly in Figure 10b, the relativizer who becomes the dative of the predicate buy so labeled as r-dat, implying that there exists John who is the real dative of buy. These referent relations are represented by the secondary dependencies to avoid cyclic relations. The constituency trees do not provide such referent information; we again use the heuristic provided by NLP4J,4 which has been used to pseudo generate such annotation in PropBank, LINK-SLC. Coordination Arguments in coordination structures are shared across predicates. These arguments can be identified in constituency trees; they are either the siblings of the coordinated verbs (e.g., the book and last year in Figure 11a) or the siblings of the verb phrases that are the ancestors of these verbs (e.g., John in Figure 11a). When the coordination is not on the same level, right node raising is used, which can be identified by the empty category *RNR*-d. In Figure 11b, John is coordinated across the verb phrase including value and the preposition phrase including for. Unlike the coordinated verbs in Figure 11a that are siblings, these are not siblings so need to be coordinated through right node raising. The coordinated arguments are represented by the secondary dependencies to avoid multiple heads. 3.3
Modal adjective Modal adjectives are connected with the class of modal verbs such as can, may, or should that are used with non-modal verbs to express possibility, permission, intention, etc: able likely willing unable An adjective am is considered a modal if 1)it is a non-verbal predicate (i.g., if it belong to an adjective phrase with the function tag PRD), 2)it is followed by a clause whose subject is an empty category e, and 3)the antecedent of e is the subject of am. In Figure 12, able and about are considered modal adjectives because they are followed by the clauses whose subjects are linked to the subjects of the adjectival predicates, John. Modal adjectives together with modal verbs give another level of abstraction in DDG.
Raising verb Distinguished from most of the previous work, raising verbs modify the “raised” verbs in DDG. A verb is considered a raising verb if 1)it is followed by a clause whose subject is the empty category *-d, and 2)the antecedent of the empty category is the subject of the raise verb. In Figure 13, the raising verbs go, have, and keep are followed by the clauses whose subjects are the empty categories *-1, *-2, and *-3, which all link to the same subject as the raised verb, study. have go continue need As shown in Figure 9a, semantic roles are extracted from certain function tags and added to the terminal heads of the phrases that include such function tags. The function tags used to extract semantic roles are: DIR: directional, EXT: extent, LOC: locative, MNR: manner, PRP: purpose, and TMP: temporal. 4 4.1
Six corpora that consist of the Penn Treebank style constituency trees are used to
generate deep dependency graphs: OntoNotes (Weischedel et al. [38]), the English
Web Treebank (Web; Petrov and McDonald [
SC WC
OntoNotes
138,566 2,620,495 Table 4 shows the distributions of the primary and secondary dependencies generated by our deep dependency graph conversion. At a glimpse, the portion of the secondary dependencies over the entire primary dependencies seems rather small (about 2.3%). However, when only the core arguments (*sbj, obj, dat, comp) and the adverbials (adv*, neg, ppmod) are considered, where the secondary dependencies are mostly focused on, the portion increases to 8.4%, which is more significant. Few of the secondary dependencies are generated for unexpected relations such as acl, appo, and attr; from our analysis, we found that those were mostly caused by annotation errors in constituency trees. We present a conversion method that automatically transforms constituency trees into deep dependency graphs. Our graphs consist of three types of relations, primary dependencies, secondary dependencies, and semantic roles, which can be processed separately or together to produce one unified dependency representation. The primary dependencies form dependency trees that can be generated by any non-projective dependency parser. The secondary dependencies together with the primary dependencies form deep dependency graphs. The semantic roles together with the syntactic dependencies form rich predicate argument structures. Our conversion method is applied to large corpora (over 4.6 times larger than the original Penn Treebank), which provides big data with much diversities. We plan to further extend this approach to more semantically-oriented dependency graphs by utilizing existing lexicons such as PropBank and VerbNet.
We gratefully acknowledge the support of the Kindi research grant. A special thank is due to professor Martha Palmer at the University of Colorado Boulder, who had encouraged the author to develop this representation during his Ph.D. program. Type
Label
Description csbj expl nsbj comp dat obj acl appo attr num relcl adv advcl advnp neg ppmod case conj com dep disc meta prn root r-adv r-advcl r-advnp r-comp r-nsbj r-ppmod
Total clr [18] Julia Hockenmaier and Mark Steedman. Acquiring Compact Lexicalized Grammars from a Cleaner Treebank. In Proceedings of the 3rd International Conference on Language Resources and Evaluation, LREC’02, 2002. [19] Julia Hockenmaier and Mark Steedman. CCGbank: A Corpus of CCG Derivations and Dependency Structures Extracted from the Penn Treebank. Computational Linguistics, 33(3):355–396, 2007. [20] Liang Huang and Kenji Sagae. Dynamic Programming for Linear-Time Incremental Parsing. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, ACL’10, 2010. [21] Jena D. Hwang, Archna Bhatia, Clare Bonial, Aous Mansouri, Ashwini Vaidya, Nianwen Xue, and Martha Palmer. PropBank Annotation of Multilingual Light Verb Constructions. In Proceedings of ACL workshop on Linguistic Annotation, LAW’10, pages 82–90, 2010. [22] Richard Johansson and Pierre Nugues. Extended Constituent-to-dependency Conversion for English. In Proceedings of the 16th Nordic Conference of Computational Linguistics, NODALIDA’07, 2007. [23] John Judge, Aoife Cahill, and Josef van Genabith. QuestionBank: Creating a Corpus of Parse-Annotated Questions. In Proceedings of the 44th Annual Meeting of the Association for Computational Linguistics, ACL’06, pages 497–504, 2006. [24] Matthias T. Kromann. The Danish Dependency Treebank and the underlying linguistic theory. In Proceedings of the Second Workshop on Treebanks and Linguistic Theories, TLT’03, 2003. [25] Sandra Kübler, Ryan T. McDonald, and Joakim Nivre. Dependency Parsing.
Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers, 2009. [26] Mitchell Marcus, Grace Kim, Mary Ann Marcinkiewicz, Robert Macintyre, Ann Bies, Mark Ferguson, Karen Katz, and Britta Schasberger. The Penn Treebank: Annotating Predicate Argument Structure. In ARPA Human Language Technology Workshop, pages 114–119, 1994. [27] Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. Building a Large Annotated Corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330, 1993. [28] Adam Meyers, Ruth Reeves, Catherine Macleod, Rachel Szekely, Veronika Zielinska, Brian Young, and Ralph Grishman. Annotating Noun Argument Structure for NomBank. In Proceedings of the 4th International Conference on Language Resources and Evaluation, LREC’04, 2004. [36] Sebastian Schuster and Christopher D. Manning. Enhanced English Universal Dependencies: An Improved Representation for Natural Language Understanding Tasks. In Proceedings of the Tenth International Conference on Language Resources and Evaluation, LREC’16, 2016. [38] Ralph Weischedel, Eduard Hovy, Martha Palmer, Mitch Marcus, Robert Belvin, Sameer Pradhan, Lance Ramshaw, and Nianwen Xue. OntoNotes: A Large Training Corpus for Enhanced Processing. In Joseph Olive, Caitlin Christianson, and John McCary, editors, Handbook of Natural Language Processing and Machine Translation. Springer, 2011. [39] Fei Xia. Extracting Tree Adjoining Grammars from Bracketed Corpora. In In Proceedings of the Fifth Natural Language Processing Pacific Rim Symposium, 1999. (a) Penn Treebank (PTB). (b) Deep Dependency Graph (DDG).
(c) Universal Dependencies (UD). (b) Deep Dependency Graph (DDG).
(c) Universal Dependencies (UD). (b) Penn Treebank (PTB; top) and Deep Dependency Graph (DDG; bottom).
(c) Penn Treebank (PTB; top) and Deep Dependency Graph (DDG; bottom). (b) Universal Dependencies (UD).
(c) Penn Treebank (PTB). (d) Deep Dependency Graph (DDG).
(e) Universal Dependencies (UD). (b) Penn Treebank (PTB; top) and Deep Dependency Graph (DDG; bottom).
(c) Penn Treebank (PTB; top) and Deep Dependency Graph (DDG; bottom). (b) Penn Treebank (PTB; top) and Deep Dependency Graph (DDG; bottom).
(c) Penn Treebank (PTB; top) and Deep Dependency Graph (DDG; bottom). (a) Universal Dependencies (UD).
(b) Universal Dependencies (UD). (a) Penn Treebank (top), Deep Dependency Graph (middle), and Universal Dependencies (bottom). (b) Penn Treebank (top), Deep Dependency Graph (middle), and Universal Dependencies (bottom).
(b) Penn Treebank (PTB; top) and Deep Dependency Graph (DDG; bottom).
(b) Penn Treebank (PTB; top) and Deep Dependency Graph (DDG; bottom).
(b) Penn Treebank (PTB; top) and Deep Dependency Graph (DDG; bottom).
(b) Penn Treebank (PTB; top) and Deep Dependency Graph (DDG; bottom).