Evaluating a rule based strategy to map IMAGACT and T-PAS Andrea Amelio Ravelli Lorenzo Gregori Anna Feltracco Università di Firenze, Italy Università di Firenze, Italy Fondazione Bruno Kessler andreaamelio.ravelli lorenzo.gregori Università di Pavia, Italy @unifi.it @unifi.it Università di Bergamo, Italy feltracco@fbk.eu Abstract T-PAS is a repository of argument typed struc- tures for Italian verbs. Each verb is listed with its English. This paper presents the analy- structures, which correspond to different senses of sis of a mapping between two resources, the verb. For each structure, the specification of IMAGACT and T-PAS, made through a the expected semantic type in every argument po- rule-based algorithm which converts argu- sition (e.g. for the subject) is provided. ment structures in thematic roles. Results In this paper, we describe the results of a first are good in terms of Recall, while Pre- attempt of mapping information between these re- cision values are low: an analysis of the sources. Specifically, for each of the 248 verbs causes is proposed. analysed in both resources, we aim at matching the IMAGACT Types with the corresponding typed Italiano. Questo articolo presenta argument structures in T-PAS. We operate this l’analisi di un mapping tra le risorse IMA- mapping by applying a set of rules which convert GACT e T-PAS, realizzato attraverso un the information from the argument structure into a algoritmo basato su regole che converte le thematic-role combination, and find all the Types strutture argomentali in ruoli tematici. I that match this combination. risultati sono buoni in termini di Recall, The linking between argument and thematic mentre sono bassi i valori di Precision per structures of a predicate is a debated complex task i quali viene proposta un’analisi. in linguistic theories (Baker, 1997; Pinker, 2009; Bowerman, 1990, among others). The predictabil- ity of thematic roles from argument structure (or 1 Introduction viceversa) belongs to the syntax-semantics inter- The automatic mapping of information between face, and a study in this direction is out of the two resources is not a trivial task, but indeed scope of this paper. Our experiment is focused joining information over specific data can benefit on an empirical analysis of argument and thematic the involved resources. This paper describes the structures in Italian verbs and our aim is to evalu- analysis of a mapping between two linguistic re- ate whether, and to which extent, a rule-based sys- sources: IMAGACT and T-PAS. The motivation tem is able to produce thematic structures. We also behind this mapping starts with the observation intend to verify how these results can be exploited that both resources deal with Italian verbs disam- for a mapping purpose. biguation, are corpus-based and contain pieces of The paper is structured as follows: in Section 2 information that can be integrated with each other. we present the resources; in Section 3 we describe IMAGACT is a linguistic ontology of actions, the mapping procedure; in Section 4 we present that are grouped in concepts and related to dif- and discuss the results of the mapping, tested on ferent verb Types. For example, the action “John a gold standard; in Section 5 we provide direction takes the cup from the shelf” belongs to the con- for future work; in Section 6 we report our conclu- cept “take an object” and refers to Type 3 of the sions. verb to take. Each Type is also associated to one 2 The Resources or more thematic structures (e.g. [AGENT-verb- THEME-SOURCE]) and to videos via a set of In this section we describe IMAGACT and T-PAS. captions. Table 1 shows the total and shared quantitative data of the two resources. typed structure (henceforth t-pas), the specifica- tion of the expected semantic type (ST) for each IMAGACT T-PAS argument slot is provided. T-PAS accounts for the Total Verbs 777 1,000 following argument positions: subject, object, in- Total Types - t-pass 1,429 4,241 Shared Verbs 248 direct object, complement, adverbial and clausal. Shared Types - t-pass 421 1,153 A description of the sense, in the form of an impli- cature, is also linked to the t-pas. Table 1: Data of IMAGACT and T-PAS. Example 1 reports the t-pas#2 of the verb ab- battere: the STs [[Human]] and [[Event]] are spec- 2.1 IMAGACT ified for the subject position (as alternatives) and [[Building]] for the object position. IMAGACT1 (Moneglia et al., 2014; Panunzi et al., 2014) is a visual ontology of action that provides a (1) [[Human Event]-subj] abbattere [[Building]-obj] translation and disambiguation framework for ac- implicature:[[Human Event]] distrugge, butta giù [[Building]] tion verbs. The resource contains a fine-grained example: “Il muratore abbatte la parete.” categorization of action concepts, which are rep- (Eng.“The bricklayer knocks the wall.”) resented by one or more visual prototypes, in the The STs aim at generalizing over the set of lex- form of recorded videos or 3D animations. ical items observed in a certain position for a par- Action concepts are derived by a deep analy- ticular sense of the verb. For instance, in Example sis of the most frequent action verbs in Italian and 1, the ST [[Building]] generalizes over the lexi- English spoken corpora; this ensures the ontology cal item parete (Eng. wall). STs are drawn from to cover the most relevant actions for our every- a list of about 230 types4 and are also organized day activities. Given that no one-to-one corre- in a hierarchy, in which the elements are linked spondence can be established between an action by a “IS-A” relation (Jezek et al., 2016). Table 2 verb and an action concept (Moneglia, 1993), each presents a section of the hierarchy in which it is verb is divided in Types, which operate a seg- shown that [[Plane]] IS-A [[Vehicle]], [[Vehicle]] mentation of the predicate extension by identify- IS-A [[Machine]] and so on.5 If no generalization ing the prominent cores of the verb meaning. Verb is possible, the set of lexical items found in the Types are connected to action concepts and they argument position is listed. are the linkage point between lexical and action levels (Moneglia et al., 2012a). Types in IMA- ... GACT are inter-connected through semantic rela-  [[Artifact]]  [[Machine]] tions and gather the sentences retrieved in the spo-  [[Vehicle]] ken corpora, which have been classified and lin-  [[Plane]] guistically annotated with thematic roles and ak-  [[Road Vehicle]]  .. tionsart2 . The resource is growing continuously: by now, Table 2: Section of the STs hierarchy. it consists of a total of 1010 action concepts, each one with a visual representation (i.e. a scene), and Each t-pas corresponds to a distinct sense of the 21 covered languages (9 fully-mapped, 13 under- verb and is identified and defined by analysing in- way), with an average of 730 action verbs per lan- stances of the verb in a corpus, following the lexi- guage. cographic procedure called Corpus Pattern Analy- sis (Hanks, 2004; Hanks and Pustejovsky, 2005).6 2.2 T-PAS The corpus instances are then associated to the corresponding t-pas. T-PAS3 , Typed Predicate Argument Structures 4 (Jezek et al., 2014), is a repository of verb patterns For details on the list creation see (Jezek et al., 2014). 5 acquired from corpora by manual clustering distri- The same list has been used for the English resource PDEV (Hanks and Pustejovsky, 2005), http://pdev. butional information about Italian verbs. For every org.uk. The hierarchy can be found in http://pdev. org.uk/#onto. 1 http://www.imagact.it/ 6 According to the CPA procedure, after analysing a ran- 2 See Moneglia et al. (2012b) for details on annotated data dom sample of 250 concordances of the verb in the corpus, and ontology building process. each t-pas is defined by recognizing its relevant structure and 3 http://tpas.fbk.eu/ identifying the STs for each argument slots. Figure 1: An example of the mapping between IMAGACT and T-PAS for the verb macinare. T-PAS currently contains 1000 verbs. The refer- Datasets The rules for the conversion of a t- ence corpus is a reduced version of ItWAC (Baroni pas in a derived-ts have been manually created and Kilgarriff, 2006). by observing a sample of 15 verbs shared by the two resources (devset). We evaluated the map- 3 The Mapping ping against a gold standard manually created by pairing the Types of other 14 verbs with the corre- We aim at finding the best semantic match be- sponding t-pass. We extracted the 29 verbs from tween a verb Type in IMAGACT and the t-pass the 248 shared by the two resources. The selection of the same verb in T-PAS, the two referring to was made preserving the variability of the verbs the same action concept. Notice that it is possi- in the two resources, in terms of their number of ble that a Type in IMAGACT is mapped to more Types or t-pas. For instance, prendere (to take) is than one t-pas due, for instance, to different possi- associated with 17 t-pass in T-PAS and 18 Types ble verb alternations that can occur inside the same in IMAGACT; on the contrary bussare (to knock) Type. Figure 1 shows an example of this mapping, has only 2 t-pass and 1 Type. in which there is a match between Type 1 and t- pas#1 of the verb macinare. Conversion rules Table 3 synthesizes the rules The mapping is done as follows. By observing we adopted. The rules consider both the ST in the a sample of verbs in the resources, we first defined argument slot and the argument slot itself, and are a set of simple rules to convert the t-pas in a the- meant to associate a ST in an argument slot to a matic structure. Considering the ST in the argu- thematic role. For example, line 7 of Table 3 has ment positions of the t-pas (e.g. [Human]-subj, to be interpreted as follows: if for the subject po- [Food]-obj]), the rules aim at creating a thematic sition of the t-pas the ST [[Animate]] (or a IS-A structure for the t-pas of the kind AG-v-TH (dot- [[Animate]], according to the hierarchy of ST) is ted arrow in Figure 1). Then, we used an algorithm expected, then the AGENT role is selected (line which applies these rules to all the t-pass of a verb, 8). The rules also consider if the verb is in reflex- and map the derived thematic structure (derived- ive form (line 13). Moreover, if the t-pas regis- ts) to the thematic structures (ts) of the Types in ters the ST [[Abstract Entity]] (or a ST that IS-A IMAGACT (horizontal arrow in Figure 1). The [[Abstract Entity]]) as unique ST for any argument system thus compares all the ts in IMAGACT with position (i.e. it is the only ST expected for the po- all the derived-ts in T-PAS for the same verb, and sition), the t-pas was excluded from the mapping, retrieves the matches.7 In Figure 1, the t-pas#1 for as IMAGACT only accounts for physical actions the verb macinare have been transformed in the which do not involve abstract entities. structure AG-v-TH and then mapped to the ts of the Type. 4 Results and discussion The mapping between IMAGACT and T-PAS In order to calculate Precision (P) and Recall is made for the 248 verbs common to the two re- (R) of the algorithm, we considered that DESTI- sources. NATION (DE), SOURCE (SO) and LOCATION 7 (LO) roles can not always be discriminated (for Notice that the mapping is considering just this informa- tion of the resources and does not consider e.g. captions in example, room is a DE in “John puts a table in IMAGACT or examples in T-PAS. the room”, a SO in “John takes the table from 1 y = ST in argument slot 2 for y: 3 if y = IS or IS-A [Abstract | State | ..] 4 do not map 5 if obj: 6 y in obj = Theme TH 7 if y in subj IS or IS-A [[Animate]]: 8 subj = Agent AG 9 else: 10 subj = Causer CA 11 else: 12 if y in subj IS or IS-A [[Animate]] 13 & verb is reflexive: Figure 2: Distribution of the thematic structures. 14 subj = Actor AC 15 else: 16 subj = Theme TH ancies in the production of AG-v-TH, TH-v (too 17 for y !=subj and obj: high) and AG-v-TH-[DE|LO|SO] (too low) (see 18 x = (ImagAct Role != AG, CA, AC, Instrument IN) Figure 2). 19 x = y The critical issue is represented by the AG-v- Table 3: Rules for mapping. TH structure: this is the most frequent one among the IMAGACT Types and in our test set (112 over the room”, a LO in “John walks in the room”). 166 Types). For example, the following sentences The same happens for AGENT (AG) and ACTOR belong to 4 different Types of the verb stringere, (AC): a human can be an agent (“John sweeps but have the same ts AG-v-TH: “Marco stringe la the room”) or an actor (“John bumps his head”). mano a Luca”; “Marco stringe le gambe”; “Marco These limits can not be exceeded by an improve- stringe i pugni”; “Marco stringe la vite”. This hap- ment of the rule definitions, because they are pens also for the t-pas of stringere: 3 over the 5 strictly dependent on the verb semantics. When derived-ts are AG-v-TH, so the system produces calculating P and R, we grouped these derived 12 combinations over 3 attested in the gold stan- structures together. dard. The high frequency of this structure strongly influences the final P and R results. Moreover, the Precision (P) Recall (R) F-measure (F1) ts AG-v-TH is not distinctive of Types intra-verbs: 0.283 0.792 0.418 by taking all the verbs with more than one Type, Table 4: Precision, Recall, F1 of the mapping. and for which AG-v-TH is a possible ts, we mea- sured that in only 38,22% of them this ts is present We observe good values for R, while the P is in only one Type; in the other verbs (61.78%) very low (Table 4). A deeper analysis shows that the AG-v-TH structure appears in more than one in 34.61% of the cases, we have a full match with Type. the gold standard and in 38.46% the results from the mapping include the ones expected by the gold 5 Future work standard. This means that in many cases the sys- Given the result in terms of Precision we pre- tem is able to retrieve the correct matches. sented in the previous section, we are considering Figure 2 shows the distribution of the main the- to adopt other strategies that can be useful for the matic structures in the Types of the whole IMA- mapping of IMAGACT and T-PAS. GACT ontology (in orange), in the devset (in For instance, it would be possible to exploit the red), compared with the derived-ts from T-PAS (in examples from the corpus associated with each t- green). We verified a posteriori that the distribu- pas in T-PAS. In this sense, we hypothesize the tion of tss in the devset is strictly comparable with processing of these examples through BabelFy the one in the whole ontology, meaning that the (Moro et al., 2014), an online system for word devset is also well-balanced in terms of the the- sense disambiguation, based on the BabelNet se- matic structures coverage (see orange and red bars mantic network (Navigli and Ponzetto, 2012). Ba- in Figure 2). belNet is already linked to IMAGACT (via the By using the transformational rules we were scenes). We can use BabelFy in order to perform able to recreate all the structures that are used the disambiguation of a verb in the sentences as- in IMAGACT; however, there are some discrep- sociated to each t-pas. In this way we can ob- tain a link between the verb under examination and sources, but it provides a reliable set of mapping the corresponding BabelNet synset (i.e., a Babel- candidates: we believe that it can be fruitfully ex- Synset). The application of this method to every ploited for a first step of a mapping process, in or- example will result in a ranking of the most fre- der to filter a lot of unwanted matching possibili- quent BabelSynsets for the group of sentences of ties. We are confident that by exploiting additional each t-pas. Combining this output ranking with linguistic information from the two resources (e.g. the BabelNet-IMAGACT linking (Gregori et al., captions and occurrences in IMAGACT, lexical in- 2016), we will obtain the set of IMAGACT Types formation and examples in T-PAS), the precision that most likely match with each t-pas. of this mapping will improve sensibly. On the other way round, IMAGACT captions could also be mapped into the corresponding t- pass, by using the output of the algorithm de- References veloped in (Feltracco et al., 2016): given a sen- Mark C Baker. 1997. Thematic roles and syntactic tence of a t-pas, the algorithm identifies the lexi- structure. In Elements of grammar, pages 73–137. Springer. cal item(s) that are generalized by the ST for each argument position of every t-pas (e.g. assigning Marco Baroni and Adam Kilgarriff. 2006. Large the ST [[Building]] to “parete” in the sentence “Il linguistically-processed web corpora for multiple muratore abbatte la parete” for the t-pas [[Human | languages. In Proceedings of the Eleventh Confer- ence of the European Chapter of the Association for Event]] abbattere [[Building]]). A measure of se- Computational Linguistics: Posters & Demonstra- mantic similarity between the lexical items of an tions, pages 87–90. Association for Computational IMAGACT caption and the set of items associated Linguistics. to the same verb in T-PAS, would provide an ap- Melissa Bowerman. 1990. Mapping thematic roles proximation of which are t-pass that most likely onto syntactic functions: are children helped by in- match the given caption. The application of this nate linking rules? Linguistics, 28(6):1253–1290. method to every caption of an IMAGACT Type Anna Feltracco, Lorenzo Gatti, Simone Magnolini, will help us in the goal of mapping T-PAS with Bernardo Magnini, and Elisabetta Jezek. 2016. Us- IMAGACT. ing WordNet to Build Lexical Sets for Italian Verbs. This method added to our rule-based strategy In Proceedings of the Eighth Global WordNet Con- can be particularly useful to solve the ambigu- ference (GWC ’16), Bucharest, Romania, January. ity related to the thematic pattern AG-v-TH, for Lorenzo Gregori, Alessandro Panunzi, and An- which the use of lexical information would reduce drea Amelio Ravelli. 2016. Linking IMAGACT on- the number of possible matches. tology to BabelNet through action videos. Proceed- ings of Third Italian Conference on Computational Linguistics (CLiC-IT 2016), pages 162–167. 6 Conclusions Patrick Hanks and James Pustejovsky. 2005. A pattern In this paper we presented a first attempt of map- dictionary for natural language processing. Revue ping IMAGACT and T-PAS by using a rule-based française de linguistique appliquée, 10(2):63–82. algorithm for the automatic conversion of T-PAS Patrick Hanks. 2004. Corpus pattern analysis. In Pro- semantic types into thematic structures. We took ceedings of the Eleventh EURALEX International advantage of the strong discriminative power of Congress, Lorient, France, Universite de Bretagne- semantic types in their argument position to re- Sud. duce the possible set of allowed thematic struc- Elisabetta Jezek, Bernardo Magnini, Anna Feltracco, tures. This approach has an intrinsic limit: the- Alessia Bianchini, and Octavian Popescu. 2014. T- matic roles are determined by verb semantics and PAS: a resource of corpus-derived types predicate- their difference is not always reflected in the re- argument structures for linguistic analysis and se- lated semantic type. We also found out that the ts mantic processing. In Proceedings of the Ninth International Conference on Language Resources AG-v-TH represents the most critical issue, being and Evaluation (LREC’14), Reykjavik, Iceland, the most frequent structure, and appearing in more May. European Language Resources Association than one Type of the same verb. (ELRA). The results report a good recall and a low pre- Elisabetta Jezek, Anna Feltracco, Lorenzo Gatti, Si- cision, confirming that our algorithm is not able mone Magnolini, and Bernardo Magnini. 2016. to produce an actual mapping between the two re- Mapping Semantic Types onto WordNet Synset. In Proceedings of the Twelfth Joint ACL - ISO Work- shop on Interoperable Semantic Annotation (Isa ’12), Portorose, Slovenia, May. Massimo Moneglia, Gloria Gagliardi, Lorenzo Gre- gori, Alessandro Panunzi, Samuele Paladini, and Andrew Williams. 2012a. La variazione dei verbi generali nei corpora di parlato spontaneo. L’ontologia IMAGACT. In Proceedings of the VIIth GSCP International Conference: Speech and Cor- pora, pages 406–411. Massimo Moneglia, Gloria Gagliardi, Alessandro Pa- nunzi, Francesca Frontini, Irene Russo, and Monica Monachini. 2012b. Imagact: deriving an action on- tology from spoken corpora. In Eighth Joint ACL- ISO Workshop on Interoperable Semantic Annota- tion (isa-8), pages 42–47. Massimo Moneglia, Susan Brown, Francesca Frontini, Gloria Gagliardi, Fahad Khan, Monica Monachini, and Alessandro Panunzi. 2014. The IMAGACT Vi- sual Ontology. An Extendable Multilingual Infras- tructure for the Representation of Lexical Encod- ing of Action. In Proceedings of the Ninth Interna- tional Conference on Language Resources and Eval- uation (LREC’14), Reykjavik, Iceland, May. Euro- pean Language Resources Association (ELRA). Massimo Moneglia. 1993. Prototypical vs. non- prototypical predicates: ways of understanding and the semantic partition of lexical meaning. In Inter- national conference” Linguistics at the end of the century” Moscow State University February. Andrea Moro, Alessandro Raganato, and Roberto Nav- igli. 2014. Entity Linking meets Word Sense Dis- ambiguation: a Unified Approach. Transactions of the Association for Computational Linguistics (TACL), 2:231–244. Roberto Navigli and Simone Paolo Ponzetto. 2012. BabelNet: The automatic construction, evaluation and application of a wide-coverage multilingual se- mantic network. Artificial Intelligence, 193:217– 250. Alessandro Panunzi, Irene De Felice, Lorenzo Gre- gori, Stefano Jacoviello, Monica Monachini, Mas- simo Moneglia, Valeria Quochi, and Irene Russo. 2014. Translating Action Verbs using a Dictionary of Images: the IMAGACT Ontology. In XVI EU- RALEX International Congress: The User in Focus, pages 1163–1170, Bolzano / Bozen, 7/2014. EU- RALEX 2014, EURALEX 2014. Steven Pinker. 2009. Language learnability and lan- guage development, with new commentary by the author, volume 7. Harvard University Press.