Classification of German verbs using nouns in argument positions and aspectual features Michael Richter Jürgen Hermes Radboud University Nijmegen University of Cologne mprrichter@t-online.de hermesj@uni-koeln.de the aspect-based classification of Richter and van Abstract Hout (2015) into five classes which extends the typology of Vendler (1967), i.e. This paper provides evidence that accomplishments, achievements, states and aspectual verb classes (Vendler, 1967) activities by the class accomplishments with an can be induced from nominal fillers in affected subject. argument positions and aspectual This classification into five aspectual verb features. We classified 35 German verbs classes was derived by combining two user based in a supervised learning procedure using classifications induced by cluster analyses from a support vector machine classifier and a raters’ judgments and associations with stimulus classification into five aspectual classes verbs and two usage based classifications (Richter and van Hout, 2015) as gold induced from corpus data (Richter and van Hout, standard and observed excellent and 2015). We took this classification as gold substantial agreements. standard as we were interested in the correlation 1 Introduction of the semantics of the nominal fillers in argument positions of verbs and the aspectual This study aims to empirically validate aspectual properties of verbs thereby following Klein verb classes in German using large corpus data. (2009) who defines aspect as a grammatical Siegel (1997) and Siegel and McKeown (2000) category of verbs. induced the two aspectual classes states and In the present study we represent verbs as vectors events in the frame of a vector space model from that consist of nouns in argument positions corpora, however an induction of the complete separated into areas according to their noun Vendlerian typology has not yet been classes, which were induced by cluster analyses undertaken. We hypothesize that aspectual verb from similarity data. In addition, we added classes can be automatically induced from the aspectual features as defined by Vendler (1967) classified nominal fillers in the argument to the vectors in order to compare the predictive position of verbs. Our hypothesis refers to the power of the noun classes in argument positions Distributional Hypothesis (Rubenstein and against the predictive power of the aspectual Goodenough, 1965; Schütze and Pedersen, 1995; features, respectively. The test set of verbs was Landauer and Dumais, 1997; Pantel, 2005) classified in a supervised learning procedure which says that semantically related linguistic using a support vector machine (SVM) classifier. elements appear in semantically related contexts. In order to compare the results with aspectual The present study in the framework of a vector verbs classes as gold standard with a gold space model is also driven by the Statistical standard-classification based on concrete Semantics Hypothesis (Weaver, 1955; Furnas et semantic categories compatible with al., 1983; Turney and Pantel, 2010) which states Schumacher's typology (1986) of German verbs, that linguistic meaning can be derived from we trained the SVM classifier with a statistic linguistic patterns. In order to test our classification based on ten verb classes which hypothesis, we took a test set of verbs from comprises classes such as verbs of consumption Schumacher (1986) and determined the nominal and verbs of handicraft working (Richter and van fillers and their classes in argument positions. Hout, 2015). This classification was induced That is, in subject, direct, indirect, and prepositional object positions by parsing a very large German corpus. As gold standard we used Copyright © by the paper’s authors. Copying permitted for private and academic purposes. In Vito Pirrelli, Claudia Marzi, Marcello Ferro (eds.): Word Structure and Word Usage. Proceedings of the NetWordS Final Conference, Pisa, March 30-April 1, 2015, published at http://ceur-ws.org 177 from the co-occurrence data bank (CCDB) of the by the TF-IDF measure and classified by cluster Institut für Deutsche Sprache (IDS).1 analyses carried out on a matrix with similarity values taken from the co-occurrence data bank 2 Method (CCDB) of the Institut für Deutsche Sprache We classified 35 common German verbs used by (IDS).5 On the matrix of the similarity values, a Schumacher (1986), who defines seven lexical cluster analysis with Ward’s method and semantic macrofields and 30 subfields. We chose Euclidean distance was carried out. According to the verbs from all subfields, the only criterion the Bayesian Information Criterion there are two being the representation of every subfield in optimal noun classes for all arguments. We order to cover the total semantic range of interpreted the resulting noun classes using our Schumacher’s typology (1986). We checked the intuition thereby applying the criterion of frequency of the verbs in the first one million animacy (Croft, 2003; Aissen, 2003): The sentences containing at least one of our selected resulting two noun classes can be interpreted as verbs of the web based 880 million word denoting predominantly animate and inanimate SDEWAC corpus2. The verbs of our test set things, respectively class 1[+animate] for instance, occurred in more than one million sentences with contains nouns such as Arzt ’doctor’ Lehrkraft a mean frequency of approximately 30,000 ’teacher’ and class 2[-animate] contains nouns such occurrences per verb. 66 percent of the verbs was as Entwicklung ’development’, Organisation in the interval between 5,000 and 40,000 ’organization’ and Wahrnehmung ’perception’. occurrences, the more frequent outliers being The verbs’ vectors consist of areas for each müssen ‘to must’ with 500,965 and halten für ‘to argument type. There are four areas in total and take so./sth. for so./sth.’ with 123,595 each area is split into areas for each noun class as occurrences. We added five verbs; hämmern ‘to is depicted in (1): hammer’, schneiden ‘to cut’, aufessen ‘to eat up’, laufen ‘to walk/to run’, and zersägen ‘to saw into . pieces’ since these verbs since a previous study . (Richter and van Hout 2015) showed (i) that . laufen ‘to walk/to run’ and zersägen ‘to saw into pieces’ are typical activity and accomplishment verbs respectively and (ii) that aufessen ‘to eat = up’ is a typical accomplishment with an affected . subject verb. Schneiden ‘to cut’ and hämmern ‘to . hammer’ were ambiguous (Richter and van Hout, 2015), but we decided to classify in this . study the former as accomplishment and the ( : ℎ !! ) latter as a process verb. In order to determine the verbs’ arguments we parsed at most 30.000 sentences per verb using the Mate-Tools dependency-parser (Bohnet, Figure 1. Dimensions of verb vectors: Weighted verbs 2010)3. The whole code we used for filtering and in noun class areas. parsing the sentences, and aggregating the actants and aspectual features (see below) is In addition, the vectors were completed by available at GitHub.4 The 35 verbs of our test set aspectual features that Vendler (1967) suggested (Richter and van Hout, 2015) are represented as in order to distinguish aspectual verb classes. 139 dimensional vectors containing the 30 most The aspectual features indicate, for instance, frequent nouns in the verbs argument positions: whether the verbs occur in sentences with subjects, direct objects, indirect objects and temporal specifications of duration or a limited prepositional objects. The nouns were weighted time span with prepositions in and for, respectively, as in he wrote the letter in an hour 1 http://corpora.ids-mannheim.de/ccdb/. The similarity versus he wrote the letter for an hour, whether values were provided by Cyril Belica. the verbs can be embedded by matrix verbs such 2 The SdeWaC Corpus is available at the WaCky as persuade or whether they occur in imperative Corporadownload page at forms. In order to classify the 35 verbs we used a http://wacky.sslmit.unibo.it/doku.php?id=corpora 3 See https://code.google.com/p/mate-tools/ 4 5 https://github.com/spinfo/verbclass The similarity values were provided by Cyril Belica. 178 SVM classifier with a non-linear kernel which müssen* ‘to must’, einschlafen* ‘to fall asleep’ , achieved the best results. vergehen ‘to go (by)/ to pass/to disappear’, We first trained the SVM using the classification übersehen ‘to overlook’, fehlen ‘to lack’, of Richter and van Hout (2014) as a gold verlieren ‘to loose’, verhindern ‘to prevent’, standard and tested it with a 10-fold cross- abgrenzen ‘to mark off/ to define’, abweichen validation. The gold standard classification in ‘to deviate detail: 4. verbs of transfer (of information): mitteilen ‘to inform’, übermitteln ‘to 1. accomplishments: communicate/to forward’ aufbauen auf ‘to build on/to be based on’, 5. verbs of examination (by mental activity): herstellen ‘to produce’, schneiden ‘to cut‘, nachprüfen ‘to ascertain/to check’, erörtern ‘to zersägen ’to saw into pieces‘, verlängern ‘to debate’, untersuchen ‘to examine’ extend’, mitteilen ‘to tell/to inform’, übermitteln 6. verbs of production: ‘to communicate/to forward’, verhindern ‘to aufbauen auf ’to build on‘/acc to be based on’, prevent’, abgrenzen ‘mark off/to define’ herstellen ‘to produce’ 2. accomplishments with affected subject: 7. verbs of beginning and rising processes: untersuchen ‘to examine‘, bedenken ’to anfangen ‘to begin’, ansteigen ‘to rise/ to consider‘, erörtern ‘to debate’, nachprüfen ‘to increase’ ascertain/to check’, aufessen ‘to eat up’, essen 8. verbs of discussion and consideration: ‘to eat’ betreffen ‘to concern’, bedenken ‘to consider’, 3. activities: eingehen auf ‘to respond to so./sth.’, halten für laufen ‘to walk/to run‘, eingehen auf ‘to respond ‘to take, richten auf ‘to direct towards‘, to so./sth.’, hämmern ‘to hammer’, ansteigen ‘to orientieren an ‘to be geared to’ increase‘ 9. verbs of membership and agreement: 4. achievements: angehören ‘to belong to’, übereinstimmen mit ’to einschlafen ‘to fall asleep‘, vergehen ‘to go agree with’ (by)/to pass/to diasappear‘, übersehen ‘to 10. folgen aus ‘to follow from’, laufen ‘to overlook’, verlieren’to loose’, anfangen ‘to walk/to run’, existieren ‘to exist’, begin‘, abweichen ‘to deviate‘, sich orientieren verlängern ‘to extend’ an ‘to be geared to‘, richten auf ‘to direct towards/to focus’ 2.1 Results 5. states: existieren ‘to exist‘, fehlen ‘to lack‘, müssen ‘to In order to evaluate the consistency of the must‘, halten für ‘to take so./sth.for so./sth.‘, comparisons of the classifications against the folgen aus ‘to follow from‘, angehören ‘to gold standards we calculated both accuracy and belong to‘, übereinstimmen ‘to agree‘, betreffen Cohen’s kappa. The latter measure considers the ‘to concern’, abweichen ‘to deviate’ , verhindern number of classes which differ in the two gold ‘to prevent’ standards and, in addition, gives the significance levels. The classification into classes of concrete lexical Taking the classification with five aspectual properties which we induced from the co- verbs classes as gold standard the subject feature occurrence data bank (see above) is given below clearly outperforms the remaining features with (the class labels are compatible with .857 accuracy (which means that 30 of 35 verbs Schumacher’s labels and are assigned using our were classified correctly) and k = .812. Kappa linguistic intuitions; class 10 is incoherent and values above .61 are characterized as substantial, could not be labelled): above .81 as almost perfect agreement and therefore highly significant. The combinations 1. verbs of activities manipulating a substance subject-direct object-prepositional object- (normally with a tool): aspectual features and subject-direct object- hämmern ‘to hammer’, schneiden ‘to cut’, aspectual features yield .828 accuracy, k = .775 zersägen ‘to saw into pieces’ and k = .773, respectively. The combinations 2. verbs of consumption: subject-prepositional object-aspectual features, aufessen ‘to eat up’, essen ‘to eat‘ subject-direct object-prepositional object and 3. verbs of difference, ‘negative’ processes, non- subject- aspectual features yield .8 accuracy each existence: with k = .741, k = .739 and k = .71, respectively. 179 In contrast the remaining features, including the Figure 2. Accuracy of the argument and aspectual aspectual feature which yields .514 % accuracy, features using five aspectual verb classes vs. ten with k = .317 (fair agreement), perform poorly. classes with concrete lexical properties as gold Taking the classification according to concrete standard. semantic properties into ten classes as the gold Note: s: subject, d: direct object, i: indirect object, p: standard we observed that the hierarchy remains prepositional object, as: aspectual features, and almost the same, the subject feature outperforms combinations of predictors, for instance, das: direct the remaining features. However, the accuracy is object and aspect, sp: subject and prepositional object. considerably lower compared to the classification with 5 aspectual verb classes. The 3 Conclusion subject achieves .657 accuracy, k = .573. The The study provides evidence for the hypothesis combinations subject-direct object-aspectual that aspectual verb classes can be induced from features and subject-direct object-prepositional classified nominal fillers in argument positions. object yield .628 accuracy with k = .458, For the five aspectual verb classes used as the followed by the combinations subject-direct gold standard (Richter and van Hout, 2014) it object and subject-aspectual features with .6 turned out that noun classes in subject positions accuracy each and k = .495. These combinations have the highest predictive power compared to exhibit a moderate agreement. Again, the the nouns in the remaining argument positions aspectual feature performs poorly with .428 and the aspectual features derived from Vendler accuracy, k = .266 which is a fair agreement. In (1967). This result is surprising since the figure 2 the accuracy of the argument and Vendlerian aspectual categories were formulated aspectual features for the comparisons against in order to distinguish aspectual classes. Future both gold standard classifications are given. research should explore a comparison of the predictive power of nominal and aspectual features. Using a classification into concrete lexical fields as the gold standard of the predictive values we observed a considerable decrease in the predictive values indicated by the lower kappa values. We explain this result by the difference in information provided by the argument structures of the verbs in the 5 class-gold standard classification in contrast to the information provided by co-occurrences that is, lexical information of any type in the context of verb in the 10-class gold standard classification. The results of this study show that: 1. Aspectual verb classes can be empirically validated, 2. Classified nouns in subject argument positions are reliable predictors of aspectual verb classes, i.e. the meaning of nouns in combination with their noun classes correlates with aspectual parts of the verbal meaning. In order to confirm these results further research with an extended test set of verbs is needed. Acknowledgments Roeland van Hout suggested to evaluate the classifications with Cohen‘s Kappa and was very helpful in the calculations of the k-values. 180 References Zeno Vendler 1967. Linguistics in Philosophy. Ithaka /New York: Cornell University Judith Aissen 2003. Differential Object Marking: Press. Iconicity vs. economy. Natural Language and Warren Weaver 1955. Translation. In W. N. Linguistic Theory, 21: 435 – 483. Locke, D.A. Booth, Machine Translation Bernd Bohnet 2010. Very High Accuracy and of Languages: 15 – 23. Cambridge Mass.: Fast Dependency Parsing is not a MIT Press. Contradiction. The 23rd International Conference on Computational Linguistics (COLING 2010), Beijing, China. William Croft 2003. Typology and Universals. Cambridge University Press (2nd edition). George W. Furnas, Thomas K. Landauer, Louis M. Gomez, and Susan T. Dumais 1983. Sta- tistical semantics: Analysis of the potential performance of keyword information systems. Bell System Technical Journal, 62 (6): 1753 – 1806. Wolfgang Klein 2009. How time is encoded. In W. Klein and P. Li (eds.), The expression of time, 39 – 82. Berlin: Mouton de Gruyter. Thomas K. Landauer and Susan T. Dumais 1997. A solution to Plato's problem: The La- tent Semanctic Analysis theory of the acquisi- tion, induction, and representation of knowledge. Psychological Review, 104: 211 – 140. Patrick Pantel 2005. Inducing ontological co- occurrence vectors. In Proceedings of As- sociation for Computational Linguistics (ACL-05): 125 – 132. Michael Richter and Roeland van Hout 2015. A classification of German verbs using empirical language data and concepts of Vendler and Dowty. To appear 2015 in Sprache und Datenverarbeitung – International Journal for Language Data Processing. Herbert Rubenstein and John B. Goodenough 1965. Contextual correlates of synonomy. In Communications of the ACM, Vol 8 (10): 627 – 633. Helmut Schumacher 1986. Verben in Feldern. Valenzwörterbuch zur Syntax und Seman- tik deutscher Verben. Berlin & New York: De Gruyter. Hinrich Schütze and Jan Pedersen 1993. A vector model for syntagmatic and paradigmatic relatedness. In Making Sense of Words: 104 – 113. Ninth Annual Conference of the UW Centre for the New OED and Text Research, Oxford. Peter D. Turney and Patrick Pantel 2010. From Frequency to Meaning: Vector Space Models of Semantics. Journal of Artificial Intelli- gence Research, 37: 141 – 188. 181