A Cognitive Linguistics analysis of Phrasal Verbs' representation in Distributional Semantics

Abstract

Phrasal Verbs (PVs) constitute a peculiar feature of the English language and represent a challenge for both language learners and computational models because of their complex and idiomatic nature, which has made them appear unsystematic and unpredictable. Recently, Cognitive Linguistics has ofered a more systematic explanation of the semantics of PVs by relating their non-compositional meanings to the metaphorical extensions of the particle's meaning. In order to assess the computational suitability of this approach using Distributional Semantics, we analyzed three diferent semantic spaces to understand how PVs and particles are represented and whether any of the embeddings capture the significance of particles in the semantics of the entire construction. The results indicate that phrase embeddings are efective in representing the meanings of PV constructions, while word embeddings excel at capturing particle meanings and additionally support the Cognitive Linguistics hypothesis. Since improving the semantic representation of PVs can benefit various NLP applications, further research is necessary to validate these findings.

Keywords

• eol>Phrasal Verbs
• Cognitive Linguistics
• Distributional Semantics

1. Introduction

lar its metaphorically extended meaning [ 9 ]) that plays a crucial role in shaping the overall meaning of the PV [ 2 ].

Phrasal Verbs (PVs) represent a distinctive peculiarity of Given that this approach has shown promising results in the English language and are defined as a lexicon unit language learning [ 10 ], and that computational models composed of a verb (e.g. look) and a particle (e.g. out), of language face dificulties that are similar to English whose meaning is often non-compositional (e.g. look out as a Second Language (ESL) learners in understanding means ’to beware’) [ 1, 2 ]. PVs comprise a significant por- the semantic complexity of PVs, we wanted to examine tion of the verb vocabulary [ 3 ] and are commonly used in whether such Cognitive Linguistics account holds also everyday language, particularly in spoken and informal from a Distributional Semantics perspective, where the contexts [ 4, 5 ]. In addition, they are highly productive, representation of words’ meanings and semantic relawith new ones continually being coined to reflect societal tionship have repeatedly been proved to be similar to the changes (e.g. google up) [ 6 ]. They are also characterized way they are represented in the human cognitive system by their polysemy, with each phrasal verb having mul- [ 11 ]. With this aim, we analysed three diferent semantic tiple meanings on average [ 5 ]. To further enhance their spaces –word embeddings, phrase embeddings and POScomplexity, for a long time, linguists and grammarians tagged embeddings– to determine the most accurate way have claimed that the selection of verb and particle in of representing PVs and particles and whether the Cogthe PV construction is totally unsystematic and unpre- nitive Linguistics hypothesis was accounted for in any dictable [ 6, 7, 8 ], therefore the traditional pedagogical of them. The importance of the particle in shaping the approach to PVs has always been based on memorization meaning of PVs was confirmed but the results appeared of the verb-particle combination and the corresponding to vary across the diferent semantic spaces, suggesting meaning causing a general discouragement of learners the need for further and more detailed research. around PVs. Recent research in Cognitive Linguistics, however, has proposed a more systematic explanation of the association between these verb-particle combina- 2. Related Works tions and their apparently randomly assigned idiomatic meaning, by suggesting that it is the particle (in particu

Phrasal Verbs have for long been a hot topic among linguists and lexicographers who have largely debated on their definition and classification, proposing various

CLiC-it 2023: 9th Italian Conference on Computational Linguistics, theories based on their syntactic and semantic features N$ovm3e0li—ssaD.deocn0a2t,i2@02st3u,dVieon.uicnei,bItoa.ilty(M. Donati); [e-mail] [ 6, 12, 4, 13, 14 ]. Corpus linguistics has also played a cru(C. Strapparava) cial role in studying PVs, providing insights into their fre© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License quency and meaning distribution, thus aiding language CPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g ACttEribUutRion W4.0oInrtekrnsahtioonpal (PCCroBYce4.0e).dings (CEUR-WS.org) teacher by identifying the most frequently used PVs and words can be measured as the geometrical distance betheir meanings [ 4, 5, 15, 16, 17 ]. However, to develop tween the vectors representing such words (for a more efective teaching strategies for PVs, it is essential to con- detailed explanation of the diferent frameworks see [ 11 ]; sider the cognitive processes involved in storing and re- [29]). Even though this approach works extremely well trieving these structures from the mental lexicon and that for representing the meaning of single words, it faces is where the Cognitive Linguistics approach comes into some challenges when representing the meaning of PVs play ofering a new perspective that considers PVs as con- and multi-word expressions more in general due to their ceptually motivated constructions rather than arbitrary non-compositional and often polysemous meaning. Most combinations [ 18, 19, 20, 2, 21 ]. This account is based on studies addressed this issue by developing strategies to one of the cornerstones of Cognitive Linguistics which detect compositionality using dictionary-based [30, 31] is Metaphor Theory. According to this view, metaphors and distributional similarity methods [32, 33, 34]. While play a fundamental role in conceptualization and think- providing efective working solutions, these compensaing as they allow us to understand and experience ab- tion strategies do not fix the root problem at the level stract concepts by mapping them onto concrete entities of semantic representation. Recently, DS has been exthat we can bodily perceive [ 9 ]. In the case of PVs, the tended to incorporate larger units, such as multi-word Cognitive Linguistics account considers the metaphorical expressions and phrases, thus creating more informative extension of the particle’s literal meaning as responsi- embeddings and leading to better performance in NLP ble for the idiomatic meaning of the entire construction tasks [35, 36, 37, 38]. It is against this background that [ 2, 22 ], unlike traditional approaches that often neglect we framed our research questions and decided to investhe semantic role of particles, and function words more tigate which type of embeddings could better represent in general [23]. According to this view, the prototypical the complex semantics of PVs and whether the distrimeaning of particles, which is usually related to spatial lo- butional semantic space manages to capture the role of cations and orientations, can be metaphorically extended the particle’s meaning in shaping the meaning of the ento abstract non-physical domains that are thought of in tire construction, as posited by the Cognitive Linguistics terms of space, such as attitudes, knowledge, completion, account. or increase [ 2 ]. For example, the particle “UP" literally denotes a physical upward motion (e.g. to pick up), but can be used to denote a number of abstract domains that 3. Methodology we categorize assigning (spatial) values along a vertical line such as temperature, ranks, attitudes, knowledge etc. In order to investigate how the semantics of PVs is repTherefore, the metaphorical extension of ’UP’ can indi- resented within the Distributional Semantics framework, cate improvement (e.g. to brush up), higher visibility and and more specifically to test whether distributed representations can capture the importance of particles in PV accessibility (e.g. to turn up), completion (e.g. to fill up ), and reaching a a boundary (e.g. to be fed up) [ 2 ]. This per- constructions, as suggested by the Cognitive approach, spective allows to get insights into the systematicity and we analyzed three types of embeddings: predictability of PVs’ semantics and demonstrates that id- Word embeddings: we selected the pre-trained word veciomatic and polysemous meanings of PVs are connected tors released by Google1, which are 300-dimensional vecthrough a network of senses derived from the prototypi- tors trained using a Skip-gram model on a portion of the cal meaning of the particle. Empirical studies have shown Google News dataset, with a window-size of 5. The Skipthat adopting this more cognitively plausbile approach gram model was selected because it has shown superior in the instruction of PVs can benefit ESL learners as it performance in semantic tasks compared to other models helps them grasp the relationship between idiomatic and like CBOW, NNLM, and RNNLM [39]. The window-size literal meanings of PVs thus facilitating the processing of 5 allows capturing broader semantic information beand acquisition of these constructions [ 24, 25, 26, 27, 10 ]. yond immediate context, which is suitable for investigatHaving briefly discussed the Cognitive Linguistics per- ing the semantics of PVs with separable particles [40]. spective concerning the semantics of PVs, we now briefly The vector size of 200-300 dimensions strikes a balance explore how the meaning of PVs is processed computa- between informativeness and computational complexity tionally, adopting a Distributional Semantics approach. [41].

Distributional Semantics is a computational approach to language meaning where words are represented as Phrase embeddings: we selected the embeddings for distributional vectors in a semantic space based on their generalized phrases introduced by [38]. They collected contextual usage. The underlying assumption, referred two-word phrases, categorized them as continuous or to as the Distributional Hypothesis [28], is that words discontinuous, and trained a Skip-gram model to learn that occur in similar contexts tend to purport similar 1https://drive.google.com/file/d/0B7XkCwpI5KDYNlNUTTmeanings and that the semantic similarity between two lSS21pQmM/edit?resourcekey=0-wjGZdNAUop6WykTtMip30g embeddings for both words and phrases. They showed that phrase embeddings outperformed word embeddings in semantic tasks, demonstrating their better representative power for such multi-word expressions, because considering them as linguistic units allows to capture the attributes of their real contexts of usage and thus to create accurate semantic representations that account for their non-compositional meaning.

We conducted exploratory analyses on the three diferent semantic spaces to investigate how PV constructions and particles are distributionally represented. We selected as target verbs the 150 most frequent PVs identified by previous corpus-based studies [ 15 ]. To determine the meanings of these PVs, we referred to the PHaVE List [ 16 ], which provides key meaning senses based on frequency distributions. Similarly, we addressed the challenge of representing particle meanings, which are often overlooked in Distributional Semantics, by turning to [ 2 ] work for detailed meaning descriptions of particles and we selected simplified synonyms to evaluate the accuracy of particles’ semantic representations. For both PVs and particles, we evaluated their meaning representation using cosine similarity measures. The final lists of particle meanings used in our analysis can be found in Appendix A, while in Appendix B we reported a sample of PVs that were used in our analysis with the corresponding meanings selected from the PHaVE List [ 16 ]. Having deifned the meaning of reference for PVs and particles, we designed three types of analyses to answer our research questions.

4.3. Verb vs particle in the semantic representation of PVs

In order to test whether the distributional representations of meanings successfully capture the cognitive peculiarity of PVs’ semantics, specifically the fact that particles play a significant role in shaping the meaning of the entire construction compared to the verb proper [ 2 ], we compared the similarity between the particle and the verb proper with the whole PV construction. For instance, considering the PV set up if the distributional representation efectively captures the Cognitive Linguistics account 5.2. Semantic representation of particles of PVs’ semantics, we would expect the cosine similarity score between the entire PV and the particle (sim(set When evaluating the similarity between the particles up–up)) to be higher than that between the PV and the and their meanings, we obtained similarity scores that verb proper (sim(set up–set)). In other words, the vector were overall below the 0.5 significance threshold across representing the meaning of the PV should be more sim- the three types of embeddings. Surprisingly, comparing ilar, and therefore closer, to the vector representing the the results, word embeddings performed best, followed particle than to the vector representing the verb proper. by phrase embeddings, while POS-tagged embeddings showed very poor performance (see Table 2 for a sample of the results). This was unexpected because it is only 5. Results in POS-tagged embeddings that we could represent particles in isolated vectors, therefore they were supposed Since one of the primary objectives of this work was to to capture more precisely their meaning. Conversely, in understand what could be the most appropriate way to word embeddings the meaning representation of partibuild a distributional semantic representation of PVs that cles was collapsed in a single vector that included also truthfully accounts for their complex semantics, we will occurrences of the same words when used with diferent now briefly 2 present and discuss the results of the three syntactic functions (i.e. prepositions or adverbs), while types of analyses that were carried out comparing the in phrase embeddings the vectors representing the partiresults obtained with the three semantic spaces. cle was actually built excluding the occurrences of the words as particles because those where captured within 5.1. Semantic representation of PVs the phrase-vectors themselves. These findings point to two possible conclusions which are not mutually exclusive but rather complementary: on the one hand, they suggest that for building distributional representation of words that occur frequently in diferent syntactic roles –such as particles, and function words more in general– collapsing all the occurrences within a single vector representation might lead to better capturing their meaning, and on the other, they hint that particles used in PVs may have a core (prototypical) meaning that transcends their syntactic role, aligning with the Cognitive Linguistics hypothesis [ 2 ].

When evaluating the similarity between the PVs and their

meanings to assess the quality of the semantic representation, overall significance was not high in any embeddings. However, phrase embeddings outperformed the others as they showed higher similarity scores for PV-meaning pairs (32%) compared to word (27%) and POS-tagged (17%) embeddings (see Table 1 for an example). This might be explained by the fact that phrase embeddings treat PVs as a single unit, capturing their real contexts of usage and therefore represent their semantic complexity more accurately. Conversely, word and POS-tagged embeddings, which summed the verb and particle vectors, fell short in capturing the full meaning of PVs. In conclusion, phrase embeddings proved to be the most suitable for representing PV semantics.

5.3. Verb vs particle in the semantic representation of PVs

In the final analysis, we compared the similarity between the PV and the particle and the PV and the verb across the three types of embeddings to test whether any of them

2For the sake of brevity, just a sample of the results is reported herein,

for a comprehensive overview and a more detailed analysis see [45].

Table 3 outperformed both phrase and POS-tagged embeddings. Sample of our results showing the similarity scores obtained These results were unexpected because we anticipated with the three types of embeddings for the verb take out com- better performance from POS-tagged embeddings, which pared to the verb proper (take) and the particle (out). The were designed to isolate particle occurrences. We idenhighest similarity scores, highlighted in bold, are obtained tified diferent possible explanations for these results, with word embeddings. including limitations in integrating POS-tag information

Similarity scores (that reduces the number of occurrences) compared to PV Embeddings PV – v PV – prt including occurrences with diferent syntactic functions, word 0.784 0.714 the issues related to the general dificulty of representtake out phrase 0.476 0.373 ing function words and also the possibility that words POS-tagged 0.645 0.685 used as particles carry a unique core meaning regardless of their syntactic functions. Further research is needed, along these lines, to disentangle these factors and undersupported the Cognitive Linguistics hypothesis about the stand what could be the most efective way to represent role of particles in PV semantics. Broadly speaking, as particles and function words in Distributional Semantics. expected word and POS-tagged embeddings performed Finally, the results of the third and last type of analbetter than phrase embeddings. They both gave over- ysis, that was designed precisely to test the Cognitive all significant similarity scores but showed the opposite Linguistics claim on the role of the particle, showed that patterns of similarity (see Table 3 for an example). In- when separate vector representations are built for partideed, while in word embeddings it is the verb proper that cles, i.e. distinguishing the occurrences of the same words resulted to be more similar to the PV compared to the with other syntactic functions, as was the case with POSparticle, the opposite is true for POS-tagged embeddings tagged embeddings, particles do appear to play a greater where it is the particle that resulted to be more similar to role than the verbs proper in the semantics of the PVs. the PV compared to the verb proper. These contrasting Conversely, when the vector representation of particles is patterns align respectively with the traditional view and less accurate, i.e. includes occurrences of the same words with the Cognitive Linguistics view (Section 2) on PVs with other syntactic functions, as it was the case with meaning, and suggest that in a semantic space in which word embeddings, it is the verb proper that appears to particles are accurately (i.e. separately) represented, the be more crucial in the semantics of the PV in most cases. Cognitive Linguistic view claiming the higher signifi- Overall our findings align with the literature, in that cance of the particle (vs the verb proper) in shaping the they support the idea that vectors for PVs should be PVs meaning, is supported and accounted for. treated as single tokens rather than splitting them into individual words [38] and that representing the meaning 6. Conclusion and Future of particles is challenging, justifying their removal in many NLP applications [46]. However, we believe that Directions understanding how to build appropriate semantic representations for particles is crucial for analyzing their The aim of this work was to analyze the distributional contribution to larger constructions, such as PVs. In orrepresentation of PVs from a Cognitive Linguistics per- der to do so, future studies can explore diferent types spective. More specifically we wanted to examine three of embeddings and testing whether refining POS-tagged diferent semantic spaces (word embeddings, phrase em- embeddings (for example by weighting each POS-tag feabeddings and POS-tagged embeddings) using simple vec- ture according to the task or alternatively using Neural tor combination (sum) and mathematical computations Networks for combining these features into a unique (cosine similarity) to evaluate whether: 1) the meaning meaningful hidden representation) could improve repreof the PV construction is properly represented; 2) the sentation accuracy and thus lead to better performance particles’ embeddings truthfully capture their meaning; of the models in specific semantic tasks. 3) the greater role of the particle in shaping the seman- Last but not least, our results provide initial evidence tics of the PV, as posited by the Cognitive Linguistics supporting the Cognitive Linguistics account of PV seapproach, is accounted for. The current results showed mantics from a Distributional Semantic perspective, althat, as expected, phrase embeddings performed best in though further confirmation is needed. Adopting the capturing the complex semantics of PVs, supporting the Cognitive Linguistics approach to PVs in education and idea of treating PVs as single tokens when training the leveraging NLP applications in this direction can facilembeddings so as to capture the true context of occur- itate the acquisition of this complex English structure rence and obtain more accurate meaning representation for ESL learners. Additionally, capturing and representthat account also for the less compositional meanings. ing PV meanings and the semantic roles of their com

As far as particles are concerned, word embeddings ponents can benefit NLP tasks involving semantic and morphosyntactic relations (such as machine translation, question-answering, summarization, automatic synonym detection, etc.). For these reasons, we hope this work stimulates further advanced research in this area, leveraging the insights from Cognitive Linguistics and Computational Linguistics.

A. Appendix A. Particle’s meanings

List of particles’ meanings adapted from [2] that were used for the analysis.

Particles on up back out in down of ahead over a/round through about along

Meanings contact/continuation positive verticality/increasing/completing returning/past leaving/exhaustion entering/being inside negative verticality/decreasing/ending separation progressing crossing/overcoming vicinity/proximity crossing/completing dispersion parallel/accompanying

B. Appendix B. Sample of PVs’ meanings

Small sample of PVs-meanings pairs as extracted from the PHaVE list [16].

PV get up take out go down look out Meaning 1 rise remove move observe / contemplate give out give Meaning 2 Meaning 3 invite decrease take care / protect make public obtain go collapse / fail

Acknowledgements

We acknowledge the support of the PNRR project FAIR Future AI Research (PE00000013), under the NRRP MUR program funded by the NextGenerationEU.

References

[1] S. Thornbury , How to teach vocabulary, Pearson Education India , 2006 .

[2] B. Rudzka-Ostyn , Word power: Phrasal verbs and compounds: A cognitive approach , Walter de Gruyter, 2008 .

[3] W. Li , X. Zhang , C. Niu , Y. Jiang , R. K. Srihari , An expert lexicon approach to identifying english phrasal verbs , in: Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics , 2003 , pp. 513 - 520 .

[4] D. Biber , S. Johansson , G. Leech, S. Conrad , E. Finegan, Longman grammar of spoken and written english , 1999 .

[5] D. Gardner , M. Davies , Pointing out frequent phrasal verbs: A corpus-based analysis , TESOL quarterly 41 ( 2007 ) 339 - 359 .

[6] D. L. M. Bolinger , The phrasal verb in English, Cambridge University Press, 1971 .

[7] L. Lipka , Semantic structure and word-formation ( 1972 ).

[8] K. A. Sroka , The syntax of English phrasal verbs , volume 129 , Walter de Gruyter GmbH & Co KG , 2016 .

[9] G. Lakof , M. Johnson, Metaphors we live by, University of Chicago press, 2008 .

[10] B. J. White , A conceptual approach to the instruction of phrasal verbs , The Modern Language Journal 96 ( 2012 ) 419 - 438 .

[11] A. Lenci , et al., Distributional semantics in linguistic and cognitive research , Italian journal of linguistics 20 ( 2008 ) 1 - 31 .

[12] R. Quirk , S. Greenbaum , G. Leech, J. Svartvik , A comprehensive grammar of the english language: Longman group limited ( 1985 ).

[13] M. C. Murcia , D. L. Freeman , The grammar book , Heinle & Heinle Publishers, 1999 .

[14] C. M. Darwin , L. S. Gray , Going after the phrasal verb: An alternative approach to classification , Tesol Quarterly 33 ( 1999 ) 65 - 83 .

[15] D. Liu , The most frequently used english phrasal verbs in american and british english: A multicorpus examination , Tesol Quarterly 45 ( 2011 ) 661 - 688 .

[16] M. Garnier , N. Schmitt , The PHaVE list: A pedagogical list of phrasal verbs and their most frequent meaning senses , Language Teaching Research 19 ( 2015 ) 645 - 666 .

[17] D. Liu , D. Myers , The most-common phrasal verbs with their key meanings for spoken and academic written english: A corpus analysis , Language Teaching Research 24 ( 2020 ) 403 - 424 .

[18] S. J. Lindner , A lexico-semantic analysis of English verb particle constructions with out and up , Uni-