Electrophysiological correlates of idiom comprehension: semantic composition does not follow lexical retrieval

-

Idiomatic expressions, such as break the ice, are pervasive in everyday communication. They are frequently co-occurring sequences of words with a conventional meaning that is not derived from word-by-word semantic composition, but rather can be retrieved as such from semantic memory. Idioms are often read faster compared to literal sentences [e.g., Siyanova-Chanturia et al., 2011 ] and also lexical decision times are faster on idiom related words than on literal related targets [e.g., Cacciari & Tabossi, 1988 ]. Recent EEG data further suggest that semantic composition processes of idiomatic constituents might be not fully engaged during comprehension [Rommers et al, 2013]. Finally brain-imaging studies reported stronger and more widespread activation of the language network when reading idioms compared to non-idiomatic sentences [ Zempleni et al., 2007 ; Lauro et al., 2008 ; Boulenger et al., 2009 ], suggesting that idiom comprehension might involve more cognitive resources. From these fragmented results, it is not clear yet how idiomatic semantic processing differs from literal semantic processing and this might be due to the paradoxical nature of idioms [e.g., Libben & Titone, 2008 ], which seem to be at the same time amenable of direct memory retrieval and wordby-word compositional analysis.

The two main questions of the present research thus concern two aspects of idiom comprehension: one relates to how the meaning of the whole is retrieved and integrated in the sentence representation; the second relates to what happens to word-by-word semantic composition of the literal meanings of the expression: is it carried out or suspended? To answer these questions we used EEG measures (with the analysis of Event-Related Potentials and oscillatory dynamics of Time-Frequency representations) because of their temporal precision [e.g., Luck, 2014 ], and because of the possibility of disentangling between memory gling between memory retrieval and semantic integration processes [e.g., Hoecks & Brower, 2014]. 2

The present Study

We carried out two Experiments in which short and literally plausible idioms (e.g., break the ice), i.e. having a literal well-formed meaning and a conventional meaning, were embedded in literal or idiomatic contexts. Notably, materials were designed in such way that the sentential context would constrain expectations on the upcoming target words to a similar extent across conditions. By doing so we minimized the impact of differential sentence constraints, known to elicit N400 effects, and we carried out a comparison between sentences that were semantically well-formed and for which contextual expectations on upcoming words were always fulfilled. Experiment 1 used EEG measures as dependant variable to investigate the time course of idioms comprehension and was followed up by Experiment 2 in which a cross modal priming paradigm was implemented, in order to confirm the activation of the literal meaning of the idiomatic constituents in both types of contexts.

On the basis of the previous ERP literature we hypothesized that meaning retrieval processes would affect the N400 component [e.g., Federmeier, 2007 ]: more demanding retrieval processes should be associated to larger N400 effects. The debate about the role of the N400 in semantic integration vs. retrieval mechanisms [see semantic unification processes in Hagoort & Van Berkum, 2007 ] makes it hard to exclude that the N400 component is not associated with the semantic integration of the meaning of the whole; however, given the available evidence on figurative language processing, we could also expect an effect on later occurring positivities, previously associated with metaphor (Late Positive Complex, LPC) [e.g., Coulson & Van Petten, 2002 ; Lai et al., 2009 ] or irony (P600) [ Regel et al., 2010 ] processing, or semantic pragmatic reanalysis (frontal Post-N400 Positivity) [e.g., Van Berkum et al., 2009 ; Molinaro et al., 2012 ]. Another result that has been previously reported in the ERP literature of idioms processing is the finding of an involvement of the P300 component. The P300 is generally associated with cognitive mechanisms of context update [ Donchin & Coles, 1988 ] or context closure [ Verleger, 1988 ]: Vespignani et al. (2010) found that the brain’s electrical response to the correct idiom constituent was different if recorded before or after the idiom recognition point (RP, e.g., prendere il toro per leRP … corna -- take the bull by theRP … horns). The match to the correct idiom word was associated with an N400 reduction before recognition, but the electrophysiological response led to a P300 effect after the recognition of the idiom. Such effect would mirror a qualitative change in readers’ expectations about upcoming words, after the expression has been recognized. We also expected to replicate Rommers et al (2013) results in the time-frequency domain of the EEG. The authors observed a power increase in the upper gamma frequency band after the presentation of the expected target words in literal but not in idiomatic contexts, supporting the hypothesis that semantic unification mechanisms are less engaged in idioms comprehension. 3 3.1

Method

Participants 380 students at Università degli studi di Modena e Reggio Emilia participated to the study set up to norm the experimental materials. 32 different students took part in Experiment 1. 42 students volunteered in experiment 2. 3.2

Materials

Experiment 1 materials were 90 idiomatic expressions of similar structure (VP+NP idioms) embedded in sentences. Idioms were selected for being highly Familiar and correctly paraphrased. Three sentential contexts for each expression were created so that the last word of the expression was highly predictable in the three contexts (above 85% cloze probability). ERPs were timelocked to the presentation of the first word of the expression (W1), and epochs comprising W1, W2 and W3 were extracted from the EEG. In Experiment 2 a subset of 44 idioms was used. 1a) La maestra aveva notato che Nicola disturbava i compagni, ma la prima volta chiuse un occhio e continuò la lezione. (The teacher saw Nick was bothering his desk mate but for the first time she closed an eye (turned a blind eye) and kept on teaching.) 1b) Alla visita oculistica Enrico, prima di leggere le lettere indicate sulla lavagna luminosa, chiuse un occhio per valutare la miopia. (At the Ophthalmological visit, before starting to read the letters on the panel aloud Henry closed an eye in order to evaluate his nearsightedness.) 1c) Giovanni ha rotto gli occhiali durante la rissa perché ha preso un pugno in un occhio e gli sono caduti a terra. (Jack broke his glasses during the fight because got a punch in his eye and fell on the ground.) 3.3

Procedure

In Experiment 1 sentences were presented wordby-word at the centre of the screen (SOA=600ms). In Experiment 2, contexts sentences were auditorily presented via headphones until the last word of the expression. Targets that could be related or unrelated to the literal meaning of the last word of the expression were visually presented at the offset of the audio file. 4

Results

expression (e.g., ice).

Experiment 1 showed that: - No N400 differences emerged between literal and idiomatic context, during the processing of the three constituent words. - Differences between Idiomatic vs. Literal, and Idiomatic vs. Control conditions emerged during the presentation of the last word of the expression (e.g., ice), and occurred in the 400 to 600 ms time interval. - Consistently with Rommers et al (2013) study, the Time-Frequency analysis of the EEG revealed power differences in the higher gamma frequency band (60-80Hz) between expressions embedded in literal vs. idiomatic contexts: no power increase was associated with the idiomatic condition.

Experiment 2 showed that: - Target words related to the literal meaning of the idiomatic constituents obtained faster lexical decision times with respect to unrelated targets, regardless of type of context. 5

Discussion

Concerning the question related to how the meaning of the whole idiom is integrated in the sentence representation, our results suggest that integration mechanisms occur only upon presentation of the last constituent word, when the idiomatic expression has very likely been recognized. On the last constituent, ERP differences between idiomatic and literal contexts emerged between 400 and 600 ms in frontal electrodes. The timing and scalp distribution of the effect suggest that it affected a positive component (the frontal Post-N400 Positivity) occurring soon after the peak of the N400. These results could be accommodated elaborating the framework proposed by the Retrieval-Integration hypothesis [Hoecks & Brower, 2014], which holds that semantic - pragmatic integration processes are reflected in P600 like positivities. One possible interpretation is that the observed frontal positive shift might be part of a larger family of positive components reflecting the engagement of a semantic/pragmatic wrap-up mechanism that is performed at end of the expression to assign a full interpretation to the incoming input.

Concerning the second experimental question related to the composition of individual constituent words we argue that Experiment 2 showed that the literal meaning of the last word of the expression was at least accessed, and confirms other evidence supporting the idea that readers process the literal meaning of idiomatic constituents (Boulenger, Shtyrov & Pulvermüller, 2012) . Moreover, the lack of N400 differences across conditions and word positions, suggests that lexical retrieval processes similarly occurred in literal and idiomatic contexts. However, the analysis of the frequency domain replicated Rommers et al’s findings of a larger power increase in the high gamma frequency band for literal compared to idiomatic contexts, which, consistently with their interpretation, could signal that word-by-word composition mechanisms are less engaged in idioms comprehension.

Conclusions

When presented with idiomatic expressions readers retrieve the literal meaning of the constituent words. However, word-by-word semantic composition mechanisms are idling, and, only at the end of the expression, a semantic/pragmatic wrap-up of the idiom is carried out to update the sentence representation.

References

Siyanova-Chanturia , A. , Conklin , K. , & Schmitt , N. ( 2011 ). Adding more fuel to the fire: An eye-tracking study of idiom processing by native and non-native speakers . Second Language Research , 27 ( 2 ), 251 - 272 .

Cacciari , C. , & Tabossi , P. ( 1988 ). The comprehension of idioms . Journal of Memory and Language , 27 ( 6 ), 668 - 683 .

Zempleni , M.- Z. , Haverkort , M. , Renken , R. , & A. Stowe , L. ( 2007 ). Evidence for bilateral involvement in idiom comprehension: An fMRI study . NeuroImage , 34 ( 3 ), 1280 - 1291 .

Lauro , L. J. R. , Tettamanti , M. , Cappa , S. F. , & Papagno , C. ( 2008 ). Idiom Comprehension: A Prefrontal Task? Cerebral Cortex , 18 ( 1 ), 162 - 170 .

Boulenger , V. , Hauk , O. , & Pulvermüller , F. ( 2009 ). Grasping Ideas with the Motor System: Semantic Somatotopy in Idiom Comprehension . Cerebral Cortex , 19 ( 8 ), 1905 - 1914 .

Rommers , J. , Dijkstra , T. , & Bastiaansen , M. ( 2012 ). Context-dependent Semantic Processing in the Human Brain: Evidence from Idiom Comprehension . Journal of Cognitive Neuroscience , 25 ( 5 ), 762 - 776 .

Libben , M. R. , & Titone , D. A. ( 2008 ). The multidetermined nature of idiom processing . Memory & Cognition , 36 ( 6 ), 1103 - 1121 .

Luck , S. J. ( 2014 ). An Introduction to the EventRelated Potential Technique . MIT Press.

Hoeks , J. C. J. and Brouwer , H. ( 2014 ). Electrophysiological Research on Conversation and Discourse Processing . In: Holtgraves, T. (Ed.), Oxford Handbook of Language and Social Psychology , pp. 365 - 386 . New York: Oxford University Press.

Hagoort , P. , & Berkum , J. van. ( 2007 ). Beyond the sentence given . Philosophical Transactions of the Royal Society B: Biological Sciences , 362 ( 1481 ), 801 - 811 .

Federmeier , K. D. ( 2007 ). Thinking ahead: The role and roots of prediction in language comprehension . Psychophysiology , 44 ( 4 ), 491 - 505 .

Coulson , S. , & Petten , C. V. ( 2002 ). Conceptual integration and metaphor: An event-related potential study . Memory & Cognition , 30 ( 6 ), 958 - 968 .

Lai , V. T. , Curran , T. , & Menn , L. ( 2009 ). Comprehending conventional and novel metaphors: An ERP study . Brain Research , 1284 , 145 - 155 .

Regel , S. , Gunter , T. C. , & Friederici , A. D. ( 2010 ). Isn't It Ironic? An Electrophysiological Exploration of Figurative Language Processing . Journal of Cognitive Neuroscience , 23 ( 2 ), 277 - 293 .

Berkum , J. J. A. V. , Holleman , B. , Nieuwland , M. , Otten , M. , & Murre , J. ( 2009 ). Right or Wrong? The Brain's Fast Response to Morally Objectionable Statements . Psychological Science , 20 ( 9 ), 1092 - 1099 .

Molinaro , N. , Carreiras , M. , & Duñabeitia , J. A. ( 2012 ). Semantic combinatorial processing of non-anomalous expressions . NeuroImage , 59 ( 4 ), 3488 - 3501 .

Donchin , E. , & Coles , M. G. H. ( 1988 ). Is the P300 component a manifestation of context updating? Behavioral and Brain Sciences, 11 ( 03 ), 357 - 374 .

Verleger , R. ( 1988 ). Event-related potentials and cognition: A critique of the context-updating hypothesis and an alternative interpretation of the P300 . Behavioral and Brain Sciences , 11 , 343 - 427 .

Vespignani , F. , Canal , P. , Molinaro , N. , Fonda , S. , & Cacciari , C. ( 2009 ). Predictive Mechanisms in Idiom Comprehension . Journal of Cognitive Neuroscience , 22 ( 8 ), 1682 - 1700 .

Boulenger , V. , Shtyrov , Y. , & Pulvermüller , F. ( 2012 ). When do you grasp the idea? MEG evidence for instantaneous idiom understanding . NeuroImage , 59 ( 4 ), 3502 - 3513 .