In our laboratory, we have attempted to measure subtle differences in human subjects partially by using the richness of Hebrew in both homophonic and heterophonic homographs (in standard orthography Hebrew is written without vowels) and measuring the difference in response when presenting homographs directly to one hemisphere or the other. To compare our human results with computational ones, we designed and present here a connectionist (neural network) model of each hemisphere for lexical disambiguation based on the well-known Kawamoto [1993] model.
Our model includes two separate networks, one for each hemisphere. One network incorporates Kawamoto's version in which the entire network is completely connected. (Thus orthographic, phonological and semantical "neurons" are not distinguished architecturally.) This network successfully simulated the time course of lexical disambiguation in the Left Hemisphere. In the other network, direct connections between orthographic and phonological units are removed. The speed of convergence in resolving ambiguities were studied in these two networks under a variety of conditions simulating various kinds of priming. The comparative results presented are analagous to the results obtained under our human subject testing thereby strengthening our belief in the correctness of our psychological explanation of the processing.
Neuropsychological studies have shown that both cerebral hemispheres process orthographic, phonological and semantic aspects of written words, albeit in different ways. Behavioral studies have shown that the LH is more influenced by the phonological aspect of written words whereas lexical processing in the RH is more sensitive to visual form. In addition, semantically ambiguous words (e.g., "bank") were found to result in different time-lines of meaning activation in the two hemispheres. However, computational models of reading in general and of lexical ambiguity resolution in particular, have not incorporated this asymmetry into their architecture.
A large amount of psycholinguistic literature indicates that readers utilize both frequency and context to resolve lexical ambiguity [e.g., Duffy, Morris & Rayner 1988; Titone 1998; Peleg, Giora & Fein 2001, 2004]. The idea that multiple sources of evidence (relative frequency as well as context) affect the degree to which a particular meaning is activated and the eventual outcome of the resolution, as well as the process, can be nicely captured within a neural network (connectionist) approach to language processing. In connectionist terminology, the computation of meaning is a constraint satisfaction problem: the computed meaning is that which satisfies the multiple constraints represented by the weights on connections between units in different parts of the network. 2.1
A connectionist account of lexical ambiguity resolution was presented by Kawamoto [1993]. In his fully recurrent network, ambiguous and unambiguous words are represented as distributed pattern of activity over a set of simple processing units. Each lexical entry is represented over a 216 - bit vector divided into separate sub-vectors representing the “spelling”, ”pronunciation”, "part of speech" and “meaning”. The network is trained with a simple error correction algorithm by presenting it with the pattern to be learned. The result is that these patterns (the entire word including its orthographic, phonological and semantic features) become attractors in the 216-dimensional representational space. The network is tested by presenting it with just part of the lexical entry (e.g., its spelling pattern) and testing how long various parts of the network take to settle into a pattern corresponding to a particular lexical entry. Kawamoto trained his network in such a way that the more frequent combination for a particular orthographic representation was the "deeper" attractor; i.e. the completion of the other features (semantic and phonological) would usually fall into this attractor. (This was accomplished by biasing the learning process of the network.). However, using a technological analogy of "priming" to bias the appropriate completion, the resulting attractor could in fact be the less frequent combination – which corresponds nicely to human behavioral data. Indeed, consistent with human empirical results, after the network was trained, the resolution process was affected by the frequency of the different lexical entries (reflected in the strength of the connections in the network) and by the context.
Kawamoto’s model uses perhaps the simplest architecture that can suffice for LH processing during reading in general and ambiguity resolution in particular. Thivierge, Titone and Schultz (2005) recently presented a connectionist model of LH involvement during ambiguity resolution, in which the representations of the words were identical to the vectors used by Kawamoto. (Other computational models of reading have included interconnections between orthographic, phonological, and semantic representations [e.g., Seidenberg & McClelland 198]). The model proposed below incorporates two networks, the first architectural identically to Kawamoto’s original model, and the second architecturally modified in order to account for RH language processing.
Note that Kawamoto's network, however, does not model hemispheric differences.
In this paper, we present a preliminary model for lexical disambiguation in the two cerebral hemispheres that is based on the above work of Kawamoto. The model includes two separate networks. One network incorporates Kawamoto’s version, and successfully simulates the time course of lexical disambiguation in the LH. In the other network based on the behavior of the disconnected RH of split brain patients [Zaidel & Peters, 1982], we made a change in Kawamoto's architecture, removing the direct connections between orthographic and phonological units. Taken together, the two networks produce processing asymmetries comparable to those found in the behavioral studies.
In Latin orthographies (such as English), the orthographic representation (the spelling) of a word is usually associated with one phonological representation. Thus, most studies of lexical ambiguity have used homophonic homographs (homonyms - a single orthographic and phonological representation associated with two meanings). As a result, models of hemispheric differences in lexical processing have focused mainly on semantic organization [e.g., Beeman 1998]. We suggest that this reliance on homonyms may have limited our understanding of hemispheric involvement in meaning activation, neglecting the contribution of phonological asymmetries to hemispheric differences in semantic activation and has limited the range of models proposed to describe the process of reading in general.
Visual word recognition studies demonstrate that, even though both hemispheres have access to orthographic and phonological representations of words, the LH is more influenced by the phonological aspects of a written word [e.g., Zaidel, 1982; Zaidel & Peters 1981; Lavidor and Ellis 2003], whereas lexical processing in the RH is more sensitive to the visual form of a written word [e.g., Marsollek, Kosslyn & Squire, 1992; Marsolek, Schacter & Nicholas 1996; Lavidor and Ellis 2003]. Given that many psycholinLH: RH: guistic models suggest that silent reading always includes a phonological factor [e.g., Berendt & Perfetti, 1995; Frost 1998; Van Orden, Pennington & Stone, 1990; Lukatela and Turvey 1994], it is conceivable that such asymmetries may also impact the assignment of meaning to written words during on-line sentence comprehension.
This study takes advantage of Hebrew orthography that in contrast to less opaque Latin orthographies, offers an opportunity to compare different types of ambiguities within the same language [e.g., Frost and Bentin 1992].
In Hebrew, letters represent mostly consonants, and vowels can optionally be superimposed on consonants as diacritical marks. Since the vowel marks are usually omitted, readers frequently encounter words with more than one possible interpretation. Thus, in addition to semantic ambiguities (a single orthographic and phonological form associated with multiple meanings), the relationship between the orthographical and the phonological forms of a word is also frequently ambiguous. For example, the printed letter string "חלמ" in Hebrew has two different pronunciations (/melach/ or /malach/), each of which has a different meaning (‘salt’ or ‘sailor’). 3
We propose a model that incorporates a right hemisphere structure (i.e. network) and a left hemisphere structure (i.e. network) that differ in the coordination and relationships between orthographic, phonological and semantic processes. The two structures are homogeneous in the sense that all computations involve the same sources of information. However, the time course of meaning activation and the relative influence of different sources of information at different points in time during this process is different, because these sources of information relate to each other in different ways. A graphic representation of the model is presented below:
Phonology
Semantics
LH Structure: Orthographic, phonological and semantic codes are fully connected. The connections between these different sources of information are bi-directional and the different processes may very well run in parallel. However, the model incorporates a sequential ordering of events that results from some processes occurring faster than others. For example, in the LH, orthographic codes are directly related to both phonological and semantic codes. However, because orthography is more systematically related to phonology than to semantics, the phonological computation of orthographic representations is faster than the semantic computation of these same representations. As a result, meaning activation in the LH is initially influenced primarily by phonology [e.g., Lavidor & Ellis, 2003] resulting in immediate exhaustive activation of all meanings related to a given phonological form, regardless of frequency or contextual information [e.g., Burgess & Simpson 1988; Titone 1998; Swinney & Love, 2002].
RH Structure: Phonological codes are not directly related to orthographic codes and are activated indirectly via semantic codes. This organization predicts a different sequential ordering of events in which the phonological computation of orthographic representations begins later than the semantic computation of these same representations. As a result, lexical access in the RH is initially influenced by orthography [e.g., Lavidor & Ellis, 2003] and by semantic information, so that less frequent or contextually inappropriate meanings are not immediately activated. Nevertheless, these meanings can be activated later when phonological information becomes available [e.g., Burgess & Simpson 1988; Titone 1998]. 4
This model is tested according to the philosophy describe in the abstract in two complementary ways: (i) By psychophysical experiments with human subjects. (ii) By a computational neural network model. (In this paper we mainly describe the computation network and its results).
If our ideas are correct and orthographic codes activate phonological codes directly in the LH and indirectly in the RH, we should observe that the distinction in processing the two kinds of word types (i.e. homophonic and heterophonic homographs) should occur at different stage in processing in the LH and RH.
Specifically within the LH these differences will be seen in the early stage of lexical access, where as with RH, these differences will only be seen at a later point in time.
In our lab, we have recently investigated the role phonology plays in silent reading by examining the activation of dominant and subordinate meanings of homophonic and heterophonic homographs (a single orthographic representation associated with two phonological representation, each associated with a different meaning) in the two hemispheres. We used a divided visual field paradigm that allows the discernment of differential hemispheric processing of tachistoscopically presented stimuli. Heterophonic and homophonic homographs were used as primes in a lexical decision task, where the target words were either related to the dominant meaning or to the subordinate meaning of the ambiguous word, or were unrelated. We measured semantic facilitation by response times. A significant interaction between visual field of presentation (right or left), type of stimulus (heterophonic or homophonic homograph) and g inm100 ir 80 p to 60 e 40 u d 20 n tio 0 tia-20 l ic-40 a F RH LVF * * LH
RVF g Heterophonic Homographs n i im100 rp 80 to 60 edu 2400 * n tio 0 tiilac--2400 RH aF LVF
LH RVF dom sub dom sub
The units in the LH and RH network were implemented as described by Kawamoto [1993] with the following changes: (a) the original 48 4-letters words were replaced with 48 patterns representing 24 pairs of polarized Hebrew 3-letter homographs, half heterophonic and half homophonic. (b) 45 features (instead of 48) represented the word's spelling and 60 features (instead of 48) represented its pronunciation. This is because the pronunciation includes the vowels that were omitted from the spelling. The representation for "part of speech" (all nouns) and "meaning" remains the same as in the original model. Overall, each entry is represented as a vector of 270 binary-valued features. Both networks were trained on the same input with a simple error correction algorithm [1, 2]: Δ W ij = η (t i − i i )t j ii = ∑ W ij t j j [1] [2] Where η is a scalar learning constant fixed to 0.0015, ti and tj are the target activation levels of units i and j, and ii is the net input to unit i. The magnitude of the change in connectype of target words suggested that heterophonic and homophonic homographs were disambiguated differently in the two visual fields, and by implication, in the two hemispheres. With homophonic homographs, targets related to both dominant and subordinate meanings were activated in the RVF/LH, while in the LVF/RH only dominant meanings evoked facilitated responses (panel A in Figure 1). Alternatively, with heterophonic homographs only dominant meanings evoked facilitated responses, and only in the LVF/RH (panel B in Figure 1).
Homophonic Homographs tion strength is determined by the magnitude of the learning constant and the magnitude of the error (ti - ii.).
The activity of a single unit in both networks is represented as a real value ranging between -1.0 and + 1.0.
1 LIMIT = − 1 The activity of a unit is computed from three different sources: the 1st is the sum of all outputs of other units in the net; the 2nd is the direct input from the external environment; and the 3rd is the output of the unit in the previous iteration multiplied by the decay rate.
Since all units are mutually connected these influences lead to changes in the activity of a unit as a function of time (where time changes in discrete steps). That is, the activity of a unit (a) at time t + 1 is:
a(t + 1) = LIMIT δa(t ) + ∑ wij (t )a j (t ) + si (t ) [4] j Where δ is a decay variable that changes from 0.7 to 1. si(t) is the influence of the input stimulus on unit ai at time (t+1), and LIMIT bounds the activity to the range from -1.0 to +1.0.
a(t + 1) = LIMIT δa(t ) + ∑ wij (t )a j (t ) j [5]
In each simulation, 12 identical LH and RH networks were used to simulate 12 subjects in an experiment. Each network was trained on 1300 learning trials. On each learning trial an entry was selected randomly from the lexicon. Dominant and subordinate meanings were selected with a ratio of 5 to 3. After the networks were trained they were tested by presenting just the spelling part of the entry as the input (to simulate neutral context) or by presenting part of the semantic sub-vector together with the spelling (to simulate prior contextual bias). In each simulation the input sets the initial activation of the units. The level was set to +0.25 if the corresponding input feature was positive, -0.25 if it was negative and 0 otherwise. In order to assess lexical access, the number of iterations through the network for all the units in the spelling, pronunciation or meaning fields to become saturated, was measured. A response was considered an error if the pattern of activity did not correspond with the input, or if all the units did not saturate after 50 iterations.
hetero 17.69
18.58
When homographs are presented without a biasing context, only the dominant meaning is accessed in both networks. However, in the LH network, meanings are accessed faster. This is consistent with LH advantage for lexical processing reported in the literature. More importantly, homophonic and heterophonic homographs are processed differently in the two networks. In the LH network, lexical access is longer for heterophonic homographs then for homophonic homographs (Table 1) due to the timeconsuming competition between the two phonological representations. Indeed, more iterations were needed for the phonological units to become saturated in the case of heterophonic homographs than for homophonic homographs (Table 2). This is consistent with the idea that in the LH, phonological information guides early stages of meaning activation. Alternatively, in the RH network, phonological differences are less pronounced (Table 2) and processing times of homophonic and heterophonic homographs are similar (Table 1). This is consistent with the idea that in the RH, orthographic and semantic sources of information exert their influence earlier than phonological information.
When homographs are presented with a biasing context, only the contextually compatible meaning is accessed in both networks, In addition dominant meanings in dominant contexts are accessed faster than subordinate meanings in subordinate contexts (Table 1). Interestingly, in the LH network, homophonic advantage in processing time disappears when a biasing context is provided. Moreover, when homographs are presented with a subordinate context, it takes longer to access the subordinate meaning of homophones homographs compare to heterophones homographs (Table 1). In both cases, as predicted phonological disambiguation precedes meaning disambiguation (Table 2).
Because heterophonic homographs have different pronunciations, these homographs involve the mapping of a single orthographic code onto two phonological codes. As a result, when no context is presented, the speed of lexical access is slower for heterophonic homographs then for homophonic homographs. On the other hand, when context is provided, the single phonological code of homophonic homographs is still associated with both meanings, whereas the phonological representation of heterophonic homographs is associated with only one meaning. As a result, when homographs are presented in a subordinate context, a longer period of competition between dominant and subordinate meanings is observed in the case of homophonic homographs. In contrast, in the case of heterophonic homographs, meanings are accessed immediately after a phonological representation is computed. 5
These results have important implications for the role phonology plays in accessing the meaning of words in silent reading. One class of models suggests that printed words activate orthographic codes that are directly related to meanings in semantic memory. An alternative class of models asserts that access to meaning is mediated by phonology [for reviews see Frost 1998; Van Orden and Kloos 2005]. Our results supports the idea that in the LH words are read more phonologically (from orthography to phonology to meaning), whereas in the RH, words are read more visually (from orthography to meaning).
Overall, the two networks produce processing asymmetries comparable to those found in behavioral studies. In the LH network, orthographic units are directly related to both phonological and semantic units. However, because orthography is more systematically related to phonology than to semantics, the phonological computation of orthographic representations is faster than the semantic computation of these same representations. As a result, meaning activation in the LH is initially influenced primarily by phonology. In the RH network, phonological codes are not directly related to orthographic codes and are activated indirectly via semantic codes. This organization results a different sequential ordering of events in which the phonological computation of orthographic representations begins later than the semantic computation of these same representations. As a result, lexical access in the RH is initially more influenced by orthography and by semantic.