Elevation in typing latency for the initial letter of the second constituent of an English compound, relative to the latency for the final letter of the first constituent of the same compound, provides evidence that implementation of a motor plan for written compound production involves smaller constituents, in both semantically transparent and semantically opaque compounds. We investigate here the implications of this evidence for algorithmic models of lexical organisation, to show that effects of differential perception of the internal structure of compounds and pseudo-compounds can also be simulated as peripheral stages of lexical access by a self-organising connectionist architecture, even in the absence of morphosemantic information. This complementary evidence supports a maximizationof-opportunity approach to lexical modelling, accounting for the integration of effects of pre-lexical and lexical access.
Il rallentamento nel tempo di battitura del primo carattere del secondo costituente di un composto inglese, rispetto al tempo dell’ultimo carattere del primo costituente, dimostra che l’implementazione del programma motorio per la scrittura di un composto è influenzata dai costituenti del composto stesso, siano essi semanticamente trasparenti o opachi. Il presente contributo offre un modello computazionale di questa evidenza, e ne valuta l’impatto sull’organizzazione del lessico mentale: la percezione del confine di morfema tra i due costituenti è analizzata come il risultato dell’interazione dinamica tra processi di accesso pre- e post-lessicale. 1
A key question concerning the representation and
processing of compound words has focused on
whether (and, if so, how) morphological
structure plays a role. The bulk of the research on this
issue has come from recognition or
comprehension tasks such as lexical decision or reading.
However, written production provides a useful
counterpart and allows researchers to examine
whether morphemes are used even after a word
has been accessed. One advantage of a typing
task (in which the time to type each letter of a
word is recorded) is that researchers can examine
differences in processing difficulty at various
points in the word. Previous research found an
elevation in typing latency for the initial letter of
the second constituent relative to the latency for
the final letter of the first constituent for English
The time to initiate the first letter was equivalent for monomorphemic and compound words. Typing times got faster across the word for both word types, but the rate of change was faster for monomorphemic words than for compound words. This difference was not observed when comparing monomorphemic words and pseudocompounds.
For compounds, the rate of speed-up was slower when the first constituent was transparent than when it was opaque, but was unaffected by the transparency of the second constituent. The elevation in typing latency at the morpheme boundary was larger when the first constituent was transparent than when it was opaque, but was unaffected by the transparency of the second constituent. This difference is due to the final letter of the first constituent when the first constituent requiring less time to type when it was transparent than when it was opaque.
The data for the pseudo-compounds indicated that embedded morphemes influence production, even when they do not function as morphemes. Typing latency increased one letter prior to the end of the first constituent of a pseudocompound and remained elevated through the boundary (e.g., both r and c in scarcity were elevated relative to the a). 1.1
Implications for lexical architectures The reported evidence clearly indicates that morphemic structure is involved in written word production. The production of compounds differs from that of monomorphemic words and the semantic transparency of the two constituents leads to different effects. Furthermore, embedded pseudo-morphemes appear to influence the production of pseudo-compounds, but not in the same way that the embedded morphemes affect the production of compounds.
This appears to lend only partial support to
models of lexical architecture where both
compounds and their constituents are represented and
processed as independent access units (Figure 1).
In panel A, following
Temporal Self-Organising Maps
A TSOM consists of a grid of memory nodes with two layers of connectivity. The first layer (or I-layer) fully connects each node to the input vector, where symbols are sampled at discrete time ticks as patterns of activation ranging in the [0, 1] interval. Weights on the I-layer are adjusted in training for individual nodes to develop specialised sensitivity to particular input symbols. Each node is also connected to all other nodes through a layer of re-entrant connections (or T-layer), whose weight strength determines the amount of influence that activation of one node has on other nodes at a one-tick delay.
When an input symbol is presented at time t, the level of activation ! of node i is a function of: (a) the node’s sensitivity to the current input symbol (!_!"#$%,! ), and (b) the re-entrant support the node receives from the map activation state at t-1 (!_!"#$%,! = !( − 1) , where f is a linear function and j ranges over all map nodes). More formally:
! = ∙ !_!"#$%,! + 1 − ∙ !_!"#$%,! The node responding most strongly to the input symbol S at time tick t is called Best Matching Unit (hereafter BMU(S, t) or BMU(t) for short).
The map’s response to a sequence of input symbols like carpet is a chain of consecutively firing BMUs, each responding to a letter in carpet. During training, connection weights between consecutive BMUs are adjusted to the frequency distribution of input symbols in the training set, according to Hebbian principles of correlative learning. Given the bigram ab, the connection strength between BMU(a, t-1) and BMU(b, t) increases if a often precedes b (entrenchment) and decreases if b is often preceded by a symbol other than a (competition) (Figure 2, left). Combination of entrenchment and competition yields selective specialisation of chains of BMUs (Figure 2, right). If the same input symbol follows different contexts, it will tend to be responded to by more BMUs, one for each context. The stronger the probabilistic support that the input symbol receives from its preceding context, the more likely the recruitment of a dedicated BMU, The 200 compounds and 50 pseudo-compounds used by Gagné & Spalding were used to train a 40x40 node TSOM for 100 learning epochs. Besides compounds and pseudo-compounds, the training set included 500 (pseudo)constituents as individual words (e.g. car and wash in carwash, car and pet in carpet), for a total amount of 750 items. At each training epoch, monomorphemic words were shown 10 times as often as compounds. We ran 5 repetitions of the experiment, and results were analysed using linear mixed effects models (LME), with experiment repetitions and training items as random variables.
To analyse differential processing effects for pseudo-compounds and compounds, we focused on two types of evidence: (i) per-letter performance of a trained TSOMs in incrementally anticipating compounds and pseudo-compounds; (ii) structural connectivity of BMUs responding to letter bigrams at the C1-C2 boundary.
To anticipate a progressively presented input word, a TSOM propagates the activation of the current BMU(t) through its forward temporal connections, and outputs, at each time tick, the symbol !"#(!!!) encoded on the I_layer of the most strongly (pre)activated node: ( + 1) = argmax !,! ℎ = !!!,…,! where !,! is the weight value on the forward temporal connection from node h to node i. Each correctly predicted symbol in the input word is assigned the prediction score of the preceding symbol incremented by 1. Otherwise, the symbol receives a 0-point score.
Figure 3 (top panel) illustrates the rate of letter anticipation across the word for both compounds and pseudo-compounds, plotted by distance to the morpheme boundary. The steeper rate for pseudo-compounds than for compounds shows that pseudo-compounds are easier to predict/anticipate than compounds. We take this evidence to be in line with evidence of a faster speedup rate in the typing of monomorphemic vs. compound words. A closer look at anticipation rates for individual constituents (Figure 3 bottom panel) shows a drop of anticipation at the C1-C2 boundary (more prominent for compounds than pseudo-compounds) with a steeper increase in C1 and C2 for pseudo-compounds, which happen to be, on average, shorter than C1 and C2 in real compounds.
To look for structural correlates of anticipation rates in the map, we conducted, for each item, a letter-by-letter analysis of values of pointwise entropy (PWH) for the connections between consecutive BMUs, namely h=BMU(t-1) and i=BMU(t): !,! = − !,! ! !,!
The value of PWH for the connection between end-C1 and start-C2 (x = 0) has a local peak in compounds only (Figure 4). Since PWH provides a measure of how unexpected the activation of BMU(t) is, this structural evidence can account for a delay in processing and a drop in anticipation at the morpheme boundary of compounds, but not of pseudo-compounds. Trained on both compounds and pseudocompounds, TSOMs develop a growing sensitivity to surface distributional properties of input data, turning chains of randomly connected, general-purpose nodes into specialised sub-chains of BMUs that respond to specific letter strings at specific positions. Compounds not only tend to occur, on average, less frequently than their C1/C2 constituents do as independent words (Ji et al. 2011), but they tend to present lowerfrequency bigrams at the C1-C2 boundary than do pseudo-compounds. Principles of Hebbian learning allow TSOMs to capitalise on both effects. Entrenchment makes expectations for highfrequency bigrams stronger and expectations for low-frequency bigrams weaker. At the same time, the competition between C1 as an independent word and C1 as the first constituent in a C1-C2 compound biases the map’s expectation towards the most frequent event (C1 in isolation). Compound families, i.e. sets of compounds sharing C1 (windmill, windshield etc.) or C2 (snowball, basketball etc.), magnify these effects, making the map more sensitive to formal discontinuity at morpheme boundaries. When more C2s can follow the same C1 in complementary distribution, the left-to-right expectation for a particular C2 to occur, given C1, decreases. Likewise, when more C1s competitively select the same C2, the individual contribution of each C1 to the prediction of C2 decreases. We conjecture that more global effects of lexical organisation like these may eventually blur local memory effects based on position-independent bigram frequencies.
Our simulations with TSOMs can model the correlation between continuously varying distributional regularities in the input data and peripheral levels of routinized recognition and production patterns. These patterns are in line with Gagné & Spalding’s evidence of (a) the influence of embedded pseudo-morphemes on cascaded models of written word production, and (b) faster anticipation rates for monomorphemic vs. compound words.
Further experimental results (not reported
here), obtained by including compound families
in the training data, confirm slower anticipation
rates for true compound constituents, due to the
combined effect of word frequency distributions
and word compositionality in compound
families. The size of a compound family can arguably
be a function of the degree of productivity and
semantic transparency of its members
A full account of Gagné & Spalding’s
evidence of a graded influence of semantic
transparency on compound processing is beyond the
reach of the computational architecture presented
here. Surface effects of discontinuity in the
internal structure of compounds (as opposed to
pseudo-compounds) appear to provide a purely
formal, pre-lexical scaffolding for truly
morphosemantic effects to emerge at later processing
stages. To model these effects, we appear to be
in need of a parallel processing architecture able
to effectively integrate several representational
levels (orthographic, phonological,
morphological, and conceptual) and different processing
steps within a single distributed system