In spite of considerable converging evidence of the role of inflectional paradigms in word acquisition and processing, little efforts have been put so far into providing detailed, algorithmic models of the interaction between lexical token frequency, paradigm frequency, paradigm regularity. We propose a neurocomputational account of this interaction, and discuss some theoretical implications of preliminary experimental results.
Over the last fifteen years, growing evidence has
accrued of the role of morphological paradigms in
the developmental course of word acquisition.
Children have been shown to be sensitive to
subregularities holding among paradigm cells
Such evidence supports a view of the mental
lexicon as an emergent integrative system,
whereby words are concurrently, redundantly and
competitively stored
In spite of such a wealth of converging evidence, however, little efforts have been put so far into providing detailed, algorithmic models of the interaction between word frequency, paradigm frequency, paradigm regularity and lexical familiarity in word acquisition and processing. We offer here such an algorithmic account, and discuss some theoretical implications on the basis of computational simulations. 2
In the present contribution, we use Temporal Selforganising Maps (TSOMs) to simulate dynamic effects of lexical storage, organisation and competition.
TSOMs, a variant of classical Kohonen’s SOMs
(Kohonen, 2001), are dynamic memories that are
trained to store and classify time-series of
symbols through patterns of activation of fully
interconnected nodes
When a symbol is shown on the input layer at a certain time tick, all map nodes are fired synchronously, their overall pattern of activation representing the processing response of a TSOM to the symbol at that time tick. Due to principles of topological organisation of map’s responses, similar input stimuli (i.e. two instances of the same symbol in different contexts) tend to be associated with largely overlapping memory traces (e.g. the two p nodes activated by pop in Fig. 1). During training, nodes get gradually specialised to respond most strongly to specific time-bound instantiations of symbols, while remaining relatively inactive in the presence of other stimuli. A recurrent activation pattern associated with an input symbol occurring in a specific context can thus be seen as the map’s memory trace for that symbol in that context.
An input word is administered to a TSOM as a time series of symbols, i.e. a sequence of letters or sounds presented on the input layer one at a time. The map’s response to a word stimulus is the overall activation pattern obtained through integration of the activation patterns triggered by the individual symbols making up the word (see Fig. 1 for a simplified example with the word pop). Accordingly, if two input strings present some symbols in common (e.g. pop and cop, write and written), they will tend to activate largely overlapping patterns of strongly responsive nodes. Like in the case of individual symbols, the integrated activation pattern for an input word is, at the same time, the systematic processing response of the map to an input stimulus, and the word’s memorised representation (or memory trace) in the map.
To investigate issues of
“frequency-byregularity” interaction
Fifty German and fifty Italian verb
(sub)paradigms were selected among the most
highly ranked paradigms by cumulative frequency
in a reference corpus
As a general trend, TSOMs acquire word forms by token frequency, with higher-frequency words being successfully recalled at earlier learning epochs. However, when it comes to the actual timing of paradigm acquisition, things get considerably more complex, with the notion of morphological regularity interacting non-trivially with token frequency distributions. In fact, in both German and Italian, the vast majority of paradigms are acquired earlier (p<.005) in a UD regime than in an SD regime (Fig. 2).
Our simulations show that, in both languages, word forms in regular paradigms tend to be acquired earlier (significantly earlier learning epochs, p<.001), and regular paradigms are acquired more quickly (significantly shorter learning spans, i.e. lower number of epochs between the acquisition time of the first and the last member of a paradigm, p<.005) than irregular paradigms are. In German data, regular paradigms are less sensitive to token frequency effects than irregular paradigms are, as witnessed by the strong correlation (r=.95, p<.00001) between the time course of acquisition of regular paradigms in SD and UD regimes (Fig. 2, bottom left panel). Token frequency affects the acquisition of regular paradigms to a lesser extent than the acquisition of irregular ones, because regular stems can take advantage of their cumulative frequency across the whole paradigm. In fact, forms in regular paradigms exhibit a significant correlation between stem cumulative frequency and time of acquisition (r=-.40, p<.00001). Similarly, also German irregular paradigms, which exhibit a predictable stem allomorphy due to a limited number of alternants, show a correlation between stem cumulative frequency and acquisition time (r=-.24 p<.00001).
Conversely, in Italian, where verb
conjugation exhibits more extensive and less
predictable patterns of allomorphy than in
German
Our data cannot be explained away as a simple by-product of word-frequency effects. Experiments provide, in fact, evidence of interactive processing effects in word acquisition, whereby morphological regularity modulates frequency. Data analysis shows that recurrent patterns appear to determine global coorganisation of stored word forms and distributed, overlapping memory traces, which ultimately favour generalisation in lexical acquisition. Forms containing recurrent patterns can take advantage of the memory traces shared with other related forms, namely forms sharing the same stem, and connections between the nodes making up their memory traces are strengthened since patterns are shown more often in training, similarly to highfrequency isolated words.
This is particularly true for regular, highly entropic paradigms, i.e. those regular paradigms whose members exhibit uniform frequency distributions, and for irregular highly systematic paradigms. Conversely, where memory traces overlap less systematically, this effect is considerably reduced, as witnessed by the difference in time of acquisition between regular and irregular paradigms, particularly in Italian conjugation.
In TSOMs, the effects are the dynamic result
of two interacting dimensions of memory
selforganisation: (i) the syntagmatic or linear
dimension, which controls the level of
predictability and entrenchment of memory traces
in the lexicon through the probabilistic
distribution of weights over inter-node Hebbian
connections; and (ii) the paradigmatic or vertical
dimension, which controls for the number of
similar, paradigmatically-related word forms that
get co-activated when one member of a paradigm
is input to the map
High-frequency words develop quick entrenchment of Hebbian connections, which eventually cause high levels of node activation in their memory traces and sparser co-activation of memory traces of other words. Strong connections and high activation levels mean high expectations for frequently activated memory traces, which are thus recalled more easily and are less confusable with other neighbouring words. Likewise, in regular and sub-regular paradigms, sharing memory traces can strengthen connections and raise node activation levels, since all related forms can take advantage of the memory traces shared with other members of the same paradigm.
This dynamic provides an algorithmic account of the observation that regularity favours acquisition of both high- and low-frequency words, as shown in Fig. 3, where we compare average levels of activation for four classes of training word forms: low-frequency regulars, low frequency irregulars, high-frequency regulars and high-frequency irregulars.1
Activation levels of low-frequency words appear to be significantly stronger within regular paradigms than within irregular paradigms (Fig. 3, top). Stronger activation levels make patterns less confusable and easier to be accessed, as witnessed by the lower level of filtering2 required for activation patterns to be recalled accurately (Fig. 3, bottom). We observe, in fact, a highly significant correlation (r=.49, p<.00001 for both datasets) between levels of filtering and words’ learning epochs.
High-frequency words predictably show higher activation levels than low-frequency words, with an interesting difference of the interaction of frequency and activation levels of regulars and irregulars. High-frequency, highly irregular words (e.g. German ist or Italian è) are stored in isolation, with highly-activated memory nodes and no co-activation with other words. As a result, they require little filtering to be recalled and are acquired considerably quickly. Highfrequency regular paradigms, despite in both Italian and German training sets their average frequency is nearly half the average frequency of high-frequency irregulars, show comparable levels of activation with high-frequency irregulars, due to the facilitatory effect of having more words that consistently activate the same pattern of nodes.
This evidence shows that regularity indeed modulates the interaction between frequency and activation strength, and it gives a strong indication that acquisition of regulars is typically paradigmbased, whereas acquisition of irregulars is mostly item-based.
Surely, as the notion of paradigm regularity is inherently graded, some verb systems show higher sensitivity to these effects than others. This is illustrated by German sub-regular paradigms, which present fewer and more predictable stem alternants than Italian sub-paradigms, and thus larger stem-sharing word families. Accordingly, TSOMs allocate comparatively higher levels of activation to low-frequency German sub-regulars and acquire them earlier than their Italian homologues.
The evidence reported here establishes, in our view, an important connection between aspects of morphological structure, frequency distributions of words in paradigms, and lexical acquisition in concurrent, competitive storage. Acquisition of redundant morphological patterns play an increasingly important role in an emergent lexicon, shifting acquisitional strategies from rote memorisation (typical of irregular low-entropy paradigms) to dynamic memory-based generalisation. 1 Frequency thresholds are set below the first quartile (low frequency) and above the third quartile (high frequency) in the frequency distribution of training word forms. 2 Filtering an integrated activation pattern refers to the process of bringing down to zero the levels of activation of nodes that do not reach a set threshold.