This contribution pursues two objectives. First, a short summary of the computational approaches for identifying Middle English afixes comprising diferent AI methods will be devised reaching from standalone application [1] and (semi-automated) shared-work solutions [2]1 to requesting the OED RESTful API.2 Second, the role of AI and how it is entertained in the context of diachronic afix identification will be shown.
The methodological procedure improved in bumps and leaps over the year as both the linguistic knowledge of Middle English (ME) was enhancing (very moderately) and the inventory of computational methods increased (rather exuberantly) over the years. Hence the phenomenon under investigation as well as the methods helping to understand the phenomenon are constantly changing over time. The researcher has to keep track of all advancements on either aspect, integrate technological innovations in ongoing methodological work, but also consider new ifndings in the field of derivational morphology altogether. It needs little justification to see that the longer a research project takes, the more problematic it will be to constantly adjust the methodological set up. Yet, besides increasing one’s personal learning, such challenge bears the advantage of pervading the subject to an utmost degree and by that diminishing the chance of encountering inconsiderate evidence. It is under this lens that roughly four phases of diferent approaches to the collection of data can be observed since the inception of the project about one decade ago.
The first phase is characterized by the basic idea of having the machine support manual data
collection in text corpora. It derived from elaborated corpus searches for selected derivational
morphemes. Realizing that the variations in writing would not produce correct quantities, it
soon became clear that a more elaborated algorithm needs to be developed. What was needed
foremost was a comprehensive list of all known afixes and possibly all of their known variations.
Already at the time, the OED online version (4.0) provided a seemingly complete collection of
about 800 sufixes and 300 prefixes. Although this early study of English afix usage could not
manage to reliably find all tokens that are really required for generating productivity measures
over time, at least the existence of each of the morphemes, i.e. type frequencies, could be proven
in textual data in predefined time spans [
The second phase started with the production of the Morphilo Toolset [
Furthermore, to avoid the embeddedness problem, the length of an afix chain is taken as a simple criterion to prioritize possible matching candidates, i.e. -ation beats -tion in e.g. temptation leaving as the root tempt. The simple logic behind this decision results from probability measures and the relation between the length of a word and its frequency of occurrence. And it follows from Zipf’s law.
More precisely, the workflow goes as follows. The file containing lexically annotated words is created by an automaton, which extracts all nouns, verbs, and adjectives from tagged corpora and marks the grammatical morpheme, if regular (plural, 3rd person sing, possessive, progressive/gerund, past, and participles), otherwise it is already tagged in the standard PENN annotation. Enumerated lists serve as a data structure for eficiently keeping sufix and prefix morphemes and their allomorphic representations. There are more than 800 sufix variations; the prefixes comprise about one third of the sufixes.
These afixes were copied from the OED (version 4.0). At the time of devising the architecture, the time of usage, inception, and possibly attrition of the afix was not provided (other than in later versions) and, hence, could not be incorporated into the data type. So the algorithm could be refined with knowledge of the word class, but not on the time of usage, which is only then included in automatic matching if existent in the data base, i.e. it was added by manual post processing. The enumerated lists are processed according to the above algorithm.
Post-processing is done after the words of a text were synchronized with existing data in the master data base. All non-matches are displayed as a result of the algorithm. Corrections can be done by click and drop. Confirming a correct lexical segmentation means that they are saved to the data base together with the number of occurrence in the corpus and the time span of the text.
The gist of phase three is a community based approach [
Finally the fourth phase implemented the new RESTful API of the OED (see figure 2). It includes the main functionality of its predecessors although it is completely refactored. Generally it follows the idea known in information science as micro services. Once the OED decided to ofer a functional API and make it available for research use free of any charge, the future way to go in using this resource was strikingly obvious. Because this approach successfully collected suficient amounts of language data from a given text corpus and at the same time is the most advanced and exact method predominating the solution of Middle English afix extraction while incorporating some (but not all) of the previous algorithmic solutions, it is the prioritized candidate to be recommended for future use.
The above explications on a given (and long standing) problem in Diachronic Linguistics depicts a picture of how the existing inventory of AI-methods is typically applied. There are hardly any straight imperatives of proceedings that we could follow. It cannot be predicted with a higher or lower probability or plausibility as to which method fits better than the other. Indeed, it is possible to make a reasonable selection from the method inventory – i.e. exclude neural networks because the data does not fulfill its very basic requirements – but it still leaves the researcher with too many alternatives from which it is impossible to estimate a success rate. What seems to be an trial-and-error approach from the outside, it is a kind of systematic polling from the inside perspective. In the concrete case described here, one could learn that, on the one hand, the right combination from a semi-automatic method (first generation) enriched with a smart algorithm (2nd generation) would only be eficient if extended with a quality resource (4th generation). On the other hand, none of these components can be missed out, however, as the third generation showed, not all methods are equally optimal.