The following contribution summarizes a number of problems associated with a methodological focus on statistical measures in corpus linguistics, specifically in the subfield of lexical and lexicosyntactic analysis: the practical impossibility of collecting suficient amounts of data to fully account for all of the factors that are known to interact with linguistic output; the reliance on mathematical assumptions that are generally not met by language data; and the epistemological limitations of considering corpora language samples from a presumed superpopulation versus the evolutionary and non-ergodic nature of language. It then proposes graph metrics as a viable alternative to statistical measures. Rather than comparing groups of factors, graph metrics capture relational aspects of the whole dataset, and unlike inferential statistics, they do not project to a presumed external totality or population. Neither do they rely on assumptions of randomness, independence, stationarity, or ergodicity. They thus avoid many of the concessions to the linguistic model that are inherent to probabilistic models of the lexicon, which in turn may result in an epistemologically sounder operationalization and quantification of corpus data. However, the high level of abstraction inherent in graphs, and especially in applications of
graph theory, comes with its own caveats. Drawing on the example of Kobalt, a mid-sized corpus of texts written by learners of German, two approaches to the application of graph metrics in corpus-linguistic research are outlined and demonstrated in this paper, including a detailed discussion of the steps necessary for validation and linguistic embedding. 1
Quantitative corpus linguistics has been a prolific field of research since the
early 1990s. A number of large, general-purpose corpora, such as the British
National Corpus
While a wide-range of phenomena are analyzed in linguistics, a particularly prevalent strand of research in corpus linguistics addresses the cooccurrence of words and the formation of meaningful larger units that do not appear to follow higher-order syntactic rules, such as collocations (words that habitually co-occur despite potentially available alternatives, like strong cofee vs. powerful cofee ), idioms (word bundles with elements of meaning that cannot be derived from the words themselves, like hit the road), and constructions (semi-syntactic elements that are partially lexically specified and tend to be expandable by analogy, like He worked his way up, She studied her way to the top, and They sang their way into our hearts).
The major methodological focus of this work has been on frequentist
statistics over words, lexemes (base word forms, such as to be for is, are, were, etc.)
and relatively coarse syntactic categories (part-of-speech tags, such as
preposition, or dependency labels, such as accusative object) that rely on automatic
extraction. The term ‘frequentist’ here refers to the ‘traditional’
understanding of statistics, whereby probability is defined as the convergence to the
limit of an expected relative frequency in an infinite series of experiments. 1
1Bayesian models have not played a role in corpus-linguistic analysis so far. Bayesian
statistics defines probability in a diefrent manner, namely as a range of reasonable belief based
on prior experience. This increases the suitability to the modeling of dynamic systems and
interacting factors. Whether or not this also allows for an improved quantification of
corpus data remains to be seen. Although highly relevant to the methodological development
of corpus linguistics, this question will be set aside for the remainder of this paper in
orFor example, if I toss a fair coin an infinite number of times, the relative
frequency of it landing on either side will approximate 0.5, which is also the
probability value. Corpus linguistics operates within precisely such a
framework, relying on various types of quantitative analysis such as productivity
measures, which look at word distributions and quantify the openness of
slots to accept new members
In the wake of these conceptualizations of corpora as plausibly representative samples of language, a dynamic interplay has developed between amassing larger amounts of data to allow for statistical analysis and an increasing reliance on automatic modes of data extraction and classification. There are, however, a number of issues with viewing language through the lens of stochastics, which can be broken down into three general categories: a) practical – controlling for all interacting factors tends to greatly limit sample size as well as linguistic depth; b) mathematical – the lexicon is likely not a stochastic system; and c) epistemological – corpora frequently do not qualify as samples that allow inferences to a population, especially those that have been compiled in elicited or quasi-experimental settings. 1.1
A major reason for the reliance of corpus linguistics on surface or surfacenear forms lies in the dificulty and high cost of in-depth annotation. Linguistic categories are generally ambiguous, fuzzy-edged, and often only implicitly represented in surface forms.
For example, while the grammatical roles of subject and direct object in the sentence The boy broke the vase are easily extractable – especially in English, where the position in left adjacency to the finite verb is reserved for the grammatical subject – the semantic, pragmatic, textual, and intertextual implications can be much harder to pinpoint. Consider, for example, the sentence The vase broke, in which the former direct object to the verb break is now the grammatical subject, while the process of breaking still belongs to the same object semantically.2 der to allow for a more in-depth discussion of the pitfalls of frequentist approaches and the potential of graph-based modeling as a structural alternative.
2This is an example of a so-called unaccusative verb. Features and problems of categor
Even aspects of limited ambiguity, such as the morphological features of a
word (for example grammatical gender or types of inflection) or the syntactic
structure of a sentence, cannot always be parsed with high accuracy in
languages with flexible word order and a high degree of inflection
In a research setting that works with rich morphology and/or syntactic
lfexibility, untrained, and non-canonical data (as is the case in the analysis
of German learner language), automatic classification and parser
performance generally do not sufice for research purposes. This is even more so
the case for analyses that go beyond surface-near forms, where the accuracy
of automatic parsing of semantic or pragmatic information is generally not
very high. For example,
Annotation of deeper level information generally requires large amounts of tedious manual or, at best, semi-automatic annotation. Rather than being a simple labeling task, most linguistically interesting annotation is a complex categorization process. It requires the development of guidelines for ambiguous cases, measurements of inter-annotator agreement, and the iterative revision of previous annotations. With this amount of manual input, deep linguistic annotation is generally not feasible for large corpora.
At the same time, a number of recent and ongoing studies
There is thus an inevitable trade-of between the depth of linguistic analysis and corpus size. In other words, large corpora generally underexplore the potentials of linguistic analysis. Statistical measures were introduced into corpus linguistics in order to capture gradual eefcts and to distinguish between true diefrences and random fluctuation – but in reality, a strong focus on these metrics greatly limits the analytical capacity of linguistic research. However, as I will argue in the following sections, the problems with such an approach run deeper. 1.2
There are a number of debates centered around statistics in linguistics which
deserve to be mentioned here, but cannot be discussed in detail due to spatial
constraints:
• There appears to be a general lack of understanding of the underlying
distribution of words in corpora. It has been assumed that words are
Zipf-distributed, but recent research suggests that this might not be the
case, and/or that the Zipf distribution is itself an artifact
Beyond these already grave concerns, there are two even deeper problems concerning aspects that are intrinsic not only to corpus compilation, but to the nature of language itself.
First, frequentist statistics is rooted in a concept of probability that
derives overall probabilities from previous observations – it predicts the future
from the past. The bridge between the two is provided by the central limit
theorem, which states that in suficiently large samples, regardless of the
underlying distribution, relative frequencies will reach limits and those limits
can be idealized to probabilities. This is expressed in the assumption that
the overall system will reach expected values if the experiment is repeated ad
infinitum: if a fair coin is tossed a suficient number of times, its idealized
property of being 0.5 heads and 0.5 tails will become defined over time, even
if heads or tails cluster in parts of the series. However, for the central limit
theorem to hold true, the underlying system must be stochastic – it must
have probabilities – which means it must be stationary and ergodic, with
stationarity referring to the property of a system to have stable and unchanging
probabilities. If, for example, the coin is damaged after a few tosses, the
outcome may be skewed and the system is no longer stationary. Language is
obviously not stationary, since it evolves significantly over time. If this was
the whole extent of the problem, then it would sufice to define stationary
subsets of language, e.g. corpora spanning only a decade or less – but in fact,
there are reasons to believe that stationarity is not reached in thematically
diverse corpora even over short periods of time
Since language is at least in part a cognitive function, it underlies
perceptual biases and cognitive influences. One major factor in this is priming. It
is well-known that speakers are very easy to prime on all linguistic levels –
syntactically
Priming also contributes to inter-speaker convergence or alignment, a
phenomenon in which the participants in a dialogue adapt to one another in
their linguistic expression. Alignment has been shown to exist in contexts as
diverse as abstract frames of reference (e.g. up/down vs. north/south in map
description tasks, as investigate
A more global factor influencing the frequency of occurrence of linguistic elements is the ebb and flow of discourse interaction, by which debates gain momentum and then die down. This can play out within days, hours, or in the space of a single document, and is inextricably linked to the issue of intention: speakers do not utter certain words to fix imbalances in the probability distribution, but rather consciously choose to speak in specific ways and/or about a specific topic, which in turn increases or decreases the probability for any given word to occur.
Lastly, perhaps the strangest eefct is produced by productivity (the process of coining new words incidentally), and creativity (the intentional creation of new words). Nouns, verbs, and adjectives are open classes, meaning new lexemes can be introduced to the system at any time. However, in stochastic terms, this is the equivalent of rolling dice that keep changing the number of their sides – with every new word, the relative frequencies for all words change. Far from being a marginal phenomenon, productivity is highly prominent in all spoken and written language, and is intrinsically problematic for the concept of stationarity in language.
The second major issue with respect to the mapping of the linguistic to
the statistical model is ergodicity or path-independence. It is also the second
necessary condition for the upholding of the central limit theorem. In an
ergodic system, no matter which path I take through the system, relative
frequencies for all elements can approximate limits and the overall system will
reach expected values. For example, by rolling a fair die a suficient number
of times, noting down the values, and then averaging over them, I will find
an approximation of the expected value 3.5. No matter which path I take,
whether I roll a 6 seven times in a row and then a 1 seven times in a row,
or any other combination – with suficient repetitions, the system will
converge. In a non-ergodic system, this is not the case: if I roll the die once and
define the final result as “1” for 1, 2, or 3, and “6” for 4, 5, or 6, the
expected value is still 3.5. But this value can never actually be reached, as the path
taken through the system determines the outcome. There is some research
on the mathematics of non-ergodicity in cognitively oriented research fields,
including cognitive neuroscience
Without ergodicity and/or stationarity, the central limit theorem fails. This is not a minor inconvenience, but equivalent to the collapse of the single bridge that spans the space between two tall mountains – without the central limit theorem, mathematically, there is no connection between prior observations and a presumed population. There is no reason to assume that words counted in one corpus hold any precise, probabilistic information about what will be found in another. In a changing system, the past has limited capacity to predict the future. While one may still choose to trust corpus data as empirical evidence, there is little justification to rely on concepts like p-values or eefct size to safeguard against chance results. 1.3
The epistemology of statistics over words is unsatisfying for reasons that go beyond mathematical concerns. Suggesting that the production of words can be modeled stochastically, i.e. as a distribution of probabilities, entails at least two philosophical extensions: 1. There is a stochastic system, perhaps in a latent superpopulation3 (as it is sometimes conceptualized in the social sciences), that produces unchanging and stochastically deterministic output; 2. Linguistic data collection is capable of capturing samples of this output.
Both of these extensions are, in fact, rather strange. The first suggests that
while we cannot know when certain things will be said, or which words will
co-occur; the conversations we have are eefctively pre-determined in the
stochastic system. This implies that for all new word forms that are
productively generated, and for all new processes and items in the world that did
not exist until recently and are now named (such as to google or Tesla), the
stochastic system has always held a quantitatively predefined slot. This idea is
utterly alien to linguistics, and is unlikely to be entertained by anyone in the
ifeld. On the contrary: usage-based linguistics metaphorically draws on
systems theory by suggesting that language is a complex, adaptive (i.e. evolving)
system – one that is neither stationary, nor ergodic
The second extension touches on the problem of the representativeness of
corpus data, and specifically of corpus data that is compiled in what is
sometimes called a quasi-experimental design. For example, in studying learner
language, one may try to control for certain variables such as topic, task,
setting, degree of formality, etc., which typically results in small to
midsized corpora like the learner corpora Kobalt
This approach is epistemologically risky in two ways. First, unlike newspaper archives or historical sources, it does not involve naturally occurring data that has been collected from existing language – it works with language that has been intentionally created and that may never have existed without the researcher’s intervention. This means that the data collection created the presumed superpopulation from which certainty or eefct size is then derived, rendering the argument circular. Second, if the amount of data turns out to be too small for certain types of analysis (as it typically does for collocations, see section 1.2), one may be tempted to collect more data of the same kind. Doing so is not the same as taking samples, however. Rather, it means expanding language in a way in which it probably would not have developed on its own. Continuing with this method until data sizes of two higher orders of magnitude are reached – for example by collecting tens of thousands of texts written by learners of German instead of a few hundred – would: a) skew the proportions of all documented data of learner German significantly; b) interfere in the writing development of hundreds of thousands of learners of German; and c) create a population, rather than sampling from one.
Consider for a moment the following analogy: if a naturalist went into the woods to count black and blue birds belonging to a particular species, the birds would have existed without their involvement. Provided that the proper methodology is chosen and applied well, reliable insights into the features of this population are to be expected. However, if the naturalist began breeding the birds in order to gain control over data collection, and accidentally bred green birds never before observed in the wild, no amount of new data would allow them to infer from their laboratory population to the original population, as the existence of these birds has permanently changed the entire species. In other words, in an attempt to create representative amounts of data (birds), our naturalist will have created irresolvable pathdependency in their research. The population that could have been inferred to from the initial sample in the wild is now no longer representative of the whole species, and thus of little to no use for comparison or significance testing.
At first glance, this may seem like a far-fetched and purely theoretical problem, but the reality of learner language is that most of it is not spoken in the context of the argumentative essays that are typically found in learner corpora. Instead, it is found in: a) immigration ofices and areas with high density immigrant populations; b) middle schools; and c) tourist destinations – none of which are prone to eliciting essay-length argumentative texts on controversial topics in a formal written setting. This means that the compilation of learner corpora as they exist today already interferes in learner language and influences projections of a latent superpopulation.
This is not to say that there are no quantifiable patterns or measurable diefrences between language cohorts. However, it is important to map the demands of the mathematical model to the subject matter, and it appears as though frequentist statistical interpretations of lexical distributions oefr a rather unlinguistic perspective. The same is not necessarily true of other types of linguistic distributions. More abstract syntactic features like word order, cases, part-of-speech distributions, determiner systems, and so on, are much more stable historically, and converge quickly even in data of limited size. They also exhibit lower variance than word distributions and are greatly influenced by changes in subcategories, as well as in any of the complementary categories. For example, phonological changes may influence the obligatoriness of articles, thereby also influencing the case system, as is the case in diachronic language change. Using diefrent words and inventing new ones, on the other hand, does not have the same eefct. It is therefore quite possible that probabilistic analysis of syntactic aspects in corpora is mathematically and conceptually valid. For largely lexically oriented analysis, this does not appear to be the case.
It should be noted here that criticizing the application of statistical methods on lexical aspects of corpora is not merely a matter of methodological taste or belief. If words do not have probabilities, probability-based measures are meaningless in the same way that it is meaningless to measure the temperature of a philosophical debate (even if it is heated) or the width of the history of Europe (even if it is wide-ranged). Validity matters in scholarly research, and that includes both the internal validity of the mathematical model, and the mapping of mathematical to subject-specific concepts and empirical observations. 2
With an abundance of problems around statistical measures in corpus-based
lexical and lexicosyntactic analysis, and with the practical consideration that
reliable measures of smaller data are necessary, graph metrics would appear
to possess intrinsic appeal to corpus linguistics. Yet they are practically
unused. In fact, graph theory in general is underrepresented in
linguistics, despite some early work centered mainly around Markov chain
modeling
The first example shows the modeling of lexical information and its
measurement through the metric of Louvain modularity
In conducting this analysis, the main research question was whether
graphs of this type show diefrent degrees of structuredness for diefrent
stages of foreign language acquisition in learners, as well as between learners
and native speakers. The underlying concept of coselectional constraint
or idiomaticity describes the tendency of native speakers to constrain their
choice in word combinations to a relatively small set out of a potentially
large combinatorial space. For example, the verb chosen for describing the
action of cleaning teeth in English is brush, whereas the German equivalent
is putzen (‘to clean’). While clean teeth would be understood and is
semantically acceptable, the combination is highly unlikely to occur in a text written
by an English native speaker (unless it was describing the act of having one’s
teeth professionally cleaned by a dentist). This particular example would
typically be learned early in both unguided learning and in language classes.
However, there are vast amounts of subtle constraints of this kind, and they
are known to be very dificult to master even at an advanced stage of second
language acquisition
The central hypotheses of the present study are as follows: 1. Graphs are more structured in more vs. less advanced learners; 2. Graphs are more structured in native speakers than learners; 3. Learners show a u-shaped learning development in their coselectional structuredness.5 4Since these graphs are generally to large and detailed to be legible in print, further visualizations, alongside the data and scripts for analysis, are available in a separate Zenodo repository (doi: 10.5281/zenodo.3584091).
5The linguistic background of this is discussed in detail in
The measure used to capture these eefcts is Louvain modularity
6It appears that Louvain modularity prefers certain community sizes and constellations over others, raising the question of whether other algorithms may be better suited for corpus-linguistic analysis. This issue has not been considered in the analysis presented and remains to be addressed in future research. many nodes that are more randomly connected to other parts of the graph. A graph of negative modularity contains fewer edges between nodes than would be expected by chance, which does not appear to be the case in lexical graphs generally. Importantly, modularity is not an artifact of graph size. As exemplified in Figure 3, graphs of the same size in terms of nodes and edges can be more or less structured, and thus possess higher or lower modularity.
An analysis of a range of Kobalt subcorpora, each represented by five samples in Figure 4, shows that hypotheses 1 and 2 are met by the data, and hypothesis 3 is met in the Belarusian, but not in the Chinese subcorpus. A sliding window analysis of the same data is presented in Figure 5. Windows of 15 texts were used for each data point. Window 1 is created from texts 115 arranged by test scores, window 2 from texts 2-16 and so on. This analysis shows that, beyond the grouped samples, a clear trajectory emerges over test scores, which suggests that test scores and graph structures indeed correlate in meaningful ways. If they did not, the trajectory would be expected to be much more erratic.
For comparison, a statistical analysis of the same data based on lexical association measures was also performed. However, despite cohort homogeneity and the limitation to an identical task/topic, there are very few similar or identical collocational pairs across subcorpora. Absolute occurrences quickly dwindle into very low numbers (below 10), which means that a high and low rate of co-occurrence would falsely be identified from the same order of magnitude. While 6 is of course three times as many as 2, the absolute diefrence is not very telling in terms of the underlying diefrence in coselectional association. In addition, the combinatorial power of lexical material as it occurs even in small corpora is counter-intuitively large and poses a hindrance to statistical interpretation. In fact, the number of verb and accusative object lexemes as they are used in a small subcorpus of Kobalt (21 texts, 148 unique verbs, 304 unique nouns in accusative object position) results in a potential of 44,992 combinations. Even if one were to assume that only 10% of those are semantically possible, and that only 10% of the total 726 realizations of verbs and adjective objects in the subcorpus are freely combined in the first place, this results in 72 draws from 449 elements. The combinatorial potential of this is higher by several orders of magnitude than the estimated number of atoms in the universe (3.352 · 1089 vs. 1078 – 1082). Constructing a scalar measure of coselectional constraint statistically or stochastically would require a definition of thresholds for relative frequencies at which the occurrence of word pairs would be considered within or outside the range of expectation. However, drawing any specific combination from this vast combinatorial space is extremely unlikely. Overall, results from the statistical analysis remain inconclusive and dificult to interpret. This is discussed in detail in Shadrova (2020, chapter 4).
In contrast, the graph-based analysis shows clear trajectories and eefcts in line with hypotheses 1 and 2, as well as a linguistically meaningful divergence from hypothesis 3. 2.2
The second example is not yet quantified and serves to demonstrate the underlying modeling problem – a suggestion as to its quantification appears at the end of this section.
In the previous analysis, syntax is involved as a hierarchical lexical filter:
only those words that occur in certain syntactic slots of the verb are
considered in the structural analysis. Unlike positional models of collocation,
which consider words in a window of n tokens around the target word, a
syntactic model of this kind is able to eefctively filter for the relevant
collocations in languages with flexible word order such as German. However, in the
theoretical framework of usage-based linguistics that many corpus linguists
ascribe to, a strict division of lexicon and syntax is not usually presumed.
Instead, a number of models suggest the existence of a continuum, partial
overlap, and/or interdependence of lexical and syntactic elements. In fact,
some strands of grammar theory are largely focused on the very interface of
lexicon and syntax – these include construction grammar, which is entirely
based on the idea of the inseparability of lexicon and syntax
While the idea of the syntax-lexicon continuum is appealing at first glance, it quickly runs into conceptual problems and is dificult to operationalize. For one thing, it requires clarity over whether the compared elements constitute categories/variables or exemplars. Since many words are exemplars (like on, after, if ), but many are also categories (like to be with its paradigm are, were, is, etc.), this is not trivial. Second, syntax and higher-order functions structure text in such a way that once a category is decided, many others fall into place. At the same time, there is plenty of space for inter- and intraindividual variation.
So far, corpus studies have tended to look at the coselectional patterns of
individual lexemes or types of lexemes, or individual verb-argument
structures or classes thereof. For example,
Technically, the studies mentioned do not model syntax and lexicon in the same space – rather, they discuss the combinatorics of two sets of elements, where the sets are selected by semantic or syntactic rules, similar to the filtering function of syntax in the previous analysis.
The mapping of two sets of elements, however, can be summarized and
made explicit as one system in a graph. An example is provided in Figures
6 and 7. In these graphs, lexemes and syntactic functions such as
grammatical subject (SUBJ) and direct or accusative object (OBJA) are modeled as
nodes, and the syntactic dependency between all syntactic functions is
encoded explicitly. At the same time, forces of association between lexemes
and syntactic functions are also represented in the graph. Thus, a
connection between what would be rows of a table of factor combinations are made
explicit in the graph structure. The visualization is produced by a so-called
force algorithm, a physics simulation in which higher frequency of
occurstructure grammar (HPSG,
A closer look at the two figures reveals that the graph for intermediate Belarusian learners of German in Figure 7 shows much more proximal positions (i.e. shared lexemes) for subjects and accusative objects compared to the native speaker graph in Figure 6, while objects appear to cluster in a more coherent group in the native speaker graph. From a linguistic perspective, this is interesting in two ways.
First, as has already been noted, syntactic category distributions tend to converge quickly in corpus data, and in many cases do not diefr much between diefrent cohorts. Despite some minor pattern deviations, when the same data is analyzed for distributions of syntactic categories, it does not yield any substantial diefrences according to native language group. However, through the combination of syntactic categories and lexical items, substantial diefrences in the graphs of the two cohorts do emerge. This suggests that graph structure is capable of capturing linguistically meaningful eefcts that are not encoded in the individual co-occurrences of lexemes in the syntactic structures themselves, but only in their interrelation: it appears that lexicosyntax lives in the cross-systemic structure, and not in the combinations of words.
Second, the specific diefrences between the two graphs correspond to higher-level linguistic concepts: a higher interchangeability of subject vs. object lexemes may indicate passivization, or it might point towards diefrences in anaphoricity (i.e. the types and structure of referentiality to previously introduced discourse referents). This is particularly interesting because Russian and Belarusian are partially pro-drop languages, meaning that subjects can be left out or go unrealized in many contexts, which raises the plausibility for diefrent encodings of the relation between subjects and objects in the minds of native speakers of those languages.
Figure 8, which shows the same type of graph for intermediate Chinese learners of German in Kobalt, underscores this point. Here, subjects and predicates are more closely aligned, which may be rooted in typological aspects of Mandarin Chinese; a theme-rheme language in which a sentence topic is presented and then commented upon, and which can thus be plausibly mapped to a subject-copula-predicate construction in German. However, since these analyses are post-hoc, and the graphs involved are opportunistically derived from previously analyzed data, more research is needed to verify their genuine usefulness in the study of lexicosyntax.
This includes the need for quantification. Even if the visualization suggests certain eefcts, these can only be verified through a comparison with other graphs from the same as well as other cohorts. Assessing diefrences of this kind visually can be tedious and unreliable. Furthermore, a quantification would allow for an analysis of variance (i.e. more or less similar graphs of the same cohort), while a visual assessment can only extract obviously diverging patterns.
The visualization encodes structural information that is also present
quantitatively in the graph itself, and may be used for external quantification.
One possible way to approach this would be through non-negative matrix
factorization (NMF,
While conceptually and computationally well-implemented, the central question arising here is what should be encoded in the matrix to ensure linguistic validity. One approach would be to span a matrix for all nodes times all nodes (i.e. both lexemes and syntactic functions). But since words can only occur in syntactic slots, this would leave most cells empty, perhaps overestimating similarity. The other would be to abstract from the concrete lexemes and span the matrix for syntactic nodes only, entering only the number of shared lexemes in each field. However, this implies a double dimensionality reduction and leaves out not only the exemplar information, but also structural information that is relevant to the graph visualization and relates to linguistically meaningful aspects of the graph structure, such as the issue of how node clusters are distributed in relation to other node clusters with which they do not share information.
This goes to show that computing just any graph metric will not sufice for the development of a linguistically interpretable method of graph measurement. As for all methodological development, in-depth validation, replication, and extension to new data are of crucial importance - but perhaps more so is the embedding into the theoretical frameworks of linguistic study, and the triangulation with results obtained through other methods. 3
Graph-based modeling could bring new perspectives into quantitative corpus linguistics. Its major advantages over frequentist statistical modeling are that it: (a) captures relational information directly, without the internal triangulation of various measures; and b) does not infer to concepts such as stationary and ergodic stochastic systems or a stable and existing superpopulation. However, it is going to take extensive further research to fully assess the potential of graph metrics. This is particularly true since unlike in mathematical graph theory or network analysis, the words of a language are both representative of linguistic categories and relevantly individual. That is to say, each item diefrs in important aspects from all other items, which means that, generally speaking, a full abstraction over all words will not be well-aligned with the linguistic model. Detailed theoretical modeling and empirical conceptual validation are therefore necessary to ensure that graph metrics do indeed measure linguistic aspects and not spurious information of a hyperstructure that is only marginally important in linguistic terms.
An example of such an eefct is the measurement of the in- and out-degree
of lexemes in corpora, a measurement that consistently detects so-called
small-world-efects . These are graph structures in which some nodes have
many in- or outgoing edges, while most others have only few, and have been
measured consistently through a range of published research across many
corpora
With the development of suficient computing power, graph-based analysis has become more advanced and central not only in traditional STEM subjects, but also in the humanities and social sciences. Accordingly, there is an ongoing development of new metrics. Some of these metrics are derived from graph theory directly, which means that they abstract more strongly from individual nodes and focus on the more abstract properties of node and edge classes. Isographs, i.e. structurally identical (sub-)graphs that may diefr significantly in their node contents, are a particularly relevant issue in this context. Their existence may or may not have implications for the underlying theoretical model, and should be considered in any application of graph metrics to a subject-specific research question.
In order to make the most eficient use of graph metrics in linguistics and other fields of study, it is important to consider implications of this kind not only mathematically, but also epistemologically by asking questions like the following: 1. Is the theoretical model well-represented by the mathematical model? 2. Are there aspects to the mathematical model that may interfere with subject-specific theoretical underpinnings and/or interpretations of the data? Much more validation and research is required to identify concerns in this regard, including: the application of measures to more data; the detailed modeling and mapping of theoretical to mathematical concepts; the calibration and triangulation of metrics; and the development of evaluative frameworks for all of the above. 4
Graph metrics are very new to the field of corpus linguistics, especially in core-lingustic (rather than psycholinguistic or computational) research. The analyses discussed in this paper suggest that graph-based modeling and quantification may provide an alternative to operationalizations in the more traditionally applied framework of frequentist statistics. They may even provide solutions to some of the common practical problems of corpus linguistics, since by encoding information at a higher density, graphs allow for the quantification of small data. By abstracting from individual entities such as lexemes they also allow for structural assessment without the triangulation and comparison of a large number of words or word pairs, which implies fewer artifacts from text length, a problem that is notorious with all corpus research.
With advances in computing power and the ease provided by graph
infrastructures such as neo4j, the corpus search engine graphANNIS
Kobalt was preprocessed with TreeTagger8 and Malt Parser
8https://cis.uni-muenchen.de/~schmid/tools/TreeTagger/
I would like to thank Joris van Zundert for his very kind and constructive comments on a previous version of this essay, which have greatly helped to improve its focus and clarity. All remaining errors are of course my own.
Leibniz-Institut für Deutsche Sprache (2019). Deutsches Referenzkorpus/Archiv der Korpora geschriebener Gegenwartssprache 2019-I (Release vom 18.03.2019). http://www.ids-mannheim.de/DeReKo.
R Core Team (2015). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, https://www.
R-project.org/.
Reznicek, M., Walter, M., Schmidt, K., Lüdeling, A., et al. (2010). Das Falko-Handbuch: Korpusaufbau und Annotationen. Institut für deutsche Sprache und Linguistik, Humboldt-Universität zu Berlin.
RStudio Team (2015). Rstudio: Integrated Development Environment for R.
RStudio, Inc., Boston, MA, http://www.rstudio.com/.
Sag, I. A., Boas, H. C., and Kay, P. (2012). Introducing Sign-Based Construction Grammar. In Boas, H. C. and Sag, I. A., editors, Sign-Based Construction Grammar, pages 1–30. CSLI Publications.
Wachs-Lopes, G. A. and Rodrigues, P. S. (2016). Analyzing Natural Human Language from the Point of View of Dynamic of a Complex Network. Expert Systems with Applications, 45:8–22, DOI: 10.1016/j.eswa.2015.09.020. Wickham, H. (2007). Reshaping Data with the reshape Package. Journal of Statistical Software, 21(12):1–20, http://www.jstatsoft.org/v21/i12/.