In this contribution we present a use case of the application of big language data and digital methods such as natural language processing, machine learning, and network analysis in the fields of digital humanities and linguistics by characterizing and modeling the diachronic dynamics of lexical networks. The proposed analysis will be based on two corpora containing 20 years of data with billions of tokens. 2012 ACM Subject Classification Human-centered computing → Social network analysis; Computing methodologies → Natural language processing; Computing methodologies → Machine learning Evidently, languages are constantly subject to change. For example, on the word level, new items enter the vocabulary (i.e. the lexical system) of a language, others cease to be used by speakers, and some established words may change their meaning. Characterizing and modeling these dynamics has a broad field of applications including linguistics, natural language processing, digital humanities, artificial intelligence, computer sciences and cognitive sciences. In the project Diachronic Dynamics of Lexical Networks we therefore want to investigate, 1) how and why lexical systems of natural languages change, thereby considering social factors such as influential individuals as well as cognitive factors [3, 6, 11]; and 2) how language change in the lexical domain can be measured. Here, approaches such as corpus analysis and statistical analysis of word-frequency trajectories are typically employed in the field of diachronic linguistics (i.e. the analysis of language over time). Figure 1, for example, shows frequency trajectories of two lexical innovations. Recently, however, network-based approaches [1] have become increasingly important in this context [16, 9, 10, 4]. The advantage of network-based approaches for the analysis of lexical dynamics is that they allow to study the semantic properties of words in addition to word frequency, since the meaning of a word is closely related with its context, i.e. other words it co-occurs with frequently. So, we can track lexical innovations (i.e. new words) introduced by influential individuals (politicians) and systematically analyze contextual, i.e., semantic, changes of these words. More specifically, our project focuses on the following research questions:
Funding The project Diachronic Dynamics of Lexical Networks (DYLEN) is funded by the ÖAW go!digital Next Generation grant (GDNG 2018-020).
What is the role of influential innovators (e.g. politicians) in lexical change?
Does network science give more detailed answers about language change than traditional frequency based methods? Which computational method is most suitable to analyze the evolution of lexical networks through time?
How can we enrich the digital-humanities toolbox with the output of the project? 2
As data sets we use two diachronically layered big text corpora available for Austrian German: the Austrian Media Corpus (AMC), containing more than 20 years of journalistic prose [15] and the ParlAT corpus, covering the Austrian parliamentary records of the last 20 years [21]. The journalistic prose included in the Austrian Media corpus comprises Austrian press agency releases, most Austrian periodicals such as all daily national newspapers as well as a large number of the major weekly and monthly magazines, in total 53 different newspapers and magazines.
Moreover, the Austrian Media Corpus contains also transcripts of Austrian television news programs, news stories and interviews [15]. In total, the AMC contains 10.500 million tokens with 40 million wordforms and 33 million lemmas. The ParlAT corpus contains the stenographic records, in German called “Stenographische Protokolle” from the XX to the XXV legislative period (1996 – 2017). So they are not transcripts of recordings but shorthand records. The corpus size is 75 million tokens with over 0.6 million word forms and 0.4 million lemmas [21]. Both corpora are tokenized, part-of-speech tagged and lemmatized.
Crucially, the two corpora cover lexical innovations both directly in the linguistic output of politicians as well as indirectly in media texts. Thus, the two corpora provide an ideal testing ground for the hypotheses outlined above.
To address the questions mentioned in section 1, we analyze the above described data sets, namely the Austrian Media Corpus (AMC), and the ParlAT corpus. In addition, we will provide an easy-to-use online tool to enable researchers to do diachronic analyses of lexical networks by themselves. Our approach requires the following steps, which are schematically depicted in Figure 2: 1. NLP pre-processing and data model development: For both corpora (i.e. AMC and ParlAT) a number of data pre-processing steps have already been conducted, i.e. tokenization, part-of-speech tagging, segmentation, lemmatization, named-entity (NE) recognition. Parts of the existing NE recognition will be enhanced using machine learning and semantic knowledge bases, e.g. Wikidata [19]. Furthermore, we will introduce a comprehensive data model combining both corpora and all metadata. In addition we will compile a list of relevant Austrian politicians as we want to analyze their impact on language change. 2. Network construction and description: A systematic procedure will be defined to 1) construct different co-occurrence networks (i.e. networks, where nodes represent identified entities, e.g. politicians, as well as nouns, verbs or adjectives and edges represent the co-occurrence of these nodes in a sentence, paragraph or document) for different time intervals (i.e. all documents within a week, a month, a year, etc.); and 2) extract basic properties (e.g. number of nodes/edges, clustering, centrality) to describe the networks. Together with frequency of occurrence, these properties can be interpreted cognitively and semantically [9, 10]. 3. Network analyses and comparisons: In-depth analyses of the resulting networks will be conducted using network analysis and visualization. As the number of networks is assumed to be quite large, an approach will be developed to systematically compare these networks over time and across the two corpora. Therefore, different methods from network analysis, machine learning and statistical modeling will be tested. This will allow to identify relevant parameters (e.g. network properties) to capture diachronic developments. 4. Modeling diachronic developments: Statistical models including time-series analysis with generalized additive models and time-series clustering techniques for analyzing the co-evolution of parameters (see 3.) in multiple networks will be employed. 5. Interactive web application: A web-based interactive tool will be developed that retrieves the constructed networks and allows to explore, analyze and visualize them.
The technical implementation, which will build on an existing prototype [7], will mainly be based on Python and appropriate libraries [5, 8, 12, 14], on Neo4j [20] to store the network and on software for big data analysis , e.g. Apache Spark [23], Hadoop Yarn [17, 18], HDFS [17]. Gephi [2] will be used to visualize the graphs, and R for the statistical analyses [13, 22].
We expect our project, which has to face specific challenges such as NE recognition for Austrian German and the analysis of two large-scale diachronic corpora, to contribute to the understanding of the role that influential speakers and other linguistic factors play in lexical change by analyzing big amounts of language data. Since we cover both the linguistic output of influential speakers (ParlAT) as well as their linguistic reflex (AMC), we can test if lexical innovations introduced by these individuals behave differently than other lexical innovations. This allows us to disentangle social effects from cognitive effects in the process of lexical spread. For example, by analyzing the evolution of the clustering coefficients of
Figure 2 Work flow in the DYLEN project. Lexical networks are generated from diachronically layered corpus data. Network properties of lexical items, such as semantic neighborhood density, are then investigated across time to derive insights into semantic change. networks around lexical innovations, we can test if increase in frequency is accompanied by semantic widening effects; a correlation which is expected given results from research on language change [3, 6, 11].
We also seek to foster network theory as a suitable tool to analyze and make sense of diachronic language data in the linguistic research community.