As our world becomes more globalized and interconnected, the boundaries between languages become increasingly blurred (Bullock, Hinrichs & Toribio, 2014). But to what degree? To date, researchers have no objective way to measure the frequency and extent to which languages might be mixed. While Natural Language Processing (NLP) tools process monolingual texts with very high accuracy, they perform poorly when multiple languages are involved. In this paper, we offer an automated language identification system and intuitive metrics-the Integration, Burstiness, and Memory indices-that allow us to characterize how corpora are mixed.
When multilinguals are in interaction with one another, some degree of language
mixing is likely to take place (talk & Choudhury, 2016). Indeed, the phenomenon
has been attested since the ancient world
Bullock & Toribio, 2013; Toribio & Bullock, 2016 ] 2. diana@Dianier1019 Oct26 for some reason I’m starting to talk Spanglish like
I’ll start off talking American despues mi mexicana quiere salir [ Twitter ] 3. . . . but she had the posture and speech (and arrogance) of una muchacha respetable [ Junot Díaz, Brief Wondrous Life of Oscar Wao ] 4. Après, c’était bien easy de l’embarquer pour tuer les autres fuckers... qui ont détruit notre great game. [ Bon Cop Bad Cop ] (Ball et al. 2015) ‘After, it was real easy to set out to kill the other fuckers... who destroyed our great game.’ For those interested in the forms, meanings, and dispersion of multilingual language use, observing variation in C-S in reliable, reproducible, and language independent ways is essential. In seeking to understand C-S, it would be advantageous to have the ability to compare the frequency and the degree to which the languages represented in different corpora are intermingled. Herein we present our methods for quantifying and visualizing language mixing in corpora and apply our methods to the analysis of mixed language texts of various genres and of different language pairings. Our contributions in this paper are as follows: (i) to provide a brief explanation of the models that we built to identify the language of each word token in a corpus; (ii) to describe the metrics that we use to calculate and to visualize the frequency and degree of language mixing found in a corpus and sub-corpora; (iii) to describe the corpora that we model; and (iv) to demonstrate the application of the metrics to these corpora to quantify and visualize the results. We conclude with implications for future work in the digital humanities, in linguistics, and in NLP research. 2
Language Identification. Corpora may contain more than one language for a variety
of reasons, including a change in author from one sub-corpus to the next
Our system is an adapted version of the language identification system of
Solorio & Liu
The Integration Index. Barnett et al. (1999) developed the Multilingual Index (M-Index) to quantify the ratio of languages in oral speech corpora based on the Gini coefficient to measure inequality of a distribution 1. The values range from 0 (monolingual) to 1 (perfectly balanced between two or more languages), permitting a measure of how ‘monolingual’ or, for present purposes, ‘bilingual’ a given text is. The M-index is calculated as follows where k is the total number of languages represented in the corpus, p j is the total number of words in the language j over the total number of words in the corpus, and j ranges over the languages present in the corpus:
(k 1
å p j2 1) å p j2 : To supplement the M-Index, we created the Integration-index, a metric that describes the probability of switching within a text (Guzmán et al. 2016) (see also Gambäck & Das (2014, 2016)). We calculate the I-index from summing up the probabilities that there has been a language switch (from Lang1 ! Lang 2 or viceversa). The values of the I-Index range from 0 (a monolingual text in which no switching occurs) to 1 for a text in which every other word comes from a different language, i.e. every word represents a switch in language. Given a corpus composed of tokens tagged by language flig where i ranges from 1 to n, the size of the corpus, and j ranges from i + 1 to n, the I-index is calculated by the following expression:
n 1 1 iå<j n
S(li; l j); where S(li; l j) = 1 if li 6= l j and 0 otherwise, and the factor of 1=(n 1) reflects the fact that there are n 1 possible switch sites in a corpus of size n.
Muysken (2000) presents a typology of mixing, identifying three types of patterns: insertion, in which an other-language item is inserted within the a string of a base language (A A A B A A), alternation, in which the base language changes (A A A B B B), and congruent lexicalization, in which the structures of the two contributing languages overlap (A\B A\B A\B) so that either language can occupy a position in a string. The M-index and the I-index are calculated at the lexical level, which does not capture the contribution of syntax. Nonetheless, we use the I-index as a proxy measure of how much CS is in a document, where the value 0 represents a monolingual text with no switching and 1 a text in which every word switches language, a highly unlikely real-world situation. It is an empirical question whether or not there is a threshold of integration beyond which a C-S is perceived as inauthentic.
1A reviewer points out that Shannon entropy may also be used for measuring diversity in text.
Intermittency. To refine our profile of C-S within a corpus, we utilize measures
of intermittency from research on complex systems
Burstiness (st=mt 1) (st=mt + 1) = (st mt) ; (st + mt) where st is the standard deviation of the language spans and mt is the mean of the language spans.
Burstiness, by considering the length of these language spans, provides one measure of the intermittency of C-S. However, the ordering of these language spans in time is important, as it is possible for two corpora to have identical language span distributions—and thus the same Burstiness-index—that nonetheless appear very different to a reader due to how the switch points are ordered in each corpus. In Goh & Barabási’s system, this is measured as Memory, a measure of first order autocorrelation between the language spans. The computation of memory involves going through the language spans in order, measuring the extent to which the length of one language span is influenced by the length of the previous language span. Memory is calculated as:
Memory nr 1 nr 1 (ti å 1 i=1 m1)(ti+1 s1s2 m2) ; where nr is the number of language spans in the distribution, ti is the current language span, ti+1 is the language span after ti, s1 is the standard deviation of all language spans but the last one, s2 is the standard deviation of all language spans but the first, m1 is the mean of all language spans but the last, and m2 is the mean of all language spans but the first. Memory is bounded within [-1, 1]: a signal closer to -1 indicates that the language spans are negatively autocorrelated, meaning that spans of discourse in one language tend not to be similar in length to the span of discourse in the language preceding it. That is, long spans are followed by short spans and short spans are followed by long spans. Conversely, a signal closer to 1 indicates that the language spans are positively autocorrelated, meaning that the span of discourse in one language tends to be similar in length to the span of discourse in the language preceding it. In summary, Memory and Burstiness are mechanisms that give a complete signature of the intermittency—the time order and frequency—of C-S for a corpus, allowing for meaningful comparison of C-S behavior between corpora. It is important to note that this method is not exclusive to C-S behavior, and is a time series analysis that may be applied more generally to any type of events that may occur in corpora. The crux of the strategy is to iterate over the corpus, marking when the events occur, thereby generating the distribution of time spans between the events. The memory and burstiness metrics then describe the intermittency of that event.
Data and Analysis. The data that we analyzed comprises three texts of distinct genres and languages, each of which is touted for its bilingualism. The first is the film transcript of the FR-ENG bilingual buddy movie Bon Cop Bad Cop (BCBC) (2006). The French and English versions of the transcript were downloaded from subtitles.com and the final transcript (n = 13,502 words) was pieced together by watching the film frame by frame and choosing the appropriate language from the subtitles. The other two are Spanish—English written texts that are available online: Killer Crónicas (KC) is an 40,469-word novel by Susana Chávez-Silverman in multilingual (and multi-dialectal) emails that present extensive SP-ENG C-S, and Yo-Yo-Boing (YYB) is 58,494-word novel comprised of alternating and mixed SP-ENG poetry, and essays by Giannina Braschi. We annotated each text for language and quantified the switching as outlined above. 3
The results of these metrics as applied to the three texts are found in Table 1. A
comparison of the M-index for these texts reveals that the novels YYB and KC are
nearly equally balanced between SP and ENG, with M-index values that are close
to 1; the film, BCBC, with an M-Index of 0.86, is less balanced between languages
than the novels. The I-index serves to differentiate the two balanced texts and
indicates that the languages are more closely integrated in KC than in YYB despite
their similar M-indices. BCBC shows an integration value that is intermediate
between YYB and KC. In terms of burstiness, BCBC has the highest value of the
three texts, indicating that there is not a regular pattern to the C-S but rather there
are moments in the film in which characters switch languages frequently followed
by moments where little switching occurs. Overall, the Spanish—English novels
are very different from one another; while YYB shows bursts of C-S throughout
the text, the low Burstiness value for KC shows that C-S occurs with regularity
throughout the text. Finally, both KC and BCBC, texts in which the probability of
C-S is relatively high compared to YYB show a neutral value for memory, which
appears to be the normal complexity measure for texts
The metrics that we have proposed and tested here are useful for distinguishing the types of mixing patterns found in corpora. They tell us, for instance, that any random selection from KC, but not YBB, is likely to contain frequent switching events since the text is characterized by short spans between switching events that recur regularly. The Canadian movie, BCBC, would also be a good candidate for the study of C-S but because switching is burstier relative to KC, one would need a larger sample of that text than of KC in order to capture language alternation. Finally it is much less probable that choosing a random section from YYB would yield any switching phenomenon because there are long spans within the book in which no C-S occurs. These methods and models can be applied to any languagetagged corpora in which more than one language appears. This would allow us to compare patterns of language mixing across various corpora in a standard and reliable way, a task that cannot currently be achieved in a straightforward fashion. Additionally, these metrics enable scholars from any discipline in the humanities to visualize their data before they begin to analyze or model it. Since these measures quantify the actual frequency and degree to which languages are intermixed in a sample, they may aid in dispelling popular (and sometimes scholarly) misconceptions about the nature and extent of C-S among multilingual societies, communities, and individuals. In our future work, we intend to compare across corpora produced with the same language pairings, for example, to quantify and visualize the differences between the ‘Spanglishes’ of Miami, El Paso, Los Angeles, and New York, and to compare these, in turn, to ‘Hinglish’ (Hindi—English) corpora in India and England and to French—Arabic in Europe and the Maghreb with the intent to model the variation inherent in code-switching worldwide. [3] Kalika Bali and Monojit Choudhury. NLP for code-switching: why more data is not necessarily the solution. In Empirical Methods in Natural Language Processing (EMNLP). The Association of Computational Linguistics, 2016. [4] Kelsey Ball, Barbara E. Bullock, Gualberto Guzmán, Rozen Neupane, Kristopher S. Novak, and Jacqueline L. Serigos. Bon cop, bad cop: A tale of two cities. In Transcultural Urban Spaces, 2015. [5] Ruthanna Barnett, Eva Codo, Eva Eppler, Montse Forcadell, Penelope Gardner-Chloros, Roeland van Hout, Melissa Moyer, Maria Carme Torras, Maria Teresa Turell, Mark Sebba, Marianne Starren, and Sietse Wensing. The LIDES Coding Manual: A document for preparing and analyzing language interaction data Version 1.1–July, 1999. International Journal of Bilingualism, 4(2):131–132, June 2000. [6] Mirjam Broersma and Kees De Bot. Triggered codeswitching: A corpusbased evaluation of the original triggering hypothesis and a new alternative. Bilingualism: Language and cognition, 9(01):1–13, 2006.