The paper discusses the differences between collocations extracted from a number of Russian dictionaries paying attention to their frequency characteristics based on corpora. The aim of the study was, first, to analyze how collocations and set expressions are described in Russian explanatory and specialized dictionaries and to what extent their data coincide with each other, and, secondly, to investigate how collocations presented in dictionaries are reflected in text corpora. This will make it possible to examine the interrelation between the “manually” collected data and modern corpora (the Russian National Corpus and ruTenTen). We tested the following hypothesis, i.e. high collocation frequencies correspond to the fact that the item is represented in several dictionaries. In our paper we considered 180 collocations built according to the “adjective / participle + noun” model. The results show the heterogeneity of the dictionary data while the choice of lexical items does not coincide with its frequency characteristics: the examples are low-frequency and about 34% are absent in the disambiguated subcorpus. Explanatory dictionaries and collocation dictionaries show the smallest overlap.
Our project deals with the process of building a database that will represent Russian
collocations extracted from dictionaries and corpora [
Since dictionaries are an important source of data about collocations, it is important to analyze them in order to understand the possible distinctions. At the same time, it can be tricky to compare different dictionaries.
In the paper we consider collocations which were extracted from six Russian dictionaries, analyzing how they are reflected in corpora of the Russian language. Within the framework of our research, we dwell on two tasks. First, to analyze how recurrent word combinations are presented in different dictionaries and how much they coincide with each other. Secondly, to investigate the extent to which collocations that are reflected in dictionaries can be found in corpora and, therefore, trace the intersection between “manually” collected data and modern corpora. 2
The tradition of Russian lexicography has a rich history, however, there are not many projects dealing with collocability in Russian and moreover based on corpus data or created by automatic methods. At the same time corpora are seen as the main source of language data in Western lexicography and up-to-date projects implement them (Macmillan Dictionary).
The issue of selecting collocations is crucial in lexicography, and not only for
monolingual dictionaries, representing “the most controversial and vulnerable part of
almost every bilingual dictionary” [2, p. 61]. Atkins and Rundell [
What kind of phrases to include in a computer dictionary is discussed, for example,
in [
The idea of comparison between dictionaries and corpora attracted attention from
scholars a few decades ago. The interest was focused on automatic extraction (either
rule-based or statistical one) from the sources and their further evaluation. The results
of the analysis complement each other and can be applied for constructing NLP
lexicons [
There is also a number of works involving comparison between text corpora (for
example, [
The merging of dictionaries from different sources implies not only a single lexicographic format, but also raises the question of data relevance. When describing collocations, a lexicographer needs to select examples taking into account their representativeness in corpus, coverage in dictionaries, and also suitability for language users and their purposes.
We identified items from several Russian dictionaries of different types:
1. Explanatory dictionaries, i.e. the Dictionary of the Russian Language (DRL [
In our research, we tested the following hypothesis: high collocation frequencies in
the corpus correspond to high values of the dictionary index introduced in [
To assess the extracted data across text corpora, we randomly selected 20 collocations from groups with dictionary indices 2, 3 and 4 (see Table 1 for the examples) resulting in 60 collocations. During the statistical analysis we implemented nonparametric Friedman and Kruskal-Wallis tests for the comparison between corpora.
We also analyzed 20 collocations that are present only in one dictionary (i.e., their dictionary index was 1) resulting in 120 collocations.
As a material for our study we used the Russian National Corpus (RNC), i.e. a
disambiguated subcorpus of 6 mln tokens and the main corpus of 321 mln tokens. The
given subcorpora represent a “classical” (traditional) approach to corpus building
compared to an automatic one but are not so large, as opposed to other Russian
corpora. Therefore we also consulted ruTenTen corpus of more than 18 bln tokens that was
crawled automatically [
Collocation
Borisova 1995 1. adskaya bol’ ‘hellish pain’ 2. glubokaya drevnost’ 1 1 and 0 stand for presence or absence of the collocation in the dictionary.
LEDR
Dictionary index 0 0 2 3
an‘great tiquity’ 3. goryachaya lyubov’ ‘burning love’ 4. ostraya diskussiya ‘heated discussion’ 5. vysokoye masterstvo ‘superior skill’ 6. zverinaya zhestokost’ ‘monstrous cruelty’ 1 1 0 0 1 1 1 1 1 1 1 1 0 0 0 0 1 1 1 0 0 0 0 0 4 4 3 2 In our study we will also address to the following questions: 1) can we use corpora of a smaller volume for collocation analysis or in tasks dealing with their automatic processing? 2) do “traditional” and large web corpora produce the same results? 3) do dictionaries present homogeneous data, i.e. collocations of the same language nature demonstrate similar quantitative features? 4 4.1
Even for the high value of dictionary index, the results are not uniform (see examples in Table 2). 4 collocations are absent in the disambiguated subcorpus, although they are present in several dictionaries. Spearman’s rank correlation coefficient is about 0.81 for all three pairs of corpora, which indicates a strong positive relationship between them and their similar ranking of collocations. The average frequency in the subcorpus of the RNC was 1.34, the main corpus of the RNC and ruTenTen showed 1.93 and 2.01 respectively, but the differences between the data are statistically insignificant (p > 0.05 according to the Friedman test). Hence, the frequencies are homogeneous but two collocations with the lexeme bol’shoy ‘large’ (bol’shoye znacheniye ‘great meaning’ and bol’shoy uspekh ‘great success’) show outliers. 4.2
The frequencies of this group of collocations (see Table 3) show lower correlation (Spearman’s rank correlation coefficient varies between 0.62 and 0.79), while the differences between them in corpora are significant (p < 0.05 according to the Friedman test). For collocations selected from two dictionaries, we see that 12 units out of 20 (60%) were not found in the smallest corpus, and 3 of them were not recorded in the main RNC corpus either (see Table 4 for the results). 14. 15. 16. 17.
Thus, in the case of the above given examples that are present in three dictionaries, the frequencies tend to reveal more diversity. Spearman’s correlation coefficient increased up to 0.92 for ruTenTen and the main RNC corpus while it decreased to 0.53 for ruTenTen and the disambiguated RNC subcorpus. Hence we can register differences in ranking in the latter case. But the fluctuations in the frequencies between all three corpora are not significant (p > 0.05 according to the Friedman test). This result enables us to suggest that collocations found only in two dictionaries are rare. 4.4
Despite the fact that the online dictionary of idiomatic expressions [
The collocations extracted from the dictionary of lexical intensifiers [
The dictionary of set expressions [
Analysis of the data in the collocations dictionary [
The results for collocations from both explanatory dictionaries suggest that the
sources differ to a certain degree in how they represent unique phrases. For units from
the LEDR, the distribution of frequencies in three corpora is characterized by outliers
and a large range of values (for example, aktsionernoye obscestvo ‘joint stock
company’, organicheskoye veschestvo ‘organic matter’, pochtovyy yaschik ‘letterbox’).
Collocations from the DRL have smaller deviations from the mean values. Both
explanatory dictionaries show very little overlap with other lexicographic sources. This can
be explained by the fact that dictionaries are aimed at describing different vocabulary:
for example, the dictionary [
The analysis shows that in total there are no significant differences between corpora in frequencies of collocations with the same dictionary indexes (or from the same dictionary). Thus the analyzed items prove to be rare units. About 34% of the considered collocations are absent in the RNC disambiguated subcorpus, i.e. it can be assumed that the volume of 6 mln tokens is not enough to study collocability. About 12% of the analyzed collocations yield less than 0.01 occurrences per million even in the largest ruTenTen corpus.
It should be mentioned that collocation frequencies in corpora are steadily decreasing with the decrease of dictionary index (the differences are statistically significant, p < 0.05 according to the Kruskal-Wallis test), and the value of the Spearman’s rank correlation coefficient also decreases. Collocations represented in four dictionaries tend to be more widespread in corpora but also have low frequencies.
It is worth noting that with the rise of corpus volume the unique collocations (with
dictionary index equal to 1) tend to show more diversity in their frequencies. Only
the collocation dictionaries [
The ruTenTen corpus proved to have the largest number of pairs with significant
differences in frequencies (here we can name additionally, firstly, the dictionary of
idiomatic expressions [
With the exception of a few collocations (bol’shoye znacheniye ‘great importance’,
bol’shoy uspekh ‘great success’, vyssheye obrazovaniye ‘higher education’ and
nervnaya sistema ‘nervous system’ the examples turn out to be low-frequency in all three
corpora. The hypothesis is confirmed that with the decrease of the dictionary index,
the relative frequencies of collocations in the corpus decrease (with the exception of
unique collocations in the dictionary [
In our study we examined the Russian collocations which were extracted from six dictionaries. Their quantitative characteristics obtained on corpora of different volumes show that the analyzed examples turn out to be low-frequency and demonstrate their ambiguous nature. The overwhelming majority of dictionary collocations are unique, i.e. presented in only one dictionary; hence such items are difficult to be identified in corpora by using automatic methods.
The issue of data volume deserves much more attention, and the very phenomenon of collocability must be investigated in larger corpora as small volume does not show any occurrences for a number of collocations. Automatically crawled large corpora reveal more fascinating findings as well as peculiarities obscured in smaller text collections. Hence we can assume that machine learning algorithms that process word combinations should be based on large datasets counting several bln tokens.
The number of dictionary collocations depends on the dictionaries used. Unfortunately, despite the processing of several sources, the volume of the extracted data is still insufficient, therefore it is important to analyze other dictionaries and lexicographic sources and extract examples from them. Explanatory dictionaries may contain set expressions in other parts of dictionary entries (in the texts of quotations or illustrative examples), therefore, their further analysis is necessary, which will be performed at the next stage of our work. In future we plan to consider other resources and to study collocations based on other syntactic models.