The paper is aimed at studying the texts of Russian Constitutional Court decisions, issued from 1992 to 2018. We analyzed the corpus, consisting of 584 decisions or 3,426,747 tokens (incl. punctuation marks) and tested the hypothesis about increasing lexical complexity of the documents. Using the R package stylo and MFW statistics, we got a picture that reflects the differences of the texts by years. The results of cluster analysis show that the texts of the 90s and 2000s are combined into the first large cluster. The second large cluster includes the texts of the 2010s. Using the R package quanteda, we obtained the values of 11 lexical diversity measures. We chose the index K (Yule's K) as a basic measure, relatively more reliable and independent of the text length, and then interpreted the values of this measure. In general, the value of K decreases over the years, except for the texts of 2006, in which there is a noticeable increase in the index value, and the texts of 1993, in which the outlier is observed. The calculation hapax proportion shows a picture of a gradual decrease in the share of hapaxes. If we apply the traditional approach to the interpretation of TTR values and derived metrics, we can conclude that, as the lexical diversity decreases and the proportion of hapaxes decreases, the texts become easier to read.
This paper is aimed at studying the texts of Russian Constitutional Court decisions, issued from 1992 to 2018. The purpose is to verify the hypothesis, according to which the texts became more complex during the specified period (i.e. in shallow diachrony). At the moment, we were primarily interested in the lexical complexity.
The Constitutional Court is one of the youngest legal institutions in Russia. Its appearance in 1991 was associated with large-scale changes in the legal and political
Copyright ©2020 for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). system caused by the rejection of Soviet legal and political system and the creation of a new democratic state in Russia.
The Constitutional Court, more than any other courts, was perceived as an “alien body” in the judicial system, since the Court was significantly different from all other judicial bodies in its objectives and duties. The task of the Constitutional Court was neither to solve a specific case, nor to draw conclusions about the rights and obligations of a particular citizen, but to compare the norms of a law challenged by a citizen and the provisions of the Constitution, ensuring the protection and implementation of constitutional principles.
The major function of the Constitutional Court is direct application and appropriate interpretation of the constitutional text. Citizens and legal entities’ complaints about the violation of their constitutional rights noticeably prevail among the cases considered by the Russian Constitutional Court. For example, in 2019, out of more than 3,500 judgments and rulings of the Constitutional Court, only three were made at the request of state bodies, about 30 were made at the request of the courts, and all the rest were made on complaints.
At the same time, it cannot be concluded that decisions of the Constitutional Court, initiated by citizens, are addressed specifically to these citizens. Of course, an applicant should understand whether the Constitutional Court supported her arguments as well as the outcome of the case consideration. However, only a small (operative) part of а Constitutional Court decision is devoted to this.
The rest is primarily addresses those bodies that must restore the violated rights, and not only and not so much the citizen who applied directly to the Constitutional Court, as those who find themselves in a similar situation. In this regard, the main addressees of Constitutional Court decisions are the legislative bodies, which should amend corresponding laws, and law-enforcement bodies (both executive and judiciary), which should interpret the legal provisions that they apply in accordance with decisions of the Constitutional Court.
However, it would be wrong to exclude citizens and legal entities from the addressees of the decisions. The Constitutional Court very rarely finds itself concluding the absolute, complete and unconditional contradiction of examined norms of the Constitution. More often, conclusions about unconstitutionality are made in relation to a particular interpretation of the impugned norm. Acts of the Constitutional Court become part of existing law, shall be applied along with statutes, the interpretation of which they strongly influence. Coming to court and demanding application of a provision, for example, establishing a social payment, citizens often must refer not only to the provisions of the law, but also to their constitutional interpretation in the practice of the Constitutional Court. That is why the decisions of the Constitutional Court should be clear to all citizens.
On the one hand, the field of activity for the Constitutional Court is an area of refined jurisprudence, free from description of factual circumstances, proving their existence and assessment of such evidence; on the other hand, it is a part of the existing legal regulation, along with statutes. The decisions of the Constitutional Court have the most significant difference with the latter, as these decisions are a result of the work of judges (i. e. professional lawyers). Many of the Constitutional Court judges are professors of law, others were appointed to the Constitutional Court after many years career in other courts.
A draft of each decision is prepared by a judge-rapporteur, while the final text of any decision becomes a result of the collective creativity of all judges and has no authorship.
That is why the assessment of the decisions’ complexity is important and indicative. Such an assessment demonstrates the ability of professional lawyers to be clear, to write specialized texts addressing to a wide range of citizens, in an accessible manner. 1
To test the hypothesis of texts becoming more complex with time, we used the
capabilities of two R software packages (stylo and quanteda) [
Using the stylo package, we got a general picture that reflects the differences between the texts by year. Using the quanteda package, we received more detailed information about the lexical complexity of texts by different time periods.
The diachronic study of legal documents is the actively developing area. In
particular, there are diachronic corpora of legal texts, for example, Corpus of Historical
English Law Reports (CHELAR) [
A research on the readability of texts of Constitutional Court is presented in [
The Flesch-Kincaid formula for the Russian looks as follows:
FRE = 206,836 − 60,1 × ASL − 1,3 × ASW, (1) where ASL is the average sentence length in words, and ASW is the average word length in syllables.
However, two points should be emphasized. Firstly, the coefficients of the
Oborneva’s formula were obtained by calculating the statistical characteristics of about 100
works of famous English-language literary classics (and translating these works into
Russian). Thus, the formula is not quite universal, but is applicable primarily for the
analysis of the complexity of texts of (translated) fiction; about the indicated problem,
see [
In this paper to assess the text complexity (lexical complexity) we also use the traditional method of assessing, calculating the TTR (type-token ratio), more precisely, we use a number of derived metrics.
TTR is the ratio of the number of unique tokens (types) to all document tokens. It
is known, however, that the values of the TTR measure are not independent of text
length, that is, documents of equal length should be compared to obtain relevant
results. The solution to this problem can also be the use of derivative measures, see
[
In addition, to assess the lexical complexity, we use information on hapax richness and hapax proportion (the hapax is a token that appears in a sample once). Hapax richness is a measure that describes text from the same perspective as the TTR measure. 2
There is a rather long tradition of applying methods for assessing complexity
(readability) to texts in Russian; for a review, see, for example, [
To assess the lexical complexity, we can use information on:
• lexical density, the proportion of various content words in the texts;
• lexical richness and lexical diversity, measured by calculating the values of TTR or
derived measures;
• number of words with abstract or concrete meaning;
• number of ambiguous words;
• number of function words (particularly prepositions);
• number of abbreviations
etc., see [
We analyzed a collection of judgments, consisting of 584 documents relating to the period since 1992, when the first judgment of the Court appeared. The distribution of decisions by year is described in Table 1.
The full texts of the decisions were taken from the database of the ConsultantPlus
information system [
There are no 1994 decisions in the text collection. This is due to the fact that the Constitutional Court suspended work at the end of 1993. The reason was the need to adopt a new law, regulating the Constitutional Court activities. As a result, in 1994 the Federal Constitutional Law “On the Constitutional Court of the Russian Federation” was adopted. The appointment of judges also took some time. In an updated form, the Constitutional Court resumed its work in 1995. We used the corpus, which contains texts combined by years. Accordingly, the corpus files received names like “1992”, “2003”, “2018”.
The text collection consists of 3,426,747 tokens (including punctuation), see Table 2.
The stylo package was created for quantitative studies of writing style and can be used
in authorship verification (including forensic linguistics) and diachronic studies,
see [
We performed unsupervised multivariate analysis. Using the basic functions does
not imply preprocessing and markup of the text collection (segmentation into
sentences, lemmatization, etc.). We downloaded text data directly from the corpus files. Text
metadata were included in the file names. Using the basic stylo() function, it is
possible to analyze a corpus with the assistance of the following methods.
• Cluster analysis (CA), the results of which are visualized as a dendrogram, or a
graph, showing the clustering of texts.
• Multidimensional scaling (MDS), as a result of which texts are displayed as
ordered on the basis of several variables, so that similar texts are placed next to each
other, and heterogeneous texts are separated, see [
So, by means of the package one can find out how much the analyzed texts or text collections differ. As features for the analysis, n-gram sequences of tokens and characters can be used. 4.2
At the stage of preprocessing, we performed tokenization and removal of stop words.
When tokenizing, we used the built-in features of the package. To remove stop words,
we took a stop word list from [
The corpus size after the removal of stop words was 2,103,608 tokens. We formed
a list of 1000 frequent features, then found the features that are used in at least 90% of
the texts. In this way, we got a list of 1684 MWF, and then in the analysis we used
100 or from 100 to 1600 of them.
1 We used a list of stop words to remove units that are not able to characterize the lexical
peculiarity of a text or text collection. The list [
Then we performed cluster analysis, multidimensional scaling, principal
component analysis. As a measure of distance, where relevant, we used the Eder’s Delta
measure, which is recommended for highly inflected languages [
The results of the PCA, MDS, and CA (see Fig. 1 below) show that the texts of the 90s, 2000s, and 2010s can be described as two separate groups. One can see the following pattern: the texts of the 90s and 2000s are combined into the first large cluster. The second large cluster combines primarily the texts of the 2010s.
Thus, MFW statistics shows, that in general the texts before 2010 and after 2010 are clearly opposed, but the texts of 2005 and 2007 are adjacent to the group of texts of the 2010s. In addition, the texts of 1992 and 1993 are opposed to the rest of the texts written before 2010. On the whole, the results of calculating the Euclidean distance on normalized token frequency demonstrate a similar, but non identical patterns, see Fig. 2 (the dendrogram displaying normalized token frequency was obtained using quanteda package). The texts of 1992 and 1993 are contrasted with the rest of the texts in the corpus; in addition, the texts of 2012 fell into a large cluster containing the remaining texts of the 1990s and texts of the 2000s.
Finally, BCT allowed us to obtain the combined results of a cluster analysis (see Fig. 3). 5 5.1
The quanteda package provides tools for a range of natural language processing tasks,
see [
When forming the corpus, we performed the removal of stop words, numbers and
punctuation marks (since by default numbers were considered as tokens). The
package uses the “Snowball” list of stop words [
Using the package, we obtained the values of 11 measures of lexical diversity listed
above.
It is well known that the value of a simple TTR is affected by the text (or the sample)
length, see for example [
1. It is possible to use samples of the same length.
2. It is possible to apply formulas with logarithms or with other transformations
of variables N and V (Herdan’s C, Guiraud’s Root TTR, Carroll’s Corrected
TTR, Dugast’s Uber Index, Maas’ indices).
3. It is possible to apply measures that make use of elements of the frequency
spectrum (for example, the “Yule’s K” measure), e.g. measures that take into account
the number of hapax legomena (as in the “Honoré’s R” measure) or hapax
dislegomena, see [
The values of the indices TTR, C, S, K, Vm, Maas demonstrate a general decrease in time. The values of the indices R, CTTR, logV0 demonstrate a general increase in time. In general, the interpretation of the data is quite tricky, since the values of different measures are somewhat contradictory.
Therefore, based on the findings of [
The value of the index K varies in the range from 23.40 to 83.69 (and the value of 83.69 observed in 1993 should be considered an outlier, see Fig. 4 and 5). In general, we can say that the value of K decreases over the years (except 2006 and 1993). Accordingly, it can be argued that the lexical diversity of the texts of in time is decreasing.
The calculation of hapax richness (the number of tokens that appear in the sample only once) and the proportion of hapaxes show that the share of hapaxes gradually decreases, see Table 4, Fig. 6. However, the texts of 2006, 2008 and, to a lesser extent, 2007, 2005 and 2016 do not correspond to this general scheme. Thus, the share of hapaxes decreases over the years, the lexical diversity decreases over the years (texts have more and more repeating words). These two text evaluation options are easy to interpret in a consistent manner.
As a result of analyzing the corpus of Constitutional Court decisions we found out the following. • MFW statistics shows that, in general, the texts before 2010 and after 2010 inclusive are clearly opposed. • The texts of 1992 and 1993 are contrasted with all other texts (see, in particular, the clustering results after calculating the Euclidean distance on normalized token frequency). This can be explained by the fact that in 1994 the composition of the Constitutional Court was updated. • The hapax proportion decreases over the years, the lexical diversity of texts also decreases.
If we apply the traditional approach to the interpretation of TTR values and derivative metrics, we can make a general conclusion that, since the lexical diversity is reduced and the proportion of hapaxes decreases, the texts become easier to read. Thus, our hypothesis of an increase in lexical complexity has not been confirmed.
There is an opposite approach to the interpretation of TTR values, we quote: “a lot
of formal repetitions of the same words denoting legal entities and various legal terms
interfere with the perception of the meaning of the sentence. In this case, we can say
that reducing diversity not only does not simplify the text, but also causes the opposite
effect” [
Legal texts use many repetitions. Though the presence of repetitions tires, it also allows to avoid problems with the interpretation of coreferential expressions. In addition, the process of text perception is affected by the priming effect (in particular, lexical priming).
Apparently, for the successful application of vocabulary-based measures for text complexity assessment, it is necessary to take into account at least some words’ characteristics, that is, their semantics (first of all, abstractness/concreteness), their belonging to a certain part-of-speech class and general-language frequency. Acknowledgement. The research was supported by the Russian Science Foundation, project #19-18-00525 “Understanding official Russian: the legal and linguistic issues”.