Censorship is a phenomenon that affected availability and content of much published material in different countries in modern history. To better understand the composition of our corpora, it is important to detect and measure the effects of censorship in texts.

The current consensus in humanities converges on understanding censorship not just as a repressive apparatus of the government, but rather as a special kind of collective imagination about the (in)acceptable [3,10,21]. In formal modelling, notably in economics and political science, censorship is typically conceptualized as a governmental ban on certain media or topics [17,7]. While essentially ban on some topics accords well with the notion of inacceptable, it does not cover it in full. The content of published materials that passed all the censorship filters is still affected by censorship in the abovementioned theoretical sense. We would like to find a way to detect the effect of censorship in this situation, too. While considering the effects of censorship, we choose to evaluate how topics relate to one another, rather than the ways in which any particular topic is expressed.

Our central idea is that censorship affects the ability of the authors to navigate the topical space, so that juxtaposition of certain topics (e.g. literature and politics) are specifically avoided. The ecological validity of this idea is supported by the well-known historical cases when direct and sometimes unconscious proximity of certain articles, themes or ideas could and was interpreted by censors as an intentional connection of meanings, the establishment of a causal relationship between them (see a thorough review of different censorship cases [11] or one of the most well-known books on the history of Imperial censorship and the Russian press [16] for examples).

To measure the avoidance of topic juxtaposition we suggest the notion of topical dissociation, operationalized as a probability that either one or the other topic appear in the same article, but not both. We expect that the higher the censorship pressure, the higher the dissociation for some pair of politically sensitive topics would be, even when both topics are discussed in legitimate (censorship-wise) ways.

In this paper we present an empirical study that tests this idea on the corpus of the 19 th century Russian literary magazine Sovremennik (The Contemporary) (1847-1866). During this period censorship policies changed significantly, so that we can hypothesize about the timing and direction of the censorship effect on topical dissociation.

To identify the systemic effects of censorship, we turn to the materials of Sovremennik, one of the most influential Russian literary magazines of the period. This magazine, by the middle of the 19 th century, received exceptional recognition from the readers and absorbed most of the contemporary fiction works that later became canonical. As such, it was highly visible to the government, and fully experienced censorship practices of the period.

The history of Sovremennik spans a few institutional and ideological changes in respect to censorship in Russia [5,4,6]. We believe that these changes could be categorized into three distinct periods that we call 'censorship regimes'. They differ by the idea of what is acceptable or inacceptable, inherent in a particular social and political context, and influenced the logic of decision-making by different actors.

The Gloomy Seven Years: due to the revolutionary unrest that occurred in Europe in 1848, the Russian government feared a similar uprising and began to tighten its grip on civil liberties, higher education, and the press [18].

Liberalisation of domestic policies after the change in reign and the conclusion of the Crimean War, the preparation and implementation of Great Reforms.

The suspension of Sovremennik's activities after the Saint Petersburg fires. Preparation and implementation of the Censorship Reform of 1863-1865, which reorganized the system of censorship agencies and introduced punitive censorship [2].

It is also known that different periods in the history of Sovremennik vary in terms of editorial policy [5,4,6]. These periods do not entirely coincide with the boundaries of censorship regimes, and they can be linked to changes in the editorial team.

From January 1847, Sovremennik was under the editorship of Nikolay Nekrasov and Ivan Panaev. The end of the first editorial period corresponds with the cancellation of the so-called binding agreement with Ivan Turgenev and other leading writers of liberal views who started leaving the magazine in March 1858 [13].

Between April 1858 and May 1862, Sovremennik was led by the authors of social-democratic views with Nikolay Chernyshevsky and Nikolay Dobrolyubov in the forefront. The period ends with the suspension of publishing activities for Sovremennik.

Finally, from January 1863 to May 1866, Sovremennik resumed its activities, but Dobrolyubov by the time was already dead (November 1861), and Chernyshevsky, arrested in July 1862, was still detained.

Mikhail Saltykov-Shchedrin took a lead role, but the final closure of the magazine was not long in coming. It followed after the first attempt on Alexander II's life that caused ofÏcial panic and tough reactionary policy.

During the period of Sovremennik, literature was one of the mediums where the formation of public opinion that is independent of the government took place [1]. Our test case is based on the expectation that censorship pressure would cause authors and editors of Sovremennik to avoid language related to politically sensitive topics while discussing literary subjects. At the same time, the relative strength of censorship pressure differed within the three censorship regimes listed in the previous section. Summarising historical changes in a single dimension of censorship strength we would say it was at its highest in the first period (1847-1855), then hit the lowest point after the death of Nikolai I (1855-1862) to be partially reinstated in the third period though not attaining its initial strength. Hence, we hypothesize that the strength of dissociation of literary and political topics in Sovremennik would follow this trajectory of censorship pressure (see figure 1).

As a control condition, we include a test for dissociation of literary and non-political topics that should not be sensitive to changes in the censorship pressure. For this, we use topics of theatre and fine arts that were also prominent in the content of Sovremennik. We base our topical dissociation measurements on a LDA topic model of a comprehensive digital corpus of Sovremmenik's publications.

The aim of our analysis is the direct causal effect of the censorship regime on topic dissociation, i.e. self-censorship by the authors realized as avoidance of topical juxtaposition. Due attention should be paid to possible confounders. One alternative causal path would involve editorial policy that is both affected by a censorship regime and influences the topical composition of the corpus through selection of articles and direct editing. The use of LDA for operationalizing a topic's presence in a document produces a strong negative association between document length and topical dissociation (a longer document contains more LDA topics that surpass a certain minimal word number threshold, for our model the correlation is 𝜌 = 0.91). This creates another causal path since article length also depends on editorial policy. Finally, it is reasonable to assume the existence of unmeasured confounders that affect both document length and editorial policy, for instance, a fashion for longer literary texts or economic considerations of the publisher. These assumptions are summarized in a DAG in the figure 2. Given this DAG, to estimate the direct causal effect of censorship on topic dissociation one needs to stratify the data by editorial policy and article length.

In this paper we focus on the whole corpus of Sovremennik publications, including fiction, scientific and critical articles from the magazine's program sections, but also numerous essays and short stories by minor authors, news items, chronicles, reviews, and other materials. We used a digitized version of this corpus [19]. In total, 5412 materials were published in Sovremennik (excluding appendices to the magazine) between 1847 and 1866. The corpus includes texts for 4686 of them (87%). Since the quality of the originals varied and the OCR results were not corrected, the data quality is inferior for some documents. In total, 16,4% of tokens were identified as non-words (most likely OCR errors).

Topical dissociation for a pair of topics is defined for a corpus where either of the topics may occur in any given document. The strength of topical dissociation is expressed by the probability that either one or the other topic occur in a document, but not both. A direct measure for the share of such documents in a corpus is a well-known Jaccard distance

where 𝑇 1 and 𝑇 2 stand for the sets of documents which contain the first and the second topic, respectively. Importantly, by this definition the documents where neither of the topics occur do not attest for or against the topical dissociation, and should be regarded as irrelevant to the measurement. In our case, a single publication in Sovremennik (an article) serves as a document.

The limitation of the Jaccard distance is that it gives a single point estimate for the whole corpus. To properly account for the uncertainty of estimation and document-level confounders we need a way to define the probability of topical dissociation for a single document. This is achieved by recognizing that Jaccard distance essentially aggregates a set of binary outcomes for every document 𝑑 in a corpus

Every document-level outcome is interpreted here as a Bernoulli trial for which a conditional probability model can be defined. Then topical dissociation for a pair of topics 𝑡 1 and 𝑡 2 is given by Δ𝑇 (𝑡 1 , 𝑡 2 ) = 𝑝(𝑑𝑇 |𝜃 𝑑 ), for 𝑑 ∈ 𝑇 1 ∪ 𝑇 2

where 𝑝(𝑑𝑇 |𝜃 𝑑 ) is the probability that a document occurs in a topical dissociation set, and 𝜃 𝑑 is a vector of parameters that describe document-level predictors (article length, editorial policy, censorship regime). To estimate the topical dissociation probability we apply a Bayesian generalized linear model (logistic regression). All posterior distributions are estimated using STAN's Hamiltonian Monte Carlo algorithm. More details on the definition of the statistical model can be found in the appendix A.

For LDA modeling all text was converted into modern orthography and lemmatized with automatic disambiguation. All text was split into chunks of 500 tokens or less in case of shorter articles. Only nominal and verbal lemmas occurring in less than 50% of chunks were kept for modeling. For interpretation, we selected a model with 200 topics. The topic was considered to occur in an article when its share exeeded a threshold corresponding to a 1/3 of a journal page.

To measure dissociation, we selected three sets of topics from the LDA model. The first set (11 topics) represents the discussion of literary subjects, including journalistic polemics, writing manner, literary talent, poetry, book distribution, etc. The second set (16 topics) represent politically sensitive topics important for public opinion at the time: government (state), peasants, public life, nation, politics, justice. The third set (4 topics) serves as a control condition and represents discussion of theatre and fine arts. In these sets, we included only the topics relevant throughout the magazine's entire publication period, omitting those topics that were mostly localized in a short period of time. The summary of the topics is given in the appendix B.

Summaries of the posterior probability distributions of the topic dissociation for our focal (literature vs. politics) and control (literature vs. art/theatre) conditions are displayed in figure 3. Data points correspond to all possible combinations of censorship regime and editorial policy. As expected, the dissociation between literature and politics is highest in the first censorship regime. During the second censorship regime (1855-1862) Sovremennik has undergone changes in editorial team and policy, therefore we observe two separate states here. After the death of Nikolai I and still under the leadership of Nekrasov and Panaev the average expected dissociation dropped by 10 percentage points. This effect can be attributed to the diminishing censorship pressure. The editorial board led by Chernyshevsky and Dobrolyubov was inclined to include public issues in literary discourse more than the previous generation of authors as they believed that literature could change reality by presenting positive ideas and models for progress. The observed topical dissociation diminished accordingly (11 percentage points) even during the same censorship regime. Contrary to our expectations, more regulated censorship policy during the third regime did not result in growth in topical dissociation for literature and politics.

While we expected more or less constant dissociation rates for our control condition (literature vs. art/theatre), the data proved us wrong. Instead, we observe the trend that is opposite to the dissociation of literature and politics. It appears that the probability of dissociation steadily increases over time, across all censorship periods. Thus the topical pair of literature and arts displays an effect of censorship that is similar in strength though inverse in direction. If anything, it means that the dissociation of literary topics from the other arts was both an effect of diminishing censorship pressure and part of the Sovremennik's editorial policy after 1858.

The conceptual simplicity of the suggested measure for topic dissociation (which is essentially an enhanced version of Jaccard distance) is both a strength and a weakness. One important concern may be raised when we compare topical dissociation between two periods. Our measure ignores the difference in overall corpus size and keeps track only of the overlap of two topics relative to each other. In our view, the possible change in the coverage of other topics should not affect the dissociation measure for the two focal topics if their relative overlap remains the same. Moreover, since the size of our data corpus is not balanced across time periods, any dependence of topic dissociation on corpus size would be undesirable. Hence, we prefer our definition to measures of association that depend on corpus size, notably, to pointwise mutual information as widely applied in NLP for measuring co-occurrence strength.

Another debatable decision we took was to binarize topic presence in a document using 1/3 page threshold (see appendix B for details). An alternative approach would be to use LDA topic probabilities for each document to estimate topic association, as suggested, for instance, in definition of topic linkage in [15]. Topic linkage is closely related to mutual information, and depends on both document-level topic probabilities and corpus size. We consider LDA as a noisy measure of topics and we would like to avoid relying too heavily on its artifacts. That is why we choose to ignore the topics with low document-level probability estimates.

The definition of topical dissociation that we suggest allowed us to trace separate effects of the changing strength of censorship pressure and the editorial team's policy during the publishing period of the Sovremennik magazine.

The empirical results only partially supported our initial hypotheses. While the drop of the topical dissociation after the first period is in line with the expectation of the lowered censorship pressure, there was no rise of dissociation in the third period when the censorship policy has been updated again. One possible explanation is that the preparation and implementation of the 1863-1865 censorship reform primarily involved institutional change (reorganization of censorship departments), but did not imply stricter content censorship policy. Institutional changes appear to have had a minimal impact on the topical dissociation.

Moreover, the data contradicts our assumption that topical dissociation of literature and theatre/arts should not be sensitive to changes in the censorship regime. Apparently, depolitization of literature and placement of the literary discourse into the topical space of arts was characteristic of censorship and might be one of the effects of higher censorship pressure. During the last years of Nikolai I's reign, the censorship committee found even the most innocent things objectionable. According to one of the authors of this time period, the censors referred to the censorship committee as a scarecrow, who threatens punishment for every printed word [14].

In our research, we aimed not only to understand a specific case from the history of Russian press in the 19 th century, but also to identify general patterns of how censorship influenced journalism. Our results suggest that censorship pressure may affect the topical dissociation in some predictable way. But it also showed that censorship effect on the published content can be decoupled from the institutional changes in the censorship procedures.

On a more general level, we would like to further the theoretical reflection on what censorship is and how it can be operationalized. By defining censorship as a dissociative force, we take into account, on the one hand, the liberal understanding of censorship as an instrument of the state to limit the freedom of individuals. On the other hand, we consider the more recent research that have challenged the traditional binary opposition between power and free speech, or truth, that is assumed by the model of censorship as a governmental limitation of freedom. In the latter view, power is not seen as a form of limiting the truth, but rather as a monopoly over it, and censorship serves as an expression of that truth [9,8]. Using our definition of the censorship through topical dissociation, we were able to see the consequences of these rules imposed by authorities, even in printed publications that have gone through the censorship process. As the method itself is general enough, it has a potential for wider application to study the manifestation of censorship effects in the content of published materials. are considered irrelevant for the measurement of the topic dissociation and are excluded from the data. 𝑝 𝑖 stands for the expected probability of dissociation for in article 𝑖, and 𝐿 𝑖 is the standardized log-length of an article 𝑖 in pages. Effects of censorship regime and editorial policy are captured by 𝛼 𝑐,𝑒 . A separate coefÏcient is computed for each attested combination of the censorship regime and editorial policy (4 in total).

The LDA topic inference was performed using R wrapper for MALLET package [12]. We were choosing between topic models with 100, 200, and 300 topics. All these models had the same beta hyperparameter value 𝛽 = 0.01 while alpha parameters differed: 0.05, 0.025, and 0.0167, respectively. The model with 𝑘 = 200 was selected for analysis due to comparatively greater interpretability of the topics (45 topics were hard to interpret).

For our analysis, we selected only the topics relevant to literature, arts, and politics, and grouped them into three corresponding categories. The category was considered to be present in an article if any topic from that category in an article exceeded word limit corresponding to a 1/3 of a journal page. The top 10 words for the topics in each category are listed below. literature 18 th and 19 th century writers, literary reputation, periodicals, literary criticism, genres, poetry and prose, versification, literary language art fine arts, theatre, music 13. sovereign (gosudar'), great (velikij), decree (ukaz), majesty (velichestvo), emperor (imperator), empress (imperatrica), note (zapiska), Gavriil Derzhavin (derzhavin), imperial (imperatorskij), Russia (rossiya) 14. people (narod), state (gosudarstvo), law (pravo), power (vlast'), freedom (svoboda), law (zakon), public (obshchestvennyj), government (pravitel'stvo), society (obshchestvo), state (gosudarstvennyj) 15. community (obshchestvo), member (chlen), meeting (sobranie), committee (komitet), shared (obshchij), meeting (zasedanie), board (sovet), shareholder (akcioner), board (pravlenie), chairman (predsedatel') 16. Russian (russkij), literature (literatura), Russia (rossiya), Moscow (moskovskij), question (vopros), statement (vedomost'), cause (delo), society (obshchestvo), reader (chitatel'), public (obshchestvennyj)

Following our operationalization of a presence of a topic in an article (≥ 1/3 page) we can define what proportion of articles in each period contains discussion of our major topical categories: literature, arts, and politics. The proportions are shown in the figure 4. The data displays a clear upward trend for political topics while the share of the Sovremennik space devoted to literary topics remained remarkably stable throughout its history. Arts, on the contrary, were on the steady decline.