Unseen species models from ecology have recently been applied to censored historical cultural datasets to estimate unobserved populations. We extend this approach to historical criminology, analyzing the police registers of Brussels' Amigo prison (1879-1880) using the Generalized Chao estimator. Our study aims to quantify the 'dark number' of unarrested perpetrators and model demographic biases in policing eforts. We investigate how factors such as age, gender, and origin influence arrest vulnerability. While all examined covariates contribute positively to our model, their small efect sizes limit the model's predictive performance. Our findings largely align with prior historical scholarship but suggest that demographic factors alone may insufÏciently explain arrest patterns. The Generalized Chao estimator modestly improves population size estimates compared to simpler models. However, our results indicate that more refined models or additional data may be necessary for robust estimates in historical criminological studies. This work contributes to the growing field of computational methods in humanities research and ofers insights into the challenges of quantifying hidden populations in historical datasets.
statistical modeling approaches. While empirical validation of these estimates remains challenging, they provide valuable insights that can inform policy decisions and guide law enforcement strategies. In recent years, criminology has increasingly turned to advanced quantitative methods to address the negative bias in available data. Although these models have produced valuable insights into contemporary crime patterns, their application to historical data remains limited.
Our study aims to bridge this gap by applying one such modern technique, the Generalized
Chao method [
Furthermore, we leverage the demographic information available in the Amigo register to construct a model that predicts crime detection rates across diferent social groups. This approach allows us to explore historical vulnerability to arrest, potentially revealing diferential treatment by the police and broader societal biases of the era. By applying contemporary statistical methods to historical data, we aim to provide new insights into the dynamics of crime and law enforcement in 19th-century urban environmen1ts.
Our research aligns with a growing scholarly interest in measuring the amount of
underdetection in historical data, known aÜs berlieferungschance [
Recent scholarship has turned to unseen species models from ecology as an innovative
approach to estimating the extent of lost or forgotten data, particularly in cultural heritage
studies [
The primary source for our study is the Amigo register, a serial, handwritten source housed in the City Archives Brussels, within the records of the Brussels municipal po3liTceh.is unique historical resource, illustrated in F1i,gt.akes a tabular form, with each row recording information about an individual who spent the night in the Amigo priso4nU.nfortunately, like many archival records of urban police forces, the Amigo register has sufered from poor conservation. For Brussels, only a few registers from the 1880s have survived. Our study focuses on the register covering the period from October 31, 1879, to November 1, 1880, encompassing all individuals detained in the Brussels police prison during this time.
A critical distinction in the Amigo register data is between two types of entries: 1. Involuntary detentions: Individuals forcefully arrested and detained by the police; 2. Voluntary stays n( uit sur demande or ‘night on request’): Individuals who voluntarily sought shelter at the Amigo.
The latter category primarily comprised socially vulnerable people lacking financial means or family support. These individuals requested to spend the night at the Amigo for shelter, rather than being brought in through active law enforcement eforts. The voluntary nature of tnhueit sure demande entries contrasts sharply with the involuntary detentions, potentially impacting our interpretation of arrest patterns and policing eforts in late 19th-century Brussels.
Our analysis primarily focuses on involuntary detentions, specifically individuals arrested and locked up in the Amigo police prison on charges of ’vagrancy’. At the end of the 19th century, the Belgian economy sufered a depression, which heightened concerns about the influx of foreign ‘vagrants’ and other perceived undesirable newcomers. in response, urban municipal authorities implemented stricter local ordinances targeting poor migrants. These included entry regulations and passport requirements at the city gates, as well as mandatory reporting by landlords and innkeepers about their lodgers’ characteristics and previous settlements. Concurrently, new anti-vagrancy legislation criminalized several conditions: (a) the inability to prove stable attachment to a local community, (b) failure to register as required by law, and (c) lack of a steady income. From 1866 onwards, a police arrest and judicial conviction for vagrancy or begging resulted in forced internment in State Vagrancy ColonRieijsks(weldadigheidskolonies). Before being transported to the vagrancy colonies, arrested vagrants were held in the police prison of the city where the arrest was made: these were the individuals who were registered as ‘vagrants’ or ‘beggars’ in the Amigo register, and who constitute our research population9[]. 3City Archives Brussels (SAB), Records of the Brussels Municipal Police, Amigo, register “Vagabonds et Mendiants” and register “Ivrognes, logés, prostituées”, 1880 (no series or piece number). 4The historical building of the Amigo prison in Brussels still exists today and currently houses a luxury hotel, centrally located, adjacent to the Grand-Place of Brussels. The name is a corruption of the older Dutch word Vrunte, which meant ‘place of detention’, misinterpreted during the Spanish occupation as ‘friend’ (Duvtcrihend), hence the Spanish ‘amigo’.
Recidivism is a prominent feature in our dataset, with many individuals experiencing multiple arrests. Table1 illustrates the distribution of arrest counts. In total, the data holds evidence for 8,367 forceful arrests of 6,016 unique individuals. The mean number of nights an individual was locked up in the Amigo is therefor∼e 1.39. However, this average masks a highly uneven distribution of arrests across perpetrators. While most individuals were registered only once, repeat ofenders were common, with one extreme case involving 35 arrests for public intoxication within a single year.
Fig. 2 visualizes the temporal patterns of arrests for several individuals, revealing an interesting phenomenon of ‘burstiness’ in the data. Arrest events involving the same individual often form local temporal clusters. To investigate this pattern, we applied a simple Poisson model to all individuals arrested at least twice. This model predicts the time lag (in days) between an individual’s arrests based on their number of previous arrests. As illustrated in3,Ftihge. time lag generally decreases as the number of previous arrests increases. One way to interpret this is that the police gradually lost their patience with recidivists. Additionally, this suggests that the policing efort in arrests was not entirely neutral or random, but was guided by certain biases. The Amigo register holds information on the date of entry, but not on the release date: historians assume that the typical imprisonment was typically restricted to a short time period (i.e. a single night). The re-arrest data shows that individuals could indeed be re-registered after a short interval, which lends credibility to the assumption that incarceration was generally short-lived.
There exist valid reasons to assume that these data do not cover all of the individuals which theoretically could have been arrested by the police. One of these is the elevated number of singletons and doubletons in this distribution: if so many individuals were only recorded once or twice in the data, this renders it statistically quite likely that many perpetrators were never captured at all and don’t appear in the register. The arrest data must therefore be considered incomplete and, in all likelihood, severely underestimate the true number of perpetrators in Brussels for the time period considered.
The methodology described below enables us to estimate a lower bound on these “dark numbers” in historical Brussels, i.teh.e number of perpetrators who were criminally active in this time period, but who were never formally registered by the police. While this estimand presents a valuable scholarly objective in itself, this paper is also interested in characterizing this unobserved share of the criminal population in terms of demographics. Were men more likely to be arrested than women? Was the police efort more heavily geared towards younger individuals? Is there evidence of an active bias towards foreigners in comparison to local citizens? And, are there any intersections in these biases? Answering these questions would shed more light on the subjective drivers behind arrests and, thus, the historical vulnerability to arrest across demographic groups. On a more abstract level, explaining the drivers of under-detection in such historical datasets represents a major advance with respect to previous work in this area, which could only estimate the mere size of the non-observed share of the population, but not explain or characterize its composition.
Based on previous historical studies of the Amigo registe9r],[we formulated several hypotheses to guide our experimental design: H.1 Impact of voluntary stays: We hypothesize that a previous confinement on request (nuit sur demande) might have increased the visibility of socially fragile individuals to the local police, thus increasing their vulnerability to future arrest. For example, Joseph Janssens, a 25-year-old construction worker, was granted night shelter on March 20, 1880, but was arrested for public drunkenness just days later on March 28.
H.2 Age-related arrest patterns: We expect that vulnerability to arrest generally increased with an individual’s age. Prior work suggests that the police were more lenient towards younger individuals, particularly children, although minors were surprisingly often incarcerated. Among recidivists, there is a high presence of elderly citizens, who were more socially vulnerable due to a lack of suitable care institutions.
H.3 Gender disparities: While there appear to be fewer criminally active women in the city, many women are among the recidivists. This suggests that the police efort may have been negatively biased towards socially vulnerable women.
H.4 Migratory status: Contrary to older assumptions about discriminatory police biases against migrants, recent work found no clear evidence for this in the register’s data. In fact, the data suggest that local, sedentary citizens were much more vulnerable to arrest than perpetrators born outside of Brussels.
H.5 Family-based biases: We hypothesize that there existed negative biases against members of perceived ”criminal families.” The data shows frequent reappearance of the same family names across diferent individuals, sometimes with family members co-arrested. For instance, in autumn 1880, we find frequent arrests of 10 children (aged 8-10) from the same migrant family from Naples, Italy, who were active as beggars in the city. After preprocessing and manual disambiguation of individuals appearing in the dataset, the following data was available for each individual: .1 .2 .3 .4 .5 prior: Binary indicator (’prior’ (reference level) and ’no prior’) whether the individual was granted at least one ”night on request”n(uit sur demande) prior to the date of their first arrest; age: An individual’s mean age at the time of arrest, measured in years (scalar variable, centered and standardised). sex: An individual’s biological sex (binary: ‘female’ or ‘male’ as reference level). We deliberately use the term ‘sex’ as the data contains no information on an individual’s gender role. origin: Manually coded factor based on the individual’s place of birth (‘ABROAD’ for individuals born outside of Belgium, ‘BE’ for those born in Belgium, and ‘BXL’ (reference level) for locally born individuals from Brussels). family: Binary indicator (‘no family’ (reference level) and ‘family’) whether an individual with the same family name occurs in the dataset. This variable aims to capture bias against known ”criminal families” but should be interpreted cautiously, because we cannot rule out the possibility that unrelated individuals shared the same last name.
In this paper, we apply a so-called unseen species model to the Amigo data, borrowed from the
biostatistical literature in ecology, to help solve the problem of estimating the number of
unobserved perpetrators. Capture-recapture surveys are important bioregistration instruments in
the field of ecology, used to monitor aspects of biodiversity, such as species richness (the
number of unique species living in a certain area1)2[]. During such campaigns, field workers use
a variety of trapping devices (e.g. cameras) to register animals, mark them and release them
again, so that they can be re-sighted at a later time. This process results in what is known
as “abundance data”: counts that record how often animal types have been observed, such as
singletons ( 1 or the number of species sighted exactly once), doubleton s2 (or the number
of species sighted exactly twice), etc. Because of the imperfect observation process, however,
many animal types will not be observed during such campaigns, leading to an underestimation
of the true ecological diversity (“unseen species”). The resulting count data must therefore be
treated as censored, because it is zero-truncated: the number of relevant species which exist
in the area but which were never observed 0() are missing. Statistical methods are therefore
used to estimate 0 as 0̂ and correct for the observation bias, by addi ng0̂ to (the number of
is a widely used estimator that estimates a lower bound o0nas follows : 0̂ ≥ 12/2 2.
observed species) to obtain an estimate of the true population sizê. Chao1 [
For theoretical reasons, it is important to stress that Chao1 only estimates a lower bound on
the true 0, i.e. it estimates the minimum number of unobserved crimina ls0;, in reality, could
in fact could have been larger than0̂ . (Readers should take time to convince themselves of the
fact that, conversely, the detection ratio/( 0̂ + ) is an upper bound for that reason.) The fact
that Chao1 only ofers a lower bound is related to the fact that it does not take into account any
heterogeneity in the data [
In this section, we discuss the Generalized Chao: a generalization of the Chao1 method that can take into account statistical covariates and thus model heterogeneity across individual perpetrators in a dataset. This estimator, originally proposed by Böhning et al., aims to model heterogeneity in the data by modeling the detection probabilities as a regression problem. The method is characterized by a similar focus on low-frequency spec ie1sa(nd 2) as Chao1, capturing the intuition that such uncommon species carry the most information about species which were not observed at all. These data consisting o1f and 2 counts can be said to arise from a Poisson distributio n, ∼ Poisson() , but a truncated one with ∈ 1, 2.
Böhning and colleagues show that by maximizing the binomial likelihood, we can obtain an estimate for using ̂ = 2 ̂
(1− ̂ ) [
∼ Binomial(1, ) logit( ) = + + … =̂ + ∑1+ 2 1 =1 ̂ + ̂2/2
We employ Bayesian logistic regression models as implemented in the Python package
Bambi[
We fit a series of models for diverse (additive) combinations of the available covariates. No
interactions or random efects are considered. We include linear and quadratic terms for the
age predictor. We compare all models against an intercept-only model which, as it ignores
any heterogeneity in the data, should produce estimates equal to the Chao1 model, and can
thus be considered a baseline in these experiments. We compare the predictive performance
of the models using LOO-CV 2[
In Fig. 6, we show a forest plot for the coefÏcients included in the highest ranked model, which is also the most complex in terms of parameters. The predictor coefÏcients’ HDI values do not intersect with zero suggesting that they each contribute meaningfully to the model. Given the strong class imbalance, however (cf. the intercept estimate), their efect sizes are however small, which is both reflected in the non-distinctness of the model comparison above and the marginal efect plots in Fig. 7. The coefÏcients show how this particular model assesses the likelihood of an individual being a doubleton (as opposed to a singleton) (, i.e2.). First of all, the arrest rates vary with gender: we see that female individuals were more likely to be a doubleton than men. There appears to be a non-linear relationship with “age”, which peaks around the age of 50. Doubletons are also slightly common in the local population from Brussels, who sufered higher arrest rates than people born elsewhere in Belgium – the vulnerability to arrest of native Belgians appears comparable to individuals born outside of the country. People who previously stayed the night in the Amigo at their own request also saw a higher doubleton incidence than others. Finally, this model finds weak evidence for a bias against individuals with a known last name, potentially indicating more police efort against members from families perceived as “troublemakers”.
Given the lack of decisive arguments to conclusively prefer one model over the rest, we opt
for the common practice of (weighted) model averaging. This approach allows us to account
for model uncertainty by combining the predictions of multiple models, rather than relying
solely on a single ‘best’ model. Specifically, we employ the stacking of predictive distributions
method [
The main results are presented in in Tabl2e, which is primarily insightful for the categorical predictors. Here, the adjusted population size estimates are shown with 89% upper and lower credible intervals, both for the highest ranked model and the ensemble model, together with the observed counts and the original Chao1 estimate for each predictor level. Detection rates are presented for each predictor. Asterisks in the table indicate that the lower end of the credible interval for a level does not include the corresponding Chao1 estimate: this is of particular interest, because these are instances where the novel estimators deviate meaningfully from the baseline estimator, identifying demographic categories where the detection rate is potentially underestimated by Chao1. Crucially, we see that the Generalized Chao estimates consistently yield an upwards adjustment for the total population in comparison to Chao1: as the estimated Tabular overview of the Generalized Chao estimates (best performing model ( ̂, and model averaging MA) across the diferent demographic levels in the data, including per-level observed counts and the original Chao1. Asterisks in the table indicate that the lower end of the credible intervals for a level does not include the corresponding Chao1 estimate.
Pop. total population total went up from 16,937 to 18,414, our estimated detection rate dwindled down from 33%, to 31%. While the adjustments can be limited in some instances, this suggests that the Generalized Chao has successfully managed to capture at least some of the heterogeneity across the demographic group in the data. As expected, the model averaging yields slightly lower estimates than the single-model approach and presents a more cautious approach that might be preferable in the case of ranking ties between models, like here.
When we compare the detection rates across the levels of specific factors, interesting trends can be discerned. These insights are further supported by the marginal efects plots in Fi7g.. The estimated number of female perpetrators is much lower than for male perpetrators, but we see that women’s vulnerability to arrest was considerably higher. In the case of migratory status, we see that local citizens form the largest group, but this time we see that the arrest efforts clearly targeted this group more intensely than people born outside of the city or country. Again, no bias against migrants is manifest in the data. A factor that considerably increased detect rates was the stay of a prior night in the Amigo. Somewhat weaker is the efect of belonging to a family of perceived troublemakers: again, a smaller demographic group, but with much more policing efort geared against them. Often, we see that the efects are strongest for the smallest, most underrepresented subgroups in the criminalized population, such as women, which are heavily outnumbered by men. This might help explain why even the best performing model in the evaluation in the end does not clearly outperform the baseline model.
Our application of the Generalized Chao method to the Amigo prison register data has yielded valuable insights into the ‘dark number’ of criminality in late 19th-century Brussels. By incorporating demographic covariates into our model, we’ve been able to estimate not only the total number of unobserved perpetrators but also to characterize patterns of arrest vulnerability across diferent social groups. Our finding suggests that the true criminal population was likely much larger than can be observed in the register data, and that the Generalized Chao, albeit slightly, adjust the population estimate relative to the Chao1 model.
However, it’s crucial to acknowledge the limitations of our approach. Our statistical models primarily focus on relatively stable individual characteristics and do not incorporate eventspecific information such as the date, location, or reason for arrest. The inclusion of age as a predictor, while informative, is only justified by the narrow one-year time frame of our dataset. In ecological terms, our model considers species-level covariates but lacks observation-level predictors. This constraint represents a significant limitation of the method and highlights an area for future methodological development.
Despite these limitations, our rigorous model comparison revealed valuable insights. Distinguishing between singletons and doubletons proved to be a complex task, with considerable model uncertainty. While our models incorporating demographic covariates didn’t markedly outperform the baseline model in terms of out-of-sample predictive ability, this finding itself is valuable. It underscores the complexity of historical criminality and arrest patterns, suggesting that simple demographic factors alone may not fully explain arrest vulnerability.
While our model comparison revealed limitations in predictive power, it’s crucial to consider additional factors that may influence our results. One such factor is the potential violation of core assumptions underlying the Chao1 estimator and its generalizations, particularly the assumption of a closed population during the observation period. In many applications, including ecology, this assumption is often violated due to births and deaths within the population. However, in our historical urban context, migration presents a more significant concern. Late 19th-century Brussels experienced extreme population turnover, with historians estimating annual rates as high as 10% 9[]. This dynamic population flux challenges the closed population assumption inherent in our models. Furthermore, the specific nature of our data introduces additional complexities. Some arrested vagrants were temporarily sent to vagrancy colonies outside Brussels. While many of these individuals eventually returned to the city, they were temporarily unobservable9][. This periodic absence of individuals from the observable population further complicates our modeling eforts.
These factors likely contributed to one-inflation in our data, an excess of singletons (individuals observed only once) compared to doubletons (individuals observed twice). This phenomenon requires careful consideration, as standard models, including non-parametric lower bound estimators, tend to overestimate the true population size in such case3s].[ Several mechanisms can contribute to one-inflation in a population: 1. Behavioral change: Identified individuals may alter their behavior after initial detection. In our context, the experience of incarceration might have had a strong dissuasive efect, reducing the likelihood of repeat ofenses and contributing to one-inflation; 2. Mismatching: Individuals actually detected twice might be incorrectly recorded as single detections, artificially inflating the singleton category. Given the nature of our historical data, such mismatches are plausible. For instance, an intoxicated arrestee with impaired speech might have been difÏcult for police to accurately identify and record. Population turnover: As discussed earlier, the high rate of population flux in late 19thcentury Brussels could contribute to one-inflation by introducing many individuals who were only briefly present in the city. 3. Temporary absences: The practice of sending some arrested vagrants to colonies outside Brussels, as previously mentioned, could also contribute to one-inflation by interrupting the observability of these individuals.
The presence of one-inflation in our data may partially explain the limited improvement in predictive power we observed when incorporating demographic covariates into our models. It also underscores the importance of developing and applying methods that can account for such data characteristics in historical criminology research.
Looking beyond the specific challenges of our study, this work has significant implications for the broader field of computational humanities, particularly from the perspective of critical data studies. A persistent challenge in historical scholarship is the under-representation of marginalized groups in archival records. This systemic bias not only skews our understanding of historical societies but also perpetuates the silencing of under-represented voices. Our work with the Generalized Chao estimator ofers a concrete, quantitative approach to addressing this issue. By estimating the ‘dark numbers’ and characterizing demographic patterns in arrest vulnerability, we provide a method for: (a) quantifying the extent of under-representation in historical records, (b) identifying specific demographic groups that may be disproportionately under-represented, and (c) adjusting historical narratives to account for these ‘invisible’ populations.
While our study focused on historical criminology, the methodology has potential applications across various domains of historical research. It could be adapted to estimate underrepresentation in census data, literary corpora, or other historical datasets where certain groups may be systematically excluded or under-reported. Clearly, statistical methods alone cannot fully rectify historical biases. They must be used in conjunction with critical historical analysis, interdisciplinary collaboration, and a commitment to amplifying marginalized voices. Future research should explore how unseen species models can be integrated with other quantitative and qualitative methods in the humanities to provide a more comprehensive and equitable view of historical societies. The full code and data for replicating our analysis will be made available from the following GitHub repository upon the publication of the paper: https://doi.org/10.5281/zenodo.13969373.