A vast amount of our cultural heritage is at risk of getting lost because it resides in old books that are difÏcult to access. It is therefore important to make this information available to human readers but also to machine analysis, so that new representations and insights based on this knowledge can be constructed. In our case study, we use a host of digital tools to extract and analyze a corpus of 19th century French texts about the practices of market gardening in Paris, and to apply a variety of possible visualizations in an integrated interface. Our work includes a Named Entity and Linking procedure for creating maps of the locations mentioned in these texts as well as the social networks of people cited in the books. We also consider how the analysis of verbs can approximate and represent the knowhow of market gardening: we analyze the statistics of those verbs compared to their usage in a general corpus for French, and map the verbs using word embeddings. Finally, we also consider a semantic frame analysis to extract causal relations from texts to evaluate how well these relations support the biological knowledge embedded in those texts (such as how too much exposure to the sun may afect the quality of the garden's produce). Altogether, we show how the visualizations based on Natural Language Processing and Textual Statistics could support a convivial navigation through the corpus.
Digital libraries gather large corpora of texts which are beyond human possibilities of reading.
One of the tasks of digital humanities [
A large collection of works dedicated their eforts in this direction, applied to literary texts
[
19th century texts are particularly interesting because shortly after that period, from the second part of the 20th centuray onwards, agriculture went through radical changes with the green revolution and the introduction of chemicals to control the growth and the environment of plants. These changes, which were driven by the agronomical institutions, were so sweeping that we can reasonably ask whether some part of the old knowledge was lost. To answer this question, it is necessary to mine the older texts; and their analysis will also help visualize some interesting aspects of the history of agriculture.
We present here how we built the corpus, the preprocessing of the data and some analysis we did on the texts. First, we performed Named Entity Recognition and Linking to gather information on the places and people cited in those books. Then, we analyzed the verbs appearing in the corpus through semantic embeddings. And finally, we collected sentences expressing causal relations as those are most susceptible of containing agronomical knowledge. For each of these analyses, we provide visualization which can help navigate the corpus in an interactive manner.
The gardening manuals of the 19th century are a memory of the development of very efÏcient
methods for growing vegetables in an urban environment (as many of these books are focused
on the practices in the Paris area). These methods of cultivating very densely mixtures of crops
on small plots of land have inspired a movement in California and more recently in Europe
commonly referred to as the Biointensive French Method [
The first selection of books was collated by looking at the recommended readings accessible on an online platform about agroecological practices. The GOM1 corpus is thus composed of seven books listed in the table below. Additionally, we included 14 more books in the full GOM corpus after discussions with specialists of market gardening. All books are related to market gardening and were published between 1802 and 1912. For the following textual analysis, we only consider the GOM1 section of the corpus. The list of books included in the full GOM corpus is available on the companion website 1.
The first step in our analysis is to extract the layout of each page, identifying regions of the page
occupied by text paragraphs, title, figures or tables using an image segmentation algorithm
based on Faster RCNN trained on a large collection of publications [
As expected, the resulting text still includes many mistakes, so a first preprocessing was done
to substitute characters unlikely to appear in the text by their most likely replacement (for
example ä->à). Next, to correct spelling mistakes from the OCR, we filtered out-of-vocabulary
words (using the reference lexicon MORPHALOU3; [
It is important to identify the places and people cited in the GOM corpus so that the texts can be properly situated in their appropriate geography and history. For this, we used the out-of-vocabulary words, and selected the ones written starting with a capital letter. We then matched this list to a dictionary of geographical locations including their localization as GPS coordinates. In the remaining words, we checked manually, through web search, in the most commonly cited if those correspond to personalities.
Additionally, for places, there is a common ambiguity in our corpus on whether the name of a location is used to refer to the location or to a variety of plant originating from this location. To disambiguate this, we manually annotated all the mentions of names of locations as referring to the location or to a variety of plant originating from this location.
Based on this recognition of places and people, we were able to visualize both aspects. First, in a graph on Fig.2, we represented the authors and the most cited people (more than 2 times). We drew an edge between an author and a cited person if this person was cited by the author. We see that some authors cite generously, while some others only mention a few people. For example, in the Moreau & Daverne, only Héricart de Thury and Mr Gontier are cited. The book they wrote was a response to a call emitted by the Royal Society of Horticulture, whose director was Héricart de Thury; and Mr Gontier was a market gardener in the region of Nantes and who was among the first to experiment with an innovative technology of the time, the thermosiphon. For places, on Fig.3, we placed circles on a map of France with the radius denoting the frequency of occurrence of the name of place in the GOM1 corpus. We notice that there are many mentions of places in the Paris region, which is expected since a lot of the practices we are interested in are originating from the Paris region.
It is interesting to focus on the verbs mentioned in the GOM corpus as they reflect the actions
that are important to a market gardener on their farm. We are particularly interested in the
verbs that are specific to market gardening, which can be considered as a keyword identification
problem. For this, we first lemmatize and POS tag the texts using spacy, a widely used tool for
2We used the confusion matrix available at https://github.com/shaneweisz/OCR-Character-Confusion/blob/mast
er/confusion_matrix/confusion_matrix_base.pkl
people are listed in the central columns. Names in purple refer to people mostly on the knowledge side
(professors of agronomy or botany for example) and names in yellow refer to people involved on the
practical side (market gardeners, seed sellers,...).
(
)
various NLP tasks 3. Then, similarly to the keyness commonly used in corpus linguistics [
The word cloud in Fig.4 shows the verbs with a size proportional to this index in yellow and the verbs not appearing in FRANTEXT in red with a size proportional to the log of the frequency of occurrences in GOM1.
In the previous representation the location of words has no interpretation and we also want
to represent the words in a space where two words located close together would have similar
meaning (in the distributional sense). That representation can be useful, for example, to show
groups of words clustered together having a similar meaning. We represented each verb using
its embedding in a word2vec model trained on a large French corpus [1] and we visualize the
map of verbs after reducing the dimension of the embedding to 2 dimensions using UMAP [
The GOM corpus gathers an rich mixture of practical advice and practical knowledge. It is interesting to study whether the discourse in those books reflects this dichotomy between practices and knowledge. A key feature of the transition of discourse from practice to knowledge is nominalization, a linguistic process where nouns are derived from verbs [11]. Thus in the particular example of the verb arroser (“to water”), we plot the usage statistics in each of the 7 books of the GOM1 corpus. We see, in Fig.6 top panel, that some authors favor much more the use of the verb than the noun, denoting a more practical and less abstract discourse. Also, it is interesting to note that in the case of the verb arroser, there were actually two forms for the corresponding noun: arrosage and arrosement (both meaning “the watering (of crops)”). By plotting the frequencies of occurrence of these two terms in large corpora (Gallica and Google books), it shows that the 19th century is precisely the time during which those 2 terms coexisted, arrosement being used more frequently before; and arrosage becoming dominant after the 19th century. Some references (ATILF) mention a small diference in the meaning of those 2 terms, arrosement being more related to a passive manner for plants to receive water and arrosage referring to a more active process from a human to provide the water.
We were also interested in capturing the parts of the discourse reflecting causal relations because in the sentences expressing causality, we may find elements of biological knowledge. For example, let us consider the following sentence from Moreau et Daverne:
”Autre observation : la pratique nous a appris que, pendant l’été, si nous arrosons nos romaines durant le grand soleil avec l’eau froide de nos puits, quand elles sont près de se coifer ou déjà coifées, cela détermine dans leur intérieur des taches de pourriture; nous disons alors que la romaine est mouchetée : dans cet état, elle n’est plus bonne pour la vente.” “Another observation: practice has taught us that, during the summer, if we water our romaine plants in the hot sun with cold water from our wells, when they are about to be capped or have already been capped, this causes spots of rot inside them; we then say that the romaine is speckled: in this state, it is no longer fit for sale.”
Here, the authors draw a causal relation between on the one hand the watering of the crops with cold water when it’s hot at a specific growth stage of the crops; and on the other hand the rotting of their leaves. Even though knowledge was too scarce at the time to fully explain this phenomenon, namely that these conditions are favoring the growth of fungi, it is clearly some kind of knowledge about biology that is encapsulated in the text.
To detect such causal relationships in a systematic matter, we are currently performing a
Frame-Semantic analysis [
Our Semantic Frame Extractor has been implemented in Fluid Construction Grammar (FCG;
[
In its current form, a Causal Frame extractor is already useful because it can search through a text for instances of causal language, and then present the results to the human reader. We are currently evaluating how well a frame extractor trained on contemporary French data can be applied to the Good Old Manual corpus. For this, we are annotating a test set of causal expressions that can be found in the corpus. Moreover, as can be seen in Figure 7, the Frame Extractor currently identifies Frame Elements through syntactic relations, so the syntactic subject cela was assigned the role of Cause rather than the semantic subject (printed in italics), which is what really matters for extracting knowledge. Future work will therefore have to include anaphor resolution and tracking entities across longer spans in the discourse.
Old texts are often treasure troves of past knowledge that has become almost inaccessible or even forgotten as societies evolve. Especially “good old” manuals, which have so far been neglected, ofer a great potential source of information about the knowledge and practices of a given time and place. In this paper, we have illustrated how a suite of techniques from Digital Humanities, natural language processing, statistical analysis and data visualization, can be exploited to make such texts not only accessible, but also more meaningful to human readers.
More specifically, we have introduced the Good Old Manual corpus of 19th century texts about French market gardening, particularly in the Paris region. These techniques have recently gained a renewed interest because they ofer insights into increased efÏciency for farming on small plots of lands, known as the French Method. We have demonstrated how the most prominent actors at the time can be situated in a social and geographic network through named entity linking; how activities that are relevant and meaningful to specific topics such as market gardening can be visualized through word clouds and word embedding spaces, and how more fine-grained knowledge could potentially be mined through semantic parsing.