Digital humanities still need to unlock the potential of images anlysis algorithms to a large extent. Modern deep learning images processing can contribute much to quantify knowledge about visual components in books. In this study, we report on experiments carried out for historical print. The illustrations in books offer much for humanities research. Object recognition systems can identify the portfolio of objects in book illustrations. In a study with several hundreds of books, we applied systems to find illustrations and classify them. Results show that persons are shown in illustrations within fiction books with a higher frequency than in non-fiction books. We also show the classification results for an analysis of the printing technology. This expert task can still not be perfectly modeled by a CNN. A class activation map analysis can be used to analyze the performance qualitatively.
Digital Humanities integrates automatic processing and analysis into research practices
in the Humanities. Image analysis is a growing area within Digital Humanities. The
analysis of books is of great interest to many disciplines. Digital historical corpora
allow the automatic access to illustrations in books and their analysis in large quantities.
This can lead to innovative research questions and quantitative results [
Historical children’s and youth books have yet not often been the subject of research.
Children books typically contain more images than adult books typically. As a
consequence, they are of special interest for an analysis of images. In addition, they form a
closed category on the one hand which contains sufficient variety on the other hand [
Illustrated books have played a significant role in knowledge dissemination. The declining production costs for printed images have led to a growing exposure of more and more people to rich visual resources. Research in this area could identify trends in the objects depicted.
Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
The progress in computer vision and images analysis in the last decade was substantial.
Deep learning as a new direction in machine learning now can be considered as state of
the art and delivers excellent results in many domains. Deep learning refers to a
departure from feature engineering. Algorithms instead find a representation space for the
problem at hand by themselves. Neural networks have proven to be very successful for
this task. Already early approaches intended to input compresses feature spaces into
backpropagation neural networks [
Convolution Neural Networks (CNN), the recent state of the art technology is known
to be very effective in automated feature detection and subsequent classification in
many domains. CNNs are composed of recurring sets of two layers: a convolution layer
and a pooling layer. The CNN combines pixels locally and by working through many
layers, more complex features can be extracted [
The applications of computer vision to the digital humanities is a growing area.
However, there are still not very many publications in this domain. One influential
experiment in the art domain by Salah & Elgammel is dedicated to classify the painter of
artistic work. Such work is highly dependent on the type of paintings in the collection
[
A recent project is focusing on research on graphic novels. Current state of the art
CNNs are applied to tasks like author identification with very good success [
A recent approach shows that the visual analysis of a page structure can be carried
out successfully with CNNs. A system can detect elements on a page and analyze tables
from heterogeneous layouts [
One goal for the research of images in historical children books lies within the
production technologies. As a classification problem with few classes, it seems like a
challenge which could be solved with current technology. The classification of production
technology in the 19th century is still a hard task [
This research is exploiting two collections of books that are digitized. The first
collection is the Wegehaupt corpus maintained by the Staatsbibliothek in Berlin [
We present some results for using a system with pre-defined classes and one with a self-trained model. All models use deep learning systems and in particular some variant of a CNN architecture.
After extracting images from the book pages, we use them as input for a object
classification system pre-trained on modern photographs. In previous work, we could see
that the classification results are not perfect and that the greatly differ between books.
Typical performance metrics can lie between 30% and 90% for different books [
Yolo to the images and record the recognized object types. The results of Yolo [
1.143 0.737 1.446 1.151 It needs to be stressed that the distribution of the classes is highly skewed. The most frequent classes are humans and a few animals. These results currently do not allow a quantitative tracing of many different motifs through the century.
For the study of historical print, questions of materiality are of great importance. Issues of aesthetic design needs to be considered in relation to the techniques and printing technologies available. Printing technology like wood cut, wood engraving and lithography allowed different levels of elaboration. Finding the technique is a tedious task. It is often not stated in the meta data and it requires experts to identify it from digitized books.
Therefore, the automatic identification is a important requirement. We trained a
model for distinguishing between three classes and managed to achieve only a
reasonable performance. However, for a statistical analysis, this can be sufficient.
To further analyze the errors and to observe how the algorithms differs from human
experts, we applied a qualitative analysis with class activation mapping (CAM)[
We thank the Fritz Thyssen Foundation for their funding for the research project Distant Viewing. We thank the library of the Technische Universität Braunschweig, the BBF | Bibliothek für Bildungsgeschichtliche Forschung Bibliothek Bildungsgeschichtliche Forschung, Abteilung des DIPF | Leibniz-Institut für Bildungsforschung und Bildungsinformation and the Staatsbibliothek Berlin (Preußischer Kulturbesitz) for providing and facilitating access to their digitized collections.