The Voynich Manuscript, an enduring enigma that has challenged scholars for centuries, remains a formidable hurdle in linguistic decryption and historical cryptography. This study presents an innovative artificial intelligence-driven methodology, aligned with the overarching goals of the CHANGES project, specifically targeting symbol recognition. It tackles the intricate task of decoding this ancient manuscript through a refined computational analysis framework. Employing a convolutional neural network trained on an extensive dataset encompassing over 6000 symbols extracted from the manuscript, this approach demonstrates notable strides in both the classification and interpretation of these symbols. This computational tool represents a significant advancement in supporting scholars in symbol recognition and stands as evidence of the CHANGES initiative's commitment, providing invaluable assistance to researchers striving to unveil the historical and linguistic context embedded within the manuscript's symbols.
CEUR ceur-ws.org
of the ceurart style. LGOBE (S. Ferilli) The CHANGES project stands at the forefront of advancing techniques in digital imaging, preservation, recognition, and fostering accessibility of textual, text-image sources, and tangible as well as intangible linguistic heritage. It aims to develop innovative methodologies for the recognition of handwritten symbols. This article aims to introduce a novel tool aligned with the goals of CHANGES, focusing on its application to the unique and enigmatic case of the Voynich manuscript. At the core of CHANGES lies the primary objective of promoting the use of cutting-edge techniques in digital imaging while also ensuring preservation and recognition of texts. Furthermore, it aims to enhance accessibility to tangible and intangible linguistic heritage. To achieve these goals, the project envisions establishing an open-source web environment for automated recognition—both in terms of layout (HTR - Handwritten Text Recognition) ⋆You can use this document as the template for preparing your publication. We recommend using the latest version https://www.uniba.it/it/docenti/eleonora-bernasconi (E. Bernasconi); http://lacam.di.uniba.it/people/ferilli.html CEUR Workshop Proceedings
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). and character recognition, further training HTR engines for the automated recognition of linguistic features and loci critici required for digital recensio. The tool developed within this project represents a pivotal step towards achieving these ambitious objectives by enabling the recognition of handwritten symbols, addressing a significant challenge in textual analysis. The specific focus on the Voynich manuscript is noteworthy as it represents a corpus of writing devoid of an available Optical Character Recognition (OCR), providing a pristine and unexplored ground for the application of this innovative tool. Through the illustration of this use case, this paper aims to showcase how the tool developed within the CHANGES project can be efectively applied to a calligraphy lacking an existing OCR tool. This demonstration not only paves the way for potential recognition of previously unexplored scripts but also establishes an important precedent for applying this technology to similar contexts where the absence of OCR tools poses a substantial challenge.
The Voynich Manuscript, discovered by the bookseller and collector Wilfrid Voynich in 1912, represents one of humanity’s most intriguing mysteries. This document, dated to the 15th century, stands out for its main characteristic: a text written in an indecipherable language or code, accompanied by illustrations of plants, celestial bodies, and human figures.
Despite the eforts of cryptanalysts, linguists, and historians for over a century, the Voynich
Manuscript remains an unsolved enigma. Its language, structure, and meaning elude any
traditional interpretation attempts. This challenge has fueled academic curiosity and led to
various theories, yet no concrete solution has been reached [
The immense complexity of the Voynich Manuscript calls for an innovative approach to
overcome the obstacle of its indecipherability. The advent of modern technologies, such as
artificial intelligence, image analysis, and machine learning, ofers a new horizon of possibilities
in exploring and understanding this ancient text. These technological tools pave the way for
new perspectives in analyzing the symbols and linguistic structures present in the manuscript,
shedding new light on its interpretation [
Over the decades, eminent scholars, cryptanalysts, and linguists have devoted tremendous eforts to deciphering the Voynich Manuscript. Approaches based on statistics [ 6, 7, 8, 9], linguistic analysis [10], and historical hypotheses [11] have formed the bedrock of research. However, no traditional method has led to a satisfactory understanding of its enigmatic content. Attempts to associate it with known languages, historical ciphers, or recognized writing styles have only resulted in deadlocks.
The symbols present in the Voynich Manuscript constitute the crux of its indecipherability. The peculiar graphical representations, intricately intertwined with the text, are considered the keys to understanding its meaning. Analyzing the symbols, their frequency, arrangement, and potential correlations with known concepts or languages are crucial elements in unraveling the mystery hidden within these ancient pages.
This paper aims to examine the crucial role of innovative technologies, particularly an automatic symbol recognition tool, in approaching the decoding of the Voynich Manuscript. Through the application of this technology and the analysis of the obtained results, the goal is to highlight the efectiveness and value of such tools in the realms of historical and linguistic research. The primary objective is to contribute to the research and understanding of one of the greatest historical mysteries, ofering a new perspective through the synergy between artificial intelligence and the humanities.
Our approach to symbol recognition in the Voynich Manuscript is based on a structured methodology that synergistically leverages image processing techniques and machine learning. The operational sequence adopted in the application follows a systematic pathway comprising the following phases: • Import of the target image: This initial phase involves importing the Voynich
Manuscript images for analysis. • Image filtering: Images undergo filtering processes to enhance quality and prepare them for subsequent analysis phases. Processing includes noise removal, histogram equalization, and applying filters to improve sharpness. • Region of interest extraction: Using advanced image segmentation algorithms, relevant areas containing handwritten symbols are identified and isolated. This phase requires a combination of thresholding, contours, and connected regions techniques. • Contour detection: This phase focuses on precise identification and tracing of contours of symbols present in the image. The approach includes edge detection algorithms and contour extraction to precisely define the boundaries of symbols. • Clustering: Employing advanced clustering techniques, the extracted symbols are aggregated based on common characteristics, such as shape and structure, preparing them for further processing. • Training a convolutional neural network: A crucial step involves training a convolutional neural network using data extracted from preceding phases. This learned model facilitates the identification and assignment of Voynich symbols to specific categories within the selected images. • Performance evaluation: Post-training, the model’s performance is evaluated and optimized to ensure accurate results. Various evaluation metrics like precision, recall, and F1-score are executed to assess the model’s efectiveness. • Prediction and decoding: Finally, the trained convolutional neural network is employed to predict and decode the Voynich symbols present in the target images, assigning identified symbols with corresponding labels to their classification.
We emphasize that our methodology is designed to be entirely accessible to humanists, providing an intuitive interface supporting users throughout the process. This allows humanists to fully manage the entire procedure, from dataset assembly to convolutional neural network training to identification of Voynich symbols in selected images.
Furthermore, we focus on Explainable AI aspects [12, 13, 14, 15], ensuring transparency in the decision-making processes of the convolutional neural network. This approach, utilizing artificial intelligence for humanists’ benefit through an Explainable AI-empowered tool, allows more active participation and critical interpretation of results. Such an approach raises crucial ethical and theoretical questions concerning the intersection of technology and humanities, laying the groundwork for further reflections and insights in interdisciplinary research.
Users have the ability to upload images containing handwritten symbols found in the Voynich manuscript for analysis and recognition purposes. This initial step allows the introduction of the image into the application’s working environment.
The selection of images constitutes a crucial aspect for constructing the dataset. The dataset is formulated based on symbols automatically detected and extracted from the chosen initial images. Proper image selection directly influences the quality and representativeness of the dataset, contributing to the robustness of the entire process of training and recognizing Voynich symbols.
The acquired image undergoes a series of filtering and preprocessing processes. These processes aim to optimize the visual quality of the image and adequately prepare it for subsequent stages of symbol analysis and recognition.
The involved filtering operations encompass various techniques. The Canny Edge Detection focuses on precise edge detection, reducing noise [16]. However, it is sensitive to lighting changes and requires careful calibration. The Gaussian Blur reduces noise and enhances details but might compromise sharpness due to blurring.
The Bilateral Filter reduces noise without compromising details, although it may increase processing time [17]. The Median Filter removes noise without compromising major details but has limitations in handling complex noise [18]. The CLAHE technique dynamically adjusts contrast, emphasizing details, but it might overload the image with information [19].
Histogram Equalization promotes an even distribution of grayscale levels but could lead to excessive contrast enhancement, reducing the image’s naturalness [20]. Lastly, Unsharp Masking enhances details and contrast, yet excessive application can generate visual artifacts [21].
These diverse techniques, used in sequence or combination, aim to standardize and enhance the image, making details clearer and facilitating the analysis of symbols present in the Voynich manuscript. This filtering phase represents a crucial step for decoding and interpreting its enigmatic contents.
Currently, scholars can manually extract regions of their interest directly from the interface. However, we are planning to simplify this task by employing image segmentation algorithms. These algorithms will suggest relevant regions containing handwritten symbols. This processing phase identifies key sections of the image, preparing them for subsequent stages of symbol recognition and interpretation.
A crucial step in analyzing the Voynich manuscript is detecting and tracing the contours of symbols present in the image. This phase is fundamental for accurately and automatically identifying and analyzing individual symbols.
Figure 1 illustrates the contour detection process. Initially, the original RGB color image is loaded. Subsequently, the Canny edge detection filter is applied to highlight the edges of symbols in the image. The identified contours are then distinctly colored on the initial image.
In more detail, the contour detection phase begins with loading the original image, followed by creating two copies of this image: one to display thin contours and the other to label the identified contours with distinctive IDs. To facilitate analysis, the color image is converted to grayscale.
Next, the Canny edge detection filter, an efective algorithm for detecting object edges in the image, is applied [22, 23, 24]. This filter generates an image with highlighted edges, enabling precise detection of present symbols. Subsequently, the ‘findContours‘ algorithm from OpenCV [25] is used to identify and trace the contours of the detected symbols in the Canny image.
Once the contours are identified, each of them is distinctly colored on the image with thin contours, and a corresponding ID is added to another copy of the image, facilitating symbol identification.
Finally, the area, appearance, and cropped image for each contour are saved in a list to allow further analysis and visualization. The function also enables filtering results based on the number of points, area, and appearance of contours, providing users with the option to select specific contour IDs for detailed analysis.
An important phase in processing the dataset for training artificial intelligence models is the clustering and grouping process of symbols present in historical documents. This step plays a crucial role in making categorization and dataset preparation easier for humanists.
The developed interface ofers a range of functionalities to simplify this complex task. By implementing clustering algorithms such as K-Means and Agglomerative Clustering, combined with binarization methods like Global, Adaptive, Otsu, Gaussian, and Inverse, the user has the ability to manage and divide symbols into clusters based on their visual characteristics [26, 27].
A representative image of the cluster identification process is shown in Figure 2. This figure displays the division of symbols into clusters 4 and 5, identified through the interface. The graphical representation provides users with a visual indication of the symbol categorization process.
The functionality to display the Silhouette Score or the Elbow Method [28] gives users a clear indication of the optimal number of clusters to use for symbol categorization, making the decision-making process more intuitive and informative.
The image binarization process [29] is of fundamental importance for dataset preparation. This step reduces noise in images and facilitates subsequent training of machine learning models.
Finally, the interface ofers the ability to create a structured dataset with symbols divided into identified clusters, simplifying further analysis and preparing a dataset ready for training artificial intelligence models.
During this process, machine learning algorithms are applied to identify and recognize symbols extracted from the Voynich manuscript image. The model is constructed using a sequential architecture composed of convolutional layers [30], MaxPooling layers, Fully Connected layers (Dense), and Dropout layers. The use of these layers allows the model to learn the salient features of the symbols for their classification.
Conv2D Layer: These layers represent the convolutional filters that perform convolution on the input image to extract various features. The number of filters indicates the depth of the output. More filters mean more features can be extracted. MaxPooling2D Layer: These layers perform pooling to reduce the spatial dimension of the output, decreasing the number of parameters and the risk of overfitting. Pooling also helps retain the main features while reducing redundant information. Flatten Layer: This layer flattens the output from a three-dimensional shape into a one-dimensional vector. It is a necessary step before moving to densely connected layers. Dropout Layer: This layer is used to reduce overfitting during training by randomly ”turning of” some neurons during each iteration, forcing the model to use all its pathways and not overly rely on specific features. Dense Layer: These layers are fully connected, performing the final computation and producing the output. More neurons in these layers imply a higher capacity for the model to learn complex features and relationships in the input data. These combined layers allow the model to learn and recognize increasingly complex hierarchical features in the images, progressively enhancing its ability to identify Voynich manuscript symbols. The code below implements the creation of the CNN model using TensorFlow and Keras. Additionally, it includes functions to load images from the dataset, split the dataset into training and test sets, and display crucial dataset information such as class distribution and data statistics. The loading of images occurs through the function that loads images from the specified path, converts them into NumPy arrays, and encodes them. Subsequently, the dataset is split into training and test sets. The percentage of data to be used for training and testing can be configured through the user interface. This allows for flexible customization of the data split, adapting it to the specific needs of the problem or model being addressed. The visualization of dataset information, such as image shapes, label information, and class distribution, is provided to better understand the composition and structure of the data. The CNN model is trained with the ’adam’ optimizer and ’sparse categorical crossentropy’ loss function.
In this phase, the application ofers the ability to evaluate the performance of the trained model in recognizing symbols and the accuracy of the predictions made. Upon application startup, the ”Results” section displays graphs illustrating the accuracy and loss trends during the model training. These graphs provide an overview of the model’s performance on the training and test sets, allowing an assessment of its generalization capability. Subsequently, the results of testing the trained model using test data are displayed. The model’s loss and accuracy on the test data are reported. Additionally, some predictions made by the model for certain test samples are shown 3. This allows users to visually understand the correspondence between the model’s predictions and the true labels of the symbols.
The graphical representation of predictions and true labels of test images enables the evaluation of the model’s accuracy in identifying symbols. Images are presented along with the predicted class by the model and the corresponding true class, facilitating understanding of inaccurate or accurate predictions. Finally, the confusion matrix 4 is provided to ofer an overall view of the model’s performance, displaying how many times the model correctly or incorrectly predicted each symbol class.
All these visualizations are designed to ofer humanist users a clear and efective representation of the symbol recognition system’s performance, allowing for accurate evaluation and in-depth analysis of the model’s predictions.
The final phase of our methodology involves rigorous testing of the trained artificial intelligence system specifically designed for the identification and interpretation of symbols within the Voynich manuscript. The initial symbol extraction method, reliant on contour detection, is consistently applied. Following extraction, symbols are organized in left-to-right rows and subjected to a series of filtering criteria, as outlined in Figure 5, Part A. These filters are based on area size, contour point count, and aspect ratio. Post-filtration, the methodology computes recurring sequences of detections, as depicted in Figure 5, Part C. Concurrently, the count of detections per class is systematically recorded. This methodical process ensures a precise and thorough analysis of Voynich manuscript symbols, contributing to a nuanced understanding of its symbolic representations.
The trained artificial intelligence undergoes rigorous evaluation using previously unseen data to test its accuracy and reliability in correctly identifying manuscript symbols. This phase represents a crucial moment as it measures the system’s efectiveness in consistently interpreting a text with multiple interpretations.
The results obtained from applying the methodology described in Section 3 to the Voynich manuscript images have yielded significant results in symbol detection, highlighting various strengths and important insights. Various techniques were applied to improve the quality of images from approximately 10 Voynich manuscript images. 6383 symbols were identified and extracted. The K-Means clustering algorithm, coupled with Otsu binarization [31], showcased efectiveness owing to its capacity to handle multivariate data while minimizing intra-cluster variability. This combination, further enhanced by Dimensionality Reduction using PCA (Principal Component Analysis) [32], automatically identified 26 clusters. The application of PCA aided in reducing the dataset’s dimensions while retaining essential features, contributing to more eficient clustering by preserving critical information. It is noteworthy that the distribution of the number of images associated with each cluster and therefore each class was uneven, as shown in Figure 6.
The parameters used for training the convolutional neural network described in Section 3.6 were chosen considering test-size of 0.20 and random-state of 42. These values were selected to ensure an adequate division of data into training and test sets while maintaining consistency in result reproducibility. The number of epochs for training was set to 100, allowing the model to learn from the data for a suficient number of iterations. During the training of the neural network, a progressive improvement in performance metrics was observed, as highlighted in the trend of accuracy and loss over the 100 epochs, depicted in Figure 7.
The results obtained from the methodology applied to the Voynich manuscript images highlight significant progress in symbol detection and text interpretation. This approach stands out for its efectiveness in identifying 26 symbol classes. In comparison with previous studies, significant progress is evident in the quantity of extracted and identified symbols. However, the uneven distribution of images among the 26 clusters identified poses a challenge in correctly assigning symbols to specific classes. The training of the convolutional neural network model showed a consistent improvement in performance metrics over the 100 epochs. The trend of accuracy and loss indicated a positive trend, demonstrating the model’s efectiveness in learning and recognizing symbols. The efectiveness of this tool in interpreting the Voynich manuscript could have a significant impact in aiding humanists in historical and linguistic research. The improvement in symbol detection could facilitate the identification and understanding of meanings behind this enigmatic writing, opening new research perspectives in historical and linguistic ifelds. However, it is important to emphasize that the uneven distribution of classes may require further eforts from humanists in accurately categorizing symbols. Therefore, improving the clustering methodology combined with validation from humanists could contribute to better class organization and more precise interpretation of the manuscript. In conclusion, while ofering significant progress in symbol identification, our tool requires further developments to address remaining challenges, thus providing a significant contribution to advancing historical and linguistic research of the Voynich manuscript.
In conclusion, the CHANGES project has taken significant strides in advancing techniques related to digital imaging, preservation, and recognition, with a specific focus on promoting accessibility to linguistic heritage. The development of our novel tool aligns seamlessly with the overarching goals of CHANGES, particularly when applied to the intricate case of the Voynich manuscript. Through our work, we have showcased the tool’s eficacy in recognizing handwritten symbols, addressing a critical gap in the realm of linguistic heritage preservation. Summarily, our tool has played a crucial role in decoding the mysterious Voynich manuscript. The identification of 26 symbol classes from a limited set of images demonstrates the methodology’s potential in interpreting enigmatic scripts. The trained convolutional neural network’s proficiency in learning symbols signifies significant progress in detecting and associating complex symbols. However, challenges related to the uneven distribution of classes necessitate further optimization for precise symbol categorization.
This research was partially supported by projects FAIR – Future AI Research (PE00000013), spoke 6 – Symbiotic AI, and CHANGES – Cultural Heritage Active innovation for Next-GEn Sustainable society (PE00000020), Spoke 3 – Digital Libraries, Archives and Philology, under the NRRP MUR program funded by the NextGenerationEU. [6] T. Timm, A. Schinner, A possible generating algorithm of the voynich manuscript, Cryptologia 44 (2019) 1–19. URL: http://dx.doi.org/10.1080/01611194.2019.1596999. doi:10.1080/ 01611194.2019.1596999, 4 cites:. [7] D. R. Amancio, E. G. Altmann, D. Rybski, O. N. Oliveira, L. da F. Costa, Probing the statistical properties of unknown texts: Application to the voynich manuscript, PLoS ONE 8 (2013). URL: http://dx.doi.org/10.1371/journal.pone.0067310. doi:10.1371/journal. pone.0067310, 40 cites:. [8] M. A. Montemurro, D. H. Zanette, Keywords and co-occurrence patterns in the voynich manuscript: An information-theoretic analysis, PLoS ONE 8 (2013). URL: http://dx.doi. org/10.1371/journal.pone.0066344. doi:10.1371/journal.pone.0066344, 38 cites:. [9] O. Golubitsky, S. M. Watt, Distance-based classification of handwritten symbols, International Journal on Document Analysis and Recognition (IJDAR) 13 (2010) 133–146. [10] F. Crowe, The voynich manuscript: Decoded, Journal of Historical Archaeology & Anthropological Sciences 7 (2022) 94–131. URL: http://dx.doi.org/10.15406/jhaas.2022.07. 00262. doi:10.15406/jhaas.2022.07.00262, query date: 2023-11-30 09:13:24. [11] A. Schinner, The voynich manuscript: Evidence of the hoax hypothesis, Cryptologia 31 (2007) 95–107. URL: http://dx.doi.org/10.1080/01611190601133539. doi:10.1080/ 01611190601133539, 27 cites:. [12] W. Samek, K.-R. Müller, Towards explainable artificial intelligence, Explainable AI: Interpreting, Explaining and Visualizing Deep Learning (2019) 5–22. URL: http://dx.doi. org/10.1007/978-3-030-28954-6_1. doi:10.1007/978-3-030-28954-6_1, 209 cites:. [13] H. Hagras, Toward human-understandable, explainable ai, Computer 51 (2018) 28–36. URL: http://dx.doi.org/10.1109/mc.2018.3620965. doi:10.1109/mc.2018.3620965, 192 cites:. [14] A. Holzinger, A. Saranti, C. Molnar, P. Biecek, W. Samek, Explainable ai methods - a brief overview, xxAI - Beyond Explainable AI (2022) 13–38. URL: http://dx.doi.org/10.1007/ 978-3-031-04083-2_2. doi:10.1007/978-3-031-04083-2_2, 60 cites:. [15] M.-A. Clinciu, H. Hastie, A survey of explainable ai terminology, Proceedings of the 1st Workshop on Interactive Natural Language Technology for Explainable Artificial Intelligence (NL4XAI 2019) (2019). URL: http://dx.doi.org/10.18653/v1/w19-8403. doi:10. 18653/v1/w19-8403, 27 cites:. [16] R. Song, Z. Zhang, H. Liu, Edge connection based canny edge detection algorithm, Pattern Recognition and Image Analysis 27 (2017) 740–747. URL: http://dx.doi.org/10.1134/ s1054661817040162. doi:10.1134/s1054661817040162, 42 cites:. [17] S. Paris, P. Kornprobst, J. Tumblin, F. Durand, et al., Bilateral filtering: Theory and applications, Foundations and Trends® in Computer Graphics and Vision 4 (2009) 1–73. [18] B. Justusson, Median filtering: Statistical properties, Two-Dimensional Digital Signal
Prcessing II: Transforms and Median Filters (2006) 161–196. [19] S. Sahu, A. K. Singh, S. Ghrera, M. Elhoseny, et al., An approach for de-noising and contrast enhancement of retinal fundus image using clahe, Optics & Laser Technology 110 (2019) 87–98. [20] G. Malik, A. S. Sappal, Adaptive equalization algorithms: an overview, International journal of advanced computer science and applications 2 (2011). [21] G. Ramponi, N. K. Strobel, S. K. Mitra, T.-H. Yu, Nonlinear unsharp masking methods for image contrast enhancement, Journal of electronic imaging 5 (1996) 353–366. [22] F. J. P. Montalbo, D. P. Y. Barfeh, Classification of stenography using convolutional neural networks and canny edge detection algorithm, in: 2019 International Conference on Computational Intelligence and Knowledge Economy (ICCIKE), IEEE, 2019, pp. 305–310. [23] P. Kumar, T. Narendra, N. Vinay, Short hand recognition using canny edge detector,
International Journal 7 (2017). [24] H. Koyuncu, A comparative study of handwritten character recognition by using image processing and neural network techniques, Hittite Journal of Science and Engineering 8 (2021) 133–140. [25] G. Bradski, The opencv library., Dr. Dobb’s Journal: Software Tools for the Professional
Programmer 25 (2000) 120–123. [26] N. J. Wala’a, J. Rana, A survey on segmentation techniques for image processing, Iraqi
Journal for Electrical and Electronic Engineering 17 (2021) 73–93. [27] F. U. Siddiqui, A. Yahya, Clustering techniques for image segmentation, Springer, 2022. [28] Y. Jung, H. Park, D.-Z. Du, B. L. Drake, A decision criterion for the optimal number of clusters in hierarchical clustering, Journal of Global Optimization 25 (2003) 91–111. [29] C. Tensmeyer, T. Martinez, Historical document image binarization: A review, SN
Computer Science 1 (2020) 173. [30] Z. Li, F. Liu, W. Yang, S. Peng, J. Zhou, A survey of convolutional neural networks: analysis, applications, and prospects, IEEE transactions on neural networks and learning systems (2021). [31] A. Moghimi, S. Khazai, A. Mohammadzadeh, An improved fast level set method initialized with a combination of k-means clustering and otsu thresholding for unsupervised change detection from sar images, Arabian Journal of Geosciences 10 (2017) 1–18. [32] M. Greenacre, P. J. Groenen, T. Hastie, A. I. d’Enza, A. Markos, E. Tuzhilina, Principal component analysis, Nature Reviews Methods Primers 2 (2022) 100.