This research focuses on evaluating the effectiveness of the Attention-Gated-CNN-BGRU model for Handwritten Text Recognition from historical documents, specifically from the archive of Khmelnytskyi Oblast, written in Ukrainian and Russian languages between 1861 and 1913. The methodology involved preprocessing, data augmentation, deep learning with attention mechanism, and expert assessment. The obtained results showed an average percentage of correctly recognized characters at 71.7%, demonstrating the high effectiveness of the model. A strong negative correlation between text complexity and recognition accuracy underscores the need for further improvement in Optical Character Recognition technologies. The main direction of future research will be adapting the model for recognizing texts written in the Ukrainian language using the Latin alphabet, which is crucial for preserving Ukraine's cultural heritage.
The modern world of digital technologies opens new horizons for the preservation and study of historical documents, offering unique tools for exploring cultural heritage. One of the key directions in this field is the development and application of OCR systems, which automate the process of converting image-based text into machine-readable format. This, in turn, facilitates easier access to historical documents, their analysis, and interpretation. However, the characteristics of historical manuscripts, such as variability in writing styles, degree of preservation, and material erosion, pose challenges for researchers that require the development of specialized OCR algorithms and methods.
This scientific article is dedicated to analyzing the effectiveness of the
Attention-Gated-CNNBGRU model [
Given the importance of preserving historical heritage and the development of digital humanities, the results of this research aim not only to demonstrate the potential of modern OCR technologies in historical text analysis but also to provide valuable insights for further improvement of digital humanities tools.
The formulation of the problem. In the context of the development of digital technologies and the study of cultural heritage, there arises the problem of effectively recognizing text from historical manuscripts using OCR systems. This problem arises due to the diversity of writing styles, differences in the preservation state of documents and their materials, complicating the process of automated conversion of images into machine-readable form. Such technical challenges create the necessity for the development and enhancement of specialized OCR algorithms to effectively process historical manuscripts.
This article presents a method for evaluating the effectiveness of the Attention-Gated-CNNBGRU model for text recognition from historical manuscripts, based on deep learning with attention mechanism. Chapter 2 discusses a review of related works; Chapter 3 describes the research methodology, including dataset description and document recognition approach; Chapter 4 is dedicated to the implementation of the algorithm and detailed analysis of the obtained results. Chapter 5 presents the research conclusions summarizing the main findings and emphasizing the prospects for further research in this field.
The development of artificial intelligence (AI) technologies is one of the key and significant trends
of the present day. This is primarily explained by the ability of AI, or more accurately
"computational" or "electronic" intelligence, to learn and self-learn, recognize and synthesize
language and images. Worldwide practice already has results of AI activity in the field of historical
research. For example, Swedish scientists were able to reconstruct a complex handwritten text
from the 18th century, stored in the Swedish National Archives [
In Ukraine, specialists from not only the Institute of Artificial Intelligence Problems of the Ministry of Education and Science of Ukraine and the National Academy of Sciences of Ukraine are working on the development and improvement of AI capabilities, but also scientific and pedagogical workers and researchers from departments and divisions of AI, computer science and applied mathematics, computer technologies and economic cybernetics, innovative technologies, intellectual information systems, mathematical modeling, AI systems, information security of many higher education institutions and research institutes of Ukraine. A leading role in this field is played by the team of the Research Institute of Intelligent Computer Systems of the Western Ukrainian National University (Ternopil) and the V.M. Glushkov Institute of Cybernetics of the National Academy of Sciences of Ukraine (Kyiv).
Modern research in the field of AI is actively developing, especially in the area of document
interpretation, where information technologies are used for analysis, search, and interpretation
of relevant texts. Significant important informational potential for our research[
Innovative approaches to preserving cultural heritage through the use of image recognition
and augmented reality, as seen in the researchs [
Research [15] explored offline handwritten text recognition (HTR) with reduced training
datasets, proposing a model trained on crossed-out text to effectively recognize such words
without compromising accuracy. In [16], methods for classifying handwritten words into a digital
format were investigated, combining direct word classification using Convolutional Neural
Networks (CNN) and character segmentation with Long Short-Term Memory (LSTM) networks.
Additionally, the study in [17] addressed handwritten text conversion and storage in ASCII
format, covering preprocessing, feature extraction, and classification using deep learning
methods. The research in [18] enhanced HTR performance on lines with crossed-out words, while
the approach in [19] proposed outperformed traditional OCR systems. The integration of HTR
into OCR systems was discussed in [20], alongside outlining an HTR competition focusing on
historical documents [21]. Effective practices in HTR, including basic architectures and datasets
like IAM and RIMES, were described in [22], while the importance of image processing in
handwritten text detection was emphasized in [23], showing potential in forgery protection and
handwriting analysis. Lastly, the Attention-Gated-CNN-BGRU model [
The Attention-Gated-CNN-BGRU model [
For this study, documents of varying readability levels were collected from the State Archives of Khmelnytskyi Oblast. Among the proposed documents, digitized descriptions of ancient records from the following funds were taken: Fund 442, Kamianets-Podilskyi County Treasury for 18611913; Fund 507, Office of the Chief of the South-Western Customs District for 1907-1913; Fund 596, Podilia Branch of Princess Tetiana Mykolaivna's Committee for Providing Temporary Assistance to Victims of Military Actions for 1914-1915; Fund 598, Investigative Court of the 2nd Division of Kamianets-Podilskyi County for 1875-1880; Fund 616, Kamianets-Podilskyi County Military Affairs for 1884-1919; as well as Fund 309, Isakovets Customs for 1931-1915, written in Ukrainian and Russian languages.
The dataset [24] for recognizing ancient handwritten text consists of 75 PNG-format images, examples of which are shown in Figure 1.
This research focuses on analyzing the effectiveness of the Attention-Gated-CNN-BGRU model for recognizing text from historical manuscripts. Modern challenges in OCR in historical documents include a variety of writing styles, document preservation levels, and character variability. This approach involves applying deep learning with attention to the context of characters and their interactions within words or text fragments, which enhances recognition accuracy compared to traditional methods. The implementation was done using the Python programming language. The approach consists of the following stages: Stage 1. Data preprocessing: 1.1. Digital transformation: Each text sample I from historical manuscripts is converted into a digital format through scanning or photography, where I is represented as a matrix of pixels ( , ), where x and y are pixel coordinates. 1.2. Normalization: Applying normalization to each image I to standardize sizes and color intensities. The normalized image can be represented as where is the normalization function. 1.3. Data augmentation: Generating new samples Iaug from Inorm using transformations such as rotation, scaling, shifting, etc., to increase data diversity: ), where is the augmentation function.
Stage 2. The Attention-Gated-CNN-BGRU
model combines CNN for efficient visual feature extraction with gated recurrent units (GRU) and attention mechanism for modeling dependencies between characters. Step-by-step, this is presented as follows: 2.1. CNN: Using CNN to extract visual features ( feature extraction function implemented using CNN.
attention mechanism to model character dependencies and consider context: ) from images, where is the where is the function implementing GRU and attention mechanism.
Stage 3. Model validation: 3.1. Word Error Rate (WER): where S is the number of substitutions, D is the number of deletions, I is the number of insertions, and N is the total number of words in the correct text. 3.2. Character Error Rate (CER): where M is the total number of characters in the correct text. 3.3. Levenshtein CER: Using the Levenshtein distance to calculate CER, where the Levenshtein distance between two strings is the minimum number of single-element edits (insertions, deletions, substitutions) required to transform one string into another.
( (
)),
WER= + + + + 4.1. Performance evaluation: Analyzing the distribution of WER and CER errors to evaluate the model's effectiveness in recognizing text from historical manuscripts. Statistical methods are used to compare the model results with baseline indicators. 4.2. Impact of attention
mechanism: Evaluating the contribution of the attention mechanism to the model's ability to identify complex characters and their contextual relationships through detailed analysis of corrected errors and improvements in recognition accuracy.
Further, this approach is described as an algorithm (Figure 2) for evaluating the effectiveness of the
Attention-Gated-CNN-BGRU model for recognizing text from historical manuscripts, starting with data preparation, which includes data collection, digital transformation, and dataset augmentation from manuscripts. Next, model configuration involves integrating convolutional neural networks and gated recurrent units with attention
mechanism for text analysis. Model validation is done using independent test datasets, and performance evaluation is based on word and character error rates. Results analysis includes comparison with baseline indicators, detailed recognition analysis, and identification of directions for further model improvement.
Within the framework of this study, an expert evaluation methodology was used to assess the effectiveness of OCR models on historical manuscript materials. Five qualified experts analyzed the document samples, evaluating the difficulty of text recognition, the number of correctly recognized characters, the total number of recognized characters, and the total number of characters in each document. The approach proposed by the authors for this research for evaluation consists of the following stages: Stage 1. Data preparation for analysis.
following criteria: difficult. recognized. 1.1. Collection of evaluations from experts ( 1, 2, 3, 4, 5) based on the 1.1.1. Assessment of text recognition difficulty from 1 to 5, where 1 - very easy, 5 - very 1.1.2. Correctly recognized characters: the number of characters that were correctly = ∑ =1 ( ℎ _ 1.1.3. Recognized characters: the total number of recognized characters. 1.1.4. Total characters: the total number of characters in the original document. 1.2. Calculation of indicators: 1.2.1. Average assessment of text recognition difficulty ( ): where n is the number of experts, and Ratingi is the rating from expert i. 1.2.2. Average percentage of correctly recognized symbols ( ): Stage 2. Analysis of the results: 2.1. Analysis of the overall effectiveness of the OCR model:
for the evaluation of the overall effectiveness of the OCR model. 2.1.2. Measuring the dispersion and standard deviation for each indicator helps understand the diversity of expert ratings and the variability of recognition results. 2.2. Impact of text complexity on recognition quality: Using correlation analysis between SORT and SVPRS helps determine how text complexity affects recognition quality. A positive correlation may indicate that as the complexity of the text increases, recognition efficiency decreases. , and for all documents allows Stage 3. Interpretation of the obtained results: 3.1. The overall efficiency of the OCR model is determined through the analysis of the average values of , , and . High values of and with low
indicate the model's ability to adapt to various conditions of handwritten text. 3.2. The results of analyzing the impact of text complexity on recognition quality provide insights into the limitations of the OCR model and directions for further improvement. Enhancing recognition accuracy under conditions of high text complexity may become a key aspect of model optimization.
The algorithm (Figure 3) for the experimental evaluation of OCR models on historical manuscripts begins with collecting expert ratings, where experts analyze text samples from manuscripts based on defined criteria such as text recognition complexity and the number of correctly recognized symbols. The second stage involves analyzing the overall efficiency of the models by calculating average values, dispersion, standard deviation of indicators, and conducting correlation analysis between text complexity and recognition quality. In the final stage, the obtained results are interpreted, allowing for the assessment of the overall efficiency of the models and determining optimization directions to improve recognition accuracy under conditions of high text complexity.
1.2.3. Average number of recognized symbols (ANRS) = _
section 3.1.
In this section, a detailed analysis of the experiment results is presented, allowing for the evaluation of the effectiveness of applying OCR models to historical manuscripts, as described in manuscripts using the OCR model. The data include links to the photos of the originals as well as the text recognized by the model. A comprehensive analysis of the entire dataset and detailed research results are available for review on GitHub [24], where an in-depth investigation of the OCR model's effectiveness on various fragments of historical documents is presented.
Upon completing the initial analysis of the text recognition results, a comprehensive expert evaluation was conducted, as detailed in section 3.3 of the documentation. The expertise involved a thorough examination of text complexity, recognition quality, and symbol identification accuracy, aimed at gaining a deeper understanding of the effectiveness of applying OCR models to historical documents. Prominent experts involved in the analysis included Senior Lecturer of the Department of Information and Socio-Cultural Activities at the Western Ukrainian National University, Volodymyr Yarych, Doctor of Historical Sciences Anatoliy Filinyuk from Ivan Ogienko Kamianets-Podilskyi National University, as well as candidates of Historical Sciences Serhiy Sydoruk and Serhiy Trubchaninov. Additionally, Valentina Filinyuk, a candidate of Philological Sciences and Associate Professor at Khmelnytsky Humanitarian-Pedagogical Academy, contributed to the expert assessment. Their professional approach and deep knowledge facilitated the identification of key aspects for further improvement of OCR technologies in the context of working with historical texts.
Table 2 provides a statistical overview of the collected data, including the mean, variance, and standard deviation for the number of successfully recognized characters, the average complexity rating of texts, and the average accuracy percentage of recognition.
The analysis (Table 2) of the statistical indicators of text recognition results by the OCR model indicates overall effectiveness and challenges associated with processing historical documents. The arithmetic mean of the number of recognized characters is 107.33, with an average complexity rating of text recognition at 2.93, suggesting a moderate level of document complexity. The average percentage of correctly recognized characters is quite high at 71.7%, indicating a reasonably high accuracy of the OCR model. However, significant variance in the number of recognized characters (1908.14) and the average percentage of correctly recognized characters (576.72), as well as the standard deviation for these indicators (43.68 and 24.01, respectively), underscore the variability in recognition quality among different documents. The correlation value of -0.76 indicates a strong negative relationship between the average complexity rating of texts and the average percentage of correctly recognized characters, demonstrating that as the text complexity increases, the recognition efficiency decreases.
The detailed analysis of the data [24] revealed variability in the accuracy of text recognition by the OCR model, which correlates with the complexity rating of the text, ranging from 1.0 to 4.6. Higher SVPRS values, reaching up to 100%, are observed in texts with lower complexity ratings, indicating the high efficiency of the model in recognizing less complex documents. However, a significant decrease in recognition accuracy to 1.74% and below is observed in texts with the highest complexity ratings (4.4 and above), highlighting the limitations of the current OCR model when working with highly complex historical materials. Documents numbered 69-75[24], written in Ukrainian using the Latin alphabet, demonstrated significantly lower recognition accuracy compared to others, as reflected in the average percentage of correctly recognized characters (SVPRS) ranging from 1.74% to 6.99%. Meanwhile, the complexity rating of these texts was the highest among all analyzed documents (4.4 and above), indicating significant challenges that the OCR model faces when working with texts written in the Latin alphabet but in the Ukrainian language. These results underscore the particular challenges associated with recognizing texts in languages using non-standard or less common alphabets and indicate a critical need for the development of specialized approaches and algorithms to enhance OCR accuracy in such conditions.
In conclusion, the analysis of the expert evaluation results and the statistical overview of the collected data regarding the efficiency of the OCR model confirm its significant potential utility in the study of historical texts. However, the high level of negative correlation between text complexity and recognition accuracy emphasizes the importance of further refinement and adaptation of OCR technologies to optimize working with complex historical materials. Special attention to texts written in non-standard alphabets, such as Ukrainian using the Latin alphabet, underscores the necessity for the development of specialized approaches to overcome these unique challenges, paving the way for improving accessibility and preservation of valuable historical heritage.
Within this study, an analysis of the effectiveness of the Attention-Gated-CNN-BGRU model for HTR from historical documents stored in the State Archives of Khmelnytskyi Oblast was conducted, primarily focusing on documents written in Ukrainian and Russian languages. The research concentrated on digitized descriptions of ancient deeds from 1861 to 1913. The utilized model demonstrated an SVPRS of 71.7%, indicating significant efficiency in the context of the complexity of the analyzed documents.
Comparing our results with similar solutions [
One of the main directions for future research is the development and adaptation of the model for effective recognition of texts written in Ukrainian using the Latin alphabet. This aspect is crucial for the preservation and study of Ukraine's cultural heritage, as a considerable number of historical documents of significant importance are written in this alphabet. The results of our study indicate the potential feasibility of effectively adapting existing technologies for recognizing such texts, but also emphasize the need for further developments in this area.
In conclusion, our research not only confirmed the high effectiveness of the Attention-GatedCNN-BGRU model in recognizing handwritten text from historical documents but also identified promising directions for future work. Specifically, the development of models for recognizing texts written in Ukrainian using the Latin alphabet could be a key step in preserving and making important historical resources accessible, enriching our understanding of the past and promoting cultural exchange.
[13] Yatskiv, V., Jun, S., Yatskiv, N., Sachenko, A., & Osolinskiy, O. (2011). Multilevel method of data coding in WSN. У 2011 IEEE 6th International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS). IEEE. https://doi.org/10.1109/idaacs.2011.6072894 [14] Sachenko, A., Osolinskyi, O., Bykovyy, P., Dobrowolski, M., & Kochan, V. (2020). Development of the Flexible Traffic Control System Using the LabView and ThingSpeak. У 2020 IEEE 11th International Conference on Dependable Systems, Services and Technologies (DESSERT).
IEEE. https://doi.org/10.1109/dessert50317.2020.9125036 [15] Aradillas Jaramillo, J. C., Murillo-Fuentes, J. J., & M. Olmos, P. (2018). Boosting Handwriting Text Recognition in Small Databases with Transfer Learning. У 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR). IEEE. https://doi.org/10.1109/icfhr-2018.2018.00081 [16] Balci, Batuhan, Dan Saadati, and Dan Shiferaw. "Handwritten text recognition using deep learning." CS231n: Convolutional Neural Networks for Visual Recognition, Stanford University, Course Project Report, Spring (2017): 752-759. [17] Yintong Wang1,2, Wenjie Xiao2 and Shuo Li2 Journal of Physics: Conference Series, Volume 1848, 2021 4th International Conference on Advanced Algorithms and Control Engineering (ICAACE 2021) January 29-31, 2021, Sanya, China [18] Nisa, H., Thom, J. A., Ciesielski, V., & Tennakoon, R. (2019). A deep learning approach to handwritten text recognition in the presence of struck-out text. У 2019 International Conference on Image and Vision Computing New Zealand (IVCNZ). IEEE. https://doi.org/10.1109/ivcnz48456.2019.8961024 [19] Yang, J., Ren, P., & Kong, X. (2019). Handwriting Text Recognition Based on Faster R-CNN. У 2019 Chinese Automation Congress (CAC). IEEE. https://doi.org/10.1109/cac48633.2019.8997382 [20] Ingle, R. R., Fujii, Y., Deselaers, T., Baccash, J., & Popat, A. C. (2019). A Scalable Handwritten Text Recognition System. У 2019 International Conference on Document Analysis and Recognition (ICDAR). IEEE. https://doi.org/10.1109/icdar.2019.00013 [21] Sanchez, J. A., Romero, V., Toselli, A. H., & Vidal, E. (2016). ICFHR2016 Competition on Handwritten Text Recognition on the READ Dataset. У 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR). IEEE. https://doi.org/10.1109/icfhr.2016.0120 [22] Retsinas, G., Sfikas, G., Gatos, B., & Nikou, C. (2022, May). Best practices for a handwritten text recognition system. In International Workshop on Document Analysis Systems (pp. 247259). Cham: Springer International Publishing. [23] Mishra, P., Pai, P., Patel, M., & Sonkusare, R. (2020). Extraction of Information from Handwriting using Optical Character recognition and Neural Networks. У 2020 4th International Conference on Electronics, Communication and Aerospace Technology (ICECA). IEEE. https://doi.org/10.1109/iceca49313.2020.9297418 [24] GitHub - kniosu/handwriting-text-recognition: Handwriting text recognition. (б. д.). GitHub.
https://github.com/kniosu/handwriting-text-recognition [25] I. Paliy, V. Turchenko, V. Koval, A. Sachenko and G. Markowsky, "Approach to recognition of license plate numbers using neural networks," 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541), Budapest, Hungary, 2004, pp. 2965-2970 vol.4, doi: 10.1109/IJCNN.2004.1381137 [26] V. Lytvyn, V. Vysotska, P. Pukach, Z. Nytrebych, I. Demkiv, A. Senyk, O. Malanchuk. Analysis of the developed quantitative method for automatic attribution of scientific and technical text content written in Ukrainian - Eastern-European Journal of Enterprise Technologies. 2018. - № 6(2). - P. 19-31.