Companies of any size and industry still struggle in automatic business processes where human cognitive and contextualization capabilities are required to read and understand complex documents. Ongoing progress in the fields of Computer Vision and Natural Language Processing, where (large) language models are becoming increasingly and freely available, have made possible to create a new generation of Intelligent Document Processing technologies that allow automatically analyzing and understanding both documents layout and contents. In this paper we present an Intelligent Document Processing platform that makes use of hybrid AI techniques to allow document reading comprehension by means of a combination of Document Layout Analysis and recognition, table recognition and detection, context free grammars, and question answering techniques. Such a technology combines also no-code principles with high performance computing based on micro-services to streamline the execution of tasks such as document and text classification, document segmentation, entity extraction, sentiment analysis, question answering, and more.
2 presents the state of the art of technologies on which Intelligent Document Processing workflows are built; approach for layout analysis, using deep learning techsection 3 describes the Altilia Intelligent Automation niques to detect various components in a document, such platform; finally, section 4 concludes the paper. as headings, paragraphs, tables, and images. By incorporating both textual and spatial features, LayoutLM is capable of understanding the structure and seman2. Related work tics of a document more efectively, thereby improving the performance of various NLP tasks. LayoutML has Recent advances in AI and machine learning, particu- demonstrated its eficacy in handling complex document larly in the fields of Natural Language Processing (NLP) layouts and multi-column documents, which are often and Computer Vision, have led to significant improve- challenging for traditional layout analysis methods. Anments in document processing techniques. The past few other notable work in Document Layout Analysis is DiT, years have witnessed the development of various tech- a self-supervised pre-trained Document Image Transniques and architectures that address diferent aspects former model for Document AI tasks[5]. DiT serves as of document processing, extracting and understanding the backbone network for various vision-based Docuinformation from diverse document types. Techniques ment AI tasks, including document image classification, include Optical Character Recognition (OCR) for text DLA, and table detection. The model is trained on one of extraction, layout analysis for structural understanding, the largest datasets for Document Layout Analysis by[6], and syntactic and semantic retrieval for relevant infor- the PubLayNet dataset. mation extraction.
Optical Character Recognition (OCR) is a fundamental cant role in document processing, as they are responsible
technology in document processing, as it enables the con- for identifying and extracting relevant information from
version of scanned images or printed text into machine- unstructured or semi-structured documents. A popular
readable and editable text format. OCR techniques have approach for information extraction from documents is
improved significantly over the last few years, thanks to the Retriever-Reader[7] pipeline. The Retriever-Reader
the advancements in deep learning and Computer Vision. pipeline is a common approach for addressing document
Recent works such as DBNet[1] and RobustScanner[2] processing tasks, this pipeline consists of two main
comhave contributed significantly to improving the accu- ponents: a retriever, responsible for selecting relevant
racy and eficiency of OCR techniques. The first work documents or passages from a large corpus based on
proposes a novel approach for text detection in natural the given query, and a reader, which extracts specific
scenes that uses diferentiable binarization and a tailored information from the retrieved documents.
loss function. This approach has demonstrated high ac- Syntactic keyword retrieval and neural retrieval are
escuracy and real-time performance in detecting text from sential techniques for eficiently searching and retrieving
complex backgrounds and challenging conditions, such relevant information from large document collections.
as low resolution or distorted images. The second intro- Syntactic keyword retrieval methods, such as BM25[8],
duces a method to dynamically enhance positional clues rely on term frequency and document frequency to
idenfor robust text recognition. By combining visual and tex- tify relevant documents. These methods are often
efectual features. Both works have achieved state-of-the-art tive in situations where documents contain structured
results on several benchmark datasets and demonstrated information, such as tables, where keywords play a
crurobustness to challenging scenarios such as low resolu- cial role in the retrieval process. On the other hand,
tion, significant distortion, occlusion and varying text neural retrieval methods, such as Dense Retriever[9],
styles and sizes. These recent advancements in OCR tech- leverage dense embeddings to capture semantic
relationnology, along with the end-to-end Transformer-based ships between queries and documents. These methods
OCR approach proposed by TrOCR[
Document Layout Analysis (DLA) is another critical retrieval algorithms enable information extraction from aspect of document processing, focusing on identifying vast collections of documents by filtering relevant ones the structure and organization of text and visual elements for further analysis. Combining OCR and layout analywithin a document. Layout analysis techniques aim to sis with the Retriever-Reader pipeline can significantly understand the structure and organization of a docu- enhance the eficiency of document processing systems. ment, which is crucial for accurate information extrac- For instance, OCR technology can be used to convert tion. Recently, LayoutLM[4] has emerged as a promising scanned images into text, which can then be processed by retrieval algorithms to identify relevant elements of the layout. Subsequently, reader models can be applied to extract information from the selected documents.
focuses on extracting relevant information from the retrieved documents. Some reading techniques include token classification, text classification, and question answering. These task are typically addressed leveraging Transformer architectures[10] which have enabled the development of powerful reader models. These models, such as BERT[11], RoBERTa[12], and GPT[13], have been widely adopted in these tasks due to their strong performance in capturing the semantic meaning of text, achieving state-of-the-art performance in a wide range of Natural Language Processing tasks, demonstrating their suitability for document processing. Readers can also be implemented by traditional rule-based approaches, such as regular expressions, wich rely on hand-crafted rules and patterns to extract information. However, these methods are limited in their ability to scale and adapt to new document types and variations.
address all document processing needs of users: Teach, Automate, Understand. Figure 1 shows a snap of the platform, in particular the Altilia Labels tool (deeply presented in section 3.1.1). On the left of the screen the user can find labels containers, a way to logically group
Token classification techniques have been used to iden- the data that needs to be annotated and then extracted. tify specific types of information within a document, such On the right there are annotations filtered by page, the as dates, amounts, or named entities. These techniques user is leveraging the auto annotation functionality (in leverage pre-trained language models, such as BERT, to order to speedup the data labeling process), a feature ifne-tune and adapt them to specific tasks and domains. that suggests passages to annotate. Finally, in the cenCommon token classification tasks include named en- ter of the screen documents are shown along performed tity recognition (NER), as discussed in the paper by[14]. annotations.
NER automatically scans entire articles to extract and classify fundamental entities in a text into predefined categories, such as organizations, quantities, monetary 3.1. Teach values, person names, and locations.
ness domain experts to transfer their knowledge into AI
Text classification techniques have been employed to models. Such tools enable users to create sophisticated AI categorize documents or sections of documents based on skills for reading and understanding complex documents, their content. These techniques typically involve the use starting from a set of pre-trained ML models than can of pre-trained language models and transfer learning to be fine-tuned for the specific use case application. More adapt the model to specific classification tasks[ 15]. Text in detail, tools in the Teach module allow annotating classifiers, using NLP, automatically analyze text and as- documents (Altilia Labels), training and fine-tuning ML sign pre-defined tags or categories based on its content. models (Altilia Models), and creating complex AI skills Common examples and use cases for automatic text clas- (Altilia Skills). sification include sentiment analysis, topic detection, and language detection.
Question-Answering (QA) techniques[16] have been applied to extract specific information from a document in response to a given question. Recent advancements in language models, such as BERT, have enabled the development of highly accurate question-answering models that can understand and extract relevant information from complex documents.
ness domain experts to conduct point-and-click document annotations as shown in figure1. It enables the creation of annotated datasets that can be used as examples to train AI models for numerous Natural Language Processing (NLP) and Computer Vision tasks. This tool is powered by active learning and auto-labeling algo- can be used for data and concepts identification within rithms to reduce annotation workloads and speed up both standard form-like documents and complex docAI model deployment. A key feature of this tool is the uments with mixed layouts (including framed and ununiversal document ingestion capability, i.e. users can framed tables, text, charts, columns, and backgrounds). seamlessly ingest documents of any type and format. It provides multi-language support and has the capability 3.2. Automate to recognize complex document layouts and structures (e.g., tables, lists, multi-column documents, charts, and The Automate module of AIA allows the integration of images). Another important feature is that it enables multiple AI skills and models with RPA capabilities to semantic annotation which is used to enhance concepts deploy Intelligent Document Processing workflows. This and attributes recognition by leveraging sophisticated module also allows setting up connectors to integrate knowledge representation mechanisms and techniques, the workflows with external systems and applications it also uses full-text and neural information retrieval to across the enterprise, and provides users with a review facilitate the recognition of document portions to anno- and validation tool, to give feedback and enhance the tate and provides optimized annotation techniques for ML models’ accuracy and performance. The Automate diferent machine learning tasks. module provides three main tools to design and build process automation workflows, to connect workflows 3.1.2. Altilia Models – AI Models Training and with external systems and tools, and most importantly to Fine-Tuning enable users to review and validate data extracted by AI skills. This last tool concretely enables the adaptive AI Altilia Models gives access to a library of out-of-the-box paradigm implemented in AIA because it allows the platpre-trained models and algorithms that can be fine-tuned form to leverage user feedback to re-train AI algorithms with the customer’s datasets (based on real-world doc- improving workflows eficiency and accuracy over time. uments) according to the specific business use case and workflow requirements. Additionally, customers can im- 3.2.1. Altilia Workflows – Process Automation port custom AI models and train them with their datasets.
Altilia Models provides important features like robust By the Altilia Workflows tool users can easily create transfer learning, connection with Hugging Face1 repos- and design automated workflows that streamline comitory, that is the largest library of pre-trained state-of- plex, document-intensive processes that normally require the-art models, to easily find models to fine-tuning on significant human understanding and intervention. AI custom datasets to solve specific tasks. Another key fea- skills enable to process, analyze, and understand complex ture is constant tracking of behavior and performance documents with human-level accuracy, no matter what of models by validating on a rich set of metrics, it pro- format or layout. This tool provides workflow templates vides the way to visually track the training, validation, that can be customized by a few clicks. Users can build and testing process of a model. Continual learning and workflows by combining diferent tasks available in the few-shot learning are other features used to visually re- platform. Users can configure input/output connectors, view labels predicted by models and use active learning execution behavior, scheduling, accuracy thresholds, and to improve model. Altilia Models provides support for workflow SLA, to enable execution strategies that exactly over 50 languages. match specific business process needs. Workflow can be executed in attended and unattended mode.
Processing information from complex documents, regardless of format and layout, requires the use of highly sophisticated AI capabilities. This module allows combining multiple AI models, data functions, and knowledgegraph capabilities to build sophisticated AI skills, capable of recognizing and understanding all the key contents and components contained within documents. It provides a way to create advanced document processing skills, which can combine multiple AI models, predefined data manipulation functions, search queries, rules-based hybrid AI functions, and custom algorithms. Altilia Skills
The-Loop (HITL) AI mechanism, that allows reviewing and validating workoflw results. The Human-In-TheLoop AI cycle is one of the main enabler of the adaptive AI method implemented in the platform. AI algorithm predictions can be validated manually or by specific validation rules, such feedback is used to re-train and enhance AI models. In particular, human validation and feedback are essential to enable the continuous monitoring and training of AI models to enhance models capabilities, improve data quality and accuracy. More in detail, this tool provides a point-and-click interface to review semantic document indexing and data conceptualizaand validate data, metadata, and concepts obtained from tion to support intelligent decision-making. It allows the execution of workflows. It uses semantic faceted an object/concept-first modeling approach that combine search, queries in natural language, and knowledge rep- symbolic AI with machine learning to better address resentation functions to validate the most sophisticated training of AI models, automation of document-driven data, metadata, objects, and concepts recognized in doc- processes, and empowerment of decision-making. uments. This way this module provides also explainable AI features supporting trustworthy AI applications.
Altilia Knowledge Base to simplify the access, search, To execute an Intelligent Document Processing automa- and filtering of all the available information. The Altilia tion workflow, it is necessary to gather and exchange data Searches tool constitutes a user-friendly neural search between internal and external sources. AIA platform pro- engine equipped by powerful document filtering mechvides pre-built input and output connectors that make anisms to easily locate the exact information users are it possible to connect and integrate workflows with all looking for. The tool makes use of an advanced semantic external applications and systems, including RPA, CMS, search engine that leverages both full text and neural ERP, CRM systems, and even custom-built tools. It pro- search to help users find and retrieve documents that vides support for documents and contents ingestion from contain specific terms and/or concepts. It supports multiany internal or external source by using predefined tem- faceted searches, to all types of documents, that combine plate connectors to FTP folders, local or remote folders, content queries with filters applied to data, entities, and REST API, RPA, ERP, CRM systems, and any other doc- objects produced by the machine learning algorithms. ument source. Some special connectors allow applying For example, users can formulate queries to search a speweb wrapping and scraping techniques to gather docu- cific concept within tables. The tool also provides a way ments and page contents of entire (deep) web sites. These to search by similarity, and to answer to natural language type of connectors are particularly useful in ESG data col- questions. In this case, users write a question or insert a lection applications where documents and contents must piece of text in the search bar and get accurate answers be gathered from company websites, online newspapers, or a list of relevant documents. social media, online document sources, aggregators, etc.
tion (AIA), a platform that ofers a promising solution for businesses and organizations seeking to streamline their All the documents and data that the platform intakes and document processing workflows. With its comprehengenerates are indexed, stored, and handled by a knowl- sive suite of tools, AIA caters to all document processing edge base grounded on a multi-structured attribute graph needs of users, providing a one-stop-shop for businesses that enables multi-faceted semantic searches, querying, looking to automate their processes. The Teach module’s and reporting. The Altilia Knowledge Graph simplifies ability to create sophisticated AI models for complex 3.3. Understand
Document Processing technology depends on the ability to analyze and understand the data contained within processed documents. The Understand module of the AIA platform allows for the extraction of meaningful information and insights from all the given document data, empowering intelligent decision-making. This module is composed of three main tools meant, to create an internal representation of the data and metadata extracted from documents, to enable keyword and semantic queries, and to synthesise information into reports and dashboards to facilitate informed decision-making.
The Altilia Insights module is a conversational decision intelligence tool that makes it possible to understand all the data and metadata available within the Altilia Knowledge Graph, thus giving a complete overview of all contents of the processed documents, to drive smarter, more eficient, and better-informed decision making. We can create multi-dimensional reports by using natural language queries. You write a query in natural language and the system creates a dashboard with multiple charts describing all aspects of the questions known in the knowledge base. It allows exporting charts and reports on your desktop, and sharing dashboards with coworkers. 4. Conclusions document understanding, without requiring extensive tion for Computational Linguistics, 2017, pp. 1870– coding knowledge, makes AIA accessible to a wide range 1879. URL: https://doi.org/10.18653/v1/P17-1171. of users. The Automate module’s intuitive visual work- doi:10.18653/v1/P17-1171. lfow editor and wide range of connectors simplify the [8] S. Robertson, H. Zaragoza, The probabilistic releautomation process, enabling users to automate their vance framework: Bm25 and beyond (2009). workflows with ease. Lastly, the Understand module’s [9] V. Karpukhin, B. Oğuz, S. Min, P. Lewis, L. Wu, semantic search and knowledge graph capabilities make S. Edunov, D. Chen, W. tau Yih, Dense passage reretrieving documents and insights from large datasets trieval for open-domain question answering, 2020. fast and straightforward. Overall, the AIA platform ofers arXiv:2004.04906. a powerful combination of ease of use, automation, and [10] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkorintelligence, making it an excellent choice for businesses eit, L. Jones, A. N. Gomez, L. Kaiser, I. Pololooking to boost eficiency and productivity. sukhin, Attention is all you need, CoRR
The AIA platform behavior will be further improved, abs/1706.03762 (2017). URL: http://arxiv.org/abs/ enhancing the Document Layout Analysis and recogni- 1706.03762. arXiv:1706.03762. tion algorithms. In the future, we will extend the Platform [11] J. Devlin, M. Chang, K. Lee, K. Toutanova, capabilities towards a complete machine reading com- BERT: pre-training of deep bidirectional transprehension system providing full QA features, where we formers for language understanding, CoRR can extract the information in a question-answer manner abs/1810.04805 (2018). URL: http://arxiv.org/abs/ from documents. 1810.04805. arXiv:1810.04805. [12] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen,
O. Levy, M. Lewis, L. Zettlemoyer, V. Stoyanov,
References Roberta: A robustly optimized bert pretraining
ap[1] M. Liao, Z. Wan, C. Yao, K. Chen, X. Bai, Real-time [13] pAr.oRaacdhf,o2rd01,K9.. aNraXraisvim:1h9a0n7,T..1S1a6li9m2a.ns, I. Sutskever,
scene text detection with diferentiable binarization, Improving language understanding by generative
2019. arXiv:1911.08947. pre-training (2018).
[2] X. Yue, Z. Kuang, C. Lin, H. Sun, W. Zhang, [14] R. Hanslo, Deep learning transformer architecture
Robustscanner: Dynamically enhancing posi- for named entity recognition on low resourced
lantional clues for robust text recognition, 2020. guages: State of the art results, 2021. URL: https:
[
C. Zhang, Z. Li, F. Wei, Trocr: Transformer-based [15] 2K1o1w1s.ar0i0,J8.3M0e.imandi, Heidarysafa, Mendu, Barnes, optical character recognition with pre-trained mod- Brown, Text classification algorithms: A survey els, 2022. arXiv:2109.10282. (2019). URL: https://doi.org/10.33902Finfo10040150. [4] Y. Xu, M. Li, L. Cui, S. Huang, F. Wei, M. Zhou,
LayoutLM: Pre-training of text and layout for doc- [16] dPo.Ri:a1j0p.u3rk3a9r0,J/.iZnhfaon1g,0K0.4L0o1p5y0re.v, P. Liang, SQuAD: ument image understanding, in: Proceedings 100,000+ questions for machine comprehension of the 26th ACM SIGKDD International Confer- of text, in: Proceedings of the 2016 Conference ence on Knowledge Discovery & Data Mining, on Empirical Methods in Natural Language ProACM, 2020. URL: https://doi.org/10.11452F3394486. cessing, Association for Computational Linguis3403172. doi:10.1145/3394486.3403172. tics, Austin, Texas, 2016, pp. 2383–2392. URL: https: [5] J. Li, Y. Xu, T. Lv, L. Cui, C. Zhang, F. Wei, Dit: //aclanthology.org/D16-1264. doi:10.18653/v1/ Self-supervised pre-training for document image transformer, 2022. URL: https://arxiv.org/abs/2203. D16-1264.
02378. doi:10.48550/ARXIV.2203.02378. [6] X. Zhong, J. Tang, A. J. Yepes, Publaynet: largest dataset ever for document layout analysis, 2019. URL: https://arxiv.org/abs/1908.07836. doi:10.
48550/ARXIV.1908.07836. [7] D. Chen, A. Fisch, J. Weston, A. Bordes, Reading wikipedia to answer open-domain questions, in: R. Barzilay, M. Kan (Eds.), Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, Associa