Business corporations manage tons of unstructured data daily, such as PDFs and websites. Recent advances in the deep learning field help find insight from this unstructured information. New models leverage the power of the Transformer architecture to accomplish natural language understanding tasks on these data, jointly using the raw image and its text content or directly the image without OCR. We propose an extraction pipeline that employs question-answering models to get insight from unstructured data, allowing fast and eficient information retrieval from diferent sources. We show an application of this technique to a specific set of documents and how we can scale this infrastructure to diferent types of records. Our solution can efectively handle large document corpora robustly, helping corporations exploit all the power coming from their data.
eficiency of document analysis. In particular, the
Transformers model [
Recent difuser algorithms [
Language models based on transformers are highly
efective for text embedding, such as Sentence Bert [
Also, the NER (Named Entity Recognition) problems
can be covered by Bert [
The T5 [
To extract text from documents OCR (Optical Char- other tasks, we select it for question answering by reach2.1. Scraping
essary to build a dataset from scratch that contains an 1https://www1.agenziaentrate.gov.it/servizi/geopoi_omi/index.php 2https://pvp.giustizia.it/pvp/it/homepage.page The information extraction pipeline is tailored to specific documents, namely businesses’ semi-annual reports. The algorithm starts by adding an OCR layer to the original document thanks to o c r m p y p d f 3, a Python library that takes care of rasterizing, performing OCR, and saving the 3https://github.com/ocrmypdf/OCRmyPDF result back to PDF format. This new document is then Table 1 read in memory using an extended version of c a m e l o t 4, Execution time of diferent OCRs calculated by generating a PDF table extraction utility written in Python. c a m e l o t some random document with 1000 characters. exposes its internal API to handle PDFs manipulation and OCR CPU time (s) GPU time (s) layout reading. This API has been extended to support, in addition to table reading, horizontal and vertical text Tesseract 0.71 extraction. A set of query functions has been added to al- EasyOCR 0.89 0.1 low eficient information retrieval, like extracting text in range, extracting words at a specific location, matching lines with a specific set of words, and extracting text in In the second step of the conversion, the text is split column format. Once a PDF is loaded as raw text in mem- into sentences using a sentence segmentation process ory, the information extraction phase takes place. An provided by a convolutional neural network based on extractor is defined as an atomic information searcher, word2vec and GloVe 5. This step leads to the generawhich is a single class specialized in the retrieval and tion of sentences that will be processed for correct incleaning of chunks of text. A set of ad hoc extractors is formation retrieval. The sentence segmentation process created to perform an exhaustive search of all the infor- defines semantically relevant sentences in the context mation. Since documents might have several formats, an from which the desired information can be retrieved. The extractor for each document type is defined to retrieve next step after sentence splitting is the application of a the same information but with diferent queries. For each mapping algorithm that retrieves the desired information PDF, the whole set of extractors is executed to retrieve from the sentences by searching for relevant words and as much information as possible. Since more than one contemplating the results found by matching regular exextractor might return successfully, a majority voting pressions in a neighbourhood of n words from the matchphase ensures that only one among the valid extractor is ing index. This ensures that part of the irrelevant values used. In the case of a tie, each extractor has a reliability is filtered out by the first extraction layer. Subsequently, score that determines its accuracy, and only the most the relevant values for each sentence are entered into a accurate is chosen. list related to the reference sentence and mapped into a dictionary by the key-value structure, where the keys are represented by the sentences that possibly contain 2.3. Document extraction Deep searched values and as values the list of values contained Creating a scalable model with an algorithmic approach in the sentence. It is noticeable that retrieved values is a challenging task, as documents containing the de- here can present some inconsistencies, as pattern matchsired information often have no pattern or similar regular ing has an inherent limitation related to the language patterns. For this reason, we present an approach that property. From this step, artificial intelligence comes combines a pattern-matching algorithm and an artificial into play, with a further layer consisting of a fine-tuned intelligence algorithm to efectively scale data extraction T5 transformer for question answering and trained for from documents. the Italian language 6. The transformer makes it pos
The algorithm in detail implements a first part, or text sible to search for values in the filtered sentences and extractor, comprising two OCRs and one extractor to then retrieve the data having as input the keys of the retrieve text from documents. This first part of the algo- dictionary mapped in the previous step, guaranteeing a rithm can recognise pdf documents. Where a document second skimming of irrelevant data and checking that the has no scanned text, the PDF Reader (Fig. 1) retrieves the value retrieved from each sentence is consistent with the text. Where the first extractor does not find text one of values retrieved from the text. To further improve the two OCR engines, Tesseract and EasyOCR, comes into results, an adjacency check to remove repeated values play. Both have advantages and disadvantages. Tesser- was implemented on the strings, which discards strings act has about 94% accuracy in identifying numbers, and with the same values if their index is equal to n +1 where 98% accuracy in identifying letters, with an image-to-text n is the index of the sentence under consideration. In conversion, instead EasyOCR has an accuracy of 98% in this way, the algorithm can return the sentences with identifying numbers and 95% in identifying letters, with results filtered on the text with their relative values. In an image-to-text. Another important diference is that the case of documents concerning the real estate market, EasyOCR can use GPU; so we decide to use Tesseract the need to calculate the total square footage for each (that has more accuracy on most of the text) when we run property led to the integration of a final layer that checks scripts on CPU, while we use EasyOCR on GPU devices by means of regular expressions and pattern matching to speedup the works 1. whether the values in the text report a total quantity, 4https://camelot-py.readthedocs.io/en/master/ 5https://spacy.io/models/it#it_core_news_lg 6https://huggingface.co/it5/it5-base-question-answering calculating the value if this is not returned by the first check. The algorithm is being improved by implementing an AI Named Entity Recognition module and a second transformer-based module that exploits sentence similarity logic on the tokenization of words. The idea would allow considerable improvements in terms of accuracy and completeness in handling the information sought without having to resort to pattern matching and regular expressions layers.
ments. We run the pure algorithmically extraction phase on the entire dataset. The machine used for the experiments is equipped with 20 CPUs at 3.50GHz, 128GB of RAM, running Ubuntu 7 with kernel 4 . 4 . 0 - 1 7 0 - g e n e r i c . Table 2 shows the experimental results. Apart from 542 unprocessable documents (they are encrypted, or they are not PDFs), we successfully parsed 17 929 PDFs, extracting all the relevant parts. The remaining 41 468 documents turned out to be invalid relations or relations with an unrecognized format. A manual investigation shows that less than 5% are false negatives. As a side note, the experiment required about 7 days of full computation, taking into account the overhead introduces by storing the OCR ifles and the extracted information on an external cloud provider.
With the purely algorithmic approach, we just reach to extract any information from data from documents with high precision, but it does not scale quickly to others document formats. The deep proposed approach is the ifrst idea to scale document extraction information to all documents and start to fill the gap between Deep Neural models and this research field. 7https://ubuntu.com/
if accepted.X