The Notarial Archives in Valletta houses a collection of Latin Notarial deeds that has not been exploited yet. In this paper, Machine Learning techniques are proposed and implemented to extract entities such as people, place names, dates, deed types and keywords from these historical texts. Both supervised and unsupervised techniques are considered and compared with baseline models. Experimental results on a subset of these documents are already showing results that outperform the baselines for Latin text such as those from CLTK. Evaluation was carried out using indexes of four published Notarial Registers.
Archives around the world are a source of hidden information. One of these
archival gems is found at the Notarial Archives in Valletta, Malta and houses
around 20,000 notarial deeds dating back to the 13th century. It is typically the
case that archives publish high quality images and metadata of the structure
of historical documents, but the content itself is not exposed in a meaningful
way to aid historians. In literature, some researchers [
Most of the notarial deeds found in the Valletta archives fall under categories
such as wills, dowry and transfer of land. Although some notaries used to write
the deed type, this was not a requirement. Thus, text classi cation can be used
to maximize the use of the remaining words to predict the presented deed type.
Automatic keyphrase extraction can be used to express a document as a set of
keywords/keyphrases[
In our research we use four Latin notarial transcribed registers entitled
'Documentary Sources of Maltese History' and compiled by Professor Stanley
Fiorini[
In the rest of the paper, we discuss some related work in Section 2 and present the adopted methodology in the following section. In Section 3 we present some initial evaluation which is followed by some future work. 2
Latin poses a great challenge for Named Entity Recognition (NER). Annotated
corpora that can be used for training are scarce and most of them focus on
classical texts. Conditional Random Fields (CRF) have been used successfully for
training Latin models as in Aguilar et al. [
Both supervised and unsupervised models for Keyphrase detection and
extraction have been used successfully[
We used the indexes found in [
Extracting Information from Medieval Notarial deeds copies. Furthermore, there is no index available for dates. A dictionary of all possible mentioned entities was compiled using the Ratcli -Obershelp distance algorithm3 to annotate the text with the relevant tag to be used for evaluation.
Dates are presented using indictions for the year and thus we had to work
out the indiction cycle of each act. Notaries tend to use shorthand when writing
dates such as eodem (same date as before) and penultimo (day before the last).
For this reason a rule based entity extraction was implemented to convert the
dates to modern dates. Extraction of person and place entities was performed
using a trained CRF model based on su xes, uppercase, title, digit, POS, lemma,
next and previous words. The POS and lemma tags were derived using Schmidt's
treetagger4 using parameters for Latin to improve accuracy. We used the Fiorini's
register [
Deeds can have a variety of categories and these were generalized for text classi cation. In total there are 981 deeds and 79 di erent categories with an average of 12 deeds per category. Some of the categories include only one deed, making the training set highly imbalanced. Di erent feature vectors were tried out including count vectors, word level TF-IDF, n-grams and character level vectors. Di erent models were trained for deed classi cation using Naive Bayes, Linear Classi er, SVM and Random Forest. The model with the highest accuracy was saved.
The index of keyphrases was used to annotate the corpus for keyphrase
extraction. Lemmas were used for comparisons as Latin often uses declensions. The
index was merged with the deed text using exact, lemma and stem matches. A
list of annotated/non-annotated words was kept for each deed to be used for
evaluation. Generic unsupervised approaches were used for keyphrase extraction
including TextRank, RAKE and TF-IDF, however these yielded unsatisfactory
results with RAKE giving the best results as shown in Table 1. We then used
a variant of the supervised approach presented by [
The results achieved for the extracted entities are already very promising. Both NER and keyphrase extraction were done using 100 acts (indexes of other registers are yet to be used for annotations) with a 70%-30% split (results in Table 1).
Since we used a domain speci c corpus, supervised models gave better results. The dataset was highly imbalanced and the text classi cation was performed on the whole corpus of 981 records with a 75%-25% split. A Linear classi er was used that leveraged on CLTK's stopword list and the count vector features giving an accuracy of 72%. The removal of stopwords improved slightly the achieved results across all trained models. Purpose Method Precision Recall F1 score NER People/Places CLTK 0.339 0.113 0.17 NER People/Places Spacy with multilingual model 0.414 0.922 0.572 NER People/Places Stanford NER with Spanish model 0.152 0.947 0.263 NER People/Places Conditional Random Fields Model 0.956 0.957 0.956
RAKE with CLTK and
Purpose Method Keyphrase extraction Conditional Random Fields
4
In this paper, we presented our initial research on extracting entities and keyphrases from historical Latin texts. The results are very encouraging even though the datasets are fairly small. We plan to digitize the indexes of the other registers in Fiorini's collection so that we can train the models with more data. We will furthermore be using the extracted information to create a knowledge graph for the Notarial Archives.
O-KEY 0.982 0.15 F1 score B-KEY I-KEY 0.751 0.465