In recent times, it was made a great investment in transfer from physical ancient Portuguese texts to digital support. This support transfer allows not only the access to the texts, bringing them to the public in general, but also the possibility of texts to be readable and processed by machines. NLP tools are addressed, mainly, to contemporary Portuguese and the application of NLP to classic texts has several difficulties. The elaboration of big lexical corpora of forms previous to modern Portuguese is an opportunity for multidisciplinary field of studies allowing the enlargement of linguistic studies and also the possibility of obtaining, by NLP, validated corpora, collections and ontologies, that can be input in NLP tools for ancient Portuguese texts. In this work we will present, briefly, the problem of lexical variation of forms in processing classic Portuguese texts, the challenges that emerge from them and future perspectives of work.
The aim of this paper is to demonstrate the problem of lexical variation in classic period of Portuguese in Natural Language Processing (NLP) of ancient Portuguese texts. Therefore, the main purpose of this work is to present a preliminary essay of a major work in systematization of lexical variants, orthographical, typographical, historical or semantic, and proposal of equivalence/grouping of forms. The typology of lexical variation will enable a more accurate processing of these texts, as there are no known NLP tools able to process any classic Portuguese texts fully automatically. 2
Some challenges to NLP in classic Portuguese texts
Lately, it has been done a great effort to transfer textual documents booked in Patrimonial Libraries to a digital support, bringing them from restricted access bookshelves and provide them to public in general. With this investment and the progress in Natural Language studies, the access to the “inside” of these texts is a very interesting field of study for Digital Humanities.
Digital Humanities (DH) is an area of confluence of Computer Sciences and
Language Sciences
Some tools, the VARD2 and the TICCL, have been adapted and used to
Portuguese in ancient texts, ported to the Portuguese, discussed by
Recently, the CARDS-FLY project
The classic linguistic period of Portuguese is very rich in variation. It has
remarkable lexical renovation, with the entry of many new terms in Portuguese language
Concerning Portuguese orthography of classic period, as there was not still a standardization, the variation is very expressive, for example, in nasal diphthongs, in the vocalic or consonantal use of <j> and <v>, in pseudo-etymological spelling or in the use of double consonants, sometimes in a very “creative” registration of Portuguese language.
Also, the printing press process at that time produced a great variety of different spelling, as printers, limited by the availability of typography types or the lack of linguistic criteria, use freely allographs, increasing the variation of Portuguese forms.
The description and registration of the lexical variants of this period of the Portuguese language is fundamental for tasks of pre-processing and post-processing of texts in this period. 3
An example of lexical variation in classic Portuguese in dictionaries (classic period)
The lexical variation in a text is noticed by a human reader but, in an automatic processing, the variants are treated as autonomous forms when, in fact, they are related, linked by the history of language. The example of the forms giolho, geolho, joelho, and juelho, all used during the classic period of Portuguese, present, clearly, a good example of this linguistic variation. The search of each of these forms in Corpus Lexicográfico do Português produces interesting results, showing the lexicographical and textual registration of forms across time.
The form giolho is used by Jerónimo Cardoso (1569-70), Bento Pereira (1697), Bluteau (1712-1728) and Madureira Feijó (1734) and others.
However, geolho is only registered by Bento Pereira, in the Latin-Portuguese dictionary Prosodia (1697). The lexical form joelho is not registered in this Corpus by Jerónimo Cardoso and, in Bento Pereira, it is only register in Prosodia. This form is register in all the subsequent dictionaries:
Dictionarium Tesouro Prosodia giolho geolho joelho 5 --1 -1 -5 23
Vocabulario 2 -120
Parvum lexicon --15
Bluteau (1712- 1728) and Folqman (1755) dictionaries also register the form juelho and they are the only ones in this Corpus that register this variation. The forms giolho, geolho and juelho are no longer active in contemporary language and they don’t have actual lexicographical registration. 4
Linguistic variation in historical online corpora
The search in historical corpora is made with queries for each word separately. In Corpus do Português the forms giolho and joelho are bothe registered. The variants geolho and juelho, registered in dictionaries, are not mentioned in this corpus.
Concerning the corpus Tesouro do léxico Patrimonial Galego e Português, although it is not an historical corpus for the Portuguese, gives information about dialectal variants. The query for one of the forms gives back also all the equivalent forms and their location. The variants giolho, joelhe, joelho are found in this corpus. 5
Some considerations and future perspectives
The NLP of lexical variants in Classic Portuguese must assume, in pre-processing and/or post-processing of texts, among other steps, an exhaustive lexical description of variants made from dictionaries and a verification of other possible variants by use included in texts of the classic Portuguese. The registration of lexical forms in dictionaries is much more enlarged than the occurrences available in texts in actual online corpora. Probably, this is due to the fact that some corpora choose representative texts, and, in consequence, they may not have enough volume of different lexical forms. The probability of existence of a particular word in a corpus may depend much more of the amount and variety of texts that constitute the corpus than the probable existence of that lexical variant, itself. For that, only the combination of these two steps can contribute for a validation of lexical forms from the use in texts.
Before the great variance of Portuguese classic forms, the need of having historical vocabularies made with strong linguistic criteria is essential so vocabularies and ontologies that will be used in NLP of ancient texts may respond to the demands of processing and parsing.
Also, facing this range of variation of historic forms during time, when lexical variants appear in some texts in certain dates but are no longer used in other texts from posterior dates, the automatic recognition of variant lexical forms in a text may also validate the authenticity and probable date of documents.
We pretend to collect a lexical classic Portuguese corpus, from texts of that period, obtained automatically or semi automatically, using Optical Character Recognition (OCR). It will be processed and annotated, and it will allow a validated study of lexical variants in context, not only orthographical. This corpus will also help to a better optimization in character capture, supported by the results obtained.
To enlarge our lexicon in pre-processing tasks, we collected a list of 46 000 words
from the linguistic corpus of the Latin-Portuguese-Latin dictionary Prosodia, from its
7th edition of 1697, one of the largest corpora from bilingual dictionaries in classic
Portuguese. We transcribed the complete text to a Word document ans, in format .txt,
we processed it with AntConc© tool, using conventional information to separate Latin
from Portuguese. This corpus, that was not lemmatized, is representative of Classic
Portuguese
Although the application of NLP to classic Portuguese texts may need an initial effort in order to have validated vocabularies and ontologies, in future, this investment may provide very interesting results allowing automatic access to the inside of texts and all that may result of that.