In this paper we present a Multilingual Ontology-Driven framework for Text Classi cation (MOoD-TC). This framework is highly modular and can be customized to create applications based on Multilingual Natural Language Processing for classifying domain-dependent contents. In order to show the potential of MOoD-TC, we present a case study in the e-Health domain.
The large amount of digital data made available in the last years from a wide
variety of sources raises the need for automatic methods to extract meaningful
information from them. The extracted information is precious for many purposes,
and especially for commercial ones. Jackson and Moulinier [
The problem of classifying multilingual pieces of text was addressed since the
end of the last millennium [
The use of ontologies to classify multilingual texts [
In this paper we present MOoD-TC (Multilingual Ontology Driven Text
Classi er [
The paper is organized as follows: Section 2 introduces three motivating scenarios where an ontology-driven multilingual text classi cation may prove useful, Section 3 analyzes the state of the art, Section 4 describes MOoD-TC, Section 5 provides examples and experimental results, and Section 6 concludes. 2
Alice is enjoying her holidays in Stockholm. Suddenly, she feels a painful spasm to her stomach and in a few minutes a strong feeling of nausea appears. Spasms go on for half an hour, and she starts to feel worried. She does not think it is the case to go to the hospital, but she would at least ask for advice over the phone. However, she cannot speak Swedish and, in the stressful situation she is experiencing, she cannot recall how to express her health problems in English. She could speak in her native language Italian, but it is not so likely that the doctor can speak Italian as well.
Bob is making a walk in his town. He notices a young man bending over his knees, with a scared expression on his face. He runs to help him, and he understands that the problem is with his chest. The young man speaks French only and Bob cannot understand him: he calls the rst aid emergency number and explains what he is seeing and what he supposes to be taking place. If he could understand what the young man says, he would be de nitely more helpful.
Carol is a volunteer in Honduras. She is neither a physician nor a nurse. She has a very basic knowledge of rst aid procedures and a rst aid kit with medicines that she knows how to administer, given a clear diagnosis. A woman runs towards her asking for her assistance. The woman's small boy has a problem with his head and he has a high fever but, without understanding the other symptoms that the woman is trying to explain in Spanish, Carol cannot recognize and classify the problem. In the remote place where she is, she cannot contact the doctor. Carol should need to understand the other symptoms besides fever and headache, in order to select the correct medicine.
The three scenarios above are all characterized by the impossibility for the doctor to visit the patient on-the- y and the need for the patient to be understood despite language barriers, in order to get advice for minor problems or to speed up the assistance procedure for major ones. The patient's need could be suitably addressed by identifying and translating symptoms from her language to the assistant's or the doctor's one. If automatic tools for facing this issue were available, for example as an app installed on the mobile phone, the three situations could evolve in the following way: { Scenario 1: through the use of an app, the person needing care communicates with the \health emergency" software application in her own language. The application performs a speech-to-text translation, identi es the symptoms in the text based on a standard ontological representation of symptoms, and sends the list of symptoms expressed in the doctor's language to a center where they are managed either by intelligent software agents or by human experts. { Scenario 2: the \health emergency" software application is not directly used by the person needing care, but by the one who assists her. Like before, the assisted person can \tell" her problems to the application which performs a speech-to-text translation and identi es the symptoms represented in a domain ontology which appear in the text. The symptoms, translated into the language of the person who his giving the rst assistance, may be read on the screen. That person can call the national rst aid number, telling what is happening, what she sees, and the symptoms which have been understood, classi ed, and translated by the app. { Scenario 3: also in this case, besides a speech-to-text translation, the symptoms expressed in the language of the patient are identi ed w.r.t. a symptoms ontology and translated into the target language. The way this information is used can require a further automatic processing stage, if the doctor cannot be involved in the loop and the person providing aid needs an automatic support for making a diagnosis and identifying the right therapy to administer.
In all the three situations above, a standard machine translation application and a symptoms classi er based on machine learning might not be powerful enough: the pre- and post-processing stages require to have a machine-readable explicit representation of symptoms, in some vocabulary agreed upon by all the application components and by the humans involved in the loop, in order to share them among the application components (both at the client and at the server side) and to reason about them if needed. A multilingual ontology-driven text classi cation approach seems the right way to face these challenging scenarios. 3
According to [
In this paper we are interested in Natural Language Processing (NLP)
techniques for solving multilingual term identi cation and text classi cation
problems in the e-Health domain where extracting information from clinical notes
has been the focus of a growing body of research in the past years [
Signs and symptoms have seldom been studied for themselves in the eld
of biomedical information extraction. Indeed, they are often included in more
general categories such as \clinical concepts" [
Another recent project in e-Health NLP context is the IBM Watson for Oncology1. It has an advanced ability to analyze the meaning and context of structured and unstructured data in clinical notes and reports, easily assimilating key patient information written in plain English that may be critical to select a treatment pathway. These works are di erent from ours because they do not address multilingual aspects and, furthermore, because they have to manage the di erences between the \signs", which are identi ed by clinicians, and the \symptoms", which can be described directly by the sick person.
In our work we do not have to manage clinical records but directly the information provided by the person who feels sick. This di erence is crucial in works using an ontology-driven approach, because clinical reports contain many more technical words2 compared to a text written (or a sentence told) by a normal person describing how she feels. This allows us to use simpler ontologies. Especially from the multilingual viewpoint, having an ontology containing simple concepts, omitting useless technicalities, allows achieving better results with less e ort, considering that a technical word could be less supported by the tools we use during our text classi cation pipeline.
The assumption upon which MOoD-TC relies, is the availability of
ontologies in the domain of interest. Even if the application developer might design and
implement her own domain ontology from scratch, integrating well-founded and
widely used ontologies into MOoD-TC would be the most modular, reusable
and scienti cally acceptable approach. Luckily, many domain ontologies exist, in
particular in the biomedical domain. Panacea [
MOoD-TC has been developed as part of Silvio Beux' Masters Thesis [
The Language Detector Library detects, with a precision greater than 99%, 53 languages making use of Naive Bayesian lters. It is devoted to recognize the language Lo of the ontology o and the language Ld of the textual document d. The TreeTagger tool performs the tagging of d in order to obtain, for each word w 2 d di erent from a stop word, its lemma (the canonical form of the word) and its part of speech (POS). This information is used by BabelNet to perform the translation of w into the ontology language. Finally, the translated word w0 is searched inside the ontology and contributes to the classi cation of d in the category modeled by the ontology concept c having the same semantics as w0. The Classi erObject is the object that stores a correctly classi ed word (and additional information) of the document d with respect to o. TC returns a list of such objects. ADM specializes the text classi er task by implementing functionalities for pre- and post- processing a multilingual textual document. If an ADM is used, the entire system specializes its behaviour in the domain represented by that particular ADM (e.g., from text classi er to disease recognizer). In our system TC can work alone, but an ADM is meant to work in close connection with the core system. The core modules are implemented to work for the European languages (which share some common features like, for example, the relationship between noun and adjective), but they could be extended to cope with the peculiar features of other languages; in fact, thanks to the modularity of the system, it is possible to integrate di erent algorithms created speci cally to handle that peculiarities, without modifying the entire system. The ADM processes the TC input and output in order to obtain a new domain oriented tool. An ADM is composed by two sub-components: pre-processing and postprocessing. The pre-processing component takes as input a digital object (for
example a spoken sentence, in the scenarios discussed in Section 2) and returns a new processed text, while the post-processing component takes as input the TC output and returns a domain dependent result. Figure 1 shows the entire pipeline of the integration process between the TC and the ADM. 5
As illustrated in Section 2, the scenarios we aim to address require that disease symptoms appearing in a text are correctly identi ed w.r.t. a domain ontology. The pre-processing stage consists of moving from a spoken sentence to a text and the post-processing in translating the identi ed symptoms into a target language and, depending on the scenario, moving back from text to speech and/or reasoning over them. In the sequel we discuss the experiments related with our main task, namely that of symptoms identi cation.
The domain ontology used for describing symptoms is a standard ontology named the symptoms ontology 7, partially shown in Figure 2. It is an ontology of disease symptoms with symptoms encompassing perceived changes in function, sensations or appearance reported by a patient and indicative of a disease. We stress that our experiments in exploiting MOoD-TC for symptom identi cation did not require to build any new ontology. Rather, consistently with the good principle of reusing existing software whenever available and, in particular, reusing existing ontologies, we just passed the symptoms ontology as input to the TC, obtaining the results discussed in the next section.
In the sequel we discuss our initial experiments with phrases in ve di erent languages (English, French, German, Italian, Spanish), where symptoms are
identi ed by the TC module. The classi cation of two sample sentences is shown below, where the TC GUI screenshot associated with each sentence shows the ontology concepts which appear in the text along with the number of their occurrences in the text.
Phrase 1 (Italian language): \Credo di avere la febbre, continuo a sudare e ho i brividi. Non la smetto di tossire e fatico a mangiare a causa del male alla gola, come un forte bruciore. Mi sento stanchissimo e ho dolore a tutti i muscoli." Phrase 3 (Spanish language): \Me siento fatal. Tengo temperatura, vomito y diarrea. Hace dos d as que no consigo comer nada. Tengo nausea y mareos."
The experiments have been carried out on 32 sentences for each of the 5
languages, for a total of 160 sentences. Each sentence describes symptoms
related to one of the following sixteen disease: tinnitus, food allergy, cervical,
dehydration, hyperthyroidism, u, appendicitis, food poisoning, labyrinthitis,
narcolessia, pneumonia, diabetes type 1, hyperglycemia, hypoglycemia,
bronchitis, jet lag (two sentences for each disease). To cover the widest range of cases
we considered the diseases with the most varied symptoms. The description of
symptoms associated with each disease has been retrieved from [
Since the nal purpose of this work is to provide an automatic diagnostic system with as many symptoms as possible, in order to devise the correct diagnosis, we were mainly interested in symptoms which appear in the text but which are not identi ed by our classi er (false negatives). We also looked for false positives, but their number is so low to be irrelevant for our experiments. Also, false positives are due to an under classi cation, rather than an actually wrong classi cation: if the text contains the \abdominal cramp" symptom, for example, and it is classi ed with the more general \abdominal symptom" concept, we consider this result a false positive as a more speci c concept could have been returned. Figure 3 shows the average number of symptoms that should have been identi ed w.r.t the correctly identi ed symptoms in the ve considered languages. Figure 4 shows the number of false negatives (y axis) for disease (x axis). Figure 3 demonstrates that the results greatly vary with the disease. For example, symptoms related to tinnitus are hardly classi ed, but this can be easily explained by observing the ontology we used, where problems related to ears are not modeled at all. By carefully analyzing the obtained results, we also realized that sometimes the performances of the classi er are worsened by the presence of a symptom in the text which has a di erent grammatical role than the symptom in the ontology (usually a noun), making their matching impossible although the word root and the meaning are the same. For example, the ontology contains the noun \irritability", but if the text contains the adjective \irritable" (in any language), the identi cation fails. This problem is due to the way the root of a word is computed, and to the way words are managed in BabelNet.
What emerges from Figure 4 is that false negatives have a very similar behavior despite the language of the sentence. This is again due to the two reasons discussed above. Despite these problems, which have a clearly understood motivation and which can be addressed by extending the ontology and by re ning the management of word root extraction, MOoD-TC has demonstrated to be a exible and ready-to-use approach for multilingual symptoms identi cation driven by a standard ontology we retrieved on the web. 6
In this paper we presented the MOoD-TC architecture showing its possible use
in the symptoms identi cation problem. The speech-to-text pre-processing stage
can be faced using existing tools, and the post-processing stage with a translation
of the identi ed symptoms into the doctor's language can be addressed using
BabelNet, in the same way we exploit BabelNet for bridging the text, whatever its
language, and the ontology. The more challenging post-processing stage of
supporting the user in providing a diagnosis given a set of identi ed symptoms could
be addressed by means of sophisticated expert system such as the old and well
known MYCIN [
Our framework does not face many well known open problems in multilingual
text classi cation and information extraction such as negation [
In the short time, our work will be devoted to overcome the problems that limit the performances of MOoD-TC in the considered scenario: we will make the word identi cation more exible and we will extend the symptoms ontology with those symptoms which have not been modeled so far.
In the future, it would be interesting to run an experimental comparison between our approach and a machine learning one. In case of a limited number of labeled examples, in fact, it would be feasible to apply semi-supervised learning methods. Depending on the comparison results, we will also consider to combine both approaches, using a domain ontology to improve the results of a traditional machine learning approach.