- Before physicians prescribe medicines, they must take into consideration the patient's diseases and medicines they use. This is done to avoid complications that may occur. All information about possible contraindications is written in the medicine package leaflet. A system that can automatically detect contraindication mention in the Lithuanian text of leaflet applying natural language parsing is presented. This system gives a possibility to shorten the time needed for medicines prescription decision making. The results of the experiment showed that the created system successfully detected 56 per cent contraindications.
When a patient is diagnosed with a new disease,
additionally physician asks the patient about his allergies,
previous health problems, chronic deceases, what medications
and food supplements he is using. After taking gathered
information into consideration and evaluation of possible
contraindications with prescribed medication physician
assigns treatment and, if needed, changes previous
assignments. Almost all information about contraindications
can be found in the medicine package leaflet. According to
Lithuania’s medicines registration procedure [
Automatic information extraction tools can extract
biomedical data, save it in a structural way, and minimize
information search problem. However, automatic text analysis
and information extraction from unstructured text in the
medical domain is a challenging task [
© 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0)
A system that automates the extraction of contraindications from leaflet text is described is in Section 3. Using this system all leaflets of medicines registered in Lithuania were analyzed. The results of this analysis (contraindications extracted) are used in a commercial medications information system that is used by Lithuanian physicians for prescription of medications. The evaluation of the obtained results is presented in Section 4.
In Lithuania, it is established that each medicine registered in Lithuania must contain a package leaflet describing therapeutic indications, possible contraindications, safety precautions, and usage information in the Lithuanian language. In order to be sure that the patient does not suffer from possible contraindication, the physician should read through all leaflet text before prescribing the medicine. Usually, the analysis of leaflets is time-consuming, so physicians tend to skip it and rely on the knowledge and experience they have gained.
There are lots of systems developed for analysis and
information extraction from the biomedical text in the English
language. But there is no solution for the detection of
contraindication (i.e. contraindication with disease or
contraindication with the pharmacological group) mentions in
Lithuanian written text. We have analyzed articles that
describe similar problems when analyzing biomedical text.
For example, a tool Semantator [
Although all these authors work on texts written in
English, we found that common rules and approaches could
be applied to Lithuanian texts as well. In order to extract
information from text, preprocessing is needed using natural
language processing: text segmentation, a morphological
analysis should be performed and then a syntactic parse tree
or the dependency graph [
In this section, a system for the detection of contraindication mentions in the medicine leaflet text written in Lithuanian is presented. The system implements a text analysis pipeline of four analysis stages: extraction of contraindication text block, morphological analysis, noun phrase detection, and annotation.
Additionally, all annotated phrases are checked is it in the database of noun phrases to be ignored or not. This database is manually filled and helps to obtain more precise results. The overall pipeline for the detection of contraindication mentions is shown in fig. 1.
Below each stage of text analysis is discussed in more detail.
In Lithuania, when describing the medicine, a producer
must follow a certain template of the package leaflet [
The information which, the patient should be aware of before he or she takes the medicine, is presented in section number two. An example of this section is shown in fig. 2 with highlighted contraindications phrases. So, the first task of our system is to find this section and extract its text for further analysis.
A morphological analysis forms a background for
information extraction about contraindications. In this stage, a
given text is split into lexical units (e.g. sentences, lexemes)
and analyzed morphologically. For this task, a web service
provided by the system “http://semantika.lt” [
Phrases that express a specific contraindication usually are
noun phrases, for example, heart attack, type one diabetes,
pancreatitis, and so on. Therefore, we chose a phrase structure
grammar method because it better fits for noun phrase
detection than dependency grammar as it was suggested by
Axel Halvoet in his monography [
No 1 2 3
An algorithm implemented for the noun phrase detection checks every lexeme in the sentence for the satisfaction of conditions of at least one rule presents in Table 2. If the condition is satisfied a lexeme is included in the noun phrase. The workflow of analysis of the noun phrase Lėtinis reumatinis perikarditas (Chronic rheumatic pericarditis) is shown in Table 3.
When the construction of the noun phrase is complete the
form of the head noun in the phrase is changed to its canonical
form (lemma). This is done because the name of item
registered in the International Classification of Diseases (ICD)
[
All noun phrases identified in the previous stage are reviewed and checked for contraindication. If a contraindication is identified, the phrase is annotated. For annotation three databases are used: ICD, ATC and the lists of active substances. The algorithm compares the noun phrase and name of the item from the database. If the noun phrase matches the name in ICD the phrase is tagged as contraindication with the disease. If the phrase matches the ATC item name, it is tagged as contraindication with a pharmaceutical chemical group, and if the phrase matches the name of the active substance, it is tagged as contraindication with an active substance.
It is worthy to mention that before comparison of the noun phrases all identified phrases are checked against phrases in the database of noun phrases to be ignored. In the text of medicine package leaflet, a lot of words (i.e. illness, hand and so on) that are irrelevant (do not express a contraindication) but are used in ICD, ATC and active substances lists could be found. The database of noun phrases to be ignored was filled manually with the help of a professional pharmacist.
The aim of the experiment is to evaluate the created system and check if a tool can achieve its target - to give physicians the possibility of a faster and more accurate way of finding contraindications. The experiment was done by manually annotating contraindications mentions in the package leaflet text block and comparing results with the system’s results. This was done by a professional pharmacist who works in JSC Skaitos kompiuterių servisas.
The experiment was organized as follows. From medicines database ten randomly selected leaflets were analyzed using the system created. The results of the analysis were automatically gathered into the table, which example is presented in Table 4 In the first column the code of item automatically found in the text of leaflet by the system is indicated. The second column represents the database (ATC, ICD or active substances) where the item is registered. The third column was used for the evaluation of annotation correctness.
The same randomly selected leaflets were analyzed and annotated manually, and the table of the same structure was filled in with manual annotation results. Manually found contraindications were not interpreted or changed to synonyms. For example, heart attack and myocardial infarction are the same diseases. But ICD contains only one name of this disease - myocardial infarction. The created system is not able to recognize the heart attack as a synonym of myocardial infarction.
Additionally, the active substances, mentioned in the leaflet, were translated into the Latin language (nominative and genitive grammatical cases). This was done because the database of active substances, that was provided, has three versions of translation: Lithuanian, Latin in the nominative case and Latin in the genitive case.
The results of the evaluation are presented in Table 5. The precision, recall and F-Score metrics have been calculated for each leaflet analyzed. Additionally, the ratio between the number of correctly detected contraindications and overall automatically detected contraindications was calculated as well. This metric allows to evaluate how accurate the results are and to use them in further calculations.
Results showed that the system developed is able to correctly detect 56% of relevant contraindications. The average number of links detected automatically is 1482.8 while manually detected links are 197.9. The number of links detected automatically in one leaflet is average four times higher, than detected manually. The average number of erroneous links to ICD is 72%, to ATC - 90%, and links to the list of active substances - 61%.
Calculations show that the system is able to achieve 0.25(±0.23) precision, 0.56(±0.32) recall, and 0.31(±0.19) Fscore value. To give a better perspective where the system’s failures were and possible reasons for that, Pearson correlation coefficient calculations between various indicators were done (Table 6). The biggest impact on F-Score had incorrectly detected links to ICD, a coefficient was -0.89. The reason why precision was so low is that of the high ratio between automatically and manually detected links.
Auto. correctly detected links
Man. detected links 100% 82% 81% 17% 98% 77% 46% 45% 87% 90% 82% 54% 89% 72% 27% 17% 100% 100% 100% 100% 100% 25% 100% 88% 100% 100% 87% 100% 91% 100% 90% 23% 25% 100% 65% 58% 100% 24% 21% 43% 49% 100% 100% 51% 55% 45% 91% 61% 30% 21% 100%
The experiment shows that the system automatically successfully detected more than half of the relevant contraindication links (56%). But 75% of links were erroneous and the system lacks precision. The reason for that is a high number of incorrect links to ICD (r=-0.9655), this indicator has the most negative impact on the precision and FScore results. This might be because of commonly used phrases that are not contraindications but used in the ICD list. For example, the word allergy does not imply that this is a contraindication and must be ignored. Another reason for low estimates results is, the number of detected contraindications phrases. Calculations show, that the higher is the difference between automatically and manually detected contraindications phrases, the lower are precision and F-Score results. The reason for that is, a high number of noun phrases that are irrelevant to contraindications noun phrases, for example, pill, driving.
Additionally, considering why F-Score is so low (0.31) the assumption that this is because of low precision (0.25) can be done. To raise this indicator the list of phrases to be ignored (common word and phrases) must be used. The most frequent reasons for the incorrect detection of contraindications are: the context of the phrase in the sentence is not taken into account; Conjunctions are not taken into account and two or more noun phrases (i.e. “…kidney and liver diseases…”) are not identified; Brackets that are used to specify contraindication are not taken into account (“…liver tumor (malignant or benign)…”).
To avoid errors caused by those reasons, users of “https://gydytojams.vaistai.lt” IS will be able to mark contraindication as erroneous and if the pharmacist approves that it will be removed from the database.
In this paper, the system which automatically detects contraindications and links them to existing “Skaitos kompiuterių servisas” databases have been introduced. System analyses text of medications leaflets, it extracts noun phrases and links them to corresponding items in ATC, ICD, and active substances list. The system presented was used for the extraction of contraindications from leaflets of all medications registered in Lithuania. Extracted data was used in the pilot project for extending the functionality of the system “https://gydytojams.vaistai.lt”. The additional function supports physicians in search of possible contraindications that are relevant to patient medical records. Moreover, physicians have the possibility to give feedback about erroneous contraindications presented. In such a way they help in expanding the list of phrases to be ignored and eliminating incorrect contraindication links.
The experiment shows that approximately 56% of contraindications are found but only every fourth is correct. Several changes in the algorithm remain for future work. First, before the noun phrase is looked up in databases, a context must be identified. This would reduce the number of incorrect links. Second, to detect phrases that refer to medication analyzed and to ignore them.
kompiuterių servisas was provided by JSC Skaitos