English. We propose a system to extract entities and relations from a set of clinical records in Italian based on two preceding works (Alicante et al., 2016b) and (Alicante et al., 2016a). This approach does not require annotated data and is based on existing domain lexical resources and unsupervised machine learning techniques.
The digitization of medical documents in hospitals has produced plenty of information which should be adequately organized. While part of the material, mainly including international scientific publications, is in English, increasingly more material is being created in the language of the country of the medical institution. The main part of the local language material is represented by patient records. They contain important information not only for preparing care plans or solve problems for the particular patient, but also to extract statistics useful for research and also for logistics administration.
Automatic processing of such repositories still
can not be straightforwardly applied. One of the
principal issues to be solved is the automatic
extraction of relevant information, usually consisting
in entities and relations connecting them
General tools, such as TextPro
Therefore, ad hoc solutions represent the only
way to build effective applications to solve this
kind of problems. For example, not only domain
entities and relations can help identifying
potentially dangerous information, but also ontological
information can be exploited to better protect
patient privacy
Efforts to port systems to languages different from English require, first of all, the development of lexical resources for the considered language. However, they are not sufficient, because of the intrinsic differences between languages. A widely adopted way to tackle such difficulties is represented by machine learning approaches.
Although supervised approaches are usually more effective, they require large corpora of annotated data, which are quite expensive to obtain, as they require that domain experts invest time in a long and tedious annotation activity. In the medical domain, staff should invest part of their precious time to annotate data with information about the presence and the type of domain relevant entities and relations in records to be used for the training phase. Things would be much easier if domain experts are only required to check an automatically produced annotation. We therefore propose to integrate a knowledge-based and a text mining approaches to develop an application which requires the expert intervention only to check on medical and pharmaceutical labels associated to groups of relations.
More in detail, we propose here to integrate
the systems discussed in
The paper is organized with Section 2 detailing the approach implementation and Section 3 for conclusions and future works. 2
The framework proposed is composed by three modules, and its logical structure is depicted in Figure 1. The first one is devoted to domain entity (i.e., medical and pharmaceutical entities) identification and classification, and exploits domain related lexical resources and standard natural language tools. The second one is based on an unsupervised machine learning approach, namely clustering, to avoid the necessity of annotating data, for the relation extraction. A potential relation is hypothesized among all pairs of the entities identified in the preceding phase. Clustering is then applied to group similar entity pairs. Small clusters indicate the lack of repetitive patterns and will therefore be considered as entity pairs which are not in relation to each other, while larger clusters are likely to correspond to different relation types.
Relations are clustered and labeled using the
approach proposed in
The third module of the framework is based on
Word Embeddings (WEs)
For each entity, we then consider the
embeddings corresponding to each token. As shown
in
2https://en.wikipedia.org/wiki/ Special:Export
3medialab.di.unipi.it/Project/ SemaWiki/Tools/WikiExtractor.py *#"-%,'(&3%$#&$( 4"5&-$#) !"-',50 #)+$,-.$(
*(%&%2)*+%,$-%&'( !"#$%&!'()* ')8!#733$( 9$::7&-.$(
97;$00-,3!
6$<-=70!>,&-&--$2
"?698!5!'@A*
entity and input this representation into a k-means
clustering
The text, processed by our system, is extracted from anonymized medical records, in the form of plain text encoded in UTF-8. The text includes a small set of special characters, used as delimiters and/or formatters. The largest part of these medical records has been produced by an HL7compatible information system. At the end of each medical record, there is often an ICD9M (International Standard for Encoding and Classifying Diseases) disease code, which we disregard together with the rest of the structured part of the records.
The text is initially preprocessed for extracting textual parts from the medical records, and to get rid of non-textual characters. The plain text, produced by this preprocessing step, is passed to the natural language processing suite TextPro to perform tokenization, sentence splitting, PoS tagging and lemmatization. 2.2
Entity extraction is crucial for our analysis, and a specific module has been implemented with the goal of extracting entities which are relevant for the application domain: biomedical and pharmaceutical entities in our case. The module follows a pattern matching approach by identifying each occurrence of a number of PoS patterns in the input text as a candidate to be further analysed.
Afterwards, for each token occurring in the identified pattern, we search for matches of the corresponding lemma in the dictionaries. In case of multi-word expressions, when several patterns apply to overlapping strings of tokens, we apply a greedy approach by choosing the longest one matching the input.
The output is produced following the TextPro format, that is a line for each token, and a column for each analysis level. In our system these files are enriched by the information about Medical and Pharmaceutical entities obtained from the dictionaries provided by UMLS4 and PRB5. These information are labeled as MED for the medical entities, and FAR for the pharmaceutical ones (the whole entity tag list is shown in the Table 1).
In addition to a label indicating whether the entity is medical (MED) or pharmaceutical (FAR), we also add to each medical entity annotation the sub-categories included in the UMLS database in correspondence to the dictionary entry. The list of sub-categories labels are summarized in Table 1. A side-effect of such sub-categorization is that the number of potential relations increases while it becomes possible to find more specific relations.
4Unified Medical Language System, http://www. nlm.nih.gov/research/umls
5Pharmaceutical Reference Book, officially mantained by Agenzia Italiana del Farmaco We apply the k-means approach that identifies groups of relations of the same type appearing in the data set. Each pair of entities occurring in the same sentence identifies a potential relation, therefore all possible entity pairs must be considered. We then apply a clustering algorithm to the set of all the potential relations identified. We will disregard all entity pairs belonging to clusters having a size smaller than a given threshold.
We then concentrate on the remaining entity
pairs, which are likely to represent actual relations
and semantically cluster them. The approach
proposed for this is structured in three main modules:
Feature Construction, Clustering, and Cluster
Labeling. The first module builds a feature vector
based on WEs for each relation candidate; for
doing this, first it constructs a WE dictionary by
using a large collection of unannotated texts, in our
case extracted from Wikipedia. This module is
based on word2vec6
After that, the k-means clustering is applied to the set of feature vectors obtained by the first module. For every entity pair we then construct a Feature Vector (FV) starting from the WE of each word involved. Each entity can be composed by one or more words, as for example conati di vomito: in this case, for each entity, we take the average among the WEs of the words composing the entity associated to the entity pair. Finally, we concatenate the FVs of the two entities, obtaining a FV of 1; 000 entries.
The clustering algorithm is then applied to the
FV data set by means of the C Clustering
library
Eventually, to label each cluster we ordered the pairs in each cluster according to its cosine simi6The software is freely available at https://code. google.com/p/word2vec/ larity from the cluster centroid: the first four pairs are then chosen to characterize the cluster.
As discussed above, each FV can be partitioned in two parts: the first half corresponds to the first entity in the pair, the second one to the other. Such partition is consistently maintained during the whole processing. Also in the computation of centroids in the k-means clustering algorithm, the former half of each centroid derives from the average of the former half of the involved FVs and then corresponds to the first entity. Correspondingly, the latter half of each centroid vector only depends on the second entity of each involved pair.
The choice of the cluster to which a given item is assigned is based on the cosine similarity. Its computation can be divided in three parts: the dot product of the part of the two FVs corresponding to the first entity, the same for the second entity and eventually the normalization with respect to the whole FV. Therefore, the evaluation of the cosine similarity is based on a trade-off between how similar are the first and the second entities in each pair. In other words, they represent actual entities pairs which are similar to the (abstract) cluster representative, corresponding to the centroid. 3
In this paper we presented a system for the extraction of information from clinical records in Italian. A first part of the system aims to extract domain relevant entities from medical reports by a pattern matching approach. A second part takes the output of the former step and applies a clustering approach to explore possible relations between such entities. A third part is based on WE and aims to give cues about the type of the relations.
Interestingly, the approach does not require annotated data, but only easily available data such as Wikipedia and off-the-shelf tools in addition to the documents to process. Naturally, available tools have been trained on annotated data, but without any adaptation to the specific domain. It would therefore be interesting to port it to a new language, possibly different from English, which represents the most widely studied among all languages.
The research presented in this paper was partially supported by the national projects CHIS - Cultural Heritage Information System (PON), and BIG4H - Big Data Analytics for E-Health Applications (POR).