Hospital systems routinely assign disease codes (ICD10 codes) to medical records. The challenge stands on treating natural and nonstandard language in which doctors express their diagnoses and, additionally, to solve a large-scale classi cation problem, as there are thousands of possible codes. In this working notes paper, we present our system and the results of the CLEF 2018 eHealth Evaluation Task 1 on Multilingual Information Extraction - ICD10 coding. This benchmark addresses information extraction in written text with focus on several languages, speci cally Hungarian, Italian and French. The goal is to automatically assign ICD10 codes to diagnostic terms of death certi cates. The problem can be cast in di erent ways, for example as a multilabel classi cation task or as sequence-to-sequence prediction. Our proposal follows this last approach, with promising results, well above the average results for the task. It only relies on the material provided by the task organizers, allowing the application of the same system to all datasets.
The aim of this paper is to explore computer aided approaches to classify Medical
Records following the World Health Organization`s International Classi cation
of Diseases (ICD). These records are written in di erent languages and the
CLEF 2018 eHealth Evaluation Task 1 on Multilingual Information Extraction
consists of assigning the right ICD10 coding [
However, the encoding of diagnoses with ICD codes is a di cult, time consuming and expensive task for health services. These records are written in a non-standard medical language causing problems for retrieving and exchanging information due to elements such as misspellings or colloquial and speci c language. This lack of standardization also poses a challenge for the automatic classi cation process due to: { Acronyms: the adoption of non standard contractions for the word-forms. { Abbreviations. { Omissions: often prepositions, articles or verbs are omitted in an attempt to write the word-form quickly. { Synonyms: some technical words are typically replaced by others. { Misspells: sometimes words are incorrectly written.
The IxaMed group has approached the automatic ICD10 coding for French, Italian and Hungarian with a neural model that tries to map the input text snippets with the output ICD10 codes. Our solution does not make assumptions about the content of the input and output data, treating them by means of a machine learning approach that assigns a set of labels to any input line. The solution is language-independent, in the sense that treating a new language only needs a set of (input, output) examples, making no use of language-speci c information apart from terminological resources such as ICD10 dictionaries, when available. 2
Computer aided classi cation of medical records can be seen as a pattern recognition task, as the aim is to recognize unknown instances of expressions and assign them one or more elements from a set of possible labels. This problem has been approached in several tasks and challenges using di erent techniques.
The 2007 Computational Medicine Challenge [
Perez et al. [
Perez et al. [11] tackle diagnostic term normalization employing Weighted
Finite-State Transducers (WFSTs) that learn how to translate sequences into
standard representations given a set of samples. They are highly exible and
easily adaptable to terminological singularities of di erent hospitals and
practitioners. They also implemented a similarity metric to enhance
spontaneousstandard term matching. Looking at their results, they found that only 7.71%
of the diagnostics were written in their standard form matching the ICD. This
WFST-based system enabled matching spontaneous ICD codes with a Mean
Reciprocal Rank of 0.68, which means that, on average, the right ICD code for
each diagnosis is found between the rst and second position among the
normalized set of candidates. Similarly, Almagro et al. [
CLEF eHealth 2017 Task 1 is a similar challenge but multilingual since the
texts were both English and French, more extensive because it was not limited to
an speci c service and employed ICD-10-CM for coding instead of ICD-9-CM. [
Other systems presented at the CLEF 2017 shared task made use of varied
approaches. In [
To summarize, we can say that the problem presents a complex characterization due to multiple factors, like non-standard language variation, spontaneous writing, or large-scale multilabel classi cation. Accordingly, there are plenty and varied approaches to tackle it, ranging from knowledge-based solutions to statistical and deep learning ones. 3 3.1
In the present challenge [
The sets of document-ICD10 codes come in two di erent formats: raw and aligned, though the aligned version is only available for French. For the raw version, the diagnostic terms as expressed in the original death certi cate are stored in one le (CausesBrutes, see Table 1) separately from the coding which is stored in another one (CausesCalculees, see Table 2). The link between them can be carried out through indexing information common to both: document identi er, year of the death certi cate, and line number within the death certi cate representing the exact location in the text. As previously stated, diagnostic terms in the CausesBrutes les appear as originally expressed in the death certi cates and therefore they show orthographic misspellings (infacrtus vs. infarctus ) and abbreviations (HTA vs. hypertension artrielle, see Table 1).
It is important to notice that a one-to-one correspondence between the raw diagnostic term and the ICD is not assured. Missmatches occur like the ones shown in document 100569, where line 5 in Table 1 has no correspondence in Table 2. The correspondence appears in line 6. It might happen to nd more than one diagnostic term in one line separated by commas, and coordination by means of complementizers or prepositions.
In the aligned version, the original text is accompanied with the standard text and the ICD associated (see Table 3). 3.2
Preprocessing. With the aim of boosting the ICD assignation, we preprocessed the raw corpora to organize the information at three levels: document level, line level and nally ICD level. At the document level and line level, we grouped all diagnostic terms and ICD codes by document and by line respectively, hoping that the system could capture dependencies among the di erent ICD codes. It seems logic to think that ICD codes within a document or within a line are related to each other and, if so, ensemble recognition might be helpful. The preprocess to obtain the line level information consisted mostly in trying to overcome the alignment mistakes in the original corpus as shown in Tables 1 and 2. At the ICD level, we treated separately each (diagnostic term ICD) pair aiming to simplify the assignment process but at the cost of missing any interrelation that could exist. This level required a more re ned preprocessing since the original information was set at the line level. Remember that certain lines showed several diagnostic terms.
As a rst step in normalization, the input texts were preprocessed in the following order: tokenization, lowercasing and substitution of accents. These are standard operations in sequence-to-sequence learning, that help to improve the results.
ICD10 coding. In neural sequence-to-sequence modeling, the encoder-decoder model has been used to encode a variable-length input sequence of tokens into a sequence of vector representations, and to then decode those representations into a sequence of output tokens, in this case ICD10 codes.
This decoding is conditioned on information from both the latent input vector encodings as well as its own continually updated internal state, motivating the idea that the model should be able to capture meanings and interactions beyond those at the word level [12, 13].
The supplied data was divided in three subsets. A training set was iteratively evaluated on a second hold-out evaluation set and, nally, the best performing system was evaluated on an independent third set. For the nal submission, the training and hold-out sets were merged, using the third subset for iterative evaluation, and applying the best system on the unseen test set. 4
the diagnostic codes which is kept at the line level but cannot be assumed at document level. This work tackles the classi cation of medical records following the ICD10 standard. The classi cation problem is tough for several reasons: 1) the gap between spontaneous written language and standard one; and 2) it is a large-scale classi cation system, being the number of possible classes the number of di erent diseases within the ICD10 catalogue.
Our best system showed high-quality results, and this fact opens a promising avenue for the task of automatically assigning ICD10 codes to medical documents. Moreover, the method is language independent and it allows e cient training, given only a set of annotated documents, not requiring complex feature engineering. 6
This work has been partially funded by:
We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan X Pascal GPU used for this research. 11. Perez, A., Atutxa, A., Casillas, A., Gojenola, K., Sellart, A.: Inferred joint multigram models for medical term normalization according to ICD. International Journal of Medical Informatics, 110, pp. 111{117 2018. 12. Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to Sequence Learning with Neural Networks. Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems (NIPS), pp. 3104{3112 2014. 13. Cho, K., van Merrienboer, B., Gulcehre, G., Bougares, F., Schwenk, H., Bengio, Y.: Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. Conference on Empirical Methods in Natural Language Processing (EMNLP 2014) 2014.