Machine learning to detect ICD10 codes in causes of death

Abstract

In this paper we present our rst participation as SINAI research group from the Universidad de Jaen at Task 1 "Multilingual Information Extraction - ICD10 coding". Our main goal is make a system based on Natural Language Processing (NLP) techniques to detect International Classi cation Diseases (ICD10) codes using di erent machine learning algorithms. First, we nd all the possibles ICD10 codes mentioned in the text. Next, we calculate several measures of quality of these codes. With these metrics we trained di erent machine learning algorithms and we choose the best model to use in our system. Most of the techniques used are independent of the language, therefore our system are easily adaptable to other languages.

Keywords

• Natural Language Processing
• Named Entity Recognition
• Text Classi cation
• ICD10
• French
• Biomedical Text

-

The goal of the task Multilingual Information Extraction [ 11 ] is to automatically assign International Classi cation Diseases (ICD10) codes to the text content of death certi cates. This task can be treated as a named entity recognition and normalization task, but also as a text classi cation task. This builds upon the 2016 [ 13 ] and 2017 [ 12 ] tasks which already addressed the analysis of French biomedical text.

This task focuses on the Named Entity Recognition (NER), one of the basic problems of text mining [ 2 ]. Therefore, information extraction and Natural Language Processing (NLP) techniques will be developed to manage the information contained in a medical record [ 8 ]. There are many investigations related to the detection of medical entities [ 7,10,16 ]. Tools such as Uni ed Modeling Language MetaMap Transfer (UMLS MMTx) [ 14 ] is a con gurable tool commonly used by system developers in biomedicine and Text Analysis and Knowledge Extraction System (cTAKES)1 [ 15 ]. However, both tools are implemented to be used in English language collection. There are few automatic tools for identifying concepts in languages such as French, Hungarian or Italian.

1 ctakes.apache.org

This group has a large experience participating in several tasks of other editions of CLEF task related to the medical domain [ 3,6,4,5 ] and by participating in this task we try to expand our horizons.

We have focused on a part of the task in French and have worked with the aligned data and dictionaries provided by the organization.

This paper is organized as follows: In the next section, we introduce the material provided by the organizers. Our method is described in section 3. In the section 4 showing the experiments performed and we are done with the conclusions and future work. 2

French aligned dataset

The CLEF e-Health 2018 Task 1 CepiDC Gold Standard Training data [ 9 ] composed of the following death certi cates: { Aligned causes 2006-2012: 65.843 death certi cates and associated gold standard ICD10 codes { Aligned causes 2013: 27.850 death certi cates { Aligned causes 2014: 31.690 death certi cates

The les provided are in CSV format and contain one line for each death certi cate. Each line of the le is structured as shown in the table 1 below.

Methodology

In this section, we present the di erent strategies that we have followed in our participation in CLEF eHealth 2018 Task 1: Multilingual Information Extraction ICD10 coding.

We have created a tool to recognize ICD10 codes stored in dictionaries. This tool performs a Natural Language Processing (NLP) on the input text and tries to match with the dictionary terms. 3.1

System description

For both the input text and the dictionary, the system performs the following steps: 1. Normalize: to unify the form of terms we used unicode canonical decomposition (NFD)[ 1 ]. 2. Tokenize: the tokenizer used for this task is WhitespaceTokenizer from the Python Natural Language Toolkit (NLTK)2 Library. The whitespace tokenizer breaks text into terms whenever it encounters a whitespace character. 3. Stemmer: the algorithm used to make the stemmer in French is Snowball.

Snowball3 is a small string processing language designed for creating stemming algorithms for use in Information Retrieval.

The system returns a list of relevant ICD10 codes with a value between 0 and 1 corresponding with the percentage of success between the term detected within the text entered, the gure 1 shows an example of a system output. 100000;Demence type Alzheimer a un stade severe;F03; demence severe;1 100000;Demence type Alzheimer a un stade severe;G309;Alzheimer;1 100000;Demence type Alzheimer a un stade severe;G309;demence type Alzheimer;1 100000;Demence type Alzheimer a un stade severe;G309;demence type maladie Alzheimer severe;0.8 100000;Demence type Alzheimer a un stade severe;G309;demence type Alzheimer debutante;0.75 ...

For the rst three recognized concepts, the system returns 1 because the percentage of success of that term appears in the input text, for the case of "demence type maladie Alzheimer severe " the system returns 0.8 because of the fact that of the 5 words that the recognized term matches, 4 appear in the text.

2 https://www.nltk.org/ (last visited: May 30, 2018) 3 https://snowballstem.org/

3.2

Measures for recognized terms

After developing the system proposed above, we created several measures to treat the words of the recognized terms: { Exact words. This measurement is set to 1 if the term appears exactly the same in the cause of death text and assigns 0 otherwise. { Number of words between the rst and last word detected. This measure takes into account the total word number of the recognized term divided by the number of words between the rst word of the term and the last word. With this measure, we are able to give a higher score to those terms with a higher number of words. { Weight according to position in the detected term. Assigns a weight to each position of the recognized term:

X 1=i where:

i = word text position. { Weight by category ICD10. The last measure used is related to the categories of ICD10 (the rst letter in the code: A,B, ..., Z). We think that if the same category is detected many times, it will be more important and we should give it more weight. For this reason, we divide the number of occurrences of the detected category by a number of codes chosen , in our case, = 10 (that is, the system is con gured to return the top 10 relevant codes). 4 4.1

Experiments

Linear regression for classi cation codes ICD10

Our goal was to use predictive analysis to improve the accuracy of the results. We use linear regression because is one of the most well known algorithms in machine learning.

In order to choose the best classi er we tested several machine learning techniques. The table 2 shows the results obtained. We trained with the 2013 and 2014 documents and tested with 2006-2012.

We used all the measures described above to build a training model on the data provided, for every cause of death and code detected in the collections: 2006-2012, 2013 and 2014. Then we predicted the results of the new ICD10 codes for the causes of death in 2015. 4.2

Adding measurements

We found that the sum of all measures calculated for each recognized term also worked well. To perform the sum, the value of the measures were normalized.

The results obtained with the 2013 and 2014 training collections are shown in the table 3.

Conclusion and future works

In this work, the SINAI group has participated for the rst time in the task of recognition of medical entities and classi cation in ICD10 codes in French. We have developed an automatic system that identi es medical terms using the dictionaries provided and almost independent of the language. We have created some measures for the treatment of the recognized concepts, with these measures, we have tested our system using di erent machine learning techniques to predict the nal results. The technique with the best result obtained was Linear Regression.

For the next tasks of the CLEF, among others, we will want to improve our results train our systems by providing new information, will explore word embeddings models and we will incorporate new sources of knowledge such as ontologies specialized in the medical domain in French. In addition, other types of measurements that use bigrams and trigrams could be used. It will also be studied how much each measure used contributes to the nal result.

Acknowledgments

This work has been partially supported by a grant from Fondo Europeo de Desarrollo Regional (FEDER) and REDES project (TIN2015-65136-C2-1-R) from the Spanish Government.

References

1. Atkin , S.E. : Meta normalization for text (Apr 19 2005 ), uS Patent 6 , 883 , 007

2. Cohen , A.M. , Hersh , W.R.: A survey of current work in biomedical text mining . Brie ngs in bioinformatics 6(1) , 57 { 71 ( 2005 )

3. D az-Galiano , M.C. , Garc a-Cumbreras, M. , Mart n-Valdivia, M. , Urena-Lopez , L. , Montejo-Raez , A. : SINAI at ImageCLEFmed 2008 . In: Peters, C. , Ferro , N. (eds.) CLEF (Working Notes) . CEUR Workshop Proceedings , vol. 1174 . CEUR-WS.org ( 2008 )

4. D az-Galiano , M.C. , Garc a-Cumbreras, M.A. , Mart n-Valdivia, M.T. , MontejoRaez , A.: Knowledge integration using textual information for improving imageclef collections . In: ImageCLEF , pp. 295 { 313 . Springer ( 2010 )

5. D az-Galiano , M.C. , Mart n-Valdivia, M.T. , Mar a , S., Jimenez-Zafra , A.A. , Lopez , L.A.U. : Sinai at clef ehealth 2017 task 3 . In: CLEF 2017 Working Notes. CEUR Workshop Proceedings , vol. 1866 . CEUR-WS.org ( 2017 )

6. D az-Galiano , M.C. , Mart n-Valdivia, M.T. , Uren~a- Lopez , L. : Query expansion with a medical ontology to improve a multimodal information retrieval system . Computers in biology and medicine 39(4) , 396 { 403 ( 2009 )

7. Finkel , J. , Dingare , S. , Nguyen , H. , Nissim , M. , Manning , C. , Sinclair , G.: Exploiting context for biomedical entity recognition: From syntax to the web . In: Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications . pp. 88 { 91 . Association for Computational Linguistics ( 2004 )

8. Hahn , U. , Romacker , M. , Schulz , S. : How knowledge drives understanding|matching medical ontologies with the needs of medical language processing . Arti cial Intelligence in Medicine 15 ( 1 ), 25 { 51 ( 1999 )

9. Lavergne , T. , Neveol , A. , Robert , A. , Grouin , C. , Rey , G. , Zweigenbaum , P.: A dataset for icd-10 coding of death certi cates: Creation and usage . In: Proceedings of the Fifth Workshop on Building and Evaluating Resources for Biomedical Text Mining (BioTxtM2016) . pp. 60 { 69 ( 2016 )

10. Leser , U. , Hakenberg , J.: What makes a gene name? named entity recognition in the biomedical literature . Brie ngs in bioinformatics 6(4) , 357 { 369 ( 2005 )

11. Neveol , A. , Robert , A. , Grippo , F. , Morgand , C. , Orsi , C. , Pelikan , L. , Ramadier , L. , Rey , G. , Zweigenbaum , P. : Clef ehealth 2018 multilingual information extraction task overview: Icd10 coding of death certi cates in french, hungarian and italian . In: CLEF 2018 Evaluation Labs and Workshop: Online Working Notes, CEUR-WS ( 2018 )

12. Neveol , A. , Anderson , R.N. , Cohen , K.B. , Grouin , C. , Lavergne , T. , Rey , G. , Robert , A. , Rondet , C. , Zweigenbaum , P. : Clef ehealth 2017 multilingual information extraction task overview: Icd10 coding of death certi cates in english and french . In: CLEF 2017 Evaluation Labs and Workshop: Online Working Notes, CEUR-WS. p. 17 ( 2017 )

13. Neveol , A. , Cohen , K.B. , Grouin , C. , Hamon , T. , Lavergne , T. , Kelly , L. , Goeuriot , L. , Rey , G. , Robert , A. , Tannier , X. , et al.: Clinical information extraction at the clef ehealth evaluation lab 2016 . In: CEUR workshop proceedings . vol. 1609 , p. 28 . NIH Public Access ( 2016 )

14. Osborne , J.D. , Lin , S. , Zhu , L.J. , Kibbe , W.A. : Mining biomedical data using metamap transfer (mmtx) and the uni ed medical language system (umls) . Gene Function Analysis pp. 153 { 169 ( 2007 )

15. Savova , G.K. , Masanz , J.J. , Ogren , P.V. , Zheng , J. , Sohn , S. , Kipper-Schuler , K.C. , Chute , C.G. : Mayo clinical text analysis and knowledge extraction system (ctakes): architecture, component evaluation and applications . Journal of the American Medical Informatics Association 17 ( 5 ), 507 { 513 ( 2010 )

16. Zhou , G. , Zhang, J., Su , J. , Shen , D. , Tan , C. : Recognizing names in biomedical texts: a machine learning approach . Bioinformatics 20 ( 7 ), 1178 { 1190 ( 2004 )