Semantic indexing of animal experiment summaries is the process of annotating the summaries with its medical codes. Semantic indexing is helpful in reducing time and performance in knowing the context and finding relevant summaries. Indexing the Non-Technical Summaries (NTP)s using codes from the German version of the International Classification of Diseases (ICD-10) is a challenging task. ICD-10 codes, which is a comprehensive way of storing the health conditions are useful for the identification of many disorders, diseases and other health related problems. Thus, annotating the NTPs with codes will make the way of storing, organising, retrieval and comparing the health information more easier. In our paper, we have approached the problem using deep neural network. This work is evaluated on the dataset given by eHealth@CLEF2019. The test set given by the task is used to evaluate our methodology which attains precision, recall and f1 score of 0.19, 0.27 and 0.23 for Run 1 , 0.19, 0.27 and 0.22 for Run 2 and 0.13, 0.34 and 0.19 for Run 3 respectively. The performance of our method can further be increased by considering other recurrent units.
In this task, Non Technical Summaries(NTPs) are given which are to be
indexed with their International Classification of Diseases (ICD-10) in German
version. ICD-10 codes are helpful in many ways to diagnose various diseases
and identify its drugs [
The dataset is given for the task 1 of eHealth-CLEF@2019[
C50-C50|C00-C97|C00-C75|II where ’|’ separates one code from another one. 3
A Deep neural network model [
A deep neural network model was built using a multi-layer RNN (Recurrent Neural Network) in which LSTM (Long Short Term Memory) as its recurrent
unit. Layers namely embedding layer, encoder-decoder layer, projection layer
and loss layer are used to build the deep neural network. The input lines in
the document and its corresponding code labels in the embedding layer are
used to learn the weight vectors based on their vocabulary. Two hidden layers
are used for encoding and decoding.The attention mechanism such as Normed
Bahdanau(NB) and Scaled Luong(SL) models [
indexing. We have implemented two broad variations of the Seq2Seq model by varying the attention models with a batch size of one hundred and twenty eight, two encoder-decoder layers, dropout of 0.2 and bi-directional. Further variations are done by different considerations in post processing and type of input(attention model) given. The different variations are explained in the Table 1.
The performance obtained for these variations are evaluated using the evaluation script provided by the CLEF-ehealth@2019 in shown Table 2.
From the above models, model 3, 4 and 6 are submitted as Run 1, Run 2 and Run 3 respectively for the shared task.
The evaluation script provided by the CLEF-eHealth@20194 is used to evaluate our models with respect to both development set and test set. The devel
opment set has more accuracy than compared to the test set. Testing accuracy can also be improved by building the model using test set as its development set. From the Table 3, Run 3 has more true postive score than the other runs for both development set and test set. 4
The final evaluation is done on the dataset provided by CLEF-eHealth@2019. The test set contains 407 summaries which should be annotated.
Semantic indexing is the indexing of animal experiment summaries with ICD-10 codes in German version. We have made use of deep neural network with two different attention models for the indexing the summaries with their medical codes such as C50-C50|C00-C75|II. We have splitted the documents into sentences and generated the codes with respect to the documents and combined them according to minimum of two occurences with NB attention for Run 1 which has 0.19, 0.27 and 0.23 and the same with SL attention for Run 2 which has 0.19, 0.27 and 0.22 and considering minimum of two occurences and considering the whole codes generated if nothing is generated as its code as Run which has 0.13, 0.34 and 0.19 as its precision, recall and F1 score which is evaluated on the test set of eHealth@CLEF2019. Further improvements can be done by considering Google Neural Machine Translation (GNMT), Gated Recurrent Unit (GRU) as recurrent units instead of LSTM. 10. Luong, M.T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025 (2015) 11. Neves, M., Butzke, D., Dörendahl, A., Leich, N., Hummel, B., Schönfelder, G., Grune, B.: Overview of the CLEF eHealth 2019 Multilingual Information Extraction. In: Crestani, F., Braschler, M., Savoy, J., Rauber, A., et al. (eds.) Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the Tenth International Conference of the CLEF Association (CLEF 2019). Lecture Notes in Computer Science. Springer, Berlin Heidelberg, Germany (2019) 12. Skull, S.A., Andrews, R.M., Byrnes, G.B., Campbell, D.A., Nolan, T.M., Brown, G.V., Kelly, H.A.: Icd-10 codes are a valid tool for identification of pneumonia in hospitalized patients aged 65 years. Epidemiology & Infection 136(2), 232–240 (2008) 13. Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in neural information processing systems. pp. 3104–3112 (2014) 14. Thygesen, S.K., Christiansen, C.F., Christensen, S., Lash, T.L., Sørensen, H.T.: The predictive value of icd-10 diagnostic coding used to assess charlson comorbidity index conditions in the population-based danish national registry of patients. BMC medical research methodology 11(1), 83 (2011)