In this work, we describe a statistical graph matching method for semantic indexing of documents from large-scale biomedical repositories in Spanish language provided at the MESINESP 2020 task (8th BioASQ Workshop [15]). The results obtained show enough accurate behavior, especially with respect to the rest of the results in the task. The execution time and computational requirements have been a priority in our approximation, which has proved to be e cient and robust for tackle further improvements.
Although the number of medical data is growing at an exponential rate, literature in the medical domain is often found as unstructured or semi-structured data. In these cases, it is necessary to nd methods to automatically extract and categorize the data contained in them, using di erent techniques as, for example, biomedical semantic indexing.
The BioASQ [15] is an EU-funded support action [
In this paper, we propose a statistical graph matching method implemented as a module into the HESML framework [9{11]. This method obtains information on the frequency with which DeCS codes are annotated to rank the list of candidates that are extracted from the text following two di erent methods described in Section 2.1.
The results are encouraging enough, especially when compared to the rest
of the experiments and knowing the main di culties. We will continue working
in this task with mixed approaches ([
The rest of the paper is organized as follows. In section 2, we describe the architecture of the system. Section 3 describes the evaluation process and the results obtained. Finally, section 4 outlines the conclusions and future work. 2
The MESINESP task, is the rst task on semantic indexing of Spanish medical texts, provides a dataset divided in training (318.658 documents), development (750 documents) and test (23.509 documents) sets. The average of DeCS codes per document is 8.12, and the document with the maximum number of codes has a total of 53 di erent ones. At a glance, the training set give us the idea of a scattered distribution of the codes annotated in the documents, as described in Table 1. Our proposed method try to overcome this problem using statistical information about the frequency of a DeCS code annotated in a document.
Total codes that appear more than 10 of documents% 6 Total codes that appear more than 1 of documents% 48 Total codes that appear less than 1 of documents% 33654 Total codes that never appear 22523 Total codes 33702
Table 1. Frequencies of the codes in the training set2 2.1
Proposed method The method proposed herein is a rst approximation focused on the e ciency and robustness of the system. Figure 1 represents the information ow to annotate the test set.
The process stages are the following: 1. Creation of the frequencies graph from the training and development data.
In this step, a directed graph is developed, where each DeCS code represent
a node, and the edges are the number of times the codes co-occur in each
document.
2. Parse the DeCS ontology and the list of codes and descriptors provided for
the competition 3.
3. Split the sentences and identi cation of the chunks for each sentence using
the Stanford CoreNLP library [
Once the test dataset is processed, the next step is the alignment of the documents with a list of DeCS code candidates. In this work, there has been carried out using two di erent methods, (a) exact matching and (b) graphbased matching. In the rst one, every possible descriptor is matched with each document. If the descriptor exists in the text, it is selected as a candidate. In the second method, every chunk is compared with the list of possible descriptors, aligning as much DeCS codes as possible.
Other experiments have been planned but could not be carried out due to
time constraints. The rst one is the alignment of chunks with DeCS codes
using a semantic sentence similarity measure, as for example, the Jaccard
similarity [
MESINESP task has been evaluated using the following measures: (a) Accuracy (Acc.), (b) Example Based Precision (EBP), (c) Example Based Recall (EBR), (d) Example Based F-Measure (EBF), (e) Macro Precision (MaP), (f) Macro Recall (MaR), (g) Macro F-Measure (MaF), (h) Micro Precision (MiP), (i) Micro 3 Downloaded from https://temu.bsc.es/mesinesp/index.php/resources/ Recall (MiR) and (j) Micro F-Measure (MiF), but we only include in this section the Micro F-measure, since it is the o cial evaluation measure for this task.
Our results are shown in Table 2 including the rst position and the baseline results of the task.
A total of 25 methods have been submitted to the MESINESP competition. Our work has focused on the e ciency and robustness of the method and executes the whole process in less than 30 minutes without requiring a training process. We have used the DeCS ontology and our new hypothesis is that the results will improve using another ontology-based similarity measure to the concept alignment without losing e ciency of the system.
The main di culty in this task is derived from the use of a purely statistical method that prioritizes the most frequent terms and considers neither the ontology hierarchy for avoiding the annotation of redundant child-parent terms nor the less frequent codes that the experts annotate in the gold standard. For example, the terms "tumor de mediastino" and "mediastino" are annotated using our approximation for the document ID "biblio-1000005", but the experts only annotate the terms "mediastino" and "neoplasias del timo". But, it happens that the term "tumor de mediastino" is explicitly written in the title of the document and, for this reason, it is considered as relevant for our algorithm. On the other hand, the term "pesar" is wrongly considered as relevant for our algorithm in most of the documents, because no semantics is considered in the selection of candidates. 4
In this work, we describe a statistical graph matching method for semantically index documents from large-scale biomedical repositories in Spanish language provided at the MESINESP 2020 task [15]). The execution time and computational requirements have been priority factors in our approximation, giving us a rst approach that is e cient and su ciently robust to improve the results in the future.
Addressing the task, we understand that the Spanish language has not been
thoroughly studied in a semantic indexing task, and there are only a few available
tools. For example, there are some Name Entity Recognizers (NER) that nd
UMLS concepts in Spanish biomedical documents, such as QuickUMLS [14] or
IXAMedTagger [
In the future work, we are going to focus on the integration of the parser for the DeCS ontology in HESML. We will try to prove that our proposal will overcome the problems with the running time of the experiments based on sentence similarity measures by allowing the use of di erent ontology-based measures. Finally, we want also to test a new model that recognizes co-occurrence patterns beyond the basic measure of the frequency of occurrence of terms. 13. P. Jaccard: Nouvelles recherches sur la distribution orale. Bull. Soc. Vaud. sci.
nat. 44, 223{270 (1908) 14. Soldaini, L., Goharian, N.: Quickumls: a fast, unsupervised approach for medical concept extraction. In: MedIR workshop, sigir. pp. 1{4. ir.cs.georgetown.edu (2016) 15. Tsatsaronis, G., Balikas, G., Malakasiotis, P., Partalas, I., Zschunke, M., Alvers, M.R., Weissenborn, D., Krithara, A., Petridis, S., Polychronopoulos, D., Almirantis, Y., Pavlopoulos, J., Baskiotis, N., Gallinari, P., Artieres, T., Ngomo, A.C.N., Heino, N., Gaussier, E., Barrio-Alvers, L., Schroeder, M., Androutsopoulos, I., Paliouras, G.: An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition. BMC Bioinformatics 16, 138 (Apr 2015)