Detection of drug-drug interaction (DDI) is crucial for identification of adverse drug effects. In this paper, we present a range of new composite kernels that are evaluated in the DDIExtraction2011 challenge. These kernels are computed using different combinations of tree and feature based kernels. The best result that we obtained is an F1 score of 0.6370 which is higher than the already published result on this same corpus.
The DDIExtraction2011 challenge3 provides a platform to identify the state of
the art for drug-drug interaction (DDI) extraction from biomedical articles. We
have participated in this challenge applying a range of new composite kernels.
These kernels combine different combinations of mildly extended dependency tree
(MEDT) kernel [
The best result we have obtained is an F1 score of 0.6370 by combining MEDT, PST and GC kernels on the unified format of the data. From the preprocessing of data to the extraction of DDIs using kernel compositions, our objective is to exploit the maximum information that could be learned from different representations of the data.
In the remaining of this paper, we discuss how we have addressed the DDI
extraction task. In Section 2, we briefly discuss the dataset. Then in Section 3,
the pre-processing steps are described. Following that, in Section 4, we mention
the individual kernels which are the building blocks for our kernel compositions.
Section 5 defines the proposed composite kernels. Evaluation results are
discussed in Section 6. Finally, in Section 7 we summarize our work and present
ideas for future work.
"##
+ ) ,
+ ( (
. / (
The DDIExtraction2011 challenge task requires the automatic identification of
DDIs from biomedical articles. Only the intra-sentential DDIs (i.e. DDIs within
single sentence boundaries) are considered. The challenge corpus [
These datasets are made available in two formats: the so-called unified format and the MMTx format. The unified format contains only the tokenized sentences, while the MMTx format contains the tokenized sentences along with POS tag for each token.
We have used the unified format data. We have found out that, in both training and evaluation datasets, there are some missing special symbols, perhaps due to encoding problems. The position of these symbols can be identified by the presence of the question mark “?” symbol. For example:
<sentence id=”DrugDDI.d554.s14” origId=”s14” text=”Ergotamine or dihydroergotamine?acute ergot toxicity characterized by severe peripheral vasospasm and dysesthesia.”>
We have tried to randomly check whether the unified format and MMTx format datasets contain the same sentences. We have found that one of the randomly chosen sentences 4 does not include a “>” character which exists as a token of the corresponding sentence inside the corresponding MTMx file. This suggests that there might be missing characters inside some sentences due to conversion errors of the html/xml special characters. 3
Our system is trained and evaluated on the unified format. We use the Stanford
parser5 [
Some of the characteristics of the data sets have required pre-processing steps to correctly handle the texts. Having “?” in the middle of a sentence causes parsing errors since the syntactic parser often misleadingly considers it as a sentence ending sign. So, we replaced all “?” with “@”. Additionally, to reduce tokenization errors, if a drug name does not contain an empty space character immediately before and after its boundaries, we insert space characters in those positions inside the corresponding sentence.
The SPECIALIST lexicon tool is used to normalize tokens to avoid spelling variations and also to provide lemmas. The dependency relations produced by the parser are used to create dependency parse trees for corresponding sentences. 4 DrugDDI.d151.s11 of the file Flumazenil ddi.xml. 5 http://nlp.stanford.edu/software/lex-parser.shtml
The approach adopted for our participation to the challenge is to exploit systems
(or methodologies) that already obtained state-of-the-art results in the
proteinprotein interaction (PPI) extraction task and also in other RE tasks in domains
such as newspaper articles. One of these systems [
Giuliano et al. [
KSL (R1, R2) = KLC (R1, R2) + KGC (R1, R2) where KSL, KGC and KLC correspond to SL, global context (GC) and local context (LC) kernels respectively. The GC kernel exploits contextual information of the words occurring before, between and after the pair of entities (to be investigated for RE) in the corresponding sentence; while the LC kernel exploits contextual information surrounding individual entities.
The jSRE system6 provides an implementation of these kernels. It should be noted that, by default, jSRE uses the ratio of negative and positive examples as the value of the cost-ratio-factor7 parameter during SVM training.
Segura-Bedmar et al. [
One of the tree kernels that we have used is called mildly extended dependency
tree (MEDT) kernel, proposed by Chowdhury et al. [
The MEDT kernel addresses this issue using some linguistically motivated
extensions. The best settings for the MEDT kernel, that we used in our
experiments for DDI extraction, observed by the authors on the AIMed protein-protein
interaction dataset [
If the root of the minimal subtree is the head word of one of the interacting entities, then add the parent node (in the original DT tree) of the root node as the new root of the subtree.
Apart from that, we have also used a phrase structure tree (PST) kernel
which is basically the path-enclosed tree (PET) proposed by Moschitti [
We propose the following composite kernels for DDI extraction: – KMP (R1, R2) = w1 * KMEDT (R1, R2) + w2 * KP ST (R1, R2) – KLMP (R1, R2) = KLC (R1, R2) + w3 * KMP – KGMP (R1, R2) = KGC (R1, R2) + w3 * KMP – KSMP (R1, R2) = KSL (R1, R2) + w3 * KMP
where KSL, KMEDT , KP ST , KLC and KGC represent SL, MEDT, PST, LC and GC kernels respectively, and wi represents multiplicative constant(s). The values for all of the wi used during our experiments are equal to 1. All the composite kernels are valid according to the closure properties of kernels.
The motivation behind using these new composite kernels is to combine varying representations (i.e. tree structures and flat structures) of different types of information (i.e. dependencies, syntactic information, and shallow linguistic features), and to see whether they can complement each other to learn a more robust model.
To compute the feature vectors of KSL, KLC and KGC , we used the jSRE
system. The tree kernels and composite kernels are computed using the
SVMLIGHT-TK toolkit8 [
We have tuned all the composite kernels on the training data using 10-fold cross validation9. The results of these experiments are shown in Table 1. The experiments show that the best result is gained using the KGMP . Both the KGMP and KSMP perform much better than the other kernels.
Precision
Recall F1 Score 9 To obtain the overall performance we sum up the true positives, false positives, and false negatives of all the 10 folds, and then measure precision, recall and F1 score from these figures. + ) , + ( ( . / ( have been evaluated on the DDIExtraction2011 challenge, and have achieved encouraging results.
Due to time constraints, we have not been able to perform extensive parameter tuning. We are confident that tuning of the multiplicative constant(s) (i.e. wi) might produce even better performance. We also predict these kernels would be able to learn more accurate training models using a bigger training data, and would produce results better than that of the individual kernels which are their building blocks.
This work was carried out in the context of the project “eOnco - Pervasive knowledge and data management in cancer care”.