Artificial neural networks (ANNs) have been developed to predict the clinical significance of drug-drug interactions (DDIs) for a set of 35 pharmaceutical drugs using data compiled from the Web-based resources, Lexicomp® and Vidal®, with inputs furnished by various drug pharmacokinetic (PK) and/or pharmacodynamic (PD) properties, and/or drug-enzyme interaction data. Success in prediction of DDI significance was found to vary according to the drug properties used as ANN input, and also varied with the DDI dataset used in training. The Lexicomp® dataset is found to give predictions marginally better than those obtained using the Vidal® dataset, with the best prediction of minor DDIs achieved using a multi-layer perceptron (MLP) model trained using enzyme variables alone (F1 82%) and the best prediction of major DDIs achieved using a MLP model trained on PK/PD properties alone (F1 54%). Given a more comprehensive and more consistent dataset of DDI data, we conclude that machine learning tools could be used to acquire new knowledge on DDIs, and could thus facilitate the regulatory agencies' pharmocovigilance of newly licensed drugs.
Recent years have witnessed an increasing number of publically available databases for drug knowledge including chemical and pharmacological information [1], drug-protein relationships and drug mechanisms of action [2,3] and adverse effects [4,5].
The creation of these various resources has afforded new opportunities in drug discovery and development, wherein data mining techniques are employed, for example, to deduce combinations of chemical and biological characteristics of use in drug repurposing [6,7] – that is, the identification of new therapeutic indications for approved drugs – and also to allow the prediction of drug-protein relationships [8], and likely drug side effects [9]. Such predictions as these are of great relevance in pharmacovigilance, providing for the detection, assessment, understanding, and prevention of the adverse effects of drugs (and any other drug-related problems) [10]. Licensed drugs are frequently seen to cause adverse effects that are not observed in the clinical trials conducted prior to approval when used in large populations comprising individuals with very different physiological and pathological characteristics. The early detection of these adverse effects is crucial to ensure patient safety and is the primary purpose of pharmacovigilance. Among the various adverse effects of interest, those that arise as a consequence of drug-drug interactions (DDIs) are arguably the most common. These adverse effects are seen in patients that are prescribed two or more drugs that interact in some way. In these cases, one of the drugs taken affects the blood levels and/or efficacy of a coadministered drug, thereby giving rise to unexpected toxicological problems or therapeutic failure [11]. Problems of this nature have seen increased frequency of late, partly because of the increased numbers of elderly patients that suffer multiple co-morbidities [12] or the common use of cocktails of drugs to treat complex pathologies [13]. Historically, information on DDIs was collated in manually curated compendia [11], [14], and many of these resources are nowadays accessible online. There are many such pharmacological databases and semistructured resources that are available to assist healthcare professionals in the prevention of DDIs (e.g., Vidal® and Lexicomp®) but their quality is variable and the consistency of their contents rather limited. One of the most relevant discrepancies among the different DDI information sources is their assessment of the significance of the recorded interactions. Here, significance refers to the clinical relevance of the DDI, and describes the risk that the DDI might pose for a patient’s health [14]. For any given DDI, the clinical significance will vary according to the nature of the patient – their age, ethnicity, and genetic profile – and also on the drugs’ pharmacological characteristics – including their target(s), metabolism, and side effects. The grading of DDI significance is generally assessed subjectively, through a preestablished set of evaluation criteria, and thus discrepancies among different DDI compendia and information sources are very common. Indeed, different researchers have identified important discrepancies between different information compendia [15] and between these and criteria laid down by clinicians [16]. These studies highlight the extreme difficulty of assessing the severity of DDIs. One might expect, however, that most of the information sources would show a high degree of overlap, at least for those DDIs that would have severe health consequences (interactions that we may thus consider as major DDIs), and a similar degree of overlap for those DDIs that would not be expected to do so (interactions which we might thus call minor DDIs). The development of an in-silico system that could automatically identify DDIs and provide an initial assessment of their likely clinical significance (classifying each as major or minor) would likely be of great benefit, therefore, in the field of signal detection in pharmacovigilance. The prediction of DDIs through the application of machine learning methods is an active research field. Cheng et al. [17] used phenotypic, therapeutic, chemical structure and genomic similarity between drugs as input properties to train and evaluate different machine learning methods: naive Bayes (NB), decision tree (DT), k-nearest neighbors (kNN), logistic regression (LR), and support vector machines (SVM). The last of these proved to lead to the highest performance for the prediction of DDIs between 721 drugs. Recently, Sridhar et al. [18] described a probabilistic approach using Probabilistic Soft Logic (PSL) and different drug-drug and protein-protein similarity measures as input variables. Focusing on enzyme-related DDIs only, Hunta et al. [19] evaluated the performance of ANN, SVM and k-NN for the prediction of DDIs, while Polak et al. [20] constructed several ANN models using drug physicochemical and metabolic properties of drugs as input data. One of the challenges in all these projects is the identification of reliable negative examples of non-interacting drug pairs. So far, these approaches have used pairs of drugs not included in the selected information source as examples of non-DDIs. Because of the aforementioned limitations of manually created DDI compendia and databases [15,16], it is impossible to know if one pair of drugs is not described in the selected source because it is not known yet or because there is no DDI between the drugs [21]. In contrast, prediction of significance of DDIs based on examples of graded DDIs could overcome this issue, enabling the distinction between minor (or potentially non-harmful) and major (or potentially serious) DDIs. However, to the best of our knowledge there has been no attempt made to date to develop a more sophisticated reasoning engine to furnish systematic predictions of DDI severity.
In the work reported here, we describe a novel approach for the identification of clinically relevant DDIs using machine learning techniques; we use input provided by relevant chemical and pharmacological drug characteristics extracted from online information sources, together with a bespoke DDI dataset containing information extracted from known DDI compendia. 2 2.1
A total of 35 drugs were selected for study, according to the criteria that each of those selected exhibit a high potential for interaction with other drugs and/or are representative of a major therapeutic class and had no missing data for any of the input variables. Many of the selected drugs were previously used in related work by Vitry et al. [15].
Through a review of the general literature on DDIs [11], [14], those drug characteristics considered relevant to their interaction profiles were identified. The 20 drug characteristics selected included both pharmacokinetic (PK) properties − those descriptive of drug disposition in the body − and pharmacodynamic (PD) properties − those descriptive of the drugs’ effects in patients. The drug properties data were taken from the manually-curated database DrugBank [1], the online versions of Martindale [22] and Clarke's Analysis of Drugs and Poisons [23] and the Database of Intravenous Pharmacokinetic Parameters in Humans [24].
Given the frequency with which DDIs result as a consequence of
druginduced changes in the activity of metabolic enzymes [
Information on DDIs was collated from the Lexicomp® and Vidal® compendia. In the former case, the online Lexi-Interact™ Online Interaction Service, was used to acquire the DDI information [27]. DDI information from both compendia were graded according to their respective five and four point scales. In the case of Lexicomp®, the DDIs were graded from ‘no known interaction’ through to ‘avoid combination’. In the case of Vidal®, the DDIs were graded from ‘none’ to ‘contraindication’ [28] (Table 1). The two sets of DDI data were compiled separately, and are referred to below as the DDI-L and DDI-V datasets. 2.3
Artificial neural network (ANN) models to predict DDIs were developed using the data mining tools provided in Statistica®. This application provides a wide selection of network types and the training algorithms BFGS (Broyden-Fletcher-Goldfarb-Shanno) and Scaled Conjugate Gradient algorithms [29].
Separate models were trained and tested using the datasets described above, which included a total of 142 variables, and using different combinations of drug properties. Experiments were initially performed to predict interacting vs. non-interacting drug pairs and these then repeated to predict the significance/grade of the DDIs.
In an initial analysis, we explored two types of ANN architecture: Multilayer Perceptron Neural Networks (MLP) and Radial Basis Function Neural Networks (RBF). Only MLP networks with a number of hidden units in the range of 8-20 were retrieved as the best ones.
For the construction of the final models, therefore, the methodology adopted was as follows: 200 MLP networks were trained with a range of 8-20 hidden units retrieving the best five networks. Activation functions were not restricted, so we explored the set of neuron activation functions for the hidden and output neurons available in Statistica: identity, logistic sigmoid, tanh and exponential. The error function used was either sum of squares (SOS) or cross entropy. Overfitting was prevented by manually dividing the input data into training (70% of the dataset), test (10%) and validation (20%) datasets, ensuring a balanced representation in each of these for all classes.
From the five best networks per analysis, we selected the one with best performance for training, test and validation sets. The generalization ability of the models was quantified by means of precision (P), recall (R) and F1 in the validation dataset. 3
The two DDI datasets differ considerably in terms of both their coverage and their significance gradings. The total number of drug pairs is ® 561, of which 210 (37%) are labelled as interacting in Lexicomp , while in Vidal® they are only 124 (22%). The overlap between them is small (421 coincidences), with only 97 DDIs and 324 non-DDIs in common. Regarding significance grading, the number of coincidences is also limited (Table 1). Because of the different scales used in the two DDI datasets and the small number of examples for some types (such as contraindication/avoid combination), we combined the examples into two gradings: minor and major DDIs. As shown in Table 2, results vary for the different datasets and the different input variables. In the case of the DDI-L dataset, the best results are achieved using enzyme properties alone with a MLP network with 204-10-2 input, hidden and output neurons, respectively. The hidden activation function is exponential and the output activation function is logistic. The training algorithm is BFGS and the error function SOS. Although the results show higher relevance of enzyme properties alone compared to a combination of all variables, this difference is very small (F1 64% vs 60%). In contrast, the use of PK/PD variables alone outperformed the other models in the DDI-V dataset, mainly because of a decrease in recall. As with the previous dataset, the performance of the best model does not differ greatly from that achieved through use of all combined properties (F1 58% vs 50%). This model is a MLP with 10114-2 input, hidden and output units trained using a BFGS algorithm. The hidden and output activation functions are exponential and logistic respectively, while the error function is SOS. Regarding the prediction of the significance of DDIs, we created another six different models using the same datasets DDI-L and DDI-V but excluding the non-interacting pairs. As with the results presented above, it is not possible to establish a relationship between a set of variables and better models’ performance (Table 3). The two datasets are unbalanced and both showed better performance for the majoritarian class (minor in DDI-L dataset and major in DDI-V dataset). A larger DDI dataset would solve this issue and would allow us to establish more significance classes. Although the best results correspond to the model based on PK/PD variables in the DDI-V dataset, the small number of instances (194 DDIs) and the high results suggest that the model might be over-fitted. Therefore, we believe that the most reliable results correspond to the model trained using enzyme variables alone for minor DDIs (F1 82%) and the model based on PK/PD properties alone for major DDIs (F1 54%) in the DDI-L dataset. The first model is a MLP with 204-16-2 input, hidden and output units and hidden and output activation function than. The second one is a MLP with 100-13-2 input, hidden and output neurons with exponential and identity hidden and output activation functions. Both models are trained with a BFGS algorithm and use SOS as error function. 4
Here, we have described a preliminary analysis for the prediction of DDIs and their clinical significance through the creation of machine learning models that exploit drug information collected from different information sources available on the web.
Different research groups have applied machine learning for the prediction of DDIs. These projects differ considerably in the original datasets, the properties used as input variables, the machine learning methods studied and the evaluation of their performance. Thus, a straight comparison with our results is difficult and beyond the scope of this project. The closest work in terms of evaluation metrics is the prediction of DDIs based on a probabilistic approach using PSL, which reported an F1 of 67% on a dataset of 4,293 known interactions between 315 drugs. This approach outperformed state-of-the-art works for DDI prediction that obtained F1 values of 51% and 60% [18].
In our case, the ANN models led to F1 of 64% and 59% for the validation instances in the DDI-L and DDI-V datasets, respectively. We believe that there is still room for improvement, in part because the enzyme properties included in our approach represent only a relatively small selection of those likely to lead to DDIs. However, there are many different DDI mechanisms not related to metabolic processes [11]. Therefore, in our future work we plan to include other drugprotein relationships – including targets, transporters and carriers – that will enable the identification of DDIs occurring by other mechanisms. Also, we believe that representation of adverse effects profiles will be very useful to identify DDIs due to the addition of side effects [30]. The previous approaches rely on unknown DDIs as examples of noninteracting pairs, which may lead to incorrect predictions and hinder the identification of new DDIs. In contrast, we have proposed a new strategy based on the prediction of the significance of known DDIs. Our model yielded interesting results, with an F1 of 82% for the best model. To the best of our knowledge, this is the first work attempting the prediction of DDI significance. The main limitations are, however, the size of the current dataset, the inconsistent information between different DDI sources and the frequent missing data for some input variables. Automatic methods for knowledge extraction from the web is crucial for the creation of a larger dataset of graded DDIs combining consistent information from different sources, which will lead to more sophisticated prediction models.
We believe that further improvements in this area could represent an important tool in pharmacovigilance, for example as an initial signal detection tool for the editorial boards maintaining and updating current DDI compendia.
María Herrero-Zazo holds a C. W. Maplethorpe Postdoctoral Fellowship for Pharmaceutical Education and Research at King’s College London. Maxime Lille was supported by an internship provided from the Fondation de l'Universite d'Auvergne, France. [23] [24]