Neurological adverse drug reactions (NADRs) pose a significant clinical challenge as they can have a profound impact on patient health and treatment outcomes. While diverse drug descriptors have been employed for neurological ADR prediction, the potential of using 17 molecular descriptors for this purpose has not been explored. To address this, a multilabel NADR-Net and MLSMOTE-based framework have been proposed for neurological adverse drug reaction prediction. The data for 17 MD and ADRs were extracted from PubChem and ADRECs databases and then mapped based on drug ID. The resulting dataset contained information on 2160 drugs, including their molecular properties, and 1030 ADRs. The methodology was then applied to this dataset, and it showed promising results in terms of hamming loss, precision, true positive rate, f1 score, and ROC-AUC. This study highlights the potential of using molecular descriptors for predicting neurological ADRs, which could improve patient outcomes and drug safety.
An Adverse Drug Reaction (ADR) is characterized as a Superfluous or deleterious reaction
experienced after the administration of a medication [
To achieve this aim, a deep neural network-based framework, NADR-Net, is designed to process and analyze the 17 MD characteristics of drugs to identify potential neurological adverse efects. To handle class imbalance, MLSMOTE is applied. The eficacy of this framework is validated using five performance metrics: precision, true positive rate, f1-score, AUC-ROC score, and hamming loss. Furthermore, the performance of the proposed architecture is compared with two other DNN architectures, namely, the standard DNN and the DNN with a 1-skip connection. Subsequently, the framework’s performance, with and without the use of MLSMOTE, is compared to demonstrate the influence of the MLSMOTE technique on the model’s proficiency in accurately predicting NADRs.
The remainder of this paper is structured as follows: Section 2 presents related work, Section 3 outlines the dataset and proposed methodology for predicting NADR-Net, Section 4 details the experimental setup and results, and Section 6 ofers concluding remarks.
In the last few years, several proposals have been made to use machine learning and deep learning to address the problem of ADR prediction. One such study [7] employed a binary classification approach and evaluated 176 machine-learning models for 22 NADRs. The models were trained using drug properties, including biological, chemical, and phenotypic aspects. SMOTE was applied to mitigate class imbalance. Relief-based feature selection techniques were used to identify relevant drug properties. Lee et al. [8] introduced a three-interval method integrating chemical and biological drug properties, outperforming k-nearest neighbor, naïve Bayes, and random forest classifiers. Another study [ 9] suggested a hybrid clustering-based approach to analyze the quantitative relationships between adverse drug reactions and patient attributes. Further, Jamal et al. [10] focused on predicting adverse drug reactions for cardiovascular drugs through biological and chemical information, therapeutic indications, and their combined datasets, using techniques specifically RF, SVM, and Sequential Minimization Optimization. This approach addressed the issue of class imbalance through SMOTE. Another study describes an approach to generating DL-based, systematic ADR prediction models [11] [12]. In a related study, Dey et al. [13] use ML models, including a DL framework, to simultaneously predict ADRs and identify the underlying mechanisms. The models were trained on chemical-protein binding and gene expression datasets to improve prediction performance. Zheng et al. [14] examined and pinpointed the adverse efects associated with medications through the use of Highly Credible Negative Samples (HCNS), which were derived from various sources, including pathways, target proteins, chemical substructures, substituents, and the connection between genes and diseases. Their research dataset included 1,048 drugs and 1,276 diferent side efects. Das et al. [ 15] explored a multi-label ML strategy using drug functions and the MLSMOTE technique for handling class imbalances. In a recent review article, Lee et al. [16] provides an overview of the detection and classification of side efects using deep learning approaches. They show that deep learning approaches can help reduce or prevent the occurrence of ADRs by detecting and predicting them during post-marketing surveillance. Martenot et al. [17] proposed a DLbased pipeline for ADR monitoring in the biomedical literature that was introduced to detect serious ADRs in relevant documents at the sentence level. It relies on a modular architecture of open-source fine-tuned models and drug entities. Wang et al. [ 18] developed a deep learning model to examine drug side efects using descriptors from various sources, including biomedical literature from MEDLINE, drug-like and biological properties from PubChem and DrugBank, respectively. These elements were combined into a dataset for analysis using an MLP model with two hidden layers. The model outperformed others like Gaussian NB, Linear SVM, and PMF, achieving a top AUC of 84.40
The investigation of NADRs remains primarily focused on chemical and biological properties in existing literature, with only one study [7]conducted in this domain. However, a notable lack of research using a set of 17 specific molecular properties to explore these ADRs exists. Prior research has employed traditional machine learning models and the SMOTE for data balancing, limiting their scope to 22 non-neurological ADRs (NADRs). To date, the potential of deep learning models in studying neurological ADRs remains largely unexplored, highlighting a significant gap in current research.
This section presents the acquisition of 17 MD datasets to predict neurological adverse drug reactions. Furthermore, the problem statement for predicting NADR and the proposed framework for solving the stated problem has been demonstrated.
Let = {1, 2, . . . , } be a set of drugs, where is the total number of drugs. Each drug is characterized by a feature vector x ∈ R17, representing the 17 molecular properties of the drug. The objective is to map each drug to a set of neurological ADRs. Define the set of all possible ADRs as = {1, 2, . . . , 1030}, where each represents a unique neurological ADR. The task is to learn a function : R17 → {0, 1}1030 such that for each drug , the function predicts a binary vector y = (x), where y ∈ {0, 1}1030. In this vector, = 1 indicates the presence of the ADR for the drug , and = 0 indicates its absence.
In this section, the preparation of datasets for validating the proposed methodology is carried out by integrating 17 molecular drug descriptors and neurological adverse drug reactions (NADRs) data, as illustrated in Figure. 1. The properties of 17 molecules from drugs are retrieved from the PubChem [19] database using the PubChemPy Python package, resulting in a dataset of 2310 drugs, each characterized by 17 distinct features. These features include exact mass, hydrogen bond donor count, molecular weight, covalently-bonded unit count, rotatable bond count, undefined bond stereocenter count, complexity, monoisotopic mass, defined bond stereocenter count, topological polar surface area, isotope atom count, hydrogen bond acceptor count, formal charge, heavy atom count, defined atom stereocenter count, and XlogP3. The data on neurological adverse drug reactions, comprising 2160 drug samples and information on the occurrence and non-occurrence of 1030 ADRs, is extracted from the ADRECs [20] database. The dataset is created by mapping the 17 molecular drug descriptors with ADR data on drug ID.
In this study, we applied the MLSMOTE [21] to mitigate the challenge of underrepresented data in NADRs. MLSMOTE enhances the dataset by creating synthetic samples, specifically chosen for its ability to consider the multi-label characteristics of our data. The process begins with identifying minority labels, which are those with fewer average data samples compared to others. Subsequently, all data samples associated with these minority labels are identified. We then randomly select a sample from this minority group and identify its k-nearest neighbors to form a reference neighborhood. The generation of synthetic data samples involves an interpolation method for attributes and a majority voting approach for labels, utilizing the selected minority sample and its neighbors. In this context, the number of nearest neighbors (k) was set to 3.
No.
Adverse neurological drug reactions are predicted using a deep neural network-based architecture (NADR-Net), as detailed in Table. 1. This architecture utilizes a sequential and concatenated approach to process 17 molecular properties. The model begins with an input layer, indicating the dataset’s feature count, followed by dense layers. Each layer employs a ReLU (Rectified Linear Unit) activation function, introducing non-linearity and enhancing the model’s ability to learn complex patterns.
The input data is transformed into a compressed representation in the described neural network model through a series of dense layers. Initially, the input is compressed into a 1024dimensional space by the first layer, which is then reduced to 528 dimensions by the second layer. Following this, the output of the second layer is concatenated with the original input, resulting in a 545-dimensional vector. This vector undergoes further dimensionality reduction through two additional dense layers, ultimately bringing it down to 128 dimensions. Another concatenation operation combines the output of the third layer with that of the second layer to leverage low-level and high-level features from the previous layers. Finally, the resulting vector is transformed by one last dense layer into a 64-dimensional output. For the final output, a 1030-dimensional vector is produced using a sigmoid activation function, indicating the model’s capacity for multi-label classification. By combining earlier and deeper features in the network, this architecture enhances the learning process through feature reusability and representation learning.
This section presents the evaluation measures, experimental setup, and findings from analyzing 17 molecular properties in predicting neurological adverse drug reactions using the proposed framework, NADR-Net.
The performance of the proposed framework was evaluated using the following measures. Consider = {(test , )| = 1, 2, . . . , } as the multi-label dataset, where signifies the actual label set for the test instance test , and represents the labels predicted by the classifier.
Is defined as the frequency at which the model inaccurately predicts a label pair for a given sample . In this context, ∆ represents the symmetric diference between two sets, which is used to calculate the mismatch between the predicted and actual labels [22].
1 ∑︁ 1 Hamming Loss = =1 || | ∆ | (1) (2) (3) (4)
Is the ratio of actual positive predictions to the total number of positive predictions made by the model [22].
Precision = 1 ∑︁ | ∩ | =1 | |
Is defined as the ratio of the number of positive samples correctly identified by the model (True Positives) to the total number of actual positive samples in the data (the sum of True Positives and False Negatives) [22].
Precision = 1 ∑︁ | ∩ | =1 | | 4.1.4. F1-Score: Is the harmonic average of TPR and Precision [22].
F1 Score = 1 ∑︁ 2 × | PL ∩ AL| =1 |PL| + |AL|
It indicates a model’s ability to discriminate across classes, with the ROC curve plotting true versus false positive rates and the AUC quantifying the overall classification accuracy. Higher AUC indicates better model performance.
The implementation of the proposed framework was carried out using the Scikit-learn package in Python 3.7.4. An Intel (R) i5-5300U CPU and 512 GB of RAM were utilized by the deep learning framework. In the DNN architecture, the ReLU activation function and six hidden layers (1024, 528, 256, 128, 64, 32) were employed. For the DNN with one skip connection, two hidden layers (1024, 528) were used in conjunction with the ReLU activation function, followed by a concatenation layer that combines the input layer with the second hidden layer (528). Subsequently, four hidden layers (256, 128, 64, 32) were utilized. ReLU and Sigmoid activation functions were adopted to address non-linearities and probabilistic outputs, crucial for multi-label classification. A learning rate of 0.01 was set to ensure eficient convergence. Binary cross-entropy was selected as the loss function due to its efectiveness in independent label predictions. The Adam optimizer was used for its capability to manage sparse gradients. The training was conducted over 150 epochs with a batch size of 32, aiming to balance learning eficiency and prevent overfitting, optimizing the model’s performance for the multi-label dataset. These hyperparameters were kept consistent across all three architectures. Additionally, the cross-validation technique was used for validation, where 80% of the data was allocated for training and 20% for testing.
This section analyses the efectiveness of the proposed framework, comparing it with DNN and DNN with 1-skip connection architectures, and evaluates the impact of using the MLSMOTE technique. The model is evaluated on test data and calculates various performance measures. Table 2 presents a comparative performance analysis of deep neural network, DNN with a 1-skip connection, and NADR-NET. Each architecture’s performance is evaluated based on various metrics: precision, actual positive rate (TPR), f1-score, ROC-AUC, and hamming loss. Precision showed an incremental increase from 89.06% to 91.24%, TPR improved from 93.12% to 95.77%, and the f1-Score rose from 92.01 to 93.45. ROC-AUC and hamming loss saw similar trends, with ROC-AUC escalating from 95.65% to 97.02% and loss decreasing from 2.32 to 2.12. These results suggest that NADR-NET architecture significantly enhances model eficacy with increments ranging from approximately 2% to 5% across five metrics.
As delineated in Figure. 2, precision, true positive rate (TPR), f1-score, and ROC-AUC are presented for each fold, ranging from 1 to 7. Precision percentages fluctuate slightly, maintaining above 90% across all folds. TPR predominantly remains in the mid-95% range, peaking at 96.77% in fold 3. The F1-Score, indicative of test accuracy, consistently stays above 93%, reflecting robust model performance. ROC-AUC values, representing the model’s class distinction ability, are uniformly high, with all folds scoring above 96%. It can be observed from the above graph that the predictive model exhibits a high degree of accuracy and reliability, as evidenced by the performance metrics across all folds in the cross-validation process.
The incorporation of MLSMOTE was evaluated both with and without the NADR-Net’s performance in the research study. The results, as demonstrated in Figure. 3, indicated a substantial enhancement in the model’s performance. More than a 42% increase was shown in precision, while the rate of correctly identified positive instances experienced a growth of over 55%. A nearly 49% improvement was exhibited in the aggregate measure of test accuracy, and the model’s ability to diferentiate between classes improved by approximately 28%. Furthermore, the occurrence of incorrect labels decreased by over 50%, underscoring the significant impact of MLSMOTE on the overall efectiveness of the NADR-Net.
The proposed methodology efectively showcased the capability of the NADR-Net, a deep neural network, to predict adverse neurological drug reactions utilizing 17 molecular properties. A significant aspect of the study was addressed by applying the Multilabel Synthetic Minority Over-sampling Technique (MLSMOTE) to overcome the challenge of class imbalance. The eficacy of NADR-Net was evaluated rigorously using five performance metrics: a high precision of 91.24 ± 0.0028, TPR of 95.77 ± 0.0051, F1 score of 93.45 ± 0.0017, ROC-AUC of 97.02 ± 0.0023 and maintaining a hamming loss at a minimal 2.12 ± 0.0004 were notably achieved. A comparative analysis revealed that the model’s performance was notably enhanced by including the MLSMOTE-enhanced dataset. In the future, an expanded set of drug properties could be utilized to enhance prediction capabilities and integrate mechanisms to demystify the black box model’s decision-making process. adverse drug reaction monitoring centre in central india: A 7-year surveillance study, BMJ open 11 (2021). [5] K. Svendsen, M. Wood, E. Olsson, H. Nordeng, Reported time to onset of neurological adverse drug reactions among diferent age and gender groups using metoclopramide: an analysis of the global database vigibase®., European Journal of Clinical Pharmacology 74 (2018) 627–636. [6] P. A. Thuermann, R. Windecker, J. Stefen, M. Schaefer, U. Tenter, E. Reese, H. Menger, K. Schmitt, Detection of adverse drug reactions in a neurological department: comparison between intensified surveillance and a computer-assisted approach, Drug safety 25 (2002) 713–724. [7] S. Jamal, S. Goyal, A. Shanker, A. Grover, Predicting neurological adverse drug reactions based on biological, chemical and phenotypic properties of drugs using machine learning models, Scientific reports 7 (2017) 872. [8] W.-P. Lee, J.-Y. Huang, H.-H. Chang, K.-T. Lee, C.-T. Lai, Predicting drug side efects using data analytics and the integration of multiple data sources, IEEE Access 5 (2017) 20449–20462. [9] Yogita, J. W. Sangma, S. N. Anal, V. Pal, Clustering-based hybrid approach for identifying quantitative multidimensional associations between patient attributes, drugs and adverse drug reactions, Interdisciplinary Sciences: Computational Life Sciences 12 (2020) 237–251. [10] S. Jamal, W. Ali, P. Nagpal, S. Grover, A. Grover, Computational models for the prediction of adverse cardiovascular drug reactions, Journal of translational medicine 17 (2019) 1–13. [11] A. Mohsen, L. P. Tripathi, K. Mizuguchi, Deep learning prediction of adverse drug reactions in drug discovery using open tg–gates and faers databases, Frontiers in Drug Discovery 1 (2021) 768792. [12] P. Das, Yogita, V. Pal, Integrative analysis of chemical properties and functions of drugs for adverse drug reaction prediction based on multi-label deep neural network, Journal of Integrative Bioinformatics 19 (2022) 20220007. [13] S. Dey, H. Luo, A. Fokoue, J. Hu, P. Zhang, Predicting adverse drug reactions through interpretable deep learning framework, BMC bioinformatics 19 (2018) 1–13. [14] Y. Zheng, H. Peng, X. Zhang, Z. Zhao, J. Yin, J. Li, Predicting adverse drug reactions of combined medication from heterogeneous pharmacologic databases, BMC bioinformatics 19 (2018) 49–59. [15] P. Das, Y. Thakran, S. N. Anal, V. Pal, A. Yadav, Brmcf: Binary relevance and mlsmote based computational framework to predict drug functions from chemical and biological properties of drugs, IEEE/ACM Transactions on Computational Biology and Bioinformatics (2022). [16] C. Y. Lee, Y.-P. P. Chen, Prediction of drug adverse events using deep learning in pharmaceutical discovery, Briefings in Bioinformatics 22 (2021) 1884–1901. [17] V. Martenot, V. Masdeu, J. Cupe, F. Gehin, M. Blanchon, J. Dauriat, A. Horst, M. Renaudin, P. Girard, J.-D. Zucker, Lisa: an assisted literature search pipeline for detecting serious adverse drug events with deep learning, BMC medical informatics and decision making 22 (2022) 1–16. [18] C.-S. Wang, P.-J. Lin, C.-L. Cheng, S.-H. Tai, Y.-H. Kao Yang, J.-H. Chiang, Detecting potential adverse drug reactions using a deep neural network model, Journal of medical Internet research 21 (2019) e11016. [19] S. Kim, P. A. Thiessen, E. E. Bolton, J. Chen, G. Fu, A. Gindulyte, L. Han, J. He, S. He, B. A.
Shoemaker, et al., Pubchem substance and compound databases, Nucleic acids research 44 (2016) D1202–D1213. doi:10.1093/nar/gkv951. [20] M.-C. Cai, Q. Xu, Y.-J. Pan, W. Pan, N. Ji, Y.-B. Li, H.-J. Jin, K. Liu, Z.-L. Ji, Adrecs: an ontology database for aiding standardization and hierarchical classification of adverse drug reaction terms, Nucleic acids research 43 (2015) D907–D913. [21] F. Charte, A. J. Rivera, M. J. del Jesus, F. Herrera, Mlsmote: Approaching imbalanced multilabel learning through synthetic instance generation, Knowledge-Based Systems 89 (2015) 385–397. [22] D. Krstinić, M. Braović, L. Šerić, D. Božić-Štulić, Multi-label classifier performance evaluation with confusion matrix, Computer Science & Information Technology 1 (2020).