<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Generating Knowledge Graph Based Explanations for Drug Repurposing Predictions</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Elif Ozkan</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Remzi Celebi</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Arif Yilmaz</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Vincent Emonet</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Michel Dumontier</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Institue of Data Science, Maastricht University</institution>
          ,
          <addr-line>Maastricht</addr-line>
          ,
          <country country="NL">The Netherlands</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Over the past years, computer assisted drug repurposing methods have started to gain more attention as they ofer a faster and a more efective way to treat many diseases. While these methods are quite promising in terms of power of prediction, the hesitation regarding the use of these methods in practice still remains due to their highly complex working mechanisms, which limits their interpretability. Explainable Artificial Intelligence (XAI), which takes transparency, interpretability, informativeness as its main foundations, could address the limitations of the black-box models. In this context, Knowledge Graphs (KGs) could leverage the explanations provided to the user in the biomedical domain, as they are capable of represent relations between the entities in a semantically consistent way. Knowledge Graphs have the potential to generate graph-based representations, while providing the context, which make it easily interpretable by humans. In this paper, we propose an approach, which is a KG based explainable AI framework in the field of drug repurposing as an extension of the PREDICT Method. The approach is centered on generating similarity-based explanations by extracting the relevant paths from the input, which consists of a disease and a predicted drug for the treatment of the disease. To demonstrate the utility of this approach, we demonstrate how the graphical operations used in the KG could be used to generate plausible explanations, by conducting a use case on Alzheimer Disease. Our findings suggest that the utilization of biomedical KGs and this approach has a great potential to provide transparent explanations as it is able to illustrate the relations between drug, disease entities which are quite relevant to the target input. Application of this approach to the drug repurposing and to other similar domains, could be helpful to overcome the limitations caused by the black-box nature of the computational drug repurposing models and could be a powerful tool to enhance the understanding of decision making process of models and simplify scientific communication among domain experts and computer scientists.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Knowledge Graph</kwd>
        <kwd>Explainable AI</kwd>
        <kwd>XAI</kwd>
        <kwd>drug repurposing</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        The advancements in the field of Artificial Intelligence (AI) have been successfully utilized
in the computer-assisted biomedical tasks in the past few years. AI and Machine learning
methods applied in this field bears significant promise for drug discovery and repurposing as
they significantly accelerate and ofer new alternatives for the process of treatment [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Drug
companies and researchers started to pay more attention to the computational drug repurposing
methods in the recent years in order to find a faster and efective path for the treatment of
COVID-19 [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. Although the performance of many computational drug repurposing frameworks
are quite promising such as the PREDICT method which utilizes similarity search based on
the principle of "guilt by association" [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ], their inner working mechanisms are still seen as a
black-box as the way these frameworks make decisions are not entirely evident [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. Hence, this
limitation restricts the full adoption process of computational drug repurposing methods by
institutions.
      </p>
      <p>
        Explainable AI could play a critical role in addressing to the limitations which are caused
by the highly complex, non-transparent nature of the computational drug repurposing models
and help us understand and interpret the underlying models, in order to mitigate the lack
of interpretability of certain machine learning models and to augment human reasoning and
decision-making. In the biomedical context, alongside the natively interpretable models such
as Random Forests or Decision Trees, Knowledge Graphs (KG), which are semantically rich,
interlinking data structures that formally represents the relationship between diferent entities,
have started to be leveraged [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].Knowledge Graphs (KGs), graph-based representations of
knowledge, are capable of encoding the complex in the form of structured statement, in the
way that it is human interpretable [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ].
      </p>
      <p>
        In this paper, our approach, which is a knowledge graph based explainable AI approach
in the specific context of drug repurposing, is proposed. The proposed approach involves
extracting relevant subgraph, given a drug and a disease pair, in order to provide
similaritybased explanations in the form of a knowledge graph as an explainable extension of the PREDICT
method [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] which is based upon the principle of "Guilt by Association" and easily adaptable
to the knowledge graph structure. This method is then evaluated by conducting a case study
on drug candidates, which could potentially treat Alzheimer disease. It was made accessible
to the user as a branch of the OpenPredict [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] model, which is the concrete implementation
of a drug-repositioning framework. The outline of this paper is as follows. A more in depth
information about the related work done by other researches is provided in Section 2, and it is
followed by the methodology which is adopted in this research in Section 3. The results and
discussion are presented in Sections 4 and 5 respectively.
      </p>
    </sec>
    <sec id="sec-2">
      <title>2. Related Work</title>
      <p>AI-based drug repurposing is defined as the identification, prediction and evaluation of new use
cases and indications for existing and approved drugs using computational methods, such as
Machine Learning and Deep Learning. One of the most efective computational approaches utilized
in the context of drug repurposing is consideration similarities between entities, specifically by
analyzing the drug and disease based similarities, as well as the their combined similarities.</p>
      <p>
        In this sense, the PREDICT method presents a framework that provides predictions on novel
associations between desired drugs and diseases [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. The framework is mainly based on the
‘Guilt by Association’ (GBA) approach which was first proposed by Chiang and Butte, which
involves the measurements of similarities among the known drug and disease to drug-disease
pairs, given a target query drug and disease. The known associations between entities and the
later formed associations are used as features and then are fed into a classification algorithm in
order to provide a final prediction. PREDICT is a quite efective framework as it enables the
incorporation of additional features related to similarity between drugs and diseases. However,
the usability of this efective framework itself holds some limitations as the underlying features
and reasoning behind the final predictions made by the framework, since both predictive and
interpretative features are isolated by the complex classification algorithms [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ].
      </p>
      <p>
        Augmentation of Knowledge Graphs into AI systems in the biomedical field, specifically for
the drug discovery and repurposing tasks, allows for generating explanations of the system by
providing informed and labeled visualizations by converting knowledge formalization rules and
logic into a form, which is more suitable for human comprehension, to the user. For instance,
Edwards et. al [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. in their study on Explainable Biomedical recommendations via reinforcement
learning, propose a neuro-symbolic approach which involves the application of multi-hop neural
driven recommendation to complex biomedical knowledge graphs . They conclude that such KG
based approaches has a great potential to generate explanations and improve the performance
of the black-box methods.
      </p>
      <p>
        Similarly, Liu et.al. [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] in their study regarding Neural Multi-Hop Reasoning with logical
rules on Biomedical Knowledge graphs propose a novel neuro-symbolic approach PoLo
(PolicyGuided Walks With Logical Rules) that leverages the interpretability and the structure of
Knowledge Graphs to conduct guided policy walks. The experimental findings that they have
found for this specific approach based on KGs, on the use case of drug repurposing of the
novel disease COVID-19, demonstrated that path-based reasoning methods outperform existing
black-box methods on the drug repurposing task as well as providing a natural transparency
mechanism which makes this approach more transparent to the existing black-box methods.
      </p>
      <p>
        Moreover, Wang et. al. [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ], in their study on discovering the potential reactions of antitumor
drugs adopted a Tumor-Biolink knowledge graph (TBKG) based method which is comprised
of four main steps including (1) graph building, (2) reaction discovery, (3) graph verification,
(4)clinical validation, and in which they explored the relations among tumors, biomarkers and
drugs. It is concluded in the study that the generated knowledge graphs could have successfully
been interpreted and validated by the domain experts and therefore, their approach is capable
of providing explanations and transparency of their reaction discovery process.
      </p>
      <p>
        Inherently explainable predictive models such as Decision Trees and Classification Rules as
well as biomedical Knowledge Graphs are utilized for the drug interaction tasks to bring its
explainability to a higher level [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]. Bresso et.al. utilize these simple classification methods
to develop an explainable AI system for investigating drug interactions and they have used
Decision Trees to make predictions from the generated Knowledge Graphs. Along with the
quantitative performance metrics they also conduct qualitative experiments with the domain
experts for explainability, similarly to the clinical validation step in Wang et. al’s study. It
demonstrates that the synthesis of knowledge graphs with inherently explainable prediction
methods provide explainable and comprehensible models to explore activity reactions of drugs.
      </p>
    </sec>
    <sec id="sec-3">
      <title>3. Method</title>
      <p>In order to provide an interpretable drug repurposing framework, we developed a knowledge
graph based pipeline. The primary purpose of this pipeline is to generate a knowledge graph
which indicates two types of relationship; similar_to which is the similarity between the
drugdrug and disease-disease pairs, and the treats relationship between a drug and a disease.</p>
      <p>The base information regarding the similarity between drug-drug &amp; disease-disease pairs and
the relations between drug-disease pairs are curated into a dataset which includes the vector
embeddings of 593 drugs obtained from DrugBank and 313 associated diseases from Online
Mendelian Inheritance in Man, (OMIM) databases.</p>
      <p>The overall strategy includes identifying a set of ranked paths through the generated
Knowledge Graph that provide plausible explanations for a predicted drug indication based on their
similarities to known drug-disease pairs. The generated explanation is based on a input which
is composed of a desired drug-disease pair. Our approach generates a KG of ranked paths in
three steps : Path Generation, Path Ranking and the Generation of Explanation Graph.</p>
      <p>Path Generation step involves creating a set of paths based on a given input. Each path has
n-length, consisting of two types of relations; similar_to between drug-drug and treats between
drug-disease entities. The level of similarity between two entities, 1 and 2, is obtained by
taking the cosine similarity  of their vector embeddings which is given by :
‖1‖ ‖2‖</p>
      <p>As the paths of length  &gt; 3 are less biochemically relevant and the increasing path lengths
become increasingly dificult to understand, 0, 1 and 2-hop relation paths are used to connect
the drug and the disease.</p>
      <p>Five cases are taken into account during the path formation. As Figure 1 demonstrates,
the treats relation among the known drug-disease pairs are retrieved immediately
(drug2disease1). For the unknown drug-disease pairs, for instance drug1-disease1, the structural
similarity between the homotypic times should also be considered in order to form an edge
which represents the relation treats. In this specific case, drug1 is similar to drug2, which is
known to be indicated for the treatment of disease1, and similarly, it is known that disease1
is similar to disease2. Therefore, it is possible to form an edge between the entities drug1 and
disease1. However, the plausibility of this edge depends on the similarity scores that the other
existing edges have, and in practice, the number of paths between two predicted entities is quite
high due to the size of the data sets. Therefore, an additional graphical action is needed to rank
the weight of the formed paths and select the most relevant paths to form the treats relation,
based on an input drug-disease pair.</p>
      <p>The path ranking operation is achieved by the adoption of principle of parsimony, which
suggests that explanations with simpler and shorter paths are more relevant compared to the
paths that are longer, and might have relatively less indirect information. Each path formed in
the previous phase are ranked according to their assigned weight. The weight   , assigned to
each path   is computed by :
(1)
(2)</p>
      <p>where the edge weight  is set to 1 in case of a relation of type treats, and to (1 − )
in the edge connects two entities of the same type. Once a weight is assigned to each path in
the KG, they are ranked by their weights in an ascending order. As a last step, top  weighted
paths with highest weights are included in the explanation graph. Moreover, to ensure the
relevance and significance of the included entities in the final explanation, as well as simplifying
the graphs to provide more readable explanations, we introduce an additional binary variable
"min_similarity_threshold", which restricts the amount of included entities further, according to
a desired similarity threshold. The restriction process is achieved by taking the entities, which
are in the top  percentile of all entities, in terms of their similarity scores to the target. If there
are no entities which satisfy a certain similarity threshold, i.e, the similarity between the found
entities are too weak, an empty explanation graph is returned.</p>
    </sec>
    <sec id="sec-4">
      <title>4. Use case</title>
      <p>
        In order to observe the efectiveness of the generated explanation through the Knowledge
Graphs, a case study for Alzheimer Disease (OMIM:104300), which is indicated to carry similar
characteristics with diseases such as dementia and Parkinson’s disease [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ], was conducted
through the OpenPredict API, using the PREDICT dataset and model.
      </p>
      <p>The drug Amandatine is suggested by the PREDICT model as a potential treatment for
Alzheimer’s disease. In order to understand the relation between the drug Amandatine and
Alzheimer Disease, we use this pair as an input to the pipeline as shown in Figure 2.</p>
      <p>Considering the above input, the generated explanation graph, the pipeline first extracts the
individual edges, then augments them into paths, as illustrated in Figure 3.</p>
      <p>
        Amandatine is structurally similar to Donepezil, and Rivastigmine which are directly
indicated for the treatment of Alzheimer Disease. It is also shown to be similar to drugs
Carbidopa, Zonisamide and Haloperidol which treat Parkinson’s Disease, Epilepsy and Dementia
&amp; Schizophrenia , which are similar to Alzheimer, respectively. The generated paths are then
merged into a single knowledge graph, displaying the relationships between the drug-disease
entities as a complete semantic network as shown in Figure 4. The resulting explanation graph
provides plausible explanations as the entities included in the graph are closely related to the
target pair (Amandatine-Alzheimer). For instance, many studies have shown that there is strong
evidence that Parkinson’s Disease and Alzheimer Disease have overlapping similarities in terms
of clinical and neuropathologic features [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] and Carbidopa is indicated for treatment of early
symptoms of Alzheimer [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ] [16].
      </p>
      <p>The min_similarity_threshold is taken as 10 in this case study, considering the availability of
the instances in the data. In order to observe whether the variable min_similarity_threshold
causes loss of information in this specific case study, an alternative explanation is generated
without taking min_similarity_threshold into consideration. The paths formed, that are not
restricted by a certain threshold, turned out to be indeed more populated with entities, as seen
in Figure 5. The result is quite interesting as in this example, min_similarity_threshold indeed
reduced the size of the explanation graph in a way that the entities included in the graph are
more relevant.In Figure 5, Clonidine is shown to be similar to Amandatine. Clonidine is known
to be indicated for the treatment of Gilles La Tourette Syndrome, which is a disease mostly
related to neuropsychiatric movement and typically starts developing from childhood [17]. For
the entity Carbidopa, in comparison with the explanation graph restricted with the similarity
threshold, Multiple Sclerosis, a neuroskeletal disorder [18], is also shown to be similar to the</p>
      <p>Alzheimer Disease. In this context, it is possible to say that these diseases are relatively less
related to Alzheimer, therefore the utilization of min_similarity_threshold enabled excluding
less relevant entities.</p>
    </sec>
    <sec id="sec-5">
      <title>5. Discussion</title>
      <p>The results that have been obtained from the conducted case study demonstrated that the
building semantic connections using Knowledge Graphs could provide meaningful and efective
explanations in the biomedical, specifically drug repurposing, domain. Although this study is
mainly focused on drug repurposing domain, it is intended to show that the proposed pipeline,
which takes the Knowledge Graph structure as its main baseline, is a powerful pipeline to
generate plausible explanations.</p>
      <p>Although, the literary sources and previous studies were taken as a basis to qualitatively
evaluate how efective the proposed pipeline is, in generating explanations, evaluation of
conducted case studies by domain experts could be a further and a more reliable justification.
In this sense, this comes as the main limitation in the evaluation process. Furthermore, another
limitation in terms of applicability in other domains and cases could be that the path ranking
process might be problematic as an edge could have a dominantly large weight, especially, if
the weights are not normalized. Therefore, it would be sensible to consider the alternative path
ranking strategies, such as finding the shortest path in the graph, as well as the one used in this
pipeline. Exclusion of entities with lower similarities through determined thresholds simplify
the outputted knowledge graphs, allowing for easier interpretations by the domain experts. In
order to prevent the possible hindrances, knowledge graphs generated using diferent thresholds
could be observed.</p>
      <p>
        Computational drug repurposing methods are still not fully adopted by the institutions due
to the lack of explainability behind the sophisticated methods [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. In this sense, utilization of
Knowledge Graphs could help domain experts to augment the explanations provided with their
expertise and reasoning to gain more insight on the studied subjects. It could also encourage
considering the drug-disease relations that have not been studied yet as the Knowledge Graph
explanation visualizes not only the entities related to the target but also the entities related to
the intermediate entities along the paths.
      </p>
      <p>This method has also drawn some challenges that are still yet to be tackled. For instance,
considering more complex relations such as the interaction between the target and the
intermediate drugs may foster obtaining a deeper level of understanding of the treatment potential of
the target disease by the prospective drug. Another challenge might be the augmentation of
new drug and disease information to the pipeline. The vector embedding conversion is easily
performed as a reproducible strategy is adopted, however augmentation of large information
could cause redundancy and sometimes loss of information due to the larger filtering and
simplification which would be performed in parallel with the increasing search space.</p>
      <p>Overall, the case study conducted on the proposed pipeline is an indicator of the promising
potential of Knowledge Graphs, and semantic operations that come with it, in providing
transparent and understandable explanations in the biomedical domain, and the challenges that it
introduces are an incentive to enhance KG-Based Explainable AI methods in the domain.</p>
    </sec>
    <sec id="sec-6">
      <title>6. Conclusion</title>
      <p>In this work, a knowledge graph based explanation framework is proposed for drug repositioning
task. The proposed approach could be utilized to provide explanations and improve the main
principles of Explainable AI, by providing accountability, reliability and transparency regarding
the decisions that were made through computational methods.</p>
      <p>The proposed framework took the PREDICT method as a baseline in providing the
explanations. This way, by enhancing the Guilt by Association strategy that PREDICT method uses by
augmenting KGs, along with several graphical operations, the relations between the related
entities to the given drug-disease pairs are demonstrated as transparent explanations to the
users in the form of structured predicates.
administration of donepezil hcl and levodopa/carbidopa in patients with parkinson’s
disease: assessment of pharmacokinetic changes and safety following multiple oral doses,
British journal of clinical pharmacology 58 (2004) 41–49.
[16] P. Chopade, N. Chopade, Z. Zhao, S. Mitragotri, R. Liao, V. Chandran Suja, Alzheimer’s
and parkinson’s disease therapies in the clinic, Bioengineering &amp; Translational Medicine
(2022) e10367.
[17] E. Jakubovski, K. R. Müller-Vahl, Gilles de la tourette syndrome: symptoms, causes and
therapy, Psychotherapie, Psychosomatik, Medizinische Psychologie 67 (2017) 252–268.
doi:https://doi.org/10.1186/s12911-021-01518-6.
[18] N. Ghasemi, S. Razavi, E. Nikzad, Multiple sclerosis: pathogenesis, symptoms, diagnoses
and cell-based therapy, Cell Journal (Yakhteh) 19 (2017) 1.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>J.</given-names>
            <surname>Jiménez-Luna</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Grisoni</surname>
          </string-name>
          , G. Schneider,
          <article-title>Drug discovery with explainable artificial intelligence</article-title>
          ,
          <source>Nature Machine Intelligence</source>
          <volume>2</volume>
          (
          <year>2020</year>
          )
          <fpage>573</fpage>
          -
          <lpage>584</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>S.</given-names>
            <surname>Ekins</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Mottin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P. R.</given-names>
            <surname>Ramos</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B. K.</given-names>
            <surname>Sousa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B. J.</given-names>
            <surname>Neves</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. H.</given-names>
            <surname>Foil</surname>
          </string-name>
          ,
          <string-name>
            <surname>K. M. Zorn</surname>
            ,
            <given-names>R. C.</given-names>
          </string-name>
          <string-name>
            <surname>Braga</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Cofee</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <string-name>
            <surname>Southan</surname>
          </string-name>
          , et al.,
          <article-title>Déjà vu: stimulating open drug discovery for sars-cov-2</article-title>
          , Drug discovery today
          <volume>25</volume>
          (
          <year>2020</year>
          )
          <fpage>928</fpage>
          -
          <lpage>941</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>A.</given-names>
            <surname>Gottlieb</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G. Y.</given-names>
            <surname>Stein</surname>
          </string-name>
          , E. Ruppin,
          <string-name>
            <given-names>R.</given-names>
            <surname>Sharan</surname>
          </string-name>
          ,
          <article-title>Predict: a method for inferring novel drug indications with application to personalized medicine</article-title>
          ,
          <source>Molecular systems biology 7</source>
          (
          <year>2011</year>
          )
          <fpage>496</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>W. J.</given-names>
            <surname>Murdoch</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Singh</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Kumbier</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Abbasi-Asl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Yu</surname>
          </string-name>
          , Definitions, methods, and
          <article-title>applications in interpretable machine learning</article-title>
          ,
          <source>Proceedings of the National Academy of Sciences</source>
          <volume>116</volume>
          (
          <year>2019</year>
          )
          <fpage>22071</fpage>
          -
          <lpage>22080</lpage>
          . doi:https://doi.org/10.1073/pnas.1900654116.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>A.</given-names>
            <surname>Callahan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. H.</given-names>
            <surname>Shah</surname>
          </string-name>
          ,
          <article-title>Machine learning in healthcare</article-title>
          ,
          <source>in: Key Advances in Clinical Informatics, Elsevier</source>
          ,
          <year>2017</year>
          , pp.
          <fpage>279</fpage>
          -
          <lpage>291</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>I.</given-names>
            <surname>Tiddi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Schlobach</surname>
          </string-name>
          ,
          <article-title>Knowledge graphs as tools for explainable machine learning: A survey</article-title>
          ,
          <source>Artificial Intelligence</source>
          <volume>302</volume>
          (
          <year>2022</year>
          )
          <fpage>103627</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>R.</given-names>
            <surname>Celebi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. R.</given-names>
            <surname>Moreira</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. A.</given-names>
            <surname>Hassan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Ayyar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Ridder</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Kuhn</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Dumontier</surname>
          </string-name>
          ,
          <article-title>Towards fair protocols and workflows: the openpredict use case</article-title>
          ,
          <source>PeerJ computer science 6</source>
          (
          <year>2020</year>
          )
          <article-title>e281</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>E.</given-names>
            <surname>Bresso</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Monnin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Bousquet</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.-E.</given-names>
            <surname>Calvier</surname>
          </string-name>
          , N.-
          <string-name>
            <surname>C. Ndiaye</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          <string-name>
            <surname>Petitpain</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <article-title>SmaïlTabbone, A</article-title>
          . Coulet,
          <article-title>Investigating adr mechanisms with explainable ai: a feasibility study with knowledge graph mining, BMC medical informatics and decision making 21 (</article-title>
          <year>2021</year>
          )
          <fpage>1</fpage>
          -
          <lpage>14</lpage>
          . doi:https://doi.org/10.1186/s12911-021-01518-6.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>G.</given-names>
            <surname>Edwards</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Nilsson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Rozemberczki</surname>
          </string-name>
          , E. Papa,
          <article-title>Explainable biomedical recommendations via reinforcement learning reasoning on knowledge graphs, 2021</article-title>
          . doi:
          <volume>10</volume>
          .48550/ARXIV. 2111.10625.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Liu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Hildebrandt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Joblin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Ringsquandl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Raissouni</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Tresp</surname>
          </string-name>
          ,
          <article-title>Neural multi-hop reasoning with logical rules on biomedical knowledge graphs</article-title>
          ,
          <source>in: European Semantic Web Conference</source>
          , Springer,
          <year>2021</year>
          , pp.
          <fpage>375</fpage>
          -
          <lpage>391</lpage>
          . doi:https://doi.org/10.48550/arXiv. 2103.10367.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>M.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Ma</surname>
          </string-name>
          , J. Si,
          <string-name>
            <given-names>H.</given-names>
            <surname>Tang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Li</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Ouyang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Gong</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Tang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>He</surname>
          </string-name>
          , et al.,
          <article-title>Adverse drug reaction discovery using a tumor-biomarker knowledge graph</article-title>
          ,
          <source>Frontiers in genetics 11</source>
          (
          <year>2021</year>
          )
          <article-title>625659</article-title>
          . doi:
          <volume>10</volume>
          .3389/fgene.
          <year>2020</year>
          .
          <volume>625659</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>E.</given-names>
            <surname>Bresso</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Monnin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Bousquet</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.-E.</given-names>
            <surname>Calvier</surname>
          </string-name>
          , N.-
          <string-name>
            <surname>C. Ndiaye</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          <string-name>
            <surname>Petitpain</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <article-title>SmaïlTabbone, A</article-title>
          . Coulet,
          <article-title>Investigating adr mechanisms with explainable ai: a feasibility study with knowledge graph mining, BMC medical informatics and decision making 21 (</article-title>
          <year>2021</year>
          )
          <fpage>1</fpage>
          -
          <lpage>14</lpage>
          . doi:https://doi.org/10.1186/s12911-021-01518-6.
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>C.</given-names>
            <surname>Reitz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Brayne</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Mayeux</surname>
          </string-name>
          , Epidemiology of alzheimer disease,
          <source>Nature Reviews Neurology</source>
          <volume>7</volume>
          (
          <year>2011</year>
          )
          <fpage>137</fpage>
          -
          <lpage>152</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>D. P.</given-names>
            <surname>Perl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. O.</given-names>
            <surname>Warren</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Calne</surname>
          </string-name>
          ,
          <article-title>Alzheimer's disease and parkinson's disease: distinct entities or extremes of a spectrum of neurodegeneration?</article-title>
          ,
          <source>Annals of neurology 44</source>
          (
          <year>1998</year>
          )
          <fpage>S19</fpage>
          -
          <lpage>S31</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>C. S.</given-names>
            <surname>Okereke</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Kirby</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Kumar</surname>
          </string-name>
          ,
          <string-name>
            <surname>E. I. Cullen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. D.</given-names>
            <surname>Pratt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W. A.</given-names>
            <surname>Hahne</surname>
          </string-name>
          , Concurrent
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>