<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>An entropy heuristic to optimize decision diagrams for index-driven search in biological graph databases</article-title>
      </title-group>
      <contrib-group>
        <aff id="aff0">
          <label>0</label>
          <institution>Nicola Licheri Elvio Amparore Vincenzo Bonnici University of Turin University of Turin University of Verona 10149</institution>
          ,
          <addr-line>Turin, Italy 10149, Turin Italy 37134, Verona</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <abstract>
        <p />
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>Graphs are a widely used structure for
knowledge representation. Their uses range from
biochemical to biomedical applications and are
recently involved in multi-omics analyses. A key
computational task regarding graphs is the search
of specific topologies contained in them. The
task is known to be NP-complete, thus indexing
techniques are applied for dealing with its
complexity. In particular, techniques exploiting paths
extracted from graphs have shown good
performances in terms of time requirements, but they
still su er because of the relatively large size of
the produced index. We applied decision
diagrams (DDs) as index data structure showing a
good reduction in the indexing size with respect
to other approaches. Nevertheless, the size of a
DD is dependent on its variable order. Because
the search of an optimal order is an NP-complete
task, variable order heuristics on DDs are applied
by exploiting domain-specific information. Here,
we propose a heuristic based on the information
content of the labeled paths. Tests on well-studied
biological benchmarks, which are an essential part
of multi-omics graphs, show that the resultant size
correlates with the information measure related to
the paths and that the chosen order allows to
effectively reduce the index size.</p>
    </sec>
    <sec id="sec-2">
      <title>Introduction</title>
      <p>Graphs are mathematical objects used to represent items,
also called vertices, and relations between them. In the
bioinformatics context, they are exploited to express
relationships at any biochemical, biological, and medical level.
For example, graphs can represent physical molecule
structure by expressing chemical bonds among atoms [Tri18].
At the cellular system level, graphs are instead applied
to represent biological actors, such as genes, proteins, or
RNAs, and their relations, such as physical interactions or
causal inference [HPRL08, DL05, BDCC+18]. Di erently,
in medical applications, graphs are exploited in decision
support systems to connect patient data with disease states
and treatments [XWJF19]. For what concerns integration
and analysis of multi-omics data, graphs are becoming
popular for integrating biomedical information with data
regarding multiple omics. In such a model, items compose
a heterogeneous set of biological and meta-biological
objects. Graphs of genetic interactions are enriched by
embedding their relationship with diseases, drugs, anatomic
phenotypes, biological functions, or cellular localization.
Then, multi-omics data linked to the genetic actors are
integrated. The result is a knowledge base that can be exploited
for drug repurposing, for prioritizing disease-associated
genes, or for patient classification and biomarker
identification [HB15, FWY+21, WSH+21].</p>
      <p>Among the computational tasks that can be performed
on top of such structures, the search of specific topologies
within biological graphs is one of the most challenging
problems. In particular, the subgraph isomorphism
problem is known to be NP-complete [Coo71]. In this
context, indexing of labeled graphs is a widely used technique
for dealing with such complexity. In fact, it provides a
good compromise between precision in filtering
unmatching parts of the graphs and time to compute such an
operation [LVCF21, GBB+13]. Practically, indexing approaches
store topological features, ranging from paths to frequent
substructures, in order to provide a fast pruning of the
labeled graphs or parts of them that do not contain the queried
topology. However, this kind of approaches may lead to
relatively large indexes, that can compromise performance.</p>
      <p>Recently, Decision Diagrams (DDs) have successfully
been applied for reducing the indexing size [LBBG21].
In fact, DDs are particularly e cient for detecting
common portions among the paths and storing e ciently them.
Their e ciency is known to be strongly a ected by the
ordering of variables describing a path: a good ordering
can substantially reduce the memory consumption and the
execution time to generate and encode the indexing.
Unfortunately, discovering the optimal variable ordering is
known to be NP-complete [BW96a]. Thus, various
heuristics depending on the specific application field for the
selection of (sub)optimal orderings were proposed in the
literature [FFM93].</p>
      <p>Starting from this observation, in this paper, we extend
the results presented in [LBBG21] by investigating how
the variable ordering may a ect the performance of such
an approach in terms of memory consumption. This task
was carried out by first proposing a new metric called Sum
of Entropies (SOE), which experimentally highlighted a
medium-to-strong anti-correlation value with respect to the
final size of the DD encoding the indexing. Then, the
metric was exploited as starting point to derive a sub-optimal
heuristic EntropyHeu that finds a variable order greedily
by optimizing the SOE metric.</p>
      <p>In detail, Section 2 introduces the concepts of indexing
of labeled graphs and DDs. Moreover, the section recalls
how DD can be e ciently exploited for encoding graph
indexing and we discuss how its e ciency is strongly
affected by the choice of a “reasonably good” variable order.</p>
      <p>In Section 3, the new metric SOE and the derived new
heuristic are formally introduced.</p>
      <p>Then, the e ectiveness of the new heuristic is assessed
in Section 4 reporting its performance on a set of
wellknown biological benchmarks. Finally, Section 5
concludes the paper.
2</p>
    </sec>
    <sec id="sec-3">
      <title>Background</title>
      <p>In this section, we firstly introduce the concepts of graphs,
labeled paths and path-based graph indexing. Then, the
Multi-Terminal Multi-way Decision Diagrams (MTMDDs)
are introduced as an e cient data structure to encode and
manipulate a set of paths with their occurrences. Finally,
we describe GRAPES-DD, a tool using MTMDDs for
effective searching in graphs.
2.1</p>
      <sec id="sec-3-1">
        <title>Graphs and paths as indexing features</title>
        <p>Formally, a graph is a pair G = (V; E) where V is the set
of vertices and E : V V is the set of relations, also called
edges. Given a set of labels , labeled graphs are enriched
with a function f : 7! V which maps each vertex to a
label in . The same label can be associated with di erent
vertices. A path p of length l is a vector (v1p; v2p; : : : ; vlp) such
that vip 2 V, for 1 i l, and (vip; vip+1) 2 E, for 1 i &lt; l.
A labeled path pˆ is obtained by mapping the vertices of a
path to their corresponding labels via the f function, thus
pˆ = ( f (v1p); f (v2p); : : : ; f (vlp)) = ( 1p; 2p; : : : ; lp).</p>
        <p>Given a query graph GQ = (VQ; EQ) and a target graph
GT (VT ; ET ), the subgraph isomorphism problem consists
in finding the occurrences of GQ within GT . An occurrence
is a mapping m : VQ 7! VT , thus between the vertices
of VQ and the vertices in VT , which preserves label
compatibility and graph topology. Label compatibility ensures
that, for each v 2 VQ, f (m(v)) = m(v). Topology
compatibility is ensured by asserting that, for each (u; v) 2 EQ,
(m(u); m(v)) 2 ET . The search space of the problem is at
most O(VQVT ), because each possible combination of
assignment of a target vertex to a query vertex must be
explored and verified. However, several techniques can be
used to reduce the search space by avoiding visiting the
unfeasible parts of it. One of these techniques consists in
extracting features of graph vertices for computing
compatibility between target and query vertices. Features of a
target graph are thus extracted and stored in an index with
the aim of reusing it for multiple queries. A key property
of indexes is the costs for building and querying them, as
well as the size they require in memory [Din17].</p>
        <p>In particular path-based indexing uses labeled paths as
features that describe the topological neighborhood of a
vertex. According to this, labeled paths are stored together
with the identifier of their starting vertex [GS02].</p>
        <p>In this way, a set of target vertices to be candidates to
match to a given query vertex v can e ciently be retrieved.
All labeled paths starting from v are extracted, then the
set of target vertices that are starting points of the same
labeled paths in the index are retrieved. After such a
filtering phase,all the exact occurrences are retrieved by a
subgraph isomorphism solver, such as VF2 [CFSV01] or
RI [BGP+13]. Because the number of paths can
exponentially grow on increasing their length, a maximum length is
usually set for the paths stored.
2.2</p>
      </sec>
      <sec id="sec-3-2">
        <title>Decision diagrams in a nutshell.</title>
        <p>Decision diagrams (DDs) are a family of data structures
proposed to encode and manipulate a set of values e
ciently. Multi-Terminal Multi-way Decision Diagram
(MTMDD) is a type of DD that can be e ectively exploited
to encode the function counting the occurrences of an
element into a multiset1, where elements are tuples with
format hv1; : : : ; vni with vi 2 N . Let O : N ! [1 : : : n]
be a bijective variable ordering function that assigns a
unique DD level in [1 : : : n] to each encoded variable.
Formally, an MTMDD is a rooted directed acyclic graph
ordered by O, where the first n levels represent the variables
hO(v1); : : : ; O(vn)i of the encoded tuple, and the terminal
level the number of occurrences of each tuple in the
multiset. Let us count the levels of an MTMDD in a bottom-up
fashion, so that the first level is above the terminal one and
the root node is at n-th level.</p>
        <p>The high storage e ciency of DDs is strongly
conditioned by the choice of a “reasonably good” variable
order, i.e. the assignment of the problem variables to the DD
levels. It is known [BW96b] that finding the optimal order
is a NP-complete problem. Some heuristics exist to help
searching at least sub-optimal orders [FFM93], but these
algorithms typically use problem-specific information.
However, to the best of our knowledge, no such heuristic is
currently available for reordering the variables of DDs
encoding biological graph databases.
2.3</p>
      </sec>
      <sec id="sec-3-3">
        <title>GRAPES-DD: a tool using Decision Diagrams for searching in graphs.</title>
        <p>In [LBBG21] we proposed a new version of GRAPES, a
path-based graph indexing tool [GBB+13], which exploits
the decision diagram (i.e MTMDD) to achieve a substantial
reduction of the memory footprint of the index graphs. The
goal was reached thanks to DD ability to e ciently handle
the presence of similar patterns in the indexed graph paths.</p>
        <p>Roughly speaking the GRAPES-DD workflow is
composed of three main phases: (1) the index building phase in
which MTMDD indexing the set of target graphs is
created by extracting all the labeled paths up to length lp,
(2) the filtering phase in which, given a query graph, the
set of target graphs is restricted to those subgraphs
potentially containing the query, and (3) the verification phase
in which subgraph isomorphism algorithm (i.e. VF2
algorithm [CFSV01] or RI [BGP+13]) is applied only on the
subset of candidate targets.
3</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Methods</title>
      <p>In this section, after introducing the GRAPES-DD
strategy, we focus on the formal definition of a new
suboptimal heuristic inspired to decision tree learning in
machine learning [Qui86].</p>
      <p>GRAPES-DD exploits an MTMDD with n variables:
n 1 variables vi, i = 1; : : : ; n 1 encoding the paths’
ith label; and vn encoding the identifier of the starting node
of the labeled path. Note that vn and the vi, 1 i &lt; n, are
di erent because they belongs to di erent domain spaces
1Multiset extends the concept of a set allowing for multiple instances
for each of its elements.
1: function EntropyHeu
2: O EntropyHeuLabels()
3: minSize
4: for i
5:
6:
7:
8:
9:
10:</p>
      <p>1
[1 : : : n] do
O0 InsertAt(O; fvn 7! ig)
DD0 BuildDD(O0)
if SizeDD(DD0) &lt; minS ize then
minSize SizeDD(DD0)</p>
      <p>O O0
return O
11: function EntropyHeuLabels
12: U fv1 : : : vn 1g
13: O fg
14: for i from 1 to n 1 do
15: null
16:
17:
18:
19:
20:
21:
22:
vsel
Hsel 1
for v0 2 U do:</p>
      <p>U0 U n fv0g
H0 Entropy(U0)
if H0 &gt; Hsel then
vsel v0</p>
      <p>Hsel H0</p>
      <p>Algorithm 1: Variable ordering selection heuristic
(graph vertices and labels, resp.) with largely di erent
domain cardinalities. Given a fixed position vn 7! k for the vn
variable, we define the stratum k as the subset of variable
orders fOgk sharing the fixed position for vn. We shall see
that the strata show significant clustering of the results in
Section 4.</p>
      <p>Each tuple x = hv1 : : : vni has an associated integer
multiplicity mult(x). Let X be the multiset of all the encoded
tuples x. Given a multiset X, let H(X) be the entropy of X,
defined according to the standard definition [Sha01]
H(X) =</p>
      <p>X p(x) log2 p(x); with: p(x) =
x2X
(1)</p>
      <p>Let U N be a subset of the problem’s variables. Let
x0 = x=U be a new tuple x0 obtained from a tuple x by
removing all the variables not in U. Let X=U be the
projection of the multiset X over the sole variables U, with
mult(x)
P mult(x0)
x02X
mult(x0) =
mult(x)</p>
      <p>(2)</p>
      <p>X
x2X; x0=x=U
the multiplicity of each tuple x0.</p>
      <p>Given a variable order O = fk1 : : : kng, we define the i-th
variable subset UO;i as the set of the first i variable indices
of O. We define the SOE metric for a variable order O as
SOE(O) =
n
X H X=UO;i
i=1
(3)
Research question R1: The size of the MTMDD (i.e. the
sum of its nodes and edges) correlates with the SOE
function. To test this hypothesis, we construct the MTMDD for
all the variable orders (which is factorial in the number n
of variables), and compute a correlation score between the
value (3) and the final MTMDD size.</p>
      <p>Unfortunately, finding the optimal MTMDD by
constructing all the permutations is not feasible in practice,
except for a limited number of encoded variables. Therefore,
to make the technique broadly applicable in a real world
context, we define a sub-optimal heuristic EntropyHeu that
searches a variable order O by applying a greedy
optimization the local entropy sum at every projection step i.</p>
      <p>The pseudo-code of EntropyHeu is shown in
Algorithm 1. The function EntropyHeu first computes the
ordering for the fv1 : : : vn 1g label variables. It then tries to
insert the identifier variable vn in all the positions, returning
the order O that minimizes the final DD size. The function
EntropyHeuLabels is the core heuristic algorithm,
performing the greedy search. It starts by defining an empty
variable order O and by taking into account the full set of
label variables U. At each outer iteration (lines 14-23), a
variable vsel 2 U is removed and assigned to position i in
the order O. The variable vsel is chosen to be the one that
maximizes the entropy given by the remaining set of
variables U n fvselg, namely:
vsel = arg max H(U n v)
v2U
(4)</p>
      <p>We assume that BuildDD(O) generates the MTMDD
for the projected variables subset with order O, and
Entropy(U) computes (1) on the projected multiset X=U.</p>
      <sec id="sec-4-1">
        <title>Research question R2: The function EntropyHeu se</title>
        <p>lects reasonably good variable orders, comparable with the
theoretical-optimal order derived by the SOE metric.</p>
        <p>In the next section, results are analysed to find answers
to the research questions R1 and R2.
4</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>Results</title>
      <p>We empirically tested our two research questions R1 and
R2 using a set of well-known biological benchmarks,
described hereafter. The first 5 benchmarks are
proteinprotein interaction (PPI) networks of 5 di erent species:
Caenorhabditis elegants (CE), Drosophila melanogaster
(DROSOFILA), Homo sapiens (HOMO), Mus musculus
(MUS) and Saccaromyces cerevisae (YEAST) [SFK+10].
Vertices are proteins and edges are predicted physical
interactions between them. For each species, di erent
thresholds on the accurateness of the prediction were applied,
ranging from 0.4, 0.5, 0.6 to 0.7. The retrieved graphs have
from 2k to 10k vertices, and from 2k to 89k edges, with
average degrees ranging from 1.3 to 15. PPIs belonging to
the same species were then merged into a single benchmark
to be indexed. The obtained benchmarks have from 13k to
21k vertices, and from 39k to 260k edges.</p>
      <p>We also included in the benchmark the standard
database for Antiviral Screen(AIDS) [ci]. It consists of 40k
chemical structures representing small molecules. Vertices
are atoms and edge are the chemical bounds linking them.
Vertex labels represent atomic elements, and there are a
total of 62 distinct elements. The average number of vertices
per graph is 44.98, and the average degree is 4.17.</p>
      <p>We conducted a set of experiments over the graph
databases described above. We indexed each database
using labeled paths up to length 4, so that each index
MTMDD is defined over 5 variables. Then, for each collection,
we obtain the size of the index MTMDD for all possible
variable orders.</p>
      <p>Figure 1 reports the results for the R1 question on the 6
benchmarks. Each dot represents one of the 120 possible
variable orders. Dots are colored by their respective
stratification induced by the level of the identifier variable. In
each stratum, a dashed line represents the trend of the
relation between the SOE metric and the final DD size. The
number indicates the value Spearman’s correlation
coefficient. We can observe that the metric has
medium-tostrong anticorrelation values in all stratum except for the
one where the identifier is positioned at the bottom, whose
sizes are almost insensible to the reordering of the label
variables. The figure shows a very positive result, because
it shows that a heuristic that maximizes the SOE metric has
an high chance of selecting a good order that minimizes the
DD size. Moreover, the cross on each stratum identifies the
ordering that would be selected by the proposed heuristic
EntropyHeu when fixing the position of the identifier
variable.</p>
      <p>Figure 2 shows the results for the R2 question on the
e ectiveness of the EntropyHeu heuristic on the 6
benchmarks. Relative DD sizes are shown on the y-axis, while
the x-axis has no meaning (it is only used for visualization
purposes to separate the dots). The green cross identifies
the relative DD size of the selected order, while the blue
bar identifies the average size that would be obtained by
taking an order randomly among the possible 120 orders.
We can observe that the greedy heuristic that follows the
metric SOE is actually capable of selecting almost-optimal
orders in all the tested cases, showing the e ectiveness of
the proposed information-based strategy.
5</p>
    </sec>
    <sec id="sec-6">
      <title>Discussion and conclusions</title>
      <p>In this paper, we extended the approach proposed
in [LBBG21] investigating how the MTMDD variable
order may a ect the performance of such an approach in
100
) 80
%
(
s
egd 60
e
+
sed 40
o
n
D
D 20
0
90
85
terms of memory consumption. To achieve this task we
first proposed the new metric SOE based on the
Shannon entropy which experimentally showed a
medium-tostrong anticorrelation with respect to the DD size
encoding the graph indexing. Then we developed the
suboptimal heuristic EntropyHeu inspired to the information
gain which is able to derive a variable order comparable
with the theoretical-optimal order derived by the SOE
metric. As a future extension, we will apply the EntropyHeu
heuristic on a bigger set of benchmarks coming from
different research fields and we will evaluate its performance
by increasing the length of labeled paths.</p>
      <sec id="sec-6-1">
        <title>Acknowledgements</title>
        <p>This work is partially supported by “Creation of a
computational framework to model and study West Nile Disease”
project supported by “Fondazione CRT”.</p>
        <p>DROSOFILA</p>
        <p>HOMO
1.0 1.5</p>
        <p>× 106
DD nodes + edges
2.5 5.0 7.5</p>
        <p>× 106
DD nodes + edges
[BDCC+18] Vincenzo Bonnici, Giorgio De Caro, Giorgio
Constantino, Sabino Liuni, Domenica D’Elia,
Nicola Bombieri, Flavio Licciulli, and
Rosalba Giugno. Arena-idb: a platform to build
human non-coding rna interaction networks.</p>
        <p>BMC bioinformatics, 19(10):25–38, 2018.
[BW96a]
[BW96b]
[CFSV01]
[ci]
[Coo71]</p>
        <p>Vincenzo Bonnici, Rosalba Giugno, Alfredo
Pulvirenti, Dennis Shasha, and Alfredo Ferro.</p>
        <p>A subgraph isomorphism algorithm and its
application to biochemical data. BMC
bioinformatics, 14(7):1–13, 2013.</p>
        <p>Beate Bollig and Ingo Wegener. Improving
the variable ordering of obdds is np-complete.</p>
        <p>IEEE Trans. Computers, 45(9):993–1002,
1996.</p>
        <p>Beate Bollig and Ingo Wegener.
Improving the variable ordering of OBDDs is
NPcomplete. IEEE Trans. Comp., 45(9):993–
1002, September 1996.</p>
        <p>Luigi Pietro Cordella, Pasquale Foggia, Carlo
Sansone, and Mario Vento. An improved
algorithm for matching large graphs. In 3rd
IAPR-TC15 workshop on graph-based
representations in pattern recognition, pages 149–
159, 2001.</p>
        <p>National cancer institute. National cancer
institute. Accessed: 2021 september 21.</p>
        <p>Stephen A Cook. The complexity of
theoremproving procedures. In Proceedings of the
[LBBG21]
[Qui86]</p>
        <p>David Luaces, Jose´ RR Viqueira, Jose´ M
Cotos, and Julia´n C Flores. E cient
access methods for very large distributed graph
databases. Information Sciences, 573:65–81,
2021.</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <source>third annual ACM symposium on Theory of computing</source>
          , pages
          <fpage>151</fpage>
          -
          <lpage>158</lpage>
          ,
          <year>1971</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <string-name>
            <given-names>Hamed</given-names>
            <surname>Dinari</surname>
          </string-name>
          .
          <article-title>A survey on graph queries processing: techniques and methods</article-title>
          .
          <source>International Journal of Computer Network and Information Security</source>
          ,
          <volume>9</volume>
          (
          <issue>4</issue>
          ):
          <fpage>48</fpage>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <given-names>Eric</given-names>
            <surname>Davidson</surname>
          </string-name>
          and
          <string-name>
            <given-names>Michael</given-names>
            <surname>Levin</surname>
          </string-name>
          .
          <article-title>Gene regulatory networks</article-title>
          .
          <source>Proceedings of the National Academy of Sciences</source>
          ,
          <volume>102</volume>
          (
          <issue>14</issue>
          ):
          <fpage>4935</fpage>
          -
          <lpage>4935</lpage>
          ,
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <given-names>Masahiro</given-names>
            <surname>Fujita</surname>
          </string-name>
          , Hisanori Fujisawa, and
          <string-name>
            <given-names>Yusuke</given-names>
            <surname>Matsunaga</surname>
          </string-name>
          .
          <article-title>Variable ordering algorithms for ordered binary decision diagrams and their evaluation</article-title>
          .
          <source>IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems</source>
          ,
          <volume>12</volume>
          (
          <issue>1</issue>
          ):
          <fpage>6</fpage>
          -
          <lpage>12</lpage>
          ,
          <year>1993</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [FWY+21]
          <string-name>
            <surname>Jiansong</surname>
            <given-names>Fang</given-names>
          </string-name>
          , Qihui Wu, Fei Ye, Chuipu Cai, Lvjie Xu, Yong Gu, Qi Wang,
          <string-name>
            <surname>Ai-lin</surname>
            <given-names>Liu</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wenjie Tan</surname>
          </string-name>
          , and
          <string-name>
            <surname>Guan-hua Du</surname>
          </string-name>
          .
          <article-title>Networkbased identification and experimental validation of drug candidates toward sars-cov-2 via targeting virus-host interactome</article-title>
          .
          <source>Frontiers in Genetics</source>
          ,
          <volume>12</volume>
          :
          <fpage>1590</fpage>
          ,
          <year>2021</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <given-names>Rosalba</given-names>
            <surname>Giugno</surname>
          </string-name>
          , Vincenzo Bonnici, Nicola Bombieri, Alfredo Pulvirenti, Alfredo Ferro, and
          <string-name>
            <given-names>Dennis</given-names>
            <surname>Shasha</surname>
          </string-name>
          .
          <article-title>Grapes: A software for parallel searching on biological graphs targeting multi-core architectures</article-title>
          .
          <source>PloS one</source>
          ,
          <volume>8</volume>
          (
          <issue>10</issue>
          ),
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <given-names>Rosalba</given-names>
            <surname>Giugno</surname>
          </string-name>
          and
          <string-name>
            <given-names>Dennis</given-names>
            <surname>Shasha</surname>
          </string-name>
          .
          <article-title>Graphgrep: A fast and universal method for querying graphs. In Object recognition supported by user interaction for service robots</article-title>
          , volume
          <volume>2</volume>
          , pages
          <fpage>112</fpage>
          -
          <lpage>115</lpage>
          . IEEE,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <string-name>
            <surname>Daniel S Himmelstein and Sergio E Baranzini</surname>
          </string-name>
          .
          <article-title>Heterogeneous network edge prediction: a data integration approach to prioritize disease-associated genes</article-title>
          .
          <source>PLoS computational biology</source>
          ,
          <volume>11</volume>
          (
          <issue>7</issue>
          ):e1004259,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <string-name>
            <given-names>Luke</given-names>
            <surname>Hakes</surname>
          </string-name>
          , John W Pinney, David L Robertson, and Simon C Lovell.
          <article-title>Proteinprotein interaction networks and biologywhat's the connection? Nature biotechnology</article-title>
          ,
          <volume>26</volume>
          (
          <issue>1</issue>
          ):
          <fpage>69</fpage>
          -
          <lpage>72</lpage>
          ,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <string-name>
            <given-names>Nicola</given-names>
            <surname>Licheri</surname>
          </string-name>
          , Vincenzo Bonnici, Marco Beccuti, and
          <string-name>
            <given-names>Rosalba</given-names>
            <surname>Giugno</surname>
          </string-name>
          .
          <article-title>GRAPES-DD: exploiting decision diagrams for index-driven search in biological graph databases</article-title>
          .
          <source>BMC bioinformatics</source>
          ,
          <volume>22</volume>
          (
          <issue>1</issue>
          ):
          <fpage>1</fpage>
          -
          <lpage>24</lpage>
          ,
          <year>2021</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <source>[Din17] [DL05] [FFM93] [GBB+13] [GS02] [HB15] [HPRL08] [SFK+10] [Sha01] [</source>
          <string-name>
            <surname>Tri18] J. Ross</surname>
          </string-name>
          <article-title>Quinlan. Induction of decision trees</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <source>Machine learning</source>
          ,
          <volume>1</volume>
          (
          <issue>1</issue>
          ):
          <fpage>81</fpage>
          -
          <lpage>106</lpage>
          ,
          <year>1986</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <string-name>
            <given-names>Damian</given-names>
            <surname>Szklarczyk</surname>
          </string-name>
          , Andrea Franceschini, Michael Kuhn, Milan Simonovic, Alexander Roth, Pablo Minguez, Tobias Doerks, Manuel Stark, Jean Muller,
          <string-name>
            <given-names>Peer</given-names>
            <surname>Bork</surname>
          </string-name>
          , et al.
          <article-title>The STRING database in 2011: functional interaction networks of proteins, globally integrated and scored</article-title>
          .
          <source>Nucleic acids research</source>
          ,
          <volume>39</volume>
          (
          <issue>suppl 1</issue>
          ):
          <fpage>D561</fpage>
          -
          <lpage>D568</lpage>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          <string-name>
            <given-names>Claude</given-names>
            <surname>Elwood Shannon</surname>
          </string-name>
          .
          <article-title>A mathematical theory of communication</article-title>
          .
          <source>ACM SIGMOBILE mobile computing and communications review</source>
          ,
          <volume>5</volume>
          (
          <issue>1</issue>
          ):
          <fpage>3</fpage>
          -
          <lpage>55</lpage>
          ,
          <year>2001</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          <string-name>
            <surname>Routledge</surname>
          </string-name>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [WSH+21]
          <string-name>
            <surname>Tongxin</surname>
            <given-names>Wang</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wei</surname>
            <given-names>Shao</given-names>
          </string-name>
          , Zhi Huang, Haixu Tang, Jie Zhang, Zhengming Ding, and
          <string-name>
            <given-names>Kun</given-names>
            <surname>Huang</surname>
          </string-name>
          .
          <article-title>Mogonet integrates multi-omics data using graph convolutional networks allowing patient classification and biomarker identification</article-title>
          .
          <source>Nature Communications</source>
          ,
          <volume>12</volume>
          (
          <issue>1</issue>
          ):
          <fpage>1</fpage>
          -
          <lpage>13</lpage>
          ,
          <year>2021</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [XWJF19]
          <string-name>
            <given-names>Xiayu</given-names>
            <surname>Xiang</surname>
          </string-name>
          , Zhongru Wang,
          <string-name>
            <surname>Yan Jia</surname>
            , and
            <given-names>Binxing</given-names>
          </string-name>
          <string-name>
            <surname>Fang</surname>
          </string-name>
          .
          <article-title>Knowledge graph-based clinical decision support system reasoning: a survey</article-title>
          .
          <source>In 2019 IEEE Fourth International Conference on Data Science in Cyberspace (DSC)</source>
          , pages
          <fpage>373</fpage>
          -
          <lpage>380</lpage>
          . IEEE,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>