<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Ontology‐based  expansion  of  virtual  gene  panels  to  improve  diagnostic efficiency for rare genetic diseases </article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Jae-Moon Shin</string-name>
          <email>shin@dbcls.rois.ac.jp</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Toyofumi Fujiwara</string-name>
          <email>fujiwara@dbcls.rois.ac.jp</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Atsuko Yamaguchi</string-name>
          <email>atsuko@tcu.ac.jp</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Database Center for Life Science</institution>
          ,
          <addr-line>Kashiwa, Chiba</addr-line>
          ,
          <country country="JP">Japan</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Tokyo City University</institution>
          ,
          <addr-line>Setagaya, Tokyo</addr-line>
          ,
          <country country="JP">Japan</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>   Recently, to diagnose rare genetic diseases, Virtual Gene Panels (VGP) comprising sets of disease-related causal genes have been used to interpret candidate genes detected through whole-genome sequencing and whole-exome sequencing. In a pilot study of the UK 100,000 Genome Project for diagnosing rare diseases, VGPs from PanelApp software were used to filter candidate genes instead of manual interpretation to improve diagnostic efficiency. However, in about 50% of cases, the PanelApp VGPs also filtered out disease-causing genes. Here, we propose several methods using the hierarchical structure of Mondo disease ontology to design VGPs that avoid filtering disease-causing genes with high probability. To determine the best VGP design method among those proposed, we evaluated our methods with various parameters via computational experiments with an evaluation data set composed of 74 patients. Our results show that our proposed method can contribute significantly to automatically filtering candidate genes as well as shortening the interpretation time for diagnosing rare diseases.</p>
      </abstract>
      <kwd-group>
        <kwd>1  Rare disease</kwd>
        <kwd>Ontology</kwd>
        <kwd>Genetic testing</kwd>
        <kwd>Virtual gene panel</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction </title>
      <p>
        Rare diseases are those with a very low prevalence rate, of which there are estimated to be about
10,000 in the world [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Worldwide, the total number of patients with these diseases is estimated to be
more than 400 million, many of whom have been undiagnosed for years [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. Approximately 80% of
rare diseases are believed to have a genetic origin [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. The advancement of next-generation sequencing
(NGS) technology has decreased both the cost and time of decoding genetic sequences. As a result,
using NGS for genetic testing is considered a powerful tool for diagnosing rare diseases [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. However,
diagnosing rare diseases using NGS involves a labor-intensive process of searching through literature
to identify a single candidate disease-causing gene that can best explain a patient's symptoms. This
manual interpretation process can take hours, even for trained experts [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ].
      </p>
      <p>
        Recently, to reduce the time needed for manual search, a Virtual Gene Panel (VGP) comprising a
set of disease-related causal genes has been used to filter candidates. By taking the intersection of
candidate genes and the set of genes of the VGP corresponding to an initial diagnosis, the number of
potential candidates can be reduced significantly. As part of the UK 100,000 Genomes Project, a pilot
study for rare disease diagnosis, the PanelApp software, including 332 VGPs, was developed and
applied to the manual interpretation of whole-genome sequencing results [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. However, they failed to
effectively filter out candidates in 50% of cases [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ].
      </p>
      <p>There are two reasons why disease-causing genes were not effectively filtered in half of the pilot
study cases. First, there were only 332 VGPs included in PanelApp. VGPs in PanelApp are designated
through manual curation, which takes time to design. Therefore, there were not always appropriate
VGPs corresponding to initial diagnoses. Furthermore, if the initial diagnosis is even slightly wrong,
the disease-causing gene may not be included in the VGP.</p>
      <p>
        In this study, we propose a method to design VGPs automatically using a knowledge graph.
Additionally, we propose several methods to expand VGPs using the hierarchical structure of Mondo
disease ontology (Mondo) [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. The basic premise of the expanded VGP design is adding genes
associated with the superclass or sibling of the initially diagnosed disease in Mondo into an original set
of genes. We evaluated the diagnostic efficiency of these methods using 74 cases of rare genetic
diseases. Our results show that the best method succeeded in automatically filtering candidate genes
while still containing the disease-causing genes with high probabilities.
      </p>
    </sec>
    <sec id="sec-2">
      <title>2. Methods  2.1.</title>
    </sec>
    <sec id="sec-3">
      <title>Knowledge graph </title>
      <p>
        We constructed a knowledge graph based on the Resource Description Framework to design VGPs
automatically (https://integbio.jp/rdf/dataset/pubcasefinder), using existing classes for interoperability
with other knowledge graphs. All diseases and genes were defined as instances of "med2rdf:Disease"
and "med2rdf:Gene" classes, respectively, by Med2RDF-ontology (http://med2rdf.org/) [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ].
Genedisease associations were defined as instances of the "sio:SIO_000983" class defined by Semantics
Science Integrated Ontology [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ].
      </p>
      <p>We collected gene-disease associations from the following three data sources.</p>
      <p>MIM2Gene (https://ftp.ncbi.nlm.nih.gov/gene/DATA/mim2gene_medgen).</p>
      <p>Associations between MIM</p>
      <p>Numbers with type "phenotype" and NCBI Gene IDs were
extracted.</p>
      <p>OrphaData (http://www.orphadata.org/data/xml/en_product6.xml).</p>
      <p>Associations between OrphaCodes and Gene symbols were extracted.</p>
      <p>
        GenCC (https://search.thegencc.org/download) [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ].
      </p>
      <p>Associations between disease IDs including MIM Numbers, OrphaCodes, and Mondo IDs and
HGNC IDs were extracted.</p>
      <sec id="sec-3-1">
        <title>Then, we define</title>
        <p>∪ ∪
2.2.</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Development of VGPs </title>
      <p>
        We connected to MIM numbers in Online Mendelian Inheritance in Man [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] and OrphaCodes in
Orphanet [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] to Mondo IDs using the "equivalentTo" classes in Mondo. In our constructed knowledge
graph, genes are connected for a disease ID in Mondo through med2rdf:Disease and sio:SIO_000983.
For a disease ID d in Mondo, we denote a set of genes by  
connected in the knowledge graph.
′
.
      </p>
      <p>Initially, when a disease, d, is diagnosed, G(d) can be regarded as a VGP for d. Original set (OR):
For a given disease d, output a set G(d) of genes. Using OR, we can obtain as many VGPs as the number
of Mondo IDs. However, like PanelApp, OR also has the problem of filtering out the disease-causing
gene if the initial diagnosis is slightly different. To overcome this problem, we propose an expansion
of the VGP designed by OR.</p>
      <p>Mondo forms a directed acyclic graph, in which nodes represent diseases and edges represent
"rdfs:subClassOf" relationships between diseases. Using this structure, we expanded the VGPs
designed by OR by adding genes of superclasses in Mondo.</p>
      <p>Here we describe four simple rolling-up methods to expand VGP using rdfs:subClassOf. One class
up
for
all
paths
(1UAP):</p>
      <p>For
disease
d,
compute
a
set</p>
      <p>of</p>
      <p>Two classes up for all paths (2UAP): For disease d, compute a set of
genes 
′,  rdfs:subClassOf ′.</p>
      <p>|  ∈ 
set of genes  
′,  rdfs:subClassOf ′.
minimum paths (1UMP): For disease d, compute a set of genes  

,  rdfs:subClassOf ′</p>
      <p>′ rdfs:subClassOf ′′.</p>
      <p>Two classes up for minimum paths (2UMP): For disease d, compute a
|G(d'')| |  ∈ ′′,  rdfs:subClassOf ′
 ′ rdfs:subClassOf ′′.</p>
      <p>genes  |  ∈
One class up for
|G(d')|  |  ∈




if|′| 
else</p>
      <p>∪ ′
 ∪  ∈  
return 
 ∅
while
′′ 
 ′′
return 
if|′| 
else
  ∪ ′
′.push′′ for ′′ such that ′ is a subclass of ′′</p>
      <sec id="sec-4-1">
        <title>Algorithms 2 TH k MP </title>
        <p>Input: a Mondo ID d, the Mondo Ontology
Output: a set G of genes
 .push //′ is a stack
is not empty
.pop</p>
        <p>Additionally, we describe the following two methods to expand VGP by rolling-up until the number
of genes exceeds the threshold k.</p>
        <p>
</p>
        <p>All paths up with threshold k (TH k AP)</p>
        <p>Minimum path up with threshold k (TH k MP)</p>
        <p>For an initially diagnosed disease d, TH k AP and TH k MP compute a set D of Mondo IDs, such
size of ′</p>
        <p>. The algorithms of TH k AP and TH k MP are shown in Algorithms 1 and 2.
that each</p>
        <p>in D is d or ancestor of d and the size of</p>
        <sec id="sec-4-1-1">
          <title>TH k AP outputs the union of ′</title>
          <p>for all ′ in D, and TH k MP outputs ′
is equal to or greater than the threshold k.
for ′
with minimum</p>
        </sec>
      </sec>
      <sec id="sec-4-2">
        <title>Algorithms 1 TH k AP </title>
        <p>Input: a Mondo ID d, the Mondo Ontology
Output: a set G of genes
 ∅
 .push //′ is a stack
while
is not empty
.pop
′.push′′ for ′′ such that ′ is a subclass of ′′
|G(d)| |  ∈</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>3. Results </title>
      <p>To evaluate the diagnostic efficiency of VGPs developed according to the proposed methods, a data
set from 74 patients obtained via whole-exome sequencing (WES) was used. The data set includes
candidate genes from WES, initially diagnosed diseases, and disease-causing genes. The median of the
number of candidate genes in the 74 patients was 384. For each patient, the number of disease-causing
genes, which would be included in G, was one. The number of initially diagnosed diseases, which is
the input of the methods, takes from one to three, with a median of one.</p>
      <p>Using the knowledge graph, VGPs for about 25,000 diseases contained in Mondo can be
automatically designed by the OR method. To evaluate the performance of the seven expansion methods,
we computed Coverage, Median, and Expectation. Table 1 shows the experimental results of OR, 1UAP,
1UMP, 2UAP, and 2UMP. Table 2 and Table 3 show the results for TH k AP and TH k MP when
increasing the threshold from 50 to 300 at increments of 50. Coverage refers to the ratio of VGPs that
include disease-causing genes. Median is the median of the sizes of intersections of the patient's
candidate genes and VGPs. Expectation is the expected number of genes to analyze for determining the
final disease-causing gene. If VGPs include disease-causing genes, only genes in intersections of the
patient's candidate genes and VGPs should be analyzed. However, if VGPs do not include
diseasecausing genes, all candidate genes should be analyzed. Therefore, the expectation can be computed
using the following formula.</p>
      <p>Expectation  ∈ VGP |  ∩ VGP |  ∉ VGP |  | / 74
where, for patient  , a set of genes in VGP, the disease-causing gene, and a set of candidate genes are
denoted by VGP( ),  , and  , respectively.</p>
      <sec id="sec-5-1">
        <title>Table 1  </title>
        <p>The results of the experiments using OR, 1UAP, 1UMP, 2UAP and 2UMP 
 
Coverage 
Median 
Expectation 
2UAP performed best in terms of Coverage, although this had the worst Median. Conversely, OR
had the best Median but worst Coverage. Regarding Expectation, TH 200 MP performed best.</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>4. Discussion </title>
      <p>Among the seven methods, the best performance observed was for TH k MP, especially when k =
200. As k increases, both the coverage of TH k MP and the median of the number of genes to be analyzed
become larger. Therefore, there is a trade-off between coverage and the median. By selecting suitable
values of k, users can design a VGP with preferred coverage and median.</p>
      <p>
        Using VGPs, the number of candidate genes to be analyzed can be reduced. If a VGP was not used,
all candidate genes had to be analyzed (median of 388). However, by using VGPs designed by TH 200
MP, the median of the number of candidate genes was reduced to 14.5 if the VGPs included
diseasecausing genes. Therefore, VGPs designed by our proposed methods may be useful for gene ranking
systems, such as PubCaseFinder [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ], by filtering candidate genes using VGPs.
      </p>
    </sec>
    <sec id="sec-7">
      <title>5. Conclusion </title>
      <p>In this study, seven methods were presented to develop expanded VGPs for initial clinical diagnosis
using knowledge graphs including Mondo. As a result, the TH 200 MP method, i.e., the minimum path
navigation using a threshold of 200, achieved the best performance regarding Expectation. We found
that this could contribute significantly to automatically reducing candidate genes as well as shortening
the interpretation time for diagnosing rare diseases. We expect our methods to be widely used by
clinicians to diagnose rare diseases with NGS analysis technology.</p>
    </sec>
    <sec id="sec-8">
      <title>Acknowledgments </title>
      <p>We are very grateful to Prof. Hirotomo Saitsu for providing the evaluation dataset. This work was
supported by the National Bioscience Database Center of the Japan Science and Technology Agency,
by "Challenging Exploratory Research Projects for the Future" grant from Research Organization of
Information and Systems and by JSPS KAKENHI grant number 21K12148.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>M.</given-names>
            <surname>Haendel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Vasilevsky</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Unni</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Bologa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Harris</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Rehm</surname>
          </string-name>
          , et al,
          <article-title>How many rare diseases are there?</article-title>
          ,
          <source>Nat Rev Drug Discov</source>
          <volume>19</volume>
          (
          <year>2020</year>
          )
          <fpage>77</fpage>
          -
          <lpage>78</lpage>
          . doi:
          <volume>10</volume>
          .1038/d41573-019-00180-y.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>S.</given-names>
            <surname>Marwaha</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. W.</given-names>
            <surname>Knowles</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E. A.</given-names>
            <surname>Ashley</surname>
          </string-name>
          ,
          <article-title>A guide for the diagnosis of rare and undiagnosed disease: beyond the exome</article-title>
          ,
          <source>Genome Med 14</source>
          (
          <year>2022</year>
          )
          <article-title>23</article-title>
          . doi:
          <volume>10</volume>
          .1186/s13073-022-01026-w.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>D.</given-names>
            <surname>Bick</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Jones</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. L.</given-names>
            <surname>Taylor</surname>
          </string-name>
          , R. J.
          <string-name>
            <surname>Taft</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Belmont</surname>
          </string-name>
          ,
          <article-title>Case for genome sequencing in infants and children with rare, undiagnosed or genetic diseases</article-title>
          ,
          <source>J Med Genet</source>
          <volume>56</volume>
          (
          <year>2019</year>
          )
          <fpage>783</fpage>
          -
          <lpage>791</lpage>
          . doi:
          <volume>10</volume>
          .1136/jmedgenet-2019-106111.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>A. L.</given-names>
            <surname>Wise</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T. A.</given-names>
            <surname>Manolio</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G. A.</given-names>
            <surname>Mensah</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. F.</given-names>
            <surname>Peterson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. M.</given-names>
            <surname>Roden</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Tamburro</surname>
          </string-name>
          , et al,
          <article-title>Genomic medicine for undiagnosed diseases</article-title>
          ,
          <source>Lancet</source>
          <volume>394</volume>
          (
          <year>2019</year>
          )
          <fpage>533</fpage>
          -
          <lpage>540</lpage>
          . doi:
          <volume>10</volume>
          .1016/S0140-
          <volume>6736</volume>
          (
          <issue>19</issue>
          )
          <fpage>31274</fpage>
          -
          <lpage>7</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>A. R.</given-names>
            <surname>Martin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Williams</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. E.</given-names>
            <surname>Foulger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Leigh</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. C.</given-names>
            <surname>Daugherty</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            <surname>Niblock</surname>
          </string-name>
          , et al,
          <article-title>PanelApp crowdsources expert knowledge to establish consensus diagnostic gene panels</article-title>
          ,
          <source>Nat Genet</source>
          <volume>51</volume>
          (
          <year>2019</year>
          )
          <fpage>1560</fpage>
          -
          <lpage>1565</lpage>
          . doi:
          <volume>10</volume>
          .1038/s41588-019-0528-2.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>Investigators</surname>
            <given-names>GPP</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Smedley</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K. R.</given-names>
            <surname>Smith</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Martin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E. A.</given-names>
            <surname>Thomas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E. M.</given-names>
            <surname>McDonagh</surname>
          </string-name>
          , et al,
          <volume>100</volume>
          ,000
          <article-title>Genomes pilot on rare-disease diagnosis in health care - preliminary report</article-title>
          ,
          <source>N Engl J Med</source>
          <volume>385</volume>
          (
          <year>2021</year>
          )
          <fpage>1868</fpage>
          -
          <lpage>1880</lpage>
          . doi:
          <volume>10</volume>
          .1056/NEJMoa2035790.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>K. A.</given-names>
            <surname>Shefchek</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. L.</given-names>
            <surname>Harris</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Gargano</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Matentzoglu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Unni</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Brush</surname>
          </string-name>
          , et al,
          <article-title>The Monarch Initiative in 2019: an integrative data and analytic platform connecting phenotypes to genotypes across species</article-title>
          ,
          <source>Nucleic Acids Res</source>
          <volume>48</volume>
          (
          <year>2020</year>
          )
          <fpage>D704</fpage>
          -
          <lpage>D715</lpage>
          . doi:
          <volume>10</volume>
          .1093/nar/gkz997.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>M.</given-names>
            <surname>Kamada</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Katayama</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Kawashima</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Kojima</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Nakatsui</surname>
          </string-name>
          ,
          <string-name>
            <surname>Y. Okuno,</surname>
          </string-name>
          <article-title>Med2RDF: Semantic biomedical knowledge-base and APIs for the clinical genome medicine</article-title>
          ,
          <source>in: Proceedings of the 12th. International Conference on Semantic Web Applications and Tools for Health Care and Life Sciences. SWAT4HCLS</source>
          <year>2019</year>
          , Edinburgh, Scotland,
          <year>2019</year>
          , pp.
          <fpage>161</fpage>
          -
          <lpage>162</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>M.</given-names>
            <surname>Dumontier</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. J.</given-names>
            <surname>Baker</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Baran</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Callahan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Chepelev</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Cruz-Toledo</surname>
          </string-name>
          , et al,
          <article-title>The Semanticscience Integrated Ontology (SIO) for biomedical research and knowledge discovery</article-title>
          ,
          <source>J Biomed Semantics</source>
          <volume>5</volume>
          (
          <year>2014</year>
          )
          <article-title>14</article-title>
          . doi:
          <volume>10</volume>
          .1186/2041-1480-5-14.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>OMIM</surname>
          </string-name>
          ,
          <article-title>An online catalog of human genes and genetic disorders</article-title>
          ,
          <year>2022</year>
          . URL: https://www.omim.org/
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Orphanet</surname>
          </string-name>
          ,
          <article-title>The portal for rare diseases</article-title>
          and orphan drugs,
          <year>2022</year>
          . URL: https://www.orpha.net/consor/cgi-bin/index.php
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>GenCC</surname>
          </string-name>
          ,
          <source>The gene curation coalition</source>
          ,
          <year>2022</year>
          . URL: https://thegencc.org/
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>T.</given-names>
            <surname>Fujiwara</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Shin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Yamaguchi</surname>
          </string-name>
          ,
          <article-title>Advances in the development of PubCaseFinder, including the new application programming interface and matching algorithm</article-title>
          ,
          <source>Hum Mutat</source>
          <volume>43</volume>
          (
          <year>2022</year>
          )
          <fpage>734</fpage>
          -
          <lpage>742</lpage>
          . doi:
          <volume>10</volume>
          .1002/humu.24341.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>