<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Features of a FAIR Vocabulary</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Fuqi Xu</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>k Juty</string-name>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Simon Jupp</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>rkinson</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Courtot</string-name>
          <email>mcourtot@gmail.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>European Molecular Biology Laboratory, European Bioinformatics Institute</institution>
          ,
          <addr-line>Wellcome Genome Campus, Hinxton, Cambridge CB10 1SD</addr-line>
          ,
          <country country="UK">United Kingdom</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>SciBite, BioData Innovation Centre</institution>
          ,
          <addr-line>Wellcome Genome Campus, Hinxton, Cambridge CB10 1DR</addr-line>
          ,
          <country country="UK">United Kingdom</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>University of Manchester,</institution>
          ,
          <addr-line>Manchester M13 9PL</addr-line>
          ,
          <country country="UK">United Kingdom</country>
        </aff>
      </contrib-group>
      <fpage>1</fpage>
      <lpage>02</lpage>
      <abstract>
        <p>The FAIR Principles explicitly require the use of FAIR vocabularies, but what precisely constitutes a FAIR vocabulary remains unclear. Here we provide de nitions for FAIR vocabularies, examine the application of the FAIR Principles to vocabularies, align their requirements with the Open Biomedical Ontologies (OBO) Principles, and propose FAIR Vocabulary Features (FVFs). We also design assessment approaches for FAIR vocabularies by mapping the FVFs with existing FAIR assessment indicators. Finally, we demonstrate how FVFs can be used for evaluating and improving vocabularies using exemplar biomedical vocabularies.</p>
      </abstract>
      <kwd-group>
        <kwd>FAIR principles</kwd>
        <kwd>Vocabulary</kwd>
        <kwd>Ontology</kwd>
        <kwd>Assessment</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>
        The Findable, Accessible, Interoperable and Reusable (FAIR) Principles [
        <xref ref-type="bibr" rid="ref40">40</xref>
        ]
have gained traction in the biomedical community since their publication in
2016, with many groups attempting to improve their data quality, develop FAIR
capable data resources, and design generic FAIR assessment tools for biomedical
data[
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ] [
        <xref ref-type="bibr" rid="ref39">39</xref>
        ]. Due to the heterogeneous nature and broad scope of biomedical
data, from molecules to human studies via interdisciplinary analysis, stringent
requirements for its FAIRness need to be met to ensure its usefulness toward
bene ting human health. While assessing the FAIR level of datasets and data
resources [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ], we noted a futile cycle with respect to the `Interoperable' FAIR
Principle, "I2 - (Meta)data use vocabularies that follow FAIR principles". To
comply with that principle, datasets need to use FAIR vocabularies, which
themselves need to be FAIR. FAIR vocabularies promote the exchange of biomedical
∗These authors contributed equally to this paper.
      </p>
      <p>†To whom correspondence should be addressed.</p>
      <p>Copyright © 2022 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
data, which are usually generated, annotated and used by di erent groups of
researchers. FAIR vocabularies promote biomedical data FAIRness throughout the
data life cycle, during the data generation, curation, and distribution processes,
and support data exchange and integration across data resources.</p>
      <p>
        Multiple e orts have been made to develop standards for FAIR
vocabularies. The FAIRsFAIR recommendations provide guidance[
        <xref ref-type="bibr" rid="ref24">24</xref>
        ] on FAIR
semantic artefacts, as well as supporting vocabulary search engines and repositories.
      </p>
      <p>
        Garijo and Poveda-Villalon[
        <xref ref-type="bibr" rid="ref22">22</xref>
        ] discussed detailed requirements of ontology URI
and versioning strategies, as well as the formatting of the ontologies. Ten
simple rules[
        <xref ref-type="bibr" rid="ref15">15</xref>
        ] for converting print-based or other forms of legacy vocabularies to
FAIR vocabularies have also been proposed.
      </p>
      <p>
        Researchers have also developed approaches to assess the FAIR level of digital
objects both manually and automatically; FAIRsharing hosts FAIR indicators
for automated tests in their FAIR Maturity Evaluation Service[
        <xref ref-type="bibr" rid="ref39">39</xref>
        ], FAIR
metrics in the F-UJI Automated FAIR Data Assessment Tool[
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], and the Research
Data Alliance(RDA) Data Maturity Model Speci cation and Guidelines[
        <xref ref-type="bibr" rid="ref5">5</xref>
        ], also
known as the RDA indicators. Among them, the RDA indicators are a set of
representative and descriptive indicators to evaluate the FAIR level of data and
have been used in many projects and with many types of data. Some automated
assessments of the FAIRness of vocabulary have been developed to measure the
FAIR level of public, machine-readable vocabularies, such as FOOPS![
        <xref ref-type="bibr" rid="ref21">21</xref>
        ] To
the best of our knowledge, there have not yet been quanti able FAIR
assessment approaches developed to measure the FAIR level of di erent formats of
vocabularies objectively.
      </p>
      <p>
        Therefore, in this paper, we distinguish the concepts of FAIR data, FAIR data
resources, and FAIR vocabularies, and propose a set of general FAIR Vocabulary
Features (FVFs) as a set of satis able features for vocabularies. We also adapted
the RDA indicators to measure the FAIR level of vocabularies. Further, we
provide example assessments based on selected ontologies available from the
EMBL-EBI Ontology Lookup Service (OLS)[
        <xref ref-type="bibr" rid="ref25">25</xref>
        ] and other vocabulary resources.
2
      </p>
      <p>FAIR Data and FAIR Vocabulary
In the execution of this work, we note the distinction between FAIR data and
FAIR capable data resources. In our analysis, data can be FAIR, to a greater or
lesser extent, and data resources and data vocabularies are capable of supporting
FAIRness (FAIR capable) at di erent levels. Data vocabularies are designed to
support FAIR data, and they can also be considered as FAIR data resources.</p>
      <p>The orthogonality of these concepts is an important context for this work when
determining the features of a FAIR vocabulary. We must also determine whether
a vocabulary itself is 1) FAIR in terms of its application to FAIR data 2) FAIR
in the context of FAIR capable resources 3) FAIR in the context of other
vocabularies. A FAIR vocabulary has a set of FAIR features and have a list of associated
FAIR indicators. It is usable for annotation, analysis and presentation of data,
and is deployable in the context of a FAIR capable data resource or tools. It also
serves 'aggregation' use cases where data originates from di erent domain, and
enables data interoperability where di erent vocabularies are used.</p>
      <p>
        Vocabularies come in di erent forms, such as lists, thesaurus, taxonomies,
and ontologies; each at di erent levels of semantic maturity and FAIR
requirements. The International Classi cation of Diseases (ICD-11)[
        <xref ref-type="bibr" rid="ref31">31</xref>
        ] is a large
taxonomy of disease and is the global standard for diagnostic information, disease
de nitions and synonyms. The Gene Ontology (GO)[
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] is a well established and
highly regarded and utilised biomedical resource. It contains over 43000 terms
and has been cross referenced in other classi cation systems, such as UniProt[
        <xref ref-type="bibr" rid="ref13">13</xref>
        ],
HAMAP[
        <xref ref-type="bibr" rid="ref32">32</xref>
        ], and InterPro[
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. GO is also a reference OBO Foundry ontology[
        <xref ref-type="bibr" rid="ref34">34</xref>
        ]
and has been reused in many other resources. The Experimental Factor Ontology
(EFO)[
        <xref ref-type="bibr" rid="ref27">27</xref>
        ], on the other hand, is an application ontology built for communities
like the Open Targets[
        <xref ref-type="bibr" rid="ref29">29</xref>
        ] for describing experimental variables.
3
      </p>
      <p>
        Existing Vocabulary Standards
In determining features of FAIR vocabularies, we considered previous
standardisation work by the Open Biomedical Ontology (OBO) community to determine
whether the OBO Principles[
        <xref ref-type="bibr" rid="ref18">18</xref>
        ] addressed elements of vocabulary FAIRness.
      </p>
      <p>The OBO Principles aim to coordinate the development of biomedical
ontologies, which focus speci cally on ontologies, covering both the development of
ontologies and ontology themselves. Despite the OBO principles predating the
FAIR principles, a comparison of the two aided us in de ning FVFs. Table 1
summarises the key points of the OBO Foundry principles, and assesses their
suitability as FVFs. We also noted that not all the OBO principles possess the
same level of maturity or granularity, and therefore some were unmappable and
excluded from the comparison. As a result, this analysis did not include OBO
Principle 1, 4, 6, 9 - 12 and 20. The rationale for suitability as FVFs is discussed
in detail below and further in Supplemental Table 1.
4</p>
      <p>FAIR Vocabulary Features
Based on the analysis of OBO foundry practices and our previous experience
working with and developing ontologies, we propose eleven features for FAIR
vocabulary in Table 2, covering requirements for identi ers, access protocols,
knowledge representation, etc. Supplementary Table 2 shows the relationship
among FAIR Vocabulary Features, the FAIR principles and requirements for
FAIR vocabularies.</p>
      <p>Table 2also provides examples for each FAIR feature, but does not
exhaustively cover all current practices across the various vocabularies; each feature is
represented in di erent formats and at varying FAIRness levels amongst those
vocabularies. For example, for FVF-6: versioning and persistent vocabularies, of
all ontologies indexed and updated in OLS, 59.3%§ of vocabularies use a date
format of "yyyy-mm-dd" in the "versionIRI", such as "http://purl.obolibrary.org/</p>
      <p>§See details in Supplementary Material 3
ID OBO Principle Summary Suitable
as
FAIR
Vocabulary
Feature?
Principle 1: Open The ontology MUST be openly available to be used by No
all without any constraint other than (a) its origin must
be acknowledged and (b) it is not to be altered and
subsequently redistributed in altered form under the original
name or with the same identi ers.</p>
      <p>Principle 2: Com- The ontology is made available in a common formal lan- Yes
mon Format guage in an accepted concrete syntax.</p>
      <p>Principle 3: Each class and relation (property) in the ontology must Yes
URI/Identi er have a unique URI identi er.</p>
      <p>Space
Principle 4: Ver- The ontology provider has documented procedures for ver- No
sioning sioning the ontology, and di erent versions of ontology are</p>
      <p>marked, stored, and o cially released.</p>
      <p>Principle 5: Scope The scope of an ontology is the extent of the domain or Yes
subject matter it intends to cover. The ontology must have
a clearly speci ed scope and content that adheres to that
scope.</p>
      <p>Principle 6: Tex- The ontology has textual de nitions for the majority of No
tual De nitions its classes and for top level terms in particular
Principle 7: Rela- Relations should be reused from the Relations Ontology Yes
tions (RO).</p>
      <p>Principle 8: Docu- The owners of the ontology should strive to provide Yes
mentation as much documentation as possible. The documentation
should detail the di erent processes speci c to an ontology
life cycle and target various audiences (users or
developers).</p>
      <p>Principle 9: Docu- The ontology developers should document that the ontol- No
mented Plurality ogy is used by multiple independent people or
organizaof Users tions.</p>
      <p>Principle 10: OBO Foundry ontology development, in common with Yes
Commitment to many other standards-oriented scienti c activities, should
Collaboration be carried out in a collaborative fashion.</p>
      <p>Principle 11: Lo- There should be a person who is responsible for communi- No
cus of Authority cations between the community and the ontology
developers, for communicating with the Foundry on all
Foundryrelated matters, for mediating discussions involving
maintenance in the light of scienti c advance, and for ensuring
that all user feedback is addressed.</p>
      <p>Principe 12: Nam- Naming conventions are used No
ing Conventions
Principle 16: The ontology needs to re ect changes in scienti c consen- Yes
Maintenance sus to remain accurate over time.</p>
      <p>Principle 20: Re- Ontology developers MUST o er channels for community No
sponsiveness participation and SHOULD be responsive to requests.
-obo/scdo/releases/2021-04-15/scdo.owl". 2.51% of vocabularies use semantic
versioning (x.x.x) such as "http://www.ebi.ac.uk/efo/releases/v3.34.0/efo.owl"
or other forms of numeric versioning, such as http://www.orpha.net/version3.2.
31.66% of vocabularies do not provide valid machine-readable versioned IRIs. For
FVF-1: identi ers, 74% of vocabularies use OBO-format PURLs, identi er.org,
w3id.org identi ers, as well as other domain-speci c identi ers. For FVF-5:
accessible using standard protocols, of all 199 selected ontologies, only one ontology
used the HTTPS protocol; the rest use HTTP protocols.
5</p>
      <p>FAIR Vocabulary Feature Indicators
FAIR vocabulary Features outline general characteristics of a FAIR vocabulary,
however, those features need to be objectively quanti ed to be useful in
vocabulary selection, development and assessment. Hence, we propose aligning FVFs
with FAIR indicators to enable computation of a discrete FAIR score, with the
aim to o er an objective quantitative evaluation of vocabularies and to guide
subsequent improvements.</p>
      <p>We mapped the RDA indicators to FAIR Vocabulary Features, ltered out
indicators that do not apply to vocabularies (see details in Supplementary Table
4, speci ed the digital object which the indicator refers to, and identi ed within
each indicator the relevant standards used in corresponding domains. It is worth
noting that when mapping the RDA indicators on datasets, metadata refers to
the metadata to which the vocabulary can be applied, while in the context of
vocabularies, metadata and data refer to the description of the vocabulary and
the vocabulary information. Therefore, we combined the indicators evaluating
data and metadata in the mapping, wherever possible. The FVFs, associated
with selected indicators, can be used as indicators for FAIR Vocabulary as shown
in Table 3.
6</p>
      <p>Assessment against Indicators for FAIR Vocabulary</p>
      <p>Features
We tested the FAIR Vocabulary Features and corresponding indicators on three
representative vocabularies, GO, EFO and ICD-11, as shown in Table 4. For
each FVF, three compliance levels are assigned; if a vocabulary meets the
requirements of all indicators, full compliance is achieved. Otherwise, depending
on the scoring within each FVF, partial compliance or no compliance results are
given. The percentages of full compliance, partial compliance and no compliance
features are also calculated. Supplementary Table 5-7 provide the assessment
details.</p>
      <p>From the assessment results, both the Gene Ontology and Experimental
Factor Ontology are vocabularies of high FAIR level, with over 80% FVFs ful lled.</p>
      <p>The Gene Ontology only partially complies with `FVF-6: Vocabularies and their
terms are persistent over time and are appropriately versioned ', with a Fail in</p>
      <p>FVF-4: Vocabularies and their terms are reg- RDA-F4-01M
istered or indexed in a searchable engine or a
resource.</p>
      <p>FVF-5: Vocabularies and their terms are
retrievable using a standardised communications
protocol, preferably open, free and universally RDA-A1-04M
implementable protocols. and allows for
authentication and authorisation, where
necessary.</p>
      <p>FVF-6: Vocabularies and their terms are
persistent over time and are appropriately versioned.</p>
      <p>FVF-7: Vocabularies and their terms use a
formal, accessible and broadly applicable, and
preferably machine-understandable language
for knowledge representation.</p>
      <p>FVF-8: Vocabularies and terms use quali ed RDA-I3-02D
references to other vocabularies.</p>
      <p>FVF-9: Vocabularies and terms are described RDA-R1-01M
with a plurality of accurate and relevant
attributes.</p>
      <p>FVF-10: Vocabularies are released with a
standard data usage licence, preferably machine- RDA-R1.1-01M
readable licence.</p>
      <p>FVF-11: Vocabularies meet domain relevant
community standards.</p>
      <p>RDA
ID
RDA-F1-01D
RDA-F1-02M
RDA-A1-02M
RDA-A1-02D
RDA-A1-03M
RDA-A1-03D
RDA-A1-05D
RDA-A1-04D
RDA-A1.1-01M
RDA-A1.1-01D
RDA-A1.2-01D
RDA-A2-01M
RDA-R1.2-01M
RDA-R1.2-02M
RDA-I1-01M
RDA-I1-01D
RDA-I1-02M
RDA-I1-02D
RDA-I3-03M
RDA-R1.1-02M
RDA-R1.1-03M
RDA-R1.3-01M
RDA-R1.3-01D
RDA-R1.3-02M
RDA-R1.3-02D</p>
      <p>
        Metadata is identi ed by a persistent identi er
Data is identi ed by a persistent identi er
Metadata is identi ed by a globally unique
identi er
Data is identi ed by a globally unique identi er
Rich metadata is provided to allow discovery
Metadata contains information to enable the
user to get access to the data
Metadata can be accessed manually (i.e. with
human intervention)
Data can be accessed manually (i.e. with
human intervention)
Metadata identi er resolves to a metadata
record
Data identi er resolves to a digital object
Data can be accessed automatically (i.e. by a
computer program)
Metadata is o ered in such a way that it can
be harvested and indexed
Metadata is accessed through standardised
protocol
Data is accessible through standardised
protocol
Metadata is accessible through a free access
protocol
Data is accessible through a free access protocol
Data is accessible through an access protocol
that supports authentication and authorisation
Metadata is guaranteed to remain available
after data is no longer available
Metadata includes provenance information
according to community-speci c standards
Metadata includes provenance information
according to a cross-community language
Metadata uses knowledge representation
expressed in standardised format
Data uses knowledge representation expressed
in standardised format
Metadata uses machine-understandable
knowledge representation
Data uses machine-understandable knowledge
representation
Data includes quali ed references to other data
Metadata includes quali ed references to other
metadata
Plurality of accurate and relevant attributes
are provided to allow reuse
Metadata includes information about the
licence under which the data can be reused
Metadata refers to a standard reuse licence
Metadata refers to a machine-understandable
reuse licence
Metadata complies with a community standard
Data complies with a community standard
Metadata is expressed in compliance with a
machine-understandable community standard
Data is expressed in compliance with a
machine-understandable community standard
`Indicator RDA-R1.2-02M: Metadata includes provenance information according
to a cross-community language '. `FVF-2: Vocabularies and their terms have rich
metadata' was not complied with since no general description of the ontology is
provided in the released artefact. Compared with the ontologies, the taxonomy,
ICD-11, fully complies with 18.18% FVFs, and partially complies with 36.36%
FVFs. This is because ICD-11 neither refers to other vocabularies, nor adheres
to other community standards, such as vocabulary formats. ICD-11 was selected
for this evaluation as it already o ers signi cant FAIR improvements over
ICD10[
        <xref ref-type="bibr" rid="ref30">30</xref>
        ], such as providing a standard licence.
7
      </p>
    </sec>
    <sec id="sec-2">
      <title>Discussion</title>
      <p>The FAIR Vocabulary Features we propose integrate multiple FAIR
vocabulary requirements and can be used as FAIR vocabulary standards to guide the
development and maintenance of vocabularies. Each FVF is associated with
indicators enabling its quanti able, objective assessment. Those indicators can be
connected to related standards and extended to existing or emerging standards
in other domains. For example, FVF-8: cross-referencing other vocabularies can
be linked to the ontology cross-reference standards, MIREOT. We focused on
how FVFs can be applied to ontologies, and demonstrated the potential for using
them across other forms of vocabularies; for example, with the ICD-11,
assessment would inform authors on the means to enrich their resources. Because of our
expertise and requirements, this manuscript focuses on the biomedical domain;
however, we anticipate this framework could be reused in other domains.</p>
      <p>Integrating the FAIR vocabulary features with FAIR indicators makes it
possible to assess the FAIR level of vocabularies, identify progressive ontology
development use cases, and improve those vocabularies. We selected the RDA
indicators as it has proven to be useful in many datasets and has been referenced
by other assessment approaches in FAIRassist.org; yet, FVFs could alternatively
be aligned to other FAIR-principle based indicators which would similarly re ect
the guiding principles proposed by Wilkinson et al. Besides manual assessment,
quanti able formal indicators are also amenable to becoming machine actionable.</p>
      <p>Some e orts already exist, and reusing shared indicators will make it possible to
perform automated FAIR vocabulary assessments.</p>
      <p>The indicators were proposed to objectively measure the FAIR level of
ontologies, yet this score does not re ect an absolute FAIR level for the
vocabulary. Indeed, depending on the purpose and requirements of the vocabulary,
some FVFs can be more or less important than other features. For example, for
internal vocabularies which are used and shared within an institution, having
global identi ers (FVF-1) is not a mandatory requirement. Instead of comparing
the FAIR score of di erent vocabularies to nd the `FAIRer' one, we propose
the FAIR score should be used to measure and guide the evolution of FAIR
vocabularies by successively comparing the FAIR levels of iteratively developed
versions. For example, compared to ICD-10, its successor, ICD-11 has
incorporated many features to make it FAIRer, such as providing APIs for easier access,
having a machine-readable license, etc.</p>
      <p>
        From the assessment results of the two ontologies and ICD-11, ontology-based
vocabularies follow stricter semantics and therefore fared better in the scoring of
FAIR features. For example, many ontology-related standards have been
established, including formats, such as OWL, guidelines such as the OBO principles,
minimum information standards, such as MIBBI[
        <xref ref-type="bibr" rid="ref36">36</xref>
        ], and mechanisms for
crossreferences or incorporating external ontologies, such as MIREOT. This naturally
re ects in a high score for compliance with community standards, which is a core
part of FAIR Vocabulary Features and which improves the interoperability and
reusability of a vocabulary.
      </p>
      <p>The FAIR Vocabulary features and assessments provide insights on how to
improve vocabularies. For example, based on the EFO assessments, the FAIR
level of EFO could easily be improved by adding a description of the aim and
function of EFO. This way, di erent vocabulary management services can
harvest the information.</p>
      <p>Acknowledgements
This work is funded by the IMI-FAIRplus project (Grant number 802750) and
the European Molecular Biology Laboratory - European Bioinformatics Institute
core funds.
8
Supplementary Table 1: The suitability of OBO principles used as FAIR</p>
      <sec id="sec-2-1">
        <title>Vocabulary Features</title>
        <p>OBO-1: The ontology MUST be openly available to be
used by all without any constraint other than (a) its origin
must be acknowledged and (b) it is not to be altered and Suitable as FAIR
subsequently redistributed in altered form under the Vocabulary Feature?
original name or with the same identifiers. No
While it is highly desirable to have open source semantic artefacts, as it is to have open
source code and ultimately open research. This is not a prerequisite for vocabularies to
be FAIR, nor is it well aligned with the FAIR principles which require licensing
information to be provided but do not mandate it be open source.</p>
        <p>OBO-2: The ontology is made available in a common
formal language in an accepted concrete syntax.</p>
        <p>Common formats define minimum standards for accessing an ontology, support using
ontologies in a FAIR capable resource, and are the foundation of interoperable
vocabulary. Many common formal languages have been proposed during the
development history of ontologies. OWL format is currently used as a W3C standard.
OBO-3: Each class and relation (property) in the
ontology must have a unique URI identifier.</p>
        <p>Identifiability is a core FAIR principle, and in order to be fulfilled, all elements of the
ontology, such as classes and relationships, should be clearly and uniquely identified.
OBO-4: The ontology provider has documented
procedures for versioning the ontology, and different
versions of ontology are marked, stored, and officially
released.</p>
        <p>Proper versioning improves the findability and reusability of the vocabulary. We
proposed a corresponding FVF to cover this aspect. Yet, this principle evaluates the
development, especially the documentation process, of the ontology rather than FAIR
vocabulary.</p>
        <p>OBO-5: The scope of an ontology is the extent of the
domain or subject matter it intends to cover. The
ontology must have a clearly specified scope and
content that adheres to that scope.</p>
        <sec id="sec-2-1-1">
          <title>Suitable as FAIR Vocabulary Feature? Yes</title>
          <p>Users must be able to determine which ontologies meet their needs in order to
implement these in FAIR capable resources and to be interoperable with other
vocabularies. A clear specification of this aids in FAIR implementation; vocabularies do
not exist in isolation.
OBO-6: The ontology has textual definitions for the
majority of its classes and for top level terms in
particular
While textual definitions provide human-readable content and are generally desirable,
this is not essential as an FVF as ontology content has definitions in terms of logical
axioms (e.g., position in the hierarchy) and term labels.</p>
          <p>OBO-7: Relations should be reused from the Relations
Ontology (RO).</p>
          <p>Relation standards promote interoperability across different ontologies. This principle
focuses on the relationships within and across ontologies. It can be adapted and used
in a broader range of FAIR vocabularies.
OBO-8: The owners of the ontology should strive to
provide as much documentation as possible. The
documentation should detail the different processes
specific to an ontology life cycle and target various
audiences (users or developers).
Rich metadata of the vocabulary, such as the purpose and status of the ontology,
promotes the reuse of the ontology.</p>
          <p>OBO-9: The ontology developers should document that
the ontology is used by multiple independent people or
organizations.
Usage of vocabularies depends on the content and there are strategies for
interoperating ontologies in what is anyway a crowded semantic space. Evidence that
an ontology is highly used - when measurable - assumes a level of maturity that is
unlikely for some starting communities, and would not reflect fairness of the resource.
OBO-10: OBO Foundry ontology development, in
common with many other standards-oriented scientific
activities, should be carried out in a collaborative
fashion.</p>
        </sec>
        <sec id="sec-2-1-2">
          <title>Suitable as FAIR Vocabulary Feature? Yes</title>
          <p>Vocabularies should be implementable in FAIR data resources and should reflect
community needs. Responsiveness to community needs comes via collaboration and
development should therefore be collaborative.</p>
          <p>OBO-11: There should be a person who is responsible
for communications between the community and the
ontology developers, for communicating with the
Foundry on all Foundry-related matters, for mediating
discussions involving maintenance in the light of
scientific advance, and for ensuring that all user
feedback is addressed.</p>
        </sec>
        <sec id="sec-2-1-3">
          <title>Suitable as FAIR Vocabulary Feature? No</title>
          <p>Having a designed contact is important for the community to provide feedback and
ensures someone is having an ongoing editorial responsibility for the ontology. While it
does pertain to sustainability of the resource and its possible evolution, it doesn’t
directly speak to its level of fairness.</p>
          <p>OBO-12: Naming conventions are used
Consistency of naming is a best practice feature of ontologies but does not detract from
deployment in support of the FAIR principles.</p>
          <p>OBO-16: The ontology needs to reflect changes in
scientific consensus to remain accurate over time.</p>
          <p>Vocabularies codify knowledge. To fulfil one of their primary functions, they must
evolve. Dead ontologies fail to support FAIR capable resources to interoperate.
OBO-20: Ontology developers MUST offer channels for
community participation and SHOULD be responsive to
requests.
Having communication channels to collect and respond to community requirements
supports the maintenance and evolution of the ontology. However, it is not directly
linked to the FAIRness of the vocabulary.
Supplementary Table 2: FAIR Vocabulary Features mapped to FAIR
principles and FAIR vocabulary requirements</p>
          <p>Findability Accessibility Interoperability Reusability
FAIR in terms of application to FAIR data.</p>
          <p>FAIR in terms of serving as a FAIR data
resource.</p>
          <p>FAIR in the context of interacting with other
vocabularies.</p>
          <p>FVF-1
FVF-4
FVF-6</p>
          <p>FVF-3
FVF-5
FVF-10</p>
          <p>FVF-11
FVF-7
FVF-8</p>
          <p>FVF-2
FVF-6
FVF-9
FVF-10
FVF-11
FVF-2
FVF-6
FVF-7
FVF-9
Supplementary Materials 3: VersionIRI analysis
We fetched ontologies indexed in the OLS repository and selected those that are
successfully loaded and up-to-date. OLS contains 266 biomedical ontologies by the time we
access the database (https://www.ebi.ac.uk/ols/api/ontologies). We filtered out ontologies
which could not be indexed automatically (without a valid loaded timestamp), and removed
inactive ontologies based on the date information in the versionIRI section. 200 ontologies
are selected based on these criteria.</p>
          <p>We recognise the limitations of the ontology selection approaches. The filtering relys on the
metadata collected by OLS instead of the ontology itself, and therefore might not correctly
reflect the ontology status. We filted out some inactive ontologies based on the loading time
(only ontologies with a loading timestamp after 2019-01-01 are choosen) and date
information in the versioneIRI (ontologies with date before 2019-01-01 in the verionIRI are
removed). But these criterias does not ensure all vocabularies selected are up-to-date. For
example, for ontologies using semantic versioning format where no date information is
provided in the versionIRI, or some update information are collected in other metadata fields
such as 'annotation' 'editor comments', etc.</p>
          <p>Despite the constraints of the analysis, it still provides enough information to showcase the
status of current vocabularies. A complete list of selected ontologies are provided in the
table below.</p>
        </sec>
      </sec>
      <sec id="sec-2-2">
        <title>Supplementary Table 4: RDA data mapped to FAIR Vocabulary Features maturity indicators that are not</title>
        <sec id="sec-2-2-1">
          <title>ID Indicator</title>
          <p>RDA-F3-01M Metadata includes the identifier for the data
RDA-I2-01M Metadata uses FAIR-compliant vocabularies
RDA-I2-01D Data uses FAIR-compliant vocabularies
RDA-I3-01M Metadata includes references to other metadata
RDA-I3-01D Data includes references to other data
RDA-I3-02M Metadata includes references to other data</p>
          <p>RDA-I3-04M Metadata include qualified references to other data
Supplementary Table 5: FAIR assessment results of Gene ontology
https://github.com/geneontology/go-ontology and
http://geneontology.org/
FAIR vocabulary feature summary
FVF, full compliance
FVF, partial compliance
RDA-F1- Metadata is identified by
01M a persistent identifier
RDA-F1- Data is identified by a
01D persistent identifier</p>
          <p>Metadata is identified by
RDA-F1- a globally unique
identifier
1
1
FVF-1: Vocabulary and 02M 1
their terms are assigned Full http://purl.obolibrary.org/
globally unique and RDA-F1- Data is identified by a Complia obo/go.owl is globally
persistent identifiers. 02D globally unique identifier 1 nce unique identifier.</p>
          <p>Assessment details
Metadata are provided
in
http://geneontology.org/
docs/ontology-documen
tation/. It can also be
found in the OBO
foundry repository
https://github.com/OBO
Foundry/OBOFoundry.g
ithub.io/edit/master/onto
logy/go.md. But they
are not standard
persistent identifiers.</p>
          <p>Gene ontology uses
PURL identifiers
http://purl.obolibrary.org/
obo/go.owl
http://geneontology.org/
docs/ontology-documen
tation/ is globally unique
identifier.</p>
          <p>Metadata contains
information to enable
RDA-A1- the user to get access
01M to the data</p>
          <p>Metadata can be
accessed manually (i.e.</p>
          <p>RDA-A1- with human
02M intervention)</p>
          <p>Data can be accessed
RDA-A1- manually (i.e. with
02D human intervention)</p>
          <p>Metadata identifier
RDA-A1- resolves to a metadata
03M record
FVF-3: Vocabularies
and their terms can be 03D</p>
          <p>accessed using the
identifiers, preferably by
both human and
machine.</p>
          <p>RDA-A1- Data identifier resolves
to a digital object</p>
          <p>Data can be accessed
RDA-A1- automatically (i.e. by a
05D computer program)</p>
          <p>Data can be
Full downloaded using</p>
          <p>Complia command line tools,
1 nce such as curl, wget, etc.</p>
          <p>Gene ontology has
been indexed by EMBL
OLS, BioPortal and
other semantic
Full repositories. Also it is</p>
          <p>Complia indexed in Google
1 nce search.</p>
          <p>Descriptive text is
provided in
http://geneontology.org.</p>
          <p>Rich metadata for
indexing and reuse is
provided in
https://github.com/OBO
Full Foundry/OBOFoundry.g</p>
          <p>Complia ithub.io/edit/master/onto
1 nce logy/go.md.</p>
          <p>The metadata includes
data download links
http://geneontology.org/
docs/download-ontology
/.</p>
          <p>The metadata can be
accessed from the gene
ontology website.</p>
          <p>Data can be
downloaded from
http://geneontology.org/
docs/download-ontology
/
http://geneontology.org/
docs/ontology-documen
tation// is resolvable and
directs to the metadata.</p>
          <p>The vocabulary
identifier
http://purl.obolibrary.org/
obo/go.owl resolves to
the ontology source
files. Identifiers such as
http://purl.obolibrary.org/
obo/GO_0098743
resolves to ontology
terms.
1
1
1
1
1
FVF-5: Vocabularies
and their terms are
retrievable using a</p>
          <p>standardised
communications
protocol, preferably
open, free and</p>
          <p>universally
implementable
protocols. and allows for
authentication and
authorisation, where
necessary.</p>
          <p>Data is accessible
RDA-A1- through standardised
04D protocol</p>
          <p>Metadata is accessible
RDA-A1. through a free access
1-01M protocol</p>
          <p>Data is accessible
RDA-A1. through a free access
1-01D protocol</p>
          <p>Data is accessible
through an access
protocol that supports
RDA-A1. authentication and
2-01D authorisation</p>
          <p>Metadata is guaranteed
to remain available after
RDA-A2- data is no longer
01M available</p>
          <p>Metadata includes
provenance information
according to
RDA-R1. community-specific
2-01M standards
FVF-6: Vocabularies
and their terms are
persistent over time and
are appropriately
versioned.</p>
          <p>Metadata includes
provenance information
according to a
RDA-R1. cross-community
2-02M language</p>
          <p>Partial</p>
          <p>Complia
0 nce
Metadata uses
knowledge
representation</p>
          <p>RDA-I1- expressed in
FVF-7: Vocabularies 01M standardised format
and their terms use a
formal, accessible and
broadly applicable, and</p>
          <p>preferably
machine-understandabl 01D
e language for</p>
          <p>knowledge
representation.</p>
          <p>Data uses knowledge
representation
RDA-I1- expressed in
standardised format</p>
          <p>Metadata uses
RDA-I1- machine-understandabl
02M e knowledge</p>
          <p>The metadata can be
accessed through the
HTTP protocol.</p>
          <p>Data can be accessed
through HTTP protocol.</p>
          <p>HTTP is a free access
protocol.</p>
          <p>HTTP is a free access
protocol.</p>
          <p>Metadata and data can
be found in version
controlled repositories
on Github.</p>
          <p>The metadata includes
links to access different
snapshots of the
ontology. The
snapshots are in
owl/obo format and has
PURL identifiers.</p>
          <p>The metadata is
provided in a standard
format and can be
harvested by major
vocabulary serivices,
such as OLS and
BioPortal.</p>
          <p>The data uses OWL
and OBO standards.</p>
          <p>HTTP HTTP allows
access control. But
authentication and
Full authorisation are not</p>
          <p>Complia required by Gene
NA nce Ontology.</p>
          <p>Full</p>
          <p>Complia Basic metadata is
1 nce provided in OWL.
1
1
1
1
1
1
1
1</p>
          <p>FVF-8: Vocabularies
and terms use qualified
references to other</p>
          <p>vocabularies.</p>
          <p>FVF-9: Vocabularies
and terms are
described with a
plurality of accurate
and relevant
attributes.</p>
          <p>FVF-10: Vocabularies
are released with a
standard data usage
licence, preferably
machine-readable</p>
          <p>licence.</p>
          <p>representation</p>
          <p>Plurality of accurate and
RDA-R1 relevant attributes are
-01M provided to allow reuse</p>
          <p>Metadata includes
information about the
RDA-R1. licence under which the
1-01M data can be reused
RDA-R1. Metadata refers to a
1-02M standard reuse licence</p>
          <p>Metadata refers to a
RDA-R1. machine-understandabl
1-03M e reuse licence
RDA-R1. Metadata complies with
3-01M a community standard
RDA-R1. Data complies with a
3-01D community standard</p>
          <p>Metadata is expressed
in compliance with a
RDA-R1. machine-understandabl
3-02M e community standard
1
1
1
1
1
1
1
1
1
1</p>
          <p>The GO data uses OWL
and OBO formats,
which are
machine-readable
community formats.</p>
          <p>The Gene ontology
cross reference policy is
here:
http://geneontology.org/
docs/download-mappin
gs/ Data from other
vocabularies are
provided as 'xref'/
The Gene ontology
cross reference policy is
provided here:
Full http://geneontology.org/
Complia docs/download-mappin
nce gs/</p>
          <p>Gene Ontology includes
sufficient term
Full attributes.</p>
          <p>Complia http://geneontology.org/
nce docs/GO-term-elements</p>
          <p>Gene Ontology
Consortium data and
data products are
licensed under the
Creative Commons
Attribution 4.0 Unported</p>
          <p>License.
1
FAIR vocabulary
Feature
RDA-F1- Metadata is identified by
01M a persistent identifier
RDA-F1- Data is identified by a
01D persistent identifier</p>
          <p>Metadata is identified by
FVF-1: Vocabulary and RDA-F1- a globally unique
their terms are assigned 02M identifier 1
globally unique and RDA-F1- Data is identified by a
persistent identifiers. 02D globally unique identifier 1
Supplementary Table 6: FAIR assessment results of Experimental Factor</p>
        </sec>
      </sec>
      <sec id="sec-2-3">
        <title>Ontology</title>
        <p>FAIR vocabulary feature summary</p>
        <p>FVF-3: Vocabularies
and their terms can be
accessed using the
FVF-4: Vocabularies
and their terms are Metadata is offered in
registered or indexed in such a way that it can
a searchable engine or RDA-F4- be harvested and
a resource. 01M indexed
FVF-5: Vocabularies
and their terms are
retrievable using a
standardised
communications
protocol, preferably
open, free and
universally
implementable
protocols. and allows for
authentication and
authorisation, where
necessary.</p>
        <p>Metadata is accessed
RDA-A1- through standardised
04M protocol</p>
        <p>Data is accessible
RDA-A1- through standardised
04D protocol</p>
        <p>Metadata is accessible
RDA-A1. through a free access
1-01M protocol</p>
        <p>Data is accessible
RDA-A1. through a free access
1-01D protocol</p>
        <p>Data is accessible
through an access
protocol that supports
RDA-A1. authentication and
2-01D authorisation</p>
        <p>Metadata is guaranteed
to remain available after
RDA-A2- data is no longer
01M available
1
1
1
1
1
1
1
1
1
1</p>
        <p>The metadata is
provided in OWL format
Full and has been harvested
Complia by both OLS and
nce BioPortal</p>
        <p>EFO can be accessed
using HTTP protocol,
and it is an open-acess
ontology.</p>
        <p>EFO follows vocabulary
release guidelines, and
its versioned copies can
be found on Github. But
it doesn't strictly follows
Partial cross community
Complia language standards,
nce such as rdfs, xmls
FVF-7: Vocabularies
and their terms use a
formal, accessible and
broadly applicable, and
preferably
machine-understandabl
e language for
knowledge
representation.</p>
        <p>FVF-8: Vocabularies
and terms use qualified
references to other
vocabularies.</p>
        <p>FVF-9: Vocabularies
and terms are described
with a plurality of
accurate and relevant
attributes.</p>
        <p>FVF-10: Vocabularies
are released with a
standard data usage
licence, preferably
machine-readable
RDA-I3- Data includes qualified
02D references to other data 1</p>
        <p>Metadata includes
RDA-I3- qualified references to
03M other metadata</p>
        <p>Plurality of accurate and
RDA-R1 relevant attributes are
-01M provided to allow reuse</p>
        <p>Metadata includes
information about the
RDA-R1. licence under which the
1-01M data can be reused
1
0
1
1
1
1
1
1</p>
        <p>Full
Complia
nce
Full
Complia
nce
Full
Complia
nce</p>
        <p>EFO can be
downloaded in OWL
and OBO, which are
standardised format and
machine-understandabl
e.</p>
        <p>EFO reuses terms from
other vocabularies and
provides suffient
reference, such as
source of the external
term.</p>
        <p>FVF-11: Vocabularies
meet domain relevant
community standards.</p>
        <p>Metadata refers to a
RDA-R1. machine-understandabl
1-03M e reuse licence
RDA-R1. Metadata complies with
3-01M a community standard
RDA-R1. Data complies with a
3-01D community standard</p>
        <p>Metadata is expressed
in compliance with a
RDA-R1. machine-understandabl
3-02M e community standard</p>
        <p>Data is expressed in
compliance with a
RDA-R1. machine-understandabl
3-02D e community standard
1
1
1
1
1</p>
        <p>Full
Complia
nce</p>
        <p>EFO uses the standard
OWL format, complies
with OBO principles and
imports terms following
the MIREOT standards.</p>
        <p>Supplementary Table 7: FAIR assessment results of ICD-11
FAIR vocabulary feature summary
05/2021
ICD-11 browser and ICD11 print
version:https://icd.who.int/en print
version:https://icd.who.int/browse11/Downloads/Download?
fileName=print_en.zip
FAIR vocabulary
Feature
RDA-F1- Metadata is identified by
01M a persistent identifier
RDA-F1- Data is identified by a
01D persistent identifier</p>
        <p>Metadata is identified by
RDA-F1- a globally unique
02M identifier
FVF-1: Vocabulary and
their terms are assigned
globally unique and
persistent identifiers.</p>
        <p>RDA-F1- Data is identified by a
02D globally unique identifier 0</p>
        <p>Metadata is accessed
RDA-A1- through standardised
04M protocol</p>
        <p>Data is accessible
RDA-A1- through standardised
04D protocol</p>
        <p>Metadata is accessible
RDA-A1. through a free access
1-01M protocol</p>
        <p>Data is accessible
RDA-A1. through a free access
1-01D protocol</p>
        <p>Data is accessible
through an access
protocol that supports
RDA-A1. authentication and
2-01D authorisation
1
1
1
0
1
1
0
1
1
1
1
FVF-3: Vocabularies
and their terms can be
accessed using the
identifiers, preferably by
both human and
machine.</p>
        <p>FVF-4: Vocabularies
and their terms are Metadata is offered in
registered or indexed in such a way that it can
a searchable engine or RDA-F4- be harvested and
a resource. 01M indexed
FVF-5: Vocabularies
and their terms are
retrievable using a
standardised
communications
protocol, preferably
open, free and
universally
implementable
protocols. and allows for
authentication and
authorisation, where
necessary.</p>
        <p>ICD-11 uses HTTPS
protocol.https://icd.who.i
nt/browse11/l-m/en#http
%3a%2f%2fid.who.int%
2ficd%2fentity%2f91170
7612</p>
        <p>Metadata includes
provenance information
according to
RDA-R1. community-specific
2-01M standards</p>
        <p>Metadata uses
knowledge
representation
RDA-I1- expressed in
01M standardised format</p>
        <p>Data uses knowledge
representation
RDA-I1- expressed in
01D standardised format</p>
        <p>Metadata uses
machine-understandabl
RDA-I1- e knowledge
02M representation</p>
        <p>Data uses
machine-understandabl
RDA-I1- e knowledge
02D representation
RDA-I3- Data includes qualified
02D references to other data 0</p>
        <p>Metadata includes
RDA-I3- qualified references to
03M other metadata
1
1
0
0
0
0
0
1</p>
        <p>FVF-8: Vocabularies
and terms use qualified
references to other
vocabularies.</p>
        <p>Previous versions of
ICD-11 can access at
https://icd.who.int/brows
e11/l-m/en/releases.</p>
        <p>However, the versioning
style does not follow
common community
standards.</p>
        <p>ICD-11 is published
mainly as a pdf
document and doesn't
use standard
vocabulary formats.</p>
        <p>No
Complia
nce</p>
        <p>Terms in ICD-11 doesn't
refer to other terms.</p>
        <p>Partial The ICD-11 description
Complia refers to other projects
nce and publications.</p>
        <p>Plurality of accurate and
RDA-R1 relevant attributes are
-01M provided to allow reuse</p>
        <p>ICD-11 contains only a
minimum description of
each disease.</p>
        <p>FVF-10: Vocabularies
are released with a
standard data usage
licence, preferably
machine-readable
licence.
RDA-R1. Metadata refers to a
1-02M standard reuse licence 1</p>
        <p>Metadata refers to a
RDA-R1. machine-understandabl
1-03M e reuse licence</p>
        <p>Data is expressed in
compliance with a
RDA-R1. machine-understandabl
3-02D e community standard 0</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <article-title>[1] Creative commons | attribution 4.0 international | CC BY 4</article-title>
          .0, https: //creativecommons.org/licenses/by/4.0/
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <article-title>[2] ID policy</article-title>
          , http://www.obofoundry.org/id-policy
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <article-title>[3] The MIT license j open source initiative</article-title>
          , https://opensource.org/ licenses/MIT
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>F-</surname>
          </string-name>
          <article-title>UJI automated FAIR data assessment tool (</article-title>
          <year>2020</year>
          ), https://www. fairsfair.eu/f-uji
          <article-title>-automated-fair-data-assessment-tool</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <article-title>[5] FAIR data maturity model: speci cation and guidelines - draft (</article-title>
          <year>2020</year>
          ), https://www.rd-alliance. org/group/fair
          <article-title>-data-maturity-model-wg/outcomes/ fair-data-maturity-model-specification-and-guidelines</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>FAIRassist.</surname>
          </string-name>
          org (
          <year>2021</year>
          ), https://fairassist.org/#!/
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>Antoniou</surname>
          </string-name>
          , G., van
          <string-name>
            <surname>Harmelen</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          :
          <article-title>Web ontology language: OWL</article-title>
          . pp.
          <volume>67</volume>
          {
          <fpage>92</fpage>
          .
          <source>International Handbooks on Information Systems</source>
          , Springer (
          <year>2004</year>
          ). https://doi.org/10.1007/978-3-
          <fpage>540</fpage>
          -24750-04
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>Arnaud</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cooper</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shrestha</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          , et al.:
          <article-title>Towards a reference plant trait ontology for modeling knowledge of plant traits and phenotypes (</article-title>
          <year>2012</year>
          ), http://wrap.warwick.ac.uk/59831/
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <surname>Blum</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chang</surname>
            ,
            <given-names>H.Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chuguransky</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , et al.:
          <article-title>The InterPro protein families and domains database: 20 years on 49, D344{d354 (</article-title>
          <year>2021</year>
          ). https://doi.org/10.1093/nar/gkaa977
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>Bodenreider</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          :
          <article-title>The uni ed medical language system (UMLS): integrating biomedical terminology 32</article-title>
          ,
          <issue>D267</issue>
          {
          <volume>270</volume>
          (
          <year>2004</year>
          ). https://doi.org/10.1093/nar/gkh061
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Burdett</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Xu</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Courtot</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          , et al.:
          <source>FAIRplus: D3</source>
          .
          <article-title>2 IMI FAIR metrics publication</article-title>
          . https://doi.org/10.5281/zenodo.4428633
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>Consortium</surname>
            ,
            <given-names>T.G.O.:</given-names>
          </string-name>
          <article-title>The gene ontology resource: 20 years and still GOing strong 47, D330{d338 (</article-title>
          <year>2019</year>
          ). https://doi.org/10.1093/nar/gky1055
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>Consortium</surname>
          </string-name>
          , T.U.:
          <article-title>UniProt: the universal protein</article-title>
          knowledgebase in
          <year>2021</year>
          49,
          <issue>D480</issue>
          {d489 (
          <year>2021</year>
          ). https://doi.org/10.1093/nar/gkaa1100
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <surname>Courtot</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gibson</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lister</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , et al.:
          <article-title>MIREOT: the minimum information to reference an external ontology</article-title>
          term pp.
          <volume>1</volume>
          {
          <issue>1</issue>
          (
          <year>2009</year>
          ). https://doi.org/10.1038/npre.
          <year>2009</year>
          .
          <volume>3576</volume>
          .1
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <surname>Cox</surname>
            ,
            <given-names>S.J.D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gonzalez-Beltran</surname>
            ,
            <given-names>A.N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Magagna</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          , et al.:
          <article-title>Ten simple rules for making a vocabulary FAIR 17(6</article-title>
          ),
          <year>e1009041</year>
          (
          <year>2021</year>
          ). https://doi.org/10.1371/journal.pcbi.1009041
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <surname>Drysdale</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cook</surname>
            ,
            <given-names>C.E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Petryszak</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          , et al.:
          <article-title>The ELIXIR core data resources: fundamental infrastructure for the life sciences 36(8</article-title>
          ),
          <volume>2636</volume>
          {
          <fpage>2642</fpage>
          (
          <year>2020</year>
          ). https://doi.org/10.1093/bioinformatics/btz959
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <surname>Foundry</surname>
            ,
            <given-names>T.O.</given-names>
          </string-name>
          :
          <article-title>The OBO at le format speci cation</article-title>
          ,
          <source>version 1</source>
          .2, https: //owlcollab.github.io/oboformat/doc/GO.format.obo-
          <volume>1</volume>
          %5F2.html
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <surname>Foundry</surname>
            ,
            <given-names>T.O.</given-names>
          </string-name>
          :
          <article-title>OBO foundry principles, overview</article-title>
          , http://www. obofoundry.org/principles/fp-000-summary.html
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <surname>Foundry</surname>
            ,
            <given-names>T.O.</given-names>
          </string-name>
          :
          <article-title>PURL administration</article-title>
          , https://purl.prod.archive.org/
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <surname>Foundry</surname>
            ,
            <given-names>T.O.</given-names>
          </string-name>
          :
          <source>Versioning (principle 4)</source>
          , http://www.obofoundry.org/ principles/fp-004-versioning.html
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <surname>Garijo</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Corcho</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Poveda-Villalon</surname>
            ,
            <given-names>M.:</given-names>
          </string-name>
          <article-title>FOOPS!: An ontology pitfall scanner for the FAIR</article-title>
          principles p.
          <volume>4</volume>
          (
          <issue>2021</issue>
          ), http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>2980</volume>
          /paper321.pdf
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <surname>Garijo</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Poveda-Villalon</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Best practices for implementing FAIR vocabularies and ontologies on the web (</article-title>
          <year>2020</year>
          ), http://arxiv.org/abs/
          <year>2003</year>
          .13084
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <surname>Hastings</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Owen</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dekker</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , et al.:
          <article-title>ChEBI in 2016: Improved services and an expanding collection of metabolites 44,</article-title>
          <issue>D1214</issue>
          {
          <volume>1219</volume>
          (
          <year>2016</year>
          ). https://doi.org/10.1093/nar/gkv1031
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <surname>Hugo</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Le Franc</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Coen</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          , et al.:
          <source>D2</source>
          .
          <article-title>5 FAIR semantics recommendations second iteration (</article-title>
          <year>2020</year>
          ). https://doi.org/10.5281/zenodo.4314321
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <surname>Jupp</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Burdett</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Malone</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , et al.:
          <article-title>A new ontology lookup service at EMBL-</article-title>
          EBI p.
          <volume>2</volume>
          (
          <issue>2015</issue>
          ), http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>1546</volume>
          /paper%5F29.pdf
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <surname>Krajewski</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chen</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cwiek</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          , et al.:
          <article-title>Towards recommendations for metadata and data handling in plant phenotyping 66(18</article-title>
          ),
          <volume>5417</volume>
          {
          <fpage>5427</fpage>
          (
          <year>2015</year>
          ). https://doi.org/10.1093/jxb/erv271
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <surname>Malone</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Holloway</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Adamusiak</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          , et al.:
          <article-title>Modeling sample variables with an experimental factor ontology 26(8</article-title>
          ),
          <volume>1112</volume>
          {
          <fpage>1118</fpage>
          (
          <year>2010</year>
          ). https://doi.org/10.1093/bioinformatics/btq099
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <surname>Mungall</surname>
            ,
            <given-names>C.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Torniai</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gkoutos</surname>
            ,
            <given-names>G.V.</given-names>
          </string-name>
          , et al.:
          <article-title>Uberon, an integrative multi-species anatomy ontology 13(1</article-title>
          ),
          <source>R5</source>
          (
          <year>2012</year>
          ). https://doi.org/10.1186/gb-2012
          <source>-13-1-r5</source>
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [29]
          <string-name>
            <surname>Ochoa</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hercules</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Carmona</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          , et al.:
          <article-title>Open targets platform: supporting systematic drug{target identi cation and prioritisation 49, D1302{ d1310 (</article-title>
          <year>2021</year>
          ). https://doi.org/10.1093/nar/gkaa1027
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [30]
          <string-name>
            <surname>Organization</surname>
            ,
            <given-names>W.H.</given-names>
          </string-name>
          :
          <article-title>International classi cation of diseases for mortality and morbidity statistics (10h revision) (</article-title>
          <year>2010</year>
          ), https://www.who.int/ classifications/icd/ICD10Volume2_en_
          <year>2010</year>
          .pdf
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          [31]
          <string-name>
            <surname>Organization</surname>
            ,
            <given-names>W.H.</given-names>
          </string-name>
          :
          <article-title>International classi cation of diseases for mortality and morbidity statistics (11th revision) (</article-title>
          <year>2021</year>
          ), https://icd.who.int/ browse11/l-m/en
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          [32]
          <string-name>
            <surname>Pedruzzi</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rivoire</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Auchincloss</surname>
            ,
            <given-names>A.H.</given-names>
          </string-name>
          , et al.:
          <article-title>HAMAP in 2015: updates to the protein family classi cation and annotation system 43, D1064{d1070 (</article-title>
          <year>2015</year>
          ). https://doi.org/10.1093/nar/gku1002
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          [33]
          <string-name>
            <surname>Sansone</surname>
            ,
            <given-names>S.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>McQuilton</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rocca-Serra</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          , et al.:
          <article-title>FAIRsharing as a community approach to standards, repositories</article-title>
          and policies
          <volume>37</volume>
          (
          <issue>4</issue>
          ),
          <volume>358</volume>
          {
          <fpage>367</fpage>
          (
          <year>2019</year>
          ). https://doi.org/10.1038/s4158701900808
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          [34]
          <string-name>
            <surname>Smith</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ashburner</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rosse</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          , et al.:
          <article-title>The OBO foundry: coordinated evolution of ontologies to support biomedical data integration 25(11</article-title>
          ),
          <volume>1251</volume>
          {
          <fpage>1255</fpage>
          (
          <year>2007</year>
          ). https://doi.org/10.1038/nbt1346
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          [35]
          <article-title>Snomed: SNOMED home page</article-title>
          , https://www.snomed.org/
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          [36]
          <string-name>
            <surname>Taylor</surname>
            ,
            <given-names>C.F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Field</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sansone</surname>
            ,
            <given-names>S.A.</given-names>
          </string-name>
          , et al.:
          <article-title>Promoting coherent minimum reporting guidelines for biological and biomedical investigations: the MIBBI project 26(8</article-title>
          ),
          <volume>889</volume>
          {
          <fpage>896</fpage>
          (
          <year>2008</year>
          ). https://doi.org/https://doi.org/110.1038/nbt.1411
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          [37]
          <article-title>W3id: w3id.org - permanent identi ers for the web</article-title>
          , https://w3id.org/
        </mixed-citation>
      </ref>
      <ref id="ref38">
        <mixed-citation>
          [38]
          <string-name>
            <surname>Whetzel</surname>
            ,
            <given-names>P.L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Noy</surname>
            ,
            <given-names>N.F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shah</surname>
            ,
            <given-names>N.H.</given-names>
          </string-name>
          , et al.:
          <article-title>BioPortal: enhanced functionality via new web services from the national center for biomedical ontology to access and use ontologies in software applications 39, W541{w545 (</article-title>
          <year>2011</year>
          ). https://doi.org/10.1093/nar/gkr469
        </mixed-citation>
      </ref>
      <ref id="ref39">
        <mixed-citation>
          [39]
          <string-name>
            <surname>Wilkinson</surname>
            ,
            <given-names>M.:</given-names>
          </string-name>
          <article-title>The FAIR maturity evaluation service (</article-title>
          <year>2021</year>
          ), https:// fairsharing.github.io/FAIR-Evaluator-FrontEnd/#!/
        </mixed-citation>
      </ref>
      <ref id="ref40">
        <mixed-citation>
          [40]
          <string-name>
            <surname>Wilkinson</surname>
            ,
            <given-names>M.D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dumontier</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Aalbersberg</surname>
            ,
            <given-names>I.J.</given-names>
          </string-name>
          , et al.:
          <article-title>The FAIR guiding principles for scienti c data management and stewardship 3(1</article-title>
          ),
          <volume>160018</volume>
          (
          <year>2016</year>
          ). https://doi.org/10.1038/sdata.
          <year>2016</year>
          .18
        </mixed-citation>
      </ref>
      <ref id="ref41">
        <mixed-citation>
          [41]
          <string-name>
            <surname>Wimalaratne</surname>
            ,
            <given-names>S.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Juty</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kunze</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , et al.:
          <article-title>Uniform resolution of compact identi ers for biomedical data 5, 180029 (</article-title>
          <year>2018</year>
          ). https://doi.org/10.1038/sdata.
          <year>2018</year>
          .
          <volume>29</volume>
          <fpage>2021</fpage>
          -
          <lpage>08</lpage>
          -
          <issue>02</issue>
          <year>Release 2021</year>
          -
          <volume>07</volume>
          -02
          <fpage>2021</fpage>
          -
          <lpage>08</lpage>
          -02 ICD11 provides licensing
        </mixed-citation>
      </ref>
      <ref id="ref42">
        <mixed-citation>
          documentation. https://icd.who.int/en/do cs/ICD11-license.pdf https://icd.who.int/brows e11.
          <article-title>Licensed under Creative Commons Attribution-NoDerivative s 3.0 IGO licence (CC BY-ND 3.0 IGO)</article-title>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>