<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Building Standardized Semantic Web RESTful Services to Support ICD-11 Revision</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Guoqian Jiang</string-name>
          <email>jiang.guoqian@mayo.edu</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Harold R. Solbrig</string-name>
          <email>solbrig.harold@mayo.edu</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Christopher G. Chute</string-name>
          <email>chute@mayo.edu</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Health Sciences Research, Division of Biomedical Statistics &amp; Informatics, Mayo Clinic College of Medicine</institution>
          ,
          <addr-line>Rochester, MN.</addr-line>
          <country country="US">U.S.A</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>1999</year>
      </pub-date>
      <abstract>
        <p>For the beta phase of the 11th revision of International Classification of Diseases (ICD-11), the World Health Organization (WHO) intends to accept public input through a distributed model of authoring. To enable social computation and collaboration among broader communities, it is imperative but remains challenging to have an open-data access solution for ICD-11 contents. The objective of the present study is to describe our experiences and challenges on building the Simple Knowledge Organization System (SKOS) compliant Semantic Web RESTful services that aim to support the authoring application development of ICD-11. We propose a framework of building such services, which is composed of four layers: a semantic normalization layer, a semantic repository layer, a semantic services layer and an application layer. For the prototype implementation, we first generated the normalized ICD-11 linked RDF data through utilizing WHO ICD URI scheme and SKOS RDF data model. We used the Linked Data API and developed simple RESTful services for the standardized ICD-11 contents. We performed a preliminary evaluation on the usefulness of the system in terms of the feasibility of the services and its potential in support of building the Common Terminology Services 2 (CTS2) compliant services. In conclusion, we developed an effective framework of building standardized Semantic Web RESTful services on ICD-11 linked data, which would be useful to support ICD-11 revision.</p>
      </abstract>
      <kwd-group>
        <kwd>ICD-11 Revision</kwd>
        <kwd>Common Terminology Services 2 (CTS2)</kwd>
        <kwd>Linked Open Data (LOD)</kwd>
        <kwd>Simple Knowledge Organization System (SKOS)</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1 Introduction</title>
      <p>
        The 11th revision of International Classification of Diseases (ICD-11) was officially
launched by the World Health Organization (WHO) in March 2007 [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Historically,
ICD was developed to support international comparison of mortality statistics. WHO
has embraced a broadened set of use cases to drive ICD-11 development, including
scientific consensus of clinical phenotype (definition and criteria), public health
surveillance (e.g. mortality and morbidity) and clinical data aggregation [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. Instead
of using a traditional manual revision process leveraging international expertise, the
ICD-11 revision aims to digitize the content of ICD in a scientific way with modern
knowledge representation standards.
      </p>
      <p>
        The beta phase of the ICD-11 revision started in May 2012, and WHO intends to
accept public input through a distributed model of authoring. For the beta phase, there
are a number of major challenges to the existing authoring framework and workflow
[
        <xref ref-type="bibr" rid="ref3 ref4">3-4</xref>
        ]. First, the user community will potentially be large. Users will want to be able to
access and download target ICD-11 contents into their own evaluation tools and/or try
alternatives in their own environment. Second, the system must enable a sophisticated
user role definition with a hierarchy of authority and a fine-grained permission matrix.
This will be very important for supporting efficient social computing and for building
communities in collaboration. Third, the system must support an open-data solution to
lower the barriers for ICD data access. This openness will form the basis to enable
human social computation and will empower the users to create their own views of
data to satisfy different job roles and changing needs [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. The present study is focused
on the third challenge, i.e. building an open-data solution for ICD-11 data access.
      </p>
      <p>
        Semantic Web technology provides a scalable framework for facilitating semantic
data integration of heterogeneous resources and enabling semantic sharing through
the standard query services. In particular, an emerging Semantic Web technology,
known as Linked Open Data (LOD), has the potential to enable a revolution in how
data is accessed and utilized. In the biomedical domains, Linked Data principles and
practices have been adopted by an increasing number of projects. For instance,
Linking Open Drug Data (LODD) is a task force within the W3C Health Care and
Life Sciences Interest Group (HCLS IG). LODD has created Linked Data
representations of the data sets about drugs, and identified interesting scientific and
business questions that can be answered once the data sets are connected [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. The
National Center of Biomedical Ontology (NCBO) is embracing Semantic Web
technologies as a common backend, and to extend the scope of its ontology access
services to include RDF/OWL capabilities [
        <xref ref-type="bibr" rid="ref2 ref7">2, 7</xref>
        ]. Scalable and standardized
knowledge bases are beginning to come into play, as typified by the repository of
adverse drug events (ADEs) using the Linked Data technology [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. In this study, we
consider that the Linked Data technology will provide a scalable framework and
solution for dealing with the challenge on the open data access of ICD-11 content.
      </p>
      <p>
        The Simple Knowledge Organization System (SKOS), a W3C recommendation
and standard, is a common data model for sharing and linking knowledge
organization systems via the Web [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. The SKOS data model provides a standard,
low-cost migration path for porting existing knowledge organization systems to the
Semantic Web.
      </p>
      <p>The objective of the present study is to describe our experiences and challenges on
building SKOS compliant Semantic Web RESTful services that aim to support the
authoring application development of ICD-11. We propose a framework of building
such services, which is composed of four layers: a semantic normalization layer, a
semantic repository layer, a semantic services layer and an application layer. For the
prototype implementation, we first generated the normalized ICD-11 linked RDF data
through utilizing WHO ICD URI scheme and SKOS RDF data model. We used the
Linked Data API and developed simple RESTful services for the standardized ICD-11
contents. We performed a preliminary evaluation on the usefulness of the system in
terms of the feasibility of the services and its potential in support of building the
Common Terminology Services 2 (CTS2) compliant services.</p>
    </sec>
    <sec id="sec-2">
      <title>2 Background</title>
    </sec>
    <sec id="sec-3">
      <title>2.1 ICD-11 Content Model</title>
      <p>
        The purpose of ICD-11 content model is to present the knowledge that underlies the
definitions of an ICD entity. The content model is composed of three layers: a
foundation component, a linearization component and an ontological component [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ].
The foundation component stores the full range of knowledge of all classification
units in ICD. The linearization component corresponds to the classical print versions
of ICD. The ontological component provides references to formal definition of terms
and relationships. For the ontological component, WHO has sought to reuse existing
ontologies including the Systematized Nomenclature of Medicine – Clinical Terms
(SNOMED CT). SNOMED CT is the most comprehensive clinically oriented medical
terminology system. It is owned and maintained by the International Health
Terminology Standard Development Organization (IHTSDO). The IHTSDO and the
WHO signed a collaborative agreement in July 2010, aiming at enabling
harmonization – as complementary tools – of WHO Classifications and SNOMED
CT, which essentially establishes SNOMED CT as the core of the ontological
component of ICD [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. The distributed authoring process envisioned for ICD will
focus on the foundation layer, drawing on SNOMED terms for definitional context
and populating the details of a rubric such as anatomy, severity, histopathology, or
symptoms. Attributes within the foundation layer will enable the algorithmic
derivation of a linearization from the foundation layer, thus unifying all linearizations
with a common foundation core [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ].
      </p>
    </sec>
    <sec id="sec-4">
      <title>2.2 ICD URI Scheme</title>
      <p>
        A Uniform Resource Identifier (URI) is commonly used to name and identify a
resource, which has been treated as one of core principles of Semantic Web Linked
Open Data by Tim Berners-Lee [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]. A Restful, resource-oriented service exposes a
URI for every piece of data the client might want to operate on. A well-designed URI
structure/scheme allows users easily explore and extend ICD data through an
implementation of the RESTful services. Recently, an ICD URI scheme is proposed
for naming and supporting web services by WHO. A base URI of http://id.who.int
has been proposed, with http://id.who.int/icd/schema as the prefix for the vocabulary
terms that related to ICD classification efforts maintained by WHO,
http://id.who.int/icd/entity for the fundamental foundation entities related to ICD
concepts.
      </p>
    </sec>
    <sec id="sec-5">
      <title>2.3 SKOS RDF Model</title>
      <p>
        The SKOS data model views a knowledge organization system as a concept scheme
comprising a set of concepts [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. The vocabulary used in the SKOS data model is a set
of URIs that specifies the notion of SKOS concepts, concept schemes, lexical labels,
notations, documentation properties and semantic relations. SKOS data are expressed
as RDF triples. An increasing number of SKOS datasets in RDF are publicly
available. A typical example is the SKOS version of AGROVOC Agricultural
Thesaurus provided by the Food and Agriculture Organization (FAO) of the United
Nations (http://aims.fao.org/).
      </p>
    </sec>
    <sec id="sec-6">
      <title>2.4 CTS2 Standard</title>
      <p>
        The Common Terminology Services 2 (CTS2) standard defines the functional
requirements of a set of service interfaces to allow the representation, access, and
maintenance of terminology content either locally, or across a federation of
terminology service nodes [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. Note that the CTS2 standard has also incorporated the
principles of Representational State Transfer (REST) web service in its specification.
The CTS2 platform independent model (PIM) is rendered in the Unified Modeling
Language (UML), which is publicly available for download [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. The CTS2 PIM
model describes the formal model and semantics of the various components. In the
present study, we evaluate the potential of the SKOS model in support of building
CTS2 compliant services on ICD-11 linked data.
      </p>
    </sec>
    <sec id="sec-7">
      <title>2.5 Linked Data API</title>
      <p>
        Linked Data API is a configurable API specification intended to support the creation
of simple RESTful APIs over RDF triple stores [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]. The API specification is
designed to be deployed as a proxy in front of a SPARQL endpoint to support: 1)
generation of documents (or information resources) for the publishing of Linked Data;
2) provision of sophisticated querying and data extraction features, without the need
for end-users to write SPARQL queries; 3) delivery of multiple output formats from
these APIs, including a simple serialization of RDF in JSON syntax. Figure 1 shows
system architecture of the Linked Data API. A prototype implementation of the
Linked Data API, known as Elda, is publicly available at [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ].
      </p>
    </sec>
    <sec id="sec-8">
      <title>3 System Architecture</title>
    </sec>
    <sec id="sec-9">
      <title>4 Prototype Implementation</title>
      <p>In the prototype implementation, we focused on normalizing the ICD-11 linked data
using the SKOS-based RDF model and building the SKOS compliant Semantic Web
RESTful services. Note that in the present study, we did not implement the CTS2
model as illustrated in the system architecture, instead we evaluated the potential of
the SKOS-based approach in support of building the CTS2 compliant services for the
ICD-11 linked data in the next section.</p>
    </sec>
    <sec id="sec-10">
      <title>4.1 Normalize the ICD-11 Data Using a SKOS-based RDF Model</title>
      <p>The first part of the normalization is to generate the unique identifier for each ICD-11
foundation entity. In this study, we utilized the uniform entity IDs generated by the
WHO. For example, for the entity “Acute myocardial infarction” in the current ICD
OWL-based rendering, the original ICD-10 code I21 is used as the entity ID, and the
URI of the entity is http://who.int/icd#I21. The uniform entity IDs 1634725920 is
generated by the WHO for this entity.</p>
      <p>The second part of the normalization is to implement the ICD URI scheme
proposed by the WHO. For example, the normalized URI for the entity “Acute
myocardial infarction” becomes http://id.who.int/icd/entity/1634725920.</p>
      <p>The third part of the normalization is to utilize the Simple Knowledge Organization
System (SKOS) RDF data model. In the current prototype implementation, we
defined the mappings between the ICD-11 model signatures and the SKOS model
signatures (Table 1). Figure 3 shows a normalized ICD-11 entity “Acute myocardial
infarction” using the SKOS data model.
icd - http://id.who.int/icd/schema/
skos - http://www.w3.org/2004/02/skos/core#/</p>
      <p>SKOS Predicate
skos:ConceptScheme,
skos:inScheme
skos:Concept
skos:notation
skos:prefLabel
skos:altLabel
skos:definition
skos:broaderTransitive
skos:narrowerTransitive</p>
    </sec>
    <sec id="sec-11">
      <title>4.2 Build Semantic Web RESTful Services for the ICD-11 Linked</title>
    </sec>
    <sec id="sec-12">
      <title>Data</title>
      <p>We used the Elda implementation of Linked Data API. We created an API
specification file as required by the Elda. Basically we specified the following
modules for generating a RESTful endpoint as a series of steps. First, we defined two
URI templates that identified the set of URLs to which it should respond; one is a List
Template that selects all ICD foundation entities and the other is an Item Template
that selects a single ICD entity resource. Second, we specified a number of Selectors
that describe how to retrieve a list of resources from the configured SPARQL
endpoint.</p>
      <p>
        Here, we use the SPARQL endpoint enabled by a 4store built-in SPARQL server.
The 4store is an open source RDF store developed at the Garlik [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]. By defining the
Selectors, the RESTful endpoint actually converted them to the corresponding
SPARQL queries to retrieve the desired resources. Third, we defined a set of named
Viewers that describe specific sets of properties to retrieve about each resource. There
is a default viewer that can retrieve all properties available for a specific category. In
the prototype implementation, we also defined three customized viewers, the first one
called “label viewer” that shows the skos:notation and skos:prefLabel for a specific
entity; the second one called “definition viewer” that shows the definition of an entity
in addition to those defined in the “label viewer”; the third one called “hierarchy
viewer” that shows the children and parent of an entity in addition to those defined in
the “label viewer”. Fourth and finally, we used the default named Formatters that
define the possible representation formats that can be delivered to a client. Table 2
shows the RESTful APIs for an ICD entity “1634725920”, its definition view and
available formats.
      </p>
      <p>Using the Elda implementation of Linked Data API, we successfully established a
RESTful endpoint over the ICD-11 Linked Data. The RESTful endpoint can be
accessed as either a List Endpoint or an Item Endpoint. For the List Endpoint, it will
provide the RDF graph data for all ICD entities with pagination using the following
URL: {baseURL}/icd?_page=0. For the Item Endpoint, it will provide a RDF graph
data for a single ICD entity resource. An example URL is:
{baseURL}/icd/entity/1634725920. To access the renderings in different formats such
as json, xml, rdf, ttl, etc., end users just need to specify the URLs as illustrated in
Table 2.</p>
      <p>Table 2. The example of RESTful APIs for an ICD entity “1634725920”</p>
      <p>RESTful APIs
return json format
return xml format
return rdf format
return turtle format
return plain text format
return plain html format
return html format
The ultimate goal of building the standardized Semantic Web RESTful services on
the ICD-11 linked data is to support the authoring application development of the
ICD-11. We believe the usefulness of the system can be evaluated in the following
two aspects: 1) to test whether it is feasible to build AJAX (i.e. an acronym for
Asynchronous JavaScript and XML) widgets utilizing the RESTful services
developed in this study; and 2) to evaluate the potential for extending the
SKOSbased approach in support of building CTS2 compliant services on the ICD-11 linked
data.</p>
      <p>
        For the first aspect, we invited two experienced web developers at Mayo Clinic to
evaluate the feasibility of the services to support the AJAX widget development. A
subjective 1-5 scale survey is used for both the feasibility evaluation, with 1
indicating “strongly disagree” and 5 indicating “strongly agree”. As the results, both
of web developers strongly agreed (i.e. scored as 5) that it is feasible to utilize the
RESTful services to build the AJAX widgets in support of collaborative authoring of
ICD-11. In addition, we are working on testing the services using a real world AJAX
application - an ICD Proposal Authoring Widget based on the SmartGWT (developed
in our ongoing study at Mayo Clinic[
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]). The widget will interface with the
standardized Semantic Web RESTful services developed in the present study for the
ICD-11 content access. Figure 4 shows a screenshot of the ICD Proposal Authoring
Widget.
      </p>
      <p>
        For the second aspect, we tried to define the mappings between the SKOS model
predicates and the CTS2 model predicates. Since the SKOS data model is a proper
subset of the CTS2 model, we were able to define the mappings between the SKOS
model predicates and the CTS2 model predicates. Table 3 shows the mappings
between CTS2 predicates and SKOS predicates for this preliminary evaluation. The
CTS2 predicate is defined in CTS2 entity description model [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ].
      </p>
      <p>SKOS Predicate
rdf:about
icd11:(skos:notation)
skos:inScheme
skos:prefLabel,
skos:altLabel,
skos:hiddenLabel</p>
    </sec>
    <sec id="sec-13">
      <title>6 Discussion</title>
      <p>In this study, we developed a framework of building standardized Semantic RESTful
services on ICD-11 linked data that are transformed from an ICD-11 relational
database. In the prototype implementation of the framework, we have utilized a
number of open source Semantic Web applications: Protégé OWL/RDF exporter,
4store RDF store and Elda implementation of Linked Data API. As shown in the
semantic normalization layer in Figure 3, we used the Protégé OWL/RDF exporter to
generate an OWL/RDF rendering of ICD-11 contents for subsequent normalization.
The 4store RDF store enabled the storage of ICD-11 linked data and a built-in
SPARQL endpoint server. The Elda Linked Data API provides a highly configurable
mechanism that allowed us to build the simple RESTful services on top of a SPARQL
endpoint.</p>
      <p>
        The original rationale of the Linked Data API is that simple RESTful APIs are well
supported and understood by a large community of web developers and SPARQL is a
power tool whose sophistication is unnecessary for many users [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]. The goal of the
API is to provide some basic functionality “out-of-the-box”, but provide a range of
configuration options to support additional, domain-specific customization and
functionality. Basically, the Linked Data API supports encoding the requests from the
end users in a API specification file and the underlying engine will convert the
specification into the corresponding SPARQL queries, retrieve back the requested
data in RDF graph and expose the data as the RESTful services in a number of
easyto-process representation of resources such as simple JSON, or XML, etc. In the
biomedical domains, for instance, the similar approach was used to build a
lightweight web service interface for Semantic Web contents integrating multiple life
science databases in a Semantic-JSON project [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ].
      </p>
      <p>
        Combining the strengths of the Semantic Web application components above, we
actually developed an open-data solution that would be useful to maximize the data
accessibility to support the authoring application development of the ICD-11 for the
beta phase. This solution has made the infrastructure of ICD-11 data available as the
RESTful web services such that the ICD-11 linked data are pretty friendly to be
consumed by many existing AJAX or AJAX-like frameworks and toolkits. The
Exhibit, for example, is a lightweight framework for publishing structured data on
standard Web servers that requires no installation, database administration, or
programming [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ]. Exhibit has been used widely in many web applications and
projects such as a patient record data collection project [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ], a collaborative platform
for representation and harmonization of clinical data elements [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ], etc. The Google
Web Toolkit (GWT) is another example of such an AJAX-based framework that has
been used to develop the WHO ICD-11 alpha authoring tool [
        <xref ref-type="bibr" rid="ref3 ref4">3-4</xref>
        ]. In addition, WHO
is currently using a .NET-based browser to expose ICD-11 alpha content [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]. The
browser provides simple commenting functionality to allow the domain professionals
to make comments on existing contents, and it intends to introduce more social
computing capabilities. As the browser is actually using some AJAX widgets, we
consider that the browser should be able to consume the ICD-11 contents through the
standardized services developed in the present study. As part of our preliminary
evaluation, we evaluated the feasibility of the RESTful services developed in this
study in support of the AJAX widgets. Two experienced web developers all strongly
agreed that it is highly feasible to build the AJAX widgets by utilizing the RESTful
services.
      </p>
      <p>We have made efforts on normalizing the ICD-11 linked data using the SKOS RDF
model along with the WHO ICD URI scheme. The SKOS model provides a set of
Semantic Web friendly signatures with well-defined semantics. For instance, the
SKOS not only provides a rich set of labeling and document properties (e.g.,
skos:prefLabel, skos:altLabel, skos: note, skos:definition, etc), but also provides
welldifferentiated hierarchical semantic relationships, e.g., differentiating between
skos:broader and skos:broaderTransitive and between skos:narrower and skos:
narrowerTransitive. We consider that the use of the SKOS model would facilitate the
access and use of ICD-11 classification by many existing SKOS-aware Semantic Web
tools. In addition, the SKOS RDF data model is defined as an OWL full ontology, so
it is highly expressive and extensible. We consider that this data model is particularly
suitable for representing the foundation component in the ICD-11, which is based on
a highly complex model involving many complicated use cases. Along with the
SKOS RDF model, we have implemented the ICD URI scheme proposed by WHO.
Note that the URI scheme is under active discussion in the WHO ICD revision
community. We consider that this implementation played a role in examining the
proposed URI scheme. For example, in the SKOS model, an ICD entity is represented
as an instance of skos:concept, in which a property skos:inScheme is used to specify
the context information about concept scheme. We found that the URI for naming
such concept scheme is required though original proposal did not cover this
requirement.</p>
      <p>In addition, we have made efforts in evaluating the potential of the ICD-11 SKOS
model in support of building the CTS2 compliant Semantic Web RESTful services.
We envision that the CTS2 standard will provide additional layer of standardization
that would potentially optimize the interoperability between downstream authoring
applications. The goal of the CTS2 standard is to provide a common model and
semantics for the representation, interchange and federation of terminological
resources. For the ICD-11 revision use case, part of the input process involves a way
for users to download, test and evaluate ICD-11 in a variety of situations, including its
alignment with SNOMED CT and other terminologies. Another component of that
process involves the ability to propose additions and changes – something that, if
successful, will need to be managed in a semi-automated fashion. The key is, in part,
a public, open-data model and associated contents that cover not just ICD-11 but
ICD-10, SNOMED CT, ICD and others, and allow proposed changes to be accepted,
reviewed, revised and adopted or rejected in a semi-automated fashion. The emerging
CTS2 standard embodies many of these characteristics already, which is why our
study evaluates it as a possible solution. One of CTS2 efforts is on defining a
canonical RDF specification that establishes the RDF tags and structure to represent
the various CTS2 resources in RDF format. The SKOS rendering in this paper is
considerably close to this target. In the preliminary evaluation, we have identified an
initial set of mappings between the SKOS predicates and the CTS2 signatures. We
consider the mappings would serve as a starting point in support of building the CTS2
compliant services in the future.</p>
      <p>There are a number of limitations in this study. Firstly, the mappings among
ICD11 content model, SKOS data model, CTS2 standard and have not yet been
extensively discussed in the ICD-11 community, although we consider the draft
mappings presented in this study are accurate in certain degree as the authors have
been involved in the specifications of these models. Secondly, although the
preliminary evaluation results are promising, it seems clear that we could obtain more
reliable results by inviting more web developers and CTS2 experts for the evaluation.
Thirdly, the RESTful services developed in this study are limited to a number of
common use cases. A systematic review and collection of the use cases and
requirements from the ICD-11 revision community would be helpful to make the
services more useful to the broader parties in the community.</p>
      <p>In summary, we described our experiences and challenges on building the SKOS
compliant Semantic Web RESTful services that aim to support the collaborative
authoring application development of ICD-11. We have demonstrated the usefulness
of the proposed framework in terms of its feasibility and its potential in support of
building CTS2 compliant services. In the future studies, we plan to integrate CTS2
standard to consolidate and standardize the RESTful services based on the evaluation
from the ICD-11 community and to build reusable AJAX widgets library utilizing the
ICD-11 RESTful services. We expect our efforts in the present study would facilitate
data accessibility and system interoperability that have been challenging in the
ICD11 community.</p>
      <p>Acknowledgement
The authors would like to thank two experienced web developers: Kevin Peterson and
Cory Endle and two CTS2 experts: Craig Stancl, Harold Solbrig, who participated in
the evaluation of the study.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1. WHO.
          <article-title>Revision of the International Classification of Diseases (ICD)</article-title>
          . Available from: http://www.who.int/classifications/icd/ICDRevision/en/index.html.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Chute</surname>
            <given-names>CG</given-names>
          </string-name>
          .
          <article-title>Distributed biomedical terminology development: from experiments to open process</article-title>
          .
          <source>Yearb Med Inform</source>
          .
          <year>2010</year>
          :
          <fpage>58</fpage>
          -
          <lpage>63</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <given-names>T.</given-names>
            <surname>Tudorache</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. I.</given-names>
            <surname>Nyulas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. F.</given-names>
            <surname>Noy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Redmond</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Musen</surname>
          </string-name>
          .
          <source>iCAT: A Collaborative Authoring Tool for ICD-11. Workshop on Ontologies come of Age in the Semantic Web at ISCW</source>
          <year>2011</year>
          , Bonn, Germany. Published in
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Tudorache</surname>
            <given-names>T</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Falconer</surname>
            <given-names>S</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nyulas</surname>
            <given-names>C</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Storey</surname>
            <given-names>MA</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ustün</surname>
            <given-names>TB</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Musen</surname>
            <given-names>MA</given-names>
          </string-name>
          .
          <article-title>Supporting the Collaborative Authoring of ICD-11 with WebProtégé</article-title>
          .
          <source>AMIA Annu Symp Proc. 2010 Nov</source>
          <volume>13</volume>
          ;
          <year>2010</year>
          :
          <fpage>802</fpage>
          -
          <lpage>6</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <source>WHO Software Development Meeting</source>
          <year>2011</year>
          : http://sites.google.com/site/icd11revision/home/face-to
          <article-title>-facemeetings/software-development-</article-title>
          <string-name>
            <surname>meeting-</surname>
          </string-name>
          11
          <string-name>
            <surname>-</surname>
          </string-name>
          april-2011
          <source>; last visited at September</source>
          <volume>28</volume>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Samwald</surname>
            <given-names>M</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jentzsch</surname>
            <given-names>A</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bouton</surname>
            <given-names>C</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kallesøe</surname>
            <given-names>CS</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Willighagen</surname>
            <given-names>E</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hajagos</surname>
            <given-names>J</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Marshall</surname>
            <given-names>MS</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Prud'hommeaux</surname>
            <given-names>E</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hassanzadeh</surname>
            <given-names>O</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pichler</surname>
            <given-names>E</given-names>
          </string-name>
          and
          <string-name>
            <surname>Stephens</surname>
            <given-names>S.</given-names>
          </string-name>
          <article-title>Linked open drug data for pharmaceutical research and development</article-title>
          .
          <source>Journal of Cheminformatics</source>
          <year>2011</year>
          ,
          <volume>3</volume>
          :
          <fpage>19</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7. BioPortal: http://bioportal.bioontology.org/;
          <source>last visited September 28</source>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Jiang</surname>
            <given-names>G</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Solbrig</surname>
            <given-names>H</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chute</surname>
            <given-names>CG</given-names>
          </string-name>
          .
          <article-title>ADEpedia: A scalable and standardized knowledge base of adverse drug events using Semantic Web technology</article-title>
          .
          <source>AMIA Annu Symp Proc</source>
          .
          <year>2011</year>
          ;
          <year>2011</year>
          :
          <fpage>607</fpage>
          -
          <lpage>16</lpage>
          . Epub 2011 Oct 22.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9. SKOS Reference: http://www.w3.org/TR/2009/REC-skos-reference20090818/.
          <source>Last visited at October 1</source>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10. ICD-11 Content Models in UML: http://informatics.mayo.edu/icdmodel.
          <source>Last visited at September</source>
          <volume>27</volume>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <article-title>Agreement between IHTSDO and</article-title>
          WHO URL: http://www.who.int/classifications/AnnouncementLetter.pdf.
          <source>Last visited at August 8</source>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Linked</surname>
          </string-name>
          Data - Design issues: http://www.w3.org/DesignIssues/LinkedData.html; last visited at September 27,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13. CTS2 wiki: http://informatics.mayo.
          <source>edu/cts2. Last visited at September</source>
          <volume>28</volume>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Linked Data</surname>
            <given-names>API</given-names>
          </string-name>
          : http://code.google.com/p/linked-data-api/; last visited at September 28,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <article-title>Elda implementation of Linkded Data API</article-title>
          : http://code.google.com/p/elda/; last visited at September 28,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16. 4store RDF store: http://4store.org.
          <source>last visited at September</source>
          <volume>27</volume>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Jiang</surname>
            <given-names>G</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Solbrig</surname>
            <given-names>HR</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chute</surname>
            <given-names>CG</given-names>
          </string-name>
          .
          <article-title>Using semantic web technology to support ICD-11 textual definitions authoring</article-title>
          .
          <source>ACM International Conference Proceeding Series</source>
          .
          <year>2012</year>
          ;
          <fpage>38</fpage>
          -
          <lpage>44</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Kobayashi</surname>
            <given-names>N</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ishii</surname>
            <given-names>M</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Takahashi</surname>
            <given-names>S</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mochizuki</surname>
            <given-names>Y</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Matsushima</surname>
            <given-names>A</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Toyoda</surname>
            <given-names>T</given-names>
          </string-name>
          .
          <article-title>Semantic-JSON: a lightweight web service interface for Semantic Web contents integrating multiple life science databases</article-title>
          .
          <source>Nucleic Acids Res</source>
          .
          <year>2011</year>
          Jul;
          <volume>39</volume>
          (
          <issue>Web Server issue</issue>
          ):
          <fpage>W533</fpage>
          -
          <lpage>40</lpage>
          . Epub 2011 Jun 1.
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Huynh</surname>
            <given-names>D</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Karger</surname>
            <given-names>D</given-names>
          </string-name>
          and
          <string-name>
            <surname>Miller R. Exhibit</surname>
          </string-name>
          :
          <article-title>Lightweight structured data publishing</article-title>
          .
          <source>In Proceedings of the 16th international conference on World Web Web; page 746. ACM</source>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Ogbuji</surname>
            <given-names>C.</given-names>
          </string-name>
          <article-title>A role for Semantic Web technologies in patient record data collection</article-title>
          . In the book - Linking
          <string-name>
            <surname>Enterprise Data (Wood D</surname>
          </string-name>
          , Ed.),
          <source>2010 - ISBN 978-1-4419-7664-2.</source>
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <surname>Jiang</surname>
            <given-names>G</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Solbrig</surname>
            <given-names>HR</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Iberson-Hurst</surname>
            <given-names>D</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kush</surname>
            <given-names>RD</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chute</surname>
            <given-names>CG</given-names>
          </string-name>
          .
          <article-title>A Collaborative Framework for Representation and Harmonization of Clinical Study Data Elements Using Semantic MediaWiki</article-title>
          .
          <source>AMIA Summits Transl Sci Proc. 2010 Mar</source>
          <volume>1</volume>
          ;
          <year>2010</year>
          :
          <fpage>11</fpage>
          -
          <lpage>5</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22. WHO ICD-11 Beta Browser: http://apps.who.int/classifications/icd11/browse/f/en.
          <source>Last visited at October 1</source>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>