<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>FedViz: A Visual Interface for SPARQL Queries Formulation and Execution</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Syeda Sana e Zainab</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Muhammad Saleem</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Qaiser Mehmood</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Durre Zehra</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Stefan Decker</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Ali Hasnain</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Insight Centre for Data Analytics, National University of Ireland</institution>
          ,
          <addr-line>Galway</addr-line>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Universita ̈t Leipzig, IFI/AKSW</institution>
          ,
          <addr-line>PO 100920, D-04009 Leipzig</addr-line>
        </aff>
      </contrib-group>
      <fpage>49</fpage>
      <lpage>60</lpage>
      <abstract>
        <p>Health care and life sciences research heavily relies on the ability to search, discover, formulate and correlate data from distinct sources. Over the last decade the deluge of health care life science data and the standardisation of linked data technologies resulted in publishing datasets of great importance. This emerged as an opportunity to explore new ways of bio-medical discovery through standardised interfaces. Although the Semantic Web and Linked Data technologies help in dealing with data integration problem there remains a barrier adopting these for non-technical research audiences. In this paper we present FedViz, a visual interface for SPARQL query formulation and execution. FedViz is explicitly designed to increase intuitive data interaction from distributed sources and facilitates federated as well as non-federated SPARQL queries formulation. FedViz uses FedX for query execution and results retrieval. We also evaluate the usability of our system by using the standard system usability scale as well as a custom questionnaire, particularly designed to test the usability of the FedViz interface. Our overall usability score of 74.16% suggests that FedViz interface is easy to learn, consistent, and adequate for frequent use.</p>
      </abstract>
      <kwd-group>
        <kwd>SPARQL</kwd>
        <kwd>Life Sciences (LS)</kwd>
        <kwd>Query Federation</kwd>
        <kwd>Visual Query Formulation</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        The researchers in health care, life sciences and biomedical (also known as domain
users) adopted Semantic Web and Linked Data technologies due to the data integration
challenges faced as a result of excessive data produced [
        <xref ref-type="bibr" rid="ref16 ref6">6,16</xref>
        ]. Different researchers
recommended the use of SPARQL services for publishing biomedical resources [
        <xref ref-type="bibr" rid="ref19 ref2 ref20">2,20,19</xref>
        ].
The use of these technologies facilitate the domain users for issuing structured SPARQL
queries over highly heterogeneous data spread over diverse data sources [
        <xref ref-type="bibr" rid="ref1 ref5">5,1</xref>
        ]. Such
structured queries are vital, not only in order to query relevant data regarding different
entities e.g. Drugs, Molecules and Pathways but also to drive meaningful biomedical
correlations such as Drug Drug Interactions and Protein Protein Interactions etc. Such
retrieved information can subsequently be applied to various bioinformatics tasks such
as functional analysis, protein modelling or image analysis. As pointed out earlier that
in the most of cases, the required information to draw any biological correlation or to
answer a biological question involve querying multiple data source, provided by different
providers, sometimes available in different format with different accessing mechanism.
Meaningful biological query such as “Find out the Diseases that causes due to the
deficiency of Iodine” can only be answered by querying and aggregating data from
multiple reliable data sources. The use of Semantic Web and Linked Data technologies
are commonly exploited by computer scientists, who can formulate structured SPARQL
queries to access data from different SPARQL endpoints, the ultimate end-users and
the domain experts either biologists or clinical researchers, remain unable to assemble
complex queries in order to access such data [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. Making complex SPARQL queries to
drive necessary information to support clinical experiments and observations poses a
barrier in health care and life sciences domain that confront the adoption and acceptance
of such technologies. Moreover, even for computer scientists, assembling a federated
SPARQL query is time-consuming and technical process since it requires the knowledge
of underlying datasets schema and the connectivity between the datasets [
        <xref ref-type="bibr" rid="ref10 ref9">9,10</xref>
        ]. An
alternative to this is an intuitive and interactive platform that can facilitate domain
users to assemble complex but meaningful SPARQL query through visual interface. To
this end, we introduce FedViz which enables a user to formulate and execute complex
federated SPARQL queries using intuitive visual query interface. FedViz allows user to
select concepts and properties from multiple datasets using nodes and edges, assemble
SPARQL query in a background independent of user involvement and allow users to
edit the resultant SPARQL query before sending it to the SPARQL query federated
engine. Assembled query is executed through FedX- a state of the art engine [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ], that
federates the query to relevant data sources and retrieves the results. The choice of FedX
was due to the fact it can execute both federated (both SPARQL 1.0 and SPARQL 1.1)
and non federated queries. At present, six real time biomedical data sources, i.e., Kegg,
Drugbank, DailyMed, Medicare, Sider, and Diseasome are selected to visually construct
the SPARQL query. However, FedViz can be generalise to any set of datasets.
      </p>
      <p>The remaining part of this paper is organised as follows: we highlight the related
work in section 2. Later we present the motivational use case in section 3. We introduce
our methodology and FedViz salient features in section 4. Subsequently, we present
a thorough evaluation of FedViz in section 5. We finally conclude the paper with an
overview of future work.</p>
    </sec>
    <sec id="sec-2">
      <title>2 Related work</title>
      <p>
        Several approaches have been proposed for Visual query formulation over Linked data.
Form-based querying is one of the famous paradigm, where Form elements (i.e. filters,
variables, identifiers) are used for query formulation. Example of this approach is
SPARQLViz [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. However it is less flexible and allows only those users with some knowledge
of RDF and SPARQL language. In Graph-based querying paradigm query is formulated
using node-link diagrams and this approach is more flexible as compared to Form-based
paradigm and requires the RDF notations of subject-predicate-object cause barrier for
users with limited semantic web knowledge. Examples for such approaches include
NITELIGHT [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ], iSPARQL1, RDF-GL [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] and ReVeaLD [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. QueryVOWL[
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] uses
      </p>
      <sec id="sec-2-1">
        <title>1 http://oat.openlinksw.com/isparql/</title>
        <p>Listing 1.1: Find all the drugs and their interactions for curing thyroid disease.
PREFIX drugbank : &lt;h t t p : / / www4. wiwiss . fu−b e r l i n . de / drugbank / r e s o u r c e / drugbank/&gt;
PREFIX d i s e a s o m e :&lt; h t t p : / / www4. wiwiss . fu−b e r l i n . de / d i s e a s o m e / r e s o u r c e / d i s e a s o m e /&gt;
S e l e c t D i s t i n c t ? i n t e r a c t i o n D r u g 1 ? i n t e r a c t i o n D r u g 2 ? t e x t ? name
WHERE
{? Drugbank0 a drugbank : d r u g i n t e r a c t i o n s ;
drugbank : i n t e r a c t i o n D r u g 1 ? i n t e r a c t i o n D r u g 1 ;
drugbank : i n t e r a c t i o n D r u g 2 ? i n t e r a c t i o n D r u g 2 ;
drugbank : t e x t ? t e x t .
? i n t e r a c t i o n D r u g 1 drugbank : p o s s i b l e D i s e a s e T a r g e t ? p o s s i b l e D i s e a s e T a r g e t .
? p o s s i b l e D i s e a s e T a r g e t d i s e a s o m e : name ? name .</p>
        <p>FILTER ( regex ( ? name , "thyroid" , "i" ) )
}
LIMIT 100
specific language and graph database. Most of aforementioned available systems focused
on query formulation using specific graphs, available predicate links and user may need
sufficient SPARQL knowledge using such system. FedViz is a step towards interactively
and intuitively formulating federated SPARQL queries using class and property links
visually presented per dataset.
3</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Motivation</title>
      <p>We believe FedViz enables a variety of use cases, of which one is explained as follows:
Drug-Drug Interaction for Medication of Certain Disease: When patients are
diagnosed with certain disease, a large number of drugs are associated with that depending
upon its stage and condition. It is imperative that physician are thoroughly educated about
drug-drug interaction before prescription for certain disease. Take hypothyroidism
for example. It is a disease which results from an under-active thyroid, leading to the
necessity of taking extrinsic thyroxine hormone to maintain normal bodily functions. One
treatment option for hypothyroidism is using Levothyroxine, which is a synthetic
thyroid hormone similar to T4 hormone, which is intrinsically produced by the thyroid
gland, deficiency of which leads to the disease in the first place.Levothyroxine has
many drug interactions, especially with the warfarin family and similar drugs, including
Acenocoumarol. It is an anticoagulant that functions as a Vitamin K antagonist, and
so controls clot formation in the body. Simultaneous use of Levothyroxine with
Acenocoumarol can sensitise the body to the latter, which may put the patient at an
increased risk of bleeding. This is just an example how FedViz can be used to monitor
interactions of a drug, in this particular case Levothyroxine, by creating a visual
query, making it easier for the physician to have a comprehensive look at the potential
contraindications to using the drug in particular patients (Listing 1.1).
4</p>
    </sec>
    <sec id="sec-4">
      <title>FedViz</title>
      <p>FedViz is an online application that provides Biologist a flexible visual interface to
formulate and execute both federated and non-federated SPARQL queries. It translates
the visually assembled queries into SPARQL equivalent and execute using query engine.
At present, FedViz visualises Life Sciences datasets and facilitates complex query
formulation and execution in order to draw meaningful biological co-relations including
drug-drug interaction, drug-disease interaction and drug-side effect correlations. Through
FedViz Biologist can formulate simple queries that typically involve single or multiple
concepts from one dataset as well as complex federated queries that might involve more
than one datasets with multiple constraints.
4.1</p>
      <sec id="sec-4-1">
        <title>Methodology</title>
        <p>Our methodology consists of two steps namely: 1) building visual interface and 2) result
retrieval using query engine (Figure 1).</p>
        <p>
          Building visual interface A concise graphical representation is needed to display
datasets to facilitate biologist in order to formulate query. We chose the concept map
approach [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ] for building the visual interface, which is a graphical method representing
the relationship between nodes and links, and has been used in various domains for
organising knowledge [
          <xref ref-type="bibr" rid="ref24">24</xref>
          ]. Using this approach in FedViz, we represent concepts as
big circular nodes (drugs, disease etc) and properties as small circular nodes (protein
sequence, possible disease target etc). As mentioned earlier, currently FedViz contains six
datasets and their concepts with associated properties are visualised for query formulation
also known as catalogue (Fig 1). Each dataset represented in catalogue is marked with
unique colour. The nodes are modelled as objects in a two-dimensional system using a
force-directed layout[
          <xref ref-type="bibr" rid="ref23">23</xref>
          ]. In force-directed layout nodes repel each other based on their
sizes that prevents overlapping and increases concept-property visibility to end-user.
Result Retrieval Using Query Engine To process the FedViz query request, FedX
the state of the art efficient SPARQL query federation engine [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ] is chosen to execute
both federated (SPARQL 1.1 and SPARQL 1.0) and non-federated queries. FedViz
provides the set of required SPARQL endpoints (i.e., data sources) URLs in order to
enable FedX’s query execution. Overall, the query execution works as follow: (1) FedViz
formulate SPARQL query and sends to FedX, (2) FedX executes the query and sends
back the results to FedViz, (3) FedViz presents the results to end user.
        </p>
        <p>
          Technologies FedViz is browser-based client application that provides biologist a
flexible front-end. To build this application variety of web technologies are used
including HTML5, CSS, JavaScript, JQuery2, Java Servlet, SVG3, AJAX4 and JSON5. The
datasets visualisation is based on SVG (Scaler Vector Graphics) with Javascript usage.
In catalogue, datasets are represented in JSON format and displayed as nodes (Concept
and Properties). The communication between the client query and federated query
engine(FedX) has done by AJAX calls through middle layer. Open source Javascript library
D3.js[
          <xref ref-type="bibr" rid="ref4">4</xref>
          ] is used to implement force-directed layout for datasets visualisation.
        </p>
        <sec id="sec-4-1-1">
          <title>2 https://jquery.com/</title>
          <p>3 www.w3schools.com/svg/
4 http://api.jquery.com/jquery.ajax/
5 http://json.org/
Availability The FedViz application can be accessed at http://srvgal86.deri.ie/FedViz/
index.html. Example queries both simple (include single dataset) and complex (include
more than single dataset) are provided at https://goo.gl/AOJGpu.
4.2</p>
        </sec>
      </sec>
      <sec id="sec-4-2">
        <title>Datasets</title>
        <p>
          Current version of FedViz supports a total of 6 real-world datasets. All the datasets were
collected from Life Sciences domains. We began by selecting two real world datasets
from Fedbench [
          <xref ref-type="bibr" rid="ref21">21</xref>
          ] namely Drugbank6 a knowledge base containing information of
drugs, their composition and their interactions with other drugs and Kegg Kyoto
Encyclopedia of Genes and Genomes (KEGG)7 which contains further information about
chemical compounds and reactions with a focus on information relevant for geneticists.
Apart from aforementioned selected datasets four other datasets were chosen that had
connectivity with the existing ones that enabled us to include real federated queries.
These datasets include Sider8- that contains information on marketed drugs and their
        </p>
        <sec id="sec-4-2-1">
          <title>6 http://www.drugbank.ca/</title>
          <p>7 http://www.genome.jp/kegg/
8 http://wifo5-03.informatik.uni-mannheim.de/sider/
adverse effects, Diseasome9- that publishes a network of 4,300 disorders and disease
genes linked by known disorder-gene associations for exploring all known phenotype
and disease gene associations, indicating the common genetic origin of many diseases.,
Dailymed10- provides information about marketed drugs including the chemical structure
of the compound, its therapeutic purpose, its clinical pharmacology, warnings,
precautions, adverse reactions, over dosage etc., and Medicare11. Figure 2, shows the topology
of all 6 datasets while some other basic statistics like the total number of triples, the
number of resources, predicates and objects, as well as the number of classes and the
number of links can be found in table 1.
4.3</p>
        </sec>
      </sec>
      <sec id="sec-4-3">
        <title>Query Formulation</title>
        <p>In this section, an example scenario is discussed to demonstrate our visual query
formulation process.</p>
        <p>Drug-Disease and Drug-Compound interaction: Drugs with their compound mass for
curing disease Anemia. This query requires data integration from Drugbank (containing
drugs information), Diseasome (containing disease information) and Kegg(containing
compound mass information) and can be formulated by using the following step-by-step
approach (ref., Fig. 3):</p>
        <sec id="sec-4-3-1">
          <title>9 http://wifo5-03.informatik.uni-mannheim.de/diseasome/</title>
          <p>10 http://dailymed.nlm.nih.gov/dailymed/index.cfm
11 http://wifo5-03.informatik.uni-mannheim.de/medicare/
Dataset</p>
          <p>Triples Subjects Predicates Objects Classes
1. The first step is to identify how Drugbank, Diseasome and Kegg datasets are
connected to each other? This connectivity (i.e., via classes drugbank:drug,
diseasome:disease and kegg:compound can be found by using the Class
visualisation view of FedViz that shows all classes of datasets along with there
connectivity (ref., Fig. 4).
2. User selects Drugbank from the Datasets Selection box (window A).
3. The visualisation for Drugbank dataset can be seen in window B where he selects
drugbank:drug class and its properties(i.e., drugs:possibleDiseaseTarget
and drugs:keggCompoundId).
4. Step 2 and 3 are now followed for Diseasome dataset, i.e., select diseasome:disease
class and it’s name property (window C) and for Kegg dataset, i.e., select kegg:compound
class and it’s mass property (window D).
5. Selected Concepts are shown in status bar (window E).
6. Next, FedViz SPARQL Query Editor allows user to add constraints to the
formulated federated query such as select projection variables, apply SPARQL LIMIT,
FILTER(in this scenario disease name Anemia), ORDERY BY clauses, and can
further edit the query according to his choice (window Fa, Fb).
7. The final query can be seen on submission (window G).
8. Query is executed over FedX and the retrieved results are displayed by FedViz
(Result window H).
9. Finally, by selecting any URI from the retrieved result, FedViz can provide detailed
information regarding that instance (Data Exploration window I).
4.4</p>
        </sec>
      </sec>
      <sec id="sec-4-4">
        <title>Query Execution</title>
        <p>
          On dispatching from FedViz, SPARQL query is received and handled by an intermediate
layer (IL) built on top of FedX [
          <xref ref-type="bibr" rid="ref22">22</xref>
          ]. The IL acts as an adopter, which allows the FedX to
communicate with outer world (i.e, Web). FedX requires the set of endpoints URLs as
input to query execution engine. The FedViz request incorporates the set of endpoints
required by the query. The IL forwards the endpoints to FedX query engine by selecting
endpoints from request. FedX executes a SPARQL ASK requests on set of endpoints.
Furthermore, FedX optimise the query by splitting it into sub-queries. The selected
endpoints are requested to run these sub-queries to generate the results. Finally, all the
retrieved results from various sub-queries are integrated and displayed through FedViz
interface.
5
        </p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>Evaluation</title>
      <p>
        The goal of our evaluation is to quantify the usability and usefulness of FedViz graphical
interface. We evaluate the usability of the interface by using the standard System Usability
Scale (SUS) [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] as well as a customised questionnaire designed for the users of our
system. In the following, we explain the survey outcomes.
5.1
      </p>
      <sec id="sec-5-1">
        <title>System Usability Scale Survey</title>
        <p>
          In this section, we explain the SUS questionnaire12 results. This survey is more general
and applicable to any system to measure the usability. The SUS is a simple, low-cost,
reliable 10 item scale that can be used for global assessments of systems usability[
          <xref ref-type="bibr" rid="ref14 ref17">14,17</xref>
          ].
As of 10th July 2015, 15 users13 including researchers and engineers in Semantic Web
were participated in survey. According to SUS, we achieved a mean usability score of
74.16% indicating a high level of usability according to the SUS score. The average
scores (out of 5) for each survey question along with standard deviation is shown in
Figure 5.
        </p>
        <p>The responses to question 1 (average score to question 1 = 3.8 ± 0.86) suggests that
FedViz is adequate for frequent use. The responses to question 3 indicates that FedViz is
easy to use (average score 4 ± 0.84) and the responses to question 7 (average score 4.06
12 SUS survey can found at: http://goo.gl/forms/bhReuNgd6O
13 Users from AKSW, University of Leipzig and INSIGHT Centre, National University of Ireland,
Galway. Summary of the responses can be found at: https://goo.gl/ZOrJx9</p>
        <p>I needed to learn a lot of things before I could get going with this system (10)</p>
        <p>I felt very confident using the system (9)</p>
        <p>I found the system very cumbersome to use (8)
I would imagine that most people would learn to use this system very quickly (7)</p>
        <p>I thought there was too much inconsistency in this system (6)</p>
        <p>I found the various functions in this system were well integrated (5)
I think that I would need the support of a technical person to be able to use this system (4)</p>
        <p>I thought the system was easy to use (3)</p>
        <p>I found the system unnecessarily complex (2)
I think that I would like to use this system frequently (1)
0
1
2
3
4
5
6
± 0.96) suggests that most people would learn to use this system very quickly. However,
the slightly higher standard deviation to question 9 (standard deviation = ± 1.05) and
question 10 (standard deviation = ± 1.16) suggest that we may need a user manual to
explain the different functionalists provided by the FedViz interface.
This survey14 was particularly designed to measure the usability and usefulness of the
different functionalists provided by FedViz. In particular, we asked users to formulate
both federated and non-federated SPARQL queries and share their experience through
question 10 and question 11. As of 10th July 2015, 10 researchers including Computer
Scientist15 and Bioinformaticians were participated in survey. The average scores (out of
5 with 1 means strongly disagree and 5 means strongly agree) for each survey question
along with standard deviation is shown in Figure 6. The average scores to question
10 (i.e., 4.2 ± 0.91) and question 11 (i.e., 3.9 ± 0.73) show that most of the user feel
confident in formulating simple and federated queries, respectively. The responses to
question 2 (average score = 4.4 ± 0.69) suggests that navigating on different datasets
are much easy by using FedViz ”Selection Box”. A slightly lower scores to question
7 (average score = 3.5 ± 0.70) suggests that we need to further improve the datasets
visualisation component of the FedViz.</p>
        <p>As an overall usability evaluation, our SUS and custom surveys outcome suggest
that FedViz interface is easy to use, consistent, adequate for frequent use, easy to learn,
and the various functions in the system are well integrated.
14 Custom survey can be found at: http://goo.gl/forms/2DWvK2qYsV
15 Summary of the responses can be found at: https://goo.gl/tT8TXF</p>
        <p>How easy is the visualisation to formulate complex federated SPARQL Query? (11)</p>
        <p>How easy is the visualisation to formulate simple SPARQL Query? (10)
How relevant do you think is the federated results you getare for your daily research? (9)</p>
        <p>How easily you can explorefurther details of theretreiveresults? (8)
How would you categorize your experiencewhileusing the Dataset Visualization? (7)</p>
        <p>How easy is itfor you to getresults of your query? (6)
How easy is itfor you to make federated query on different datasets by using Query Edit</p>
        <p>page? (5)
How easy is itfor you to make query of individual Dataset by using Query Edit page? (4)
How easy is itfor you to exploreeach datasetwhile clicking on its conceptand find their</p>
        <p>properties? (3)
How easy is itfor you to navigateon differentdatasets using Selection box on the top? (2)
How easy is itfor you to hover on all datasets and find their links with each other on main
page? (1)
0
1
2
3
4
5
6</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>6 Conclusion and Future Work</title>
      <p>In this paper we introduce FedViz as a online interface for SPARQL query formulation
and execution. We evaluate our approach and usability of our system using the standard
system usability scale as well as through domain experts. Our preliminary analysis and
evaluation revels the overall usability score of 74.16%, concluding FedViz an interface,
easy to learn and help users formulating complex SPARQL queries intuitively. As a future
work we aim to extend FedViz with Faceted browsing and also provide visualization at
entity level e.g, Genes and Molecules where user can see the Gene sequences and 3D
structure for Molecules.</p>
    </sec>
    <sec id="sec-7">
      <title>7 Acknowledgement</title>
      <p>The work presented in this paper has been partly funded by Science Foundation Ireland
under Grant No. SFI/08/CE/I1380 (Lion-2).</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <given-names>J.</given-names>
            <surname>Almeida</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Deus</surname>
          </string-name>
          , and
          <string-name>
            <given-names>W.</given-names>
            <surname>Maass</surname>
          </string-name>
          .
          <article-title>Development of integrative bioinformatics applications using cloud computing resources and knowledge organization systems (kos)</article-title>
          .
          <source>Nature proceedings</source>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <given-names>F.</given-names>
            <surname>Belleau</surname>
          </string-name>
          , M.
          <article-title>-</article-title>
          <string-name>
            <surname>A. Nolin</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          <string-name>
            <surname>Tourigny</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          <string-name>
            <surname>Rigault</surname>
            , and
            <given-names>J. Morissette.</given-names>
          </string-name>
          <article-title>Bio2rdf: towards a mashup to build bioinformatics knowledge systems</article-title>
          .
          <source>Journal of biomedical informatics</source>
          ,
          <volume>41</volume>
          (
          <issue>5</issue>
          ):
          <fpage>706</fpage>
          -
          <lpage>716</lpage>
          ,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <given-names>J.</given-names>
            <surname>Borsje</surname>
          </string-name>
          and
          <string-name>
            <given-names>H.</given-names>
            <surname>Embregts</surname>
          </string-name>
          .
          <article-title>Graphical query composition and natural language processing in an rdf visualization interface</article-title>
          .
          <source>Erasmus School of Economics and Business Economics</source>
          , Vol. Bachelor. Erasmus University, Rotterdam,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <given-names>M.</given-names>
            <surname>Bostock</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Ogievetsky</surname>
          </string-name>
          , and
          <string-name>
            <surname>J. Heer.</surname>
          </string-name>
          <article-title>D3 data-driven documents</article-title>
          .
          <source>Visualization and Computer Graphics</source>
          , IEEE Transactions on,
          <volume>17</volume>
          (
          <issue>12</issue>
          ):
          <fpage>2301</fpage>
          -
          <lpage>2309</lpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <given-names>B.</given-names>
            <surname>Chen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. J.</given-names>
            <surname>Wild</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Q.</given-names>
            <surname>Zhu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Ding</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Dong</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Sankaranarayanan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Wang</surname>
          </string-name>
          , and
          <string-name>
            <given-names>Y.</given-names>
            <surname>Sun. Chem2bio2rdf</surname>
          </string-name>
          :
          <article-title>A linked open data portal for chemical biology</article-title>
          .
          <source>arXiv preprint arXiv:1012.4759</source>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <given-names>H.</given-names>
            <surname>Chen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Yu</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J. Y.</given-names>
            <surname>Chen</surname>
          </string-name>
          .
          <article-title>Semantic web meets integrative biology: a survey</article-title>
          . Briefings in bioinformatics,
          <volume>14</volume>
          (
          <issue>1</issue>
          ):
          <fpage>109</fpage>
          -
          <lpage>125</lpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <given-names>F.</given-names>
            <surname>Haag</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Lohmann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Siek</surname>
          </string-name>
          , and
          <string-name>
            <given-names>T.</given-names>
            <surname>Ertl</surname>
          </string-name>
          .
          <article-title>Visual querying of linked data with QueryVOWL</article-title>
          .
          <source>In Joint Proceedings of SumPre 2015</source>
          and
          <article-title>HSWI 2014-15</article-title>
          . CEUR-WS, to appear.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <given-names>A.</given-names>
            <surname>Hasnain</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Fox</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Decker</surname>
          </string-name>
          , and
          <string-name>
            <given-names>H. F.</given-names>
            <surname>Deus</surname>
          </string-name>
          .
          <article-title>Cataloguing and linking life sciences LOD Cloud</article-title>
          . In EKAW,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <given-names>A.</given-names>
            <surname>Hasnain</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. R.</given-names>
            <surname>Kamdar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Hasapis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Zeginis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. N. Warren</given-names>
            <surname>Jr</surname>
          </string-name>
          , et al.
          <article-title>Linked Biomedical Dataspace: Lessons Learned integrating Data for Drug Discovery</article-title>
          . In International Semantic Web Conference (In-Use Track),
          <year>October 2014</year>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <given-names>A.</given-names>
            <surname>Hasnain</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. S. E.</given-names>
            <surname>Zainab</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. R.</given-names>
            <surname>Kamdar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Q.</given-names>
            <surname>Mehmood</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. Warren</given-names>
            <surname>Jr</surname>
          </string-name>
          , et al.
          <article-title>A roadmap for navigating the life scinces linked open data cloud</article-title>
          .
          <source>In International Semantic Technology (JIST2014) conference</source>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <given-names>F.</given-names>
            <surname>Hogenboom</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Milea</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Frasincar</surname>
          </string-name>
          , and
          <string-name>
            <given-names>U.</given-names>
            <surname>Kaymak</surname>
          </string-name>
          .
          <article-title>Rdf-gl: a sparql-based graphical query language for rdf</article-title>
          .
          <source>In Emergent Web Intelligence: Advanced Information Retrieval</source>
          , pages
          <fpage>87</fpage>
          -
          <lpage>116</lpage>
          . Springer,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>D. H. Jonassen</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          <string-name>
            <surname>Beissner</surname>
            , and
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Yacci</surname>
          </string-name>
          .
          <article-title>Structural knowledge: Techniques for representing, conveying, and acquiring structural knowledge</article-title>
          . Psychology Press,
          <year>1993</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>M. R. Kamdar</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          <string-name>
            <surname>Zeginis</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Hasnain</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Decker</surname>
            , and
            <given-names>H. F.</given-names>
          </string-name>
          <string-name>
            <surname>Deus</surname>
          </string-name>
          .
          <article-title>Reveald: A user-driven domain-specific interactive search platform for biomedical research</article-title>
          .
          <source>Journal of biomedical informatics</source>
          ,
          <volume>47</volume>
          :
          <fpage>112</fpage>
          -
          <lpage>130</lpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <given-names>J. R.</given-names>
            <surname>Lewis</surname>
          </string-name>
          and
          <string-name>
            <surname>J. Sauro.</surname>
          </string-name>
          <article-title>The factor structure of the system usability scale</article-title>
          .
          <source>In HCD</source>
          .
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <given-names>A.</given-names>
            <surname>Russell</surname>
          </string-name>
          and
          <string-name>
            <given-names>P.</given-names>
            <surname>Smart. Nitelight</surname>
          </string-name>
          :
          <article-title>A graphical editor for sparql queries</article-title>
          .
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <given-names>A.</given-names>
            <surname>Ruttenberg</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Clark</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Bug</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Samwald</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            <surname>Bodenreider</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Chen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Doherty</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Forsberg</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Gao</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Kashyap</surname>
          </string-name>
          , et al.
          <article-title>Advancing translational research with the semantic web</article-title>
          .
          <source>BMC bioinformatics</source>
          ,
          <volume>8</volume>
          (
          <issue>Suppl 3</issue>
          ):
          <fpage>S2</fpage>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>M. Saleem</surname>
            ,
            <given-names>M. R.</given-names>
          </string-name>
          <string-name>
            <surname>Kamdar</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Iqbal</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Sampath</surname>
            ,
            <given-names>H. F.</given-names>
          </string-name>
          <string-name>
            <surname>Deus</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.-C. N.</given-names>
            <surname>Ngomo</surname>
          </string-name>
          .
          <article-title>Big linked cancer data: Integrating linked tcga and pubmed</article-title>
          .
          <source>Web Semantics: Science, Services and Agents on the World Wide Web</source>
          ,
          <volume>27</volume>
          :
          <fpage>34</fpage>
          -
          <lpage>41</lpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>M. Saleem</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          <string-name>
            <surname>Khan</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Hasnain</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          <article-title>Ermilov, and</article-title>
          <string-name>
            <given-names>A.-C. N.</given-names>
            <surname>Ngomo</surname>
          </string-name>
          .
          <article-title>A fine-grained evaluation of sparql endpoint federation systems</article-title>
          .
          <source>Semantic Web Journal</source>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>M. Saleem</surname>
            ,
            <given-names>S. S.</given-names>
          </string-name>
          <string-name>
            <surname>Padmanabhuni</surname>
            ,
            <given-names>A.-C. N.</given-names>
          </string-name>
          <string-name>
            <surname>Ngomo</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Iqbal</surname>
            ,
            <given-names>J. S.</given-names>
          </string-name>
          <string-name>
            <surname>Almeida</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Decker</surname>
            , and
            <given-names>H. F.</given-names>
          </string-name>
          <string-name>
            <surname>Deus</surname>
          </string-name>
          . Topfed:
          <article-title>Tcga tailored federated query processing and linking to lod</article-title>
          .
          <source>Journal of Biomedical Semantics</source>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>M. Samwald</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Jentzsch</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <string-name>
            <surname>Bouton</surname>
            ,
            <given-names>C. S.</given-names>
          </string-name>
          <string-name>
            <surname>Kallesøe</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          <string-name>
            <surname>Willighagen</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Hajagos</surname>
            ,
            <given-names>M. S.</given-names>
          </string-name>
          <string-name>
            <surname>Marshall</surname>
            , E. Prud'hommeaux,
            <given-names>O.</given-names>
          </string-name>
          <string-name>
            <surname>Hassanzadeh</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          <string-name>
            <surname>Pichler</surname>
          </string-name>
          , et al.
          <article-title>Linked open drug data for pharmaceutical research and development</article-title>
          .
          <source>Journal of cheminformatics</source>
          ,
          <volume>3</volume>
          (
          <issue>1</issue>
          ):
          <fpage>19</fpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <surname>M. Schmidt</surname>
            , O. Go¨rlitz, P. Haase,
            <given-names>G.</given-names>
          </string-name>
          <string-name>
            <surname>Ladwig</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Schwarte</surname>
            , and
            <given-names>T.</given-names>
          </string-name>
          <string-name>
            <surname>Tran</surname>
          </string-name>
          .
          <article-title>Fedbench: A benchmark suite for federated semantic data query processing</article-title>
          .
          <source>In The Semantic Web-ISWC</source>
          <year>2011</year>
          , pages
          <fpage>585</fpage>
          -
          <lpage>600</lpage>
          . Springer,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <string-name>
            <given-names>A.</given-names>
            <surname>Schwarte</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Haase</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Hose</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Schenkel</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Schmidt</surname>
          </string-name>
          . Fedx:
          <article-title>Optimization techniques for federated query processing on linked data</article-title>
          .
          <source>In The Semantic Web, ISWC</source>
          <year>2011</year>
          , volume
          <volume>7031</volume>
          of Lecture Notes in Computer Science, pages
          <fpage>601</fpage>
          -
          <lpage>616</lpage>
          .
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23.
          <string-name>
            <given-names>R.</given-names>
            <surname>Tamassia</surname>
          </string-name>
          .
          <article-title>Handbook of graph drawing and visualization</article-title>
          . CRC press,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          24.
          <string-name>
            <surname>J. D. Wallace</surname>
            and
            <given-names>J. J.</given-names>
          </string-name>
          <string-name>
            <surname>Mintzes</surname>
          </string-name>
          .
          <article-title>The concept map as a research tool: Exploring conceptual change in biology</article-title>
          .
          <source>Journal of research in science teaching</source>
          ,
          <volume>27</volume>
          (
          <issue>10</issue>
          ):
          <fpage>1033</fpage>
          -
          <lpage>1052</lpage>
          ,
          <year>1990</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>