There are many data sources containing overlapping information about Intrinsically Disordered Proteins (IDP). IDPcentral aims to be a registry to aid the discovery of data about proteins known to be intrinsically disordered by aggregating the content from these sources. Traditional ETL approaches for populating IDPcentral require the API and data model of each source to be wrapped and then transformed into a common model. In this paper, we investigate using Bioschemas markup as a mechanism to populate the IDPcentral registry by constructing the Intrinsically Disordered Protein Knowledge Graph (idp-kg). Bioschemas markup is a machine-readable, lightweight representation of the content of each page in the site that is embedded in the HTML. For any site it is accessible through a HTTP request. We harvest the Bioschemas markup in three IDP sources and show the resulting idp-kg has the same breadth of proteins available as the original sources, and can be used to gain deeper insight into their content by querying them as a single, consolidated knowledge graph.
One of the goals of the ELIXIR Intrinsically Disordered Protein (IDP)
community is to create a centralised registry for IDP data to support the community in
their data analyses. The registry will aggregate data contained in the
community's numerous specialist data sources, such as DisProt [
Bioschemas is a community e ort to provide machine-readable markup within
life sciences resources to increase their discoverability [
In this paper, we demonstrate that Bioschemas markup can be harvested to create a central repository of IDPs by creating the Intrinsically Disordered Proteins Knowledge Graph (idp-kg3). It is not su cient to simply harvest all the markup into a single data repository. The concepts within the markup need to be identi ed and reconciled since the sources contain overlapping, and potentially con icting, information about proteins but use di erent identi ers for the proteins. Therefore, the provenance of each statement should be tracked so that users of the registry can retrieve full details from the original data source. 2
We will now discuss the background material for our work. Note that throughout this paper we will use CURIEs to link to items in databases. These can be resolved using Identi ers.org. Similarly, ontology terms will be given as CURIEs that correspond to the widely used pre xes given in https://pre x.cc. 2.1
Schema.org provides a way to add semantic markup to web pages to enable
those web pages to become more understandable by the search engines that
index them, and therefore to improve search results [
The Schema.org vocabulary provides types which correspond to the things we can describe, and properties which capture the characteristics of those things. The majority of the vocabulary is focused on generic web search, e.g. books, movies, or places, but it also includes types relevant to science, e.g. Dataset, and most recently types have been added for Bioinformatics4 such as Gene, Protein, and Taxon. A major bene t of this approach is that the markup is accessible to all through a common API, i.e. HTTP Get requests, there is no need to learn and code for the REST API of each individual source.
The Bioschemas community (Bioschemas.org) promotes the use of Schema.org
markup within life sciences web resources to improve their Findability, and
provide lightweight Interoperability (c.f. the FAIR Data Principles) [
4 Schema.org v13.0 https://schema.org/version/13.0 accessed 30 Aug 2021. 1. Proposing extensions to the Schema.org vocabulary to include types and properties relevant for life sciences resources; and 2. Providing recommended usage pro les over Schema.org types.
Seven types covering key life sciences areas have been included in the Schema.org pending vocabulary. The Bioschemas community continue to work to add more types, e.g. the annotation of genes or proteins using a SequenceAnnotation type. The goal is not to replace existing life sciences ontologies, but to provide a lightweight vocabulary to aid discovery of resources. Once discovered it is expected that detailed biological models, captured with rich Interoperable ontologies, will be used to accurately describe the data.
For any given type in Schema.org, there can be a large number of properties available to use, many of which can be inherited from parent types. For example, the Dataset type has over 100 properties due to the inheritance from CreativeWork and Thing. This can make it di cult for developers of markup to know which properties to use, and certainly they are unlikely to use all. Bioschemas pro les5 provide usage guidelines for Schema.org types; identifying the most critical properties to aid search, and important properties for disambiguation; presented as minimal and recommended properties respectively. This provides a much smaller pool of properties for types relating to the life sciences.
The ELIXIR IDP community6 curates and maintains many data resources that function as the basis of the IDPcentral registry. These specialist data sources are built around a subset of proteins having the interesting property of being unstructured or structurally disordered. The structural and functional aspects of such proteins are covered in three distinct resources.
DisProt [
MobiDB [
The Protein Ensemble Database (PED) [
The creation of the idp-kg requires two steps. First we must harvest the markup from each of the data sources. Second we need to transform the source markup into the model for the knowledge graph, reconciling the multiple identi ers for a speci c protein into a single concept. 3.1
The markup was extracted from the three data sources using the Bioschemas Markup Scraper and Extractor (bmuse). Markup is extracted using HTTP Get requests which means that resource speci c APIs do not need to be coded for. To verify the correctness of the harvesting, we developed three datasets. BMUSE. The Bioschemas Markup Scraper and Extractor (bmuse7) is a data harvester developed speci cally to extract markup embedded within web pages. bmuse has been developed to extract markup embedded as either JSON-LD or RDFa, and also supports the use of both in the same page. The pages to be harvested can be static, or be single page applications (dynamic) that require JavaScript processing on the client side to generate the page content. bmuse harvests data from a given list of URLs or sitemaps; it does not perform web crawling by following links embedded within pages. A maximum number of pages to harvest per sitemap is also required.
For each page extracted, bmuse generates an n-quad le containing: 1. The extracted markup stored in an RDF named graph with the IRI of the named graph being uniquely constructed based on the date of the scrape and the page visited. Where the markup does not contain a subject IRI for the data, i.e. the JSON-LD markup does not include an @id attribute, bmuse substitutes in the page URL to avoid the use of blank nodes. 2. Provenance data about the data harvesting. This is stored in the default graph and describes the named graph in which the data is stored. The provenance data provided is: { URL of the page visited using pav:retrievedFrom { Date of extraction using pav:retrievedOn { The version of bmuse used to harvest the data using pav:createdWith
The pav:retrievedFrom property can be used to provide the links back from individual pieces of data to the source from which it came. The other two properties are primarily used for debugging purposes, although the retrieval date can also be used to ensure that the most up to date data is available in the generated idp-kg.
Harvested Data. To develop and test the data processing pipeline to be applied
to the harvested data, we used a series of test datasets. These correspond to data
harvested from the three data sources on 28 September 2021. We note that in
the initial run of bmuse 13 pages produced errors due to timeouts. These pages
were harvested in a second run with just those pages listed as targets.
Test-8: This dataset consists of eight sample pages that correspond to those
used in [
Full: This dataset contains all pages that could be harvested from the sitemaps of the three data sources. This contains the 5 to 9 pages of site structure per data source and all protein pages listed in the sitemap. This dataset is used to construct the idp-kg. 3.2
After the data has been harvested, it is processed so that information about a
particular protein, which can come from multiple sources, is consolidated into
a single concept for the protein, with links back to where each piece of data
originated. The data transformation process is available as a Jupyter Notebook9.
This is an extended version of the notebook presented in [
The notebook uses SPARQL CONSTRUCT queries to extract the data from the harvested pages and convert them into the idp-kg model, based on the Bioschemas vocabulary. While the queries are based on the properties listed in the corresponding Bioschemas pro le, they make extensive use of OPTIONAL clauses since the data does not always exactly correspond to the pro le. 8 The sitemap of each source is split into two entries in the bmuse con guration le. 9 https://github.com/AlasdairGray/IDP-KG/blob/main/notebooks/ETLProcess.
ipynb accessed Sept 2021
Bioschemas Pro les. Within the three data sources, we expected to nd markup conforming to the following Bioschemas pro les: { DataCatalog (v0.3-RELEASE) { Dataset (v0.3-RELEASE) { Protein (v0.11-RELEASE) { SequenceAnnotation (v0.1-DRAFT) { SequenceRange (v0.1-DRAFT) Additionally, within these pro les there are uses of the Schema.org types PropertyValue and DefinedTerm, which must be processed separately, and references to pages of type ScholarlyArticle.
While all the data conforms to the same data vocabulary, there are di erences in the underlying usage. DisProt and MobiDB provide protein centric representations of the data. PED provides a cluster of proteins on a single page. These di erences need to be consolidated into a coherent knowledge graph model centred around proteins.
Instance Merging. Each of the data sources uses their own identi er scheme to identify concepts in their data. Within the idp-kg, we need to aggregate the data from the multiple sources into a single consolidated entry, which will need its own identi er. In considering the di erent entity types, it was decided that only the proteins would be merged, as there is no clear way to decide when two annotations are equivalent and it is not expected that multiple instances of the Dataset and DataCatalog data would appear in the di erent datasets.
The IDPcentral team have decided they will use UniProt accessions [
https://idpcentral.org/id/<accession> where <accession> is replaced by the UniProt accession for the protein. Knowledge Graph Construction. While constructing the IDPcentral knowledge graph, it was assumed that the data sources would contain declarations of the same property of information, e.g. the name of the protein. However, we do not assume that they are consistent in their content. There are two cases to consider. The rst is that each source contains di erent values but these compliment each other, e.g. a list of synonyms where no source will necessarily have a complete set but by merging the data from the sources the IDPcentral knowledge graph would have a more complete set. The second case is where two sources have di ering values for a property which should have a single speci c value, e.g. protein name. Rather than decide that a speci c source's value should be used, we have decided to include all values available in the sources together with the provenance. Users of the data can then decide on the correct value, and feedback issues to the source with the erroneous value.
To support providing statement level provenance, we adopted the named
graph approach that was used in the Open PHACTS platform [
To verify the generated knowledge graph, we performed various data analyses.
These build on the queries from [
We rst give an overview of the idp-kg using the statistics recommended in the
HCLS Community Pro le for Dataset Descriptions [
Table 2 presents a comparison between the number of proteins found in the original data sources and the number in the idp-kg. The comparison gives the number of proteins in the di erent intersections of the data sources. The table shows that the data harvesting completely recreates the information available in the data sources.
Now that we have veri ed that the idp-kg is complete with respect to the content of the sources, we can use it to analyse the data available about IDPs. 10 https://github.com/AlasdairGray/IDP-KG/blob/main/notebooks/
AnalysisQueries.ipynb accessed Sept 2021 11 We have deployed the \Snorql - Extended Edition (https://github.com/ ammar257ammar/snorql-extended)" query interface at https://swel.macs.hw.ac. uk/idp with access to the same queries that are used in the analysis notebook.
KG Triples Subjects Properties Objects Classes Literals Graphs The answers presented in this section are possible due to the aggregation of the data into a single knowledge graph. We only perform the following analysis over the full idp-kg. The full set of responses to these queries are available through the notebook or idp-kg SPARQL endpoint.
From Table 1 we can see that there are 15,767 annotations on the proteins. These correspond to 11,046 from DisProt, 4,488 from MobiDB, and 233 from PED (annotations per dataset query). Using the annotations in multiple datasets query, we can see that there are 912 proteins with annotations from more than one dataset, with https://idpcentral.org/id/P04637 having a total 77 annotations, contributed by all 3 datasets. Using the annotations per article query, we nd that there are 2,578 distinct scholarly articles referenced in the annotations, with the article pubmed:20657787 providing 80 annotations. Finally, using the annotations per term code query, we found that 149 codes from the Intrinsically Disordered Protein Ontology are used, with IDPO:00076 (Disorder) being the most common with 7,542 instances, followed by IDPO:00063 (Protein Binding) with 1,325 instances. 5
Schema.org markup is extensively used by search engines (Google, Microsoft,
and Yandex) to optimise search results (SEO) [
The work presented in this paper relies on the ability to harvest markup
embedded within web pages. The common crawl [
In this work, we have shown that Bioschemas markup can be harvested, transformed using a standard API (c.f. HTTP Get), and used to generate a community focused knowledge graph. We veri ed that the breadth of coverage was equivalent to the original sources, and showed that the resulting knowledge graph can be used to gain further insight into the domain. As future work, we plan to extend the number of sources from which we harvest data and to further exploit the idp-kg to gain further insights into IDPs. We also intend to extend our transformation framework so that it can be applied in other life sciences communities with Bioschemas markup. 12 https://gleaner.io/ accessed September 2021
Acknowledgements. This work was funded through the ELIXIR Strategic Implementation Study Exploiting Bioschemas Markup to Support ELIXIR Communities https://elixir-europe.org/about-us/commissioned-services/ exploiting-bioschemas-markup-support-elixir-communities. Early stages of this work were carried out during the BioHackathon Europe 2020 organized by ELIXIR in November 2020.