<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Conversion of Bulgarian Observational Data to OMOP Common Data Model: Initial Results</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Petko Kovachev</string-name>
          <email>az@petko.info</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Evgeniy Krastev</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Dimitar Tcharaktchiev</string-name>
          <email>dimitardt@gmail.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Emanuil Markov</string-name>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Ivan Evg. Ivanov</string-name>
          <email>ivan.evgeniev@gmail.com</email>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Medical University of Sofia, University Hospital of Endocrinology</institution>
          ,
          <addr-line>Zdrave street No. 2, Sofia, 1431</addr-line>
          ,
          <country country="BG">Bulgaria</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Sofia University “St. Kliment Ohridski”, Faculty of Mathematics and Informatics</institution>
          ,
          <addr-line>James Bourchier blvd., No. 5, Sofia, 1164</addr-line>
          ,
          <country country="BG">Bulgaria</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Technical University of Sofia, Faculti of Automatics</institution>
          ,
          <addr-line>Kliment Ohridsky blvd. No. 8, Sofia</addr-line>
          ,
          <country country="BG">Bulgaria</country>
        </aff>
      </contrib-group>
      <fpage>113</fpage>
      <lpage>125</lpage>
      <abstract>
        <p>Common data models (CDMs) offer a standardized approach for data persistence and exchange. This is especially useful when nowadays-clinical data is distributed among heterogeneous sharing systems. Besides, OHDSI provides software tools in support on each stage of the ETL and ensure quality control. Therefore, data presented in CDM possesses all the features of a reliable source for a broad range of statistical analyses. This paper presents initial results of a research work done with the objective to transfer outpatient records from the Bulgarian Diabetes register into the OMOP CDM. One of the major challenges has been the extraction of clinical data from native language text as well as the use of international OMOP concepts to annotate data recorded in a Bulgarian context. The mapping of national encoding for drug codes was one of the serious obstacles to conceptual mapping that requires adaptation of such codes to corresponding drug codes in the International Classification of Diseases 9th Revision.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;eHealth</kwd>
        <kwd>observational health data</kwd>
        <kwd>common data model</kwd>
        <kwd>ETL</kwd>
        <kwd>data harmonization</kwd>
        <kwd>electronic health records</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>Digital health technologies produce huge amounts of data related to patient
health collected as part the execution of routine healthcare services under
realworld conditions. Data collected from such sources is collectively known as
observational data (OD) OD is generated from a number of sources such as
electronic health. OD is a valuable source for clinical evidence, which can be used to
evaluate the safety and eefctiveness of medical products in treatment of socially
signicfiant diseases like diabetes or cancer. Moreover, results from analysis of OD
provide evidence in support for clinical decision-making [1]. Therefore, many
research groups around the world attempt to integrate OD into a common data
model that can serve as a reliable source for analyses of healthcare data [2] [3].</p>
      <p>The Observational Health Data Sciences and Informatics [4] [5] (or OHDSI,
pronounced “Odyssey”) program is a multi-stakeholder, interdisciplinary
collaborative initiative that aims to bring out the value of health data through
largescale analytics. OHDSI objective is to generate accurate, reproducible, and
wellcalibrated evidence and promote better health decisions and better care.</p>
      <p>
        The Observational Medical Outcomes Partnership (OMOP) [
        <xref ref-type="bibr" rid="ref1">6</xref>
        ]Common
Data Model (CDM) [
        <xref ref-type="bibr" rid="ref2">7</xref>
        ] is an open community data standard, designed to
standardize the structure and content of observational data and to enable eficient
analyses that can produce reliable evidence. It is a unified database model that
allows integrating various OD sources including EHRs based on the standard.
      </p>
      <p>
        The European Health Data and Evidence Network project (EHDEN) [
        <xref ref-type="bibr" rid="ref3">8</xref>
        ]
under the Innovative Medicines Initiative (IMI) drives the adoption of the
OMOPCDM in Europe in close collaboration with OHDSI.
      </p>
      <p>
        The OHDSI community and the software tools it is using follow the FAIR
Data Guiding Principles [
        <xref ref-type="bibr" rid="ref4">9</xref>
        ] [
        <xref ref-type="bibr" rid="ref5">10</xref>
        ]:
• Findability – Any healthcare database that is mapped to OMOP and used
for analytics should persist for future reference and reproducibility.
Therefore, data are described with rich metadata, where metadata early and
explicitly include a globally unique and persistent identifier of the data it describes.
• Accessibility – Accessibility of OMOP mapped data through an open
protocol is typically achieved through the SQL interface. The protocol must
provide a procedure for authentication and authorization.
• Interoperability – data use a formal, accessible, shared, and broadly
applicable language for knowledge representation. Additionally, data must be
accompanied with vocabularies that follow FAIR principles with qualified
references to other data.
• Reusability – Metadata and data should be well described so that they
can be replicated and/or combined in diferent settings. Moreover, data must
satisfy domain-relevant community standards.
      </p>
      <p>
        The OMOP Common Data Model (CDM) [
        <xref ref-type="bibr" rid="ref2">7</xref>
        ] is an open community data
standard, designed to standardize the structure and content of observational data
and to enable eficient analyses that can produce reliable evidence. A central com
ponent of the OMOP CDM are the OHDSI standardized vocabularies (Figure 1).
      </p>
      <p>The OMOP Common Data Model allows systematic analysis of disparate
observational databases. The concept behind this approach is to transform data
contained within those databases into a common format (data model) as well as a
common representation (terminologies, vocabularies, coding schemes), and then
perform systematic analyses using a library of standard analytic routines that
have been written based on the common format.</p>
      <p>Routine health databases, based on routine electronic health records (EHRs)
difer in purpose, content and design. Common Data Models (CDM) can enable
standardized analysis of disparate data sources simultaneously.</p>
      <p>The CDM contains standardized tables grouped in 16 Clinical Event tables,
10 Vocabulary tables, 2 metadata tables, 4 health system data tables, 2 health
economics data tables, 3 standardized derived elements, and 2 Results schema tables.
• The development of the CDM follows the following design elements:
• Suitability for purpose – CDM data is organized in a way that is suitable
for analysis
• Data protection – Personalized data, such as names, birthdays, and living
address and so on is anonymized format.
• Design of domains – The domains are modeled in a person-centric
relational data model, where for each record refers to a person and the date when
the OD is captured.
• Rationale for domains – Domains are identified and separately defined
in an Entity-relationship diagram, where each domain has specific attributes
that are not otherwise applicable. All other data can be preserved as an
observation in an entity-attribute-value structure.
• Standardized Vocabularies – CDM relies on the Standardized
Vocabularies such as SNOMED containing all necessary and appropriate
corresponding standard healthcare concepts.
• Reuse of existing vocabularies- definitions of codes drugs, diseases from
national, industry standardization or vocabulary definitions are mapped to
international coding systems or reused.
• Maintaining source codes – The original source code is persisted
together with their corresponding codes in Standardized Vocabularies, so that the
model loses no information from the OD.
• Technology neutrality – The CDM does.t depend on specific technology.
It can be implemented on any relational database, such as MS SQL Server,
Oracle etc.
• Scalability – The CDM is optimized for data processing and
computational analysis to accommodate data sources that vary in size, including
databases with up to hundreds of millions of persons and billions of clinical
observations.
• Backwards compatibility – All changes from previous CDMs are
clearly delineated. Older versions of the CDM can be easily created from this
CDMv5, and no information is lost that was present previously
There are implicit and explicit conventions that adopted in the CDM:
• General Conventions of the Model – The CDM is considered a
“personcentric” model, meaning that all clinical Event tables are linked to the
PERSON table.
• General Conventions of Schemas – Most of the schemas are considered
as read only, writable tables are only COHORT and
COHORT_DEFINITION in “Results” schema.
• General Conventions of Data Tables – The CDM is platform
independent. Data types are defined generically using ANSI SQL data types (VAR
CHAR, INTEGER, FLOAT, DATE, DATETIME, and CLOB). Precision is
provided only for VARCHAR.
• General Conventions of Domains – Events of diferent nature are orga
nized into Domains. These Events are stored in tables and fields, which are
Domainspecific, and represented by Standard Concepts that are also
Domainspecific as defined in the Standardized Vocabularies.
2.</p>
    </sec>
    <sec id="sec-2">
      <title>Methods</title>
      <sec id="sec-2-1">
        <title>2.1. Environment preparation</title>
        <p>
          Preparing the CDM database environment by installing a PostgreSQL
DBMS, Java JDK and Docker compose in a Linux workstation. The database
setup includes:
a) Creating the required database users;
b) Creating the OMOP CDM tables with the Common Data
Model/PostgreSQL DDL scripts; and
c) Importing the standard OMOP vocabularies from athena.ohdsi.org.
d) All the OHDSI sources are available at github.org/OHDSI. We
installed the White Rabbit, Rabbit-In-a-Hat, Usagi, [
          <xref ref-type="bibr" rid="ref6">11</xref>
          ] Achilles [
          <xref ref-type="bibr" rid="ref7">12</xref>
          ] and
Broadsea repositories required for the OHDSI web applications,
configured the addresses and JDBC URLs and started their respective Docker
containers.
        </p>
      </sec>
      <sec id="sec-2-2">
        <title>2.2. ETL – Extract Transform Load</title>
        <p>In order to get from the native/raw data to the OMOP Common Data Model
(CDM) we have to create an extract, transform, and load (ETL) process. This
process should restructure the data to the CDM, and add mappings to the
Standardized Vocabularies with these steps (Figure 2).</p>
        <p>
          1. Design the ETL – To initiate an ETL process on a database we need to
understand source data, including the tables, fields, and content. The White
Rabbit software from OHDSI to perform a scan of the source data. The scan
generates a report used as a reference when designing the ETL. With the
White Rabbit scan in hand, we have a clear picture of the source data. We
also know the full specification of the CDM. Rabbit-In-a-Hat [
          <xref ref-type="bibr" rid="ref8">13</xref>
          ] is used in
next step is to perform mapping of source fields to the target in CDM data
base. Rabbit-In-a-Hat is designed to read and display a White Rabbit scan
document and generates documentation for the ETL process but it does not
generate code to create an ETL.
2. Create the Code Mappings- With Usagi form OHDSI tools we perform
manual process of creating a code mappings with standard source codes to
Vocabulary concepts.
3. Implement the ETL – Once the design and code mappings are
completed, the ETL process is implemented with ETL-CDM Builder. As result,
we have CDM relevant database populated with the data from the source
database.
4. Quality Control – For the extract, transform, load process, quality
control is iterative. The typical pattern is to write logic &gt; implement logic &gt; test
logic &gt; fix.
        </p>
        <p>The result of the ETL process is a CDM compliant database/schema ready to
be used for analyses.</p>
      </sec>
      <sec id="sec-2-3">
        <title>2.3. Study execution</title>
        <p>
          The most convenient and precise approach to perform observational study
against CDM database is to use ATLAS [
          <xref ref-type="bibr" rid="ref9">14</xref>
          ]- free, publicly available, web based
tool developed by the OHDSI that facilitates the design and execution of analyses
on standardized, patient level, observational data in the CDM format. ATLAS is
deployed as a web application in combination with the OHDSI WebAPI and is
hosted on Apache Tomcat and could be deployed and started as Docker container
or cloud service.
        </p>
        <p>The screenshot of ATLAS in Figure 3 shows the various functionalities
provided by ATLAS:
• Data Sources – provides the capability review descriptive, standardized
reporting for each of the configured data sources.
• Vocabulary Search – provides the ability to search and explore the OMOP
standardized vocabulary.
• Concept Sets provides the ability to create collections of logical
expressions that can be used to identify a set of concepts to be used throughout your
standardized analyses.
• Cohort Definitions – ability to construct a set of persons who satisfy one
or more criteria for a duration of time.
• Characterizations – an analytic capability that allows you to look at one or
more cohorts and to summarize characteristics about those patient populations.
• Cohort Pathways Cohort pathways is an analytic tool that allows you to
look at the sequence of clinical events that occur within one or more
populations.
• Incidence Rates – a tool that allows you to estimate the incidence of
outcomes within target populations of interest.
• Profiles – tool that allows exploring of an individual patients longitudinal
observational data to summarize what is going on within a given individual.
• Population Level Estimation- a capability that allows defining a popu
lation study for level efect estimation using a comparative cohort design
whereby comparisons between one or more target and comparator cohorts
can be explored for a series of outcomes.
• Patient Level Prediction – allows to apply machine-learning algorithms
to conduct prediction analyses at patient level whereby you can predict an
outcome within any given target exposures.
• Jobs – used to explore the state of processes that are running through the
WebAPI.
• Configuration – to review the configured data sources that have been in
the source configuration section.</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. Results</title>
      <p>In Bulgaria outpatient records are produced by the General Practitioners
(GPs) and the Specialists from Ambulatory Care for every contact with the
patient. Outpatient records from patients with diabetes are maintained by the
Bulgarian Diabetes Register. These records represent a true example of OD that can
be produce valuable evidence for improving the treatment and management the
healthcare services for such patients. The outpatient records are semi-structured
ifles with predefined XML schema. The source XML documents included more
than 1 600 000 pseudonymized outpatient records. The most important indicators
in the records like Age, Gender, Location, Diagnoses are stored in explicit tags.
The Case history is presented as free text in the Anamnesis. Additionally, these
records include in native text information about the Patient status described the
patient state, symptoms, syndromes, patients’ height and weight, body mass
index (BMI), blood pressure and other clinical concepts. The values of clinical tests
and lab data are enumerated also as free text in a separate section of the XML
document. A special section is dedicated to the prescribed treatment.</p>
      <p>This paper presents first results from a research work whose objective is to
convert this OD into OMOP CDM version 5.3.0. Here we will shortly describe
the first stage of the ETL process (Design the ETL) that will be used for mapping
of the source fields to the target in CDM database.</p>
      <p>Our first task has been to parse data from the available XML documents and
store it in a relational database, where the relational model matches the XML
schema of the source XML documents. Each one of the XML documents
contains a set of outpatient records of patients with diabetes. Therefore, the relational
database is referred to as Diabetes2018. Both the source database Diabetes2018
and the target CDM database are MS SQL Server databases. The Entity
Relationship Diagram of the source database and a description of the tables in the source
database are shown correspondingly in Table 1 and Figure 4.
MDdirections
Patients
PrimaryVisit
Procedures
Profilact
RecipeBooks
Recipies
Reimbursables
SecondaryVisit
SIMPConsults
VSDSIMPConsults</p>
      <p>MDdirections
Patients
PrimaryVisit
Procedures
Profilact
RecipeBooks
Recipies
Reimbursables
SecondaryVisit
SIMPConsults</p>
      <p>VSDSIMPConsults
BloodPressure</p>
      <p>BloodPressure
BloodSugar</p>
      <p>BloodSugar
BMIdata</p>
      <p>BMIdata
HbAc1Data</p>
      <p>HbAc1Data
DiabetTimeData</p>
      <p>DiabetTimeData
TrigData</p>
      <p>TrigData</p>
      <p>Direction for medical examination by
Specialist in Ambulatory care
Patient description data
Initial visit to a Specialist in
Ambulatory Care
Procedures assigned in Outpatient
record
Describes a visit for Disease prevention
Recipe Books of Patient
Recipes for reimbursable medicinal
products. These are stored in the
patient’s Recipe Book
Reimbursable medicinal products
Secondary visit to a Specialist in
Ambulatory Care
Specialized Medical care
Highly Specialized Medical care
Blood pressure values measured in
mmHg extracted from natural
language text description of patient status
and examination data
Blood sugar profile first value mea
sured in mmol/l extracted from natural
text description of patient status and
examination data
Body Mass Index data extracted from
natural language text description of
patient status and examination data.</p>
      <p>Table includes also Height and Weight
measurements in sm and kg, when
Height and Weight data are found in
the text
Contains values of HbAc1 extracted
from natural language text description
of patient status and examination data
measured in mmol/mol
Contains the number of years before
the illness diabetes has been
established for the first time, extracted from
natural language text description of
patient status and examination data
Contains values of triglycerides
extracted from natural language text
description of patient status and
examination data and measured in mmol/L</p>
      <p>Thus, clinical data for more than 502 000 patients with diabetes have been
loaded in Diabetes2018. In order to execute this task, we had to write programs in
Java and Python scripts that parse the XML documents and extract clinical data as
blood pressure, glucose, height and weight body mass index and many other
measurements recorded in the source XML documents in natural language text. Next,
we used the Rabbit-In-A-Hat tool for the mapping denfiition of data (Figure 5).</p>
      <p>
        The mapping rules identified in Figure 5 allow proceeding with the extract,
transform, and load stages of the ETL process using SQL queries implemented
in Microsoft SQL Server 2019. For completeness, with the help of White Rabbit
we generated a data dictionary for all the tables and fields that have been profiled.
This data dictionary includes the English translation of the local name of fields
and their description (Table 1). All the tasks at this stage have been
accompanied by continuous and tedious quality control so that the CDM database has all
the attributes of a reliable evidence, namely, repeatable, reproducible, replicable,
generalizable, robust and calibrated [
        <xref ref-type="bibr" rid="ref10">15</xref>
        ].
      </p>
    </sec>
    <sec id="sec-4">
      <title>4. Conclusion</title>
      <p>Common data models (CDMs) ofer a standardized approach for data per
sistence and exchange. This is especially useful when nowadays-clinical data
is distributed among heterogeneous sharing systems. Besides, OHDSI provides
software tools in support on each stage of the ETL and ensure quality control.
Therefore, data CDM possesses all the features of a reliable source for a broad
range of statistical analyses.</p>
      <p>This paper presents initial results of a research work done with the objective
to transfer outpatient records from the Bulgarian Diabetes register into the OMOP
CDM. One of the major challenges has been the extraction of clinical data from
native text as well as the use of international OMOP concepts to annotate data
recorded in a Bulgarian context. The mapping of national encoding for drug codes
was one of the serious obstacles to conceptual mapping that requires adaptation
of such codes to corresponding drug codes in the International Classification of
Diseases 9th Revision.</p>
    </sec>
    <sec id="sec-5">
      <title>5. Acknowledgements</title>
      <p>The presentation of this paper is supported by the National Scientific Pro
gram “Electronic Healthcare in Bulgaria” (eHealth).</p>
    </sec>
    <sec id="sec-6">
      <title>6. References</title>
      <p>Y. Jeon, Y. Choi, E. Kim, S. Oh and H. Lee, “Common data model-based
real-world data for practical clinical practice guidelines,” Transl Clin
Pharmacol., vol. 28, no. 2, pp. 67–72, 2020.</p>
      <p>A. Lamer, N. Depas, M. Doutreligne, A. Parrot, D. Verloop, M. Defebvre,
G. Ficheur, E. Chazard and J. Beuscart, “Transforming French Electronic
Health Records into the Observational Medical Outcome Partnership’s
Common Data Model: A Feasibility Study,” Appl Clin Inform., vol. 11, no.
1, pp. 13–22, 2020.</p>
      <p>B. Ryu, E. Yoon, S. Kim, S. Lee, H. Baek, S. Yi, H. Na, J. Kim, R. Baek, H.
Hwang and S. Yoo, “Transformation of Pathology Reports Into the
Common Data Model With Oncology Module: Use Case for Colon Cancer.,”
J Med Internet Res., vol. 22, no. 12:e18526, 2020.</p>
      <p>OHDSI, “Observational Health Data Sciences and Informatics – OHDSI,”
www.ohdsi.org, 2022. [Online]. Available: https://www.ohdsi.org/web/
wiki/doku.php?id=welcome. [Accessed 10 April 2022].</p>
      <p>Observational Health Data Sciences and Informatics, The Book of OHDSI,
https://ohdsi.github.io/TheBookOfOhdsi, 2021.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>OHDSI</given-names>
            <surname>CDM Working</surname>
          </string-name>
          <string-name>
            <given-names>Group</given-names>
            , “Welcome to OMOP,
            <surname>”</surname>
          </string-name>
          <string-name>
            <surname>OHDSI</surname>
          </string-name>
          ,
          <year>2022</year>
          . [Online]. Available: https://ohdsi.org/omop.
          <source>[Accessed 2 April</source>
          <year>2022</year>
          ].
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>Observational</given-names>
            <surname>Health</surname>
          </string-name>
          <article-title>Data Sciences and Informatics, “OMOP Common Data Model,” ohdsi</article-title>
          .org,
          <year>2022</year>
          . [Online]. Available: https://ohdsi.github.
          <source>io/ CommonDataModel. [Accessed 10 April</source>
          <year>2022</year>
          ].
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [8] EHDEN, “
          <article-title>The European Health Data &amp; Evidence Network,”</article-title>
          <string-name>
            <surname>EHDEN</surname>
          </string-name>
          ,
          <year>2022</year>
          . [Online]. Available: https://www.ehden.
          <source>eu. [Accessed 10 April</source>
          <year>2022</year>
          ].
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [9] OHDSI, “FAIR Principles,”
          <year>2016</year>
          . [Online]. Available: https://www.go-fair. org/fair-principles.
          <source>[Accessed 10 April</source>
          <year>2022</year>
          ].
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [10]
          <string-name>
            <surname>M. D. Wilkinson</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Dumontier</surname>
            ,
            <given-names>I. J.</given-names>
          </string-name>
          <string-name>
            <surname>Aalbersberg</surname>
          </string-name>
          et al., “
          <article-title>The FAIR Guiding Principles for scientific data management and stewardship,” Scientific Data</article-title>
          , vol.
          <volume>3</volume>
          :
          <issue>160018</issue>
          , no.
          <issue>1</issue>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <source>[11] Observational Health Data Sciences and Informatics</source>
          , “Software Tools,” ohdsi.org,
          <year>2022</year>
          . [Online]. Available: https://www.ohdsi.org/software-tools.
          <source>[Accessed 10 April</source>
          <year>2022</year>
          ].
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [12]
          <article-title>Observational Health Data Sciences and Informatics, “ACHILLES for data characterization,” ohdsi</article-title>
          .org,
          <year>2022</year>
          . [Online]. Available: https://www.ohdsi. org/analytic-tools/
          <article-title>achilles-for-data-characterization</article-title>
          .
          <source>[Accessed 10 April</source>
          <year>2022</year>
          ].
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <source>[13] Observational Health Data Sciences and Informatics</source>
          , “
          <article-title>Rabbit-In-a-</article-title>
          <string-name>
            <surname>Hat</surname>
          </string-name>
          ,”
          <year>2022</year>
          ,
          <issue>15</issue>
          <year>February 2022</year>
          . [Online]. Available: http://ohdsi.github.io/WhiteRabbit/RabbitInAHat.html.
          <source>[Accessed 10 April</source>
          <year>2022</year>
          ].
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [14] OHDSI,
          <string-name>
            <surname>”</surname>
            <given-names>ATLAS</given-names>
          </string-name>
          ,”
          <year>2022</year>
          . [Online]. Available: https://github.com/OHDSI/ Atlas/wiki. [
          <source>Accessed 10 April</source>
          <year>2022</year>
          ].
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>C.</given-names>
            <surname>Blacketer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Kallfelz</surname>
          </string-name>
          and
          <string-name>
            <given-names>P.</given-names>
            <surname>Rijnbeek</surname>
          </string-name>
          , “
          <fpage>WP5</fpage>
          - Data
          <source>Workflow Implementation &amp; Service Deployment. D5.2 Report on Quality Assurance and Control Procedures,” EHDEN</source>
          ,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>