<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Ten Years of the e-Health System in Estonia</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Janek Metsallik</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Peeter Ross</string-name>
          <email>peeter.rossg@taltech.ee</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Dirk Draheim</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Gunnar Piho</string-name>
          <email>gunnar.pihog@taltech.ee</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Health Technologies Tallinn University of Technology Akadeemia tee 15a</institution>
          ,
          <addr-line>12618 Tallinn</addr-line>
          ,
          <country country="EE">Estonia</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Information Systems Group Tallinn University of Technology Akadeemia tee 15a</institution>
          ,
          <addr-line>12618 Tallinn</addr-line>
          ,
          <country country="EE">Estonia</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>The e-health system in Estonia, called the Estonian nationwide Health Information System (EHIS) has been operational since the end of 2008. The main success factors for the e-health system in Estonia are clear governance, legal clarity, a mature ecosystem, agreement about access rights, and standardization of medical data and data exchange rules. We present a short history, outline the general business and technical architecture and discuss the lessons learned.</p>
      </abstract>
      <kwd-group>
        <kwd>e-Health system</kwd>
        <kwd>e-state system</kwd>
        <kwd>Estonian nationwide Health Information System (EHIS)</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>There are 1.3 million citizens in Estonia and every citizen and every resident
has a unique ID-number. In Estonia, 88% of households have a broadband
connection (2015), 82% of households use a mobile Internet connection (2016), 96%
of income tax declarations are made via the e-tax board (2016), 32% of votes
were cast over the Internet (2017), and 99% of bank transfers are carried out
electronically. The NATO Cooperative Cyber Defense Centre is in Estonia and
Skype, Transferwise and Taxify have been developed there.</p>
      <p>The healthcare system in Estonia is based on health insurance, paid by
employers. Healthcare providers in Estonia can be private, municipal or
governmental. Most hospitals are publicly owned, and most general practitioners are
private entrepreneurs. Healthcare costs make up to 6% of the GDP (9.5% in
the OECD area). The e-health system in Estonia, called Estonian nationwide
Health Information System (EHIS), has been operational since the end of 2008.
EHIS, by containing the health data of every Estonian resident virtually from
birth to death, integrates di erent healthcare databases and services and makes
it possible to access medical data, prescriptions and medical images online in
a secure and trusted way. The goal of the Estonian e-health system is to
develop patient-friendly, e cient and high-quality healthcare services. In addition,
it aims to make time-critical medical information accessible for physicians and
to decrease the level of bureaucracy in the daily routine of physicians.</p>
      <p>The three main layers of the EHIS are the data layer, the data transfer layer
and the application layer. The data layer consists of the data repositories for
storing the medical documents and images. The data transfer layer provides a secure
Internet-based infrastructure for data exchange both for citizens and healthcare
providers. The developing and open-ended application layer is to provide services
for di erent parties (citizens, healthcare providers, government authorities,
policy makers, etc.) according to their demands now and in the future. Ten years of
experience has shown that both the citizens and healthcare professionals, as well
as politicians and government authorities, have accepted the e-health system.</p>
      <p>In Sect. 2, we present a short history of the e-health system in Estonia. Next,
in Sect. 3, we describe the services and the architecture of the Estonian e-health
system. In Sect. 4 we present the e-state infrastructure, including the security
and legal infrastructure, that is used in the e-health system. Finally, in Sect. 5 we
conclude by discussing some lessons learned from the Estonian e-health system
operation.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Historical Overview</title>
      <p>The story of the Estonian e-health system states back to the rst years of
independence of Estonia and is closely related to the activities and in uence of the
rst Prime Minister, Mr. Mart Laar, and his team. Mr. Laar served as Prime
Minister of Estonia from 1992-1994 and 1999-2002. In those years, information
technology was seen as an opportunity to develop the economy and politics in
Estonia { a small and developing country. Firm foundations were built for many
of the initiatives that today form the e-state of Estonia3, including e-banking,
e-health, e-documents [1{4], e-school, e-taxation, e-voting, etc.</p>
      <p>As Estonia lacked legacy software at that time, and information
technology was underdeveloped in the Soviet period, the utilization and deployment of
information technology began to evolve rapidly during the rst years of
independence. Society believed in information technology and started using information
technology in all domains, including healthcare. Fig. 1 illustrates the history of
the e-health system in Estonia. Between 1990 and 2000, hospitals, general
practitioners and other health providers started developing their own information
systems and introducing the use of electronic health records. Several small and
medium sized software companies, focusing on the development of healthcare
systems, were founded at that time. In the same decade, the informal planning
and the rst ideas of the development of a nationwide e-health system were
initiated.</p>
      <p>The active preparation of the Estonian e-health project by the government
authorities and proponents of e-health system took place between 2003 and 2005,
however some important events took place earlier. In 2001, the digital invoicing</p>
      <sec id="sec-2-1">
        <title>3 https://e-estonia.com/</title>
        <sec id="sec-2-1-1">
          <title>Planning initiated</title>
        </sec>
        <sec id="sec-2-1-2">
          <title>Project</title>
          <p>preparation
(2003-2005)
eHealth</p>
        </sec>
        <sec id="sec-2-1-3">
          <title>Foundation established eHealth</title>
        </sec>
        <sec id="sec-2-1-4">
          <title>Projects (2006-2008)</title>
        </sec>
        <sec id="sec-2-1-5">
          <title>Digital stamp ePrescription</title>
        </sec>
        <sec id="sec-2-1-6">
          <title>Implementation of new e-services</title>
        </sec>
        <sec id="sec-2-1-7">
          <title>Funding decision by</title>
        </sec>
        <sec id="sec-2-1-8">
          <title>Ministry of Economic</title>
        </sec>
        <sec id="sec-2-1-9">
          <title>Affairs</title>
          <p>2008</p>
        </sec>
        <sec id="sec-2-1-10">
          <title>National</title>
          <p>HIS
2000
2003
2006
2010
2015</p>
        </sec>
        <sec id="sec-2-1-11">
          <title>Electronic</title>
        </sec>
        <sec id="sec-2-1-12">
          <title>Health Record</title>
        </sec>
        <sec id="sec-2-1-13">
          <title>Digital</title>
        </sec>
        <sec id="sec-2-1-14">
          <title>Images</title>
        </sec>
        <sec id="sec-2-1-15">
          <title>Digital</title>
        </sec>
        <sec id="sec-2-1-16">
          <title>Prescription</title>
        </sec>
        <sec id="sec-2-1-17">
          <title>Digital</title>
        </sec>
        <sec id="sec-2-1-18">
          <title>Registration</title>
          <p>system for electronic transfer of reimbursement claims, called Estonian Health
Insurance Fund (EHIF), was launched. In 2002, all pharmacies were obliged by
law to transmit the prescription information for reimbursement to the EHIF
electronically. Over 75% of healthcare providers and 45% of all pharmacies had
signed data transmission contracts. In 2005 all the reimbursement claims and
prescription data in Estonia were submitted electronically.</p>
          <p>The foundation for EHIS was established in 2005, when the Ministry of Social
A airs launched a concept for the e-health system. This concept postulated four
main projects, i.e., electronic health records, digital images, digital registration
and digital prescription. Soon after that, the o cial body for the development
of EHIS, called the e-Health Foundation, was established. The e-Health
Foundation was responsible for the development, nancing and management of the
the system. EHIS was funded by the EU (e 1,196,206) and Estonia (e 398,735)
and was launched at the end of 2008.</p>
          <p>EHIS continues to develop and add new functionalities and services.
Eprescription, digital stamps, a driver's license health certi cate application,
drugdrug interaction services and e-registration are examples of such e-services.
Today, EHIS is operated and developed by TEHIK4, a government-owned private
company. The system contains the health records of all the residents in Estonia
and more than 10,000 healthcare professionals use the system on a daily basis5.
In September 2017, the actual number of medical documents in the system was
more than 30 million.</p>
          <p>Although Estonia has about 1.32 million citizens, EHIS contains health
information for 1.54 million people. Overall, 14 di erent medical document types
are in use, covering more than 17 million out-patient case summaries, around
2 million stationary case summaries, and more than 8 million di erent medical</p>
        </sec>
      </sec>
      <sec id="sec-2-2">
        <title>4 https://www.tehik.ee/</title>
        <p>5 https://e-estonia.com/e-health-estonian-digital-solutions-for-europe/
Patient queries per month</p>
        <p>Healthcare professional queries per month</p>
        <p>
          Documents per month
2500000
2000000
1500000
1000000
500000
0
diagnostic examination reports, including radiology reports and laboratory
results [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ]. Fig. 2 illustrates the growth in the number of the queries in EHIS per
month. It was close to zero when the system started in 2008 and now more than
1.2 million queries per month are made by patients (black line) and more than
1.6 million by the healthcare professionals (gray line). In these queries, about
800,000 di erent healthcare documents per month (dotted line), are utilized.
3
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Services and Architecture</title>
      <p>
        EHIS is not a big centralized database but a federated system of mutually
independent yet integrated healthcare-related software services [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. Fig. 3 illustrates
the main elements of the system.
      </p>
      <p>The most widely implemented e-health project in Estonia is a nationwide
health information exchange platform, which is called the nationwide Electronic
Health Record (EHR) system. The EHR platform, which is based on widely
accepted international standards such HL7 CDA (HL7 Clinical Document
Architecture)6, DICOM (Digital Imaging and Communications in Medicine)7, LOINC
(Logical Observation Identi ers Names and Codes)8, etc., enables exchange of
digital health documents in a standardized way. The EHR project began the
ongoing standardization of digital health data artifacts in Estonia. By the
beginning of 2017, a number of 1,163 healthcare institutions were sending and</p>
      <sec id="sec-3-1">
        <title>6 http://www.hl7.org/ 7 https://www.dicomstandard.org/ 8 https://loinc.org/</title>
        <p>retrieving medical data using the EHR platform. The average number of queries
was close to 50,000 queries per day. It is important to note that all medical data
entered is digitally signed either by the physicians or healthcare institutions.
Digital signing is also discussed in Sect. 4.</p>
        <p>
          The other widely used healthcare service provided by EHIS is e-prescription.
Physicians upload prescriptions, in electronic form, to the prescription center
database, from where any pharmacist can request currently valid or previously
dispensed prescriptions. The e-prescription system, that has been launched at
the beginning of 2010, was very quickly accepted by all parties and today
approximately 99% of medical prescriptions in Estonia are issued electronically.
For more information about the e-prescription experience in Estonia, see [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ].
        </p>
        <p>The Picture Archiving and Communication System (PACS) is for sharing
medical images between health institutions. Today all radiology facilities in
Estonia have the duty to send, achieve and retrieve radiology images using the
central PACS. Radiologists and all referring physicians have web-based access
to PACS once they have signed the necessary contract with the Estonian Health
Image Archive Foundation, the responsible authority of PACS.</p>
        <p>S
E
IICN sceend
FEDM treen ii
YCNO igdnoC frsenom
EAG -C ld
TEA -aH
T
S</p>
        <p>LTEECRRBAAADHHO li-trrrscvaaeeehpdoH lilf-trsssaaeehpnooH iii-tsssscgeenpnhDm
.
n
o
itc
.SR ifrn .c</p>
        <p>a
IILSTTYEERAG il.rcyaadoM il.trsscEeubo
UQ I.rVH Tu
cen
a
C</p>
        <p>R
E
T
S
I
G
E
R
N
O
I
LTP
A
U
O
P</p>
        <p>R
E
T
S
I
G
E
R
S
S
E
N
I
S
U</p>
        <p>B
DRIVING LICENCE
HEALTH CERTIFICATE</p>
        <p>APPLICATION
2015
PATIENT PORTAL
2009</p>
        <p>S
L
ISTPAO 2009
H</p>
        <p>S
R
O
T
LYCDO 2090
I
M
A
F</p>
        <p>ISC rya
RAAM Ja0un
PH 201</p>
        <p>SSER reb</p>
        <p>m
UN te</p>
        <p>p
L e
O S
O 0
SCH 201</p>
        <p>E
C
I
V
R
LEC
S
A
I
YEDM 1420
C
N
E
G
R
E
M</p>
        <p>E</p>
        <p>SFINX</p>
        <p>Drug-drug
Interaction database</p>
        <p>2016
PRESCRIPTION</p>
        <p>CENTRE
2010 January
Secure data exchange layer provided by the state</p>
        <p>X-Road, ID-card, mobile-ID, State IS Service Register</p>
        <p>X-ROAD GATEWAY</p>
        <p>SERVICE
2009
PHARMACIES AND
FAMILY DOCTORS
2009</p>
        <p>NATION-WIDE</p>
        <p>HEALTH</p>
        <p>INFORMATION
EXCHANGE PLATFORM
2008 December</p>
        <p>EHIS also hosts many central registers and databases such as of hospitals,
family doctors (general practitioners), pharmacies, school nurses, medicine
interactions, and di erent quality registers (cancer, HIV, tuberculosis, etc.).
Furthermore, it utilizes several nationwide registers such as the population and the
business register.</p>
        <p>One of the crucial parts of EHIS is the patient portal [8{11]. Using the patient
portal, the user can:
log in with ID card or mobile ID;
view and update personal data and add contact data of close relatives;
view his/her medical data from healthcare providers;
view electronic referral letters and electronic prescriptions;
add representatives for him/herself for actions such as collecting e-prescriptions;
make declarations of intent (e.g. donation of organs);
access health insurance data;
hide sensitive health data from doctors and representatives;
complete a health declaration form before an appointment;
view the log of who has accessed his/her data.</p>
        <p>Feedback from the healthcare providers' and the Estonian e-Health
Foundation's helpdesks shows that, when patients do not have access to their health data
(for example during system upgrade), they immediately contact the helpdesk.
They are periodically interested in their data and want to view their test results
before appointments. This information supports the idea that making health
data easily accessible to patients will encourage them to take a more active role
in monitoring their health.
4</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>The Infrastructure</title>
      <p>
        EHIS is not a separate system but an integrated part of the Estonian e-state
system used by the public and the private sector. The e-state system, by
secure data exchange and authentication methods, provides a mature ecosystem
for the e-services in Estonia. E-banking, e-school, e-taxation, e-voting and other
e-services are all using this ecosystem. The most important parts of this
ecosystem are the X-Road [12{14] (governmental service bus) and the e-identity [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ]
PKI (Public Key Infrastructure). X-Road (or X-Tee in Estonian) 9 is a data
exchange layer for enabling a secure Internet-based data exchange between
information systems. To ensure secure transfers, all outgoing data from X-Road
is digitally signed and encrypted, and all incoming data is authenticated and
logged. X-Road is based on the protocols and patterns of the standard SOA
(Service-Oriented Architecture) stack [
        <xref ref-type="bibr" rid="ref16 ref17">16, 17</xref>
        ], i.e., SOAP (Simple Object Access
Protocol) messages and WSDL (Web-Service Description Language) and utilizes
the following principles:
it enables the information systems of X-Road members to communicate with
the information systems of data service providers across any software
platforms;
X-Road members are able to request access to any data services provided
through X-Road;
9 Both X-Road and X-Tee are registered trademarks.
in X-Road, international standards and protocols are used wherever possible;
exchanging data through X-Road does not a ect the integrity, availability
or con dentiality of the data.
      </p>
      <p>Since 2002, in Estonia, every resident has had a digital identity. This identity
is based on the unique identi er (personal ID number), digital certi cation
organizations (police, certi cation center), and physical security devices like smart
card (ID card), mobile SIM card (mobile ID) and smart ID. The digital identity
has two functions: authentication and digital signature. The digital signature is
available also for companies in the form of digital stamping.</p>
      <p>Besides X-Road and e-identity, the important infrastructure for e-health is
a legal environment initiated by the Estonian government and implemented by
the Parliament. The rst idea was to create separate legislation for the e-health
system. However, due to the natural relationship between the e-health system
and the healthcare system, and also due to the desire to direct healthcare
professionals to accept and to use the e-health system, the relevant legislation was
made part of the healthcare legislation.</p>
      <p>The Health Services Organization Act, which regulates the healthcare service
provision, was extended by a new chapter for EHIS. This chapter lays down the
responsibilities of patients, health service providers, and provides requirements
for document standards, etc. For example, all healthcare providers must send
certain health data to EHIS. The set of documents is de ned by the law. The
Act also states that access to patient data is available only to licensed medical
professionals, legal representatives or patients trustees. In the Estonian e-health
system, the concept of the attending doctor has been introduced. This means
that the physician or a nurse must prove the treatment relation to the patient,
when accessing the patient's data in EHIS. The Act also states (and this is
realized in the patient portal), that the patient has the right to hide their data so
that healthcare professionals are no longer able to view them. This could be done
either by hiding a single document or by hiding all their personal data in EHIS.
All attempts to view healthcare data in EHIS are monitored by the government
authorities and reported to the patients in the patient portal. In case of
suspicions of unlawful access to the data, necessary actions are taken immediately.
According to the Act, the ethical committee was created to lead the discussions
on patients rights and to select the proper system for the EHIS. Citizens can
access their own data, declare intentions and preferences, and monitor logs.
5</p>
    </sec>
    <sec id="sec-5">
      <title>Discussion and Conclusion</title>
      <p>The Estonian e-health system is unique as it is nationwide, integrates de ned
healthcare data of all healthcare providers and provides an overview of the health
condition of every resident from birth to death. Such a comprehensive data
system requires a robust security system. The security of the Estonian e-health
system is ensured by the following six techniques:</p>
      <p>A secure authentication and authorization of all users with ID card,
mobile ID or smart ID;
Digital signing (by individuals) or digital stamping (by institutions) of all
medical documents;
Accountability and transparency provided by an untamperable and
unremovable secure log (audit trail) containing all actions;
Coding of personal data ensures separation of personal data from medical
data;
Encrypted database records allow a minimal con dentiality risk from the
technical administrators of the system;
Monitoring of all actions together with the corresponding countermeasures
(both organizational and technical) allows identi cation of fraud and misuse
quickly and de nitely.</p>
      <p>Huge change management issues that digitalization brings to healthcare is
always a challenge. The observations and di culties that were related to the
Estonian e-health system were (and are) as follows:</p>
      <p>Physicians and other professionals must change the way they ll out medical
les to some extent { the trend is towards more uniform language;
Semantic interoperability of medical data is hard to achieve;
Data quality and secondary usage of data is still challenging;
General acceptance of hospital personnel to share medical data in patient
portal with patient is problematic;
Much attention must be paid to the security and electronic authentication
of the users;
User interface development must not be underestimated;
Medical data is not what people are looking for { they are interested in
services.</p>
      <p>
        As an example, related to the last claim in the list above, e-health
services that are crossing institutional and/or sectoral borders such as e-referral,
e-consultation, e-prescription or lling in health declarations for a driving
license application are the most popular services among the users. These types of
services make healthcare processes more e ective and save time for both
healthcare professionals and individuals. The exchange of data and digital documents
between institutions also puts high demands for data quality [
        <xref ref-type="bibr" rid="ref18 ref19">18, 19</xref>
        ] and has a
clear potential to increase healthcare quality in general.
      </p>
      <p>The aim of the Estonian e-health system was to develop a platform for health
information exchange. However the solution had to provide some tangible
services as well. During the rst phase of the system, the scope to provide certain
standardization and applications for most valuable documents and data was set,
including outpatient summary notes, inpatient discharge letters, demographics,
time-critical health data reports, and diagnostic image references.</p>
      <p>Even though Estonia is considered an IT-mature society, it is important to
understand that, similar to other European countries, Estonian society has
members whose computer skills do not allow them to use e-services comfortably. This
means that the digital society ecosystem should be accompanied by activities to
educate society members and to decrease the digital divide. New e-services must
not replace ordinary services but should complement them to make processes
more e cient and a ordable for the whole society. This is the leading concept
in the implementation of EHIS in Estonia, meaning that patients still have the
right and the opportunity to receive services in conventional ways.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Ansper</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Buldas</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Freudenthal</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Willemson</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          :
          <article-title>High-performance qualied digital signatures for X-Road</article-title>
          . In Nielson,
          <string-name>
            <given-names>H.R.</given-names>
            ,
            <surname>Gollmann</surname>
          </string-name>
          , D., eds.:
          <source>NordSec 2013 { the 18th Nordic Conference on Secure IT Systems</source>
          . Volume
          <volume>8208</volume>
          of Lecture Notes in Computer Science., Springer (
          <year>2013</year>
          )
          <volume>123</volume>
          {
          <fpage>138</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Draheim</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Koosapoeg</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lauk</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pappel</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pappel</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tepandi</surname>
            ,
            <given-names>J.:</given-names>
          </string-name>
          <article-title>The design of the Estonian governmental document exchange classi cation framework</article-title>
          . In Ko},
          <string-name>
            <given-names>A.</given-names>
            ,
            <surname>Francesconi</surname>
          </string-name>
          , E., eds.:
          <source>Electronic Government and the Information Systems Perspective</source>
          , Cham, Springer International Publishing (
          <year>2016</year>
          )
          <volume>33</volume>
          {
          <fpage>47</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Pappel</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pappel</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tepandi</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Draheim</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Systematic digital signing in Estonian e-government processes. Transactions on Large-Scale Data- and KnowledgeCentered Systems XXXVI: Special Issue on Data and Security Engineering (</article-title>
          <year>2017</year>
          )
          <volume>31</volume>
          {
          <fpage>51</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Pappel</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Oolu</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Saarevet</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lauk</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Draheim</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>The digital archiving process in estonia assessment and future perspectives</article-title>
          . In Dang, T.K.,
          <string-name>
            <surname>Wagner</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          , Kung, J.,
          <string-name>
            <surname>Thoai</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Takizawa</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Neuhold</surname>
          </string-name>
          , E.J., eds.
          <source>: FDSE'2107 - the 4th International Conference on Future Data and Security Engineering</source>
          , Cham, Springer International Publishing (
          <year>2017</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Piho</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tepandi</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Parman</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Towards LIMS (laboratory information management systems) software in global context</article-title>
          .
          <source>In: MIPRO'2012 { the 35th International Convention on Information and Communication Technology, Electronics and Microelectronics</source>
          , IEEE (May
          <year>2012</year>
          )
          <volume>721</volume>
          {
          <fpage>726</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Parv</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Saluse</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Aaviksoo</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tiik</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sepper</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ross</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Economic impact of a nationwide interoperable e-health system using the PENG evaluation tool</article-title>
          . In Mantas, J.,
          <string-name>
            <surname>Andersen</surname>
            ,
            <given-names>S.K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mazzoleni</surname>
            ,
            <given-names>M.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Blobel</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Quaglini</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Moen</surname>
          </string-name>
          , A., eds.: MIE'2012 {
          <article-title>Quality of Life through Quality of Information. Studies in Health Technology and Informatics (</article-title>
          <year>2012</year>
          )
          <volume>876</volume>
          {
          <fpage>880</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Parv</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kruus</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          , Mo~tte,
          <string-name>
            <given-names>K.</given-names>
            ,
            <surname>Ross</surname>
          </string-name>
          ,
          <string-name>
            <surname>P.:</surname>
          </string-name>
          <article-title>An evaluation of e-prescribing at a national level</article-title>
          .
          <source>Informatics for Health and Social Care</source>
          <volume>41</volume>
          (
          <issue>1</issue>
          ) (
          <year>2014</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Essen</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Scandurra</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gerrits</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Humphrey</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Johansen</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kiergegaard</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Koskinen</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Liaw</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Odeh</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ross</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ancker</surname>
          </string-name>
          , J.:
          <article-title>Patient access to electronic health records: Di erences across ten countries</article-title>
          .
          <source>Health Policy and Technology</source>
          <volume>7</volume>
          (
          <issue>1</issue>
          ) (
          <year>2018</year>
          )
          <volume>44</volume>
          {
          <fpage>56</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9. de Lusignan,
          <string-name>
            <given-names>S.</given-names>
            ,
            <surname>Ross</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            ,
            <surname>Shifrin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            ,
            <surname>Hercigonja-Szekeres</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            ,
            <surname>Seroussi</surname>
          </string-name>
          ,
          <string-name>
            <surname>B.</surname>
          </string-name>
          :
          <article-title>A comparison of approaches to providing patients access to summary care records across old and new europe: an exploration of facilitators and barriers to implementation</article-title>
          . In Lehmann,
          <string-name>
            <given-names>C.</given-names>
            ,
            <surname>Ammenwerth</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            ,
            <surname>Nohr</surname>
          </string-name>
          , C., eds.
          <source>: MEDINFO 2013 { the 14th World Congress on Medical and Health Informatics</source>
          . (
          <year>2013</year>
          )
          <volume>397</volume>
          {
          <fpage>401</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Sepper</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ross</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tiik</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Nationwide health data management system: a novel approach for integrating biomarker measurements with comprehensive health records in large populations studies</article-title>
          .
          <source>Journal of Proteome Research</source>
          <volume>10</volume>
          (
          <issue>1</issue>
          ) (
          <year>2011</year>
          )
          <volume>97</volume>
          {
          <fpage>100</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Tiik</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ross</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Patient opportunities in the Estonian electronic health record system</article-title>
          .
          <source>Medical and Care Compunetics</source>
          <volume>6</volume>
          (
          <year>2010</year>
          )
          <volume>171</volume>
          {
          <fpage>177</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Kalja</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>The X-Road : a key interoperability component within the state information system</article-title>
          . In Odrats, I., ed.:
          <source>Information technology in public administration of Estonia { yearbook</source>
          <year>2007</year>
          ,
          <article-title>Tallinn, Ministry of Economic A airs and Communications (</article-title>
          <year>2008</year>
          )
          <volume>19</volume>
          {
          <fpage>20</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Kalja</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>The rst ten years of X-Road</article-title>
          . In Kastehein, K., ed.:
          <source>Information technology in public administration of Estonia { yearbook</source>
          <year>2011</year>
          /
          <year>2012</year>
          , Tallinn, Ministry of Economic A airs and
          <source>Communications</source>
          (
          <year>2012</year>
          )
          <volume>78</volume>
          {
          <fpage>80</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Kalja</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Robal</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vallner</surname>
            ,
            <given-names>U.</given-names>
          </string-name>
          :
          <article-title>New generations of Estonian eGovernment components</article-title>
          .
          <source>In: PICMET'2015 { the 15th Portland International Conference on Management of Engineering and Technology</source>
          , IEEE (
          <year>2015</year>
          )
          <volume>625</volume>
          {
          <fpage>631</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Tsap</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pappel</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Draheim</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Key success factors in introducing national eidenti cation systems</article-title>
          . In Dang, T.K.,
          <string-name>
            <surname>Wagner</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          , Kung, J.,
          <string-name>
            <surname>Thoai</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Takizawa</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Neuhold</surname>
          </string-name>
          , E.J., eds.
          <source>: FDSE'2107 - the 4th International Conference on Future Data and Security Engineering</source>
          , Cham, Springer International Publishing (
          <year>2017</year>
          )
          <volume>455</volume>
          {
          <fpage>471</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Draheim</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>The service-oriented metaphor deciphered</article-title>
          .
          <source>Journal of Computing Science and Engineering</source>
          <volume>4</volume>
          (
          <issue>4</issue>
          ) (
          <year>2010</year>
          )
          <volume>253</volume>
          {
          <fpage>275</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Atkinson</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bostan</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Draheim</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Foundational MDA patterns for serviceoriented computing</article-title>
          .
          <source>Journal of object technology 14(1)</source>
          (
          <year>2015</year>
          )
          <volume>1</volume>
          {
          <fpage>30</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Tepandi</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lauk</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Linros</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Raspel</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Piho</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pappel</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Draheim</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>The data quality framework for the Estonian public sector and its evaluation. Transactions on Large-Scale Data-</article-title>
          and
          <string-name>
            <surname>Knowledge-Centered</surname>
            <given-names>Systems</given-names>
          </string-name>
          35 (
          <year>2017</year>
          )
          <volume>1</volume>
          {
          <fpage>26</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Ross</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Importance of data quality in nation-wide e-health system for personalized medicine decision support applications</article-title>
          .
          <source>In: the 7th Baltic Atherosclerosis Society Congress</source>
          . (
          <year>2018</year>
          )
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>