<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Analyzing the Tra jectories of Patients with Sepsis using Process Mining</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Felix Mannhardt</string-name>
          <email>f.mannhardt@tue.nl</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Daan Blinde</string-name>
          <email>daan@blijnder.nl</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Blijnder</institution>
          ,
          <country country="NL">The Netherlands</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Eindhoven University of Technology</institution>
          ,
          <addr-line>Eindhoven</addr-line>
          ,
          <country country="NL">The Netherlands</country>
        </aff>
      </contrib-group>
      <fpage>72</fpage>
      <lpage>80</lpage>
      <abstract>
        <p>Process mining techniques analyze processes based on event data. We analyzed the trajectories of patients in a Dutch hospital from their registration in the emergency room until their discharge. We considered a sample of 1050 patients with symptoms of a sepsis condition, which is a life-threatening condition. We extracted an event log that includes events on activities in the emergency room, admission to hospital wards, and discharge. The event log was enriched with data from laboratory tests and triage checklists. We try to automatically discover a process model of the patient trajectories, we check conformance to medical guidelines for sepsis patients, and visualize the flow of patients on a de-jure process model. The lessons-learned from this analysis are: (1) process mining can be used to clarify the patient flow in a hospital; (2) process mining can be used to check the daily clinical practice against medical guidelines; (3) process discovery methods may return unsuitable models that are difficult to understand for stakeholders; and (4) process mining is an iterative process, e.g., data quality issues are often discovered and need to be addressed.</p>
      </abstract>
      <kwd-group>
        <kwd>Process Mining</kwd>
        <kwd>Medical Guidelines</kwd>
        <kwd>Patient Trajectories</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>Process models are used by organizations to document, specify, and analyze their
processes. A process model describes the expected behavior of a process. Any set
of related activities that are executed in a repeatable manner and with a defined
goal can be seen as process. For example, in a health care context the trajectory
of a patient from arrival in the emergency ward to the admission to a hospital
ward and up to the discharge can be seen as a process. Often the execution of
such a processes is supported by information systems. For example, the hospital
may record medical information such as symptoms, the condition upon arrival of
the patient, and the results of blood tests. Moreover also logistical information
are recorded such as the movement of patients between wards and different types
of discharge.</p>
      <p>
        Process mining uses data recorded by such systems to analyze the actual
execution of processes. Common goals of process mining are: process discovery,
the discovery of accurate process models that help in understanding the process
execution and conformance checking, the diagnosis of problems in the process
execution by comparing the real execution (as-is process ) to a process model
that defines the desired process execution (to-be process ). Process mining has
been previously used in the health care context [
        <xref ref-type="bibr" rid="ref1 ref2 ref3">1,2,3</xref>
        ]. However, it is often stated
that the flexibility of the work in a hospital makes direct application of process
mining methods difficult [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ].
      </p>
      <p>
        We used state-of-the-art process mining methods to analyze the trajectories
of 1050 patient that are admitted to the emergency ward of a Dutch hospital1
because they display symptoms of a sepsis. Sepsis is a life-threatening condition
that requires immediate treatment [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. The goals of the process mining project
were:
1. to get insights in the patient trajectories,
2. to validate the compliance with medical guidelines for sepsis treatment, and
3. to evaluate the use of process mining methods in this context.
2
      </p>
    </sec>
    <sec id="sec-2">
      <title>Context and Data Collection</title>
      <p>
        We conducted the process mining project in a regional hospital in The Netherlands.
The hospital has about 700 beds at several locations and is visited by about
50,000 patients per year. The scope of our project was on the patient trajectories
of patients that are admitted to the emergency ward. We focused on a specific
sub group of patients toTotaalEvRPoatiiendts a common problem in process mining for health
care processes: the inherent complexity and flexibility of health care processes [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ].
Scoping our analysis to a single group of patients, for which a specific treatment
is to be expected, avoids some complexity.
      </p>
      <p>Which SIRS criteria?
IV therapy: Yes/No</p>
      <p>Antibiotics: Yes/No</p>
      <p>Type of visit?</p>
      <p>Which diagnostics</p>
      <p>are ordered?
Filter: Time, Delay, ..</p>
      <p>Discharge Home</p>
      <p>Is there organ dysfunction?</p>
      <p>Continuation
Admission
Normal Ward</p>
      <p>Admission ICU</p>
      <p>Admission from ER
to Normal Ward +
Admission to ICU
up to 72 hours after
ER visit
1 For privacy reasons we do not disclose the name of the hospital.
1. are particular medical guidelines for the treatment of sepsis patients followed:
– patients should be administered antibiotics within one hour,
– lactic acid measurements should be done within three hours;
2. visualize and investigate the following specific trajectories:
– discharge without admission,
– admission to the normal care,
– admission to the intensive care,
– admission to the normal care and transfer to intensive care;
3. investigate the trajectory of patients that return within 28 days.
Moreover, the document in Figure 1 facilitated the collection of event data. Based
on the document, we could identify several sources for events:
– the triage document filled in for sepsis patients with information on
the time the triage was conducted,
the symptoms present (SIRS criteria for sepsis),
the diagnostics ordered,
the time infusions of liquid and antibiotics were administered;
– documents created by the laboratory for several blood tests,
– information about the further trajectory of the patient recorded by financial
systems.</p>
      <p>All documents and records are stored in different databases of the ERP system
that is used by the hospital.</p>
      <p>emergency ward</p>
      <p>triage documents
record
record
record
laboratory
lab documents
consolidate
(SQL, Text
mining)</p>
      <p>extract
data warehouse
event log
other wards</p>
      <p>financial records
Fig. 2. Consolidation of data from several source systems to a single event log
Based on the identified sources, we collected data about several activities that are
executed for this group of patients from three different systems of the hospital.
The activities can be coarsely categorized into medical activities and logistical
activities.</p>
      <p>
        Figure 2 shows how we collected the event data and consolidated it into a single
anonymized event log covering the traces that were recorded for 1050 patients
over the course of 1.5 years in the hospital information systems. The resulting
event log is made available for further process mining research purposes [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. The
event log contains events for 16 activities:
– 3 activities regarding the registration and triaging in the emergency ward;
– 3 activities regarding measurement of leukocytes, CRP, and lactic acid;
– 2 activities regarding admission or transfer to normal care or intensive care;
– 5 activities for variants of discharge from the hospital; and
– an activity concerned with returning patients at a later time.
3
      </p>
    </sec>
    <sec id="sec-3">
      <title>Applied Process Mining Methods</title>
      <p>
        We applied several process mining methods on the event log. We started by
applying the Inductive Miner (IM) [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] as a state-of-the-art process discovery
method. Figure 3 shows the returned process model. Some of the structures that
we expected were discovered by the IM. For example, the process starts with
patient registration (ER Registration) and ends with some variants of discharge
(Release C-E). However, most of the activities that occur in between these
registration and discharge activities are modeled very imprecisely. Almost all
activities can be skipped and repeated. We highlighted the loop edge in Figure 3.
      </p>
      <p>Therefore, the process model returned by IM is difficult to be used for the
communication with doctors and nurses. At a first glance it appears that administering
antibiotics (IV Antibiotics) and filling in the triage document (ER Sepsis Triage)
is in an exclusive choice. Yet, it is possible to repeat the entire middle part of
the process, thus, both activities can occur in the same process instance. Using
different parameter settings for the IM did not help. Even though we could
obtained a more precise process model, unfortunately, that process model does not
allow the laboratory tests to be repeated. Moreover, it does not allow to visualize
the patient trajectory: first admission to normal care and, then, a transfer to
intensive care. This trajectory is important to address Question 2. Since we use
process discovery to address concrete questions, we did not rely on measures
such as fitness and precision to estimate the quality of the model. Instead, we
determined the model quality based on whether it is suited to answer our process
questions and whether it is helpful in communicating with doctors and nurses.</p>
      <p>As an alternative to automatic process discovery, we iteratively designed a
suitable process model based on domain knowledge obtained from doctors and
nurses working in the emergency ward. One example for such domain knowledge
are the descriptions shown in Figure 1. In each iteration, we used alignment-based
conformance checking techniques to validate whether the hand-drawn process
model reflects the observations in the event log. Since the iteratively designed
model follows the expectations of doctors and nurses working in the emergency
ward, it is can be used (1) to communicate with the stakeholders, and (2) to
answer the process questions.</p>
      <p>
        Figure 4 shows the iteratively designed model. It fits 98.3% of the event log,
i.e., almost all of the events can be correctly replayed on the model. We used the
ProM plug-in Multi-perspective Process Explorer (MPE) [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] to project the event
log onto the process model and explore the process. Figure 4 also depicts the
output of the MPE. The edges are scaled according to the number of cases that
are projected on the edge. The Petri net transitions are colored according to the
conformance level, the darker a transition the more conformance violations. We
used the MPE to explore the three identified questions about the process.
      </p>
      <p>Question 1: medical guidelines There are two time rules specified by the sepsis
guideline:
1. between ER Sepsis Triage and IV Antibiotics should be less than 1 hour,
2. between ER Sepsis Triage and LacticAcid should be less than 3 hours.</p>
      <p>
        We used Data Petri nets (DPN) to encode both time-perspective rules using
three variables timeTriage, timeAntibiotics, and timeLacticAcid. We added guard
expressions to both activities IV Antibiotics (timeAntibiotics’ timeTriage +
60) and LacticAcid (timeLacticAcid’ timeTriage + 180). Then, we used the
multi-perspective conformance checking technique that is described in work [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] to
check conformance. Rule 1 regarding administering antibiotics within one hour is
sometimes violated: IV Antibiotics and timeAntibiotics in Figure 4 are colored
orange. The average time between ER Sepsis Triage and IV Antibiotics is 1.7
hours and the rule is violated 58.5% of the cases. Regarding the second rule the
situation is much better. Rule 2 regarding the timely measurement of lactic acid
is only violated in 0.7% of the cases. The high number of violations regarding
rule 1 can be explained be two factors:
– Guidelines recommend to administer antibiotics within one hours, however,
acknowledged that this is not always feasible [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. Moreover, not all patients
in our event log show symptoms of a severe sepsis. Thus, the one hour rule
can be considered very strict.
– The data about the infusion of antibiotics is entered manually by nurses and
doctors in a form as shown in Figure 5. When looking at the data, we found
some cases in which the entered antibiotics timestamp is 24 hours after the
triage and other cases in which the antibiotics timestamp was before the
triage document was filled in. Thus, it is difficult to conclude actions from
the measurement, since it might be caused by poor data quality.
      </p>
      <p>Question 2: patient trajectories Using the MPE, we could visualize the four
trajectories of interest. In Figure 4 they are represented as an exclusive choice
in the main branch of the process on the right side. The choice regarding the
admission of patients is made after filling in the general triage document (ER
Triage), the triage document for patients with suspicion for a sepsis (ER Sepsis
Triage), and, possibly, giving infusions of antibiotics and liquid (IV Liquid and
IV Antibiotics ). It can be seen that 18.1% of the patients leave the emergency
ward without admission to the hospital. Most patients (70.6%) are admitted
to the normal care ward (Admission NC ), less patients (6.8%) are admitted to
the intensive care ward (Admission IC ), and a small group of patients (3.6%)
is first admitted to the normal care ward and, directly afterwards, admitted to
the intensive care ward. The latter group of patients is of interest since their
condition got worse. Therefore, it would be beneficial to avoid patients following
this trajectory.</p>
      <p>Volgende</p>
      <p>
        pagina
Question 3: returning patients We tried decision mining techniques such as
described in [
        <xref ref-type="bibr" rid="ref10 ref9">9,10</xref>
        ] to discover rules regarding patients that return within some
time to the emergency ward. In total 27.8% of the patients return to the emergency
room. On average it takes 81.6 days for a patient to return. We were only interested
in patients that returned within 28 days, thus, we filteredKwtahlieteietsvheanndtblooegk: aAcSczoArcdutienzgolryg.Medische behandelprotocollen
Out of all patients 12.6% return within 28 days. Unfortunately, we did not find
any decision rule based on the available data attributes of the event log, which
include checkboxes marked in the triage form and the values of blood tests.
4
      </p>
    </sec>
    <sec id="sec-4">
      <title>Lessons Learned</title>
      <p>We now report on the lessons learned from analyzing an event log recorded for
sepsis patients in a regional hospital with process mining technology. The project
was successful in visualizing the trajectories of patients with a suspicion for sepsis.
In fact, a doctor found the process mining analysis to be "[a] magnificent way to
clarify patient flow".</p>
      <p>
        Unfortunately, little actionable results were obtained. We attribute this to
the lack of a set of initial hypotheses that could lead to actionable result. We
started the project with some process questions such as the compliance to the
medical guidelines and patients returning within 28 days. Still, it turned out to
be difficult to follow-up on the findings due to lack of data quality regarding the
time when antibiotics were given and the general lack of data that could explain
the returning patients. In conclusion, we found that:
1. Process mining can be used to clarify and visualize the flow of patients
in a hospital. This confirms previously reported results in [
        <xref ref-type="bibr" rid="ref1 ref2 ref3">1,2,3</xref>
        ]. However,
we show that when focusing on a specific group of patients and a mix of
medical and logistical activities, the often cited flexibility of health care
processes [
        <xref ref-type="bibr" rid="ref1 ref2 ref3">1,2,3</xref>
        ] can be avoided.
2. Often hospitals monitor their processes by looking only at quality indicators
(e.g., length of the hospitalization, percentage of on-time surgeries). Process
mining can be used to check the conformance to medical guidelines by
hospitals in the context of treatment and patient logistics processes. Medical
guidelines [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] can be encoded as part of the process model and the whole
process (i.e. pathways, outcomes, waiting times, and guidelines) can be
monitored with an alignment technique [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. Similar results have been presented
for a declarative modeling notation in [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. We show that procedural notation
are also suitable when focusing on a specific process.
3. Process discovery methods may return models of the observed behavior that
are unsuitable to communicate with stakeholders and answer questions. The
discovered process model might correctly represent the observed behavior.
However, its structure may not be recognizable to the people working in the
process. Iteratively creating a hand-made model proved to be useful and
feasible in this case.
4. Obtaining the data and analyzing the process is an iterative process. Data
quality issues and further questions may arise only after some initial data
has been collected. Regular feedback from process stakeholders to validate
assumptions is important. Iterative analysis and data enrichment has also
been proposed in process mining methodologies such as [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]. In our case we
also extracted more data from the source systems after an initial analysis
iteration.
      </p>
      <p>Acknowledgments We would like to thank the hospital and the doctor for the
support with the process mining project and the provided insights.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Mans</surname>
            , R.S., van der Aalst,
            <given-names>W.M.P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vanwersch</surname>
            ,
            <given-names>R.J.B.</given-names>
          </string-name>
          : Process Mining in Healthcare. Springer (
          <year>2015</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Rebuge</surname>
          </string-name>
          , Á.,
          <string-name>
            <surname>Ferreira</surname>
            ,
            <given-names>D.R.</given-names>
          </string-name>
          :
          <article-title>Business process analysis in healthcare environments: A methodology based on process mining</article-title>
          .
          <source>Information Systems</source>
          <volume>37</volume>
          (
          <issue>2</issue>
          ) (
          <year>2012</year>
          )
          <fpage>99</fpage>
          -
          <lpage>116</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Rovani</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Maggi</surname>
            ,
            <given-names>F.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>de Leoni</surname>
          </string-name>
          , M.,
          <string-name>
            <surname>van der Aalst</surname>
          </string-name>
          , W.M.:
          <article-title>Declarative process mining in healthcare</article-title>
          .
          <source>Expert Syst. Appl</source>
          .
          <volume>42</volume>
          (
          <issue>23</issue>
          ) (
          <year>2015</year>
          )
          <fpage>9236</fpage>
          -
          <lpage>9251</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Rhodes</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , et al.:
          <article-title>Surviving sepsis campaign: International guidelines for management of sepsis and septic shock: 2016</article-title>
          .
          <source>Intensive Care Med</source>
          .
          <volume>43</volume>
          (
          <issue>3</issue>
          ) (
          <year>2017</year>
          )
          <fpage>304</fpage>
          -
          <lpage>377</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Mannhardt</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          :
          <string-name>
            <surname>Sepsis Cases - Event Log</surname>
          </string-name>
          . Eindhoven University of Technology. Dataset. (
          <year>2016</year>
          ) doi:10.4121/uuid:
          <fpage>915d2bfb</fpage>
          -7e84
          <string-name>
            <surname>-</surname>
          </string-name>
          49ad
          <source>-a286-dc35f063a460.</source>
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Leemans</surname>
            ,
            <given-names>S.J.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fahland</surname>
            , D., van der Aalst,
            <given-names>W.M.P.</given-names>
          </string-name>
          :
          <article-title>Discovering block-structured process models from event logs - A constructive approach</article-title>
          .
          <source>In: Petri Nets</source>
          . Volume
          <volume>7927</volume>
          of LNCS., Springer (
          <year>2013</year>
          )
          <fpage>311</fpage>
          -
          <lpage>329</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Mannhardt</surname>
          </string-name>
          , F.,
          <string-name>
            <surname>de Leoni</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Reijers</surname>
            ,
            <given-names>H.A.</given-names>
          </string-name>
          :
          <article-title>The multi-perspective process explorer</article-title>
          . In Daniel, F.,
          <string-name>
            <surname>Zugal</surname>
          </string-name>
          , S., eds.:
          <source>BPM (Demos)</source>
          .
          <source>Volume 1418 of CEUR Workshop Proceedings., CEUR-WS.org</source>
          (
          <year>2015</year>
          )
          <fpage>130</fpage>
          -
          <lpage>134</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Mannhardt</surname>
          </string-name>
          , F.,
          <string-name>
            <surname>de Leoni</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Reijers</surname>
          </string-name>
          , H.A.,
          <string-name>
            <surname>van der Aalst</surname>
            ,
            <given-names>W.M.P.</given-names>
          </string-name>
          :
          <article-title>Balanced multiperspective checking of process conformance</article-title>
          .
          <source>Computing</source>
          <volume>98</volume>
          (
          <issue>4</issue>
          ) (
          <year>2016</year>
          )
          <fpage>407</fpage>
          -
          <lpage>437</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9. de Leoni, M.,
          <string-name>
            <surname>van der Aalst</surname>
            ,
            <given-names>W.M.P.</given-names>
          </string-name>
          :
          <article-title>Data-aware process mining: Discovering decisions in processes using alignments</article-title>
          .
          <source>In: SAC'13. SAC '13</source>
          , New York, NY, USA, ACM (
          <year>2013</year>
          )
          <fpage>1454</fpage>
          -
          <lpage>1461</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Mannhardt</surname>
          </string-name>
          , F.,
          <string-name>
            <surname>de Leoni</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Reijers</surname>
          </string-name>
          , H.A.,
          <string-name>
            <surname>van der Aalst</surname>
            ,
            <given-names>W.M.P.</given-names>
          </string-name>
          :
          <article-title>Decision mining revisited - discovering overlapping rules</article-title>
          . In Nurcan, S.,
          <string-name>
            <surname>Soffer</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bajec</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Eder</surname>
          </string-name>
          , J., eds.:
          <source>CAiSE</source>
          . Volume
          <volume>9694</volume>
          of LNCS., Springer (
          <year>2016</year>
          )
          <fpage>377</fpage>
          -
          <lpage>392</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Peleg</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tu</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bury</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ciccarese</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fox</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Greenes</surname>
            ,
            <given-names>R.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hall</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          , Johnson, P.D.,
          <string-name>
            <surname>Jones</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kumar</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Miksch</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Quaglini</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Seyfang</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shortliffe</surname>
            ,
            <given-names>E.H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stefanelli</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Comparing computer-interpretable guideline models: A case-study approach</article-title>
          .
          <source>J. Am. Med</source>
          . Inform. Assoc.
          <volume>10</volume>
          (
          <issue>1</issue>
          ) (jan
          <year>2003</year>
          )
          <fpage>52</fpage>
          -
          <lpage>68</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12. van Eck,
          <string-name>
            <given-names>M.L.</given-names>
            ,
            <surname>Lu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            ,
            <surname>Leemans</surname>
          </string-name>
          ,
          <string-name>
            <surname>S.J.J</surname>
          </string-name>
          ., van der Aalst,
          <string-name>
            <surname>W.M.P.:</surname>
          </string-name>
          <article-title>PM2: A process mining project methodology</article-title>
          .
          <source>In: CAiSE</source>
          . Volume
          <volume>9097</volume>
          of LNCS., Springer (
          <year>2015</year>
          )
          <fpage>297</fpage>
          -
          <lpage>313</lpage>
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>