<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>Hertogenbosch, The Netherlands
* Corresponding author.
$ f.a.bukhsh@utwente.nl (F. Bukhsh); p.v.naguine@student.utwente.nl (P. Naguine);
j.a.jayasinghearachchige@utwente.nl (J. Jayasinghe)</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Subpopulation process comparison with the help of ontological foundation: A discussion</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Faiza Bukhsh</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Priya Naguine</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Jeewanie Jayasinghe</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>University of Twente</institution>
          ,
          <addr-line>Enschede</addr-line>
          ,
          <country country="NL">The Netherlands</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2024</year>
      </pub-date>
      <volume>000</volume>
      <fpage>0</fpage>
      <lpage>0001</lpage>
      <abstract>
        <p>Process modelling and mining frameworks have demonstrated their efectiveness across diverse domains, including healthcare. However, existing frameworks often lack explicit guidance on learning from best practices. For instance, the case of Frozen shoulder (FS), a condition with multiple treatment options and varying outcomes. Understanding how care paths difer among patient groups and determining the most efective approach remains a challenge. By identifying this gap, our research employs the Process Mining Project Methodology in Healthcare (  2) alongside the MIMIC-IV dataset to uncover distinctions in care paths among diferent age groups and genders. This experimental validation seeks to identify optimal strategies for addressing Frozen shoulders through ontological concepts. The study concludes by presenting a set of open challenges, aiming to guide future research in healthcare by integrating ontological concepts to learn from the best and optimal care paths. It is important to note that while this research doesn't ofer a singular solution, it contributes significantly by opening a new dimension of ontological research. Specifically, it delves into how various care paths can be compared and aligned with the help of ontological foundation.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;subpopulation comparison</kwd>
        <kwd>ontology</kwd>
        <kwd>frozen shoulder</kwd>
        <kwd>process mining</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Medical professionals often question whether there is a diference in the treatment procedures
followed by subgroups of patients diagnosed with the same disease [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. In this case, a subgroup
refers to a group of patients with a common characteristic, e.g., all female patients diagnosed with
frozen shoulder. When comparing subpopulations, experts’ knowledge is essential. However,
Ontology-based interpretation is a valuable technique for capturing better insight into a complex
domain like healthcare.
      </p>
      <p>
        In the context of information science and knowledge representation, an ontology is a formal
and explicit specification of a shared conceptualization [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. It provides a structured framework
for representing knowledge in a particular domain by defining the entities, their properties,
and the relationships between them. Ontologies aim to capture a common understanding
of a domain and facilitate communication and interoperability among diferent systems and
applications [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ].
      </p>
      <p>
        This research aims to identify carepath for diferent subpopulations and learn from best
practices. Since age and gender play a role in the development of a disease such as FS [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ],
these can be chosen as the subgroups. There has not been much research on comparing these
subpopulations. Ontological concepts are rich in nature and can provide a methodological way
to compare subpopulations [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
      </p>
      <p>Therefore, the objective of this research is to find the diferences and similarities between
the care paths for diferent subpopulations and learn from best practices with the guidance of
ontological foundations.</p>
      <p>
        As an example scenario, we used the MIMIC-IV database and analysed the procedures
followed by FS patients; this database contains data on approximately 300,000 patients that
were admitted to a tertiary academic medical centre in Boston, the USA, between 2008 and 2019
[
        <xref ref-type="bibr" rid="ref6 ref7">6, 7</xref>
        ]. Specifically, process mining takes data from hospital information systems (HIS) when
applied in the healthcare domain [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. Event logs are then created using the data from the HIS to
show the sequence of processes followed by patients. The event logs created can then be used
to find the diferences in the care paths followed by subgroups of patients with FS. Further,the
diferences and similarities of the care paths were analyzed with BPMNDif Viz tool 1 to show
comparison examples.
      </p>
      <p>This paper is structured as follows. The state of the art will be described in section 2.
Section 3 will describe the methodology used with the title of "An example Scenario: Frozen
Shoulder Exploration with the Application of Process Mining (PM)". Section 4 will discuss the
subpopulation comparison through ontological foundations. Finally, in the last section 5, the
paper concludes with opening insightful research directions to the reader.</p>
    </sec>
    <sec id="sec-2">
      <title>2. State of the art</title>
      <p>
        Process mining techniques can be used for various purposes in the healthcare domain, e.g., with
BPMN diagrams to get the graph edit distances. [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] used process mining techniques specifically
for the process comparison of subgroups. There was a focus on the application of process
mining for subpopulation process comparison between patients diagnosed with diferent types
of cancer.
      </p>
      <p>
        The tool BPMNDif Viz can be used to find graph similarity measures. It takes as input two
BPMN diagrams and gives the minimal graph edit distance (GED) as a result. The GED can
be defined as the minimum number of steps required to transform one graph into another [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ].
This tool makes use of Business Process Model and Notation (BPMN) 2.0, which is one of the
frequently used notations used for process modelling [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] states that although the use of
BPMN diagrams in medicine is a recent development, it can be used to model clinical pathways
to teach and train medical staf.
      </p>
      <p>
        Visual comparison can be used to diferentiate between the care paths followed by subgroups
of patients and the tool BPMNDif Viz can be used for that. BPMNDif Viz allows for a choice
between six comparison algorithms: Greedy, TabuSearch, Genetic, AStar, Ants and simulated
annealing. [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] compares the algorithms mentioned except Genetic and concludes that the
Greedy algorithm gives the best performance results while the TabuSearch algorithm gives
      </p>
      <sec id="sec-2-1">
        <title>1https://pais.hse.ru/en/research/projects/CompBPMN/</title>
        <p>
          more precise and accurate results. The Genetic algorithm only gives an approximation of the
GED [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ].
        </p>
        <p>Subpopulation comparison based on visual aspects or graph edit distances provides us with
an initial view. However, the robustness of these comparisons should be strengths beyond the
statistical figures for making decisions in real situations, especially in complex domains like
healthcare.</p>
        <p>
          Ontological foundation is one of the potential approaches that can be used to ensure the
accuracy of domain structures. Ontological concepts are hierarchical domain structures that
provide a domain theory, have a syntactically and semantically rich language, and a shared and
consensual terminology [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ].
        </p>
        <p>
          The work of [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ] explores ontology learning, a dynamic research field crucial for efective
ontology engineering. It distinguishes ontology-based definitions from conventional
labelcentric ones, emphasizing the interconnected nature of objects. This shift allows for advanced
functionalities such as scenario search, ontology fusion, and recommendation through nuanced
relation labelling. Moreover work of [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ] discusses the potential of ontology-based process
modelling (OBPM) to enhance business process management theoretically.
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. An example scenario: Frozen shoulder exploration with the application of process mining (PM)</title>
      <p>
        In the following section, we will methodically elaborate on a specific scenario to illustrate the
design and comparison of subpopulations. Throughout this example, we will highlight the
potential role that could be played by ontological concepts in shaping and evaluating these
subpopulations. The methodology to be used in this research is called Process Mining Project
Methodology in Healthcare (  2) [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ].   2 involves 6 phases: planning, extraction,
data processing, mining and analysis, evaluation, and improvement and support.
      </p>
      <sec id="sec-3-1">
        <title>3.1. Planning</title>
      </sec>
      <sec id="sec-3-2">
        <title>3.2. Extraction</title>
        <p>
          During this phase, we chose specific subgroups to explore various care paths and organized
the sequence of events. Additionally, we conducted thorough background research on frozen
shoulder and process mining in healthcare, as detailed in the [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ].
        </p>
        <p>
          In order to get access to and query the MIMIC-IV database, Google Cloud Platform BigQuery2
was used. Since the MIMIC-IV database stores the diagnoses given to the patients at the end
of their ICU stay using the International Classification of Diseases (ICD) Version 9 and 10
codes, the first step was to find the ICD codes associated with frozen shoulder. This was
found in the D_ICD_DIAGNOSES table [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ] using the keywords frozen shoulder and adhesive
capsulitis for the long_title. The ICD codes are 7260, M750, M7500, M7501, and M7502 and
their corresponding diagnoses are "Adhesive capsulitis of shoulder", "Adhesive capsulitis of
        </p>
        <sec id="sec-3-2-1">
          <title>2https://cloud.google.com/bigquery</title>
          <p>shoulder", "Adhesive capsulitis of unspecified shoulder", "Adhesive capsulitis of right shoulder"
and "Adhesive capsulitis of the left shoulder." It is important to note that there is a possibility
that a patient is given more than one diagnosis associated with the frozen shoulder in a single
hospitalization, e.g., a patient can be diagnosed with both M7501 and M7502.</p>
          <p>To apply process mining algorithms to the data, the cases, events, start times and end times
have to be defined. For both the subgroup process comparison and bottleneck analysis, a case is
a patient’s admission to the hospital and the events are the procedures that the patients were
billed for.</p>
          <p>Since the start and end times were not stored for the subgroup process comparison, the
sequence number was used instead to indicate the order in which the procedures were carried
out.</p>
        </sec>
      </sec>
      <sec id="sec-3-3">
        <title>3.3. Data processing</title>
        <p>In this phase, the CSV files on the subgroups were entered into ProM, converted into XES files
and visualised using the LogVisualiser (LogDialog) plugin. Table 1 gives an overview of the
number of cases and events per subgroup, given by the LogDialog. Also, further filtering was
required to find the diferences in care paths between the diferent patient groups. This was
done using the Filter Log on Event Attribute Values plugin, where specific procedures were
ifltered out from the care paths.</p>
      </sec>
      <sec id="sec-3-4">
        <title>3.4. Mining and analysis</title>
        <p>This phase involved finding the diferences in the care paths between the diferent subgroups
and the bottlenecks in the medications taken and the procedures followed by patients during
their ICU stays. To do this, process models were created in ProM3 and Disco4.</p>
        <p>
          The Inductive Miner plugin was chosen because it gives the best fitness, i.e., the degree by
which the process models generated can recreate the cases in the event log [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ]. At first, the
plugin Mine with Inductive visual Miner was used because it can create animations showing
the order in which the processes occur; it was used with the activities slider set to 1 and the
        </p>
        <sec id="sec-3-4-1">
          <title>3https://promtools.org/</title>
          <p>4https://fluxicon.com/disco/
paths slider set to 0.8. These settings were chosen so that the Petri net and the Inductive visual
Miner models are equivalent. Secondly, Mine Petri net with Inductive Miner was used to
create static process models that can be used for visual comparison, with a noise threshold of
0.2 to allow for slight deviations. Lastly, in order to convert the Petri net models into BPMN
diagrams so that they can be loaded into BPMNDif Viz to get the GED, Convert Petri net to
BPMN diagram was used.</p>
          <p>The process models created in ProM and Disco for the subgroup process comparison and
bottleneck analysis can be found in the author’s GitHub repository5.</p>
          <p>When comparing the care paths of the subgroups, three keywords will be used. Firstly,
parallel will be used when two procedures occur but the order in which they occur does not
matter. Secondly, sequence is used when one procedure follows another. Lastly, exclusive will
be used when only one of two procedures can occur.</p>
          <p>
            Also, visual comparison is performed in BPMNDif Viz using the TabuSearch algorithm with
maximum expansions and tabu list size set to 100 as this gives precise results faster than other
algorithms [
            <xref ref-type="bibr" rid="ref9">9</xref>
            ]. The activities in the BPMN diagrams are encoded with diferent colours: blue
denotes elements that match between the subgroups, green denotes elements that should be
added to transform one diagram into the other and red denotes elements that should be deleted
to transform one diagram into the other.
          </p>
          <p>Visual comparison was made in BPMNDif Viz for the care paths followed by male and female
FS patients, resulting in a final score of 167 using the TabuSearch algorithm. 37% of the elements
matched between the care paths, 33% of the elements were deleted and 30% of the elements
were added. Table 2 shows the procedures that are only performed on either female or male FS
patients, but not both.</p>
          <p>The procedure "Other repair of shoulder" can be done in parallel with "Division of joint
capsule, ligament, or cartilage, shoulder" in male patients while in female patients, these
procedures are performed in sequence. Furthermore, it is performed in sequence with "Rotator</p>
        </sec>
        <sec id="sec-3-4-2">
          <title>5https://github.com/PriyaNaguine/Complete-Process-Models-Frozen-Shoulder</title>
          <p>localhost:8080/ru_pais_vkr_war/comparison/fourth_step 1/2
localhost:8080/ru_pcais_uvkr_wfar/rcomeparispon/fouarth_sitepr" in male FS patients. However, in fe1/2male patients, these processes are exclusive. This
can be seen in figures 1a and 1b.</p>
          <p>As can be seen in figures 1a and 1b, the procedure "Synovectomy, shoulder" is always the
last process in male FS patients, in case it is performed. In female patients, it is exclusive to
"Rotator cuf repair", while in male patients, they can occur in sequence, where "Rotator cuf
repair" is the first procedure and "Synovectomy, shoulder" is the last procedure to take place.</p>
        </sec>
      </sec>
      <sec id="sec-3-5">
        <title>3.5. Evaluation and improvements</title>
        <p>In this phase, the insights obtained in the previous phase were used to suggest improvements
and learn from care path by considering the best practices. In this phase, the stakeholders, e.g.,
medical professionals, decide on the path to be followed to implement the improvements.</p>
        <p>This phase was conducted with an expert physiotherapist at Fysiotherapie Polman in
Enschede, The Netherlands, in order to discuss and evaluate the results of this research. Thereby,
based on the discussion of the results found with the physiotherapist, which is based on his 8
years of experience working as a physiotherapist for FS, more insight was gained on patients of
FS. In particular, there is a ratio of approximately 7:3 between female and male FS patients. This
could be because female patients ask for help earlier on. Based on his experience, there is no
diference in the care paths followed by male and female FS patients. Also, it was mentioned that
the age group between 40 and 60 years old is more prone to developing FS and this applies to
both genders. Furthermore, older people, i.e., those aged above 60, are more likely to experience
FS after shoulder trauma. In this age group, they are less likely to get surgery as it is an invasive
procedure. In general, depending on the health conditions of the patient, the older they are, the
more they are at risk of developing complications.</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Subpopulatoin comparison through ontological foundation</title>
      <p>Section 3 of our research shows how diferent care paths of subpopulations derived using
process mining and how they compare using BPMNDif Viz tool. Subpopulation comparison
based on visual aspects or graph edit distances provides us with an initial view. There is no
doubt, that we can argue the robustness of these comparisons in terms of statistical figures.
However, the question is, whether these statistical figures are suficient for making decisions in
real situations, especially in complex domains like healthcare. Ontological foundation is one of
the good approaches that can be used to ensure not only the structural correctness but also the
accuracy of domain knowledge in the derived models.</p>
      <p>In our research case study, we analyzed the treatment procedures and care paths for FS within
two distinct subpopulations. The central focus of this study revolves around understanding
the variations in care paths among diferent patient groups and determining the most efective
approach. Naturally, the expertise of domain professionals serves as the primary and most
iftting source of knowledge for these investigations. Secondly, the ontological foundation can
be employed to determine the best care path. Surprisingly, research work is scarce on using
ontologies for comparing (assessing similarities or diferences) diferent care paths based on
subpopulations.</p>
      <p>To address this, our case study explores a research direction on establishing a method for
comparing subpopulations within a given knowledge domain, along with defining appropriate
evaluation criteria. These criteria encompass the ontological richness and the reliability of
methodologies in conceptualization, shareability in terms of sources and granularity, explicitness
and formality through implementation tools and formalization language, and adherence to
design criteria within the methodological process of building ontologies.</p>
      <p>In essence, our proposal leverages the significance of the ontology definition as a foundation
for comparison features, ensuring a comprehensive evaluation that goes beyond traditional
similarity metrics.</p>
    </sec>
    <sec id="sec-5">
      <title>5. Food for thought</title>
      <p>The use of ontology in subpopulation comparison involves various dimensions. Ontologies
prove beneficial in comprehending and conceptualization. Below are key discussion points
highlighting the ways in which ontology can be utilized for comparing subpopulations.
• Conceptual Clarity: Ontologies help to define and clarify the concepts related to
subpopulations. By establishing a common understanding of terms, attributes, and relationships,
ontology ensures clarity in the representation of diverse sub-groups.
• Semantic Interoperability: Ontological representations facilitate semantic interoperability,
allowing for the integration of diverse data sources and the comparison of
subpopulations across diferent datasets. This is crucial for ensuring consistency and accuracy in
comparisons.
• Granular Attribute Definition: Ontologies allow for the granular definition of attributes
associated with subpopulations. This includes demographic information, medical conditions,
or any relevant factors. This granularity enhances the precision of comparisons.
• Relationship Modeling: Ontologies capture relationships between entities, enabling the
modelling of complex interactions within subpopulations. This is particularly valuable
when comparing the influence of diferent factors on health outcomes or other relevant
criteria.
• Automated Inference: Ontologies support automated reasoning and inference, allowing
for the deduction of additional information based on the defined relationships. This
capability aids in uncovering hidden patterns or correlations within subpopulations.
• Consistent Terminology: Ontologies promote the use of consistent and standardized
terminology, reducing ambiguity in the description of subpopulations. Consistency in
terminology is crucial for accurate and meaningful comparisons.
• Facilitating Data Integration: Ontologies provide a common framework for integrating
data from diverse sources, making it easier to compare subpopulations across diferent
studies or datasets. This promotes a more comprehensive understanding of variations
and similarities.
• Enabling Query and Retrieval: Ontologies enhance the eficiency of querying and
retrieving relevant information about subpopulations. Researchers can formulate queries using
ontological terms, streamlining the comparison process.</p>
      <p>While we acknowledge that this list may not be exhaustive, it represents our initial efort to
address the multifaceted nature of this complex research. In essence, ontology serves as a
powerful tool in subpopulation comparison by ofering a structured, standardized, and
semantically rich representation of entities and their relationships. This approach contributes to more
meaningful, accurate, and eficient comparisons across diverse subsets of a population.</p>
    </sec>
    <sec id="sec-6">
      <title>Acknowledgments</title>
      <p>We would like to thank the physiotherapist at Fysiotherapie Polman in Enschede for sharing
important insights on frozen shoulder.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>R. S.</given-names>
            <surname>Mans</surname>
          </string-name>
          ,
          <string-name>
            <surname>W. M. P. van der Aalst</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. J. B.</given-names>
            <surname>Vanwersch</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. J.</given-names>
            <surname>Moleman</surname>
          </string-name>
          ,
          <article-title>Process mining in healthcare: Data challenges when answering frequently posed questions</article-title>
          , in: R.
          <string-name>
            <surname>Lenz</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Miksch</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Peleg</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Reichert</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          <string-name>
            <surname>Riaño</surname>
          </string-name>
          , A. ten Teije (Eds.),
          <source>Process Support and Knowledge Representation in Health Care</source>
          , Springer Berlin Heidelberg, Berlin, Heidelberg,
          <year>2013</year>
          , pp.
          <fpage>140</fpage>
          -
          <lpage>153</lpage>
          . doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>642</fpage>
          -36438-9_
          <fpage>10</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>N.</given-names>
            <surname>Guarino</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Oberle</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Staab</surname>
          </string-name>
          ,
          <article-title>What is an ontology?</article-title>
          , Handbook on ontologies (
          <year>2009</year>
          )
          <fpage>1</fpage>
          -
          <lpage>17</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>T.</given-names>
            <surname>Gruber</surname>
          </string-name>
          ,
          <source>What is an ontology</source>
          ,
          <year>1993</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>R.</given-names>
            <surname>Koorevaar</surname>
          </string-name>
          , E. Riet,
          <string-name>
            <given-names>M.</given-names>
            <surname>Ipskamp</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Bulstra</surname>
          </string-name>
          ,
          <article-title>Incidence and prognostic factors for postoperative frozen shoulder after shoulder surgery: a prospective cohort study</article-title>
          ,
          <source>Archives of Orthopaedic and Trauma Surgery</source>
          <volume>137</volume>
          (
          <year>2017</year>
          ).
          <source>doi:10.1007/s00402-016-2589-3.</source>
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>F.</given-names>
            <surname>Marazza</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F. A.</given-names>
            <surname>Bukhsh</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Geerdink</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            <surname>Vijlbrief</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Pathak</surname>
          </string-name>
          ,
          <string-name>
            <surname>M.</surname>
          </string-name>
          v. Keulen,
          <string-name>
            <given-names>C.</given-names>
            <surname>Seifert</surname>
          </string-name>
          ,
          <article-title>Automatic process comparison for subpopulations: Application in cancer care</article-title>
          ,
          <source>International Journal of Environmental Research and Public Health</source>
          <volume>17</volume>
          (????).
          <source>doi:10</source>
          .3390/ ijerph17165707.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>A.</given-names>
            <surname>Goldberger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Amaral</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Glass</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Hausdorf</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Ivanov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Mark</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Mietus</surname>
          </string-name>
          , G. Moody, C. Peng, H. Stanley, PhysioBank, PhysioToolkit, and
          <article-title>PhysioNet: components of a new research resource for complex physiologic signals</article-title>
          ,
          <source>Circulation [Online] 101</source>
          (
          <year>2000</year>
          )
          <article-title>"e215- e220"</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>A.</given-names>
            <surname>Johnson</surname>
          </string-name>
          , L. Bulgarelli,
          <string-name>
            <given-names>T.</given-names>
            <surname>Pollard</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Horng</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. A.</given-names>
            <surname>Celi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Mark</surname>
          </string-name>
          ,
          <article-title>MIMIC-IV (version 1</article-title>
          .0),
          <year>2021</year>
          . doi:
          <volume>10</volume>
          .13026/s6n6-
          <fpage>xd98</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>J.</given-names>
            <surname>Munoz-Gama</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Martin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Fernandez-Llatas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O. A.</given-names>
            <surname>Johnson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Sepúlveda</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Helm</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Galvez-Yanjari</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Rojas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Martinez-Millana</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Aloini</surname>
          </string-name>
          ,
          <string-name>
            <given-names>I. A.</given-names>
            <surname>Amantea</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Andrews</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Arias</surname>
          </string-name>
          ,
          <string-name>
            <given-names>I.</given-names>
            <surname>Beerepoot</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Benevento</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Burattin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Capurro</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Carmona</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Comuzzi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Dalmas</surname>
          </string-name>
          , R. de la Fuente,
          <string-name>
            <given-names>C.</given-names>
            <surname>Di Francescomarino</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Di Ciccio</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Gatta</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Ghidini</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Gonzalez-Lopez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Ibanez-Sanchez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H. B.</given-names>
            <surname>Klasky</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. Prima</given-names>
            <surname>Kurniati</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Lu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Mannhardt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Mans</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Marcos</surname>
          </string-name>
          , R. Medeiros de Carvalho,
          <string-name>
            <given-names>M.</given-names>
            <surname>Pegoraro</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. K.</given-names>
            <surname>Poon</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Pufahl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H. A.</given-names>
            <surname>Reijers</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Remy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Rinderle-Ma</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Sacchi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Seoane</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Song</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Stefanini</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Sulis</surname>
          </string-name>
          ,
          <string-name>
            <surname>A. H. ter Hofstede</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P. J.</given-names>
            <surname>Toussaint</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Traver</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Z.</given-names>
            <surname>Valero-Ramon</surname>
          </string-name>
          , I. van de Weerd, W. M. van der Aalst, R. Vanwersch,
          <string-name>
            <given-names>M.</given-names>
            <surname>Weske</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. T.</given-names>
            <surname>Wynn</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Zerbato</surname>
          </string-name>
          ,
          <article-title>Process mining for healthcare: Characteristics and challenges</article-title>
          ,
          <source>Journal of Biomedical Informatics</source>
          <volume>127</volume>
          (
          <year>2022</year>
          ). doi:https://doi.org/10.1016/j.jbi.
          <year>2022</year>
          .
          <volume>103994</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>A.</given-names>
            <surname>Skobtsov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Kalenkova</surname>
          </string-name>
          ,
          <article-title>Eficient algorithms for finding diferences between process models</article-title>
          ,
          <year>2019</year>
          , pp.
          <fpage>60</fpage>
          -
          <lpage>66</lpage>
          . doi:
          <volume>10</volume>
          .1109/ISPRAS47671.
          <year>2019</year>
          .
          <volume>00015</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>S.</given-names>
            <surname>Ivanov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Kalenkova</surname>
          </string-name>
          , W. Aalst,
          <article-title>Bpmndifviz: A tool for bpmn models comparison 1418 (</article-title>
          <year>2015</year>
          )
          <fpage>35</fpage>
          -
          <lpage>39</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>H.</given-names>
            <surname>Scheuerlein</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Rauchfuss</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Dittmar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Molle</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Lehmann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Pienkos</surname>
          </string-name>
          , U. Settmacher,
          <article-title>New methods for clinical pathways-business process modeling notation (bpmn) and tangible business process modeling (t</article-title>
          .bpm),
          <source>Langenbeck's archives of surgery / Deutsche Gesellschaft für Chirurgie</source>
          <volume>397</volume>
          (
          <year>2012</year>
          )
          <fpage>755</fpage>
          -
          <lpage>761</lpage>
          . doi:
          <volume>10</volume>
          .1007/s00423-012-0914-z.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>K.</given-names>
            <surname>Riesen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Fischer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Bunke</surname>
          </string-name>
          ,
          <article-title>Improving approximate graph edit distance using genetic algorithms</article-title>
          ,
          <year>2014</year>
          , pp.
          <fpage>63</fpage>
          -
          <lpage>72</lpage>
          . doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>662</fpage>
          -44415-
          <issue>3</issue>
          _
          <fpage>7</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>C.</given-names>
            <surname>Corea</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Fellmann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Delfmann</surname>
          </string-name>
          ,
          <article-title>Ontology-based process modelling-will we live to see it?</article-title>
          , in: Conceptual Modeling: 40th International Conference, ER 2021,
          <string-name>
            <given-names>Virtual</given-names>
            <surname>Event</surname>
          </string-name>
          ,
          <source>October 18-21</source>
          ,
          <year>2021</year>
          , Proceedings 40, Springer,
          <year>2021</year>
          , pp.
          <fpage>36</fpage>
          -
          <lpage>46</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>M.</given-names>
            <surname>Somodevilla García</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. Vilariño</given-names>
            <surname>Ayala</surname>
          </string-name>
          ,
          <string-name>
            <surname>I. Pineda,</surname>
          </string-name>
          <article-title>An overview of ontology learning tasks</article-title>
          ,
          <source>Computación y Sistemas</source>
          <volume>22</volume>
          (
          <year>2018</year>
          )
          <fpage>137</fpage>
          -
          <lpage>146</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>G.</given-names>
            <surname>Pereira</surname>
          </string-name>
          , E. Santos,
          <string-name>
            <given-names>M.</given-names>
            <surname>Maceno</surname>
          </string-name>
          ,
          <article-title>Process mining project methodology in healthcare: a case study in a tertiary hospital</article-title>
          ,
          <source>Network Modeling Analysis in Health Informatics and Bioinformatics</source>
          <volume>9</volume>
          (
          <year>2020</year>
          ). doi:
          <volume>10</volume>
          .1007/s13721-020-00227-w.
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>P.</given-names>
            <surname>Naguine</surname>
          </string-name>
          ,
          <source>Subpopulation Process Comparison and Bottleneck Analysis: A Case Study of Frozen Shoulder, B.S. thesis</source>
          , University of Twente,
          <year>2022</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17] d_icd_diagnoses, https://mimic.mit.edu/docs/iv/modules/hosp/d_icd_diagnoses/,
          <year>2020</year>
          . [Online; accessed 17. May.
          <year>2022</year>
          ].
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>A.</given-names>
            <surname>Bogarín</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Cerezo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Romero</surname>
          </string-name>
          ,
          <article-title>Discovering learning processes using inductive miner: A case study with learning management systems (lmss</article-title>
          ),
          <source>Psicothema</source>
          <volume>30</volume>
          (
          <year>2018</year>
          )
          <fpage>322</fpage>
          -
          <lpage>329</lpage>
          . doi:
          <volume>10</volume>
          .7334/psicothema2018.
          <fpage>116</fpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>