=Paper= {{Paper |id=None |storemode=property |title=Design and Prototypical Development of a Web Based Decision Support System for Early Detection of Sepsis in Hematology |pdfUrl=https://ceur-ws.org/Vol-727/eics4med13.pdf |volume=Vol-727 |dblpUrl=https://dblp.org/rec/conf/eics/WichtMK11 }} ==Design and Prototypical Development of a Web Based Decision Support System for Early Detection of Sepsis in Hematology== https://ceur-ws.org/Vol-727/eics4med13.pdf
                  Design and Prototypical Development of a
                  Web Based Decision Support System for
                   Early Detection of Sepsis in Hematology

          Andreas Wicht                             Gerrit Meixner                                    Ulrike Klein
 Heidelberg University Hospital,             German Research Center for                      Heidelberg University Hospital,
  Institute of Medical Biometry              Artificial Intelligence (DFKI)                   Department of Hematology
          and Informatics                      Trippstadter Strasse 122                              and Oncology
    Im Neuenheimer Feld 305                 Kaiserslautern, 67663 Germany                      Im Neuenheimer Feld 410
   Heidelberg, 69120 Germany                   Gerrit.Meixner@dfki.de                         Heidelberg, 69120 Germany
      Andreas.Wicht@med.                                                                          Ulrike.Klein2@med.
         uni-heidelberg.de                                                                         uni-heidelberg.de

ABSTRACT                                                             decades, the practical impact remains low for several
Physicians do not always make optimal decisions.                     reasons, e.g. [3]:
Computer based clinical support systems are intended to                - Systems failed to cover an entire medical domain
provide clinicians with decision aids, but their practical             - Poor practicability and integration into the clinical
impact remains low. We introduce a software architecture                   workflow
which might overcome key barriers and present the                      - Poor availability of digital patient data
prototypical implementation of a web based knowledge                   - Poor acceptance
module for early detection a life-threatening medical                Within the ESGOAB1 project, which will be described in
condition, sepsis.                                                   more detail in the next section, we are trying to overcome
                                                                     the weaknesses mentioned above. In this paper we will
Keywords
                                                                     briefly describe the ESGOAB software architecture, which
Clinical decision support, Knowledge-based systems in
                                                                     provides an electronic health record (EHR) and is designed
medicine, hematology, sepsis, fever, web-based application,
                                                                     to provide the base to interact with knowledge modules. We
knowledge maintenance, ESGOAB.
                                                                     focused on two specific clinical challenges: supporting the
INTRODUCTION                                                         physicians’ order entry process (CPOE) and supporting
The information overload physicians are confronted every             early detection of sepsis (a serious and life-threatening
day with makes it impossible for them to keep up with all            medical condition) on patients with hematological
the information and knowledge that would be potentially              underlying diseases. In this paper we will describe the
useful in making optimal clinical judgments. Empirical               second challenge. We introduce our conceptual design of
studies have shown that physicians do not always make                the sepsis knowledge module and present the currently
optimal decisions [17] [6]. Clinical decisions are often             implemented web-based prototype.
made under time pressure, without having all information
                                                                     Project Background
and knowledge needed in the right place at the right time.
                                                                     The ESGOAB project is a 2-year public funded joined
Computer-assisted clinical decision support systems
                                                                     research project between two scientific partners
(CDSS) are intended to provide “clinicians, patients or
                                                                     (Heidelberg University Hospital, DFKI) and two industrial
individuals with knowledge and person-specific or
                                                                     partners (COPRA System GmbH, Dosing GmbH).
population information, intelligently filtered or presented
at appropriate times, to foster better health processes,             A survey at the Department of Hematology and Oncology
better individual patient care, and better population health.        of the Heidelberg University Hospital has shown a range of
CDS interventions include alerts, reminders, and order sets          problems concerning various applications of Information
[…]” [11].                                                           and Communications technologies (ICT) within the hospital
Although research has been done in the field of CDSS for             (response rate = 70.5 %, 36 of 51 of the medical personnel)


                                                                     1
Copyright © 2011 for the individual papers by the papers'                ESGOAB = „Entwicklung einer Softwareumgebung zur Generierung
authors. Copying permitted only for private and academic                 von organisationsspezifischen Anwendungen zum Behandlungs-
                                                                         prozessmanagement“; english: Development of a Software Environment
purposes. This volume is published and copyrighted by                    for Generation of Organization-Specific Applications for Treatment
the editors of EICS4Med 2011.                                            Process Management

                                                                69
[20]. The study revealed problems concerning e.g., time                          stress, and optimization of clinical decisions, which
consuming searches, redundant data entries, use of various                       altogether may improve quality of care.
software applications to perform the various tasks, different
                                                                                 Related work
user interfaces, and only marginal decision support.
                                                                                 Research in the field of medical knowledge-based systems
However we found a promising openness towards CDSS
                                                                                 has been done since the early 1960s. Many different
[20]. In general the staff was open-minded towards new
                                                                                 research approaches have been explored, but yet the degree
Information Technology (IT) systems (88 % indicated to be
                                                                                 of impact of clinical decision support systems still remains
“rather open-minded” or “open-minded”), concerning
                                                                                 low [5]. Only few knowledge-based systems are widely
CDSS, the potential benefit was assessed by the majority
                                                                                 used day-to-day, such as automated electrocardiogram
(72 %) as “rather high” or “high”, as well as prospects of
                                                                                 (ECG) interpretation [18]. However, systems utilizing a
success (53 %). At least 47 % rated reliability as well as
                                                                                 broad variety of individual patient data had to fail due to
acceptance as “rather high” or “high”.
                                                                                 poor availability of digital data. Providing an EHR
The ESGOAB project aims at encapsulating various data                            eliminates this obstacle [3].
sources like e.g., hospital information system (HIS),
                                                                                 Research in the field of monitoring and analyzing vital
laboratory data or drug information systems (see Figure 1)
                                                                                 signs in Intensive Care Units (ICU) for early warning of
into one integrated software system which provides one
                                                                                 patient deterioration or sepsis were done by [15] and [12].
consistent user interface. The second main aim is the
                                                                                 However, including microbiological findings as well as the
development and integration of knowledge bases into the
                                                                                 accurate handling of the specifics of the hematological
ESGOAB system.
                                                                                 patients remained unconsidered.
The ESGOAB system is based on a 3-layer software
                                                                                 Clinical Background
architecture concept. A data collector (layer 1) is
                                                                                 The treatment of patients with hematological diseases has
responsible to capture and gather data from various existing
                                                                                 advanced enormously in the past years. Nevertheless such
sub systems which are illustrated below the three layers.
                                                                                 infections pose a serious life-threatening risk for these
Adapters for each of the sub systems are transforming data
                                                                                 immunocompromised patients. Beside various other clinical
into defined structures. The knowledge carrier (layer 2)
                                                                                 and laboratory parameters, fever is an essential factor,
analyzes incoming data from the data collector layer or
                                                                                 which indicates a manifest or beginning infection.
handles requests triggered by user interactions. The
                                                                                 Therefore a refined assessment of the body temperature is
knowledge carrier consists of various knowledge bases
                                                                                 needed. The responsible physician has to distinguish
(e.g., about drug information) which are connected
                                                                                 between innocent fever as immunologic reaction, fever of
following a modular concept. If there are e.g., new blood
                                                                                 unknown origin, fever caused by bacteremia or the onset of
values these will be evaluated by the knowledge carrier and
                                                                                 a severe sepsis. Hereby assists the combination of lab-
according hints or warnings will be immediately displayed
                                                                                 values, microbiological findings and vital signs. The
by the visualizer (layer 3) following a defined alerting
                                                                                 emphasis and valuation of the combination of single-values
concept, taking into account the importance, severity etc. of
                                                                                 and the experience of the doctor partly determine the
the alert [4].
                                                                                 treatment course and the outcome of the patient. Clinical
                                                                                 studies demonstrated that the survival probability of
             Layer 3
         Visualizer                       Patient X                              patients with sepsis depends most essentially on the period
                                          ….              Dynamic Desktop

                                          …
                                                                                 of time between diagnosis and start of effective antibiotic
                                                                                 treatment [7]. Sepsis is not only a problem of hematological
             Layer 2
                                                                                 patients. It’s rather a challenge for the population. Severe
                                                          Workflow Model
 Knowledge Carrier                                                               sepsis is considered to be the most common cause of death
                                                          Decision Model         in non-coronary critical care units. Approximately 150.000
                                                                                 persons die annually in Europe and more than 200.000 in
             Layer 1                                                             the United States [1]. The problematic nature of a timely
    Data Collector                                        Meta Data Model        recognition is not that data is missing, but it is detached
                                                          Adapters
                                                                                 from one another, generated at various places and different
                                                                                 times. The responsible physician has to link the separated
                                                                                 information for the plurality of patients. Ward rounds,
                       HIS   Lab   PACS               …   Sub Systems
                                                                                 printouts with lab values, calls from the microbiologist and
                                                                                 signs from monitors serve as instruments for this
Figure 1: 3-layer architecture of the ESGOAB system                              connection. This work has to be done by the medical
At the end of the ESGOAB project we expect an increase of                        personnel even in the future, but the model we present in
efficiency of tasks and processes at the Department of                           this article offers the convincing advantage of automated
Hematology and Oncology at the Heidelberg University                             joining of relevant data and usable presentation, resulting in
Clinic (due to time-savings), a reduction of performance                         efficient and faster decisions.


                                                                            70
APPROACH                                                                   the flexibility to use the same implementation (objects) in
Conceptual Requirements                                                    two kinds of settings:
When deploying and operating knowledge-based systems, a                     1. Interactive data retrieval with the user (module
weak point is often poor practicability, in particular in terms                execution)
of maintenance. Either the knowledge model is
implemented statically, there is no way experts of the                      2. Running in background through a web service
certain domain (in the present case hematologists) can                         retrieving data from the EHR
modify the model on their own. This leads to the fact that                 Knowledge Model
each adaptation has to be done by a software engineer. Or                  Sepsis accompanies with several symptoms, such as fever,
the user interfaces do not provide intuitive means to modify               increased heart rate, low blood pressure etc. Further
the knowledge base; thus users have to be instructed and the               important parameters are signs of infections such as specific
system becomes error-prone. Between designing and using                    blood values and microbiological findings. The sequence of
knowledge-based systems, a long-lasting cyclic process of                  appearance and the severity of these manifestations differ
modeling, testing, adapting, and retesting of the core engine              from patient to patient. We have to deal with fuzzy and
has to be passed through. While operating knowledge-based                  uncertain information. However, some signs are more
systems the focus shifts towards maintenance issues.                       important than others and certain value ranges are
Maintaining the knowledge within the system is critical to                 supporting sepsis more than other diagnoses. So the idea
successful delivery of decision support [3]. In this context               was to design a decision model, which balances between a
practicability plays an important role. Being aware of this                set of differential diagnoses and specifies the one which can
issue, easy knowledge maintenance was an important goal.                   be explained best by the observed findings. The set
The clinical expert should be able to modify the underlying                covering model is a potentially useful approach, which was
knowledge model without extensive training. It has to be                   introduced by [13], as well as the more abstract view on
simple and intuitive to use. Furthermore it should be                      multiple diagnose problems by [9]. Our approach is based
possible to test the constructed or adapted model right                    on the set covering model, extended by the possibility to
away. Beyond that, the knowledge model should be generic,                  define parameters which contradict certain diagnoses since
so it might be useful for other diagnostic problems.                       we experienced a further need for accuracy.
Knowledge Engineering Process                                              Two sub modules based on a rule engine were required to
Knowledge Engineering is the systematic approach for the                   handle two specific problems:
development of knowledge based systems. The process may                     1. Interpretation of microbiological findings: Presence of
be divided into two main phases (Figure 2): knowledge                          an infection or suspected contamination?
acquisition and knowledge operationalization [14]. It
should be noted, that typically the process of acquisition                  2. Interpretation of white blood cell count (leukocytes):
and operationalization is not a linear process but rather a                    May we take this parameter into account?
cyclic process characterized by continuous, iterative                      Implementation
refinement.                                                                Initially, the basic idea was demonstrated using a prototype
                                                                           realized in Microsoft Excel (Figure 3). Taking advantage of
  Knowledge Acquisition                Knowledge Operationalization        quick implementation possibilities, this prototype helped us
                                                                           to refine our knowledge model.
Capturing                                       Design
                          Knowledge Model
Structuring                                     Implementation
Formalization
                          Information Model

Figure 2: The Knowledge Engineering Process
Knowledge acquisition is the process of capturing,
structuring and formalizing knowledge. The result of the
acquisition phase is a knowledge model and an information
model which both serve as a base for the system design of
an implementation [5]. Sources of knowledge are typically                  Figure 3: Prototype developed with Microsoft Excel
domain experts, medical literature and patient data                        (result presentation in the upper left corner, value
repositories. Our approach was based on expert opinion and                 selection in the mid-area)
literature research.                                                       The decision to proceed with the development of a web-
Information Model                                                          based application was made due to the following main
A specification of the kinds of information that were                      reasons:
required was created, including the data format and the                     -   Easy Accessibility from all clients via the clinic’s
taxonomy. The resulting information model – which will be                       intranet; no need to install software on clients
implemented as an object-oriented data model – provides us
                                                                            -   Quick deployment of new versions

                                                                      71
We used a XAMPP [2] installation on Windows including
an Apache 2.2.14 web server and a MySQL database
system. We used the scripting language PHP and the Ajax
toolkit xajax [19].
The application is based on the Model-View-Controller
(MVC) design pattern; the PHP code architecture follows
an object-oriented approach. The application is
implemented using the open source relational database
management system MySQL, with use of the InnoDB
storage engine. An initial database model was designed
using the database-modeling tool MySQL Workbench [10].
The model was refined iteratively during the
implementation of the application.                                 Figure 5: Screenshot of a weight table (columns:
                                                                   diagnoses, rows: parameters and values)
WEB-BASED PROTOTYPE
                                                                   The process of creating a new model facilitates the
Application Structure
                                                                   following steps:
The application is composed of three modules: Sepmod,
Leukomod and Mibimod (Figure 4). Sepmod represents the              1. Add diagnoses
generic core model, which implements the weight model as            2. Add parameters
specified before and interacts with the sub modules
                                                                    3. Add value ranges for each parameter
Leukomod and Mibimod. Leukomod is based on a rule
engine, which is responsible for the white blood cells’             4. Create weight relations between value and diagnosis
assessment. Mibimod is also based on a rule engine and              5. Define Equivalencesets
performs the assessment of the microbiologic findings.
                                                                    6. Define Minimalsets
                                                                   These steps are described in more detail below.
                                                                   Step 1: Firstly we need to add diagnoses, by providing its
                                                                   name and optionally a description. For each diagnose
                                                                   added, the table will be expanded by one column.
                                                                   Example definition:

                                                                     Name:     SIRS
                                                                     Range:    Systemic Inflammatory Response Syndrome

                                                                   Step 2: In the second step, symptoms or “parameter” can be
                                                                   defined, specifying its name, type, unit, value range,
                                                                   validity, importance. The type tells us, which kind of data
                                                                   we deal with, e.g. integer, float or special medical
Figure 4: Structure of the application                             classifications like the anatomic therapeutic classification
The web-front-end provides the following two main                  (ATC) code. The value range defines the valid value range
sections:                                                          for the parameter and is used to perform plausibility checks.
 -   Maintenance: Provides editors for each module for             Regarding the process of diagnosis, an important and
     construction and modification of the knowledge base.          relevant issue is always the time context. How long can I
                                                                   rely on a measured value? The answer depends on each
 -   Module-Execution: Interactive tool, which requests for        parameter. In the present model we therefore define for
     data or selection of options by the user and presents         each parameter a time frame (validity), entering a numerical
     the results.                                                  value for the absolute time (minutes, hours or days) within
The interactive tool is basically designed for testing             this parameter remains valid or in other words we can
purposes. The impact of adaptions of the knowledge base            assume that the measured value may be used for the
can be explored right away.                                        diagnostic assessment. If a parameter exceeds the defined
Knowledge Maintenance Section                                      time frame, it will be ignored and treated, as it would be not
                                                                   available. Alternatively we can define a relative time frame
Sepmod
In this section the user can create a new and edit existing        such like “Valid until next measurement”. Some parameters
knowledge models. Created models can be loaded, saved,             may have a more significant importance than other. Thus
and deleted. The core element is the weight table                  for each parameter the importance can be defined, choosing
(Figure 5).                                                        from given weights “very important”, “fairly important”,
                                                                   “important”.

                                                              72
Example definition:                                                  least a certain set of parameters is available. In the section
                                                                     minimalset the clinical experts can define, which
  Name:          bodytemperature                                     parameters they consider as being essential for performing a
  Type:          float                                               profound assessment. It is possible to define more then one
  Unit:          °C                                                  set.
  Range:         30.0 to 43.0                                        Leukomod
  Validity:      6 hours                                             Leukomod is based on a rule engine. The rules were
  Importance:    very important                                      defined by clinical experts. In the current version they are
                                                                     implemented statically and can be activated or deactivated
Step 3: In the next step, for each parameter, several values         through the web-front-end in the section leukomod/rules.
have to be defined. The values are specified by its name             Further we can adapt various parameters (like thresholds) of
and its value range.                                                 the rules. For the assessment, if the white blood cells may
                                                                     be included, the underlying disease is of high importance. A
Example definition:
                                                                     dynamic list of diagnosis codes (ICD, International
  Name:          fever                                               Statistical Classification of Diseases and Related Health
                                                                     Problems) can be maintained in the section diagnoses.
  Range:         38.0 to 43.0
                                                                     Mibimod
Step 4: Adding diagnoses, parameter and its values results           Mibimod is based on a rule engine. A dynamic list of germs
in a count(diagnoses) x count(values) matrix. For each pair          which support the suspicion of a contamination can be
of a value and a diagnosis we can now define a weight                maintained in this section. The rules of this module are
relation between a specific value and a diagnosis by                 currently implemented statically.
clicking on the corresponding button. A weight relation is
                                                                     Module Execution Section
the symbol for the strength a specific value supports a
                                                                     For testing purposes the web-front-end provides a section to
diagnosis. The currently implemented model supports five
                                                                     run the modules. The current version has one section to run
different weights, ranging from H0 (value does not support
                                                                     each of the modules separately and one, which encapsulates
the diagnosis or even contradicts) to H4 (value strongly
                                                                     all three of them. These execution modules are
supports the diagnosis or is even essential) depending on
                                                                     implemented as interactive tools, requesting each parameter
the currently used weight model which can be defined in a
                                                                     step-by-step.
separate section. For each weight symbol the value can be
selected through a slide control.                                    For each input field a check for plausibility is performed
                                                                     while entering data, considering two main issues
Step 5: In the section called equivalencesets we define sets
of previously defined parameters, which are clinically                -   Valid characters (depending on the data type)
equivalent (Figure 6). In other words all of these parameters         -   Valid value range (as defined in the model).
support a specific context diagnosis, such as low blood
                                                                     To keep track of entered values, we show breadcrumbs
pressure.
                                                                     horizontally across the top of the input form (Figure 7).




                                                                     Figure 7: Entering values using the interactive mode
Figure 6: Screenshot of the equivalenceset dialog                    The user can easily go back to previously entered values
                                                                     and change them if necessary. Each breadcrumb item shows
In this case we would define that low blood pressure exists
                                                                     the name of the parameters as well as the entered value and
if at least one of these blood pressure parameters takes a
                                                                     unit. Once all values are entered, the system performs the
specific value (and a corresponding high weight) or in this
                                                                     assessment and presents the results, giving explanations and
special case we also assume low blood pressure if
                                                                     a visualization of the results (Figure 8). Presenting results in
vasopressors (substances which result in an increase in
                                                                     an appropriate way means finding a balance between (1)
blood pressure) are given what we can specify through a list
                                                                     simple, clear and aggregated representations, supporting
of ATC codes.
                                                                     e.g. quick task completion and (2) comprehensive and
Step 6: A decision model might be sophisticated but its              detailed representations, which are needed for making
accuracy highly depends on the available parameters. So it           comprehensible decisions.
only makes sense to perform a diagnostic assessment if at

                                                                73
                                                                           Elsevier Academic Press, Amsterdam; Boston, 2007.
                                                                      6.   Kohn, L.T. (Ed.). To err is human. National Academy
                                                                           Press, Washington, DC, 2000.
                                                                      7.   Kumar, A., Roberts, D., Wood, K.E., et al. Duration of
                                                                           hypotension before initiation of effective antimicrobial
                                                                           therapy is the critical determinant of survival in human
Figure 8: Result presentation (diagram and textual                         septic shock. Critical care medicine 34, 6 (2006),
explanation)                                                               1589-1596.
DISCUSSION & FUTURE WORK                                              8.   Lo, H.G., Matheny, M.E., Seger, D.L., Bates, D.W.,
We identified two important and promising factors that will                and Gandhi, T.K. Impact of non-interruptive
help overcome key barriers limiting more widespread use of                 medication laboratory monitoring alerts in ambulatory
CDSS: Firstly we did not experience the “physician                         care. Journal of the American Medical Informatics
resistance” using decision support systems; they are rather                Association 16, 1 (2009), 66–71.
convinced of their usefulness as described in the
introduction. Secondly we can take advantage of the                   9.   Miller, R.A. A Heuristic Approach to the Multiple
software infrastructure we presented which has the potential               Diagnoses Problem. Proceedings of the 6th
to improve clinical processes and to integrate and interact                Conference on Artificial Intelligence in Medicine in
with sharable knowledge modules conceived to support the                   Europe, Springer-Verlag (1997), 187–198.
physicians in their decisions. We also introduced a web               10. MySQL Workbench visual tool. Available                 at
based prototype of a knowledge module for early detection                 http://www.mysql.com/products/workbench/.
of sepsis. We believe to contribute important elements to             11. Osheroff, J.A., Teich, J.M., Middleton, B., Steen, E.B.,
lift computer-aided decision support into widespread                      Wright, A., and Detmer, D.E. A roadmap for national
practice.                                                                 action on clinical decision support. Journal of the
However, our approach still needs to be refined and                       American medical informatics association 14, 2
evaluated. It has to be shown that our knowledge modules                  (2007), 141-145.
are able to provide accurate and traceable support,                   12. Pilz, U., and Engelmann, L. Früherkennung von
characterized by high sensitivity. A test concept as well as              septischen Krankheitsverläufen durch Einsatz eines
test scenarios will be defined. Further alerting concepts will            Expertensystems. Intensivmedizin und Notfallmedizin
be tested in practice since empirical studies have shown that             43, 7 (2006), 575-579.
too many generated alerts could lead to “alert fatigue”,
whereas “non-interruptive” alerts only have low impact and            13. Reggia, J.A., Nau, D.S., and Wang, P.Y. Diagnostic
are not effective [8].                                                    Expert Systems Based on a Set Covering Model.
                                                                          International Journal of Man-Machine Studies,
ACKNOWLEDGMENTS                                                           (1983), 437-460.
This work was funded by the German Federal Ministry of
                                                                      14. Spreckelsen, C., and Spitzer, K. Wissensbasen und
Education      and    Research     under grant   number
                                                                          Expertensysteme in der Medizin. Vieweg + Teubner,
01 IS 09027 C. The responsibility for the content of this
                                                                          Wiesbaden, 2008.
publication lies with the authors.
                                                                      15. Tarassenko, L., Hann, A., and Young, D. Integrated
REFERENCES
                                                                          monitoring and analysis for early warning of patient
1.   Angus, D.C., Linde-Zwirble, W.T., Lidicker, J.,
                                                                          deterioration. British Journal of Anaesthesia 97, 1
     Clermont, G., Carcillo, J., and Pinsky, M.R.
                                                                          (2006), 64-68.
     Epidemiology of severe sepsis in the United States:
     analysis of incidence, outcome, and associated costs of          17. Warner, H.R., Sorenson, D.K., and Bouhaddou, O.
     care. Critical care medicine 29, 7 (2001), 1303-1310.                Knowledge engineering in health informatics.
                                                                          Springer, New York; Berlin; Heidelberg, 1997.
2.   Apache Friends. XAMPP distribution. Available at
     http://www.apachefriends.org/xampp.html.                         18. Willems, J.L., Abreu-Lima, C., Arnaud, P., et al.
                                                                          Evaluation of ECG interpretation results obtained by
3.   Bates, D.W., Kuperman, G.J., Wang, S., et al. Ten
                                                                          computer and cardiologists. Methods of Information in
     commandments for effective clinical decision support:
                                                                          Medicine 29, 4 (1990), 308-316.
     making the practice of evidence-based medicine a
     reality. Journal of the American Medical Informatics             19. xajax Project. xajax framework.           Available    at
     Association 10, 6 (2003), 523–530.                                   http://www.xajax-project.org/.
4.   Durieux, P. Electronic Medical Alerts – So Simple, So            20. Zuehlke, D., Meixner, G., and Klein, U. Smart Medical
     Complex. New England Journal of Medicine 352, 10                     Software Systems For Dummies? The Case For A
     (2005), 1034-1036.                                                   User-Centered Systems Design. Proceedings of the 3rd
                                                                          International Conference on Health Informatics 2010
5.   Greenes, R.A. (Ed.). Clinical decision support.
                                                                          (Valencia ESP, 2010), 350-354.

                                                                 74