=Paper=
{{Paper
|id=Vol-1419/paper0094
|storemode=property
|title=Diagrammatic Reasoning Meets Medical Risk Communication
|pdfUrl=https://ceur-ws.org/Vol-1419/paper0094.pdf
|volume=Vol-1419
|dblpUrl=https://dblp.org/rec/conf/eapcogsci/BrunsteinBM15
}}
==Diagrammatic Reasoning Meets Medical Risk Communication==
https://ceur-ws.org/Vol-1419/paper0094.pdf
Diagrammatic Reasoning Meets Medical Risk Communication
Angela Brunstein (angela@brunstein.net)
Department of S.A.P.E. - Psychology, American University in Cairo, AUC Ave, PO Box 74, New Cairo 11835, Egypt
Joerg Brunstein (joerg@brunstein.net)
Department of S.A.P.E. - Psychology, American University in Cairo, AUC Ave, PO Box 74, New Cairo 11835, Egypt
Ali Marzuk (AM12334@rcsi-mub.com)
Royal College of Surgeons in Ireland – Bahrain, PO Box 15503, Adliya, Bahrain
Abstract 2006). Also medical students perform better for displays on
Informed consent for medical procedures requires that
accumulation problems than non-medical students for some,
patients understand risks associated with diagnostic and but not all medical scenarios (Brunstein, Gonzalez, &
treatment options. Similar to performance for diagrammatic Kanter, 2010).
reasoning and system dynamics, patients, physicians and In this study, we aimed to combine the lessons learned
medical students are reported to perform poorly on from diagrammatic reasoning for understanding a treatment
understanding medical risk-related information. At the same scenario on ventricular fibrillation with medical students
time, different presentation formats seem to support different and non-medical undergraduates.
kind of conclusions across domains. In this research, we
investigated different formats of presenting risk information
For decision whether or not to undergo surgery to get an
related to a treatment scenario with 22 medical and 50 non- implantable cardioverter defibrillator (ICD) after surviving a
medical students. As expected, medical students performed heart attack, patients need to evaluate the risk of having
better than non-medical students for all versions of the another heart attack (i.e., severity of disease) and how likely
problem, while non-medical students could partially an ICD can save their life during that heart attack (i.e.,
compensate missing medical knowledge with displays that effectiveness of treatment).
reduce complexity and allow reducing cognitive load. This
In the medical literature, common measures for those
implies that it is possible to support patients’ decision
making, but also highlights the need to educate patients on values come from clinical trials with number of patients
potential risks and benefits. surviving versus dying in treatment, in this case ICD, versus
control groups, in this case heart medication only.
Keywords: medical decision making, diagrammatic
For physicians, critical values impacting treatment
reasoning, risk evaluation
decisions are absolute and relative risk reduction as
estimated proportion of patients surviving due to treatment
Applying Diagrammatic Reasoning to Medical and number needed to treat as estimates of the number of
Risk Communication patients who are exposed to potential side effects for saving
Diagrammatic reasoning and understanding complex system one patient’s life.
has been demonstrated to be difficult for several task For illustrating these measures, several formats of
domains (e.g., Cronin, Gonzalez, & Sterman, 2009). information presentation are used in the medical literature
Performance for these kinds of tasks is better if information and on patient information leaflets or websites. These
can be directly read out from diagrams compared to include tables, frequency arrays or bar graphs with numbers
inferring it (e.g., Larkin & Simon, 1987). The other way or proportions of patients dying versus surviving in different
around, specific presentation formats seem to support conditions. As for the miles per gallon example, patients
different kinds of conclusions. For example, for judging a perform better for understanding outcomes of clinical trials
car’s fuel efficiency, presentation of gallons per mile is when presented in terms of frequencies and not in terms of
more promising than presentation of miles per gallon probabilities (Gigerenzer et al., 2007). Unfortunately, the
(Larrick & Soll, 2008). decision whether or not undergo surgery to receive an ICD
For the medical domain, understanding information requires both, understanding the research results and
related to diagnostic and treatment choices is essential to estimating the probability of success for themselves. Also,
informed consent, evidence-based medicine and doctor- when presented with several treatment options, comparing
patient shared decision-making. As for understanding several pairs of 100 smileys each can become very
complex systems in general, there is evidence that both, confusing. Therefore, it is not evident which kind of display
physicians and patients have difficulties understanding risk might help patients best to make informed decisions.
related information (Mazur & Hickam, 1993; Windish, For each of these three domains, participants display a
Huot, & Green, 2007). Similar to the diagrammatic tendency for bias or errors. For the miles per gallon illusion
reasoning literature, patients’ and undergraduates’ (Larrik & Soll, 2008), participants tend to assume a linear
performance varies between different presentation formats instead curvilinear relationship and, therefore, they do not
(Gigerenzer et al., 2007; Shapira, Nattinger, & Mc Auliffe, appreciate the increase of efficiency by replacing least
572
efficient cars. For health information, participants tend to suited to support performance on proportion versus
neglect base rates or to confuse variables for calculations number of patients, but also vary in complexity and in
and, therefore, do not understand the value of screening or how intuitive they are to interpret, we had no specific
underestimate the risks associated with treatment. hypotheses which presentation should be best for this
Different kinds of visualization do not change the task.
concepts or required calculations, but they do change the
likelihood of error in participants’ responses. Miles per Method
gallons come with decimals instead naturals and with
constant base rates. This makes the comparison much easier. Participants
Frequencies instead conditional probabilities have the base
Fifty non-medical students (26 female, 24 male), mean age
rates already integrated and require one step less for
21 years (SD =3) participated for course credit in this study
processing. At the same time, naturals are more convenient
for calculations and comparisons than proportions with and were randomly assigned to one of five presentation
varying base rates. formats for a scenario on treating ventricular fibrillation
Applying these considerations to medical risk with ICD (10 per condition).
Forty medical students (19 female, 16 male, 18 dropped
communication, all presentation formats are isomorph and
before demographic information) mean age 21 years (SD =
display the same information. All of them allow
2) agreed to participate in this study. Due to high dropout,
comparison. At the same time, each of them invites for
only 22 participants (4-5 per condition) completed the
different strategies:
scenario. All participants who quitted before completing the
Tables explicitly display number of patients, invite to
calculate, but require numeracy and statistical literacy to get survey, did so before performing the risk information
the correct number and to understand that result. It is scenario at the consent page or at the demographics page.
challenging to visualize patients in different conditions from Data from these participants were excluded from analysis.
numbers. And patient numbers need to be converted to
Design
estimate personal odds of surviving or dying.
Bar graphs illustrate proportions of patients and are This study implemented a 5 (versions of visualization) x 2
therefore closest to personal risk. They invite to estimate, (medical knowledge) between participants design for
but they can be problematic given reported difficulties with answering 4 questions associated with severity of disease
probabilities (e.g., Gigerenzer et al., 2008). As for tables, it and effectiveness of treatment.
is difficult to imagine number of patients in different
conditions. Materials
Frequency arrays as favored by Gigerenzer and Participants were presented with scenario on 5-year survival
colleagues (2008) work with naturals and do not require of patients with ventricular fibrillation from a clinical trial
considering base rates. They illustrate numbers of patients (based on Moss et al., 1996). According the description, one
and are intuitive to understand. As for tables, number of hundred patients had an ICD implanted in addition to
patients need to be converted to estimate personal risk. traditional heart medication (treatment condition). The
Similarly, separate or integrated displays are associated remaining 100 patients received heart medication only
with different advantages and disadvantages. Separate (control condition). This was the topic of a group project for
displays can be directly mapped onto treatment conditions, first year medical students at RCSI. Therefore, medical
but require comparing and integrating displays to derive students were familiar with the topic, but not with the
conclusions on treatment effectiveness. Integrated displays specific data.
highlight the difference and take processing away from Data were presented in one of five presentation formats: a
participants, but make it more difficult to read the display. table (see Figure 1a) or frequency arrays for number of
This holds especially for the part of patients that have patients dying versus surviving in treatment and control
survived in treatment condition, while the corresponding conditions (see Figure 1d and 1e) or bar graphs on
number of patients had died in control condition. proportions patients surviving versus dying (see Figure 1b
1. Based on the diagrammatic reasoning and system and 1c). In addition for arrays and bar graphs, we either
dynamics literature, we expected that medical students presented a pair of individual displays of patients surviving
should outperform non-medical students due to their versus dying per condition or a combined display for both
greater knowledge associated with medical risk conditions.
communication and treatment options for that disease. Participants answered four questions on simplified
2. Given that background knowledge, we expected little versions of risk reduction and number needed to treat
impact of presentation formats on medical students’ (NNT), on severity of disease and estimated effectiveness of
performance for evaluating outcomes of clinical trials. treatment. These questions are relevant for patients, for
3. In contrast, we expected, differences in performance for example, with ventricular fibrillation, for making informed
different presentation formats for non-medical students. decisions on their treatment. The first question asked for a
Because tables, arrays and bar graphs are differently number, the remaining questions were true/false statements:
573
Figure 1: Displays for patients surviving versus dying in treatment condition (ICD) and control condition (heart
medication only) as (a) table, (b) integrated bar graph, (c) separate bar graphs, (d) integrated frequency array, and (e)
separate frequency arrays.
Table 1: Average performance (and standard deviations) for medical students (med; N = 22) and non-medical
students (non-med; N =40) for different presentation formats (maximum score was 4).
Table 1 Array 2 Arrays 1 Bar 2 Bars Total
med 3.0 (1.0) 3.2 (0.8) 3.7 (0.6) 3.0 (0) 3.3 (0.5) 3.2 (0.3)
non-med 1.6 (0.7) 2.5 (0.7) 2.0 (0.7) 2.1 (0.9) 1.7 (0.5) 2.0 (0.4)
The first question asked for the difference of survivors effectiveness and as a component of absolute and relative
between both conditions as a proxy for treatment risk reduction. Correct answer is 20.
574
The second question asked whether more patients die than students performed better for evaluating treatment options
survive as a proxy for the severity of the disease or for the than non-medical students.
necessity to treat. Correct answer is “false”. However, not all medical students scored 4 of 4 questions
The third question asked whether more patients die with correctly. As for diagrammatic reasoning literature, in our
implant than without. The correct answer is “false”. study different formats were differently supportive for
These 3 questions are needed to calculate the number evaluating medical risk information in non-medical
needed to treat to save one patient’s life (NNT). For this students. Potentially, knowledge of the task domain and
scenario, 1 of 5 patients survives due to treatment and the presentation formats that match the required task can help
remaining 4 of 5 suffer from potential side effects of with understanding risk information. For our task that
treatment without benefit. One of five will die in both required understanding of research results in terms of
conditions and 3 of 5 will live in both conditions. The frequencies and estimating the patient’s own chances of
correct answer is “true”. success in terms of probability, there is not one single
Because all four questions ask for aspects that are relevant format of diagram that serves all aspects of the task best.
for the treatment decision and because of the small number Therefore, it seems that displays that are intuitive (arrays of
of participants among medical students, we report patient numbers) and displays that allow reducing cognitive
accumulated scores below. For patients, the next question load (a combined bar graph on patient proportions) seem to
would be whether they want to have an ICD implanted. foster non-medical students’ performance. In contrast,
medical students tended to profit from separate displays that
Procedure can serve as external memory when calculating the
The study was conducted as an online experiment on statistical values. This means we should not leave it to the
surveymonkey.com. After providing informed consent and doctor to choose the presentation format for the patient
demographic information, participants answered the four because what is best for the expert does not match what is
questions on the treatment scenario. Total time was about 10 best for the layperson.
to 15 min. For our study, dropout rates for medical students (18 of
The IRB/ethics boards of the American University in 40 dropped) were very different from non-medical students’
Cairo and the Royal College of Surgeons in Ireland – dropout rate. If eliminating the corresponding proportion of
Bahrain had approved this research. low-performing non-medical students, the effect of
presentation format becomes weaker, but does not disappear
Results completely. In addition, even the best performing group of
non-medical students performs worse than any of the groups
As expected, medical students performed better than non- of medical students.
medical students, F (1, 62) = 47.12, p < .001, η2 = .43. This indicates that presentation format can promote
Given the low number of participants for the group of performance for medical risk evaluation, but cannot replace
medical students, we analyzed the effect of versions on domain knowledge for understanding implications of
participants’ performance for the four questions separately illustrated data. Therefore, when supporting patients’
for groups. For medical students, presentation format had no treatment choices, we will need to educate them on potential
impact on performance, F (4, 17) = 0.64, η2 = .13. In risks and benefits in addition to provide intuitive displays.
tendency, medical students performed better for separate
displays than for integrated displays (Bonferoni, all p’s >
Acknowledgements
.10). However, presentation format impacted performance
of non-medical students, F (4, 45) = 2.96, p < .05, η2 = .21. This research was made possible by the Royal College in
Non-medical students performed better for the array than 2 Ireland – Bahrain’s generous support. The data collection on
bars or table (p’s < .01) and in tendency better for integrated medical students at RCSI Bahrain was conducted when Dr.
displays than for separate displays (p’s > .10, see Table 1). Angela Brunstein was a Senior Lecturer in Psychology at
This indicates that displays that reduce complexity and have RCSI Bahrain. The contents of this report are solely the
potential for visual imagery better support non-medical responsibility of the authors and do not necessarily represent
students’ performance for evaluating medical risk related the official views of RCSI Bahrain.
information.
Discussion References
Understanding medical risk communication is essential for Brunstein, A, Gonzalez, C, & Kanter, S. (2010). Effects of
informed consent for treatment choices, evidence-based domain experience in the stock-flow failure. System
medicine and physician-patient shared decision-making. At Dynamics Review, 26, 347–354.
the same time, literature on medical decision-making Cronin, M., Gonzalez, C., & Sterman, J.D. (2009). Why
indicates that patients’ performance is far from perfect. This don’t well-educated adults understand accumulation? A
is matched by reported difficulties for diagrammatic challenge to researchers, educators, and citizens.
reasoning and system dynamics. For our study, medical Organizational Behavior and Human Decision Processes,
108, 116–130.
575
Gigerenzer, G., Gaissmaier, W., Kurz-Milcke, E., Schwartz,
L.M., & Woloshin, S. (2007). Helping doctors and
patients make sense of health statistics. Psychological
Science in the Public Interest, 8, 53-96.
Larkin, J.H., & Simon, H.A. (1987).Why a diagram is
(sometimes) worth ten thousand words. Cognitive
Science, 11, 65-99.
Larrik, R.P., & Soll, J.B. (2008). The MPG illusion.
Science, 320, 1593-94.
Mazur, D.J, & Hickam, D.H. (1993). Patients’ and
physicians’ interpretations of graphic data displays.
Medical Decision Making, 13, 59–63.
Moss, A.J., Jackson Hall, W., Cannon, D.S., Daubert, J.P.,
Higgins, S.L., et al. (1996). Improved survival with an
implanted defibrillator in patients with coronary disease at
high risk for ventricular arrhythmia. New England
Journal of Medicine, 335, 1933-1940.
Shapira, M.M., Nattinger, A.B., McAuliffe, T.L. (2006).
The influence of graphical format on breast cancer risk
communication. Journal of Health Communication, 11,
569-82.
Windish, D.M., Huot, S.J., & Green, M.L. (2007). Medicine
residents’ understanding of the biostatistics and results in
the medical literature. JAMA, 298, 1010–1022.
576