=Paper=
{{Paper
|id=Vol-2001/paper5
|storemode=property
|title=A formative evaluation of an eHealth service for patients with COPD
|pdfUrl=https://ceur-ws.org/Vol-2001/paper5.pdf
|volume=Vol-2001
|authors=Anita Das,Jarl Reitan,Silje Bøthun,Yngve Dahl
|dblpUrl=https://dblp.org/rec/conf/pahi/DasRBD17
}}
==A formative evaluation of an eHealth service for patients with COPD==
https://ceur-ws.org/Vol-2001/paper5.pdf
A Formative Evaluation of an eHealth Service for Patients with
COPD
Anita Das, Silje Bøthun, Jarl Reitan
SINTEF Technology and Society, Trondheim, Norway
Abstract. The number of people suffering from chronic conditions such as
COPD is increasing globally because of demographic changes worldwide.
People suffering from severe stages of COPD commonly experience
exacerbations leading to hospital admissions. Telehealth solutions can be
used as a means for improved healthcare delivery to this patient group with a
possible impact on preventing COPD exacerbations and hospital admissions.
We conducted a pilot study where patients with COPD were offered an
eHealth service from the hospital for three months. The eHealth service
involved the patients reporting daily on fixed health parameters, which they
sent to the clinic through an eHealth tablet computer application. Nurses at
the clinic received the patients’ reports on a computer-based system and
provided follow-up based on these reports. We here report from a formative
evaluation of the pilot study, where data collection was done through
workshops with patients and healthcare providers, and where participatory
design approaches enabled active engagement and involvement of the users
in evaluating the service and proposing future adjustments of the solution.
1 Introduction
Chronic Obstructive Pulmonary Disease (COPD) is one of the leading causes of
mortality and morbidity worldwide and is the fourth leading cause of death [1].
People suffering from severe stages of COPD commonly experience
exacerbations leading to hospital admission; for some, this occurs several times a
year [2,3]. Such hospital admissions are associated with deteriorated health status
of the individual and involve considerable health care and societal costs [1,4].
One main objective in COPD management is to improve or preserve the patients’
health status. Increasingly, patients with COPD are being managed at home to
reduce health related costs while trying to increase their comfort [5,6]. The use of
Internet based tools and eHealth solutions have increased in the last decade and
Copyright © 2017 by the paper's authors. Copying permitted for private and academic
purposes. In: H. Gilstad, S. Khodambashi, J. Bjerkan (eds.):
Proceedings of the 4th European Workshop on Practical Aspects of Health Informatics (PAHI
2017), Levanger, Norway, MAY-2017, published at http://ceur-ws.org
people use these to search for information, manage their own health and illness,
and communicate with peers and healthcare providers [7]. As the populations’
competence in using such tools increases, people suffering from chronic diseases
use these as important resources in their daily lives. It is expected that healthcare
services will increasingly implement and adapt to using eHealth solutions to
facilitate patients at a distance and provide quality care services in a cost-efficient
manner.
Participatory design is an approach where various stakeholders are actively involved
in the design process to identify, create and ensure that new solutions are according to
their needs. Such approaches are randomly used in the design of eHealth services,
resulting in solutions that fail to engage the end-users, and that lead to low adaptation
and use. Health care is a complex domain with unforeseen incidents, changing use
contexts and multiple stakeholders with specific needs, interests, roles, expectations
and power. These factors are important to take into consideration when designers
design and develop solutions for this specific domain.
The HelsaMi innovation project comprised of the development of an eHealth service
for integrated care at home for patients with COPD. The overall project aim was to
improve the efficacy and quality of the health services while improving the patients’
quality of life. In the project, a new health-care service was proposed, involving
eHealth follow-up of patients with COPD. We here report from a formative evaluation
as part of the first pilot study in the project, where the eHealth service was
implemented in a specialist healthcare clinic and offered to patients. The objective of
the formative evaluation was to improve the project’s design and effects, to
understand which parameters influenced use and usability, and to understand internal
and external factors that might influence the project.
2 Methods
The pilot study was carried out in spring 2014 and lasted for three months. The
evaluation comprised of workshops with the nurses and patients that had
participated in the pilot study. The Norwegian Social Science Data Services
approved the study and all participants provided written informed consent at
participation.
2.1 The pilot study: ‘HelsaMi’ (MyHealth)
The pilot study involved five patients (three men and two women) enrolled to
participate and receive the eHealth service from the hospital for a period of three
months. Two nurses who were specialized in respiratory nursing at the clinic were
responsible for handling the received patient reports and providing follow-up to the
patients. In addition, respiratory physicians were available for decision support.
The patients were included in the study when they were discharged from
hospital after an incidence of COPD exacerbation. The eHealth service involved the
patients sending daily health reports through a tablet computer application (app)
to the clinic. The eHealth app included predefined questions about health-related
parameters (e.g. respiration, expectorate, use of medications etc.) which the
patients were to report on. The patients were instructed to send their health reports
every working day (Monday– Friday) before noon, a timing that was set to ensure
that the clinic could handle and provide follow-up to the patients during office
hours. The clinic was closed during weekends; this also applied to the eHealth
service.
The nurses at the clinic had the responsibility of receiving the patients’ health
reports, evaluating them and providing further follow-up to the patients if
required. To access the patient reports, the nurses logged into a Web-based portal,
which they accessed through their stationary computer at the clinic. Since the
nurses were not merely stationed in the office during their working hours, they
could also receive patients’ reports on their mobile phone, for immediate
evaluation.
2.2 Participants and data collection
Data collection involved two separate workshops with the involved stakeholders,
in total five participants: one with the nurses (n=2) and the other with patients
(n=3). All patients (n=5) in the pilot study were requested to participate, but two
were prevented from attending. The workshops lasted 60–120 minutes, were
audio-recorded and parts of the recordings were transcribed verbatim.
Participatory design (PD) activities such as brainstorming sessions, sketching and
plenary discussions were conducted as part of the workshops. Evaluating interface
design and creating redesign proposals were also included. Two researchers
facilitated each workshop, where one had the responsibility to lead the
conversations and facilitate the design activities, while the other had the
responsibility to make notes. The material was analyzed inductively.
3 Results
The analysis revealed four main themes: (1) the healthcare professionals mapping
of the patients’ health status and needs; (2) the importance of the professionals’
qualification and competence regarding facilitating patients through such an
eHealth service; (3) the patients’ experience that the eHealth service is
safeguarding; (4) usability and the requirement for added features of the eHealth
system.
3.1 Mapping the patients’ health status and needs
The patients sent their health reports based on a subjective evaluation of their
health status by answering a set of predefined questions. The nurses reported that
the information they acquired about the patients through the app was minimal for
being able to do a sufficient evaluation of the patients’ health condition. In cases
when a patient reported that his/her health condition was as normal or as usual,
the nurses did not conduct any further actions. If a patient reported on symptoms
of change in health status, the nurses required additional information to get a
sufficient understanding about the patient’s condition. In these cases, the nurses
would contact the patient by phone to obtain further information:
“We ask about much more, because if they have heavy
breathing; we check or ask if they have chest pain; if
their feet are swollen; and find out if it can be something
else.” (nurse)
The nurses conveyed that the phone call provided valuable non-verbal
information about the patients that they did not capture in any way other than by
talking to the person; by listening to the patient’s tone of voice, the respiration
(breathing) and coughing, the nurses acquired additional information that was
relevant to attain a sufficient understanding about the patient’s situation. When
asked if additional questions could be included in the technical solution in the app
to elucidate these aspects, the professionals did not consider it necessary. They
elaborated that they would prefer to contact the patients by phone for a further
investigation anyway: “no, those questions do not need to be in the app” (nurse).
Patients suffering from COPD commonly have several comorbidities. This was also
the case for those participating in this study. The patients expressed that they found it
difficult to merely report on one disease, as they had several illnesses which might
affect the other and thus cause a change in health status and illness deterioration: “It
feels awkward to merely report on COPD […] One thing affects the other” (patient).
As a result, the patients would prefer to report on more general parameters or
questions about their health, and not merely be limited to COPD related
parameters/symptoms. Further, the patients conveyed that a COPD exacerbation
could occur in various ways for the individual, with a great specter of individual
variations, not always appearing with the standard symptoms of such
exacerbations. Due to this fact, the nurses underlined the need for acquiring more
information, thus having a telephone conversation with the patients to identify the
cause of the reported health deterioration and symptoms. The combination of
information from the app together with a follow-up phone call became the
standard procedure to map the patients’ situation sufficiently.
Fig. 1: Illustration of the graphical interface of the tablet application used in
the pilot study.
3.2 Professionals’ qualifications and competence
The nurses that were responsible for managing the patient reports were
specialized in respiratory nursing. Being able to consider various factors that
could influence the patients’ situation by the professionals was crucial to be able
to evaluate the patients’ health condition. If a patient had reported on illness
deterioration or on symptoms of worsened health condition, the nurses would use
the subsequent telephone conversation as an additional source to map the person’s
condition:
“The one that talks with them (patients) needs to be
experienced, that you both listen… it is good to hear
when they talk, if they are breathing heavily, hear if they
cough… […] and I check what medications they have
taken, what kind of inhalations, perhaps they haven’t
taken it at all… different things.” (nurse)
Their personal experience of working with this patient group would influence the
way the nurses would conduct the telephone conversation with the patient:
“We start [the phone call] with repeating what they have
registered, [I say] that I see that you have reported that
you are like this and that… how are you actually? And
then they explain from that. And then we ask the additional
questions. The first thing is to check according to the
differential diagnosis.” (nurse)
However, in cases where the patients had reported on illness symptoms, the
nurses reported that they needed decision support in quite a few cases and stated a
need for decision support from the pulmonary physician in about 50% of
instances during the study period. They elaborated that the need for decision
support was primarily in cases concerning initiating or dose adjustments of
medications.
The patients on the other hand, expressed that they very much appreciated the
way they were approached and handled by the professionals. They reported
having full confidence that their needs were addressed and that the decisions
made would be correct.
3.3 The eHealth service as safeguarding
The new eHealth service involved increased accessibility to healthcare
professionals and specialist healthcare that was valued by the patients. One of the
nurses experienced this in her daily work: “We have our office phone (cell phone)
with us all the time, so they can call us. And they do call” (nurse). The patients
experienced that this service made them feel safe. This was emphasized as
safeguarding because of the way the service was designed, with the health
professionals making contact with the patient if s/he had reported feeling ill
through the tablet application. One of the participants expressed: “Because of my
health condition, the app is my safety in my daily life’ (patient). The service was
particularly appreciated by those who lived alone and who did not have a shared
household with anyone. All the patients conveyed that they experienced it as
reassuring that someone else knew how s/he was coping, which made them feel
safe and catered for. The nurses occasionally experienced not receiving the
patient report before noon, if so, they would contact the patient by phone to check
that everything was okay:
“It depends, the same day at least, but they are supposed
to report before noon or one o’clock. It says that in our
protocol. So, we follow that, so I call, I consider… half
past one, or one o’clock. I call in sufficient time before
the working day is over. To check, give them a chance
to…” (nurse)
The patients experienced that it was reassuring to be contacted by phone if they
had not reported through the tablet application. The nurses reported that they had
experienced that patients had not been able to deliver their daily report through
the tablet app because they were in bad shape. In other instances, the users had
just forgotten to report. All the patients described being sincerely satisfied with
the follow-up and care they attained during the pilot study period.
3.4 Usability and added features
The patients reported that the app was easy to use and time efficient, as one
participant said: “I think it is good as it is. Honestly” (patient). However, when
going into details about the app and when presented with alternatives for how
questions could be visualized, the participants reported that they would prefer
more graphical illustrations and not merely text based questions. The patients
could be categorized as novice users, with limited experience with the Internet,
applications and online solutions. Therefore, they had low expectations and little
to compare with considering what to expect. However, both patients and
professionals remarked that the text used in the app could be improved to avoid
misunderstandings. Further, they reported that some of the questions about
medication were imprecise and could lead to misunderstandings; they were to
report if they had taken their ‘standard medication’, but what each individual
considered this to be differed from person to person. The participants created
sketches of suggested app features, such as exercise and information videos, and
social networking features. The nurses informed of several technical challenges,
some more critical than others, which had been identified during the pilot study.
These had been adjusted continuously but needed further improvements. The
graphical user interface (GUI) of the receiving system in the clinic was easy to
use, but required improvements for increased usability. Particularly, the GUI of
the patient reports on the recipients’ cellphones needed improvement. The cell
phones appeared to be a crucial working tool for the nurses due to their mobile
working situation: “It is important that we receive [the patient reports] on the cell
phone. We are not stationed here all the time” (nurse). They considered the GUI
of their stationary (pc) access to the patient reports as satisfying, as one of the
nurses expressed: “I don’t use it much, NN [the other nurse] checks every day, so
I only do it the few days she is not here. And so… if I can manage to use it, then it
is quite good.”
4 Discussion
The findings from this formative evaluation revealed some important aspects
concerning both interaction design and service design of the HelsaMi-solution for
patients suffering from COPD. By using PD approaches as part of the workshops
we could identify the participants’ experiences and improvement suggestions of
importance. By using a combination of individual brainstorming activities and
plenary discussions, we allowed the participants express their experiences and
perspectives that otherwise might not have come to the surface. Commonly in
such workshops some participants are more verbal and engaged than others,
taking more attention on behalf of those who are more silent and shy. Considering
small-scale pilots in healthcare, such as the one described in this project,
presupposes acquiring the views and voices of the participants to evaluate and
give directions for the next phase. Using PD approaches are useful but require
consideration for those involved. We observed that some participants had
difficulties participating in the more creative tasks, such as drawing suggestions
for a redesign of the app, but were easier to engage after being presented with
some suggestions that could trigger their creativity and discussions. This implies
that the use of various techniques is important to accomplish the planned PD-
activities. This understanding complements previous experiences of using PD-
techniques with patients [8].
Our results revealed the need for improvements of the GUI of the patient
application and the receiving system at the clinic. These requirements were of
both aesthetical and functional character. Such need for improvements could have
been identified by usability testing of the eHealth solution earlier in the
development process, and thus prevented technical hustle during the pilot study.
This finding underlines the importance of structured usability testing early in a
project. The fact that the nurses required additional information from the patients
by phone, supplementing the received info through the eHealth app, emphasizes
the need to plan sufficient resources and time to conduct essential activities that
result from introducing new tools such as eHealth solutions in clinical care.
Considering the nurses’ high need for decision support reveals the necessity of
having qualified personnel that have the competence to identify specific illness
symptoms, and thus enable decision support to implement interventions. This
implies the requirement of having qualified personnel to receive and handle such
patient data and information, but also the importance of having qualified decision
support available. The tested eHealth solution was based on self-reported patient
data, but for further studies, it would also be interesting to add objective data
measures from the patients, such as pulse oximetry, body temperature etc. to
supplement the patient self- reported data. These could act as mediators for the
professionals to facilitate decision support. The service was considered valuable
and important for the participating patients, and indicates that an eHealth solution
for COPD management and follow-up has potential for positive outcomes,
particularly regarding feeling safe and concerning quality of life for this patient
group. This study is limited to its qualitative approach and the results cannot be
generalized. Nevertheless, the study provides some implications about using PD -
approaches as important means to reveal users’ experiences, perspectives and
demands that are important for system and service design of eHealth solutions in
COPD management. Further studies are needed to explore the potential and
impact of using eHealth solutions in integrated health services for people with
COPD.
Acknowledgements The Norwegian Research Council funded this project.
References
1. Mannino, D.M and V.A. Kiriz, Changing the burden of COPD mortality. Int J Chron
Obstruct Pulm Dis, 2006. 1 (3): pp. 219–33, Publisher Location, 1995
2. Wesseling, G. and H.J. Vrijhoef, Acute exacerbations of COPD: recommendations for
integrated care. Expert Rev Respir Med, 2008. 2 (4): pp. 489–94. DOI:
10.1586/17476348.2.4.489
3. Menn, P., N. Weber, and R. Holle, Health-related quality of life in patients with severe
COPD hospitalized for exacerbations – comparing EQ-5D, SF12 and SGRQ. Health
Qual Life Outcomes, 2010. 17 (6): p. 39. DOI: 10.1186/1477-7525-8-39
4. Zhang, J., Y.-L. Song, C.-X. Bai. MIOTIC study: a prospective, multicenter, randomised
study to evaluate the long-term efficacy of mobile phone-based Internet of Things in the
management of patients with stable COPD. International Journal of Chronic Obstructive
Pulmonary Disease, 2013. 8: pp. 433–8. DOI: http://dx.doi.org/10.2147/COPD.S50205
5. Bolton, C.E., et al., Insufficient evidence of benefit: a systematic review of home
telemonitoring for COPD. J. Eval. Clin. Pract., 2010. 17 (6): pp. 1216–1222. DOI:
10.1111/j.1365-2753.2010.01536.x
6. Titova, E. et al., Long term effects of an integrated care intervention on hospital
utilization in patients with severe COPD: a single centre controlled study. Respiratory
Research, 2015. 16 (8). DOI: 10.1186/s12931-015-0170-1
7. Das, A. and A. Faxvaag. What influences patient participation in an online forum for
weight loss surgery? A qualitative case study. Interactive Journal of Medical Research,
2012. 3 (1). DOI: 10.2196/ijmr.2847.
8. Das, A., Bøthun, S., Reitan, J., Dahl, Y. The use of generative techniques in co-design of
mHealth technology and healthcare services for COPD patients. International
Conference of Design, User Experience, and Usability (pp. 587–595). Springer
International Publishing. 2015.