=Paper=
{{Paper
|id=Vol-2903/IUI21WS-HEALTHI-4
|storemode=property
|title=Lena: a Voice-Based Conversational Agent for Remote Patient Monitoring in Chronic Obstructive Pulmonary Disease
|pdfUrl=https://ceur-ws.org/Vol-2903/IUI21WS-HEALTHI-4.pdf
|volume=Vol-2903
|authors=David Cleres,Frank Rassouli,Martin Brutsche,Tobias Kowatsch,Filipe Barata
|dblpUrl=https://dblp.org/rec/conf/iui/CleresRBKB21
}}
==Lena: a Voice-Based Conversational Agent for Remote Patient Monitoring in Chronic Obstructive Pulmonary Disease==
https://ceur-ws.org/Vol-2903/IUI21WS-HEALTHI-4.pdf
Lena: a Voice-Based Conversational Agent for Remote
Patient Monitoring in Chronic Obstructive Pulmonary
Disease
David Cleresa , Frank Rassoulic , Martin Brutschec , Tobias Kowatscha,b and Filipe Barataa
a
Center for Digital Health Interventions, Department of Management, Technology, and Economics, ETH Zurich, Zurich, Switzerland
b
Center for Digital Health Interventions, Institute of Technology Management, University of St. Gallen, St. Gallen, Switzerland
c
Lung Center, Cantonal Hospital St. Gallen, St. Gallen, Switzerland
Abstract
Chronic obstructive pulmonary disease (COPD) is one of the leading causes of death worldwide. To manage the increasing
number of COPD patients and reduce the social and economic burden of treatment, healthcare providers have sought to
implement remote patient monitoring (RPM). Screen-based RPM applications, such as filling self-reports on the smartphone
or computer, have been shown to increase the quality of life, reduce the frequency and severity of exacerbations, and increase
physical activity in patients with COPD. These applications, however, are not without challenges for the elderly target
population. They are often used on devices designed by and for a different age group, which makes filling out self-reports
prone to error and induces fears of technology malfunctions. Voice-based conversational agents (VCAs) are available on more
than 2.5 billion devices and are increasingly present in homes worldwide. Aside from their commercial success, VCAs are
also credited with several functionalities, such as hands-free use, that make their adoption in healthcare attractive, especially
for the elderly. In this work, we investigate the potential of VCAs for RPM of COPD. Specifically, we designed and evaluated
Lena, a single-board computer-based VCA framed as a digital member of the medical team. Lena acts as RPM for the early
prediction of COPD exacerbations by asking ten symptom-related questions to determine the patient’s daily health status.
This paper presents the patients’ feedback after their interaction with Lena. Patients evaluated the acceptability of the system.
Notably, all patients could imagine using the system once a day in the context of a larger study and wished to integrate Lena
into their daily routine.
Keywords
Chronic obstructive pulmonary disease, Voice-based Conversational Agents, Remote Patient Monitoring, Single-Board
Computer, Ubiquitous Computing
1. Introduction COPD exacerbations occur on average one to four times
per year [4] and represent the acute worsening of symp-
COPD is the third leading cause of death worldwide [1]. It toms such as shortness of breath and cough [5]. Approx-
is responsible for 251 million reported deaths per year [2]. imately 70 % of health care costs associated with COPD
COPD is a chronic, progressive disease caused by airway are due to emergencies and hospitalizations for treatment
inflammation due to smoking or long–term exposure to of exacerbations. RPM has the potential to reduce the fre-
pollutants (e.g., dust, fumes, poor air quality) [2]. quency and severity of COPD exacerbations [6], thereby
To cope with the increasing number of COPD patients reducing healthcare costs [7]. While recent studies have
and to reduce the pressure on health services, providers shown potential benefits (e.g., increased quality of life [8],
have sought to introduce RPM for COPD patients [3]. reduced frequency and severity of exacerbations [6], im-
RPM is the automatic, continuous transmission and pro- proved physical activity [9]) of RPM for patients with
cessing of physiological data, decision support, prediction COPD, physiological parameters of COPD patients are
of deterioration, and alerting. not continuously monitored outside of hospitals, except
COPD patients are usually treated as outpatients, except for research purposes [10]. Moreover, symptoms are usu-
in cases of hospitalization due to an exacerbation [4]. ally self–reported by the patients using pen and paper
Joint Proceedings of the ACM IUI 2021 Workshops, April 13–17, 2021, diaries [10]. Considering that COPD patients belong to
College Station, USA the older part of the population, current screen–based
Envelope-Open dcleres@ethz.ch (D. Cleres); frank.rassouli@kssg.ch applications (e.g., filling self–reports on smartphones or
(F. Rassouli); martin.brutsche@kssg.ch (M. Brutsche); computers) do not come without challenges. Older adults
tkowatsch@ethz.ch (T. Kowatsch); fbarata@ethz.ch (F. Barata) often have low IT–literacy, fear of malfunction [11], and
Orcid 0000-0002-7534-7960 (D. Cleres); 0000-0003-2426-3545
(F. Rassouli); 0000-0002-1612-3609 (M. Brutsche);
a lack of confidence in their ICT abilities [12]. In addi-
0000-0001-5939-4145 (T. Kowatsch); 0000-0002-3905-2380 tion, the fact that most commercially available software
(F. Barata) is developed by and for a different age group limits the
© 2021 Copyright for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0). inclusion of more digital applications in the lives of older
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
people, even though they recognize the benefits that percent of all global deaths [2]. The disease’s prevalence
come with increased ICT capabilities [13]. Further, there is on the rise due to higher smoking prevalence and the
is evidence that older generations prefer voice–based to aging population [2]. In consequence, the burden on
screen–based communication [14]. health care providers is expected to increase in the com-
From The Voder, the first attempt to electronically syn- ing years. Evidence suggests that RPM for COPD patients
thesize human speech by using a console with fifteen can reduce hospitalizations [6] and costs associated with
touch–sensitive keys and a pedal to select the appro- this disabling disease by at least 14 percent [7]. Moreover,
priate bandpass filters to convert the hisses and tones McLean et al. showed in a Cochrane review that RPM can
into vowels, unveiled at the 1939 New York World’s Fair, increase the quality of life of COPD patients and reduce
to today’s Alexa, Siri, or Google Assistant, the way and the number of exacerbations [8]. There is also evidence in
frequency humans interact with VCAs have evolved dra- a review by Lundell et al. that RPM can improve physical
matically. VCAs are now available on more than 2.5 activity levels in COPD patients [9]. Pedone and Lelli, in
billion devices such as smartphones and tablets, smart their review [24], reported a positive but nonsignificant
speakers, computers and have even been embedded in effect of remote care on hospital admissions and emer-
wearables or cars, thus nearly becoming ubiquitous in gency department visits. Similarly, McDowell et al. [25]
our daily lives [15]. One in four Americans owns a smart showed that RPM with self–management improves the
speaker, and in 2018 alone, their ownership increased quality of life but does not significantly improve emer-
by 40 percent [16]. Beyond their commercial success, gency care.
VCAs are also credited with several functionalities that More recently, Rassouli et al. investigated the association
make their use in healthcare attractive. VCAs enable between the COPD Assessment Test (CAT) and the risk
hands–free interaction, allow input from individuals with of exacerbation of COPD [26]. In this study, patients com-
low literacy or with intellectual [17], motor and cogni- pleted an online questionnaire focused on detecting an
tive disabilities [18], or provide more natural support acute exacerbation of COPD (daily) and the CAT (weekly).
for routine health tasks when in–person healthcare is The authors found that by completing the questionnaires
not possible [19]. Voice interaction also enables passive regularly, patients could assess their health status more
monitoring and analysis of audio samples for healthcare accurately. Also, the evolution of the CAT could help
applications, such as Alzheimer’s [20], depression [21], assess the risk of future exacerbations.
and schizophrenia [22]. In addition, recent work sug-
gested the potential of speech (e.g., pause time, pause 2.2. VCAs for healthcare
frequency, prosodic features, among others) as a marker
of exacerbations in patients with COPD [23]. Humans apply social rules in interactions with comput-
With this in mind, we argue that VCAs have the potential ers [27, 28]. VCAs enable this behavior by mimicking
to enable RPM for COPD patients by facilitating the col- interpersonal conversation. Therefore, compared to text–
lection and sharing of health–related information with based interaction, they are perceived as more socially
healthcare professionals, thereby improving quality of present (i.e., the perception of interacting with an in-
life, reducing exacerbations, and thus, reducing the costs telligent being) [29, 30] and even more believable than
in COPD care. humans when it comes to information retrieval [31]. Sim-
For these reasons, the authors developed and evaluated a ilar to existing screen–based conversational agents [32,
single–board computer (SBC)–based VCA framed as Lena, 33, 34], VCAs can form a working alliance [35] with pa-
a digital member of the medical team. This pilot study tients, which has a positive impact on treatment out-
investigates the acceptability of a voice–based approach comes [36, 37]. Recent works by Balasuriya et al. [38],
to alleviating patients’ communication burden while fill- Ireland et al. [39], and Kadariya et al. [40] also suggest
ing questionnaires. More specifically, the VCA’s ability that non–commercial dedicated VCAs can meet user ex-
to formulate its questions simply and understandably, pectations when supporting the prevention and man-
as well as its ability to understand the patient’s answer agement of chronic or mental illnesses. Furthermore,
and respond accordingly, were assessed with four COPD speech interaction also enables passive monitoring and
patients. analysis of voice samples for health applications, such
as Alzheimer’s [20], depression [21], schizophrenia [22],
and COPD [23].
2. Related Work
2.1. RPM for COPD 3. Methods
In 2015, the World Health Organization estimated that Lena’s design and evaluation follow an iterative design
3.17 million people died from COPD, accounting for five science approach, in which single capabilities are con-
Figure 2: Illustration of Lena’s embedded system. The sky
blue box represents the external inputs coming from the pa-
tient. The white boxes show the hardware components. The
light gray color highlights all the elements from which Lena
Figure 1: Detail view of Lena – A) Patient interacting with is built. Grey represents the operating system. The features
Lena. B) Front–view of Lena with open front. C) Front–view of the developed Android app appear in dark grey.
of Lena without its housing. D) Raspberry Pi 4 with all the
connected components.
3.2.2. How Lena communicates
tinuously improved and eventually integrated into one To engage with a patient, Lena must be able to (i) recog-
system. nize when the patient talks with it, (ii) understand the
patient’s speech, and (iii) to respond in an intelligible way.
The next three paragraphs describe the implementation
3.1. Hardware of these points.
Lena consists of three hardware components: a Single–
Board Computer (SBC), a USB microphone, and a pair of Key phrase recognition To activate Lena, a patient
USB speakers. For aesthetics and protection, a wooded must utter the key phrase “Hello, Lena”. The developed
loudspeaker housing contains Lena’s core components, app recognizes this key phrase in real–time. After rec-
as shown in Figure 1. More precisely, Lena’s core consists ognizing the key phrase, Lena starts speaking, explains
of a Raspberry Pi 4 Model B (cf. Figure 1. D.; Specifica- the conversation’s goal to the patient, and asks the first
tions: 8 GB RAM and 1.5 GHz processor) and a SanDisk question.
Ultra microSDHC of 16 GB. The USB microphone has The key phrase recognition uses Vosk, an offline open–
a sensitivity of −67 dBV/pBar, −47 dBV/Pascal ±4 dB, source speech recognition toolkit that understands 17
the frequency ranges from 100 − 16kHz, and on–device languages and dialects.
noise–canceling filters out the background noise. Finally,
the USB–powered speakers are connected to the 3.5 mm Speech recognition To reliably understand the pa-
jack port of the SBC. Their frequency ranges from 50 to tient’s responses, Lena uses Android’s API. More specifi-
20′ 000 Hz. cally, the SpeechRecognizer, RecognitionService, and Rec-
ognizerIntent Android speech classes handle the speech–
3.2. Software to–text transformation of the patient’s speech. We im-
plemented speech recognition to operate purely offline.
3.2.1. Operating system In this way, the patient’s personal and sensitive data re-
The SBC used to power Lena runs LineageOS, a free mains on the device and does not need to be anonymized
and open–source operating system based on the Android by another algorithm, which could affect the quality of
mobile platform. This has the advantage that all apps the recording. To enable this functionality, the authors
already available in the Google Play Store can also be had to specify the flag EXTRA_PREFER_OFFLINE in the
used on the Raspberry Pi after installing Open GApps. Android RecognizerIntent class. Furthermore, the offline
The Open GApps Project is an open–source effort that speech recognition package must be downloaded on the
provides pre–built Google Apps packages. By installing device from the Language and Input section of the Lin-
Open GApps, the Google Play Store became available, as eageOS system settings. Finally, the Android speech
well as Google’s speech functionalities and APIs, which recognition framework must be set to the one provided
will be discussed in more detail below. by Google.
Speech synthesis Lena’s offline text–to–speech (TTS) Lena would react differently, e.g., by telling a joke if the
capabilities are also based on the Android API. The An- patient was not feeling well. At the beginning of the
droid class TextToSpeech converts Lena’s predefined con- interview, Lena would ask the patient whether to use
versational turns from text to speech. Other TTS APIs formal or casual language throughout the interview. Fi-
(e.g., Amazon Polly, Google Cloud TTS, IBM Watson nally, the eight other binary questions were aimed at
TTS) could have been used to make Lena’s speech more evaluating the patient’s health status.
melodic and less monotone. However, the authors de- At the end of the dialog, Lena would thank the patient
cided to use the Android API, which allows speech to be for sharing today’s symptoms with the study team and
synthesized offline without installing additional software switch back into an idle mode, waiting for the patient to
packages while delivering an intelligible speech. say the key phrase.
3.3. Experimental set–up 3.3.2. Evaluation and interview details
3.3.1. Patient recruitment & interaction with the After the interaction with Lena, the patients were inter-
VCA viewed by the study team for 20 minutes.
The first part consisted of filling out a pen and paper
Four COPD patients (three male, one female) of age 69±5 survey with six questions evaluated on a 7–point Likert
interacted with Lena. In practice, Lena was framed as a scale (see Table 1). The questions aimed to evaluate Lena
digital member of the medical team. and were based on the technology acceptance model [42].
Three different hospital rooms served to conduct the in- Perceived enjoyment was defined as the degree to which
terviews (a furnished hospital bedroom, a doctor’s office, the activity of using technology is perceived to be en-
and an examination room) to evaluate Lena under var- joyable [43]. Finally, relative advantage represented the
ious circumstances. Also, one patient received oxygen degree to which a novel application is perceived as being
therapy while interacting with Lena. Before the interac- better than the state of the art [44]. The second part
tion with Lena began, the authors instructed the patients of the interview consisted of a face–to–face interview
on the nature of the questions that would be asked. The between the patient and a member of the study team (see
patients were randomly recruited based on their previous Table 2).
or current participation in a COPD–RPM study [41, 26].
In this study, patients completed a daily questionnaire via
their personal computer or smartphone. Rassouli et al. 4. Results
were able to detect 60 out of 63 acute exacerbations of
COPD with a sensitivity, specificity, positive and negative 4.1. Acceptability
predictive value of 95, 98, 26, and 99.9 %, respectively [41].
The patients understood Lena’s questions accurately (EOU1)
During the interaction, Lena asked and the patients an-
and felt that Lena understood their responses (EOU2).
swered the following questions in German: (i) Do you
They enjoyed conversing with Lena (ENJ1) and could
have more dyspnoea today, exceeding your usual varia-
imagine using Lena at least once a day to complete the
tion?, (ii) Do you have more sputum today, exceeding your
questionnaire (ITU1), and found Lena useful (USE1). Last
usual variation?, (iii) Is your sputum today more yellow or
but not least, patients would prefer to use Lena than the
green, exceeding your usual variation?, (iv) Do you have
existing screen–based solution (RA1). The entire conver-
more cough today, exceeding your usual variation?, (v)
sation lasted an average of two minutes.
Do you feel febrile today?, (vi) Do you feel like having a
After the conversation with Lena, the authors conducted
common cold today?, (vii) Do you feel unwell today, ex-
face–to–face interviews with patients to understand the
ceeding your usual variation? and (viii) Did you start to
current system’s strengths and weaknesses. Table 2
take your emergency medication within the last 24 h?. This
shows a condensed summary of these interviews. All pa-
time, however, the questions were asked and answered
tients owned a smartphone (TECH1). Half of the patients
verbally. By selecting these patients, it was possible to
were already intentionally using a VCA in the context
objectively compare the speech–based solution with the
of driving or reading a text aloud (TECH2). All patients
previously used screen–based solution.
could imagine using Lena instead of the screen–based
Concretely, the patients sat in front of Lena, as shown in
application for a one–year study (ITU2). When asked
Figure 1. After being triggered with the key phrase “Hello
about the advantages and disadvantages of completing
Lena”, Lena began asking questions to capture a ques-
the questionnaire orally, patients emphasized the ease of
tionnaire of the patient’s perceptions of current COPD
use of Lena and that no login is required, which is the
severity. The questionnaire consisted of nine binary ques-
case with the screen–based study (ENJ3). Another patient
tions and one open–ended question. The latter referred
responded that interacting verbally with Lena would in-
to the patient’s mood. Depending on the patient’s mood,
crease compliance as it felt more engaging (ENJ3). Two
Table 1
Pen and paper survey questions and aggregated patient responses. SD: Strongly Disagree, SA: Strongly Agree
Construct Code Question–Item Answer Options Results (M ± SD)
Perceived ease of use EOU1 I understand Lena’s questions SD (−3) – SA (3) 2.75 ± 0.5
Perceived ease of use EOU2 Lena understands my answers SD (−3) – SA (3) 2.25 ± 0.96
Perceived enjoyment ENJ1 I was happy to respond to Lena’s questions SD(−3) – SA (3) 2.5 ± 0.58
Intention to use ITU1 I can imagine Lena to answer a few SD(−3) – SA (3) 2.00 ± 1.41
questions daily
Perceived usefulness USE1 I find Lena useful to give my daily SD (−3) – SA (3) 2.25 ± 0.96
assessment on COPD control study
Relative Advantage RA1 In the context of the planned SD (−3) – SA (3) 1.50 ± 1.73
study, I would rather answer Lena
once a day than fill a questionnaire
on a smartphone for this purpose
Table 2
Face–to–face interview questions and patient answers. P: Patient, NI: No Idea
Construct Code Question–Item Answers (P1, P2, P3, P4)
Tech. Affinity TECH1 Do you possess a smartphone? Yes, Yes, Yes, Yes
Tech. Affinity TECH2 Do you already use a voice No, In car, No, Read aloud
assistant (Alexa, Cortana, Siri)?
Intention to use ITU2 Would you be willing, as part of a study Yes, Yes, Yes, Yes
on self–management of COPD, to answer Lena’s
questions by voice before going to bed or
when getting up for a period of one year?
Perceived ENJ3 What did you like and dislike about Lena, especially in Engagement, Ease–of–use,
enjoyment comparison to the questionnaire on the smartphone? NI, No login process
Perceived ENJ4 Would you prefer to interact in Swiss German dialect Yes, No, No, Yes
enjoyment instead of German?
Improvements IMP1 What would definitely need to be improved about Lena? Intonation, NI, NI, NI
Improvements IMP2 What could possibly go wrong with Power outage, NI,
Lena’s questions or with your answers? What NI, Holidays
should we consider in the development of Lena?
shortcomings were that Lena sometimes took too long each interview in JSON format. The patients could freely
to interpret her utterances, and one of the two testers answer Lena’s questions. However, to understand the
highlighted Lena’s intonation as a disadvantage (IMP1). positive or negative connotation of a patient’s answer,
In their opinion, these two factors made the dialogue the algorithm searched for a keyword such as “yes” or
sluggish at some moments. One patient was concerned “no”.
that Lena would not fit in his luggage when traveling All the interviews could be initiated with the key phrase
on vacation, and that in this case, completing the ques- and completed by the patients. It took, on average, 2±1.41
tionnaire would require a different solution (IMP2). P4 attempts to trigger the interaction. Finally, the patient’s
also pointed out possible risks caused by power outages speech transcribed by Lena perfectly matched the au-
since the system needs to be plugged in (IMP2). Finally, thors’ transcription; all responses were correctly identi-
whether Lena should rather address the patients in Swiss fied and transcribed.
German dialect instead of German was answered posi-
tively by two patients (ENJ4).
5. Discussion
4.2. Speech recognition All COPD patients succeeded in triggering, understand-
To evaluate Lena’s speech recognition capabilities, the ing, and interacting with Lena to complete the question-
authors recorded and transcribed the interviews using a naire. The speech recognition results indicate that the
smartphone phone. Lena saved the recognized speech of prototype can understand the patients’ responses with
perfect accuracy and lead the discussion accordingly.
Also, all patients declared their willingness to use Lena symptoms and make information exchange seamless with
daily over a longer period (e.g., 12 months) (ITU1, ITU2) the medical team. The combination of the voice–based
and favored this voice–based solution over the original questionnaires and the passively recorded data should
screen–based application [41] (RA1). This suggests that not only open a wide range of new research directions
Lena qualifies for validation in a longitudinal study with but, more importantly, provide better support for COPD
COPD patients. patients.
Lena’s voice assistant capabilities rely on Android’s speech
recognition and speech synthesis and Vosk’s speech recog-
nition APIs. All three can be used offline without trans- 6. Conclusion
ferring patients’ recorded speech samples to an external
This pilot study proposes Lena, a state of the art VCA
server. With this approach, we recognize the sensitive
for COPD RPM. Lena interacts with the patient in a spo-
nature of medical and speech data and ensure privacy
ken natural language to collect daily self–reports. This
and security.
first evaluation yielded promising acceptance results of
Although the healthcare sector already uses VCAs [45,
a VCA–based RPM application for COPD. All patients
46], this study provides the first insights regarding the
also showed a willingness to integrate Lena into their
feasibility, relevance, and acceptability for such an appli-
daily routine and saw its potential to improve future RPM
cation for COPD patients. In contrast to the proposed
solutions. We plan to integrate and evaluate Lena in a
pen and paper or screen–based applications to collect
longitudinal observational study with COPD patients.
self–reports’ information, Lena provides a tailored ap-
This research is a first step towards enabling scalable and
plication for the elderly target population by verbally
natural–language–delivered RPM to facilitate access to
capturing patient information without requiring interac-
health–related self–management services. It may also
tion with another person via a phone call or in–person
help overcome limitations of text–based systems, such
visit [41, 24, 9]. Moreover, recent work has suggested
as the lack of literacy of users in countries with low edu-
the potential of speech (e.g., pause time, pause frequency,
cation index, accessibility for the elderly population, or
prosodic features, a.s.o.) and cough [47] as a marker
even empowerment of patients with mental, motor, or
of COPD exacerbations [23]. Considering that recent
cognitive disabilities.
research has also shown the feasibility to detect cough
events with high accuracy on devices with limited com-
puting power (e.g., smartphones [48]) and the ability to References
continuously listen in the background, detect and count
coughs comparable to human annotators [49], we argue [1] T. G. I. for Chronic Obstructive Lung Disease, 2020
that such a smart speaker system could not only collect GOLD REPORTS, 2020. URL: https://goldcopd.org/
subjective information but also provide passively and gold-reports/.
objectively measured parameters related to the health [2] WHO, Chronic respiratory diseases, 2017. URL:
status of patients with COPD. https://bit.ly/39y6qQN.
The study population received Lena well in terms of use- [3] C. E. Bolton, C. S. Waters, S. Peirce, G. Elwyn,
fulness, ease and intention of use, and enjoyment. Never- Insufficient evidence of benefit: a systematic re-
theless, there is a need for a systematic and comparative view of home telemonitoring for copd, J Eval
evaluation of the system (e.g., test the distance and direc- Clin Pract 17 (2011) 1216–22. doi:1 0 . 1 1 1 1 / j . 1 3 6 5 -
tion in which the microphone was effective, test on older 2753.2010.01536.x.
adults with an absence of basic digital literacy skills) to [4] I. Tomasic, N. Tomasic, R. Trobec, M. Krpan,
understand which aspects of Lena are best for patient T. Kelava, Continuous remote monitoring of
acceptance and better outcomes than standard in–person copd patients—justification and explanation of
care. the requirements and a survey of the available
Patients welcomed completing the questionnaire without technologies, Medical & Biological Engineering
having to sign in. However, for Lena to meet the same & Computing 56 (2018) 547–569. URL: https://
data security standards as the state of the art screen– dx.doi.org/10.1007/s11517-018-1798-z. doi:1 0 . 1 0 0 7 /
based system, voice–based authentication could be used, s11517-018-1798-z.
as it prevents unauthorized use and protects patient pri- [5] S. D. Sullivan, S. D. Ramsey, T. A. Lee, The
vacy [38]. economic burden of copd, Chest 117 (2000)
Finally, Lena’s current version did not allow patients to 5S – 9S. URL: http://www.sciencedirect.com/
freely express their symptoms since the questionnaire science/article/pii/S0012369215527487. doi:h t t p s :
consisted only of binary questions. Nonetheless, using //doi.org/10.1378/chest.117.2_suppl.5S.
a voice–based system allows patients to communicate [6] F. C. Cordova, D. Ciccolella, C. Grabianowski,
more naturally to provide more information about their J. Gaughan, K. Brennan, F. Goldstein, M. R. Jacobs,
G. J. Criner, A telemedicine-based intervention among low income housing residents in new york
reduces the frequency and severity of copd exacer- city, BMC Research Notes 10 (2017) 517. URL: https:
bation symptoms: A randomized, controlled trial, //doi.org/10.1186/s13104-017-2850-z. doi:1 0 . 1 1 8 6 /
Telemed J E Health 22 (2016) 114–122. doi:1 0 . 1 0 8 9 / s13104-017-2850-z.
tmj.2015.0035. [15] B. Kinsella, A. Mutchler, Voice assistant consumer
[7] G. Paré, P. Poba-Nzaou, C. Sicotte, A. Beaupré, adoption in healthcare, Technical Report, 2019.
E. Lefrançois, D. Nault, D. Saint-Jules, Com- URL: https://voicebot.ai/voice-assistant-consumer-
paring the costs of home telemonitoring and adoption-report-for-healthcare-2019.
usual care of chronic obstructive pulmonary [16] B. Kinsella, A. Mutchler, Smart Speaker
disease patients: A randomized controlled Consumer Adoption Report, Techni-
trial, European Research in Telemedicine / cal Report March, 2019. URL: https:
La Recherche Européenne en Télémédecine 2 //voicebot.ai/wp-content/uploads/2019/03/
(2013) 35–47. URL: http://www.sciencedirect.com/ smart_speaker_consumer_adoption_report_2019.pdf.
science/article/pii/S2212764X13000241. doi:h t t p s : [17] S. S. Balasuriya, L. Sitbon, A. A. Bayor,
//doi.org/10.1016/j.eurtel.2013.05.001. M. Hoogstrate, M. Brereton, Use of voice
[8] S. McLean, U. Nurmatov, J. L. Liu, C. Pagliari, activated interfaces by people with intellectual
J. Car, A. Sheikh, Telehealthcare for chronic disability, in: Proceedings of the 30th Australian
obstructive pulmonary disease, Cochrane Conference on Computer-Human Interaction,
Database Syst Rev (2011) Cd007718. doi:1 0 . 1 0 0 2 / OzCHI ’18, Association for Computing Ma-
14651858.CD007718.pub2. chinery, New York, NY, USA, 2018, p. 102–112.
[9] S. Lundell, A. Holmner, B. Rehn, A. Nyberg, URL: https://doi.org/10.1145/3292147.3292161.
K. Wadell, Telehealthcare in copd: a system- doi:1 0 . 1 1 4 5 / 3 2 9 2 1 4 7 . 3 2 9 2 1 6 1 .
atic review and meta-analysis on physical out- [18] F. Masina, V. Orso, P. Pluchino, G. Dainese, S. Vol-
comes and dyspnea, Respir Med 109 (2015) 11–26. pato, C. Nelini, D. Mapelli, A. Spagnolli, L. Gam-
doi:1 0 . 1 0 1 6 / j . r m e d . 2 0 1 4 . 1 0 . 0 0 8 . berini, Investigating the accessibility of voice
[10] F. Pitta, T. Troosters, V. S. Probst, M. A. Spruit, assistants with impaired users: Mixed methods
M. Decramer, R. Gosselink, Quantifying physical study, Journal of Medical Internet Research
activity in daily life with questionnaires and mo- 22 (2020) e18431. URL: https://dx.doi.org/10.2196/
tion sensors in copd, Eur Respir J 27 (2006) 1040–55. 18431. doi:1 0 . 2 1 9 6 / 1 8 4 3 1 .
doi:1 0 . 1 1 8 3 / 0 9 0 3 1 9 3 6 . 0 6 . 0 0 0 6 4 1 0 5 . [19] E. Sezgin, L. K. Militello, Y. Huang, S. Lin, A scop-
[11] J. Kowalski, A. Jaskulska, K. Skorupska, K. Abram- ing review of patient-facing, behavioral health in-
czuk, C. Biele, W. Kopeć, K. Marasek, Older terventions with voice assistant technology tar-
adults and voice interaction: A pilot study with geting self-management and healthy lifestyle be-
google home, in: Extended Abstracts of the 2019 haviors, Translational Behavioral Medicine 10
CHI Conference on Human Factors in Comput- (2020) 606–628. URL: https://dx.doi.org/10.1093/
ing Systems, CHI EA ’19, Association for Com- tbm/ibz141. doi:1 0 . 1 0 9 3 / t b m / i b z 1 4 1 .
puting Machinery, New York, NY, USA, 2019, p. [20] M. L. B. Pulido, J. B. A. Hernández, M. A. F.
1–6. URL: https://doi.org/10.1145/3290607.3312973. Ballester, C. M. T. González, J. Mekyska,
doi:1 0 . 1 1 4 5 / 3 2 9 0 6 0 7 . 3 3 1 2 9 7 3 . Z. Smékal, Alzheimer’s disease and auto-
[12] W. Kopeć, R. Nielek, A. Wierzbicki, Guide- matic speech analysis: A review, Expert Systems
lines towards better participation of older adults with Applications 150 (2020) 113213. URL:
in software development processes using a new https://dx.doi.org/10.1016/j.eswa.2020.113213.
spiral method and participatory approach, in: doi:1 0 . 1 0 1 6 / j . e s w a . 2 0 2 0 . 1 1 3 2 1 3 .
Proceedings of the 11th International Workshop [21] J. Wang, L. Zhang, T. Liu, W. Pan, B. Hu, T. Zhu,
on Cooperative and Human Aspects of Soft- Acoustic differences between healthy and depressed
ware Engineering, CHASE ’18, Association for people: a cross-situation study, BMC Psychiatry 19
Computing Machinery, New York, NY, USA, (2019). URL: https://dx.doi.org/10.1186/s12888-019-
2018, p. 49–56. URL: https://doi.org/10.1145/ 2300-7. doi:1 0 . 1 1 8 6 / s 1 2 8 8 8 - 0 1 9 - 2 3 0 0 - 7 .
3195836.3195840. doi:1 0 . 1 1 4 5 / 3 1 9 5 8 3 6 . 3 1 9 5 8 4 0 . [22] N. B. Mota, M. Copelli, S. Ribeiro, Thought disor-
[13] A. Aula, Learning to use Computers at a Later Age, der measured as random speech structure classifies
in: HCI and the Older Population, Univ. of Glasgow, negative symptoms and schizophrenia diagnosis
Leeds, UK, 2004. 6 months in advance, npj Schizophrenia 3 (2017).
[14] D. T. Duncan, W. C. Goedel, J. H. Williams, B. Elbel, URL: https://dx.doi.org/10.1038/s41537-017-0019-3.
Acceptability of smartphone text- and voice-based doi:1 0 . 1 0 3 8 / s 4 1 5 3 7 - 0 1 7 - 0 0 1 9 - 3 .
ecological momentary assessment (ema) methods [23] V. Nathan, K. Vatanparvar, M. M. Rahman, E. Ne-
mati, J. Kuang, Assessment of chronic pulmonary //dx.doi.org/10.1089/cyber.2018.0110. doi:1 0 . 1 0 8 9 /
disease patients using biomarkers from natural cyber.2018.0110.
speech recorded by mobile devices, in: 2019 IEEE [33] C. M. Schwartzman, J. F. Boswell, A narrative re-
16th International Conference on Wearable and Im- view of alliance formation and outcome in text-
plantable Body Sensor Networks (BSN), 2019, pp. based telepsychotherapy, Practice Innovations 5
1–4. doi:1 0 . 1 1 0 9 / B S N . 2 0 1 9 . 8 7 7 1 0 4 3 . (2020) 128–142. doi:1 0 . 1 0 3 7 / p r i 0 0 0 0 1 2 0 .
[24] C. Pedone, D. Lelli, Systematic review of telemoni- [34] T. Bickmore, A. Gruber, R. Picard, Establishing the
toring in copd: an update, Pneumonol Alergol Pol computer–patient working alliance in automated
83 (2015) 476–84. doi:1 0 . 5 6 0 3 / P i A P . 2 0 1 5 . 0 0 7 7 . health behavior change interventions, Patient Edu-
[25] J. E. McDowell, S. McClean, F. FitzGibbon, S. Tate, cation and Counseling 59 (2005) 21–30. URL: https:
A randomised clinical trial of the effectiveness of //dx.doi.org/10.1016/j.pec.2004.09.008. doi:1 0 . 1 0 1 6 /
home-based health care with telemonitoring in pa- j.pec.2004.09.008.
tients with copd, J Telemed Telecare 21 (2015) 80–7. [35] A. O. Horvath, L. Luborsky, The role of the
doi:1 0 . 1 1 7 7 / 1 3 5 7 6 3 3 x 1 4 5 6 6 5 7 5 . therapeutic alliance in psychotherapy, J Consult
[26] F. Rassouli, F. Baty, D. Stolz, W. Albrich, Clin Psychol 61 (1993) 561–73. doi:1 0 . 1 0 3 7 / / 0 0 2 2 -
M. Tamm, S. Widmer, M. Brutsche, Longi- 006x.61.4.561.
tudinal change of copd assessment test (cat) [36] D. J. Martin, J. P. Garske, M. Katherine Davis,
in a telehealthcare cohort is associated with Relation of the therapeutic alliance with out-
exacerbation risk, International Journal of come and other variables: A meta-analytic re-
Chronic Obstructive Pulmonary Disease Volume 12 view, Journal of Consulting and Clinical Psychology
(2017) 3103–3109. URL: https://dx.doi.org/10.2147/ 68 (2000) 438–450. URL: /record/2000-15699-009.
copd.s141646. doi:1 0 . 2 1 4 7 / c o p d . s 1 4 1 6 4 6 . doi:1 0 . 1 0 3 7 / 0 0 2 2 - 0 0 6 X . 6 8 . 3 . 4 3 8 .
[27] C. Nass, J. Steuer, E. R. Tauber, Computers are so- [37] N. Mead, P. Bower, Patient-centred consultations
cial actors, in: Proceedings of the SIGCHI Confer- and outcomes in primary care: a review of the
ence on Human Factors in Computing Systems, CHI literature, Patient Educ Couns 48 (2002) 51–61.
’94, Association for Computing Machinery, New doi:1 0 . 1 0 1 6 / s 0 7 3 8 - 3 9 9 1 ( 0 2 ) 0 0 0 9 9 - x .
York, NY, USA, 1994, p. 72–78. URL: https://doi.org/ [38] U. U. Rehman, D. J. Chang, Y. Jung, U. Akhtar,
10.1145/191666.191703. doi:1 0 . 1 1 4 5 / 1 9 1 6 6 6 . 1 9 1 7 0 3 . M. A. Razzaq, S. Lee, Medical instructed real-
[28] B. Fogg, Persuasive computers: Perspectives and time assistant for patient with glaucoma and di-
research directions, in: Proceedings of the SIGCHI abetic conditions, Applied Sciences 10 (2020)
Conference on Human Factors in Computing Sys- 2216. URL: https://dx.doi.org/10.3390/app10072216.
tems, CHI ’98, ACM Press/Addison-Wesley Publish- doi:1 0 . 3 3 9 0 / a p p 1 0 0 7 2 2 1 6 .
ing Co., USA, 1998, p. 225–232. URL: https://doi.org/ [39] D. Ireland, C. Atay, J. Liddle, D. Bradford, H. Lee,
10.1145/274644.274677. doi:1 0 . 1 1 4 5 / 2 7 4 6 4 4 . 2 7 4 6 7 7 . O. Rushin, T. Mullins, D. Angus, J. Wiles, S. McBride,
[29] E. Cho, Hey google, can i ask you something in pri- A. Vogel, Hello harlie: Enabling speech moni-
vate?, in: Proceedings of the 2019 CHI Conference toring through chat-bot conversations, in: Stud-
on Human Factors in Computing Systems, CHI ’19, ies in Health Technology and Informatics, volume
Association for Computing Machinery, New York, 227, IOS Press, 2016, pp. 55–60. doi:1 0 . 3 2 3 3 / 9 7 8 - 1 -
NY, USA, 2019, p. 1–9. URL: https://doi.org/10.1145/ 61499-666-8-55.
3290605.3300488. doi:1 0 . 1 1 4 5 / 3 2 9 0 6 0 5 . 3 3 0 0 4 8 8 . [40] D. Kadariya, R. Venkataramanan, H. Y. Yip,
[30] K. Kim, N. Norouzi, T. Losekamp, G. Bruder, M. Kalra, K. Thirunarayanan, A. Sheth, kbot:
M. Anderson, G. Welch, Effects of patient Knowledge-enabled personalized chatbot for
care assistant embodiment and computer me- asthma self-management, IEEE, ???? URL:
diation on user experience, IEEE, ???? URL: https://dx.doi.org/10.1109/smartcomp.2019.00043.
https://dx.doi.org/10.1109/aivr46125.2019.00013. doi:1 0 . 1 1 0 9 / s m a r t c o m p . 2 0 1 9 . 0 0 0 4 3 .
doi:1 0 . 1 1 0 9 / a i v r 4 6 1 2 5 . 2 0 1 9 . 0 0 0 1 3 . [41] F. Rassouli, M. Pfister, S. Widmer, F. Baty, B. Burger,
[31] A. Pradhan, A. Lazar, L. Findlater, Use of intelligent M. H. Brutsche, Telehealthcare for chronic obstruc-
voice assistants by older adults with low technol- tive pulmonary disease in switzerland is feasible
ogy use, ACM Transactions on Computer-Human and appreciated by patients, Respiration 92 (2016)
Interaction 27 (2020) 1–27. URL: https://dx.doi.org/ 107–13. doi:1 0 . 1 1 5 9 / 0 0 0 4 4 8 3 7 7 .
10.1145/3373759. doi:1 0 . 1 1 4 5 / 3 3 7 3 7 5 9 . [42] F. D. Davis, Perceived usefulness, perceived ease
[32] B. Liu, S. S. Sundar, Should machines express sym- of use, and user acceptance of information tech-
pathy and empathy? experiments with a health nology, MIS Quarterly 13 (1989) 319. URL: https:
advice chatbot, Cyberpsychology, Behavior, and //dx.doi.org/10.2307/249008. doi:1 0 . 2 3 0 7 / 2 4 9 0 0 8 .
Social Networking 21 (2018) 625–636. URL: https: [43] F. D. Davis, R. P. Bagozzi, P. R. Warshaw, Extrinsic
and intrinsic motivation to use computers in the
workplace 1, Journal of applied social psychology
22 (1992) 1111–1132.
[44] E. M. Rogers, Diffusion of Innovations, 5th ed., Free
Press, 2003.
[45] Aiva Health, Virtual health assistant | aiva health,
2020. URL: https://www.aivahealth.com/.
[46] Mayo Clinic, Skills from mayo clinic, 2020. URL:
https://www.mayoclinic.org/voice/apps.
[47] M. G. Crooks, A. den Brinker, Y. Hayman, J. D.
Williamson, A. Innes, C. E. Wright, P. Hill, A. H.
Morice, Continuous cough monitoring using
ambient sound recording during convalescence
from a copd exacerbation, Lung 195 (2017)
289–294. URL: https://doi.org/10.1007/s00408-017-
9996-2. doi:1 0 . 1 0 0 7 / s 0 0 4 0 8 - 0 1 7 - 9 9 9 6 - 2 .
[48] F. Barata, K. Kipfer, M. Weber, P. Tinschert,
E. Fleisch, T. Kowatsch, Towards device-agnostic
mobile cough detection with convolutional neural
networks, in: 2019 IEEE International Conference
on Healthcare Informatics (ICHI), 2019, pp. 1–11.
doi:1 0 . 1 1 0 9 / I C H I . 2 0 1 9 . 8 9 0 4 5 5 4 .
[49] F. Barata, P. Tinschert, F. Rassouli, C. Steurer-Stey,
E. Fleisch, M. A. Puhan, M. Brutsche, D. Kotz,
T. Kowatsch, Automatic recognition, segmentation,
and sex assignment of nocturnal asthmatic coughs
and cough epochs in smartphone audio record-
ings: Observational field study, Journal of Med-
ical Internet Research 22 (2020) e18082. URL: https:
//dx.doi.org/10.2196/18082. doi:1 0 . 2 1 9 6 / 1 8 0 8 2 .