=Paper=
{{Paper
|id=Vol-3124/paper13
|storemode=property
|title=Techno-Spiritual Engagement: Mechanisms for Improving Uptake of mHealth Apps Designed for Church Members
|pdfUrl=https://ceur-ws.org/Vol-3124/paper13.pdf
|volume=Vol-3124
|authors=Hye Sun Yun,Shou Zhou,Everlyne Kimani,Stefan Olafsson,Teresa K. O'Leary,Dhaval Parmar,Jessica Hoffman,Stephen Intille,Michael Paasche-Orlow,Timothy Bickmore
|dblpUrl=https://dblp.org/rec/conf/iui/YunZKOOPHIPB22
}}
==Techno-Spiritual Engagement: Mechanisms for Improving Uptake of mHealth Apps Designed for Church Members==
https://ceur-ws.org/Vol-3124/paper13.pdf
Techno-spiritual Engagement: Mechanisms for Improving
Uptake of mHealth Apps Designed for Church Members
Hye Sun Yun1 , Shuo Zhou1 , Everlyne Kimani1 , Stefan Olafsson1 , Teresa K. O’Leary1 ,
Dhaval Parmar1 , Jessica Hoffman1 , Stephen Intille1 , Michael Paasche-Orlow2,3 and
Timothy Bickmore1
1
Northeastern University, Boston, MA, USA
2
Boston University School of Medicine, Boston, MA, USA
3
Boston Medical Center, Boston, MA, USA
Abstract
Keeping users engaged with mHealth applications is important but difficult to achieve. We describe the development of a
smartphone-based application designed to promote health and wellness in church communities, along with mechanisms
explicitly designed to maintain engagement. We evaluated religiously tailored techno-spiritual engagement mechanisms,
including a prayer posting wall, pastor announcements, an embodied conversational agent for dialogue-based scriptural
reflections and health coaching, and tailored push notifications. We conducted a four-week pilot study with 25 participants
from two churches, measuring high levels of participant acceptance and satisfaction with all features of the application.
Engagement with the app was higher for users considered to be more religious and correlated with the number of notifications
received. Our findings demonstrate that our tailored mechanisms can increase engagement with an mHealth app.
Keywords
engagement, tailoring, field study, church communities, mHealth,
1. Introduction number of users who complete an intervention [6]. In our
work, we focus on the number of voluntary interactions
Smartphones provide a useful platform for health and by users during an intervention period.
wellness interventions given their ubiquity and accessi- We explore a range of strategies for maintaining user
bility [1]. However, the use of mobile health applications engagement with an mHealth app designed specifically
(mHealth apps) typically decreases over time as users for African American church communities. These com-
disengage and lose interest [2]. Many factors may lead munities typically have strong social support networks,
to loss of engagement, such as poor interface design per- common religious and spiritual practices, shared religious
ceived to be too simple and generic [3]. These design beliefs and attitudes, and a predominate racial identity,
problems may ultimately hinder the user from reaping all of which can be leveraged to tailor the mHealth in-
the health benefits the application would otherwise pro- tervention, to increase its efficacy [7] and to maintain
vide [4]. engagement with the app over time. In particular, we
Long-term engagement with digital health behavior focus on two kinds of engagement mechanisms: 1) the
change interventions, in particular, has been shown to incorporation of non-health related, techno-spiritual [8]
have significant positive effects on health outcomes [5]. functions, both social and individual, designed specifi-
There are several related measures of long-term user cally for this community; and 2) the inclusion of a virtual
engagement with a system in longitudinal interventions, agent health counselor that has both its appearance and
such as the number of voluntary interactions that users language tailored for the church community. The virtual
choose to have over a given time period, the length of coach is an embodied conversational agent (ECA), an
time they adhere to the system recommendations, or the animated character that simulates face-to-face conver-
sation using nonverbal behaviors such as facial displays
Joint Proceedings of the ACM IUI Workshops 2022, March 2022,
Helsinki, Finland of empathy along with her speech [9]. We conducted a
$ yun.hy@northeastern.edu (H. S. Yun); four-week field study to assess how well the engagement
zhou.sh@northeastern.edu (S. Zhou); kimani.e@northeastern.edu mechanisms worked.
(E. Kimani); stefanola@ru.is (S. Olafsson);
oleary.t@northeastern.edu (T. K. O’Leary);
d.parmar@northeastern.edu (D. Parmar); 2. Related Work
j.hoffman@northeastern.edu (J. Hoffman);
s.intille@northeastern.edu (S. Intille); mpo@bu.edu Various strategies for promoting engagement with mobile
(M. Paasche-Orlow); t.bickmore@northeastern.edu (T. Bickmore)
© 2022 Copyright for this paper by its authors. Use permitted under Creative health (mHealth) systems have been explored. These
Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org) strategies include social support [10, 11] and reminders
Figure 1: Screenshots of the mobile app: a) the virtual agent Clara; b) the home screen; c) the Pastor Announcements; and d)
the Prayer Wall.
[12, 13]. This section explores research specifically on theBased on these participatory design sessions, we de-
use and effectiveness of these strategies for promoting veloped an app with multiple features, including an ECA
engagement with mHealth systems. named Clara (Figure 1a). When the app is launched for
Studies have shown that mobile applications for pro- the first time, Clara introduces herself to the user and
moting healthy behaviors that implement features for provides an overview of the features and intervention
social support can have positive effects on health out- content within the app. On subsequent launches, the
comes [10, 11]. For example, a formative evaluation of app shows a menu of options for quick access to its core
a mobile cardiovascular health app designed in partner- features (Figure 1b). One of the options initiates a con-
ship with church-going African Americans found that a versation with Clara. In the first session of the day, Clara
forum-style social networking feature was supportive, en-greets the user and engages in rapport building with so-
joyable, encouraging, and motivating to maintain healthy cial chat on topics ranging from the weather to movies
behaviors, such as diet and physical activity [14]. and sharing important events that have occurred since
Push notifications have also been used in mHealth apps
the last time they spoke such as their church pastor post-
to increase engagement. Research has shown that users ing an announcement. Finally, Clara presents a list of
who received push notifications have better health out- topics that users can choose to talk about, such as exer-
comes or higher engagement with mHealth apps overall cise, nutrition, or scriptural meditation.
[2] than those not receiving any notifications in vari- The mHealth functions of the app promote nutrition,
ous mHealth contexts, such as mental disorder treatment including fruit and vegetable consumption and healthy
[12] or weight loss maintenance [13]. Based on previ- hydration, and physical activity. These are primarily
ous research, components such as notification content provided by coaching from the ECA following the stages-
can be important factors to influence responsiveness to of-change model as a theoretical framework [20], with
notifications [15]. Bidargaddi et al. [16] showed that no-
goal setting and behavior logging for users in later stages
tifications with tailored health suggestions improved theof change. Several of the topics related to healthy living
overall likelihood of users interacting with an mHealth are designed to provide users with the tools they need
app while tailored insights from self-monitored data led to feel motivated to improve their health, to set goals,
to greater engagement amongst frequent app users and and to keep track of their progress. Each topic has an
showed the importance of tailoring content to increase introduction where Clara provides information on the
engagement. topic at hand, for example, the current recommendations
for exercise or fruit consumption, as well as getting to
know the user’s current engagement in these health-
3. Application Design related behaviors. Clara encourages the user to engage
in conversation about health behavior change by asking
In order to successfully implement the mHealth app in
them what they think about exercise or healthy eating,
our stakeholders’ communities, we conducted nine fo-
their reasons for change and why they are interested in
cus groups to better understand the participants’ church
change, how they would like Clara to help them, and the
communities and to gain insights into how to effectively
benefits of the health behavior for them personally. The
tailor the app [17, 18, 19].
conversations are tailored according to the user’s stage of
change, which indicates their intentions to change their
health behavior [20].
4. Design of Engagement
Mechanisms
Figure 2: The variants of push notification content to remind
To encourage members of church communities to regu- users to talk to Clara: a) first person voice; b) third person
larly use the mHealth app, we implemented six engage- voice; and c) third person voice with a Bible verse.
ment mechanisms that are community-based and reli-
giously tailored. These are the ECA, Prayer Center, Pas-
tor Announcements, Bible Story of the Day, Scriptural information and engage with them through regular up-
Meditation, and push notifications. dates (Figure 1c). Whenever a new pastor announcement
is received, the user receives a push notification if they
4.1. Embodied Conversational Agent are not currently using the app, and sees a numbered
(ECA) indicator in the main app screen showing the number of
new and unread announcements.
The ECA speaks using a synthesized voice and her di-
alogue is driven by a hierarchical task network-based
4.4. Bible Story of the Day & Scriptural
dialogue manager. Her non-verbal behavior is automati-
cally generated using BEAT [21]. We implemented a reli- Meditation
giously tailored social chat module and representations To incorporate scriptures as a way to engage users, we
of agent-led faith-based activities, as well as designing implemented the Bible Story of the Day and Scriptural
the agent dialogue to include both a community-situated Meditation features. When users select the option to
greeting (e.g., “Hello Sister Kathy!”) and farewell (e.g., talk to Clara, they are provided with an option to either
“Blessings to you. Take care!”). At the end of each con- hear a Bible story or engage in scriptural meditation
versation, Clara shares a scripture verse in the same way led by Clara. For the Bible Story of the Day, all users
that church members often end church-based activities. are presented with the same Bible story for a particular
day to encourage discussing the story with others in
4.2. Prayer Center their church. Clara provides a summary of the story and
allows the user to engage by providing various responses
We implemented a set of features collectively referred to the story. For the Scriptural Meditation, users have
to as the Prayer Center that enables users to connect the option to choose the Bible passage they would like
with their fellow congregants. The first feature is the to read and are guided by Clara through the process of
Prayer Wall, where users can post textual messages that reflection and meditation.
include requests for prayers or praise (e.g., expressions
of gratitude or joy) to share with other members of their
church (Figure 1d). The messages are displayed as cards, 4.5. Push Notifications
similar to posts on a forum, and can be updated, replied
The final engagement mechanism we designed is the
to, and added to one’s own private list of prayers. The
ability to send push notifications to users’ mobile devices.
second feature is a private Prayer List to remind users of
We tailored the push notifications to encourage users to
the prayers they intend to say for others, allowing use
engage with the health features or the other religiously
of the app as a tool to support private prayer practices.
tailored engagement mechanisms. We used Google’s
Users can add their own prayers to the list manually and
Firebase Cloud Messaging to send three different types of
remove them or add prayer requests and praise reports push notifications to both iOS and Android devices with
from the Prayer Wall. a server-based notification service and decision rules.
Reminder Notifications: To remind users to regularly
4.3. Pastor Announcements engage with Clara, we send check-in reminders to talk
to her if the user has not used the app or received other
To further engage app users with their church commu- notifications in the past 24 hours. We religiously tailored
nity, we developed a mechanism for the church pastors the push notifications by randomly assigning certain
to reach out to members in a one-way broadcast com- notifications to include a Bible verse. To further diversify
munication. These Pastor Announcements give church the content of these notifications, we randomly delivered
leaders the means to provide members with news and different variations of messages by framing some in first
person with an image of Clara and others in third person Technology literacy was assessed at the beginning of the
without an image. Figure 2 shows some examples of the study with the following questions: (1) “About how many
variations. texts and/or messages via applications do you send in a
Social Notifications: We also designed notifications day?” and (2) “Which applications are currently installed
to encourage users to engage with features that serve on your smartphone or tablet?” The religiosity and spiri-
social functions, such as the Pastor Announcements and tuality of the participants were assessed using the Brief
the Prayer Center. Users receive a notification when Multidimensional Measure of Religiousness/Spirituality
the church pastor posts an announcement and when (BMMRS) [22]. Specifically, the sub-scale measuring indi-
new prayers are posted on their church’s Prayer Wall in viduals’ private religious practices (five-item, eight-point
the app. Instead of receiving a notification every time a composite measure) was used as it aligned well with pre-
prayer request is posted, these notifications are combined dicting the use of the religiously tailored engagement
into two batches per day to avoid burdening users with mechanisms in our app. Assessments of health behav-
many notifications. Lastly, the app notifies the user when ior change were conducted with pre-post measures fol-
a member of the church engages with the user’s prayer lowing the Transtheoretical Model [20], including stage
post by either replying or adding the prayer to a private of change, decisional balance, and self efficacy for each
prayer list. health behavior the app addressed.
Health Notifications: Notifications related to the Satisfaction with each engagement mechanism and the
health behavior interventions were sent when users had overall app was assessed using a four-item, seven-point
set a physical activity or nutrition goal. Reminder noti- composite measure (Anchors 1=“not at all satisfied”, An-
fications were sent when they should check-in with the chor 7=“very satisfied”) per mechanism. The number of
ECA regarding their physical activity goal, while notifi- logins, sessions with the ECA, interactions with different
cations for nutrition goals included relevant tips for fruitfeatures, and push notifications were all derived from
and vegetable intake and hydration. The tips also address data logged by the system for each individual user.
user-reported barriers to goal attainment. Seventeen participant post-evaluation interviews were
audio recorded and transcribed by a professional tran-
scription service. We conducted a deductive analysis
5. Evaluation Study guided by sensitizing concepts which focused on par-
ticipant satisfaction with each engagement mechanism,
We evaluated the effectiveness of our religiously tailored
authenticity of the religious components, and participant
engagement strategies in keeping users engaged with
usage of the system. We used elements of grounded the-
our smartphone-based mHealth app in a single-group,
ory method, including open coding, selective coding, and
quasi-experimental field study. The duration of the study
memoing [23].
was four weeks (28 days), with continuous logging of
system usage and weekly self-report assessments of en-
gagement and satisfaction. The study was approved by 5.2. Participants
our institution’s IRB. A total of 25 participants from two churches were re-
Due to COVID-19 restrictions, participants were en- cruited for the study (14 from Church A and 11 from
rolled into the study via video conference sessions during Church B). Participants self-identified primarily as Black
which they provided consent, were guided on installation (19 Black, 2 Black & Hispanic, 2 Black & Native Amer-
and use of the app, and filled out baseline questionnaires. ican, 1 Hispanic/Latinx, 1 White). Eighteen of the par-
They were asked to use the app every day during the ticipants identified as women, 5 identified as men, 1 as
four-week study. After each of the first three weeks, elec- genderqueer, and 1 as non-binary/third gender. Partici-
tronic surveys were emailed to participants to measure pants were aged 19 to 75 (Mean=55, Median=62, SD=18).
their satisfaction with the app. All participants but one had at least some post-secondary
After four weeks, participants completed a final exit education.
questionnaire and were invited to participate in an op-
tional qualitative interview with a member of the re-
search team. Participants were paid a fixed amount once, 5.3. Results
at the end of the study; they were not paid based on how 5.3.1. Usage Statistics and Satisfaction
much they used the app.
During the four-week study period, participants (N=25)
used the app a total of 560 times with each participant log-
5.1. Measures & Data Collection ging in to the app an average of 22 times (Min=4, Max=54,
We assessed user engagement and satisfaction using SD=13.3). They logged in to the app, on average, half of
self-reported measures and system-logged usage metrics. the 28 study days. Participants posted 82 prayer requests,
Table 1
Mean, minimum, maximum, and standard deviation of user sessions per user and mean satisfaction scores for the app overall
and for each engagement feature, respectively. Mean satisfaction scores were calculated based on four-item composite scales.
Anchors: 1=“not at all satisfied” to 7=“very satisfied”. One-sample t-tests demonstrating scores significantly different from
neutral of 4.
Feature Mean Minimum Maximum SD Mean Satisfaction Score p-value
Application 22.4 4 54 13.3 5.2 p < 0.001*
ECA (Clara) 8.2 0 31 8.3 5.1 p < 0.01*
Prayer Center 6.8 0 27 7.0 5.4 p < 0.001*
Pastor Announcements 1.4 1 2 0.5 5.1 p < 0.001*
Scriptural Meditation 3.4 0 17 4.9 6.0 p < 0.001*
Bible Story of the Day 2.0 0 13 3.0 6.0 p < 0.001*
added 43 prayer requests to private prayer lists, and wrote Clara resonated with them while younger participants
45 replies to prayer requests. Participants in Church A did not necessarily view Clara to be very relatable.
each received 1 pastor announcement, whereas Church Bible stories told by Clara were viewed as calming,
B received 2 pastor announcements. Each participant en- relaxing, and reflective. P7 stated, “Clara gave me room
gaged with the ECA 8 times on average, comprising 36.6% to think.” They were seen as accurate and increased
of their sessions with the app. During their sessions with participants’ confidence in Clara more generally. Satis-
the ECA, they chose the topic of Scriptural Meditation 84 faction with the Bible Story of the Day was positively
times, Bible Story of the Day 50 times, Physical Activity related with satisfaction of Clara (rho=0.479, p<0.05). In
26 times, and Nutrition 25 times. Furthermore, partici- addition, perception by the participants of Clara being
pants engaged with the health behavior tracking features religious, spiritual, and of faith was positively correlated
(e.g., tracking of physical activity, hydration, etc.) a total
with their satisfaction of the Scriptural Meditation feature
of 801 times, with each participant, on average, using (rho=0.774, p<0.001). However, participants expressed
the feature 32 times. Overall, participants were satisfied wanting to see more scriptures related to different topics
with the app features, as well as the overall app, scoring or categories so that they could easily find personally tai-
significantly higher than a neutral rating for all features lored scriptures for what they were feeling in any given
(Table 1). moment. Some participants felt the stories were limited
The ECA was the most-used part of the app (Table 1). and child-like and would like some tailoring based on
During the post-study interview, participants reflected level-of-knowledge.
on their perceptions of Clara and her role. The second-most-used part of the app was the Prayer
Center (Table 1). In general, the Prayer Center increased
P7: It may seem like a script because opportunities for the religious practice of supporting the
it...she was programmed to do this, but it community in prayer, because participants were able to
still felt good, because it just felt genuine, interact with the prayer wall anytime and anywhere. It
because she asked so many questions to also served as a memory aid because participants were
kinda gauge you, kinda like, where are able to use the prayer list to keep track of their prayer
you at right now, how can I help to get concerns. P2 said, “I don’t have to remember, what I want
you in that direction type of thing. That’s to pray for is right there.” Participants stated that initially
the feeling that I liked a lot. The feeling they were interested in seeing who from the church was
like, god man, I appreciate you. using the Prayer Center and wanted to connect with other
members through the feature. All participants reported
One participant viewed Clara as a member of the church
wanting more of a critical mass and wanting a notification
who took part in the culturally-situated practices: “Clara
system that would remind them to pray. P3 said, “It’s nice
felt like home. Even if we are laughing and rolling on the
to be connected to the church, especially right now. I just
floor. We still end in prayer.” P11 explored how details of
wish more members had been there.”
the interaction, e.g., Clara ending the conversation with
For health features such as setting health goals with
scripture, or Clara discussing Bible stories, mirrored how
Clara or tracking health behaviors, we observed that
physical church events were run in her church. Partici-
22 out of the 25 participants used at least one health
pants said that Clara was able to gain their trust by not
feature. Among the 22 participants who used the health
only signaling values but being factually accurate, espe-
features, users who rated high in the private religious
cially around health topics. Furthermore, participants
practices sub-scale of the BMMRS had higher usage of
who were retired or near retirement age expressed how
Table 2 For participants who received health tip notifications,
Top 5 notification types leading to the most app use. Click- they appreciated them overall. However, participants
through rate is the ratio of number of push notifications found it difficult to refer back to the tips as they disap-
clicked/tapped over number of push notifications received. peared after being viewed, and some were not necessar-
Push Notification Type Click-through Rate
ily sent during a time participants could actively act on
them. Participants also suggested other types of health
Social: Pastor announcement 31.8% tips such as congratulatory messages that would allow
Reminder: Check-in with the ECA 31.3% them to know what goal they had reached (P3), and ac-
Health: Physical activity goal 27.3%
countability notifications to help get them to their goals
check-in
Social: Prayer added to prayer list 25.0% (P2).
Social: Prayer request reply 18.4%
5.3.3. Engagement: Usage Over Time
To explore which factors led to the most longitudinal use
health features within the app (mean=50.7) compared to of the app, we developed a multilevel model, to account
those who rated low (mean=23.3), based on a two-sample for individual differences. The intraclass correlation coef-
t-test (p<0.05). Private religious practices ratings were ficient (ICC) was calculated to be 15.4% for daily number
categorized into high and low based on the median score. of logins, indicating that the majority of the variance
existed within a user. The best fitting model was se-
5.3.2. Push Notifications lected based on Akaike information criterion (AIC) and
Bayesian information criterion (BIC) values and model
For the four weeks of the study, a total of 635 push no-
interpretability. The number of logins on any given day
tifications were sent and 17.5% (N=111) of those were
was modeled as a linear pattern of change, with study
clicked on by participants. Each participant received
day, daily number of notifications received, and partici-
a mean of 25.4 notifications in total (Min=15, Max=48,
pants’ religiosity based on the private religious practices
SD=8.5) and, on average, clicked on 17.2% of those no-
sub-scale from the BMMRS [22] as predictors. For the
tifications (Mean=4.4, Min=0, Max=23, SD=6.3). Push
selected model, intercept and slope for study day were
notifications that promoted engagement with Pastor An-
modeled as random effects to capture individual differ-
nouncement, the Prayer Center, and talking to the ECA,
ences at baseline and over time. Table 3 shows the fixed
led to the most app use (Table 2). Notifications alert-
effects parameter estimates of the selected model for pre-
ing users of new pastor announcements had the highest
dicting daily number of logins. The final equation is:
click-through rate (=31.8%).
Participants were satisfied with the notifications they
received, scoring significantly above neutral on a satisfac- Daily # Of Logins = 0.696 − 0.039 * SD
tion rating scale (Mean=5.3, seven-point composite scale
+ 0.164 * N + 0.117 * R
with 4=neutral, parametric test, p<0.001). Participants
were also asked about their perceived pleasantness of where SD is the study day, N is the daily number of noti-
notifications (single-item scale, Anchor 1=“Unpleasant”, fications, and R is the private religious practices score.
Anchor 7=“Pleasant”). The ratings were significantly Participants’ religiosity ratings were centered at the
different from neutral (=4), with a median score of 5 (one- lowest score (coded as 0-7). Study day was centered at
sample Wilcoxon signed rank test, p<0.01), indicating the first day (day 0). The intercept can be interpreted
they felt receiving notifications was a mostly pleasant as the estimated number of logins on the first day, for a
experience. participant who received no notification on that day, and
During the post-study interviews, participants found
the reminder notifications helpful.
Table 3
P7: I did get some notifications from time- Fixed parameter estimates of the selected model for predicting
to-time since there were times I would daily number of logins.
forget it, but I would get a notification
from Clara, and I’m like, you know what, Parameter Value Std. Error p-value
I’m sorry shorty, forgot, hold on! Intercept 0.696 0.276 p < 0.05*
Study Day -0.039 0.007 p < 0.001*
Also participants found notifications related to the Prayer Daily Number of Notifi- 0.164 0.039 p < 0.001*
Center and Pastor Announcements bringing them back cations Received
to the app because they wanted to see what was going Private Religious Prac- 0.117 0.052 p < 0.05*
on. tices Score
rated the lowest on the private religious practices sub- We did not find that the use of the app led to signif-
scale. Based on the model, the number of notifications icant pre-post improvements in health attitudes or be-
received on a given day has a positive effect on the out- haviors, likely due to the short duration of the study and
come, as with 1 unit increase, the outcome increases by a small convenience sample. We plan to evaluate the
0.164 (p<0.001, Table 3). Study day has a negative effect effectiveness of our app in improving health behaviors
on the outcome, as usage decreases slightly over time. through future studies and used this pilot study to mainly
Religiosity has a positive effect on daily number of logins, evaluate the engagement mechanisms. Also, due to the
as with 1 unit increase on the private religious practices participants of our pilot study being financially compen-
sub-scale, the outcome increases by 0.117 (p<0.05, Table sated, we can expect some level of response bias where
3). engagement and satisfaction with our app can partly be
Using Pearson’s r correlation tests, we observed that associated with being compensated rather than the en-
participants with lower technology literacy had higher gagement mechanisms. In addition, we did not explore
private religious practices scores (r=-0.477, p<0.05). Sim- the duration of ECA sessions or scrolling through the
ilarly, those who were older had higher private reli- Prayer Wall as a measure of engagement. Measuring
gious practices scores (r=0.748, p<0.001). These findings session duration in light of app suspension, exits, and
showed that participants who were older or used fewer interrupts is very error-prone, and we felt that these data
smartphone-based messaging technologies, were more were too noisy to warrant analysis. Finally, this pilot
active in their private religious practices and logged into study lacked a control condition, and a series of studies
the app more often when notifications were sent to them. to systematically evaluate each engagement mechanism
relative to a control is ultimately needed.
5.3.4. Health Behavior Change Outcomes
We found no significant pre-post changes in health atti- Acknowledgments
tude or health behavior measures in our pilot study.
This material is based upon work supported by the Na-
tional Science Foundation under Grant No.1831755. Any
6. Conclusions and Limitations opinions, findings, and conclusions or recommendations
expressed in this material are those of the author(s) and
Maintaining user retention with mobile interventions is do not necessarily reflect the views of the National Sci-
essential for affecting longitudinal outcomes in health, ence Foundation.
education, and other application domains. Utilizing well-
tailored engagement mechanisms that meet the interests
and needs of a priority population may be useful in main- References
taining engagement with the app over time and more
effective at providing help and interventions to users. [1] T. W. Bickmore, E. Kimani, H. Trinh, A. Pusateri,
We explored a range of religiously tailored engagement M. K. Paasche-Orlow, J. W. Magnani, Manag-
strategies that can be used to motivate church community ing chronic conditions with a smartphone-based
members to interact with a smartphone-based mHealth conversational virtual agent, in: Proceedings
app. of the 18th International Conference on Intel-
We described and evaluated six religiously tailored ligent Virtual Agents, IVA ’18, Association for
engagement mechanisms in this pilot study: an ECA, Computing Machinery, New York, NY, USA, 2018,
Prayer Center, Pastor Announcements, Bible Story of the p. 119–124. URL: https://doi.org/10.1145/3267851.
Day, Scriptural Meditation, and push notifications. We 3267908. doi:10.1145/3267851.3267908.
demonstrated that push notifications were effective at [2] N. Bidargaddi, D. Almirall, S. Murphy, I. Nahum-
driving the use of the app, with the number of notifica- Shani, M. Kovalcik, T. Pituch, H. Maaieh, V. Strecher,
tions per day and the user’s private religious practices To prompt or not to prompt? a microran-
score being significant factors in predicting the number of domized trial of time-varying push notifications
user logins per day throughout the study. We found that to increase proximal engagement with a mo-
user satisfaction with all elements of the app was high bile health app, JMIR mHealth and uHealth 6
with several participants finding Clara to be relatable (2018) e10123. URL: https://mhealth.jmir.org/2018/
and having personal resonance with the religiosity, spir- 11/e10123. doi:10.2196/10123.
ituality, and role she exhibits. Our study demonstrated [3] T. McCurdie, S. Taneva, M. Casselman, M. Yeung,
that religiously tailored engagement mechanisms that C. McDaniel, W. Ho, J. Cafazzo, mhealth consumer
fulfill techno-spiritual functions can help religious users apps: the case for user-centered design, Biomedical
engage with an mHealth application. instrumentation & technology 46 (2012) 49.
[4] G. J. Norman, M. F. Zabinski, M. A. Adams, D. E. [14] L. C. Brewer, A. Kumbamu, C. Smith, S. Jenk-
Rosenberg, A. L. Yaroch, A. A. Atienza, A review ins, C. Jones, S. N. Hayes, L. Burke, L. A. Cooper,
of ehealth interventions for physical activity and C. A. Patten, A cardiovascular health and well-
dietary behavior change, American journal of pre- ness mobile health intervention among church-
ventive medicine 33 (2007) 336–345. going african americans: Formative evaluation
[5] S. Michie, L. Yardley, R. West, K. Patrick, F. Greaves, of the faith! app, JMIR Formative Research 4
Developing and evaluating digital interventions to (2020) e21450. URL: https://formative.jmir.org/2020/
promote behavior change in health and health care: 11/e21450. doi:10.2196/21450.
Recommendations resulting from an international [15] A. Mehrotra, M. Musolesi, R. Hendley, V. Pejovic,
workshop, J Med Internet Res 19 (2017) e232. Designing content-driven intelligent notification
[6] T. Bickmore, D. Schulman, L. Yin, Main- mechanisms for mobile applications, in: Proceed-
taining engagement in long-term interventions ings of the 2015 ACM International Joint Confer-
with relational agents, Applied Artificial In- ence on Pervasive and Ubiquitous Computing, Ubi-
telligence 24 (2010) 648–666. URL: https://doi. Comp ’15, Association for Computing Machinery,
org/10.1080/08839514.2010.492259. doi:10.1080/ New York, NY, USA, 2015, p. 813–824. URL: https:
08839514.2010.492259, pMID: 21318052. //doi.org/10.1145/2750858.2807544. doi:10.1145/
[7] R. P. Hawkins, M. Kreuter, K. Resnicow, M. Fishbein, 2750858.2807544.
A. Dijkstra, Understanding tailoring in communi- [16] N. Bidargaddi, T. Pituch, H. Maaieh, C. Short,
cating about health, Health education research 23 V. Strecher, Predicting which type of push noti-
(2008) 454–466. fication content motivates users to engage in a self-
[8] G. Bell, No more sms from jesus: Ubicomp, reli- monitoring app, Preventive medicine reports 11
gion and techno-spiritual practices, in: P. Dourish, (2018) 267–273.
A. Friday (Eds.), UbiComp 2006: Ubiquitous Com- [17] T. K. O’Leary, E. Stowell, E. Kimani, D. Parmar,
puting, Springer Berlin Heidelberg, Berlin, Heidel- S. Olafsson, J. Hoffman, A. G. Parker, M. K. Paasche-
berg, 2006, pp. 141–158. Orlow, T. Bickmore, Community-based cultural
[9] J. Cassell, Embodied conversational agents: rep- tailoring of virtual agents, in: Proceedings of the
resentation and intelligence in user interfaces, AI 20th ACM International Conference on Intelligent
magazine 22 (2001) 67–67. Virtual Agents, IVA ’20, Association for Computing
[10] J. M. Petersen, E. Kemps, L. K. Lewis, I. Prichard, Machinery, New York, NY, USA, 2020. URL: https:
Psychological mechanisms underlying the relation- //doi.org/10.1145/3383652.3423875. doi:10.1145/
ship between commercial physical activity app 3383652.3423875.
use and physical activity engagement, Psychology [18] E. Stowell, T. K. O’Leary, E. Kimani, M. K. Paasche-
of Sport and Exercise 51 (2020) 101719. URL: Orlow, T. Bickmore, A. G. Parker, Investigating
https://www.sciencedirect.com/science/article/pii/ opportunities for crowdsourcing in church-based
S1469029219308568. doi:https://doi.org/10. health interventions: A participatory design study,
1016/j.psychsport.2020.101719. in: Proceedings of the 2020 CHI Conference on
[11] F. D. Barber, Social support and physical activity Human Factors in Computing Systems, Association
engagement by cancer survivors., Clinical Journal for Computing Machinery, New York, NY, USA,
of Oncology Nursing 16 (2012) E84 – E98. URL: 2020, p. 1–12. URL: https://doi.org/10.1145/3313831.
http://libproxy.clemson.edu/login?url=https: 3376833.
//search.ebscohost.com/login.aspx?direct=true& [19] T. K. O’Leary, E. Stowell, J. A. Hoffman, M. Paasche-
db=a2h&AN=76169513. Orlow, T. Bickmore, A. G. Parker, Examining the
[12] F. Gravenhorst, A. Muaremi, J. Bardram, intersections of race, religion & community tech-
A. Grünerbl, O. Mayora, G. Wurzer, M. Frost, nologies: A photovoice study, in: Proceedings of the
V. Osmani, B. Arnrich, P. Lukowicz, et al., Mobile 2021 CHI Conference on Human Factors in Com-
phones as medical devices in mental disorder puting Systems, Association for Computing Ma-
treatment: an overview, Personal and Ubiquitous chinery, New York, NY, USA, 2021, p. 1–19. URL:
Computing 19 (2015) 335–353. https://doi.org/10.1145/3411764.3445418.
[13] A. Hernández-Reyes, F. Cámara-Martos, G. M. Re- [20] J. O. Prochaska, W. F. Velicer, The transtheoretical
cio, R. Molina-Luque, M. Romero-Saldaña, R. M. model of health behavior change, American Journal
Rojas, Push notifications from a mobile app to im- of Health Promotion 12 (1997) 38–48. URL: https:
prove the body composition of overweight or obese //doi.org/10.4278/0890-1171-12.1.38. doi:10.4278/
women: randomized controlled trial, JMIR mHealth 0890-1171-12.1.38, pMID: 10170434.
and uHealth 8 (2020) e13747. URL: https://mhealth. [21] J. Cassell, H. H. Vilhjálmsson, T. Bickmore, Beat:
jmir.org/2020/2/e13747. doi:10.2196/13747. the behavior expression animation toolkit, in:
H. Prendinger, M. Ishizuka (Eds.), Life-Like Char-
acters: Tools, Affective Functions, and Applica-
tions, Springer Berlin Heidelberg, Berlin, Hei-
delberg, 2004, pp. 163–185. URL: https://doi.
org/10.1007/978-3-662-08373-4_8. doi:10.1007/
978-3-662-08373-4_8.
[22] K. S. Masters, Brief Multidimensional Measure of
Religiousness/Spirituality (BMMRS), Springer New
York, New York, NY, 2013, pp. 267–269. URL: https:
//doi.org/10.1007/978-1-4419-1005-9_1577. doi:10.
1007/978-1-4419-1005-9_1577.
[23] J. M. Corbin, A. Strauss, Grounded theory research:
Procedures, canons, and evaluative criteria, Quali-
tative sociology 13 (1990) 3–21.