=Paper=
{{Paper
|id=Vol-2662/BCSS2020_paper5
|storemode=property
|title=Deconstructing Persuasive Strategies in Mental Health Apps Based on User Reviews using Natural Language Processing
|pdfUrl=https://ceur-ws.org/Vol-2662/BCSS2020_paper5.pdf
|volume=Vol-2662
|authors=Oladapo Oyebode,Rita Orji
|dblpUrl=https://dblp.org/rec/conf/persuasive/OyebodeO20b
}}
==Deconstructing Persuasive Strategies in Mental Health Apps Based on User Reviews using Natural Language Processing==
https://ceur-ws.org/Vol-2662/BCSS2020_paper5.pdf
Deconstructing Persuasive Strategies in Mental Health
Apps Based on User Reviews using Natural Language
Processing
Oladapo Oyebode*[0000-0002-5797-7790] and Rita Orji[0000-0001-6152-8034]
Faculty of Computer Science, Dalhousie University, Halifax NS B3H 4R2, Canada
oladapo.oyebode@dal.ca, rita.orji@dal.ca
Abstract. Text Mining is concerned with extracting interesting and significant
patterns or knowledge from unstructured text data. In this paper, we applied the
text mining approach using natural language processing (NLP) techniques, es-
pecially topic modelling (with automated topic labelling), in deconstructing the
persuasive strategies implemented or employed by 100 mental health apps
based on user reviews. We focus on the persuasive strategies in the primary task
support category of the Persuasive Systems Design (PSD) framework. We used
the Latent Dirichlet Allocation (LDA) topic modelling algorithm, in conjunc-
tion with semantic attributes, to achieve our goal. Our experimental results re-
vealed that self-monitoring is the most employed persuasive strategy. Finally,
we compare our findings with that obtained using manual coding method and
found significant similarities.
Keywords: Text mining, Persuasive strategy, Mobile apps, Mental health, User
reviews, Natural language processing, Topic modelling.
1 Introduction
Smartphone ownership continues to increase rapidly in both emerging and advanced
economies [1, 2]. Evidence shows that there are 3.06 billion smartphone users global-
ly in 2018 and this figure is expected to surpass 4 billion by 2025 [3]. As a result,
mobile applications (or apps) are proliferating and download rate is projected to keep
increasing steadily. For instance, 2.8 million and 2.2 million apps are available for
download on Google Play and App Store respectively [4] and the number of down-
loads on Google Play was 21.3 billion [5] as at 2019. Among these are mobile health
(mHealth) apps that leverage embedded sensors in smartphones and connected wear-
ables, as well as technology-assisted self-reporting, to deliver health interventions to
patients [6]. For instance, mental health apps can use global positioning system (GPS)
and accelerometer data to compute physical and spatial activity measures for detect-
ing depressive symptoms and anxiety [7, 8]. In addition, integration with wearable
sensors for real-time monitoring of heart rate, blood pressure, temperature, skin con-
ductance, etc. can help identify stress [9, 10]. This shows that mHealth apps have a
considerable potential to support early detection and treatment of medical conditions,
Copyright © 2020 for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
2
thereby promoting healthy lifestyles and behaviours in patients, including improved
mental health. In addition, mHealth apps designers utilize persuasive techniques that
influence patients to imbibe these health behaviours in their daily lives.
Over the years, previous research has conducted systematic reviews to identify per-
suasive strategies employed in mHealth apps [11–13]. The widely used approach is to
have expert reviewers download the apps and manually code them using the Persua-
sive Systems Design (PSD) framework [14] or behaviour change theories [15]. While
this approach may seem to be effective, it can be very costly or impracticable for
wider studies that involve hundreds or thousands of apps. To address this issue, we
applied the text mining approach using natural language processing (NLP) techniques
and topic modelling (with automated topic labelling) on app reviews (i.e., user re-
views or comments about apps) which is a more efficient and practicable, but less
costly, way of detecting the persuasive strategies implemented in large number of
mobile apps. Although research in this area is still emerging, it has the potential to
yield successful and reliable outcomes since text mining has been widely applied on
social media posts and online/app reviews to generate insights for addressing chal-
lenges (or supporting decision making) in many domains [16–18], including health
[19, 20]. Furthermore, user reviews of mobile technologies capture information about
functionality [21] which is useful in detecting the persuasive features present in apps.
Our methodology combines several well-known computational techniques as fol-
lows: (1) Extract user reviews for mental health apps on both Google Play and App
Store using Heedzy tool1 [22], and preprocess the data using NLP techniques to pre-
pare it for analysis; (2) Apply the Latent Dirichlet Allocation (LDA) topic modelling
algorithm on user reviews to detect main topics or themes; (3) Label each topic with
persuasive strategies (in the primary task support category of the PSD framework)
using semantic attributes depicting each strategy. Next, we compare our findings with
that obtained using manual coding method and found significant similarities. Finally,
we discuss the implication of our findings.
2 Related Work
Existing research has used the Persuasive Systems Design (PSD) framework or Be-
haviour Change Techniques (BCTs) to deconstruct the persuasive strategies employed
in mobile apps. For example, Alqahtani et al. [13] reviewed mental health apps to
identify persuasive strategies employed. Similarly, Chang et al. [23] and Oyebode et
al. [12] reviewed mental health apps and mHealth apps respectively using the PSD
framework, while Langrial et al. [24] reviewed apps for personal wellbeing using the
same framework. However, all the researchers coded the apps manually which is less
efficient, time-consuming, and prone to coding mistakes.
Al-Ramahi et al. [25] adopted a text mining approach to identify existing persua-
sive strategies and propose new ones. They applied the topic modelling technique
(using LDA algorithm) on online user reviews to extract main topics or themes de-
1
User reviews on both stores were extracted in May 2018 using the paid version of the tool
3
scribing the reviews. However, they manually labelled the topics with appropriate
persuasive strategies based on the words associated with each topic. This manual
labelling of topics is also less efficient and automated options should be considered.
To address these gaps, our work applies the text mining approach using NLP tech-
niques and topic modelling (with automated topic labelling) in deconstructing the
persuasive strategies (in the primary task support category of the PSD framework)
implemented or employed by mental health apps, thereby eliminating manual coding
of apps and manual labelling of topics. The automatic labelling was achieved using
semantic attributes that represent various persuasive strategies.
3 Methodology
The main goal of this paper is to identify persuasive strategies employed by mental
health apps based on user reviews. To achieve this, we used well-known computa-
tional techniques. As shown in Figure 1, the steps taken to achieve our goal are sum-
marized below.
1. We extracted user reviews for mental health apps on both Google Play and App
Store using Heedzy tool.
2. We preprocessed the data using NLP techniques to prepare it for analysis.
3. We classified each review as either positive, negative, or neutral using user rating-
based criteria.
4. We vectorized the user reviews (i.e., converting them into numeric form) using the
Term Frequency Inverse Document Frequency (TF-IDF) weighting technique.
5. We applied the Latent Dirichlet Allocation (LDA) algorithm on the vectorized user
reviews to detect main topics or themes.
6. We labelled each topic with persuasive strategies using semantic attributes depict-
ing each strategy.
Data Collection Data Preprocessing
(user reviews)
Data Vectorization Sentiment
(TF-IDF) Classification
Topic Labelling
Topic Modelling
Latent Dirichlet Allocation Persuasive Strategies
(LDA) algorithm
Semantic Attributes
Fig. 1. Approach for detecting persuasive strategies in user reviews
4
3.1 Data Collection
To identify eligible apps, we first performed a search on Google Play (for Android
apps) and App Store (for iOS apps) using various keywords including anxiety, stress,
depression, emotion, mental health, and mood. We got a total of 183 and 254 apps
from Google Play and Apple Store respectively as search results. Secondly, we ex-
cluded apps whose description do not relate to mental health, apps with less than five
user reviews, and non-English apps. Afterwards, we collected 101,715 user reviews
for 105 eligible apps using the Heedzy tool. If an app is published on both Google
Play and App Store, we merged the reviews on both platforms.
3.2 Data Preprocessing
Next, we applied the following preprocessing steps using NLP techniques to clean the
data and prepare it for analysis:
1. Convert words to lowercase
2. Reduce repeated characters (e.g. “goooood” becomes “good”)
3. Remove numbers
4. Expand contractions (e.g. “there's” becomes “there is”, etc.)
5. Replace slangs with English words using online slang dictionaries [26, 27] which
contain 5,434 entries combined
6. Remove punctuation, special characters, and extra whitespaces
7. Remove stopwords (e.g. the, an, will, shall, let, may, can, it, with, of, this, and, as,
etc.)
8. Lemmatize words using the WordNet Lemmatizer (which is part of the nltk module
of Python and uses WordNet [28] behind the scene) so that words can be converted
to their root form (e.g., “better” becomes “good”, while “statistics” becomes
“statistic”)
9. Remove duplicates
After data preprocessing, the total reviews reduced to 88,125 across 104 apps.
3.3 Sentiment Classification
Next, we classified each review as either positive, negative, or neutral sentiment po-
larity based on user ratings. On Google Play and App Store, users assign star ratings
to apps on a scale of 1 to 5 (where 1 star represents “very dissatisfied” and 5 stars
represents “very satisfied”). In our dataset, a total of 68,247 reviews have user ratings
(representing 77.4% of total reviews). Table 1 shows user ratings and the correspond-
ing sentiment polarity, as well as the total reviews for each rating. This approach has
been used by previous research, such as [21, 29], to determine the sentiment polarity
of app reviews.
5
Table 1. Criteria for sentiment classification based on user rating
Rating Description Number of reviews Polarity
5 Very satisfied 47994 Positive
4 Satisfied 9711 Positive
3 Okay 3491 Neutral
2 Dissatisfied 2237 Negative
1 Very dissatisfied 4814 Negative
Since feature request is one of the most common issue type in negative reviews [29,
30], we excluded them from our analysis. We retained only positive reviews
(n=57705) since they mostly reflect user opinions or experience about features and
strategies already implemented in the apps. The positive reviews are distributed
across 100 apps, as shown in Table 2.
Table 2. Summary of Positive Reviews
Total Positive Reviews Number of Apps
Less than 1000 89
1000 – 5000 9
Above 5000 2
3.4 Topic Modelling using Latent Dirichlet Allocation (LDA) Algorithm
To identify main topics or themes describing the reviews, we applied Latent Dirichlet
Allocation (LDA) which is a widely used and efficient topic modelling algorithm [31,
32]. We implemented the algorithm in Python programming language. Prior to apply-
ing the LDA, we vectorized the documents (i.e., reviews) using the popular Term
Frequency Inverse Document Frequency (TF-IDF) weighting scheme since it consid-
ers both frequency and relevance when assigning weight to words [33]. The idea is to
pass the vectorized reviews for each app to the LDA algorithm to generate topics.
The LDA algorithm returns top 𝐾 topics in the reviews, along with top 𝑁 words for
each topic (where K and N are set to 50 and 10 respectively). We set the value of 𝐾 to
50, based on previous research which shows that perplexity reduces with an increase
in number of topics but later converges to a fixed value (approximately 50) [25].
In summary, for each of the 100 apps, we retrieved top 50 topics and top 10 words
that describe each topic. The words corresponding to each topic will be used to de-
termine appropriate labels (i.e., persuasive strategies), as described in the next section.
3.5 Topic Labelling, Semantic Attributes, and Persuasive Strategies
The final step is to infer persuasive strategies from the words associated with the top-
ics, and then assign the persuasive strategies as topic labels. Instead of achieving this
manually, we employed the approach proposed by [34] which involves assessing the
existence of persuasive strategies using semantic attributes. We generated the seman-
tic attributes for each persuasive strategy in the primary task support category of the
6
PSD framework using the WordNet lexical database [28]. The attributes were verified
and validated by two persuasive technology experts for appropriateness. Table 3
shows the persuasive strategies and the corresponding semantic attributes.
Table 3. Persuasive Strategies and the corresponding Semantic Attributes
Persuasive Strategy Semantic Attributes
Personalization and Tailoring personalize, relevant, personalization, personal,
individual, profile, personality, personalise,
individualize, personify, relevance, suitable,
suit, fit, adjust, change, adjustable, edit, edita-
ble, amend, modify, flexible, control, customi-
zation, customize, customise, customizable,
customisable, modifiable, changeable, adapt,
adaptable, refine, alter, tailor, tailored
Self-monitoring track, statistics, statistic, measure, progress,
goal, history, view, display, activity, analysis,
record, monitor, graph, chart, log, logging, jour-
nal, diary, duration, speed, pace, time, insight,
insightful, recording, journaling
Reduction simple, simplify, automatically, automatic,
quickly, immediately, instantly, quick, instant,
immediate, simplicity, straightforward
Tunneling step-by-step, guide, stepwise, gradual, step,
instruction, procedure, procedural, process,
journey, stage, plan, guided, guidance
Simulation and Rehearsal simulate, virtual, imitate, visualization, sound,
audio, video, image, observe, simulator, effect,
game, animation, environment, voice, practice,
train, practise, learn, drill, rehearsal, intro, in-
troduction, rehearse
We developed a Python program to match the words corresponding to each topic with
the semantic attributes, and then label the topic with the appropriate persuasive strate-
gy (or strategies) based on the matching attribute(s).
4 Results
Based on our experimental results, the self-monitoring persuasive strategy emerged as
the most employed overall (n=92/100), followed by personalization and tailoring
(n=83/100) and simulation and rehearsal (n=81/100), as shown in Figure 2. However,
reduction (n=77/100) followed by tunneling (n=53/100) are the least employed strate-
gies.
7
Fig. 2. Persuasive strategies (x-axis) and the corresponding number of apps that employed each
strategy
Table 4 presents sample topics (out of the 5000 extracted topics) for some of the
apps including the top 10 words associated with each topic, the matching semantic
attributes, topic label (i.e., persuasive strategy (PS)), and sample user reviews.
Table 4. Sample topics (showing words associated with each topic), topic label (PS), matching
semantic attributes, and sample user reviews
Topic Words Topic Label Matching Sample user reviews App Name
(PS) Attribute(s)
personalization personality “It gives good reminders for
and tailoring a type A personality who
needs to learn to slow
down”2 [R37]
[good, personality, simulation and animation, “The animation which let me
magically, animation, rehearsal learn know when we breathe and 3 Minute
hold, application, let, when we hold works magi- Mindfulness
slow, type, learn] cally” [R42]
“Great little application to
learn deep breathing without
having to count…” [R50]
[helps, good, issue, self- diary “It has everything I need to Cognitive
diary, mindful, pre- monitoring keep an anxiety diary…” Diary CBT
sent, solve, job, read, [R290] Self-Help
useful]
[job, pain, med, track, self- track, “Awesome application Daylio
mood, statistic, appli- monitoring statistic beautiful designs and it is a
cation, use, great, great way to keep on track of
love] the moods!” [R253]
“An amazing application I
just loved the way it gives me
my emotional statistics…”
[R391]
2
User reviews are included verbatim throughout the paper, including spelling and grammatical mistakes.
8
4.1 Comparing our results with that of manual approach
Next, we compared our findings with the results obtained by Alqahtani et al. [13]
after manually coding the mental health apps using PSD framework and the Behav-
iour Change Techniques (BCTs). To aid comparison, we refer to our method as “au-
tomated approach”, and Alqhatani et al.’s method as “manual approach”. According
to both results, self-monitoring emerged as the most employed strategy in both auto-
mated and manual approaches. Similarly, personalization (called “personalization and
tailoring” in the automated approach) emerged as the second most employed persua-
sive strategy in both approaches. However, while reduction and tunneling are the least
employed strategies in the automated approach, rehearsal and reduction are the least
employed in the manual approach.
In summary, both approaches agreed that self-monitoring and personalization are
the top 2 (most employed) persuasive strategies in mental health apps. On the other
hand, both approaches slightly agreed on the least employed strategies which may be
due to human coding errors in the manual approach.
5 Discussion
In this paper, we applied the text mining approach in identifying persuasive strategies
employed by mental health apps based on user reviews. The results of our experiment
revealed that self-monitoring is the most employed strategy, closely followed by per-
sonalization and tailoring. This aligns with the results of a prior research that applied
the manual method of deconstructing persuasive strategies in mental health apps (see
Section 4.1). We discuss the implications of our findings in subsequent sub-sections.
5.1 Self-monitoring and Personalization as key strategies for mental health
interventions
Self-monitoring is an essential requirement for digital health interventions, especially
mHealth apps for emotional and mental wellbeing. For example, research shows that
apps for self-monitoring of mood (i.e., mood tracking) increase emotional self-
awareness which causes a reduction in depressive and anxiety symptoms [35–37]. In
addition, Hetrick et al. included the mood monitoring feature, as well as personalized
interventions, in their app design for young adults to aid self-treatment of depression
[38]. Thus, by employing both self-monitoring and personalization, developers can
deliver effective persuasive technological interventions that promote better health in
patients with mental health issues.
5.2 Text Mining as a useful approach for persuasive technology research
Text mining can be useful for research involving persuasive technologies. Most per-
suasive technology researchers still rely on empirical studies that require manual ef-
forts, and this technique usually limits the scope of research due to various factors
9
such as cost, study population size and diversity challenges, generalizability issues
due to limited data, etc. In addition, manual approaches are prone to errors. Text min-
ing algorithms usually leverage large textual datasets (such as social media data, user
reviews on app stores, open data, organizational data, etc.) for analysis. As a result,
their outputs can uncover valuable and useful insights that may be impossible to
achieve through other methods. Text mining has been applied in healthcare [39–41]
and other domains [42, 43] over the years, and can be exploited by PT researchers to
generate data-driven insights that inform the design, development, and evaluation of
persuasive systems. We demonstrated the applicability of text mining on mental
health app reviews from persuasiveness standpoint, but it can be extended to other
areas of interest.
5.3 User Reviews as a way of evaluating persuasive apps
Two of the limitations hindering the advancement of the field of persuasive technolo-
gy are the lack of study on the effectiveness of persuasive apps over a long-term and
the limited number of participants that are usually involved in the app evaluation [44,
45]. Most existing persuasive apps in the literature did not evaluate the effectiveness
of their apps, and even those that did, the evaluation is often in a controlled environ-
ment and for a short-term. This is mainly due to the cost of conducting a long-term
study in terms of time and other resources. Also, most evaluation results are not gen-
eralizable due to limited number of participants and their similarity. We argue that our
findings, which show the possibility of applying text mining on user reviews to un-
cover insights about the apps, suggest that user reviews could be a reliable and cost-
effective alternative for evaluating the effectiveness of persuasive apps both in short
and long-term. The findings are more generalizable considering the diversity of re-
viewers. Hence, public user reviews of apps and text mining are promising techniques
for evaluating the effectiveness of persuasive applications.
5.4 Tunneling and Reduction as potential strategies designers of mental
health apps should employ
Our findings revealed that reduction followed by tunneling are the least employed
strategies. This is surprising since mental health patients are often advised to avoid
stressful conditions, as well as complex tasks that may raise their stress levels and
worsen their health. Also, lack of guidance in mental health apps can impair concen-
tration and make users to be easily frustrated [46]. Therefore, designers of mental
health apps should consider their users by reducing tasks into simpler steps (reduc-
tion) and providing necessary guidance (tunneling) until they complete those tasks.
6 Limitations
First, user reviews or comments about each app reflect opinion about features the
users have interacted with or noticed. However, it is possible there are other features
10
not highlighted in the reviews since they may not be noticeable or personally relevant
for a user. As a result, certain persuasive strategies could be more obvious or preva-
lent than others. Second, the one-word semantic attributes used in our analysis may
not be exhaustive; hence, including phrasal attributes may help to boost coverage.
7 Conclusion and Future Work
In this paper, we applied the text mining approach using natural language processing
(NLP) techniques and topic modelling (with automated topic labelling) in decon-
structing the persuasive strategies (in the primary task support category of the PSD
framework) implemented or employed by mental health apps based on user reviews.
We used the Latent Dirichlet Allocation (LDA) algorithm and Semantic Attributes to
achieve our goal. Our experimental results revealed that self-monitoring is the most
employed persuasive strategy overall, followed by personalization and tailoring, and
simulation and rehearsal. However, reduction followed by tunneling emerged as the
least employed strategies.
In our future work, we plan to conduct a large-scale study on mental health apps,
as well as apps in other health domains, based on user reviews using the text mining
approach to investigate the persuasiveness and effectiveness of these apps. We will
extend the semantic attributes to cover all the 28 persuasive strategies of the PSD
framework for our study. Finally, we will explore negative reviews to investigate the
effect of how persuasive strategies are implemented on overall user experience, and
then recommend design solutions to address identified gaps.
References
1. Poushter, J.: Smartphone Ownership and Internet Usage Continues to Climb in Emerging
Economies. (2016). http://www.pewglobal.org/2016/02/22/smartphone-ownership-and-
internet-usage-continues-to-climb-in-emerging-economies/
2. Taylor, K., Silver, L.: Smartphone Ownership Is Growing Rapidly Around the World, but
Not Always Equally. (2019). https://www.pewresearch.org/global/2019/02/05/smartphone-
ownership-is-growing-rapidly-around-the-world-but-not-always-equally/
3. GSM Association: The Mobile Economy 2019. (2019).
https://www.gsmaintelligence.com/research/?file=b9a6e6202ee1d5f787cfebb95d3639c5&
download
4. Blair, I.: Mobile App Download and Usage Statistics. (2019). https://buildfire.com/app-
statistics/
5. Clement, J.: Mobile app usage - Statistics & Facts. (2019).
https://www.statista.com/topics/1002/mobile-app-usage/
6. Areán, P.A., Ly, K.H., Andersson, G.: Mobile technology for mental health assessment.
Dialogues Clin. Neurosci. 18, 163–169 (2016)
7. Tucker, I., Smith, L.-A.: Topology and mental distress: self-care in the life spaces of home.
J. Health Psychol. 19, 176–83 (2014). https://doi.org/10.1177/1359105313500260
11
8. Mason, M.J., Korpela, K.: Activity spaces and urban adolescent substance use and
emotional health. J. Adolesc. 32, 925–939 (2009).
https://doi.org/10.1016/j.adolescence.2008.08.004
9. Sano, A., Taylor, S., McHill, A.W., Phillips, A.J.K., Barger, L.K., Klerman, E., Picard, R.:
Identifying objective physiological markers and modifiable behaviors for self-reported
stress and mental health status using wearable sensors and mobile phones: Observational
study. J. Med. Internet Res. 20, e210 (2018). https://doi.org/10.2196/jmir.9410
10. Yu, B., Hu, J., Funk, M., Feijs, L.: DeLight: biofeedback through ambient light for stress
intervention and relaxation assistance. Pers. Ubiquitous Comput. 22, 787–805 (2018).
https://doi.org/10.1007/s00779-018-1141-6
11. Fadhil, A., Wang, Y.: Health Behaviour Change Techniques in Diabetes Management
Applications: A Systematic Review. (2019). http://arxiv.org/abs/1904.09884
12. Oyebode, O., Ndulue, C., Alhasani, M., Orji, R.: Persuasive Mobile Apps for Health and
Wellness : A Comparative Systematic Review. In: International Conference on Persuasive
Technology. pp. 1–12 (2020)
13. Alqahtani, F., Khalifah, G. Al, Oyebode, O., Orji, R.: Apps for Mental Health : An
Evaluation of Behavior Change Strategies and Recommendations for Future Development.
Front. Artif. Intell. 2, 1–11 (2019). https://doi.org/10.3389/frai.2019.00030
14. Oinas-Kukkonen, H., Harjumaa, M.: Persuasive Systems Design: Key Issues, Process
Model, and System Features. Commun. Assoc. Inf. Syst. 24, 96 (2009)
15. Orji, R., Orji, F., Oyibo, K., Ajah, I.A.: Personalizing health theories in persuasive game
interventions to gamer types: An African perspective. In: Proceedings of the Second
African Conference for Human Computer Interaction: Thriving Communities. pp. 45–56
(2018)
16. Du, X., Kowalski, M., Varde, A.S., de Melo, G., Taylor, R.W.: Public opinion matters:
Mining Social Media Text for Environmental Management. ACM SIGWEB Newsl. 1–15
(2020). https://doi.org/10.1145/3352683.3352688
17. Pejić Bach, M., Krstić, Ž., Seljan, S., Turulja, L.: Text Mining for Big Data Analysis in
Financial Sector: A Literature Review. Sustainability. 11, 1277 (2019).
https://doi.org/10.3390/su11051277
18. Kuhzady, S., Ghasemi, V.: Factors influencing customers’ satisfaction and dissatisfaction
with hotels: A text-mining approach. Tour. Anal. 24, 69–79 (2019).
https://doi.org/10.3727/108354219X15458295631972
19. Bollegala, D., Maskell, S., Sloane, R., Hajne, J., Pirmohamed, M.: Causality Patterns for
Detecting Adverse Drug Reactions From Social Media: Text Mining Approach. JMIR
public Heal. Surveill. 4, e51 (2018). https://doi.org/10.2196/publichealth.8214
20. Toulis, A., Golab, L.: Social media mining to understand public mental health. In: Data
Management and Analytics for Medicine and Healthcare. pp. 55–70. Springer Verlag
(2017)
21. Gebauer, J., Tang, Y., Baimai, C.: User requirements of mobile technology: Results from a
content analysis of user reviews. Inf. Syst. E-bus. Manag. 6, 361–384 (2008).
https://doi.org/10.1007/s10257-007-0074-9
22. Heedzy: Download app reviews from iTunes App Store & Google Play,
https://heedzy.com/
23. Kaasinen, E., Mattila, E., Lammi, H., Kivinen, T., Välkkynen, P.: Technology Acceptance
Model for Mobile Services as a Design Framework. In: Human-Computer Interaction and
Innovation in Handheld, Mobile and Wearable Technologies. pp. 80–107. IGI Global
(2011)
12
24. Langrial, S., Lehto, T., Oinas-Kukkonen, H., Harjumaa, M., Karppinen, P.: Native Mobile
Applications For Personal Well- Being: A Persuasive Systems Design Evaluation. In:
PACIS 2012 Proceedings (2012)
25. Al-Ramahi, M.A., Liu, J., El-Gayar, O.F.: Discovering design principles for health
behavioral change support systems: A text mining approach. ACM Trans. Manag. Inf.
Syst. 8, (2017). https://doi.org/10.1145/3055534
26. Slang Words Dictionary, https://raw.githubusercontent.com/sifei/Dictionary-for-
Sentiment-Analysis/master/slang/acrynom.csv
27. Slang Lookup Table,
https://raw.githubusercontent.com/felipebravom/StaticTwitterSent/master/extra/SentiStren
gth/SlangLookupTable.txt
28. Fellbaum, C.: WordNet and wordnets. In: Encyclopedia of Language and Linguistics. pp.
665–670. Elsevier Science (2005)
29. McIlroy, S., Ali, N., Khalid, H., E. Hassan, A.: Analyzing and automatically labelling the
types of user issues that are raised in mobile app reviews. Empir. Softw. Eng. 21, 1067–
1106 (2016). https://doi.org/10.1007/s10664-015-9375-7
30. Khalid, H., Shihab, E., Nagappan, M., Hassan, A.E.: What do mobile app users complain
about? IEEE Softw. 32, 70–77 (2015). https://doi.org/10.1109/MS.2014.50
31. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3,
993–1022 (2003). https://doi.org/10.1016/b978-0-12-411519-4.00006-9
32. Hoffman, M.D., Blei, D.M., Bach, F.: Online Learning for latent Dirichlet allocation
(Supplementary Material). Nature. 1–9 (2010). https://doi.org/10.1.1.187.1883
33. Juan Ramos: Using TF-IDF to determine word relevance in document queries. Proc. first
Instr. Conf. Mach. Learn. 242, 133–142 (2003)
34. Yoganathan, D., Kajanan, S.: Designing fitness apps using persuasive technology: A text
mining approach. Pacific Asia Conf. Inf. Syst. PACIS 2015 - Proc. (2015)
35. Bakker, D., Rickard, N.: Engagement in mobile phone app for self-monitoring of
emotional wellbeing predicts changes in mental health: MoodPrism. J. Affect. Disord. 227,
432–442 (2018). https://doi.org/10.1016/j.jad.2017.11.016
36. Morris, M.E., Kathawala, Q., Leen, T.K., Gorenstein, E.E., Guilak, F., Labhard, M.,
Deleeuw, W.: Mobile therapy: Case study evaluations of a cell phone application for
emotional self-awareness. J. Med. Internet Res. 12, 1–21 (2010).
https://doi.org/10.2196/jmir.1371
37. Kauer, S.D., Reid, S.C., Crooke, A.H.D., Khor, A., Hearps, S.J.C., Jorm, A.F., Sanci, L.,
Patton, G.: Self-monitoring using mobile phones in the early stages of adolescent
depression: Randomized controlled trial. J. Med. Internet Res. 14, 1–17 (2012).
https://doi.org/10.2196/jmir.1858
38. Hetrick, S.E., Robinson, J., Burge, E., Blandon, R., Mobilio, B., Rice, S.M., Simmons,
M.B., Alvarez-Jimenez, M., Goodrich, S., Davey, C.G.: Youth codesign of a mobile phone
app to facilitate self-monitoring and management of mood symptoms in young eople with
major depression, suicidal ideation, and self-harm. J. Med. Internet Res. 20, 1–14 (2018).
https://doi.org/10.2196/mental.9041
39. Boit, J., El-Gayar, O.: Topical Mining of Malaria Using Social Media. A Text Mining
Approach. In: Proceedings of the 53rd Hawaii International Conference on System
Sciences. Hawaii International Conference on System Sciences (2020)
40. Simmons, M., Singhal, A., Lu, Z.: Text mining for precision medicine: Bringing structure
to ehrs and biomedical literature to understand genes and health. In: Advances in
Experimental Medicine and Biology. pp. 139–166. Springer New York LLC (2016)
13
41. Payton, F.C., Yarger, L.K., Pinter, A.T.: Text Mining Mental Health Reports for Issues
Impacting Today’s College Students: Qualitative Study. JMIR Ment. Heal. 5, e10032
(2018). https://doi.org/10.2196/10032
42. Deng, T., Lee, Y.J., Xie, K.: Management Responses to Online Hotel Reviews: Text
Mining to Lift Reputation and Revenue. SSRN Electron. J. (2019).
https://doi.org/10.2139/ssrn.3416224
43. Sezgen, E., Mason, K.J., Mayer, R.: Voice of airline passenger: A text mining approach to
understand customer satisfaction. J. Air Transp. Manag. 77, 65–74 (2019).
https://doi.org/10.1016/j.jairtraman.2019.04.001
44. Orji, R., Moffatt, K.: Persuasive technology for health and wellness: State-of-the-art and
emerging trends. Health Informatics J. 24, 66–91 (2018).
https://doi.org/10.1177/1460458216650979
45. Orji, R., Mandryk, R.L., Vassileva, J.: Improving the efficacy of games for change using
personalization models. ACM Trans. Comput. Interact. 24, (2017).
https://doi.org/10.1145/3119929
46. Alqahtani, F., Orji, R.: Usability Issues in Mental Health Applications. Adjun. Publ. 27th
Conf. User Model. Adapt. Pers. - UMAP’19 Adjun. 343–348 (2019).
https://doi.org/10.1145/3314183.3323676