Persuasive Strategies in Mental Health Apps: A Natural Language Processing Approach Oladapo Oyebode*[0000-0002-5797-7790] and Rita Orji[0000-0001-6152-8034] Faculty of Computer Science, Dalhousie University, Halifax NS B3H 4R2, Canada oladapo.oyebode@dal.ca, rita.orji@dal.ca Abstract. We present a natural language processing (NLP) approach to detect- ing the persuasive strategies employed by 100 mental health apps based on 57705 user reviews. We focus on the persuasive strategies in the primary task support category of the Persuasive Systems Design (PSD) framework. We used the Latent Dirichlet Allocation (LDA) topic modelling algorithm, in conjunc- tion with semantic attributes, to achieve our goal. Keywords: Persuasive strategies, Natural language processing, Topic model- ling, Latent dirichlet allocation, LDA, Mental health, Mobile apps. 1 Introduction Over the years, previous research has adopted the manual coding method to identify persuasive strategies employed in mHealth apps [1–3]. This approach requires expert reviewers to download the apps and manually code them using the Persuasive Sys- tems Design (PSD) framework [4] or behaviour change theories [5]. However, this may be very costly or impracticable for wider studies that involve hundreds or thou- sands of apps. In this paper, we applied the natural language processing (NLP) techniques, includ- ing topic modelling (with automated topic labelling) on user reviews of 100 mental health apps (from both Google Play and App Store) to detect the persuasive strategies they employed. This automated approach is a more efficient, practicable, and less costly way of identifying the persuasive strategies employed by large number of mo- bile apps. The main contribution of this paper is to eliminate manual coding of apps by automatically deconstructing persuasive strategies (in the primary task support category of the PSD framework) based on user reviews using natural language pro- cessing (NLP) techniques and the Latent Dirichlet Allocation (LDA) topic modelling algorithm. The paper also contributes to research by creating semantic attributes rep- resenting various persuasive strategies to automatically label topics or themes gener- ated by the LDA algorithm. Persuasive 2020, Adjunct proceedings of the 15th International conference on Persuasive Technology. Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0) 2 2 Methodology and Results To identify the persuasive strategies implemented by mental health apps based on user reviews, we applied well-known computational techniques. First, we used the Heedzy tool [6] to extract 101715 user reviews of 105 eligible mental health apps on both Google Play and App Store. Second, we preprocessed the data using NLP techniques (such as converting words to lowercase, reducing repeated characters, removing numbers, expanding contractions, replacing slangs with English words, removing punctuation and special characters, removing stop words, lemmatiz- ing words, and removing duplicates) to prepare it for analysis. After data prepro- cessing, which was automatically done using Python scripts, total reviews reduced to 88125. Third, we classified each review into either positive, negative, or neutral sen- timent polarity. We retained only positive reviews (n=57705) since they mostly re- flect user opinions or experience about features and strategies already implemented in the apps. The positive reviews are distributed across 100 apps. Fourth, we vectorized the user reviews for each app using the Term Frequency Inverse Document Frequency (TF-IDF) weighting technique which considers frequency and relevance when assign- ing weight to words [7]. Fifth, we applied the Latent Dirichlet Allocation (LDA) algo- rithm (which is widely used and efficient [8, 9]) on the vectorized reviews for each app to identify main topics or themes describing the reviews. The LDA algorithm returns top 𝐾 topics in the reviews, along with top 𝑁 words for each topic (where K is set to 50 based on evidence on perplexity [10], and N set to 10). In other words, for each of the 100 apps, we retrieved top 50 topics and top 10 words that describe each topic. Sixth, we automatically inferred persuasive strategies from the words associat- ed with each topic using semantic attributes. The semantic attributes were generated, using the WordNet lexical database [11], for each persuasive strategy in the primary task support category of the PSD framework. Table 1 shows the semantic attributes for the self-monitoring and tunneling strategies. The attributes were verified and vali- dated by two persuasive technology experts for appropriateness. Finally, a Python program was developed to match the words corresponding to each topic with the se- mantic attributes, and then label the topic with the appropriate persuasive strategy (or strategies) based on the matching attribute(s). Table 1. Semantic Attributes for the Self-monitoring and Tunneling persuasive strategies Persuasive Strategy Semantic Attributes Self-monitoring track, statistics, statistic, measure, progress, goal, history, view, display, activity, analysis, record, monitor, graph, chart, log, logging, journal, diary, duration, speed, pace, time, insight, insightful, recording, journaling Tunneling step-by-step, guide, stepwise, gradual, step, instruction, procedure, procedural, process, journey, stage, plan, guided, guidance Our experimental results showed that self-monitoring is the most employed persua- sive strategy overall (n=92), followed by personalization and tailoring (n=83) and 3 simulation and rehearsal (n=81). However, reduction (n=77) followed by tunneling (n=53) are the least employed persuasive strategies. Our findings aligned with the results of a prior research that applied the manual coding method which showed that self-monitoring and personalization are the top 2 persuasive strategies employed by mental health apps [3]. Table 2 shows sample topics for one of the apps including the top 10 words associ- ated with each topic, the matching semantic attributes, topic label (or persuasive strat- egy), sample user reviews, and the app name. Table 2. Sample topics (showing words associated with each topic), topic label, matching semantic attributes, sample user reviews, and app name. Topic Words Topic Label Matching Sample user reviews App Name Attribute(s) [job, pain, med, self-monitoring track, “Awesome application Daylio track, mood, statistic beautiful designs and it is a statistic, great way to keep on track application, use, of the moods!”1 [R253] great, love] “An amazing application I just loved the way it gives me my emotional statis- tics…” [R391] 3 Conclusion In this paper, we presented a natural language processing (NLP) approach to decon- structing persuasive strategies employed by mental health apps based on user reviews. This eliminates manual coding of apps which has been a common phenomenon among persuasive technology researchers. In addition, our approach showed that user reviews could be a reliable and cost-effective alternative for evaluating the effective- ness of persuasive apps both in short and long-term. Our experimental results revealed that self-monitoring is the most employed per- suasive strategy in mental health apps, followed by personalization and tailoring, and simulation and rehearsal. Reduction followed by tunneling emerged as the least em- ployed persuasive strategies. References 1. Fadhil, A., Wang, Y.: Health Behaviour Change Techniques in Diabetes Management Applications: A Systematic Review. (2019) 2. Oyebode, O., Ndulue, C., Alhasani, M., Orji, R.: Persuasive Mobile Apps for Health and Wellness : A Comparative Systematic Review. In: International Conference on Persuasive Technology. pp. 1–12 (2020) 1 User reviews are included verbatim throughout the paper, including spelling and grammatical mistakes. 4 3. Alqahtani, F., Khalifah, G. Al, Oyebode, O., Orji, R.: Apps for Mental Health : An Evaluation of Behavior Change Strategies and Recommendations for Future Development. Front. Artif. Intell. 2, 1–11 (2019). https://doi.org/10.3389/frai.2019.00030 4. Harri, O.-K., Marja, H.: Persuasive Systems Design: Key Issues, Process Model, and System Features. Commun. Assoc. Inf. Syst. 24, 96 (2009) 5. Orji, R., Orji, F., Oyibo, K., Ajah, I.A.: Personalizing health theories in persuasive game interventions to gamer types: An African perspective. In: Proceedings of the Second African Conference for Human Computer Interaction: Thriving Communities. pp. 45–56 (2018) 6. Heedzy: Download app reviews from iTunes App Store & Google Play, https://heedzy.com/ 7. Juan Ramos: Using TF-IDF to determine word relevance in document queries. Proc. first Instr. Conf. Mach. Learn. 242, 133–142 (2003) 8. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003). https://doi.org/10.1016/b978-0-12-411519-4.00006-9 9. Hoffman, M.D., Blei, D.M., Bach, F.: Online Learning for latent Dirichlet allocation (Supplementary Material). Nature. 1–9 (2010). https://doi.org/10.1.1.187.1883 10. Al-Ramahi, M.A., Liu, J., El-Gayar, O.F.: Discovering design principles for health behavioral change support systems: A text mining approach. ACM Trans. Manag. Inf. Syst. 8, (2017). https://doi.org/10.1145/3055534 11. Fellbaum, C.: WordNet and wordnets. In: Encyclopedia of Language and Linguistics. pp. 665–670. Elsevier Science (2005)