Persuasive Strategies in Mental Health Apps: A Natural Language Processing Approach

Abstract

We present a natural language processing (NLP) approach to detecting the persuasive strategies employed by 100 mental health apps based on 57705 user reviews. We focus on the persuasive strategies in the primary task support category of the Persuasive Systems Design (PSD) framework. We used the Latent Dirichlet Allocation (LDA) topic modelling algorithm, in conjunction with semantic attributes, to achieve our goal.

Keywords

• Persuasive strategies
• Natural language processing
• Topic modelling
• Latent dirichlet allocation
• LDA
• Mental health
• Mobile apps

Introduction

Over the years, previous research has adopted the manual coding method to identify persuasive strategies employed in mHealth apps [ 1–3 ]. This approach requires expert reviewers to download the apps and manually code them using the Persuasive Systems Design (PSD) framework [ 4 ] or behaviour change theories [ 5 ]. However, this may be very costly or impracticable for wider studies that involve hundreds or thousands of apps.

In this paper, we applied the natural language processing (NLP) techniques, including topic modelling (with automated topic labelling) on user reviews of 100 mental health apps (from both Google Play and App Store) to detect the persuasive strategies they employed. This automated approach is a more efficient, practicable, and less costly way of identifying the persuasive strategies employed by large number of mobile apps. The main contribution of this paper is to eliminate manual coding of apps by automatically deconstructing persuasive strategies (in the primary task support category of the PSD framework) based on user reviews using natural language processing (NLP) techniques and the Latent Dirichlet Allocation (LDA) topic modelling algorithm. The paper also contributes to research by creating semantic attributes representing various persuasive strategies to automatically label topics or themes generated by the LDA algorithm.

Methodology and Results

To identify the persuasive strategies implemented by mental health apps based on user reviews, we applied well-known computational techniques.

First, we used the Heedzy tool [ 6 ] to extract 101715 user reviews of 105 eligible mental health apps on both Google Play and App Store. Second, we preprocessed the data using NLP techniques (such as converting words to lowercase, reducing repeated characters, removing numbers, expanding contractions, replacing slangs with English words, removing punctuation and special characters, removing stop words, lemmatizing words, and removing duplicates) to prepare it for analysis. After data preprocessing, which was automatically done using Python scripts, total reviews reduced to 88125. Third, we classified each review into either positive, negative, or neutral sentiment polarity. We retained only positive reviews (n=57705) since they mostly reflect user opinions or experience about features and strategies already implemented in the apps. The positive reviews are distributed across 100 apps. Fourth, we vectorized the user reviews for each app using the Term Frequency Inverse Document Frequency (TF-IDF) weighting technique which considers frequency and relevance when assigning weight to words [ 7 ]. Fifth, we applied the Latent Dirichlet Allocation (LDA) algorithm (which is widely used and efficient [ 8, 9 ]) on the vectorized reviews for each app to identify main topics or themes describing the reviews. The LDA algorithm returns top topics in the reviews, along with top words for each topic (where K is set to 50 based on evidence on perplexity [ 10 ], and N set to 10). In other words, for each of the 100 apps, we retrieved top 50 topics and top 10 words that describe each topic. Sixth, we automatically inferred persuasive strategies from the words associated with each topic using semantic attributes. The semantic attributes were generated, using the WordNet lexical database [ 11 ], for each persuasive strategy in the primary task support category of the PSD framework. Table 1 shows the semantic attributes for the self-monitoring and tunneling strategies. The attributes were verified and validated by two persuasive technology experts for appropriateness. Finally, a Python program was developed to match the words corresponding to each topic with the semantic attributes, and then label the topic with the appropriate persuasive strategy (or strategies) based on the matching attribute(s).

Our experimental results showed that self-monitoring is the most employed persuasive strategy overall (n=92), followed by personalization and tailoring (n=83) and simulation and rehearsal (n=81). However, reduction (n=77) followed by tunneling (n=53) are the least employed persuasive strategies. Our findings aligned with the results of a prior research that applied the manual coding method which showed that self-monitoring and personalization are the top 2 persuasive strategies employed by mental health apps [ 3 ].

Table 2 shows sample topics for one of the apps including the top 10 words associated with each topic, the matching semantic attributes, topic label (or persuasive strategy), sample user reviews, and the app name. App Name Daylio 3

Conclusion

In this paper, we presented a natural language processing (NLP) approach to deconstructing persuasive strategies employed by mental health apps based on user reviews. This eliminates manual coding of apps which has been a common phenomenon among persuasive technology researchers. In addition, our approach showed that user reviews could be a reliable and cost-effective alternative for evaluating the effectiveness of persuasive apps both in short and long-term.

Our experimental results revealed that self-monitoring is the most employed persuasive strategy in mental health apps, followed by personalization and tailoring, and simulation and rehearsal. Reduction followed by tunneling emerged as the least employed persuasive strategies. 1 User reviews are included verbatim throughout the paper, including spelling and grammatical mistakes.

References

1. Fadhil , A. , Wang , Y. : Health Behaviour Change Techniques in Diabetes Management Applications: A Systematic Review . ( 2019 )

2. Oyebode , O. , Ndulue , C. , Alhasani , M. , Orji , R.: Persuasive Mobile Apps for Health and Wellness : A Comparative Systematic Review . In: International Conference on Persuasive Technology. pp. 1 - 12 ( 2020 )

3. Alqahtani , F. , Khalifah , G. Al, Oyebode , O. , Orji , R.: Apps for Mental Health : An Evaluation of Behavior Change Strategies and Recommendations for Future Development . Front. Artif. Intell. 2 , 1 - 11 ( 2019 ). https://doi.org/10.3389/frai. 2019 .00030

4. Harri , O.-K. , Marja , H.: Persuasive Systems Design: Key Issues, Process Model, and System Features . Commun. Assoc. Inf. Syst . 24 , 96 ( 2009 )

5. Orji , R. , Orji , F. , Oyibo , K. , Ajah , I.A. : Personalizing health theories in persuasive game interventions to gamer types: An African perspective . In: Proceedings of the Second African Conference for Human Computer Interaction: Thriving Communities . pp. 45 - 56 ( 2018 )

6. Heedzy: Download app reviews from iTunes App Store & Google Play, https://heedzy.com/

7. Juan Ramos : Using TF-IDF to determine word relevance in document queries . Proc. first Instr. Conf. Mach. Learn . 242 , 133 - 142 ( 2003 )

8. Blei , D.M. , Ng , A.Y. , Jordan , M.I. : Latent Dirichlet allocation . J. Mach. Learn. Res . 3 , 993 - 1022 ( 2003 ). https://doi.org/10.1016/b978-0 -12-411519-4 . 00006 - 9

9. Hoffman , M.D. , Blei , D.M. , Bach , F. : Online Learning for latent Dirichlet allocation (Supplementary Material) . Nature . 1- 9 ( 2010 ). https://doi.org/10.1.1.187.1883

10. Al-Ramahi , M.A. , Liu , J. , El-Gayar , O.F. : Discovering design principles for health behavioral change support systems: A text mining approach . ACM Trans. Manag. Inf. Syst. 8 , ( 2017 ). https://doi.org/10.1145/3055534

11. Fellbaum , C. : WordNet and wordnets . In: Encyclopedia of Language and Linguistics . pp. 665 - 670 . Elsevier Science ( 2005 )