Mental health issues are an increasing global public health concern. With the rise of social media, there is growing interest in the early detection of mental disorders by analyzing user posts. This research project aims to leverage Artificial Intelligence (AI) to enhance mental wellness. We operate initially under two primary objectives: first, that Natural Language Processing (NLP) can efectively identify signs of mental disorders or emotional distress in user messages over time; and second, that Large Language Models (LLMs) can provide high-quality information to mental health professionals. To test these hypotheses, we explore key research questions, including the feasibility of detecting emotional symptoms in text messages and the capability of chatbots to collect high-quality data for professional use. This study focuses primarily on the Spanish language.
According to the World Health Organization (WHO), mental health is a state of mental well-being that
enables individuals to cope with life’s stresses, realize their abilities, learn and work efectively, and
contribute to their communities. The absence of mental health could carry on mental health problems
such as eating disorder (ED), depression or anxiety disorder, being the last two most prevalent nowadays.
In 2019, one in eight people in the world was diagnosed with one or more mental health issues and
it is estimated that only one year before COVID-19 the number of people sufering from anxiety or
depression has increased by about 30% [
In addition, due to the increasing use of social media, people often express their emotional problems
or thoughts on the Internet in search of comfort, support or to unwind [
For this reason, our research focuses on applying NLP and AI to identify in an early way the risk of a user sufering from a disorder in social media and to develop systems and tools to help professionals with mental health in their work. By harnessing the potential of these technologies, we strive not only to improve our understanding of the dynamics of mental health in digital spaces but also to provide scientific and professional communities with the results of our mental well-being eforts. Ultimately, this work aims to contribute to the advancement of the development of tools in this field for Spanish speakers, since the research carried out so far has focused on the English language.
The detection of whether a person is sufering from a mental disorder and the implementation of treatment are two of the important tasks the scientific community has been tackling in recent years. Still, the speed factor is one of the most important factors that is being taken into account. The early detection of signs of mental disorders is important, since, undetected, mental disorders can develop into more serious consequences, constituting a major predictive factor of suicide [12]. Therefore, both the early detection of a person’s risk of sufering from a mental health problem and the rapid implementation of treatment are two growing tasks.
The concept of early risk detection involves promptly identifying potential signs of mental health issues, hate speech, and other concerns on social networks [13]. This concept is central to the well-known eRisk shared task eRisk1 which began in 2017 and is hosted annually at the Conference and Labs of the Evaluation Forum (CLEF). Many studies on early risk prediction originate from eRisk, which has addressed early risk detection of gambling [14], self-harm [15], and disorders such as anorexia or depression [16]. These tasks consist of sequentially processing subjects’ posts on Reddit. This simulates a real-time analysis of social media to evaluate the early risk detection capabilities of various systems teams.
So far, several datasets have been developed to identify mental health problems like stress, depression
or anxiety [17, 18, 19], as well as the severity of depression [20, 21] or suicide risk [22]. However,
to carry out the task of early detection it is necessary to get a user-level dataset that permits the
application of measures such as Early Risk Detection Error (ERDE) [23]. Twitter and Reddit are the
most studied platforms for mental health research [24], especially in early detection. All eRisk datasets
come from Reddit users but there are other datasets annotated at the user level extracted from other
social networks like Twitter-STMHD [25] which is a collection of user Twitter profiles sufering from
mental health disorders. These datasets often focus on English-language data, but there are exceptions.
For instance, Villa-Pérez et al. [26] created two datasets in English and Spanish that comprise the
timeline of Twitter users who explicitly reported in one or more of their posts having been diagnosed
with one disorder. Additionally, the dataset MentalRiskES [
However, one of the emerging branches in recent years comes from the need for explainability of a
risk prediction [
In the medical field, conversational agents (CAs) are gaining popularity, particularly for addressing
health-related inquiries, a functionality that can also be expanded to mental health concerns [
Replika2 [
Studies have shown chatbots can significantly benefit both young people and adults by reducing
symptoms of anxiety and depression and improving overall mood [
Despite their benefits, the deployment of chatbots in mental health care presents challenges and ethical considerations. Although people who frequently use these tools trust them and their security more than people who have never used any of them, these chatbots can also be disruptive and introduce risks for users with sensitive questions or disclosure of information [47]. Privacy concerns, the accuracy of the chatbots’ responses, and the need for human oversight are significant issues that researchers and developers must address to ensure chatbots provide reliable and safe support.
However, chatbots do not necessarily have to be emotionally involved with the individual, sometimes they are simply useful tools for gathering information beyond applying a self-reported test [48] since users find language is more precise in communicating their mental health issues, preferring it to rating scales [49].
Given the large number of existing applications of NLP and the use of more advanced techniques such as LLM in the mental health field, our research elaborates on the following premises as scientific hypotheses: • H1: NLP techniques allow identifying and tracking signs of mental disorders or emotional problems in user-generated text messages over time. • H2: LLM can provide high-quality, contextually relevant information and support to mental health professionals, enhancing their ability to diagnose and treat patients efectively. 2https://replika.ai/ 3https://woebothealth.com/ 4https://www.youper.ai/ 5https://www.wysa.com/ To prove these hypotheses we asked ourselves the following research questions: • Q1: Can NLP models accurately identify symptoms of emotional problems in text messages? • Q2: Are there identifiable linguistic features or patterns that are most indicative of these symptoms in the Spanish language? • Q3: Can we utilize LLMs to induce user self-disclosure in mental health? • Q4: Can we assess the feasibility of collecting high-quality, clinically relevant data through interactions with chatbots? • Q5: Can we build chatbots to elicit meaningful mental-health-related information from users while maintaining user trust and engagement?
As detailed below, in previous research work I developed resources, such as a corpus or a basic dialogue system, which I have been able to develop in this pre-doctoral period. These resources will serve as the foundational elements and support for future work. Additionally, several experiments have been conducted to validate and extend these initial developments.
As a preliminary work associated with a research project called BigHug6, focused on the early detection of disorders and misbehaviours in online social networks, a dialogue system was developed. For this project, the author of this paper developed a novel chatbot [50] to chat about several mental disorders for young Spanish on the Telegram Platform. The most novel aspect of this chat, apart from its ability to converse in Spanish, is that it allows for both closed and open dialogue and also does not present itself as a therapist but as one more teenager who wants to talk about his or her problems. For the open dialogue, we integrated the Generative Pre-trained Transformer (GPT-3) trained mostly on English texts, so we also used DeepL7 to translate. For the controlled dialogue, we used some questions and sentences established by psychologists and specialists in mental disorders in teenagers.
The dialogue system creation involved a major collaboration, where the full development of the dialogue system was carried out by the author of this paper and now forms a key part of the thesis. It will be a central component of ongoing and future research work related to H2 above. The basic corpus obtained from this experimentation has been used in initial experiments to test and refine the functionalities of the system and to detect needs and strengths. Moreover, nowadays there are generative models of language with greater capacity, which is why we propose a more updated development focused on the needs of a therapist, focusing on the ability to contextualise and synthesise useful information.
A new extensive dataset entitled MentalRiskES [
The creation of this collection of Spanish-language message threads aimed at early risk detection for mental health disorders is directly relevant to H1 of the thesis. The author of the thesis has been directly involved in all phases of the development of this corpus. As future work, it is planned to work in more depth and analyse the dataset created and to develop a new dataset that will allow the detection of symptoms for certain disorders and messages for Spanish.
We have engaged in the shared task eRisk organized during CLEF conferences to test the hypothesis
and gain access to annotated data. We plan to continue to participate in future editions of the task.
• eRisk 2022 [51]. Two of the proposed tasks were addressed: early detection of signs of
pathological gambling, and measuring the severity of the signs of eating disorders. The approach presented
for the first task is based on the use of sentence embeddings from Transformers with features
related to volumetry, lexical diversity, complexity metrics, and emotion-related scores, while the
approach for the second task is based on text similarity estimation using contextualized word
embeddings from Transformers [52].
• eRisk 2023 [
On the other hand, I was part of the organising committee of MentalRiskES, a shared task organized at IberLEF (edition 39) as part of the International Conference of the Spanish Society for Natural Language Processing (SEPLN). MentalRiskES aim to promote the early detection of mental risk disorders in Spanish.
• MentalRiskES 2023 [
My research is still at the beginning of the way so there are a lot of questions to address and elements to be proposed and discussed. Some of them are the following: • Early detection of mental disorders and emotional problems: What are the challenges and limitations of detecting signs of mental health problems in text data? Can early detection methods be generalised to diferent mental health conditions or should they be disease-specific? Is it possible to identify symptoms of emotional problems in text messages? How accurately can NLP models detect early signs of specific mental health disorders in text communications? • Efectiveness of dialogue systems in mental health support: What are the key features and functionalities that should be included in the dialogue system to ensure it is both supportive and safe for users? Can a chatbot be designed to collect high-quality data that is valuable and reliable for mental health professionals? How can the impact of the dialogue system on users’ mental health and well-being be measured accurately? In what ways can a LLM be designed to encourage user self-disclosure in a manner that promotes well-being? How do users perceive the use of chatbots and automated systems in their mental health care? What factors influence their acceptance and trust, and how can these systems be designed to align with the standards and practices of certified mental health diagnostics and treatments? • Ethical and legal considerations: What are the ethical implications of using automated systems for detecting and responding to mental health issues? Is it possible to ensure the system does not inadvertently cause harm or distress to users?
My sincere thanks to my thesis tutors, Arturo Montejo-Raéz, Miguel Ángel García-Cumbreras and Manuel García-Vega, for guiding me along this process, to the doctoral programme of the University of Jaén and the Centre for Advanced Studies in Information and Communication Technologies (CEATIC for its acronym in Spanish) for their support in this research experience.
This work has been supported by project MODERATES (TED2021-130145B-I00) funded by Plan Nacional I+D+i from the Spanish Government. [11] S. Henry, M. Yetisgen, O. Uzuner, Natural Language Processing in Mental Health Research and Practice, Springer International Publishing, Cham, 2021, pp. 317–353. URL: https://doi.org/10.1007/ 978-3-030-70558-9_13. doi:10.1007/978-3-030-70558-9_13. [12] World Health Organization, Depressive disorder (depression), https://www.who.int/news-room/ fact-sheets/detail/depression, 2023. Accessed: 07.06.2024. [13] D. E. Losada, F. Crestani, J. Parapar, erisk 2017: Clef lab on early risk prediction on the internet: experimental foundations, in: Experimental IR Meets Multilinguality, Multimodality, and Interaction: 8th International Conference of the CLEF Association, CLEF 2017, Dublin, Ireland, September 11–14, 2017, Proceedings 8, Springer, 2017, pp. 346–360. [14] J. Parapar, P. Martín-Rodilla, D. E. Losada, F. Crestani, Overview of erisk at clef 2021: Early risk prediction on the internet (extended overview)., CLEF (Working Notes) (2021) 864–887. [15] D. E. Losada, F. Crestani, J. Parapar, Overview of erisk 2019 early risk prediction on the internet, in: Experimental IR Meets Multilinguality, Multimodality, and Interaction: 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9–12, 2019, Proceedings 10, Springer, 2019, pp. 340–357. [16] D. E. Losada, F. Crestani, J. Parapar, Overview of erisk: early risk prediction on the internet, in: Experimental IR Meets Multilinguality, Multimodality, and Interaction: 9th International Conference of the CLEF Association, CLEF 2018, Avignon, France, September 10-14, 2018, Proceedings 9, Springer, 2018, pp. 343–361. [17] D. Owen, J. C. Collados, L. Espinosa-Anke, Towards preemptive detection of depression and anxiety in twitter, arXiv preprint arXiv:2011.05249 (2020). [18] A. Haque, V. Reddi, T. Giallanza, Deep learning for suicide and depression identification with unsupervised label correction, in: Artificial Neural Networks and Machine Learning–ICANN 2021: 30th International Conference on Artificial Neural Networks, Bratislava, Slovakia, September 14–17, 2021, Proceedings, Part V 30, Springer, 2021, pp. 436–447. [19] S. Ji, X. Li, Z. Huang, E. Cambria, Suicidal ideation and mental disorder detection with attentive relation networks, Neural Computing and Applications 34 (2021) 10309–10319. doi:10.1007/ s00521-021-06208-y. [20] I. Pirina, Ç. Çöltekin, Identifying depression on Reddit: The efect of training data, in: G. Gonzalez-Hernandez, D. Weissenbacher, A. Sarker, M. Paul (Eds.), Proceedings of the 2018 EMNLP Workshop SMM4H: The 3rd Social Media Mining for Health Applications Workshop & Shared Task, Association for Computational Linguistics, Brussels, Belgium, 2018, pp. 9–12. doi:10.18653/v1/W18-5903. [21] U. Naseem, A. G. Dunn, J. Kim, M. Khushi, Early identification of depression severity levels on reddit using ordinal classification, in: Proceedings of the ACM Web Conference 2022, Association for Computing Machinery, New York, NY, USA, 2022, p. 2563–2572. doi:10.1145/3485447. 3512128. [22] M. Gaur, A. Alambo, J. P. Sain, U. Kursuncu, K. Thirunarayan, R. Kavuluru, A. Sheth, R. Welton, J. Pathak, Knowledge-aware assessment of severity of suicide risk for early intervention, in: The World Wide Web Conference, Association for Computing Machinery, New York, NY, USA, 2019, p. 514–525. doi:10.1145/3308558.3313698. [23] D. E. Losada, F. Crestani, A test collection for research on depression and language use, in: N. Fuhr, P. Quaresma, T. Gonçalves, B. Larsen, K. Balog, C. Macdonald, L. Cappellato, N. Ferro (Eds.), Experimental IR Meets Multilinguality, Multimodality, and Interaction, Springer International Publishing, Cham, 2016, pp. 28–39. [24] K. Harrigian, C. Aguirre, M. Dredze, On the state of social media data for mental health research, arXiv preprint arXiv:2011.05233 (2020). [25] A. K. Singh, U. Arora, S. Shrivastava, A. Singh, R. R. Shah, P. Kumaraguru, et al., Twitter-stmhd: An extensive user-level database of multiple mental health disorders, in: Proceedings of the International AAAI Conference on Web and Social Media, volume 16, 2022, pp. 1182–1191. [26] M. E. Villa-Pérez, L. A. Trejo, M. B. Moin, E. Stroulia, Extracting mental health indicators from english and spanish social media: A machine learning approach, IEEE Access 11 (2023) 128135– 2022, pp. 1035–1047. URL: https://aclanthology.org/2022.findings-acl.83. doi: 10.18653/v1/2022. findings-acl.83. [43] T. Blose, P. Umar, A. Squicciarini, S. Rajtmajer, Privacy in crisis: A study of self-disclosure during the coronavirus pandemic, 2020. URL: https://arxiv.org/abs/2004.09717. arXiv:2004.09717. [44] D. Catona, K. Greene, Self-Disclosure, 2015. doi:10.1002/9781118540190.wbeic162. [45] Y.-C. Lee, N. Yamashita, Y. Huang, W. Fu, "i hear you, i feel you": Encouraging deep self-disclosure through a chatbot, in: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI ’20, Association for Computing Machinery, New York, NY, USA, 2020, p. 1–12. doi:10.1145/3313831.3376175. [46] J. Meng, N. Dai, Emotional support from ai chatbots: Should a supportive partner self-disclose or not?, Journal of Computer-Mediated Communication 26 (2021) 207–222. doi:10.1093/jcmc/ zmab005. [47] P. Chametka, S. Maqsood, S. Chiasson, Security and privacy perceptions of mental health chatbots, in: 2023 20th Annual International Conference on Privacy, Security and Trust (PST), IEEE, 2023, pp. 1–7. [48] A. Schick, J. Feine, S. Morana, A. Maedche, U. Reininghaus, Validity of chatbot use for mental health assessment: experimental study, JMIR mHealth and uHealth 10 (2022) e28082. [49] V. Varadarajan, S. Sikström, O. Kjell, H. Schwartz, ALBA: Adaptive language-based assessments for mental health, in: K. Duh, H. Gomez, S. Bethard (Eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Association for Computational Linguistics, Mexico City, Mexico, 2024, pp. 2466–2478. URL: https://aclanthology.org/2024.naacl-long.136. [50] A. M. Mármol-Romero, M. García-Vega, M. Á. García-Cumbreras, A. Montejo-Ráez, An empathic gpt-based chatbot to talk about mental disorders with spanish teenagers, International Journal of Human–Computer Interaction (2024) 1–17. [51] P. Martın-Rodilla, D. E. Losada, F. Crestani, Overview of eRisk 2022: Early Risk Prediction on the Internet, in: Experimental IR Meets Multilinguality, Multimodality, and Interaction: 13th International Conference of the CLEF Association, CLEF 2022, Bologna, Italy, September 5–8, 2022, Proceedings, volume 13390, Springer Nature, 2022, p. 233. [52] A. M. Mármol-Romero, S. M. J. Zafra, F. M. P. del Arco, M. D. Molina-González, M. T. M. Valdivia, A. Montejo-Ráez, Sinai at erisk@ clef 2022: Approaching early detection of gambling and eating disorders with natural language processing., in: Working Notes of CLEF), 2022, pp. 961–971. [53] A. M. Mármol-Romero, F. del Arco, A. Montejo-Ráez, Sinai at erisk@ clef 2023: Approaching early detection of gambling with natural language processing, Working Notes of CLEF (2023) 18–21. [54] J. Parapar, P. Martín-Rodilla, D. E. Losada, F. Crestani, erisk 2024: Depression, anorexia, and eating disorder challenges, in: Advances in Information Retrieval: 46th European Conference on Information Retrieval, ECIR 2024, Glasgow, UK, March 24–28, 2024, Proceedings, Part V, Springer-Verlag, Berlin, Heidelberg, 2024, p. 474–481. doi:10.1007/978-3-031-56069-9_65. [55] A. M. Mármol-Romero, A. Moreno Muñoz, P. Álvarez-Ojeda, K. M. Valencia-Segura, M.-C. Eugenio, M. García-vega, A. Montejo-Ráez, Sinai at erisk@ clef 2024: Approaching the search for symptoms of depression and early detection of anorexia signs using natural language processing., in: Working Notes of CLEF, 2024. To appear. [56] A. M. Mármol-Romero, A. Moreno-Muñoz, F. M. Plaza-del Arco, M. D. Molina-González, M. T.
Martín-Valdivia, L. A. Ureña-López, A. Montejo-Raéz, Overview of MentalriskES at IberLEF 2024: Early Detection of Mental Disorders Risk in Spanish, Procesamiento del Lenguaje Natural 73 (2024). To appear.