Elomia Chatbot: the Effectiveness of Artificial Intelligence in the Fight for Mental Health Oleksandr Romanovskyi, Nina Pidbutska and Anastasiia Knysh National technical university “Kharkiv polytechnic institute”, Kyrpychova str. 2, Kharkiv, 61002, Ukraine Abstract The article is presenting results of controlled study of effectiveness of Elomia chatbot in reducing tendency to depression, anxiety and negative emotional effects. Elomia was developed using artificial intelligence technologies. In the process of developing a chatbot following technologies were used: RoBERTa(NER) for identification names and locations; COSMIC for identification of human emotions; DIALOGPT for answers generation; DistilBERT (SQuAD) for pointing the most relevant information in the script; GECToR for spelling check. Elomia is able to identify the main psychological problems of the client and offer him the most suitable support option using first aid techniques and cognitive-behavioral psychotherapy. To check the effectiveness of the chatbot it was conducted a study that included three stages: 1) the formation of experimental and control samples; 2) baseline testing; 3) final testing. The study used psychological research methods: 1) Patient Health Questionnaire-9 (PHQ-9) to diagnose a tendency to depression; 2) General Anxiety Disorder- 7 (GAD-7) to diagnose a tendency to generalized anxiety disorder; 3) Positive and Negative Affect Schedule (PANAS) to diagnose of prevailing (positive / negative) emotional affects. 412 volunteers (202 women, 210 men, ages 19 to 23 years old) who identified their tendency to depression, anxiety and low mood were selected in students social networks groups of Kharkiv region as participants of research. It was found that regular usage of Elomia contributes to a significant reduction in the high tendency to depression (up to 28%), anxiety (up to 31%), and negative affects (up to 15%). Keywords1 Chatbot, artificial intelligence, depression, anxiety, negative states correction, mental health. 1. Introduction Depression and anxiety are the two most common disorders that seriously affect the quality of life of people of different age and social groups. The prevalence of anxiety and depression problems is evidenced by WHO data, as well as an increasing number of studies on this issue [1]. In recent years, scientists have revised the tools for diagnosing anxiety and depression [2, 3, 4], and ways to prevent and treat them [5, 6, 7], the influence of anxiety and depression on the quality of life of individual age and social groups [2, 8]. The consequences of depressive and anxious conditions are particularly painful for people who are going through age-related crises and experience reassessment of their own values and changes in self- image. Depressive and anxiety disorders spectrum include: F31 Bipolar disorder, F32 Major depressive disorder, F33 Recurrent depressive disorder, F41.1 Generalized Anxiety Disorder, F41.2 Mixed anxiety-depressive disorder and other anxiety disorders. All of these diagnoses can only be made by a psychiatrist and can be treated using a combination of psychotherapy and medication. According to COLINS-2021: 5th International Conference on Computational Linguistics and Intelligent Systems, April 22–23, 2021, Kharkiv, Ukraine EMAIL: romanovskiy_a_khpi@ukr.net (O. Romanovskyi); podbutskaya_nina@ukr.net (N. Pidbutska); n_knysh@ukr.net (A. Knysh) ORCID: 0000-0002-0602-9395 (O. Romanovskyi); 0000-0001-5319-1996 (N. Pidbutska); 0000-0003-0211-2535 (A. Knysh) ©️ 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) the WHO, about 16% of all mental illnesses on the planet are caused by disorders of the anxiety- depressive spectrum. Moreover, hundreds of thousands of people around the world suffer from the so-called “precursors” of such disorders: increased levels of situational anxiety, low mood, anhedonia, feelings of loneliness, acute dissatisfaction with interpersonal relationships. People who suffer with these issues rarely seek help from psychotherapists, psychologists and counselors. They even tend to hide their emotional struggling from loved ones. Long-term attempts to cope with such experiences without outside help often have negative consequences for mental health, which results in the occurrence of disorders of the anxiety-depressive spectrum. People of the described group need additional support, which can be provided by “conversational” services based on artificial intelligence and psychology techniques. 2. Elomia In recent years, more and more psychological services aimed at providing psychological assistance online have occurred (Betterhelp, Helppoint, Talkspace, Youtalk, etc.). These services can be divided into 2 groups:  “chatbots” which are based on active listening techniques  online psychologists. Each of these groups has its advantages and disadvantages. So, the work of "chatbots" often does not imply an individual approach to the needs of the client and allows you to simply talk. Online psychologists, by contrast, provide expert assistance, but their work is quite expensive, which is an obstacle to the continued use of such a service for large group of potential clients. Awareness of the shortcomings of the existing models of online psychological assistance led to the creation of Elomia chatbot, in which the developers tried to combine the advantages of psychological online services: cheapness, round-the-clock availability and an individual approach to solving client problems. Elomia was developed using artificial intelligence technologies. In the process of developing a chatbot following technologies were used: 1. RoBERTa(NER) for identification names and locations; 2. COSMIC for identification of human emotions; 3. DIALOGPT for answers generation. 4. DistilBERT (SQuAD) for pointing the most relevant information in the script; 5. GECToR for spelling check. Nowadays, artificial intelligence is widely used in medicine: 1) in the diagnosis of cancer through the analysis of MRI and ultrasound images; 2) to predict the likelihood of Parkinson's disease; 3) to find the optimal way of surgical intervention; 4) to build a course of treatment. This is just a short list of the areas in which the use of artificial intelligence can save lives today.Elomia is able to identify the main psychological problems of the client and offer him the most suitable support option using first aid techniques and cognitive-behavioral psychotherapy. Elomia's arsenal includes:  exercises for calming;  exercises for falling asleep;  grounding technique;  exercises to reduce anxiety;  breathing exercises;  exercises to improve self-esteem. The algorithm determines the user's need for one or another help while communicating with the chatbot and suggests an exercise to ease his emotional state. The aim of the study was to test the effectiveness of Elomia chatbot in reducing the tendency to anxiety, depression and experiencing negative emotional states. 3. Research design The study consisted of three stages: 1. The formation of experimental and control samples; 2. Baseline testing; 3. Final testing. The study used psychological research methods: 1) Patient Health Questionnaire-9 (PHQ-9) to diagnose a tendency to depression; 2) General Anxiety Disorder-7 (GAD-7) to diagnose a tendency to generalized anxiety disorder; 3) Positive and Negative Affect Schedule (PANAS) to diagnose of prevailing (positive / negative) emotional affects. The PHQ-9 questionnaire reveals a tendency to depression at 5 levels: minimal depression (1-4 points), mild depression (5-9 points), moderate depression (10-14 points), moderately severe depression (15-19 points), severe depression (20-27 points) [9]. The GAD-7 questionnaire allows you to diagnose one of four levels of anxiety: minimal (0-4 points), mild (5-9 points), moderate (10-14 points), severe (15-21 points) [10]. The PANAS questionnaire allows to evaluate the tendency to positive and negative affects in the range from 10 to 50 points [11]. For statistical processing of the results methods of descriptive statistics, chi-square, t-test for paired and unpaired samples were used. The research sample was formed in several steps. At the first step 412 volunteers (202 women, 210 men, ages 19 to 23 years old) who identified their tendency to depression, anxiety and low mood were selected in students social networks groups of Kharkiv region. Another criterion for including respondents in the testing group was English proficiency at a B2 level (Upper Intermediate) and higher. After preliminary testing, 320 respondents (162 men and 158 women) who really showed a tendency to anxiety and depression according to testing were selected. At the second step, a group of 82 respondents (39 women and 43 men) was randomly selected from 320 respondents. At the third step, the respondents were randomly divided into 2 groups: experimental (42 people) and control (40 people). To check the distributions equality of respondents in groups by age and gender a chi square test was used (table 1). Table 1 Results of age and gender distributions equality checking with chi square Amount % Chi- Experimental Control Experimental Control p square group group group group Age 19 3 4 7.142 10 20 12 12 28.571 30 21 14 12 33.333 30 1.249 0.871 22 12 12 28.571 30 23 1 0 2.380 0 Gender Male 22 21 52.380 52.5 0.01 0.922 Female 20 19 47.619 47.5 Statistical analysis with usage of chi-square criterion confirmed the equality of the subjects distributions by age and gender (no significant differences in the distribution of data were found). Among selected participants there were: 13,41% with tendency to mild, 47,56% with tendency to moderate, 18,29% with tendency to moderate severe and 20,73% with tendency to severe depression; 18,29% with tendency to moderate and 81,7% with tendency to severe anxiety. The obtained groups were once again checked for compliance with the selection criteria of respondents (table 2). Table 2 Descriptive statistics of indicators of depression, anxiety and a tendency to negative affects at the first stage (baseline) of the study Std. Std. Error Group N Mean Deviation Mean Experimental group 42 14.952 5.516 0.851 PHQ-9 Control group 40 14.100 5.405 0.854 Experimental group 42 17.833 2.978 0.459 GAD-7 Control group 40 17.675 3.237 0.511 Positive Experimental group 42 26.619 7.190 1.109 affect Control group 40 27.750 5.960 0.942 Negative Experimental group 42 33.761 8.731 1.347 affect Control group 40 33.725 5.746 0.908 The data in the table clearly show that the average indicator of depression in the experimental and control group corresponds to the indicators of tendency to moderate (10-14 points) depression; the average anxiety score corresponds to a tendency to a severe (15-21 points) level of anxiety; the average indicator of tendency to negative affects corresponds to a moderate level (20-40 points); the average indicator of the tendency to positive affects corresponds to a moderate level (20-40 points). A comparison was also made of the average indicators of the respondents of the two groups using the T-test for independent samples on the level of tendency to anxiety, depression and emotional affects (table 3). Table 3 T-test results for independent samples in the experimental and control group at the first stage (baseline) of the study Levene's Test for Equality of t-test for Equality of Means Variances 95% Confidence Sig. Std. Interval of the Mean F Sig. t df (2- Error Difference Diff. tailed) Diff. Lower Upper PHQ-9 1.092 0.299 0.706 80 0.482 0.852 1.206 -1.549 3.254 GAD-7 0.050 0.824 0.231 80 0.818 0.158 0.686 -1.207 1.524 Positive 0.144 0.705 -0.773 80 0.442 -1.130 1.462 -4.041 1.779 affect Negative 1.269 0.263 0.022 80 0.982 0.036 1.640 -3.228 3.302 affect The obtained results of the T-test indicate the absence of significant differences in the level of the studied characteristics at the first stage of the study. After the completion of baseline testing, the respondents of the experimental group were given unlimited access to Elomia chatbot for 4 weeks. Respondents were instructed that they can use Elomia at any time of the day to the extent that they need. Thus, each participant in the study had the opportunity to control the amount of communication with Elomia, depending on their needs. Respondents in the control group were asked to use the Depression self-help guide developed by The National Health Service of the United Kingdom to correct negative emotional states [12]. This guide also contains a set of cognitive-behavioral techniques that help to deal with non-adaptive thoughts and to reduce anxiety and depression. Respondents were instructed that they can contact the guide at any time when they feel the need for it without restrictions. 4. Research results As a result of a survey after 4 weeks of the study it was revealed that the respondents of the experimental group used chatbot with different frequencies and intensities (Figure 1). Never used 2% Every day 14% Once per 4-5 times week per week 22% 17% 2 times per week 45% Figure 1: Frequency of Elomia usage in Experimental Group Retesting was performed in study groups 4 weeks after the baseline. To identify changes in the level of anxiety and depression in the study groups the T-test for paired samples was conducted separately in the experimental (table 4) and control (table 5) study groups. Table 4 T-test for paired samples for experimental group data Paired Differences 95% Confidence Std. Sig. (2- Std. Interval of the t df Mean Error tailed) Deviation Difference Mean Lower Upper Baseline -4 weeks PHQ-9 9.380 5.193 0.801 7.762 10.999 11.706 41 0.00 later GAD-7 13.476 4.203 0.648 12.166 14.786 20.776 41 0.00 Positive -7.023 5.993 0.924 -8.891 -5.155 -7.594 41 0.00 affect Negative 16.880 8.396 1.295 14.264 19.497 13.030 41 0.00 affect T-test showed significant differences in all study indicators. This means that users of Elomia after 4 weeks of its usage noted a significant decrease in the symptom of anxiety, depression and negative affects. Moreover, there is an increase in positive affects, which manifests itself in a more calm, balanced state, high self-esteem, confidence in the future. The participants in the experimental group noted that after using Elomia, they became calmer, more self-confident. Many noted that they have become less likely to experience aggression towards others, fear, sense of hopelessness. More than 70% of the participants noted that they returned to using the chatbot in moments of increased anxiety, panic attack, self-doubt, loneliness. Table 5 T-test for paired samples for control group data Paired Differences 95% Confidence Std. Std. Sig. (2- Interval of the t df Mean Deviatio Error tailed) Difference n Mean Lower Upper PHQ-9 -0.225 3.892 0.615 -1.469 1.019 -0.366 39 0.717 Baseline -4 weeks later GAD-7 1.175 3.713 0.587 -0.012 2.362 2.001 39 0.052 Positive 0.325 6.638 1.049 -1.798 2.448 0.310 39 0.758 affect Negative 0.775 2.536 0.401 -0.036 1.586 1.932 39 0.061 affect A similar comparison made in the control group did not detect significant statistical shifts, which indicates the inefficiency of introspection methods in dealing with signs of depression, anxiety and negative affects compared to using Elomia chatbot. PHQ-9 16.00 14.00 12.00 10.00 8.00 6.00 4.00 2.00 0.00 Baseline 4 weeks later Experimental group Control group Figure 2: Comparison of means in depression level in experimental and control group Comparison of the average indicators of depression tendency in the experimental and control groups allows us to evaluate the effect of the work of respondents with Elomia chatbot. So, after 4 weeks of using the chatbot the average indicator of depression tendency moved from the “Moderate Depression” zone to the “Mild Depression” zone, while in the control group it remained at the same level. The main reason for such results is that Elomia does not immerse the client in dull and lonely thoughts. On the contrary, it creates the conditions for self-disclosure of the person, gives the opportunity to speak out. In addition, the techniques presented in the application allow a person with depressive tendencies to reduce stress levels and feel a positive attitude towards themselves. GAD-7 20.00 15.00 10.00 5.00 0.00 Baseline 4 weeks later Experimental group Control group Figure 3: Comparison of means in anxiety level in experimental and control group The average indicator of tendency to anxiety in the experimental group moved from a zone of severe to a zone of moderate anxiety. In the control group the anxiety rate remained as high as during the baseline testing. These results are associated with the systematic use of grounding techniques that AI offers to anxious clients. While in a state of anxiety, a person often becomes obsessed with unproductive experiences and thoughts that prevent him from finding contact with a calming reality. Grounding techniques allow a person to return to reality through simple, repetitive actions (drawing, creating shapes with simple objects). Elomia allows clients to use these techniques on their own in times of anxiety or fear. PANAS (Positive affect) 40.00 30.00 20.00 10.00 0.00 Baseline 4 weeks later Experimental group Control group Figure 4: Comparison of means in positive affect level in experimental and control group The tendency to positive affects in the experimental group increased, while in the control group there was a slight decrease in this indicator. PANAS (Negative affect) 40.00 35.00 30.00 25.00 20.00 15.00 10.00 5.00 0.00 Baseline 4 weeks later Experimental group Control group Figure 5: Comparison of means in negative affect level in experimental and control group The tendency to negative affects in the experimental group decreased, while in the control group remained unchanged. The participants in the experimental group, after using the chatbot, stopped experiencing fear, nervousness, shame and distress as the leading emotions. In addition, there was a decrease in emotions such as irritability, guilt and hostility. Respondents noted that a decrease in negative experiences led to better relationships with family and friends, which in turn created an additional circle of psychological support around them. Thus, the use of the chatbot contributed not only to the reduction of negative feelings directed by a person towards himself, but also the manifestation of negative emotions directed to others. 15.00% 10.00% 5.00% 0.00% Experimental -5.00% PHQ-9 GAD-7 PANAS PANAS Group (Positive (Negative -10.00% affect) affect) -15.00% Control Group -20.00% -25.00% -30.00% -35.00% -40.00% Figure 6: Comparison of changes in highest level of study indicators (%) During the final interview, the respondents who used the chatbot noted that during communication they became more self-confident, started to understand life problems differently, began to think about how they live, felt “the ground under their feet”. As the most significant change, participants noted the change in the outlook on the world around them as more positive, friendly and filled with resources. Having received support from the chatbot, the participants realized that it was not a shame to ask for help and it was absolutely normal, which took their thinking out of a dead corner. Respondents in the control group also noted the usefulness of the depression self-help guide. But they also pointed out that in the process of reading the manual they had questions and ideas that they had no one to discuss with, and attempts to introduce changes into their lives without outside support came up against resistance from relatives. The inability to bring changes to life reduced the motivation to re-visit the guide and increased the overall level of frustration and self-discontent. 5. Conclusions Regular usage of Elomia contributes to a significant reduction in the high tendency to depression (up to 28%), anxiety (up to 31%), and negative affects (up to 15%). This reduction is achieved through the use of "conversational therapy" with elements of cognitive-behavioral techniques. At the same time usage of such techniques without a conversational elements does not lead to significant changes in the emotional state of clients within the indicated time frames. Elomia is a powerful tool for providing first aid to people suffering from anxiety and depression. Its systematic use can reduce the level of negative affects. It is important to note that it cannot act as a full-fledged substitute for psychotherapy or medical treatment for depression. Rather, it reduces the chances of the person using it to experience serious mental illness. 6. References [1] Depression, 2020. URL: https://www.who.int/news-room/fact-sheets/detail/depression. Accessed on: February 22, 2021. [2] A. Biaggi, S. Conroy, S. Pawlby, and C. Pariante, "Identifying the women at risk of antenatal anxiety and depression: A systematic review", Journal of Affective Disorders, volume 191, 2015, pp. 62–77. doi:10.1016/j.jad.2015.11.014. [3] D. Chisholm, K. Sweeny, P. Sheehan et al., "Scaling–up treatment of depression and anxiety: a global return on investment analysis", Lancet Psychiatry, volume 3, 2016, pp. 415–424. [4] K. Kroenke, F. Baye, and S. G. Lourens,"Comparative validity and responsiveness ofPHQ- ADS and other composite anxiety-depression measures", Journal of Affective Disorders, volume 246, 2019, pp. 437–443. https://doi.org/10.1016/j.jad.2018.12.098. [5] V. Brenninkmeijer, S. E. Lagerveld, R. W. B. Blonket al., "Predicting the Effectiveness of Work-Focused CBT for Common Mental Disorders: The Influence of Baseline Self-Efficacy, Depression and Anxiety", Journal of Occupational Rehabilitation, volume 29, 2019, pp. 31–41. https://doi.org/10.1007/s10926-018-9760-3. [6] J. Firth, W. Marx, S. Dash et al., "The Effects of Dietary Improvement on Symptoms of Depression and Anxiety: A Meta-Analysis of Randomized Controlled Trials", Psychosomatic Medicine, volume 81(3), 2019, pp. 265–280. doi:10.1097/PSY.0000000000000673. [7] M. G. Craske, A. E. Meuret, T. Ritz et al.,"Positive affect treatment for depression and anxiety: A randomized clinical trial for a core feature of anhedonia", Journal of Consulting and Clinical Psychology, volume 87(5), 2019, pp. 457–471. [8] A. Werner-Seidler,Y. Perry, A. L. Calear, J. M. Newby, and H. Christensen, "School-based depression and anxiety prevention programs for young people: A systematic review and meta- analysis", Clinical Psychology Review, volume 51, 2016, pp. 30–47. doi:10.1016/j.cpr.2016.10.005. [9] N. V. Pogosova, T. V. Dovzhenko, A. G. Babin, A. A. Kursakov, and V. A. Vygodin, "Russian version of PHQ-2 and 9 questionnaires: sensitivity and specificity in detection of depression in outpatient general medical practice", Cardiovascular Therapy and Prevention, volume 13(3), 2014, pp. 18–24. https://doi.org/10.15829/1728-8800-2014-3-18-24. [10] R. L. Spitzer, K. Kroenke, J. B. Williams, and B. Löwe, "A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7", Archives of Internal Medicine, volume 166(10), 2006, pp. 1092–1097. [11] Ye. N. Osin, "Izmerenie pozitivny`kh i negativny`kh e`moczij: razrabotka russkoyazy`chnogo analoga metodiki PANAS" [Measuring positive and negative emotions: developing a Russian- language analogue of the PANAS methodology], Psychology: journal of higher school of economy, 9, volume 4, 2012, p. 91–110. [12] Depression self-help guide: Work through a self-help guide for depression that uses cognitive behavioural therapy (CBT), 2020. URL:: https://www.nhsinform.scot/illnesses-and- conditions/mental-health/mental-health-self-help-guides/depression-self-help-guide. Accessed on: February 22, 2021.