Real-time sentiment analysis allows to monitor social networks and process negative comments before the situation worsens, gives an opportunity to gather customer response to the marketing campaigns or product launches and get an overview of how customers react to the product or prevent negative ones events determining the mood of people (posts on social networks, videos on YouTube, Twitch or live). The development of this system aims at testing the capabilities of the natural language processing system in the recognition of the Ukrainian language.
As computer technology goes beyond its artificial limitations, organizations are looking for new
ways to reap the benefits. The sharp increase in computing speeds and capabilities has led to new, highly
intelligent software systems, some of which are ready to replace or increase human services based
natural language processing (NLP) technology and Ukrainian dictionary [1-6]. The objective of our
research is to build an algorithm for recognizing emotions behind user text messages written in
Ukrainian based on linguistic analysis technology [
market.
In this section we provide a thorough analysis of analogue systems which are available on the
2022 Copyright for this paper by its authors.
The first one under discussion is Daylio programme. Its main advantages include the following points. The application has a special calendar that allows its users to record the mood by date and add notes (possibly about the events that affected it). It means that the application is effective in subsequent trips to a psychologist to prescribe treatment or analysis of the client. What is more, it supports interface localization in 28 languages (including Ukrainian). Its main drawback is that users can simply enter their mood manually without having support for virtual assistants or language recognition.
The next one considered is the MoodKit system. Among its benefits we should mention that it is an intuitive and easy-to-use application which is based on a well-studied therapy model and designed for people struggling with anxiety, stress and depression. Also, the advantage of this application is that it offers tips for dealing with a bad mood. One of its main disadvantages is that only English language is supported there. Besides, the MoodKit application is limited to iOS users and there is no free version of it. The user can enter his mood manually, not having support for virtual assistants and language recognition. The application has not yet gained enough users, so there is no rating for it in the App Store.
The last application we discuss is Worry Watch programme. While acknowledging its positive features, we note that the application works close to a personal diary and allows to record worries by date and set reminders after the event that caused the worries has passed. Thus, it helps to trace the analytics on how many of the experiences were correct. Its users’ rating is rather high. What comes to the negative effect of the programme, we should specify that it supports for 16 languages, but not including Ukrainian. What is more, Worry Watch is only available for iOS users in a paid version. And as it is with the mentioned above programmes, users can enter their mood manually without having support for virtual assistants and language recognition.
We have chosen Dialogflow ES to develop this system. Dialogflow Customer Experience (Dialogflow CX) was recently launched by Google. It provides a new way of designing agents considering the approach of the state machine to the design of agents. Such an approach gives a clear and sharp conversation control combined with a better end-user experience and workflow. The older version of Dialogflow, Dialogflow ES, short for Dialogflow Essentials, is still supported, but Dialogflow CX should allow higher-complexity chatbots to build more seamlessly with a visual editor and not to require developers to write complex code.
Another possible option which fits our needs is Cognigy programme. It is a corporate software platform that helps automate artificial intelligence on the conversational level. Thus, there are three services provided. They are Dialogflow ES, Dialogflow CX and Cognigy. Since Cognigy only works on a subscription basis for their services, we also exclude this service.
Among the two remaining services, it would be better to choose the version that has a built-in flow editor, but, unfortunately, at the moment Dialogflow CX does not support integration and limits the possibilities for processes not built in English. Therefore, the construction of this system is chosen to work with Dialogflow ES and integrate this system with a free flow editor. The designed system has three constituents: user, dialogflow, and system. They are shown in Fig.1.
The actions the user can do include the following: start working with the bot, answer the bot's questions, get a recommendation / motivation. Dialogflow recognizes the user's mood after the user answers the bot's questions. What comes to the system functions, they record user data (name, preferences) and select personal phrases for the user. After the user answers the bot's questions, the system records the user's data (name, preferences) and Dialogflow programme identifies the user's mood.
With the help of a cooperation diagram, you can describe the full context of interactions as a kind of time "slice" of a set of objects interacting with each other to perform a specific task or business goal of the software system. This diagram shows 3 objects: system, user, dialogflow (Fig. 2). They are connected by the following connections: 1. The user initiates work with the bot 2. The system asks questions to the user 3. The user answers the bot's questions 4. The system processes user data 5. The system sends responses to Dialogflow recognition 6. Dialogflow defines intent and entities 7. The system formulates a motivational response to the user 8. The system sends a response to the user
We develop and add training phrases to intents (Fig. 3-4).
In Figure 3, you can see that some words are highlighted in the training phrases. It shows that these entities carry extra emotional or informational meaning. For this purpose, entities with positive and negative shades of meaning are placed in one domain. With the aim of further dialogue analysis and gathering some analytics, they will be placed in different entities. First, all emotionally positive words are put into good domain. The domain name is written in English, as Dialogflow platform only recognises the characters A-Z, a-z, 0-9 for giving a name. And Fig. 4 includes a list of emotionally positive words of good domain.
To demonstrate how it works we will show here the results of the Test case #1. The description of the task requires to ensure that the Dialogflow recognizes a positive answer to the question "How are you today?" The Dialogflow console will be used to conduct a testing.
Instructions: to perform this test, one needs to log in to the Dialogflow console and type in the answer "good" in the input field.
2. Log in to the Dialogflow console 3. Select the input field on the right 4. Type in “good” 5. Get the result Expected Result: after recognizing the word “good”, Dialogflow should identify:
One of the three text answers: a. Thank you for trusting me! Did you have the opportunity to meet friends? b. I'm very interested, let’s continue)) In fact, to meet your loved ones helps a lot to relieve a daily stress. Have you had such an opportunity recently? c. I'm here to talk to you about it. Did you have a chance to meet your family or friends last week? 2. Source contexts 3. A “good” domain
Case status. As based on the case results, we should admit that the expected output is achieved. The test results are presented in Fig. 5. In this test case, a positive answer to the question “How are you today?” is successfully recognized by the Dialogflow programme. The identification of the required intent and domain is completed.
Output. To reset the system, select the Reset Context option in the Dialogflow console.
The next test we have conducted is the Test case #2. The description of the task requires to ensure that the Dialogflow recognizes a negative answer to the question “How are you today?” The Dialogflow console will be used to perform the testing.
Instructions: to perform this test, one need to log in to the Dialogflow console and write “bad” in the input field.
2. Log in to the Dialogflow console 3. Select the input field on the right 4. Type in “bad” 5. Get the result
Expected Result: After recognizing the word “good”, Dialogflow should determine:
One of three text answers: a. Thank you for trusting me! Did you have the opportunity to meet friends? b. I'm very interested, let’s continue)) In fact, to meet your loved ones helps a lot to relieve a daily stress. Have you had such an opportunity recently?
c. I'm here to talk to you about it. Did you have a chance to meet your family or friends last week? Source contexts
A “good” domain
Case status. Considering the case results, we may confirm that the expected output is gained. The test results are presented in Fig. 6. In this test case, a negative answer to the question “How are you today?” is successfully recognized by the Dialogflow programme. The identification of the required intent and domain is completed.
Output. To reset the system, select the Reset Context option in the Dialogflow console.
To check whether the Dialogflow programme identifies a complex positive answer to the question “How are you today?”, the Test case #3 is conducted. The necessary condition is that the training phrases are not included in the list. The Dialogflow console will be used to perform the testing.
Instructions: to perform the test, one needs to log in the Dialogflow console and give an answer “Actually, I feel great today” in the input field.
Steps:
Expected result: after recognizing the sentence “Actually, I feel great today” Dialogflow should determine:
1. One of three text answers: a. Thank you for trusting me! Did you have the opportunity to meet friends? b. I'm very interested, let’s continue)) In fact, to meet your loved ones helps a lot to relieve a daily stress. Have you had such an opportunity recently? c. I'm here to talk to you about it. Did you have a chance to meet your family or friends last week?
Source contexts 3. A “good” domain
Case status. The case corresponds to the expected result. The test results are presented in Fig. 7. In this test case, a positive answer to the question “How are you today?” is successfully recognized by the Dialogflow programme with the identification of the required intent and domain.
Output. To reset the system, select the Reset Context option in the Dialogflow console.
To review the ratio of the number of recognized phrases to the number of unrecognized phrases among the total number of phrases used in testing, a diagram is developed (see Fig. 8). The total number is 70 phrases, 54 out of them are successfully recognized, and 16 belong to the Default Fallback Intent.
On the pie chart it is shown that more than 77% of the entire sample of 70 phrases is successfully recognized. Such statistics indicate that the agent has been trained successfully.
Our next step is to consider the number of training phrases which contain keywords – entities to improve recognition. Of the sample of 70 phrases, only 26 phrases include entities (see Fig. 9).
Taking into account the ration in Fig. 9, we may conclude that the agent is well trained to recognize the context of phrases even if they do not include predefined entities. It is proved by the fact that the number of 62.9% of recognized phrases has been successfully identified without keywords.
In Fig. 10, it is shown a bar chart with the number of training phrases differentiated according to the level and their recognizability.
The results derived in the diagram indicate that the 1st level intent is called the most times, and the 5th level intent is the least one. In addition, we can conclude that among all the intents, the best recognized is the intent of the 3rd level, and the lowest – of the 5th level. It means that further research and investigation of training phrases should be conducted.
To conclude, we would like to state that the development of NLP is experiencing its rapid progress and its application in various areas of science contributes to the general technology-oriented approach. In our research, its application resulted in developing of 32 custom intents, 2 system intents, 2 custom entities, one system domain, and 300+ training phrases as based on Dialogflow programme. Besides that, 16 intents are set as the end of the conversation. Because of the complexity of the Slavic word formation system it is a challenging task to use NLP systems in this field. In practice, it means that the recognition of the Ukrainian language by NLP systems is highly complicated due to its extensive system of inflections. Further research in this area would be a valuable contribution both to the development of NLP and Ukrainian language software advances in technology.
[1] N. Tmienova, B. Sus, System of Intellectual Ukrainian Language Processing, CEUR Workshop
Proceedings Vol-2577 (2019) 199-209. URL: http://ceur-ws.org/Vol-2577/paper16.pdf. [2] Large Electronic Dictionary of the Ukrainian Language (VESUM) as a tool of NLP. URL: https://www.researchgate.net/publication/344842033_Velikij_elektronnij_slovnik_ukrainskoi_m ovi_VESUM_ak_zasib_NLP_dla_ukrainskoi_movi_Galaktika_Slova_Galini_Makarivni_Gnatuk [3] Ukrainian tonal dictionary. URL: https://github.com/lang-uk/tone-dict-uk. [4] Brown Corps of the Ukrainian language. URL: https://github.com/brown-uk/corpus. [5] NER-text markup. URL: https://github.com/lang-uk/ner-uk/blob/master/doc/README.md. [6] Microservices lang-uk. URL: https://lang.org.ua/uk/services/. [7] P. Zhezhnych, A. Shilinh, V. Melnyk, Linguistic analysis of user motivations of information content for university entrant’s web-forum, International Journal of Computing 18 (2019) 67-74. [47] M. Maree, M. Eleyat, Semantic graph based term expansion for sentence-level sentiment analysis,
International Journal of Computing 19(4) (2020) 647-655. [48] S. Shrivastava, K. V. Lakshmy, C. Srinivasan, On the Statistical Analysis of ZUC, Espresso and
Grain v1, International Journal of Computing 20(3) (2021) 384-390. [49] V. Turchenko, V. A. Golovko, A. Sachenko, Parallel Batch Pattern Training of Recirculation Neural Network, in: Proceedings of the 9th International Conference on Informatics in Control, Automation and Robotics, ICINCO, 2012, 1, pp. 644–650. [50] Md Maksudur R. Mazumder, C. Phillips, Partitioning known environments for multi-robot task allocation using genetic algorithms, International Journal of Computing 19(3) (2020) 480-490. [51] M. Patil, T. Abukhalil, S. Patel, T. Sobh, UB SWARM: hardware implementation of heterogeneous swarm robot with fault detection and power management, International Journal of Computing 15(3) (2016) 162-176. [52] V. M. Hung, V. Mihai, C. Dragana, I. Ion, N. Paraschiv, Dynamic computation of haptic-robot devices for control of a surgical training system, International Journal of Computing 17(2) (2018) 81-93. [53] V. Vysotska, O. Kanishcheva, Y. Hlavcheva, Authorship Identification of the Scientific Text in Ukrainian with Using the Lingvometry Methods, in: Proceedings of the International Conference on Computer Sciences and Information Technologies, CSIT, 2018, pp. 34-38. doi: 10.1109/STCCSIT.2018.8526735. [54] V. Vysotska, V.B. Fernandes, V. Lytvyn, M. Emmerich, M. Hrendus, Method for Determining Linguometric Coefficient Dynamics of Ukrainian Text Content Authorship, Advances in Intelligent Systems and Computing 871 (2019) 132-151. doi: 10.1007/978-3-030-01069-0_10. [55] V. Vysotska, Ukrainian Participles Formation by the Generative Grammars Use, CEUR workshop proceedings Vol-2604 (2020) 407-427. [56] V. Vysotska, S. Holoshchuk, R. Holoshchuk, A comparative analysis for English and Ukrainian texts processing based on semantics and syntax approach, CEUR Workshop Proceedings Vol-2870 (2021) 311-356. [57] K. Tymoshenko, V. Vysotska, O. Kovtun, R. Holoshchuk, S. Holoshchuk, Real-time Ukrainian text recognition and voicing, CEUR Workshop Proceedings Vol-2870 (2021) 357-387. [58] A. Dmytriv, V. Vysotska, M. Bublyk, The Speech Parts Identification for Ukrainian Words Based on VESUM and Horokh Using, in: Proceedings of the IEEE 16th International Conference on Computer Sciences and Information Technologies (CSIT) :, 22-25 Sept., Lviv, Ukraine, 2021, Vol. 2, pp. 21–33. doi: 10.1109/CSIT52700.2021.9648813. [59] L. Savytska, N. Vnukova, I. Bezugla, V. Pyvovarov, M. Turgut Sübay, Using Word2vec Technique to Determine Semantic and Morphologic Similarity in Embedded Words of the Ukrainian Language, CEUR Workshop Proceedings Vol-2870 (2021) 235-248. [60] M. Sazhok, A. Poltieva, V. Robeiko, R. Seliukh, D. Fedoryn, Punctuation Restoration for Ukrainian Broadcast Speech Recognition System based on Bidirectional Recurrent Neural Network and Word Embeddings, CEUR Workshop Proceedings Vol-2870 (2021) 300-310. [61] O. Cherednichenko, O. Kanishcheva, Readability Evaluation for Ukrainian Medicine Corpus (UKRMED), CEUR Workshop Proceedings Vol-2870 (2021) 402-412. [62] A. Luchyk, O. Taran, O. Palchevska, N. Sharmanova, G. Demydenko, Corpus-Driven Approaсh to Ukrainian Е-Anecdotes Study, CEUR Workshop Proceedings Vol-2870 (2021) 424-434. [63] V. Starko, Implementing Semantic Annotation in a Ukrainian Corpus, CEUR Workshop
Proceedings Vol-2870 (2021) 435-447. [64] Н. Sytar, O. Vietrov, V. Diachenko, Synonymizer of the Ukrainian Language: Stage of Creation, Features of Database Update and Software Implementation, CEUR Workshop Proceedings Vol2870 (2021) 448-458.