Trends in any commercial activity show that making marketing decisions that will provide the greatest impact on the consumer in making decisions to maximize profits are relevant today. In recent years, in the area of marketing, the subject of intensive research has become nonverbal information, namely the study of facial expressions. It is known from psychology that all human emotions can be classified into six basic emotions, which are most used to obtain nonverbal information. The ability to automatically recognize this kind of information will simplify the interpretation of emotions on a person's face while watching advertising, product testing or using the service.
The proposed approach will help to understand whether the consumer really liked the product, what color, size or smell he prefers, etc., as the survey can often get inaccurate information. The proposed approach will allow you to see the informal reaction of users, which will help to understand what necessary to focus on and what to improve.
The development of emotion recognition in the vast majority of methods occurs in three steps [1]. In the first step, functions are defined from fixed images, and in the second step, emotions are detected with the help of already developed classifiers and in the third step is face recognition itself. The most common are Local Binary Patters (LBP) [2] is a description of the pixels around the central pixel in binary form. The COLINS-2021: 5th International Conference on Computational Linguistics and Intelligent Systems, April 22-23, 2021, Kharkiv, Ukraine EMAIL: spivak.iruna@gmail.com (I. Spivak); msya220189@gmail.com (S. Krepych); fedorov.oleks@gmail.com (O. Fedorov); spivaksm@ukr.net (S. Spivak) ORCID: 0000-0003-4831-0780 (I. Spivak); 0000-0001-7700-8367 (S. Krepych); 0000-0002-8080-9306 (O. Fedorov); 0000-0002-7160-2151 (S. Spivak) LBP operator is applied to the central pixel of the image, it uses 8 pixels that are around it taking the central pixel as the main. The main disadvantage of this method is that the image needs high-quality preprocessing due to high sensitivity to noises. Geometric approach to face recognitionis one of the first developed methods it consists of choosing the key points of the face such as lips, center of the eye, etc. This method does not require an expensive equipment but it affects the low reliability of the method. Viola-Jones method and Haar features [3,4,13,15] -is the most popular method of finding the facial area in images because of its relatively high speed and efficiency. Face recognition in this method is based on three basic principles: Active appearance model (AAM) -are statistical models of images that can be adjusted to the real image by various deformations. Fitting the model to a specific image of the face is performed in the process of solving the optimization problem, the essence of which is to minimize the functionality.
However, these approaches require a lot of time and resources for training, which limits their use in a large sample of input data [7-9, 11, 12]. The article proposes an approach to recognizing the emotions of the human face by definition and changing the positions of key points of the eyes, mouth and eyebrows, which does not require large computational resources for its implementation.
Emotion is one of the basic elements of the human psyche. A human's emotions are distinguished depending on the state of satisfaction of his needs. They can be both positive and negative, as well as neutral, when a person doesn't react in any way and remains in its original state. For example, looking the advertising -some people feel anger or disgust, while others -pleasure and interest. A person's behavior also changes depending on what emotions the person is experiencing.
When making marketing decisions, the automated system must be able to identify and recognize one of the six basic human emotions:
Surprise -a short-term feeling and state of a person that occurs during a sudden and unexpected situation.
Fear is a state of anxiety and restlessness caused by the expectation of something undesirable or unpleasant.
Disgust -a feeling of disapproval towards someone or something.
Angry is a strong feeling of dissatisfaction that arises when a person's needs or expectations have not satisfied.
Happy is a feeling of satisfaction that arises when a person's needs or expectations have been met.
Sad is a feeling opposite to happy, which arises in case of loss and helplessness of a person. In the future, depending on the emotion reflected on the face, this area will increase or decrease. For example, let's take two emotions: disgust and happy, which are very similar when identifying key points. In both, the oral area increased and the eye area decreased. Once the program determines this, the search for the next key points will be narrowed only to these two emotions, and will not pass all six, which will save time to identify the required final emotion, in contrast to existing methods and approaches [5,6,14].
Our performed analysis of this issue showed that to determine a human's emotions it is enough to choose the key elements on the face, namely the eyes, eyebrows, nose and mouth, and not to identify the whole face. The algorithm of proposed approach includes the following steps:
Step 1. Determining a face image from a photo or video and convert it into black and white using the capabilities of the CSS-filter function, which can be implemented on any PC hardware, as it doesn't require large resources.
Step 2. Selection of key elements of the face and their processing in the HSL (Hue Saturation Luminance) color model (Fig. 6), where Hue -means both color and hue; Saturation -indicates the amount of gray color; Luminance is the intensity of light projected on a given area and direction. Step 3. Each pixel of the photo is replaced by its numerical value (the darker the hue of black, the smaller the number and vice versa) (Fig. 7). These numerical values will be used to search for key points of the selected facial elements based on the algorithm for finding the nearest neighbor, namely the hue of black. When the hue of darkness decreases by more than 15% (perhaps more, its need to check by the software)this means that the next pixel (depending on its direction) doesn't need to be estimated. This will give us clear contour of the element we are estimating.
Based on the emotion of fear, we will show how a human's face will change, and how software will be able to detect it. For example, from the fixed image of the face the element "eye" for research is determinate. 9a) shows the image of the human eye in a neutral state, while Figure 9b) -the expansion of the eye during the action of the emotion of fear. It is known that when a person is afraid, his eyes expand and their area increases accordingly. Counting the number of pixels from the extreme left point (1) to the extreme right (2) we get 31 pixels in the neutral state and 32 pixels in the fear state. The difference of 1 pixel is not significant, because during fear the eyes cannot increase in width, only in height. Counting the number of pixels from the uppermost point (3) to the lowermost point (4) we get 17 pixels in the neutral state and 23 pixels in the fear state. Now that we have this data, we can calculate the area of the eyes, which will show us its increase during the action of the emotion of fear.
The next key point is the eyebrows. 10b) -under the influence of fear. As we can see in a neutral emotional state, the distance between the eyebrows (from point 2 to point 1a) is 37 pixels, the distance between the eyebrows in a state of fear -32 pixels. You should also pay attention to the distance from the eyes to the eyebrows. In neutral state, the distance for the left eyebrow is 13 pixels and for the right 8. In a frightened person, this distance is 4 pixels for the right and left eyebrows.
The last key point in recognizing the emotion of fear is the corners of the lips. 11a) shows the area of the human mouth in a neutral state and it is 21 pixels. In the state of fear person opens his mouth and, accordingly, changes its size. In our case it is 28 pixels (Fig. 11b).
According to the study, we can generate table 1, which contains data on the change of distance in pixels between the extreme points of the selected elements of the face to facilitate the identification of the image of emotion. According to the data in Table 1, the programing search for the emotion fixed in the image will be as follows. First, for example, choose one element "eyes" and check it for conformity with the characteristics of emotions when they are detected (Fig. 12).
The figure shows that the further search will continue in one of three directions if eyes are dilated (Fear, Surprise or Disgust) or (Angry or Sad) if they are squinted, and for the last if eyes are unchanged (Happy or Neutral Emotion) can be applied. The next steps are to cut off unnecessary emotions by checking other characteristics. To determine the change in the position of key points of the eyes, eyebrows and mouth, the ranges of these changes should be set in the interval forms [10], which will take into account the physiological characteristics of human faces.
Research in the field of psychology has shown that the emotional state of all people has common external features. This made it possible to develop a universal classifier of emotions, which can be used to determine person's state. The article proposes an approach to the recognition of visualized human emotions using a pixel color model, which has the ability to adapt to changes in input data less time consuming compared to other existing methods, high speed and low resource usage.
The proposed approach has practical value in marketing decision-making systems based on the analysis of a human's emotional state while viewing or testing a particular product or service. The article presents an algorithm for step-by-step identification of visualized human emotion based on the comparison of changes in the positions of key points of the selected element in accordance with changes in the characteristics of this element. Further research will focus on the development of automated methods and algorithms for recognizing human emotions, taking into account the physiological characteristics of the human face, gender, age and more. This consideration is necessary and appropriate, because the physiological features of the facial structure of men and women are different: the location of the eyebrows, their width, the shape of the nose, the shape of the lips and their thickness.