Introduction

Increasing the number of e-commerce entities rises a difficulty for a company to be competitive. In order to solve this problem, recommendation systems (RSs) are being actively developed to overcome situations of the uncertainty of users' choice. Significant development is gaining hybrid recommendation systems (HRSs). Modern RSs are created to overcome a certain problem (for example, a cold start or a long tail of products). But the scope of product evaluation in recommendation systems remains insufficiently developed. Approaches to evaluation are based on the characteristics of specific types of RSs (content-based, collaborative filtering, hybrid) and the goal that the evaluation should provide. There is a requirement to form an alternative evaluation model that would consider the specifics of the RSs. Therefore, this work is devoted to the development of a new model of product evaluation in RSs for e-commerce. An analysis of the most popular e-commerce sites in Ukraine shows that a limited number of the simplest algorithms are used. Sites use in most cases only direct user information (history of purchases, products viewed and products, that are added to the wish list). Other recommendations are based on information collected from all consumers of the company. An urgent task for the current e-commerce market in Ukraine is to improve the forms and models of product evaluation. The study aims to form an alternative approach to develop product evaluation in hybrid recommendation systems and to propose a model of recommendation system of such a type that is relevant to the needs of the modern e-commerce market.

According to the purpose, the following tasks are formed:  outline the key features of modern HRSs;  analyze existing approaches to product evaluation;

Review of the current state of development of hybrid recommendation systems

In this section, we will consider progressive modern examples of HRSs and point out the necessity to develop an approach to increase the accuracy of evaluation models. The study of the presented systems allows us to analyze the state and vector of development of RSs in general. On the other hand, the review of these systems will help to form the uniqueness of the developed model. Using methods of one type of recommendation system to calculate a feature or set of features that further is the input to a recommendation system of another type Cascade

The result of one type of the recommendation system uses as an input signal to the methodology of another type of the recommendation system Meta-level

The technique of one type of recommendation system is used, that establishes a certain model, which further is being used as a contribution of the technique of the other type of the recommendation system

Collaborative RS to overcome the problems of cold start. The authors' study [5] focuses on improving traditional similarity measurements for co-element-based filtering to address and mitigate cold start situations. The paper proposes an algorithm that is designed to balance three modern traditional measurement metrics, such as cosine-based similarity, Pearson correlation similarity and adjusted cosine similarity in the direction of cold-start situations.

HRS with a focus on the long tail. Research [15] considers the influence of preferences of individual consumers on new products. The research uses an extension of the matrix factorization model to predict the rating. Models (PM-1, PM-2) are proposed, which demonstrate the suitability to 1) provide personalized recommendations to the user, considering the appropriate taste of objects with a long tail; 2) advertise items with a long tail to idiosyncratic users HRS based on recommendations and tonality. In [10], a RS based on a hybrid analysis of recommendations and tonality is proposed. F-metric is used -weighted average harmonic accuracy and completeness, which allows more effectively assess the work of the model in terms of a fuller perspective. The model proposed in the study is more powerful in its ability to identify relevant films to the user.

HRS to active users. The authors of the study [11] built a novel RS, which combines the methods of collaborative filtering and matrix factorization. The RS analyzes only users and items that are related to active users. Novel RS uses genetic algorithms to estimate the rates of invaluable items of the active user. The considered RS for joint filtering uses neighborhood models and hidden factor models to recommend elements for the active user (customer).

HRS for finding educational materials. The study [16] proposes an improved method for the existing system of e-learning recommendations with a combination of content-based filtering and collaboration filtering with good student ratings (CBF-CF-GL method). The method of admission for analysis of students with only high scores can be used in e-commerce, for example, through the selection for analysis of users with costs in a specific range that is relevant to the costs of the target user. HRS based on visualization. For this purpose, [2] combines a collapsed neural network and a Bagdanau attention mechanism. As a result, the method allows identifying areas that were particularly important for the image that is recommended.

Telemetry HRS for a web browser. The authors [17] presented a recommendation add-on system with telemetry information (Telemetry-Aware Add-on Recommender), which provides recommendations for Firefox users. Three separate models are used, based on three data sources: a set of add-ons that the user has already installed; usage and interaction data (browser telemetry); user browser language settings. The research builds individual models of recommendations for each model separately and summarizes the recommendations that they generate using the method of linear ensemble composition. HRS with a combination of taxonomy and folksonomy. According to [13], an algorithm for reconciling the establishment of general similarity between elements is developed, where tag information is integrated for semantic analysis of taxonomy attributes. The research proposes a unique model of random sampling on a heterogeneous graph, built by user nodes, element nodes and different types of relationships -user-element, element-element.

HRS of consideration of commodity networks. Research [18] analyzes the relationship between product networks generated by RSs and the convergence of product ratings. The paper considers how different types of product networks are related to consumers' perception of quality between product pairs in the product network. In addition, this study examines whether the types of product networks are differently related to the convergence of attitudes towards products in the product network.

HRS considers the coefficients of similarity of profiles and demographic differences. In [9], the method of calculating the similarity coefficients of user profile vectors and object profile vectors was further developed, which, in contrast to the basic ones, uses the demographic characteristics of users, which allows increasing the accuracy of forecasting recommendations. Based on the concept of application in one method of categorical, mixed and numerical clustering, a method of searching for user groups has been developed, which adapts to the sparseness of the user-object matrix.

HRS with partial variation autoencoder method. The authors [14] proposed a method of hybrid recommendation, which basically processes the missing data and uses the amortized output for quick forecasting. The method calls as Partial Varial Autoencoder (P-VAE). The method uses a new probabilistic generative model to process a different number of user ratings in principle.

HRS for comparing product description and user profile. The study [19] developed a hybrid system of recommendations for e-commerce, which implements content-based filtering and collaborationfiltering that calculates the similarity of product description and user profile.

HRS with partial variation autoencoder method. The authors of [20] used the built-in categories obtained in the content-based approach and the embedding of clients in the shared filtering approach. It is used as input to the new neural network and will check if it is relevant to the recommendations. The developed model can be used in two different modes: use to organize the recommendations classified by previous approaches; use to make your own recommendations.

The study proposes HRS that differs from previous models by improving the modeling accuracy of product evaluation. The proposed evaluation model considers a wider range of relevant indicators.

Review of product evaluation methods and elaboration of an alternative evaluation model

The development of effective RSs requires the use of appropriate mathematical apparatus. An average rate is the simplest type of metric used in recommendations. It is applied in the case when it is necessary to obtain the minimum calculation time. The rating 𝑟 𝑖𝑗 ′ is the arithmetic mean of the ratings 𝑟 𝑖𝑗 , 𝑖, 𝑗 = 1, 𝑛 ̅̅̅̅̅ . A simple recommendation has the following formula:

𝑟 𝑖𝑗 ′ = ∑ 𝑟 𝑖𝑗 𝑛 𝑗=1 𝑛 . (1)

Another form of metric employs the proportion of positive ratings to offset the uncertainty of a limited number of observations. The limits of Wilson score of the confidence interval for the Bernoulli parameter [21,22] is referred to one type of mathematical equipment. The goal is to use the generated rating after considering the current set of user ratings as a statistical sample of a hypothetical set of user ratings from all users.

𝑝̂+ 𝑧 2 𝛼 2 2𝑛 ± 𝑧 𝛼 2 √[𝑝̂(1 − 𝑝̂)] + 𝑧 2 𝛼 2 1 + 𝑧 2 𝛼 2 2𝑛 , (2)

where 𝑝̂ -the observed share of positive evaluations; 𝑧 2 𝛼 2 -2 quantile of the standard normal distribution; 𝑛 -the total number of rates.

Wilson's rating has the drawback of assigning zero value to products that have not gotten any positive customer feedback, as well as new products, that have received any feedback.

The Bayesian approximation is another metric that can be used to evaluate a product when measured on a K-star scale [22]. The lower limit of the normal approximation to the Bayesian confidence interval for the mean estimate is given by the following formula.

𝑠(𝑛 1 , … , 𝑛 𝑘 ) = ∑ 𝑠 𝑘 𝐾 𝑘=1 𝑛 𝑘 + 1 𝑁 + 𝐾 − 𝑧𝛼 2 √ ((∑ 𝑠 𝑘 2 𝐾 𝑘=1 𝑛 𝑘 + 1 𝑁 + 𝐾 ) − (∑ 𝑠 𝑘 𝐾 𝑘=1 𝑛 𝑘 + 1 𝑁 + 𝐾 ) 2 ) 𝑁 + 𝐾 + 1 ,(3)

where 𝑠 𝑘 to 1-point, .... N-point scale; 𝑁 = total score with 𝑛 𝑘 scores for scale 𝑘.

One of the simplest techniques to assign nonzero probabilities to invisible terms is Laplace smoothing, which assumes that each element "by default" has 1 thumb up and 1 thumb down [23].

𝑠(𝑛 ↑, 𝑛 ↓) = Pr[↑ |𝑀] = 𝑛 ↑ +1 (𝑛 ↑ +1) + (𝑛 ↓ +1) (4)

Although Laplace smoothing avoids most of the disadvantages of popular methods (for example, gaining zero probability for invisible user ratings), this is probably too large for pseudo-counts. A more rational choice is a more generalized form -Leadstone smoothing, which assumes that each element "by default" has fingers up and fingers down:

𝑠(𝑛 ↑, 𝑛 ↓) = Pr[↑ |𝑀] = 𝑛 ↑ +∈ (𝑛 ↑ +∈) + (𝑛 ↓ +∈) ,(5)

where ∈> 0 -parameter.

Studies have shown that the efficiency of Lidstone smoothing with 0 <∈< 1 usually exceeds ∈= 1 (Laplace smoothing) [23]. The Hacker metric is primarily designed to rank news for Hacker News to address hot and new articles using gravity (g) and time (t). However, this algorithm can be applied to e-commerce, considering the period between the provision of recommendations to the user and the considered comments, evaluations of other users [24].

𝑟 = 𝑝 − 1 (𝑡 + 2) 𝑔 (6)

where 𝑝 -points by position (p -1 -to cancel the vote of participants); 𝑡 = time since submission (in hours); 𝑔 = gravity, default 1.8 in news.arc.

The most widely used in terms of collaborative filtering is the formation of the user similarity function based on Pearson correlation and cosine similarity [24]. The cosine measure of similarity is popular in many collaborative RSs for calculating similarity coefficients between object profile vectors. It gives an unsatisfactory result in the presence of several subjects with the same ratings. Also poorly considers differences in the values of the components of rating vectors. Pearson's correlation coefficient gives incorrect values if both profile vectors have several identical rates. It also calculates incorrect values for a small quantity of the set of components of the profile vectors between which the degree of similarity is calculated. This coefficient may have a large or low value of similarity despite the similarity or difference in ratings. The limited Pearson correlation coefficient makes a slight improvement in the normal Pearson correlation coefficient. It also calculates incorrect values in the presence of even a small number of identical ratings in both profile vectors. The root mean square difference does not consider the same values of ratings in the vectors of user profiles. Jacquard's similarity factor does not consider the absolute values of the ratings. In the assessment of Jacquard + root mean square there are the same shortcomings as in each assessment. The study proposes an original product evaluation model based on such methods as Wilson 's score, Bayesian approximation and Hacker ranking algorithm. The proposed evaluation model can be calculated by the formula:

𝑅 = 𝑎 (𝑡 + 2) 𝑔 √𝑟_𝑤 2 + 𝑏 × 𝑟_𝑏 2 ,(7)

where 𝑟_𝑤 -Wilson's score; 𝑟_𝑏 -Bayesian approximation; 𝑟_ℎ -Hacker ranking algorithm.

In the calculation of the rate R, the Wilson and Bayes metrics are combined as the root mean square value of their values by their normalization. The coefficient b normalizes the values of these estimates among themselves. The user may prefer a particular estimation method or set the coefficient so that the maximum value of the estimates is within the same range. The Hacker method is used to adjust the number of days the product is published. Instead of the indicator (p-1) the root mean square value is taken from the normalization of Wilson and Bayes metrics. The coefficient 𝑎 is set by the user only to expand the scope of the assessment and does not affect the essence of the calculation. Because the adjusted estimate is designed for e-commerce, the degree of adjustment for the number of days of publication should be less than for news (equal to 1.1). This assessment takes advantage of each of the methods and adjusts the value of the assessment if one of the methods would be equal to 0. At the same time, this allows you to form an assessment as the average for the entire sample and considers the division into positive and negative assessments (on a five-point scale). It is assumed that grades 4, 5positive, grades 1, 2, 3 -negative).

Conceptual bases of development of the hybrid recommendation system

The study proposes a version of the complex of RSs that provides recommendations based on content and collaborative filtering and solves current problems in e-commerce. The model is especially relevant for those markets where the state of development of RSs is low, such as in Ukraine. Thus, the Ukrainian e-commerce market is characterized by the fact that sites are limited to direct user information (history of purchases, viewed products and those added to the wish list). Other recommendations are based on information collected from all consumers of the company. There is a limited number of the simplest algorithms used to provide recommendations.

The developed RS provides recommendations in four categories, such as "Personalized recommendation", "Best buy", "News", and "Recommendation according to the survey". The key advantage is that the user has more choice, as he can choose which recommendation he wants to receive at a particular time.

Personalized recommendation. The HRS of the type "addition of features" has been developed: a demographic method of the content filtering of the system and a factorization matrix of the collaborative filtering. The method of calculating the similarity coefficients of user profile vectors and object profile vectors has been further developed, which allows increasing the accuracy of forecasting forecasts. The first step is to calculate the user's monthly costs on this platform for the last six months. Based on this data, other users who have monthly expenses in the same interval are filtered (intervals are set by the author). The next step is to form a user similarity function. The following methods of similarity formation were used in the study: Pearson correlation coefficient, Jacquard coefficient, inverse Euclidean distance, cosine measure of similarity. The similarity feature searches for multiple users with a similar purchase history. Subsequently, the recommendations are passed to the current user from similar users. Based on the calculation of the rating of products according to the developed model of score evaluation, recommendations and the sequence of their provision are formed. The advantage of this RS is that recommendations are appropriate for consumers` tastes and financial capabilities. The disadvantage is the need for constant recalculation with any purchase of new or similar users. The key parameters based on which the assessment is formed present in the Figure 1.

News. A model of a HRS has been developed, which combines a sequence of content recommendation filtering. In the terms of the approach, it is proposed to provide recommendations based on the similarity of the characteristics of past views of products by the user and the characteristics of objects. Modifications of the Pearson coefficient methods and the Hacker ranking algorithm are used together. Thus, the lack of a sufficiently large number of responses to a new product (cold start of production) is offset by its high time priority. At the same time, in this section of the recommendations, the analysis does not consider new products, which in turn have more reviews, but less time priority. As a result, the model balances newer and older products according to their benefits for the user. The advantage of this method is to solve the problem of cold start products and personalize the recommendations of not yet purchased, but revised products. The disadvantage is not considering the interest in the revised product. In the future, this problem can be solved by expanding the ability to track various e-commerce sites` time of active browsing. Despite the possibility of implementing this item, most companies do not enter this item in their user database. The key parameters based on which the assessment is formed present in the Figure 2. Best buy. A developed model of a direct collaborative RS with a direct connection. The first stage is the formation of a database of smaller dimensions than the initial number of purchased products in the last week. The next step is to form recommendations based on the calculation of the Wilson score of the confidence interval for the Bernoulli parameter. As a result, users receive recommendations in descending order of the adjusted product rating among those purchased the most in the last seven days. The advantage of having a "Best Buy" category is solving the problem of the cold start of the user. The disadvantage of this approach is the lack of personalization of recommendations. The key parameters based on which the assessment is formed present in the Figure 3.

Recommendation according to the survey. A developed model of HRS with a cascade approach method of content direct approach of the RS with a subsequent situational application of the recommendation systems of existing species. Within the approach, it is proposed to ask questions to the user for a clear knowledge of their needs and desires. This format of surveys must be conducted at the time of user registration, once a month and with the constant opportunity to use the function of updating answers. The advantage of using the method of recommendations for surveys is the presence of direct user responses, rather than simulated by the program of probable user choices.

The disadvantages are that not all users always know what they want to buy; the survey recommendation limits the recommendations in a narrower direction. This type of recommendation is most widespread on music and telecommunications platforms. This is due to the limited focus on consumer tastes in certain areas. The key parameters based on which the assessment is formed present in the Figure 4. In the developed HRS, several the following questions are formed:

 Planned monthly cost budget on the platform. The default average monthly user budget.

 The planned number of purchased units of products. By default, the average number of units purchased by users in one month. Considering the previous paragraph, the user chooses the pricing policy of the product.  Product categories that interest you (unlimited quantity).  Product features that interest you (unlimited quantity).

Justification of the need to use adjusted rates

To test the developed model of product evaluation used an Amazon database [25]. The proposed conceptual model includes four different HRS: 3 independent (Personalized recommendation, News, Best buy) and 1 (Recommendation according to the survey), which includes one of the three previous RSs of the user's choice. The following key features of the developed RSs can be distinguished:

 specificity of combining different methods of recommendation;  uniqueness of the developed rates to each RS. This part of the research describes the causality of the use of such adjusted rates within each system separately and demonstrates their advantages in comparison. The environment for the implementation of the developed RSs is the programming language R. Hacker method is a way to consider the time of publication of the product in the existing evaluation. It adjusts the specific evaluation, rather than a number in the aggregate. As the importance of considering the time of publication increases, the Hacker score becomes steepermore sensitive to duration. In the RS "News", it is important to consider the time of publication of products on the site to provide recommendations to the user on the latest products. Within the RS, the degree of sensitivity to the duration of publications is 1.8, which allows to adjust the evaluation in the right direction. For example, the Hacker rating is the same for a product with a standard rate of 4 for a publication time of 2 days and a product with a standard rate of 5 for a publication time of 3 days. As the score decreases, the Hacker score decreases accordingly (there is a direct relationship).

In the RS "Personalized recommendation", the use of the proposed model of adjusted evaluation based on such methods as Wilson's score, Bayesian approximation and Hacker ranking algorithm is implemented. The adjusted rate formed based on the Wilson (r_w) and Bayes (r_b) method are equivalently taken as the root mean square value, adjusted for the degree of importance (g) of days (t) Product attributes:  Categories according to the answers to the questions;  Characteristics according to the answers to the questions;  The number of recommended products according to the answers to the questions  Product prices according to the answers to the questions of the Hacker (r_h) publication. To test the effectiveness of the overall adjusted rate, the analysis of each method was performed separately depending on the input parameters in comparison.

Wilson's score includes two main indicators on which the value of the score depends: the share of positive rates and the number of rates in general. Wilson's score is higher, when the share of positive rates is higher and overall rate is higher. Fluctuation limits from 0 (in the case when there are no rates or no of the, are positive) to 1 (in the case of many rates and they are all positive).

According to the results of the analysis, it is shown that a sharp increase in the assessment occurs when the number of rates increases in the range from 0 to 100 (the steeper the growth, the higher the share of positive feedback). With an increase in the number of rates, more than 100, the intensity of growth of Wilson's score is lower. Another important trend in Wilson's assessment is its unambiguous growth with an increase in the share of positive rates. This is especially evident with many assessments.

In the case when the number of estimates is not large (for example, up to 20) (Figure 5), the change in Wilson's estimate is not so sharp and more fluid. For example, Wilson's score will be almost the same when the number of scores is 5 and 80% of them are positive to when the number of scores is 2 and 100% positive. Bayesian estimation is adapted to the K-dimensional evaluation system. The research introduces a 5-point grading system, as this is the most used. In contrast to Wilson's rate, the current metric does not divide into positive and negative rates, but it is based on the number of points of each species. The maximum Bayesian score is 5 (the higher the number of scores and the more of them the highest score, the faster the score approaches 5). However, the Bayesian score is always greater than 0.

Decreasing the total number of scores not only reduces the Bayesian score regardless of the internal distribution but softens it. For example, in the case of unambiguous choice of score 5 with a total number of scores of 10,000, 100, 10, the Bayesian score is respectively equal to 4.9997; 4.8046; 3.7222 -there is a tendency to decrease. Otherwise, the unambiguous choice of grade 1 with a total of 10,000, 100, 10 Bayes score, respectively, is 0.9999; 0.9951; 1.0555 -with a small number of overall scores, the Bayesian score is higher than with many scores.

In more real situations -the attraction to a particular score, rather than their unambiguous choice, the Bayesian score is directly proportional to the total number of scores and the attraction to a particular score. In the "Personalized Recommendation", the rate adjusted for the Hacker denominator with a degree of sensitivity to the duration of operations is 0.5 (Figure 6). This value allows to adjust the rate, but the reduction limits are small. Within the approach, not only the duration of publication is a key factor, so the degree of importance of the duration of publication is only 0.5. The developed model of product evaluation combines the methods of Wilson, Bayes and Hacker on the following principles:

 the Bayesian method score is divided by 5 for normalization with the Wilson rate;  the scores of the Bayesian and Wilson methods are taken as the root mean square of the sum;  degree of sensitivity to the days of publication 0.5;  to reduce the adjusted score to a 5-point rating scale, a coefficient is set before the calculation formula 5.

Comparative analysis of product evaluation results

The analyzed Amazon database performed a comparative characterization of ratings by different methods for randomly selected products. Since the database is formed for 2017-2018, the calculation

Conclusions

Examples of modern methods of HRSs focus on overcoming the problems of cold start, long-tail products, data tone, considering telemetry, combined taxonomy and folksonomy, considering the similarities of profiles and demographic differences, comparing product descriptions and user profiles.

For the market with a low level of development of RSs (including Ukrainian) there are certain problems, such as the use of only direct user information or general information of all users together, limited algorithms of RSs, calculation of product rates by the simplest metrics. They can be partially solved using specific RSs and specific methods of product evaluation, that demonstrated in this study.

The proposed model of product evaluation makes it possible to obtain a more accurate product evaluation. This allows users to get more personalized recommendations, which better meets their demand. For companies, this ensures the growth of the brand image because companies scan each consumer in more detail. As a result, it increases sales and revenue growth, respectively.

The study presents the concept of HRSs in four categories, such as "Personalized recommendation", "Best buy", "News", "Recommendation according to the survey". In the "Personalized Recommendation" RS, evaluation is based on a combination of Wilson, Bayes, and Hacker methods in their normalization. The comparative analysis of the proposed metric in the RS "Personalized recommendation" showed that the developed rates are more accurate due to their sensitivity to the considered criteria. It is higher than the others if all the criteria are met at a high level. At the same time, the score is lower than most other scores if at least one of the criteria is low. Therefore, it should be used when the products presented on the e-commerce site, that have a wide range of indicators such as time, number of ratings, distribution of positive and negative ratings, and so on. To test the results, we used a database that tracks such indicators that can be traced in markets with a low level of development of RSs in e-commerce. For further research in this direction and the final confirmation of the prospects of the proposed approach, we will monitor and collect data from Ukrainian businesses.