=Paper= {{Paper |id=Vol-3132/Paper_5 |storemode=property |title=Product Valuation Modeling in Hybrid Recommendation Systems |pdfUrl=https://ceur-ws.org/Vol-3132/Paper_5.pdf |volume=Vol-3132 |authors=Galyna Chornous,Tetiana Lem |dblpUrl=https://dblp.org/rec/conf/iti2/ChornousL21 }} ==Product Valuation Modeling in Hybrid Recommendation Systems== https://ceur-ws.org/Vol-3132/Paper_5.pdf

Product Valuation Modeling in Hybrid Recommendation
Systems
Galyna Chornous and Tetiana Lem
Taras Shevchenko National University of Kyiv, Faculty of Economics, Department of Economic Cybernetics,
Vasylkivska str. 90a, Kyiv, 03022, Ukraine

Abstract
The study proposes an alternative approach to product evaluation in hybrid recommendation
systems and the model of such a recommendation system that is relevant to the needs of the
modern e-commerce market. The concept of hybrid recommendation systems includes four
categories, such as "Personalized recommendation", "Best buy", "News", and
"Recommendation according to the survey". In the "Personalized Recommendation"
recommendation system, evaluation is based on a combination of Wilson, Bayes, and Hacker
methods in their normalization. The advantage of the developed model of product evaluation
is the formation of recommendations that are more personalized for users with limited time
upon their publication. As a result, the customer receives a recommendation of products that
are more interesting for him (personalized), considering the relevance of the product (time of
publication), the share of positive feedback. In this way, users will buy more products. It is an
advantage for businesses due to increased purchases and revenue as the main goal of the
activity.

Keywords 1
hybrid recommendation system, product evaluation, model, metrics, e-commerce

1. Introduction
Increasing the number of e-commerce entities rises a difficulty for a company to be competitive. In
order to solve this problem, recommendation systems (RSs) are being actively developed to overcome
situations of the uncertainty of users’ choice. Significant development is gaining hybrid
recommendation systems (HRSs). Modern RSs are created to overcome a certain problem (for example,
a cold start or a long tail of products). But the scope of product evaluation in recommendation systems
remains insufficiently developed. Approaches to evaluation are based on the characteristics of specific
types of RSs (content-based, collaborative filtering, hybrid) and the goal that the evaluation should
provide. There is a requirement to form an alternative evaluation model that would consider the
specifics of the RSs. Therefore, this work is devoted to the development of a new model of product
evaluation in RSs for e-commerce. An analysis of the most popular e-commerce sites in Ukraine shows
that a limited number of the simplest algorithms are used. Sites use in most cases only direct user
information (history of purchases, products viewed and products, that are added to the wish list). Other
recommendations are based on information collected from all consumers of the company. An urgent
task for the current e-commerce market in Ukraine is to improve the forms and models of product
evaluation. The study aims to form an alternative approach to develop product evaluation in hybrid
recommendation systems and to propose a model of recommendation system of such a type that is
relevant to the needs of the modern e-commerce market.
According to the purpose, the following tasks are formed:
 outline the key features of modern HRSs;
 analyze existing approaches to product evaluation;

Information Technology and Implementation (IT&I-2021), December 01–03, 2021, Kyiv, Ukraine
EMAIL: chornous@univ.kiev.ua (G.Chornous); tetiana.lem@gmail.com (T.Lem)
ORCID: 0000-0003-4889-1247 (G. Chornous)
©️ 2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)

50
 develop an effective alternative evaluation model;
 outline the concept of a RS that uses the proposed approach;
 analyze the effectiveness of developed models of product evaluation.

2. Features of modern recommendation systems and approaches to their
evaluation
RSs are a subclass of information filtering systems that aim to predict the potential needs of users
by analyzing their preferences and providing personalized referral services. Explicit and implicit user
feedback data can be used to provide recommendations. The most popular usage of RSs is in spheres
like movies, books, news, research articles, normal goods, and luxuries.
The purpose of RSs is to help consumers discover new and most relevant products outside their area
of interest. It is in the interests of platforms (business) and consumers to apply such RSs that increase
the novelty and diversity of the list of recommendations. The functioning of the recommendation system
consists of four key stages: accumulation of information about users and items; teaching; forecasting
and feedback. Depending on the method of information selection, recommendation systems are
classified as content-based methods; methods based on collaborative filtering, and hybrid methods [1].
Content recommendation systems include working with a specific user profile. For the
recommendations of a particular element, many descriptions of the properties of previous elements are
considered, as well as their evaluations, preferences, tags, keywords. The result is an assessment of the
level of relevance, which determines the level of user interest in the subject. Research [2, 3, 4] are
devoted to the problems of using this type of RSs.
Recommendation systems based on collaborating filtering are based on finding and analyzing past
user behavior, which later predicts elements on the similarity of the type of ratings given by like-minded
users to the target user. Most existing co-filtering methods rely heavily on explicit feedback.
Collaborating filtering methods can be classified into memory-based and model-based. The results of
research on such systems are presented in [5, 6, 7].
Hybrid recommendation systems combine co-filtering and content-based methods. Based on
research [8, 9, 10, 11], we can identify seven basic principles that demonstrate different ways of
combining methods based on content and collaborative filtering in HRSs (Table 1).
In terms of creating a HRS, a different step-by-step combination of methods of different types of
recommendation systems is possible to apply. The most common are parallel and monolithic
construction [12]. The monopoly type has the following features: only one component of the
recommendation; hybridization is "virtual" in the sense that it combines the features/sources of
knowledge of recommendation systems of different types. The parallel type has the following features:
the result is formed based on several existing implementations of different types of recommendation
systems; the least invasive design; there is a scheme of weighing or voting to combine several sources
of knowledge. The system of hybrid recommendations integrates different types, which helps to
overcome shortcomings and improve performance. Most modern systems eliminate the most common
problems such as cold start and sparsity. Empirical studies show higher efficiency and quality of the
hybrid approach and prove that such approaches provide recommendations more accurately than
content-based or collaborative approaches [10, 11, 16, 18]. The effectiveness of recommendation
algorithms is measured using mechanisms for evaluating them. The complexity of evaluating the
recommended system depends on the number of approaches or algorithms. The most common
evaluation metrics in recommendation systems: rating and forecast accuracy used, user and space
coverage, user and product temperature, ranking, advanced metrics, and online metrics.

3. Review of the current state of development of hybrid recommendation
systems
In this section, we will consider progressive modern examples of HRSs and point out the necessity
to develop an approach to increase the accuracy of evaluation models. The study of the presented
systems allows us to analyze the state and vector of development of RSs in general. On the other hand,
the review of these systems will help to form the uniqueness of the developed model.

51
Table 1
Classification of hybrid recommendation systems depending on the methods of combining content-
based methods and collaborative filtering methods
Hybridization Description
techniques
Weighted Combining components of different types of recommendation systems
with specific weights to create a common single result
Switching The type of recommendation system changes depending on the current
situation
Mixed Combination of results of different types of recommendation systems
Feature Functions from different types of recommendation sources, combined
Combination and presented as input to a single algorithm of recommendations
Feature Using methods of one type of recommendation system to calculate a
Augmentation feature or set of features that further is the input to a recommendation
system of another type
Cascade The result of one type of the recommendation system uses as an input
signal to the methodology of another type of the recommendation
system
Meta-level The technique of one type of recommendation system is used, that
establishes a certain model, which further is being used as a
contribution of the technique of the other type of the recommendation
system
Collaborative RS to overcome the problems of cold start. The authors' study [5] focuses on
improving traditional similarity measurements for co-element-based filtering to address and mitigate
cold start situations. The paper proposes an algorithm that is designed to balance three modern
traditional measurement metrics, such as cosine-based similarity, Pearson correlation similarity and
adjusted cosine similarity in the direction of cold-start situations.
HRS with a focus on the long tail. Research [15] considers the influence of preferences of individual
consumers on new products. The research uses an extension of the matrix factorization model to predict
the rating. Models (PM-1, PM-2) are proposed, which demonstrate the suitability to 1) provide
personalized recommendations to the user, considering the appropriate taste of objects with a long tail;
2) advertise items with a long tail to idiosyncratic users
HRS based on recommendations and tonality. In [10], a RS based on a hybrid analysis of
recommendations and tonality is proposed. F-metric is used - weighted average harmonic accuracy and
completeness, which allows more effectively assess the work of the model in terms of a fuller
perspective. The model proposed in the study is more powerful in its ability to identify relevant films
to the user.
HRS to active users. The authors of the study [11] built a novel RS, which combines the methods of
collaborative filtering and matrix factorization. The RS analyzes only users and items that are related
to active users. Novel RS uses genetic algorithms to estimate the rates of invaluable items of the active
user. The considered RS for joint filtering uses neighborhood models and hidden factor models to
recommend elements for the active user (customer).
HRS for finding educational materials. The study [16] proposes an improved method for the existing
system of e-learning recommendations with a combination of content-based filtering and collaboration
filtering with good student ratings (CBF-CF-GL method). The method of admission for analysis of
students with only high scores can be used in e-commerce, for example, through the selection for
analysis of users with costs in a specific range that is relevant to the costs of the target user. HRS based
on visualization. For this purpose, [2] combines a collapsed neural network and a Bagdanau attention
mechanism. As a result, the method allows identifying areas that were particularly important for the
image that is recommended.
Telemetry HRS for a web browser. The authors [17] presented a recommendation add-on system
with telemetry information (Telemetry-Aware Add-on Recommender), which provides

52
recommendations for Firefox users. Three separate models are used, based on three data sources: a set
of add-ons that the user has already installed; usage and interaction data (browser telemetry); user
browser language settings. The research builds individual models of recommendations for each model
separately and summarizes the recommendations that they generate using the method of linear ensemble
composition. HRS with a combination of taxonomy and folksonomy. According to [13], an algorithm
for reconciling the establishment of general similarity between elements is developed, where tag
information is integrated for semantic analysis of taxonomy attributes. The research proposes a unique
model of random sampling on a heterogeneous graph, built by user nodes, element nodes and different
types of relationships - user-element, element-element.
HRS of consideration of commodity networks. Research [18] analyzes the relationship between
product networks generated by RSs and the convergence of product ratings. The paper considers how
different types of product networks are related to consumers' perception of quality between product
pairs in the product network. In addition, this study examines whether the types of product networks
are differently related to the convergence of attitudes towards products in the product network.
HRS considers the coefficients of similarity of profiles and demographic differences. In [9], the
method of calculating the similarity coefficients of user profile vectors and object profile vectors was
further developed, which, in contrast to the basic ones, uses the demographic characteristics of users,
which allows increasing the accuracy of forecasting recommendations. Based on the concept of
application in one method of categorical, mixed and numerical clustering, a method of searching for
user groups has been developed, which adapts to the sparseness of the user-object matrix.
HRS with partial variation autoencoder method. The authors [14] proposed a method of hybrid
recommendation, which basically processes the missing data and uses the amortized output for quick
forecasting. The method calls as Partial Varial Autoencoder (P-VAE). The method uses a new
probabilistic generative model to process a different number of user ratings in principle.
HRS for comparing product description and user profile. The study [19] developed a hybrid system
of recommendations for e-commerce, which implements content-based filtering and collaboration-
filtering that calculates the similarity of product description and user profile.
HRS with partial variation autoencoder method. The authors of [20] used the built-in categories
obtained in the content-based approach and the embedding of clients in the shared filtering approach. It
is used as input to the new neural network and will check if it is relevant to the recommendations. The
developed model can be used in two different modes: use to organize the recommendations classified
by previous approaches; use to make your own recommendations.
The study proposes HRS that differs from previous models by improving the modeling accuracy of
product evaluation. The proposed evaluation model considers a wider range of relevant indicators.

4. Review of product evaluation methods and elaboration of an alternative
evaluation model
The development of effective RSs requires the use of appropriate mathematical apparatus.
An average rate is the simplest type of metric used in recommendations. It is applied in the case when
it is necessary to obtain the minimum calculation time. The rating 𝑟𝑖𝑗′ is the arithmetic mean of the
ratings 𝑟𝑖𝑗 , 𝑖, 𝑗 = ̅̅̅̅̅
1, 𝑛. A simple recommendation has the following formula:
∑𝑛𝑗=1 𝑟𝑖𝑗 (1)
𝑟𝑖𝑗′ = .
𝑛
Another form of metric employs the proportion of positive ratings to offset the uncertainty of a
limited number of observations. The limits of Wilson score of the confidence interval for the Bernoulli
parameter [21, 22] is referred to one type of mathematical equipment. The goal is to use the generated
rating after considering the current set of user ratings as a statistical sample of a hypothetical set of user
ratings from all users.
𝛼 (2)
𝑧2 2 𝛼 𝛼
𝑝̂ + ± 𝑧 √[𝑝̂ (1 − 𝑝̂ )] + 𝑧 2
2𝑛 2 2
𝛼 ,
𝑧2 2
1 + 2𝑛

53
𝛼
where 𝑝̂ - the observed share of positive evaluations; 𝑧 2 2 - 2 quantile of the standard normal
distribution; 𝑛 - the total number of rates.
Wilson's rating has the drawback of assigning zero value to products that have not gotten any
positive customer feedback, as well as new products, that have received any feedback.
The Bayesian approximation is another metric that can be used to evaluate a product when measured
on a K-star scale [22]. The lower limit of the normal approximation to the Bayesian confidence interval
for the mean estimate is given by the following formula.

𝑛 +1 𝑛 +1 2 (3)
𝐾 ((∑𝐾
𝑘=1 𝑠𝑘2 𝑁𝑘 + 𝐾 ) − (∑𝐾
𝑘=1 𝑠𝑘 𝑁𝑘 + 𝐾 ) )
𝑛𝑘 + 1 √
𝑠(𝑛1 , … , 𝑛𝑘 ) = ∑ 𝑠𝑘 − 𝑧𝛼 ,
𝑁+𝐾 2 𝑁+𝐾+1
𝑘=1

where 𝑠𝑘 to 1-point, .... N-point scale; 𝑁 = total score with 𝑛𝑘 scores for scale 𝑘.
One of the simplest techniques to assign nonzero probabilities to invisible terms is Laplace
smoothing, which assumes that each element "by default" has 1 thumb up and 1 thumb down [23].
𝑛 ↑ +1 (4)
𝑠(𝑛 ↑, 𝑛 ↓) = Pr[↑ |𝑀] =
(𝑛 ↑ +1) + (𝑛 ↓ +1)
Although Laplace smoothing avoids most of the disadvantages of popular methods (for example,
gaining zero probability for invisible user ratings), this is probably too large for pseudo-counts. A more
rational choice is a more generalized form - Leadstone smoothing, which assumes that each element
"by default" has fingers up and fingers down:
𝑛 ↑ +∈ (5)
𝑠(𝑛 ↑, 𝑛 ↓) = Pr[↑ |𝑀] = ,
(𝑛 ↑ +∈) + (𝑛 ↓ +∈)
where ∈> 0 - parameter.
Studies have shown that the efficiency of Lidstone smoothing with 0 <∈< 1 usually exceeds ∈= 1
(Laplace smoothing) [23]. The Hacker metric is primarily designed to rank news for Hacker News to
address hot and new articles using gravity (g) and time (t). However, this algorithm can be applied to
e-commerce, considering the period between the provision of recommendations to the user and the
considered comments, evaluations of other users [24].
𝑝−1 (6)
𝑟=
(𝑡 + 2)𝑔
where 𝑝 - points by position (p -1 - to cancel the vote of participants); 𝑡 = time since submission
(in hours); 𝑔 = gravity, default 1.8 in news.arc.
The most widely used in terms of collaborative filtering is the formation of the user similarity
function based on Pearson correlation and cosine similarity [24]. The cosine measure of similarity is
popular in many collaborative RSs for calculating similarity coefficients between object profile vectors.
It gives an unsatisfactory result in the presence of several subjects with the same ratings. Also poorly
considers differences in the values of the components of rating vectors. Pearson's correlation coefficient
gives incorrect values if both profile vectors have several identical rates. It also calculates incorrect
values for a small quantity of the set of components of the profile vectors between which the degree of
similarity is calculated. This coefficient may have a large or low value of similarity despite the similarity
or difference in ratings. The limited Pearson correlation coefficient makes a slight improvement in the
normal Pearson correlation coefficient. It also calculates incorrect values in the presence of even a small
number of identical ratings in both profile vectors. The root mean square difference does not consider
the same values of ratings in the vectors of user profiles. Jacquard's similarity factor does not consider
the absolute values of the ratings. In the assessment of Jacquard + root mean square there are the same
shortcomings as in each assessment. The study proposes an original product evaluation model based on
such methods as Wilson 's score, Bayesian approximation and Hacker ranking algorithm. The proposed
evaluation model can be calculated by the formula:

54
𝑎 (7)
𝑅= √𝑟_𝑤 2 + 𝑏 × 𝑟_𝑏2 ,
(𝑡 + 2)𝑔
where 𝑟_𝑤 – Wilson’s score; 𝑟_𝑏 – Bayesian approximation; 𝑟_ℎ - Hacker ranking algorithm.
In the calculation of the rate R, the Wilson and Bayes metrics are combined as the root mean square
value of their values by their normalization. The coefficient b normalizes the values of these estimates
among themselves. The user may prefer a particular estimation method or set the coefficient so that the
maximum value of the estimates is within the same range. The Hacker method is used to adjust the
number of days the product is published. Instead of the indicator (p-1) the root mean square value is
taken from the normalization of Wilson and Bayes metrics. The coefficient 𝑎 is set by the user only to
expand the scope of the assessment and does not affect the essence of the calculation. Because the
adjusted estimate is designed for e-commerce, the degree of adjustment for the number of days of
publication should be less than for news (equal to 1.1). This assessment takes advantage of each of the
methods and adjusts the value of the assessment if one of the methods would be equal to 0. At the same
time, this allows you to form an assessment as the average for the entire sample and considers the
division into positive and negative assessments (on a five-point scale). It is assumed that grades 4, 5 -
positive, grades 1, 2, 3 - negative).

5. Conceptual bases of development of the hybrid recommendation system
The study proposes a version of the complex of RSs that provides recommendations based on content
and collaborative filtering and solves current problems in e-commerce. The model is especially relevant
for those markets where the state of development of RSs is low, such as in Ukraine. Thus, the Ukrainian
e-commerce market is characterized by the fact that sites are limited to direct user information (history
of purchases, viewed products and those added to the wish list). Other recommendations are based on
information collected from all consumers of the company. There is a limited number of the simplest
algorithms used to provide recommendations.
The developed RS provides recommendations in four categories, such as "Personalized
recommendation", "Best buy", "News", and "Recommendation according to the survey". The key
advantage is that the user has more choice, as he can choose which recommendation he wants to receive
at a particular time.
Personalized recommendation. The HRS of the type "addition of features" has been developed: a
demographic method of the content filtering of the system and a factorization matrix of the collaborative
filtering. The method of calculating the similarity coefficients of user profile vectors and object profile
vectors has been further developed, which allows increasing the accuracy of forecasting forecasts. The
first step is to calculate the user's monthly costs on this platform for the last six months. Based on this
data, other users who have monthly expenses in the same interval are filtered (intervals are set by the
author). The next step is to form a user similarity function. The following methods of similarity
formation were used in the study: Pearson correlation coefficient, Jacquard coefficient, inverse
Euclidean distance, cosine measure of similarity. The similarity feature searches for multiple users with
a similar purchase history. Subsequently, the recommendations are passed to the current user from
similar users. Based on the calculation of the rating of products according to the developed model of
score evaluation, recommendations and the sequence of their provision are formed. The advantage of
this RS is that recommendations are appropriate for consumers` tastes and financial capabilities. The
disadvantage is the need for constant recalculation with any purchase of new or similar users. The key
parameters based on which the assessment is formed present in the Figure 1.
News. A model of a HRS has been developed, which combines a sequence of content
recommendation filtering. In the terms of the approach, it is proposed to provide recommendations
based on the similarity of the characteristics of past views of products by the user and the characteristics
of objects. Modifications of the Pearson coefficient methods and the Hacker ranking algorithm are used
together. Thus, the lack of a sufficiently large number of responses to a new product (cold start of
production) is offset by its high time priority. At the same time, in this section of the recommendations,
the analysis does not consider new products, which in turn have more reviews, but less time priority.
As a result, the model balances newer and older products according to their benefits for the user. The
advantage of this method is to solve the problem of cold start products and personalize the

55
recommendations of not yet purchased, but revised products. The disadvantage is not considering the
interest in the revised product. In the future, this problem can be solved by expanding the ability to track
various e-commerce sites` time of active browsing. Despite the possibility of implementing this item,
most companies do not enter this item in their user database. The key parameters based on which the
assessment is formed present in the Figure 2.

User Product

User attributes: Product attributes:
 Monthly expenses  Product rank
 Purchase history
Rates

Figure 1: Input parameters for evaluation in the RS "Personalized recommendation"

User Product

User attributes: Product attributes:
 Purchase history (product  Product
characteristics) characteristics
Rates  Product categories

Figure 2: Input parameters for evaluation in the RS "News"
Best buy. A developed model of a direct collaborative RS with a direct connection. The first stage is
the formation of a database of smaller dimensions than the initial number of purchased products in the
last week. The next step is to form recommendations based on the calculation of the Wilson score of
the confidence interval for the Bernoulli parameter. As a result, users receive recommendations in
descending order of the adjusted product rating among those purchased the most in the last seven days.
The advantage of having a "Best Buy" category is solving the problem of the cold start of the user. The
disadvantage of this approach is the lack of personalization of recommendations. The key parameters
based on which the assessment is formed present in the Figure 3.
Recommendation according to the survey. A developed model of HRS with a cascade approach
(a method of content direct approach of the RS with a subsequent situational application of the
recommendation systems of existing species. Within the approach, it is proposed to ask questions to the
user for a clear knowledge of their needs and desires. This format of surveys must be conducted at the
time of user registration, once a month and with the constant opportunity to use the function of updating
answers. The advantage of using the method of recommendations for surveys is the presence of direct
user responses, rather than simulated by the program of probable user choices.
The disadvantages are that not all users always know what they want to buy; the survey
recommendation limits the recommendations in a narrower direction. This type of recommendation is
most widespread on music and telecommunications platforms. This is due to the limited focus on
consumer tastes in certain areas. The key parameters based on which the assessment is formed present
in the Figure 4. In the developed HRS, several the following questions are formed:
 Planned monthly cost budget on the platform. The default average monthly user budget.
 The planned number of purchased units of products. By default, the average number of units
purchased by users in one month. Considering the previous paragraph, the user chooses the pricing
policy of the product.
 Product categories that interest you (unlimited quantity).
 Product features that interest you (unlimited quantity).

56
 Method by which you want to receive recommendations: Personalized recommendation (for
active users), News, Most buy.

User Product

Product attributes:
User attributes  Numbers of rates;
 Rates (positive or negative).
Rates

Figure 3: Input parameters for evaluation in the RS "Best buy"

User Product

Product attributes:
User attributes:  Categories according to the answers to
 Answers to questions of the Rates the questions;
survey  Characteristics according to the answers
 Selected method of RSs. to the questions;
 The number of recommended products
according to the answers to the questions
 Product prices according to the answers
to the questions

Figure 4: Input parameters for evaluation in the RS "Recommendation by survey"

6. Justification of the need to use adjusted rates
To test the developed model of product evaluation used an Amazon database [25]. The proposed
conceptual model includes four different HRS: 3 independent (Personalized recommendation, News,
Best buy) and 1 (Recommendation according to the survey), which includes one of the three previous
RSs of the user's choice. The following key features of the developed RSs can be distinguished:
 specificity of combining different methods of recommendation;
 uniqueness of the developed rates to each RS.
This part of the research describes the causality of the use of such adjusted rates within each system
separately and demonstrates their advantages in comparison. The environment for the implementation
of the developed RSs is the programming language R. Hacker method is a way to consider the time of
publication of the product in the existing evaluation. It adjusts the specific evaluation, rather than a
number in the aggregate. As the importance of considering the time of publication increases, the Hacker
score becomes steeper – more sensitive to duration. In the RS "News", it is important to consider the
time of publication of products on the site to provide recommendations to the user on the latest products.
Within the RS, the degree of sensitivity to the duration of publications is 1.8, which allows to adjust the
evaluation in the right direction. For example, the Hacker rating is the same for a product with a standard
rate of 4 for a publication time of 2 days and a product with a standard rate of 5 for a publication time
of 3 days. As the score decreases, the Hacker score decreases accordingly (there is a direct relationship).
In the RS "Personalized recommendation", the use of the proposed model of adjusted evaluation
based on such methods as Wilson's score, Bayesian approximation and Hacker ranking algorithm is
implemented. The adjusted rate formed based on the Wilson (r_w) and Bayes (r_b) method are
equivalently taken as the root mean square value, adjusted for the degree of importance (g) of days (t)

57
of the Hacker (r_h) publication. To test the effectiveness of the overall adjusted rate, the analysis of
each method was performed separately depending on the input parameters in comparison.
Wilson's score includes two main indicators on which the value of the score depends: the share of
positive rates and the number of rates in general. Wilson's score is higher, when the share of positive
rates is higher and overall rate is higher. Fluctuation limits from 0 (in the case when there are no rates
or no of the, are positive) to 1 (in the case of many rates and they are all positive).
According to the results of the analysis, it is shown that a sharp increase in the assessment occurs
when the number of rates increases in the range from 0 to 100 (the steeper the growth, the higher the
share of positive feedback). With an increase in the number of rates, more than 100, the intensity of
growth of Wilson's score is lower. Another important trend in Wilson's assessment is its unambiguous
growth with an increase in the share of positive rates. This is especially evident with many assessments.
In the case when the number of estimates is not large (for example, up to 20) (Figure 5), the change
in Wilson's estimate is not so sharp and more fluid. For example, Wilson's score will be almost the same
when the number of scores is 5 and 80% of them are positive to when the number of scores is 2 and
100% positive.

Figure 5: Change in Wilson's grade depending on the number of rates (from 0 to 20) and the share of
positive rates
Bayesian estimation is adapted to the K-dimensional evaluation system. The research introduces a
5-point grading system, as this is the most used. In contrast to Wilson's rate, the current metric does not
divide into positive and negative rates, but it is based on the number of points of each species. The
maximum Bayesian score is 5 (the higher the number of scores and the more of them the highest score,
the faster the score approaches 5). However, the Bayesian score is always greater than 0.
Decreasing the total number of scores not only reduces the Bayesian score regardless of the internal
distribution but softens it. For example, in the case of unambiguous choice of score 5 with a total number
of scores of 10,000, 100, 10, the Bayesian score is respectively equal to 4.9997; 4.8046; 3.7222 - there
is a tendency to decrease. Otherwise, the unambiguous choice of grade 1 with a total of 10,000, 100, 10
Bayes score, respectively, is 0.9999; 0.9951; 1.0555 - with a small number of overall scores, the
Bayesian score is higher than with many scores.
In more real situations - the attraction to a particular score, rather than their unambiguous choice,
the Bayesian score is directly proportional to the total number of scores and the attraction to a particular
score. In the "Personalized Recommendation", the rate adjusted for the Hacker denominator with a
degree of sensitivity to the duration of operations is 0.5 (Figure 6). This value allows to adjust the rate,
but the reduction limits are small. Within the approach, not only the duration of publication is a key
factor, so the degree of importance of the duration of publication is only 0.5. The developed model of
product evaluation combines the methods of Wilson, Bayes and Hacker on the following principles:
 the Bayesian method score is divided by 5 for normalization with the Wilson rate;
 the scores of the Bayesian and Wilson methods are taken as the root mean square of the sum;
 degree of sensitivity to the days of publication 0.5;
 to reduce the adjusted score to a 5-point rating scale, a coefficient is set before the calculation
formula 5.

7. Comparative analysis of product evaluation results
The analyzed Amazon database performed a comparative characterization of ratings by different
methods for randomly selected products. Since the database is formed for 2017-2018, the calculation

58
of the duration of publication is calculated as the difference between the date of publication of the
current product and the latest date of publication of a particular product. All estimations are normal so
that their maximum value is 5 (for ease of comparison).

Figure 6: Change the Hacker score depending on the number of days of publication (importance 0.5)
Product 1 (id = AVqVGZNvQMlgsOJE6eUY) has 96 rates, among which 88.54% are positive. This
provides a high score of Wilson 4.03. Among the distribution of rates (Table 2) there is an attraction to
5 - because of this, the Bayesian score is high and is 4.52. The average value is 4.41. Hacker score is
only 0.31 since the product was published more than 1 year ago. In the developed model, the adjustment
for the time of publication is made not by days, but by weeks, which allows to consider the time, but
does not make it the main criterion. The adjusted rate by authors is 1.51. The last rate is lower than
Wilson's, Bayesian, and arithmetic mean because it considers the time of publication. The visual
distribution of product evaluations 1 is shown in the Figure 7.
Products 2 (id = AWFUWc8THh53nbDRF6YO) has the following characteristics: 650 rates, 616
positive rates, tendency to rate 5, 89 days of publication. Estimates of Wilson, Bayes, arithmetic mean
is in the range from 4.6 to 4.7 (Table 2). Due to Hacker's low score, adjusted rate is 2.97. Products 2
have the rate above average. The visual distribution of product ratings 2 is shown in the Figure 7.
Table 2
Product evaluations by different metrics
Normalized rates Product 1 Product 2 Product 3 Product 4
Wilson 4.0319 4.6390 4.4777 4.7209
Bayes 4.5224 4.7076 4.7093 4.7831
Hacker 0.3083 0.7002 4.7051 4.7909
Average 4.4063 4.6677 4.6462 4.7492
By authors 1.5069 2.9656 4.5949 4.7521
Products 3 (id = AWK8z0pOIwln0LfXlSxH) has the following characteristics: 195 rates, 93.85%
of them are positive, tendency to rate 5, 0 days of publication. All evaluations (Table 2) contain high
values due to the satisfactory conditions of the criteria of each of them. The developed rate is 4.59,
which is a higher value than in other methods. This means that the developed evaluation model responds
to a greater extent to both negative values and positive values (considering the effects of different
criteria in synergy). The visual distribution of product evaluations 3 is shown in the Figure 8.
Products 4 (id = AWMjT0WguC1rwyjrFh3) has the following characteristics: 590 rates, 568
positive rates, tendency to rate 5, 0 days of publication. The values of all estimates are in the range from
4.72 to 4.87 (Table 2). The adjusted rate is 4.75, that is the highest from other metrics. Products 4 have
a high rating. The visual distribution of product evaluations 4 is shown in the Figure 8. Comparative
analysis of evaluations showed that the proposed evaluation is more adaptable to use due to its
sensitivity to the considered criteria. It is higher than the others if all the criteria are met at a high level.
At the same time, the score is lower than most other scores if at least one of the criteria is low. The
advantage of the developed model is the formation of recommendations that are more personalized for
users with a low time of their publication. As a result, the customer receives a recommendation of
products that are more interesting for him (personalized), considering the relevance of the product (time
of publication), the share of positive feedback. In this way, users will buy more products. It is an
advantage for businesses due to increased purchases and revenue as the main goal of the activity.

59
Figure 7: Rates for products 1 and 2 by different evaluation methods

8. Conclusions
Examples of modern methods of HRSs focus on overcoming the problems of cold start, long-tail
products, data tone, considering telemetry, combined taxonomy and folksonomy, considering the
similarities of profiles and demographic differences, comparing product descriptions and user profiles.
For the market with a low level of development of RSs (including Ukrainian) there are certain
problems, such as the use of only direct user information or general information of all users together,
limited algorithms of RSs, calculation of product rates by the simplest metrics. They can be partially
solved using specific RSs and specific methods of product evaluation, that demonstrated in this study.
The proposed model of product evaluation makes it possible to obtain a more accurate product
evaluation. This allows users to get more personalized recommendations, which better meets their
demand. For companies, this ensures the growth of the brand image because companies scan each
consumer in more detail. As a result, it increases sales and revenue growth, respectively.
The study presents the concept of HRSs in four categories, such as "Personalized recommendation",
"Best buy", "News", "Recommendation according to the survey". In the "Personalized
Recommendation" RS, evaluation is based on a combination of Wilson, Bayes, and Hacker methods in
their normalization. The comparative analysis of the proposed metric in the RS "Personalized
recommendation" showed that the developed rates are more accurate due to their sensitivity to the
considered criteria. It is higher than the others if all the criteria are met at a high level. At the same time,
the score is lower than most other scores if at least one of the criteria is low. Therefore, it should be
used when the products presented on the e-commerce site, that have a wide range of indicators such as
time, number of ratings, distribution of positive and negative ratings, and so on. To test the results, we
used a database that tracks such indicators that can be traced in markets with a low level of development

60
of RSs in e-commerce. For further research in this direction and the final confirmation of the prospects
of the proposed approach, we will monitor and collect data from Ukrainian businesses.

Figure 8: Rates for products 3 and 4 by different evaluation methods
Areas of improvement of recommended by authors set of HRSs for the possibility of their
implementation on various e-commerce sites involve greater use of internal characteristics of products
and users of the complex of RSs; deeper analysis of the Ukrainian e-commerce market, testing of
systems on the data of domestic platforms and considering their features.

9. References
[1] M. Sridevi, R.Rajeshwara Rao, M.Varaprasad Rao, A Survey on Recommender System.
International Journal of Computer Science and Information Security (IJCSIS) (2016) 14(5). 265-272.
[2] P. Woldan, P. Duda, Y. Hayashi, Visual Hybrid Recommendation Systems Based on the Content-
Based Filtering, in: Proceedings of the 19th International Conference on Artificial Intelligence and
Soft Computing, 2020, pp. 455-465. doi: 10.1007/978-3-030-61534-5_41
[3] L. Barolli, F. D. Cicco, M. Fonisto, An Investigation of Covid-19 Papers for a Content-Based
Recommendation Syste, in: Proceedings of International Conference on P2P, Parallel, Grid, Cloud
and Internet Computing, 2021, pp. 156-164. doi:10.1007/978-3-030-89899-1_16
[4] N. V. Dat, P. V. Toan, T. M. Thanh, Solving distribution problems in content-based
recommendation system with gaussian mixture model, Applied Intelligemce (2021). URL:
http://eprints.lqdtu.edu.vn/id/eprint/8787/ doi: 10.1007/s10489-021-02429-9
[5] C. Ajaegbu, An optimized item-based collaborative filtering algorithm. Journal of ambient
intelligence and humanized computing, 12 (2021) 10629-10636.

61
[6] L. V. Nhuen, M-S. Hong, J. J. Jung, B-S. Sohn, Cognitive Similarity-Based Collaborative Filtering
Recommendation System, Applied Sciences 10 (12) (2020). doi: 10.3390/app10124183
[7] J. Shen, T. Zhou, L. Chen, Collaborative filtering-based recommendation system for big data.
International Journal of Computational Science and Engineering, 21.2 (2020) 219-225.
doi:10.1504/IJCSE.2020.105727
[8] J. Feng, Z. Xia, X. Feng, J. Peng, RBPR: A hybrid model for the new user cold start problem in
recommender systems, Knowledge-Based Systems, 214 (2021) doi:
10.1016/j.knosys.2020.106732
[9] M.E. Schwartz, Hybrid models and methods of forecasting recommendations for online shopping,
Ph.D. thesis , Lviv Polytechnic National University, Lviv, Ukraine, 2019 (in Ukrainian).
[10] K. Y. Chorna, System of recommendations with use of social networks, Master’s thesis, National
Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv, Ukraine, 2018
(in Ukrainian).
[11] Y. Kilani, A. F. Otoom, A. Alsarhan, M. Almaay, A genetic algorithms-based hybrid recommender
system of matrix factorization and neighborhood-based techniques. Journal of Computational
Science, 8 (2018). doi: 10.1016/j.jocs.2018.08.007
[12] Jiang. Hybrid Recommendation Approaches. iCAMP. UCI Interdisciplinary Computational and
Applied Mathematics Program. URL:
https://www.math.uci.edu/~icamp/courses/math77b/lecture_12w/
[13] M. Mao, S. Chen, F. Zhang, Hybrid ecommerce recommendation model incorporating product
taxonomy and folksonomy. 214 (2021). doi: 10.1016/j.knosys.2020.106720
[14] C. Ma, J. M. Hernández-Lobato, N. Koenigstein, S. Nowozin, C. Zhang, Partial VAE for Hybrid
Recommender System (2018). Preprint. Work in progress.
[15] B. Kumar, P. K. Bala, Fattening The Long Tail Items in E-Commerce. Journal of Theoretical and
Applied Electronic Commerce Research, 12(3) (2016) 27-49. doi: 10.4067/S0718-
18762017000300004
[16] R. Turnip, D. Nurjanah, D. S. Kusumo, Hybrid recommender system for learning material using
content-based filtering and collaborative filtering with good learners' rating, in: Proceedings of
IEEE Conference on e-Learning, e-Management and e-Services (IC3e), 2017, pp. 61-66. doi:
10.1109/IC3e.2017.8409239
[17] M. Lopatka, V. Ng, B. P. Miroglio, A. P. Placitelli, L. Thomson, Telemetry-Aware Add-on
Recommendation for Web browser customization, in: Proceedings of the 27th ACM Conference
on User Modeling, Adaptation and Personalization, 2019, pp. 166-175. doi:
10.1145/3320435.3320450
[18] D. R. Stockli, H. Khobzi, Recommendation systems and convergence of online reviews: The type
of product network matters!, Decision Support Systems, 142 (2020).
doi:10.1016/j.dss.2020.113475
[19] T. Badriyah, E. T. Wijayanto, I. Syarif, P. Kristalina, A hybrid recommendation system for E-
commerce based on product description and user profile, in: Proceedings of the Seventh
International Conference on Innovative Computing Technology (INTECH), 2017, pp. 95-100. doi:
10.1109/INTECH.2017.8102435
[20] R. Kwok, Recommendation System for E-commerce customers – Part III (Hybrid
Recommendation), 2019. URL: https://medium.com/analytics-vidhya/content-based-
collaborative-and-hybrid-recommender-for-e-commence-customers-part-iii-2e225d2e6d21
[21] E. Miller, How to Sort By Average Rating. Evanmiller. 2009. URL:
https://www.evanmiller.org/how-not-to-sort-by-average-rating.html
[22] A. Kumar, Wilson Lower bound Score and Bayesian Approximation for K star scale rating to Rate
products, 2020. URL: https://medium.com/tech-that-works/wilson-lower-bound-score-and-
bayesian-approximation-for-k-star-scale-rating-to-rate-products-c67ec6e30060
[23] D. Zhang, R. Mao, H. Li, J. Mao, How to Count Thumb-Ups and Thumb-Downs: User-Rating
based Ranking of Items from an Axiomatic Perspective, in: Proceedings of Conference: Advances
in Information Retrieval Theory - Third International Conference, ICTIR 2011, Bertinoro, Italy,
September 12-14, 2011. doi: 10.1007/978-3-642-23318-0_22
[24] A. Sallihefendic, How Hacker News ranking algorithm works, 2015. URL:
https://medium.com/hacking-and-gonzo/how-hacker-news-ranking-algorithm-works-
1d9b0cf2c08d
[25] J. McAuleym, UCSD. Amazon product data. URL: https://nijianmo.github.io/amazon/index.html