The Cold-Start problem refers to the initial sparsity of data available to Recommender Systems that leads to poor recommendations to users. This research compares a Deep Learning Approach, a Deep Learning Approach that makes use of social information and Matrix Factorization. The social information was used to form communities of users. The intuition behind this approach is that users within a given community are likely to have similar interests. A community detection algorithm was used to group users. Thereafter a deep learning model was trained on each community. The comparative models were evaluated on the Yelp Round 9 Academic Dataset. The dataset was pruned to consist only of users with at least 1 social link. The evaluation metrics used were Mean Squared Error (MSE) and Mean Absolute Error (MAE). The evaluation was carried out using 5-fold cross-validation. The results showed that the use of social information improved on the results achieved from the Deep Learning Approach, and grouping users into communities was advantageous. However, the Deep Learning Approach that made use of social information did not outperform SVD++, a state of the art approach for recommender systems. However, the new approach shows promise for improving Deep Learning models.
Recommender Systems based on collaborative filtering [
3. System cold start; recommending a new item to a new user.
There have been a lot of research efforts to address the cold start problem in literature and these include; (i) integrating collaborative filtering and content-based filtering (ii) applying popularity based recommendation strategy (iii) real-time updating of user/item latent factors, and (iv) users manually suggesting what they may like.
Recently, there have been some efforts using social information to solve the cold
start problem [
Deep learning is also one of the techniques that have recently been used to solve the recommender systems problem in literature, with the hope of solving the cold start problem. There also has not been any work comparing deep learning solutions with SVD++. There have also been minimal work on combining the social media data and deep learning to solve the cold start problem.
This research aims to establish whether it is necessary to enhance deep learning solutions with social information and give insights on whether the recommender systems that use social information can outperform SVD++ in solving the cold start problem. This research therefore compares the performance of a community-based deep learning recommender system solution, a purely deep learning solution, and SVD++ in solving the cold start problem.
The remainder of this paper is organised as follows: Section 2 surveys existing literature of various existing solutions to the cold-start problems, Section 3 discusses the design and implementation of the proposed solution, Section 4 outlines the research methodology, Section 5 provides the results and Section 6 concludes this paper with a conclusion, discussion of results and future work. 2
This section looks at related work that focused on the cold-start problem in
recommender systems. The works of [
In order to compare the performance of a community-based deep learning
recommender system solution, a purely deep learning solution, and the SVD++ in solving the
cold start problem, this research adapted concepts from Matrix Factorisation to create
a deep learning solution to recommender systems, and extended the created solution to
develop a community based solution. For the SVD++ the Surprise python package [
The users’ latent factors matrix is called the user-concept matrix and items’ latent factors matrix is called the item-concept matrix.
3.2
It was decided to make use of a deep convolutional neural network architecture over recurrent neural networks or multilayer perceptron, due to the ability of a convolutional neural networks to learn latent factors, and optimize towards the desired output.
At the inception of this research, [
Deep Convolutional Neural Network architecture used in this research, as shown in Fig. 2. The specification of the Deep Learning Architecture is as shown below. Input Layer: 2-Dimensional Convolutional Layer ─ Dimension nr x nc, where nr number of rows and nc number of columns ─ Kernel size is 3 with filters of 8 ─ Activation Function: Rectified Linear Unit (ReLU) Max-Pooling Layer Flatten Layer Dense Layer returns vector of size 50
─ Activation Function: ReLU Dropout Layer with a dropout rate of 0.5 (50%) Output: Predicted rating scaled between 0 and 1, ─ Dense Layer returns vector of size 1 ─ Activation Function: Sigmoid
Upon training SVD++ [
Key nf Number of users ni Number of items
Lf Number of latent factors
The Deep Convolution model input, Algorithm 1, consisted of vectors from the userconcept and item-concept matrices. The input was comprised of the target user’s and item’s vectors from the user-concept and item-concept matrices respectively. Thereafter, the next nf rows were the users who were the most similar (Cosine Similarity) to the target user. They were ranked according to how close they were to 1 and the top nf user vectors being used. Thereafter, the next ni rows were the items that were the most similar (Cosine Similarity) to the target item. They were ranked according to how close they were to 1 and the top ni item vectors being used. The benefit of using a Deep Convolutional Neural Network over traditional matrix factorization is that as new users or new items join the recommender system the matrix dimension for each of the matrices for the matrix factorization algorithm would need to be increased to cater for more users and items. A Deep Convolution matrices dimension would not need to be increased, however the vector representation for users and items would need to be updated as a user rates items.
The Deep Community Learning model consisted of two main components, community detection and the deep learning components as shown in Fig. 3.
The community aspect of the proposed solution made use of friendship information
to form a relationship between users called a social graph. Thereafter, Multi-Level [
The two main aspects, community and deep learning, needed to be linked through some algorithm so that the deep learning aspect could make use of the community information in predicting a rating for a given user item pair. It was decided to map each user and item to a vector that best represents the user and item within a community. SVD++ was ran for each community to get a user-concept and an item-concept matrix based on the community information. The user-concept matrix consisted of users in the given community and the item-concept matrix consisted of items found in the recommender system. This information for each community was passed to the deep learning component.
The deep learning model used was a Deep Convolutional Neural Network. It received as input a data matrix that consisted of the target user vector, target item vector, most similar users’ vectors, and most similar items’ This information was used to predict the rating for the target user-item pair. A challenge was that the number of users per community varied and could be below nf which led to the input matrix for the convolutional network having empty rows. This was solved by setting the empty rows as vectors of zeros since a vector of zeros hold no significance as input, they served to represent empty rows or padding to ensure that the input matrix had the correct dimension for the network. 4
This section discusses how the experiments were carried out. It gives details on the dataset used, and how it was pre-processed to showcase the cold-start problem existence. It also discusses how the models were trained and evaluation metrics used. 4.1
The Yelp Academic dataset Round 91 was used. The dataset includes reviews of businesses and restaurants. The reviews were conducted by users who provide a review and rating for a business and/or restaurant that they have searched for on the Yelp website. Since this research focuses on solving the cold-start problem, users who have 1 review
in the dataset are not discarded as they are key to us ascertaining how well does each
of the compared solution perform in solving the cold start problem. Table 2 shows the
statistics of the dataset used in this research and the evidence of the cold-start problem.
Dataset Sparsity = (Number of missing ratings / Number of expected ratings) %
(2)
The dataset was pre-processed to remove information that was not used by this research
such as user reviews and for the items the information about where the business (item)
was located was removed. The reviews a user had written were removed since it meant
a review would need to be written by a user first in order to predict how well a user
would like an item and due to hardware limitations it was not used to aid in training the
model to be able to better predict what rating a user would give for an unrated item.
Future work would look at using the reviews written by users to reinforce the rating
prediction and recommendation accuracy. Thereafter the dataset was split into 5 equal
parts into order to carry out 5-fold cross-validation more efficiently. Users who did not
have friends are removed. The remaining users who have friends were used to form a
social communities. The Multi-Level community detection algorithm [
The input for the deep learning model for the Deep Community Learning model and
the Deep Convolution model had a relatively high computational time. The high
computational time was due to the amount of checks involved in selecting the most similar
users and items to the target user and item respectively. Therefore, the solution to this
was to parallelize the input. This was possible by the CHPC [
The dataset for this research was formed by pruning users who had few friends (11),
the cold-start problem still did exist. A recommender system has the cold-start problem
if the sparsity of ratings is at least 85% [
The mean squared error (MSE) Equation (3) and mean absolute error (MAE) Equation (4) metrics were used to evaluate the performance of each of the models. MSE has the benefit of penalizing large errors, but has the drawback to greatly penalizing outliers. In this research the max error possible between the true and predicted value was 4, this prevented any extreme outliers and MSE from growing to be large because of any extreme outliers. MAE served as a second evaluation metric, since it does not penalizing the model by being far from the true value and it better describes the average error.
MSE = MAE = ∑| ∑( − − 2 ) |
Experimentation was carried out on the research dataset with users that had friends listed being considered. To evaluate whether social networks are beneficial and improve on a deep learning approach. The proposed model was compared against a noncommunity-based Deep Convolutional Neural Network. The state of the art benchmark for the cold-start problem is the matrix factorization model SVD++. The comparison models made use of all the training data from the research dataset. The parameter values for nf, ni, and Lf are 19, 19 and 50 respectively. The test set consisted of a user and item with their matching rating, and the community the user fell into. Each review was considered independent to the next review in the test set and as such it did not alter the community structure and change the modularity value. MSE and MAE were used to evaluate the testing error and used as a measure to evaluate how well the models being compared performed.
This section presents the results that were obtained from the experiments carried out in this research and provides a discussion on the results gathered is provided as well as the shortcomings of the relevant models.
The comparison between Deep Community Learning and Deep Conv was to evaluate whether social information enhanced a deep learning recommender system in solving the cold-start problem. Thereafter, evaluate how well the Deep Community Learning performs against the state of the art recommender system model, SVD++, in solving the cold-start problem. As shown in Table 3 and Table 4, SVD++ on average outperforms Deep Community Learning and Deep Conv at solving the cold-start problem.
Error Model
Deep Community Learning
Deep Conv SVD++
Error Model
Deep Community Learning
Deep Conv SVD++
User Cold-Start 2.4015
The results in Table 3 and Table 4 show that, for the user cold-start problem, social information improved the deep learning model by achieving a lower MSE and MAE. SVD++ achieved a lower MSE than both models. For this cold-start problem the community-based solution performed better than a deep learning model without social information.
For the item cold-start problem, it is not clear if the use of social information improves the deep learning model, since the community-based solution achieved a lower MAE but higher MSE compared to deep learning model without social information. SVD++ was found to perform slightly better than the Deep Community Learning and Deep Conv solutions. For this cold-start problem social information does not provide evidence as to whether it improves a deep learning model.
For system cold-start problem, the use of social information improved the deep learning model. For MSE, SVD++ achieved better than Deep Community Learning, but for MAE Deep Community Learning achieved a lower error than SVD++. The system cold-start is the hardest problem to solve for since there is minimal information to no information available about users and items, and the use of social information to enhance a deep learning is proving to be useful. 6
The results showed that on average the Deep Community Learning enhanced the Deep
Learning Approach, and grouping users into communities was advantageous. However,
the approach did not outperform SVD++, a state of the art approach for recommender
systems. The enhancement of social information shows promise for improving Deep
Learning models at solving the cold-start problem. This finding is corroborated by other
related work which include [
Out of the 3 cold-start problems, the use of social information enhanced the deep learning approach for two of them. Social information achieved lower error values than deep learning at solving the cold-start problem. The shortfalls of the social information and deep learning models exist because deep learning models generally have a black box and pose the challenge of designing the ideal architecture to solve the recommender system problem and cold-start problem. Future work would look at blending communities so a user can fall into more than one community. SVD++ was used to map a user or item to a vector, future work would look at designing a deep learning model to map a user or item to a vector while forming a correlation between users and items.