The massive number of mobile games available necessitates a technique to help the consumer find the right game at the right time. This paper introduces HybridRank, a novel hybrid algorithm to deliver recommendations for mobile games. This technique is based on a personalised random walk approach, with the incorporation of both content-based and user-based information in the formulation of the recommendations. This technique is evaluated against traditional neighbourhood based collaborative filtering and content-based recommendation algorithms. This paper also explores the fact that this algorithm can also be used to help alleviate the cold start problem that is associated with little user data.[1] Online evaluations were conducted and results yield that the approach presented performed the best in both a controlled testing environment as well as in live production. This algorithm is currently implemented in a live mobile game platform developed by Singapore Telecommunications Ltd called WePlay.
Singapore Telecommunications Ltd (SingTel) launched WePlay, a mobile game app store in early 2014. With an increasing number of mobile games available to the consumer, there is a need for the development of a mobile game recommendation system. Key work on this domain has been done by Xbox [2] and others [3]. This paper presents a novel approach, HybridRank, which is a hybrid content-based recommender system using a biased random walk model. This is an adaptation of the ItemRank algorithm [10] for the incorporation of both content-based and user-based signals to generate recommendations. Online evaluations were conducted and results yield that the approach presented performed best when compared against user-based collaborative filtering[16] and content-based filtering approaches.[15]
Recommendation algorithms can be broadly classified into
two well known techniques: collaborative filtering methods
and content-based methods. User-user collaborative filtering
starts by placing the user in a vector space of their explicit
and implicit activities. A nearest neighbour algorithm with
a defined distance metric is then applied to identify what
items the users might like based on behaviours of users that
are most similar to them. Such an algorithm typically faces
issues of data sparsity [9] and algorithm complexity[
There is also an increasing focus on the ability to combine
both user and content metadata together in a hybrid way to
generate recommendations such as using Naive Bayes [1] or
clustering techniques [7]. Graph based approaches of using
concept graphs [12] and Markov Chains [
This paper explores new approaches towards the graph-based hybrid recommender system problem. It draws heavily on the Pagerank algorithm [8]. This algorithm has been adapted by ItemRank [10], as well as LBSNRank[4].The Pagerank algorithm computes an importance score for each node, and one can use this important score as a measure of importance within the network to be used to provide recommendations.
A recommender system deals with a set of users ui where i = 1, ..., n and a set of items pj where j = 1, ..., m. For each user, item pair, (ui, pj ) the system generates a score that will describe the relationship between ui and pj captured in a relationship score rij. To generate this score, this paper proposes HybridRank, an algorithm that combines both items features and user behaviours in a novel way for recommendations. The key steps in setting up the algorithm is to generate two matrices based on user and feature correlation respectively.
The user correlation graph Gu draws the correlation between games via user co-occurrence, i.e, a link appears between 2 games gi and gj if one or more users have downloaded both games. It is noted that multiple user co-occurrence matrices can be developed, where links between users can be drawn not only when they have downloaded both games, but also the frequency of which they have viewed and played the games on the platform. For these co-occurrence matrices, define matrix M 2 Rn⇤ m where n is number of games and m is number of users. Mxij represents the number of times a user ui has conducted an action x on the game gj. Examples of actions would be viewed, downloaded or played. Their respective correlation matrix can then be generated via the inner product of the matrices as follows: | Ux = Mx · Mx (1) The feature correlation graph Gf on the other hand draws the correlation between games via feature co-occurrence, i.e. a link appears between two games gi and gj if both games share one or more metatags. For example, two games that share the same developer, or price points will share a link. The feature set Fij is defined as the set of features which belong to both gi and gj where i 6= j and i, j 2 Savailablegames. These features can typically be generated from two sources. The first source will be structured information provided by the developer, the second being user generated content like reviews. This paper focuses on utilising structured metatags provided by the former source.
It is noted there are some features that are more important in determining game similarity as compared to others, for example two games sharing the same game mechanics is considered more similar than two games sharing the same price point. [14]. As such, a set of weights k associated with each feature can be defined. This set of weights can be learned, defined via empirical experiments or assigned via a TF-IDF on the content vector of the items[13]. Specifically for the experiments conducted a hierarchy of metadata was defined, and weights were assigned from qualitative user feedback. With that, the feature correlation matrix can be defined as follows:
n Fij = X k=1 k1k 1k will be 1 if both games gi and gj share feature k, and 0 otherwise. We normalise both matrices, U and F columnwise to generate stochastic matrices U˜ and F˜, such that each column sums up to 1. While the former is a symmetrical (2) matrix, the normalised matrices are not symmetric. The diagonals are also 0 by definition. These two correlation matrices become valuable graphic model to indicate correlations between games. The weights associated with the links provide approximate measures of games relation.
The idea underlying HybridRank is that a hybrid combination of both the user and feature correlation graph can be used to forecast the user preferences in a content-based approach. The personalised page rank algorithm has been shown to be a good algorithm to be used for such a use case as in [10]. This algorithm oe↵rs key properties of propagation and attenuation. Utilising the relationships between games, captured by both the feature and user correlation matrices, U˜ and F˜, the personalised page rank algorithm is able to propagate preferences through the graph from a given starting point. As the preferences move further away from the seed nodes, the influence of the user preferences diminishes, and such an attenuation property is aptly captured by the said algorithm. The personalised page rank algorithm is defined as below:
P Rui = ↵ · M · P Rui + (1 ↵ ) · dui (3) P Rui refers to the personalised page rank vector for a particular user ui, which gives an indication of the importance the di↵erent nodes in the system to the user ui. M refers to the stochastic matrix which captures the connectivity between all the nodes in the system. This paper uses the feature correlation matrix F˜ to represent the connections between the games. The vector dui is often referred to as the teleport vector, which allows the introduction of bias into the system to a given user ui. This generates a static score distribution vector of all the items that user has consumed or has an opinion for. For example, the jth element of the vector dui will be 1 if the user ui has downloaded the game, and 0 otherwise. The vector will then be normalised to sum to 1. The HybridRank algorithm builds on this idea by introducing the user correlation matrix U to build this vector dui . Let the set Dudli , Dui and Dui be the set of games that the v p user ui has downloaded, viewed and played respectively. The vector dui can be defined as follows: djui = ·
X k2D udli
Ud˜ljk + ⌘ ·
Uv˜jk + ✓ · X k2D uvi
X k2D upi
Up˜jk (4) Next normalise the vector to sum to one, to obtain d˜ui . For this paper, equal weights have been assigned to the weights , ⌘ and ✓ and further optimisation is underway. In the simple case, where the user has only downloaded one game, the vector d˜ui will simply be the jth column of the matrix U corresponding to the game that the user has downloaded. This draws upon not only the user preferences, but also assigns a bias towards games that are close in relationship to the games selected via a simple collaborative approach or captured via the user co-occurrence matrix.
Linear algebra approaches via power iteration can be used to solve equation 3. There has been research to improve computation eciency, one of them being in [17]. Also, in terms of complexity, [10] has shown that such a computation is ecient from both computation and memory resources.
The HybridRank algorithm was developed and deployed in two separate live experiments in relation to mobile game recommendations. The first experiment was done in a controlled fashion with a preloaded web prototype with a group of 526 users. The second experiment was done on the production app WePlay with over 100,000 users in Indonesia.
The seed dataset has 78099 users and 199 games. Each game also comes with its set of 149 set of metadata, including tags that are temporal in nature, i.e. whether it is trending, top grossing or curated by marketing team for that week. The following shows the distribution of the tags available. As from equation 2, weights were assigned to several top-level categories. This was purely done via qualitative assumptions and reasoning. A testing portal was developed and sent to 526 digitally savvy members of SingTel Digital Advisor Panel1. These users were asked to select up to five mobile games that they like, and four separate lists of recommendations were provided generated by HybridRank, kNN Collaborative Filtering, Top Grossing and Baseline. The baseline algorithm randomly selects games from the entire catalog. Each set of recommendations exposed seven games. The users were asked to then choose the mobile games they like across all the lists provided.
To evaluate the results, users were segmented2 across the dimensions of externalised gratifications and internalised fulfilment. The former comprises of factors associated with basic progression in the game, scoring, beating the competition. The latter involves the altruistic sharing of knowledge and experience, helping others in game progression and gaining respect and trusted recognition. Table 2 gives the definition and distribution of users across the segments. 1This is a panel of 15,000 users across South East Asia maintained by SingTel Group Digital Life to help in testing of new digital products. The users in this study have been screened to have played at least a mobile game in the past one month. 2This framework is an ongoing research by the team in Group Digital Life SingTel in an eo↵rt to deeper understand the gamer’s psyche, fundamentally based on Maslow0s Hierarchy of Needs [11]
Seeker and Contributor (Grp1) Trend Seeker (Grp2) Contributor (Grp3)
testers Testers are at the forefront and ac- 34 tively contribute online reviews to share knowledge
less actively/seldom contribute online reviews to share knowledge Testers who are not at the fore- 41 front but are actively involved in information exchanges with likeminded gamers through online reviews Testers who take heed from what 128 their friends/social circle play Testers who just want to stay in 246 the game and are indi↵erent to what others say Table 3 shows the performance results of the four algorithms across the five di↵erent segments. Success of the algorithms were measured by comparing the lift in average number of games selected against baseline. It can be seen that HybridRank provided maximal lift in segments of users who are indie↵rent and social gamers. For the small segment of users who are trend seekers, it appears that the top grossing algorithm performed the best. This could be because the HybridRank did not take into consideration global market features in the development of the item metadata.
In this second evaluation, the algorithm was exposed to over 100,000 users in Indonesia in the live WePlay app with over 900 games to recommend from. The section evaluated recommends games that are similar to the selected game. This particular use-case can be likened to the cold start problem, where there are no previous preferences of the user and the only preference being the current selected game. The HybridRank algorithm was compared with two other algorithms. The first being a commonly used content-based algorithm via an euclidean distance metric on the feature vector of the games as in [15] and the second being a baseline that chooses the more popular items within the same category as the selected game. The experiment was conducted live on the platform in Indonesia for a period of one month in an out of time validation fashion. The entire base was exposed to the algorithms in an alternating day fashion. The click through rates of the suggested game were measured - the higher the click through rate, the more e↵ective the algorithm was considered to be.
6.3% 0
Content-based 7.1% +0.127
HybridRank 13.3% +1.11 From the results HybridRank was shown to serve as a better algorithm in recommending games in a user cold-start scenario. The hybrid approach of using both user and feature correlation proved superior to the typical content-based approach.
This paper presents HybridRank, a personalised pagerank approach that incorporates both content metadata and user collaborative features in a novel approach. The algorithm was compared against state of the art collaborative filtering algorithms as well as content based approaches in live environment, with the conclusion that the hybrid approach performs better against the algorithms that were compared against. Also the algorithm proved to be able to help alleviate the cold start problem. Future work will include the incorporation of context such as user location, global trends in mobile gaming as well as custom curated metadata to the approach.
The authors would like to thank the WePlay team for the data and allowing us to conduct evaluation of our algorithms on the live app.
[1] Andrew I. Schein et al. Methods and metrics for cold-start recommendations. Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR ’02), pages 253–260, 2002. [2] Noam Koenigstein et al. The xbox recommender system. Proceedings of the sixth ACM conference on Recommender systems (RecSys ’12), pages 281–284, 2012. [3] Pavle Skocir et al. The mars - a multi-agent recommendation system for games on mobile phones. KES-AMSTA’12 Proceedings of the 6th KES international conference on Agent and Multi-Agent