Recommender systems traditionally focus on maximizing user satisfaction by suggesting preferred products to the users. This approach aims to increase the likelihood of immediate sales and relevancy for users. However, this focus can often neglect broader business strategic goals crucial for stakeholders. This paper presents a novel dual-objective Bundle Recommendation system tailored for King's ”Candy Crush Saga” designed to balance the relevancy of recommendations for players with game economy returns. We propose a two-step methodology combining an attention-based model to accurately capture diverse player behaviors and preferences, followed by a clustering algorithm that defines a representative pool of in-game bundles, tailored to the game platform's constraints. The eficacy of our approach is assessed through a controlled A/B testing framework, measuring the take rate (TR) for economic impact and the click rate (CR) for user engagement. We report significant performance gains, with a 41% increase in TR and a 33% increase in CR, efectively balancing user satisfaction with economic returns.
Recommender systems have become ubiquitous across various sectors, enhancing decision-making processes by facilitating eficient discovery and access to new products, services, and content. Traditional systems predominantly adopt a receiver-centric (user-centric) approach, focusing on maximizing user satisfaction without considering the broader strategic or business objectives that are crucial for system stakeholders.
However, the landscape of recommender systems is evolving. The distinction between ”organic”
recommender systems, which prioritize personalized user experiences, and ”strategic” or ”utility-aware”
recommender systems is becoming more pronounced. The latter aims to balance user relevance with
additional utilities, be they economic, social, or ethical, to optimize overall system value [
In the gaming industry, especially in systems featuring in-game economies and virtual marketplaces, utility-aware recommender systems aim to consider both user engagement and economic returns. This paper explores a method employed by King to achieve this alignment in Candy Crush Saga: a strategic Bundle Recommendation of in-game items, designed to appeal to players’ desires and keep the bundle economy balanced, while increasing transactions from the recommended items.
Bundle Recommendation (BR) [
CEUR
ceur-ws.org
(CR) [
Our approach employs a two-step, game economy-aware methodology for in-game bundle recommendation: We first employ an attention-based recommendation model to accurately capture individual user preferences. This step focuses on diverse player behaviors and preferences at a granular level. Given the constraints of the gaming platform, where only a limited number of bundles can be available in the game, in the second step we employ a clustering algorithm to define a pool of the most statistically representative in-game bundles. To evaluate the efectiveness of these bundles with respect to the game economy, we monitor the take rate (TR). User relevance is assessed in an ofline setting by measuring the model’s performance using cosine distance, and in an online setting, monitored through click rate (CR).
The contributions of our work are outlined below: • We introduce a novel two-step approach to Bundle Recommendation systems, that employs a combination of a supervised and an unsupervised approach. This utility-aware BR system is designed to consider both user preferences and the game economy. • We validate the online performance of our method using a controlled A/B testing framework, demonstrating the correlation between user relevancy and game economy. Click rate is employed to measure user relevancy, while take rate serves as a measure of economic impact. • We present how this solution is deployed in the real-world in order to serve millions of users on a daily basis, detailing the design of data collection, inference, training, and monitoring pipelines, ensuring maintainability.
Current recommender algorithms mainly suggest individual items by analyzing user-item interactions. However, recommending item sets, or bundles, particularly within online gaming environments, has received less focus. The integration of Bundle Recommendation (BR) in online games, which aims to simultaneously cater to user preferences and business goals, is an emerging research area. This field is still in its early stages of development, especially in the industrial applications of online games.
Within the context of bundle recommendations, several methodologies have been explored. [
Despite these advancements, considering both broader user engagement and economic returns
mentioned in the Introduction in bundle recommendations remains underexplored. In the gaming sector,
previous research has primarily focused on recommending game titles based on historical user data [
Our approach seeks to fill this gap by designing a Bundle Recommendation system with the dual objective of optimizing for user relevancy and considering the game economy and the constraints of the game system, while ensuring through careful monitoring that our recommendations not only achieve user satisfaction but also drive substantial business value.
Candy Crush Saga developed by King— is an online mobile game with millions of users, where players advance through a sequential map of progressively challenging levels by solving match-3 puzzles. The pace of advancement is contingent upon the player’s skill level, determined by their ability to strategically choose optimal moves, with the option of utilizing appropriate boosters and timing their usage efectively. As in other free-to-play games, players have the option to buy virtual items with real money through in-app purchases (IAPs). Users are presented with a range of bundles that consist of in-game currency and other in-game power-ups like boosters, time-limited boosters, and unlimited lives. The quantity of in-game currency and other in-bundle items can vary. An example of how bundle recommendations are presented to the users is depicted in Fig. 1.
user , data true labels ( , ) prediction function
( ) supervised learning
minimize ∑ cos_dist( , ) (a) Training flow (step 1) (b) Clustering flow (step 2) user data user data
for every user evaluate ( ) = = [ ,1 , ... , ] cluster vectors for = 1, .., into clusters emerging clusters 1, ... rounding of clusters
1, ... into bundles 1, ..., for every user evaluate ( ) = serve bundle to user according to
= argmin {cos_dist( , )} (c) Inference flow (executed daily)
In this section we explain in two-steps how we optimize for user relevance in an ofline setting, to ensure that the recommended bundles reflect the user preference on a global scale, using historical data. We also describe how we streamline our experimentation and deployment processes with a platform that supports the complete ML workflow.
Suppose we have users = { ∣ = 1, 2, … } and items = { ∣ = 1, 2, … }. Our solution comprises two sequential steps.
Step 1. In the first step, we predict one D-dimensional vector per user , denoted as = [ ,1 , ,2 , … , , ], where each value , represents the quantity of a bundle item purchased by the user . To predict this vector, we adopt a supervised learning approach. We formulate the task as a multi-output regression problem, where the target consists of numerical values representing the quantities of each respective item purchased by the user. During training, we aim to minimize the cosine distance between true preference and prediction as follows ∑ cos_dist(
, ) = .
Given that the targets are normalised, we use cosine similarity as our evaluation metrics as it ignores the overall scale of the predicted vectors, which is beneficial if the magnitude of the model predictions is not directly comparable to the targets. Moreover, the proportionality of the items’ values in the vectors matters to us in this use-case. The cosine distance metric enables a scale-invariant comparison of the proportions of the diferent items present in the predicted vector and the actual label vector. The training flow is depicted in Fig. 2.
Step 2. We expect to find many similar combinations of in-game item proportions in the predictions yielded from the model in Step 1, so in this step we employ an unsupervised clustering approach to define a discrete preference space. Since the quantities of the items in a bundle are discrete, the clustering approach serves the double purpose of discretizing the problem and resolving the data sparsity. The goal here is to define a set of preference clusters,
= { ∣ = 1, 2, … } .
Parameter is usually chosen between 5 and 30 so we preserve meaningful diferences between the bundles. These diferences depend on user perception and can be only studied qualitatively. In the experiments section = 20 is used. The distance from the raw prediction to the closest cluster centroid is:
At this point we have real-valued vectors, but given that the elements of the vectors represent actual in-game products, we need to round the values so that they describe the actual quantities of the various in-game items to be shown in the bundles. In this step, we convert the clusters to bundles that will be recommended to our users. By the end of this step, we will have defined a set of bundles = { 1, 2, ... }, = ( 1, ..., ) ∈ ℕ
with ∈ ℕ is the volumes of each item for every = 1, .., .
The distance between the cluster centroid and the final product with rounded values is: cos_dist( , ) = cos_dist(, ) = . .
The error to be minimized from this whole procedure is: cos_dist( , ) <= + + .
The process of creating clusters and bundles is visualized in Fig. 2. This 2-step process enables the segregation of the model predictions and the delivery of personalized results through bundles, providing lfexibility to easily modify and market diferent ofers.
Our ML model of choice for Step 1 is TabNet [
To enhance the ease of experimenting and deploying machine learning models, King has developed a
platform designed to support and automate various aspects of the ML workflow. Employing a self-service
approach, the platform provides machine learning practitioners with a range of modular components
and tools that streamline the modeling workflow. These resources are integrated under a unified system,
akin to previously described ML systems [
The system in Fig.3 is composed of four distinct pipelines: a data pipeline for daily data extraction, a training pipeline, an inference pipeline, and a monitoring pipeline. Our internal data notification service notifies our in-house pipelines orchestrator of new data availability and in turn, the orchestrator triggers all pipelines.
We use infrastructure-as-code tools, leveraging standardized modules that we apply to all environments to ensure consistency of the environments in development and production, allowing for more rigorous testing and minimizing one of the most common pain points of ML practitioners [32]. As we only need to make daily predictions, we opt for a batch prediction system that executes all of the pipelines daily.
The initial data extraction, including the retrieval of raw model features and the computation of labels, is handled by the data pipeline which draws from the data warehouse. The feature transformation pipeline is configured to ensure that the feature generation process is idempotent, even in the case of data backfilling.
The training pipeline fetches data prepared by the data pipelines to train models and generates artifacts which are consumed by an experiment tracker and the monitoring and inference pipelines. The type of data we hold within the artifacts are model weights, parameters and metadata, the git hash of the training code, evaluation metrics, training datasets and settings (e.g. learning rate, seed, optimizer parameters, etc.). This artifact choice enables us to tackle the reproducibility challenge inherent in operationalizing ML projects [33] by providing all the necessary elements to recreate the model.
The inference pipeline produces predictions for each feature for the next day and stores them in a database. After we have generated our predictions, they are converted to the bundles using the unsupervised clustering approach described in Step 2 of 3.1.
To prevent issues related to training/inference skew, we rely on parameterized queries as the input to the model for both training and inference pipeline. Together with a common data pipeline, this allows for a common source of truth, removing discrepancies in the data preparation. 3.2.3. Serving Once the personalized bundles have been generated, they are uploaded to the game services to be delivered to our players. The recommendation is cached in the game for the whole user session and the user is displayed the recommended bundle consistently through the assigned placements. To ensure reliability, the game system incorporates two fail-safe mechanisms: first, if there is no prediction available for a user in the most recent batch of bundles, the latest available bundle for that user is displayed. Second, if no bundle is available at all, we employ a fallback, non-personalized bundle. Once the recommendations are live, we validate their performance following relevant metrics with an A/B test setup. To avoid issues stemming from inconsistent definitions [ 32], we have a standard validation process for all models at various stages of the ML process (which is also version-controlled). Models undergo the same A/B test setup, and once in production, we monitor the same business metrics for all of them, this ensures reliable comparisons between models and an increase in iteration speed.
Model monitoring is essential for reliability and production-level machine learning systems [34]. Our system monitoring relies on a third-party platform. The monitoring pipeline is responsible for uploading daily predictions, features, and labels to this external platform. In addition to the standard monitoring policies to address training/serving skew, changes in feature distributions or relationship between features and labels [35], we track key business metrics to ensure the model’s relevance to the business [36]. In particular we monitor the bundles’ take rate, click rate, and recommendation diversity associated with the model’s usage. Furthermore, we implement feature importance monitoring to ensure that the contributions of features remain consistent during serving, fostering transparency in understanding the correlation between input data and model outcomes.
Upon any monitoring policy violation, an alert prompts an investigation, followed by the model retraining; post-retraining, a detailed manual investigation informs the decision to promote the updated model to production, utilizing CI/CD pipelines for automatic deployment.
In this section, we describe the experimental setup and outcomes of both the ofline training of TabNet and the online A/B testing of bundles suggested by our two-step approach introduced in Section 3.1. For the ofline analysis, we evaluate TabNet against XGBoost to determine efectiveness w.r.t. cosine distance. In the online scenario, we extend the comparison to include both TabNet and XGBoost, alongside a heuristic approach. The heuristic approach, known as heuristics, is crafted manually by subject matter experts and is based on game domain knowledge rather than personalized user data. Additionally, in the second online experiment, we investigate the impact of injecting varying levels of randomness—referred to as contamination—into our model recommendations.
The training process is carried out in batches, with the input being a matrix = { () } =1 ∈ × , where represents the number of samples in each batch, and denotes the number of input features in each sample. The input features include data on player behavior. We do not use or compare with public datasets as they do not have relevant properties and user actions that are required for our solution architecture, moreover, they cannot be used for online experiments. In our dataset, we only keep users who have been active for more than or equal to 30 days and aggregate all features by averaging them over an -day period. The target consists of numerical values [ 1, 2, … , ] representing the quantities of each respective item purchased by the user on their next active day after the -day period. We train our models over hundreds of thousands of users, for D=13. To simulate the production setting where users exhibit diverse activity levels, we do not aggregate the test set over a -day period. Instead, we include all users regardless of the amount of active days they have had. If a user has been active for less than days, we aggregate their corresponding input features over as many days as they have been active for. We train two distinct models: • TabNet with = 15 days • TabNet with = 30 days These two models are diferentiated by their respective number of aggregation days. We evaluate the models’ performance using the mean cosine distance as our evaluation metric, as outlined in Section 3.1. A lower value of this metric indicates better model performance. Using cosine distance for both optimization and evaluation ensures consistency since there is a clear alignment between what the model is learning during training and how it’s judged in evaluation. It also simplifies performance interpretation, as the model is directly optimized to minimize this metric. However, it may overly specialize the model, potentially missing broader patterns that enhance user satisfaction or real-world performance, since cosine distance focuses on direction rather than magnitude or other factors that could afect user preferences.
Based on the results shown in Table 1, we proceed with the TabNet model that uses a 30-day
aggregation period. It is worth mentioning that the mean cosine similarity is not normally distributed
on [
Model XGBoost baseline
TabNet 15day TabNet 30day
Mean Cosine Distance 0.234 0.124 0.103
4.3.1. Metrics In our setting we want to study especially the relationship between the relevancy and game economy. Therefore, we define a set of online metrics that can quantify the relevancy of the recommendation.
Click volume: Click volume ( , ) is the total number of clicks on product and day .
Acceptance volume: Acceptance volume ( , ) denotes the number of takes of product and day .
Click rate: Click rate () is the proportion between () and number of impressions on day . Take rate: Take rate () is the proportion between () and number of impressions on day .
We are using the click rate to measure the relevancy of our recommendation, while the take rate measures the performance of the game economy. Both of these metrics are scaled according to the number of impressions per day, since those can vary from user to user. We want to validate that increased relevancy for the user translates into increase in , while the overall efect on the game economy increases .
To be able to understand the online performance of our approaches to recommendation, we have tested the predictive models using A/B experiments. The A/B testing methodology allows us to compare the performance of key metrics between the treatment group and the control group in a single experiment. We denote the uplift of metric M as a percentage diference of the absolute values of M in the treatment group and control, respectively, scaled to the size of these groups. We denote the aggregate uplift in metric as Δ .
We define a pool of bundles as a set = { 1, 2, ... }, where is a bundle. We define a random bundle as a discrete random variable uniformly distributed on , i.e. ∼ U( ). Recommended bundle is defined as argmin{cos_dist( , ) ∶ ∈ } for model prediction and pool of bundles . Let be a uniformly distributed continuous random variable on [0, 100] and let be a random recommendation. The recommendation with contamination % is defined as a random variable = I( < ) ⋅ + ( ≥ ) ⋅ where I is the indicator function, i.e. I(true) = 1 and I(false) = 0 and is recommended bundle. Throughout the experiments section, we’ll be using four kinds of treatment groups that we denote : • TabNet model recommendation ( 0) • Recommendation with 10% contamination ( 10) • Recommendation with 30% contamination ( 30) ΔTR for recommended bundle ΔTR for base bundle 0 10 30
40 20 days
Prior to conducting experiments, Candy Crush Saga had a heuristic (rule-based) approach to serve bundle content to players. Here we conducted two experiments, where we tested the performance of the recommendation against the heuristic approach, which is our primary control group. Alongside this, we want to understand the relationship between the model accuracy and relevancy. We conducted experiment with random group as a source treatment, while serving contaminated recommendation in the target treatment. We use K=20 as the number of total bundles. The setting, including the source treatment and target treatment, is shown in Tab. 2.
Experiment 1 This experiment tests the model recommendation against a heuristic recommendation. Our source treatment serves heuristic bundle recommendation, while the target treatment is decided using the TabNet recommendation model. While the take rate uplift has increased significantly by 41% and the corresponding click rate has increased by 33%, as shown in Tab. 3
We visualize the trend in the increase of take rate of the recommended product in Fig. 4. The experiment started on day 11, and while the recommended product gradually gained popularity, the novelty efect stabilized roughly 20 days after the experiment started. This experiment has demonstrated the potential of varying bundle content with respect to the user engagement metrics. While the user engagement increased, the game economy metric increased even more, resulting in more successful transactions. However, it is necessary to understand further the relationship of how quickly decreases if the relevance deteriorates. This is the objective of the next experiment.
Experiment 2 In experiment 2 we study the relation between a random recommendation and model
recommendations with diferent levels of contamination. This enables us to understand how the key
engagement metrics uplifts deteriorate when the model is contaminated with a random recommendation
in some cases, being the source treatment a random recommendation in all cases. We take advantage of
the artificially created treatment group with a random recommendation as a method to compare to
other treatment groups 0, 10, 30. The changes in key metrics are presented in Tab. 3. We can see that
with increasing levels of contamination, the and uplifts decrease, however, at a non-linear pace.
The level of contamination ∈ [
Results interpretation The experiments aimed to study the relationship between the click rate as a measure of relevancy together with the take rate as a measure of game economy. In the first experiment, we’ve observed significant increase in both metrics, while the take rate uplift was higher than the click rate uplift. We’ve observed similar trend also in experiment 2, while we’ve achieved a much higher uplift in take rates when compared to the random group.
The question one could be asking is why we didn’t obtain a higher click-rate uplift when compared to the random group in experiment 2. The proportion of Δ /Δ is about 1.27 in experiment 1, while for 0 in experiment 2 it is around 3.35. We get almost three times higher in comparison to in experiment 2 is because the random control group created diferent recommendations for a user every day, which has increased clicks on that particular bundle, without increasing its relevancy. Additionally, these numbers are cleaned up from the novelty efect, reported after the initial 20-day period. We proved experimentally when serving diferent content every day, regardless if relevant or not, the resulting metrics achieve higher click rates than static or heuristic recommendations, which are the same every day. Therefore, to preserve the importance of click rate as a relevancy metric, experimentation with contamination is necessary, to prove a diminishing relationship between the recommendation performance and corresponding online metrics in experiment 2.
TabNet [
The model’s architecture uses a sequential attention mechanism that dynamically identifies and prioritizes important features for each sample. Specifically, by examining attention weights, we can ensure the model prioritizes relevant features. This helps identify and correct biases where the model might overemphasize features linked to unrelated targets, leading to more accurate, target-focused predictions.
TabNet’s self-supervised pre-training involves training on an unsupervised task without labels [
In the current 3-step approach described in 3.1, we use TabNet as the model of choice. The downside of the solution is its heavier computational load, which slows down the training pipeline. While real-time retraining is not a constraint in the current formulation, we are interested in exploring lighter models and the balance between ofline performance evaluation and computation costs for the training across diferent architectures. XGBoost or other vanilla Neural Networks like a simple feed-forward neural network, have been considered, and initial experiments reveal comparable results in our key metric, cosine distance, during ofline evaluation.
Adding smart processing to the same data can improve how our models identify player preferences. This section explores a few approaches for achieving this.
When exploring player’s preferences in historical data, it is important to capture both short-term patterns as well as more long-term in-game habits and preferences of a user. Currently, our model’s 30-day input feature aggregation captures long-term player behavior but doesn’t give higher weight to more recent events. Incorporating shorter aggregation periods may improve performance. Alternatively, a hierarchical model could automatically capture short-term patterns in lower levels and aggregate long-term behavior in higher levels.
The Cold Start Problem: Our training data focuses on in-game purchases, targeting users who made a purchase on a given day. We aim to determine the preference vector for items in the purchased bundle, so only paying and new users are included, while non-paying players are excluded. This creates a cold start problem, as recommendations require at least one purchase. Addressing this by developing models for underrepresented players could be included in the scope of a future iteration.
The loss function uses cosine distance between output and true label vectors but doesn’t account for weight diferences between items, which can cause overestimation when some items are overrepresented in training. Our analysis shows a 60% drop in performance when excluding one target, with in-game currency included in training but not in evaluation. This suggests in-game currency contributes 60% of the cosine distance. To better understand the players’ preferences and cater to individual play styles, we could explore target-specific weighting, separate models for distinct targets, or fixed allocation for specific targets, and focusing the model on the preferences for the remaining.
In this paper, we present a novel two-step approach to item recommendations in mobile games, which was applied and tested on a bundle recommendation problem in Candy Crush Saga. First, we’ve defined the general methodology and architecture of the solution, which was specially designed for the mobile game environment. Apart from ofline validation, the architecture was also tested in several online experiments, empirically modeling the relationship between the click- and take rates and model accuracy. The robust architecture and technical debt prevention strategies allowed the system to be deployed in two in-game placements, one of them being illustrated in Figure 1.
The novelty of this approach lies not only in the item recommendation methodology, which is subsequently applied to bundle recommendation but also in the implementation. The robust and fail-safe pipeline is designed to scale for millions of players and implements many policies that can prevent the delivery of inaccurate recommendations. We continuously monitor the system performance, both in the ofline and online environment, where we focus on understanding the click change- and take rates, but also other underdeveloped metrics, such as the impact of degenerate feedback loops and a corresponding deterioration in recommendation diversity.
The scale-invariant system defined in the methodology is an eficient tool both for the generalization of the system for other tasks and presents a responsible AI solution that makes sure fairness resulting from the generation of the recommendation is in place regardless of user’s level of activity or spending.
We’re grateful for the support from King we’ve been given while preparing the manuscript and the openness we work in that contributes to a good level of insight sharing. In addition to the authors who have contributed hands-on to the success of this solution, we also thank other teams in King for their help and support, specifically ML Special Projects, MLOps Accelerator, Core Data, AI Labs, CCS IAP&E and CCS Operators teams. We also thank Pradyumna Prasad for his significant support and contributions. https://doi.org/10.1145/3292500.3330658. doi:10.1145/3292500.3330658. [32] S. Shankar, R. Garcia, J. M. Hellerstein, A. G. Parameswaran, Operationalizing machine learning:
An interview study, arXiv preprint arXiv:2209.09125 (2022). [33] D. Sculley, G. Holt, D. Golovin, E. Davydov, T. Phillips, D. Ebner, V. Chaudhary, M. Young, J.-F.
Crespo, D. Dennison, Hidden technical debt in machine learning systems, Advances in neural information processing systems 28 (2015). [34] E. Breck, S. Cai, E. Nielsen, M. Salib, D. Sculley, The ml test score: A rubric for ml production readiness and technical debt reduction, in: 2017 IEEE International Conference on Big Data (Big Data), IEEE, 2017, pp. 1123–1132. [35] C. Huyen, Designing Machine Learning Systems: An Iterative Process for Production-ready Applications, O’Reilly Media, Incorporated, 2022. URL: https://books.google.se/books?id= YISIzwEACAAJ. [36] T. Schröder, M. Schulz, Monitoring machine learning models: a categorization of challenges and methods, Data Science and Management 5 (2022) 105–116.