Recommender systems usually leverage multi-task learning methods to simultaneously optimize several objectives because of the multi-faceted user behavior data. The typical way of conducting multi-task learning(MTL) is to establish appropriate parameter sharing across multiple tasks at lower layers while reserving a separate task tower for each task at upper layers. With such design, the lower layers intend to explore the structure of task relationships and mine valuable information to be used by the task towers for accurate prediction. Since the task towers exert direct impact on the prediction results, we argue that the architecture of standalone task towers is sub-optimal for promoting positive knowledge sharing. First, for each task, attending to the input information of other task towers is beneficial. For instance, the information useful for predicting the "like" task is also valuable for the "buy" task. Furthermore, because diferent tasks are inter-related, the training labels of multiple tasks should obey a joint distribution. It is undesirable for the prediction results for these tasks to fall into the low density areas. Accordingly, we propose the framework of Deep Mutual Learning across task towers(DML), which is compatible with various backbone multi-task networks. At the entry layer of the task towers, the shared component of Cross Task Feature Mining(CTFM) is introduced to transfer input information across the task towers while still ensuring one task's loss will not impact the inputs of other task towers. Moreover, for each task, dedicated network component called Global Knowledge Distillation(GKD) are utilized to distill valuable knowledge from the global results of the upper layer task towers to enhance the prediction consistency. Extensive ofline experiments and online A/B tests are conducted to evaluate and verify the proposed approach's efectiveness.
Recently, we have seen the widespread application of recommender systems, which involve
diferent types of user feedback signals, such as clicking, rating, commenting, etc. Moreover, no
single feedback signal can accurately reflect user satisfaction. For example, over-concentrating
on clicking may aggravate the click-bait issue. Therefore, it is highly desirable to be able to
efectively learn and estimate multiple types of user behaviors at the same time. And Multi-Task
Learning is a promising technique to address this challenge. Given several related learning
tasks, the goal of multi-task learning is to enhance the overall performance of diferent tasks by
leveraging knowledge transfer among tasks. With the Multi-Task Learning(MTL) paradigm[
Most existing methods of Multi-Task Learning(MTL) appropriately share parameters across
multiple tasks at lower layers while keeping separate task towers at upper layers. These methods
can be roughly classified into four categories. The first category comprises the methods of
hard parameter sharing. Among them, embedding sharing is the most intuitive structure to
share information. For instance, the ESSM model [
AITM [
Admittedly, the aforementioned methods achieve impressive performance. However, as the task towers exert a direct efect on the prediction results, the standalone task towers tend not to be the most efective design for promoting positive knowledge transfer by exploiting the task relationships. First, for each task, the information selected by the relevant tasks is extremely valuable. Accordingly, we introduce the shared component of Cross Task Feature Mining(CTFM), which utilizes delicate attention mechanisms to extract relevant information from other tasks at the entry layer of the task tower. With the common attention mechanisms, the explicit task-specific information distilled by lower-level networks are mingled together and one task’s loss will undesirably afect the inputs of other task towers, which is the task awareness missing problem and can hinder the learning of lower-level networks. In contrast to the usual attention mechanisms, our design can ensure appropriate information separation. We argue that reserving explicit task-specific knowledge has a positive efect on performance, which is validated in the experimental section. Second, because the tasks for recommender systems are related, the training labels of multiple tasks should obey a joint distribution. The prediction results for these tasks should not densely fall into the low-density areas. Therefore, a dedicated network named Global Knowledge Distillation(GKD) is introduced for each task to distill valuable global knowledge from the results of the upper layer task towers. For each task, the distilled global information helps to ensure consistent predictions with other tasks. We summarize our main contributions below.
• We propose the framework of Deep Mutual Learning across task towers(DML), which is compatible with various backbone multi-task models. • The proposed novel sharing structure helps to enhance efective knowledge transfer across diferent tasks. • We conduct ofline experiments and online A/B testing to evaluate and understand the efectiveness of our method.
In this section, we first introduce the problem of multi-objective ranking for recommender systems. Second, we describe the general design of DML. Finally, we elaborate on the introduced components.
Given a set of candidates with items = {}1≤ ≤ , the ranking model for recommender systems is to rank and recommend the top items = {}1≤ ≤ ⊆ for user so as to optimize the overall utility and enhance user experience. First, for each pair of user and item , the input feature is derived. Second, a multi-task learning model is utilized to estimate objectives corresponding to multiple user feedback signals. Furthermore, to compute the overall reward, we need to merge the multiple predictions with a function Φ shown in equation (2) to derive the item’s final reward score for greedy ranking.
, , ..., = (; ) 1 2 = Φ( , , ..., ) 1 2 (1) (2) where denotes the model parameters and Φ is usually a function manually tuned to reflect the reward of recommending item to user based on the business goals. With the estimated reward scores {}1≤ ≤ for the items in , the ranking model can recommend the item sequence consisting of the top items to the user .
With the existing MTL algorithms[
= (; ℎ)
where represents the lower level networks that encodes to diferent latent spaces with
partially or fully shared parameters . And is the upper-level network for task , which
accepts as input to model objective with task-specific higher level parameter ℎ. Multiple
candidate models [
In this research, rather than , we focus on the enhancement of upper-level networks for improved prediction performance. First, the shared component of CTFM is introduced, which leverages the attention mechanism to extract relevant information from the inputs of other task towers (the results of Equation (3)) as a complement to the target task. Please note that this attention is well-designed to solve the task-awareness missing issue, for which the excessive encouragement of knowledge sharing is not conducive to the extraction of taskspecific knowledge. With our design, the gradients computed from the target task’s loss will not impact the inputs of other task towers.
ˆ1, ˆ2, ..., ˆ = (1, 2, ..., ; ) where shared parameters is employed across diferent tasks.
Moreover, a separate multi-layer network is introduced for each task to process each element of {ˆ }1≤ ≤ and generate the hidden representation, based on which accurate prediction can be made.
ℎ = (ˆ ; 0 ) where denotes the task-specific MLP for task with separate parameters 0 .
Finally, for each task, a dedicated component named GKD is utilized to distill information from the hidden representations for both itself and other tasks to promote prediction consistency across tasks and more precisely model the corresponding objective.
= (ℎ, {ℎ }1≤ ≤ ; 1 ) (7) where is the dedicated component for task . In contrast to at Equation (5), we utilize task-specific parameters 1 here as specialization is usually helpful for upper layer networks. Furthermore, proper operation is implemented to ensure the prediction error of one task does not impact the hidden representations of other tasks.
(a) Cross Task Feature Mining (Shared) (b) Global Knowledge Distillation (for Task k)
Please refer to Figure 1(a) for the detailed process of CTFM. From the perspective of task towers,
the outputs from lower-level networks at Equation (3) can be recognized as the mined features
for them. First, for each task, the features mined for related tasks can be leveraged to enhance
its prediction performance. Thus, trainable task embeddings, namely {}1≤ ≤ , are derived
for the tasks to facilitate the learning of general task relations. Second, the importance of
features from related tasks can vary per sample. Accordingly, we stack together the items of
the set { + }1≤ ≤ to derive the matrix ∈ ℛ× 0 where 0 is the size of and
. Third, we use the projection parameters , , ∈ ℛ0× 0 to transform to the
query, key, and value matrix of , , ∈ ℛ× 0 . And gradient backpropagation
from and to is forbidden. Moreover, the scaled dot-product attention [
Please refer to Figure 1(b) for the detailed process of GKD. In contrast to CTFM, each task is assigned dedicated parameters for global knowledge distillation. Acting as the last step, we would like to facilitate more flexible modeling by promoting task specialization here. This module accepts the hidden representations for both the corresponding task and other tasks as input. First, a multilayer perceptron(MLP) is utilized to extract valuable global knowledge ( ∈ ℛ1 ) for the target task from the concatenation of all these hidden representations ({ℎ }1≤ ≤ ). Since the design goal here is to train task-specific MLPs to distill relevant global knowledge while not impacting the hidden representations, the gradient backpropagation from to the MLP’s input is prohibited. Second, we input and ℎ to another MLP with Sigmoid as its last activation function. Then, the weights of ’s diferent latent dimensions can be dynamically adapted for each sample. The weights are denoted by ∈ ℛ1 . Moreover, the weighted global knowledge ( ∈ ℛ1 ) is computed with the hadamard product of and . Finally, and ℎ are concatenated together as the input for the last MLP to make predictions for task .
In this section, we conduct extensive ofline experiments 1 and online A/B testing to prove DML’s efectiveness.
We evaluate our methods on two public datasets.
• MovieLens-1M[
For ML-1M, we introduce the binary classification task of positive rating prediction (>=4) and
the regression task of rating estimation. These two tasks are strictly correlated. For electronics,
following [
The merge function Φ in Equation (2) assumes that the model can estimate accurate interaction
probabilities for binary classification tasks (e.g. clicking) and absolute values for regression
tasks (e.g. watch time). Therefore, instead of the ranking metrics, such as NDCG [19] and MRR
[20], we use the metrics of AUC [21] for classification tasks and Mean Squared Error (MSE) [ 22]
for regression tasks. Please note that many other recommendation literature, such as [
For each feature, we use the embedding size 8. As suggested by the original papers, we use 1 level bottom sub-networks for MMOE, MSSM, and SB while 2 levels for PLE. For SB, a sub-network of 1 layer structure with 128 output dimensions is shared by the tasks. For other multi-task models, each bottom level includes three sub-networks, which have the same aforementioned architecture. For MSSM and PLE, task-specific and shared sub-networks are designated. For multi-task models, each task tower is of the three layers MLP structure (128,80,1) and each task is assigned equal loss weight. For the single-task model, each task utilizes the four layers MLP structure (128,128,80,1). For the first two MLPs at Figure 1(b), we utilize the one layer structure with 80 as the output dimension. For the last MLP at Figure 1(b), a one layer structure with 1 as the output size is used. If not explicitly specified, RELU [ 23] is used as the default activation function. All models are implemented with tensorflow [ 24] and optimized using the Adam [25] + + + 0 + + + + 0 + optimizer with learning rate 0.001 and mini-batch size 512. We run 20 times for each test to report the results.
Please refer to Table 1 for the overall results. First, DML achieves significant gains across all these tested multi-task models on the two public datasets, which shows DML’s wide compatibility. Second, DML-enhanced PLE and MMOE get the best performance for MovieLens and Electronics respectively. Considering their wide application in recommender systems, the results are as expected. Third, the multi-task models perform better than the single task models thanks to the knowledge transfer between tasks.
Besides AUC and MSE, DML should help to foster task consistency with CTFM and GKD. As the tasks of MovieLens are rigorously correlated, we verify whether DML really enhances task consistency on this data. First, we construct pairs of samples with diferent rating scores and count the pair numbers. Second, we count the number of pairs, for which the prediction scores of both tasks are in the same pair order as the rating score. The enhancement of the pair order consistency among the two prediction scores and rating score should positively contribute to the performance. Then, we can compute the metric of ’Consistency Ratio’. The listed data in Table 1 agree with our anticipation. (For Shared-Bottom, we also observe more pairs, for which predictions of both tasks are in rating score’s reverse order. This can explain its worse performance in spite of the better consistency ratio.)
We select the two latest algorithms of PLE[
DML is applied to the ranking stage[26] of an industrial large-scale news recommender system.
PLE [
In this papaer, we propose the framework of Deep Mutual Learning across task towers(DML), which is compatible with various backbone multi-task networks. Extensive ofline experiments help to verify DML’s efectiveness on multiple real-world datasets and across various base models. Moreover, thorough ablation studies are carried out to verify and understand the value of each newly introduced module. Finally, DML achieves significant online gains and has already been deployed to the online platform. [19] Y. Wang, L. Wang, Y. Li, D. He, W. Chen, T.-Y. Liu, A theoretical analysis of ndcg ranking measures, in: Proceedings of the 26th annual conference on learning theory (COLT 2013), volume 8, 2013, p. 6. [20] Wikipedia contributors, Evaluation measures (information retrieval) — Wikipedia, the free encyclopedia, https://en.wikipedia.org/w/index.php?title=Evaluation_measures_ (information_retrieval)&oldid=1095286224, 2022. [Online; accessed 9-January-2023]. [21] P. A. Flach, J. Hernández-Orallo, C. F. Ramirez, A coherent interpretation of auc as a measure of aggregated classification performance, in: ICML, 2011. [22] Wikipedia contributors, Mean squared error — Wikipedia, the free encyclopedia, https://en. wikipedia.org/w/index.php?title=Mean_squared_error&oldid=1127519968, 2022. [Online; accessed 31-January-2023]. [23] R. Arora, A. Basu, P. Mianjy, A. Mukherjee, Understanding deep neural networks with rectified linear units, in: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings, OpenReview.net, 2018. URL: https://openreview.net/forum?id=B1J_rgWRW. [24] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, et al., Tensorflow: a system for large-scale machine learning., in: Osdi, volume 16, Savannah, GA, USA, 2016, pp. 265–283. [25] D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, in: Y. Bengio, Y. LeCun (Eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL: http://arxiv.org/abs/ 1412.6980. [26] P. Covington, J. Adams, E. Sargin, Deep neural networks for youtube recommendations, in: Proceedings of the 10th ACM conference on recommender systems, 2016, pp. 191–198.