Deep Sequential Modeling for Recommendation (DISCUSSION PAPER) Giuseppe Manco2 , Ettore Ritacco2 , Noveen Sachdeva1 , and Massimo Guarascio2 1 International Institute of Information Technology, Hyderabad, India noveen.sachdeva@research.iiit.ac.in 2 ICAR-CNR, Via P. Bucci, 8/9c, Rende, Italy {name.surname}@icar.cnr.it Abstract. We propose a model which extends variational autoencoders by exploiting the rich information present in the past preference history. We introduce a recurrent version of the VAE, where instead of passing a subset of the whole history regardless of temporal dependencies, we rather pass the consumption sequence subset through a recurrent neural network. At each time-step of the RNN, the sequence is fed through a series of fully-connected layers, the output of which models the proba- bility distribution of the most likely future preferences. We show that handling temporal information is crucial for improving the accuracy of recommendation. 1 Introduction Learning of accurate models able to suggest interesting items to the user is an important problem for many scientific and industrial applications. To address this problem, collaborative filtering approaches to recommendation were extensively inves- tigated by the current literature. Among these, latent variable models [5, 14, 17, 6] gained substantial attention, due to their capabilities in modeling the hid- den causal relationships that influence user preferences. Recently, however, new approaches based on neural architectures were proposed, achieving competitive performance with respect to the current state of the art. Also, new paradigms based on the combination of deep learning and latent variable modeling [7, 12] were proven quite successful in domains such as computer vision and speech pro- cessing. However, their adoption for modeling user preferences is still unexplored, although recently it is starting to gain attention [9, 8]. The aforementioned approaches rely on the “bag-of-word” assumption: when considering a user and her preferences, the order of such preferences can be ne- glected and all preferences are exchangeable. This assumption works with general Copyright c 2019 for the individual papers by the papers’ authors. Copying per- mitted for private and academic purposes. This volume is published and copyrighted by its editors. SEBD 2019, June 16-19, 2019, Castiglione della Pescaia, Italy. user trends which reflect a long-term behavior. However, it fails to capture the short-term preferences that are specific of several application scenarios, espe- cially in the context of the Web. Sequential data can express causalities and dependencies that require ad-hoc modeling and algorithms. And in fact, efforts to capture this notion of causality have been made, both in the context of latent variable modeling [11, 1, 3] and deep learning [4, 2, 18, 15]. In this paper, we consider the task of sequence recommendation from the perspective of combining deep learning and latent variable modeling. Inspired by the approach in [9], we assume that at a given timestamp the choice of a given item is influenced by a latent factor that models user trends and preferences. However, the latent factor itself can be influenced by user history and modeled to capture both long-term preferences and short-term behavior. Our contribution can be summarized as follows: (i ) we extend the frame- work to the case of sequential recommendation, where user’s preferences exhibit temporal dependencies; (ii ) we evaluate the proposed framework on standard benchmark datasets, by showing that (a) approaches not considering tempo- ral dynamics are not totally adequate to model user preferences, and (b) the combination of latent variable and temporal dependency modeling produces a substantial gain, even with regard to other approaches that only focus on tem- poral dependencies through recurrent relationships. The paper is organized as follows. Sections 2 proposes the modeling of user preferences in a variational setting, by describing how the framework can be adapted to the case of temporally ordered dependencies. The effectiveness of the proposed modeling is illustrated in section 3, and pointers to future developments are discussed in section 4. 2 Variational Autoencoders for User Preferences The reader is referred to [13] for more details of the approach illustrated here. We shall use the following shared notation: u ∈ U = {1, . . . , M } indexes a user and i ∈ I = {1, . . . , N } indexes an item for which the user can express a preference. We model implicit feedback, thus assuming a preference matrix X ∈ {0, 1}N ×M , so that x u represents the (binary) row with all the item preferences for user u. Given x u , we define Iu = {i ∈ I|xu,i = 1} (with Nu = |Iu |). Analogously, Ui = |{u ∈ U |xu,i = 1}| and Mi = |Ui |. We also consider a precedence and temporal relationships within X . First of all, the preference matrix induces a natural ordering relationship between items: i ≺u j has the meaning that xu,i > xu,j in the rating matrix. Also, we assume the ×N existence of timing information T ∈ IRM + ∪{∅}, where the term tu,i represents the time when i was chosen by u (with tu,i = ∅ if xu,i = 0). Then, i