In recent years, users have increasingly taken to consuming streaming video services via applications (i.e. Netflix, Hulu, Youtube, etc.) on so called over the top (OTT) platforms (i.e. AppleTV, Roku, Amazon FireTV, etc.). Given the very large (and still growing) number of streaming services, there is fierce competition to attract new customers, while maintaining customer satisfaction. Unfortunately, there is significant cost to attracting new users; thus, service providers are very invested in retaining end-users and keeping them engaged with their products. These customer retention efforts focus on providing exclusive and engaging content, personalized recommendations and intuitive user interfaces. When such efforts fail, the operators experience customer churn, wherein a subscriber stops using the service. By examining high level data collected on one such OTT hardware platform, we propose feature engineering techniques for modeling user behavior and leverage these features to develop application-level churn prediction models. Specifically, given a user u who installs an application a at a given time on their OTT device, our model predicts whether u will be engaged (or not engaged) with a after a particular time window.
Users may decide to abandon a streaming service for any number of reasons such as a limited time budget to consume content, an increasing affinity for a different application or a lack of compelling new content. Yet another reason may be that the user experiences more hardware faults (i.e. reboots, poor wifi, high memory usage, etc.) when a particular service is being used. In such cases, the user perceives these faults as being caused by the application and quits out of frustration. The key observation here is that both application-level and device-level behavior can influence user churn. Consequently, it is critical to model these heterogeneous factors along with temporally correlated features reflecting usage of different applications to accurately predict churn. To achieve this, we first analyze a dataset that captures high level events on OTT devices and examine the signals of application churn. These events could be user-initiated (i.e. opening or closing an application, restarting the device, putting the device to sleep, etc.) or device-specific events (i.e. automatic reboots, wifi drops, software/firmware updates, etc.) With this data, we examine questions such as: (1) how users allocate time across applications on their OTT device, (2) how often and for how long users engage with the device (specific applications), and (3) how long do users go dormant, and how likely is it for a dormant user to become reactive. In the second part of our work, we leverage these statistical insights to design interpretable models that are effective in predicting churn across a wide range of scenarios.
The naive approach to building a model that predicts churn would be to fix an observation time window T , extract a number of features of interest |m| from this window and then deploy a suitable classification algorithm that predicts whether a subscriber will quit a service after a period of time T . While this is entirely viable, there are two important drawbacks. First, the data is inherently noisy and high-dimensional; OTT devices send out periodic device-level and application level summaries (the start time and duration of an application session), and events observed on the box. This results in a feature space of size T × |m|. Second, there is significant inherent temporal correlation in the data; if a user spends a significant amount of time inside an application on successive days, there is a strong signal that he/she will engage with the same application on the next day. Flattening data over the entire window T into a single representation vector will lead to this information being lost. An alternative, more principled approach, is to learn latent attributes of the data using time-series models such as recurrent neural networks (RNN) [3] and use them as features for churn prediction. A potential issue with this approach is that the compressed latent vector is inefficient at capturing all the necessary information that leads to churn. Furthermore, it is extremely difficult to interpret the results of vanilla RNNs.
In this paper, we propose a model that address the drawbacks of the more conventional approaches. First, we introduce Attention LSTM (ALSTM), where we modify the neural machine translation (NMT) model [2] for churn prediction; ALSTM models the local attention. Second, we propose Neural Churn Prediction Model (NCPM), which incorporates two levels of attention i.e., local and global. ALSTM uses temporal-level attention (local attention) where different (sub) observation windows contribute different weights towards predicting churn. For example, in a particular week we might observe the subscriber slowly starting to watch more and more content on a new application, and spending less and less time in another in which they were previously engaged (and eventually abandon). Thus, features collected in this week might require higher priority when compared to other time frames. NCPM on the other hand is a more comprehensive model that captures each feature using a separate ALSTM. The individual ALSTMs are then combined with a feature-level attention layer (global attention). Global attentions are much better at prioritizing weights across different temporal features. For example, churning could be more influenced by device-level issues such as periodic reboots rather than application engagement. Although attention-based RNNs have been extensively used in the field of natural language processing [7,25], they have very rarely been applied to the problem of churn prediction. To our knowledge, the only work that appears to address this area is [26]; however, the attention mechanism used in their work is quite different than ours. We summarize the major contributions of our work as follows:
• Understanding user behavior: Through data engineering and statistical analysis, we provide several insights that explain the behavior of users in our OTT dataset. We begin by introducing our dataset in Section 2 and in Section 3, we model the behavior of OTT customers as observed in this dataset. The churn prediction models ALSTM and NCPM are proposed in Section 4 followed by the results of our experiments in Section 5. Finally, we review related work in Section 6 and conclude our paper in Section 7.
Our dataset consists of high level application session data (start time and durations) and device level events that spans from Sept 2018 to April 2019. The data was collected from a sample of 31k
AndroidTV based OTT devices deployed in homes and operated by a large provider. Each of these devices comes pre-installed with a set of applications; apart from these, users (u) can download others from a large catalog on the app store. We observed over 3.7k distinct applications being used in the dataset. However, some devices and applications are used very sporadically and account for very little data. To remove these, we pre-processed the data to filter out devices that were active for less than 90 days in total, and removed applications that were used on fewer than 15 devices in our population. This filtering resulted in 14,082 unique users (i.e., OTT devices) 1 and 462 unique apps; this dataset is denoted by D. Note that the churn prediction model builds features over the lifetime of the application on the device; this starts from the time the application was installed, to when the user is deemed to have abandoned it. Unfortunately, for the pre-installed applications such as Netflix, YouTube and SlingTV there is no install date. One can exclude such apps from the data; however, this leads to removing a significant number of users. This is because a large proportion of users tend to confine themselves to using the pre-installed apps (which also happens to be the popular streaming services). Alternatively, having all users in a single bin could lead to some serious bias in our modeling since for default apps, we do not have a clear signal on when it was downloaded. The user might have been using the app well before the onset of our study. Therefore, besides D, we create a separate dataset D e that completely excludes the default apps, while for (u, a) ∈ D that do not have a start date, we simply take the first log entry of a by u as a proxy for the actual install date. The subset D e covers 8,223 unique devices using 397 distinct applications.
Clearly, the earlier a service provider is able to predict churn (of u), the more effectively they can take action and address the underlying reasons as to why a customer might be departing. Consequently, we divide D and D e into different days of activities A, where A = {t |t ≤ T ,T ∈ {5, 10, 15, 20}}. For instance, T = 5 means we consider a maximum of five days of user activity to predict the churn. Table 1 shows the characteristics of dataset D and D e across different activity days. In the upcoming section, we explain the methodology of determining the churners and non-churners (i.e., columns four and five in Table 1). Here, one can see that as T increases, the number of users decrease. This is because the number of users that continuously use the OTT for say 20 days is far less than than those who use for just 5 days.
App and Device Usage: Figure 1 (a) shows the ten most popular apps seen in our dataset, based on the number of users that regularly use them. Sling Tv, Netflix, Youtube and Google games occupy the top four spots. Figure 1 (b) plots the distribution of daily time spent inside each of the applications. We find that users spend 3-4 hours on average with the OTT device, with a very small fraction of users that spend more than 8 hours. In fig. 1 (c), we break this daily spend into four different parts of a day, corrsponding to morning (5am-12pm), afternoon (12pm-5pm), evening (5pm-10pm) and night (10pm-5am). Unsurprisingly, we observe that users tend to spend more time in the evening when compared to other periods of a 1: Statistics of the churn prediction datasets D and D e for different range of activity days T . For example, T = 5 implies for a given user-app (u, a) tuple, u actively used a upto 5 days before churning.
day (median of 2.2 hrs). However, it is not significantly higher than afternoon, which has a median of 1.8 hrs and morning with a median of 1.4 hrs. Another key statistic of interest is the inter arrival time between application sessions. Note that there may not be an explicit indicator of a user quitting an application. Very often, users just stop using the application that is installed, or deactivate their accounts but keep the application installed. Thus, churn must be detected implicitly, i.e., by the fact of the application not being started by the user for a sufficiently long time. We calculate the arrival time between successive start times of an application session on a device (across all applications) and plot the maximum values, across all the devices, in fig. 1 (d). We observe that users, after an absence, return to the OTT applications within a median time of 6 days (100-200 hours). The 75%-ile value of this distribution is about 10 days. Later in this section, we leverage this to establish an inactivity threshold when we define churn more precisely.
User Engagement Patterns: Here we try to understand how users spend time on their OTT device. We wish to explore the following aspects: (a) Are there users who consistently use the box for same number of hours every day? (b) Are there dormant users who don't use their device for a while, but then reactivate it? (c) Are there users that engage with their device, but only intermittently and for brief periods of time? To answer these, we carry out the following analysis. First, for each day that our dataset spans, we compute the cumulative time (in terms of cummulative distribution function -CDF) that the user engaged with the device. Specifically, we compute (i, c i ) for each user, where i = 1, 2, .., |D| represents each day in our dataset, and c i is the total number of hours spent on the device upto day i. Next, we carry out a non-linear fit on this data for each user, recording the learned slope, intercept and standard error as derived features for each user. Subsequently, we cluster the derived features using K-means; the number of clusters is determined based on the silhouette score [23]. This analysis yields four main behavior patterns that cover the vast majority of users, and are depicted in fig. 2. Each plot is based on the original data of cumulative device engagement time (x-axis is days elapsed, y-axis is cumulative time spent). These four patterns can be labeled as follows: (1) mid bloomers (a), these are users who are initially silent and do not use the OTT box heavily, but then suddenly start using during the middle phase of their total period. (2) late bloomers (b), these users remain dormant for a longer duration with minimal activity, but suddenly start using the device towards the end. (3) potential churners (c), these are users who are of interest to us. As explained by the plot, these users start using the device heavily at first, but then stop using the box for various reasons. Please note that since y-axis is the CDF, the flat line here indicates minimal or no activity (also indicated by very low standard deviation, since there is no activity). ( 4) consistent users (d), finally, these are users who are consistent and regularly use their OTT boxes to watch different shows.
Understanding Churning Behavior: Before introducing our prediction models, we briefly explain how we label a user (or device) having churned, i.e., left the service. This is fundamentally a difficult task because there is no explicit signal for this behavior. Further complicating things, (a) some users don't use the app for a few days, but return back after a brief period of inactivity, and (b) some users simply download the app once (or spend a brief amount of time in it) and never use it again. Figure 3 depics, at a high level, all the information for a user (u) application (a). The dotted lines at either end capture the time data was collected and each of the green vertical lines in the middle indicate the start of application sessions (a 1 is the first session, a n is the last). Here, we see that the application was downloaded after the start of the data collection and used several times, the last instance is at t 3 . Somewhat infrequently, we see the device itself disappears from the dataset; we consider this a signal that the user has disconnected the device and is no longer using it. In this scenario, we capture this event having occured at t 4 .
With this depiction, we can now define churn in very specific terms by addressing the two challenges previously discussed. First, we require that the application not be used for a period of time after the last use. Following the example in fig. 3, we impose the condition ∆ 2 ≥ T 3q . Here T 3q -an inactivity threshold -is the 3rd quartile of the distribution in fig. 1 (d) and turns out to be 10 days. Second, we require a minimum number of sessions to be recorded for an application and user. Specifically, u should have engaged with a at least K times, i.e., n ≥ K and we set K = 3. Figure 4 illustrates the characteristics of devices that are exclusively labeled as churn. We see that the median inter-arrival times for the top 5 apps is around 30 days (Figure 4 (b)), which is significantly higher than the generic inter-arrival characteristics shown in Figure 1 (d). In figure 4 (c) we notice that users who download more apps tend to have higher churning rate, we obtained a Pearson correlation coefficient of 0.67. It is also interesting to observe that as the churn increases, users tend to switch between apps more frequently, where the app switch is indicated by the session feature (y-axis).
Given a user u and an app a, our objective is to predict if u will continue or stop using a. We realize each user-app entity as a tuple (X , M x , Y ) where X = {x 1 , ..., x t } is a stream of events (or logs) that spans a time t ∈ T . Each event x comprises of M features, and Y = {y 1 , ..., y t } are the binary labels that indicate churn (or non-churn) at t. When designing our churn prediction model we had two main objectives. First, since our data is highly temporal, it is important to learn the latent characteristics of churners (and nonchurners) in such a way that it embeds the temporality of events.
Second, not only should we predict the churn with good accuracy, but also produce highly interpretable results. In other words, we should reason out as to why a user is churning. To achieve this, we propose the following models: Both the models are based on recurrent neural network (RNNs) that have shown to be effective in modeling time-series data [5,8].
RNNs take a series of temporally dependent inputs and learn their latent representation (or hidden state vector) using the following expression:
where h t is the hidden layer at time t and f is some non linear function. For our application, we model f using long short-term memory network (LSTM) [13]. LSTM has four states that are defined as follows:
where t is the time step (i.e, days) , h t is the hidden state at t , c t is the cell state at t , x t is the hidden state of the previous layer at time t, i t , f t , o t are the input, forget and out gates, respectively.
Attention LSTM (ALSTM): Obviously one can predict churn by simply providing the input X to a vanilla LSTM, get the latent vectors h from the final layer, and use them as features for prediction.
A potential issue with this approach is that the compressed (or low dimensional) latent vector h is inefficient in capturing all the necessary information that attributes to churn. As explained in Section 1, in a particular week we might observe the subscriber slowly starting to navigate towards a new application and spend less and less time in one that they were previously engaged with (and eventually abandon). So, it is important to give high priority to these time windows when compared to other weeks. Modeling churn using vanilla LSTM networks fails to prioritize such key events. Inspired by recent developments in neural machine translation (NMT) [2], we incorporate attentions into LSTM to overcome this issue. Since our application is very different from natural language processing, we introduce two modifications over NMT. First, we replace the decoder part with a single layer neural network (NN) with sigmoid activation for churn prediction and second, we change the attention mechanism to suit our problem. The proposed ALSTM model is shown in Figure 5 (a). Here, the attention block A outputs a vector of weights α that emphasizes the importance of the latent vector h for a given time frame t. The weighted latent vector p is defined as follows:
where the weight α j for a time instance t is defined by
Neural churn prediction model (NCPM): One drawback of AL-STM is that it is unable to prioritize across features. For example, churning could be more influenced by the consistency of users (see Figure 2), while the number of downloads might have little importance. To overcome this problem, we incorporate both temporallevel attention and feature-level attention. As depicted in Figure 5, instead of treating the features as a single vector, we decouple the features and model them using individual ALSTMs. Similar to ALSTM, the attention block p m of an a feature m captures the influence (or weight) of the latent features from different slices of time. On the other hand, the feature-level attention is capture by the block B, which is defined by the following expressions:
In the above expression, β denotes the individual attention weights that is defined as follows:
where z = p 1 ⊕ {p i } m 2 is the concatenation (indicated by ⊕) of the feature-level latent vectors . Finally, to predict the churn, a linear projection with a sigmoid function is connected to the output of the last layer to produce user churn prediction as follows:
The loss for both ALSTM and NCPM is computed using binary cross entropy, that is defined as follows:
Obviously, from Table 1, one can notice that our dataset is biased towards negative samples (i.e., #non-churns). Therefore, to create a balanced dataset, for every positive data point (i.e, churns) for an app a, we randomly sample a corresponding negative data point. We test our models by varying the number of days in the training sample (explained in Section 2). This helps us to see how quickly can our models predict the churn. For all our experiments, we use 10 fold cross validation, where eight folds are used for training, one for validation, and one for testing. The deep learning models are implemented using Keras with Tensorflow as the back-end.
We compare the performance of the proposed ALSTM and NCPM with three baseline methods. Unlike the proposed models (i.e., AL-STM and NCPM) the following baselines do not capture the temporality in data. Therefore, the inputs to these model are flattened vectors across the time frames.
Logistic Regression: the classic model for binary classification problem. Albeit simplistic, it helps us to understand if a linear decision boundary is sufficient to capture the churners and the non-churners. We use L2 norm as the regularizer and Stochastic Average Gradient (SAG) as the solver.
Multi-layer Perceptron (MLP): We consider a simple two layer neural network and a dropout layer to avoid overfitting. A linear projection with a sigmoid function is connected to the output of the last layer to produce user churn. Similar to logistic regression, MLP does not capture the temporal dependencies between the data. The number of neurons are set to 80 for each intermediate layer.
Random Forest (RF): Despite the rapid advancements in the field of deep learning, ensemble techniques such as RF [4] remain highly competitive in producing excellent results on data with several modalities. In our experiments, the number of decision trees are set as 50 and the maximum depth as 10.
Classification accuracy: Tables 2-3 shows that NCPM outperforms all other models for both datasets D e and D, achieving an accuracy of upto 92%. As we increase the number of days the accuracy increases for all models (except logistic regression). Here, CA-5 implies the classification accuracy with just 5 days of data, while CA-20 implies 20 days of data. We can also see that the proposed ALSTM is not as good as NCPM which proves the following: (1) it is important to learn the latent attributes of each individual features separately and (2) incorporating both global and local attention is necessary. That being said, ALSTM clearly outperforms MLP, which emphasizes the necessity of learning the temporal actions of OTT users. The worst performing model is the logistic regression, which is just slightly better than a random selection. This illustrates the difficulty of our churn prediction task. The performance of RF is very close to that of ALSTM, which indicates that ensemble models are still a strong candidate for our problem.
In general, the performance of models over non-continuous data is much better than its continuous counterpart, this can be explained using the following example. let us say that u uses an app for six days before churning and we have the following data for u {m 1 , m 4 , m 6 , m 7 , m 8 , m 10 }, where m is some feature and the suffix indicates the day. Our objective is to predict the outcome on sixth day, using the first five days. Since the user does not use the OTT box for days two, three, and five, the continuous data that is fed to our models (i.e., both NCPM and ALSTM) is essentially a sparse vector {m 1 , 0, 0, m 4 , 0}, which has several missing values. On the contrary, for non-continuous dataset, we will have the actual usage values for five days. This obviously means that the model gets to train with more observed data points, which leads to better prediction accuracy. Another interesting observation is that the performance of all models (except logistic regression) is noticeably better on the all-apps dataset. One key reason for this outcome is the popularity of the default apps. Apps such as Sling TV, Netflix and Youtube are significantly popular than other non-default apps. Therefore, the models are able to effectively learn the churn patterns for such apps more effectively. AUC and ROC characteristics: Figures 6 and 7 compare the ROC characteristics of the proposed models with other baselines. The corresponding AUC values are listed in Table 4, due to space constraints, only the non-default case is furnished. Similar to the accuracy scores, for most scenarios, NCPM remains dominant over other models. We also notice that RF tends to perform better than NCPM and ALSTM when the temporal length of data is low (i.e., with just five days). However, as we incorporate more days for training, there is a clear increase in the performance of our models. The outcome for dataset D is much different than D e where we are able to achieve an AUC of almost 89% with just five days of data; additionally, ALSTM seems to perform very similar to NCPM. Interpreting the churn prediction: One of the key strengths of our model is interpretability. As explained in Section 1, ALSTM provides single level of interpretability, which indicates which days are important when predicting churn. NCPM on the other hand, has two levels; besides telling the important days, it also tells us which features are important. We present the interpretability scores as heatmaps in Figure 8. Due to the lack of space, we only furnish the results of non-continuous dataset. Heat maps (a)-(d) explains that the influence of features are not uniform across apps; for instance, when we have less days for prediction, the churn is influenced by two main attributes namely, the number of downloads and the cluster types (Figures 8 (a) and (c)). As we incorporate more data for training (i.e., the number of days) the attention tends to get more focused towards a few key features. For non-default apps, there seems to be more attention on the inter-arrival time, while for all-apps the influence seems to be more towards the number of reboots. It could be possible that these apps experience a higher number of app crashes, which could lead to user rebooting the device. For non-default apps, Hulu, Plot Tv and Kodi is heavily influenced by the cluster id feature that we engineered in Section 3. When it comes to temporal attention (Figures 8 (e)-(h)), for dataset D, the influence is mainly concentrated on a few selective days, i.e., day 4 for non-continuous case, and day 2 for continuous case. Contrary to this, for D e this influence is spread across almost all days.
In Section 3 we explained that the engagement pattern of users could have a strong impact on churn. To show this effect, for each user, we get the final attention score from individual RNNs of NCPM and plot the outcome in Figure 9 (a). Here, we can see that consistent users are the highest indicators of non-churn, while potential churners are the highest indicators of churn. Interestingly, mid bloomers seem to have higher attention over late bloomers when it comes to predicting non-churners, while the opposite is true for churners. Figure 9 (b) provides a more deeper look into this outcome by emphasizing on the importance of temporal progression on the user types. Unsurprisingly, during the initial phase (elapsed duration of 10-20%) almost all types have less attention weights. This is because, during the early phase, we do not have enough data about the user type. As the time progresses, around 20-50% of the elapsed duration, we see that potential churners have the strongest impact on the outcome followed by consistent users and late bloomers. Around 50-80% , the impact of potential churners drastically reduces, while mid and late bloomers increase. At the final stage (i.e., 80-100%) almost all user types have less importance (or attention). This is because, during the last phase, there is more available data in the form of other features such as number of
The problem tackled in this paper is related to the following topics:
(1) churn prediction (2) user behavior modeling and (3) interpretable neural networks. We now detail some existing research that correspond to these topics. Churn Prediction: User retention (or churn) has been extensively studied in the field of social computing and human computer interaction (HCI) [9,14,27]. However, developing predictive models for churn is still at infancy. Au et. al. [1] adopt a rule based learning technique for early churn prediction. In [29], the authors tackle the problem of churn prediction in mobile apps. They find that application performance such as energy consumption and latency have a significant impact on retention. [16] use a social influence based approach for churn prediction. Recently, [26] develop an interpretable framework that constraints the objective of RNN with the outcome of K-means clustering to predict the retention of users in Snap Chat.
Modeling user behavior: There are numerous research on behavior modeling [10,15,21]. For example, [6] predict user intents by leveraging the activity logs in Pinterest. [12] predict the likelihood of a successful search in web search queries. They show that user behavior are more predictive of goal success than those using document relevance. [22] model the behavior of users in the Kickstarter crowdfunding domain using a heterogeneous combination of social communities, popularity of projects and the impact of reward categories. Studies such as [11,24] and [21] model user behavior from sequencial actions such as click streams and social Interpretable Sequence Modeling: RNNs have become the stateof-the-art technique for sequential modeling [5,13]. Albeit a plethora of research in the NLP domain [2,19], extending interpretable RNNs for other real world applications is still an emerging field of research. In a recent work, [18] predict the engagement of users in the Snap Chat app by capturing the in-app action transition patterns as a temporally evolving action graph. [17] develop an interpretable LSTM to learn multi-level graph structures in a progressive and stochastic manner. [20] propose a dual stage attention model for medical diagnostics such as heart failure prediction. Zhou et. al. [28] propose an attention-based RNN that predicts the purchase probability of users for targeted ads. Albeit having a similar NN architecture as ours, their problem is quite different than churn prediction. Additionally, they modeling of local and global attention is quite different than ours. To the best of our knowledge, the only research that closely resembles our work is the churn prediction model proposed by Yang et. al. [26]. However, the attention mechanism used in their work is quite different than ours.
In this paper we proposed interpretable recurrent neural network based models for prediction churn in over the top media (OTT) devices. In the first part of the paper, we analyzed the behavioral characteristics of users and found that they can be categorized into four main types: mid bloomer, late bloomers, potential churners and consistent users. In the second part, we introduced two models for churn prediction, namely Attention LSTM (ALSTM) and Neural Churn Prediction Model (NCPM). In ALSTM, the prediction of churn was done by weighting on individual time frames (temporallevel attention) and (2) NCPM, we used two levels of attentions namely, feature-level and temporal-level. We showed that NCPM outperforms all other models over a wide range of test cases and achieves an accuracy of upto 89% and AUC of 92%.