Massive Open Online Courses (MOOCs) are becoming a complementary, or even preferred, method of learning compared to traditional education among learners. While MOOCs enable learners to access a wide range of courses from various disciplines, anytime and anywhere, a significant number of course enrollments in MOOCs end up in dropouts. To increase learners' engagement in MOOCs, they need to interact with the courses that match their preferences and needs. A course Recommender System (RS) models learners' preferences and recommends courses based on their previous interactions within the MOOC platform. Dropout events in MOOCs, like other time-to-event predictions, can be effectively modeled using survival analysis methods. The objective of this talk is to illustrate the benefits of employing survival analysis in enhancing the performance of collaborative filtering-based course recommendations in MOOCs.
Massive Open Online Courses (MOOCs) platforms offer a diverse selection of online courses to learners worldwide, promoting the concept of equitable learning by removing barriers of location and time. Despite its considerable advantages, a significant portion of MOOC enrollments end up in dropouts. It has been reported that dropout rates for courses offered by prestigious institutions like MIT and Harvard can be as high as 90% [2]. While dropouts may result from various reasons, such as accessing only the free portions of the courses, finding the course or topic irrelevant, or insufficient competencies, this information is valuable for modeling users' preferences in MOOCs and would provide more infromed recommendations.
Recommender Systems (RSs) are intelligent information retrieval algorithms that utilize users' past interactions to suggest the most relevant items to them. Generally, RSs can be categorized into two main types: Content-based filtering and collaborative filtering. Content-based filtering RSs recommend items whose features match those of items that the target user has previously liked. On the other hand, collaborative filtering RSs model users' preferences based on similarities between the past interactions of users and items.
In a MOOC platform, a collaborative filtering-based RS can be applied to recommend courses to users based on their previous enrollments in the platform. While previous enrollments are informative to model users' preferences, the dropout information is still missing in this kind of recommendations. The dropout event in MOOCs is crucial as a significant portion of enrollments result in dropout. This additional information about user-course interactions can be useful to better model users' preferences or needs regarding the courses in MOOC platform. Survival analysis (SA) comprises a set of statistical methods that model the time until an instance experiences a specific event such as death or machine failure [3]. The key characteristic of survival data is that some instances have unobserved events, referred to as censored data. The most common form of censoring in SA is right-censoring, where the target event is not observed during follow-up or the instance is lost before the end of the follow-up period. The main strength of SA is its utilization of such partial information during the learning process by considering instances with censored events, which are usually discarded in classification and regression tasks. We believe that time to dropout is highly informative in modeling users' preferences in the context of course recommendations, as it provides valuable insights regarding students' engagement in MOOCs [4].
In a previous study [5], the authors demonstrated that SA can improve the performance of a specific RS, namely Bayesian Personalized Ranking (BPR), when the predictions of a SA method, trained based on time to dropouts, are embedded in the BPR algorithm. In this invited talk, we discuss how to generalize the usage of SA in any type of collaborative filtering-based RS. In the next section, we briefly report the existing literature around dropout in MOOCs and then in Section 3, elaborate on the research questions that could be tackled by researchers regarding enhancing MOOC recommendations using dropout information. Finally, in Section 4, we illustrate the possible experimental setup to conduct such research studies.
The task of dropout prediction in the context of MOOCs has been mainly modeled as a classification task [6,2]. While in this studies the task was predicting the event of dropout they ignored the time information, i.e., time to dropout, in their predictions. SA can be used to incorporate the time information in modeling dropout in MOOCs and there are some promising examples in the literature. In [7] survival analysis was used to model dropout risk in the context of MOOCs and unveil social and behavioral feature impacts on the outcome. Xie [8] utilized survival analysis to examine the hazard function of dropout, employing the learner's course viewing duration on a course in MOOCs. Labrador et al. [9] specified the fundamental factors attached to learners' dropout in an online MOOCs platform using Cox Proportional Hazard regression. Wintermute et al. [10] applied Weibull survival function to model the certificate rates of learners in a MOOCs platform, assuming that learners "survive" in a course for a particular time before stochastically dropping out. In [11] a more sophisticated SA deep learning approach was proposed to tackle volatility and sparsity of the data, that moderately outperformed the Cox model.
While SA has been already applied in literature to model dropout in MOOCs, to the best of our knowledge, such time to dropout from courses has never been incorporated in course recommendations in the context of MOOCs. The research gap this invited talk aims to investigate is whether utilizing this time-to-event information would enhance the performance of typical collaborative filtering approaches, and how it would do so.
In this invited talk, the focus is to discuss the merits of SA in course recommendations when the time-to-events information is available in the context of MOOCs. The following research questions would be interesting to tackle:
1. Does a SA method trained based on time to dropout have a positive impact on the performance of course recommendations in MOOCs when combined with a regular RS? In a previous paper [5], authors followed the most straightforward approach to combine the SA method and the RS. They enhanced the training data of the RS using the predictions of the SA method. What would be the other possible ways of combining the SA method and the RS? a) Is it possible to generalize the proposed augmentation idea in [5] to the other possible learning-to-rank recommendation approaches such as Weighted Approximate-Rank Pairwise (WARP) [12]? b) Is it possible to use the predictions of SA directly for recommendations, i.e., sort the items based on their risk scores predicted by SA? Can SA directly compete collaborative filtering approaches? c) How effective it is if the predictions of SA being used to post-process the output of a collaborative filtering method, i.e., to adjust the predictions of RS based on the predictions of SA? d) Does it make sense to model the whole prediction task in MOOCs as a multi-task learning problem where the tasks are predicting time-to-dropout or the risk scores for the SA method and ranking the courses for the recommendation task? Do these two mentioned tasks benefit from a partially shared learning process? e) To select the SA method, which objective should be followed? Should the SA method be selected/tuned based on SA objectives such as c-index, or RS objectives such as recall, or a combination of both? f) Can we define other events of interests in the context of MOOCs such as course completion or milestone achievement? g) How to recommend a completely new published course to the learners? How to address the item cold-start problem [13]? 2. How to generalize the discussed idea to other contexts and applications? a) To the best of our knowledge, there are only three publicly available MOOC datasets (described in Table 1) that can be used to validate course RSs. Is it possible to apply the same idea to other applications, for instance series recommendations, using the time to stop watching the series? b) Is it possible to extend the current idea with multiple time to events, for instance course completion and course dropout in the context of MOOCs? How to make an ensemble of recommendations [14] based on these different methods?
To conduct research studies to answer the raised research questions in the previous section, powerful collaborative filtering RSs should be employed as the competing approaches. Following the findings of the award winning paper [15], traditional collaborative filtering methods such as BPR [16], WARP [12], User-and Item-based k Nearest Neighbors (UKNN [17] and IKNN [18]) and Sparse Linear Methods (SLIM [19]) outperform more recent neural network based RSs and therefore can be used as baselines. These baselines can be used to benchmark performance of a SA-based RS, a collaborative filtering RS enhanced with SA post-processing, or a multi-task method that learns both the time-to-event prediction task and the recommendation task. Researchers could apply the competing methods on the three publicly available datasets, namely Xuentangx [20], Canvas [21] and KDD-CUP [20] and possibly an additional dataset from another domain such as series recommendations. The competing methods shall be evaluated based on the typical evaluation measures for RSs such as NDCG and recall.