Taking more time than expected to complete university degree programs is a global and known problem. The average time to degree 1 is 1.36 [1] in Latin America, and 1.31 in Chile, a number that has not experienced a relevant variation in the last 10 years. This statistic not only means that obtaining a degree takes on average about 31% more time than expected but also reveals that career delays are a serious problem with economic considerations: this situation causes in Chile an additional expense for families and the government estimated at US$ 500 million [2,3]. Delay in completing a degree program may have multiple causes such as the failure and repetition of subjects, temporary suspension of studies, or assuming less course load than expected according to the curricular plan [4]. Special importance is given to academic reasons behind academic delay because pressure exists to complete degrees on time: high education in Chile is financed by the student (or her family) with the help of scholarships or other funding benefits that require good performance and usually do not tolerate delays [5].

In this work, we explore academic delay in higher education programs, in particular an engineering program, and its relation with academic information summarizing the trajectory of students along with the academic program. While similar work focused in performance variables such as grades [6,17,19], we include other relevant aspects of the academic information such as the course load taken by the student each term, the difficulty associated with the courses taken, and repetition of courses which academic situation may determine the risk of punitive actions (for example, elimination of the study program by failing a subject more than twice, or failing, in the same semester, more than two subjects), among others. The first research question of this work is:

1) What is the relation between delay in obtaining the degree and academic information along the trajectory of the student?

We build prediction models on academic delay considering different features that can describe academic trajectories. As mentioned before, we seek to represent such trajectories in terms of different academic information spanning performance, course workload, and difficulty. These academic factors are relevant not only because they could predict academic delay, but because they could characterize the trajectories in an actionable manner, that is, further analysis of such trajectories could bring insights that could support counseling practices and curricular re-design actions. Thus, we state a second research question:

2) Can the academic trajectories relate to delay be characterized in a manner that provides information about student behavior?

Researchers have used different approaches to analyze academic information and typically centered around performance measures. Based on grading data and dismissing the background characteristics of the students, [9] identified curriculum subjects that can serve as effective indicators of academic performance. Using X-means [7] to group students yearly, [9] discovered typical progress patterns and evaluated the predictive capacity of the explanatory subjects. Considering the results in an entrance self-assessment test and the academic performance of the first year, [11] used K-means [10] to group students and follow their performance trajectories in the following 2nd and 3rd year, measuring the influence of the first year behavior in the progression of the curriculum [11]. [6] explored multiple personal and social factors that can affect the academic performance of university students and, using the Grade Point Average (GPA) as an explained variable through decision trees [14] proposed a qualitative model to classify and predict it [6].

Dropout has also been investigated. Aiming to relieve dropout, [16] used recommendation systems techniques to predict the grades that students will obtain in future subjects. The predictions were made using personalized multiple linear regressions (PLMR) [15] on student participation data in both traditional classes and Massive Open Online Courses (MOOC) [16]. [13] examined demographic variables as family characteristics; pre-university and university academic performance factors; and the participation or not in recovery courses, to predict the persistence of the students in the study programs. Using as explanatory variables the scores of the ACT standardized test for college admission, the average of grades in high school, the average grades of the first semester of the university; and using analysis of variance (ANOVA), Pearson product-moment correlations, and multiple regression analysis [12] showed that students who were academically prepared to take college-level courses were more likely to persist than students assigned to mandatory recovery courses [13].

Researchers have also focused on performing analyses of students'trajectories to inform the design of curricula. [17] used student performance data from a specific program to perform an analysis of the curriculum design of the program. In particular, they modeled the difficulty of each subject as its contribution (negative or positive) to the students' GPA and then contrasting this measure to a survey of student perception. Using the same performance data they also performed a dropout and enrollment path analysis [17]. Important to consider is that most of the previous work has been carried related to a flexible course-credit systems where the degrees are obtained as the sum of core and optional approved subjects [8,11,13,19]. This context differs from our sequential-non-flexible curricula, where the flow of subjects to take is pre-defined for all the terms of the program.

The representation of the academic trajectories of the students is not trivial, and it is necessary to consider the temporal dependence of the variables under study. Following this idea, [19] used frequent pattern mining [18] to reveal academic trajectories to understand the sequences of subjects to take that could improve student performance. In [20], a sequential data model is proposed that explicitly captures the temporal dependencies of the academic performance characteristics to form fingerprints or signatures that are constructed allowing different analytical interpretations and the development of predictive models for the risk of academic delay.

As presented, several of the proposed models -all of the regression type-considered the performance of the students as explanatory variables, without taking into account the dependence that exists in the performance of a student in var-ious subjects within the same semester, as well as in successive semesters. Our work distinguished from previous work in several aspects: in its multivariate nature considers in addition to academic performance aspects such as the academic workload and the difficulty of the subjects in each semester; incorporating the temporal dimension in the modeling of the students' trajectories; and exploring such trajectories in the context of a curriculum with a sequential structure, where about 90% of the courses to be completed are mandatory.

Academic information at the level of the degree program includes data of courses taken, passed, failed (with their grades), and dropped in each term of the academic life of the student. We combine these records with historic data and the expected curricular progress2 to generate a series of academic features in each term of the student academic life. These features represent different dimensions related to academic performance, academic workload (courseload), the relative difficulty of the courses, and the consistency between the courses taken and their theoretical order in the curriculum. Nine features for each term of each student were computed and are defined in Table 1. More details about the definition of the variables can be found in Appendix A.

We understand by academic trajectory to all this information organized in a term by term sequence. To allow comparisons between trajectories, we considered only the activity of the first 8 semesters (8 terms) for each student, counted from their first enrollment. These 8 semesters also represent a milestone of the study program because contain the required subjects to obtain the Licenciatura degree 3 . Considering this, the explained variable will be the delay in the 8th term (DELAY8), which measures how far is the student of having completed all courses of the first 8 terms of the plan in her first 8 terms of academic life. A student who passed all planned courses of the first 8 semesters of the program in her first 8 semesters of academic life, has delay zero.

To reduce inconsistencies in the comparisons of the trajectories (because of the dependency between the features and the structure of the program curricula), it was decided to analyze only one study program: Engineering in Computer Science. For the period of available data, this program implements three curriculum versions (2008, 2010, and 2015) each with 11 semesters of duration and an average of 6 subjects per semester. To study the performance of the students during their first 8 semesters, only whose admitted between 2008 and 2015 were DIF F IC A Difficulty of the semester, as an additive measure ("Alpha difficulty").

[0, max(HF Rj)SU Bi] DIF F IC B Difficulty of the semester, as a geometric measure ("Beta difficulty").

[0, 1]

The disparity of subjects enrolled in the semester: the difference, in semesters, between the highest level subject and the lowest level subject.

[0, 1]

Measurement of the delay between the theoretical and actual (average) semester of the student given their date of admission.

[0, 1]

considered. The resulting data set was composed of 14,199 records of academic activity of 365 different students.

We are interested in modeling the behavior of the 8th-semester delay (DELAY8) as a function of the other eight features defined for each semester. We limit the scope of the independent variables (the features) to the first 4 semesters because of two reasons. First, the idea of predicting delay (and also predicting dropout) gain relevance if a prediction can be done early, thus it seems reasonable to predict a delay in the 8th semester with information from the four first terms. Second, the four initial semesters correspond to the "Bachillerato" milestone in the engineering programs of the university, where the foundational courses of math and physics are concentrated and which have the higher failure rates, thus are strongly related with academic failure or success. The analyses will be performed in three steps. The first step is to perform an exploratory data analysis (EDA) for all the variables. We summarize the main characteristics of each variable (mean, median, quantiles, and range) together with their box plots, following with correlation matrix and principal component analysis (PCA) to gain an understanding of the structure of the set of variables and identify the most significant variables which can explain the academic delay.

The second step includes building predictive models using two supervised algorithms, linear regression (LR) and support vector machine (SVM) on the delay at 8th semester with other features of the first 4 semesters, such as difficulty and workload as predictors. These analyses target research question 1.

The third step is to characterize academic trajectories in terms of the relation of the term features and delay. An adaptation of the sequential model proposed by [20] is implemented. In our case, this model is built as follows. Each student trajectory is represented as a sequence of 4 nodes (semesters 1 to 4), where each node is a single value representing a delay score. To compute the i semester's delay score, first all students are clustered using k-means on their i semester's features. Then the score of the i semesters is the average delay of all students in the cluster (cluster members).

We explored the behavior of all the features in their semester-by-semester progression. Box plots and summary statistics for the variables can be found in Appendix B. Figure 2 presented here as a sample shows box plots of the features GPA (not cumulative weighted grade average) and DIFFIC A (an additive measure of difficulty) for the first 8 semesters.

From the analysis can be observed that the DELAY variable shows distributions with increasing medians and variability through the terms, as students on average accumulate more delay as they stay more terms. It can be observed that the distributions of the average grade (GPA) have medians that increase slightly but consistently through the terms. PASSRATE shows the lowest median in the 2nd semester with a value of 0.5. In the following semesters, the values of the medians increase progressively meanwhile the median passing rate for the 1st semester is much higher than the rest, with a value of 0.7. Dispersion is similar in all semesters.

In the case of the academic workload (WKLOAD), the median distribution in semesters 2nd to 4th is below the academic workload defined by the curriculum and it approaches that value in semester 5. The dispersion of these distributions increases between the 6th and 8th semesters. The relative difficulty, both DIFFIC A, and DIFFIC B, show medians that descend as students progress in their semesters, with quite similar dispersion. The disparity (DIS-PAR) shows only two median values: 0.143 for semesters 2 through 5; and 0.286, for semesters 6 through 8. Dispersion appears biased (positively or negatively) for all semesters, except for semester 8 in which symmetry is appreciated.

Correlation matrices and principal component analysis (PCA) were obtained to explore options for reducing the set of variables. The results of the analyzes for semester 3 are described in Figure 4. Biplots and correlograms for all the features can be found in Appendix B. In all semesters it is observed that the delay is negatively correlated with the variables of performance, curricular progress, and academic load, namely: GPA, PASSRATE, PROGRE, AVG, and WKLOAD. On the other hand, it has a very weak positive correlation with the measures of difficulty of the subjects (DIFFIC A and DIFFIC B) and weak negative with the measure of disparity (DIS). Two main components of the PCA shows two groups of variables: i) those that are more related to the individual performance of the students: AVG, PASSRATE, PROGRE and GPA, and ii) those which are related to the characteristics of the study program: WKLOAD, DIFFIC A, and DIFFIC B. It is interesting the distinct components represented by performance and workload. To represent each of the groups in the subsequent analyzes of this work, we selected GPA and DIFFIC A respectively based on a better degree of interpretation than they may have compared to the other features.

To analyze the predictive capacity of the explanatory variables, two families of models where built. The SGD i models use predictors GP A i and DIF F IC A i adding them incrementally from first to fourth semester. In this way, SGD1 implements:

and so on. The SAi models use as predictors all the features in each semester in the same incremental way. Thus, SA1 implements:

and so forth. Table 2 summarizes the results obtained. Regressions were built using a Generalized Linear Model (GLM) with Gaussian errors and an identity link function. The corrected Akaike Information Criterion (AICc) of every model and the difference with the lower value was computed. We can observe that this indicator declines, as expected, with the inclusion of the variables of the consecutive semesters. An important finding is that the predictive power in the first two semesters is quite similar between the model with two variables and the model with all variables. In contrast, by including semesters 3 and 4, the predictive power of the complete model is much greater. In the case of SVM, the RSME, RSq, and MAE indicators are calculated and ratify the observed with AICc. In particular, with the complete model until the fourth semester, an RSq of 0.805 is obtained which is high.

Trajectories were built for each student as a sequence of 4 values, one per each of the 4 first terms, each representing the average delay of all students who have similar academic features in a semester. To do this, we performed a clustering at each semester with all its features. Each term, the student falls into one cluster, from which the average delay marks his/her delay trajectory. The number of clusters obtained varied from 3 to 4.

Figure 5a shows the trajectories for all students who completed 8 semesters. To understand the delay behavior along time, the trajectories were presented into 3 groups: those students who reached their 8th semester with 2 semesters of delay or less (DELAY 8 ≤ 0.29), students who reached the 8th semester with a delay between 2 and 4 semesters (0.29 < DELAY 8 ≤ 0.57), and students who have a delay of more than 4 semesters (DELAY 8 > 0.57).

Figures 5b to 5d shows the trajectories for every one of those groups. It can be observed -as in Figure 5a-that for the first semester, the prediction of delay for all students is 2 or 4 semesters.

Most of the students who complete their 8th semester with a delay lower or equal than 2 semesters (Figure 5b), were projected with 2 semesters or less of delay throughout the entire program.

On the other hand, students who finished the 8th semester with a delay greater than 4 semesters (Figure 5d), maintained similar forecasts during the 4 semesters under study.

It can be seen that in both groups, the less and more delayed, the proportion of "good" and "bad" results for their 1st semester is quite similar, which would make the 1st semester a not reliable indicator of the final delay result.

In this research paper, we applied statistical and data mining methods to understand the different behaviors in the progression of students across a sequential curriculum program related to delay in obtaining a degree. First, features that summarize different aspects of the academic records information such as performance, workload, and course difficulty were built for each term a student stays in the academic program. A measure of the delay was custom-made in relation to the expected progress at term 8th. Second, we performed an exploratory data analysis of the features which revealed two different sets of variables that appeared orthogonal within the two principal components of a PCA: a group with all performance variables, and a group with workload and difficulty. The weighted average grade (GPA) and one measure of difficulty (DIFFIC A) were selected to represent these groups. Third, the predictive capacity of the features was explored, revealing that the selected two variables can predict the 8th-semester delay close enough as a model using all predictors. Fourth, delay sequences were modeled and represented as trajectories showing distinguishable groups of students'behavior. The evidence provided shows that the delay is not strictly determined by the students'performance during their first semester. Low initial performances can follow a path of progressive improvement and reduce their potential delay while ends the program. In conclusion, this work provides valuable insight into a better understanding of the dynamics of the progress in a sequential curricular program, potentially contributing to the decision-making of institutions, directors, and students.

= The number of CTS credits of the subject j. CT S Avg = The average number of CTS credits per semester of the study program. HF R j = The historical failure rate of the subject j.