Worldwide, a signi cant concern of universities is to reduce academic dropout rate. Several initiatives have been made to avoid this problem; however, it is essential to recognize at-risk students as soon as possible. In this paper, we propose a new predictive model that can identify the earliest moment of dropping out of a student of any semester in any undergraduate course. Unlike most available models, our solution is based on academic information alone, and our evidence suggests that by ignoring socio-demographics or pre-college entry information, we obtain more reliable predictions, even when a student has only one academic semester nished. Therefore, our prediction can be used as part of an academic counseling tool providing the performance factors that could in uence a student to leave the institution. With this, the counselors can identify those students and take better decisions to guide them and nally, minimize the dropout in the institution. As a case study, we used the students' data of all undergraduate programs from 2000 until 2019 from a public high education university in Ecuador.
Worldwide, one main concern in all Higher Education Institutions (HEI) is
student dropout [
Copyright c 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
In the speci c case of Latin America, the World Bank reports a similar rate
than the OECD with a 22% dropout rate [
The available data about students' academic performance, creates an
opportunity to identify what factors a ect student dropout and take preventive
measures to solve this problem. This is the reason why studies have been done using
prediction algorithms to solve this situation [
The rest of the paper is organized as follows. Section 2 provides a brief overview of the relevant literature. Section 3 describes the data used in our predictive model and summarises our work. Section 4 shows the experimental results from the study. The discussion about our work is presented in Section 5. Finally, Section 6 draws conclusions from this analysis, and suggests possible directions for future work in this area. 2
When it comes to input variables to predict dropout, there are di erent models and input features found in the literature, some focused on analyzing a single type of data, while others mix di erent groups of data.
Regarding a single type of data, for example, in studio [
In terms of mixing di erent groups of data, in [
While all of the above models are accurate in their contexts, they are not scalable because they use small samples, only use the rst academic years, or focus on speci c elds or undergraduate programs. Moreover, a mix of di erent variables, such as psychological, demographic and academic, could cause biased results or less accuracy prediction. Besides, certain models cannot be replicated because the variables are not speci ed but rather mentioned in a general way. Therefore, a model that can be scalable and replicable is needed to understand the student's path from beginning to end and consider variables that cause an impact on learning performance. 3
The study took place in Escuela Superior Politecnica del Litoral, ESPOL, a public engineering-oriented university located in Ecuador. The model was implemented in Spyder3, an open-source platform, and we used Python V3.7 as programming language and made use of the Scikit-learn library, an open-source library implements many machine learning algorithms. The proposed model and each of its phases is described below. 3.1
The information was provided by the Information Technology and Systems Management4. For the model, we considered the bachelor's degree in Industry Engineering, which has 753 students enrolled from 2000 until the rst semester of 2019. This is one of the undergraduate programs with the most signi cant number of students enrolled during this period. The years selected were used as boundaries to allow ve years of full studies from the time the students began their rst academic semester until obtaining a professional degree.
This phase focused on the information acquired, with the purpose of generating consistent data that could be used for the algorithm. The data was categorized as follows: socio-demographic data of the student (gender, marital status, employment status, city of residence, among others), data about the undergraduate program (number of total credits, code of the course, among others) and data on the student's performance (courses taken, state of the course, number of times the course has been taken, GPA, credits of the course, among others). During the pre-processing phase, cleaning, transformation and integration procedures were performed due to the origin of the information.
A recognition process was carried out, in which the academic terms were appropriately identi ed because the period of study varied depending on the term. Therefore, the periods were the rst and second term, also called ordinary periods, and third term also called extraordinary period. For the predictive model, ordinary periods were selected as they had the same duration. Subsequently, we enumerated the semesters in which students had been studying with their respective academic record, in order to execute the analysis per semester.
After the pre-processing, the most representative academic performance variables were calculated and used as inputs for the model, which are detailed in Table 1. Besides, Section 4.1 explains the reasons why the model uses academic information alone, and discards socio-demographics information. Other indicators as undergraduate program performance indicators per semester were calculated in order to obtain additional information to compare the results gathered from the prediction. V3 aNfutemrbfaerilionfgttimheesseocfotnhdirtdimenerollment in a subject V4 The period in years that the student takes to return
to study V5 GPA average of all subjects V6 tGaPkAenaivnertahgeecoufrrseunbtjeucntdsergraduate program
GPA average of subjects with number of credits greater V7 than zero, penalized for the number of times
the subjects are taken by the student V8 Ratio of the approved subjects taken by the student V9 Ratio of the failed subjects taken by the student V10 Ratio of the canceled subjects taken by the student
For this preliminary work, we de ne the concept of academic dropout when a
student has not been enrolled in any course for some time since his/her last
registration. Based on ESPOL's internal guidelines and regulations5, the period
is ve years. Therefore, those students have a dropout value equal to one. On the
other hand, students who graduated had a dropout value equals to zero. Also,
students with 90% of their undergraduate degrees completed are assigned with
dropout equal to zero. This percentage based on other studies [
After pre-processing, the process started with the dropout assignment equal
to one or zero, according to the de nition of academic dropout indicated above.
Subsequently, the semester selection was made cumulatively. This means,
selecting the academic history of all students in their rst semester and
entering this data into the model; then, selecting the students in their rst and
second semesters and entering this data into the model; and so on, until the
fth semester, which represents two and a half years of studies. After the fth
semester, the evaluation of all students who had more than ve semesters in
their undergraduate programs was carried out. In Ecuador, according to [
Every group of semesters was used to calculate the academic performance indicators, and consequently, de ne the set of data that was taken for the training and testing. The training data was built with the group of students whose dropout was previously de ned, and the algorithm was trained; while, for testing data, the set of students who did not have a dropout de ned was considered.
In general, high probability values mean that the student has a high probability of dropping out of the undergraduate program, while low values indicate a high probability of nishing the degree. Fig. 1 depicts the steps that were carried out to produce meaningful information. 4 4.1
For data acquired from the bachelor's degree in Industry Engineering, we applied a correlation matrix to analyze how the di erent variables were a ected within the proposed model. As consequence, the socio-demographic variables were discarded, such as gender, married status, school type, and work status, because this variables had a correlation coe cient lower than 0.1; therefore, were
not included into the model. Additionally, we use the RFC with the following parameters: number of trees equal to 300, depth of trees equal to 6, minimum number of samples required to split equal to 6, minimum number of samples required to be at a leaf node equal to 7. These parameters were chosen based on the results in accuracy when the model was applied to our testing set with an accuracy of more than 95%.
In order to evaluate the e ectiveness of the proposed algorithm, accuracy,
area under curve (AUC), F1-score, recall and precision were used as evaluation
criteria. Accuracy is the proportion of correct prediction of dropout. Precision
is the proportion of dropout learners predicted correctly by the classi er. Recall
is the proportion of dropout learners predicted correctly by the classi er in all
real dropout learners. F1-score is the harmonic mean of precision and recall [
A total of 65 undergraduate programs were predicted within the model. The model predicted the dropout degree by degree, and as explained before, every group of semesters was used to calculate the academic performance indicators, and consequently, de ne the set of data that was taken for the training and testing. In this way, in parallel to testing our proposal, we proved that sociodemographic data are unnecessary for dropout prediction, and eliminate typical biases from the learning process. A summary of the ve evaluation criteria of the model for the 10 undergraduate programs with the largest of number of students enrolled are shown in Table 3. Our proposed model was based on academic data alone and the analysis of the academic semesters of students throughout the student's undergraduate program. The initial results indicated a strong signal in the use of data purely academic for predicting student dropout, with an accuracy of more than 95%. In addition, the experimental results proved the e ectiveness and scalability of our algorithm due to the di erent numbers of students enrolled.
Unfortunately, these results can not be compared with similar studies that
have only used academic data as well because of the following reasons. First, the
prediction model focuses on the rst semesters only [
We also acknowledge the limitation that prevents us from generating a
better model. These limitations arise from the data gathered. Although, it is true
we only focus on the use of academic data for the model, it should be noted
the importance of the students' nancial status, because this factor is one of
the motivation behind students' decisions to stop their studies [
Dropout prediction is an essential prerequisite to make interventions of at-risk students. To address this issue, we proposed a predictive model, which used academic data alone and analyzed the information semester by semester to obtain more reliable predictions. Besides, it was able to identify the earliest moment of dropping out of a student. To the best of our knowledge, this is the rst proposal of a predictive model able to predict even for students who have just nished their rst semester.
Our model was also tested with real data of ESPOL and was also applied to other large data sets of the di erent undergraduate programs. Experimental results showed that our model has better results than existing proposals in terms of input features and accuracy. Therefore, we have proven that our model allows to predict with su ciently high accuracy but, at the same time, can predict the intention of dropping out early enough, so that some corrective actions can be attempted.
Additionally, the fact that the variables that worked are speci cally mentioned can help other universities to replicate this model or adapt it to their needs.
Our future work aims to make technical improvements to our prediction model and use other machine learning algorithms to compare them as other researches do. Furthermore, we plan to use this model in postgraduate programs and in academic data sets from other types of universities where attrition rates tend to be much higher. Additionally, we are currently working on a visualization prototype to integrate the results of the predictive model into the Academic Counseling System. Using visualization techniques with explanatory models, allows counselors to have the opportunity to identify those students who have the possibility of an early dropout, and be able to adapt a recommendation strategy to minimize the risk of failure of a student. Lastly, we plan to analyze and visualize which input variables have the most substantial in uence on predicting dropout.
Work partially funded by the LALA project (grant no.586120-EPP-1-2017-1-ESEPPKA2-CBHE-JP). This project has been funded with support from the European Commission. This work has also been partially funded by the Madrid Regional Government through the e-Madrid-CM Project under Grant S2018/TCS4307, a project which is co-funded by the European Structural Funds (FSE and FEDER). This work has also been partially funded by the FEDER/Ministerio de Ciencia, Innovacion y Universidades{Agencia Estatal de Investigacion, through the Smartlet Project under Grant TIN2017-85179-C3-1-R. This publication reects only the views of the authors, and the funders cannot be held responsible for any use which may be made of the information contained therein.