Online education often deals with the problem related to the high students' dropout rate during a course in many areas. There is huge amount of historical data about students in online courses. Hence, a relevant problem on this context is to examine those data, aiming at nding e ective mechanisms to understand student pro les, identifying those students with characteristics to drop out at early stage in the course. In this paper, we address this problem by proposing predictive models to provide educational managers with the duty to identify students whom are in the dropout bound. Four classi cation algorithms with di erent classi cation methods were used during the evaluation, in order to nd the model with the highest accuracy in prediction the pro le of dropouts students. Data for model generation were obtained from two data sources available from University. The results showed the model generated by using SVM algorithm as the most accurate among those selected, with 92.03% of accuracy.
Every year, the registration marks in E-learning modality
has increased considerably, in 2013, 15.733 courses were
offered, in E-learning or semi-presence modality.
Furthermore, the institutions are very optimistic, 82% of researched
places, believe that the amount of registration marks will
have a considerable expansion in 2015 [
Beyond the hard task on identifying the students who can have possible risk of dropping out, the same dropout also brings a huge damage to current nancial and social resources. Thus, the society also loses when they are poorly managed, once the student lls the vacancy but he gives up the course before the end.
Online education often deals with the problem related to the
high students' dropout rate during a course in many areas.
There is huge amount of historical data about students in
online courses. Hence, a relevant problem on this context is to
examine those data, aiming at nding e ective mechanisms
to understand student pro les, identifying those students
with characteristics to drop out at early stage in the course.
In this paper, we address this problem by proposing
predictive models to provide educational managers with the duty
of identifying students who are in the dropout bound. This
predictive model took in consideration academic elements
related with their performance at the initial disciplines of
the course. Data from System Information course at
Federal University of Alagoas (UFAL) were used to build this
model, which uses a very known LMS, called Moodle.
A tool to support the pre-processing phase was used in order
to prepare data for application of Data Mining algorithms.
The Pentaho Data Integration [
Several studies have been conducted in order to nd out the
reasons of high dropout indices in online courses. Among
them, Xenos [
A model for predicting students' performance levels is
proposed by Erkan Er [
Dekker [
In summary, several studies investigating the application of EDM techniques to predict and identify students who are at risk dropout. However, those works share similarities: (i) identify and compare algorithm performance in order to nd the most relevant EDM techniques to solve the problem or (ii) identify the relevant attributes associated with the problem. Some works use past time-invariant student records (demographic and pre-university student data). In this study, contribution to those presented in this section, makes the junction between two di erent systems, gathering a larger number of attributes, variables and time invariant. Besides being concerned with the identi cation and comparison of algorithms, identify the attributes of great relevance and solve the problem the predict in more antecedence the likely to dropout students.
This subsection presents an environment for construction for a predictive model for supporting educators in the task of identifying prospective students with dropout pro les in online courses. The environment is depicted in Figure 1. The proposed environment in this work is composed by three layers: Data source, Model development and Model. The data sources are located in the rst layer. Data about all students enrolled at the University are stored in two data sources: The rst one contains students' personal data, for example: age, gender, income, marital status and grades from the academic control system used by the University. Information related with frequency of access, participation, use of the tools available, and grades of students related the activities proposed within the environment are kept in second data source.
In the second layer, the pre-processing [
Data extraction: The extraction phase establishes the connection with the data source and performs the extraction of the data.
Data cleaning: This routine tries to ll missing values, smooth out noise while identifying outliers, and correct data inconsistencies.
Data transformation: In this step, data are transformed and consolidated into appropriate forms for mining by performing summary or aggregation operations. Sometimes, data transformation and consolidation are performed before the data selection process, particularly in the case of data warehousing. Data reduction may also be performed to obtain a smaller representation of the original data without sacri cing its integrity.
Data selection: In this step, relevant data to the analysis task are retrieved from the database.
Choice of algorithm: An algorithm to respond with quality in terms of accuracy, which has students elusive pro le, was considered the algorithm that best applies to the model. Finally, the last layer is the presentation of the model. This layer is able to post-processing the result obtained in the lower layer and presenting it to the end-user of a most understandable way.
The main objective of this present research is to build a
predictive model for supporting educators in the hard task
of identifying prospective students with dropout pro les in
online courses, using Educational Data Mining (EDM)
techniques [
The research question that we would like to answer is: RQ.Is our predictive model able to early identify the students with dropout risk? In order to answer this question, EDM techniques with four di erent classi cation methods were used, aiming to get a predictive model which answers us with quality in precise ways which students have a dropout pro le, taking in consideration only data about the initial disciplines of a speci ed course.
The Federal University of Alagoas o ers graduation courses,
postgraduate courses and E-learning courses. In the on line
courses, there are more than 1800 registered students[
In this work to predict student dropouts, four machine
learning algorithms were used, using di erent classi cation
methods. The methods used were: simple probabilistic classi er
based on the application of Bayes' theorem, decision tree,
support vector's machine and multilayer neural network.
These techniques have been successfully applied to solve
various classi cation problems and function in two phases: (i)
training and (ii) testing phase. During the training phase
each technique is presented with a set of example data pairs
(X, Y), where X represents the input and Y the respective
output of each pair [
The Pentaho Data Integration [
For execution of selected classi cation algorithms (see
Section 4.2.2), the data mining tool Weka was selected. Such
algorithms are implemented on Weka software as NaiveBayes
(NB), J48 (AD), SMO (SVM), MultilayerPerceptron (RN)
[
Some features were taken in consideration for Weka [
The evaluation of experiment was executed on HP Probook 2.6 GHz Core-I5 with 8Gb of memory, running Windows 8.1.
Real-world data tend to be dirty, incomplete, and
inconsistent. Data preprocessing techniques can improve data
quality, thereby helping to improve the accuracy and e ciency
of the subsequent mining process [
Currently, the data is spread in two main data sources:
LMS Moodle, utilized by the University as assistance on
Elearning teaching, including data which show the access
frequency, student's participation using the available tools, as
well as the student's success level related to proposed
activities. Meanwhile, student's personal les as age, sex, marital
status, salary and disciplines grades are kept in the
Academic Control System (ACS), which is a Software designed
to keep the academic control of the whole University [
Careful integration can help reduce and avoid redundancies
and inconsistencies in the resulting data set. This can help
improve the accuracy and speed of the data mining
process [
To maintain the integrity and reliability between data, a mandatory attribute, with unique value and present between in both data sources, was chosen. Thus, the CPF attribute was chosen to make data uni cation between the two selected data sources, once it permits the unique identi cation among selected students.
In order to facilitate algorithms execution and comprehension of results,predicting the dropout in an early stage of the study. In order to achieve a high rate of accuracy and minimum of false negatives, i.e. students that have not been recognized to be in danger of dropout. Some attributes were transformed, as we can seen below:
The corresponding attributes related with discipline grades were discretized in a ve-group-value (A,B,C,D e E), depending on the discipline's achieved grades. The student with a grade higher or equal 9, was allocated for \A" group. Those ones who had their grades between 8,99 and 7 were allocated for \B" group. the \C" students are those that had a grade between 6,99 and 5, and those who had grades under 5,99 stayed at \D" group and nally those that doesn't have a grade associated were allocated in "E" group.
Every student was labelled as approved or reproved based on the situation informed by the academics registers. The nal score of each discipline is composed by two tests, if the student did not succeed in obtaining the minimum average, he will be leaded to the nal reassessment and nal test.
In the \City" attribute, some inconsistencies were found, where di erent data about the same city were registered in database. For instance, the instances of Ouro Branco and Ouro Branco/AL are related to same city. This problem was totally solved, with application of techniques for grouping attributes.
The attribute \age" had to be calculated. For this, the student's birth date, registered in database, was taken in consideration.
When all the attributes were used the accuracy was low. That is why we utilized feature selection methods to reduce the dimensionality of the student data extracted from dataset. We improved the pre-processing method the data. In order to preserve reliability of attributes for classi cation after the reduction. We use InfoGainAttributeEval algorithm that builds a rank of the best attributes considering the extent of information gain based on the concept of entropy.
After this procedure, we reduced the set of attributes from 17 to 13 most relevant. The list of the re ned set of attributes in relevance ordercan be found in Table 1. Taking in consideration that the main objective is to predict student's nal situation with the earlier advance as possible inside the given discipline, to this study we will only use data until the moment of the rst test.
The Figure 2 presents all the executed stages, during the preprocessing phase, in order to generate a compatible le with the mining software.
The k-fold method was applied to make a assessment the
model generalization capacity, with k=10 (10-fold cross
validation). The cross validation method, consists in splitting
of the model in k subgroups mutually exclusive and with the
same size, from these subgroups, one subgroup is selected for
test and the remaining k-1's are utilized for training. The
average error rate of each training subgroup can be used as
an estimate of the classi er's error rate. When Weka
implements the cross validation, it trains the classi er k times to
calculate the average error rate and nally, leads the build
classi er back utilizing the model as a training group. Thus,
the average error rate provides a better solution in terms of
classi er's error accuracy reliability [
In order to get the best results of the algorithms without losing generalization, some parameters of SVM algorithms were adjusted.
The rst parameter was set the parameter \C". This
parameter is for the soft margin cost function, which controls
the in uence of each individual support vector; this process
involves trading error penalty for stability [
The default kernel used by Weka tool is the polynomial we
changed to the Gaussian setting the parameters Gamma.
Gamma is the free parameter of the Gaussian radial basis
function [
After several adjustments to the values of the two parameters mentioned above, which showed the best results in term of accuracy and lower false positive rate, was C = 9.0 and Gamma = 0.06 parameter.
For comparison of results related to selected algorithms, we
used Weka Experiment Environment (WEE). The WEE
allows the selection of one or more algorithms available in the
tool as well as analyse the results, in order to identify, if a
classi er is, statistically, better than the other. In this
experiment, the cross validation method, with the parameter
\k=10" [
In this section, the results of the experiment, described in Section 4, are analyzed.
The WEE tool calculated the average accuracy of each
classi er. Table 2 shows the result of each algorithms
execution. The accuracy represents the percentage of the test
group instance which are correctly classi ed by the model
built during training phases. If the built model has a high
accuracy, the classi er is treated as e cient and can be put
into production [
As we can see on table 2 the algorithm SVM presented a low false positive rate and better accuracy. Therefore, only the best algorithm was considered to our solution. The Naive Bayes classi er had the worst result in terms of accuracy and a high false positive rate. The other ones had an error average of 8%, and then, we end up with 8% of the students with dropout risk not so correctly classi ed.
As can be seen in table 2, in our experiment, the SVM
algorithm obtained 92% of accuracy. According to Han J. et
al. [
We often need compare di erent learning schemes on the
same problem to see which is the better one to use. This
is a job for a statistical device known as the t-test, or
Student's t-test. A more sensitive version of the t-test known
as a paired t-test it was used. [
The experiment has taken in consideration data from the Information System course and the Data Structure Algorithm discipline. However, the aforementioned discipline was chosen, based on its importance in the context of Information System course.
Understand the reasons behind the dropout in E-learning education and identify in which aspects can be improved is a challenge to the E-learning. One factor, which has been pointed as in uencer of students' dropout, is the academic element related with their performance at the initial disciplines of the course.
Comparing the results among the four algorithms, we can verify that the accuracy oscillates around 85.5 to 92.03%. Furthermore, a classi er which has a high error rate to false This research has addressed dropout problem by proposing predictive models to provide educational managers with the duty to identify students whom are in the dropout bound. The adopted approach allowed us to perform predictions at an initial discipline phase. The preliminaries results has shown that prediction model to identify students with dropout pro les is feasible. These predictions can be very useful to educators, supporting them in developing special activities for these potential students, during the teaching-learning process.
As an immediate future work, some outstanding points still should be regarded to the study's improvement, as apply the same model in di erent institution databases with di erent teaching methods and courses, including new factors related to dropout as: professional, vocational and family data, execute some settings in algorithms' parameters in order to have the best achievements. Furthermore, a integrated software to LMS, to provide this feedback to educators, will be developed using this built model.