When bachelor and master degrees were introduced in the Italian system, the Engineering Faculty of Universita Politecnica delle Marche dropped mandatory prerequisites between modules from its degrees. Since then, we periodically witness intense debates among professors, who think this led to unstructured learning practices with consequent performance issues, and students who are concerned of an overly constrained course of study. This paper brie y describes an ex-post analysis of the carriers of students enrolled in the bachelor degree in Computer and Automation Engineering based on process mining techniques, the resulting evidence about typical patterns followed by students, and sketches possible actions that can be put in practice to support students without constraining their course of study.
Digital technologies in learning received much attention in the last years. In
particular, educational data mining is a research eld by itself [
In this paper we brie y describe the application of process mining techniques
to the carriers of students enrolled in the bachelor degree in Computer and
Automation Engineering, with the aim to understand the typical paths followed
by students in taking exams, and their relations with nal outcomes. The work
is motivated by the intense debate among professors and students that arise
from time to time on mandatory prerequisites. Mandatory prerequisites among
? This work has been partially funded by the ITEA2 project M2MGrid (No. 13011).
modules have been dropped from courses of the Faculty of Engineering of the
Universita Politecnica delle Marche. While somebody would like to re-introduce
them in order to limit unstructured learning practices, students are concerned
of an overly constrained course of study. The work is part of the monitoring
activities in charge of the Faculty committee named \Commissione Paritetica"
[
The rest of the paper is organized as follows: in Section 2 we provide a brief introduction to Process Mining and techniques used in the paper. Section 3 describes data, their processing and the results of mining activities. Section 4 introduces some re ection and draw future work. 2
Process Mining encompasses a wide range of techniques for the analysis of
business processes from run-time information recorded in event logs [
A process consists of a set of activities that have to be performed by some actors to reach a certain goal. Welfare grant approval is an example of business process, that involves activities like application submission, the analysis of documentation, possibly split in financial check and check for illness conditions performed by two di erent units in parallel, and a nal approval decision. A process schema is a general schema representing the overall control- ow of activities. The main elements of a process schema are activities, and control- ow operators like sequence, parallelization and merging, alternative choices, and loops. Many di erent process modeling formalisms exist, among them Petri Nets is the best known theoretical model. A process instance is a speci c execution of the process. In our example, each single welfare grant application represents a process instance. Process instances are recorded by information systems in event logs in the form of traces, i.e. sequences of events generated by the execution of activities in their strict order of occurrence. The core information recorded in an event log covers the unique id of the event occurred, the associated activity, the time of occurrence and the instance, or case, it belongs to. The name of the actor executing the activity, or its role, can be also reported. Depending on the information system, other information can be recorded, like for instance the data processed by the activity, the case category and so forth. This kind of information is often referred to as data attributes and this kind of logs are also called data annotated logs.
In Process Mining, two major tasks can be recognized:
{ Process Discovery: synthesize a process schema from event logs.
{ Conformance Checking: analyze the event log to check whether it ts a
given process. If the model is a normative or prescriptive (hence a-priori
correct) model, conformance checking is interpreted as a way to measure the
adherence of real process executions to those prescribed or planned, however
it can also be seen as a way to evaluate the accuracy of a model mined by a
Process Discovery technique in capturing the content of the event log.
The basic idea of Process Discovery techniques is to exploit event logs to
reconstruct causal relations among events that are then used to build a model of
the process. In order to deal with variability and noise in real-life logs, ltering
approaches have been introduced. The infrequent Inductive Miner (iIM)
algorithms [
Conformance Checking is based on the notion of replay: each trace is
simulated on the process model one event at a time. When the current event cannot
be found at a given step, an alignment procedure is adopted to continue the
replay procedure, by either skipping the event in the log or the activity in the
process schema. Besides, a measure of the tness of the trace to the model is
calculated [
The major questions we like to answer can be stated as follows: \Do students take (exams of) modules in the suggested order?" and \Which are best practices and worst practices followed by students?". The order among teaching modules is related to prerequisites suggested in modules programs, so that module A should be taken before module B if A's contents are fundamental or useful to understand B's contents. Of course there is only a partial order among modules, since courses may deal with di erent, unrelated topics. Furthermore, best/worst practices are de ned with respect to careers performances, as expressed by the nal grade mark and length of the period of study.
With these questions in mind, we exploited process mining as a descriptive mining technique to obtain a synthetic view of the paths followed by students from enrollment to bachelor degree, in the form of process schemas. The methodology can be summarized as follows: after some data pre-processing, we obtain a set of traces representing students' careers. Traces are segmented into three disjoint groups depending on the nal grade mark and length of the period of study. Then, for each group, a process model is synthesized in the form of Petri Net by the iIM algorithm. Accuracy of the models is assessed by measuring its tness over the entire set of traces in the group. In the following we provide some details of these steps and of the results obtained. 3.1
Data Selection and Preparation Data are mainly taken from ESSE3, a software suite commonly adopted by Universities to manage students' careers and in particular registration to exam sessions and record of grades. We focused the analysis on the latest educational system, as de ned by D.M. 270/2004, considering students enrolled from Academic year 2009/10 on. This provides us with a set of homogenous curricula. Furthermore only complete and correct curricula are considered, discarding students who have not taken the degree yet, or students with exams con rmed from previous degrees (these exams do not appear as database records, determining oversimpli ed and short curricula). Further cleaning procedures were necessary in order to deal with inconsistencies and null values generated during database population at the time of ESSE3 software adoption. At the end, the careers of 187 graduated students were available for analysis. In order to reduce noise, a further step was to delete from curricula the diverse modules freely chosen by students from other courses of Universita Politecnica delle Marche.
The nal dataset is a table with the following attributes: student ID, module name, exam date, exam mark, duration of the period of study (calculated as the di erence between the nal exam date and the enrollment date, conventionally xed to the 1st of November of the Academic year of enrollment). The dataset can be interpreted as a very simple event log, where each student is a case, and exam passing is the set of events with their timestamp.
Additional data attributes related to exam mark and duration are exploited
to segment the event log. As a matter of fact, a plain application of process
mining techniques to the whole event log did not lead to meaningful process
schemas. This can be expected, since the many optional choices and the lack
of mandatory prerequisites make the course of study very exibly customized
by students according to their needs and contingencies. In order to deal with
this kind of unstructured processes, di erent methodologies have been devised,
among which trace clustering and its process cube variant are pre-processing
techniques that are related to the approach followed in this study. In both
approaches the idea is to generate di erent groups of traces with homogeneous
characteristics such that better models can be obtained for each group. While
in trace clustering similarity is based on the sequence of events inside traces
(inner or structural trace similarity) [
The model is even more interesting when contrasted with that generated from low performing careers, shown in Figure 3. Here, the only regularity is that students choose to take their rst exam among six courses taught in the rst year (Fisica 2 being the one missed in the list), and that in a number of cases the sequence of exam for Automazione Industriale, Progettazione Assistita da Calcolatore, and Laboratorio di Automazione is followed. No other regularity can be appreciated, suggesting the lack of any criteria in exams ow, and enlightening striking di erences in the behaviors of students in the two groups. 4
The one presented is a preliminary analysis of students careers, and conclusions must be taken cautiously, in particular for what concerns the cause-e ect relationship between high performance and structured course of study: is it that following the \right" order of exams leads students to high performance, or is it the other way around, i.e. students with a natural propensity to computer and automation engineering (which in principle lead to high performance somewhat independently from other factors) have a \structured mindset" and thus follow the order proposed in course regulations? Nevertheless, the evidence of such striking di erences between the two groups led us to disseminate results among students, both inside di erent Faculty committees and during the degree introductory activities, and they have been welcomed by students and professors as the rst evidence-based discussion on the advantages of prerequisites. The results also stimulated the board of degree in Computer and Automation Engineering to spread among students information about modules' prerequisites, and to develop actions able to increase awareness on the potential bene ts of a thoughtful schedule of exams.
Several further analyses can be performed to support and re ne results: rst
of all, we started replicating the methodology to other degrees, with
encouraging results. Second, more re ned clustering techniques, based both on ner
categories and on structural trace similarity may allow us to discover more
precise models. Third, we plan to apply di erent mining techniques: on the one
hand we will consider local process mining techniques, tackling unstructuredness
of processes by discovering simpler and more precise models of just parts of the
process, instead of start-to-end models [
We thank the Commissione Paritetica of the Faculty of Engineering, Universita Politecnica delle Marche for his support in the development of the analysis presented in this paper. We also wish to thank Dott. Ing. Matteo Marzioli for the work done in carrying out experiments.