The educational process is constantly changing. On the one hand, traditional educational methods have been modified and, on the other hand, the model of educational transmission has also changed. According to different authors, technological resources, specifically the eLearning platforms, and social interaction are responsible for these changes. Based on these approaches, this article applies Learning Analytics techniques with the aim of analyzing social interaction in blended-learning environments. For this, an exploratory analysis will be carried out in the messages published in the forums with the objective of qualitatively analyzing the students' interaction with the educational platform.
integrate a set of tools for the online teaching-learning process, with the aim of allowing the creation and management of teaching and learning spaces on the Internet, where teachers and the students can interact during their training process. The boom of these platforms has been so great that they are currently used in different educational levels and in different parts of the world. In fact, according to the combination of technological resources with the degree of presence that the student has while learning, we can find three widely accepted teaching modalities: traditional modality, e-learning and blended-learning.
The traditional modality is when the student receives the knowledge in its entirety
inside the classroom, in the same space-time as the teacher without the presence of
technological resources provided by the (ICTs). The e-learning modality is also called
online education modality. In this, the teaching is taught entirely remotely over the
Internet, without the need for students to interact with the platform at the same time or
in the same geographical location as the teacher. This allows the student to advance at
his own pace, making his learning process more flexible and favoring his autonomy
[
These platforms have the capacity to store an innumerable amount of data from the interaction of users (students and teachers) with them and through them. Despite the success and acceptance that these platforms are having, these do not have per se any tool to facilitate the interpretation or analysis of this data. However, these data have aroused the interest of many researchers, thus emerging two specialized fields of study: Learning Analytics (LA) and Educational Data Mining (EDM).
According to the First International Conference on Learning and Knowledge
Analysis (LAK 2011) [
Both areas share different challenges. However, this work has been developed
under the proposal of LA, based on the approach proposed by [
It is also necessary to emphasize that not only the teaching modality has changed,
also the model of transmission of knowledge has been transformed. According to
[
Several authors support the idea that interpersonal interaction provide the
advantages of the second model. This type of interaction is generated when students
react to the content and share concerns, they teach each other learning in a tangible
way when they express with words (through publications on the platform) their own
understanding and assumptions, which allows them to appropriate new skills and
ideas, at all times being focused and deepened by the lateral guide, without it
hindering the development and learning experience of the students [
Starting from the premises that e-Learning platforms and interpersonal interaction
(or also called, social interaction) are of great importance in the new changes that are
arising in the educational process. In this paper we will study the application of LA
techniques in blended-learning environments focused on university studies. To this
end, a methodology will be presented that allows qualitatively analyzing the
interpersonal interaction of students in the e-Learning platform. Our approach tries to take
advantage of the information exchange in the online forums to discover new
knowledge about the students’ way of learning or behave. In this paper, our work
done in [
To do this, the data extracted from the official platform of the University of Vigo belonging to a programming course along three different academic years of Telecommunications Engineering will be used.
This document is structured as follows. The following section (section 2) provides a description of the data set and the methodology. Subsequently, in section 3 we will analyze the students qualitatively, analyzing their messages and publications in the forums of the e-Learning platform. Finally, in section 4 the results will be analyzed. 2
To perform our experiments, we use data from a course related to the programming skills of the third year course of a Bachelor Degree on Telecommunication Engineering. This is a blended course of fourteen weeks between September and January. The dataset was gathered from the official e-Learning platform, Moodle-based, of the university where the subject was taught.
The assessment mechanism of this course is based on three mandatory assignments distributed along the course (from the fourth to the last week) as Fig. 1 shows. The forum is accessible to all students and used to debate about different aspects related to the course (content or administrative issues), answer questions, solve doubts, etc. However, it should be noticed that it does not represent any mandatory activity. The required assignments are divided in two types: • Laboratory: To determine if the student has acquired all the knowledge and skills corresponding to the laboratory practices (3 practices). • Applied: To determine if the student knows how to apply the knowledge of the course to solve problems (2 exams). The Moodle platform stores in its database not only all the information related with the courses (course contents, personal data of students and professors, students’ grades, etc.), but also all the information about the students’ interaction with the platform. In fact, Moodle distinguishes between different types of interactions, which are classified in ten different modules (Assignment, Blog, Choice, Course, Forum, Notes, Resource, Upload, User, and Quiz) as Table 1 shows. For our analysis, we gathered data (73,849 interactions) from three academic years.
We analyze data of 435 students organized from 2014 to 2017, as shown in Table 2. Academic Year 2014/2015 2015/2016 2016/2017
Total As mentioned above, we will analyze two types of interaction: interaction with content and social interaction. Initially, we have divided the events into two groups: (i) actions related with some contents or class notes and (ii) actions related to interpersonal activities. The objective is to find the group of activities that have a higher relation with one of both interaction types.
We consider the following classification: the modules Assignment, Course, Notes, Resource, Upload, and Quiz are related to content. Blog, Choice and Forum are related to interpersonal interaction. Use is outside of both classifications, because it does not provide information related to this.
Modules Blog, Choice and Forum are considered as interpersonal participation because the students can show their own ideas in module Blog. On module Choice they can choose and propose surveys and discussions, and finally, in module Forum, students can participate in a more active way.
Our methodology is divided into three stages. The qualitative analysis begins with data preprocessing, continues with the classification of the messages in three groups and ends with an exploratory analysis of the content of the messages. 3
As mentioned earlier, this analysis is divided into three stages: the first one corresponds to the data preprocessing; the second one is a classification of the messages in three categories (content, code and other); and finally the exploratory analysis of the content of these categories. 3.1
It is necessary to prepare and transform the gathered information to classify the messages. Initially, a corpus of specific content has been created for the experiment, extracting the main words (topic words) from 12 pdf files: teaching material (4 pdf files), educational resources (3 pdf files), notes (3 pdf files), slides (2 power point files), three practices (3 files) and references in the presentations of the class (2 pdf files). All information is available to any student enrolled in this subject and with access to the official e-Learning platform of the university. From these documents, we have obtained a total of 15,704 words. This set is latter reduced to a corpus of 587 words, after removing stopwords, carrying out a lemmatization and extracting the topic words. This corpus will be called “Content Corpus”.
As it was previously commented, the content of the subject is related to computing, especially to two programming technologies: Java and HTML. For this reason, the use of programming codes is very frequent. Therefore, we use a second corpus that will be called “Code Corpus”, created by RANKS NL that contains the top words of all programming languages.
RANKS NL [
As a summary, the two corpus that we will use to classify the messages are: ─ Content Corpus: created by the extraction of the main words (topic words) of the teaching material, educational resources, notes, slides, practices and references available in the e-Learning platform. It is composed of 587 words. ─ Code Corpus: this corpus will serve to classify messages that contain programming codes and it is based on the corpus armed by RANKS NL. It is composed of 2,500 words. 3.2
( | 1 = , 2 = , = ⋯ ) = ( ) ( 1 = , 2 = , = ⋯ ) ( 1 = , 2 = , = ⋯ ) (1) Where ( ) is the probability of belonging to the specific corpus (Code or Content); is the identifier of word; represents if it belongs to the corpus; and if it does not belong to it.
Our interest is the relative probabilities of the messages being a code message or content message. In other words, the exact value of the probability is not important because the classification will be assigned according to the highest percentage of belonging to any of the corpus. Therefore, we can factor out any terms that are constant, namely the denominator of the above equation is a constant because it depends on the total number of messages (from both types – content and code -). For this reason, the numerator of equation (1) can then be written as: ( ) (| 1 = , 2 = , = ⋯ ) = ( ) ( 1 = | ) ( 2 = | ) ( = ⋯ ) (2)
Each message will follow the same process. First, divide each message word by word. Second, stopwords are removed and lemmatization is executed. Third, the messages are classified using the Naïve Bayes classifier if the message has 33% membership in the code or content corpus. This percentage is recommended by RANKS NL, creator of the code corpus. This percentage is recommended when using this classifier for detecting spam in emails. Finally, we obtain three classifications. 1. Code messages: messages that 33% of its content belongs to the code corpus. 2. Content messages: respecting the same percentage, these are messages that 33% of its content belongs to the message corpus. 3. Other messages: the rest of messages that do not belong to any of the two previous classifications.
The procedure considers that the same message can contain words that belong to the two corpus. As shown in Fig. 2, first, it calculates the probability of each word of belonging to the code corpus and get the value of the probability. Then, it calculates the belonging to the content corpus, word by word, until exceeding the percentage of belonging to the code corpus or finishing by analyzing all the words of the message. Finally, the classification with the highest percentage is assigned, as long as it exceeds 33%. To check the classification, an expert in the programming area reviewed each message to classify them manually in the three identified groups, obtaining that only 7.2% of messages correspond to another category different from the one assigned by the Naïve Bayes classifier. With this information, we have calculated other interesting measures of accuracy like precision, recall and F score, summarized in Table 3.
Code Content Other As previously mentioned, we have decided to use the threshold of 33% to decide if a message would belong to the Content category, keeping the recommendation by RANKS. This value supported good result. However, we decided to perform some tests changing this threshold. After doing an exhaustive work, we detected that our results were optimized using a higher threshold of 52%: our error decreased to 5.7% and the total recall increased from 92.8% to 94.3%. This encouraged us to check what happened if the threshold of the Code category was also altered. After the same analysis, we optimized our results increasing this threshold to 35%.
Finally, Table 4 summarizes the distribution of messages per academic year: content messages are clearly the most frequently exchanges and code messages the less frequent. Since the percentages are quite similar in the three academic years, we have decided to focus the analysis in a single data set formed by the information of the three academic years (2014-15, 2015-16, and 2016-17). It is important to emphasize that we will analyze the content of the three classifications by performing an exploratory analysis. We will search the most frequently used words in the previously classified messages. This will allow us to know which are the top words and if there is a relationship between the classifications. Moreover, the next step is to plot networks of these co-occurring words, so these relationships are clearly displayed, as Fig. 3 shows. Considering that n represents the co-ocurrence of the words in Fig. 3, it was found that several words are indistinctly used in the Code category and in the Content category, such as entity, permissions, firewalls, browser, or route. Besides, there are words that appear in the Other category and in the Content category, such as exam, results, deliver, attachment or correction. Finally, there are words that appear in the three corpus: php, tomcat, query, server, etc. We can see that the words of the Other category are referring to the course administration, therefore this classification will be named as such.
Additionally, Fig. 4 shows the distribution of the messages of each category along the academic term, showing the temporal evolution of the exchanges messages. As Fig. 4 shows, we have 3 peaks (blue circles) of code messages, the first corresponds to the delivery of the first practice, the second to the revision of the second practice and the third to the delivery of the third practice. We also have 2 peaks (orange circles) in content messages corresponding to the session prior to the exams. Regarding the messages of the course administration, there is no pattern depending on the academic organization of the course.
To finish our exploratory analysis, it is important to know who initiates the posts: a teacher or a student. It was obtained that 71% of code conversations, 63% of content conversations and 55% of information conversations are started by students in each group.
Knowing that the student starts mainly the posts, the next point would be to know which messages respond most frequently, those sent by the teacher or by the other students. For this reason, we have calculated the percentage of the student's response to conversations initiated by one of their classmates, knowing that 89% of the messages sent by another student is answered. Only 11% is initially answered by the teacher.
Additionally, we have analyzed the themes and topics of the exchanged messages
with a program called DepPattern [
As a brief summary, two corpus were used in the analysis. The first (content corpus)
was created specifically with the academic content of the course, and the second
(code corpus) was taken from the one created by RANKS NL. Applying the Naïve
Bayes classifier and these two corpus, we have obtained three classes or categories:
code, content and course administration. The first two are composed of messages
with a high percentage of words related to code and course content, respectively.
Those messages which are not classified in these two categories go directly to the
third one, whose name was decided after checking that all the messages included
reference to course administration (questions and/or information about the exams,
revision dates, etc.). We chose this classifier because its structure is fixed and does
not depend on the data, it follows a generative or discriminative criterion. Like the
other Bayesian classifiers, the obtaining of the parameters is based on the maximum
likelihood or a posteriori maximum estimations [
The analysis of the messages can give feedback from the students to the teachers, remarking those topics that are considered more interesting or those in which doubts usually arise. Having a direct feedback from the student is important to be able to take more concrete actions and improve the academic course, for example, reviewing certain concepts, solving concerns, repeating dates or instructions and, consequently, supporting the student in his acquisition of knowledge from a less formal environment (forums) than the classroom. Forums can encourage shy or absent students to interact with other students and, in the same way, they can encourage the more participatory students continue to reinforce their interaction. The proposed methodology can bring improvements inside and outside the classroom. It marks an important guide in the educational process, by facilitating the content analysis of the messages in the forum, identifying the main topics of discussion, the topics that more generate doubts, the answers and the recommendations that are given between students. This allows to analyze valuable data of student behavior, with which learning models and learning analysis could be applied to improve the quality of education and the participation of students.
As a future line, on the one hand, it would be interesting to integrate the classification of students to analyze the content of the messages for each profile proposed by the classification and, on the other hand, to use these messages to try to profile the student who sent them. This methodology could be an initial step to integrate a content recommender system into the eLearning platform.
This work is funded by the European Regional Development Fund (ERDF) and the Galician Regional Government under the agreement of funding the Atlantic Research Center for Information and Communication Technologies (AtlantTIC); the Spanish Ministry of Economy and Competitiveness under the National Science Program (TEC2014-54335-C4-3-R, TEC2017-84197-C4-2-R).