In this paper we describe a study where students tagged learning objects created by their professors. The study shows that the student tags extend the professors' view of the contents of the learning objects and add a wider context for interpreting the content of the learning objects. During interviews conducted at the end of the study, the professors reported that the students' tags represented a form of feedback that would help them recognize discrepancies between the learning objects' intended purpose and the perceived purpose.
According to Kay and Knaack [
Researchers currently have different and disagreeing views on the quality of
userprovided tags and folksonomies (i.e., a vocabulary that has emerged organically
as a result of the end-user tagging activities). Guy and Tonkin [
Al-Khalifa and Davis [
The idea that what the teachers intend to communicate is received by students
is much criticized. In radical constructivism, it is claimed that it is less than
likely that the received message is congruent with the sent (Qvortrup [
In this project we studied two groups of master students at Gjøvik University College. Each group consisted of approximately ten master students. Group 1 was a group of first year students on the Master of Media Technology program participating in a course on media data coding and compression. Group 2 was a group of second year students on the same program who had previously participated in a course on semantic web. The courses were taught by two different professors – one for each course.
Both courses were offered as blended learning in which regular lectures were recorded. The recorded lectures along with lecture notes in PDF were used to produce LOs stored in an LMS. The professors assigned keywords to each learning object as content-descriptive metadata.
The project was divided in two main parts. Firstly, the students used a webbased application to tag LOs blindly (i.e., students did not get to see other students’ tags). Secondly, semi-structured interviews were conducted in which the two professors and some of the students were interviewed individually.
The key characteristics of the two groups and the generated metadata can be summarized as follows: The individual student in group 1 generated 3.5 tags per LO on average, while the average in group two was 3.3. At the same time, the professor teaching group 1 assigned 14.6 per LO on average while the professor for group 2 assigned only 6.9 keywords per LO on average. On average, 1.3 tags generated by the individual student in group 1 also appeared as keywords assigned by the professor. The corresponding value for group 2 was 1.35. The overlap between the sets of tags generated by the students and the sets of keywords assigned by the professor per LO is further illustrated in Fig. 1. The upper part of the bars shown in brighter colors exposes the number of keywords assigned by the professors that did not appear in the sets of tags generated by the groups of students. The lower part of the bars shown in darker colors exposes the number of the number of tags generated by the groups of students that did not appear in the sets of keywords assigned by the professors. The middle part shown in the darkest color illustrates the amount of overlap between student-generated tags and professor-assigned keywords.
Finally, Figure 2 illustrates the level of agreement among students with regards to the tags. The ten most popular tags are enumerated along the horizontal axis (tag 1 being the most popular for the given LO). As can be seen on the figure, student tags mostly differ from the keywords chosen by the professors. Our study therefore indicates that the students’ interpretation of content of the LO is different from the professors’. It thus seems like student-generated tags would be useful as a complementary type of metadata to professor-assigned keywords. 3
Value of Student Tagging: Beyond Content Descriptive Metadata The two professors involved in the study and six of the students from group 1 were interviewed at the end of the study. The purpose of the interviews was to investigate whether student tagging added value beyond producing content descriptive metadata.
The students all agreed that the keywords provided by the professors were useful for them in interpreting the content of the LO. They even requested that student tagging of learning object should be introduced in all the courses they were signed up for. They found the keywords especially helpful in finding what the key aspects of the LO were. A majority of the students also considered it useful to be able to view fellow students’ tags. They also thought it would be useful if they had the opportunity to view students’ tags from earlier years of the course – even though no such tags existed in the demo system.
Both professors agreed that student-defined tags were quite similar to their own keywords although they were not the same. Most importantly, however, the participating professors emphasized that the tags represented a sort of feedback that would help them understand how well the students were able to grasp the contents of the LO.
The two participating professors both stated that the feedback from the students would have a real impact. In some cases, they found that the students had chosen tags that they would like to include as one of the professor-assigned keywords in the future. In other cases, the feedback indicated that the students had missed some important messages or misinterpreted the LO. In these cases, the professors said they would either modify the original lecture and accompanying LO, or they would repeat these issues again in future lectures to help the students achieve a correct and deeper understanding.
One striking observation is that none of the professors considered removing even one single keyword from any of the LOs even if the students did not use it as one of their tags. The professors saw the student-generated tags as a possible extension to the set of keywords they assigned – not as a potential replacement.
Although the study of student tagging of LOs and the follow-up interviews were rather small, they indicate that content-descriptive metadata in the form of professor-assigned keywords and student-generated tags add value beyond supporting searchability and reusability. The students consider professor-assigned keywords as a help in interpreting the contents of the LOs and the professors see student-generated tags as valuable feedback from the students both when it comes to the quality of the LO and the quality of the accompanying set of metadata. Our study was rather small, involving some 20 students, two professors, and 16 LOs. Still, the study gives some interesting indications on the usefulness of student tagging and interesting paths for further work.
Our study shows that there is some overlap between the tags that students create for LOs and the keywords the professors assign to the same LOs. There is, however, also a significant difference between the two. This difference may provide a wider context for interpreting the content and context of the LO. Our study also shows that student tags may be utilized in the quality management of the LOs. The professors could recognize discrepancies and patterns of differences between the student-generated tags and the original set of professor-generated keywords that might call for quick or dramatic alterations of LOs. As a means for quality improvement this dimension goes far beyond the quest for retrieval and reusability question to include issues of feedback, evaluation and might improve the level of collective engagement and learning retrieval.
This project is part of an ongoing process in developing a Learning Object Repository at Gjøvik University College that will provide student tagging capabilities.