In this paper we present AMATI, (Another Massive Audience Teaching Instrument) to enhance knowledge transfer between a lecturer and their student audience as well as encourage the communication among students for both on line and on campus courses. AMATI provides a non-distractive environment for knowledge transfer and communication by using context aware teaching information for a specific classroom setup such as a lecture or exercise. Examples of teaching information are clarifications, context specific information provided at a certain time, answers to questions raised by individual students and the visualisation of the student mood. AMATI allows to use this information not only for short term knowledge transfer but also as a persistent knowledge base for students to prepare for their exams as well as for instructors to tailor future lectures. AMATI has been tested during a Massive On Campus Course (MOCC) in a Introduction to Software Engineering course attended by over one thousand students, separated in different locations such as lecture halls and home video streaming. First experiences have been elaborated providing a questionnaire to the students at the end of the lecture period.
The term of context aware software usually finds its
origin in ubiquitous computing
One of the problems with context aware teaching, however, is the volatility of the context which makes it difficult for instructors and students to recover important information after a lecture has ended. E.g. ”Does that have something to do with the model view controller pattern?” AMATI is a framework that allows to record, store and retrieve knowledge and its teaching context associated with the lectures. In particular, this allows the lecturer to reuse this information when designing new lectures or tailoring existing ones and allows students to prepare for exams. The paper is organised as follows. Section 2 describes a case study of a large software engineering lecture with more than 1000 students where we used the AMATI framework. In Section 3 we describe the AMATI framework in more detail, in particular how to deal with context aware teaching situations, storing context specific knowledge and make it accessible for lecture tailoring and exam preparation. Section 4 presents first results of the case study, evaluated through a questionnaire. Results show a large increase of student interactions using synchronous communication channels in the AMATI based lecture. However the volatile nature of these context aware teaching information indicates, that if the associated context is lost also knowledge is lost. Section 6 propose enhancements of the AMATI framework for the upcoming summer term 2017. 2
The department for Applied Software Engineering at TUM provided the instructor for the course Introduction to Software Engineering (EIST) in the summer term 2016. The class was taken by more than 1000 students from 5 different majors. Usually this course is composed of multiple theory classes and practical exercise sessions.
In the summer term 2016 the department of Applied Software Engineering focused on a new teaching methodology, consisting of a single class, combining exercises and theory. In addition to this new approach the high number of enrolled students for this course induced the separation of the attendants into multiple classrooms. This setup led to the introduction of a lecture recording infrastructure to stream the content online using a livestream service1. After post production, lecture recordings have also been offered as video files for students to prepare for upcoming lectures. Table 1 and 2 list all lectures recorded with their according views as well as live views during the lecture. This indicates that students preferred to watch the session on thursday mornings either via livestream or later as a recorded view instead of joining the class in person.
In the beginning of the course we started with the typical communication flow as been seen in traditional lectures. Class participants could ask their questions by using a raise of hand. A microphone has been provided to the according student to state their question in front of the class directly to the instructor so all students could listen the given question properly.
The introduced blend of theory and practice is a
form of experiential learning
Before the course started Moodle2 was setup to support lecture content delivery and exercise material as well as for exercise submissions. Also a Moodle forum was created to allow students to communicate with the teaching staff. In Table 4 the number of questions asked in Moodle during the class are listed and have been sorted by topics.
During the course the in class communication for content specific questions was refined by introducing a private chat tool named Slack3 using synchronous communication. This enabled synchronous information exchange between the different classrooms. At first, only tutors were granted access to share questions stated from students in the different locations to the teaching assistant. Students participating in the lecture using the provided live stream from different locations still were not able to ask questions to the teaching staff.
Later in the course the communication was refined again allowing all attendants to use the chat tool by registering through their institutional credentials. Two chat channels have been offered. One for general discussion which has not been monitored by tutors nor the teaching assistant and one question channel for content specific questions. Questions asked using the question channel have been answered by tutors and the teaching assistant. If the teaching assistant felt the need to answer a relevant question for all participants of the class the instructor was informed for further processing. This can be seen as a new form of team teaching, whereas the team consists of tutors, the teaching assistant as well as the instructor as shown in Figure 3. AMATI establishes a synchronous communication between instructors, teaching assistants, tutors and the student corpus by implementing different features. These features can be used to increase interaction using a typical class room setup, whereas the framework is time depended, but not location dependent.
AMATI can be seen as an extension to an existing classroom setup. It allows the lecturer to interactively ask questions to the student audience, directly evaluate the aggregated results of the student corpus and also present the results to the audience if desired. Furthermore the instructor can monitor the student mood of the whole audience. The participating students on the other hand can use AMATI to ask questions to the teaching assistant which takes the role of a moderator to answer questions directly or delegate questions to the instructor to be discussed in front of the whole class. The features described above will be highlighted in the following subsection in more detail. 3.2
The AMATI framework consists of three major features. The first feature is the Chat Board, which is intended for students to ask questions directly to the teaching personal, storing the question, the answer provided, as well as the context information when this question has been asked and which content has been provided at the time of the question. The second feature is the Live Quiz feature, in which the teaching staff is able to probe the knowledge of the existing audience by executing a poll mechanism with aggregates the results and either present it to the student audience or just to the teaching staff. The final and last feature is a Mood Chart which provides the teaching staff with information about the actual mood inside the student corpus. The following subsections will briefly describe each feature in more detail.
The Chat Board feature allows students to ask questions to the teaching staff in a synchronous way as seen in Figure 4. Students not stating a question are still able to participate by upvoting a stated questions to increase the chance of a fast response from the teaching staff. The teaching staff is able to answer the questions and receive votes for it, to verify if the given answer has been understood by the class. 3.2.2
The Live Quiz feature allows the teaching staff to create instant live polls to determine the level of understanding of a certain subject during class.
The teaching staff can launch a quiz, see the number of participants and display the results either for the teaching staff only, or for the full class. While the questions and quiz mechanisms collect student-to-student interaction they only represent active participants in the class. To retrieve feedback also from students not interactively participating in the class by asking or answering questions directly, each student is assigned a mood after starting the AMATI framework. This mood acts as a traffic light for their current level of understanding. Each student can choose between seven different statuses which go from understood (green light) to the request of repeating a certain topic again (red light). This way every participant can give significant feedback; student statuses get updated minutely and are displayed in the form of a bar chart to the teaching staff.
Communication and feedback are big issues in large
audience teaching environments. Key factors are
student apprehension due to intimidation by the number
of participants in large classes. Also the verbalisation
of a concrete question is an issue to students who
have not fully understood a specific topic in first place
With the introduction of the enhanced communication between students and the teaching staff as seen in Figure 3 we also introduced an additional way of synchronous communication. As shown in Table 4 only one content related question has been asked using an asynchronous way of communication. We found a significant increase in participation towards asking content related questions using synchronous communication as seen in Table 5.
One major problem found after introducing the public
chat channel to the whole class was the distractive use
the students were making of it. There is an ongoing
debate
Content related questions can be distinguished into two categories: context-free and context-dependent. The first category contains questions that are stated in a way that teaching staff members are able to answer them independent from the time, location or even lecture content when the question has been asked. For example: ”Do we actually have to rewrite inherited operations in the child classes?” ”Does a * on a UML association mean 0,.. Infinite or 1,.. Infinite?” The second category contains questions that are stated in a way that without providing the corresponding context the teaching staff is unable to answer them. ”Is returning the vehicle from startEnginge() part of the pattern, or is that just an implementational choice?” ”Does that have something to do with the model view controller pattern?”
Being in phase of development during the whole duration of the course, AMATI was beta tested only in a couple of lectures. During the semester students already had a strong opinion of what could be improved and some of the suggestions were implemented after the course ended. At the end of the semester the students were asked to evaluate the teaching methodology and the tools adopted. The evaluation has been send out to 950 students providing a questionnaire of 32 questions, resulting in 351 students of the class participating showing a 36,9% response rate. Students have also been asked about their major subject as well as their current semester however these information have not been fully evaluated. All the questions were posed as a rating based on a Likert scale where students could choose five options moving from very high to not at all.
Students appreciated the level of interaction offered by the lecture. The following charts show that the students liked the possibility of asking questions through the feedback tools proposed (see Figure 8), while the time effectiveness of the answers scored a bit lower but still positively (see Figure 9). The design of AMATI only highlights lecture relevant material when needed. AMATI automatically presents the right view for students and teaching staff without the need of manual updates of the website. This concept allows to keep student attention focused on the lecture content without introducing disturbing sideeffects which can be verified with Figure 10. The case study has been shown that the usage of the AMATI framework and other tools supporting interaction and synchronous communication can improve the participation of students and therefore increase knowledge transfer in a location-independent but time-dependent teaching environment. However, this improve of synchronous interaction comes with the cost of volatile context-dependent teaching information. This information including its context needs to properly stored to be useful as a knowledge base and information resource for later use.
The questionnaire given out to the participants as stated in Section 4 included some proposals on potential new features to be implemented. 6.1
When asked for their opinion on generating an automatic report summarising all student questions asked and all answers provided by the teaching staff for each lecture, more than 60% of the students expressed their interest in such a feature according to 11.
The introduction of a serious game
Here the voters had a a high variance distribution with 33% liking the idea very much and 25% liking it slightly or not at all as seen in Figure 12.
The most successful proposal was to integrate the live streaming and the lecture recordings in the student representation of AMATI as seen in Figure 13.
This feature was implemented after the end of the semester by adding a form to set the necessary URLs in the teacher page and by adding a content panel in the students page showing the lecture videos and links to download the slides.