Universities strive to collect feedback from students to improve their courses and tutorship. Such feedback is often collected at the end of a course via survey forms. However, such methods in collecting feedback are too controlled, slow, and passive. With the rise of social media, many students are finding online venues to group and share their experiences and seek peers' support. OUSocial2 is a platform that monitors behaviour, sentiment, and topics, in open social media groups set up by, and for, Open University students. It captures anonymous feedback from students towards their courses, and tracks the evolution of engagement behaviour and sentiment within those groups.
The Open University (OU) has around 250 thousand students, rendering it the largest university in the United Kingdom and one of the leading distance teaching institutions. Although the university provide students with several websites and applications where they can discuss their courses with their tutors and peers, many seem to be more engaged in such discussions on open social media platforms, such as Facebook groups.
Social media has become a rich source for student feedback, which could be
collected and investigated, to capture any issues and problems in real time, as well as to
monitor the engagement of students with their courses and peers. Students retention is
especially challenging in distance learning, and close monitoring of students’ activities
and involvement can greatly help to predict churn of students, and thus giving their
tutors an opportunity to intervene and support disengaging or struggling students [
OUSocial2 is a prototypical platform for collecting and analysing content from
relevant and public Facebook groups, set up by OU students. These open groups have been
set up to bring together other students who enrolled in particular OU courses or
modules. OUSocial2 extends its predecessor which is described in [
More specifically, the objectives of the OUSocial2 project are: 1. Build a data collection service for gathering, storing, and integrating data from public Facebook groups related to OU 2. Develop and train a model for identifying the behaviour of individual users based on their activities and interactions in the Facebook online groups 3. Extract the topics that emerge in Facebook group discussions 4. Track the sentiment expressed about the specific topics by the group members
This paper describes the OUSocial2 platform’s architecture, analysis components, and data enrichment with the OUs linked data portal.
Demo: A fully working OUSocial2 platform will be demoed at the conference, running over 44 groups from Facebook, with a total of 172,695 posts from 19,759 users. Audience will be able to see how the various analyses components described below can be used to assess and monitor engagement of students in course groups, their evolving sentiment, and topics. For privacy reasons, the live demo is not publicly available yet. A video recording of the demo is available at: https://dl.dropboxusercontent.com/u/17906712/ousocial2-demo.mp4 https://dl.dropboxusercontent.com/u/17906712/ousocial2-demo.avi 2
In this section we describe the three main OUSocial2 analyses components and how
their output is visualised in the demo. Facebook API is used to collect all posts and
interactions from public groups about OU courses. 44 of such groups are identified by
matching their titles to official OU course codes (e.g., T224). Collected data includes
group ID, posts’ content, owner, time of posting, whether the post is a reply to another
post, users, etc. Data collection is reactivated every 24 hours to update the database.
This component applies our behaviour analyses service (see [
This component identifies eight types of roles; Lurker, Follower, Daily User Contributor, Broadcaster, Leader, Celebrity and Super User. Figure 1 is the OUSocial2 display of the role compositions of the top 10 active groups. The slide bar at the bottom is to view the roles at different points in time. The number and percentage of each type of role in a group is displayed on the right hand side. Engagement of particular group members can also be studied (see Figure 2). The sentiment analysis component calculates the sentiment for each post. We use SentiStrength;1 a lexicon-based sentiment analyser, to estimate the strength of positive and negative sentiments in our posts. We calculate sentiment at the community and member levels. OUSocial2 users can visualise and compare the evolution of sentiment in selected groups (Figure 3). Users can also see the sentiment distribution of a given group over time 4, and upon clicking on a specific time point, the list of top positive, negative, and neutral posts are listed (Fig 5). Several named entity recognition systems have emerged recently, such as Textwise, Zemanta, DBpedia Spotlight, OpenCalais, Alchemy API, and TextRazor. OUSocial2 uses TextRazor since it seems to provide the best accuracy in our context. TextRazor 1 http://sentistrength.wlv.ac.uk/ (textrazor.com/) identifies entities from our posts, and returns the relevant URIs from DBpedia and Freebase, with confidence scores. Users of OUSocial2 can view the topics that appear in posts, in tag clouds of positive or negative entities (Fig. 6, to help them spot any issues or concerns raised by the members of these groups. 2.4
Data Enrichment OU official information about all courses already exist as linked data from data.open.ac.uk. Course information can be SPARQLed to retrieve course titles, descriptions, topic categories, relevant courses, etc. 3
OUSocial2 was demonstrated to the university’s executive board and strategy office, and was generally very well received as a tool that could enhance our collection of feedback, and speeding up our reaction to any concerns or challenges raised by students. Privacy was raised as an important issue, and further steps are planned to abstract any information that could lead to the identification of students. It was suggested that sentiment and engagement results could be compared to actual students’ performance on the courses in question, as well as to their end-of-year feedback forms. Other requests include the implementation of alerts on abnormal activities in chosen groups (e.g., drop in engagement, rise in negative sentiment), and a comparison between groups on the same course but on different years.
Sentiment analysis was done using SentiStrength; a general-purpose lexicon-based tool. However, results showed that many posts on a course about World Wars were being incorrectly flagged as negative, whereas they were simply mentioning various course topic words (e.g., war, deaths, holocaust), rather than expressing a negative opinion about the topic or course itself. We plan to investigate using course descriptions to further tune our sentiment analyses. 4
In this document we described the main components of OUSocial2; a web based tool for assessing and monitoring students’ engagement and sentiment in public social media groups about their courses. The tool enables course leaders and the university to become aware of potential concerns and factors that could lead to students to quit their courses, which is a known problem with online learning and MOOCs.