=Paper=
{{Paper
|id=None
|storemode=property
|title=OUSocial2 - A Platform for Gathering Students' Feedback from Social Media
|pdfUrl=https://ceur-ws.org/Vol-1272/paper_34.pdf
|volume=Vol-1272
|dblpUrl=https://dblp.org/rec/conf/semweb/ThomasFBA14
}}
==OUSocial2 - A Platform for Gathering Students' Feedback from Social Media==
https://ceur-ws.org/Vol-1272/paper_34.pdf
OUSocial2 - A Platform for Gathering Students’
Feedback from Social Media
Keerthi Thomas, Miriam Fernandez, Stuart Brown, and Harith Alani
Open University, UK
first.last@open.ac.uk, h.alani@open.ac.uk
Abstract. Universities strive to collect feedback from students to improve their
courses and tutorship. Such feedback is often collected at the end of a course via
survey forms. However, such methods in collecting feedback are too controlled,
slow, and passive. With the rise of social media, many students are finding online
venues to group and share their experiences and seek peers’ support. OUSocial2
is a platform that monitors behaviour, sentiment, and topics, in open social media
groups set up by, and for, Open University students. It captures anonymous feed-
back from students towards their courses, and tracks the evolution of engagement
behaviour and sentiment within those groups.
Keywords: social media, behaviour analysis, sentiment analysis
1 Introduction
The Open University (OU) has around 250 thousand students, rendering it the largest
university in the United Kingdom and one of the leading distance teaching institutions.
Although the university provide students with several websites and applications where
they can discuss their courses with their tutors and peers, many seem to be more en-
gaged in such discussions on open social media platforms, such as Facebook groups.
Social media has become a rich source for student feedback, which could be col-
lected and investigated, to capture any issues and problems in real time, as well as to
monitor the engagement of students with their courses and peers. Students retention is
especially challenging in distance learning, and close monitoring of students’ activities
and involvement can greatly help to predict churn of students, and thus giving their
tutors an opportunity to intervene and support disengaging or struggling students [4].
OUSocial2 is a prototypical platform for collecting and analysing content from rel-
evant and public Facebook groups, set up by OU students. These open groups have been
set up to bring together other students who enrolled in particular OU courses or mod-
ules. OUSocial2 extends its predecessor which is described in [1], with a completely
new interface, and lexicon-based sentiment tracking service.
More specifically, the objectives of the OUSocial2 project are:
1. Build a data collection service for gathering, storing, and integrating data from
public Facebook groups related to OU
2. Develop and train a model for identifying the behaviour of individual users based
on their activities and interactions in the Facebook online groups
3. Extract the topics that emerge in Facebook group discussions
4. Track the sentiment expressed about the specific topics by the group members
This paper describes the OUSocial2 platform’s architecture, analysis components,
and data enrichment with the OUs linked data portal.
Demo: A fully working OUSocial2 platform will be demoed at the conference, running
over 44 groups from Facebook, with a total of 172,695 posts from 19,759 users. Au-
dience will be able to see how the various analyses components described below can
be used to assess and monitor engagement of students in course groups, their evolving
sentiment, and topics. For privacy reasons, the live demo is not publicly available yet.
A video recording of the demo is available at:
https://dl.dropboxusercontent.com/u/17906712/ousocial2-demo.mp4
https://dl.dropboxusercontent.com/u/17906712/ousocial2-demo.avi
2 OUSocial2
In this section we describe the three main OUSocial2 analyses components and how
their output is visualised in the demo. Facebook API is used to collect all posts and
interactions from public groups about OU courses. 44 of such groups are identified by
matching their titles to official OU course codes (e.g., T224). Collected data includes
group ID, posts’ content, owner, time of posting, whether the post is a reply to another
post, users, etc. Data collection is reactivated every 24 hours to update the database.
Fig. 1. Distribution of behaviour roles over time for several selected groups
2.1 Behaviour Analyser
This component applies our behaviour analyses service (see [3]) which uses machine
learning and SPIN (spinrdf.org/) rules to identify the roles of users. Understanding
the behaviour composition of a group, and the evolution of behaviour roles of individ-
uals (micro) and groups (macro) is useful for assessing user engagement and future
prospects [3, 2].
This component identifies eight types of roles; Lurker, Follower, Daily User Con-
tributor, Broadcaster, Leader, Celebrity and Super User. Figure 1 is the OUSocial2
display of the role compositions of the top 10 active groups. The slide bar at the bottom
is to view the roles at different points in time. The number and percentage of each type
of role in a group is displayed on the right hand side. Engagement of particular group
members can also be studied (see Figure 2).
Fig. 2. User behaviour over time
Fig. 3. Evolution of group sentiment over time. Red line is average sentiment across all groups.
2.2 Sentiment Analyser
The sentiment analysis component calculates the sentiment for each post. We use Sen-
tiStrength;1 a lexicon-based sentiment analyser, to estimate the strength of positive and
negative sentiments in our posts. We calculate sentiment at the community and member
levels. OUSocial2 users can visualise and compare the evolution of sentiment in se-
lected groups (Figure 3). Users can also see the sentiment distribution of a given group
over time 4, and upon clicking on a specific time point, the list of top positive, negative,
and neutral posts are listed (Fig 5).
Fig. 4. Overall positive and negative Fig. 5. Topics appearing in Fig. 6. Topics in posi-
sentiment levels in a group posts with positive sentiment tive posts
2.3 Topic Analyser
Several named entity recognition systems have emerged recently, such as Textwise,
Zemanta, DBpedia Spotlight, OpenCalais, Alchemy API, and TextRazor. OUSocial2
uses TextRazor since it seems to provide the best accuracy in our context. TextRazor
1
http://sentistrength.wlv.ac.uk/
(textrazor.com/) identifies entities from our posts, and returns the relevant URIs from
DBpedia and Freebase, with confidence scores. Users of OUSocial2 can view the topics
that appear in posts, in tag clouds of positive or negative entities (Fig. 6, to help them
spot any issues or concerns raised by the members of these groups.
2.4 Data Enrichment
OU official information about all courses already exist as linked data from data.open.ac.uk.
Course information can be SPARQLed to retrieve course titles, descriptions, topic cat-
egories, relevant courses, etc.
3 Feedback and Future Work
OUSocial2 was demonstrated to the university’s executive board and strategy office, and
was generally very well received as a tool that could enhance our collection of feedback,
and speeding up our reaction to any concerns or challenges raised by students. Privacy
was raised as an important issue, and further steps are planned to abstract any informa-
tion that could lead to the identification of students. It was suggested that sentiment and
engagement results could be compared to actual students’ performance on the courses
in question, as well as to their end-of-year feedback forms. Other requests include the
implementation of alerts on abnormal activities in chosen groups (e.g., drop in engage-
ment, rise in negative sentiment), and a comparison between groups on the same course
but on different years.
Sentiment analysis was done using SentiStrength; a general-purpose lexicon-based
tool. However, results showed that many posts on a course about World Wars were being
incorrectly flagged as negative, whereas they were simply mentioning various course
topic words (e.g., war, deaths, holocaust), rather than expressing a negative opinion
about the topic or course itself. We plan to investigate using course descriptions to
further tune our sentiment analyses.
4 Conclusions
In this document we described the main components of OUSocial2; a web based tool
for assessing and monitoring students’ engagement and sentiment in public social media
groups about their courses. The tool enables course leaders and the university to become
aware of potential concerns and factors that could lead to students to quit their courses,
which is a known problem with online learning and MOOCs.
References
1. M. Fernandez, H. Alani, and S. Brown. Ou social: reaching students in social media. In Proc.
12th International Semantic Web Conference (ISWC 2013) - Demo, Sydney, Australia, 2013.
2. M. Rowe and H. Alani. What makes communities tick? community health analysis using role
compositions. In 4th IEEE Int. Conf. Social Computing (SocialCom), Amsterdam, 2012.
3. M. Rowe, M. Fernandez, S. Angeletou, and H. Alani. Community analysis through semantic
rules and role composition derivation. Journal of Web Semantics (JWS), 18(1):31–47, 2013.
4. A. Wolff, Z. Zdrahal, A. Nikolov, and M. Pantucek. Improving retention: predicting at-risk
students by analysing clicking behaviour in a virtual learning environment. In Third Confer-
ence on Learning Analytics and Knowledge (LAK 2013), Leuven, Belgium, 2013.