The SiocLog software application supports social networking and online collaboration by observing the instant messaging (IM) discussions on some public IRC channels, archiving the discussion logs, and publishing them as Linked Data on the Web. The application provides Web addresses (http URIs) for information from the discussion channels, including the daily discussions and the participants. Linked Data practices and the SIOC vocabulary provide a standard way to ensure that all parties can easily access the raw data, e.g., to create mash-ups. The chat participants can further provide their Web ID to link to their FOAF personal prole, and the FOAF prole can conrm this link in return, creating trust. Consuming FOAF data helps to avoid re-entry of personal data and searching for friends on yet another social site. The prole can also link to the person's accounts on other social forums, thus connecting their divided social presence on the Web. We present how the software is being used on some IRC channels, and describe how this work connects IRC to the greater Web of Data.
Instant messaging is one major form of social interaction and online collaboration, but it is traditionally disconnected from the Web, not least because of its immediate nature and the shortness of the messages. Various services try to bridge this gap. An area that pertains to social data is observing, archiving, analysing, and publishing of the discussions. Linked Data provides a framework for publishing data on the Web, parallel to documents.
After the background information in this section, we rst motivate and then introduce the design and implementation of the SiocLog software 4 (see Figure 1), which observes discussions on Internet Relay Chat (IRC) and publishes them as Linked Data. Subsections describe how SiocLog deploys the SIOC vocabulary and the concept of a Web ID, as well as detail the software architecture. Sections on validation of the results and comparison to related work follow before we conclude in the end.
Internet Relay Chat (IRC) [
Although the protocol lets people to choose any vacant nickname when they connect to a network and thus to freely assume a chat persona each time, some networks provide an additional service called nickserv for people to register their nicknames and to authenticate themselves repeatably.
6 http://www.w3.org/2001/12/zakim-irc-bot
1.2
Linked Data [
One way to access Linked Data is SPARQL, which is the Linked Data counterpart to and similar to SQL. SPARQL is supported by software libraries such as Redland7, as well as by utility programs such as cli.php of ARC2 Starter Pack8. Tools such as NG4J9 and Squin10 strive to make the subtask of downloading data from the Web completely implicit in the declarative SPARQL queries. 1.3
Semantically-Interlinked Online Communities (SIOC)
Semantically-Interlinked Online Communities (SIOC) [
SIOC is currently used to represent information from various types of social
software such as collaborative work environments, bulletin boards and
microblogging systems [
Web ID
A Web ID [
The standard way to describe personal prole information is the Friend of a Friend (FOAF) vocabulary.
8 http://tuukka.sioc-project.org/arc2-starter-pack 9 http://www4.wiwiss.fu-berlin.de/bizer/ng4j/ 10 http://squin.sourceforge.net/
The system described here is a specic case of the Semantic Web and Linked Data approaches, and as such, its usefulness relies mostly in the adoption of these technologies. The general pursuit of interlinking online communities using these approaches has centered on the SIOC vocabulary. Most work around SIOC this far is on publishing social data, whereas data consumers are necessary for completing most use cases. With this background, the system here is mostly a publisher, with same motivation as earlier work on SIOC publishers, but it starts to consume some data as well.
Regarding the specics of the application domain, IRC plays an important role in connecting people online and in enabling their collaboration. IRC conversations are a popular collaboration tool for Free and Open Source Software developers and other communities.
Such conversations may contain useful knowledge, such as answers to frequently asked questions, new ideas, analysis of bugs, etc. In order to preserve this information, channel managers would often record logs of conversations on the channel and post them online.
However, developers may participate in conversations on multiple channels, and similar topics may be discussed in a number of related channels (e.g., #foaf, #sioc, and #swig are all related to Linked Data). A limitation of current IRC conversation logging systems is that they just record and present what was said on each channel individually, not taking into account the relations described above.
People may also leave and join a channel a number of times a day (i.e., they may not be always present when conversations take place). Thus, they may need to re-establish the context of what was happening while they were gone.
This all motivated us to create SiocLog - a tool that not only records IRC conversations but also lets users explore relationships that exist on IRC, e.g., to collect together activities of a person across multiple IRC channels. 3
The design of the SiocLog system is mainly an application of the Linked Data principles and technologies as well as the SIOC vocabulary to the domain of publishing IRC discussion logs. The following subsections describe some of the central design decisions. 3.1
In addition to the conventional navigation links between the pages of a web site, SiocLog has Linked Data links. You can make queries that take advantage of these links, and download data by following these links. The meaning of the links is standardised via well-known vocabularies such as FOAF and SIOC. Figure 2 shows what the SiocLog site looks like initially when opened in a Linked Data browser.
Figure 3 describes how the data on the site links to other pieces of data. The topmost element is a sioc:Space, which represents an IRC network and which contains the users, channels and their posts. Linking to external Linked Data happens in two ways: a foaf:Person in a FOAF prole can be linked to (foaf:holdsAccount ) a user, and a post can link explicitly with a URI to any Linked Data resource. Similarly, URIs in channel topics could be analysed as links to the topics.
The link between a person and a user is valuable in many ways, because the FOAF prole can contain more links. For example, the foaf:knows links build an explicit social network between people, and this can now be used to infer a network and trust between IRC users. Also, the other foaf:holdsAccount links of a person imply a connection between the IRC user and the person’s user accounts on other sites. In a simple but useful case, a person in multiple IRC networks and in multiple installations of SiocLog can be integrated based on these connections.
While IRC content is plain text and as such doesn’t express rich semantics, it would be conceivable to interpret textual markup such as MicroTurtle 11, which would turn each post into an arbitrary RDF graph. SiocLog would then publish these graphs along the textual content. An issue here is how to separate the provenance of the user from the one of SiocLog in a way that the potential data consumers would support. The traditional Semantic Web solution would be triple 11 http://buzzword.org.uk/2009/microturtle/spec reication either as specied in RDF or by OWL property chains. We envision a more web-like solution where the graphs would be published at per-user subdomains of the SiocLog domain (e.g. at http://wikier.irc.sioc-project. org/). 3.3
A chat persona can claim a Web ID and thus to be associated with an actual
person. If available, SiocLog uses this information to load and display prole
information, e.g., from a FOAF prole. For example, Figure 4 shows such a
prole about a person (information about the foaf:Person that the Web ID
resolves to). This way, the person doesn’t need to enter data specically for this
site, making use of portability of Social Data [
One way to claim a Web ID is via mttlbot12, whose results SiocLog uses as one source of Web ID information. However, mttlbot doesn’t require the claiming persona to be registered, i.e., it is possible to forge this information. More importantly, we wanted claiming to be possible using only standard IRC services. 12 http://buzzword.org.uk/2009/mttlbot/#project foaf:Person foaf:holdsAccountfoaf:holdsAccount
Thus, SiocLog also includes a new way to claim a Web ID: A participant who has registered their nickname with the IRC network and is logged in ( identied ), can use the IRC command /msg nickserv set property webid SomeWebID to set their Web ID. This is a persistent setting, so the participant only needs to do this once. After this, the Web ID information is publicly available from the nickserv service as a part of the nickname’s metadata ( taxonomy ). To verify their Web ID claim, the person behind a chat persona can claim in the personal prole to be using the specic chat persona. When these claims point at each other, others can trust that the Web ID is in control of the same person as the chat persona (see Figure 5). However, people may still as usual give other false information (real name, photo, friends, etc.) about themselves in their personal proles.
The SiocLog software currently comprises circa 2000 source lines of code (SLOC) in Python and circa 1000 lines of (unabstracted) HTML page templates in the Zope Template Attribute Language (TAL). The only external dependencies are the software libraries Twisted 13 for network IO, SimpleTAL 14 for HTML template expansion, and Redland 15 for loading and querying Linked Data.
The architecture of the SiocLog software comprises four major components: the IRC interface, the data analysis, the data integration, and the Web interface. 13 http://twistedmatrix.com/ 14 http://www.owlfish.com/software/simpleTAL 15 http://librdf.org/ 4.1
The responsibility of the IRC interface is to gather information and log it for later analysis. Because of the diering requirements, this component is split into two separate IRC bot programs. One needs to be reliable and thus preferably stable and simple, mostly passive discussion logger. The other needs to gather information about the chat personas from the nickname service, responding to queries from the other components.
To provide maximum information and exibility for later data analysis, everything is logged in a text le as timestamped but otherwise raw lines of IRC protocol. The bots have been built specically for SiocLog and on top of the Twisted network IO framework. As an alternative or in addition to the bots, log les could be imported from other systems. 4.2
The data analysis processes the raw IRC log les, selecting the relevant events, analysing their textual content and relation to other events in context, translating them to the SIOC model, and calculating statistics.
As IRC data is highly linear in nature, the data analysis is implemented as a pipeline of lters that process the events and as sinks that format the output. The rst lters remove o-the-record messages and detect whether the message is from an authenticated user. Later lters select events that pertain to a specic channel, user, or time period, and analyse the textual content to extract links. The sinks aggregate statistics and produce a HTML, RDF, or raw representation of the events. 4.3
The data integration queries the Web for information needed to combine the IRC log data with other Linked Data. It fetches and combines mttlbot’s knowledge16 of nickname to Web ID mappings with SiocLog’s own information. It also fetches data pointed to by the Web IDs, combines it with SiocLog’s knowledge about chat personas, and renders the personal proles.
Currently, this all is implemented as simple queries to temporary in-memory triple stores. 4.4
Accepting HTTP requests via the CGI interface of a Web server, the Web interface depends on the data analysis and integration parts and publishes the results as Web pages in HTML and as Linked Data documents in RDF. This component does HTTP content negotiation, splits requested URIs into query parameters, drives the data analysis and integration tasks, as well as translates the results to the output format. 16 http://buzzword.org.uk/2009/mttlbot/graphs/knowledge
Listing 1.1. Example SPARQL query over IRC data provided by SiocLog. semwebquery s p a r q l "SELECT ?name WHERE { ? p e r s o n f o a f : holdsAccount
<http : / / i r c . s i o c p r o j e c t . org / u s e r s / m e l v s t e r#user > . ? p e r s o n f o a f : knows ? f r i e n d .
? f r i e n d f o a f : name ?name . }" 5
An installation of the software is in use on a few Linked Data related IRC channels since 118 days as of writing. Based on this experience, it has been reliable, and it has been convenient to be able to constantly improve the analysis, integration, and publishing parts on the y. It has collected discussions of 291 chat personas on 5 channels this far, 22418 messages in total. For 51 chat personas in total, it knows the Web ID of the person behind them (41 from mttlbot, 18 set at nickserv; 52% of #sioc personas, 15% of #swig personas). Of these 51 Web IDs, 44 point to a valid RDF document which SiocLog is able to render reasonably. This far, 7 people have conrmed their Web ID claim in their FOAF prole.
To test browsability, we checked that given the site root URI, a Linked Data
browser such as Fenre [
The project Towards linked sensor data for Hackystat 18 is already at this phase consuming Linked Data from SiocLog to track the discussion activity of software developers.
FOAF Me19 provides some simple integration with SiocLog. If a user doesn’t have a FOAF prole, SiocLog links to FOAF Me as a possible service for authoring one. FOAF Me will detect the linking and automatically pre-ll the prole with the respective IRC user as a nick name and as an external account. The FOAF Me project has further plans to consume the data from a SiocLog account in an activity feed view of the person. 5.1
For the rst two months of discussions on #swig logged by SiocLog, there were 191 participants, and 10948 messages with 662567 bytes of textual content. This is 1.6 megabytes of the raw text log format, or 5.3 megabytes of Turtle RDF format, or 3.2 megabytes of HTML web pages. 17 http://www4.wiwiss.fu-berlin.de/bizer/ng4j/semwebclient/ 18 http://socghop.appspot.com/student_project/show/google/gsoc2009/ hackystat/t124022455248 19 http://foaf.me/
The current representation produces 5 RDF triples per user, 7 triples per channel, and 6 triples plus any enrichment per post. In total, these form 67192 triples. Assuming that the activity of the channel remains similar, a linear extrapolation predicts this would become 4 million triples in 10 years, which is not a challenge for modern RDF stores. 6
Dave Beckett’s Logger 20 IRC bot has been publishing IRC discussion logs in RDF since year 2001. However, the output from Logger is somewhat outdated regarding Linked Data principles as it doesn’t give the participants http URIs and corresponding information documents with links to more information, and it uses the old RDF container properties instead of SiocLog’s implicit ordering of events based on dc:date. Further, Logger doesn’t use the standard and rich SIOC (Semantically-Interlinked Online Communities) vocabulary in its RDF. Finally, Logger doesn’t let chat participants declare their Web ID and thus doesn’t link out to and integrate with FOAF network of people or the larger Web of Data in general.
Another application that converts non-Web social data to Linked Data is
SWAML [
Another application that publishes small messages as Linked Data using
the SIOC vocabulary is SMOB [
This paper introduced SiocLogan application for collecting IRC conversations, making them available for reuse in other applications and services via the use of Linked Data, and for enabling rich access to knowledge contained in these conversations.
We described how SiocLog links together users and channels on which users engage in conversations. It can also bring in and integrate Linked Data from external sources, e.g., people’s FOAF proles.
While SiocLog as implemented deals exclusively with IRC content, a system supporting other instant messaging systems should be similar. The idiosyncrasies of each system will aect the best way to form URIs. The exact way of observing discussions will depend on the system, as protocols and chat culture vary. For example, where private chats are the norm, participants will need either to invite an observer bot, or to collect and publish the events themselves. There will also 20 http://www.dajobe.org/software/logger/ be dierences in the exact form of discussions, such as whether channels have topics or only names.
This work envisions SiocLog as a signicant part of the Web of Data, acting as an initial consumer and use case of FOAF data as well as providing other applications with real-time services such as user presence data or information about resources that people talk about right now. Further, by publishing rich, structured information about IRC conversations, SiocLog can help new studies of online communities and their behaviour to include data about real-time communication. 8
The authors would like to thank Benja Fallenstein and Dan Brickley for their insights on the requirements and suitable technologies for SiocLog.