<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Capturing Entity-Based Semantics Emerging from Personal Awareness Streams</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>A.E. Cano</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>S. Tucker</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>F. Ciravegna</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>@Anna @Paul see you at the city center. Best wkend ever!</institution>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Department of Computer Science, University of Sheffield</institution>
          ,
          <addr-line>Sheffield</addr-line>
          ,
          <country country="UK">United Kingdom</country>
        </aff>
      </contrib-group>
      <fpage>33</fpage>
      <lpage>44</lpage>
      <abstract>
        <p>Social activity streams provide information both about the user's interests and about the way in which they engage with real world entities. Recent research has provided evidence of the presence of emergent semantics in such streams. In this work, we explore whether the online discourse of user's social activities can convey meaningful contextual information. We introduce a usercentric methodology based on tensor analysis for deriving personal vocabularies given an entity-based context. By extracting entities (e.g. location, organisation, people) from the user's stream content, we explore the data structures that emerge from the user's interrelationship with these entities. Our experimental results revealed that the simultaneous correlation of entities leads to the identification of concepts which are relevant to the user given a specific context. This methodology is relevant for mobile application designers (1) in fostering user entity-based ontologies for merging user context in pervasive environments, (2) for personalising entity-based recommendations.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>The past few years have seen the launch of different social networking platforms that
allows a user to expose their online presence, create groups and build bridges for
communicating within their online social spheres. The high usage of these platforms has
generated an enormous amount of personal information online creating unprecedented
opportunities for a wide range of research related to knowledge management, user
contextualisation, and the Semantic Web.</p>
      <p>
        In this paper, we focus on the analysis of a user’s social activity streams (a.k.a
personal awareness streams [
        <xref ref-type="bibr" rid="ref32">32</xref>
        ]) generated from different social networks.We consider
a user’s social activity stream as a historical dataset from which context-sensitive items
can be derived. Users produce data streams, not only providing information regarding
the physical world (e.g. location, surrounding things) but also regarding their digital
environment (e.g. adding new friends, microblogging). Therefore, we see the user’s
social activity streams as virtual sensors that could provide valuable information not
only about the user interests but also about the user’s physical contextual situation.
      </p>
      <p>
        This paper sets out to explore whether the use of aggregations of personal
awareness streams can convey meaningful contextual information given a set of different
entities that the user has interacted with within their online discourse during a
timeline. In this paper, we introduce the Concept Selection Induced from Social Stream
Aggregations (CSISSA) methodology, which captures entity-related information (e.g.
organisations, locations, people, links) emerging from a personal awareness stream
aggregation. This methodology is based on a three-mode network of social awareness
streams (a.k.a. Tweetonomy [
        <xref ref-type="bibr" rid="ref32">32</xref>
        ]) and lightweight associative resource ontologies [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ].
CSISSA applies tensor analysis for performing a simultaneous correlation of the given
entities. Computing the decomposition of the tensor yields to conceptual structures that
characterise a user given a context.
      </p>
      <p>In this work we investigate the way in which a user refers to entities in the content of
the message he generates. These entities are interlinked to others through, for example
text and hashtags. We explore if this entity-based interrelationship can yield emerging
conceptual structures that can aid in the user modelling. Our experimental results
suggest that a key factor for successfully deriving relevant concepts for a given context is
the user’s microblogging verbosity, and the use of common vocabularies referring to
the entities involved in the context.</p>
      <p>The contributions of this paper are as follows: we study personal awareness stream
aggregations as a source of information for deriving users’ relevant concepts given an
entity-based context. We present a novel approach which enables the explicit
declaration of the context in which a user needs to be analysed. Our model abstracts the
semantics of the vocabularies introduced by the user in his social activity stream by means of
the derivation of lightweight ontologies. We make use of tensor analysis for building a
user’s entity-based context.The encapsulation of an entity-related lightweight ontology
constitutes a slice of a tensor. The decomposition of this tensor reveals concepts
relevant to the user in the analysed context. We believe that entity-based user modelling
could aid in the future integration of user context to pervasive environments.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Background</title>
      <p>In this section we start by defining concepts from principal component analysis (PCA)
and then we give a brief introduction to tensor analysis. We will follow the typical
conventions, and denote matrices with upper case bold letters (e.g. X, row vectors with
lower-case letters (e.g. v), and tensors with calligraphic font (e.g., X ).</p>
      <p>
        Principal Component Analysis (PCA) PCA [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] helps to identify patterns in data
by expressing this data in such a way that it highlights a limited number of
“components” that capture most of the information contained in the observed variables. By
performing an orthogonal linear transformation, PCA finds the best linear projections
which minimize least squares cost. For a given matrix X with zero mean (i.e. the mean
of the distribution has been subtracted from the data set), PCA can be computed by
obtaining the Singular Value Decomposition (SVD)[
        <xref ref-type="bibr" rid="ref8">8</xref>
        ][
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] of X; according to which
X = Usvd × Σsvd × VsTvd; then Y = Usvd × Σsvd and U = Vsvd. For example,
if X is a user’s status-keywords matrix taken from a user’s stream aggregation dataset,
then the Y and U matrices can be interpreted as the status-concept matrix Y, and the
keywords-concept matrix U.
#MSM2011
      </p>
      <p>A user’s post can be further analysed by considering not only keywords but also
other resources (e.g. location, people) embedded on its content; forming a
multidimensional set of parameters. An example for such analysis could study those
topics that emerge from a user’s posts generated during the morning hours at the
office (location×time×keywords). A mathematical abstraction for the representation of a
higher way structured data is a Tensor.</p>
      <p>
        Tensor Analysis Tensors[
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] are multidimensional M-ways or Mth-order arrays
which generalize the notion of vectors(1-way or first-order array) and matrices (2-ways
or second-order arrays). Tensors of order greater or equal to three are called
higherorder tensors. In order to identify patterns that emerge from the simultaneous correlation
of a set dimensions it is necessary to decompose a tensor. Tensor decomposition can be
considered as a higher-order generalisation of SVD and PCA. In this paper we will use
the Tucker decomposition approach.
      </p>
      <p>
        Tucker Decomposition The Tucker decomposition was first introduced by Tucker
in 1963 [
        <xref ref-type="bibr" rid="ref30">30</xref>
        ]. Given a tensor X ∈ RI1×...×IN PCA is performed so as to decompose
tensor X into a core tensor G ∈ RR1×..×RN multiplied by a set of matrices U(i) ∈
RIi×Ri . Therefore the Tucker decomposition of a three-order tensor X can be expressed
as.
      </p>
      <p>X ≈ G !×1QA ×2 B ×3 C =
!pP=1 q=1 !rR=1 gpqrap ◦ bq ◦ cr ≡ [[G; A, B, C]]</p>
      <p>
        One of the approaches for computing a Tucker decomposition of a three-order
vector is to start with a first approximation obtained by applying a Higher Order SVD
(HOSVD) [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ] and then apply the alternating least squares algorithm (ALS) [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ].
3
      </p>
    </sec>
    <sec id="sec-3">
      <title>Related Work</title>
      <p>
        Mika [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ][
        <xref ref-type="bibr" rid="ref28">28</xref>
        ] explores how community-based semantics, in the form of lightweight
associative ontologies, emerge from folksonomies. He introduces the semantic-social
networks model which consists of a tripartite graph of people, concept and instance
associations. Wagner and Strohmaier [
        <xref ref-type="bibr" rid="ref32">32</xref>
        ] introduce the Tweetonomy model, which is
a formalisation of social awareness streams. This model adopts a theoretic approach
similar to the one presented by Mika. However, the Tweetonomy model presents a more
complex and dynamic structure than folksonomies. Strohmaier et al[
        <xref ref-type="bibr" rid="ref18">18</xref>
        ] and Ko¨rner et
al[
        <xref ref-type="bibr" rid="ref13">13</xref>
        ], study quantitative measures for tagging motivation. In their study they found
empirical evidence that the emerging semantics of tags in folksonomies are influenced
by individual user tagging practices.
      </p>
      <p>
        Tensor decompositions have a long history and have been applied in different
research communities. In particular the Tucker decomposition has been used in
chemical analysis [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], psychometrics [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] and computer vision [
        <xref ref-type="bibr" rid="ref31">31</xref>
        ]. Tensor analysis has also
been applied in web search; Kolda et al [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] propose a method called Topical HITS
(TOPHITS) which can be considered as an extension of Kleinberg’s HITS (Hypertext
Induced Topic Selections) algorithm [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. TOPHITS analyses a semantic graph that
combines anchor text with the hyperlink structure of the web. In order to avoid losing
edge type information when modelling the adjacency structure of a semantic graph as
a matrix, they modelled it as a three-way tensor containing both the hyperlink and
an#MSM2011
chor text information. Their tensor decomposition leads to triplets of vectors containing
authority, hub scores for the pages, and topic scores for the terms.
      </p>
      <p>
        Rendle and Thieme [
        <xref ref-type="bibr" rid="ref25">25</xref>
        ] apply tensor factorisation for personalised tag
recommendation and learning. They introduce a model based on Tucker decomposition to
explicitly model the pairwise interaction between users, items and tags. More similar to our
work is the approach of Wetzker et al [
        <xref ref-type="bibr" rid="ref33">33</xref>
        ]. They follow a user-centric tag model for
deriving mappings between personal tag vocabularies (a.k.a personomies [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]) and the
corresponding folksonomies. Our approach differs from previous work in that rather
than building the tensor as a three-way tensor of items-users-tags, we generate a
threeway tensor in which each slice is a lightweight associative “resource” ontology; which
allows to store multiple stream qualifiers in the tensor.
      </p>
      <p>
        The analysis of user-generated content extracted from social media sites is an active
research area. Qualitative and quantitative studies have been carried out for leveraging
the “wisdom of crowds” [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]. Some of this research has focused on questions related
to network and community structure. For example, Krishnamurthy et al [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] present
a characterisation of Twitter social network, which includes patterns in geographic
growth and user’s social activity. In their work, they suggest that frequent updates might
be correlated with high overlap between friends and followers. Java et al [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], present an
analysis of Twitter and suggest that the differences in users’ network connection
structures can be explained by the following types of user activities: information seeking,
information sharing and social activity.
      </p>
      <p>
        Other work has presented a systematic analysis of the content of posts in social
networks. Recent work [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ], introduces the term “Social Awareness Streams” for referring
to this aggregation of short status messages. They proposed a characterisation of these
messages via a human coding of tweets into nine categories including ”Information
sharing” and “Self promotion”. By extrapolating from these categories, they induced
two types of users the “informers”, who post about non-personal information, and the
“meformers” which mostly post about themselves. Stankovic et al [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ], study
conference related tweets. They map tweets to talks an subevents that they refer to. Using
linked data they derive additional knowledge about event dynamics and user activities.
      </p>
      <p>
        Data structures emerging from the Social Web have been studied in the
Information Retrieval and Semantic Web communities. Research in this area includes the study
of content and link analysis algorithms and ontology learning algorithms. Heymann et
al [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] present an algorithm for hierarchical taxonomy generation from social tagging
systems. For generating a taxonomy of tags, they apply graph centrality in a cosine
similarity graph of tags. Ramage et al [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ], apply labelled Latent Dirichlet Allocation
(LDA) [
        <xref ref-type="bibr" rid="ref24">24</xref>
        ] for mapping content of the Twitter feed into four dimensions including
style and substance. Schmitz [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ] introduces a subsumption-based model for inducing
faceted ontologies from Flickr tag vocabulary. Our work was inspired mainly by Mika’s
[
        <xref ref-type="bibr" rid="ref20">20</xref>
        ], and Wagner and Strohmaier’s [
        <xref ref-type="bibr" rid="ref32">32</xref>
        ] work. We apply the Tweetonomy formalisation
for obtaining personal awareness stream aggregations. Our work differs from existing
work (1) through our focus on deriving person-based lightweight ontologies from
personal awareness stream; which enrich concepts and reveal structures that are meaningful
to the owner of the stream; (2) we study the content of the messages not only in terms
of traditional resources as hashtags, and links, but also in terms of entities (e.g
loca#MSM2011
tion, people, organisations); (3) we present a methodology based on tensor analysis that
allows the definition of entity-based context for deriving person-based ontologies.
4
      </p>
    </sec>
    <sec id="sec-4">
      <title>Social Stream Aggregation and Entity-Based Concept Induction</title>
      <p>Our interest is to enable a way in which a user’s social activity streams can be analysed
in order to discover concepts that can aid in profiling him. These concepts are revealed
as a combination of featuring dimensions. Example of these dimensions include e.g. a
user’s interests, user location, user’s tendencies in favouring a position in a discussion
etc. The following subsection presents the definition of three different social networks
modelled as tripartite social awareness streams.
4.1</p>
      <sec id="sec-4-1">
        <title>User’s Social Stream Aggregation</title>
        <p>
          Following the Tweetonomy model suggested by Wagner and Strohmaier[
          <xref ref-type="bibr" rid="ref32">32</xref>
          ], we
describe a social awareness stream as a sequence of tuples S, according to the following
definition:
Definition 1. A tweetonomy is a tuple
S := (Uq1, Mq2, Rq3, T, f t), where
• U,M,R are finite sets whose elements are called users, messages and resources.
• Each of these sets are qualified by q1,q2, and q3 respectively (explained below).
• T is the ternary relation T ⊆ U×M×R representing a hypergraph with ternary edges.
        </p>
        <p>The hypergraph of a tweetonomy T is defined as a tripartite graph H (T) = $V, E%
where the vertices are V = U ∪ M ∪ R, and the edges are:
E = {{u, m, r} | (u, m, r) ∈ T }. Each edge represents the fact that a given user
associates a certain message with a certain resource.
• ft is a function that assigns a temporal marker to each ternary edge.</p>
        <p>In this study we will focus on user-centric social streams generated in Facebook,
Foursquare and Twitter, according to the following qualifiers:
• The way a user can be related to a message is represented by the qualifier q1. For
this analysis we only consider the authorship relationship: U a (the author of the
message).
• The qualifier q2 represents the types of messages. This is a comment or a status in
Facebook; a broadcast message, direct message, re-tweeted message in Twitter; a
broadcast message (shout) in Foursquare are considered to be the same type. For
this experiment we don’t differentiate between types.
• The qualifier q3 for resources considers: R k (keywords), Rh (hashtags), Rli (URLs),
Rmlo (message-emitted location), Ro (organisations - entities recognised as an
organisation), Rp (people -entities recognised as a person), R l (location - entities
recognised as a location).
#MSM2011
We focus on a user given the streams he has produced within a window of time. Given
the tuples Tfacebook, Tfoursquare, Ttwitter, we define the sets U, R, M as:</p>
        <p>U = Ufacebook ∪ Utwitter ∪ Ufoursquare,
R = Rfacebook ∪ Rtwitter ∪ Rfoursquare,
M = Mfacebook ∪ Mtwitter ∪ Mfoursquare</p>
        <p>We are interested in extracting the concepts emerging from the streams produced
by a user:
uˇ ∈ U : uˇ ∈ Ufacebook ∧ uˇ ∈ Utwitter ∧ uˇ ∈ Ufoursquare
In order to do so we consider a user stream aggregation defined as a tuple:
Sa(U") = (U, M, R, Y", f t) , where
Y" = {(u, m, r) | u ∈ U" ∨ ∃u" ∈ U", ˜m ∈ M, r ∈ R : (u", m˜, r) ∈ Y}
and U" ⊆ U and Y" ⊆ Y. Sa(U"), consists of all messages related with a user u" ∈ U"
and all the resources and users related with these messages.
4.2</p>
      </sec>
      <sec id="sec-4-2">
        <title>Lightweight Associative Ontologies</title>
        <p>
          An ontology, is a shared, formal conceptualization of a domain [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ][
          <xref ref-type="bibr" rid="ref1">1</xref>
          ]. It is a data
structure which is an advancement in conceptual modelling over taxonomic structures [
          <xref ref-type="bibr" rid="ref28">28</xref>
          ].
A lightweight ontology can be considered as an evolving classification structure created
by users [
          <xref ref-type="bibr" rid="ref27">27</xref>
          ], which can be considered to be closer to a thesaurus (i.e. a structure
organising topics). We want to derive a set of concepts from a simultaneous correlation among
the resources q3 (e.g. keywords, hashtags, links) extracted from a user stream
aggregation. In order to obtain this correlation, we start identifying those bipartite graphs
(two-mode graphs) that could be of any interest to our analysis.
        </p>
        <p>Consider for instance the association between keywords and location; which can
be obtained as a combination of location×message (R l M) and keywords×messages
(Rk M). Where the location×messages (bipartite graph R l M) is defined as:
Rl M = %Rl × M, Erm&amp; = {(r, m) |r ∈ Rl ∧ ∃u ∈ U : (u, m, r) ∈ E} ,
w : E → R, ∀e = (r, m) ∈ Erm
and the keywords×message (bipartite graph R k M), is defined as:
Rk M = %Rk × M, Erm&amp; = {(r, m) |r ∈ Rk ∧ ∃u ∈ U : (u, m, r) ∈ E} ,
w : E → R, ∀e = (r, m) ∈ Erm</p>
        <p>These bipartite graphs represent the adjacency or affiliation matrices: R l M; which
links the resources (of type location) to the messages in which this resource has been
mentioned by this user. In the same way, R k M; links the resources (of type keyword)
to the messages in which this resource has been mentioned by at least one user. Each
link (edge) can be weighted following a local or global weighting function in order to
condition the data to be analysed (see Fig. 1).</p>
        <p>
          Finally, the association between keywords and location is expressed as R k Rl =
(Rk M)(Rl M)T. We can now encapsulate the information that associates locations
with keywords only in terms of keywords by multiplying R k Rl with its transpose, i.e.
O (RkRl) = (RkRl) (RkRl)T. This matrix, known as co-affiliation matrix, can be
considered as a lightweight associative location ontology [
          <xref ref-type="bibr" rid="ref20">20</xref>
          ] based on overlapping sets
of keywords.
#MSM2011
Personal Awareness Stream Aggregation
3630
status
retweet
Just checked in @De Hems Dutch Pub
22 hours ago reply
In London. Any suggestions for a wkend
pub route?
2 day ago
        </p>
        <p>De Hems Dutch Bar</p>
        <p>Rl
pub</p>
        <p>wkend
London
Rl
wkend</p>
        <p>Paul
Rp
Anna
Rp</p>
      </sec>
      <sec id="sec-4-3">
        <title>In this paper we propose the Concept Selection Induced from Social Stream Aggre</title>
        <p>
          gations technique. This technique obtains a set of concepts derived from the
simultaneous analysis of the correlation of different stream qualifiers. It is based on the analysis of
Sp3way tensors [
          <xref ref-type="bibr" rid="ref29">29</xref>
          ] in which each slice consists of a dense matrix formed by the
product of a sparse matrix and its transpose. The motivation for using this class of tensors
arises from the need of simultaneously storing multiple stream qualifier matrices.
        </p>
        <p>Given P lightweight ontologies characterising a user’s social streams consisting of
N messages; we define a tensor O ∈ RN×N×P consisting of frontal slices of the form
Op = BpBpT with p = 1, ..P , where B is a bipartite graph deriving the lightweight
ontology Op; see Figure 2.</p>
        <p>P
O = N Op= BpBpT</p>
        <p>N</p>
        <p>O(RkRh)=(RkRh)(RkRh)T
#linkedData..... #tramlines
1
RkRh=ES.WC 1</p>
        <p>K..N 1</p>
        <p>O(RkRl)=(RkRl)(RkRl)T
RkRl c=oKmE...NpSuWteCr She1feld ..... G1reece
1</p>
        <p>The computation of a Tucker decomposition (presented in subsection 2) of O yields
to an approximation of the form</p>
        <p>O ≈ G ×1 K ×2 K! ×3 C =
!iN=1 !iN=1 !pP=1 giipmi ◦ m!i ◦ cp ≡ [[G; K, K!, C]]</p>
        <p>The output has the property K ≈ K", the rows of these matrices contain feature
vectors that encapsulate a compilation of the different similarities expressed in the frontal
matrices. K and K", can be regarded as keyword × keyword-group matrices
highlighting those keywords that are more relevant to the similarities expressed in all O p. The
matrix C represents an index × index-group matrix which highlights O p matrices.
Finally the tensor G expresses how groups (keywords-group and index-group) relate to
each other. The frontal matrix K highlights those concepts.
5</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>Deriving Relevant Concepts with CSISSA</title>
      <p>The analysis with CSISSA is carried out on a user’s social stream aggregation S a(U"),
this aggregation is built upon the messages the user has posted in different social
networks. These messages are saved in a data store as the user generates them, and can be
retrieved in windows of time of n days, this is: S a(U") [ts, te] = (U, M, R, Y", ft), where
ft : Y" → N, ts ≤ ft ≤ te and |te − ts| = n days.</p>
      <p>The retrieved messages need to be pre-processed; 1) Stop words, punctuation and
numbers from the message content are removed; 2) From the message content, entities
of type: Location, Person and Organisation are extracted. Qualifiers of type: keywords,
hashtags and geocodes (when provided) are also extracted. This section presents a
concrete example in which CSISSA can be applied.
5.1</p>
      <sec id="sec-5-1">
        <title>Recurrent Entity-Concept Analysis</title>
        <p>
          Consider the problem of finding a temporal correlation among certain entities to which
a user is engaged with, through the messages he has posted within a window of time;
and from these entities induce a set of concepts to which they can be linked (this can
be applied in temporal user profiling and event detection). The selection of the correct
bipartite graphs to take part on the three-order tensor depends on the situation from
which the entity-based context needs to be extracted. For example, considering the
entities: Hashtag and Location; we define the following lightweight ontologies:
• Lightweight Associative Keyword Ontology Given a keyword × message matrix
RkM = wij , where wij is computed following a term frequency-inverse document
frequency (tf-idf) weighting function [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ]. We define the lightweight associative
keyword ontology O(RkM) as O(RkM) = (RkM)(RkM)T.
• Lightweight Associative Hashtag Ontology, we define the hashtag × message
matrix RpM following as well a (tf-idf) weighting function. The O(R hM) is
defined as O(RhM) = (RhM)(RhM)T.
• Lightweight Associative Location Ontology, we define the places × message
matrix RlM following as well a (tf-idf) weighting function. The O(R lM) is defined
as O(RlM) = (RlM)(RlM)T.
• Ligthweight associative time ontology, first, we obtain the hour × message
affiliation matrix HM = vij where vij = 1 if the time message mj was produced
during the hour hi and vij = 0 otherwise. We define the ligthweight associative
time ontology O(HM) as O(HM) = (HM)(HM)T.
        </p>
        <p>To analyse the correlation of these entities and derive the related concepts, it is
necessary to encapsulate the previous ontologies in terms of keywords (see section 4.2); i.e
to obtain O(RkRh), O(RkRl), O(RkH). These ontologies will form the slices of
#MSM2011
the tensor O. The computation of a Tucker decomposition of the O tensor will reveal
a ranked vector of concepts. By decomposing each of the tensor slices, it is possible to
derive the entities relevant to the decomposition.</p>
        <p>Table 1, presents the relevant concepts, and the highlighted entities derived from
the Tucker decomposition of a tensor built from the stream aggregation of one of the
users we followed in our evaluation (see section 6). This analysis reveals concepts that
are recurrently relevant to the user. In this case, these results expose the correlation of
the locations: Sheffield, London and Washington with the user’s work related concepts
during working hours.
CSISSA was evaluated on the grounds of the relevance of a concept induced by a given
contextual need. A contextual need was expressed by a pair of contexts, e.g.
LocationTime, Hashtag-Location. CSISSA provides a set of relevant concepts computed by the
simultaneous correlation of the entities involved in a given context. For testing this
technique, we “followed” a set of four “active” microbloggers. Three of them technology
oriented user, and one of them an active blogger in education. The stream
aggregations were recorded from 1st of July until the 25th September 2010 , and entities where
extracted using Open Calais services 1.</p>
        <p>In the absence of a gold standard, evaluating the concepts that emerge from a user’s
social aggregation given a context is a difficult task; it requires consulting the author of
the social stream whose context-induced concepts are being mapped. For evaluating the
effectiveness of CSISSA, each user was presented with a contextual need, and a set of
concepts derived by CSISSA. The users were asked to mark each concept as relevant or
irrelevant to the given context. Although CSISSA allows the simultaneous correlation
of n-entities, which define the context; we performed the evaluation on a maximum of
two entities at a time. The evaluated contexts are: hashtag-time, location-people, and
organisation-people. For example, by deriving concepts related to hashtag and time for
one of the users, the question was: In terms of the association between the hashtag
#linkeddata, and the timeslots ([12pm-5pm], 8pm), which of the following concepts do
1 Open Calais, http://www.opencalais.com/
#MSM2011
you consider relevant?. For the hashtag-time context, three different hashtags where
evaluated, and in the same way for the other two contexts.</p>
        <p>
          As it is well known, acquiring the relevance judgement of all the ranked concepts
in terms of precision/recall is a time-consuming and expensive process [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ]. Mainly
because the ranked vector can consist of hundreds of concepts that a user would not
be willing to evaluate. Therefore, we have decided to use the Mean Average Precision
(MAP) metric [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ]. MAP measures the mean of the precision scores obtained after each
relevant concept is retrieved, using zero as the precision for relevant concepts that are
not retrieved. The MAP value represents the average under the precision-recall curve
for a set of queries. MAP values were averaged for the three cases of each context. The
results are depicted in Figure 3 a), which shows a generalized MAP performance of the
relevancy of the concepts judged by each user given a context using CSISSA.
a)
b)
        </p>
        <p>These results suggest that higher lexical diversity (K/M) leads to better MAP results
(see Figure 3 b)), this is an expected result since CSISSA explores the way in which an
entity is linked to another one through keywords. We expected to discovered relevant
concepts first if the user exposed a correlation between contexts, and second if this
correlation was able to be expressed by keywords.</p>
        <p>However, although the microblogging verbosity provided a better basis for deriving
meaningful concepts, the relevance of the concepts given a context depended highly on
the user’s patterns of correlating the entities through keywords. In our experiments a
fairly naive approach was taken by not considering the ambiguity in which user’s can
relate two entities with a keyword. Future work considers the introduction of concept
disambiguation for tackling this issue.</p>
        <p>CSISSA enabled to model users’ generated patterns in their social activity streams
given an entity-based context. These patterns expose the implicit association in which
the user interlinks entities. The concepts derived with CSISSA suggests their
applicability in user modelling, and the awareness of user intentions. A main implication of
our work is that personal awareness streams can be used effectively to model context
#MSM2011
by leveraging the user’s entity affiliations. We believe that our approach can also help
in merging user contexts in pervasive environments.</p>
        <p>During the evaluation, one of the users did not remember to have tweeted about a
particular topic, until we showed him the tweet, this suggest the necessity of introducing
relevance-decay functions in our calculations. We also noticed that many of the users’
streaming topics’ relevance was in many cases volatile; further research is necessary to
address these issues. We are also planning to test this technique on a bigger corpus, and
to compare this technique against other baselines e.g. topic analysis.
7</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>Acknowledgements</title>
      <p>This work has been supported by the European Commission as part of the WeKnowIt
project (FP7-215453), and partially supported by CONACyT, grant 175203
#MSM2011</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Borst</surname>
          </string-name>
          , Pim, Akkermans, Hans, and
          <string-name>
            <surname>Top</surname>
          </string-name>
          , Jan. Engineering ontologies.
          <source>International Journal of Human-Computer Studies</source>
          ,
          <volume>46</volume>
          (
          <issue>2-3</issue>
          ):
          <fpage>365</fpage>
          -
          <lpage>406</lpage>
          ,
          <year>February 1997</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <given-names>G. H.</given-names>
            <surname>Golub</surname>
          </string-name>
          and
          <string-name>
            <given-names>C. F. Van Loan. Matrix</given-names>
            <surname>Computations</surname>
          </string-name>
          (Johns
          <source>Hopkins Studies in Mathematical Sciences)(3rd Edition)</source>
          . The Johns Hopkins University Press, 3rd edition,
          <year>October 1996</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Gruber</surname>
          </string-name>
          , Thomas R.
          <article-title>A translation approach to portable ontology specifications</article-title>
          .
          <source>Knowl. Acquis.</source>
          ,
          <volume>5</volume>
          (
          <issue>2</issue>
          ):
          <fpage>199</fpage>
          -
          <lpage>220</lpage>
          ,
          <year>June 1993</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <given-names>R.</given-names>
            <surname>Henrion</surname>
          </string-name>
          .
          <article-title>N-way principal component analysis theory, algorithms and applications</article-title>
          .
          <source>Chemometrics and Intelligent Laboratory Systems</source>
          ,
          <volume>25</volume>
          (
          <issue>1</issue>
          ):
          <fpage>1</fpage>
          -
          <lpage>23</lpage>
          ,
          <year>1994</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <given-names>P.</given-names>
            <surname>Heymann</surname>
          </string-name>
          and
          <string-name>
            <given-names>H.</given-names>
            <surname>Garcia-Molina</surname>
          </string-name>
          .
          <article-title>Collaborative creation of communal hierarchical taxonomies in social tagging systems</article-title>
          .
          <source>Technical report</source>
          , Stanford University,
          <year>April 2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <given-names>A.</given-names>
            <surname>Hotho</surname>
          </string-name>
          , R. J A˜schke, C. Schmitz, and
          <string-name>
            <given-names>G.</given-names>
            <surname>Stumme</surname>
          </string-name>
          .
          <article-title>Information retrieval in folksonomies: Search and ranking</article-title>
          . In Y. Sure and J. Domingue, editors,
          <source>The Semantic Web: Research and Applications</source>
          , volume
          <volume>4011</volume>
          of Lecture Notes in Computer Science, chapter
          <volume>31</volume>
          , pages
          <fpage>411</fpage>
          -
          <lpage>426</lpage>
          -426. Springer Berlin / Heidelberg, Berlin, Heidelberg,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <given-names>A.</given-names>
            <surname>Java</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Song</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Finin</surname>
          </string-name>
          , and
          <string-name>
            <given-names>B.</given-names>
            <surname>Tseng</surname>
          </string-name>
          .
          <article-title>Why we twitter: understanding microblogging usage and communities</article-title>
          .
          <source>In WebKDD/SNA-KDD '07: Proceedings of the 9th WebKDD and 1st SNA-KDD 2007 workshop on Web mining and social network analysis</source>
          , pages
          <fpage>56</fpage>
          -
          <lpage>65</lpage>
          , New York, NY, USA,
          <year>2007</year>
          . ACM.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <given-names>I. T.</given-names>
            <surname>Jolliffe</surname>
          </string-name>
          .
          <source>Principal Component Analysis</source>
          . Springer, second edition,
          <year>October 2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>V. M. I. Kiers HA</surname>
          </string-name>
          .
          <article-title>Three-way component analysis: principles and illustrative application</article-title>
          .
          <source>Psychol Methods</source>
          ,
          <volume>6</volume>
          (
          <issue>1</issue>
          ):
          <fpage>84</fpage>
          -
          <lpage>110</lpage>
          ,
          <year>2001</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>J. M. Kleinberg</surname>
          </string-name>
          .
          <article-title>Authoritative sources in a hyperlinked environment</article-title>
          .
          <source>Journal of the ACM</source>
          ,
          <volume>46</volume>
          :
          <fpage>668</fpage>
          -
          <lpage>677</lpage>
          ,
          <year>1999</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <given-names>T.</given-names>
            <surname>Kolda</surname>
          </string-name>
          and
          <string-name>
            <given-names>B.</given-names>
            <surname>Bader</surname>
          </string-name>
          .
          <article-title>The TOPHITS model for higher-order web link analysis</article-title>
          .
          <source>In Proceedings of the SIAM Data Mining Conference Workshop on Link Analysis, Counterterrorism and Security</source>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12. T. G. Kolda and
          <string-name>
            <given-names>B. W.</given-names>
            <surname>Bader</surname>
          </string-name>
          .
          <article-title>Tensor decompositions and applications</article-title>
          .
          <source>SIAM Review</source>
          ,
          <volume>51</volume>
          (
          <issue>3</issue>
          ):
          <fpage>455</fpage>
          -
          <lpage>500</lpage>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13. C. Ko¨rner,
          <string-name>
            <given-names>D.</given-names>
            <surname>Benz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Hotho</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Strohmaier</surname>
          </string-name>
          , and
          <string-name>
            <given-names>G.</given-names>
            <surname>Stumme</surname>
          </string-name>
          .
          <article-title>Stop thinking, start tagging: tag semantics emerge from collaborative verbosity</article-title>
          .
          <source>In WWW '10: Proceedings of the 19th international conference on World wide web</source>
          , pages
          <fpage>521</fpage>
          -
          <lpage>530</lpage>
          , New York, NY, USA,
          <year>2010</year>
          . ACM.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <given-names>B.</given-names>
            <surname>Krishnamurthy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Gill</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Arlitt</surname>
          </string-name>
          .
          <article-title>A few chirps about twitter</article-title>
          .
          <source>In WOSP '08: Proceedings of the first workshop on Online social networks</source>
          , pages
          <fpage>19</fpage>
          -
          <lpage>24</lpage>
          , New York, NY, USA,
          <year>2008</year>
          . ACM.
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>P. M. Kroonenberg</surname>
            and
            <given-names>J. D.</given-names>
          </string-name>
          <string-name>
            <surname>Leeuw</surname>
          </string-name>
          .
          <article-title>Principal component analysis of three-mode data by means of alternating least squares algorithms</article-title>
          .
          <source>Psychometrika</source>
          ,
          <volume>45</volume>
          :
          <fpage>6997</fpage>
          ,
          <year>1980</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>B. D. M. L. De Lathauwer</surname>
            and
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Vandewalle</surname>
          </string-name>
          .
          <article-title>A multilinear singular value decomposition</article-title>
          .
          <source>SIAM J. Matrix Anal. Appl.</source>
          ,
          <volume>21</volume>
          :
          <fpage>12531278</fpage>
          ,
          <year>2000</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>M. R. M. Stankovic</surname>
            and
            <given-names>P.</given-names>
          </string-name>
          <string-name>
            <surname>Laublet</surname>
          </string-name>
          .
          <article-title>Mapping tweets to conference talks: A goldmine for semantics</article-title>
          .
          <source>In Proceedings of Social Data on the Web workshop, ISWC 2010</source>
          . Shanghai, China.
          <source>ISWC</source>
          <year>2010</year>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>R. K. M. Strohmaier</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <article-title>Ko¨rner. Why do users tag? detecting users' motivation for tagging in social tagging systems</article-title>
          .
          <source>In 4th International AAAI Conference on Weblogs and Social Media (ICWSM2010)</source>
          , Washington, DC, USA,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>C. D. Manning</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          <string-name>
            <surname>Raghavan</surname>
          </string-name>
          , and H. Schu¨tze. Introduction to Information Retrieval. Cambridge University Press,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <given-names>P.</given-names>
            <surname>Mika</surname>
          </string-name>
          .
          <article-title>Ontologies are us: A united model of social networks and semantics</article-title>
          .
          <source>In In Proceedings of ISWC</source>
          <year>2005</year>
          ,
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <surname>M. Naaman</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Boase</surname>
            , and
            <given-names>C. H.</given-names>
          </string-name>
          <string-name>
            <surname>Lai</surname>
          </string-name>
          .
          <article-title>Is it really about me?: message content in social awareness streams</article-title>
          .
          <source>In CSCW '10: Proceedings of the 2010 ACM conference on Computer supported cooperative work</source>
          , pages
          <fpage>189</fpage>
          -
          <lpage>192</lpage>
          , New York, NY, USA,
          <year>2010</year>
          . ACM.
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <string-name>
            <given-names>O</given-names>
            <surname>'Reilly</surname>
          </string-name>
          ,
          <string-name>
            <surname>Tim. O'Reilly Network</surname>
          </string-name>
          :
          <source>What Is Web</source>
          <volume>2</volume>
          .0,
          <year>September 2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23.
          <string-name>
            <given-names>D.</given-names>
            <surname>Ramage</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Dumais</surname>
          </string-name>
          , and
          <string-name>
            <given-names>D.</given-names>
            <surname>Liebling</surname>
          </string-name>
          .
          <article-title>Characterizing microblogs with topic models</article-title>
          .
          <source>In ICWSM</source>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          24.
          <string-name>
            <given-names>D.</given-names>
            <surname>Ramage</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Hall</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Nallapati</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C. D.</given-names>
            <surname>Manning</surname>
          </string-name>
          .
          <article-title>Labeled lda: a supervised topic model for credit attribution in multi-labeled corpora</article-title>
          .
          <source>In EMNLP '09: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing</source>
          , pages
          <fpage>248</fpage>
          -
          <lpage>256</lpage>
          , Morristown, NJ, USA,
          <year>2009</year>
          .
          <article-title>Association for Computational Linguistics</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          25.
          <string-name>
            <given-names>S.</given-names>
            <surname>Rendle</surname>
          </string-name>
          and
          <string-name>
            <given-names>L. S.</given-names>
            <surname>Thieme</surname>
          </string-name>
          .
          <article-title>Pairwise interaction tensor factorization for personalized tag recommendation</article-title>
          .
          <source>In WSDM '10: Proceedings of the third ACM international conference on Web search and data mining</source>
          , pages
          <fpage>81</fpage>
          -
          <lpage>90</lpage>
          , New York, NY, USA,
          <year>2010</year>
          . ACM.
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          26.
          <string-name>
            <given-names>P.</given-names>
            <surname>Schmitz</surname>
          </string-name>
          .
          <article-title>Inducing ontology from flickr tags</article-title>
          .
          <source>In WWW 2006, May 22-26</source>
          ,
          <year>2006</year>
          , Edinburgh,
          <source>UK. IW3C2</source>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          27.
          <string-name>
            <given-names>L.</given-names>
            <surname>Specia</surname>
          </string-name>
          and
          <string-name>
            <given-names>E.</given-names>
            <surname>Motta</surname>
          </string-name>
          .
          <article-title>Integrating folksonomies with the semantic web</article-title>
          .
          <source>In In Proc. of the 4th ESWC</source>
          , pages
          <fpage>624</fpage>
          -
          <lpage>639</lpage>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          28.
          <string-name>
            <surname>Technology</surname>
            , Knowledge,
            <given-names>P.</given-names>
          </string-name>
          <string-name>
            <surname>Mika</surname>
            , and
            <given-names>H.</given-names>
          </string-name>
          <string-name>
            <surname>Akkermans</surname>
          </string-name>
          .
          <article-title>Towards a new synthesis of ontology</article-title>
          .
          <source>Technical report</source>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          29.
          <string-name>
            <surname>W. K. T.M. Selee</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          <string-name>
            <surname>Kolda</surname>
            and
            <given-names>J. D.</given-names>
          </string-name>
          <string-name>
            <surname>Griffin</surname>
          </string-name>
          .
          <article-title>Extracting clusters from large datasets with multiple similarity measures using imscand</article-title>
          .
          <source>In CSRI Summer Proceedings</source>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          30.
          <string-name>
            <given-names>L. R.</given-names>
            <surname>Tucker</surname>
          </string-name>
          .
          <article-title>Implications of factor analysis of three-way matrices for measurement of change</article-title>
          . C. W. Harris, University of Wisconsin Press,
          <year>1963</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          31.
          <string-name>
            <surname>Vasilescu</surname>
            and
            <given-names>D.</given-names>
          </string-name>
          <string-name>
            <surname>Terzopoulos</surname>
          </string-name>
          .
          <article-title>Multilinear analysis of image ensembles: Tensorfaces</article-title>
          . In
          <source>In Proceedings of the European Conference on Computer Vision</source>
          , volume
          <volume>1</volume>
          , pages
          <fpage>447</fpage>
          -
          <lpage>460</lpage>
          ,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          32.
          <string-name>
            <given-names>C.</given-names>
            <surname>Wagner</surname>
          </string-name>
          and
          <string-name>
            <given-names>M.</given-names>
            <surname>Strohmaier</surname>
          </string-name>
          .
          <article-title>The wisdom in tweetonomies: Acquiring latent conceptual structures from social awareness streams</article-title>
          .
          <source>In Proc. of the Semantic Search 2010 Workshop (SemSearch2010)</source>
          , april
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          33.
          <string-name>
            <given-names>R.</given-names>
            <surname>Wetzker</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Zimmermann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Bauckhage</surname>
          </string-name>
          , and
          <string-name>
            <given-names>S.</given-names>
            <surname>Albayrak</surname>
          </string-name>
          .
          <article-title>I tag, you tag: Translating tags for advanced user models</article-title>
          .
          <source>In WSDM '10: Proceedings of the Third ACM International Conference on Web Search and Data Mining</source>
          , pages
          <fpage>71</fpage>
          -
          <lpage>80</lpage>
          , New York, NY, USA,
          <year>2010</year>
          . ACM.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>