=Paper=
{{Paper
|id=None
|storemode=property
|title=Exploiting Twitter as a Social Channel for Human Computation
|pdfUrl=https://ceur-ws.org/Vol-842/crowdsearch-diaz.pdf
|volume=Vol-842
|dblpUrl=https://dblp.org/rec/conf/www/Diaz-AvilesK12
}}
==Exploiting Twitter as a Social Channel for Human Computation==
https://ceur-ws.org/Vol-842/crowdsearch-diaz.pdf
Exploiting Twitter as a Social Channel for
Human Computation
Ernesto Diaz-Aviles Ricardo Kawase Wolfgang Nejdl
diaz@L3S.de kawase@L3S.de nejdl@L3S.de
L3S Research Center / University of Hannover. Hannover, Germany
ABSTRACT Social networking sites such as Twitter3 have experienced
To fully leverage the innate problem solving capabilities of an explosion in global Internet traffic over the past years.
humans necessitates paradigm shifts towards decentraliza- As of June 2011, it is estimated that the Twitter users sur-
tion of human computation systems, making the existence of passed 300 million, and they generate more than 200 million
central authorities superfluous and even impossible. In this of 140-character Twitter messages – tweets – every day [8,
position paper, we propose a novel decentralized architecture 9]. Interestingly enough, nearly two-thirds of active Twit-
that exploits the Twitter social network as a communication ter users access the microblogging service using a mobile
channel for harnessing human computation. Our framework phone [2].
provides individuals and organizations the necessary infras- The massive amount of mobile users plus the simplicity
tructure for human computation, facilitating human task of the interactions in Twitter, together with its scalability
submission, assignment and aggregation. We presented a and real-time message exchange, make this social network-
proof of concept and explore the feasibility of our approach ing system an appealing environment to assign and collect
in the light of several use cases. feedback for human computation tasks, which fits the nature
of a tweet: short and simple.
In this work we propose MechSwarm, a decentralized frame-
Categories and Subject Descriptors work for human computation built upon Twitter’s infras-
H.3 [Information Storage and Retrieval]: Information tructure. Our primary contributions can be summarized as
Search and Retrieval—Information Filtering; K.4 follows:
[Computer and Society]: [General] • We present the building blocks necessary for a decen-
tralized crowdsourcing architecture.
General Terms
• We introduce simple yet powerful idioms for human-
Human Factors; Design intelligent-task assignment over the Twitter social net-
work, which can be considered as a protocol for human
Keywords computation over a transport layer.
Human Computation; Social Computer; Twitter • We present a conceptual design of our framework and
identify a number of use cases for human computation,
that can take advantage of the proposed approach.
1. INTRODUCTION
Today’s most successful crowdsourcing services such as The rest of this paper is structured as follows: Section 2
Amazon’s Mechanical Turk1 and CrowdFlower2 , share a com- introduces the terminology and core concepts of the frame-
mon characteristic: they are all based on centralized archi- work, as well as, its components and workflow. In Section 3,
tectures. In these services, both, users’ profile information we present a conceptual design of our approach that shows
and task distribution engine are centralized. its feasibility. Section 4 introduces several Use Case sce-
However, the Social Computer vision that we share is narios and presents how human intelligence tasks can be
more likely to be based on decentralized architectures [6], described using the MechSwarm Task Language. We discuss
like the ones provided by social networks and mobile de- current and future issues in Section 5. Section 6 presents
vices, where humans would interact via free information ex- related work. In Section 7, we conclude the paper. Finally,
change or trading to solve large-scale problems, that cannot Appendix A, includes basic terminology used in Twitter as
be easily addressed by conventional computer systems and a reference.
algorithms.
1
2. MECHSWARM FRAMEWORK
Mechanical Turk: mturk.com
2
CrowdFlower: crowdflower.com First, we introduce the key concepts of our proposed frame-
work MechSwarm. We borrow some terminology from Ama-
zon’s Mechanical Turk [1] and extend it in order to explain
our approach.
Copyright c 2012 for the individual papers by the papers’ authors. Copy- 3
ing permitted for private and academic purposes. This volume is published Twitter: twitter.com
and copyrighted by its editors.
CrowdSearch 2012 workshop at WWW 2012, Lyon, France
Human Intelligence Task (HIT): Requester
A fine-grained task such as, “Is this a picture of the Golden HIT
Gate Bridge?” or “In the video segment about sports or tech- HIT HIT
HIT HIT solved
HIT solved
nology?”, which can be easily performed by humans and it Problem solved Solution
can be rendered as part of a Web-based user interface.
HIT
1. HIT 6.
Contributor: solved
MechSwarm
A human being who is willing and able to perform a HIT. Human Computation
HIT-Solver
Each contributor has also a Twitter account that is used to 2. Optimizer
receive a HIT request and to reply with the solution. This
is equivalent to the concept of worker according to Ama- HIT HIT
zon’s Mechanical Turk terms, but we rather use the term : :
3. 5.
contributor instead, since is a more general concept, for ex-
HIT Request HIT Response
ample, volunteers which do not expect a monetary payment
for performing a given HIT are also considered contributors.
Requesters:
The individuals or organizations that need a set of HITs to
be done. Each requester has a Twitter account that is used
HIT HIT
to send HITs requests and receive HITs’ responses.
: :
HIT Assignment: HIT Request HIT Response
When a requester needs a particular HIT to be done, he uses
MechSwarm to assign it to a candidate contributor that will 4.
perform the task.
More formally, we define a Human Computation system
(HCOMP-system) as a triple (T, H, A), where
• T is called problem and corresponds to a set of Human
Intelligence Tasks (HITs),
Contributor
• H is a set of human candidates to perform a task t
(i.e.,contributors), and
Figure 1: MechSwarm Components and Workflow.
• A : T → H is a function that assigns each task t to a (1) A requester defines a problem (i.e., the set of
human A(t) ∈ H. HITs T ). Each HIT is defined using the MechSwarm
task language. (2) The requester submits the HITs
The solution to the problem T is denoted by Solution(T ). to the Human Computation Optimizer (HCO). The
Note that this definition does not impose any restriction HCO identifies for each HIT a contributor from the
on where the task submission, assignment, and completion requester’s Twitter Social Graph (i.e., from his fol-
takes place. lowers), and assigns him the HIT. (3) The HCO uses
MechSwarm provides (i) the selection of candidate con- the Twitter social network to route the HIT request
tributors H, (ii) a task assignment over this set (i.e., A) and to the contributor. (4) The contributor receives the
(iii) an aggregation mechanism to compute the final solution HIT, completes the assignment and issues a HIT
of the problem, i.e., Solution(T ). response to the requester. (5) The HIT-Solver com-
In the rest of this section we detail the different compo- ponent collects the HIT responses to the specified
nents of the framework and the system workflow. problem on behalf of the requester, and (6) com-
putes the problem’s final solution, i.e., Solution(T).
2.1 Components and Workflow
The MechSwarm Task Language, the Human Computation
Optimizer and the HIT-Solver are the fundamental compo-
The basic workflow of the system is shown in Figure 1. A
nents of the framework. They are specified as follows:
requester begins by defining a problem (i.e., T ) that can be
MechSwarm Task Language. The language used to split into several HITs easily tackled by humans. The HITs
specify the HITs and basic protocol for message ex- are expressed using the MechSwarm’s task language. The
change. requester submits the problem to the Human Computation
Optimizer, which selects a set of contributors as candidates
Human Computation Optimizer (HCO). The compo- to solve the HITs (i.e., H), and assigns to each of them a
nent that manages the HIT requests, contributor selec- task to perform (i.e., A).
tion and task assignment. Each contributor completes the HIT assigned and sends
back a response with the solution. The HIT-Solver collects
HIT-Solver. The component responsible to aggregate the the set of HITs completed and computes a final solution to
completed HITs and to compute a final solution. the problem, i.e., Solution(T ).
Function Character 3.2 Human Computation Optimizer (HCO)
Question ID start # The Human Computation Optimizer (HCO) is a core com-
Question ID end ? ponent in charge of managing HIT requests, contributor se-
Parameter delimiter & lection and task assignment. HCO exploits social proximity
Parameter terminator ! to assign HITs to contributors belonging to the requester’s
social graph.
Table 1: List of reserved characters of MechSwarm We are exploring more sophisticated methods for contrib-
Task Language to identify each part of the message. utor selection and task assignments, in particular we want
These reserved characters are configurable by the to (i) automatically identify the nature and semantics of the
developer. problem (e.g., HITs), and (ii) learn and keep a contributor
profile in order to optimize the task assignments according
the capabilities of each contributor.
In Figure 1, we can observe that the HCP and HIT-Solver 3.3 HIT-Solver
components run on the requester’s infrastructure, and not
on a centralized system. As a consequence, requesters’ pro- We consider each HIT as part of a problem. The solution
file information remains private and does not need to be of the problem does not only imply to solve each HIT, but
disclosed to third parties. also to produce an aggregated result, or a meaningful com-
In the next section we present an instance on how to re- bination of the output produced by individual HITs. For
alize these concepts. example, in order to translate into Spanish a text document
written in German, we could split the document into para-
graphs, and then create and assign a HIT to a contributor
3. CONCEPTUAL DESIGN requesting the translation of each of them. The final solu-
In this section we demonstrate the feasibility of the pro- tion corresponds to the result of each HIT plus the ordering
posed decentralized framework. We present and discuss how of the translated paragraphs, with respect to the original
each component can be realized. document.
The final step of computing the aggregated solution of a
3.1 MechSwarm Task Language problem is performed by the HIT-Solver.
One crucial point in distributing tasks among many con-
tributors is to make sure that they are familiar with the cho- 4. USE CASES
sen language (i.e., protocol ) to communicate with the frame- We reserve this section to expose a list of use cases (UC),
work. If the communication channel and the communication encompassing several human computation tasks, that can be
languages are not coordinated, any human computation is effectively solved using our framework. We use the Twitter
in vain. To this end we propose a response formatting that account “MechSwarm” in our discussion below.
is short and simple to use, is familiar to Twitter users and
is customizable. The basic format is a tweet containing a UC1: Pairwise Comparisons. The framework is pre-
Twitter mention. to the framework, followed by the iden- pared to listen to all tweets that mention the account (@Mech-
tification of a HIT, followed by the the choices from a list Swarm) and, if required, to acknowledge the received re-
of possible answers. For basic terminology used in Twitter, sponse. Additionally, HIT-Solver computes the final solu-
please refer to Appendix A. tion based on the HIT responses received. The framework
In Table 1 we list the predefined character delimiters used logs all responses received, in order not to processing them
in the MechSwarm Task Language. Note that the only char- more than once.
acter that is not customizable is the Twitter reserved symbol
(@), used for mentioning. In the end, a request and response Request: “@Contributor
should be formatted as follows: Which one is your favorite search engine?
#favSearch?Google&Yahoo!”
• Request Template:
@#? Response: “@MechSwarm #favSearch? Yahoo!”
&&. . . &!
UC2: Sound Verification. The framework can be used
• Response Template: to confirm results from unsupervised methods as automatic
@#? tagging images, videos or sounds.
&&. . . & !
Request: “@Contributor
Please note that the list of choices in the request is op- Is http:///example.com/sound.mp3 a bird?
tional. Furthermore, observe that complex and massive task #soundHIT?yes&no!”
definition require additional software tools, e.g., to select
from a database the set of questions to be asked in a ques- Response: “@MechSwarm #soundHIT? yes!”
tionnaire, but the basic idioms presented in this section can
even be input directly by the requesters.
The request and response length is restricted to 140 char-
acters, given the message limit imposed by Twitter. Con-
crete examples of HITs, specified using the MechSwarm Task
Language, can be found in Section 4.
UC3: Image Tagging Additionally, yet another applica- the functionality provided by tools like TurKit [4] using our
tion is to provide means for contributors to add correct hu- framework?
man judged metadata to resources. In particular, we are interested in use case scenario UC1:
Pairwise Comparisons, which is at the core of learning to
Request: “@Contributor Tag image rank and collaborative filtering algorithms, which can be
http://example.com/picture.png #tagImageHIT?” realized using a decentralized crowdsourcing workforce.
Response: “@MechSwarm #tagImageHIT? 6. RELATED WORK
dog&animal&nature!”
When talking about Human Computation, there are two
main concepts that come in mind: crowdsourcing and Games
UC4: Near Duplicate Detection For the task of video
With A Purpose (GWAP) [10]. Crowdsourcing is the act of
duplicate detection, the contributors could access a simple
gathering together the solutions performed by large groups
interface displaying two videos and two buttons (“yes” and
of people over some specific task. Today the most prominent
“no”). Once the contributor clicks on one of the buttons this
human computation application for is Amazon Mechanical
triggers his Twitter account to post the formatted message
Turk, a marketplace for crowdsourcing. Amazon Mechanical
understandable by the framework.
Turk works as a platform to coordinate (humans) to perform
simple tasks that usually computers cannot, in exchange for
Request: “@Contributor Are these terms/videos the same?
monetary rewards.
http://example.com/V1V2/ #matchVideoHIT?yes&no!”
Games With A Purpose, or human computation games,
exploit the idea of having human players to compete in solv-
Response: “@MechSwarm #matchVideoHIT? yes!”.
ing problems. Many domains have profit from the GWAP
approach, mostly annotation of images and music [12, 3] and
UC5: Translation The requesters can post HITs that re-
also collecting common sense facts [11, 7].
quire sentences to be translated to a certain language.
Another great example that exploits Human Computa-
tion is the reCAPTCHA project4 , which provides a captcha
Request: “@Contributor Translate to Portuguese:
service that is primarily used to identify whether is a hu-
Hello world #translateHIT?”
man accessing some online content, and at the same time,
collects the feedback to correct words in digitalized books
Response: “@MechSwarm #translateHIT? Ola’ Mundo!”.
that optical character recognition (OCR) programs fail to
recognize with certainty.
5. DISCUSSION AND FUTURE WORK First, like Amazon Mechanical Turk we propose a frame-
Twitter aggregates millions of users that are intercon- work that defines workflows and terminologies for modeling
nected through follower/followee ties. The users interactions human computational tasks. Second, from GWAPs we share
in Twitter, using mobile devices, open the opportunity to the motivational power embedded in games and social net-
achieve large scale human computations, similar than the works to leverage the distribution and completion of tasks.
ones performed in centralized crowdsourcing systems, with Lastly, like reCAPTCHA, instead of forcing users to search
the additional benefits of contributor’s social ties and real- for tasks, we bring the tasks to the users, using the Twitter’s
time information exchange. nature of pushing notifications.
Regarding the monetary motivation supported by crowd- Like a crowdsourcing marketplace, our goal is to support
sourcing systems like Amazon Mechanical Turk, we think Human Computation, but we propose a decentralized ap-
that alternative decentralized trade spaces for human com- proach based on Twitter’s architecture and social graph,
putation are possible, where rewards and incentives to indi- which is quite different from the aforementioned works.
viduals do not necessarily involve a monetary payment for
their contributions. Clear examples exist of such spaces that 7. CONCLUSION
support our vision, Wikipedia, for instance, can be consid-
ered as a massive human computation task of knowledge In this paper, we introduced a decentralized human com-
gathering, where the vast majority of contributors does not putation framework, MechSwarm, that exploits Twitter’s so-
receive money for their efforts, but are motivated by intrinsic cial network to harness the problem solving power of human
rewards that comes from work achieved itself [5]. The de- intelligence.
centralized framework we discussed here is flexible enough The framework does not require a centralized system to
to also incorporate monetary rewards if it is required, but manage task submission, assignment, and completion, but
the contract would be established directly by the requester has the potential to empower individuals and organization
and contributors, without any intermediaries. to distribute tasks across large number of human contribu-
Our work opens the door to interesting future directions. tors over Twitter’s social graph. We presented a conceptual
One interesting question is: how to exploit plurality for design and explore the feasibility of our approach in the light
error-resilient HIT solving? Additionally, one issue to be ex- of several use cases.
amined is how the public HIT responses from one contribu- We envision that decentralized architectures for human
tor influences others. It is a reasonable assumption that any computation will emerge as viable alternatives to well es-
suggestion or recommendation before the execution of a task tablished crowdsourcing services. Our approach is a small
may bias its outcome, thus should be empirically verified. step towards realizing this vision.
We plan to deploy a live implementation of our framework.
We want to explore how can more complex tasks be solved
4
using the basic idioms we proposed, is it possible to achieve reCAPTCHA: google.com/recaptcha
8. REFERENCES [11] L. von Ahn, M. Kedia, and M. Blum. Verbosity: a
game for collecting common-sense facts. In CHI’06,
[1] J. Barr and L. F. Cabrera. Ai gets a brain. Queue, pages 75–78, 2006.
4:24–29, May 2006.
[12] L. von Ahn, R. Liu, and M. Blum. Peekaboom: a
[2] Edison Research. Twitter usage in america: 2010. game for locating objects in images. In CHI’06, pages
http://www.edisonresearch.com/, 2010.
55–64. ACM Press, 2006.
[3] E. Law, L. von Ahn, R. Dannenberg, and
M. Crawford. Tagatune: a game for music and sound
annotation. In Proceedings of the 8th International APPENDIX
Conference on Music Information Retrieval (ISMIR Appendix A: Basic Twitter Terminology
2007), 2007.
[4] G. Little, L. B. Chilton, M. Goldman, and R. C. • Tweet: A message posted via Twitter containing 140
Miller. Turkit: human computation algorithms on characters or fewer.
mechanical turk. In Proceedings of the 23nd annual • @: The @ sign is used to call out usernames in Tweets.
ACM symposium on User interface software and • Mention A mention is any Twitter update that con-
technology, UIST ’10, pages 57–66, New York, NY, tains @username anywhere in the body of the Tweet.
USA, 2010. ACM.
• Follower: A follower is another Twitter user who fol-
[5] M. Poppendieck. Unjust desserts? Better Software,
lows a specific account.
pages 33–47, July/August July/August 2004.
[6] D. Robertson and F. Giunchiglia. The social • Followee: Reflects other Twitter users that a specific
computer: Combining machine and human account chose to follow.
computation (DISI-10-036). Technical report, • Lists: Curated groups of other Twitter users. Used
University of Trento, 2010. to tie specific individuals into a group on your Twitter
[7] P. Singh, T. Lin, E. T. Mueller, G. Lim, T. Perkins, account.
and W. L. Zhu. Open mind common sense: Knowledge • Reply: A Tweet posted in reply to another user’s mes-
acquisition from the general public. In On the Move to sage, usually posted by clicking the “reply” button next
Meaningful Internet Systems, pages 1223–1237, 2002. to their Tweet. Always begins with @username.
[8] C. Taylor. Social networking ‘utopia’ isn’t coming.
CNN Tech. http://goo.gl/emF5j, June 2011. • Retweet: A Tweet by another user, forwarded by
someone else.
[9] @twittereng. 200 million tweets per day. Twitter Blog.
http://goo.gl/eybp0, June 2011.
[10] L. von Ahn. Games with a purpose. IEEE Computer,
39(6):92–94, 2006.