Even though search systems are very e cient in retrieving world-wide information, they can not capture some peculiar aspects and features of user needs, such as subjective opinions and recommendations, or information that require local or domain speci c expertise. In this kind of scenario, the human opinion provided by an expert or knowledgeable user can be more useful than any factual information retrieved by a search engine. In this paper we propose a model-driven approach for the speci cation of crowd-search tasks, i.e. activities where real people { in real time { take part to the generalized search process that involve search engines. In particular we de ne two models: the \Query Task Model", representing the metamodel of the query that is submitted to the crowd and the associated answers; and the \User Interaction Model", which shows how the user can interact with the query model to ful ll her needs. Our solution allows for a top-down design approach, from the crowd-search task design, down to the crowd answering system design. Our approach also grants automatic code generation thus leading to quick prototyping of search applications based on human responses collected over social networking or crowdsourcing platforms.
While search systems are superior machines to get worldwide information, people tend to put more trust in people than in automated responses. That is why often users seek for opinions collected within friends and expert/local communities for taking an informed decision about signi cant Copyright c 2012 for the individual papers by the papers’ authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
CrowdSearch 2012 workshop at WWW 2012, Lyon, France MacroTask Description (BPMN)
M2M Transformation MicroTask Description (BPMN)
M2M Transformation
User Interaction Model (WebML) Stand-alone application
M2T Transformations Application embedded in social network or crowdsourcing platform issues. Other users' opinions can ultimately determine our decisions. While in the past people could rely on opinions given by close friends on local or general topics, the change in the social connections in our society makes users increasingly rely on online social interaction to complete and validate the results of their search activities. People often search for human help in between canonical web search steps: they rst query a search system, then they ask for an opinion on the result, maybe they also ask suggestion on the query term. We de ne this trend as crowd-searching.
In current Web systems, the crowd-search activity, i.e. looking for opinion from friends or experts, is detached from the original search process, and is often carried out through di erent social networking platforms and technologies. Moreover, people manages di erent applications, di erent virtual identities and maybe also di erent devices: they send email, ask on Twitter, Facebook or other social network, or ask to friend and people they know.
Recent works (see section 4) on crowd-based search focus on simple and atomic task, while crowd-sourced search involve a wide range of scenario, from trivial decisions, like choosing where going to eat at dinner, to more serious things like organizing a travel or even buying a house. Thus the user need a way to manage and control the whole process, from the creation of the query, the selection of the target to the gathering of the results.
In this paper we propose a model-driven, platform
independent, approach to design Web applications that support
crowd-sourced search. We de ne a top-down design
approach, as sketched in Figure 1 which applies model-driven
engineering (MDE) techniques for the speci cation of the
crowd-sourced information collection task, its splitting and
re nement, and its mapping to the Web user interaction
N
1
FieldInstance 1
value: String
speci cation. The approach starts from the task description
and applies model-to-model transformations to build the
detailed task de nitions (described e.g. in BPMN) and then
the platform independent user interaction model (described
in the domain-speci c language WebML[
The main ingredients that participate to our contribution are: 1) a metamodel of the crowd-sourced question; and 2) the models of the user interfaces needed for de ning the questions and for responding. In this short paper we focus on the aspects related to the model-driven design of the crowd-search user interactions, spanning from the question de nition to the engagement, dissemination, and ending in the response submission and collection. On the other side, we consider the task re nement and redesign problem as outside the scope of this short work.
The paper is organized as follows. Section 2 and Section 3 respectively describe our search task meta-model and user interaction model; Section 4 summarizes the related works for both the crowd-sourcing and the model-driven elds; and nally, Section 5 concludes.
The starting point of our MDE approach is the query task model. Figure 2 shows the query task meta-model to which every query task should conform. The main element is the Query submitted by a User. The Query is de ned by a Question, written in natural language, and a list of CrowdObject s, i.e. information structured according to a given schema.1
A question includes a set of Input CrowdObject s, i.e., a set of data in the user's question upon which the responder can apply his response. For example, if the user wants to collect opinions about some restaurants in Lyon, the Input CrowdObject instances comprise the restaurants subject to the comparison. The input object can be either inserted manually at query creation time by the user or extracted from a previous search (both canonical or crowd-based) step. The model of these objects is de ned by the Schema element. Input objects are not mandatory for the creation of a query, as a user can create an open question. However, we always assume the presence of a Schema. 1To ease the discussion, we assume that information is structured in relations; however, other formats (e.g. semistructured, graph, etc.) are also suitable.
The type of the query de nes how a user can answer to
the question. These have been classi ed in a taxonomy [
Finally, the Query is also related with a set of Output CrowdObject s, representing the answers to the question submitted by the crowd.
Users of a crowd-search task can be classi ed into two categories: askers and responders. The former is the user using the platform and creating questions to be submitted to the crowd, while the latter is a user involved in the query answering process using the social network or crowdsourcing platform.
Relation represents associations that can exist between
CrowdObject s. These relations can be either an Input-Input
relation or a Output-Input relation. They are created when
a query is split into sub-queries and depend on the kind of
splitting pattern that is applied. Indeed, starting from the
design of the coarse-grained task, one can re ne its
description by structuring its activities according to known
crowdinteraction patterns (ie.g., nd- x-verify [
Input-Input relations occur when the initial set of inputs
is partitioned across di erent instances of the same query,
to reduce the workload of the responder. For example, if the
original query would ask the responder to order one hundred
restaurants, it can be useful to split the task into subtask
of ten restaurants each, to be assign to di erent responders.
In this case the task performed by each responder is the
same, but it is applied on di erent sets of objects. The
initial set can be is therefore partitioned into the di erent
query instances, according to di erent strategies (e.g. , in
a uniform way or according some properties of the input
instances). The input of the new query instances are thus
mapped to the inputs of the original query, according e.g.
to a map-reduce pattern [
Output-Input relations occurs when the task requested by
the author of the query is complex or di cult, or if the
result require some kind of validation, which therefore requires
organizing the task into a sequence of subtasks [
The user interaction model describes the interface and navigation aspects of the crowdsearch application. Starting from the query task model, possibly split in a complex pattern of microtasks, a model transformation can lead to a coarse user interaction model, which in turn can be manually re ned by the designer. The user interaction must cover three fundamental phases of the crowdsearch process: the submission of the question (performed by the asker); the collection of the responses (performed by the responder); and the analysis of the results (available to the asker for getting insights).
At the current stage, our research has identi ed the
interaction patterns relevant for each phase, considering the
various options of deployment platform, task type, and
macrotask splitting pattern. For space reasons, in this section we
report one possible outcome of the user interaction design,
in case of simple query task and of deployment on the
Facebook social networking platform. We describe the phases
of query creation and of query answering, according to the
WebML notation [
Figure 3 shows the user interaction model for creating and submitting a query, according to the WebML notation. In the Create Query page, the user speci es the textual question (e.g., \What's the best museum to visit in Milan?") and sets the query type (e.g., \Like", \Add", and so on). The user can also choose the type \open question", thus assuming that no items are needed in input for the responder to select/like and so on. In both cases, an Query instance is created, and its type is set. If the query does not have inputs, then the user is directly brought to the Responder Selection page.
If, on the other hand, the user chooses to build a structured question with inputs, then he is redirected to the \Dene schema" page, where the asker can create a schema for inputs by assigning a general name to the input type and by de ning its attributes in terms of name and type. By submitting the form, the application creates a new instance for the Schema entity and its associated Fields. The asker is brought to the Add Instance page, where he can add input objects following the schema previously de ned. The speci ed instances of the Input are created and linked to the query.
Finally, in the Responder Selection page the asker can select the responders to the query: the list of possible responders is retrieved from the social network or crowdsourcing platform (in this example, the GetFriend component collects the friends from the Facebook platform). The user can select the responders through the \Friends" multi-selection list in the page. Eventually, after viewing a preview of the created question, the user can post the query on the social platform.
Figure 4 shows a compressed view of the web pages that are produced starting from the user interaction models described in Figure 3, thanks to the code generation facilities of WebRatio. The structure and content of the pages can be easily recognized and mapped to the corresponding model elements. In this particular example the user wants to know some good restaurants in Milan. Hence she de nes the question \Can you suggest me some good restaurants in Milan?" and selects the \Add" query type. Then she creates the schema the input instances of the question must conform to. In the instance list she adds a restaurant she already knows. Finally she selects the recipients of the question from the list of her friends extracted from Facebook 3.2
Figure 5 depicts the WebML model for the query answering activity, performed by a responder based on the query structure de ned by the asker. When accessing the application through the Responder Dashboard page, the responder is presented with a list of questions to answer. By clicking on a question, he is brought to the Details page, where he can provide his answer. The page shows the question text, plus the set of de ned input instances (Input component in the Details page).
Depending on the type of the question (de ned by the asker de ned during the query creation phase), di erent concrete user interfaces can be shown: in the case of a \Like" question, the responder simply selects the preferred instances in the Input list; as a consequence, a set of Output objects are created corresponding to the \likes" of the user.
In the case of \Comment" or \Add" question, the user is shown a form to respectively write a comment or add a new instance to the list. In the case of \Comment" questions, an Output object with the comment schema (i.e. a single textual eld) is created. The \Add" case is more noteworthy, as the Output objects will present a schema equivalent to the Input ones, so to add the new object instances to the list of input object of the query.
Figure 6 shows the compressed view of the \Details" page built from the user interaction model described in Figure 5. Continuing the previous example, in this page the responder of the question can add additional restaurants he knows.
This work falls into the broad eld of human
computation, i.e., the discipline that aims to use human knowledge
to ful ll tasks that are di cult or even impossible for a
machine. For example, human computation studies have been
done about using crowd's knowledge for image recognition
[
A very important aspect of a crowd-based search is the
quality of the results and the response time, therefore
several works have addressed the problem of understanding how
design features (for example the cost of the task) impacts
on these results metrics [
The novelty aspects of our approach with respect to the
existing works include: independence with respect to the
crowdsourcing platform (in particular, we allow to exploit
indi erently a social network or a crowdsourcing marketplace
of choice); model-driven design of tasks and user
interactions; model-transformation based approach that partly
automates the generation of some models, thus reducing the
cost of designing new applications; and possibility of
manually or automatically choosing the responders to a query
task. Our work can be seen as an extended social
question answering approach (as applied in Quora and other
well known platforms), where the asker has greater exibility
in de ning and sharing his questions. Our work addresses
the problem of de ning crowdsourcing tasks at the modeling
level, while existing approaches and tools typically allow for
a programming approach to the problem (e.g., see TurkIt
[
Our work is based on general purpose model-driven
techniques and on our previous work on Web application design
[
CONCLUSIONS AND FUTURE WORKS In this paper we presented a model-driven approach for crowdsourcing responses to questions. We de ned a metamodel of the query taks and a user interaction model for building and answering to a query. We apply model-driven techniques to the design of the various aspects of the query tasks and to the transformations among them.
Ongoing activities are addressing the problems of task splitting and automatic model transformations, so as to implement a model-driven approach to the design of the tasks, considering the structured crowdsourcing patterns identi ed in literature. For the future we plan to extend the coverage of the deployment to several social and crowdsourcing platforms and integration of the potential responders base from several platforms at a time.
This research is partially supported by the Search Computing (SeCo) project, funded by European Research Council, under the IDEAS Advanced Grants program; by the Cubrik Project, an IP funded within the EC 7FP; and by the BPM4People SME Capacities project. We thank all the projects' contributors.