When a group is traveling together it is challenging to recommend an itinerary consisting of several points of interest (POIs). The preferences of individual group members often diverge, but it is important to keep everyone in the group satisfied during the entire trip. We propose a method to consider the preferences of all the people in the group. Building on this method, we design explanations for groups of people, to help them reach a consensus for places to visit. However, one open question is how to best formulate explanations for such sequences. In this paper, we introduce TourExplain, an automated crowdsourcing pipeline to generate and evaluate explanations for groups with the aim of improving our initial proposed explanations by relying on the wisdom of crowds.
• Information systems → Crowdsourcing; Recommender systems; • Human-centered computing → Natural language interfaces; Empirical studies in HCI;
Recommender systems are decision support systems which help users to find one or more items in a large space of possible options that best fit their wishes and needs. The main focus of current recommender systems is to propose items to individual users. However, in tourism people often consume several items, and often do so in groups rather than individually.
A group traveling together can be recommended an itinerary
consisting of several points of interest (POIs). However, reaching a
consensus is dificult, and often compromises need to be made. Such
compromises can potentially help users expand their tastes. Mary’s
preferred POI may become John’s new favorite spot! Compromises
can also lead to rejection of the recommended items. One way to
avoid this is to explain recommendations that are surprising; or
even expected to be disliked; by an individual user [
A group traveling together can be recommended an itinerary
consisting of several points of interest (POIs). To keep the group
satisifed during the entire sequence of recommendations (e.g., POIs), we
need to consider the preferences of all the people in the group [
Ardissono et al. [
Masthof et al. [
Crowdsourcing is a practice for solving computationally hard tasks
by assigning them to an undefined (and generally large) network of
people in the form of an open call, usually through online platforms
(Mechanical Turk1, FigureEight2, etc.). This can take the form of
peer-production (when the job is performed collaboratively), but
is also often undertaken by sole individuals (crowdworkers) [
Similarly, other authors proposed a model to generate
personalized natural language explanations in the movie domain [
Bernstein et al. [
This paper builds on the Find-Fix-Verify design pattern used in
Soylent [
A group of people can use the TourExplain system when going on a trip. The group creates a new "Trip" in the system and enter trip parameters (i.e., POIs to be considered, number of participants, and whether the explanations need to be anonymous or not). Following the creation of the trip in the system, each member of the group has to enter their own preferences for each POI (in a private environment). After all of the preferences have been submitted, the system generates an itinerary, or a sequence of POIs, for the group, as well as explanations. Each explanation is then posted to the crowdsourcing system to be improved as described in Section 4.2 "Subsystem 2: Crowdsourcing". Once the crowdsourcing part of the system completes, each user will be received the recommended itinerary and its corresponding explanations. 4
Generate sequences. Here, the system generates a sequence of
POIs for the group to visit, according to previously proposed
preference aggregation algorithms [
Following are the two above mentioned algorithms that we used
to generate itineraries. Readers who wish to get a coherent overview
of the proposed algorithms is referred to our previous work [
A 2: Fairness -> Average. The arrow implies applying a tiebreaking strategy, i.e., when several items receive an equal score using only Fairness.
Generate explanations. Pure crowdsourcing approaches to explain the preference aggregation strategies used to generate the sequence of recommended items will not succeed because most crowdworkers are not domain or recommendation experts. Even if they are informed about applied algorithms we cannot expect a crowdworker to write an appropriate explanation for the recommended items. Therefore, we provide them with initial explanations in the beginning which they can improve based on specific criteria.
Using a template-based natural language generation approach, the system generates explanations for each user according to their preferences in the recommended sequence. These (personal) explanations are based on predefined templates, examples of templates are "Hello X", "we know you would love to see Y", and "however, others in your group would love to see Z".
For example, we consider a user John who has expressed a liking for seeing the Eifel Tower because John and a couple of friends are visiting Paris soon. However, John’s friends have expressed they preferred seeing the Louvre over the Eifel Tower. This could lead to a template based sentence: "Hello John, we know you would love to see the Eifel Tower, however, others in your group would love to see the Louvre first."
The system is provided with a number of templates to handle a number of predefined situations considered by the explanation generating algorithm. These automatically generated explanations are then sent to the second part of the system via an API to be reviewed by crowdworkers.
An example scenario for when an explanation may be needed is when a POI that is highly rated by person A is not chosen in the sequence of recommended POIs. The explanation for this person can be: "Even though you wanted to visit POI X, most of your friends gave a very low rating for that POI. Therefore, we did not include that into the recommended POIs for the group." 4.2
The aim of the crowdsourcing subsystem is to improve the aforementioned generated explanations by using the wisdom of crowds.
We employ the Find-Fix-Verify pattern as described by [
Guidelines for Find-Fix-Verify. We give our workers three main
criteria (based on Gricean Maxims [
Find & Fix tasks. A crowd-worker (worker henceforth) is given an explanation and asked to find any shortcomings in terms of the criteria mentioned above. After that, the worker is asked to make a suggestion to improve (fix) the sentence.
Verify task. A worker is given an explanation that is fixed by another worker in the find-fix step to evaluate in terms of the criteria mentioned above. The worker is asked to verify each criterion on a binary scale, giving their approval or disapproval for the particular metric. When the majority of the workers approve at least two criteria for a given explanation, it is considered a satisfactory explanation. Based on the number of approval/disapproval ratings, these satisfactory explanations are ranked from best to worst. A vital part of our system is that the workers who do the FindFix versus Verify steps are independent of each other. This ensures there is no bias in picking a particular explanation. The tasks are created and launched using the Figure Eight API 3.
Unlike the previous generation of explanations, the crowdsourcing part cannot be done in real-time, but it requires some more time to be done. This is due to the fact that is not possible to know when the tasks will be performed by the workers. This time is subject to multiple factors as the monetary reward for each task, or the number of workers that perform the same task. In fact even though is possible to estimate the time to perform a given task by a worker, it becomes complex to estimate when a launched task will be picked up by a worker, also this time is directly related with the monetary reward for the task.
To ensure data quality, we only select workers that are native English speakers. When it was possible we randomized the order of questions and answers to avoid possible bias. Furthermore, to limit the introduction of error by the workers we performed each step by multiple workers. The number of workers that perform the same step can be dynamically chosen. 5
We plan to use this pipeline as the basis of doctoral work
investigating how to best generate explanations for itineraries (sequences of
POIs) for groups of users. For this purpose, as suggested by Kim et al.
[
In the following sections we describe future research avenues that will be pursued in this project. We introduce the notion of group dynamics, which consider the relationship between people within a group. We also consider the influence of interaction design on the requirements for explanations. 3https://www.figure-eight.com/, retrieved June 2018
Existing group recommendation techniques usually focus on merely
aggregating individual preferences and thus do not take into
account social interactions and relationships among the group
members. Previous work has found that it is not the case, i.e., group
members are influenced in their evaluations by the combination
of the group and the interaction between and social relationships
among group members [
In order to personalized the preference aggregation algorithms
and their corresponding explanations as well as make our
recommendations group-aware, we plan to use the Thomas-Kilmann
Conflict Style Model (TKI model) [
Another important group aspect that we aim to consider is the
types of relationships within groups (c.f., [
• Communal Sharing: Somebody you share everything with • Authority Ranking: Somebody you respect highly • Equality Matching: Somebody you are on equal footing with • Market Pricing: Somebody you do deals with / compete with We can employ the group types in both aggregating preferences algorithms as well as designing explanations. For instance you might feel comfortable to reveal your preferences to somebody you are on equal footing with (such as your friends) but not with somebody you respect (such as your boss).
Individual vs Group Explanations. In this work we tried to improve automatically generated explanations by using the wisdom of crowds for a single user. However, we did not evaluate the final result with real groups of users. In our next steps, we will evaluate these explanations by presenting them to groups and compare the results with individual personalized explanations (for each group member). One can expect to find a trade-of between explanations that are suitable for the whole group, compared to personal explanations for each group member. For example, one benefit of group explanation is that we can present it on a common device viewed by the whole group. On the other hand, personal explanation can supply individual users with more personalized information about why that item is recommended to them.
Transparency vs Privacy Preserving. The requirements on explanations are also likely to be influenced by group versus individual preferences. For instance, there is a trade-of between having a high transparency while not violating the users’ privacy. So users might demand to conceal their preferences for other group members or they feel comfortable to reveal their preference depends on diferent types of groups or their personalities.
Single Item vs Sequence Explanations. In this work we provided an explanation for each single item. However it might not be convenient always depending on several things e.g., domain. For example, in the cinema domain, users would want a recommendation for a specific movie instead of a sequence whereas in the tourism domain, a sequence of POIs would be more appropriate. In our future work, we will design explanations for each POI, and compare them with an explanation for the whole itinerary.
In this paper we introduce an automated crowdsourcing pipeline to generate and evaluate explanations for groups. The proposed solution is suitable for domains where items are a) consumed in groups, and b) in a sequence. This particularly useful for the recommendation of itineraries in tourism.
Additionally, it is likely that the approach is extendable to other domains, however there is a constraint for domains which require immediate and real-time explanations. For tourism, where trips can be planned in advance of a visit, this limitation may be less severe.
While simple, the proposed approach can be extended to answer diferent research questions. In this position paper we highlighted two significant and planned extensions: • Group dynamics. How can explanations be improved by taking in account group dynamics such as conflict style or relationships within groups? • Interaction Design. How should we adapt the explanations to the way they are consumed, e.g., for an individual item or for a sequence? Or for a single user versus for the group?