When buying a mobile phone, booking an hotel, or watching a movie, many people rely on the reviews available on the Web. However, this huge amount of opinions make it di cult for users to have a comprehensive vision of the crowd judgments and to make an optimal decision. In this work we provide evidence that automatic text summarization of reviews can be used to design Web applications able to e ectively reduce the decision making e ort in domains where decisions are based upon the opinion of the crowd.
People's decision making is ever more in uenced by the \voice of the crowd". When choosing an hotel, a restaurant or a movie many people rely on the reviews available on the Web, trusting other users' opinions more than the \o cial" ones by critics, guidebooks, experts, or similar. At the same time, we are witnessing a proliferation of user generated content. For instance, Amazon collects up to 16,000 new reviews per day1, while TripAdvisor has recently passed the 150 million mark in terms of opinions posted to its site, now collecting more than 90 user contributions per minute2.
This huge mass of web based user judgements encapsulate a potentially
reliable \ground truth" that, in principle, can drive the decision making process.
Yet, too many pieces of information make it di cult for decision makers to get a
comprehensive vision of the crowd judgement and its global polarity. The goal of
our research is to help users overcome this problem. Bounded Rationality
Theory [
optimal solution, they tend towards a suboptimal result after having greatly simpli ed the choices available. In line with this theory, our approach is to decrease user's e ort by reducing the amount of information to be cognitively processed, o ering users only a distilled vision of the voice of the crowd. More precisely our approach is to present summaries of all available reviews that capture the salient aspects of the crowd's judgement and are appropriate to build at least suboptimal solutions.
Summarization is emerging as a topic of growing importance. Summly3, a
news summary service designed to help simplify the way users consume news
on mobile devices, was recently acquired by Yahoo for 30 million dollars. In
comparison with this service, as well as with other academic works [
We have performed our studies in the domain of restaurant booking. We conducted two separate user experiments: the rst to chose the best summarization technique for the target domain, the second to demonstrate the e ectiveness of the summarization technique as a tool to reduce the decision making e ort.
The rst research question we wish to answer is to assess which technique
among the proposed ones generates the summary which better represents the
entire set of reviews. To answer this question, we performed an online study
with 33 participants. For each participant: (i) a random restaurant is picked
among the entire dataset; (ii) the restaurant's reviews are provided to the user
alongside with the 12 summaries (two for each of the six techniques, one for the
positive opinions and one for the negative ones); (iii) the user is asked to rank
the summaries based on his opinion on how well the summary summarizes the
reviews. The best technique according to this study is Edmunson [
In a second user experiment we studied if polarized summarization technique provides bene ts for the user decision making process. For this purpose we have developed a web application that mimics the functionality of TripAdvisor, but limited to the restaurants in New York City. Each user involved in the experiment was asked to simulate the booking of a restaurant. Users were randomly split into two experimental conditions: (i) with original TripAdvisor reviews (Figure 1a) and (ii) with positive and negative summaries only (Figure 1b). A total of 108 users participated to this study.
In order to estimate the decision making e ort, we measured the the average time each user spent in reading the description pages of restaurants. Users in
Decision Making through Polarized Summarization of User Reviews the second experimental condition { with summaries of positive and negative reviews for each restaurant { spent, on average, half of the time with respect to the users in the rst experimental condition { with the full set of reviews. 3
The results from the second user experiment con rm that the summarization of reviews e ectively reduce the e ort of users in the decision making process.
Although this is only a preliminary study, its internal validity is supported by the accuracy of the research design. In terms of external validity, the applicability of our results might not be con ned to the speci c domain of restaurant booking, as in many decision making domains users rely their decision on the opinion of the crowd.
In order to provide better evidence of the e ectiveness of summarization as a decision making tool we need to strengthen our study by (i) providing a better engagement for users, (ii) by collecting data from a larger user base and (iii) by collecting other metrics to measure the e ectiveness of the decision making process. Romantic escape 5/5 My husband surprised me with a dinner out at Little Owl during our recent trip to NYC and it is a date that I will not soon forget. The atmosphere is wonderful, with low, warm lighting and the buzz of conversation.
Overpriced and overrated 2/5 Overpriced and overrated...Tryin to be like my spot Kitchen, but not as cool don’t expect too much 3/5 although the little owl might get rave reviews, i find the food very average. the best thing about it is that the service is down to earth and there is the cutest red door. not somewhere i’d go out of the way for, but decent. 3,64 % of people agree with: We went for a Sunday Brunch. The wait time was over 30 min. and when we were finally seated, our table was assigned to a very rude waitress who looked rather surprised that it was our first visit and we were clueless about their menu and hence was asking about some of the dishes (the names are European, so we couldnt quite figure out the dish from the names). Finally when the order came, the portion sizes were so tiny that we were thinking of heading to another restaurant to complete our meal. And they don’t have a dessert menu for brunch!! Super service&staff! Tryin to be like my spot Kitchen, but not as cool. 96,36 % of people agree with: My husband surprised me with a dinner out at Little Owl during our recent trip to NYC and it is a date that I will not soon forget. The place is super small so we were very lucky. A short menu prepared from a cupboard sized kitchen means everything was super fresh. So nice to experience such great service in a super busy city!. Nice cocktails and super service. The servers are super friendly and welcoming in a casual sort of way, you almost feel as if you’ve been invited over to their house to eat, its a good vibe all around. It was syrupy and super vanilla driven, and was great with the cheese.