FULL PAPER Proceedings of the ACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces (UCERSTI), Barcelona, Spain, Sep 30, 2010 Published by CEUR-WS.org, ISSN 1613-0073, online ceur-ws.org/Vol-612/paper5.pdf Information Overload and Usage of Recommendations Muhammad Aljukhadar Sylvain Senecal Charles-Etienne Daoust HEC Montreal HEC Montreal Cossette Communication 3000 Cote-St-Catherine 3000 Cote-St-Catherine Canada Montreal, Canada H3T2A7 Montreal, CanadaH3T2A7 1-514-340-6980 1-514-340-7012 1-514-340-6980 charles-etienne.daoust@hec.ca Muhammad.aljukhadar@hec.ca Sylvain.senecal@hec.ca ABSTRACT decision support systems are heuristics that partly alleviate This research examines the antecedents of information overload processing effort while maintaining an acceptable level of choice and recommendation agents’ consultation and their effects on accuracy [10]. Xiao and Benbasat [28 p. 137] recently provide an reactance and choice quality. We propose that information extensive review of the RA literature, and conclude that “by overload and the user need for cognition affect the tendency to providing product recommendations based on consumers’ employ decision heuristic (consulting a recommendation agent) preferences, RAs have the potential to support and improve the and shape the user reactance to recommendations. A fully quality of the decisions consumers make when searching for and randomized experiment with different levels of information loads selecting products online as well as to reduce the information that involved 466 individuals with the task of choosing a laptop overload facing consumers and the complexity of online and the option to consult a recommendation agent is performed. searches.” This explains why 40% of retailers plan to integrate Results show that users opted to consult the recommendation some personalized recommendations on their e-stores [6]. agent more as information loads and as perceived overload While research studied various designs of recommendation increases and that product recommendations were salient in agents, it has not investigated the factors triggering consumers to enhancing choice, particularly when the information was less consult the recommendations nor the cases where product diagnostic (for choice sets with proportional distribution of recommendations are vital to choice enhancement [10, 27, 28]. attribute levels across alternatives). Results further reveal that as Indeed, research is yet to assess the factors that lessen the user perceived overload increases, people show less reactance to reactance to recommendations [7]. Lurie [18 p. 484] indicates that recommendations. Whereas users consulting the recommendations “… in the age of the Internet, developing an understanding of how at higher overload levels had generally better choices, they information-rich environments affect consumer decision making showed higher confidence in their choices only when they is of crucial importance. Given the disparate ways in which conform rather than react to recommendations. product information can be presented to consumers and the high potential for information overload in online environments, it is Categories and Subject Descriptors important to use measures that capture the multiple dimensions of I.2.11 [Distributed Artificial Intelligence]: Intelligent agents, information.” Agents and Web-services. The contribution of this article is four-fold. First, the article examines the relation between the delivered information load in General Terms the choice set and perceived overload by simultaneously Management, Measurement, Human Factors, Performance, manipulating the number of alternatives, number of attributes, and Design, Theory. the distribution of attribute levels across the alternatives. Second, it assesses the role of information overload on employing decision heuristics (the tendency to consult the recommendation agent) Keywords while considering the role of need for cognition. Third, it Recommendation Agents, Information Overload Theory, investigates how information overload and need for cognition Reactance Theory. shape users’ reactance to recommendations. Fourth, it examines the impact on choice quality and confidence. We next briefly 1. INTRODUCTION review the literature and present the study conceptual framework. When making purchase decisions, users typically process large The methodology section reports the details of the pretest and the amounts of information. As people shop online to save time and experiment. Results are then presented. The paper concludes with effort, retailers are required to effectively manage product a summary of findings and implications on theory and practice. information delivered on their e-stores. The many choice possibilities associated with large choice sets represents an 2. CONCEPTUAL FRAMEWORK opportunity and challenge for consumers and retailers [7, 9]. To Research showed the effects of information overload on the help customers reduce the cognitive effort while enhancing their choice and purchase of different products: Laundry detergent [13], decision, retailers incorporate on their e-stores agents that filter, rice and prepared dinner [14], peanut butter [25], houses [19], optimize, and organize product information. Product calculators [18], and CD players [17]. Research indicates that recommendations are decision-aid tools that support rather than variations in the amount of information impact the decision replace consumer decision-making by suggesting one or more processes, which affects decision quality. Information overload product that closely matches consumer preferences [26]. In effect, Copyright © 2010 for the individual papers by the papers' authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors: Knijnenburg, B.P., Schmidt-Thieme, L., Bollen, D. 26 FULL PAPER Proceedings of the ACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces (UCERSTI), Barcelona, Spain, Sep 30, 2010 Published by CEUR-WS.org, ISSN 1613-0073, online ceur-ws.org/Vol-612/paper5.pdf happens because of humans’ limits in assimilating and processing Consumers do react to product recommendations because they information within any timeframe [13, 19]. When consumers are limit their choice freedom [7]. Under high overload levels, faced with high levels of information, their limited capacity to consumers behave as satisficers as opposed to optimizers [19]. process information becomes overloaded, which results in Because consumers are adaptive decision makers [3], we propose dysfunctional consequences such as cognitive fatigue and that the higher the information overload becomes, the more the confusion [8, 16, 20, 21, 25]. consumer will conform to recommendations (P4). This proposition finds support in the self-regulation research; Several measures were used to capture the amount of product information overload can be seen as a resource depletion information. Researchers have traditionally manipulated the mechanism that “enhances the role of intuitive reasoning by alternative and attribute levels in product choice sets [13, 19]. impairing deliberate, careful processing” of information [24, p. While this line of research has made substantial contribution, 344]. Need for cognition is also expected to shape reactance so discrepancies were noted [12, 19, 20, 21]. More recently, the that under higher levels of overload, the lower the need for concept of information structure was introduced and shown to cognition is, the less the consumer will react to product have a role in determining overload; this concept asserts that when recommendations (P5). measuring information loads, both the number and probability of outcomes should be considered (for a discussion, see [18]). When the distribution of attribute levels for instance is proportional across the alternatives (e.g., half the laptops in a given choice set are equipped with Intel and half with AMD processors), information load will be higher than for a disproportional distribution (e.g., 3/4 with Intel and 1/4 with AMD processors). This is because a disproportional distribution increases information diagnosticity [18]. Information load in a choice set can hence be affected by the number of alternatives, number of attributes, as well as the distribution of attribute levels across the alternatives (attribute distribution hereafter) [17, 18]. One purpose of this research is to manipulate these three dimensions over a range that is wider than prior work and to assess the impact on perceived overload and choice. After information-processing capacity is surpassed, information increments were found to lead to modest or insignificant reductions in decision quality [8, 18]. As research stipulates a complex rather than a linear relation between information load and perceived overload [8, 14], we expect a nonlinear relation to better describe the relation between these two factors (P1). It is plausible to assume that under high overload levels, consumers do use heuristics to maintain the cognitive effort at acceptable levels. Indeed, consumers adapt decision strategy Figure 1. Research Framework. according to product information, task, and environment [5, 23]. In complex choice situations, consumers for instance become We finally study the impact of information overload and product more selective in acquiring and processing information [23]. recommendations on choice quality and confidence. Theory posits Because consulting product recommendations can be seen as a salient role for recommendations on choice quality in complex information-processing heuristic [10, 27, 28], we theorize that the choice situations [27]. In effect, choice quality suffers when the utility of consulting product recommendations increases with processing effort exceeds processing limits [23]. As product information overload. Under high overload levels, consumers recommendations help consumers improve choice by behave as satisficers (vs. optimizers) and thus use more an concentrating on the alternatives that best match their preferences information-processing reduction strategy [19]. Therefore, we [10], product recommendations should uphold choice quality as expect that (P2) consumers will tend to consult the information overload increases (P6) [3, 10, 15, 19, 28]. Because recommendations more as (a) information load increases and as the negative role of information overload on choice is prominent (b) perceived overload increases. Figure 1 depicts the study in the case of a proportional versus disproportional attribute conceptual framework. distribution [18], we theorize that the impact of product recommendations on choice quality will be particularly salient for Consumers have divergent needs for information. Need for choice sets with proportional attribute distribution (P7). cognition (the consumer tendency to engage in effortful thinking) According to Fitzsimon and Lehmann [7], recommendations was cited as an important factor of attitudinal and behavioral reduce uncertainty for consumers who do not react to change [4]. Consumers low on the need for cognition tend to recommendations. We hence expect that consumers who consult avoid activities requiring high cognitive effort and to engage in and conform to product recommendations will have higher choice heuristic strategies [11]. We thus expect need for cognition to confidence than consumers who consult but react to attenuate the tendency to consult the recommendations such that recommendations (P8). as information overload increases, the lower the need for cognition is, the more the consumer will consult product recommendations (P3). Copyright © 2010 for the individual papers by the papers' authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors: Knijnenburg, B.P., Schmidt-Thieme, L., Bollen, D. 27 FULL PAPER Proceedings of the ACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces (UCERSTI), Barcelona, Spain, Sep 30, 2010 Published by CEUR-WS.org, ISSN 1613-0073, online ceur-ws.org/Vol-612/paper5.pdf 3. METHODOLOGY 3.2 Pretest and Measure Each participant had to choose a laptop with the option to consult 3.1 The Experimental Site and the the recommendations (between-subject design). Recommender System Recommendations consultation and if consulted whether the An e-store was created for “Portable Direct” using professional recommended product was chosen are observed variables. Web design service; a fictitious retailer name was used to control Perceived overload was measured using two seven-point items for retailer preferences [1]. The computer laptop was chosen as (There was too much information to make a choice; I wanted to product category because (a) it is a complex product thus receive more information about the different products before consumers are expected to be attentive during choice, (b) it has making my choice). Similar to [13, 19], choice confidence was many known attributes, which allows a meaningful manipulation measured using three items (I am confident that I made the best at high number of attributes, (c) it is a search product (attributes possible choice based on my needs; I am satisfied with the choice can be communicated using the Web), and (d) it is a product that I made; I am certain that I made a good choice; α=0.93). Need for consumers shop for online, which improve the ecological validity. cognition was measured using the 18-item scale ([4], α=0.82). As Though pretested (see the Appendix), manipulation levels were decision makers draw on their experience and knowledge of adapted from the literature. Three levels of alternatives (6, 18, and product category, product experience (three-item from [22], 30) were chosen because research investigating this factor along α=0.95) and product category involvement (four items adapted with attribute distribution considers only two alternative levels (18 from [2], α=0.92) were measured and controlled for. See the and 27 in [17, 18]) and because little research manipulated for Appendix for details of the pretest and manipulation checks. choice sets with low alternative level [19]. Three levels of attributes (15, 25, and 35) were chosen because research 3.3 Stimuli investigating this factor along with attribute distribution considers Participants were informed that their task consisted of choosing a only two attribute levels (9 and 18 in [17]). Whereas few studies laptop as they would in an actual purchasing situation. The task manipulated for 20 attributes or more [8, 19], including higher page described “Portable Direct” as a well-established online number of attributes is necessary as consumers consider many retailer of product category and asked the participants to navigate attributes when shopping for complex products. Akin to prior its e-store (made available through a link provided after the work [17, 18], the distribution of attribute levels across the participants entered personal attribute preferences) to choose the alternatives had two levels (proportional vs. disproportional “The laptop you would seriously consider buying”. Participants distribution); the attributes provided in a choice set were were told to take as much time as needed and to freely consult the manipulated according to one of these levels. information available on the website. A time constraint was not The participant rates the importance (weight; 1-7) of each of the imposed because this would be inconsistent with real-life 35 attributes (this step is performed before the participant is situations and because this would result in eliminating a portion of randomly assigned to one of the eighteen experimental participants based on some cut-off value. In effect, time pressure conditions). Then, the score of each potential choice (each laptop was shown to influence information overload [8]. Before a in the choice set provided under a particular condition) can be participant was randomly assigned to one of the eighteen determined by the following formula (Weighted Additive Rule; conditions, a second page asked the participant to rate the Payne, Bettman, and Johnson 1993): importance of each attribute (to estimate the participant utility function so that the recommendation agent could suggest the optimal choice; Weighted Additive Rule WADD as in [23]). Depending on the assigned condition, the e-store provided the participant with a finite choice set (e.g., six alternatives each with Where: S = Global score of alternative j for consumer k. fifteen attributes for conditions one and two in the Appendix). Similar to factual e-stores, each alternative appeared in a tabular i = Attribute; format with the attributes headed by the laptop photograph. The j = Alternative (laptop); alternatives that made the choice set were presented on the same page. To avoid presentation bias, the order of alternatives was k = Consumer; randomized for each participant in a given condition. Brand was P = Weight of attribute i for consumer k; concealed to reduce the possibility of following a brand heuristic and to entice participants to make choice using the information V = A priori value of attribute i applied by system and associated provided. This is akin to prior work [17]. Participants had the with alternative j. option to consult the recommendations by clicking on a hyper link labeled “Click here for our recommendation according to your That is, the WADD determines the score of a given alternative j preferences” located at top of the choice set provided. After (for consumer k) by multiplying the weight of each attribute making their choice, participants were presented with the measure (provided by consumer k) by its a priori value, and then adding items. the obtained values of all attributes. The alternative with the highest score (i.e. the one that optimizes consumer k’s utility 3.4 Sample function) is then suggested by the recommendation agent (should An invitation to participate in a “Study on e-commerce” was sent consumer k choose to consult the agent by clicking the link to consumers randomly chosen from a large consumer panel provided). belonging to a North American market research company. Of the 472 responses received, 466 were complete and retained. Sample demographics distribution (see the Appendix) shows that the Copyright © 2010 for the individual papers by the papers' authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors: Knijnenburg, B.P., Schmidt-Thieme, L., Bollen, D. 28 FULL PAPER Proceedings of the ACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces (UCERSTI), Barcelona, Spain, Sep 30, 2010 Published by CEUR-WS.org, ISSN 1613-0073, online ceur-ws.org/Vol-612/paper5.pdf sample was well distributed across consumer population with no Wald=8.10, p=0.004). In addition, the interaction between important bias toward a particular segment. perceived overload and need for cognition was significant (B= - 0.131, Wald=4.52, p=0.034), which shows that as perceived 4. RESULTS overload increases, the lower the consumer was on need for cognition, the less reactance to recommendations the consumer A comprehensive analysis of the data with a path model was not would exhibit. Alternatively, neither information load nor its performed because it was not feasible (i.e., central variables in the interaction with need for cognition were significant in predicting model such as RA consultation and reactance to recommendation reactance (all p’s>.34 NS). We further tested the direct impact of were binary; in addition, an important exogenous variable- the levels of alternatives, attributes, and attribute distribution on information load-is ordinal and reflected by one item). As such, reactance and found no significant effects (all p’s>.31). These ANOVA and regression analysis were used in testing the results collectively show that perceived overload, rather than propositions (except for P2 through P5 where logistical regression information loads, was the determinant factor in predicting were used because the dependent variable was binary). reactance to recommendations. The main effect for information load (called interchangeably Choice quality was measured by the distance between the information bits; [17, 18], see the Appendix section) on perceived participant actual and optimal choice (Weighted Additive Rule overload was significant (F=23.88, p<0.001); this result stays WADD; [23]). This is akin to past work [13, 16, 19]. The reliable when controlling for product involvement and experience expected interaction between information load and (only product experience was significant covariate; B= -0.085, recommendations consultation was significant (F=1.68, p=0.012; F=5.34, p=0.021). A curvilinear quadratic curve solution Figure 3 Up). Similarly, we found support to the proposition that explained more variance (R²=0.264) in the relationship between recommendations consultation upholds choice quality as information load and perceived overload than a linear (R²=0.224) perceived overload increases because the interaction between or a logarithmic (R²=0.248) solution (Figure 2). perceived overload and recommendations consultation was Binary logistical regression was performed to test the impact of significant (F=1.61, p=0.036; Figure 3 down). information load on recommendations consultation as well as the attenuating role of need for cognition. Information loads increment led to more recommendation consultation by means of main effect (B=0.164, Wald=6.00, p<0.05). In addition, the interaction between information loads and need for cognition was significant in the predicted direction (B=-0.031, Wald=5.587, p<0.05). Similarly, logistical regression was performed to test the impact of perceived overload on recommendations consultation and the attenuating role of need for cognition. Perceived overload did lead to more consultation of recommendations (B=0.344, Wald=4.06, p=0.044) and the interaction between perceived overload and need for cognition was significant in the predicted direction (B=-0.077, Wald=5.71, p=0.017). The direct effects of the alternative, attribute, and attribute distribution levels and their interactions on recommendations consultation were examined and showed insignificance (all p’s>0.10 NS). Figure 3a. Recommendations effect on choice quality (upper line: RA consulted). Figure 2. Information load effect on perceived overload. To test the impact of perceived overload and need for cognition on reactance, we applied binary logistical regression on the observations that consulted the recommendations (n=178). As expected, perceived overload was significant factor in predicting the conformation (vs. reactance) to recommendations (B=0.91, Copyright © 2010 for the individual papers by the papers' authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors: Knijnenburg, B.P., Schmidt-Thieme, L., Bollen, D. 29 FULL PAPER Proceedings of the ACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces (UCERSTI), Barcelona, Spain, Sep 30, 2010 Published by CEUR-WS.org, ISSN 1613-0073, online ceur-ws.org/Vol-612/paper5.pdf Figure 3b. Recommendations effect on choice quality (upper line: RA consulted). We then tested the proposition that product recommendations effect on choice quality is salient for choice sets with proportional attribute distribution (P7). We found support to this proposition by means of a three way interaction (Number of Attributes x attribute distribution x recommendations consultation; F=2.47, p<0.05; Figure 4). This interaction shows the recommendations to enhance choice quality for choice sets with proportional distribution of attribute levels across the alternatives at all attribute levels (Appendix for means). The interaction also highlights that recommendations consultation improved choice for all choice sets only when the number of attributes became high. We finally tested and found support to the proposition that consumers consulting and conforming to recommendations will have higher choice confidence than consumers consulting and reacting to recommendations (5.13 vs. 4.41, F=8.55, p=.004). Figure 4. Recommendations effect on choice quality for 5. DISCUSSION choice sets with proportional versus disproportional The experimental results lend support to research propositions. distribution of attribute levels across the alternatives. Results suggest a curvilinear relation between information load — = Proportional attribute distribution. and perceived overload, which indicates that the impact of additional increments in product information after some levels - - - = Disproportional attribute distribution. (condition 7 shown in the Appendix) are not as influential in driving overload perceptions. The consumer use of decision The findings show the positive effects of product heuristics at high levels of information overload helps explaining recommendations on choice quality at high levels of information this finding. Findings lend support to the notion that the utility of loads and overload perceptions. The positive impact of consulting product recommendations increases as the information recommendations on choice quality was particularly salient for load and as perceived overload increases. Consumers did use an choice sets with proportional distribution of attribute levels across information-processing heuristic by consulting product the alternatives. Finally, choice confidence improved for recommendations more as information overload increases. consumers who consulted and conformed (vs. reacted) to Moreover, this tendency was higher for consumers low on the recommendations. In effect, the recommendations might have need for cognition. Importantly, consumers appear to conform (vs. made the accuracy feedback as immediate and tangible as the react) to recommendations more at high levels of perceived effort feedback by signaling to consumers that a product in the overload. Further, the lower the need for cognition was, the less choice set is more optimal than the initially considered one [5], the consumer reacted to recommendations at higher levels of which might have triggered consumers to have lower levels of information overload. confidence in their choice if they reacted to the recommendations. This research contributes to theory by studying the relation between information loads and overload perceptions over a wide range for three factors deemed to determine the information load Copyright © 2010 for the individual papers by the papers' authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors: Knijnenburg, B.P., Schmidt-Thieme, L., Bollen, D. 30 FULL PAPER Proceedings of the ACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces (UCERSTI), Barcelona, Spain, Sep 30, 2010 Published by CEUR-WS.org, ISSN 1613-0073, online ceur-ws.org/Vol-612/paper5.pdf and by showing that consumers indeed do employ decision 6.2 Pretest and Manipulation Checks heuristics in response to information overload. People appear to A pretest was performed to ensure task and measure regard the use of product recommendation agent as information- comprehensibility [8], to check the manipulation of independent processing reduction heuristic. This research further established a variables and to inspect the distribution of control variables. The link between information overload and reactance to pretest ensured that an increment from six (and eighteen) to thirty recommendations and underlined the role of need for cognition. It alternatives resulted in a noticeable change in information load. contributes to the recommendation agents’ literature by showing The pretest included three sections: The first contained the the impact of recommendations on choice at different information manipulation checks, the second examined product experience overload levels and by showing the salient effect of level and where the product category was relevant for the recommendations on choice quality for sets with proportional participant pool (e.g., manipulating the attributes level would be distribution of attribute levels across the alternatives. realistic and meaningful). The third section helped determining Several practical implications emerge. Integrating a the 35 most important attributes (of 45 attributes identified using recommendation agent based on consumer preferences appears to two retailing websites) to be included in experiment (each be beneficial for consumers and retailers (by helping consumers attribute was evaluated using a Very Important/Not Important at make quality choices at high levels of information overload). All seven-point item). Recommendations enhance choice, particularly as information Six questionnaire versions were created for the pretest, all sharing load and perceived overload increases. In addition, the items of product experience and involvement, as well as recommendation agents appear to have particular influence on attribute importance evaluation (the versions differed only in the choice when product information is less diagnostic (attribute first section). The first two versions were developed to check the levels are proportionally distributed across the alternatives in the manipulation of number of alternatives (6, 18, and 30). The two choice set). Finally, the outcome of recommendation agents can versions differed in the order the three levels were presented to be optimized as consumers in general show less reactance to each participant (i.e., while the order was 6-18-30 in the first recommendations at higher levels of information overload. version, the order was reversed in second version). This eliminated the possibility that a respondent rated level one as This work has limitations. Although the study sample comprised having fewer alternatives than levels two and three because it was actual consumers randomly selected from large consumer panel, displayed first. Similar steps were taken in versions three and the sample was self-selected. Nonetheless, the sample distribution four, which checked the manipulation for number of attributes. across the consumer population was satisfactory. The research Versions five and six examined the manipulation for attribute considered only one product category and did not examine distribution (proportional vs. disproportional). Version five (six) whether similar effects are obtainable for less complex and for assessed the manipulation for a proportional (disproportional) experience products. Further, this research did not investigate the distribution of attribute levels across the alternatives (both for the effects of information overload and product recommendations on price attribute). shopping enjoyment and long term performance measures such as consumer loyalty and retention. These topics are potential An invitation to participate in the pretest was emailed to 116 extensions to this line of research. consumers (convenience sample). 77 useable responses were received. Because the measure (for both the alternatives level and attributes level) was within-subjects, ANOVA with repeated 6. APPENDIX measures was used to analyze the input. For attribute distribution, 6.1 Experimental conditions (Information a chi-square test was used. The 32 participants that evaluated Load*) alternatives level had to respond to a seven-point bipolar item (What do you think of the quantity of laptops offered: Not enough to make a choice/too much to make a choice) (item repeated for each of the three levels presented to the respondent). The analysis showed that participants perceived significantly different information loads between each of the three levels (M6=2.66, M18=4.81, M30=4.94; F6-18 (1, 31)=69.65, F6-30(1, 31)=139.7, F18-30(1, 31) =27.59, all p-values<0.001). Similarly, the 23 participants evaluating the attributes level had to respond to the seven-point bipolar item (What do you think of the quantity of attributes offered: Not enough to make a choice/too much to make a choice; item was repeated for each of the three levels presented to the participant). The analysis showed that participants reported significantly different information loads between each of the three levels (M15=2.87, M25=4.30, M35=4.87; F15-25(1, 22)=77.85, F25-35(1, 22)=10.33, F15-35(1, 22)= 97.32, all p-values<0.01). The 22 participants evaluating the success of attribute distribution manipulation responded to a binary item (Was the number of laptops priced at $600 different or similar to the number of laptops priced at $750 and $900?). For (dis)proportional structure, the number was (not) equal. Participants in the (dis)proportional structure condition reported (un)equal distribution of the price attribute across alternatives ( (1, 22) = 12.32, p < 0.01). Copyright © 2010 for the individual papers by the papers' authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors: Knijnenburg, B.P., Schmidt-Thieme, L., Bollen, D. 31 FULL PAPER Proceedings of the ACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces (UCERSTI), Barcelona, Spain, Sep 30, 2010 Published by CEUR-WS.org, ISSN 1613-0073, online ceur-ws.org/Vol-612/paper5.pdf The second section (shared for all participants) showed that the [5] Einhorn, H. J. and Hogarth, R. M. 1981. Behavioral Decision laptop computer is a product bought and used frequently by Theory: Processes of Judgment and Choice. Journal of participants (87 percent of participants indicated using or to have Accounting Research, 19, 1, 1-31. used a laptop regularly; 75 percent of participants have already [6] eMarketer. 2008. Retailers Take Note: Video Sells! DOI bought a laptop). This section also showed the internal =http://www.emarketer.com/Article.aspx?id=1006883 consistency for product experience items (α=0.96) and product involvement items (α=0.87) and clarified the sample distribution [7] Fitzsimon, G. J. and Lehmann, D.R. 2004. Reactance to according to these variables. Recommendations: When Unsolicited Advice Yields Contrary Responses. Marketing Science, 23, 1, 82-94. Attributes were assigned to experimental conditions using the pretest input. Attributes that have higher weights appeared more [8] Hahn, M., Lawson, R., and Lee, Y. 1992. The Effects of often in conditions with fewer attributes. This was done because Time Pressure and Information Load on Decision Quality. the inclusion of an attribute in a choice set renders the attribute Psychology & Marketing, 9, 5, 365-379. more important for the decision maker [9]. Consequently, [9] Häubl, G. and Murray, K.B. 2003. Preference Construction including less important attributes in a choice set made up of few and Persistence in Digital Marketplaces: The Role of attributes would inflate the attribute’s importance. In effect, Electronic Recommendation Agents. Journal of Consumer choice sets containing only less relevant attributes for the Psychology, 13, 3, 75-91. alternative (choice sets that do not provide basic and important [10] Häubl, G., and Trifts, V. 2000. Consumer Decision Making attributes such as price, processing speed, or memory size) are in Online Shopping Environments: The Effect of Interactive unrealistic and would reduce ecological validity. Decision Aids. Marketing Science, 19, 1, 4-21. 6.3 Sample Demographics (n=466; 56.9% [11] Haugtvedt, C. and Petty, R. 1992. Personality and females) Persuasion: Need for Cognition Moderates the Persistence Age: 11.6% ages 18-24, 26.4% 25-34, 20.0% 35-44, 19.7% 45-54, and Resistance of Attitude Changes. Journal of Personality 9.9% 55-64, 12.4% 65+. Education level: 19.6% and Social Psychology, 63, 2, 308-319. Primary/secondary education level, 70.8% Undergraduate degree, [12] Jacoby, J. 1984. Perspectives on Information Overload. 9.7% Graduate degree. Income: 14.2% less than $15K, 18.9% 15- Journal of Consumer Research, 10, 4, 432-436. 29K, 29.0% 30-44K, 19.7% 45-59K, 9.7% 60-74K, 7.5% 75K or [13] Jacoby, J., Speller, D., and Berning, C. 1974. Brand Choice higher. Marital status: 28.8% single, 57.9% married/common law Behavior as a Function of Information Load. Journal of partner, 13.3 other status. Employment: 9.5% students, 78.6% Marketing Research, 11, 1, 63-69. working full-time, 7.1% working part-time, 4% searching. [14] Jacoby, J., Speller, D., and Berning, C. 1974. Brand Choice 6.4 Choice Quality Means Behavior as a Function of Information Load: Replication and Extension. Journal of Consumer Research, 1, 1, 33-42. [15] Johnson, E. J. and Payne, J.W. 1985. Effort and Accuracy in Choice. Management Science, 31, 4, 394-414. [16] Keller, K.L. and Staelin, R. 1987. Effects of Quality and Quantity of Information on Decision Effectiveness. Journal of Consumer Research, 14, 2, 200-213. [17] Lee, B.K. and Lee, W.N. 2004. The Effect of Information Overload on Consumer Choice Quality in an On-Line Experiment. Psychology & Marketing, 21, 3, 159-181. [18] Lurie, N. H. 2004. Decision Making in Information-Rich Environments: The Role of Information Structure. Journal of Consumer Research, 30, 4, 473-486. [19] Malhotra, N. K. 1982. Information Load and Consumer 7. REFERENCES Decision Making. Journal of Consumer Research, 8, 4, 419- [1] Aksoy, L., Bloom, P., Lurie, N., and Cooil, B. 2006. Should 430. Recommendation Agents Think Like People? Journal of Service Research, 8, 4, 297-315. [20] Malhotra, N. K. 1984. Reflections on the Information Overload Paradigm in Consumer Decision Making. Journal [2] Beatty, S. E. and Talpade, S. 1994. Adolescent Influence in of Consumer Research, 10, 4, 436-440. Family Decision Making: A Replication with Extension. Journal of Consumer Report, 21, 2, 332-342. [21] Malhotra, N. K. 1984. Information and Sensory Overload. Psychology & Marketing, 1, 3&4, 9-21. [3] Bettman, J. R., Johnson, E. J., and Payne, J. W. 1990. A Componential Analysis of Cognitive Effort in Choice. [22] Oliver, R. L. and Bearden, W. O. 1985. Crossover Effects in Organizational Behavior and Human Decision Processes, the Theory of Reasoned Action: A Moderating Influence 45, 1, 111-140. Attempt. Journal of Consumer Research, 12, 3, 324-340. [4] Cacioppo, J., Petty, R., and Kao, C.F. 1984. The efficient [23] Payne, J. W., Bettman J. R., and Johnson, E. J. 1993. The assessment of need for cognition. Journal of Personality Adaptive Decision Maker. New York, Cambridge University. Assessment, 48, 1, 306–307. Copyright © 2010 for the individual papers by the papers' authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors: Knijnenburg, B.P., Schmidt-Thieme, L., Bollen, D. 32 FULL PAPER Proceedings of the ACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces (UCERSTI), Barcelona, Spain, Sep 30, 2010 Published by CEUR-WS.org, ISSN 1613-0073, online ceur-ws.org/Vol-612/paper5.pdf [24] Pocheptsova, A., Amir, O., Dhar, R., & Baumeister, R. F. [27] Swaminathan, V. 2003. The Impact of Recommendation 2009. Deciding Without Resources: Resource Depletion and Agents on Consumer Evaluation and Choice: The Choice in Context. Journal of Marketing Research, 46, 3, Moderating Role of Category Risk, Product Complexity, and 344-356. Consumer Knowledge. Journal of Consumer Psychology, 13, [25] Scammon, D. L. 1977. Information Load and Consumers. 1&2, 93-101 Journal of Consumer Research, 4, 3, 148-155. [28] Xiao, B. and Benbasat, I. 2007. E-Commerce Product [26] Senecal, S. and Nantel, J. 2004. The Influence of Online Recommendation Agents: Use, Characteristics, and Impact. Product Recommendations on Consumers’ Online Choices. MIS Quarterly, 31, 1, 137-209. Journal of Retailing, 80, 2, 159-169. Copyright © 2010 for the individual papers by the papers' authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors: Knijnenburg, B.P., Schmidt-Thieme, L., Bollen, D. 33