Serial position efects are triggered in recommendation scenarios where users focus on evaluating items shown at the beginning and at the end of a list. In this paper, we analyze these efects in the context of multi-attribute item recommendation scenarios where the recommended items are presented to users in the form of a list of relevant attributes. We conducted a user study in diferent item domains to examine if the item selection of users is afected by the order of the attributes of the recommended items presented to them. The experimental results show that the order of the attributes does not afect users' item selection. When selecting a recommended item, users tend to focus on evaluating the value of the attributes that reflect their preferences for the desired item but do not care about the order of the attributes. This finding brings us to a conclusion that in the context of multi-attribute item recommendation scenarios, the selection of a recommended item from a list of candidate items is immune to serial position efects.
Serial position efects are decision biases triggered when items are presented in the form of a list
[
In this paper, we further analyze the impacts of serial position efects in the context of
multi-attribute items, in which items are characterized by a list of attributes. We find out that
the existing studies only investigate serial position efects with the focus on the position of
recommended items themselves [
The remainder of the paper is organized as follows. In Section 2 and Section 3, we present a summary of related work and discuss our research question, corresponding recommendation scenarios, and variants of attribute order. In Section 4, we present essential steps of our user study to answer the research question. The results and discussions regarding the research question are presented in Section 5. Finally, we conclude the paper and discuss open issues for future work in Section 6.
Serial position efects (also known as primacy/recency efects ) describe the tendency of a user to
recall the items shown at the beginning and at the end of a list more often than those in the
middle [
Serial position efects can be exploited to create better user interfaces for recommender
systems. For instance, in an e-learning recommender system, these efects can increase the
frequency of interacting with questions/learning topics. The questions/learning topics can be
recommended based on the learner’s training performance or the questions’ dificulty level
[
The existing research focuses on investigating serial positions efects in the context of
noattribute item domains but does not take into account scenarios where the order of the attributes
describing the recommended items can play a role in the item selection of users. To the best
of our knowledge, there is no study where serial position efects are examined in the
multiattribute recommender systems when the recommended items with relevant attributes are shown
to the target user. In fact, there exist studies on multi-attribute recommender systems [
Coming back to Section 1 where we assumed a recommendation scenario in which a user selects the desired item characterized by a list of attributes. The user has specified his/her requirements for the attributes of the desired item. In such a scenario, we want to find the answer to the following research question: “Does the order of the attributes with regard to the recommended items afect the item selection behavior of users?” . In this context, we are interested in examining the correlation between the order of the attributes of the recommended items shown to a user and his/her item selection behavior. Especially, we want to know if users tend to evaluate attributes shown at the beginning and at the end of the list and skip attributes shown in the middle of the list when selecting an item. Answering this research question helps to figure out if serial position efects exist in the context of multi-attribute item recommendation scenarios. If this is the case, then these efects could be exploited to create position nudging that provides the particular order of the attributes, increases the ease and convenience of users when selecting the recommended items [13, 14]. It also helps to find out a solution to counteract the impacts of serial positions efects on the item selection of users.
To answer the research question, we analyze serial position efects in diferent item domains. We propose two recommendation scenarios related to two item domains - Airbnb rooms and digital cameras. In the Airbnb room domain, we assume a scenario where a user is looking for an Airbnb room for a two-day trip at a low cost. The room in this scenario is referred to as a low-involvement item that requires low cost and low decision-making efort. In the digital camera domain, we assume a scenario where a user is looking for a professional camera for his/her job. The camera mentioned in this scenario indicates a high-involvement item that requires high cost and high decision-making efort. The details of these recommendation scenarios are presented in the following:
Airbnb room recommendation scenario: “Assume you are using a recommender system to look for an Airbnb room. You want to make a two-day trip to the mountains. You need to have a relaxing trip, and the room should be therefore soundproof. The room size should be at least 12 2. Besides, the price should not be over 45 Euros per night.”
Digital camera recommendation scenario: “Assume you are using a recommender system to look for a digital camera. You are active in social media and working as a food blogger. You want to take photos with the size of at least 21 megapixels. The display should not be larger than 3 inches. The camera should not be heavier than 600 grams so that you can easily transport it. Since you are a beginner in this field, the camera should not cost more than 1500 Euros.”
With the proposed recommendation scenarios, we want to analyze the item selection behavior
of users when the recommended items are presented in a list of attributes in diferent orders.
We assume that, in each domain, the recommender system suggests five alternatives whose
attributes meet the specified requirements of a user concerning the desired item. Each alternative
is represented by a list of attributes and corresponding values. To analyze the impacts of serial
position efects, we need to propose diferent variants of attribute order. One question arising
in this context is how to place the attributes in a list? According to Wong [
In order to find the answer to the research question, we conducted an online user study with computer-science students (bachelor and master students) who were participated in our courses at Graz University of Technology, Austria. In total, there were 198 participants from 18 to 25 years old (male: 79.65%, female: 20.35%). There was a big diference between these two proportions, coming from the inequivalent numbers between male and female students in our courses. The user study was designed and conducted in the following steps:
Step 1 - Propose recommendation scenarios: We proposed two recommendation scenarios with regard to two item domains, Airbnb rooms and digital cameras, as presented in Section 3.2. For each recommendation scenario, we selected five alternatives (Alternative A ... Alternative E) whose attributes meet the requirements mentioned in the recommendation scenario. The recommended items (alternatives) are shown to the participants in a random fashion. The proposal of these recommendation scenarios for our user study was needed to provide the requirements on which the participants can select one desired item. This proposal helped to avoid a situation where the participants had no clue to select an item and therefore, just picked up an item randomly.
Step 2 - Propose diferent variants of attribute order : In each domain, we proposed four variants (variant 1 ... variant 4) showing four diferent orders of the attributes of the recommended items (see the variant description in Section 3.3). In the Airbnb room domain, the ten attributes shown in variants 1 and 2 are cleanliness, staf rating, price, smoking, quietness, modernity, mini bar, internet, room service, and room size. Besides these attributes, variants 3 and 4 in the room domain additionally show five other attributes - bathroom amenities, parking, TV, balcony, and view. In the digital camera domain, variants 1 and 2 show ten attributes - zoom, price, image resolution, HDMI, battery, video, GPS, weight, megapixels, and display. Variants 3 and 4 additionally show five other attributes - model, sensor, storage, face detection, and USB.
Step 3 - Distribute variants to participants: The study was conducted using the betweensubjects method, i.e., each participant received exactly one recommendation scenario and one variant of attribute order. The user interfaces showing the recommended items and corresponding attributes are depicted in Figure 2. The number of participants for each variant in each item domain is shown in Table 2. Each participant took a look at the recommendation scenario and assumed himself/herself to be the user in the recommendation scenario. The participant was then asked to select one item that is the most appropriate from his/her point of view.
Step 4 - Conduct the user study with an eye-tracking device: In addition to the mentioned online user study (with 198 participants), we invited 25 students to our institute to perform the user study directly on our computer with an eye-tracking device (Tobii Pro T60 XL). In order to avoid potential biases, these students did not participate in the online user study earlier (i.e., all of them saw the user study the first time). This eye-tracking user study helped us to better observe how the participants evaluated the values of the attributes of the recommended items.
Method: To answer the research question, in each item domain, we first collected items (alternatives) selected by the participants corresponding to a specific variant of attribute order. Consequently, in each item domain, we collected four data sets for four variants. Thereafter, we performed cross-tabulation analyses between the variants (e.g., variant 1 vs. variant 2, variant 3 vs. variant 4) in both domains. In these analyses, each table reports the variants in the columns, the item choices in the rows, and the corresponding number (frequency) of participants in the cells (see Figure 3). Finally, we ran Chi-Square tests ( = 0.05) to find out if there exist correlations between the item selection behavior of users and the order of the attributes.
Results: The cross-tabulation analyses and Chi-Square test results show that there were no correlations between the item selection of the participants and the order of the attributes (p > 0.05 - see Figure 3 and Table 3). In both domains, the item selection of the participants was independent of the order of the attributes shown to them. In other words, there did not exist serial position efects in the context of multi-attribute item recommendation scenarios. This can be further proven by analyzing the eye-tracking data. According to the heat map data, the participants focused on checking the attributes mentioned in the recommendation scenarios and then compared the values of these attributes in order to find the best option (see Figure 4).
Discussions: Let us have a look at the variants (with ten attributes) in the digital camera domain. The analyses in these variants show that the order of attributes did not afect how the participants chose the recommended item. Indeed, by having a look at the heat-map data of variants 1 and 2 in Figure 4, we recognized that the participants paid their attention to the values of the attributes mentioned in the recommendation scenario (i.e., mega pixel, display, weight, and price) and then compared them with each other. This way, Alternative A and Alternative C were more frequently selected by the participants since they satisfy all the requirements mentioned in the recommendation scenario (high image resolution, lightweight, and low price). In a similar fashion, Alternative D (in the room domain) was selected by the majority of the participants regardless of the order of the attributes of the room alternatives. The participants chose this alternative since this room fulfills all the requirements mentioned in the recommendation scenario. Therefore, it was preferred by the participants over other alternatives.
These results show that the participants focused on evaluating the relevant attributes without caring about how these attributes are shown to them when selecting an item. This brings us to the conclusion that serial position efects are not triggered in multi-attribute item recommendation scenarios, where users have to select recommended items characterized by a (long) list of attributes. This finding goes against what has been pointed out in the existing studies, where serial position efects are mainly taken into account in the context of no-attribute item domains. In the context of multi-attribute item recommendation scenarios, our study shows that, when the recommended items are presented in the form of their attribute lists, the item selection of users is immune to serial position efects. This finally provides the idea of designing user interfaces to present recommended items, which help counteract such efects. User interfaces showing the attribute lists of the recommended items to users can help to mitigate the impacts of serial position efects on the item selection of users.
The paper has discussed recommendation scenarios in two multi-attribute item domains Airbnb rooms and digital cameras, in which recommended items are shown to a user in a list of attributes. In each domain, we examined if serial position efects afect the item selection of users. The data analysis results show that there are no correlations between users’ item selection and the order of attributes. During the item selection process, users focus more on evaluating the values of relevant attributes but do not care about the order of the attributes presented to them. This brings us to the conclusion that serial position efects are not triggered in the context of multi-attribute item recommendation scenarios. One limitation of the paper lies in the small-size samples (on average, 25 participants for each variant) and the large diference in the numbers between the male participants (around 80%) and the female participants (around 20%). Therefore, within the scope of future work, we will collect more participants to achieve representative samples, which are the premise to provide more convincing analysis results. Besides, we will select groups of participants where the number of males and females are equivalent. Another limitation of our work is related to the selected item domains. Although the Airbnb item domain is considered a lower-involvement item domain compared to the digital camera item domain, it is not always low-stake for all users. For some scenarios where users are anxious about traveling and want to double-check the room concerning diferent attributes before choosing. In such scenarios, deciding on an Airbnb room could consume high decision making eforts, even though the price is low. For future work, we will conduct our user study with further item domains ranging from very low- to very high-involvement item domains (e.g., movies and restaurants for (very) low-involvement item domains; financial services and houses/apartments for (very) high-involvement item domains) to achieve more adequate observations from the item domain perspective. We assume that in diferent multi-attribute item domains, serial position efects could have diferent impacts on users’ item selections. [13] M. Jesse, D. Jannach, Digital nudging with recommender systems: Survey and future directions, CoRR abs/2011.03413 (2020). arXiv:2011.03413. [14] C. R. Sunstein, Nudging: a very short guide, Business Economics 54 (2019) 127–129.
doi:https://doi.org/10.1007/s10603-014-9273-1. [15] S. Bag, M. Tiwari, F. Chan, Predicting the consumer’s purchase intention of durable goods: An attribute-level analysis, Journal of Business Research 94 (2019) 408–419. doi:10.1016/ j.jbusres.2017.11. [16] S. Dolnicar, T. Otter, Which hotel attributes matter? a review of previous and a framework for future research, Faculty of Commerce - Papers 1 (2003) 1–24. [17] S. Jang, T. Liu, K. Ji Hye, H. Yang, Understanding important hotel attributes from the consumer perspective over time, Australasian Marketing Journal (AMJ) 26 (2018). doi:https://doi.org/10.1016/j.ausmj.2018.02.001.