Decoy Effect of Recommendation Systems on Real E-commerce Websites Fan Mo1, Tsuneo Matsumoto2, Nao Fukushima3, Fuyuko Kido4 and Hayato Yamana5 1 Dept. of CSCE, Waseda University, Tokyo, Japan 2 National Consumer Affairs Center of Japan, Sagamihara, Kanagawa, Japan 3 LINE Corporation, Shinjuku, Tokyo, Japan 4 Waseda Research Institute for Science and Engineering, Tokyo, Japan; National Institute of Informatics, Chiyoda-ku, Tokyo, Japan 5 Faculty of Science and Engineering, Waseda University, Tokyo, Japan Abstract Recommendations on e-commerce websites help users discover their interests and assist them in deciding on items to purchase; however, users are prone to bias when comparing and selecting items due to cognitive limitations. The decoy effect, a common user bias phenomenon, has been confirmed in previous studies to induce user selection of items by adding one other item when comparing two items. Although previous studies have confirmed the difference in item selection with and without decoy items in controlled experiments, the mechanism of decoy effect in e-commerce websites has not been elucidated. This study is the first to propose a method for evaluating the decoy effect on real e-commerce websites. We proposed a row- based decoy effect detection method inspired by users' tendency to compare items in the same row when browsing recommended items on e-commerce websites. In addition, a new metric, called intra-row decoy effect rate, is proposed to evaluate the degree of decoy effect. Our month-long study of the recommended order of items on three e-commerce sites reveals that e-commerce sites influence users' item choices regardless of whether they intentionally generate a decoy effect. Keywords 1 Decoy effect, asymmetric dominance effect, real e-commerce websites 1. Introduction On e-commerce websites, users tend to struggle with selecting an item to buy because of the massive amount of available information related to the items. Recommendation systems filter such information based on the user–item interaction history, thereby alleviating the problem of information overload. However, even if a recommendation system outputs a set of items that fit the user, the user does not select the first ranked item without consideration; instead, the user compares items before selecting the best one. Users are vulnerable to human cognitive biases during comparison and selection [11]. The decoy effect [4] [9] [10] has been studied as a common human cognitive bias in recommendation systems [11] [13]. It induces users to pay more attention to a specific item (target) when comparing two items after adding the decoy item. The decoy effect is classified into three categories: the asymmetric dominance effect (ADE), attraction effect (AE), and compromise effect (CE). ADE occurs when all attributes of the target are better than the decoy item. AE is a more general form of ADE, which is compared with the decoy item, the target item slightly reduces one attribute, but increases another "IntRS'22: Joint Workshop on Interfaces and Human Decision Making for Recommender Systems, September 22, 2022, Seattle, US (hybrid event). EMAIL: bakubonn@toki.waseda.jp (F. Mo); tsuneo.matsumoto@nifty.com (T. Matsumoto); nao.fukushima@linecorp.com (N. Fukushima); fkido@aoni.waseda.jp (F. Kido); yamana@yama.info.waseda.ac.jp (H. Yamana) ORCID: 0000-0001-7542-4826 (H. Yamana) ©️ 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) dramatically. CE explains users’ tendency to choose items with medium values in all attribute dimensions to avoid risk. To explore the decoy effect, previous studies [4] [9] [12] elaborately designed a pair of items (target and comparator) with virtual attributes, and then invited volunteers to compare the selection distributions of two items before and after adding a decoy item. Decoy effect, influences user decisions, is proposed by Simonson [8]. Mandl et al. [5] are one of pioneer researchers in studying the decoy effect in recommendation systems. They extended the decoy effect to the recommendation system domain and confirmed the effect on user selection by adjusting the attribute values of the added decoy items. However, the decoy effect in real recommendation systems for e-commerce websites has not been elucidated. Experiments with well-designed item pairs whose attributes are virtually set do not reflect the real decoy effect for two reasons. First, in the real world, the attributes of items are an inherent property created by the manufacturer, which cannot be set artificially. For instance, although monitor A (resolution: 3.85k, price: $400) can be the decoy item of monitor B (resolution: 4k, price: $350), the corresponding monitor A may not exist and needs to be verified on a real dataset. Second, no research has verified whether real recommendation systems have the probability of placing the target and decoy items closer together. For instance, monitor B (resolution: 4k, price: $350) is placed on the first page, whereas the decoy item is placed on the third page. Such far away placing may not trigger the decoy effect. To the best of our knowledge, our study is the first to propose a methodology to evaluating the decoy effect on real e-commerce recommendation systems. The main contributions of this study are as follows. 1) We obtained real recommendation results from e-commerce sites and confirmed the existence of the decoy effect in real e-commerce recommendation systems, which has never been revealed. 2) We proposed a method for extracting row-based decoy items and confirmed the effect of a decoy on a real dataset, which was inspired by our experimental analysis where users have a high tendency to compare items in the same row of the web page in which the recommended items are displayed. 3) A new metric, called intra-row decoy effect rate (IRDE Rate), was proposed to evaluate the degree of the row-based decoy effect. The remainder of this paper is organized as follows. The preliminary knowledge is introduced in Section 2, followed by a review of related works in Section 3. Details of the proposed method are described in Sections 4 and 5. Section 6 introduces the user study experiment conducted in this study on a one-month dataset using the IRDE Rate. Finally, the conclusions of the paper are presented in Section 7. 2. Preliminary This section introduces how the decoy effect, which is classified into the asymmetric dominance effect [1], attraction effect [8], and compromise effect [6], influences user decisions. The following definitions assume that there are two conflicting attributes for an item, namely, price and quality. Bateman et al. [1] proposed the asymmetric dominance effect ADE, which has the most far-reaching impact on a user’s decision. Unlike the competitor item 𝑖 𝑐 , ADE increases the user’s attention to the target item 𝑖 𝑡 by adding a decoy item 𝑖 𝑑 that is completely dominated by the target item, as shown in Fig. 1. That is, all the attributes of the target item are better (price is low and quality is better) than those of the decoy item, whereas the decoy item has at least one attribute (e.g., price) close to the target item. The attraction effect (AE) [8] is a more general form of ADE; it increases the attractiveness of the target item to the user by comparing the target item 𝑖 𝑡 with the decoy item 𝑖 𝑑 , where the target item is slightly expensive with a large quality improvement as shown in Fig. 2. The compromise effect (CE) [6] [8] explains users’ tendency to choose items with medium values in all attribute dimensions rather than items with extreme values to avoid risk. Fig. 3 shows that when the decoy item 𝑖 𝑑 is added, users estimate the target item 𝑖 𝑡 as the most favorable selection because it has a medium price and medium quality. Fig. 4 shows the relationship between price and quality with the three decoy effects mentioned above 𝑘 𝑘 𝑘 [10]. If the attribute of decoy item 𝑖 𝑑 is in the range of 𝑖𝐴𝐷𝐸 (𝑖𝐴𝐸 , 𝑖𝐶𝐸 ), item 𝑖 𝑘 is favored compared with 𝑐 competitor item 𝑖 , which demonstrates the influence of ADE (AE, CE). In this paper, we focused on ADE because: 1) it is the most likely factor to cause cognitive bias and has the most profound effect on Figure 1: Asymmetric Dominance Effect (ADE) Figure 2: Attraction Effect (AE) Figure 3: Compromise Effect (CE) Figure 4: Relationships between Attributes and Figure 5: Relationships between Attributes and Decoy Effects (balanced attributes) Decoy Effects (unbalanced attributes) users' decisions [9]; 2) the importance of two attributes is usually unbalanced, as shown in Fig. 5, which results in a smaller range of AE and CE, thereby reducing the impact of the decoy effect of AE and CE compared with ADE; 3) a nonlinear relationship between the two attributes enforces a complicated range of definition of AE and CE, which will be explored in our future work. 3. Related Work This section presents a review of previous research on the decoy effect in recommendation systems. 3.1. Decoy Effect in Recommendation Systems The decoy effect tends to cause cognitive bias and induces users. Teppan et al. [12] conducted experiments to investigate the decoy effect in recommendation systems and confirmed that a well- designed decoy item increases the selection of target items by comparing the difference in the distribution of users’ item selection before and after adding the decoy item. Similar to the study by Teppan [12], Mandl et al. [5] further investigated a technique for calculating the attractiveness of target items in recommendation systems. An attribute comparison-based method was proposed to roughly estimate the dominance of the target item compared with other items. Teppan et al. [10] proposed an algorithm to mitigate the decoy effect in recommendation systems. First, they defined two models, namely the set-independent model (SIM) and set-dependent model (SDM), to identify item-set constellations exhibiting high biasing potential. SIM reflects a set- independent utility score (i.e., objective score), whereas SDM reflects a set-dependent utility score (i.e., subjective score). Even if the objective score of item A remains the same, the set-dependent utility score of item A, that is, the subjective score, will increase depending on the nearby placed items that have lower utility scores compared with item A. Therefore, the difference between the two scores indicates the existence of the decoy effect. They verified the above hypothesis through user experiments to determine the decoy effect and mitigate it by rearranging the ordering items. The closest work to this study is that by Rafai et al. [7], which explored the decoy effect in a real environment. They evaluated the decoy effect on an online flight aggregator website with real attributes but added the fastest and cheapest flights to the actual recommended flights. The limitation of their study is that intentionally added flights cannot be guaranteed to exist, which does not reflect the actual situation or users’ behaviors. The studies mentioned above proposed a method for detecting and mitigating the decoy effect; however, the common limitation is that the mechanism and degree of decoy effect in actual recommendation systems are not known. 3.2. Attribute Expression and Decoy Effect The way in which the attributes are expressed also affects the decoy effect. Cui et al. [2] investigated the impact of price precision on the decoy effect. The researchers compared the decoy effect with a precise price presentation and rounded price presentation and demonstrated that the decoy effect increased when the price was accurately expressed. Yoo et al. [14] investigated the decoy effect when the number of competitors increased and demonstrated that an increase in the number of competitors reduces the decoy effect. Dimara et al. [3] suggested that a graphical representation of attributes, such as bar charts, can make the decoy effect less severe. They also developed a tool that allows users to discard distracting attributes and assist in item selection to avoid the decoy effect. 4. Preliminary User Study to Identify Item Comparison Tendency 4.1. Proposed Method for Evaluating Item-Comparison Tendency Before evaluating the decoy effect in actual recommendation systems, we need to emphasize the importance of understanding the users’ habits when comparing items while browsing e-commerce websites, which has been neglected in previous studies. Although Teppan et al. [10] proposed the SDM to investigate the decoy effect, they calculated the set-dependent utility score using only the top-ranked items, which is usually not the case in real recommendation systems. For instance, Amazon e-commerce sites output many items over many rows with several web pages once a user submits a search query. Therefore, defining the set of items is indispensable for detecting the decoy effect in real recommendation systems. Note that the set represents the constellations of items that users tend to compare. We hypothesize that users are more likely to compare items displayed close to each other (e.g., in the same row). To confirm the hypothesize, we adopted a method for determining users’ habits when comparing items, which reveals how the users' consecutive clicks are concentrated. We assume that two consecutive clicks are estimated as a comparison of items because consecutive clicks occur in a relatively short time interval compared with the overall time spent browsing the website. Assume that user 𝑢 clicks 𝑛𝑢 (𝑛𝑢 ≥ 2) items with the sequence 𝑆𝑢 = (𝑖1 , 𝑖2 , … , 𝑖𝑢𝑛 ), where 𝑖𝑘 indicates the k-th item clicked by user 𝑢. We define the intra-row comparison ratio (IRC ratio) as the probability that two consecutive item clicks are in the same row. For instance, if a user clicks four items 𝑖1 , 𝑖2 , 𝑖3 , and 𝑖4 sequentially, where 𝑖1 and 𝑖4 are displayed in row1, whereas 𝑖2 and 𝑖3 are displayed in row2, the IRC ratio is calculated as 1/3 because the consecutive click pair (𝑖2 , 𝑖3 ) is in the same row, whereas the other pairs (𝑖1 , 𝑖2 ) and (𝑖3 , 𝑖4 ) are in different rows. This definition can be used to calculate the IRC ratio to verify our hypothesis. Assume that there are 𝑥 rows of items and the user’s clicks are randomly distributed among the rows, the IRC ratio is 1 1 calculated as 𝑥 ∗ (𝑛𝑢 − 1)/(𝑛𝑢 − 1) = 𝑥, i.e., (the number of consecutive item clicks in the same row)/ (the total number of consecutive item clicks). If the observed IRC ratio is much higher than that in the random case, it is reasonable to conclude that users have a higher probability of comparing items displayed in the same row. 4.2. Experiment to Identify Item Comparison Tendency in Real E-commerce Websites We invited 27 university students with experience in online shopping and who understand Japanese, English, and Chinese. We then asked them to participate in 2-h experiments on three real e-commerce sites, including amazon.au, amazon.jp, and jd.com (hereafter referred to as AmazonAU, AmazonJP, and JD, respectively). Note that we used three different e-commerce sites in different countries to ensure the generalization of the experimental results. Initially, the participants imagined their real shopping process based on the assumption of buying a monitor. Then, each participant was invited to participate in the experiment with the three e-commerce sites separately. After entering the keyword “monitor” in the search form, the e-commerce site returned the relevant items on multiple web pages, each of which has multiple rows of items. AmazonJP and AmazonAU output 4 items per row, and each page has 15 rows, whereas JD outputs 5 items per row, and each page has 12 rows. Table 1 presents a summary of the statistics of the three websites. Note that the participants entered Japanese “monitor” and Chinese “monitor” on AmazonJP and JD, respectively to ensure that the search results were relevant. Finally, we asked the participants to explore the top three webpage results to compare the items to buy, followed by recording the items they were interested in clicking on while browsing the recommended items. The participants’ tendency to compare items using the IRC ratio is presented in Table 1. Table 1 shows that the consecutive clicks tend to occur in the same row at a higher probability than random comparison (4.95 times on AmazonJP, 6.23 times on AmazonAU, 4.64 times on JD), which demonstrates the large gap between the observed IRC ratio and that with random comparison. Note that the random comparison assumed that clicks occur randomly in the result items across the three pages. Besides, even if we assume the users’ all clicks stayed within the same page, i.e., the users did not click the items in the other pages except for the first page, the observed IRC ratio is still higher at least a factor of 1.55 comparing the IRC ratio when all the clicks occur randomly within the same page (the baseline IRC ratio with random comparison is tripled). Therefore, we can conclude that users tend to compare items within the same row. 5. Decoy Effect in Real E-Commerce Websites In this section, we propose a method for evaluating the decoy effect on real e-commence recommendation systems, including 1) a detection method for row-based decoy effect and 2) a new metric called IRDE Rate to evaluate the degree of the row-based decoy effect. The notations used for the decoy effect in real e-commerce websites are summarized in Table 2. Table 1 Statistics of Three Websites and Intra-row Comparison Ratio (IRC ratio) Website #Items #Rows #Pages #Rows Observed IRC ratio with per row per in total IRC ratio random page comparison (baseline) AmazonJP 4 15 15 ∗ 3 3 0.109 0.022* = 45 AmazonAU 4 15 3 15 ∗ 3 0.137 0.022* = 45 JD.com 5 12 3 12 ∗ 3 0.130 0.028* = 36 *The calculation method is mentioned in section 4.1 5.1. Preliminary Assume that the recommended results returned by an e-commerce website contain 𝑛 rows denoted as 𝑅 = {𝑟1 , 𝑟2 … , 𝑟𝑛 }, where items in multiple pages are inlined. For instance, the website returns 3 pages and 15 rows of items per page; in total, there are 45 rows. Each row 𝑟𝑘 = {𝑖 𝑘,1 , 𝑖 𝑘,2 , … , 𝑖 𝑘,|𝑟𝑘 | } contains multiple items, where 𝑖 𝑘,𝑥 is the 𝑥-th item in row 𝑘. As described in Section 4, we extend the original definition of the decoy effect [12] to propose a row-based detection method to check the existence of a decoy in row 𝑟𝑘 for real e-commerce websites, as given in Eq. 1. 1, ∃𝑖 𝑘,𝑡 , ∃𝑖 𝑘,𝑐 , ∃𝑖 𝑘,𝑑 ∈ 𝑟𝑘 , 𝑖 𝑘,𝑡 ≠ 𝑖 𝑘,𝑐 , 𝑖 𝑘,𝑡 ≠ 𝑖 𝑘,𝑑 , 𝑖 𝑘,𝑐 ≠ 𝑖 𝑘,𝑑 , (1) 𝐸𝑥𝑖𝑠𝑡𝐷𝑒𝑐𝑜𝑦 𝑟 = { 𝑘,𝑡 𝑘,𝑐 𝑘 𝑖 𝑘,𝑑 ∈ 𝑖𝐴𝐷𝐸 𝑜𝑟 𝑖 𝑘,𝑑 ∈ 𝑖𝐴𝐷𝐸 0, 𝑜𝑡ℎ𝑒𝑟𝑠 𝑘,𝑡 𝑘,𝑐 , where 𝑖𝐴𝐷𝐸 and 𝑖𝐴𝐷𝐸 denote the areas dominated by items 𝑖 𝑘,𝑡 and 𝑖 𝑘,𝑐 , respectively [10]. If item 𝑖 𝑘,𝑑 that is the decoy of 𝑖 𝑘,𝑡 or 𝑖 𝑘,𝑐 exists for any two item pairs 𝑖 𝑘,𝑡 and 𝑖 𝑘,𝑐 , we detect the presence of the decoy item in row 𝑘, where 𝑖 𝑘,𝑡 , 𝑖 𝑘,𝑐 , and 𝑖 𝑘,𝑑 are items in row 𝑘. We extract a set of rows in which the decoy item is included, thereby forming a set 𝐷𝑅 = {𝑟𝑘 | 𝐸𝑥𝑖𝑠𝑡_𝐷𝑒𝑐𝑜𝑦𝑟𝑘 = 1}. For each row 𝑟𝑘 ∈ 𝐷𝑅, we generate a decoy item set 𝐷𝑟𝑘 , as given in Eq. 2. 𝑘,𝑡 𝐷𝑟𝑘 = {𝑖 𝑘,𝑑 | 𝑖 𝑘,𝑑 ∈ 𝑟𝑘 , ∃𝑖 𝑘,𝑡 ∈ 𝑟𝑘 , 𝑖 𝑘,𝑑 ∈ 𝑖𝐴𝐷𝐸 } (2) From Eq. 2, we generate the target item set 𝑇𝑟𝑘 , as given in Eq. 3. Table 2 Notations Notation Definition 𝑅 𝑅 = {𝑟1 , 𝑟2 … , 𝑟𝑛 }; Set of rows of items returned by an e-commerce website 𝑟𝑘 𝑟𝑘 = {𝑖 𝑘,1 , 𝑖 𝑘,2 , … , 𝑖 𝑘,|𝑟𝑘 | }; Set of items in row 𝑘 𝑖 𝑘,𝑥 The 𝑥-th item in row 𝑘 𝑘,𝑥 𝑖𝐴𝐷𝐸 Areas dominated by item 𝑖 𝑘,𝑥 ; if other items are in this area, ADE decoy effect in favor of item 𝑖 𝑘,𝑥 will occur 𝐷𝑟𝑘 Set of decoy items 𝑖 𝑘,𝑑 in row 𝑘 defined by Eq. 2 𝑇𝑟𝑘 Set of target items 𝑖 𝑘,𝑡 in row 𝑘 defined by Eq. 3 𝐶𝑟𝑘 Set of competitor items in row 𝑖 𝑘,𝑡 in row 𝑘 defined by Eq. 4 𝑘,𝑡 𝑇𝑟𝑘 = {𝑖 𝑘,𝑡 | 𝑖 𝑘,𝑡 ∈ 𝑟𝑘 , ∃𝑖 𝑘,𝑑 ∈ 𝐷𝑟𝑘 , 𝑖 𝑘,𝑡 ∉ 𝐷𝑟𝑘 , 𝑖 𝑘,𝑑 ∈ 𝑖𝐴𝐷𝐸 } (3) The rest of the items in row 𝑟𝑘 are categorized into a competitor item set 𝐶𝑟𝑘 , as given in Eq.4. 𝐶𝑟𝑘 = {𝑖 𝑘,𝑐 | 𝑖 𝑘,𝑐 ∈ 𝑟𝑘 , 𝑖 𝑘,𝑐 ∉ 𝐷𝑟𝑘 , 𝑖 𝑘,𝑐 ∉ 𝑇𝑟𝑘 } (4) Each item belongs to and only belongs to one set (i.e., 𝐷𝑟𝑘 ∩ 𝑇𝑟𝑘 ∩ 𝐶𝑟𝑘 = ∅). That is, we set the priority as 𝐷 > 𝑇 > 𝐶. 5.2. Row-based Decoy Effect Detection Method In this study, we propose a method for detecting decoy items in each row of recommendation results, as described in this sub-section. Previous studies [2] [12] [14] investigated the decoy effect using an artificially arranged set of items by comparing the distributions of users’ selections of a target item and competitor item with adding and discarding a decoy item. However, the existing method cannot be directly applied to a real e-commerce website because users may be confused if an item is discarded or added. Therefore, a more effective and feasible method that replaces decoy items with a common item is proposed. The concept of detecting decoy items in a row is based on the fact that the decoy item has low attribute values and a low selection rate compared with other items in the row because the purpose of the decoy effect is to boost the selection rate of the target item. We can detect decoy items based on the above idea to confirm that 1) the decoy-suspected item has a low selection rate compared with other items in the same row and 2) the difference in the selection rate in the row is reduced if the decoy- suspected item is replaced with a decoy-free item. Details of the steps are presented in Algorithm 1. Note that we used a common item as a decoy-free item, where the common item is a randomly selected item from the same e-commerce recommended items and is not a decoy item of any items in the same row. 5.3. Intra-row Decoy Effect Rate (IRDE Rate) We further propose a metric called the intra-row decoy effect rate (IRDE Rate) for calculating the degree of the row-based decoy. The IRDE rate indicates the risk of the decoy effect by pairing items (𝑖 𝑡 and 𝑖 𝑐 ) in the same row to detect whether other items in the same row are decoys of 𝑖 𝑡 or 𝑖 𝑐 , as given in Eq. 5. 𝑂𝑐𝑐𝐷𝑟𝑘 (5) 𝐼𝑅𝐷𝐸 𝑅𝑎𝑡𝑒𝑟𝑘 = 𝑛 ∗ 𝐶𝑜𝑚𝑏(𝑛 − 1, 2) , where 𝑂𝑐𝑐𝐷𝑟𝑘 is the number of combinations in which the decoy effect exists in row 𝑘, calculated using Eq. 6, 𝑛 is the the number of items displayed in row 𝑟𝑘 , 𝐶𝑜𝑚𝑏 is the combination function, and 𝐶𝑜𝑚𝑏(𝑛 − 1, 2) denotes the number of combinations to select two out of 𝑛 − 1. For instance, there are three items 𝑖 𝑘,1 , 𝑖 𝑘,2 , 𝑖 𝑘,3 displayed in row 𝑘, 𝑖 𝑘,1 is the decoy of 𝑖 𝑘,2 . The 𝐼𝑅𝐷𝐸 𝑅𝑎𝑡𝑒 is calculated as 1 1 3∗1 = 3. A higher IRDE rate indicates a higher risk of the decoy effect. 𝑖 𝑘,𝑑 | 𝑖 𝑘,𝑑 ∈ 𝑟𝑘 , ∀ 𝑖 𝑘,𝑡 , ∀𝑖 𝑘,𝑐 ∈ 𝑟𝑘 , 𝑖 𝑘,𝑑 ≠ 𝑖 𝑘,𝑡 ≠ 𝑖 𝑘,𝑐 (6) 𝑂𝑐𝑐𝐷𝑟𝑘 = | 𝑘,𝑡 𝑘,𝑐 | , (𝑖 𝑘,𝑑 ∈ 𝑖𝐴𝐷𝐸 𝑜𝑟 𝑖 𝑘,𝑑 ∈ 𝑖𝐴𝐷𝐸 ) 6. Experimental Evaluations In this section, we describe our user study experiment of the row-based decoy effect on real e- commerce websites targeting amazon.au, amazon.jp, and jd.com (hereafter shown as AmazonJP, AmazonAU, and JD). 6.1. Experiment Preparation We constructed three pseudo-e-commerce sites with the real recommended items returned by each e-commerce website, namely, Amazon JP, Amazon AU, and JD. Although a real e-commerce site can be used in the experiment, the returned recommended items may differ over time, resulting in an unfair outcome when comparing the decoy effect. To reproduce the same recommended item pages, we continuously crawled the recommended items for “monitor” from the three e-commerce websites for thirty days from 11/25/2021 to 12/24/2021 for both AmazonAU and AmazonJP, and from 12/16/2021 to 01/14/2022 for JD. The crawler was run once daily. After entering the keyword “monitor” in the search form in English, Japanese, and Chinese (depending on the e-commerce site), we collected the recommended items from each e-commerce website. We collected the top 100 items from each e- commerce website daily. Note that we set the number of gathered items to 100 because we assumed that users usually click items within the top 100 items. The crawler also remembered each item's ranking (i.e., displayed position) on the webpage to reproduce the same recommended item list. The collected information for each item includes the item name and detailed attribute page URL. Subsequently, we collected detailed attributes for each item, including price, resolution, refresh rate, and size. Finally, we reproduced the three pseudo-e-commerce sites (AmazonJP, AmazonAU, and JD) using the collected items with their attributes. Each pseudo-e-commerce site can display one row of items at a time in the same order as the collected order. Because we examined the row-based decoy effect, each pseudo e-commerce site shows only one row. The number of items per row is also the same as that on each real e-commerce site, that is, 4 items for AmazonAU and AmazonJP, and 5 items for JD. In addition, the pseudo-e-commerce site can switch between two different item placement patterns: 1) original page and 2) decoy-free page. The original page shows the same order of items collected from real e-commerce websites. The decoy-free page replaces the decoy item in the row with a common item, where the common item is a decoy-free item for any other items in the same row. By comparing the users' selection distributions of target and competitor items in the two different patterns, we can evaluate the existence of decoy effect, where the decoy items are detected by Algorithm 1. Some examples of the original and decoy-free pages are shown in Fig. 6 and Fig. 7, where the images are dummy images, and we used the same images for all items to avoid any side effects from the images. In Fig. 6, item3 is the target item and item4 is the decoy of item3. Item3 has higher attributes and a Algorithm 1: Row-based Decoy Effect Detection Algorithm Input: Row of items in original page 𝑟𝑘 = {𝑖 𝑘,1 , 𝑖 𝑘,2 , … , 𝑖 𝑘,|𝑟𝑘 | } with attribute values ′ ′ Output: One row decoy-free items 𝑟𝑘′ = {𝑖 𝑘,1 , 𝑖 𝑘,2 , … , 𝑖 𝑘,|𝑟′𝑘 | } 1 𝐷𝑟𝑘 ← ∅ 2 𝑟𝑘′ ← 𝑟𝑘 3 for 𝑗 ← 1 𝑡𝑜 |𝑟𝑘 |do 4 for 𝑡, 𝑐 in 𝑐𝑜𝑚𝑏𝑖𝑛𝑎𝑡𝑖𝑜𝑛𝑠(|𝑟𝑘 |,2) do 5 if 𝑗 ≠ 𝑡 and 𝑗 ≠ 𝑐 then 6 if 𝑖 𝑘,𝑗 is docoy of 𝑖 𝑘,𝑡 or decoy of 𝑖 𝑘,𝑐 then 7 𝐷𝑟𝑘 ← 𝐷𝑟𝑘 ⋃ {𝑖 𝑘,𝑗 } 8 end if 9 end if 10 end for 11 end for 12 while 𝐸𝑥𝑖𝑠𝑡_𝐷𝑒𝑐𝑜𝑦𝑟𝑘 = 1 do judge the existence of decoy effect in row 𝑘 (Eq.1) 13 for 𝑗 ← 1 𝑡𝑜 |𝐷𝑟𝑘 | do 14 𝑟𝑘′ . 𝑟𝑒𝑝𝑙𝑎𝑐𝑒(𝑖 𝑘,𝑗 , 𝑐𝑜𝑚𝑚𝑜𝑛_𝑖𝑡𝑒𝑚)  replace 𝑖 𝑘,𝑗 in 𝐷𝑟𝑘 with a randomly selected common item 15 end for 16 end while 17 return 𝑟𝑘′ lower price compared with item4. Item1 and item2 are two competitors. In Fig. 7, item4 was changed to a common item because item4 in Fig. 6 was detected as a decoy of item3 (target item), whereas the other two items are two competitor items. In Fig. 6 and Fig. 7, “1222” indicates the day of data collection (12/22), whereas the text in red indicates the number of decoy rows on that day. Users can press the “TURN TO ROW” button to view different rows of items. 6.2. User Study Experiment of Row-based Decoy Effect on Real Dataset A user study experiment was conducted on our constructed pseudo-e-commerce sites; we confirmed the efficiency of the decoy effect detection algorithm described in Section 5.2 and verified the existence of the decoy effect on real e-commerce websites using the row-based decoy effect detection method proposed in Section 5.3. 6.2.1. Statistics of Decoy Rows in the Dataset The number of decoy rows detected using the proposed algorithm for each e-commerce site per day is presented in Table 3. Table 3 shows that AmazonJP and AmazonAU have a similar average number of decoy rows, whereas JD has more than double the number of decoy rows for AmazonJP and AmazonAU. item1 item2 item3 item4 Figure. 6. Example of Pseudo-e-commerce site: original page item1 item2 item3 item4 Figure. 7. Example of Pseudo-e-commerce site: decoy-free page Table 3. Statistics of the number of decoy rows (detected using the proposed algorithm) Dataset Average number of Maximum number of Minimum number of decoy rows per day decoy rows decoy rows AmazonJP 4.6 11 1 AmazonAU 4.7 13 1 JD 11.4 16 10 6.2.2. Experimental Procedures We invited 20 university students as participants for the three-hour-long experiment; these participants were different from those in Section 4.2. At the beginning of the experiment, we asked the participants to imagine a real shopping process of buying a new monitor. The participants set their budgets according to their usual expenditure. Then, we introduced the experimental procedure on the pseudo-e-commerce sites as follows: 1) The experiments were conducted on three pseudo-e-commerce sites separately. 2) The participant was asked to access the decoy-free pages between Nov. 25th (1125) and Dec. 24th (1224). Then, multiple rows of items were shown individually. 3) For each row of items, the participant must select and record the item in which he or she has the most interest. 4) The participant must repeat steps 2) and 3) to complete the selection of items for all the rows. 5) The participant was asked to access the original pages between Nov. 25th (1125) and Dec. 24th (1224). Then, repeat steps 2) to 4). 6) The pseudo e-commerce site was changed to another one to repeat steps 1) to 5) until the experiments on the three e-commerce sites have been completed. Note that we asked the participant to input the duration for JD from Dec. 16th (1216) to Jan. 14th (0114) because the collected data duration is different from the other two e-commerce sites. During the experiment, we limited the number of rows displayed to each participant in step 2) to three to ensure that the participants had sufficient time to compare and select items. 6.2.3. Analysis of the Decoy Effect on E-commerce Sites We analyzed the data recorded by the participants consisting of the selected items in each row. Specifically, we calculated the average selection rate for each item (target and competitor items) in each row on the original and decoy-free pages. For the original pages, we calculated the selection rate of the decoy items, whereas for the decoy-free pages, we calculated the selection rate of the newly added common items. We hypothesize that replacing the decoy item with a common item decreases the selection rate for target and competitor items; that is, other items except for the common item. This is because when a decoy item exists, it has a lower selection rate than the other items, which increases the selection rate of the other items. However, when the decoy item is replaced with a common item, the selection rate of the other items decreases because the common item has a higher selection ratio than the decoy item. To verify this hypothesis, we investigated the following three perspectives. 1) Lower selection rate for the decoy item: We compared the selection rates of decoy items with the selection rates of the other items, that is, competitor and target items, in (a) of Figs. 8–10, and confirmed that the selection rates of decoy items are relatively small (0.062 for AmazonJP, 0.065 for AmazonAU, and 0.054 for JD), which satisfies one of the characteristics of the decoy item. Note that the number of selections confirmed the same trend. 2) Effect of replacing with a common item: We compared the reduction in the selection rates shown in (c) of Figs. 8–10 and confirmed that the selection rates of both competitor and target items decreassed after replacing the decoy item with a newly added common item, validating our hypothesis. For instance, for AmazonJP (Fig. 8), when newly added common items replace decoy items, the average selection rates for competitor and target items decrease by 0.115 and 0.249, respectively. The same trends were observed for AmazonAU (Fig. 9) and JD (Fig. 10). Note that the number of selections confirmed the same trend. 3) Difference in the selection rates of the target and competitor items: The reduction rate of the target items is higher than that of the competitor items by a factor of 2.17 (AmazonJP), 5.02 (AmazonAU), and 4.18 (JD), which shows that the bias that makes the target items more likely to be selected has been reduced. Note that the number of selections confirmed the same trend. In Fig. 10, we notice that the selection rate of competitor items is higher than target items. The presence of decoy items can influence the selection rate of target items, but it does not necessarily mean that the selection rate of target items will increase to higher than competitor items. The selection rate also depends on user's preference and the quality of the items. The above three confirmed perspectives demonstrate that our method can detect the decoy effect and its existence on the three e-commerce websites. 6.3. Intra-row Decoy Effect Rate (IRDE Rate) Transition on a Real E- commerce Dataset Figs. 11–13 show the intra-row decoy effect rate (IRDE rate) transition for AamzonJP, AmazonAU, and JD, respectively depicting the degree of the row-based decoy effect. It can be observed from the figures that AmazonJP has the highest average IRDE rate of 0.1467 ranging from 0.1 to 0.2049 for the thirty days data. The peak IRDE rate was observed on 12/12/2021, whereas the IRDE rate decreased to (a)original page (b) decoy-free page (c) Reduction Rate Figure. 8: Item Selection Distribution on AmazonJP (a)original page (b) decoy-free page (c) Reduction Rate Figure. 9: Item Selection Distribution on AmazonAU (a)original page (b) decoy-free page (c) Reduction Rate Figure. 10: Item Selection Distribution on JD the minimum value on 12/18/2021. AmazonAU and JD also had different IRDE rates daily; they had average IRDE rates of 0.0627 and 0.0873, respectively. These IRDE rates do not indicate any intentional decoy effects; however, it should be noted that we encounter the decoy effect every day, where we may naturally gravitate toward some items in the recommended items in each row. 7. Conclusion and Future Work Unlike in previous studies on decoy effect that are based on virtual items, this study proposed a new method for evaluating the decoy effect on real e-commerce websites, including 1) a user study experiment to verify that users tend to compare items displayed in the same row, 2) a detection algorithm to extract the rows with a possible decoy effect by adopting a new metric called the IRDE Rate, and 3) a user study experiment to verify the decoy effect on real e-commerce recommendations, which affect users’ item selection decisions. We confirmed the existence of the decoy effect in real e- commerce recommendations by replacing decoy items with common items; subsequently, we confirmed a reduction in the IRDE rate by a factor of 2.17, 5.02, and 4.18 for amazon.jp, amazon.au, and jd.com, respectively. In the future, we will extend the row-based decoy effect to page-based decoy effect; that is, confirming the decoy effect among items in different rows. In addition, we plan to confirm the decoy effect in more detail, including how the decoy effect is affected by the range of values of the attributes; for instance, does the decoy effect still exist if the price difference between the target and decoy items is large? Figure 11: IRDE Rate Transition of the AmazonJP Dataset (11/25/2021 to 12/24/2021) Figure 12: IRDE Rate Transition of the AmazonAU Dataset (11/25/2021 to 12/24/2021) Figure 13: IRDE Rate Transition of the JD Dataset (12/16/2021 to 1/14/2022) 8. Reference [1] Bateman, I. J., Munro, A., and Poe, G. L. 2008. Decoy effects in choice experiments and contingent valuation: Asymmetric dominance. Land Economics, 84(1), 115-127. [2] Cui, Y. G., Kim, S. S., and Kim, J. 2021. Impact of preciseness of price presentation on the magnitude of compromise and decoy effects. Journal of Business Research, 132, 641-652. [3] Dimara, E., Bezerianos, A., and Dragicevic, P. 2016. The attraction effect in information visualization. IEEE transactions on visualization and computer graphics, 23(1), 471-480. [4] Kim, J. 2017. The influence of graphical versus numerical information representation modes on the compromise effect. Marketing Letters, 28(3), 397-409. [5] Mandl, M., Felfernig, A., Teppan, E., and Schubert, M. 2011. Consumer decision making in knowledge-based recommendation. Journal of Intelligent Information Systems, 37(1), 1-22. [6] Ouyang, M., and Mahmood, A. 2004. Does the Decoy Effect Exist in the Marketplace? An Examination of the Compromise Effect. Congrés 2004 de l'Association des Sciences Administrative du. [7] Rafai, I., Babutsidze, Z., Delahaye, T., Hanaki, N., and Acuna-Agost, R. 2022. No evidence of attraction effect among recommended options: A large-scale field experiment on an online flight aggregator. Decision Support Systems, 153 (113672), 1-11. [8] Simonson, I. 1989. Choice based on reasons: The case of attraction and compromise effects. Journal of consumer research, 16(2), 158-174. [9] Teppan, E. C., and Felfernig, A. 2009. Calculating decoy items in utility-based recommendation. In Proceeding of International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, 183-192. [10] Teppan, E. C., and Felfernig, A. 2012. Minimization of decoy effects in recommender result sets. Web Intelligence and Agent Systems: An International Journal, 10(4), 385-395. [11] Teppan, E. C., and Zanker, M. 2015. Decision biases in recommender systems. Journal of Internet Commerce, 14(2), 255-275. [12] Teppan, E. C., Felfernig, A., and Isak, K. 2011. Decoy effects in financial service e-sales systems. In Proceedings of the Workshop Decisions@ RecSys, in Conjunction with the Fourth ACM Conference on Recommender Systems, 1-8. [13] Theocharous, G., Healey, J., Mahadevan, S., and Saad, M. 2019. Personalizing with human cognitive biases. In Proceedings of Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization, 13-17. [14] Yoo, J., Park, H., and Kim, W. 2018. Compromise effect and consideration set size in consumer decision-making. Applied Economics Letters, 25(8), 513-517.