Limited transparency in targeted advertising on online content delivery platforms can breed mistrust for both viewers (of the content and ads) and advertisers. This user study (n=864) explores how explanations for targeted ads can bridge this gap, fostering transparency for two of the key stakeholders. We explore participants' preferences for explanations and allow them to tailor the content and format. Acting as viewers or advertisers, participants chose which details about viewing habits and user data to include in explanations. Participants expressed concerns not only about the inclusion of personal data in explanations but also about the use of it in ad placing. Surprisingly, we found no significant diferences in the features selected by the two groups to be included in the explanations. Furthermore, both groups showed overall high satisfaction, while “advertisers” perceived the explanations as significantly more transparent than “viewers”. Additionally, we observed significant variations in the use of personal data and the features presented in explanations between the two phases of the experiment. This study also provided insights into participants' preferences for how explanations are presented and their assumptions regarding advertising practices and data usage. This research broadens our understanding of transparent advertising practices by highlighting the unique dynamics between viewers and advertisers on online platforms, and suggesting that viewers' priorities should be considered in the process of ad placement and creation of explanations.
As content delivery platforms (e.g. video-on-demand and music streaming platforms) increasingly utilize
targeted advertisements, their influence on user engagement and content perception is undeniable
[
Within internet advertising, explanations have been touted as crucial in enhancing transparency
and addressing users’ concerns about data usage [
The level of detail in an explanation also remains a point of debate. While detailed explanations may
seem like the ideal solution for enhancing user transparency, overly complex explanations — filled with
technical jargon or excessive information — can overwhelm users. This may lead to a phenomenon
known as over-reliance, where users accept the explanation at face value without fully understanding
it. Some researchers argue that comprehensive explanations are necessary for transparency [
In this work we aim to advance transparency in targeted advertisements by evaluating essential factors to be integrated into explanations for the two of the main stakeholders of content delivery platforms: (i) viewers – customers of the platform consuming the ads; and (ii) advertisers – which produce the ads that are presented to the viewers. We seek to broaden the scope of research on transparent advertising, moving beyond its conventional focus on social media platforms, investigating the perspectives of two stakeholders regarding the content and presentation of explanatory information accompanying advertisements. Our research is guided by two research questions: • RQ1. Which features do viewers and advertisers prioritize for inclusion in advertisement explanations to improve transparency? • RQ2. Which discrepancies exist, in terms of perceived transparency and satisfaction, between the explanations provided by viewers and the explanations provided by advertisers?
To answer our research questions, we conducted a pre-registered randomized controlled trial with four between-subject factors (288 participants for each of three sessions).1 Participants were recruited using the online participant pool Prolific. 2 Proficiency in English and a minimum age of 18 were prerequisites for participation. Each participant could have taken part in the study only once. 3 The study was approved by the ethical committee of our institution.
We conducted our user study using a video-on-demand (VOD) platform as an example of a content
delivery platform with targeted ads. This approach was chosen to evoke familiar scenarios that users
frequently encounter and help them illustrate how explanations would be used. In the context of VOD
platforms, specific advertising practices are employed: ads are often brand-awareness tools, with key
targeting features based on demographic information and previously consumed content [
We apply an adapted Find-Fix-Verify crowdworker workflow [
Hence, we formalize the following hypotheses related to the sets of features selected to be included in the explanations (hypotheses related to RQ1): • H1a: The average number of the features, chosen by viewers will difer from the average number of features, chosen by advertisers. • H1b: The sets of features, chosen by viewers will be diferent from the sets of features, chosen by advertisers.
Additionally, we propose that the difering priorities of viewers and advertisers will result in variations in their evaluations, influenced by the roles of participants generating the explanations. Based on these considerations we formalize the following hypotheses related to RQ2: There is a diference in the evaluation of satisfaction for the explanation ...: • H2a: ... generated by the viewer and evaluated by the viewer versus the explanation generated by the viewer and evaluated by the advertiser. • H2b: ... generated by the advertiser and evaluated by the advertiser versus the explanation generated by the advertiser and evaluated by the viewer. • H2c: ... generated by the viewer and evaluated by the advertiser versus the explanation generated by the advertiser and evaluated by the advertiser. • H2d: ... generated by the advertiser and evaluated by the viewer, and the explanation generated by the viewer and evaluated by the viewer.
There is a diference in the evaluation of transparency for the explanation ...: • H2e: ... generated by the viewer and evaluated by the viewer, and the explanation generated by the viewer and evaluated by the advertiser. • H2f: ... generated by the advertiser and evaluated by the advertiser, and the explanation generated by the advertiser and evaluated by the viewer. • H2g: ... generated by the viewer and evaluated by the advertiser, and the explanation generated by the advertiser and evaluated by the advertiser. • H2h: ... generated by the advertiser and evaluated by the viewer, and the explanation generated by the viewer and evaluated by the viewer.
Our study consisted of three subsequent sessions, in which participants were asked to either (i) create an explanation, (ii) refine, or (iii) evaluate an explanation generated in the previous sessions. Each of the sessions consisted of a pre-survey, a task (creating/refining/evaluation of explanation) and a debrief. In all the sessions participants were assigned either the role of viewer or advertiser. Each participant engaged in only one session for a single specific scenario. A visual representation of the experimental set up is presented in Figure 2). Pre-survey. In the first step, after obtaining informed consent, we asked participants for their gender and age group. This information was collected for statistical purposes to ensure a representative sample. Role descriptions for the participants. Participants were randomly divided into two groups: (i) Viewers - representing platform users who receive ads while watching content; (ii) Advertisers representing advertisers, who are providing their ads to viewers through the platform. We explained both roles simply and neutrally, trying to avoid any implicit influence on their choices. 4 4Role descriptions can be found in the preregistration of the study.
Hypothetical scenarios. We defined eight scenarios, each consisting of a set of features and corresponding values, that participants could choose to incorporate into their explanations. While real-world applications may involve a wider range of possibilities, to ensure feasibility within this research, we provided a selection of hypothetical scenarios representing diverse user groups of a VOD platform, even those relevant to smaller segments of users. 5
These scenarios were derived from an analysis of data obtained from a commercial online platform, focusing on viewers subscribed to a plan inclusive of advertisements. However, it is important to note that our use of platform data analysis was strictly limited to informing feature selection. No personal user data from the platform was incorporated into the experimental scenarios. The participants were provided with a description of the scenario, as well as with the list of the features to choose from. 6 Variables describing the scenario were also used as features for generating explanations for advertisements. These include age group, gender, preferred content type (pref. content type), preferred genres (pref. genred), programs watched per week (programs per week), hours watched per day (hours per day), preferred days of watching (pref. type of day), preferred times of watching (pref. time of day), and preferred device (pref.device). This set of categorical features was used consistently across all scenarios, with varying feature values.
Session 1 – Generation. Participants were asked to generate an ad explanation based on their role and given information about a hypothetical platform user. They selected relevant features describing a user from a predefined set, which included usage patterns, consumption levels, demographics, preferred content, and device. Note that participants may not necessarily align with these parameters. Session 2 – Revision. After the initial explanations were generated, a new group of participants was assigned to revise and improve them. These Revision participants were given a role description, details about the hypothetical user receiving the advertisement, the set of features, and the previously created explanation. Using this information, their task was to refine and enhance the original explanation. Session 3 – Evaluation. Once the explanations were refined, yet another round of participants was recruited and asked to evaluate a refined explanation in terms of perceived transparency and satisfaction, using two 7-points Likert-scale questions.
Debriefing. Lastly, participants answered two open-ended questions about their preferences for advertisement explanations. First, they were asked if there was any additional valuable information they believed should be included in the explanations. They were then invited to share any further comments or feedback. Finally, a debriefing message was showed, with a short explanation of the objectives of the study.
Independent variables. The combinations of creators (generation / Find) and evaluators (evaluation / Verify) of explanations was used to determine the (categorical) between-subject factors. In total, four combinations were used: (i) Viewer - Viewer; (ii) Viewer - Advertiser; (iii) Advertiser - Advertiser; and (iv) Advertiser - Viewer.
Dependent Variables. For each revised explanation, we assessed two dependent variables by asking participants to rate their agreement with statements using a seven-point Likert scale ranging from “strongly agree” to “strongly disagree”.
• Perception of satisfaction (ordinal): “The user will be satisfied with the received explanation.” 5Description of the scenarios can be found in the preregistration of the study. https://osf.io/w58b2/?view_only= 9d649177d822474f85924efce36b0f82 6An example of the description, presented to the participant, can be found in the preregistration of the study.
• Perception of transparency (ordinal): “I understand why the user received an advertisement.” Descriptive Variables. We also collected demographic data from participants, who had the option to opt out of providing this information.
Participants. We recruited 1035 participants, with 864 of them (288 per stage: generation, revision, evaluation) passing all the attention checks. The number of required participants was determined through a power analysis on the basis of the statistical analysis plan and the number of tested hypotheses. 7 The average session duration was 8 minutes, exceeding the anticipated 5 minutes. To ensure fair compensation and adhere to the recommended Prolific rate of £9/hour, the remuneration for each batch was adjusted based on the observed average time spent.8 The resulting sample exhibited a well-balanced gender distribution: 454 male, 387 female, 18 non-binary participants, 7 respondents opted not to specify their gender. Regarding age groups, the distribution was as follows: 18-24 years old (225), 25-34 years old (351), 35-44 years old (172), 45-54 years old (85), and 55 years old and above (29). Two participants preferred not to disclose their age group. RQ1: diferences between features selected by two groups of participants. Our analysis found no significant diference between the count of features selected by advertiser and viewer groups ( H1a) (1.9 vs. 1.8, with standard deviations of 3.4 and 3.5, respectively) used in explanation (two-tail T-test, = − 0.57, = 0.56). We also found no significant diference in the features selected by participants when comparing the Generation and Revision explanations (two-tail T-test, = 0.131, = 0.896). Additionally, participants from both groups did not significantly difer in how often (See Fig. 4) they used specific features when creating explanations ( H1b). The analysis resulted in a Chi-Square statistic 2 of 2.538 ( = 0.959, Chi-Square Test for Homogeneity), with 8 degrees of freedom. RQ2: discrepancies in terms of perceived transparency and satisfaction between two groups of participants. While our analysis found no significant diference in satisfaction ( H2a-H2d) based 7Analysis plan can be found in the preregistration of the study. 8https://www.prolific.com/resources/how-much-should-you-pay-research-participants.
on who created and evaluated the explanations (viewers vs. advertisers) (see Table 1), it did reveal a significant efect of the evaluator’s role on the evaluation of perceived transparency ( H2e-H2h). Advertisers perceived explanations as significantly more transparent than viewers. Their evaluations were 19% higher (more positive) on a scale ranging from "Somewhat agree" to "Strongly agree." In order to investigate the efect of the explanation creator role (advertiser vs. viewer) on perceived transparency (hypotheses H2e & H2f), we conducted two separate two-tailed t-tests. The results verified hypothesis H2e, revealing that advertisers rated explanations created by viewers as significantly more transparent ( = 3.547, = 135.8, < 0.001). However, no significant diference was found in advertiser evaluations of transparency based on the explanation creator. Additionally, explanations generated by the participants were generally well-received by both viewers and advertisers. Around 79% of respondents found them transparent (rated "Somewhat agree" to "Strongly agree"), and 58% found them satisfactory (rated "Somewhat agree" or higher).
Qualitative analysis of the feedback from participants. In addition to the quantitative analysis of the responses, we also examined which information participants valued in explanations. In general, they expressed that they preferred explanations that avoid detailed information about the data used for targeting. This suggests that transparency regarding targeting may raise privacy concerns for some participants, potentially outweighing the benefits of such detail: "As a user, I would have some concerns about my personal information if ad explanations had a lot of information about me". Moreover, participants mentioned that they do not want to see their demographic information in the explanations, even when they acknowledge that this data is used for targeting purposes: "It can be somewhat unsettling that the ad mentions age and gender because it can make the user feel their privacy is being violated". Respondents also acknowledged that it is hard to find a balanced approach in the creation process: "It must balance the right amount of detail with being understandable and I think it’s challenging."
Participants generally found the presented features suficient for explanations: "I think the most valuable information is already presented.", "I believe that’s the only information that should be used in the advertisement". Additionally, several participants suggested including geolocation information used on the platform. Nevertheless, participants highly evaluated the explanations even without specific features mentioned. For example, explanations like the following were rated as high as less specific ones (see below): "You are seeing this ad, because we’ve tailored it to match your interests. With your penchant for action series, thrill-seeking adventures, and a sprinkle of horror, we figured you’d appreciate what we’re showcasing. Given your preference for watching on your phone, we want to ensure you catch our recommendation at the ideal moments—whether it’s after 6:00 PM as you wind down or before 9:00 AM to kick-start your day. Keep an eye out for our suggestion; we believe you are going to love it to keep you company during the week!".
A less specific example would be: "You are seeing this ad, because you are eligible in the population sample of our advertisement. Our product is directed to users with similar traits with you and similar interests." However, too short explanations like "You are seeing this ad, because it relates to your content preferences as a user" were evaluated low w.r.t to both transparency and satisfaction.
Age group, Pref. genres
During the initial analysis of the participants’ data, we observed variability in how they followed the instructions: the level of detail in the explanations difered, and this did not always correspond with the number of features they selected. To further explore these discrepancies, we conducted an in-depth exploratory analysis of the data, focusing on the information actually included in the text of the explanations.
Disparities between selected features and textual explanations. Further analysis revealed a discrepancy between participants’ stated feature selections and their actual usage in explanations in both Generate and Revise sessions. While instructed to incorporate specific features, participants exhibited varying levels of detail, ranging from simple feature names to in-depth descriptions. To better understand these discrepancies we examined the frequency and depth with which participants mentioned selected features and their corresponding values in their explanations. Additionally, we aimed to determine if these patterns difered between the two sessions, where participants either generated explanations from scratch or revised existing ones. To assess the features usage, we labeled explanations and features using a binary classification system: • Corresponding features: If the feature was selected and at least the name of the feature was indicated in the explanation, the label for this feature was set to 1. If the selected feature did not appear in the explanation at all, the label was 0. • Explicitly mentioned features: If the feature was selected and the value of this feature was mentioned in the explanation, the label for this feature was set to 1. Otherwise, it was labeled 0.
Based on these labels, we defined two scores for each explanation: (i) the correspondence score, which represents the ratio of corresponding features to the total number of selected features, and (ii) the explicitness score, which measures the proportion of explicitly mentioned features relative to the selected features. We provide examples of the labelling process and the obtained scores for three explanations in Table 2.
We identified a discrepancy between selected features, and the features actually included in the explanations, with many features either omitted or mentioned without explicit value specification (see Fig. 4). Moreover, focusing on the selected features, we can notice a variation between Session 1 (generate) and Session 2 (revise) regardless of assumed stakeholder type. While we did not observe a statistically significant diference in the overall number of features selected (corresponding or explicit), this might be explained by the fact that features were both added and removed in Session 2. For example, we can see from Table 3 that some of the features (e.g., Gender) were rarely mentioned (corresponded with the explanation) in Session 2, while some of them were mentioned and removed with almost the same frequency between sessions (e.g. Preferable content type). Similar patterns were observed in the explicit mention of feature values within explanations (see Table 4). We also can see that features related to demographics frequently underwent removal during Session 2, with a comparatively low rate of addition. Table 5 demonstrates that while features such as Programs watching per week also exhibited a relatively low rate of explicit value mentions, the discrepancy between explicit and corresponding mentions was less pronounced compared to features like Gender and Age. This ifnding aligns with our initial observations, where participants expressed in feedback their aversion to incorporating demographic information into explanations, despite acknowledging its role in targeted advertising. Nevertheless, we observed a significant diference in correspondence ( = 3.88, < 0.001) and explicitness ( = 3.614, < 0.001) scores between the two sessions. Both scores were higher in Session 1. Furthermore, we compared explicitness and correspondence scores and overall values between advertisers and viewers, finding no significant diferences.
Influence of the particular features on satisfaction levels. We found a statistically significant influence of Preferable content type and Preferable genres on participant satisfaction with explanations ( = 0.05). Specifically, participants expressed significantly higher satisfaction with explanations that explicitly mentioned either Preferable content type or Preferable genres ( = 2.319, = 0.021). Moreover, satisfaction was further enhanced when both features were mentioned with explicit values ( = 3.526, = 0.001). We did not observe a significant correlation between explicitness or correspondence scores and participant satisfaction levels.
Selected features for ad explanations. The quantitative hypothesis testing and the qualitative
post-hoc analysis of the data ofered two insights into participants’ explanation preferences. First,
participants generally preferred explanations that focused on information related to their content
preferences. Second, explanations often did not include the actual feature values from the scenario,
with an exception for the features Preferable genres and Preferable content type. This may be attributed
to the fact that we put participants in the context of a VOD platform, main function of which is to
deliver content to the users. Participants likely recognize that the platform already collects and uses
this information, and they were generally comfortable with its usage. However, this practice could
contrast with the real-world scenarios, in which advertiser highly prioritise demographic information
in their targeting strategies [
Our exploratory analysis of the discrepancies between selected features and textual explanations further reinforced this trend. In the revision (Session 2), participants frequently omitted features or their values from explanations, even while recognizing their importance. Participants demonstrated a reluctance to include demographic information in their explanations, and during the revision phase, such information was frequently removed.
Perceived transparency and satisfaction from explanations. We did not observe a significant diference in participants’ satisfaction ratings for the explanations when representing diferent stakeholder groups. However, in terms of transparency, the explanations received high ratings overall. This suggests that explanations written in natural language may be generally well-received, regardless of whether they were created from the viewer or advertiser perspective. This finding also suggests that perceived transparency might be less sensitive than we expected to the specific features chosen or the number of features mentioned.
Qualitative observations. During the exploratory analysis, we observed a recurring theme in many explanations: participants frequently used compliments to the user in their explanations and often emphasized the user’s “uniqueness”: “you have an amazing taste”, “you’re all about the thrill”, “born explorer”, “expert”, “you are first to seek a knowledge” . This finding also corresponds with the previous insights from participants’ feedback, where they stated that “I’ve never felt anything good when advertisers tried to fit me in some box.” Despite this, generated explanations exhibited biases related to the demographic information presented in the scenario. For example, explanations directed at users aged 45+ often assumed the presence of a family, while those targeting users aged 25-44 frequently implied the use of VOD platforms as a means of relaxation or stress relief after a "hard day of work". Limitations of the study. We acknowledge that our study design, based on an online survey, has limitations. First, to ensure experimental feasibility, we employed crowd-sourced participants to generate explanations, rather than actual advertisers. While this approach provided a scalable way to gather data, it may have introduced limitations in how well participants could authentically represent the priorities of advertisers. Despite our eforts to minimize bias in the role instructions, participants might have struggled to fully grasp the strategic concerns advertisers have, such as optimizing ad reach and targeting specific demographics. We note however that the task formulation was considered realistic and relevant by our partners at a commercial online platform (RTL Nederland B.V.).9 Second, we did not expose respondents to the actual advertisements, which could have afected their ability to imagine the hypothetical scenario efectively. This design avoided the confounding influence of perceived relevance, but this is crucial to study in future work. While we are already working with a real platform, future studies will aim to involve real advertisers to enhance the theoretical grounding of the research and better represent the two key user groups — advertisers and viewers. By integrating more authentic stakeholder involvement, we can refine the theoretical framework and ensure that the study more accurately reflects the practical realities of both groups. Nevertheless, this study yields valuable feedback and a comprehensive collection of human-generated explanations that illustrate participant perceptions of the advertiser’s role.
Ethical Implications. A key tension emerged from our study: user privacy concerns clashed with
real-world advertising practices. Participants expressed discomfort with explanations including specific
details about their habits, inclusion of demographic information, and the data collection processes behind
targeted advertising. This highlights a disconnect between viewers’ desire for transparency and the
level of detail currently employed in ad explanations. While recommender systems research is exploring
the trade-of between personalization benefits and privacy risks when disclosing user information [
Explanations have been introduced in internet advertising contexts to improve transparency and
minimize users’ concerns about the use of their personal information [
To address this potential diference, we presented a between-subject user study (N=864) investigating the preferences about the user-related features to be included in the explanations, from the point of view of both viewers and advertisers. In a crowdsourced pipeline we analyzed the explanations generated, refined, and evaluated by the two groups. The refined explanations were evaluated in terms of transparency and satisfaction. Surprisingly, we found no diferences between advertisers and viewers in terms of the number of features they wanted to see in explanations. However, advertisers overall perceived the explanations to be more transparent. It was also surprising that explanations generated by “viewers” were well-received by ”advertisers”, and that we did not see many changes in the revision stage. Furthermore, diferently from what we could expect, the viewers found simple explanations very transparent, and they highlighted the desire to use short explanations. In contrast, they expressed privacy concerns about the use of demographic information such as age and gender in both advertisements placing and explanations for these. This contrast was further confirmed through an in-depth exploratory analysis, which showed that participants frequently omitted features they considered valuable from their own explanations, especially during the revision stage of the study.
This work highlights the importance of considering multiple stakeholders in the design of explanations, in particular considering viewers’ concerns about their personal information and explanation formulations. In the future, we plan to address the limitations of this study and involve real advertisers in the process of identifying the essential information to include into the explanation and how it can be adapted to the diferent advertising practices, involving more advanced techniques for targeting like the usage of the recommender systems.
This publication is part of the project ROBUST: Trustworthy AI-based Systems for Sustainable Growth with project number KICH3.LTP.20.006, which is (partly) financed by the Dutch Research Council (NWO), RTL, DPG, and the Dutch Ministry of Economic Afairs and Climate Policy (EZK) under the program LTP KIC 2020-2024. All content represents the opinion of the authors, which is not necessarily shared or endorsed by their respective employers and/or sponsors.