This study is the first to empirically investigate the use of certification labels as a solution to communicating the trustworthiness of AI to end-users. Through interviews ( = 12 ) and a Swiss censusrepresentative survey ( = 302 ) we investigated attitudes towards certification labels and their efectiveness in conveying trustworthiness in low- and high-stakes AI scenarios. The results showed that certification labels can significantly increase end-users' trust and willingness to use AI, particularly in high-stakes scenarios. However, the study also highlighted opportunities and limitations in addressing end-users concerns regarding the use of AI. The research provides insights and recommendations for designing and implementing certification labels as a promising constituent of trustworthy AI.
deemed the AI trustworthy. This work addressed this challenge by focusing on communicating
the outcomes of auditing processes to end-users using certification labels , commonly used in
other domains to certify that a product meets certain criteria and promotes trustworthiness [
To explore the efectiveness of certification labels in communicating AI trustworthiness, we conducted a mixed-method study with both interviews ( = 12 ) and a Swiss census-representative survey ( = 302 ) with end-users. The final sample for the interviews consisted of students (P2, P3, P4, P8, P11), bike messenger (P12), waitress (P1), dancer (P9), course manager (P7), management assistant (P6), intern (P10) and retired teacher (P5). The sample was predominantly female, with ten women and two men. Following Kapania et al., we used low-stake (music preference, route planning, price comparison) and high-stake (medical diagnosis, hiring procedure, loan approval) AI scenarios and measured both trust and willingness to use AI before and after presenting end-users with a certification label. The certification label used in this study was an existing label developed by the Swiss Digital Initative (SDI) for the broader context of digital trust.
The label is based on a catalog of verifiable and auditable criteria, co-developed by an academic expert group based on a user study on digital trust. Independent third-party audits are conducted following the catalog, and if all criteria are fulfilled, the label is awarded to the service. The audits are conducted following a catalog containing, at the time of the study, 35 criteria organized in four categories: 1. Security (criteria 1 - 12): What is the security standard? The service provider shall, e.g., ensure that the data is encrypted as it transfers so that third-parties cannot access it. 2. Data protection (criteria 13 - 20): How is the data protected? The service provider shall, e.g., assume responsibility for the appropriate management of the data. 3. Reliability (criteria 21 - 29): How reliable is the service or product? The service provider shall, e.g., take all actions required to safeguard the continuity of the service. 4. Fair user interaction (criteria 30 - 35): Is automated decision-making involved? The service provider shall, e.g., ensure that all users receive equal treatment and that there is no data-based service or price discrimination.
Supplementary materials, the data, and corresponding R-scripts are available on OSF: https: //osf.io/gzp5k/?view_only=709e50a07d2f46c3a10474f1d125b32f.
The results of our study, from both the interviews and the survey, suggest that certification labels
can efectively communicate the trustworthiness of AI. Quantitative findings of the
censusrepresentative survey demonstrate that presenting end-users a certification label significantly
increases end-users’ trust and willingness to use AI in both low- and high-stake scenarios
(see Appendix). End-users were found to have a higher preference for certification labels in
high-stake scenarios, and the impact of a certification label on trust and willingness to use AI
was more pronounced in high-stakes scenarios. This suggests that compliance with mandatory
requirements for AI in high-stake scenarios could be efectively communicated to end-users
through certification labels in addition to the proposed voluntary labeling for low-stake AI
scenarios [
Qualitative findings of the interviews show that end-users have positive views of AI certification labels and that they provide the opportunity to increase trust, perceived transparency and fairness, and provide standardization for AI. Furthermore, participants indicated that certification labels can mitigate their data-related concerns regarding privacy and data protection. Certification labels were perceived by participants as efective tools for addressing their datarelated concerns, by holding certified parties accountable for complying with the standards set in the catalog. For participants, the standards set by the certification label regarding security and data protection represent the acceptable minimum to consider using an AI system. However, some limitations concerning the use of AI remain (see Appendix). The interviews also provide inhibitors and facilitators for the efective use of certification labels in the context of AI. End-users expressed a preference for independent audits and the dificulty of communicating subjective criteria such as ”fairness,” the meaning of which can be ambiguous. Certification labels may not address all end-user’s concerns (e.g., AI performance measures) and should be considered one component of a larger efort to ensure trustworthy AI.
Our study demonstrates the potential of certification labels as a promising approach to communicating AI trustworthiness to end-users. The quantitative results showed that certification labels can significantly increase both trust and willingness to use AI in low- and high-stake scenarios. Based on the qualitative findings, we further identified opportunities and limitations of certification labels, as well as inhibitors and facilitators for the efective design and implementation of certification labels. Our work provides the first empirical evidence that labels may be a promising constituent in the more extensive ”trustworthiness ecosystem” for AI.
A. Quantitative and qualitative Results
High−Stake Scenario: Trust
High−Stake Scenario: Willingness to Use
Category
Subcategory
perceived transparency
persuasiveness
”Because if it is monitored and these various criteria have to be met in order to get the label, then I as a consumer can of course trust better and also know that there are perhaps controls and random checks, so I would definitely trust more”(P6) ”I think that if there is such an established label, it will certainly help to increase transparency.” (P6) ”With the Fair User Interaction aspect, yes, probably so [fairness is increased]. … if the AI is now checked for this, and it can be determined that it is not data-based, treated diferently.” (P12) ”Because I’m not an expert in the field and the label …, gives me proof … that it’s tested by experts.” (P4) ”So I could imagine that if it is a bit more standardized, so to speak, because you have to meet certain standards, that it could introduce a general level of fairness.” (P3) ”The concern [responsibility] was covered and then just the general concern with all just how our data is also used and hopefully not misused, or yes. That is also covered.” (P10) ”[I would like to] find out what this ”Fair User Interaction” means, what it refers to, how my data is protected … how is it designed and who monitors this label. Exactly by whom was it created and by whom it is administered, awarded and so on, that’s what I would like to know.” (P12) ”Ideally, it would be an overarching body that is, for example, also external and has the competences and the knowledge …. Ideally, an NGO that runs it without any vested interest.” (P12) ”If many companies get involved in using this label. Then I think it could have an impact.” (P9) ”[The criteria] are totally comprehensible to me, in any case. It’s also something that would be important to me if I were to use such a program.” (P9) ”You could say that the label guarantees that work on AI is ongoing.” (P11) ”What you could include is a criterion for the AI. That an AI has been used enough times and has, for example, been 99% correct and always had the right answers, rather than 80%.” (P4) ”I think there are still a lot of people, or some people, who will be critical of these systems even though it has a label.” (P3) ”Because you can see that in the organic sector, there are now 20 labels and as a consumer you can almost no longer categorize them, so I think it’s so important now that there is also Bio-Suisse [an organic label] or something like that in Switzerland, they have established themselves well, but I think you always have to stick to that as a label.” (P6) ”I find these 4 points are so common. And bad news is, maybe we don’t really analyze what is written. Or don’t even read. I can’t speak of everyone, but speaking of myself. I often just don’t read that message. Beautiful words, but all blah blah blah.” (P2) ”Yes, so what is complete transparency? That brings us back to fairness … what is fair? These are all such subjective terms that, in my eyes, you can’t use like in natural sciences - where you calculate and then there’s a result - it’s soft science where you’re working in.” (P5) ”Overlap; I think it all goes a bit in a similar direction, except maybe the last point [Fair User Interaction], which is a bit diferent again.” (P10)