Trust is and has been essential to human interactions. With the rise of technology, we now live in a socio-technical environment where people frequently interact with technology as well. It is therefore natural to expect that people will also develop trust in technology. Data engineering researchers have at least assumed this when claiming certain methods they devise (e.g, explanations using provenance), likely help to foster some notion of trust. But rarely is the notion of trust clarified or this claim validated. We propose a more systematic consideration of trust in data engineering technology, compared to the ad-hoc state of the art. Therefore, we first review the notion of trust established in other disciplines, based on which we derive a model for trust in data engineering technology. We then present guidelines on how to proceed to devise a trust strategy aiming at enriching data engineering technology such that it potentially fosters trust conforming to our model. We further discuss how to possibly evaluate a trust strategy. We apply our trust model on a use case, for which we devise, implement, and evaluate a trust strategy using our proposed guidelines and methods. The results of our evaluation indicate that statements like “transparency helps build trust” should be used cautiously. This highlights the need for contributions like those we present here, as only a more systematic approach to defining, integrating, and evaluating trust in data engineering can bring us a step closer to provably fostering trust in such technologies.
gained attention – yielding approaches to possibly quantify, assess, or even improve trust – we observe that the Our society depends on us humans trusting each other. notion of trust is usually not well defined and does not From crossing the streets, to collaborating with cowork- correspond to the concept of trust established in other ers, to being treated by doctors, our society is built on disciplines, e.g., philosophy or psychology. In a first line trust. The rise of technology and its integration into our of research, the notion of trust considered in the conworld, has created a socio-technical environment where text of data engineering and data analysis reduces to a humans live together with technology. This means that possibly related metric and trust in the broader sense is we now not only have to trust other humans, we have to neither considered nor evaluated. For instance, trust as also establish a similar relationship to technology rather understood in [5] reduces back to the accuracy of a mathan second guessing its every “action”, in order to bene- chine learning model. In [10, 11, 12], trust is quantified, ift, for instance, from its improvements in eficiency or e.g., based on the similarity of information and source productivity. provenance provided by diferent data sources. While the
In an increasingly data-driven world, data engineer- resulting trust scores are measured in diferent settings,
ing, data analysis, and machine learning are software it is never validated whether or not the scores actually
technologies that can significantly afect human lives correspond to some established notion of trust. A second
(e.g., [1, 2, 3]) and for which some notion of trust has line of research considers transparency and explanations
been recognized as an aspect to consider (e.g., [4, 5, 6]). to foster trust (see, e.g., [13, 14, 15]). In this context,
This paper focuses on trust in data engineering that en- data provenance [16], which ofers transparency in data
compasses the full data preparation pipeline to get from engineering pipelines, is frequently named as relevant
raw data (as collected) to data “fit for analysis”, e.g., data for evaluating trust (e.g., in [
Clearly, we need a more nuanced and systematic discussion on trust in data engineering, to which we contribute considering the following questions: How can we incorporate the concept of trust into the development prothiness in a data engineering pipeline? While we expect 2.1.2. Trust there are many diferent types of solutions, our focus here lies on technical solutions to possibly influence trust in Considering trust, a truster A usually trusts a trustee B data engineering. Our contributions are: (1) We critically to do C [24]. As natural, familiar, and elemental it is to irceavliemwodtheleftoerrmtru“sttruinstd”a(tSaecetnigoinne2e)ritnogd(eSfineectaionth3e)o.r(e2t)- tarsuastcfoonrcuespat.s Whuhmaatnpsh, ialosscoopmhpelriscaatgerdeiet oisntoisdtehsactritbreusitt Based on this model, we describe a framework for trust entails that (1) A is somehow vulnerable to a risk when engineering that integrates trust in the data engineering they trust B, and (2) A relies on B to both be competent pipeline and serves as a guideline to develop a trust strat- and willing to do C [24]. Related to the psychological egy (Section 4). (3) We describe a general procedure one attitude of trust is the property “trustworthiness” that can use to evaluate a trust strategy (Section 5). (4) We we can ascribe to others when we think that we can apply our methods to devise a trust strategy to a use case trust them (i.e., we think that they fulfill point 2). While based on a credit scoring scenario, where explanations philosophers thus agree that trust is based on reliance are integrated into a data engineering step as evidence (see point 2), they cannot agree what the additional facto possibly foster trust. Our systematic evaluation, how- tor is that diferentiates trust from mere reliance. While ever, reveals that the explanations may not reach this some argue that the trustee’s motive must be of some goal, highlighting the importance of a more systematic moral nature such as self-interest, goodwill, or moral study of the problem with the methods we propose in integrity, others argue that the additional factor is some this paper (Section 6). Note that we are aware that it is sort of normative expectation the truster has vis a vis possible to manipulate and deceive people by creating an the trustee. It seems to depend highly on the trust relaillusion of trustworthy data engineering solutions [20] tionship example used. A diferent stance philosophers and that our contributions can lead to such deceptions use to diferentiate between trust and reliance is that if and manipulations. Countering or regulating this is how- B fails A in a reliance relationship, A feels disappointed, ever out of the scope of this paper. whereas in a trust relationship, A feels betrayed [24]. Important characteristics of trust are pro-attitude (truster wants trustee to succeed in doing C), vulnerability, lack 2. Trust perspectives of control, and active acceptance of risk [25, 24]. While trust remains an elusive concept, a widely As we motivated above, trust in data engineering and adopted model is the ABI trust model [26]. It identifies analysis has been considered in an ad-hoc manner, while three factors of perceived trustworthiness: (A)bility, that it has been systematically discussed in other disciplines, is the skills or competencies of the trustee, (B)enevolence, leading to some common understanding what trust typi- which refers to the extent to which the trustee is well cally entails. meaning to the trustor, and (I)ntegrity, which is that the trustee seems upright in the eyes of the truster because 2.1. Philosophical perspective on trust they share a common set of values or principles. As we shall see in Section 3, we incorporate the ABI characThe discussion on trust has a long history in philoso- teristics in our trust model targeting data engineering phy [21] and while the concept remains elusive, there technology rather than a human as trustee. are some underlying ideas that most philosophers seem to agree upon. One key facet of the discussion that we 2.1.3. Trust in technology highlight here is the distinction of trust and its related concept reliance. Note that while most philosophical research has dealt with interpersonal trust, our discussion will also review the philosophical perspective on trust in technology.
While philosophers have studied diferent variants of
trust (e.g., self-trust, trust in groups, trust in
organizations), they all are based on human interaction and
communication [24]. Technology strongly difers from
humans. On the one hand, it lacks human characteristics
2.1.1. Reliance such as intentionality and hope [27], it cannot use
language, and is not free to act as it will [
However, if we take a closer look, technology often is Finally, we review the perspective from computer science
more than just a simple artefact. Technology can feature on trust, with a special focus on trust with respect to data
“logical complexity, capacity to store and manipulate data, processing software that performs or relies on data
engipotential for sophisticated interaction with humans” [27] neering or data analysis. While trust is also considered
and can show unpredictable behavior [
Trust in technology might not be human trust but advocates that transparency and explanations are key
something similar, lying between interpersonal trust and factors to establish trust, which is typically not evaluated
trust as reliance [27]. It might even be on a spectrum rang- or validated though.
ing from simple machines that only aford reliance and
where the trust is based on functional criteria up to com- 2.3.1. Metric-reduced trust
plex autonomous machines with unpredictable behavior First approaches have emerged to quantify, assess, or
that cannot be verified but have to be trusted [
2.2. Psychological perspective on trust Several works discuss interpretability and explanations
for machine learning models, seen as a possible means to
While the philosophical approach is fueled by the inten- improve trust (e.g., [34, 35]). The general argument is that
tion to analyze human phenomena, psychologists attempt such methods ofer evidence and verifiability that foster
to assess why we engage in this behavior of trusting trust in a user or developer. Ribeiro et al. [35] evaluate
or distrusting another person. Psychologists also strug- their methods for trust, but this evaluation either
simugle to conceptualize and operationalize trust behavior, lates users or equates trust with which model performs
but see the same main characteristics of vulnerability, better (relating back to the metric-reduced trust).
Transrisk, uncertainty, and pro-attitude that are present in the parency and explanations in data engineering pipelines
philosophers’ view [29, 30, 31, 32]. We consider psycho- can be achieved via data provenance [16]. Also in this
logical studies on behavioral causes to not be directly area, these are frequently named as relevant for
evaluatrelevant to the development of a first model of trust in ing trust (e.g., in [
Modeling both in detail opens opportunities for a more detailed discussion of what trust in this kind of relationship entails. • Modeling influencing factors. Various factors may influence the kind of trust relationship established between a truster and a trustee, making a concise and unique definition of trust dificult (see Section 2). The model should integrate inlfuencing factors to reflect this ambiguity and incorporate the diferent nuances of trust, thereby ofering a more detailed model for a systematic and multi-facetted study.
comings, a more nuanced discussion about trust has recently emerged in the area of computer science. Siau and Wang [15] for example discuss trust in artificial intelligence (AI). They collect a set of diferent definitions for trust and derive a set of factors for trust in AI technology along multiple dimensions. They also list a variety of approaches to build and then nurture trust in AI. Having focused on methods for trust in AI, this work lacks a catalog of methods for trust for data engineering. Furthermore, it does not include an actionable process taking up their discussion to “implement” trust in AI.
Meeßen et al. [36] derive a model for trust in Management Information Systems (MIS) based on both the ABI trust model [26], which we reviewed in Section 2.1.2, and research in automation and organizations. They translate the ABI terms from interpersonal trust to trust in MIS, allowing a more diferentiated discussion about trust in 3.2. Model for trust in data engineering technology. While MIS cover data engineering applica- Given the desiderata described above, we build our novel tions, the proposed trust model is centered around the model for trust in data engineering. An overview of the trustors, mainly identifying factors such as perceived model is depicted in Figure 1. Note that it is based on trustworthiness that lead to their use of an MIS. In addi- the ABI model [26] discussed in Section 2.1.2, similarly tion, this work does not model or show what developers to [36]. While our model is more comprehensive than of MIS can actually do to build and foster trust that can previous work and tailored to data engineering, we do not lead to the decision to use the system. claim completeness (it can be extended) and leave open
Thornton et al. [37] call for a more nuanced discussion the discussion how far it applies beyond data engineering on the methods developers can use in order to foster trust, (our area of expertise). proposing what they call trust afordances : “characteristics of the technology by virtue of itself or of features designed into the technology to promote trust by pro- 3.2.1. The truster - a human viding access to evidence of (dis)trustworthiness specific In the trust relationship we consider, a human is the to a user, a technology, and their context”. As they con- truster. Based on the general notion of trust (see Secsider technology in a broad sense, the discussion remains tion 2), we define the human in a trust in data engineervery general. We build on their methodology and gen- ing relationship has to be aware of a vulnerability to some eral ideas to devise guidelines for built-in trust in data sort of risk when using the data engineering technology. engineering. Otherwise, the human will use the application as just another tool and we are looking at a “trust as reliance” 3. Trust in Data Engineering situation. A human could for example feel vulnerable and at risk when, while using a website, they are aware We build on the research presented in the previous sec- that they thereby may indirectly divulge preferences or tion to define a trust model for data engineering technol- personal information that can afect what information ogy. they will be shown, e.g., which news or which job advertisements are recommended. We argue that humans also feel vulnerability when it is not themselves but other 3.1. Desiderata people that are subjected to a risk from the trustee. The following desiderata, derived from our discussion of The trust relationship a truster may or may not engage diferent trust perspectives, underly our model of trust: in inherently depends on several influencing factors : The human could be in the role of a user of the technology, • Distinguishing trust vs. reliance. The model but also others, such as an examiner, operator, executer, should incorporate distinctive features that cap- etc. [38]. This will influence how the truster approaches ture trust as opposed to mere reliance. This dis- the trust relationship. Humans’ decisions to trust are not tinction usually implies the truster’s risk aware- only influenced by their role, but also by their general ness with respect to the trustee. disposition to trust, their past experiences in general (e.g., Awareness of vulnerability, risk
Human Truster
T r u judgement ➜ trust ➜ use ts e e i m a evidence for trustworthiness ge (Social) power, uncertainty, unpredictability, unverifiability
Ability Benevolence
Integrity
DETA Data Engineering
Technology
Company …
Developers,
Designers Trustee paRsotlee,xfapdceistropierosns,cicteiuo,lnctuotronetteruxstut,al usFaubniclittiyo,nqauliatyli,ty traCinuilntugr,es,kpilalss,tgeoxaples,ripeonlcicei,es Figure 1: Model for trust in data engineering. A human truster builds a trust relationship with a trustee, i.e., a data engineering application. The latter divides into DETAs and relates to further trust entities (e.g., company). Solid boxes surround necessary characteristics of either the truster or (parts of) the trustee to establish a trust relationship. Dashed boxes group influencing factors. based on their privileges and power) and in particular ship. Note that these are parties with which the truster with (similar) technology, and contextual factors of the can also engage individual trust relationships. However, interaction. A human’s actions are also influenced by we also include these in the model of trust with respect to the culture(s) the human is part of, shaping expectations, data engineering software, because their characteristics behaviors and beliefs [39]. can influence this trust relationship as well. Indeed, their
Note that given the large variety of human trusters ability, benevolence, and integrity have shaped the data resulting from diferent influencing factors and degree of engineering technology and can indicate to the truster risk awareness, the trust relationship to a trustee can be whether the trustee is trustworthy or not. How parties significantly diferent from one human to another. For behind the technology act when developing the product instance, one human’s relationship with the trustee may is again influenced by their culture - including organizaactually be based on reliance because they do not see nor tional and functional culture [39] - but also their past are aware of any risks involved in interacting with the experiences, training, skills, goals, and policies. All of this trustee. At the other side of the spectrum, someone else can afect the trustworthiness of the product, i.e., the data might not engage in a trust relationship at all because engineering technology, and may be taken into account they feel too vulnerable and thus decide not to use the by the truster when making the decision whether or not system. to trust the data engineering technology.
Given the context of our work, the trustee is some data We now describe the interaction of the two parties inengineering technology. For the truster to feel vulnerable, volved in establishing a trust relationship. it has to have some (social) power, element of uncertainty, When a truster judges the trustworthiness of someone, unpredictability, or unverifiability , thus preventing the they are actually assessing pieces of evidence they are assertion that the data engineering technology will not provided with to evaluate whether it is worth taking the cause any harm. risk to trust the other party and be vulnerable in some
Typically, such an application is complex and consists aspect. Whether we are in the process of judging humans of multiple diferent data engineering technology artifacts or now data engineering technology, we think the human (DETAs). These include for instance services, datasets, truster continues to act the same. Therefore, we adapt or algorithms. Note that the truster may or may not be the ABI framework by Mayer et al. [26] (Section 2) which aware of DETAs. Each DETA, as well as the data engi- states that the trustee is assessed with respect to their neering technology perceived as a whole, is characterized ability (i.e., skills and competences) to fulfil their tasks, by its functionality, usability, and quality. These have their benevolence towards the trustee, and their integrity to be suficient in order for the truster to perceive the of principles they act upon. Wile these are classically technology as reliable. Each DETA could also carry the characteristics of persons and organizations, in our setpotential to harm and therefore could also be individually ting, the truster usually creates an imaginary image of trusted or distrusted by the truster. the trustee based on visuals and communication with the
Given that technology is shaped by humans and orga- data engineering technology. Indeed, communication to nizations, parties like developers, designers, or companies developers or the company behind the application, or are part of the trustee in a trust in technology relation- access to the codebase are usually not available to the truster, so their ABI characteristics are transposed to the technology as summarized in Figure 2. The top of the figimage of the data engineering technology. Based on the ure shows the diferent steps of the data engineering protruster’s epistemic and practical judgment, the truster cess, whereas the two bottom components “accompany” then decides whether to trust and then potentially use the whole process from a technical and organizational the technology [36]. perspective, respectively.
Going from the trustee to the truster, the trustee pro- In general, before developing actual data engineering vides evidence towards the truster. In case of data en- software, the goals to reach with the use of data need to gineering applications, this could be through a modern be defined. Based on these goals, relevant data need to or old-looking visual interface, whether questions are be identified and collected. As these data may come in answered in an FAQ, etc. Opposed to interpersonal trust, various formats from diferent sources, data wrangling is trust in data engineering technology involves trust in a implemented to transform, integrate, and clean the data complex system of people, groups, institutions, who of- to obtain a unified and consistent view of the data releten cannot be judged directly but only through the pieces vant to the goal. These data can be further enriched with of technology the truster has access to. In addition to application specific data and annotations, before they are that, the truster often does not have the capabilities to distributed to downstream data consuming applications understand the inner workings of the technology they such as data analysis techniques. To monitor, document, are supposed to assess. Following the ABI model [26], and support the process, metadata are typically gathered information on ability, benevolence, and integrity of the and maintained. In addition, a data engineering process trustee with respect to the potential risk might be evi- is usually subjected to some form of governance. dence that increases the perceived trustworthiness. Following our model of trust in data engineering, the data engineering technology in its role of trustee can support a trust relationship by providing appropriate ev4. Design data processing for trust idence. This may involve evidence collected at all stages of data engineering. The methods applicable to collect Clearly, when developing data engineering technology, evidence possibly vary from one stage to another, makthe evidence that can be provided is under the trustee’s ing it important and challenging to select appropriate control, who can adapt this evidence to potentially in- methods. The collected evidence can be managed within lfuence the trust relationship. We propose guidelines on the metadata management component. While there are how to systematically integrate trust in the development many ways to possibly foster trust in data engineering of data engineering pipelines, by enriching the general applications, as well as trust in the parties behind the apdata engineering process with further steps fostering plications that can also have an efect on the considered trust. trust relationship, this paper focuses on the technical solutions targeting trust, leaving the study of trust with 4.1. Assumptions respect to governance to the future. This paper also does not aim at exhaustively reviewing how to collect and manage evidence (we mentioned some approaches in Section 2), as for diferent trust scenarios, diferent solutions apply or may need adaptation. Instead, our work here ofers guidelines on how to generally proceed to systematically integrate the consideration of trust in data engineering technology. This naturally integrates into the conceptual planning phase of data engineering processes (i.e., the leftmost step in Figure 2).
To align with the trust model we defined in Section 3, we make the following assumptions. First, to guarantee that we are fostering a trust relationship conforming to our model, we assume that the truster is aware technology is used, that it poses a risk to themselves or others, and its functionality cannot be completely verified. Second, we assume that the truster has an ambivalent attitude towards the data engineering technology and can be led to trust it. Finally, we acknowledge that the actions of developers and companies can also create an illusion of trustworthiness, e.g., through clever designed evidence. 4.3. Identify trust scenarios Here, we assume a benevolent trustee, who intends to provide actual evidence of trustworthiness and does not want to trick the user into trusting a non-trustworthy technology.
titude of scenarios in which humans with specific roles, risks, and vulnerabilities are in a trust relationship with a data engineering technology. Specific evidence will be needed - and at the same time enough - for individual 4.2. Trust-integrated data engineering trusters to perceive a particular application as trustworWith these underlying assumptions, we enrich the gen- thy. Therefore, it makes sense to identify the specific trust eral data engineering process to integrate trust in the scenarios anticipated with respect to the application goal, such that that the collection of evidence can be tailored Goal definition + identify trust scenarios + identify trust break points + devise trust strategy
Data collection + collect evidence
Data wrangling + collect evidence
Data enrichment + collect evidence Metadata management + collect evidence + manage evidence Data distribution + collect evidence + distribute evidence Governance to these. 4.5. Devise a trust strategy
At this stage, we propose to think about scenarios, relationships, or use cases where the targeted application In a sense, identifying trust scenarios and trust break(goal) has some sort of power over the truster, putting the points can be seen as a requirements analysis on how truster at risk. Modalities of power as identified in the to cover trust. This analysis forms the foundation to deifeld of political philosophy could be a starting point. Fur- vise a trust strategy, i.e., a plan to meet the requirements. thermore, diferent kinds of trusters, i.e., trusters exhibit- Referring back to the distinction of reliance and trust, it ing diferent influencing factors, should be considered. It will not be enough to provide evidence that convinces is important to identify which diferent combinations of the truster that the application is pragmatically the best influencing factors may define trusters in relevant trust option to use. Instead, following our trust model, the scenarios, as well as the specific risks they potentially trust strategy should be designed to provide suficient evface, to then devise trust strategies tailored to the dif- idence on ability, benevolence, and integrity to increase ferent kinds of trusters. For a wide coverage of possible perceived trustworthiness. trust scenarios, we recommend a diverse set of examiners The first idea that comes to mind is to transparently with a critical mindset. provide more information about the trust breakpoints, which the user can use to judge the trustworthiness of the application. This will mostly respond to the ability 4.4. Identify trust breakpoints of the trust breakpoint’s DETAs, but could also include After identifying trust scenarios, it is time to pinpoint evidence for the integrity and benevolence of the comthe critical parts for perceived trustworthiness in the pany and developers behind the application. Several (planned) data engineering process. We call these trust methods have been developed to provide metadata that breakpoints. They may comprise methods, algorithms, can serve as evidence, including plain information about or other DETAs that could expose a truster to some risk datasets [40], data provenance [16], or machine learning by not meeting specific quality, functionality, or usabil- explanations [35]. However, the problem of choosing a ity guarantees, as their behavior bears some degree of suited strategy for requirements given by trust scenaruncertainty, unpredictability, or unverifiability. ios and breakpoints remains. To systematically devise a
It is possible that one trust scenario has multiple trust strategy and identify pertinent methods, we propose to breakpoints or that diferent trust scenarios share the answer the following six questions in a structured way: same breakpoint. This leads to many-to-many relation- (Q1) What should the trust strategy enable the truster to ships between trust scenarios and trust breakpoints. For do? This refers to additional “-ility" requirements of the each application-relevant combination, we further rec- system that support the truster in their trust assessment ommend to determine the requirements each breakpoint and ultimately decision. Answers could include verifiabilin each scenario has to meet in order to minimize or avoid ity, reproducibility, traceability [41], reviewability [42], risk. accountability [43], auditability [44], or trialability [45].
Since the data engineering software is a technolog- Diferent answers will require diferent pieces of evidence ical product, the quality of its trust breakpoints is al- produced by diferent methods. For example, verifiabilways shaped by the human capabilities, thoughts, and ity of an output may require an explanation on how the attitudes of its designers, developers, and surrounding output was generated, whereas the reproducibility of an organization. Therefore, there are truster-organization algorithm asks for information about the algorithm and and truster-developer trust relationships to be identified its parameters. and addressed as well. (Q2) For what kind of component does the truster need evidence for? Diferent components of the data engi- sibly be evaluated and a trust strategy validated. Given neering technology will require diferent methods. For the complexity of human trusters through the number example, methods applying to SQL processing [46] sig- and variety of influencing factors on trust, we postulate nificantly difer from methods for data transformations that a trust reaction can hardly be simulated, as has been in Map/Reduce pipelines [47]. This question also asks for attempted for instance by Ribeiro et al. [35]. Therefore, the granularity of the component that the truster needs we suggest to resort to proper user studies, analogously evidence of. Whether it is one, multiple, or only the out- to studies conducted for instance in social sciences or put of a DETA will influence the choice of methods to human-computer-interaction, to evaluate a trust strategy. use. We provide guidelines on how to perform such studies (Q3) What is the timeframe the truster needs evidence relating to trust in data engineering. for? Depending on the trust scenario, the evidence should cover past information (e.g., evolution provenance [48]), 5.1. Study participants and goals real-time information (e.g., machine learning model explanation [35]), or future information (e.g., future use of As we have seen, a trust strategy is designed and implesensitive data [49]). mented specifically for a trust scenario. Therefore, the (Q4) What type of information is needed? To provide evaluation of the strategy should reuse this scenario in the truster with the necessary evidence, diferent types order to validate the strategy with respect to the scenario. of information can be used. Examples include factual This means that participants in the user study should information such as fairness scores [50], explanations of have the same role towards the application as the truster outcomes [35], or less technical information, e.g., limita- in the scenario. Furthermore, the participants should tions or legal considerations [40]. satisfy the modeled requirements on trusters, i.e., they (Q5) What presentation is appropriate for the truster? should be aware that the application is uncertain and Depending on the truster’s role, level of expertise, and its use is related to a specific risk, as defined in the sceother characteristics (influencing factors), the evidence nario. To ensure this, proper participant selection and has to be prepared and presented accordingly. Therefore, gauging questions in the questionnaire of the user study an appropriate level of abstraction and appearance have are possible methods one can employ. Additionally, we to be chosen, that provides the evidence without over- recommend properly introducing the participants to the whelming the truster. It could for instance be presented scenario, where they should be made aware of their role like in Datasheets for Datasets [40], where the informa- and the risk the application can pose. tion is presented as structured text and kept at a very Before deciding on the study setup or devising the technical level, or the evidence can be presented as in questionnaire, the question on what hypotheses to verify Nutritional Labels for Rankings [50], where the informa- needs to be answered. One example of such a hypothesis tion is (visually) supported using icons, diagrams, and is: “The devised trust strategy increases the perceived trustinformation boxes. worthiness of the data engineering technology compared (Q6) What other requirements have to be fulfilled? Since to the same technology without trust enrichment.”. Clearly, the trust strategy has to fit the overall development plan the hypothesis should explicitly focus on an aspect of the and requirements, other (technical) requirements may trust model, for which the impact of the trust strategy also apply. These could include storage constraints [51], is then evaluated. The impact itself also encompasses privacy considerations [52], access control [53] or execu- diferent possible aspects, e.g., perceived trustworthiness tion speed [54]. (wrt image in the model), actual use, etc. This should
After these questions have been answered for all pre- be clarified as part of the hypothesis. Finally, the scope viously determined relevant trust breakpoint-scenario of the evaluation needs to be defined, clarifying which combinations, the developers have enough information aspects of the trustee are covered (e.g., the whole data to identify or develop appropriate methods. engineering technology or just selected DETAs).
5.2. Methods for trust evaluation
After the trust strategy has been defined and imple- question on “how” to conduct the study. Here, study demented, including the collection of evidence, the question signers have to decide which methods to use to evaluate remains whether the strategy performs as expected. That the target aspects. The notion of trust is inherently difiis, whether the collected evidence helps trusters to estab- cult to quantify, which explains why a set of measurable lish a trust relationship with the trustee, in our setting a proxies is usually used that, ideally, highly correlate with data engineering technology. In this section, we discuss the aspects of interest. We review methods that have how the notion of trust we defined in this paper can posbeen used to evaluate trust and which are amenable to development in Section 6.1 and report on its evaluation our data engineering setting. in Section 6.2.
Experiments. For interpersonal trust, researchers have conducted various studies in which the participants could 6.1. Record linkage in a credit scoring wchaosoismepbleictwitleyernedlaitfeerdentototrputisotnosr[di5st5r,u5s6t]b.aEsaecdhoonf athreisske application and reward system. By tracking participants’ actions, re- Credit scores for individuals as provided by companies searches could conclude whether the participants trusted like Equifax or TransUnion are widely used to evaluate each other or not. This technique can be adjusted for the “creditworthiness” of individuals. This can have a evaluating data engineering technology by creating eval- significant impact on human’s lives, e.g., depending on uation scenarios in which the participants can actively their credit score, they may or may not be granted a loan, choose between diferent options that correlate with trust may have to pay higher or lower interest rates, may be or distrust. Recording the decisions of participants can preferred or not in the competitive housing market to be used as a proxy to measure actual use. sign a lease, etc. Therefore, it is crucial for all parties (the
Questionnaires. In designing questionnaires to evaluate human customers, banks, landlords) that a person’s credit trust in data engineering technology, we can adapt and history or report, on which the scores are based, is correct extend questionnaires that have been devised to evaluate and complete. A report itself comprises various customer trust in other settings. Examples of questions used to activities that are shared by diferent entities (banks, inmeasure trust appear in the trust section of the General surances, credit card companies, mobile phone providers, Social Survey [57] (an annually conducted study in the etc.) cooperating with the credit scoring company that US). Another option is to derive trust questions analogous are potentially related to the customers’ creditworthiness. to the questions on usability and understandability of Examples include opening of a bank account, successfully the technology acceptance model (TAM) [58]. These paying back a loan, etc. techniques allow to examine the thoughts, attitudes, etc. To ensure the data of persons’ credit reports are accuof the participants including perceived trustworthiness, rate, newly shared customer activities need to be inteintention to use, and perceived risk. grated in the consolidated master database of the credit
Structured interviews and unstructured questionnaires. storing company. This is performed by a dedicated data Information about perception, attitudes, etc. that are dif- engineering software, which we assume to be similar ifcult to express in a question with predefined answers to the pipeline for a similar goal described in [60]. Folcan be collected or captured via interviews or free text lowing the steps of the general data engineering process ifelds in questionnaires. This includes, e.g., the reasoning outlined in Figure 2, the goal definition is to correctly behind participants’ answers to structured questions or update the master database, given the data of a newly readditional comments on the study. Such answers can ported activity record. In this context, the data collection provide valuable information on aspects that study de- step includes accessing data of the master database (we signers did not anticipate and ofer insights on how to can assume an SQL query interface) and newly reported potentially improve the technology, including the trust records, e.g., obtained via an API. The subsequent data strategy. processing that will result in the transformed (updated)
Quantitative metrics. In some settings, it is possible master database is all part of data wrangling. Sub-tasks
to include quantitative metrics into the trust evaluation. of data wrangling in our use case include the
standardizaFor instance, Wintersberger et al. [
The merged record is then written back to the master use case database, which can then be queried by subsequent applications, such as an application deriving a credit score.
engineering technology as well as guidelines on how to devise and evaluate a corresponding trust strategy, we put our approach to the test by applying it on a real world use case. We describe the use case and its trust strategy
In the use case introduced above, the first step towards devising a trust strategy is to define trust scenarios. To this end, we first identify various parties (possible trusters) relationship between the employees and the data engithat have some kind of relationship with the data en- neering technology they use to consolidate credit reports. gineering application that can potentially be a trust relationship. These include, for instance, the customers, 6.1.2. Trust breakpoints whose personal data are stored and evaluated by the credit scoring company and the employees of the credit For this specific trust scenario we identified above, we scoring company that should trust the technology to sup- consider several trust breakpoints, i.e., DETAs that may port them in their task of matching and merging records. afect employees’ trust relationship with the technology.
Let us now analyze the potential trust relationship A first review reveals for instance that during data collecbetween a customer in the role of truster and the data tion, trust may be jeopardized by the reporting entities engineering technology (trustee) in more detail. Clearly, that may transmit erroneous data. During data wranthe customer relies on the credit reporting technology gling, the address standardization may sometimes be (e.g., accessible through a web interface) to be able to inaccurate, depending on which (external) address check provide the described service (maintaining the credit service is used. Next, the record linkage may match the report), e.g., to secure a loan. While the customer may be wrong records or present the employees with what can aware of the impact a (wrong) credit history can have on be perceived as misleading information to make their the loan application, the customer usually simply expects decision. Finally, the merge of records could yield an the service to work as intended, considering it as an erroneous record. We consider employees unlikely to instrument to achieve a goal. As we saw in Section 2.1.3, question the data collection or address standardization this rather qualifies as trust as reliance. Also, customers DETAs directly (they more likely may not trust extermay not be aware that the underlying technology cannot nal entities serving as data providers, which are other be completely verified and can exhibit quality issues. trust relationships). We assume their trust relationship is
This picture changes when we turn our attention to mostly afected by the internal workings of the assistance the employees involved in the “human-in-the-loop” data the system gives them during record linkage or merge. engineering technology as potential trusters in a trust To demonstrate the development of a trust strategy, we relationship. Clearly, being part of the process, they are focus on the first of these two breakpoints. well aware that the data engineering technology cannot be completely verified and can cause quality issues. 6.1.3. Trust strategy They are also aware of the risk the use of the technology In order to devise a trust strategy for the trust scenario poses, not necessarily to themselves but to their friends, and breakpoint identified above, we answer the questions their relatives, and the society in general. For their work, proposed in Section 4.5. Essentially, the trust strategy however, they rely on the technology and depending should enable employees of a credit scoring company on company policy, the use of the technology bearing who consolidate personal data to judge the trustworthisome uncertainty with respect to quality may also put ness of technology, which, in this scenario, we assume rethese employees at risk, e.g., if, in a performance review lates mostly to verifiability of its functionality and quality it turns out that these employees did match and merge (Q1). Given the trust breakpoint under consideration, we a significant amount of credit reports that have led to need evidence for the record linkage component (Q2). As claims for correction or to too generous credit scores the employees make point-wise match decisions, workfor non-creditworthy customers. Overall, we see that all ing with the technology for each individual case, the adecriteria are met by employees to be a truster in a trust quate time frame for evidence is “the now”, i.e., real-time relationship as defined by our trust model. (Q3). Considering what type of information is needed as
On the trustee side, the credit reporting technology evidence, we argue that developers are probably intercomprises several DETAs, e.g., the diferent steps of the ested in explanations on how the program came to the data engineering pipeline we described above. Given conclusion that two records could match, while design the common uses of such technology, it undoubtedly has decisions on system level and implementations are not some social power. As mentioned before, it also exhibits pertinent (Q4). In terms of presentation, employees benesome uncertainty and unverifiability on how the credit ift from simple and easy to understand explanations that reports are generated. Influencing factors relating to do not use technical terms from underlying algorithms, the DETAs are mainly their functionality and quality. as well as visual cues that support the understandability Besides the credit reporting technology, developers and of explanations (Q5). We consider no additional requiredesigners, but also the reporting entities cooperating ments (Q6). with the credit scoring company also potentially afect With the answers to the questions given above, we can the trust employees put into the trustee. determine suited methods and algorithms to implement
Given the discussion above, we focus on devising a the trust strategy, where we essentially opt to provide trust strategy for the trust scenario defined by the trust employees with an explanation of matching candidates (e.g., [61, 62, 63]). We rely both on the visualization of that serves as evidence of the trustees ABI, so that the feature importance by using diferent color highlights for employees can potentially gain trust in the system’s be- attributes that are important for making a match decision havior. and attributes that are important towards a non-match. We further provide explanations in the form of human6.2. Evaluating the trust strategy readable model approximation, listing positive semantic indicators (e.g., important fields firstname, lastname, and The goal of the trust strategy in this use case is to foster date of birth are equal) and negative semantic indicators trust of employees in the data engineering technology (e.g., contradictory gender). they use, by means of explanations. To evaluate if the In the third section of the study, each participant trust strategy implementation achieves this goal, we con- answers an exit questionnaire that covers several asduct a user study, following our discussion in Section 5. pects, including usability, by adapting questions from This section summarizes the study design, presents re- the TAM [58]. We formulate additional questions to assults, and discusses these. sess perceived risk and trustworthiness (see Figure 4), following the same rationale as TAM questions. The an6.2.1. Study design swers to these questions again follow a 7-grade Likert scale, ranging from the most positive answer “strongly agree” (1) to the most negative answer “strongly disagree” (7). The study section concludes with a free text field for additional remarks.
During the second section of the study, we capture participants’ decision time per match as quantitative metric.
sition of employees of a fictive credit scoring company and review the ambivalent decision of a record linkage DETA. Given the ongoing pandemic, we design an online study. From the diferent methods for trust evaluation (see Section 5.2), we focus mostly on questionnaires to capture the participants’ stance on the data engineering technology. The study includes three main sections, we 6.2.2. Results summarize next. Full details are available on our repeata- At the time of submission, a total of 19 participants with bility website1. a computer science background took part in our user
In its first section, the study provides an introduction study (10 without / 9 with explanations). We opted for to the setting of the study and the topic of record linkage participants with a computer science background to enin the context of credit report generation. Thereby, we sure all participants have a general understanding of data enable the participants to make informed decisions in engineering technology, to better grasp the task we ask the next section focusing on record linkage, and raise them to perform. Based on responses to the first section their awareness for the underlying potential risk. We of the study, we conclude that the participants are genfurther add questions based on 7-grade Likert scales to erally optimistic that technology can be helpful rather assess the participants’ ambivalent attitude towards the than harmful (mean of 2.7) while they are aware that the technology they evaluate and their risk awareness with technology may put others at risk (mean of 2.7). Thus, respect to the scenario. Answers to these questions allow they are aware and careful because of associated risks us to verify the assumptions stated in Section 4.1. We (mean of 2.7). also include test questions to determine if participants Determining if the explanations implemented followhave understood the problem of record linkage. ing the devised trust strategy have any efect, we ana
Next, participants are presented with potential lyze if there is some statistically significant diference matches, i.e., pairs of records the system suggests to be between the group of participants without explanations matches, for which participants, in their role as employ- and the group with explanations. Considering reaction ees, have to decide if they agree with the system or not. time, accuracy of participant match decisions, and the The study comprises 60 matches that each participant Likert scale questions relating to trust, the applicable stareviews. We ensure that these matches cover diverse tistical tests (t-tests or Wilcoxon-Mann-Whitney-Tests) real-life match situations of varying dificulty in a bal- do not reveal a diference between groups of participants anced way. The participants were shown the matches in with and without explanations. We thus cannot conclude a random order. that explanations have a significant efect on the interac
To evaluate the efect of the trust strategy, participants tion between employees and the record linkage DETA, are split into two groups: one gets to see explanations in particular, on trust. While the study may benefit from alongside matches, the other group not. Diferent op- a larger number of participants, the current results show tions for record linkage explanation have been proposed that statements of the sort “explanations are a means to improve trust” should be used cautiously, as it remains an open question in our use case (and others that have not 7 6 l5 e trsa4 c ik3 e L 2 1 witout explanation
with explanation [1] J. Angwin, J. Larson, S. Mattu, L. Kirchner, Machine bias: There’s software used across the country to predict future criminals. and it’s biased against blacks, https://propublica.org/article/machine-biasrisk-assessments-in-criminal-sentencing, 2016. [2] L. Sweeney, Discrimination in online ad delivery,
Queue 11 (2013). [3] S. Lowry, G. Macpherson, A blot on the profession.,
British medical journal (Clinical research ed.) 296 (1988) 657—-658. [4] X. L. Dong, E. Gabrilovich, K. Murphy, V. Dang,
W. Horn, C. Lugaresi, S. Sun, W. Zhang, Knowledgebased trust: Estimating the trustworthiness of web sources, Proceedings of the VLDB Endowment 8 (2015) 938–949. [5] A. Fariha, A. Tiwari, A. Radhakrishna, S. Gulwani,
A. Meliou, Conformance constraint discovery: Mea