=Paper=
{{Paper
|id=Vol-2903/IUI21WS-TExSS-14
|storemode=property
|title=Understanding How Customers Attribute Accountability in Food Delivery Breakdowns
|pdfUrl=https://ceur-ws.org/Vol-2903/IUI21WS-TExSS-14.pdf
|volume=Vol-2903
|authors=Joice Tang,Ning F. Ma,Dongwook Yoon
|dblpUrl=https://dblp.org/rec/conf/iui/TangMY21
}}
==Understanding How Customers Attribute Accountability in Food Delivery Breakdowns==
https://ceur-ws.org/Vol-2903/IUI21WS-TExSS-14.pdf
Understanding How Customers Attribute
Accountability in Food Delivery Breakdowns
Joice Tang, Ning F. Ma and Dongwook Yoon
University of British Columbia, 2366 Main Mall, Vancouver, BC, Canada
Abstract
In on-demand workplaces, the worker’s ability to work is highly dependent on a number of factors
that they have limited control over. As such, food delivery platforms are prone to unfair treatment of
workers involving incident accountability in transaction breakdowns. A breakdown in delivery trans-
actions may result in averse perception from customers towards the worker, restaurant or the platform
company, however the accuracy and fairness of these perceptions are unclear. To address the potential
effects of such crumple zones in food delivery, we will study how customers make judgements when the
responsible party is unclear and whether or not customers can identify a worker’s responsibility in a
given incident. Through an online experiment centering contentious scenarios in food delivery, we will
explore how customers may attribute accountability differently depending on the mode of explanation
and transparency of who the accountable party is. Overall, in hopes of further empowering workers, we
aim to understand the imbalanced perspectives in food delivery that can exacerbate unfair outcomes.
Keywords
Algorithmic accountability, automation, gig work, food delivery
1. Introduction to work is highly dependent on a number of
factors that they have limited control over.
On-demand workplaces support a large As such, food delivery platforms are prone
amount of temporary gigs that are sched- to unfair treatment of workers involving in-
uled, managed, delivered and paid through cident accountability in transaction break-
online APIs [1]. As a prominent sector of downs. Through this study, we hope to fur-
the on-demand economy, food delivery has ther understand how and why food delivery
gained massive growth since the beginning customers may form unfair and inaccurate
of the COVID-19 pandemic [2, 3]. Like perceptions of accountability in such break-
many other on-demand platforms, food de- downs.
livery platforms promise flexible work [4, 5], In discussion of socio-technical systems,
despite the fact that workers are managed research is increasingly recognizing the im-
through platform functionalities that strictly portance of accurately attributing the re-
dictates the what, when and where they work sponsibility of actions [7]. However, these
[6]. More importantly, the worker’s ability responsibilities cannot always be easily dis-
cerned in varied social and cultural contexts.
Joint Proceedings of the ACM IUI 2021 Workshops, April Similarly, food delivery platforms—which in-
13–17, 2021, College Station, USA
" joice.tang@alumni.ubc.ca (J. Tang); ning.ma@ubc.ca
volve multiple stakeholders (i.e., platform
(N.F. Ma); yoon@cs.ubc.ca (D. Yoon) company, worker, customer and restau-
0000-0003-4750-5965 (J. Tang); 0000-0003-2793-7429 rant)—are prone to fall into a crumple zone
(N.F. Ma); 0000-0002-3263-2972 (D. Yoon) where there might not be a single clear re-
© 2021 Copyright © 2021 for this paper by its authors. Use
permitted under Creative Commons License Attribution
4.0 International (CC BY 4.0).
sponsible party. To address the potential
CEUR
http://ceur-ws.org
CEUR Workshop Proceedings effects of such crumple zones in food de-
(CEUR-WS.org)
Workshop ISSN 1613-0073
Proceedings
livery, we will study how customers make 2. Related Work
judgements when the responsible party is
unclear and whether or not customers can 2.1. Algorithmic management
identify a worker’s responsibility in a given and consumer surveillance
incident. Furthermore, we will examine how
customers perceive particular modes of ex- Research has previously focused on the is-
planation and the extent to which this per- sues of algorithmically managed labor, es-
ception affects the level of blame. We aim to pecially in the context of fairness, account-
answer the following questions through our ability, transparency, and ethics (FATE) [8,
research: 9, 10, 11]. Existing work in gig platforms
has discussed the ways in which on-demand
workplaces take advantage of customer feed-
1) How do customers distribute account-
back as an element algorithmic management;
ability in unsatisfactory experiences, espe-
some perspectives on rating argue that com-
cially when the cause of the incident is not
panies’ valuation of customer feedback, in
immediately transparent?
tandem with features such as route surveil-
2) Do differing modes of explanations af- lance, act as a new form of “middle man-
fect customers’ perceptions in unsatisfactory agement” that can be described as consumer
experiences with multiple factors at fault? surveillance [12]. Many on-demand work-
places require certain approval ratings for
Through an online experiment center- workers to continue working: for example,
ing contentious scenarios in food delivery, Doordash requires drivers to have an av-
we will explore how customers may at- erage of 4.2 stars and UberEats requires a
tribute accountability differently depending user satisfaction rating around 75-80% de-
on the mode of explanation and overall trans- pending on the location [13, 14]. This re-
parency of the accountable party. We use quirement further imbalances the dynamic
the phrase "mode of explanation" to refer to between workers and customers, who can
the context in which the customer receives easily give a poor rating—and influence the
an explanation of the transaction breakdown: deactivation of a worker’s account—if dis-
in food delivery, this entails app design, satisfied [15, 16]. In addition to its intrinsic
communication with the driver or commu- pressure, customer feedback can become an-
nication with customer service. Meanwhile, other workplace control mechanism for com-
"transparency of accountability" refers to the panies, allowing them to encourage workers
clarity of the perpetrator. For instance, as to act in a particular manner [17]. For exam-
we are determining whether or not cus- ple, Uber sends messages to drivers suggest-
tomers unfairly attribute accountability, this ing that passengers give low or high ratings
entails scenarios in which the perpetrator based on expected behaviours, using feed-
clearly is or is not the driver, and scenar- back to nudge drivers toward delivering a
ios in which the actor at fault is unclear. In standardized service [10].
all, we aim to understand the many ways in Dependence on ratings can be especially
which misattribution manifests in food deliv- damaging in the context of on-demand work-
ery platforms, as well as the ways in which places, as these companies do not seek out
customers’ perception of particular contexts other metrics or perspectives of worker atti-
may reduce the potential of misattribution in tude and behaviour, and customer feedback
transaction breakdowns. can be inaccurate as it is typically based on
subjective interpretation [17]. A common even when a logical analysis should not”
complaint from Uber drivers is that “passen- [21]. Correspondence bias is caused by lack
gers do not understand how ratings work,” of awareness, unrealistic expectations, in-
therefore making ratings “not reflective of flated categorizations, and lack of motiva-
performance and services” [16, 8]. In fact, tion to correct inferences made. Such traits
in a study regarding the potential polar- can be found in food delivery experiences:
ity of ratings and reviews across a variety one of our presumptions coming into the
of platforms, sharing economy platform re- study is that customers are not motivated to
views were found to skew positively over- give much thought to their perception of the
all, with a tendency to be highly polarized worker, the service, and the rating and tip-
[18]. To further complicate the issue of sub- ping they provide due the fact that a pur-
jectivity, Uber drivers “noticed that passen- chasing (of service) behavior is ephemeral
gers misattributed system faults and nega- and serves a relatively non-essential portion
tive experiences that drivers could not con- of one’s goals. Furthermore, as described by
trol to drivers themselves” such as surge pric- the “illusory causation” phenomenon, people
ing, traffic jams, and GPS errors, or even the tend to overestimate the role of a particular
customers’ mood and context during the ride person in a situation when the person is the
[8, 16]. Similarly, food delivery drivers in fo- most salient actor in the interaction [22]. Due
rums often describe situations in which inci- to the fact that they are the most salient ac-
dents are caused by issues with restaurants tors for customers in food delivery interac-
or app, commonly reporting that they lose tions, compounded with the fact that they are
tips or receive low ratings in these encoun- the primary focus in the ratings in these sys-
ters, despite their lack of control of such fac- tems, drivers can become the wrongful recip-
tors. ients of blame if an order goes awry.
2.2. Misattribution in social 2.3. Moral crumple zones
psychology In an article titled “Moral Crumple Zones:
In social psychology, misattribution of neg- Cautionary Tales in Human-Robot Interac-
ative events is often attributed to the “fun- tion,” Elish uses the concept of a “moral
damental attribution error,” which describes crumple zone” to explain how responsibil-
how, when evaluating the behaviour of oth- ity and blame may be misattributed to hu-
ers, observers tend to overestimate intrinsic man actors working within complex systems
factors while underestimating extrinsic fac- when these systems breakdown. Particularly,
tors [19]. In the context of customer service, Elish discusses how systems may be designed
this means that customers “are likely to at- or set up in a manner that allows the sys-
tribute a negative outcome to an employee’s tem to take advantage of human actors to
effort and abilities, even when they know “fill in the gaps of accountability that arise
that something else, such as a computer sys- in the context of new and complex systems”
tem error, bad weather, or just bad luck, was [7]. Similarly, in the context of algorithmic
responsible for it” [20]. Misattribution can management, Moradi and Levy argue that
have a tangible effect on customer feedback, “AI-driven managerial practices redistribute
according to “correspondence bias”: people the risks and costs of these inefficiencies to
“often conclude the person who performed workers while serving a firm’s bottom line”
the behaviour was predisposed to do so... [23]. Typically, in the context of companies,
the larger organization will take accountabil- 1, we present a factorial design table with
ity for any faults: in "Computing and Ac- summaries of sample contentious scenar-
countability," Nissenbaum gives the example ios based on anecdotes from forums such
of milk producers being liable for spoiled as /r/doordash and uberpeople.net. The
milk, even if they have taken the expected dependent variables for this study will be
degree of care. The larger organization or the percentage of accountability assigned to
management is expected to compensate for each stakeholder in the scenario, as well as
the faults even if they did not enact them, participants’ estimated ratings and tips for
risk distributed to “those best able to pay, and the worker. In the survey, we will first
those best able to guard against the harm” present scenarios through narrated video
[24]. However, in the context of algorithmi- storyboards. After each scenario, we will ask
cally managed labor, the people who could be participants to provide an evaluation on each
considered best able to guard against harm of the parties in regards to responsibility in
are those who are not best able to pay. Of the scenario. A preliminary survey sample
course, if an order goes wrong in food deliv- is provided in Figure 1. The survey-based
ery, the company will refund the customer method has been studied to be effective in
and apologize to all parties for the inconve- allowing participants to accurately consider
nience. But unlike the milk producers, in their behaviours as they would in a real sit-
this scenario, the company does not take ac- uation [25]. Scenario-based studies are com-
countability for the harm caused; the worker monly used in research studying perceptions
may still lose their tips, their ratings may of fairness in algorithmic systems, and the
drop, and they are still completely respon- reach and scale of an online experiment can
sible for themselves and whatever else may be beneficial in seeking patterns in subjective
come their way. matters like perception [9, 20].
Algorithmic reliance and collection of hu- Allowing participants to rate each stake-
man stakeholders can cause food delivery holders’ percentage of accountability in each
systems to fall prey to “the problem of many scenario will give us nuanced understand-
hands”: by passing pieces of responsibility ing of the extent to which modes of expla-
amongst different, unrelated members of the nation can affect participants’ perceptions of
system, the company can rescind account- a scenario, and any qualitative data supplied
ability for any issues, leaving the matter of with these ratings will help us further under-
blame hanging in the balance [24]. A break- stand participants’ judgements and decision-
down in delivery transactions may result in making process, particularly in incidents
averse perception from customers towards where it is difficult to determine an account-
the worker, restaurant or the platform com- able party. Additionally, by surveying partic-
pany, however the accuracy and fairness of ipants’ estimated tips for each encounter, we
these perceptions are unclear. will gain a stronger understanding of the real
consequences of unfair dispersion of blame
in transaction breakdowns.
3. Method
We propose a 3x3 survey centering con-
tentious scenarios in food delivery, vary-
ing the transparency of the accountable
party and mode of explanation. In Table
Mode of explana- Transparency of accountability
tion
Clearly not driver’s Crumple zone Clearly driver’s fault
fault
Tracking map shows
Tracking map shows the driver drove near Tracking map shows
that driver is waiting the location and that the driver drove
App design at the restaurant for a says the order has in the wrong direction
long time and then or- been dropped off, but completely and order is
der is cancelled customer receives cancelled
nothing
Driver texts saying Driver texts saying
Driver texts saying the
they dropped off the they could not find the
restaurant is closing
Driver order, but the GPS led
and refuses to make location and gave up
them to the wrong on delivering the order
the order
location
Customer service con- Customer service tells Customer service tells
tacts customer saying customer the driver customer the driver
Customer service the restaurant closed accidentally delivered could not find the
and could not make the to the wrong location location and ended up
order nearby dropping the order
Table 1
Summaries of sample scenarios. Scripted based on transaction breakdown points discussed in driver
anecdotes posted on forums such as /r/doordash and uberpeople.net.
4. Expected results and why their perception may be affected by how
they receive the information, especially con-
discussion sidering elements like trust and perceived
We envision three potential outcomes for the agency. Due to the role of the customers’
study, along with potential results for each ratings in food delivery systems, we would
outcome. These potential outcomes are not closely consider the relation of participants’
mutually exclusive, and all potential results percentage ratings with their estimated rat-
may occur correspondingly. ings and tips, to consider how such percep-
tions manifest in real life. We would likely
Customers give differing distributions move forward by reaching out to interview
of accountability depending on the mode participants with thoughtful qualitative re-
of explanation. If responses present partic- sponses in hopes of gaining a deeper under-
ular patterns or skew when considering dif- standing of their thought process: How or
fering modes of explanation, we will closely why does a particular mode of explanation
consider the qualitative data given with the evoke stronger feelings of blame toward a
percentage of accountability rating to under- specific stakeholder in the food delivery sys-
stand how design may contribute to or al- tem? If a participant’s estimated ratings and
leviate misattribution in a complex system. tips tend to change with differing percent-
Participants’ reasoning can tell us how and ages of accountability, why? If a participant’s
Figure 1: A screenshot of a preliminary survey format. The given questions would be shown to the
participant after a narrated video storyboard or a written scenario depicting a transaction breakdown
in a food delivery order.
estimated ratings and tips do not change with Does the customer believe the driver could
differing percentages of accountability, why have done more to ensure the goal was met?
not? Or, is the system designed such that blame is
assigned to the driver, perhaps due to their
Customers perceive drivers as par- proximity to the customer or the fact that
tially accountable in scenarios even feedback on the platform is directed towards
when the negative factor is completely the driver?
or partially out of the driver’s control.
If participants commonly give drivers a sig- Neither of the above. If we find that nei-
nificant (≥25%) percentage of blame despite ther outcome is true, we can use the data
the fact that a scenario is intended to fail due we collect on attribution of accountability
to unrelated factors, we will further investi- as well as estimated tipping and rating be-
gate the reasoning behind the misattribution haviour to better understand why and how
of accountability. The reasoning behind this customers rate as they do. As mentioned
misattribution will give us more insight into above, we would likely conduct further inter-
customers’ mental models of food delivery views for more context of why participants’
systems; in such a complex system, we can responses. Through this study, we could gain
discover who customers see as the primary a better understanding of the correlation of
actor in the system and why. Through the blame/accountability of an encounter to a
results of the survey, and extra interviews, given rating or level of tip. Alternatively, we
we can more effectively understand the rea- could further extend the study to explore the
soning behind tendencies to blame the driver: ways in which the system can be better de-
signed to make the system more transparent the adoption of online food deliv-
to the customer. ery by 2 to 3 years, deliveroo ceo
says, CNBC (2020). URL: https://
www.cnbc.com/2020/12/03/deliveroo-
5. Conclusion ceo-says-covid-has-accelerated-
adoption-of-takeaway-apps.html.
As algorithmic management and algorith-
[3] J. Keane, How the pandemic put
mic reliance becomes increasingly prevalent
food delivery firms in the lime-
in the organization of labor, the harms of
light in 2020, Forbes (2020). URL:
misattribution of accountability will become
https://www.forbes.com/sites/
increasingly prevalent as well. Crumple
jonathankeane/2020/12/15/how-
zones, stemming from the misattribution of
the-pandemic-put-food-delivery-
accountability in large, opaque systems with
firms-in-the-limelight-in-2020/?sh=
heavy algorithmic reliance, can be used to
616b6b5f5eeb.
further take advantage of workers in systems
[4] Uber, Deliver with uber in canada,
where they may already be inherently disad-
2020. URL: https://www.uber.com/ca/
vantaged. As an industry with systems de-
en/deliver/.
pendent on a large collection of human and
[5] DoorDash, Driving opportuni-
algorithmic actors, food delivery platforms
ties with doordash, 2021. URL:
are an appropriate environment to consider
https://www.doordash.com/dasher/
the ways harm can be misattributed to par-
driving-opportunities/.
ticular human actors and the ways platforms
[6] K. Griesbach, A. Reich, L. Elliott-Negri,
can be designed to minimize such harm. By
R. Milkman, Algorithmic control in
the end of this study, we hope to understand
platform food delivery work, Socius 5
1) the ways in which customers distribute
(2019) 2378023119870041.
accountability in negative experiences, espe-
[7] M. C. Elish, Moral crumple zones: Cau-
cially when the cause of the incident may
tionary tales in human-robot interac-
be hidden by the complexities of the system
tion, Engaging Science, Technology,
and 2) the mode of explanation that most
and Society 5 (2019) 40–60.
transparently expresses context to the cus-
[8] M. K. Lee, D. Kusbit, E. Metsky, L. Dab-
tomer. Overall, in hopes of further empower-
bish, Working with machines: The
ing workers, we aim to understand the imbal-
impact of algorithmic and data-driven
anced perspectives in food delivery that can
management on human workers, in:
exacerbate unfair outcomes.
Proceedings of the 33rd annual ACM
conference on human factors in com-
References puting systems, 2015, pp. 1603–1612.
[9] M. K. Lee, Understanding perception
[1] M. Yin, S. Suri, M. L. Gray, Running out of algorithmic decisions: Fairness, trust,
of time: The impact and value of flex- and emotion in response to algorith-
ibility in on-demand crowdwork, in: mic management, Big Data & Society
Proceedings of the 2018 CHI Confer- 5 (2018) 2053951718756684.
ence on Human Factors in Computing [10] A. Rosenblat, L. Stark, Algorithmic la-
Systems, 2018, pp. 1–11. bor and information asymmetries: A
[2] S. Shead, Covid has accelerated case study of uber’s drivers, Inter-
national Journal of Communication 10
(2016) 27. agement 18 (2007) 534–554.
[11] M. L. Gray, S. Suri, Ghost work: how to [21] D. T. Gilbert, P. S. Malone, The corre-
stop Silicon Valley from building a new spondence bias., Psychological bulletin
global underclass, Eamon Dolan Books, 117 (1995) 21.
2019. [22] G. D. Lassiter, A. L. Geers, P. J. Munhall,
[12] L. Stark, K. Levy, The surveillant con- R. J. Ploutz-Snyder, D. L. Breitenbecher,
sumer, Media, Culture & Society 40 Illusory causation: Why it occurs, Psy-
(2018) 1202–1220. chological Science 13 (2002) 299–305.
[13] D. Support, Dasher ratings explained, [23] P. Moradi, K. Levy, The future of work
2021. URL: https://help.doordash.com/ in the age of ai: Displacement or risk-
dashers/s/article/Dasher-Ratings- shifting? (2020).
Explained?language=en_US. [24] H. Nissenbaum, Computing and ac-
[14] Zoomo, How to maintain a high rat- countability, Communications of the
ing on uber eats, 2020. URL: https: ACM 37 (1994) 72–81.
//www.ridezoomo.com/blog/how-to- [25] S. Woods, M. Walters, K. L. Koay,
maintain-a-high-rating-on-uber-eats. K. Dautenhahn, Comparing human
[15] D. Martin, B. V. Hanrahan, J. O’Neill, robot interaction scenarios using live
N. Gupta, Being a turker, in: Proceed- and video based methods: towards a
ings of the 17th ACM conference on novel methodological approach, in: 9th
Computer supported cooperative work IEEE International Workshop on Ad-
& social computing, 2014, pp. 224–235. vanced Motion Control, 2006., IEEE,
[16] M. Miah, G. Das, V. Hristidis, H. Man- 2006, pp. 750–755.
nila, Standing out in a crowd: Selecting
attributes for maximum visibility, in:
2008 IEEE 24th International Confer-
ence on Data Engineering, IEEE, 2008,
pp. 356–365.
[17] L. Fuller, V. Smith, Management by cus-
tomers in a changing economy, Work,
Employment and Society 5 (1991) 1–16.
[18] V. Schoenmueller, O. Netzer, F. Stahl,
The polarity of online reviews: Preva-
lence, drivers and implications, Journal
of Marketing Research 57 (2020) 853–
877.
[19] L. Ross, The intuitive psychologist
and his shortcomings: Distortions in
the attribution process, in: Advances
in experimental social psychology, vol-
ume 10, Elsevier, 1977, pp. 173–220.
[20] N. Specht, S. Fichtel, A. Meyer, Percep-
tion and attribution of employees’ ef-
fort and abilities: The impact on cus-
tomer encounter satisfaction, Interna-
tional Journal of Service Industry Man-