“Not Nice!”: Towards Understanding Dark Patterns in Com mercial Health Apps

Abstract

Commercial health apps have become more accessible, ubiquitous, and popular than ever before. These apps serve various purposes, such as enhancing health literacy, enabling continuous health tracking and management, and facilitating community engagement on health-related topics. As health apps continue to grow in popularity and adoption, scholars have raised concerns about the privacy, commodification, and exploitation of data generated within these apps. However, we know less about the presence and implications of deceptive design patterns and coercive practices in such apps from the users' perspective. We look at pregnancy as a use case and present preliminary findings from ongoing research on understanding user experiences with commercial pregnancy tracking apps. Based on this case, we argue that the context of health apps calls for a nuanced consideration of deceptive design practices because (1) deceptive patterns can intersect with users' vulnerability in ways unique to health, and (2) implications of such patterns in health can go well beyond financial losses and invasion of privacy commonly observed in e-commerce and social network services.

Keywords

• UX
• dark patterns

-

CEUR ceur-ws.org

1. Introduction

The idea of leveraging Information and Communication Technologies (ICTs) to involve patients in healthcare dates back to the late 20th century [ 1, 2 ]. However, the shift towards patients’ active participation in healthcare really gained momentum with the rise of Web 2.0, where technologies such as cloud computing, mobile apps, interactive media, and social networks are incorporated into healthcare and medicine (also referenced as Health 2.0 [ 3 ]). Foundational principles underpinning the Health 2.0 movement include (1) patient-centered healthcare— Health 2.0 advocates for shared decision-making where patients have control and agency over their own health, (2) on-demand delivery—where patients have day-and-night access to health information and healthcare services, and (3) proactive care—Health 2.0 as a vehicle to support preventive medicine by using technology for health literacy and continuous monitoring and nEvelop-O assessment, thereby anticipating and addressing potential health risks and concerns proactively. Over the past quarter-century, Health 2.0 has undergone significant evolution theoretically and practically [ 3, 4, 5, 6, 7 ], witnessing the development and adoption of new technology-enabled solutions aimed at engaging patients in health monitoring and care (e.g., [8]).

Within the broader scope of Health 2.0, health applications (apps) available on mobile devices particularly gained traction as avenues to engage patients in self-monitoring and self-care due to their ubiquity, availability, and accessibility—one US survey found that 58% of smartphone users downloaded at least one health app in 2015 [9]. Today, smartphone users can access various health apps catering to their specific health needs and goals (e.g., fitness and activity tracking, diet and nutrition, medication management, mental health and wellness, and women’s health and menstrual tracking). The popularity and adoption of health apps are projected to continue to grow globally as reports estimate a market size of USD 861.40 billion in 2030 [10]. Weaving health-tracking technologies within everyday life holds the promises of democratizing healthcare, providing new forms of self-determination, and producing insights into (and solutions for) individual and public health and well-being [11, 12].

As the market continues to grow, critiques have challenged this “Techno-Utopian” narrative of health apps. Scholars in the humanities argue that these apps can enforce a reductionist view of health and well-being, one that fails to acknowledge the complexities and ambivalences of embracing self-monitoring and self-care, withdrawing from healthcare to self-care, and reducing individuals to their mere data traces, often disregarding their humanity, personality, and individuality [ 6, 4, 13 ]. Other critiques approach the topic from a political-legal perspective, raising concerns around asymmetrical power dynamics between “those who collect, store, and mine large quantities of data and those whom data collection targets”( [14], p. 1673) and the commodification of health data, where data collected by (or for) individuals within these apps is monetized by companies that extract and combine it with others’ data to draw population-wide correlations and inferences (e.g., [15]). In that sense, Nafus and Nef argue, selves are “sliced and diced into decontextualized parts, and bought and sold”( [16], p. 62). This paper does not directly address these criticisms so much as obliquely. Instead, we seek to position design as a space to discuss (and potentially resolve) a polarized debate around self-tracking health apps.

In HCI, an extensive body of work is devoted to ethical design [17, 18], prompting the design of technology that is transparent and mindful of users’ autonomy, privacy, and psychological well-being [ 19, 20, 21 ]. Furthermore, scholars have objected to “deceptive and manipulative” design practices (often referenced as dark patterns [ 22 ]), developed taxonomies [ 23, 24 ] and conceptual foundations to describe such patterns [ 25, 26 ], and interrogated their presence in e-commerce websites [ 27 ], popular mobile apps [ 28 ], across modalities [ 29 ], in social media platforms [ 30 ] and games [ 31 ]. This body of work has been invaluable in initiating conversations and instigating regulatory initiatives to counter such unethical design practices (e.g., [32, 33]). Yet, we know little about how these patterns manifest and impact users in the specific domain of commercial health apps. A scrutiny of dark patterns angled towards commercial health apps is warranted due to factors specific to the health domain: (1) users of commercial health apps typically seek solutions to health-related concerns, which makes them particularly vulnerable to unethical design practices as these can prey on their anxiety, fear, and desire for a “quick-fix,” (2) the high-stakes context—health apps are positioned as tools to improve the health and well-being of users, however, if these apps prioritize revenue and users’ engagement metrics, this may lead to adverse health efects that necessitate a nuanced examination as such efects can go well beyond financial losses and invasion of privacy [ 25 ] commonly found in e-commerce services [ 27 ] and social networks [ 30 ], and (3) many health apps cater to intimate and sensitive topics (e.g., fertility tracking); thus, concerns around privacy and exploitation of personal data become exacerbated.

Here, drawing on ongoing research on understanding UX with commercial pregnancy apps by examining users’ reviews, we focus on users’ comments expressing concern about questionable design practices and monetization tactics within some of the commercial pregnancy apps included in our study. We use these comments to showcase the need to scrutinize commercial health apps for questionable design practices. Particularly, we surface a need to (1) broaden our conceptualization, within the HCI research community, of users’ vulnerability as a dynamic construct that changes in severity and seriousness over time, which necessitates newer methods to capture questionable design patterns in context, and (2) better understand implications of dark patterns in the context of health due to the sensitivity and value attributed to health data. Before we present our proposition on these two aspects, we briefly describe our methodology to provide context.

2. Method

In ongoing research on understanding UX with commercial pregnancy apps, we conduct a qualitative analysis of user reviews on popular pregnancy apps available on the Google Play store 1. In creating our dataset of app reviews, we conducted a keyword search for the term “pregnancy,” which resulted in identifying 250 apps, excluding e-books and movies. We chose the apps that are most popular based on the number of downloads and the number of ratings, i.e., apps that have been installed 1M+ and rated 100K+ times; these numbers represented the highest observed range of installs/ratings in the category of pregnancy apps as per the Google Play store at the time of extraction. Of the 250 apps, six satisfied the inclusion criteria and thus were included in our analysis, namely Pregnancy+, Baby Center, Pregnancy Week by Week, AMMA, Ovia, and The Asianparent. For each app, we collected a random sample of the textual reviews of the users, which we then analyzed using thematic analysis [34]. In our dataset, we found user comments pointing to questionable design practices. We present and discuss examples next. It is worth mentioning that user reviews are publicly available, and such data is considered exempt from IRB review. Nonetheless, in our reporting, we anonymize usernames, replacing those with incremental number tags to protect users against possible identification, and perform a Google search (i.e., copy-paste reviews in Google) to verify that none of the quotes that appear in this manuscript is a top hit. Further, we remove apps’ names and replace those with randomly generated IDs as our purpose is to point out questionable design practices from the users’ perspective—we acknowledge that further assessment is warranted before drawing conclusions regarding the presence of dark patterns in a specific app.

3. Vulnerability is Dynamic

While there is a general consensus that vulnerable individuals deserve special protection and safeguards, defining what vulnerability is and identifying vulnerable individuals is not as simple. An extensive body of work has discussed vulnerability in relation to social identity, calling for an intersectional approach [35] to acknowledge that people often experience vulnerabilities and disadvantages in complex and interconnected ways. Within HCI, this conceptualization of vulnerability has served to identify health disparities and pushed for health applications designed to cater to the needs of disadvantaged and marginalized communities [36]. However, more than a consequence related to some characteristic of a group (e.g., racial identity or socioeconomic status) [37], vulnerability can, more comprehensively, be considered as “occurring along a spectrum of seriousness and as a consequence of situations and context.”( [37], p. 38). In our analysis of UX with commercial pregnancy apps, we find that users describe vulnerability in relation to contextual/temporal factors (e.g., early stage of pregnancy, the state of being in labor), and (2) such “contextual vulnerability” can be exploited by questionable design practices leading to undesirable (even harmful) consequences for the users, as exemplified by the quotes below: [A5, R1038]“Paywall at the most vulnerable moment of my pregnancy. I was generally satisfied with the app during my pregnancy, but when I went into labor and urgently needed a contractions counter to understand if I should go to the hospital already or not, the app suggested an upgrade to the paid version. Frankly, I don’t mind paying couple bucks for a good app, but the moment you’ve asked for the money was terribly wrong. It felt like you were trying to take advantage of a laboring woman. Not nice!” [A5, R1024]“3 stars due to abhorrent money extraction strategy about contraction counter. If, in the due course, I was asked to pay to continue using, I would. However, trying to extract money making the Contraction Counter a paid facility and keeping this a secret until the contractions start.. was very very very low.” [A1, R2]“I don’t like that there are ads for medications right in the app. I understand ads are their revenue stream, but I think putting ads for morning sickness medication in front of nauseated women is unethical.” [A1, R284]“There are articles for cancer screening, which I don’t think is the most helpful for a pregnant woman at this time when there’s already anxiety around being healthy growing a child.” [A2, R705]“I feel this app prays too much on your fears. It has you questioning everything you put into, or on, your body while pregnant. Even if there is little evidence to back up claims of what ever is in question of being harmful, they guilt you, and scare you, into avoiding it all together. It basically makes you feel like everything your doing could be harmful to the pregnancy.” [A2, R57]“Some good info but a lot of fear-provoking and spam emails. At week 6, the main focus was miscarriage. Yes, it is common early on, but to have an entire week devoted to scare tactics is revolting.”

In reflecting on the quotes above, users describe scenarios (e.g., being in labor, nauseated, anxious) where vulnerability is contextual and dynamic—it changes in severity and over time. One significant challenge for dark patterns research is to capture questionable design practices that intersect with users’ vulnerability in context. For example, methodologies commonly used in dark pattern research [ 30, 29, 38, 28, 27 ] such as static screenshots of the user interface or short video recordings of app interaction would probably not have captured the case described by [A5, R1038] and [A5, R1024]—as the manifestation of this pattern would require using the app for an extensive period of time resembling the nine-month leading up to labor. To that end, researchers might find value in augmenting their methods with exploratory qualitative analyses of app reviews to inform their study design. App reviews are shown to carry valuable information on usability and UX [39]; our ongoing research suggests that user reviews can also point to questionable design practices that warrant further security. Furthermore, while [A5, R1038] and [A5, R1024] discuss financial losses as consequences of questionable design practices, intangible harms, such as anxiety and fear, discussed in [A1, R284], [A2, R705] and [A2, R57] are harder to operationalize; those are highly subjective experiences; what one individual perceives as anxiety-provoking or fear-inducing may not afect another person in the same way. Thus, it is dificult for researchers to have a baseline standard in mind when examining such patterns as potentially problematic. Perhaps participatory approaches to evaluate the (in)appropriateness of such designs by leveraging expertise from various perspectives, including health professionals, UX and interface designers, psychologists, and digital ethicists, would be fruitful in that regard.

4. The Sensitivity and Value of Health Data

Mathur et al. [ 25 ] propose “Individual Autonomy” as a normative lens by which researchers can describe dark patterns—“a dark pattern that infringes on individual autonomy might modify choice architecture in a way that causes users to make choices that they would not have otherwise selected absent the modified choice architecture. Alternatively, a dark pattern might deny a user choice, obscure available choices, or burden the exercise of choice”( [ 25 ], p. 12-13). Our dataset contains multiple instances where users describe design patterns that limit or obscure their choices, forcing actions that lead to undesirable outcomes; specifically concerning users’ privacy, as exemplified by the quotes below: [A5, R39]“I was forced to create an account today to continue using the app at over halfway through my pregnancy, including agreeing to them storing my weight and other info. After creating the account I then had to verify my account and now it’s lots all my data including my name, babies gender, my age etc. A Bit disappointed but can’t be bothered looking for a new app at 27 weeks pregnant.” [A1, R525] “I shouldn’t be forced to enter all of the most personal information and details of my delivery and my baby in order to claim I gave birth so that I can move onto my next pregnancy and keep it logged. I’m looking for a new app for this next baby. LOVED this app last time, but this is info I care not to share. MY CHOICE.” [A2, R681]“I am extremely disappointed that I started receiving [baby furniture company] emails because this app sold my information to third-party partners/advertisers. Upon reading the privacy policy, there is no way to opt out of this, but instead, you have to unsubscribe from every email that is sent to you. There is not a list provided of direct marketing partners, so if start receiving newsletters and ads for things you do not want or did not sign up for, this app is to blame.”

The examples above point out (1) forced action [ 24 ], a well-recognized design pattern in the dark patterns literature that describes situations where users are required to perform specific actions to use (or continue to use) the app’s functionalities, and (2) privacy-respecting choices being hard or restricted to access. While such practices are well-described in the literature, their implications in health, we propose, warrant a nuanced consideration because of the sensitivity and the value attributed to health data, which intersects with, but also extend beyond, privacy concerns, tapping into questions of accountability and data ownership as exemplified by the quotes below: [A5, R10]“Very upset! I loved this app. I used it almost daily. I tracked my weight gain and everything else on this app. When I was FORCED to “upgrade” all of my data was deleted. I’m very upset, as I have lost everything. This was a great app before. I’m not happy that I was forced to change.” [A5, R363]“I’m so angry. 20 weeks along and AAALLLLL of my data is gone except my name and due date. Half of my pregnancy is WIPED AWAY! you should not force updates if they are going to clear data, especially without telling the user beforehand. I was using this app to keep notes for my doctor. I should have known better.”

5. Conclusion

This paper argues that users’ vulnerability to deceptive design is dynamic. This calls for innovative approaches to capture questionable design practices in context. To that end, we see the potential for re-purposing user reviews in the early stage of dark patterns research, specifically when interrogating dark patterns in mobile apps. Researchers can perform preliminary qualitative analysis of app reviews to inform their study design. User reviews serve as a rich source of real-world experiences and perceptions, ofering insights into the subtle nuances of user interactions and frustrations. To that end, researchers can leverage user reviews to infer indicators on designing their protocol to adequately capture the breadth and depth of questionable scenarios described by users. Furthermore, incorporating users’ perspectives into the research methodology aligns with the user-centered design philosophy, bringing together dark patterns research objectives and the experiences and concerns of the individuals afected by questionable design practices. In addition, given the sensitivity of the context, we argue that the implications of dark patterns in health apps may go beyond the invasion of privacy and ifnancial losses commonly discussed in the context of e-commerce websites and social media platforms. To that end, expertise from diverse perspectives, including healthcare professionals, technology ethicists, law experts, designers, and HCI researchers, can contribute to evaluating the (in)appropriateness of questionable design patterns, providing collective eforts to prioritize users’ well-being and ethical design principles in the rapidly evolving landscape of digital health technologies.

6. Acknowledgments

The first author acknowledges financial support from King Saud University and the Saudi Arabian Cultural Mission to the United States. Internet Research Distributor: Journal of Medical Internet Research Institution: Journal of Medical Internet Research Label: Journal of Medical Internet Research Publisher: JMIR Publications Inc., Toronto, Canada. [8] M. Tai-Seale, N. L. Downing, V. G. Jones, R. V. Milani, B. Zhao, B. Clay, C. D. Sharp, A. S.

Chan, C. A. Longhurst, Technology-Enabled Consumer Engagement: Promising Practices At Four Health Care Delivery Organizations, Health Afairs 38 (2019) 383–390. URL: https://www.healthaffairs.org/doi/full/10.1377/hlthaff.2018.05027. doi:10.1377/hlthaff. 2018.05027, publisher: Health Afairs. [9] P. Krebs, D. T. Duncan, Health App Use Among US Mobile Phone Owners: A National

Survey, JMIR mHealth and uHealth 3 (2015) e101. doi:10.2196/mhealth.4924. [10] F. B. Insights, mHealth Apps Market Size to Surpass USD 861.40 billion by 2030, exhibiting a CAGR of 40.2%, 2023. URL: https: //www.globenewswire.com/news-release/2023/09/14/2743083/0/en/ mHealth-Apps-Market-Size-to-Surpass-USD-861-40-billion-by-2030-exhibiting-a-CAGR-of-40-2. html. [11] G. B. Jones, A. Bryant, J. Wright, Democratizing Global Health Care Through Scalable Emergent (Beyond the Mobile) Wireless Technologies, JMIR Biomedical Engineering 7 (2022) e31079. URL: https://biomedeng.jmir.org/2022/1/e31079. doi:10.2196/31079, company: JMIR Biomedical Engineering Distributor: JMIR Biomedical Engineering Institution: JMIR Biomedical Engineering Label: JMIR Biomedical Engineering Publisher: JMIR Publications Inc., Toronto, Canada. [12] P. Bravo, A. Edwards, P. J. Barr, I. Scholl, G. Elwyn, M. McAllister, C. U. the Cochrane Healthcare Quality Research Group, Conceptualising patient empowerment: a mixed methods study, BMC Health Services Research 15 (2015) 252. URL: https://doi.org/10.1186/ s12913-015-0907-z. doi:10.1186/s12913- 015- 0907- z. [13] T. Sharon, Self-Tracking for Health and the Quantified Self: Re-Articulating Autonomy, Solidarity, and Authenticity in an Age of Personalized Healthcare, Philosophy & Technology 30 (2017) 93–121. URL: https://doi.org/10.1007/s13347-016-0215-5. doi:10.1007/s13347- 016- 0215- 5. [14] M. Andrejevic, The Big Data Divide (2014). [15] D. Rosato, What Your Period Tracker App Knows About You, 2020. URL: https://www.consumerreports.org/health/health-privacy/ what-your-period-tracker-app-knows-about-you-a8701683935/. [16] G. Nef, D. Nafus, Self-Tracking, MIT Press, 2016. Google-Books-ID: b4uMDAAAQBAJ. [17] B. Friedman, P. H. Kahn, A. Borning, A. Huldtgren, Value Sensitive Design and Information Systems, in: N. Doorn, D. Schuurbiers, I. van de Poel, M. E. Gorman (Eds.), Early engagement and new technologies: Opening up the laboratory, Philosophy of Engineering and Technology, Springer Netherlands, Dordrecht, 2013, pp. 55–95. URL: https://doi.org/10.1007/978-94-007-7844-3_4. doi:10.1007/978- 94- 007- 7844- 3_4. [18] C. Knobel, G. C. Bowker, Values in design, Communications of the ACM 54 (2011) 26–28.

URL: https://dl.acm.org/doi/10.1145/1965724.1965735. doi:10.1145/1965724.1965735. [19] B. Friedman, H. Nissenbaum, User autonomy: who should control what and when?, in: Conference Companion on Human Factors in Computing Systems, CHI ’96, Association for Computing Machinery, New York, NY, USA, 1996, p. 433. URL: https://dl.acm.org/doi/ //urn.kb.se/resolve?urn=urn:nbn:se:ri:diva-24252. [32] Warner, Fischer Lead Bipartisan Reintroduction of Legislation to Ban Manipulative ’Dark Patterns’, 2023. URL: https://www.warner.senate.gov/public/index.cfm/2023/7/ warner-fischer-lead-bipartisan-reintroduction-of-legislation-to-ban-manipulative-dark-patterns. [33] FTC Report Shows Rise in Sophisticated Dark Patterns Designed to Trick and Trap Consumers, 2022. URL: https://www.ftc.gov/news-events/news/press-releases/2022/09/ ftc-report-shows-rise-sophisticated-dark-patterns-designed-trick-trap-consumers. [34] V. Clarke, V. Braun, N. Hayfield, Thematic analysis, Qualitative psychology: A practical guide to research methods 222 (2015) 248. [35] K. Crenshaw, Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of Antidiscrimination Doctrine, Feminist Theory and Antiracist Politics, University of Chicago Legal Forum 1989 (2015). URL: https://chicagounbound.uchicago.edu/uclf/vol1989/ iss1/8. [36] E. Stowell, M. C. Lyson, H. Saksono, R. C. Wurth, H. Jimison, M. Pavel, A. G. Parker, Designing and Evaluating mHealth Interventions for Vulnerable Populations: A Systematic Review, in: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, Association for Computing Machinery, New York, NY, USA, 2018, pp. 1–17. URL: https://doi.org/10.1145/3173574.3173589. doi:10.1145/3173574.3173589. [37] B. G. Gordon, Vulnerability in Research: Basic Ethical Concepts and General Approach to Review, The Ochsner Journal 20 (2020) 34–38. URL: https://www.ncbi.nlm.nih.gov/pmc/ articles/PMC7122263/. doi:10.31486/toj.19.0079. [38] C. Moser, S. Y. Schoenebeck, P. Resnick, Impulse Buying: Design Practices and Consumer Needs, Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (2019) 1–15. URL: https://dl.acm.org/doi/10.1145/3290605.3300472. doi:10.1145/3290605. 3300472, conference Name: CHI ’19: CHI Conference on Human Factors in Computing Systems ISBN: 9781450359702 Place: Glasgow Scotland Uk Publisher: ACM. [39] S. Hedegaard, J. G. Simonsen, Extracting usability and user experience information from online user reviews, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’13, Association for Computing Machinery, New York, NY, USA, 2013, pp. 2089–2098. URL: https://doi.org/10.1145/2470654.2481286. doi:10.1145/2470654. 2481286.

References

[1] P. F. Brennan , I. Strombom , Improving Health Care by Understanding Patient Preferences: The Role of Computer Technology , Journal of the American Medical Informatics Association 5 ( 1998 ) 257 - 262 . URL: https://doi.org/10.1136/jamia. 1998 . 0050257 . doi: 10 .1136/jamia. 1998 . 0050257 .

[2] A. S. Pombortsis , Communication technologies in health care environments , International Journal of Medical Informatics 52 ( 1998 ) 61 - 70 . URL: https://www.sciencedirect.com/ science/article/pii/S1386505698001257. doi: 10 .1016/S1386- 5056 ( 98 ) 00125 - 7 .

[3] T. H. V. D. Belt , L. J. Engelen , S. A. Berben , L. Schoonhoven, Definition of Health 2.0 and Medicine 2 .0: A Systematic Review , Journal of Medical Internet Research 12 ( 2010 ) e1350 . URL: https://www.jmir.org/ 2010 /2/e18. doi: 10 .2196/jmir.1350, company: Journal of Medical Internet Research Distributor: Journal of Medical Internet Research Institution: Journal of Medical Internet Research Label: Journal of Medical Internet Research Publisher: JMIR Publications Inc ., Toronto, Canada.

[4] A. Mol , J. Law , Embodied Action, Enacted Bodies: the Example of Hypoglycaemia, Body & Society 10 ( 2004 ) 43 - 62 . URL: https://doi.org/10.1177/1357034X04042932. doi: 10 .1177/ 1357034X04042932, publisher: SAGE Publications Ltd.

[5] D. Nicolini , Stretching out and expanding work practices in time and space: The case of telemedicine , Human Relations 60 ( 2007 ) 889 - 920 . URL: https://doi.org/10.1177/ 0018726707080080. doi: 10 .1177/0018726707080080, publisher: SAGE Publications Ltd.

[6] D. Lupton , The digitally engaged patient: Self-monitoring and self-care in the digital health era , Social Theory & Health 11 ( 2013 ) 256 - 270 . URL: https://doi.org/10.1057/sth. 2013 . 10 . doi: 10 .1057/sth. 2013 . 10 .

[7] G. Eysenbach , Medicine 2 .0: Social Networking , Collaboration, Participation, Apomediation, and Openness, Journal of Medical Internet Research 10 ( 2008 ) e1030 . URL: https://www.jmir.org/ 2008 /3/e22. doi: 10 .2196/jmir.1030, company: Journal of Medical 10.1145/257089 .257434. doi: 10 .1145/257089.257434.

[20] C. Norval , K. Cornelius , J. Cobbe , J. Singh , Disclosure by Design: Designing information disclosures to support meaningful transparency and accountability , in: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency , FAccT '22, Association for Computing Machinery, New York, NY, USA, 2022 , pp. 679 - 690 . URL: https://dl.acm.org/doi/10.1145/3531146.3533133. doi: 10 .1145/3531146.3533133.

[21] J. van Rest , D. Boonstra , M. Everts , M. van Rijn , R. van Paassen , Designing Privacy-byDesign, in: B. Preneel , D. Ikonomou (Eds.), Privacy Technologies and Policy, Lecture Notes in Computer Science , Springer, Berlin, Heidelberg, 2014 , pp. 55 - 72 . doi: 10 .1007/ 978-3- 642 -54069- 1 _ 4 .

[22] H. Brignull , Dark Patterns, 2019 . URL: https://www.deceptive.design/.

[23] G. Conti , E. Sobiesk, Malicious interface design: exploiting the user , in: Proceedings of the 19th international conference on World wide web, WWW '10 , Association for Computing Machinery, New York, NY, USA, 2010 , pp. 271 - 280 . URL: https://doi.org/10.1145/1772690. 1772719. doi: 10 .1145/1772690.1772719.

[24] C. Gray , Y. Kou , B. Battles , J. Hoggatt , A. Toombs , The Dark (Patterns) Side of UX Design , 2018 . doi: 10 .1145/3173574.3174108.

[25] A. Mathur , M. Kshirsagar , J. Mayer , What Makes a Dark Pattern ... Dark?: Design Attributes, Normative Considerations , and Measurement Methods, in: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems , ACM, Yokohama Japan , 2021 , pp. 1 - 18 . URL: https://dl.acm.org/doi/10.1145/3411764.3445610. doi: 10 .1145/ 3411764.3445610.

[26] T. Kollmer , A. Eckhardt , Dark Patterns, Business & Information Systems Engineering 65 ( 2023 ) 201 - 208 . URL: https://doi.org/10.1007/s12599-022-00783-7. doi: 10 .1007/ s12599-022-00783-7.

[27] A. Mathur , G. Acar, M. Friedman , E. Lucherini , J. Mayer , M. Chetty , A. Narayanan , Dark Patterns at Scale: Findings from a Crawl of 11K Shopping Websites , Proceedings of the ACM on Human-Computer Interaction 3 ( 2019 ) 1 - 32 . doi: 10 .1145/3359183.

[28] L. Di Geronimo , L. Braz , E. Fregnan , F. Palomba , A. Bacchelli , UI Dark Patterns and Where to Find Them: A Study on Mobile Applications and User Perception , Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems ( 2020 ) 1 - 14 . URL: https://dl. acm.org/doi/10.1145/3313831.3376600. doi: 10 .1145/3313831.3376600, conference Name: CHI '20: CHI Conference on Human Factors in Computing Systems ISBN: 9781450367080 Place: Honolulu HI USA Publisher: ACM.

[29] J. Gunawan , A. Pradeep , D. Chofnes , W. Hartzog , C. Wilson , A Comparative Study of Dark Patterns Across Web and Mobile Modalities , Proceedings of the ACM on HumanComputer Interaction 5 ( 2021 ) 1 - 29 . URL: https://dl.acm.org/doi/10.1145/3479521. doi: 10 . 1145/3479521.

[30] T. Mildner , G.-L. Savino , P. R. Doyle , B. R. Cowan , R. Malaka , About Engaging and Governing Strategies: A Thematic Analysis of Dark Patterns in Social Networking Services , in: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems , 2023 , pp. 1 - 15 . URL: http://arxiv.org/abs/2303.00476. doi: 10 .1145/3544548.3580695, arXiv: 2303 .00476 [cs].

[31] J. P. Zagal , S. Björk , C. Lewis , Dark Patterns in the Design of Games , 2013 . URL: https: