In today’s technology ecosystem, technology products are informed by user research that involves quantitative and qualitative methods. AI products are no exception and require accurately and robustly collected human data.
Human-centered design, evangelized by design consultancies such as the Stanford Design School and IDEO, is an approach to problem-solving using empathy and creativity to co-create usable and useful solutions. Best practice in developing high-value prototypes includes an open-ended approach that is grounded in user perspectives in the problem identification and solution validation. Problem identification includes identifying end-user, creating user personas, and conducting user interviews for understanding the true challenge. Solution validation ranges from ethnographic observations or guided click-through environments assessing product usage, satisfaction and feedback surveys, or interviews. Although user research has significant overlap with social science research techniques, the main difference is a lack of theory, methodological justification, and meticulous control in sampling procedures.
Due to the fail-fast, agile ways of working employed in rapid prototyping builds, typically less time is invested upfront into user research. Prototyping processes ask to build with the end-user in mind, often creating target personas to manage scope. A target demographic does not waive the need for user researchers to have a strong methodological justification and discussion on sample limitations or testing reliability and validity. However, user research becomes a check-box exercise that is used to validate the envisaged product with a few demographically representative end-users. Products are built with an evaluative rather than generative approach. Evaluative techniques such as interviews, surveys, or ethnography do not allow for an analysis of problem root-causes nor the relationship between different stakeholders or levels in a given system. The product only addresses immediate needs of the identified user demographic for the current problem instead of a future-proofing, systems thinking approach that maps the people and polices impacting a social problem.
Within the design community, this difference has begun to be critically discussed as what is a user-centered versus human-centered approach (Gasson 2003). Although both practices involve focusing on people in building technology, one harbors the danger of exclusion or perpetuating inequality in society. Leader in user experience research and father of the Nielsen Norman Group, Donald Norman (2005), has argued that human-centered design can cause societal harm when designing with a focus on the individual.
Oftentimes, technology products have a specific target audience. Building a successful product for that particular user group might in turn create technology that results in being harmful to a different social group. A prominent example is Google’s 2015 incident where image recognition incorrectly tagged Black people as gorillas. Algorithmic bias, where although AI is trained to properly recognize White people, falls short when identifying other groups and in turn perpetuates racism. Due to the algorithmic ‘black box’, it is even more essential for cognitive designers to document robust methodological decisions for AI explainability.
It is important to also recognize that technology is not always the right nor the only solution for a particular group or societal problem. Methodological choices must be selected with cultural understanding of end-users in mind.
Especially with technology being a recent phenomenon, attitude might vary based on demographics, geography, or ability. For example, Pivotal Acts, the foundation arm of the big tech software company, using human-centered design principles investigated and found that public toilets were not being utilized by women in a refugee camp because men were congregating at night around the vicinity as the only source of light in the camp.
By using an interdisciplinary approach to the creation of technology, products can ensure rigorous methodological processes that accounts for a diversity of perspective or limitations thereof, and most importantly in the development of AI technologies, understanding of human behavior (Barker 1964). Undoubtedly, co-creating with end-users results in more applicable products. This research pushes the Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). design community one step further. Rather than simply allowing the end-user to participate in technology design, it is essential that equality and human rights are at the heart of human-centered design to ensure that technology products result in an outcome inclusive of all (Buchanan 2001). With technology’s ultimate goal being increased efficiency, products must responsibly consider ramifications on society and strive to improve quality of life of users without impinging on the rights of other groups (Norman 2005). Effectiveness cannot come at the cost of exclusion. This research is a meta-analysis of global AI start-ups that are headquartered the Western world, evaluating their design approach. Using the ecological levels of analysis and the emphasis on both applied and theory-based research from the field of community psychology (Kelly 2006), this study will assess whether a technology company’s product mission is framed as an individual or community-level solution and subsequent societal impact. This paper calls for a more holistic and actionoriented approach to product design in order to enable greater useability, accessibility, inclusion, and ultimately human dignity and flourishing. Barker, R. G. 1964. Ecological psychology: Concepts and methods for studying the environment of human behavior. Stanford, California.: Stanford University Press.
Buchanan, R. 2001. Human Dignity and Human Rights; Thoughts on the Principles of Human-Centered Design. Design Issues 17(3): 35-39.
Gasson, S. 2003. Human-Centered vs. User-centered Approaches to Information System Design. Journal of Information Technology Theory and Application 5(2): 29-46.
Kelly, J. G. 2006. Becoming ecological: An expedition into community psychology. New York, New York.: Oxford University Press.