Intelligent algorithms have reached a new level of performance in recent years and are increasingly employed in application areas such as speech and image recognition, data analytics, or recommender systems. The proliferation of these techniques poses a range of new challenges for the design and engineering of interactive systems since they tend to act as black boxes and do not offer the transparency and level of control to the user which is considered a prerequisite for user-centered design in the HCI field. In this position paper, we provide an overview of the broad areas related to intelligent algorithms and HCI that will need further research in the future to make systems useful, usable and trustable.
Since the early days of computing, two fundamentally different concepts of the relationship between human and the technical system have been promoted by the fields of Artificial Intelligence (AI) and Human-Computer Interaction, respectively. Artificial Intelligence researchers have been aiming at automating and extensively substituting human mental (and physical) performance while the perspective of Intelligence Augmentation, as represented, among others by Douglas Engelbart, saw technology as a means of enhancing human capabilities which lead to the manifold HCI innovations that we have seen over the past decades. While AI has in the past often been charged with far-reaching promises, often followed by disillusioning results, recent developments have pointed to a breakthrough in the performance of intelligent technologies. New methods of machine learning, particularly deep learning methods based on neuronal techniques (Goodfellow et al., 2016), and the increasing availability of very large data sets ("big data") with which the systems can be trained have played a significant role in these developments (cf. O' Leary, 2013). In recent years, these techniques have produced amazing results in application areas such as natural language understanding (Otter et al., 2018, Cho et al., 2014), image comprehension (Druzhkov & Kustikova, 2016) He et al., 2015), data analytics (Najafabadi et al., 2015)), recommender systems (Zhang et al., 2019, Donkers, Loepp, & Ziegler, 2017), or the generation of photorealistic or artistic artifacts (Elgammal, Liu, Elhoseiny & Mazzone, 2017).
While the potential for automating human performance has increased significantly as a result of these successes, the new intelligent techniques also deepen the gap between user and system, since transparency and traceability of decisions generated by probabilistic techniques are mostly not given (Samek, Wiegand & Müller, 2017). Moreover, users have typically no means for interactively controlling the reasoning process. Especially with the otherwise very successful neural network techniques, the system represents a black box from the user's point of view. For system design and development methods that are oriented towards the goals and needs of the users, this represents a considerable challenge, which developers will increasingly have to face up to in the future.
In the following, we will discuss what we see as the main challenges for engineering interactive systems that are fully or partially based on intelligent algorithms. Each challenge stands for a broad range of research questions that have only initially been investigated so far. In general, the challenges ask for a much closer interdisciplinary cooperation between researchers in HCI and AI which up to now are still often separated by a large gap.
Challenges for Engineering AI-based Systems
For the presentation of and interaction with AI components, many specific questions arise beyond the classical aspects of user interface design. These include the choice of a suitable interaction style, ranging from conventional GUI solutions to conversational and language-based interaction to anthropomorphic virtual agents (Lucas et al., 2014).
In addition, embedded interactions, as they occur in the Internet of Things, in robotics or in automated vehicles, have to be considered (Kranz, Holleis & Schmidt, 2010). In many cases, the interaction is multimodal, with an extension of previously pursued approaches to multimodality in the context of new machine-learning processes. The increasingly proactive role of AI components, which can initiate situational interactions, poses a variety of research questions, among others with regard to attention guidance, the interruptibility of user activities, or the change of initiative in dialogue.
One of the central problems with regard to the usability of intelligent algorithms is the lack of transparency and explanation of automatically made decisions or recommendations (cf. Tintarev & Masthoff, 2015;Donkers et al. 2018). While with conventional techniques such as rule-based systems or decision trees the conclusions can at least in principle still be presented in a comprehensible way, this is usually not the case with neural techniques. In this case, even the developers can hardly comprehend the machine decisions. For reasons of usability as well as information self-determination and data protection (see e. g. European General Data Protection Regulation), it is therefore essential to develop mechanisms that can at least explain the decision mechanisms relevant from the user's point of view and their data basis (Doran, Schulz & Besold, 2017). Although there are initial technical approaches, the question of the type and level of detail of system-generated explanations and their presentation is still a largely open research question (cf. Dosilovic 2018). Overall, the question of user-oriented explanations for machine-made decisions forms a central aspect for the user life and acceptance of AIs.
In addition to the explanatory nature of algorithmic decisions, the further question from the user's perspective is whether and how algorithmic decision-making processes can be influenced and controlled. In principle, user control can start at different stages of the process. This ranges from the selection of data to be used for learning processes to the selection and parameterization of algorithms and feedback on proposals and decisions made by the system (Amershi et al., 2014). One example is the control of probabilistic models applied in recommender systems through textual tags associated with the items to be recommended (Loepp et al. 2019).
Future research activities on these topics should include the design of interactive methods for the control of algorithmic processes as well as the development of new algorithms that allow more far-reaching intervention possibilities and closer interaction between user and algorithm.
The old question of the division of tasks between humans and technical systems arises in a new way in the context of AI techniques, as increasingly complex mental tasks can be automated. From the point of view of human-computer interaction, AI systems should support users in their tasks in the best possible way and not pursue a pure substitution strategy. Their use raises fundamental questions regarding the agency and autonomy of users and requires new solutions regarding the interaction and cooperation between user and system. This requires research in areas such as mixed-initiative interactions, handover processes (e.g. automated driving) and mutual learning in collaborative work (e.g. robotics, cf. Lemaignan, 2017).
When machines take on cognitive tasks, a variety of questions arise that would also arise in a comparable form for human actors in the social context (Bostrom & Yudkowski 2011). A central problem here, too, is the inexplicability of AI-based decisions, such as in the granting of loans or in application procedures, which is critical both in ethical and legal terms. The demand for the explanation of algorithmically made decisions is also anchored in the European Data Protection Directive, at least in partial aspects, without the implementation having been sufficiently investigated and clarified so far. Algorithmic decisions can (intentionally or unintentionally) be biased or discriminatory for certain groups of persons (Zemel et al., 2013). Research on how such tendencies can be identified and avoided is still largely in its infancy. However, from both an ethical and a legal point of view, it is still widely open to which actors the responsibility and liability for algorithmic decisions can be attributed. Different disciplines are required to solve these problems, but human-computer interaction can make an important contribution to clarifying these questions and to their implementation in interactive systems.
The advent of new approaches such as Deep Learning has immensely enhanced the effectiveness of intelligent algorithms which have begun to pervade a wide range of application areas. A large fraction of these systems are interactive, in that they provide services and functions that are meant to support users in performing their tasks. The black-box nature of most of these techniques, however, prevents users from comprehending and controlling the system which may lead to dissatisfaction and mistrust in the system. So far, there are neither viable nor established methods for engineering AIbased interactive systems. A much closer cooperation between the different fields and an integration of their methods will be needed to overcome the current deficits of intelligent systems from the user's point of view.