Abstract

Computer-based interactive systems increasingly shape all areas of life. Scalability with respect to human computer interaction (HCI) is an umbrella term under which various developments in this regard are discussed (e.g., growing numbers of users, variety of devices). However, scalability relates not only to user interface and interaction design but to automation. Whether and how automation scales or can be scaled regarding HCI has hardly been addressed by previous research. By introducing a one-day user journey focusing on automation experience, we discuss characteristics and research directions for safe and satisfying scaled automation experiences.

1. Introduction

Computer-based interactive systems increasingly shape

all areas of life, and many people’s everyday lives [ 1, 2 ].

The associated challenges to designing interactive sys

tems and interaction concepts, such as using applications ditions, are discussed in human computer interaction (HCI) research under the umbrella term scalability [ 3, 4 ].

This discussion is still relatively young compared to scal

ing debates in other computer science disciplines, e.g., distributed systems or database management [ 5, 6 ].

However, scalability does not only mean that users must master diferent devices and interaction concepts.

They will also increasingly come into contact with automation solutions at home, at work, and on the road [7, 8]. Some even argue that ”automation comes to dominate interaction” [9].

However, whether and how automation scales or can be scaled regarding HCI has hardly been addressed by previous research. This appears necessary because the interaction between users and single automation solutions is already characterized by numerous challenges (e.g., mode confusion, loss of competence) [ 10 ]. How these phenomena develop in connection with the experience of diferent automation solutions, or to what extent others occur, needs to be explored. It is at least conceivable that the mode confusion problem occurs not only with respect to a specific level within one automation scheme but that users are confused by diferent automation concepts. The scale of devices can be viewed from diferent perspectives: the number of devices from single-device to multiple devices support (e.g., gestural text entry on multiple devices [ 14 ], multimodal information access across multiple devices [ 15 ]) and the size and complexity of devices as shown in Figure 2.

Streitz and Wichert [ 16 ] describe the dimension of contexts also as spaces and places ”where the interaction happens, where services are provided or being used and their operating range in terms of proximity, being close or distant.” The contexts range from private to public spaces such as the bathroom, the ofice to the smart home, factory, market place, train station, neighbourhood and entire countries [ 16 ] (see Figure 3).

In addition, scalability was also associated with visualizations [17], safety-critical human computer interaction [18], design patterns for multi-platform user interfaces [19], moving from smartphone to tablet applications [19], small- and large-scale interactions (finger, body) [ 20], and scope, duration and realism of evaluation of interactive systems [ 3 ].

With respect to scaling and automation, only a few HCI-related works have been identified by the authors, e.g., scalability of formal verification analyses with the aid of task-analytic models [21] or task planning [22]. In addition, automation concepts with diferent stages/levels [23] can also be understood in terms of scalability. However, as illustrated below, they do not address the individual user perspective of scaling automation experiences.

3. An One-Day Automation

Experience Journey

In Figure 4, a fictitious, but in principle indeed not ab

surd, working day is depicted from the morning routine in one’s own smart home to the end of the day on a business trip in a hotel. Note that the adjective ”smart” should also represent concepts like pervasive computing environments [24] or industry 4.0/50 [25].

As can be understood, the user encounters several au- • Users will make automation experiences in pritomation solutions on an ordinary day in various private vate and public settings. With regard to technical and public situations. mobile and wearable devices are in- solutions, the definition of private and public is tegrated in the technical infrastructures diferently. The not bound to physical spatial boundaries or other user and other family members can monitor and control directly perceptible characteristics. lighting and heating at home with their private smartphones [26]. On the way to work, navigation system of The previous example does not describe the actual dythe ebike receives information about closed bike paths, namics and diversity of a person’s automation experience: construction sites and accident blackspots and adjusts Daily routines are diferent, software and hardware comthe route accordingly [27]. Although direct use of the ponents may be faulty or updated, and user needs and private smartphone for work purposes is not permitted, capabilities may vary. These circumstances complicate the user also receives status messages from his home matters more than they facilitate them. However, the during this period and can respond remotely if necessary. validity of the aforementioned characteristics remains At work, sensor data of the private smartwatch are used essentially unafected. to detect stress situations and locations [28]. While the user is familiar with these automation solutions, there is 4.2. Research Questions initial contact with those in the highly-automated rental car and the hotel room equipment later that day [29]. The awareness that a seamless automation experience

In this context, it can be assumed that the automation does not necessarily have to grow out of the experience solutions at diferent ”stages” of the journey also come of individually matching automation solutions strung from diferent manufacturers, at least in part. Further- together raises questions for further research: more, it can be assumed at this point that the organizational and legal framework conditions, which can also impact the respective automation experience (e.g., occupational health and safety, data protection), are adhered to and resolved in the users’ interests. These aspects will not be discussed further below.

4. HCI-related Automation Scaling

Subsequently, characteristics and research questions for scaling automation from an HCI perspective are described.

4.1. Characteristics From the previous ”automation experience journey” in private and professional life, characteristic features can be derived that are relevant beyond this specific example. • Users will encounter several automation concepts in everyday life, which will difer in number and meaning of stages/levels. • Users’ mobile and wearable devices will be integrated diferently in the automation solutions, from not at all to data suppliers (e.g., sensor data) to comprehensive interaction components (e.g., remote controls, dashboards). In addition, other stationary and mobile devices will be used on a context-specific basis. • Users will make automation experiences individually, in teams and in larger groups of people. In the process, it will not always be clear to them which actors are still involved. • How can automation concepts be explained in a way that is appropriate to the user and the situation, assuming that users experience multiple forms of automation? • How can existing automation concepts be systematically transferred to new contexts, thus avoiding unnecessary new developments? • How can diferent automation concepts be compared with respect to diferent socio-technical settings, e.g., private home environment, public workspace? • How can best practices and lessons learned regarding scaling automation with respect to users, devices, and contexts be documented and shared? • What could be learned from scalability-related research in others fields of research and practice, e.g., software engineering or database systems? Parts of these questions have already been addressed in previous research, be it on the question of automation in collaborative teamwork (scaling the number of users) [30] or application of existing automation concepts in domains diferent from the one that the concept evolved from (scaling contexts of use) [31]. Design patterns and pattern languages have been developed with respect to scalability in general [32] and human-computer interaction in particular [33]. For scalable human-centered automation, such an approach is missing so far. The following scaling issue has not been addressed: There is always more than one automation for users in their life.

5. Conclusion

Questions about the scalability of automation approaches have been discussed primarily concerning technical aspects, e.g., resource requirements, and with respect to number of users, variety of devices, and diferent contexts of use. As can be seen here based on an exemplary user journey, challenges related to individual and scaled automation experiences will also have to be mastered to an increasing extent in the future. They concern the safe switching between diferent automation concepts, the use of mobile and wearable devices in diferent automation contexts, and the use of automation solutions with varying numbers of co-users.

Therefore, an answer to the question posed at the beginning, ”Does automation scale?” can be as follows.

First, it must, since users increasingly have to deal with diferent automation solutions. Second, if research and development is carried out with this awareness and on the research questions mentioned, it can. road-mapping research in ambient computing and sensors of commercial smartwatch, in: Adjunct communication environments, Towards the Human Proceedings of the 2019 ACM International Joint City: White Paper on a Future Research Agenda Conference on Pervasive and Ubiquitous Comput(2009). ing and Proceedings of the 2019 ACM Interna[17] G. Richer, A. Pister, M. Abdelaal, J.-D. Fekete, tional Symposium on Wearable Computers, 2019, M. Sedlmair, D. Weiskopf, Scalability in visual- pp. 1198–1201. ization, IEEE Transactions on Visualization and [29] H. Yang, H. Song, C. Cheung, J. Guan, How to enComputer Graphics (2022). hance hotel guests’ acceptance and experience of [18] N. Flegel, K. Van Laerhoven, T. Mentler, Scalable smart hotel technology: An examination of visithuman computer interaction in control rooms as ing intentions, International Journal of Hospitality pervasive computing environments, in: Proceed- Management 97 (2021) 103000. ings of the 33rd European Conference on Cognitive [30] M. C. Wright, D. B. Kaber, Efects of automation Ergonomics, 2022, pp. 1–5. of information-processing functions on teamwork, [19] S. R. Humayoun, S. Hess, F. Kiefer, A. Ebert, Pat- Human Factors 47 (2005) 50–66. terns for designing scalable mobile app user inter- [31] S. FakhrHosseini, C. Lee, J. F. Coughlin, Home as a faces for multiple platforms, in: Proceedings of the platform: levels of automation for connected home 28th International BCS Human Computer Interac- services, in: Human Aspects of IT for the Aged tion Conference (HCI 2014) 28, 2014, pp. 317–322. Population. Healthy and Active Aging: 6th Inter[20] A. Jaimes, N. Sebe, Multimodal human–computer national Conference, ITAP 2020, Held as Part of interaction: A survey, Computer vision and image the 22nd HCI International Conference, HCII 2020, understanding 108 (2007) 116–134. Copenhagen, Denmark, July 19–24, 2020, Proceed[21] M. L. Bolton, X. Zheng, K. Molinaro, A. Houser, ings, Part II 22, Springer, 2020, pp. 451–462.

M. Li, Improving the scalability of formal human– [32] K. S. Ahluwalia, Scalability design patterns, in: automation interaction verification analyses that Proceedings of the 14th Conference on Pattern Lanuse task-analytic models, Innovations in Systems guages of Programs, 2007, pp. 1–8. and Software Engineering 13 (2017) 1–17. [33] N. Flegel, J. Poehler, K. Van Laerhoven, T. Mentler, [22] S. Grover, S. Sengupta, T. Chakraborti, A. P. Mishra, Towards control rooms as human-centered pervaS. Kambhampati, Radar: automated task planning sive computing environments, in: Sense, Feel, Defor proactive decision support, Human–Computer sign: INTERACT 2021 IFIP TC 13 Workshops, Bari, Interaction 35 (2020) 387–412. Italy, August 30–September 3, 2021, Revised Se[23] C. D. Wickens, H. Li, A. Santamaria, A. Sebok, N. B. lected Papers, Springer, 2022, pp. 329–344.

Sarter, Stages and levels of automation: An integrated meta-analysis, in: Proceedings of the human factors and ergonomics society annual meeting, 4, Sage Publications Sage CA: Los Angeles, CA, 2010, pp. 389–393. [24] R. Jabla, M. Khemaja, F. Buendía, S. Faiz, A novel component of decision-making for context-aware applications in pervasive environments, in: Ambient Intelligence–Software and Applications–12th International Symposium on Ambient Intelligence,

Springer, 2022, pp. 127–137. [25] X. Xu, Y. Lu, B. Vogel-Heuser, L. Wang, Industry 4.0 and industry 5.0—inception, conception and perception, Journal of Manufacturing Systems 61 (2021) 530–535. [26] A. Nacer, B. Marhic, L. Delahoche, Smart home, smart hems, smart heating: An overview of the latest products and trends, in: 2017 6th International Conference on Systems and Control (ICSC), IEEE, 2017, pp. 90–95. [27] D. Namiot, M. Sneps-Sneppe, On bikes in smart cities, Automatic Control and Computer Sciences 53 (2019) 63–71. [28] P. Siirtola, Continuous stress detection using the

References

[1] M. Koch , F. Alt , Allgegenwärtige menschcomputer-interaktion, Informatik-Spektrum 40 ( 2017 ) 147 - 152 .

[2] Y. Rogers , The changing face of human-computer interaction in the age of ubiquitous computing, in: HCI and Usability for e-Inclusion: 5th Symposium of the Workgroup Human-Computer Interaction and Usability Engineering of the Austrian Computer Society , USAB 2009, Linz, Austria, November 9- 10 , 2009 Proceedings 5, Springer, 2009 , pp. 1 - 19 .

[3] L. Mamykina , A. M. Smaldone , S. R. Bakken , N. Elhadad , E. G. Mitchell, P. M. Desai , M. E. Levine , J. N. Tobin , A. Cassells , P. G. Davidson , et al., Scaling up hci research: from clinical trials to deployment in the wild ., in: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems , 2021 , pp. 1 - 6 .

[4] B. Brown , S. Bødker, K. Höök , Does hci scale? scale hacking and the relevance of hci , Interactions 24 ( 2017 ) 28 - 33 . URL: https://doi.org/10.1145/3125387. doi: 10 .1145/3125387.

[5] B. W. Boehm , R. K. Mcclean , D. Urfrig , Some experience with automated aids to the design of largescale reliable software , in: Proceedings of the international conference on Reliable software , 1975 , pp. 105 - 113 .

[6] R. H. Halstead Jr , S. A. Ward , The munet: A scalable decentralized architecture for parallel computation , in: Proceedings of the 7th annual symposium on Computer Architecture , 1980 , pp. 139 - 145 .

[7] T. B. Sheridan , R. Parasuraman , Human-automation interaction, Reviews of human factors and ergonomics 1 ( 2005 ) 89 - 129 .

[8] M. Baldauf , P. Fröhlich , S. Sadeghian , P. Palanque , V. Roto , W. Ju , L. Baillie , M. Tscheligi , Automation experience at the workplace , in: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems , 2021 , pp. 1 - 6 .

[9] G. Cockton , S. Barbosa, The good, the bad, and the techy (scaled as appropriate ), Interactions 24 ( 2017 ) 5 . URL: https://doi.org/10.1145/3131784. doi: 10 .1145/3131784.

[10] J. Bredereke , A. Lankenau , A rigorous view of mode confusion , in: Computer Safety, Reliability and Security: 21st International Conference, SAFECOMP 2002 Catania, Italy, September 10-13 , 2002 Proceedings 21, Springer, 2002 , pp. 19 - 31 .

[11] K. R. Fleischmann , Sociotechnical interaction and cyborg-cyborg interaction: Transforming the scale and convergence of hci , The Information Society 25 ( 2009 ) 227 - 235 .

[12] J. Himmelsbach , S. Schwarz , C. Gerdenitsch , B. Wais-Zechmann , J. Bobeth , M. Tscheligi , Do we care about diversity in human computer interaction: A comprehensive content analysis on diversity dimensions in research , in: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI '19 , Association for Computing Machinery, New York, NY, USA, 2019 , p. 1 - 16 . URL: https://doi.org/10.1145/3290605.3300720. doi: 10 .1145/3290605.3300720.

[13] T. Alencar , M. Barbosa , L. Machado , L. Neris , V. Neris , Considering the diversity of users in the development of a flexible bus stop , in: Proceedings of the 17th Brazilian Symposium on Human Factors in Computing Systems, IHC 2018 , Association for Computing Machinery , New York, NY, USA, 2018 , pp. 1 - 10 . URL: https://doi.org/10.1145/ 3274192.3274205. doi: 10 .1145/3274192.3274205.

[14] J. O. Wobbrock , B. A. Myers , Gestural text entry on multiple devices , in: Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility , Assets ' 05 , Association for Computing Machinery, New York, NY, USA, 2005 , p. 184 - 185 . URL: https://doi.org/10.1145/1090785. 1090821. doi: 10 .1145/1090785.1090821.

[15] G. Huebsch , K. Kadner , Multimodal information access across multiple devices , in: Proceedings of the 4th International Conference on Mobile Technology, Applications, and Systems and the 1st International Symposium on Computer Human Interaction in Mobile Technology, Mobility '07 , Association for Computing Machinery, New York, NY, USA, 2007 , p. 736 - 742 . URL: https://doi.org/10.1145/1378063. 1378188. doi: 10 .1145/1378063.1378188.

[16] N. A. Streitz , R. Wichert , Deliverable number d4. 2