Automation Space: Towards a Design Space for Everyday Automation

Abstract

No longer only experts are confronted with (semi-)automated systems, yet automation has founds its way into our everyday lifes in various forms and applications. In this paper, we introduce our ongoing work towards a design space for “Everyday Automation” to uncover the dimensions of respective approaches, identify research gaps and promising future applications as well as to allow for transferring experiences and knowledge between different types of automated systems. Based on a literature review, we derived first dimensions for such a dedicated design space, such as the domain, the task type, the type of user interaction, and the automation level. For a visual presentation of this “Automation Space”, we propose a so-called morphological box which might provide a suitable tool for overviewing the diverse manifestations of automation in everyday life and for supporting ideation of novel approaches.

-

________________________________________________________ Workshop proceedings Automation Experience across Domains In conjunction with CHI'20, April 26th, 2020, Honolulu, HI, USA Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). Website: http://everyday-automation.tech-experience.at

Author Keywords

Automation; design space; automation domain; automation level.

CCS Concepts

•Human-centered computing ! Human computer interaction (HCI);

Introduction

Whether a fully automatic vacuum cleaner in the living room or a self-sufficient service for municipal information and applications: automation appears in numerous forms in our everyday life and is constantly evolving. Everyday Automation [ 5 ] is a very broad and complex topic, which is particularly driven by recent advances in Artificial Intelligence (AI) and “smart” devices at affordable prices.

Everyday automation can be understood as a union of the definitions of automation and everyday life. It is a process in which individual functions or entire activities are transferred from humans to machines [ 15 ], and which focuses on a person in their immediate, everyday environment. The immediate everyday environment of a person is determined in particular by routines and habits, but also by mobility and social interactions, for example.

From a scientific perspective, a categorization of the numerous appearances of automation in our everyday life is relevant in order to provide a comprehensive overview and structure and to uncover possible gaps in research and promising future applications. By analyzing existing automation approaches for everyday tasks and by identifying potential variants, we strive to unfold the so-called “design space” of everyday automation. Design spaces have a long history in HCI research. Examples include the work by Buxton who introduced a taxonomy of input devices [ 2 ] and the work by Ballagas et al. [ 1 ] who presented a design space for using smartphones for ubiquitous input. Other examples include a design space for driver-based automotive user interfaces by Kern and Schmidt [ 12 ] and a design space for interactive public displays by Müller et al. [ 19 ].

In the following, we introduce our ongoing work on creating the Automation Space, a design space for Everyday Automation. We report on the method, preliminary core dimensions identified so far as well as a promising visualization approach.

Method

In order to determine this design space and the dimensions of Everyday Automation, we started to conduct a literature research. Used sources and search engines for scientific works include the ACM Digital Library, the IEEE Xplore Digital Library, the AIS eLibrary, as well as Research Gate and Google Scholar. At the center of this review are keywords and keyword combinations which were derived from contributions to last year’s CHI workshop on Everyday Automation Experience [ 5 ] and included “everyday automation”, “smart technology”, “smart devices”, “everyday interaction”, “digital assistants”, “home automation” and “smart city”. The search terms are expanded during search with newly acquired knowledge. The main inclusion criterion for a study was that the source must contain recent approaches and examples for everyday automation.

The analysis of the documents was done according to the Quantitative Content Analysis (cf. [ 16 ]). This method is considered to be particularly appropriate as the method aims at structuring certain themes and contents and filters out and summarizes aspects of the material.

Preliminary Dimensions

From this literature research, we identified and selected five preliminary core dimensions for a design space of Everyday Automation. In the following, we briefly introduce these dimensions and present corresponding examples from literature.

Presence of the System

Everyday Automation applications can be differentiated according to the presence of a physical system. Based on the analysis of the examples, a division of the applications into virtual and physical presence could be determined. For example, virtual systems are smart assistance systems in cars [ 11 ], digital representatives [ 17 ] or virtual reality indoor navigation systems [ 9 ]. Autonomous drones [ 18 ], fully-automated coffee makers [ 7 ] and automatic vacuum cleaner robots [ 14 ] are examples of physical systems, i.e. physical representations of the automated object.

Domain

The following application domains of Everyday Automation were identified from the literature examples: Education, Health and Sports, Shopping and Restaurant, Transportation, Home, Security and Government. Frequently, several different areas of application are mentioned for the same example. For instance, food recognition of smart refrigerators (for automating ordering processes, e.g.) can be used at home, but also in restaurants (e.g., [ 6 ]). Furthermore, interaction with displays based on eye movement can serve as a public information display in the museum, but can also be used as a game for waiting areas in the hospital (e.g., [ 24 ]).

Automated Task

Based on the analysis of the Everyday Automation examples, it was found that key words identified for the automated task are covered by the dimensions suggested by Parasuraman et al. [ 21 ]: Information acquisition, information analysis, decision and action selection and action implementation. While information acquisition describes purely sensory functions for capturing data from the environment, information analysis deals with processing the captured data. The decision or action selection deals with the derivation of further action steps and the action implementation includes the actual execution of an action selection and usually replaces the hand or voice of a person. For example, an autonomous delivery droid (e.g., [ 10 ]) takes over the complete delivery of orders, while a sports wearable (e.g., [ 8 ]) only signals flow state feedback and recommendations for further activity to the wearer.

User Interaction

Six different user interactions for Everyday Automation applications were identified: stationary or mobile external device, hardware buttons, touch interface, hand gestures, voice interface and eye gestures. Stationary or mobile external devices include in particular computers, tablets or smartphones, as well as hardware controllers, cameras, and wearables. Examples of the various interaction modalities include automated passport control in a stationary sluice (e.g., [ 4 ]), autonomous drone control by hand gestures (e.g., [ 18 ]), and automatic language translation via a voice interface (e.g., [ 20 ]).

Automation Level

According to Parasuraman et al. [ 21 ], the degree of automation is divided into three areas: fully manual, semiautomated and fully automated. Where manual means that a task is carried out exclusively by humans and is therefore only listed for the sake of completeness. Semiautomated means that a task is carried out by combining the advantages of human skills with the advantages of the machine [ 13 ]. In an AR-based system helping patients to test their blood at home, a combination of human action and machine support takes place (e.g., [ 3 ]). Fully automatic means that a task is completed completely and exclusively by the machine. At a sans-checkout grocery store such as Amazon Go, the scanning of the items and the payment process are carried out completely automatically (e.g., [ 22 ]).

Visualization

Everyday Automation covers a diverse and complex range of applications. Therefore it is not trivial to find an appropriate and suitable form for visualizing the corresponding design space. We propose a representation of the design space based on the concept of the morphological box, which has its origin in the creativity techniques. This form of visualization is based on the division of a subject into its elementary components, whereby the dimensions for each component are determined and a combination of the elements is ultimately displayed [ 23 ]. The aspects mentioned above reflect parallels and central elements of a design space. This form of visualization is therefore considered to be particularly suitable for compactly visualizing a design space with many dimensions and manifestations. The red line symbolizes an automatic vacuum cleaner robot [ 14 ], the green line an AR-based system helping patients to test their blood at home [ 3 ] and the blue line a virtual reality indoor navigation system [ 9 ]. Each additional path through the dimensions might inspire a novel Everyday Automation application.

Conclusion and Outlook

In this paper, we presented our ongoing work on creating a design space for “Everyday Automation”. From a literature review, we identified first core dimensions: presence of the system, domain, automated task, user interaction, and automation level. For visualizing these dimensions and the various existing specifications, we proposed a morphological box. This approach provides a compact overview of manifestations and particularly supports the ideation of novel applications.

In future work, we will complete this first version of the Automation Space by further dimensions. Additionally, we plan to evaluate complementary alternative visualization approaches beyond the currently used morphological box.

References

[1] R. Ballagas , J. Borchers , M. Rohs , and J. G. Sheridan . 2006 . The smart phone: a ubiquitous input device . IEEE Pervasive Computing 5 , 1 ( 2006 ), 70 - 77 .

[2] William Buxton . 1983 . Lexical and Pragmatic Considerations of Input Structures . SIGGRAPH Comput. Graph . 17 , 1 (Jan. 1983 ), 31 - 37 . DOI: http://dx.doi.org/10.1145/988584.988586

[3] Tom Djajadiningrat , Pei-Yin Chao , SeYoung Kim, Marleen Van Leengoed, and Jeroen Raijmakers . 2016 . Mime: An AR-based System Helping Patients to Test their Blood at Home . In Proceedings of the 2016 ACM Conference on Designing Interactive Systems . 347 - 359 .

[4] Abdulsalam Dukyil , Ahmed Mohammed, and Mohamed Darwish . 2016 . An optimization approach for a RFID-enabled passport tracking system . In Proceedings of the 4th International Conference on Control, Mechatronics and Automation . 189 - 194 .

[5] Peter Fröhlich , Matthias Baldauf, Thomas Meneweger, Ingrid Erickson, Manfred Tscheligi, Thomas Gable, Boris de Ruyter, and Fabio Paternò . 2019 . Everyday Automation Experience: Non-Expert Users Encountering Ubiquitous Automated Systems . In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19) . Association for Computing Machinery , New York, NY, USA, Article Paper W25 , 8 pages . DOI: http://dx.doi.org/10.1145/3290607.3299013

[6] Xiaoyan Gao , Xiangqian Ding, Ruichun Hou, and Ye Tao . 2019 . Research on Food Recognition of Smart Refrigerator Based on SSD Target Detection Algorithm . In Proceedings of the 2019 International Conference on Artificial Intelligence and Computer Science . 303 - 308 .

[7] Marc Hassenzahl and Holger Klapperich . 2014 . Convenient, clean, and efficient?: the experiential costs of everyday automation . In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun , Fast, Foundational. ACM, 21 - 30 .

[8] Hayati Havlucu , Terry Eskenazi, Baris¸ Akgün, Mehmet Cengiz Onbas¸lı, Aykut Cos¸kun, and Og˘uzhan Özcan. 2018 . Flow state feedback through sports wearables: A case study on tennis . In Proceedings of the 2018 Designing Interactive Systems Conference . 1025 - 1039 .

[9] Darlis Herumurti , Anny Yuniarti, Imam Kuswardayan, Wijayanti Nurul Khotimah, and Wahyu Widyananda . 2017 . Virtual reality navigation system in virtual mall environment . In Proceedings of the 3rd International Conference on Communication and Information Processing . 209 - 213 .

[10] Elin Janebäck and Matilda Kristiansson . 2019 . Friendly robot delivery: Designing an autonomous delivery droid for collaborative consumption . Master's thesis.

[11] Katharina Keller , Kim Valerie Carl, Hendrik Jöntgen, Benjamin M Abdel-Karim , Max Mühlhäuser , and Oliver Hinz . 2019 . " KITT, where are you?" why smart assistance systems in cars enrich people's lives . In Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers . 1120 - 1132 .

[12] Dagmar Kern and Albrecht Schmidt . 2009 . Design Space for Driver-Based Automotive User Interfaces . In Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '09) . Association for Computing Machinery , New York, NY, USA, 3 - 10 . DOI:http://dx.doi.org/10.1145/1620509.1620511

[13] Marcel Langer and Dirk Söffker . 2011 . Human guidance and supervision of a manufacturing system for semi-automated production . In 2011 IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT) . IEEE , 1- 6 .

[14] Hyunsoo Lee and Amarnath Banerjee . 2015 . Intelligent scheduling and motion control for household vacuum cleaning robot system using simulation based optimization . In 2015 Winter Simulation Conference (WSC) . IEEE, 1163 - 1171 .

[15] Dietrich Manzey . 2012 . Systemgestaltung und Automatisierung. Springer Berlin Heidelberg, Berlin, Heidelberg, 333 - 352 . DOI: http://dx.doi.org/10.1007/978-3- 642 -19886-1_ 19

[16] Philipp Mayring and Thomas Fenzl . 2014 . Qualitative Inhaltsanalyse. Springer Fachmedien Wiesbaden, Wiesbaden, 543 - 556 . DOI: http://dx.doi.org/10.1007/978-3- 531 -18939-0_ 38

[17] Hila Mehr , H Ash , and D Fellow . 2017 . Artificial intelligence for citizen services and government . Ash Cent. Democr. Gov. Innov. Harvard Kennedy Sch ., no. August ( 2017 ), 1 - 12 .

[18] Silvia Mirri , Catia Prandi, and Paola Salomoni . 2019 . Human-Drone Interaction: state of the art, open issues and challenges . In Proceedings of the ACM SIGCOMM 2019 Workshop on Mobile AirGround Edge

[19] Jörg Müller , Florian Alt, Daniel Michelis, and Albrecht Schmidt . 2010 . Requirements and Design Space for Interactive Public Displays . In Proceedings of the 18th ACM International Conference on Multimedia (MM '10) . Association for Computing Machinery , New York, NY, USA, 1285 - 1294 . DOI: http://dx.doi.org/10.1145/1873951.1874203

[20] Hermann Ney . 2001 . Stochastic modelling: from pattern classification to language translation . In Proceedings of the workshop on Data-driven methods in machine translation-Volume 14. Association for Computational Linguistics , 1 - 5 .

[21] Raja Parasuraman , Thomas B Sheridan, and Christopher D Wickens . 2000 . A model for types and levels of human interaction with automation . IEEE Transactions on systems, man, and cybernetics-Part A: Systems and Humans 30 , 3 ( 2000 ), 286 - 297 .

[22] Alex Polacco and Kayla Backes . 2018 . The amazon go concept: Implications, applications, and sustainability . Journal of Business and Management 24 , 1 ( 2018 ), 79 - 92 .

[23] Christian Schawel and Fabian Billing . 2018 . Morphologischer Kasten. Springer Fachmedien Wiesbaden, Wiesbaden, 219 - 221 . DOI: http://dx.doi.org/10.1007/978-3- 658 -18917-4_ 57

[24] Mélodie Vidal , Andreas Bulling, and Hans Gellersen . 2013 . Pursuits: spontaneous interaction with displays based on smooth pursuit eye movement and moving targets . In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing . 439 - 448 .