Personalized IoT’s service providers: A neurocognitive approach to assess their usability Andrea Antonio Cantone1,∗ , Alireza Mortezapour1 , Monica Sebillo1 , Genoveffa Tortora1 and Giuliana Vitiello1 1 Department of Computer Science, University of Salerno, Italy Abstract Despite the efforts of specialists to provide an IoT product, due to unforeseen challenges, these environ- ments require engagement from the end users. In the simplest case, for this personalization, they use simple recipes like if-then. Some of the most important providers of this service include IFTTT, Zapier, and Microsoft flow. Despite conducting studies with questionnaire methods to compare the usability of the systems related to these service providers, no study has yet objectively investigated their usability. The purpose of this study will be to use the fNIRS technique to assess the objective usability of the mobile systems of these service providers. Keywords End User Programming, If-then rule, fNIRS, IoT 1. Introduction The Internet of Things (IoT) connects the digital and physical worlds, making the world around us smarter and more responsive [1]. Other associated terms have been created, such as Internet of Everything (IoE), Internet of Vehicles (IoV), and Internet of People (IoP). All these have as their main goal to make humans smarter and easier to live with; in other words, empower people [2]. In general, due to the number of objects connected to each other on the one hand and the variety of people using the systems and their expectations on the other hand, it is not possible to consider all the problems and challenges of the end users when introducing a new product [3]. Expert developers cannot anticipate all the probable circumstances end users may experience while engaging with their IoT environment, meanwhile, software development cycles are still too sluggish to respond to user demands [4]. However, end users can influence IoT systems that depend on them through simple actions and commands. The entry of the end user into the cycle of providing simple commands to IoT systems is also called experience personalization [5]. Despite the numerous advantages of IoT-based technologies, the fundamental problem re- mains to tailor their behavior to meet the highly contextualized, unique, and frequently changing demands of users. One of the methods proposed in the literature to overcome this problem is IS-EUD 2023: 9th International Symposium on End-User Development, 6-8 June 2023, Cagliari, Italy ∗ Corresponding author. Orcid 0009-0001-0270-5106 (A. A. Cantone); 0000-0003-3731-6415 (M. Sebillo); 0000-0001-7130-996X (G. Vitiello) © 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings http://ceur-ws.org ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org) simple if-then commands. In this case, when a trigger occurs, a specific action that has been given to the system occurs [6]. These commands, which are called Trigger-Action rules, have been investigated in different contexts such as personalizing humanoid robot behaviors and smart homes [7][8]. It’s possible to access and use such services from different providers such as IFTTT, Zapier, and Microsoft flow [9]. The usability of these software products is critical. The cognitive approach is a method of evaluating the usability of systems that focuses on understanding users’ mental processes while using the system. This approach is based on the concept that users interact with the system through their cognitive processes, such as perception, attention, memory, and reasoning, and that understanding these processes can help design a more usable system [10][11]. To use the cognitive approach for assessing the usability of a system, one needs to identify the users’ goals and the activities performed when using the system. Next, the cognitive processes involved in these activities must be analyzed to determine whether the system supports or hinders these processes. There are several tools and techniques used in the cognitive approach to assess the usability of systems. In this work, we propose the use of fNIRS (functional near infrared spectroscopy) for the objective evaluation of User eXperience and usability of these services. 2. Background The providers IFTTT, Zapier, and Microsoft Flow have been compared from different points of view [12][13]. In recent years, the User eXperience and usability of these services have gained attention [14]. Schrepp et al. surveyed the usability of three web automation service providers with the participation of 82 users. They did a real scenario with these products. The results showed that IFTTT had the highest scores in all 6 dimensions of the User Experience Questionnaire (UEQ) [15]. Most studies use subjective methods to analyze the usability of these IoT-based products. Objective analysis of a product’s usability is a scientific field developed in the last 20 years. In the early stages of using neuroscience devices to study User eXperience and usability, we can see the importance of electroencephalogram (EEG) [16][17]. With the advent of new devices such as fNIRS, the use of this new neurophysiological approach to study user experience and usability is proposed [18][19]. In a recent study which integrates Augmented Reality into IoT, researchers use both subjective and objective measures to study the usability. Following a neuroscientific approach, they used an fNIRS device. The scenario was about two information search modes, an AR-based, and web-based. The results showed that information search using AR was more efficient and had a lower cognitive load than information search with a website. The findings of the usability testing show that AR, as an emerging retail technology, may significantly enhance consumer experiences and raise buy intent [20]. In a recent study, authors tried to capture objective data about the User eXperience of gamers and non-gamers based on fNIRS [21]. The aim of this study was about quantifying users’ cognitive parameters about the game difficulty level. Three brain regions that have the responsibility for perceptual UX are selected for data gathering. 40 university students participated in this study and a multi-channel near-infrared system was utilized to record the brain activity. The 32-channel fNIRS device had a sampling frequency of 10 Hz and its wavelengths are 760 and 850 nm (21 light sources and 20 detectors). The authors concluded that a combination of the flow state scale (subjective questionnaire) and fNIRS is a feasible approach to assess players’ UX. Considering the importance of objective studies to survey the usability of interactive products and the lack of a published study on this matter in these IoT-based services, we propose an objective technique to study the usability of these services. Based on the results obtained from this study, we can have a better judgment of end users’ relationship with these technologies. 3. Experiment design In this section, we describe the experimental design of the study. 3.1. Participants Forty participants (M=20, F=20) will be considered. They will be homogeneously selected based on several confounding parameters such as education level, programming experience and eye health status. The reason for this homogeneous selection is that these confounding variables do not distort (bias) participants’ perception of the final variable, so the usability of the products. People participating in the study will fill out the informed consent form along with the initial explanation. 3.2. Material For comparing the usability of these three software products, fNIRS will be used. The use of this technique in usability studies does not have much history and only a few articles have been published in recent years [22][23]. By using the fNIRS device we can observe the hemodynamic response, which refers to changes in blood flow and oxygenation levels in response to neural activity. During cognitive processes, such as problem-solving, attention, or memory tasks, there is an increased demand for oxygenated blood in specific regions of the brain. fNIRS can detect these changes by measuring the concentrations of oxygenated and deoxygenated hemoglobin. Typically, an increase in oxygenated hemoglobin and a decrease in deoxygenated hemoglobin are observed in brain areas involved in the cognitive task. Using an fNIRS device, during a cognitive task, an increase in oxygenated hemoglobin and a decrease in deoxygenated hemoglobin would indicate increased neural activity and oxygen consumption in the corresponding brain regions. Therefore, during software use, we expect to observe changes in the levels of oxygenated and deoxygenated hemoglobin, and we want to quantify such changes in terms of the proportion of deoxyhemoglobin/oxyhemoglobin. This technique will be used to conduct a comparative experimental study on the use of the three software products IFTTT, Zapier, and Microsoft Flow. The device used in this study will have 64 channels. To collect data, the 10-20 international system technique will be used [24]. Considering that usability/UX perception’s region of interest (ROI) is related to the frontal pole area (FPA), dorsolateral prefrontal cortex (DLPFC), and ventral lateral prefrontal cortex (VLPFC), additional data will be removed during preprocessing [22]. Through changes in the amount of hemoglobin without oxygen compared to hemoglobin with oxygen, the perception of participants with respect to those three software products will be calculated. 3.3. Tasks Due to the importance of some crucial aspects in objective usability such as efficiency, effective- ness, and learnability, we develop a scenario that can yield the results of these aspects. Our scenario is based on personalizing a simple IoT system. In our scenario, the installed version of three applications related to the three service providers is presented to participants. They will encounter a simple tangible IoT system. This system has an intelligent light, a Twitter/Face- book account, an intelligent voice command device (Google Nest), and the software which is mentioned before. The participants should do some simple tasks about opening the software and provide some simple rules in the software to connect the light to the Twitter and Facebook accounts. Then they should command by voice to the IoT system and do some predefined tasks. 3.4. Procedure First user will be informed about the required procedure. Then they will use the system based on a predefined scenario. The scenario is constant in all participants. For each software, they have the first 5 minutes for familiarization. Then they will use each software for 15 minutes. All of these applications will be installed from Google Play. Also, all of the participants will use the software in a Google Pixel 7 pro. During the total procedures of using the software and the IoT system, they will be brain scanned by fNIRS. People will have enough rest between using different software. At the end of using each software, System Usability Scale (SUS) and User Experience Questionnaire (UEQ) questionnaires will be filled for each one. 4. Conclusions In this work, we have focused on the importance of a cognitive approach to objective usability assessment. With this methodology, we expect (i) that the understanding and evaluation of the ease of use, efficiency, and satisfaction of using the software may vary based on gender. (ii) A positive and significant correlation will be observed between the results of subjective usability (questionnaires) and the results of cognitive parameters (based on brain activity), that is, the results obtained from subjective measures of usability and objective measures of cognitive parameters, such as brain activity, are related to each other. For example, if a software program is perceived as easy to use by the user, this may be reflected in increased activity in certain areas of the brain associated with attention, memory, and problem-solving; (iii) a significant difference between the usability of these software products will be observed. Some software may be more user-friendly, efficient, and effective than others. This difference in usability may arise due to factors such as the design of the software, the features and functionalities offered, and the user’s prior experience and knowledge of the software. Compared with the other devices for physiological and behavioral measures, such as fMRI and EEG, that can also provide accurate and ”objective” assessments, fNIRS can be more effectively used in real human-technology interaction scenario. Indeed, the fNIRS is more portable (in comparison to fMRI) and less sensitive to any excessive noise (in comparison to EEG). References [1] K. Sorri, N. Mustafee, M. Seppänen, Revisiting iot definitions: A framework towards comprehensive use, Technological Forecasting and Social Change 179 (2022) 121623. doi:h t t p s : / / d o i . o r g / 1 0 . 1 0 1 6 / j . t e c h f o r e . 2 0 2 2 . 1 2 1 6 2 3 . [2] P. Whig, K. Gupta, N. Jiwani, A. Velu, Ambient Intelligence Health Services Using IoT, John Wiley Sons, Ltd, 2022, pp. 53–79. doi:h t t p s : / / d o i . o r g / 1 0 . 1 0 0 2 / 9 7 8 1 1 1 9 8 2 1 8 4 7 . c h 3 . [3] F. Balducci, B. Breve, F. Cena, A. Mattioli, M. Rizvi, Empathy: 3rd international workshop on empowering people in dealing with internet of things ecosystems, in: Proceedings of the 2022 International Conference on Advanced Visual Interfaces, AVI 2022, Association for Computing Machinery, New York, NY, USA, 2022. doi:1 0 . 1 1 4 5 / 3 5 3 1 0 7 3 . 3 5 3 5 2 5 1 . [4] F. Corno, L. D. Russis, A. M. Roffarello, How do end-users program the internet of things?, Behaviour & Information Technology 41 (2022) 1865–1887. doi:1 0 . 1 0 8 0 / 0 1 4 4 9 2 9 X . 2 0 2 2 . 2071169. [5] F. Paternò, End-user creation and control of daily automations in intelligent environments, in: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems, CHI EA ’22, Association for Computing Machinery, New York, NY, USA, 2022. doi:1 0 . 1 1 4 5 / 3 4 9 1 1 0 1 . 3 5 0 3 7 5 2 . [6] K. Dong, Y. Zhang, Y. Zhao, D. Li, Z. Ling, W. Wu, X. Zhu, Real-time execution of trigger- action connection for home internet-of-things, in: IEEE INFOCOM 2022 - IEEE Confer- ence on Computer Communications, 2022, pp. 1489–1498. doi:1 0 . 1 1 0 9 / I N F O C O M 4 8 8 8 0 . 2 0 2 2 . 9796872. [7] N. Leonardi, M. Manca, F. Paternò, C. Santoro, Trigger-action programming for personal- ising humanoid robot behaviour, in: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI ’19, Association for Computing Machinery, New York, NY, USA, 2019, p. 1–13. doi:1 0 . 1 1 4 5 / 3 2 9 0 6 0 5 . 3 3 0 0 6 7 5 . [8] L. Yao, Q. Z. Sheng, B. Benatallah, S. Dustdar, X. Wang, A. Shemshadi, S. S. Kanhere, Wits: An iot-endowed computational framework for activity recognition in personalized smart homes, Computing 100 (2018) 369–385. doi:1 0 . 1 0 0 7 / s 0 0 6 0 7 - 0 1 8 - 0 6 0 3 - z . [9] I. Bastys, M. Balliu, A. Sabelfeld, If this then what? controlling flows in iot apps, in: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS ’18, Association for Computing Machinery, New York, NY, USA, 2018, p. 1102–1119. doi:1 0 . 1 1 4 5 / 3 2 4 3 7 3 4 . 3 2 4 3 8 4 1 . [10] C. Silva, J. Vieira, J. Campos, R. Couto, A. Ribeiro, Development and validation of a descriptive cognitive model for predicting usability issues in a low-code development platform, Human Factors: The Journal of the Human Factors and Ergonomics Society 63 (2020) 001872082092042. doi:1 0 . 1 1 7 7 / 0 0 1 8 7 2 0 8 2 0 9 2 0 4 2 9 . [11] A. Thorpe, K. Nesbitt, A. Eidels, Assessing game interface workload and usability: A cognitive science perspective, in: Proceedings of the Australasian Computer Science Week Multiconference, ACSW ’19, Association for Computing Machinery, New York, NY, USA, 2019. doi:1 0 . 1 1 4 5 / 3 2 9 0 6 8 8 . 3 2 9 0 7 4 9 . [12] A. Rahmati, E. Fernandes, J. Jung, A. Prakash, Ifttt vs. zapier: A comparative study of trigger-action programming frameworks (2017). [13] B. Cosar, A comparative analysis of zapier and ifttt through the lens of enterprise integra- tion (2021) 54. [14] K. Ito, H. Kogami, S. Kang, R. Yoshizaki, Y. Sakurai, K. Nakano, D. Yoshioka, J. Wu, M. Fujisaki-Sueda-Sakai, I. Sugarawa, T. Miura, K. Yabu, T. Ifukube, Configuring interface for iot: Natural user interface study on senior adults in japan, in: LifeTech 2021 - 2021 IEEE 3rd Global Conference on Life Sciences and Technologies, Institute of Electrical and Electronics Engineers Inc., 2021, pp. 200–203. doi:1 0 . 1 1 0 9 / L i f e T e c h 5 2 1 1 1 . 2 0 2 1 . 9 3 9 1 9 4 3 . [15] M. Schrepp, A. Hinderks, J. Thomaschewski, Applying the user experience questionnaire (ueq) in different evaluation scenarios, 2014, pp. 383–392. doi:1 0 . 1 0 0 7 / 9 7 8 - 3 - 3 1 9 - 0 7 6 6 8 - 3 _ 37. [16] J. Chai, Y. Ge, Y. Liu, W. Li, L. Zhou, L. Yao, X. Sun, Application of frontal eeg asymmetry to user experience research, in: 11th International Conference on Engineering Psychology and Cognitive Ergonomics - Volume 8532, Springer-Verlag, Berlin, Heidelberg, 2014, p. 234–243. doi:1 0 . 1 0 0 7 / 9 7 8 - 3 - 3 1 9 - 0 7 5 1 5 - 0 _ 2 4 . [17] H. Masaki, M. Ohira, H. Uwano, K.-i. Matsumoto, A quantitative evaluation on the software use experience with electroencephalogram, in: A. Marcus (Ed.), Design, User Experience, and Usability. Theory, Methods, Tools and Practice, Springer Berlin Heidelberg, Berlin, Heidelberg, 2011, pp. 469–477. [18] K. Pollmann, M. Vukelić, N. Birbaumer, M. Peissner, W. Bauer, S. Kim, fnirs as a method to capture the emotional user experience: A feasibility study, in: M. Kurosu (Ed.), Human- Computer Interaction. Novel User Experiences, Springer International Publishing, Cham, 2016, pp. 37–47. [19] K. Lukanov, H. A. Maior, M. L. Wilson, Using fnirs in usability testing: Understanding the effect of web form layout on mental workload, in: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI ’16, Association for Computing Machinery, New York, NY, USA, 2016, p. 4011–4016. doi:1 0 . 1 1 4 5 / 2 8 5 8 0 3 6 . 2 8 5 8 2 3 6 . [20] J. Wu, D. Zhang, T. Liu, H. Yang, Y. Wang, H. Yao, S. Zhao, Usability evaluation of augmented reality: A neuro- information-systems study, Journal of Visualized Experiments (2022) e64667. doi:1 0 . 3 7 9 1 / 6 4 6 6 7 . [21] D. Yu, S. Wang, F. Song, Y. Liu, S. Zhang, Y. Wang, X. Xie, Z. Zhang, Research on user experience of the video game difficulty based on flow theory and fnirs, Behaviour & Information Technology 42 (2023) 789–805. doi:1 0 . 1 0 8 0 / 0 1 4 4 9 2 9 X . 2 0 2 2 . 2 0 4 3 4 4 2 . [22] J. Wu, D. Zhang, T. Liu, H. Yang, Y. Wang, H. Yao, S. Zhao, Usability evaluation of augmented reality: A neuro- information-systems study, Journal of Visualized Experiments (2022) e64667. doi:1 0 . 3 7 9 1 / 6 4 6 6 7 . [23] Y. N. Ortega-Gijon, C. Mezura-Godoy, Usability evaluation of brain computer interfaces: Analysis of methods and tools, in: 2019 IEEE International Conference on Engineering Veracruz (ICEV), volume I, 2019, pp. 1–8. doi:1 0 . 1 1 0 9 / I C E V . 2 0 1 9 . 8 9 2 0 4 0 9 . [24] R. Katmah, F. Al-Shargie, U. Tariq, F. Babiloni, F. Al-Mughairbi, H. Al-Nashash, Mental stress analysis using the power spectrum of fnirs signals, in: 2022 International Conference on Electrical and Computing Technologies and Applications (ICECTA), 2022, pp. 216–219. doi:1 0 . 1 1 0 9 / I C E C T A 5 7 1 4 8 . 2 0 2 2 . 9 9 9 0 2 3 7 .