Smart ARbnb: Smarthome Interface for Airbnb with Augmented Reality and Visible Light Communication Tomas Gecevičius, Yaliang Chuang *, Jingrui An Dept. of Industrial Design, Eindhoven University of Technology, Eindhoven, the Netherlands tomas@gecevicius.lt,y.chuang@tue.nl,j.an@tue.nl ABSTRACT KEYWORDS Peer-to-peer rental services are continuously growing in the Automation, Smarthome, Feedforward, Situational Aware- sharing economy era. There also are more and more houses ness, User Experience, Mental Model with smarthome system added into the market. The hosts originally designed the automation system to improve the 1 BACKGROUND comfort and convenience of their own lives, but also shared it Smarthome recently becomes increasingly ubiquitous with with guests for enhancing their experiences while the hosts the popularity of Internet of Things (IoT) technology and were away from home. However, due to the lack of commu- products. It brings comfort, energy efficiency, and security to nication, the guests might encounter negative experiences homeowners [7] by letting them control the devices through when the automation took place without perceptible in-situ their smartphone or self-operated with automation rules. feedforward and feedback in advance. In this study, we devel- These features can provide superior user experiences in sim- oped a web-based application that combines augmented re- plifying daily routines, such as automatically switching off ality and visible light communication to control smart home appliances when users left home or adjusting the home tem- devices. The tool uses a mobile camera to scan IoT commu- perature in advance to when users are on the way back home. nicators, let users change connected devices’ settings, and Furthermore, when users are away from home for a business visualizes the current set automation. Additionally, guests trip or vacation, the system can also give users peace of mind can use presets to change the atmosphere, creating a won- by monitoring the house for users and simulating people’s derful experience. Our user study shows users preferred the presence with random lights to scare away intruders. web-based system rather than installing new apps. The inter- Smarthome is also a life-changing tool for Airbnb hosts action experience was easy to use and overall positive. Our to ensure the safety of the rental accommodations and cut AR interface provided direct controls of the appliances and off the energy costs. Hosts can remotely let guests enter by clear explanations of the functionalities. While users recog- using the smart lock or give access only for a limited amount nized the usefulness of the physical feedforward shown with of time. They can also remotely check the temperature with LED interfaces, some users could not interpret the meanings a smart thermostat and switch off the system when guests initially, and sometimes the blinking lights were distracting. are not staying inside. One of the reasons hosts are using We highlighted several design considerations for improving smart devices is to monitor the Airbnb property and guests’ the understandability and guest experiences for staying in activities, for example, by installing outdoor cameras or noise smart accommodations. sensors, hosts are checking if the visitors are not creating big parties in their place and are not bothering neighbors with CCS CONCEPTS the noise [4]. In addition, 60% of people would pay more if • Human-centered computing → User interface design; the listed rental place has a smarthome system [5]. Empirical studies in interaction design; Ubiquitous and mobile Customer satisfaction is an important factor when provid- computing design and evaluation methods. ing a service as it delivers useful behavior after the purchase, for example, helpful word-of-mouth or repeat visitations, resulting in growing profits [10]. For the guests who do not Copyright 2021 for this paper by its authors. Use permitted under Creative have a smarthome system at their home, this kind of tech- Commons License Attribution 4.0 International (CC BY 4.0). nology in Airbnb rented apartments brings an innovative experience. From accessing the property without a physical key to getting a fresh cup of coffee when they wake up it makes the stay in the rented accommodation special [8]. The smarthome system in hosting brings profits and saves energy costs, while also provides a unique experience that guests prefer. CHIIoT 2, June 8, 2021, Eindhoven, The Netherlands Gecevičius, Chuang, and An 2 RESEARCH QUESTIONS the smarthomes. To avoid creating additional mobile appli- While smarthome provides significant benefits for the hosts, cations that guests would need to download and install, a the guests might lack a corresponding understanding of the different approach was selected - to create a website that systems. Guests usually feel uncomfortable around smart could be accessible only in the local home network with devices regarding privacy and do not understand how every- the “.local” domain or by tapping the IoT communicators’ thing works. Very often, hosts are not always present to help integrated NFC tag with your phone. After reading the vi- guests check in the property and explain how the system sual light pattern the web interface gets the control from the works. The lack of guests’ experience and understanding of smarthome system and gives access to the settings of the the smarthome system can lead to unexpected losses of time, devices. money, and complaints [9]. The other functionality of the web application is atmo- sphere selection (Fig.2b). A list of automation presets that 3 METHODS the host could create and guests could choose to execute, for example, “Romantic time”, “Dinner with friends”, “Dance We applied the research through design methodology[12] to party”, could create a perfect setting for unforgettable memo- explore the "Smart ARbnb" system we developed for solving ries. The shown atmosphere should be relevant to the user’s the abovementioned problems and improve guests’ expe- current context - showing options that control only the room riences in a smart accommodation. It consists of multiple devices the user is in and based on time of the day. Moreover, devices equipped with tangible IoT communicators [3] (Fig. the interface could show suggestions that are connected with 1) and a web-based interface that could be used on various the scanned device, for example, when controlling music mobile devices. speaker, it could suggest local traditional or popular music playlists, or when interacting with motorized blinds in the Web Application of the Smart ARBnB morning, it could offer to use “Morning routine” automation. The first part of the prototype is a web application. The In addition, the UI should indicate upcoming automation that website is a tool that uses a mobile camera to detect smart is linked with the current scanned device, issues if something appliances in the environment by reading IoT communica- goes wrong, for example, the battery is low, the device lost tors’ light patterns (Fig.2). In the Airbnb context, the guests connection to the network, etc. Besides that, the host could should have a quick and simple way how to interact with attach a notification or a message to the devices, informing the guests that they changed the device settings remotely or telling other relevant information (Fig.2c). In addition, during the research it was found that hosts prefer to control the smart appliances remotely by them- selves in order not to let the guests set the settings too high, Figure 1: The “Smart ARbnb” system developed in this study. It consists of tangible IoT communicators with visible LED lights and a web-based interface with Augmented Reality feature. Figure 2: The user interface of the web application. Smart ARbnb: Smarthome Interface for Airbnb with AR and VLC CHIIoT 2, June 8, 2021, Eindhoven, The Netherlands for example, with thermostat increase the temperature, thus communicator light also acts as a visual indicator of the place- rising the energy consumption costs. For this reason, many ment of networked devices. In addition, smarthome systems hosts are not letting the guest access the smarthome inter- are not expressive, they do not visualize what is happening face at all. To find a compromise, the maximum or minimum with the system or smart devices. The IoT communicator’s value could be set by the host for each of the device settings lights can be used as a “Peripheral interaction” display [1], (Fig.2d). In this case, the host can be sure, that the guests indicating the feedforward or feedback from smarthome au- will not heat the accommodation too much or will not play tomations, showing issues or notifications from the devices music or TV in the night too loud, thus disturbing neighbors. and messages from hosts. Light patterns are shown in Fig.5. Furthermore, guests who come to the new accommodation This device’s most important part is a LED light, so this func- are not aware of the current set automation in the smarthome tionality can be easily incorporated into the smart device system. This web application should show the automation itself and no extra devices would be needed to have near overview and if needed, visualize which devices belong to each smart appliance. Nevertheless, when different smart selected automation (Fig.3). For a smarthome system, espe- appliances and home systems still do not have standardized cially in the Airbnb context, privacy is important, therefore network protocol to increase the compatibility with different people prefer to have a smarthome server locally rather than brand devices, it is highly unlikely that this kind of proposed cloud-based[11]. Our Smart ARbnb system could be fully AR interface would be incorporated inside of them, thus operated with the local network. separate IoT communicators are needed. In the Airbnb context, the guests should have a quick and Physical IoT Communicators simple way how to interact with smarthomes. To eliminate The second part is the IoT communicator which is a battery- the hassle of forcing guests to download and install a new ap- powered coin size device (Fig.4) that is attached to each plication on their devices, a different approach was selected smart appliance in the home. Each also has connected to the - to create a website that could be accessed only in the local smarthome system and equipped with integrated LED lights home network with the ".local" domain or by tapping the for providing feedback and feedforward to users. This device IoT communicators’ integrated NFC tag with your phone. uses visual light communication, a specific, morse code in- After reading the visual light pattern, the web interface gets spired, light pattern to broadcast the identification number control from the smarthome system and gives access to the of the smart appliance. While the smart appliance market devices’ settings. Research showed that Airbnb guests have is rapidly growing[8], guests might not be able to under- some difficulties with controlling appliances [6], like the stand which devices are smart and which are not, so the IoT oven, dishwasher, or washing machine. In addition, in the Airbnb accommodation, guests can come from different coun- tries, so the interface should be multilanguage and also focus Figure 4: The physical IoT Communicator. It could be linked to a smart appliance and allocated aside. It can display alter- Figure 3: After the system detects the devices with the cam- native LED lighting patterns for serving two purposes: (1) era, a user can switch to the automation overview to see the for Smart ARbnb system to detect its location and status, (2) explanations of related rules set for the devices. giving user corresponding feedforward and feedback. CHIIoT 2, June 8, 2021, Eindhoven, The Netherlands Gecevičius, Chuang, and An Figure 5: The three different lighting patterns designed for Figure 6: The layout of the studio apartment and positions the IoT Communicators to notify users the particular sys- of the six devices used in the user study. tem behaviors with users’ peripheral perceptions. easy" (P1). In addition, participants favored that the interface more on the visual elements, like icons to indicate what is is in a website - "Everything looks cool and quite cute, but I needed. By providing a simple and informative interface (Fig. don’t like to install any apps, but if it is made on the website, 3), guests should avoid usability or misuse issues. then it is awesome" (P3). Moreover, the atmosphere selection was an appreciated feature - "...the ambient settings are super User Study nice when you want the right settings in a second" (P4), "But A user study is conducted with five participants to evalu- still, like the Ambient setting it’s really cool. You just come ate the design concept and the usability of the prototype. in, feel like this and ’pop’. That’s really the best" (P1). In The research was performed in a studio-type apartment that overall, participants are positive about this project because has a working smarthome system with few smart devices - it is easy to use, creates an exciting experience, it is quickly two Philips light bulbs, a Philips hue light strip, a connected accessible and it helps them quickly change the environment speaker, a motion sensor, and a thermostat. In total, six IoT to their needs. communicators were used and scattered in the studio. The The important factor of satisfaction with this concept is smart device and IoT communicators’ placement can be seen the experience of being around the lighted up IoT communi- in Fig.6. We adapted the experimental setup used in [2] to col- cators. The user study showed that the opinions about it are lect users’ interpretations and user experiences of our design contrasting. Three out of five participants felt the physical concept with thinking aloud approach. After completing the interface was very helpful. The lights create a feeling that testing, we also conducted a semi-structured interview to the smarthome wants to interact - "They’re blinking. They collect their feedback on improving our design. try to tell me something" (P2), "It seems that they want my at- tention all the time" (P3). The IoT communicators also bring 4 FINDINGS curiosity and help to find the smart devices in the accommo- The results indicate that participants look positive about this dation - "The feeling was quite nice, it was intriguing" (P3), concept. Users value the exciting experience it creates - "I "But it is very good to find them because if it didn’t light up, can imagine if I use this in every place, I would first just try I wouldn’t have found them [i.e., smart appliances]." (P1). On out everything. Yeah, so it’s like quite exciting" (P2) and also the other hand, two participants said that the blinking IoT the gamified feeling: "It was a nice experience. I really liked communicators lights might be annoying and overwhelm- that scavenger hunt feeling, that you are walking with your ing - "They neutral well with the slightly negative though, phone and searching for the sensors, you don’t know what because there’s still a bit distracting" (P4), "I feel like these you are going to find. It’s really fun" (P3). Furthermore, users lights might be a bit too much might be a bit overwhelming. liked that it is easy to use application - "It’s incredibly easy When you’re being confronted by lights at the sudden from to learn it’s the procedure" (P4), "Aesthetically pleasing, easy every direction" (P5). to understand, allows to avoid using voice commands" (P3), Besides the interpretations, we also noticed that the light "That’s the power of this (product), like, you only need to color could be adjusted to enhance the specific meanings of learn once, and once you’ve seen it, then you know it’s quite the message. For instance, P4 mentioned that "Notification Smart ARbnb: Smarthome Interface for Airbnb with AR and VLC CHIIoT 2, June 8, 2021, Eindhoven, The Netherlands should be more direct, it is too calm for it." Similarly, P1 also the 2020 ACM Designing Interactive Systems Conference (DIS ’20). As- said, "I would use more ’Blink Blink’ solution." Since the sociation for Computing Machinery, New York, NY, USA, 2007–2019. https://doi.org/10.1145/3357236.3395450 notification light pattern color was connected more to the [3] Yaliang Chuang. 2020. IoT Communicator: A Physical Interface for thermostat functionality rather than to the pattern itself, the Expressing Situational Info of an IoT System. In Companion Publication color should be changed to more neutral, like white or violet. of the 2020 ACM Designing Interactive Systems Conference (DIS’ 20 Companion). Association for Computing Machinery, New York, NY, 5 CONCLUSION AND FUTURE STUDY USA, 369–372. https://doi.org/10.1145/3393914.3397096 [4] Rajib Dey, Sayma Sultana, Afsaneh Razi, and Pamela J. Wisniewski. After researching the smarthome systems in the Airbnb con- 2020. Exploring Smart Home Device Use by Airbnb Hosts. In Extended text, the main problems as confusion and misuse of appli- Abstracts of the 2020 CHI Conference on Human Factors in Computing ances, unawareness about the current set automation, and Systems (CHI EA ’20). Association for Computing Machinery, New lack of expressiveness of the smarthome system. Among York, NY, USA, 1–8. https://doi.org/10.1145/3334480.3382900 them, the accessibility to the smarthome interface are found, [5] August Home. 2016. Research: Smarter Hosting With Smart Home Tech. https://august.com/blogs/home/research-smarter-hosting-with- which can outweigh the benefits like energy, cost savings, smart-home-tech and pleasing home automation that could create a nice at- [6] Shrirang Mare, Franziska Roesner, and Tadayoshi Kohno. 2020. Smart mosphere or experience for the guests. To overcome those Devices in Airbnbs: Considering Privacy and Security for both Guests challenges, we developed a Smart ARbnb system for guiding and Hosts. 2 (May 2020), 436–458. https://doi.org/10.2478/ popets-2020-0035 guests’ onboarding process and facilitating them to use the [7] Sarah Mennicken, Jo Vermeulen, and Elaine M. Huang. 2014. From smart system properly. today’s augmented houses to tomorrow’s smart homes: New direc- We conducted a user study with five participants to evalu- tions for home automation research. In Proceedings of the 2014 ACM ate the usability and design concept. Their overall experience International Joint Conference on Pervasive and Ubiquitous Computing of the interaction and system appears to be positive, and all (UbiComp ’14). 105–115. https://doi.org/10.1145/2632048.2636076 [8] Stephanie. 2020. Could smart technology be useful in an Airbnb users would like to see the "Smart ARbnb" system inside the listing? https://community.withairbnb.com/t5/Hosting/Could-smart- next Airbnb accommodation place. While users recognized technology-be-useful-in-an-Airbnb-listing/m-p/1222523#M294091 the usefulness of the physical feedforward shown with vis- [9] Erose Sthapit and Jano Jiménez-Barreto. 2018. Exploring tourists’ ible LED lights, the information was not intuitive to some memorable hospitality experiences: An Airbnb perspective. Tourism Management Perspectives 28 (Oct. 2018), 83–92. https://doi.org/10. users. Sometimes the blinking lights were distracting. 1016/j.tmp.2018.08.006 In future work, the user interface and lighting patterns [10] Iis P. Tussyadiah. 2016. Factors of satisfaction and intention to use peer- can be improved and evaluated regarding the understandabil- to-peer accommodation. International Journal of Hospitality Manage- ity of the automation overview and better balance between ment 55 (May 2016), 70–80. https://doi.org/10.1016/j.ijhm.2016.03.005 notifying users and distracting their attentions. [11] Weizhi Zhang and Mohsen Kavehrad. 2012. A 2-D indoor localization system based on visible light LED. In 2012 IEEE Photonics Society Sum- REFERENCES mer Topical Meeting Series. 80–81. https://doi.org/10.1109/PHOSST. 2012.6280711 [1] Saskia Bakker and Karin Niemantsverdriet. 2016. The Interaction- [12] John Zimmerman, Jodi Forlizzi, and Shelley Evenson. 2007. Research Attention Continuum: Considering Various Levels of Human Attention through design as a method for interaction design research in HCI. In in Interaction Design. International Journal of Dsign 10, 2 (2016), 1–14. Proceedings of the SIGCHI Conference on Human Factors in Computing http://www.ijdesign.org/index.php/IJDesign/article/view/2341 Systems (CHI ’07). Association for Computing Machinery, New York, [2] Yaliang Chuang. 2020. Designing the Expressivity of Multiple Smart NY, USA, 493–502. https://doi.org/10.1145/1240624.1240704 Things for Intuitive and Unobtrusive Interactions. In Proceedings of