Vibrotactile Exploration of Indoor Objects Alexander Fickel Limin Zeng Technische Universität Dresden Technische Universität Dresden Human-Computer Interaction Human-Computer Interaction alexander.fickel@tu-dresden.de Limin.zeng@tu-dresden.de Abstract most promising regarding to users’ acceptance and perception [5]. Therefore in this preliminary study a tactile waist belt has been This paper describes our study about how to make use of a tactile used as well. belt with 8 vibrators to present rich spatial information for blind The main aim of this work is how to support visually impaired and visually impaired people while exploring indoor environ- people for exploration tasks within rooms or buildings by using a ments. In order to conduct a Wizard-of-Oz evaluation with 12 vibrotactile belt, and how to acquire object information via tactile sighted subjects, a tablet based application has been developed for patterns. The study focused on four types of common indoor setting indoor scenarios and exploration interaction. Two sets of objects: walls, doors, stairs and general obstacles. Additionally, a tactile patterns for presentation of distance, direction and type of normal tablet based application has been developed to set a room objects, have been evaluated based on one single actuator and layout and play corresponding tactile patterns conveniently. An multiple actuators, respectively. The results indicated that the evaluation with 12 sighted subjects in a Wizard-of-Oz (WoZ) subjects can acquire the rich object information through both of study has been conducted. In the future, the tablet application the two sets, and they were more sensitive on the vibration posi- should consider how to improve its accessibility and support blind tion than the vibration intensity. and visually impaired people exploration by touching, rather than Categories and Subject Descriptors in a WoZ study. H 5.2 [User Interfaces]: Haptic I/O 2. Related Work K 4.2 [Social Issues]: Assistive technology for people with disa- bilities There have been many navigation aids with tactile feedback. In recent years, various approaches have been implemented and General Terms discussed. For example, a number of previous systems make use Design, Human Factors. of a smartphone to offer tactile feedback, like PocketNavigator [7, 8], NaviRadar [9], Lund Time Machine [10] and NonVisNavi Keywords [11]. In order to provide richer tactile information, some systems Tactile Belt, Tacton, Indoor Exploration Tasks employed multiple vibrators, such as a tactile vest and a tactile belt. The researchers in [3, 4] equipped a belt with 8 actuators, 1. Introduction and the system proposed in [5] uses not only 6 actuators, but also The tactile pin-matrix displays offer novel experiences for blind activate two at the same time with different vibration intensities to and visually impaired people to explore maps [1] and the sur- interpolate a direction between them. Most of the related work rounding obstacles [2], however, due to the cost expensively a focuses on presenting navigation instructions, like turning left or large size of such display is not affordable for most of blind and right in specific angles, however, there is a few studies focusing visually impaired people. To support blind and visually impaired on presenting rich information of surrounding objects, except the people to acquire simple spatial information, like navigation study in [4] to recognition of landmarks. There are a few previous instructions, many low-cost vibrotactile displays has already been work which made use of a touch screen device and tactile vibra- taken into account in several researches [3, 4, 5, 6]. People are tors to present spatial information about surrounding objects. able to perceive tactile stimulation changes and identify the loca- To display complex information by vibrators, the concept of tion of them. As one of essential advantages for visually impaired Tactons has been introduced by Brewster et al.: people, tactile displays can represent the direction information “Tactons are structured, abstract messages that can be used to quickly and intuitively, without disturbing visual and auditory communicate complex concepts to users non-visually.” [12] perception. However, there are a few previous work focusing on Several design principles were presented in their work, espe- presenting more complex spatial information through tactile vi- cially the concept of "Transformational Tactons" which inspired brators, like various types of surrounding objects. Heuten et al. this work. There, each property is assigned to its own parameter. concluded a tactile display with vibrators worn as a belt is the As an example of this design principle, a 3-parameter has been used to notice users about upcoming appointment information Permission to make digital or hard copies of all or part of this work for personal or [13]. classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honoured. For all other uses, contact the Owner/Author. Copyright is held by the owner/author(s). TacTT ’14, Nov 16 2014, Dresden, Germany Figure 1. The tactile belt was built with 8 actuators. Each covers a range of 45° of the body of the user and is equipped with a LED to visualize the vibrations. The presentation of highly complex properties is a challenge in tion, which can be linked to specific information. From the per- the design of Tactons. There must be found a parameter with spective of tactile perception 36 actuators on the belt are possible, many distinguishable levels. Not every parameter is suitable for corresponding to a resolution of 10 ° in the horizontal plane [4]. this. The researchers in [14] investigated the distinguishability of But the aim should always be to use actuators as little as necessary rhythms. As a result of their research, they’ve presented a set of [5], in order not to inform users with so much unnecessary infor- 21 rhythms, which could be distinguished by fingers. Besides, the mation. tests with a tactile belt in [6] indicated seven different rhythms were suitable for recognition tasks with a vibrotactile waist belt. Rhythm Rhythms are generated by grouping vibrations with different durations and different intervals [13]. To create or ma- 3. A Multi-vibrator Tactile Belt nipulate a rhythm it is necessary to change the length or the num- In this work, a tactile waist belt from the company Elitac is used ber of the vibration pulses or the breaks between them [14]. From the perspective of perception, it is possible to design many differ- (see Figure 1). The detailed technical specifications can be found ent rhythms. But it should be noted that users make the more on their website1. The belt can be controlled by a host device (e.g. mistakes when distinguishing the more number of rhythms than a smartphone or a tablet) via Bluetooth. The movements of the less number of rhythms. Moreover, it is assumed that the frustra- vibrators are transmitted by the corresponding haptic patterns that tion level increases significantly with a large number of different are previously defined in a XML-based file, and are saved on the rhythms, since user have to operate a long learning curve to learn host device. This format of haptic patterns allows to configure the rhythms. start time, intensity, duration and location of a vibration. For example, Figure 2 illustrates how to make one vibrator vibrated Intensity By changing the amplitude of the sine wave dif- two times at 0 millisecond and 500 millisecond respectively. ferent vibration levels can be generated. In [16] it is reported that The tactile waist belt is equipped with eight actuators, and intensities about 26 dB can be distinguished worse than ones with each covers a range of 45° of the body of the user (see Figure 1). lower intensity values. Normally, the intensity value over 55 dB This configuration already has been used successfully in several can cause pains to users [17]. To ensure the distinguishability of prototypes and is valid for managing navigation [3, 4] and explo- different intensities, not more than 4 stages should be used [7]. ration tasks [6]. The tactile belt offers the developers various parameters. With 4. Designing Tactons for Exploration Tasks their help information can be transmit to the wearer of the belt. An indoor assisted system with a room exploration functionality The following parameters are available: should allow visually impaired users to acquire rich spatial infor- Position The 8 actuators are attached around the body of the mation of surrounded environments. In this study, three attributes subject in a horizontal plane. Each actuator occupies a fixed posi- of surrounded objects are presented:  Type: Which object it is?  Direction: In which direction the object is?
1
 Distance: How far away the object is located? 11 To convey such complex spatial information through haptic 250 feedback, the design principles of the "Transformational Tactons" [12] have been employed adaptively. In this study each of the three object properties is encoded with its own parameter.
1
11 250 In the following, the mapping strategy between the object properties and the Tacton parameters is discussed separately. Figure 2. An example of a XML-based description format for 4.1 Direction haptic patterns. In this case actuator 1 is activated twice for 250ms The Tacton parameter for coding the direction is not difficult. The with an intensity of 11 (maximum is 15). tactile belt is equipped with eight evenly distributed actuators, which are quite intuitively associated with a direction for users. 1 http://elitac.nl/products/sciencesuit.html [28.08.2014] This method was tested successfully by several researches. Thus, the 8 vibrator position can simply indicate 8 different directions. 6 and 6-8 were selected for this work (see Figure 4), because they It’s true that the parameter level of direction could theoretically be can be associated with directions (front, back, right, left) and thus increased to 36 levels [6] by adding additional actuators. may be easier to be learnt than the other ones. Within the vibrator pairs, the vibrator with a smaller serial number is the “pioneering” 4.2 Distance actuator. On both actuators the same rhythm was played: 12 repe- In previous studies, the frequency and the intensity were used for titions of a 50ms pulse, followed by a 50ms vibration break. The presenting distance information. At present, our belt does not Tacton had to be played in two successive phases, because the provide the ability to change the frequency of a single vibration. position parameter was assigned twice (for the direction and the Therefore we made use of the intensity to encode distance infor- type). The first phase showed users the direction. This pulse al- mation. Theoretically, for a common vibrator 15 different intensi- ways had the same (median) intensity. After a gap of two seconds ties can be presented. But the preliminary tests and research work a weaker or stronger pulse (compared to the direction pulse) was have shown that differences in intensity can be very poorly per- played on a pair of actuators depending on the distance. ceived. Therefore in our study the distance is encoded only with For a belt with eight actuators there are 12 pairs with a recog- two levels: “near” (all objects with a distance less than two me- nition rate of at least 80% [6]. Therefore, this method is well ters) and “far” (all objects with a distance more or equal than two suited for encoding other objects in the future. meters). 4.3 Object type In general there are various objects in rooms and buildings, but the focus of this work was on the four important ones for visually impaired people: walls, stairs, doors and obstacles. Thus, the parameter should cover at least four distinguishable stages for the 4 target types. In order to convey information about object types for users, two different haptic feedback based methods have been employed: a single actuator method and multiple actuator method. 4.3.1 Single-Actuator-Method In the single-actuator method the tactile patterns are encoded via different rhythms on a single actuator. Four distinguishable rhythms were selected for this work, as shown in Figure 3. The Figure 4. The different actuator pairs of the Multi-Actuator- grey areas represent vibration pulses, and the white fields show Method, and always two actuators are activated simultaneously to the vibration breaks. A rhythm takes a second and for a better indicate an object type. recognition it is always played twice. As a basic metric, the 1/16 pulse is the shortest one and has a length of 62.5ms, and each pulse should be followed by a pause of at least 125ms, except the 1/16 pulse, where it is followed by a pause of at least 62,5ms. It is 5 The Pilot Evaluation also possible to add additional object types in the future. To evaluate the designed Tactons we’ve developed a specific With the single-actuator method the characterizing rhythm is App, named WizzApp. 12 sighted subjects had been recruited for played on the pioneering actuator with the corresponding intensi- this evaluation. ty. Therefore, in contrast to [6] all parameters are displayed simul- taneously. This approach reduces the run time of the Tactons 5.1 ”WizzApp”: A Tablet Application significantly and thus speed up the transmission of the infor- To evaluate the tactile patterns a simulation application for An- mation. droid Tablets, called “WizzApp”, has been developed (see Figure 5). The App was implemented on a Motorola Xoom tablet with a 10.1 inch display and the Android OS. In addition to setting the room size, it also allows set the layout of the objects. The App is able to provide virtual spatial information to subjects while walk- ing and exploring the environments. In this study, we only used the Exploration Mode to test the proposed tactile patterns, and the position and the orientation of the subject were simulated by continuously touch input by a “Wizard”. Figure 3. The different rhythms of the Single-Actuator-Method. The grey areas indicates the vibration, the white ones the breaks. To support a Wizard conveniently and precisely to simulate sub- jects’ movements (i.e., position and heading direction), we devel- oped a special touch interaction method, see Figure 6. In the App, 4.3.2 Multi-Actuator-Method it was easy to configure the test room with different types of objects, and the position and the heading direction of the subject Within the multi-actuator-method actuators were always activated was represented by a red point and a black line, respectively. in pairs to indicate various types of objects. In the evaluation of When touching on the red point, an outer orientation ring is ren- [6], the method achieved even better recognition rate than the dered. The Wizard can change the orientation by moving his single-actuator-method if at the same time the direction was dis- finger around the user circle, but within the orientation ring. The played. From the seven tested actuators pairs, the pairs 2-4, 2-8, 4- Wizard also can simulate subjects’ walking paths by moving the Figure 7. Two room simulations layouts for the evaluation of the designed Tactons speak out the object type, the direction (i.e., eight cardinal direc- tions) and the distance immediately once they recognized. The accuracy of these statements and the recognition time were noted. The subjects had to evaluate the two methods in the two test Figure 5. The screenshot of the “WizzApp” on a tablet (the rooms accordingly, and the test arrangement was counterbalanced. red point is the subjects’ position, and the black line from the red point indicates the subjects’ heading orientation. In the end, the subjects had to fill in a questionnaire:  Which method was easier to learn? Why?  If you have to choose a method, which one would you choose? Why? 5.3 Results The comparison of the results between the single-actuator-method and the multi-actuator-method is shown in Table 1. The two methods have achieved similar results for encoding the direction and the distance. The object type was detected better with the multi-actuator-method, as well as a better recognition perfor- mance of the whole Tacton2. For the two methods the direction and the object type have a very high recognition accuracy (more Figure 6. The touch interaction to simulate users’ heading than 94%), while the recognition accuracy of the distance was less orientation and position than identifying the object type. However, the overall recognition accuracy of the Tactons dropped to 78% for the single-actuator- method and 81% for the multi-actuator-method, responsively. finger easily, like in the Walking Mode. With a double-tap on any Besides, the subjects spent 437ms and 634ms averagely to recog- object, the App sends the corresponding tactile patterns to the nize one Tacton, accordingly. tactile belt, by calculating the direction and the distance of this object to users automatically. Table 1. Average recognition rates of the designed tactile pattern. Two rooms were simulated and each one had 12 or 13 objects Method Property Recognition accuracy from the four types (see Figure 7). The position and orientation of Type 94,67% the subjects was the same and was fixed in each test. Additionally, Direction 97,33% Single-Actuator the order of displaying the objects was fixed. Thus, all subjects Distance 85,33% had the same conditions. Whole Tacton 78,00% Type 98,67% 5.2 Procedure Direction 97,33% Multi-Actuator Distance 86,00% The test was performed with six male and six female individuals. Whole Tacton 81,33% Two subjects were 58 years old, the others were around 23 to 30 years old. Every subject tested the two sets of Tactons. The two rooms were tested in the same order for all subjects, but the order of the two sets of Tactons were different. Regarding to the test The two-way MANOVA (at the 95% confidence level) re- order, six subjects tested the single-actuator-method firstly, and vealed only the two methods had significant multivariate main the other six ones tested the multi-actuator-method firstly. effect for the recognition accuracy of object type, direction, dis- Each subject at first received a brief introduction to their tasks tance, the whole Tacton and spending time, Wilks’ λ = 0.369, F(5, and the Tactons. Subsequently, a training phase was followed to 16) = 5.47, p = 0.04. Furthermore, the univariate ANOVA tests (at learn the Tactons in a training setting. The real test didn’t start the 95% confidence level) only found the two methods had main until the subjects claimed they had learnt all. In real tests all ob- jects were displayed one by one, triggered in the WizzApp by a 2 When all of the 3 attributes (i.e., type, direction, and distance) of one Wizard. Note that, the subjects did not allow to watch the screen object were identified correctly, we countered the Tacton was recognized at any time during the test. The subjects were asked to loudly successfully. effect for the spending time (F(1, 20) = 24.804, p < 0.001, and the ied two different methods to inform users about object type in- interaction between the two methods and the test rooms had main formation via a single actuator and multiple actuators, respectably. effect for the recognition accuracy of a whole Tacton ( F(1, 20) = Through a pilot WoZ evaluation with 12 sighted people, we 4.634, p = 0.044. The paired T-test (at the 95% confidence level) found the subjects were able to acquire the spatial information of found the mean accuracy of distance recognition had significant surrounding objects by both of the two methods. The subjects had differences to the mean accuracy of direction recognition (t = more sensitive on the vibration position than the vibration intensi- 4.252, p < 0.001) and the mean accuracy of type recognition (t = ty. Additionally, a tablet based application has been implemented 4.684, p < 0.001) both. to support the evaluation, for setting the room layout and explor- The analysis of the questionnaires showed that, on one hand ing objects one by one. the multi-actuator-method was easier to learn for 7 of 12 subjects, In the future, in addition to evaluating the tactile patterns with because they felt “less complex”, “easier perceptive”, “shorter blind and visually impaired individuals, it’s important to evaluate learning time”, or “good spatial imagination”; on the other hand, the exploration task by blind and visually impaired people direct- there were still 7 subjects who chose for the single-actuator- ly, rather than simulating the exploration behaviours by a sighted method, because they thought the Tacton were “shorter”, “more Wizard. intuitive”, “more memorable” and “more practicable”. 7 Acknowledgements 5.4 Discussion The authors were grateful to all of the subjects who took part in From the evaluation, it found the subjects were more sensitive to the evaluation. This study was supported by the Range-IT4 project the position of vibrations (e.g. for presentation of direction and within the framework of EU FP7 SME Program (Grant no. object type), than the intensity feature (e.g. for presentation of 605998). distance). It seems to be hard to distinguish the intensity of vibra- tion because of users’ clothes with different texture or fixing the References belt with different strength. In order to inform users the distance [1] Zeng, L., Mei, M. and Weber, G. 2014. Interactive audio-haptic map information, a set of extra vibrators can be placed vertically on explorer on a tactile display. Interacting with Computers, doi: other body part, like on the arm, and the closer vibration means 10.1093/iwc/iwu006. closer objects. Additionally, it might be also possible to change [2] Zeng, L., Prescher, D. and Weber, G. 2012. Exploration and avoid- the vibration frequency for indicating the distance information. ance of surrounding obstacles for the visually impaired. In Proceed- In this study, due to only 4 different object types involved, the ings of ACM ASSETS 2012, 111-118. subjects spent more or less equal time to learn the tactile patterns. When more object types are involved, we expect users have to [3] Tsukada, K. and Yasumura, M. 2004. Activebelt: Belt-type wearable spend more time for learning the single-actuator method than the tactile display for directional navigation. In UbiComp 2004: Ubiqui- multi-actuator method, and we need to further tests to confirm that. tous Computing, Springer, Berlin, Heidelberg, 384–399. Due to lack of tactile feedback, blind and visually impaired [4] Van Erp, J. B. F., Van Veen, H. A. H. C., Jansen, Chris, and Dob- people is hard to access spatial information (e.g., maps, layout of bins, Trevor 2005. Waypoint navigation with a vibrotactile waist the surroundings) through common touchscreen devices. On one belt. In ACM Trans. Appl. Percept. 2(2), ACM, New York, NY, hand, many new displays, like the pin-matrix display and the USA 106–117. under-developing BlindPad3 system, have been used to improve [5] Heuten, W., Henze, N., Boll, S., and Pielot, M. 2008. Tactile way- the accessibility of spatial information. On the other hand, we finder: A non-visual support system for wayfinding. In Proceedings think the vibrotactile feedback might be a low-cost solution to of the 5th Nordic Conference on Human-computer Interaction: reach the goal by combining touchscreen displays. Specifically, Building Bridges. NordiCHI ’08. ACM, New York, NY, 172–181. the vibrotactile feedback also can be used while walking. Alt- hough in this evaluation we let a Wizard touch the display to [6] Srikulwong, M. and O’Neill, E. 2011. A comparative study of tactile trigger and simulate an exploration task of indoor objects, we representation techniques for landmarks on a wearable device. In Proceedings of the SIGCHI Conference on Human Factors in Com- think it is possible to allow blind and visually impaired people to puting Systems. CHI’11. ACM, New York, NY, 2029–2038. explore by themselves after a few improvement of the WizzApp, like importing semantic sounds. Moreover, there should be no a [7] Pielot, M., Poppinga, B., and Boll, S. 2010. Pocketnavigator: Vibro- large delay of vibrations when exploring on the touchscreen dis- tactile waypoint navigation for everyday mobile devices. In Proceed- plays, otherwise, users might make mistakes to understand the ings of the 12th International Conference on Human Computer In- position and direction information of objects. teraction with Mobile Devices and Services. MobileHCI’10. ACM, New York, NY, 423–426. [8] Pielot, M., Poppinga, B., Heuten, W., and Boll, S. 2012. Pocketnavi- 6 Conclusion gator: Studying tactile navigation systems in-situ. In Proceedings of For blind and visually impaired people, it is challenging to acquire the SIGCHI Conference on Human Factors in Computing Systems. spatial information of surrounding environments, like how many CHI’12. ACM, New York, NY, USA, 3131–3140. objects in front of them, as well as their properties. In this paper [9] Rumelin, S., Rukzio, E., and Hardy, R. 2011. Naviradar: A novel we presented how to make use of a tactile belt with 8 vibrators to tactile information display for pedestrian navigation. In Proceedings convey spatial information of indoor objects (i.e., walls, stairs, of the 24th Annual ACM Symposium on User Interface Software and doors and general obstacles) by pre-designed tactile patterns. In Technology. UIST’11. ACM, New York, NY, USA, 293–302. addition to rendering distance and direction information, we stud- 3 BlindPad project, http://www.blindpad.eu/ 4 Range-IT project, http://www.range-it.eu/ [10] Szymczak, D., Magnusson, C., and Rassmus-Grühn, K. 2012. Guid- [14] Ternes, D. and Maclean, K. E. 2008. Designing large sets of haptic ing tourists through haptic interaction: Vibration feedback in the icons with rhythm. In Proceedings of the 6th International Confer- lund time machine. In Haptics: Perception, Devices, Mobility, and ence on Haptics: Perception, Devices and Scenarios. EuroHaptics Communication. Springer, Berlin, Heidelberg, 157–162. ’08. Springer, Berlin, Heidelberg, 199–208. [11] Nukarinen, T., Raisamo, R., Pystynen, J., and Mäkinen, E. 2012. [15] Brown, L. M., Brewster, S. A., and Purchase, H. C. 2005. A first Nonvisnavi: Non-visual mobile navigation application for pedestri- investigation into the effectiveness of tactons. In Proceedings of the ans. In Haptics: Perception, Devices, Mobility, and Communication. First Joint Eurohaptics Conference and Symposium on Haptic Inter- Springer, Berlin, Heidelberg, 214–217. faces for Virtual Environment and Teleoperator Systems, WHC ’05. [12] Brewster, S. and Brown, L. M. 2004. Tactons: Structured tactile IEEE Computer Society, Washington, DC, USA, 167–176. messages for non-visual information display. In Proceedings of the [16] Craig, J. C. and Sherrick, C. E. 1982. Dynamic tactile displays 1982. Fifth Conference on Australasian User Interface - Volume 28. AUIC In Tactual Perception: A Sourcebook. Cambridge University Press, ’04. Australian Computer Society, Inc., Darlinghurst, Australia, Aus- 209–233. tralia, 15–23. [17] Gunther, E., Davenport, G., and O’Modhrain, S. 2002. Cutaneous [13] Brown, L. M., Brewster, S. A., and Purchase, H. C. 2006. Multidi- grooves: Composing for the sense of touch. In Proceedings of the mensional tactons for non-visual information presentation in mobile 2002 Conference on New Interfaces for Musical Expression. NIME devices. In Proceedings of the 8th Conference on Human-computer ’02. National University of Singapore, Singapore, Singapore, 1–6. Interaction with Mobile Devices and Services. MobileHCI ’06. ACM, New York, NY, USA, 231–238.