We report a usability study which investigated the use of haptic versus non-speech audio interface to identify di erent geometric shapes. The study used simple graphics containing one to three geometric shapes (line, triangle, rectangle and circle). We presented the graphics to 11 participants in two di erent modes, audio and haptic, in a counterbalanced design. The participants were asked to identify the number and the types of the shapes. Error rates with audio and haptic feedback were very similar. The time to answer the overview task was generally faster with audio feedback, however it was generally faster with haptic feedback for detailed view task. These results need to be considered with some care because they were not statistically signi cant because of the small number of participants.
There have been many assistive technologies which use di erent human sensory systems. Among these systems haptic and aural systems are the most preferred ones to present graphical information because of their characteristics.
The haptic subsystem is specialised to process tactual and kinesthetic stimuli.
It has sensors that receive stimuli about touch, temperature and motion [
The aural subsystem has sensors that receive aural information such as speech
and non-speech audio [
As a result of their characteristics, these sensory systems have been used in
many di erent assistive technologies, such as [
This paper reports a usability study which investigates the use of haptic versus non-speech audio interface modes to identify di erent geometric shapes. We provide the preference, time and number of errors for participants in each of the modes, as well as the strategies that they use. 2
Comparison of Haptic and Non-Speech Audio Feedback In our rst trials we experimented with the number of ngers that we attached the vibrating motors to. We tried (i) only the right index nger, (ii) the left and right index ngers, and (iii) the left and right index and middle ngers.
Our experience, corroborated by feedback from blind participants in pilot studies was that it was bene cial to use ngers on both hands but that it was di cult to distinguish between vibration of the index and middle nger on the same hand. We rst tried attaching the vibrating devices to the underside and then to the top of the nger but this made little di erence. Our experience is that, with su cient practice, one can distinguish between vibration on all four ngers but this takes many hours of use. We therefore decided to use the tool with two ngers|the left and right index ngers|as we would not be able to give the participants the necessary time to learn to use four ngers before conducting the user study.
Given that we decided only to provide haptic feedback for the left and right index nger, a natural question to investigate was whether stereo audio feedback might be better. To determine this we implemented an audio feedback mode as an alternative to haptic feedback. This mode was restricted to the use of one nger or two ngers on di erent hands. In audio mode if the user touches an object on the screen then they will hear a sound from the headphones. If they use one nger they will hear a sound coming from both headphones while if they use two ngers then they will hear a sound on the left/right headphone if their left/right nger is on an element. The sounds associated with objects were short tones from di erent instruments played in a loop.
We conducted a usability study to investigate whether audio or haptic feedback was better for determining the geometric properties (speci cally position and shape) of graphic elements. The study used simple graphics containing one to three geometric shapes (line, triangle, rectangle and circle). Each shape had a low intensity interior colour and a thick black boundary around it. This meant that the intensity of the haptic or audio feedback was greater when the nger was on the boundary.
We used 5 di erent training graphics in total; 4 of which included di erent types of shapes (line, triangle, rectangle and circle) | see Figure 1. The last one included all of the shapes. For each shape we used a di erent audio. We changed the audio les to be sure that no kind of shape always has the same audio associated.
We used 6 graphics for the experiment whose complexity varied in the number of shapes: easy (1 shape), medium (2 shapes), and hard (3 shapes) | see Figures 2 to 4. 1. How many objects are there in the graphic? 2. What kind of geometric shape is each object?
The times taken to explore the graphic and then answer each question were recorded as well as their answers. After viewing and answering questions about the graphics presented with the audio and haptic interaction modes, the participants were asked which interaction they preferred and invited to give comments and explain the features that in uenced their preference. (a) (b) Fig. 2: Simple graphics used in comparison of audio and haptic feedback.
(a) (b) Fig. 3: Medium hard graphics used in comparison of audio and haptic feedback.
(a) (b) Fig. 4: Hard graphics used in comparison of audio and haptic feedback.
A caveat is that we slightly modi ed the presentation midway through the usability study. This was because the rst three participants had di culty identifying the geometric shapes. The reason was that they found it di cult to determine the position and number of vertices on the shape. To overcome this in subsequent experiments object vertices were given a di erent colour so that the audio and haptic feedback when touching a vertex di ered from that for the boundary and the interior of the shape. This reduced the error count to almost zero in the subsequent participants.
Another source of annoyance to the rst three participants was a delay in response from the haptic feedback due to latencies in the touch screen, Arduino circuit board, and the inertia in the vibrating motor. It was at this point that we added a predictive component to the tool which provided haptic feedback based on the expected position of the nger. 3
We recruited 11 participants, 6 born blind and 5 late blind for the study. They were aged between 17 and 63. They all had previously read a tactile graphic. 3 of the participants could not complete the experiment because of hearing and sensing problems. 8 participants completed the usability study. We found that 6 out of 8 participants preferred haptic feedback, and 2 of the 3 excluded participants also preferred haptic feedback. Error rates with audio and haptic feedback were very similar. The time to answer the question 1 (overview task) was generally faster with audio feedback, however it was generally faster with haptic feedback for question 2 (detailed view task). These results need to be considered with some care because they were not statistically signi cant because of the small number of participants.
In Table 1 and Figure 5 we give the preference, time and number of errors for each participant in each of the two modes.
There were 14 errors out of 48 diagrams which included 96 shapes. However, as we discussed earlier the rst 3 participants had di culty identifying the geometric shape. To overcome this we added vertices with a di erent colour on the shapes so that the intensity of audio and haptic feedback for a vertex di ered from the boundary and the interior of the shape. This reduced the error count signi cantly in the remaining 5 participants.
We observed that participants used two quite di erent strategies to identify shapes. The rst strategy was to nd the corners of the shapes, and then to carefully trace the boundary of the object using one or two ngers. This was the strategy we had expected.
The second strategy was to use a single nger to repeatedly perform a quick horizontal and/or vertical scan across the shape, moving the starting point of the nger between scans slightly in the converse direction to that of the scan. Scanning like this gives rise to a di erent audio or haptic pattern for di erent shapes. For instance, when scanning a rectangle, the duration of a loud sound on an edge, a soft sound inside the shape, and another loud sound on the other Median
P8 P7 P6 P5 P4 P3
P2 tna P1 p iit c raMedian P
P8 P7 P6 P5 P4 P3 P2 P1
Interface
Audio Haptic 0 100 200Time 300 400 edge are all equal as you move down the shape. But, for a triangle, the duration of the soft sound will either increase or decrease as you scan down the shape. Moreover, users could increase the speed of scanning so that they could nish the whole process quicker. However, this was harder with the tracing because the users had to adjust the direction by using the audio and the haptic feedback. With the scan strategy it was important to use the same speed for the scan, otherwise it might be confusing.
We thought that this might be a problem, but surprisingly all the participants used this strategy without any problems. The scan strategy was quite e ective and those participants who used it were faster than those using the boundary tracing strategy. 4
As a result of this usability study we decided to provide haptic feedback (through the vibrating motors) rather than audio feedback to indicate when the user was touching a graphic element. Our study showed that this was quite e ective, allowing the users to determine geometric properties of graphic elements (position and shape). The decision was due to user preferences, the slight performance advantage for haptic feedback in the detailed view task, haptic feedback being more readily generalised to more than two ngers, and because it allowed audio feedback to be used for other purposes.