A Quantitative Evaluation Approach for Cognitive Maps of Blind People Mei Miao, Gerhard Weber Technische Universität Dresden, Institut für Angewandte Informatik, Nöthnitzer Straße 46, 01187 Dresden, Germany {mei.miao, gerhard.weber} @tu-dresden.de Abstract. In the field of Human-Computer Interaction, several projects aim to develop new technological aids, which enable and provide people who are blind the ability to navigate themselves around on their own. In order to evaluate these technological aids, the created cognitive maps which are built by the help of technological aids should be evaluated at first. A new approach has been de- veloped for evaluating these reconstructed cognitive maps quantitatively. In this paper, we describe how this approach has been developed. Nine criteria are identified and weighted with help from the blind people. Using weighted Euclidean distance enables the cognitive maps to be compared with each other. Keywords: cognitive map, blind people, quantitative evaluation. 1 Introduction Cognitive maps also referred to as mental maps, express the essential structure of spatial information through learning processes [1]. One of a cognitive map’s functions is to support navigation. Research into cognitive maps is particularly useful to urban planners, mobility specialists, and navigation aid designers [2]. For most of the blind people, it is usually impossible to travel independently. Therefore, in the field of Hu- man Computer Interaction, several projects [3-5] aim to develop new technological aids to provide blind people the ability so that they can navigate indoor/outdoor on their own. In order to evaluate such technological aids, the blind people are often asked to create cognitive maps which are built by the help of technological aids. Two themes will be mostly measured: route knowledge and configurational knowledge [6]. Route knowledge means the knowledge of a route from point A to point B. Configurational knowledge means the knowledge of where the roads and landmarks are located. For blind people, there are two main ways to create cognitive maps with respect to route knowledge and configurational knowledge. We can ask them to verbally describe it [7] or to reconstruct it with help of things like Modelling kit, whiteboard, bar magnets [7, 8]. Sketch mapping is the most common method for sighted people, but it is rarely used for blind people, because most of blind people are unfamiliar with it. For evaluating the technological aids, the reconstructed cognitive maps should be evaluated. Figure 1 shows 3 maps: the original map (left, the street names are not Proceedings SKALID 2012 37 Miao & Weber displayed), cognitive map 1 reconstructed by subject A (middle), and cognitive map 2 reconstructed by subject B (right). How can we evaluate these maps with respect to configurational knowledge? original map created map 1 created map 2 Fig. 1. Examples of cognitive maps created by blind people In this paper, a new approach will be presented to quantitatively evaluate cognitive maps which are reconstructed by blind people. 2 Related Work Until now, there has been comparatively less attention paid towards the evaluation approach of cognitive maps for the blind than for the sighted people. [9, 10] describe the methods of evaluating cognitive maps quantitatively. However, the considered cognitive maps were drawn from sighted people. In the following, we just discuss the evaluation of cognitive maps of blind people. With technological aids, blind people build the cognitive maps just based on auditory and haptic cues, so their reconstructed cognitive maps differ somewhat from those based on vision. For evaluating these cognitive maps, there is no systematic approach available. In this context, the study in [11] should be mentioned. In their study, the blind subjects were asked to model the layout of labyrinths (see fig. 2) after having explored them. Fig. 2. Labyrinth 1 (left) and labyrinth 2 (right) [11] The cognitive maps were analyzed in five different criteria: Number of elements: 3 in Labyrinth 1 and 4 in Labyrinth 2. Form of elements: referring to the correct identi- fication of the L and T forms in the composition. Position: referring to the correct orientation of the elements. Placement: referring to the correct distance between the elements. Symmetry: referring to the correct axial disposition of the elements in the first labyrinth and the correct central disposition in the second labyrinth. A map of the real environment is more complex than the labyrinths in figure 2. There are features such as street names, curves on the streets, different directions, crosses and so on. Therefore, to evaluate reconstructed cognitive maps of the real environment, we need a more complex approach. 38 Proceedings SKALID 2012 A Quantitative Evaluation Approach for Cognitive Maps of Blind People 3 Development of the Approach In this section, we describe how this approach has been developed in detail. There are 3 steps: in the first step, the criteria were identified with which we can evaluate the cognitive maps quantitatively; then, we weighted these criteria; and finally, the method for the quantification of the criteria was chosen. 3.1 Identification of Criteria In this step, we analysed several cognitive maps which were reconstructed by blind people in a previous study of us. In this study, ten blind people were asked to con- struct mental maps after learning three different maps of unknown environments us- ing three different media (tactile map, iPad and tactile pin device (www.Hyper- braille.de)). Space restrictions do not allow to go into details here. Nine criteria were identified by the principle: one mistake should not be counted twice. The criteria were classified in 4 categories: • Category 1: number of elements ─ Number of correct street segments (C1): streets are divided in segments by crosses or branches. Only the correct street segments should be counted. ─ Number of correctly remembered street names (C2): This criterion is just rele- vant, if we want to evaluate how easy blind people can remember the street names with the help of technological aids. • Category 2: property of streets. Property of streets refers to the shape, name and direction of the streets. ─ Number of correct street shapes (C3): referring to, if a street is straight or has curves, the number and the directions of the curves. ─ Number of correctly assigned street names (C4): These include the street names which were not only remembered correctly, but also assigned to the right streets correctly. ─ Number of correct street direction (C5): A tolerance range for the estimation of direction is necessary for blind people. On the basis of the 12 division on a clock face, we define 2 divisions as tolerance range. In the case that a street has curve(s), we define the direction with reference to the start and end point of the street, if there is no landmark in between. • Category 3: arrangement of streets ─ Number of correct crosses and branches (C6): a cross or branch is correct, if it is found in the original map. • Category 4: number of errors ─ Number of none existing streets (C7): streets should be counted which do not exist in the original map. ─ Number of none existing crosses and branches (C8): this refers to the crosses and branches which are caused by streets which are reconstructed too long, but not by none existing streets. Proceedings SKALID 2012 39 Miao & Weber ─ Number of displacements of streets (C9): it refers to the relative position of the streets. For example, if street A crosses street B after the crossroad, it is a dis- placement, meaning that street A should cross street B before the crossroad. It should be pointed out that the length belongs to the property of a street as well. However, it cannot be used as a criterion extra. Because if a street is reconstructed too long, then the length will be taken into account in the criterion “Number of none ex- isting crosses and branches”. Otherwise, if a street is reconstructed too short, then it will be taken into account in the criterion “Number of correct crosses and branches”. 3.2 Weighting of the Criteria All of the nine criteria deal with crosses/branches (C1, 6, 7, 8, 9), street names (C2, 4), street shapes (C3), or street directions (C5). In this step, we weighted these four items with weights w1, w2, w3 and w4 by involving blind people. The weights are speci- fied as follows: (1) At first, we had to find out if it is necessary to give the 4 items different weights. According to the relevance for getting an overview of an environment, 21 blind people were invited to rank the 4 items in order. Two different items can have the same ranking. Then, the four items are ordered by the average of the rankings from blind people: crosses/branches (rank at 1), street names (rank at 2), street shapes (rank at 3), and street directions (rank at 4). (2) Then, we tested if the rankings of the items are significantly different. The fre- quency distribution of the four items and rankings was displayed in a 4x4 - contin- gency table. The items were paired (crosses/branches with street names, street names with street shapes, and street shapes with street directions) and tested by using the Chi-square test for homogeneity. The test showed a significant difference between crosses/branches and street names, street shapes and street directions, but not be- tween street names and street shapes (df = 3, p=0.95, χ23;0.95 = 7.81). However there is a significant difference between street names and street directions. Therefore, street names and street shapes should get the same weight. (3) Finally, we calculated the weights w1, w2, w3, and w4. The weights are calculated according to the frequency of rank 1 and 2 of the four items. However, the frequency of rank 1 should be given more weight than the frequency of rank 2. Therefore, the frequency of rank 1 should be multiplied by a factor f (1 < f < 3). According to the rankings from blind people, the item crosses/branches should get the highest weight. If the frequency of rank 1 is multiplied by 3, the item street names will get the highest weight. f should be therefore less then 3. In our study, we set f equal 2 (see table 1). Table 1. Weighted frequency distribution of the four items (ai) of rank 1 and 2 (H: frequency) items a1 a2 a3 a4 ∑ rank 1 H11*2 H12*2 H13*2 H14*2 H1. *2 2 H21 H22 H23 H24 H2. ∑ H11*2+ H21 H12*2+ H22 H13*2+ H23 H14*2+ H24 H1. *2+ H2. 40 Proceedings SKALID 2012 A Quantitative Evaluation Approach for Cognitive Maps of Blind People The weights were calculated as shown in equation (1). So we got following results: w1 = 0,31, w2= w3 = 0,28, w4 = 0,13. w (ai) = (H1i * 2 + H2i)/(H1. * 2 + H2.) (1) 3.3 Choice of Method for Quantification For quantifying the criteria we chose the approach weighted Euclidean distance. i d ( x, y ) = ∑ w *(x − y ) = w *(x − y ) +Λ + w *(x − y ) i =1 i i i 2 1 1 1 2 i i i 2 (2) where x = (x1, x2, … xi), y = (y1, y2, …, yi), wi = weights, and d(x, y) = the distance from point x to y. In our case, xi is the i-th value of the original map (x7, 8, 9 = 0), while yi is the i-th value of the reconstructed cognitive map, and wi is the specified weight for i- th criteria (see table 2). d (x, y) is the distance between the original map and cognitive map. In other words, we compare these two maps by measuring the distortion be- tween the original map and the cognitive map. It gives a measure of how similar cog- nitive map to the original map actually is. The smaller the value, the more similar the cognitive map to the original map is. Table 2. Criteria and their weights for evaluating cognitive map criteria weight Original Cognitive map map Number of correct street segments 0.31 x1 y1 Number of correct remembered street names 0.28 x2 y2 Number of correct street shapes 0.28 x3 y3 Number of correct assigned street names 0.28 x4 y4 Number of correct street direction 0.13 x5 y5 Number of correct crosses and branches 0.31 x6 y6 Number of none existing streets 0.31 0 y7 Number of none existing crosses and branches 0.31 0 y8 Number of displacement of streets 0.31 0 y9 As mentioned, if we do not want to evaluate how easy blind people can remember the street names with the help of technological aids, we can set its (C2) weight 0. 4 Conclusions This approach has been developed within a study in which the arrangement of streets was tested with different technological aids. There were no landmarks on the original map. It is not intended for evaluating cognitive maps according to the structure of buildings such as an airport. In this case the location of landmarks should be tested, Proceedings SKALID 2012 41 Miao & Weber the criteria have to be extended and the weight specified. In addition, we also found out that the blind people weighted the 4 items in matters of wayfinding differently than in matters of getting an overview of an environment. This indicates that if route knowledge is tested, we need other criteria and other weights for them. Acknowledgements We would like to thank all of the blind participants and collaborators for spending their time on our study. We would also like to thank Limin Zeng for the preparation of the study and Michael Schmidt for useful discussions. References: 1. Lloyd, R., Cammack, R.: Constructing cognitive maps with orientation biases. In: Portu- gali, J. (eds) The Construction of Cognitive Maps. pp. 187-213, Netherlands (1996) 2. Kitchin, RM.: Cognitive maps: What are they and why study them? Journal of Environ- mental Psychology, 14, 1-19 (1994) 3. Spindler, M., Weber, M., Prescher, D., Miao, M., Weber, G.: Translating Floor Plans into Directions. In Proceedings of the International Conference on Computers Helping People With Special Needs (ICCHP’12) (2012) 4. Daunys, G., Lauruska, V.: Sonification System of Maps for Blind – Alternative View (Vol. 2). Universal Access in Human-computer Interaction: Intelligent and ubiquitous (2009) 5. Zeng, L.; Weber, G.: Audio-Haptic Browser for a Geographical Information System. In: K. Miesenberger et al. (Eds.): Proceedings of the International Conference on Computers Helping People With Special Needs (ICCHP'10), LNCS 6180, 466-473 (2010) 6. Kitchin, R.M., Jacobson, R.D.: Techniques to Collect and Analyze the Cognitive Map Knowledge of Persons with Visual Impairment or Blindness: Issues of Validity. Journal of Visual impairments & Blindness, July-August. pp.360-376 (1997) 7. Lahav, O., Mioduser, D.: Blind Persons’ Acquisition of Spatial Cognitive Mapping and Orientation Skills Supported by Virtual Environment. In Proceedings of 5th International Conference on Disability, Virtual Reality and Associated Technologies. UK (2004) 8. Ohuchi, M., Iwaya, Y., Suzuki, Y., Munekata, T.: Cognitive-Map Formation of Blind Per- sons in a Virtual Sound Environment. In Proceedings of the 12th International Conference on Auditory Display, London, UK (2006) 9. Waterman, S., Gordon, D.: A Quantitave-Comparative Approach to Analysis of Distortion in Mental Maps, Annals of the Association of American Geographers. pp. 326-337 (1984) 10. Peake, S., Moore, T.: Analysis of distortions in a mental map using GPS and GIS. Pre- sented at SIRC 2004-16th Annual Colloquium of the Spatial Information Research Centre, University of Otago, Dunedin, New Zealand, November 29-30 (2004) 11. Passini, R., Proulx, G., Rainville, C. The spatio-cognitive abilities of the visually impaired population. Environment and Behavior 22, pp. 91-118 (1990) 42 Proceedings SKALID 2012