This paper focuses on the impact of users' mental model for the performance when interacting with mobile devices. Investigating the interrelation between the quality of the mental model and the performance, users had to accomplish standard applications with cell phones (study 1) and a PDA (study 2). Navigation routes were individually analyzed and the respective effectiveness and efficiency of menu navigation was determined. After task completion, users were requested to visualize their mental representation of the device' inherent structure. To learn the usability demands of a broad user group, younger and older adults were examined. The outcomes clearly demonstrated the importance of a correct mental model for the navigation performance. Users who had formed the correct mental representation of the respective device structure were extremely advantaged compared to users who did not have an adequate representation. In addition, it was found that the ageing disadvantage can be diminished by the correct mental model.
Mobile technologies have proliferated into most professional and private areas. Especially, small interface devices are ubiquitous and can be characterized as essential technical devices in nowadays societies. While in former times the usage of information technology was more or less voluntarily and restricted to a sophisticated user group, in the meanwhile all user groups are addressed. Further, the usage of technical devices has become an essential requirement in many working environments. Even though the mobile technology is one of the fastest industrial sectors ever, considerable friction losses with respect to usability concerns are to be noticed. [e.g. 1, 9]. Thus, the usability of small-screen devices depicts a challenging requirement for manufacturers and designers. One of the major problems associated with small screen devices is the disorientation within the menu [e.g. 2, 3, 4, 9, 10]. Users often not know where they are, where they came from and also, where they have to go next [e.g. 6, 8, 11]. However, the specific reasons for the disorientation problem are still not fully understood.
One approach is to address specific user characteristics, which account for disorientation and, as consequence, the performance decrements [e.g. 1, 9,11]. As such, technical expertise, age, emotional factors, and cognitive abilities are discussed as crucial mediators for disorientation. Another approach is to explore users’ mental representations of the devices’ inherent structure [e.g. ,5,6,7]. It is assumed that, especially due to the display’s inability to show more than a small cantle of the menu, users may fail forming an appropriate mental representation – a cognitive map – of the menu structure when navigating through the various functions and get lost. The mental representation helps users to harmonize their relative position in the menu and, at the same time, to keep the menu structure in mind while navigating through the system [e.g. 9]. Thus, having an appropriate model and a structural concept of the mental “room” that has to be navigated through, performance is distinctly supported. In small screen devices, as the cell phone and the PDA, where the overall structure of the menu is not transparent, the adequacy of a cognitive map and a proper mental model is even more crucial [e.g. 9]. However, even if there are basic similarities between both © 2006 for the individual papers by the papers' authors. Copying permitted for private and scientific purposes. Re-publication of material on this page requires permission by the copyright owners. mobile devices, there are important differences, though. In contrast to the exclusively hierarchically structured data in cell phones, PDA-applications provide both, network or hypertext character as well as hierarchical menu parts. The information in network structures is represented by nodes, which are interconnected by links. Although the navigational control in network-like systems should be higher because users can directly select the nodes they want to visit- the non-linear structure of hypertext and the number of cross-references is likely to provoke even more disorientation and hamper navigation performance (e.g. 8].
The present study addresses the interrelations between the adequacy of users mental representations and performance when interacting with cell phones and a PDA device.
Independent variable was the users’ age. Sixteen young adults (20-32 years) and sixteen older adults (50-64 years) participated in the experiment. Controlling for confounding effects stemming from different educational levels, only users with academic education were selected. The subjects processed four standard cell phone applications (calling, sending a text message, editing an entry in the phone directory and hiding the transmission of one’s own number in the phones’ display). The tasks were processed on computer emulations of two widespread real phones, the Nokia 3210 and the Siemens C35i (formerly mass models). While the software exactly mirrored the real devices, the simulations were run on a computer and were displayed on a touch screen. A logging tool enabled us to study in detail the individual navigation routes (on the key stroke level).
After task completion, users’ mental representation of the cellular phone’s menu was assessed by the card sorting technique. As a typical application, the task in which the transmission of the phone number was to be set off was selected. The tasks required the users to navigate through 22 different functions on their way to the target function. Accordingly, the users were given 22 cards with the original function names and category headers on it and were asked to arrange the cards on a table according to how they remembered having seen them in the menu. In case they did not remember, they were requested to arrange the cards according to how they felt the menu structure was like. Dependent variables were the adequateness of the mental representation of the menu structure. With respect to navigation performance, the effectiveness (number of tasks solved) and efficiency (the time on task) were analyzed.
Did the users have an adequate representation of the overall structure of the phone menu? And were the users aware of the original menu depth? The analysis showed that four out of 16 older users did not arrange the cards in a hierarchical structure. Instead, one user arranged the cards in clusters of three, without interconnections between clusters, possibly, because he or she simply mirrored the arrangement of menu functions, which were displayed (always three menu items were presented at a time). Further, two participants had no idea at all of how to arrange the cards because they were not able to imagine what was meant by the functions on the cards or how a menu could be organized at all. In the younger group, all 16 participants laid a hierarchical menu structure.
To analyze the impact of having a mental representation of the menu’s tree structure on the ability to effectively and efficiently interact with the device, the performance of those who laid a hierarchical structure in the card sorting task and those who did not was compared. First, looking at the older group, it was found that the 12 users with a hierarchical menu representation (tree structure) solved on average 80.2% of the tasks (3.2 from 4 tasks) while the others performed significantly less (65.6% (2.6 tasks), t(14) = 2.43; p < .05). Second, when considering all 32 participants, the difference between the two age groups was even bigger: Users who had a correct mental representation solved 89.7% (3.6 tasks), thus 24.1% more than those who were not aware of the tree-structure (t(30) = 3.96; p < .001) (Figure 1, left). Moreover, effects of an adequate mental model were also found for the time users spent on tasks. A next analysis was concerned with the menu depth that was reflected by participants’ card formations. (Table 1) Only one person of the older group laid the correct menu depth (4 levels), whereas six of the 16 younger users did. Comparing the group with the correct menu depth with the rest of the participants it was found that they had not only a higher effectiveness (96.4% vs. 84%, t (30) = 2.24; p < .05), but also a higher efficiency (109.9 sec vs. 234.4 sec, t(30) = 2.95; p < .01).
Concluding so far, two key results of study 1 should be kept in mind. First, the adequacy of the mental representation formed while processing the tasks was related to navigation performance. Second, the knowledge about the hierarchical nature of cell phones’ menu structure was found to be essential for a high effectiveness and efficiency.
The main goal of the second study was to validate the findings of study 1 with another device, mainly out of two reasons. First, the significance of having a correct mental model for the navigation performance should be replicated in order to strengthen the assumed relationship. Second, with respect to the requirements of cross-platform consistency, PDAs are a very interesting platform, as they have basic accordance with cell phones as well as discrepancies. In both mobile devices, users have to navigate through a quite complex menu, and –due to the limited screen space, only few functions can be seen at a time. However, in contrast to cell phones, which are completely hierarchically structured, the PDA has a mainly network-like data structure even though there are also hierarchical parts. Interacting with a PDA thus requests users to have a qualitatively different mental representation which should affect also the way of wandering around in the PDA menu.
Independent variable was the users’ age. 32 adults, 16 young (18 - 27 years.) as well as 16 older adults (50 - 69 years) participated. All participants were PDA novices.
Participants were requested to complete typical tasks of standard PDA applications. Participants worked on four prototypic tasks and had to enter (task 1 and 3) and to postpone (task 2 and 4) appointments in the digital diary. To enter a new appointment into the digital diary, a minimum of 15 steps was necessary. The postponement of an appointment required 8 steps. The PDA (iToshiba Pocket PC e740, system software Windows CE) was simulated as a software solution and displayed on a touch screen. The software prototype exactly corresponded to the real device. After completing the four experimental tasks, the users’ mental representation of the menu structure of the digital diary was assessed. Participants were offered drawings of five sample principles and asked to choose the most adequate structure principle (Figure 2).
Dependent variables were the adequateness of the mental representation of the menu structure. With respect to PDA navigation performance, the effectiveness (number of tasks solved) was analyzed. As efficiency measures, the time and the number of detour steps were determined. The number of detour steps was defined as the difference between the number of steps actually carried out and the minimum of steps that were necessary to solve the task.
It was a basic question if participants were able to build up a correct representation of the mental room of the PDA in which they have been navigating through while completing the tasks. Moreover it was of central interest to link the adequateness of the models to performance outcomes. As the menu of the PDA’s digital diary had a mixture of hierarchy and network, the answers “Hierarchy”, “Network” and “Hierarchy/Network mixture” were judged as adequate models. Accordingly, the answers “Onion”, “Pyramid” and “Chain” were judged as inadequate. In Table 3, the key results are given.
Not all users were able to build up a mental model: 6.25% of the younger and 25% of the older participants reported to have no representation at all. With respect to age effects, statistical testing revealed that young rather than older adults were able to name an adequate mental model (__ = 4.6, p < 0.05). For the older group, 37.5% had a correct representation, while 37.5% selected an incorrect mental model (pyramid, chain). Even when the younger group’s representation was much better (75% correct answers), it is quite remarkable, that still 18.8% of the younger group chose an inappropriate model. Interestingly, none of the users chose the onion-shaped model, the only structure without connections between the concentrically arranged levels. Apparently, the possibility of wandering around within the menu is not compatible with this model. A next analysis was concerned with the question if the users that had no representation of the PDA menu after the completion of the PDA applications differ from those who reported to have one. Figure 3 shows that users having no idea of the PDA menu structure were in fact disadvantaged when comparing to those users that reported to have a model (even though incorrect).
Another analysis was concerned with the specific type of model that was chosen by participants. Among the incorrect models, the pyramid and the chain-like model were quite frequently chosen as supposedly adequate (pyramid: 6.3% of the younger and 12.3 % of the older group; chain: 12.5% of the younger and 25% of the older users). Is the adequateness of the mental representation related to performance outcomes? Figure 4 shows that this is indeed the case. Regarding the adequate models, one model proved to be particularly beneficial and another one revealed to be particularly disadvantageous for navigation performance. Users with a network type representation of the PDA-menu were distinctly more effective and efficient than users that had an inappropriate model or even no mental image at all. Interestingly, even though the PDA is also hierarchically structured, it was the hierarchy model that turned out to be counterproductive, especially for older users. As can be seen from Figure 4 (right), it provoked many detour steps, accompanied by a lengthening of processing time and a smaller number of tasks solved successfully. Even if it is not in the central focus of this analysis, the absolute number of detour steps in both groups is quite remarkable, especially when taking into account that only 8 (in the postponement tasks), respectively 15 steps (in the entry tasks) were necessary to solve the tasks the most efficiently. Apparently, even standard applications on a PDA are not easy to solve, even for the younger group.
The present paper was concerned with the crucial question wehter users’ mental representation of how a technical device might be structured affects performance outcomes. Clearly it was demonstrated, that this assumption is true, for the cell phone as well as for the PDA. For both devices it was found that the appropriate knowledge of how a device is “spatially” structured, was extremely advantaging navigation performance, thus resulting in a higher effectiveness and a reduced disorientation. It is highly probable that the benefit of the mental representation is especially important for small screen devices. Characteristically, they have a rather complex menu, which is most of the time hidden from sight as the small screen restricts the amount of information displayed per screen. When knowing the device’s structure, much lesser disorientation and detouring was found to occur while navigating through the maze of functions.
Crucial for the performance using a cellular phone is the knowledge that functions are arranged hierarchically and that the menu consists of several levels in different depths.. Accordingly, users having the correct model of the cell phone showed a higher navigation performance.Furthermore, users, which had a correct representation of the path to be taken and also knew salient features of the correct menu branch showed higher effectiveness and efficiency of menu navigation in mobile phones.
Crucial for the performance of the PDA is the knowledge that the device is structured according to a network with nodes and interconnections. Users who reported to have a network-like structure as an underlying model were specifically advantaged over those who had an inappropriate representation.With respect to requirements of cross platform design, one cautionary note has to be considered. Cell phones and PDAs do share basic characteristics (small screen, mobile devices), but there is a basic difference with respect to the menu structure. Even though PDA devices are mainly network-structured, actually, the PDA menu structure is a mixture out of network and hierarchy, and thus both models are correct. However, it was found that one of the correct models proved to be particularly beneficial and another one particularly disadvantageous for navigation performance. It was the network type that was most helpful as an underlying mental model, whereas the hierarchy was extremely impedimental. Apparently, the knowledge of wandering through a network seems to be prototypical or characteristic for the PDA and forms the navigation accordingly. In contrast, the hierarchy as a major representation of the PDA was counterproductive and resulted in highly increased detour steps. Given that mobile phones and computer hard discs are exclusively hierarchically structured and that these structures are well known by many users, the ergonomic demands with respect to training matters become apparent. Users who are familiar to hierarchical structures have to face enormous switching costs when they simply transfer their understanding from one technical device to another.
This outcome is of ergonomic significance as it shows that technical devices can be used by all users rather successfully if they are supported adequately. Concluding, it should be a main goal for designers and trainers to provide structural knowledge of the device structure and to support users by forming the adequate mental representation.
Arning, K. & Ziefle, M. Cognitive and personal predictors for PDA navigation performance. Comparing younger and older adult novices. Paper submitted to Human Factors.
Arning, K. & Ziefle, M. Barriers of information access in small screen device applications: The relevance of user characteristics for a transgenerational design. Paper submitted. 9th ERCIM "User Interfaces For All". Special Theme ”Universal Access in Ambient Intelligence Environments".
Bay, S. & Ziefle, M. Effects of menu foresight on information access in small screen devices (2004). 48th Annual meeting of the Human Factors and Ergonomic Society, 1841-1845. Bay, S. & Ziefle, M. Landmarks or surveys? The impact of different instructionson children’s performance in hierarchical menus structures. Computers in Human Behavior. Paper under revision.
Conklin, E.J.. Hypertext: An introduction and survey. IEEE Computer, 20, (1987), 17-41. Gischer, G. The Importance of Models in Making Complex Systems Comprehensible. In: M.J. Tauber & D. Ackermann (Hrsg.): Mental Models and Human Computer Interaction, Elsevier (1991), 3-36.
Kieras, D.E. and Bovair, S. The role of a mental model in learning to operate a device. Cognitive Science, 8(3): (1984), 255-273.
Lin, D.M. Age differences in the performance of hypertext perusal, Proceedings of the Human Factors and Ergonomics Society 45th Annual Meeting, (2001). 211-215.
Sein, M., Olfman, L., Bostrom, R., and Davis, S. Visualization ability as a prediction of user learning success. International Journal of Man-Machine Studies, 39, (1993), 599-620. Ziefle, M. and Bay, S. Mental models of Cellular Phones Menu. Comparing older and younger users. In S. Brewster and M. Dunlop (eds.). Mobile Human computer interaction (2004), Berlin: Springer, 25-37 Ziefle, M. and Bay, S. (in press). How to overcome disorientation in mobile phone menus: A comparison oftwo different types of navigation aids. Human Computer Interaction.