Interaction with user interfaces only with usage of hands and bodies was a few years back still science ction, but now present reality. Touchless interfaces are slowly becoming mainstream and therefore it is of crucial importance to address them in a concise and standardized way. Usability is an important factor in all software quality models and a key factor during development of interactive applications. The objective of this paper is to address usability challenges we are facing during touchless application design. A conceptual usability case study is proposed with new human-computer interaction factors in mind. Factors like e ciency, ease-of-use, pleasure, fatigue, naturalness, smoothness, responsiveness and accuracy are identi ed and related to usability scenarios.
UX must not be confused with Usability as usability is a narrower term. Usability is considered the ability of the user to use the thing to carry out a task successfully: “The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” [9241-11 1998]. UX looks at the individual’s entire interaction with the 6:40
thing, as well as thoughts, feelings, and perceptions, that result from that interaction [Tullis and Albert 2013].
When discussing usability, it is important to distinguish between summative and formative usability evaluation. Formative usability has strong ties to the practice of iterative design — building something, checking to see where it could be improved, improving it, and trying again. Summative evaluations emphasize the importance of effectiveness and efficiency in the context of use and the subjective metric of satisfaction [Lewis 2014]. The main goal of summative evaluation study is to evaluate whether people can use a product for its intended purpose effectively, efficiently, and with a feeling of satisfaction. The formative evaluation study reveals the presence of usability when there is absence of usability problems.
Nielsen [1993] defined five attributes of usability: (i) efficiency (relative to the accuracy and completeness with which users achieve goals), (ii) satisfaction (freedom from discomfort and positive attitudes towards the use of the product), (iii) learnability (the system should be easy to learn so that the user can rapidly start getting work done with the system), (iv) memorability (the system should be easy to remember so that the casual user is able to return to the system after some period of not having used it without having to learn everything all over again) and (v) errors (the system should have a low error rate, so that users make few errors during the use of the system and that if they do make errors they can easily recover from them).
Harrison et al. [2013] combined the attributes from ISO 9241-11 [1998] and Nielsen [1993] and added another interesting factor, that is cognitive load. Cognitive load refers to the amount of cognitive processing required by the user to use the application. Harrison’s PACMAD model (figure 1) derives from mobile devices, where he argues that users of mobile applications may be performing additional tasks, such as walking, while using the mobile device. In our opinion the model could be well applied to touchless user UIs as those require from a user to combine usual interaction tasks with speaking and moving. In touchless UIs user needs to use predefined gestures and voice control simultaneously while executing wanted tasks and this can raise the mental load. Therefore it is important that the aspect of cognitive load is included in usability studies of touchless interfaces. More about touchless UIs is described in section 3.
All previously mentioned models recognise three factors that can affect usability of an application. These are: (i) user (users physical limitations, their knowledge and previous experience), (ii) task (goal the user is trying to accomplish) and (iii) context of use (environment in which the user will use the application including physical location and interaction with other people and objects). All of these are eligible for usability of touchless UIs.
Use experience has to be measured in order to evaluate whether it is good or bad, to discover problems and opportunities for improvement, or to compare different UIs. Many metrics are available for the 2. Comparing Products 3. Evaluating frequent use of the same product 4. Evaluating navigation and/or information architecture 5. Increasing awareness 6. Problem discovery 7. Maximizing usability for a critical product 8. Creating an overall positive experience 9. Evaluating the impact of subtle changes 10. Comparing alternative designs s s cce em Su iT sr sakT sakT rroE x x x x x x x x x x x x x x x y t i l y i ifccefinE -reaabnL x x x x x x x x x x x x x x x x x x x x x 11. Cognitive load x x
The table is adapted from [Tullis and Albert 2013] and is supplemented with cognitive load scenario. evaluation of UX. Metrics add structure to the design and evaluation process, give insight into the findings and provide information to the decision makers, they offer a way to estimate the number of users likely to experience a problem, are a key ingredient in calculation of ROI as well as can reveal patterns that are difficult or even impossible to see. For that UX metrics need to be observable (directly or indirectly), quantifiable (turned into a number or counted in some way), and they have to measure some aspect of the user experience [Tullis and Albert 2013].
Many different metrics are available for different evaluations. When evaluating UX one has to consider the goals of the study, the technology that is available to collect the data, and budget as well as time that are available for conducting evaluation. In table I we present ten common usability scenarios and the metrics that may be most appropriate for each, adapted from [Tullis and Albert 2013]. We added 11th scenario: cognitive load, as we believe it is important for evaluation of touchless UIs.
For details about the scenarios and metrics see [Tullis and Albert 2013]. At this point we describe only the newly added scenario.
Cognitive load theory (CLT) describes the relationship between the capacity of working memory and the cognitive demands of a particular task [Anderson 2012]. It is based on an idea, that cognitive capacity in working memory is limited and that if a mental task requires too much capacity knowledge acquisition and reasoning will be hindered [Jong 2009]. One solution to this is to design a UX that optimizes the use of working memory capacity and avoids cognitive overload.
CLT distinguishes between three types of cognitive load: (i) intrinsic (the inherent difficulty of the problem at hand), (ii) extraneous (generated by the representation of the content presented to the user for interpretation and action) and (iii) germane (imposed by learning a new task) [Anderson 2012]. Extraneous, intrinsic and germane cognitive load are modelled to be additive: a reduction of extraneous cognitive load frees working memory capacity that can be used for germane learning processes [Hollender et al. 2010].
The amount of extraneous load due to software use is influenced by the complexity of the software, a suboptimal software design according to traditional usability goals, and the expertise of the learner with regard to the use of the software. Load can be lowered by designing highly usable software applications and by training learners to use the software [Hollender et al. 2010].
Cognitive load can be measured in different ways. Most common measurements include task completion time and accuracy, NASA-TLX test (a survey with subjective responses), EEG-based measurement (determining cognitive load magnitude by analysing the temporal, spectral, and spatial patterns of brain activity), pupil dilation, eye tracking and blinking measurement, galvanic skin response, and heat flux [Anderson 2012; Chen et al. 2011; Haapalainen et al. 2010].
Touchless user interfaces require devices, that can either execute or sense interactive behaviour, where the interaction happens without mechanical contact between the human and any part of artificial system. Touchless interaction can be multimodal, in which case the interactive behaviour produces simultaneous events in the visual modality (colour, form, or position change), in the auditory modality (speech, sounds), or in the olfactory modality (odors) [de la Barre´ et al. 2009]. Voice control can be realized by recording the voice with a microphone and processing it through dedicated algorithms. Body gestures can be detected in different ways from using wearable sensors to environmental sensors [Jalaliniya et al. 2013].
Touchless UIs are suppose to remove the burden of physical contact with an interactive system and make interaction pleasurable [de la Barre´ et al. 2009]. To achieve that one must carefully combine characteristics of physical and digital world and must try to produce a solution with ”natural” HumanComputer Interaction (HCI). One must also consider the fact that hand movement is not equal to gesture, as gesture is a body movement which is being performed with the perceivable intention to express something [de la Barre´ et al. 2009]. For the user to learn all the gestures and voice controls, that are implemented in a certain UI, a built-in tutorial is a welcomed feature in touchless UIs.
According to Madan and Dubey [2012] usability is the most widely used concept in software engineering field and defines the software system’s demand and use. Demand for software quality and usability is increasing and there are several usability models, that can be used and tailored to our needs (for more on usability models see Madan and Dubey [2012]).
The specifics of HCI need additional factors that will help us to successfully implement a usability study with gesture interaction in mind. Touchless HCI software interacts with users by using gestures and voice commands that are tied to gesture recognition engine. Factors like efficiency, ease-of-use, pleasure, fatigue, naturalness, smoothness, responsiveness, and accuracy should be investigated in detail. Investigation should be based through simple and complex tasks. According to Farhadi-Niaki et al. [2013] gesture based systems cause more fatigue and appear less natural then finger gestures, however factors such as time, overall satisfaction, and easiness were not affected.
Gesture recognition is a rather new field of HCI, so it is normal that problems are present. The main issues are usually: (i) lack of standardization gestures, (ii) lack of cues, (iii) inability to discover operations and (iv) requiring memorization of the player (head memory). These problems were classified by OPERATING ROOM
DISINFECTION ROOM ADORA Norman et al. [2010] where he also argued that gesture based interfaces are a step backwards in usability due to lack of: (1) Visibility - available gestures at each moment are not clear, neither speeds or precision of the movement. (2) Feedback - there is not enough feedback for the user during the gesture. The user does not know if the action was because of the correct gesture or something failed in the detection. (3) Consistency and Standards - there are no standards that would define consistency of HCI menus or standardized gestures. (4) Discoverability - the user must know all the gestures in advance (head memory), or there is no feedback that would help him to connect gestures with application outcomes. (5) Reliability ”...Accidental gesture activation is common in gestural interfaces, as users happen to touch something they didn’t mean to touch. Conversely, frequently users do intend to touch a control or issue a gestural command but nothing happens because their touch or gesture was a little bit off. Since gestures are invisible, users often do not know that they made these mistakes” [Norman et al. 2010].
This is especially the case with Microsoft Kinect camera.
Therefore all the above mentioned issues need to be properly addressed in UX evaluation after they have been carefully studied, planned, and implemented in a system.
ADORA1 is an interactive physician’s assistant, that enables presentation of information about a patient before and during surgical procedures. It offers a comprehensive and integrated natural user interface experience for physicians. With its use of contact-free interaction it shortens the duration of surgeries and indirectly affects the environmental and economic aspects of healthcare.
Touchless methods of HCI (gestures and voice support) have been integrated into ADORA solution during development. Before using touchless assistant physicians had to leave the patient and the sterile field in order to access critical patient data, that was accessible through the wall PC. In order to 6:44 continue with the surgery they had to disinfect, change gloves, gown and mask), memorize the picture and then return to the patient (see figure 2). This takes time and causes additional stress for the doctor. ADORA delivers patient data in a touchless manner, so that the surgeon can stay by the patient while viewing the data. Touchless interaction was developed with the help of a Microsoft Kinect sensor.
The solution was designed together with the users, physicians. One of the main requests was a minimal set of gestures, so that they can use the solution in the most possible natural way. As the surgeon has always active hands during surgery, all the gestures and their sets are designed in a one-handed way. Table II shows most common used gestures in ADORA.
The most important thing during planning a usability study is understanding users and the goals they are trying to accomplish. The user and his expectations from the solution define core building blocks of the study. In our case study a user is a physician in an operating room who is performing long complex surgeries. His personal goal is to successfully complete a surgery with as little distraction possible.
ADORA solution is already implemented and finalized, therefore a summative usability study is in order. Tullis and Albert [2013] define summative usability as answers to the following questions: (i) Did we meet the usability goals of the project?, (ii) What is the overall usability of the product?, (iii) How does our product compare against the competition? and (iv) Have we made improvements from one product release to the next?. Main goal of summative usability study is to evaluate how well a product or piece of functionality meets its objectives.
Experiment design reflects the scenarios selected from table I and the factors that are specific for HCI and gesture interfaces. The experiment will consists of three parts. Pre-test survey: to gather participant’s data that will be used during the main part of experiment. Practice and test sessions: the core of the experiment will consist of tasks completion rate, possible errors and factors related to them. The final part will be executed with the post-task questionnaire.
In the main part of the evaluation, each participant will be instructed to perform a defined task using gestures and voice commands to control ADORA solution with the help of Kinect sensor. Tasks will include normal operations that surgeons are otherwise used carrying out on a wall PC.
Task were based on the Ux scenarios in table I. Scenario one, four and seven were addressed with selected tasks and post-task questionnaires. At this point cognitive load scenario is not included.
Tasks will be split into simple and advanced sets. For example a simple set of tasks would be selecting a surgery (pushing a button with gesture) and loading a set of medical images for the patient (a combined set of gestures). Advanced set of tasks will include changing a view, loading multiple image series into different views, combining the views and manipulation of medical images such as point based zoom and image adjusting (combined operation of brightness and contrast merged into one gesture). Table III list a set of tasks surgeon will be performing during experiment.
There will be two groups of participants (surgeons). First group will be educated about functionalities and gestures that are available in ADORA, while the second group will identify gestures with the help of a tutorial that is available in the solution. Group data will be used to determine if pre-education of the users affects the overall experience, or if the built-in tutorial is enough to gain needed operational knowledge.
During task execution data will be gathered according to Fitts’ Law Test [Zhai 2004] in terms of completion time, errors and throughput. Completion time will be measured in three iterations and then average will be calculated of the multi-directional Fitt’s law tasks. A post analysis of measured times will be done with Scheffe criterion for significance [Fleiss 1999]. Similar analysis will be made for throughput and error.
In the final phase, after task completion, users will be asked to complete a survey containing a Device Assessment Questionnaire suggested by ISO 9241-9 [2000]. All questions will be ranked with a 7-point Likert scale, from strongly agree to strongly disagree and will be fine-tuned with inclusion of HCI factors mentioned above. Only a combination of questionnaires, performance metrics, and inclusion of HCI factors can give a comprehensive and valid picture for several reasons: (i) users might be influenced not to honestly report their experience, (ii) a combination will give us more insight what system factors influenced noteworthy user ratings, (iii) for most of the uality of experience aspects described, there exist no valid and reliable metrics for the case of gesture interfaced systems, so a mixture can help to interpret the results better [Wechsung et al. 2012].
In this paper terms and definitions of Usability, User experience and Human Computer Interaction were challenged all together. There are already many models that define usability and user experience, but none of them has yet been tailored to the needs and challenges of gestural interfaces. While gestural interfaces are yet to become mainstream, a lot of them can be found in the gaming world where touchless devices have already been well accepted.
Having a natural mapping between the body actions and the reactions on the screen gives very positive reactions and in order to keep the positive interaction flow with the users, further research is needed in the gestural interfaces domain. A standardized framework is needed, that will define consistency and standards for evaluation of gestural interfaces and that will include the aspect of cognitive load. ISO 9241-11. 1998. ISO 9241-11:1998 - Ergonomic requirements for office work with visual display terminals (VDTs) – Part 11:
Guidance on usability. (1998). http://www.iso.org/iso/home/store/cataloguen tc/cataloguen detail.htm?csnumber=16883 ISO 9241-210. 2010. ISO 9241-210:2010 - Ergonomics of human-system interaction – Part 210: Human-centred design for interactive systems. (2010). http://www.iso.org/iso/home/store/cataloguen tc/cataloguen detail.htm?csnumber=52075