Reducing Risk through Human Centred Design Nigel Bevan Professional Usability Services 12 King Edwards Gardens London W3 9RG, UK www.nigelbevan.com mail@nigelbevan.com ABSTRACT 3. Iterative system development and definition: cyclic refinements of requirements, solutions, and development The National Academy of Science’s report on Human-System plans. Such iteration helps projects to learn early and Integration in the system development process (NAS HSI report) efficiently about operational and performance requirements. [12] explains how human needs can be integrated into system 4. Concurrent system definition and development: that includes design using an incremental systems engineering development concurrent engineering of requirements and solutions without process that continually assesses risks at each phase of the system waiting for every requirement and subsystem to be defined. development. This paper suggests how appropriate Human 5. Risk management – risk driven activity levels and anchor Centred Design (HCD) methods can be selected to mitigate risks point milestones. The level of detail of specific products and to project success. processes should depend on the level of risk associated with them. Principles 2, 3 and 4 are consistent with approaches to human 1. RISKS IN SYSTEMS DEVELOPMENT centred design, such as recommended in ISO 13407. The NAS HSI report points out that the ultimate goal of system The other two principles (Stakeholder satisficing and Risk development is to produce a system that satisfies the needs of its management) provide a means to determine which human centred operational stakeholders (including users, operators, design activities and methods are needed in a project to be administrators, maintainers and the general public) within confident that the final system will be acceptable to the acceptable levels of the resources of its development stakeholders operational stakeholders. (including funders, acquirers, developers and suppliers). Operational stakeholders need a system that is effective, efficient This contrasts with existing approaches to human centred design, and satisfying [1]. Developing and delivering systems that satisfy which are commonly based on a one-size-fits-all methodology all of these success-critical stakeholders usually requires managing (e.g. [5], [14]) that may be justified by a cost benefit analysis to a complex set of risks such as usage uncertainties, schedule assess the potential business benefits of producing a more usable uncertainties, supply issues, requirements changes, and system [3]. uncertainties associated with technology maturity and technical The additional expenditure needed for human centred activities is design. often difficult to justify because the budget holder for project Boehm and Lane [4] suggest five principles for managing these development usually does not personally gain from longer-term risks: benefits such as increased sales or reduced whole life costs. 1. Stakeholder satisficing; identifying the success-critical Project managers are much more likely to be influenced by the stakeholders and their value propositions (what is offered at risks of not achieving stated project objectives. It is thus useful to what cost); negotiating a mutually satisfactory set of system recast the potential cost benefits of usability as risk reduction requirements, solutions, and plans; and managing proposed strategies. Table 1 restates the list of cost benefits in [2] as changes to preserve a mutually satisfactory outcome. potential project risks. 2. Incremental growth of system definition and stakeholder commitment: incremental discovery of emergent human- system requirements and solutions using such methods as prototyping, testing with users, and use of early system 2. HUMANCENTRED DESIGN capabilities. ACTIVITIES Looking for advice on which methods to use for human centred design can be bewildering. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are Ferré [6] analyzed the methods contained in six popular not made or distributed for profit or commercial advantage and that HCI textbooks and identified 96 categories of HCD copies bear this notice and the full citation on the first page. To copy techniques. Individual textbooks each contained between otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. 21 and 43 of these categories of technique: I-USED’08, September 24, 2008, Pisa, Italy Table 1. Risks mitigated by HCD Author Number of categories Constantine [5] 31 A: Increased development costs to produce an acceptable system Hix [7] 21 Mayhew [10] 31 • Not detecting and fixing usability problems early in the Nielsen [11] 25 development process Preece [13] 43 • Increasing the cost of future redesign or radical change of the Shneiderman [15] 29 architecture to make future versions of the product more usable ISO PAS 18152 contains an exhaustive list of 125 human systems • Increased costs due to unnecessary functionality (HS) activities that are needed for all aspects of systems • Increased costs due to additional documentation development. These were derived from an analysis of best • Product fails practice in human centred design in civilian and military systems. The categories of activity are: B: Web site usability: poor web sales HS.1 Life cycle involvement activities • Users cannot find products that they want to purchase HS.1.1 HS issues in conception • Users cannot find additional information (e.g. delivery, return and warranty information) HS.1.2 HS issues in development • Dissatisfied users do not make repeat purchases HS.1.3 HS issues in production and utilization • Users do not trust the web site (with personal information and HS.1.4 HS issues in utilization and support to operate correctly) HS.1.5 HS issues in retirement • Users do not recommend the web site to others HS.2 Integrate human factors activities • Web site fails to increase sales through other channels HS.2.1 HS issues in business strategy • Increased support costs HS.2.2 HS issues in quality management HS.2.3 HS issues in authorisation and control C: Product usability: poor product sales HS.2.4 Management of HS issues • Competitors gain advantage by marketing competitive HS.2.5 HF data in trade-off and risk mitigation products or services as easy to use HS.2.6 User involvement • Dissatisfied customers do not make repeat purchases or HS.2.7 Human-system integration recommend the product to others HS.2.8 Develop and re-use HF data • Poor ratings for usability in product reviews • Brand damage HS.3 Human-centred design activities HS.3.1 Context of use D: Poor productivity: risks to purchasing organisation HS.3.2 User requirements • Slower learning and poorer retention of information HS.3.3 Produce design solutions • Increased task time and reduced productivity HS.3.4 Evaluation of use • Increased employee errors that have to be corrected later HS.4 Human resources activities • Increased employee errors that impact on the quality of HS.4.1 Human resources strategy service • Increased staff turnover as a result of lower satisfaction and HS.4.2 Define standard competencies and identify gaps motivation HS.4.3 Design staffing solution and delivery plan • Increased time spent by other staff providing assistance when HS.4.4 Evaluate system solutions and obtain feedback users encounter difficulties In [12] Table 3-A-1, the HS activities in ISO PAS 18152 have been categorised by type of system development activity: E: Increased support and maintenance costs 1. Envisioning opportunities • Increased support and help line costs 2. System scoping • Increased costs of training 3. Understanding needs • Increased maintenance costs 4. Requirements 5. Architecting solutions 6. Life-cycle planning 7. Evaluation 8. Negotiating commitments 9. Development and evolution 10. Monitoring and control 11. Operations and retirement 12. Organizational capability improvement An elaborated version of the table is included as an annex to this paper. 3. SELECTING HUMAN CENTRED • have the expertise and resources to apply a wide range DESIGN METHODS of different types of methods. The steps needed to select human-centred methods for an individual project are thus: 5. REFERENCES 1. Identify the success-critical stakeholders. [1] Bevan, N. (1999). Quality in use: meeting user needs for quality, Journal of Systems and Software, 49(1), pp 89-96. 2. Identify which potential consequences of poor usability [2] Bevan, N. (2005). Cost benefits framework and case studies. affect the success-critical stakeholders. In [3]. 3. Assess the likelihood and impact of these consequences. [3] Bias, R.G. & Mayhew, D.J. (eds) (2005). Cost-Justifying 4. Identify which categories of HS activities can reduce the Usability: An Update for the Internet Age. Morgan risks. Kaufmann. [4] Boehm, B. and Lane, J. A. (2007). Using the Incremental 5. Identify which HCD methods in each category are most Commitment Model to Integrate System Acquisition, cost-effective. The alternative methods should be assessed Systems Engineering, and Software Engineering. CrossTalk, against criteria such as: October 2007. Available at: • To what extent will each possible method address the www.stsc.hill.af.mil/crosstalk/2007/10/0710BoehmLane.ht activities that have been identified as important? ml • How cost effective is each method likely to be, given the [5] Constantine, L. L. and Lockwood, L. A. D. (1999). Software time and effort required and constraints such as available for Use: A Practical Guide to the Models and Methods of skills, access to stakeholders and other users, etc.? Usage-Centred Design. Addison-Wesley, New York. [6] Ferré, X, Juristo, N, Moreno, A.M. (2004). STATUS The needs for usability evaluation in particular should be judged Deliverable D.6.6. Final results on the Integrated Software in the broader context of the relative importance of usability Process. Universidad Politécnica de Madrid. evaluation in relation to other HS activities. For example, when designing and developing for a new context of use, the major risks [7] Hix, D. and Hartson, H.R. (1993). Developing User might be associated with requirements, so that the majority of Interfaces: Ensuring Usability Through Product and HCD resources might be devoted to early life cycle activities Process. John Wiley and Sons. (which could include evaluation of early concepts and competitive [8] ISO 13407 (1999). Human-centred design processes for evaluation). interactive systems. ISO. [9] ISO PAS 18152 (2003). A specification for the process 4. CONCLUSIONS assessment of human-system issues. This paper suggests how HCD can be justified as part of [10] Mayhew, D. J. (1999). The Usability Engineering Lifecycle. systems development and how the most appropriate HCD Morgan Kaufmann methods can be selected on a project-by-project basis. [11] Nielsen, J. (1993). Usability Engineering. AP Professional, This will enable HCD resources to be used most effectively [12] Pew, R. W. and Mavor, A. (eds.) (2007). Human-System for individual projects. The author would be happy to Integration in the System Development Process: A New advise on or support the application of this approach to Look. National Academies Press. Available at: selecting HCD methods in a real development project. books.nap.edu/openbook.php?record_id=11893 [13] Preece, J., Rogers, J., Sharp, H., Benyon, D., Holland, S., The prerequisites for successfully using this approach Carey, T. (1994). Human-Computer Interaction. Addison include having usability experts in the development team Wesley who: [14] Schaffer, E. (2004). Institutionalization of Usability: A Step- • can convince the project of the specific risks associated by-Step Guide. Addison-Wesley. with poor usability; [15] Shneiderman, B. (1998). Designing the User Interface: • have sufficient experience to be able to select the most Strategies for Effective Human-Computer Interaction. cost effective HCD methods; and Addison-Wesley. Annex A. Examples of methods that can be used to support HS best practices Activity category Best practices for risk mitigation with ISO 18152 clause reference HCD methods and techniques 1. Envisioning •Identify expected context of use of systems [forthcoming needs, trends and -Future workshop opportunities expectations]. -Preliminary field visit •Analyze the system concept [to clarify objectives, their viability and risks]. -Focus groups -Photo surveys -Simulations of future use environments -In-depth analysis of work and lifestyles 2. System scoping •Describe the objectives which the user or user organization wants to achieve -Participatory workshops through use of the system. -Field observations and ethnography -Consult stakeholders -Human factors analysis •Define the scope of the context of use for the system. -Context of use analysis 3. Understanding •Identify and analyze the roles of each group of stakeholders likely to be -Success critical stakeholder identification needs affected by the system. -Field Observations and ethnography a) Context of use •Describe the characteristics of the users. -Participatory workshop •Describe the cultural environment/ organizational/ management regime. -Work context analysis •Describe the characteristics of any equipment external to the system and the -Context of use analysis working environment. -Event data analysis •Describe the location, workplace equipment and ambient conditions. -Participatory workshops •Decide the goals, behaviours and tasks of the organization that influence -Contextual enquiry human resources •Present context and human resources options and constraints to the project stakeholders. b) Tasks •Analyze the tasks and worksystem. -Task analysis -Cognitive task analysis -Work context analysis c) Usability needs •Perform research into required system usability. -Investigate required system usability -Usability benchmarking -Heuristic/expert evaluation d) Design options •Generate design options for each aspect of the system related to its use and its -Early prototyping & usability evaluation effect on stakeholders. -Develop simulations •Produce user-centred solutions for each design option. -Parallel design (tiger testing) 4. Requirements •Analyze the implications of the context of use. -Define the intended context of use a) Context •Present context of use issues to project stakeholders for use in the development including boundaries requirements or operation of the system. b) Infrastructure •Identify, specify and produce the infrastructure for the system. -Identify staffing requirements and any requirements •Build required competencies into training and awareness programs. training or support needed to ensure that •Define the global numbers, skills and supporting equipment needed to achieve users achieve acceptable performance those tasks. c) User •Set and agree the expected behaviour and performance of the system with -Scenarios requirements respect to the user. -Personas •Develop an explicit statement of the user requirements for the system. -Storyboards •Analyze the user requirements. -Establish performance and satisfaction •Generate and agree on measurable criteria for the system in its intended goals for specific scenarios of use context of use. -Define detailed user interface requirements -Prioritize requirements (eg QFD) 5. Architecting •Generate design options for each aspect of the system related to its use and its -Function allocation solutions effect on stakeholders. -Generate design options a) System •Produce user-centred solutions for each design option. -Develop prototypes architecting •Design for customization. -Develop simulations •Develop simulation or trial implementation of key aspects of the system for the purposes of testing with users. •Distribute functions between the human, machine and organizational elements of the system best able to fulfil each function. •Develop a practical model of the user's work from the requirements, context of use, allocation of function and design constraints for the system. •Produce designs for the user-related elements of the system that take account of the user requirements, context of use and HF data. •Produce a description of how the system will be used. b) Human •Decide the goals, behaviours and tasks of the organization [that influence -Work domain analysis elements human resources] -Task analysis •Define the global numbers, skills and supporting equipment needed to achieve -Participatory design those tasks. -Workload assessment •Identify current tasking/duty -Human performance model •Analyze gap between existing and future provision -Design for alertness •Identify skill requirements for each role -Plan staffing •Predict staff wastage between present and future. •Calculate the available staffing, taking account of working hours, attainable effort and non-availability factor •Identify and allocate the functions to be performed Functional decomposition and allocation of function. •Specify and produce job designs and competence/ skills required to be delivered •Calculate the required number of personnel. •Generate costed options for delivery of training and/or redeployment •Evolve options and constraints into an optimal [training] implementation plan (4.3.5) •Define how users will be re-allocated, dismissed, or transferred to other duties. •Predict staff wastage between present and future. •Calculate the available staffing, taking account of working hours, attainable effort and nonavailability factor. •Compare to define gap and communicate requirement to design of staffing solutions. c) Hardware See a) System architecting. -Prototyping and usability evaluation elements -Physical ergonomics -Participatory design d) Software See a) System architecting. -User interface guidelines and standards elements -Prototyping and usability evaluation -Participatory design 6. Life-cycle •Develop a plan to achieve and maintain usability throughout the life of the -Plan to achieve and maintain usability planning system. -Plan use of HSI data to mitigate risks a) Planning •Identify the specialist skills required and plan how to provide them. b) Risks •Plan and manage use of HF data to mitigate risks related to HS issues. -HSI program risk analysis •Evaluate the current severity of emerging threats to system usability and other HS risks and the effectiveness of mitigation measures. •Take effective mitigation to address risks to system usability. c) User •Identify the HS issues and aspects of the system that require user input. -Identify HSI issues and aspects of the involvement •Define a strategy and plan for user involvement. system requiring user input •Select and use the most effective method to elicit user input. -Develop a plan for user involvement •Customize tools and methods as necessary for particular projects/stages. -Select and use the most effective methods •Seek and exploit expert guidance and advice on HS issues. -Customize tools and methods as necessary d) Acquisition •Take account of stakeholder and user issues in acquisition activities. -Common Industry Format e) Human •Implement the HR strategy that gives the organisation a mechanism for resources implementing and recording lessons learnt •Enable and encourage people and teams to work together to deliver the organization's objectives. •Create capability to meet system requirements in the future (conduct succession planning) •Develop and trial training solution to representative users. •Deliver final training solutions to designated staff according to agreed timetable. •Provide means for user feedback [on human issues]. 7. Evaluation •Assess the health and well-being risks to the users of the system. -Risk analysis (process and product) •Assess the risks to the community and environment arising from human error a) Risks in the use of the system. •Evaluate the current severity of emerging threats to system usability and other HS risks and the effectiveness of mitigation measures. •Assess the risks of not involving end users in each evaluation. b) Plan and •Collect user input on the usability of the developing system. -Obtain user feedback on usability execute •Revise design and safety features using feedback from evaluations. -Use models and simulation •Plan the evaluation. •Identify and analyze the conditions under which a system is to be tested or otherwise evaluated. •Check that the system is fit for evaluation. •Carry out and analyze the evaluation according to the evaluation plan. •Understand and act on the results of the evaluation. c) Validation •Test that the system meets the requirements of the users, the tasks and the -Compare with requirements environment, as defined in its specification. -Common Industry Format for usability •Assess the extent to which usability criteria and other HS requirements are reports likely to be met by the proposed design. -Performance measurement d) HSI knowledge •Review the system for adherence to applicable human science knowledge, style guides, standards, guidelines, regulations and legislation. e) Staffing •Decide how many people are needed to fulfill the strategy and what ranges of HR competence they need. •Develop and trial training solution to representative users. •Conduct assessments of usability [relating to HR]. •Interpret the findings •Validate the data. •Check that the data are being used. 8. Negotiating •Contribute to the business case for the system. -Program risk analysis commitments •Include HS review and sign-off in all reviews and decisions a) business case b) requirements •Analyze the user requirements. -Value-based practices and principles •Present these requirements to project stakeholders for use in the development (identify success critical stakeholder and operation of the system. requirements) •Identify any staffing gap and communicate requirement to design of staffing -Common Industry Specification for solutions. Usability Requirements -Environment/organization assessment 9. Development •Maintain contact with users and the client organization throughout the -Risk analysis (process and product) and evolution definition, development and introduction of a system. -User feedback on usability •Evolve options and constraints into an implementation strategy covering -Use models and simulation technical, integration, and planning and manning issues. -Guidelines: Common Industry Format • for usability reports -Performance measurement 10. Monitoring •Analyze feedback on the system during delivery and inform the organization of -Organizational and environmental and control emerging issues. context analysis •Manage the life cycle plan to address HS issues. -Risk Analysis •Take effective mitigation to address risks to system usability. -User feedback •Take account of user input and inform users. -Work context analysis •Identify emerging HS issues. •Understand and act on the results of the evaluation. •Produce and promulgate a validated statement of staffing shortfall by number and range of competence. 11. Operations •Analyze feedback on the system during delivery and inform the organization of -Work context analysis and retirement emerging issues. -Organizational and environmental •Produce personnel strategy. context analysis a) Operations •Review the system for adherence to applicable human science knowledge, style guides, standards, guidelines, regulations and legislation. •Deliver training and other forms of awareness-raising to users and support staff. •Assess the effect of change on the usability of the system. •Review the health and well-being risks to the users of the system. •Review the risks to the community and environment arising from human error in the use of the system. •Take action on issues arising from in-service assessment. •Perform research to refine and consolidate operation and support strategy for the system. b) Retirement •Collect and analyze in-service reports to generate updates or lessons learnt for the next version of the system. •Identify risks and health and safety issues associated with removal from service and destruction of the system. •Define how users will be re-allocated, dismissed, or transferred to other duties. •Plan break-up of social structures. •Debriefing and retrospective analysis for replacement system. 12. Organizational •Identify and use the most suitable data formats for exchanging HF data. -Assess and improve HSI capability capability •Have a policy for HF data management. improvement •Perform research to develop HF data as required. a) HSI capability •Produce coherent data standards and formats. data collection, •Define rules for the management of data. analysis, and •Develop and maintain adequate data search methods. improvement •Feedback into future HR procurement, training and delivery strategies. b) Organizational •Define usability as a competitive asset -Develop and maintain HSI infrastructure skill/career and •Set usability, health and safety objectives for systems and resources infrastructure •Follow competitive situation in the market place -Identify required HSI skills development •Develop user-centred infrastructure. -Provide staff with HSI skills planning and •Relate HS issues to business benefits. -Establish and communicate a policy on execution •Establish and communicate a policy for human-centeredness. HSI •Include HR and user-centred elements in support and control procedures. -Maintain an awareness of usability •Define and maintain HCD and HR infrastructure and resources. •Increase and maintain awareness of usability. •Develop or provide staff with suitable HS skills. •Take account of HS issues in financial management •Assess and improve HS capability in processes that affect usability, health and safety. •Develop a common terminology for HS issues with the organization. •Facilitate personal and technical interactions related to HS issues. •Feedback into future HR procurement, training and delivery strategies. •Create capability to meet system requirements in the future (conduct succession planning) •Identify any opportunities for redeployment. •Develop a strategy for [HR] data gathering