Eclipse Plug-in to Manage User Centered Design Yael Dubinsky Shah Rukh Humayoun Tiziana Catarci Dipartimento di Informatica e Dipartimento di Informatica e Dipartimento di Informatica e Sistemistica “A. Ruberti” Sistemistica “A. Ruberti” Sistemistica “A. Ruberti” SAPIENZA - Università di Roma SAPIENZA - Università di Roma SAPIENZA - Università di Roma Via Ariosto - 25, 00185, Roma, Italy Via Ariosto - 25, 00185, Roma, Italy Via Ariosto - 25, 00185, Roma, Italy dubinsky@dis.uniroma1.it humayoun@dis.uniroma1.it catarci@dis.uniroma1.it ABSTRACT Variations in activities arise in different UCD methods [5, 15], User-centered design (UCD) approach guides the design of user and still the Human-Computer Interaction (HCI) community lacks interface (UI) and its evaluation by integrating user experience as to agree upon a precise definition of UCD methods or process [3, part of the software development process. Involving users during 8]. However, in [8] there is a set of definition of twelve principles the development process by applying UCD techniques minimizes for designing and developing systems with focus on UCD that is risks and increases the product quality. One of the challenges obtained as: “User-centered system design (UCSD) is a process towards this is to automating the management of UCD activities focusing on usability throughout the entire development process during the development time thus to steer and control the UCD and further throughout the system life-cycle” (p. 401). The activities within the development environment of software International Organization for Standardization (ISO) has also projects. In this paper, we present a plug-in for Eclipse defined the standard guidelines to deal with different aspects of development platform to manage UCD activities at the Integrated HCI and UCD; in particular, ISO/DIS 13407 1 provides the Development Environment (IDE) level. We develop and evaluate guidance on user-oriented design process. Other relevant ISO the plug-in with teams that work according to the agile software standard guidance are ISO 9241-11 2 , ISO TR 16982 3 . A detailed development approach. Using this plug-in, the development teams discussion about the methods, processes, guidelines, and can manage UCD activities at IDE level hence developing high prototype activities in UCD can be found in ISO standards and in quality software products with adequate level of usability. [5, 15]. Lack of usability and inefficient design of the end-product are Categories and Subject Descriptors common causes amongst the others for failure of software H5.2 [User Interfaces]: User-centered design, K.6.3 [Software products [12, 14]. The software products are developed for the Management]: Software development, software process. users, and normally fail if the users find it difficult to operate them due to the lack of usability and inappropriate design. The software development teams usually work with the users either at General Terms the start of project for getting the requirements or at the end of Management, Measurement, Design. project to test and evaluate the developed product. Furthermore, normally in the testing phase, the project teams focus more on Keywords checking the functionalities of the product (such as performance, User-centered design (UCD), user experience, agile software reliability, security, robustness, etc) rather than its usability and development, Eclipse plug-in. design aspects. Checking usability or solving defects at the end of the development process needs more time, efforts, and money hence causing the failure of many software projects As a result, 1. INTRODUCTION involving the users in the design phases is a good practice to The user-centered design (UCD) approach [5, 15] is used to identify lack of usability and design defects early in development develop software products by positioning the real users of the time, in order to avoid any possibility of product failure at the system at the centre of design activities, e.g. by representing or end. The UCD approach, described above, provides methods and modeling users in some way like scenarios and personas; through techniques for involving users at early stages of development users testing of prototypes (either paper or working prototype); by [15]. So, integrating the UCD approach within software involving users in making design decisions (e.g. thorough development processes gives the benefit of including the user participatory design). The approach focus is on the increase of experience as part of the development process for producing usability for the users by involving them in design and quality products with adequate level of usability. development activities. UCD activities aim at reducing the risks of the software project and increasing the overall product quality. Analyzing the current software design practices, we identified a lack of UCD management which we define for a specific software Permission to make digital or hard copies of all or part of this work for 1 ISO/DIS 13407: Human Centered Design for Interactive personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that Systems 2 copies bear this notice and the full citation on the first page. To copy ISO 9241–11: Ergonomic requirements for office work with otherwise, or republish, to post on servers or to redistribute to lists, visual display terminals (VDTs) 3 requires prior specific permission and/or a fee. ISO TR 16982: Ergonomics of human-system interaction - Human-cantered lifecycle process descriptions I-USED ’08, September 24, 2004, Pisa, Italy project as the ability to steer and control the UCD activities 3. Roles – Different roles are defined to support software within the development environment of the project. This development environments. The agile approach adds roles management of UCD activities at IDE level is important as it will for better management and development of the project. help to integrate and automate UCD activities across different Combining agile and UCD adds the UCD roles, like for development life-cycle phases. By automating the management of example the UI Designer role. UCD activities within development environment we decrease the time and cost to test each unit and improve the overall product Using our integrated approach of UCD and agile with software quality. teams in the academia, we have gathered use cases to establish the plug-in specifications. Following are six representative use cases In this paper, we present an Eclipse plug-in to manage the user that are categorized in three themes: the development process, the involvement for different UCD activities in software development evaluation activity, and the design improvement. that can work with any software development process life-cycle. Managing UCD activities while working according to the agile software development approach [1] was already suggested [2, 4, 2.1 Development Process There is a need to involve the users in the process of development. 10, 11, 13, 7]. Our contribution is by automating UCD management at IDE level to enable, for example, creating Following are examples for use cases that relate to this category: experiments, adding users, analyzing results, and tracing the code. ƒ One of the tasks during the first planning session is as The remainder of this paper is as follows. In Section 2 we follows: ‘Explore who are the kinds of users who should use describe our framework to integrate the user experience in the the product that we develop; what are their characteristics; process of software development. Section 3 presents and explains what are their needs; what are their expectations from the how we can use our developed Eclipse plug-in to manage UCD product.’ The customer explains that this is an important task activities at IDE level during software development. We conclude since he cannot represent all users and actually he does not in Section 4. know for sure what their exact needs are (though he is sure they will like it a lot). One of the teammates asks to be assigned to this task and estimates it as 10 hours of work for 2. USER EXPERIENCE AND THE this iteration. Presenting her results after two weeks, she opens her development environment in the database of the DEVELOPMENT PROCESS User Perspective and shows the list of 20 users she talked A software development approach that has been emerging in the with (names, titles, contact details), main issues that were last decade is the agile approach that is used for constructing learned, and one new task that has emerged for future software products in an iterative and incremental manner; where iterations: ‘Prepare and run a questionnaire that will enable each iteration produces working artifacts that are valuable to the us to extract users’ needs.’ The customer sets high priority customers and to the project. This is performed in a highly for this new task. collaborative fashion in order to produce quality products that meet the requirements in cost effective and timely manner [1]. ƒ The project manager reviews the subjects for the coming reflection session, and sees that one of the subjects is ‘ways Based on our experience with guiding the implementation of the to assess the usability of our product’. She then sends agile approach [16, 6, 9], and the integration of UCD techniques invitations to seven users from the two different kinds of in the last three years in agile projects in academia [7], we gather users to join this meeting. During the reflection session, one the cases in which UCD can be supported within the IDE. decision is made that two users will participate in each The main characteristics of the integrated approach of agile and iteration planning session and their responsibility will be to UCD that we use are: give feedbacks on what presented. In addition they will help in defining three measures that will be automated thus enable 1. Iterative design activities - In many cases, when user- teammates an immediate feedback during development. centric techniques are used, the design of the system is refined according to the users’ evaluations and this is 2.2 Evaluation performed mainly during the design phase. When There is a need to perform user evaluation and to manage it along introducing the agile approach, the design is updated the process of development. regularly as the product evolves. When combining UCD activities with the agile approach, the user evaluation is Following are examples for use cases that relate to this category: fostered by performing UCD tasks in each iteration of two to ƒ The team leader browses over the details of the user four weeks, and the design is updated according to the experiments that are planned for tomorrow. He sees the evaluation of on-going outcomes that are considered as number of users that will arrive, the names, and refactoring tasks. responsibilities of the teammates that will take care of these 2. Measures – Taking measurements is a basic activity in experiments. He checks the variables that were set and the software development processes. The agile approach experiments flow. emphasizes it and suggests the tracker role. When combining ƒ One of the teammates sees that the User Perspective flushes agile and UCD, the set of evaluation tools is built and refined meaning new data have arrived. He clicks on it and sees that during the process and is used iteratively as a complement to the results of the user experiments that were conducted the process and product measures. yesterday are in. He is surprised to find a new problem with high severity ranking. Examining results from previous - A user interface to run the experiment experiments, he observes that this is a new problem and adds - Client / Server architecture for a note about it in the discussion area. During the next running the experiment iteration planning, the experiments’ results are presented and - Support automatic measures for user among others, a measure is presented that shows two evaluation that are derived from user problems that emerged from the users; one in normal experience severity and the other one in high severity. - On line help - Traceability – experiments should be 2.3 Design Improvement part of a specific project that we There is a need to improve the design of the user interfaces based develop; each specific development on the evaluation results. task that is derived from one or more experiments results should be Following are examples for use cases that relate to this category: associated with the appropriate code ƒ The designer of the user interface views the latest design parts that implement them diagrams and tries different changes that adhere to the new 3 3 Stability task in this iteration. The task was added due to the last - Testing and Refactoring problem that was found by the users. Thinking of different - Development refinement options, she talks with two users and receives their 4 5 Heuristics and User Profiling feedbacks. She shows them the possible drawings of the new - Support Nielsen heuristics technique interface and asks them to simulate trying it while thinking - Support user profiling aloud. She summarizes the results and sets her decision. - Scale with more end projects ƒ One of the teammates browses over the system reports and looks for each user experiment, which was conducted in the last two releases, what were the results and what were the 3.2 Using the UCD Management Plug-in implications on design. For each implication, he sees the The main feature of the UCD management plug-in is the ability to development tasks that are related. create and deploy user experiments from within the Eclipse IDE. We suggest that the combination of the agile and UCD Focusing on a specific software project, we can define different approaches should be supported by an extension to a kinds of experiments. One kind for example is a task-based user contemporary development environment in order to be used in a experiment in which the participant uses the target product and natural manner. This is elaborated in the next section. receives the tasks to perform along the experiment. During this experiment the system measures different performance times. Another kind of experiment is questionnaire-based experiment in 3. THE UCD MANAGEMENT PLUG-IN which the user specifies the level of his/her agreement with the 3.1 The Project presented set of statements. The development team chooses the A team of six developers in a project based course in the set of experiments according to the nature of software project and academia has developed the UCD plug-in that is presented in this then selects appropriate users from the pool of target users to paper 4 . The project took five and a half months and was perform these tasks. composed of 4 iterations, three of 5 weeks each and one of 3 We illustrate the definition of a task-based user experiment using weeks. Table 1 shows the durations and the main themes of each a view that is presented by Figures 1 and 2. iteration. Table 1. The iterations – duration and themes Iter. Duration Themes (weeks) 1 5 Experiments and Roles - End-to-end experiments: define the experiment, execute it, results view - Evaluation manager role-perspective - UI designer role-perspective - Work items can be created, assigned - The system has one data repository 2 5 Users’ interface and user experience (UX) automation Figure 1. Defining the experiment (left hand side) - Users’ management and permissions In the left hand side of the view (Figure 1) we can see the options of setting the experiment schedule and the users who are 4 This project was developed as part of the “Annual Project in involved. In the right hand side (Figure 2) we can see the options Software Engineering” course that is instructed by the first of adding tasks to the experiment, save the experiment, and author at the Computer Science Department at Technion IIT. execute it. Figure 2. Defining the experiment (right hand side) The experiment can run locally i.e., on the server on which the data is stored, or remotely. Setting the remote option causes the enlisted users to receive email with the experiment files attached, so they can perform the experiment in a way that the results are stored in the server. Figure 3 shows the results view of a specific experiment. Different kinds of experiments were developed that support appropriate results views. Figure 5. Associating code to a development task Figure 6 shows how this code is marked (left side bar) and highlighted. Figure 3. The experiment’s results view Figure 6. Associated code is marked Experiment Explorer is available to support the experiments of a specific project (Figure 4). Experiments can be shared among different projects. 3.3 Evaluating the Plug-in As part of the third iteration, the team was asked to evaluate its own product (the UCD management plug-in) using itself (“eating own cookies”). Following is the plan and the results of this preliminary evaluation. The evaluations goals as written by the team were: ƒ Examining suspicious issues like adding new users to the system and analyzing the experiments’ results (specifically for the questionnaire-based experiments). ƒ Receiving feedback on the graphical user interface (GUI) and how intuitive it is. ƒ Examining the plug-in on a large scale project. Two kinds of experiments were defined by the team for the evaluation of the plug-in. The first experiment was a task-based experiment and the second was a questionnaire-based experiment. Figure 4. The Expriement Explorer The participants were 3 students from another team in the same Managing the experiments, new development tasks are derived. course, and in addition all the six developers performed both These tasks are the results of the already conducted experiments. kinds of experiments. Each participant performed the experiment The plug-in enables associating code part/s to the appropriate by himself / herself while one observer was sitting aside for task/s and vice versa so traceability is kept. Figure 5 shows how a writing notes. code segment can be associated. Figure 7. Results of questionnaire-based experiment – participants from another team Figure 8. Results of questionnaire-based experiment – team members are the participants We focus on the questionnaire-based experiment and comments 1. “In the questionnaire-view that is presented to the of the observers and show an example of a derived task that participant, long tasks appear truncated.” emerged for further development. The questionnaire included the 2. “The participant did not know how to save the changes in the following statements: result page. He searches for a save button like appears in other screens.” 1. Logging in to the system is simple. 3. “The names of the operations in the menu of the experiment 2. Adding a user or a teammate to the system is simple. view are not clear.” 3. Switching between teammates is fast and simple. 4. The configuration page is intuitive. Analyzing the results of both experiments, associations to the 5. The Questionnaire result page displays the level of specific results were presented for each conclusion, and then agreement (per statement) in a clear way. suggested development tasks were associated to the conclusions. 6. The Questionnaire result page displays the usability One of the finding, for example, was detailed as follows: problems discovered in a clear way. 7. The different editors and views of the plug-in are uniform “It was found that there is a difficulty in identifying problems in and follow a similar theme the product out of the information that is presented in the ‘results 8. The different editors and views of the plug-in blend page’. Participants find it hard to associate the results (as seamlessly into the eclipse. presented in the ‘results page’) to the experiment goals and to the 9. I would use this plug-in to test the usability of an application practical problems that were discovered.” in development. “Association to the results: Figures 7 and 8 show the results of the three participants from ƒ In the questionnaire-based experiment the two teams marked another group and the results of the developers themselves ‘Disagree’ for statement 6 [The questionnaire result page respectively. displays the usability problems discovered in a clear way]. ƒ In the task-assignments experiment, it took long time, 84 and Following are few comments, for example, that were presented by 177 seconds in average for the two groups, to complete task the observers: 5 [According to the experiment goals, try to assess the number of usability problems indicated by the results, and [4] Detweiler, M. 2007. Managing UCD within Agile Projects. write that number as a conclusion to this experiment].” ACM Interactions May-June, 40 – 42. The development task that was defined using the plug-in is as [5] Dix, A., Finlay, J.E., Abowd, G.D., and Beale, R. 2003. follows: “Enable determining thresholds for success and failure in Human Computer Interaction, 3rd Edition, Prentice Hall. an experiment and present them clearly in the ‘results page’.” [6] Dubinsky, Y. and Hazzan, O. 2005. The construction process of a framework for teaching software development methods, 4. CONCLUSION Computer Science Education, 15:4, 275–296. In this paper, we present our Eclipse plug-in to automating the process of managing UCD activities at the Integrated [7] Dubinsky, Y., Catarci, T., Humayoun, S., and Kimani, S. Development Environment (IDE) level during the development 2007. Integrating user evaluation into software development time of software projects. To develop the framework we were environments, 2nd DELOS Conference on Digital Libraries, inspired by use cases that emerged when performing UCD Pisa, Italy. activities with the agile teams. Using this plug-in, the software [8] Gulliksen, J., Goransson, B., Boivie, I., Blomkvist, S., project team can create experiments, adding users, analyzing Persson, J. and Cajander, A. 2003. Key principles for user- results and tracing back it to code for their developed or in- centered systems design. Behaviou & Information progress product. By automating the process of managing UCD Technology, Vol. 22, No. 6, 397–409. activities the chances of creating quality products with adequate [9] Hazzan, O. and Dubinsky, Y., Agile Software level of usability become high, as it helps to get benefits of user Engineering, Undergraduate Topics in Computer Science experience during development time. Series, Springer-Verlag London Ltd, 2008, in press. In future, we intend to continue work on the developed plug-in to [10] Hudson, W. 2003. Adopting User-Centered Design within an manage more UCD activities. Further, we also intend to evaluate Agile Process: A Conversation. Cutter IT Journal, (16), 10 the developed product on big scale with different size of software http://www.suntagm.co.uk/design/articles/ucdxp03.pdf development teams. [11] Hwong, B., Laurance, D., Rudorfer, A., and Schweizer, A. 2004. User-Centered Design and Agile Software 5. ACKNOWLEDGMENTS Development Processes. Siemens Corporate Research Our thanks to the plug-in developers from Technion IIT whose http://www.scr.siemens.com/en/pdf/se_pdf/rudorfer-1.pdf. product is presented in this paper: David Ben-David, Tomer Einav, Yoav Haimovitch, Barak Nirenberg, Laliv Pele, and Alon [12] Landauer, T. K. 1995. The trouble with computers: Vinkov. usefulness, usability, and productivity, MIT Press. [13] McInerney, P., and Maurer, F. 2005. UCD in agile projects: 6. REFERENCES Dream team or odd couple?. ACM Interactions, 12(6), 19 - [1] Agile Alliance 2001. Manifesto for Agile Software 23. Development. Technical Report by Agile Alliance, [14] Norman, D. 2006. Why Doing User Observations First Is http://www.agilealliance.org. Wrong, ACM Interactions, July-August 2006. [2] Blomkvist, S. 2005. Towards a Model for Bridging Agile [15] Sharp, H., Rogers, Y., and Preece, J. 2007. Interaction Development and User-Centered Design. Published as a Design: Beyond Human-Computer Interaction. 2nd Edition. book chapter: Seffah, A., Gulliksen, J., and Desmarais, M., Willey. (eds.). Human-Centered Software Engineering – Integrating Usability in The Development Process. Springer, Dordrecht, [16] Talby, D., Hazzan, O., Dubinsky, Y. and Keren, A. 2006. The Netherlands, 217-243. Agile software testing in a large-scale project, IEEE Software, Special Issue on Software Testing, 30-37. [3] Blomkvist, S. 2006. User-Centered Design and Agile Development of IT Systems. IT Licentiate theses, Department of Information Technology, Uppsala University.