=Paper=
{{Paper
|id=Vol-407/paper-10
|storemode=property
|title=Eclipse Plug-in to Manage User Centered Design
|pdfUrl=https://ceur-ws.org/Vol-407/paper10.pdf
|volume=Vol-407
|dblpUrl=https://dblp.org/rec/conf/iused/DubinskyHC08
}}
==Eclipse Plug-in to Manage User Centered Design==
https://ceur-ws.org/Vol-407/paper10.pdf
Eclipse Plug-in to Manage User Centered Design
Yael Dubinsky Shah Rukh Humayoun Tiziana Catarci
Dipartimento di Informatica e Dipartimento di Informatica e Dipartimento di Informatica e
Sistemistica “A. Ruberti” Sistemistica “A. Ruberti” Sistemistica “A. Ruberti”
SAPIENZA - Università di Roma SAPIENZA - Università di Roma SAPIENZA - Università di Roma
Via Ariosto - 25, 00185, Roma, Italy Via Ariosto - 25, 00185, Roma, Italy Via Ariosto - 25, 00185, Roma, Italy
dubinsky@dis.uniroma1.it humayoun@dis.uniroma1.it catarci@dis.uniroma1.it
ABSTRACT Variations in activities arise in different UCD methods [5, 15],
User-centered design (UCD) approach guides the design of user and still the Human-Computer Interaction (HCI) community lacks
interface (UI) and its evaluation by integrating user experience as to agree upon a precise definition of UCD methods or process [3,
part of the software development process. Involving users during 8]. However, in [8] there is a set of definition of twelve principles
the development process by applying UCD techniques minimizes for designing and developing systems with focus on UCD that is
risks and increases the product quality. One of the challenges obtained as: “User-centered system design (UCSD) is a process
towards this is to automating the management of UCD activities focusing on usability throughout the entire development process
during the development time thus to steer and control the UCD and further throughout the system life-cycle” (p. 401). The
activities within the development environment of software International Organization for Standardization (ISO) has also
projects. In this paper, we present a plug-in for Eclipse defined the standard guidelines to deal with different aspects of
development platform to manage UCD activities at the Integrated HCI and UCD; in particular, ISO/DIS 13407 1 provides the
Development Environment (IDE) level. We develop and evaluate guidance on user-oriented design process. Other relevant ISO
the plug-in with teams that work according to the agile software standard guidance are ISO 9241-11 2 , ISO TR 16982 3 . A detailed
development approach. Using this plug-in, the development teams discussion about the methods, processes, guidelines, and
can manage UCD activities at IDE level hence developing high prototype activities in UCD can be found in ISO standards and in
quality software products with adequate level of usability. [5, 15].
Lack of usability and inefficient design of the end-product are
Categories and Subject Descriptors common causes amongst the others for failure of software
H5.2 [User Interfaces]: User-centered design, K.6.3 [Software products [12, 14]. The software products are developed for the
Management]: Software development, software process. users, and normally fail if the users find it difficult to operate
them due to the lack of usability and inappropriate design. The
software development teams usually work with the users either at
General Terms the start of project for getting the requirements or at the end of
Management, Measurement, Design. project to test and evaluate the developed product. Furthermore,
normally in the testing phase, the project teams focus more on
Keywords checking the functionalities of the product (such as performance,
User-centered design (UCD), user experience, agile software reliability, security, robustness, etc) rather than its usability and
development, Eclipse plug-in. design aspects. Checking usability or solving defects at the end
of the development process needs more time, efforts, and money
hence causing the failure of many software projects As a result,
1. INTRODUCTION involving the users in the design phases is a good practice to
The user-centered design (UCD) approach [5, 15] is used to identify lack of usability and design defects early in development
develop software products by positioning the real users of the time, in order to avoid any possibility of product failure at the
system at the centre of design activities, e.g. by representing or end. The UCD approach, described above, provides methods and
modeling users in some way like scenarios and personas; through techniques for involving users at early stages of development
users testing of prototypes (either paper or working prototype); by [15]. So, integrating the UCD approach within software
involving users in making design decisions (e.g. thorough development processes gives the benefit of including the user
participatory design). The approach focus is on the increase of experience as part of the development process for producing
usability for the users by involving them in design and quality products with adequate level of usability.
development activities. UCD activities aim at reducing the risks
of the software project and increasing the overall product quality. Analyzing the current software design practices, we identified a
lack of UCD management which we define for a specific software
Permission to make digital or hard copies of all or part of this work for 1
ISO/DIS 13407: Human Centered Design for Interactive
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
Systems
2
copies bear this notice and the full citation on the first page. To copy ISO 9241–11: Ergonomic requirements for office work with
otherwise, or republish, to post on servers or to redistribute to lists, visual display terminals (VDTs)
3
requires prior specific permission and/or a fee. ISO TR 16982: Ergonomics of human-system interaction -
Human-cantered lifecycle process descriptions
I-USED ’08, September 24, 2004, Pisa, Italy
project as the ability to steer and control the UCD activities 3. Roles – Different roles are defined to support software
within the development environment of the project. This development environments. The agile approach adds roles
management of UCD activities at IDE level is important as it will for better management and development of the project.
help to integrate and automate UCD activities across different Combining agile and UCD adds the UCD roles, like for
development life-cycle phases. By automating the management of example the UI Designer role.
UCD activities within development environment we decrease the
time and cost to test each unit and improve the overall product Using our integrated approach of UCD and agile with software
quality. teams in the academia, we have gathered use cases to establish the
plug-in specifications. Following are six representative use cases
In this paper, we present an Eclipse plug-in to manage the user that are categorized in three themes: the development process, the
involvement for different UCD activities in software development evaluation activity, and the design improvement.
that can work with any software development process life-cycle.
Managing UCD activities while working according to the agile
software development approach [1] was already suggested [2, 4,
2.1 Development Process
There is a need to involve the users in the process of development.
10, 11, 13, 7]. Our contribution is by automating UCD
management at IDE level to enable, for example, creating Following are examples for use cases that relate to this category:
experiments, adding users, analyzing results, and tracing the code.
One of the tasks during the first planning session is as
The remainder of this paper is as follows. In Section 2 we follows: ‘Explore who are the kinds of users who should use
describe our framework to integrate the user experience in the the product that we develop; what are their characteristics;
process of software development. Section 3 presents and explains what are their needs; what are their expectations from the
how we can use our developed Eclipse plug-in to manage UCD product.’ The customer explains that this is an important task
activities at IDE level during software development. We conclude since he cannot represent all users and actually he does not
in Section 4. know for sure what their exact needs are (though he is sure
they will like it a lot). One of the teammates asks to be
assigned to this task and estimates it as 10 hours of work for
2. USER EXPERIENCE AND THE this iteration. Presenting her results after two weeks, she
opens her development environment in the database of the
DEVELOPMENT PROCESS User Perspective and shows the list of 20 users she talked
A software development approach that has been emerging in the with (names, titles, contact details), main issues that were
last decade is the agile approach that is used for constructing learned, and one new task that has emerged for future
software products in an iterative and incremental manner; where iterations: ‘Prepare and run a questionnaire that will enable
each iteration produces working artifacts that are valuable to the us to extract users’ needs.’ The customer sets high priority
customers and to the project. This is performed in a highly for this new task.
collaborative fashion in order to produce quality products that
meet the requirements in cost effective and timely manner [1]. The project manager reviews the subjects for the coming
reflection session, and sees that one of the subjects is ‘ways
Based on our experience with guiding the implementation of the to assess the usability of our product’. She then sends
agile approach [16, 6, 9], and the integration of UCD techniques invitations to seven users from the two different kinds of
in the last three years in agile projects in academia [7], we gather users to join this meeting. During the reflection session, one
the cases in which UCD can be supported within the IDE. decision is made that two users will participate in each
The main characteristics of the integrated approach of agile and iteration planning session and their responsibility will be to
UCD that we use are: give feedbacks on what presented. In addition they will help
in defining three measures that will be automated thus enable
1. Iterative design activities - In many cases, when user- teammates an immediate feedback during development.
centric techniques are used, the design of the system is
refined according to the users’ evaluations and this is 2.2 Evaluation
performed mainly during the design phase. When There is a need to perform user evaluation and to manage it along
introducing the agile approach, the design is updated the process of development.
regularly as the product evolves. When combining UCD
activities with the agile approach, the user evaluation is Following are examples for use cases that relate to this category:
fostered by performing UCD tasks in each iteration of two to The team leader browses over the details of the user
four weeks, and the design is updated according to the experiments that are planned for tomorrow. He sees the
evaluation of on-going outcomes that are considered as number of users that will arrive, the names, and
refactoring tasks. responsibilities of the teammates that will take care of these
2. Measures – Taking measurements is a basic activity in experiments. He checks the variables that were set and the
software development processes. The agile approach experiments flow.
emphasizes it and suggests the tracker role. When combining One of the teammates sees that the User Perspective flushes
agile and UCD, the set of evaluation tools is built and refined meaning new data have arrived. He clicks on it and sees that
during the process and is used iteratively as a complement to the results of the user experiments that were conducted
the process and product measures. yesterday are in. He is surprised to find a new problem with
high severity ranking. Examining results from previous - A user interface to run the experiment
experiments, he observes that this is a new problem and adds - Client / Server architecture for
a note about it in the discussion area. During the next running the experiment
iteration planning, the experiments’ results are presented and - Support automatic measures for user
among others, a measure is presented that shows two evaluation that are derived from user
problems that emerged from the users; one in normal experience
severity and the other one in high severity. - On line help
- Traceability – experiments should be
2.3 Design Improvement part of a specific project that we
There is a need to improve the design of the user interfaces based develop; each specific development
on the evaluation results. task that is derived from one or more
experiments results should be
Following are examples for use cases that relate to this category: associated with the appropriate code
The designer of the user interface views the latest design parts that implement them
diagrams and tries different changes that adhere to the new 3 3 Stability
task in this iteration. The task was added due to the last - Testing and Refactoring
problem that was found by the users. Thinking of different - Development refinement
options, she talks with two users and receives their 4 5 Heuristics and User Profiling
feedbacks. She shows them the possible drawings of the new - Support Nielsen heuristics technique
interface and asks them to simulate trying it while thinking - Support user profiling
aloud. She summarizes the results and sets her decision. - Scale with more end projects
One of the teammates browses over the system reports and
looks for each user experiment, which was conducted in the
last two releases, what were the results and what were the 3.2 Using the UCD Management Plug-in
implications on design. For each implication, he sees the The main feature of the UCD management plug-in is the ability to
development tasks that are related. create and deploy user experiments from within the Eclipse IDE.
We suggest that the combination of the agile and UCD Focusing on a specific software project, we can define different
approaches should be supported by an extension to a kinds of experiments. One kind for example is a task-based user
contemporary development environment in order to be used in a experiment in which the participant uses the target product and
natural manner. This is elaborated in the next section. receives the tasks to perform along the experiment. During this
experiment the system measures different performance times.
Another kind of experiment is questionnaire-based experiment in
3. THE UCD MANAGEMENT PLUG-IN which the user specifies the level of his/her agreement with the
3.1 The Project presented set of statements. The development team chooses the
A team of six developers in a project based course in the set of experiments according to the nature of software project and
academia has developed the UCD plug-in that is presented in this then selects appropriate users from the pool of target users to
paper 4 . The project took five and a half months and was perform these tasks.
composed of 4 iterations, three of 5 weeks each and one of 3
We illustrate the definition of a task-based user experiment using
weeks. Table 1 shows the durations and the main themes of each
a view that is presented by Figures 1 and 2.
iteration.
Table 1. The iterations – duration and themes
Iter. Duration Themes
(weeks)
1 5 Experiments and Roles
- End-to-end experiments: define the
experiment, execute it, results view
- Evaluation manager role-perspective
- UI designer role-perspective
- Work items can be created, assigned
- The system has one data repository
2 5 Users’ interface and user experience (UX)
automation Figure 1. Defining the experiment (left hand side)
- Users’ management and permissions
In the left hand side of the view (Figure 1) we can see the options
of setting the experiment schedule and the users who are
4
This project was developed as part of the “Annual Project in involved. In the right hand side (Figure 2) we can see the options
Software Engineering” course that is instructed by the first of adding tasks to the experiment, save the experiment, and
author at the Computer Science Department at Technion IIT. execute it.
Figure 2. Defining the experiment (right hand side)
The experiment can run locally i.e., on the server on which the
data is stored, or remotely. Setting the remote option causes the
enlisted users to receive email with the experiment files attached,
so they can perform the experiment in a way that the results are
stored in the server. Figure 3 shows the results view of a specific
experiment. Different kinds of experiments were developed that
support appropriate results views. Figure 5. Associating code to a development task
Figure 6 shows how this code is marked (left side bar) and
highlighted.
Figure 3. The experiment’s results view Figure 6. Associated code is marked
Experiment Explorer is available to support the experiments of a
specific project (Figure 4). Experiments can be shared among
different projects. 3.3 Evaluating the Plug-in
As part of the third iteration, the team was asked to evaluate its
own product (the UCD management plug-in) using itself (“eating
own cookies”). Following is the plan and the results of this
preliminary evaluation.
The evaluations goals as written by the team were:
Examining suspicious issues like adding new users to the
system and analyzing the experiments’ results (specifically
for the questionnaire-based experiments).
Receiving feedback on the graphical user interface (GUI)
and how intuitive it is.
Examining the plug-in on a large scale project.
Two kinds of experiments were defined by the team for the
evaluation of the plug-in. The first experiment was a task-based
experiment and the second was a questionnaire-based experiment.
Figure 4. The Expriement Explorer
The participants were 3 students from another team in the same
Managing the experiments, new development tasks are derived. course, and in addition all the six developers performed both
These tasks are the results of the already conducted experiments. kinds of experiments. Each participant performed the experiment
The plug-in enables associating code part/s to the appropriate by himself / herself while one observer was sitting aside for
task/s and vice versa so traceability is kept. Figure 5 shows how a writing notes.
code segment can be associated.
Figure 7. Results of questionnaire-based experiment – participants from another team
Figure 8. Results of questionnaire-based experiment – team members are the participants
We focus on the questionnaire-based experiment and comments 1. “In the questionnaire-view that is presented to the
of the observers and show an example of a derived task that participant, long tasks appear truncated.”
emerged for further development. The questionnaire included the 2. “The participant did not know how to save the changes in the
following statements: result page. He searches for a save button like appears in
other screens.”
1. Logging in to the system is simple.
3. “The names of the operations in the menu of the experiment
2. Adding a user or a teammate to the system is simple.
view are not clear.”
3. Switching between teammates is fast and simple.
4. The configuration page is intuitive. Analyzing the results of both experiments, associations to the
5. The Questionnaire result page displays the level of specific results were presented for each conclusion, and then
agreement (per statement) in a clear way. suggested development tasks were associated to the conclusions.
6. The Questionnaire result page displays the usability
One of the finding, for example, was detailed as follows:
problems discovered in a clear way.
7. The different editors and views of the plug-in are uniform “It was found that there is a difficulty in identifying problems in
and follow a similar theme the product out of the information that is presented in the ‘results
8. The different editors and views of the plug-in blend page’. Participants find it hard to associate the results (as
seamlessly into the eclipse. presented in the ‘results page’) to the experiment goals and to the
9. I would use this plug-in to test the usability of an application practical problems that were discovered.”
in development. “Association to the results:
Figures 7 and 8 show the results of the three participants from In the questionnaire-based experiment the two teams marked
another group and the results of the developers themselves ‘Disagree’ for statement 6 [The questionnaire result page
respectively. displays the usability problems discovered in a clear way].
In the task-assignments experiment, it took long time, 84 and
Following are few comments, for example, that were presented by
177 seconds in average for the two groups, to complete task
the observers:
5 [According to the experiment goals, try to assess the
number of usability problems indicated by the results, and [4] Detweiler, M. 2007. Managing UCD within Agile Projects.
write that number as a conclusion to this experiment].” ACM Interactions May-June, 40 – 42.
The development task that was defined using the plug-in is as [5] Dix, A., Finlay, J.E., Abowd, G.D., and Beale, R. 2003.
follows: “Enable determining thresholds for success and failure in Human Computer Interaction, 3rd Edition, Prentice Hall.
an experiment and present them clearly in the ‘results page’.” [6] Dubinsky, Y. and Hazzan, O. 2005. The construction process
of a framework for teaching software development methods,
4. CONCLUSION Computer Science Education, 15:4, 275–296.
In this paper, we present our Eclipse plug-in to automating the
process of managing UCD activities at the Integrated [7] Dubinsky, Y., Catarci, T., Humayoun, S., and Kimani, S.
Development Environment (IDE) level during the development 2007. Integrating user evaluation into software development
time of software projects. To develop the framework we were environments, 2nd DELOS Conference on Digital Libraries,
inspired by use cases that emerged when performing UCD Pisa, Italy.
activities with the agile teams. Using this plug-in, the software [8] Gulliksen, J., Goransson, B., Boivie, I., Blomkvist, S.,
project team can create experiments, adding users, analyzing Persson, J. and Cajander, A. 2003. Key principles for user-
results and tracing back it to code for their developed or in- centered systems design. Behaviou & Information
progress product. By automating the process of managing UCD Technology, Vol. 22, No. 6, 397–409.
activities the chances of creating quality products with adequate
[9] Hazzan, O. and Dubinsky, Y., Agile Software
level of usability become high, as it helps to get benefits of user
Engineering, Undergraduate Topics in Computer Science
experience during development time.
Series, Springer-Verlag London Ltd, 2008, in press.
In future, we intend to continue work on the developed plug-in to
[10] Hudson, W. 2003. Adopting User-Centered Design within an
manage more UCD activities. Further, we also intend to evaluate
Agile Process: A Conversation. Cutter IT Journal, (16), 10
the developed product on big scale with different size of software
http://www.suntagm.co.uk/design/articles/ucdxp03.pdf
development teams.
[11] Hwong, B., Laurance, D., Rudorfer, A., and Schweizer, A.
2004. User-Centered Design and Agile Software
5. ACKNOWLEDGMENTS Development Processes. Siemens Corporate Research
Our thanks to the plug-in developers from Technion IIT whose http://www.scr.siemens.com/en/pdf/se_pdf/rudorfer-1.pdf.
product is presented in this paper: David Ben-David, Tomer
Einav, Yoav Haimovitch, Barak Nirenberg, Laliv Pele, and Alon [12] Landauer, T. K. 1995. The trouble with computers:
Vinkov. usefulness, usability, and productivity, MIT Press.
[13] McInerney, P., and Maurer, F. 2005. UCD in agile projects:
6. REFERENCES Dream team or odd couple?. ACM Interactions, 12(6), 19 -
[1] Agile Alliance 2001. Manifesto for Agile Software 23.
Development. Technical Report by Agile Alliance, [14] Norman, D. 2006. Why Doing User Observations First Is
http://www.agilealliance.org. Wrong, ACM Interactions, July-August 2006.
[2] Blomkvist, S. 2005. Towards a Model for Bridging Agile [15] Sharp, H., Rogers, Y., and Preece, J. 2007. Interaction
Development and User-Centered Design. Published as a Design: Beyond Human-Computer Interaction. 2nd Edition.
book chapter: Seffah, A., Gulliksen, J., and Desmarais, M., Willey.
(eds.). Human-Centered Software Engineering – Integrating
Usability in The Development Process. Springer, Dordrecht, [16] Talby, D., Hazzan, O., Dubinsky, Y. and Keren, A. 2006.
The Netherlands, 217-243. Agile software testing in a large-scale project, IEEE
Software, Special Issue on Software Testing, 30-37.
[3] Blomkvist, S. 2006. User-Centered Design and Agile
Development of IT Systems. IT Licentiate theses,
Department of Information Technology, Uppsala University.