Evaluation is a key activity in developing high level UX. This paper argues, however, that the early phases form the basis for UX, and evaluation should be seen only as a supportive role in ensuring UX. Four (4) main early activities are identified and their challenges briefly discussed.
EVALUATION IS ALWAYS A LATE, REACTIVE ACTIVITY Before any evaluation can be made, some design solutions need to be produced. Moreover, the design solutions need typically be developed to be ‘working ones’ in the sense that a user can try to use them.
The only way to make evaluation effective is to make changes to the design solutions, based on the results of the evaluation.
Making changes to design solutions is always a reactive activity where resources are needed. The more and bigger problems are found in the design solutions, the more work is needed for redesign and possibly for many sets of iterative design – evaluate cycles.
Further, as Cooper
If the product has ambitious UX targets, the less effective evaluation-driven development will be. The author argues that in-depth ‘thinking’ is needed for the generation of design solutions with high level UX. Evaluations reveal which design solutions work and which do not; but evaluations are not ‘design solution generators’.
Overall, it is more effective if the design solutions would be of ‘good quality’ before any UX evaluation is started. This would lead to less need for changes and redesign during evaluation.
Well-thought and elaborated design solutions reduce the need for redesign. But how one can achieve high-level design solutions, before evaluation? In the following, key activities are identified.
KEY PRE-EVALUATION ACTIVITIES OF UX What should then be done, to produce design solutions of good quality form the beginning, before any UX evaluation? In the following, four interrelated early activities are identified that can, and should be done. But each of them is challenging. 1. Defining the desired business impact of UX This is a key, fundamental activity. Before any project starts, one should define what do we want to achieve with good UX in the first place? What is the desired business impact of UX? This is a business issue, and is dependent on the specific business context of the product/ system. The desired UX impact should be defined in a measurable way.
As an example, in one of the author’s projects, a desired usability impact was defined as to reduce 90% of users’ support calls, compared with the old system.
The author argues that similar impact targets should be
defined for UX, too. As said, this activity is very much
business related: the appropriate impact measures and target
values are business related and a business decision.
This is an important activity for guiding and resourcing UX
work. The more important UX is for business, the more
resources one can be expect from the business management
for UX work.
2. Understanding the system’s and users’ world
Understanding users’ world is a well-known activity. It is
called the definition of context of use in ISO 9241-210:
know users’ goals, tasks and environments of use.
Wellknown techniques for understanding user’s world (work)
are interviews and contextual inquiry
The author introduces another, complementary activity: defining system’s world. This is even a more profound one to be defined. System’s world is about defining what is to be built.
The background for this activity is the author’s experience in consulting work. When joining system development projects and trying to understand ‘what system’ is to be built, the case is always that no one in the project team can explain it in a systematic and analytical manner. Not even persons who have worked in the domain for many years. The author argues – although has not done literature studies or such – that this important activity is not generally recognized. In this paper, the author does not give a more elaborated definition for what is ‘system’s world’ – because the author does not have it. The author has experience on carrying out this activity and modeling the results (system’s world), with absolute excellent customer feedback. But when asking the customers to describe, “what did we exactly produce”, they cannot find any name or term to describe it.
It is obvious that the designers need to understand ‘what is to be developed’ to be able to produce good design solutions. If this knowledge is weak, it is likely that the design solutions include (major) problems. 3. Defining measurable UX targets and giving incentives for achieving them This activity is to define ‘how good UX’ we want to achieve. This activity transforms the desired UX business impact and the understanding of users’ and system’s world into concrete, measurable UX design targets.
Further, it is useful if the design team gets some incentives for achieving the defined UX targets. The more challenging the targets are, the higher incentives the business management should consider.
The usefulness of UX targets with incentives is that the
targets drive the design team for good solutions from the
very beginning of the project (project teams anyway always
have limited budgets and tight time scales). If the UX
targets are ambitious, the design team knows from the
beginning that ‘any design’ would not be acceptable.
For measurable UX targets, one needs to define what is the
measure, what is the measuring instrument, and what is the
target value. For example, a measure may be a SUS
(System Usability Scale
A key challenge here is: how to define the appropriate UX measures and target values? 4. Designing high-quality design solutions This is the ultimate and decisive activity. The designers need to transform their in-depth knowledge of users’ world (activity 2) into design solutions that meet the UX targets (activity 3).
This is obviously dependent on the designers’ talents, creativity and knowledge of HCI. But at more detailed level, the big question is, how to do this? How to transfer a UX target such as “the average SUS score must be at least 90” into a design solution? The “trial and error” – i.e. design and evaluate with users approach might work. But the author has a hypothesis – but no evidence - that in-depth ‘thinking’ when creating the design solutions before evaluations, is required as a basis. IMPLICATIONS FOR RESEARCH The author’s understanding is that these four activities are remarkably less in the focus in UX research than UX evaluation.
The author has some experience and solutions for these activities in cases where usability has been in the development focus. The challenge is not easy, and for developing good UX the challenge may be even more demanding.
The activity 2 – understanding the system’s and users’ world – probably is quite the same, no matter whether usability or UX is the design focus. But defining valid measures for the UX business case, and valid measurable UX targets may be very challenging.
In summary, the author proposes that following activities are key ones for designing good UX, and more research is needed for how to do these in an effective and efficient way: 1.
How to define the desired business impact of UX? How to get an understanding on the system’s and users’ world? How to define UX targets for design?
How to transform this knowledge into design solutions?