In this position paper it will be argued that the changed nature of collecting user experiences and evaluation in cognitive ergonomic design implies that, in addition to existing methodology, cognitive ergonomic design and theory should move focus from evaluating designs afterwards to using on-line usage data collection as a main input to direct and steer the design process. Cognitive ergonomic products and services should be designed and implemented within the context of use, whereby the design process is early and continuously informed and evaluated by collecting user experiences and usage data from actual use of the product or service.
It is common knowledge that Cognitive Ergonomics design should or rather ought to be characterized as users-centered, iterative, evidence-based, and addressing the entire context of use, where user-centered refers to the idea that we should address user needs and requirements rather the requirements that designers or other stakeholders might be interested in, where iteration towards a set of design goals is to be understood as addressing our inability to get things right, where evidence-based refers to the practice that we have to test instead of judge for ourselves that our design suffice, and finally, where contextual intends to capture the whole relevant working-context rather then only the direct human-computer interaction aspects.
Designing in this manner is not a given practice; Steen [
In this paper I will argue that Cognitive Ergonomics needs to go one step further.
In business management it is common practice to improve the quality of business
processes by checking their effectiveness every once in a while in a systematic and
operational matter (cf. Boddy et al, [
Also in Software Engineering (SE), it is common practice to design complex
software programs in a agile manner with constant verification between business needs
and software functionality (cf. Schwaber and Beedle, [
This paper will first argue that Cognitive Ergonomics (CE) and Human Computer Interaction (HCI) are rather not very different; hence we may use the terms as exchangeable. Then we will attempt to discuss the notion of generations in methodology, just like we can -so to speak- explain the research questions asked in HCI, afterwards, by pointing at the generation of ICT devices that HCI has had to deal with. Subsequently, on a personal note, we describe which developments in CE, HCI and SE inspired me to move from a linear framework towards a more exploratory, prototypical and circular framework for design. Finally, a few approaches and cases are described about how to achieve progress in these matters. 2
In this section, it is argued that we should not distinguish between Human-Computer Interaction (HCI) and Cognitive Ergonomics (CE). HCI used to be special because of the difference between physical devices and virtual machine; that is: between how the machine presented itself to anyone using it, and the way in which the machine actually worked. In addition, HCI was relatively late to adapt the notion of social practices of use, probably because of the focus on the initial workstation and personal computer, aimed at information processing tasks and even without or with rather primitive networking facilities.
On the moment, virtually all designs for consumer good, apps, websites or production systems tend to be complex; in not revealing their internal workings, as well as social; intended for or using social relationships. Here, social can be taken to means actually doing things together with other people as in CSCW, or it can be taken as using other people's insights and experiences as in social media and, or it can be taken to refer to the necessary communication means as part of a social structure like a team, an organization or a company as in business support systems.
We can even see complexity and social aspects in some advanced AI systems that are based on recognizing successful patterns of conduct among different communities of practice. Other examples are social collections of knowledge like in Wikipedia, community-based collections of frequently-asked questions and question-answering systems.
In almost all designs in HCI and CE, two aspects are shared. First, there is no evident 'best' solution in the way how to organize things, and second, designs allow that actions, tasks and responsibilities can be (re)designed to either devices, people or the wider organization. As such, by sharing the characteristics of complexity and social order, only for very specific goals does it make sense to distinguish Cognitive Ergonomics and Human-Computer Interaction. 2.1
In an earlier paper [
“Each generation of ICT technology may be characterized by the questions it raised in the field of Human Computer Interaction (HCI):
— the mainframe technology with its expensive hardware asked for the selection of specially trained personnel for reliable data processing;
— the minicomputer era asked for software ergonomics using structured methods;
— the personal computer raised the question for usable applications and usability evaluation;
— the game computer asked how software could create an emotional appeal - the fun and immersion of software
— the smartphone app and the wireless web put the question forward about the user experience of the using application.
In the current ICT technology generation, traditionally distinct IT functions
such as data collection, processing and data access have converged
communications into small mobile, networked devices which provide functions or
services that are no longer tied to a specific time or (work) place” [
Here, we extend the notion a little further and argue that not only methods and principle solution but also research paradigms or how to investigate design options may be distinguished between generations. Regardless of claiming that such generations are 'caused' by the hard- and software used, the state of science or any cultural aspects in using information systems, it may be useful to distinguish in CE and HCI between the engineering paradigm, the problem-solving paradigm, the taskperformance paradigm, the mental-model paradigm, user experience paradigm, and the engagement and gamification paradigm, each with a fitting design approach and a main focus of attention: engineering paradigm - linear design, operator selection task-performance paradigm- guidelines and directives - human factors engineering problem solving paradigm - structured design, human factors in software design mental model paradigm - iterative design, user centered design user experience paradigm - usability engineering and prototyping user engagement and gamification paradigm - co-design and co-creation, service design, habit formation
In comparison to the hardware-based ICT generation, there is and may not be a very clear-cut distinction between the generation, also because the paradigms overlap in time since they are carried by a community and not because there is a best one, like with models or theories.
In the engineering paradigm, the core is the machine and both design and the human operator are fit to the device.
In task-performance, the human operator is seen as an element the can be made to perform better or worse, depending on the circumstances; as such, the circumstances are adapted to improve task performance.
The problem-solving paradigm is the first to acknowledge that design is complex and requires problem solving; as such, design is supported by structure and there is attention for the psychological or task-appropriate aspects of programming languages.
With the mental-model paradigm using computers is accepted as a principle cognitive endeavor and in both design and use, the focus is on user needs and requirements.
Within the user experience paradigm, the focus leaves the purely rational taskoriented realms and attention shifts towards directly measuring the usability of alpha and beta prototypes (even using discounted-discounted usability testing).
Finally, the focus shifts further away from rational task performance with on the
one side: service design aims at engaging users to maximize sales, or so-called
'conversion' and, on the other side: design for habit formation, or getting the user
"hooked" with techniques like gamification, in order to make users come back and
provide useful data, according to Eyal and Hoover [
In the user engagement paradigm, co-creation and co-design are used to acquire
proper input from each real-world stakeholder, such as the user as the expert in his or
her own life, as Sanders and Stappers [
Co-creation also refers to the design of the internet platform and algorithms that others, such as users, provide with their data to receive a set of services, as in social media. Apart from suiting external goals, the engagement paradigm may also be used to create applications that genuinely fit the users' tasks; as such, it may also be used to design a humane technology.
De Haan [
The idea of a conceptual user-centered design as exploration of the design space
comes close to Woods [
User-centeredness motivates a movement towards exploratory design, design as a
conceptual activity frees it from implementation issues and makes it possible, and the
agility and design exploration ensure that during design, the focus stays on the user.
According to Steen [
Apart from these more theoretically-driven motivations, also other sources of
inspiration turned out to be relevant. First, in usability evaluations, it is a common insight
that users are only able to reliably and validly judge the usability of designs when
they have been provided with the opportunity to actually use the design in practical
circumstances. Regarding design evaluation, Phil Turner [
In this context, the standard experimental and the questionnaire are rather
expensive but poor methods for collecting design ideas and experiences, compared to the
whole range of more-lightweight tools that are common in media design, such as
paper- and clickable prototypes, role-playing games, storyboards, etc. [
Secondly, there is the insight gathered from teaching design in a media design environment where a design virtually never concerns a final or finished product but is often an intermediate result of on an ongoing design- or design-improvement process. A website, an internet shop or a phone app are always subjected to a process of continuous quality improvement, in which, for example, sales figures for different versions of a online shop or usage data from alternative design variations of a phone apps are used to select the even marginally better design option.
As such, similar to the notion that design is for 90% concerned with redesign and
maintenance, design is a process moving from idea to prototype to improved
prototype, etc. This observation begs the question why HCI projects are so often aiming at
finished design products instead of -like in agile design- working prototypes (cf.
Schwaber and Beedle [
In the third place is the insight that more and less intelligent networks enable for
easy, effortless and almost limitless collection of data and measurement. As such,
automatic and online data collecting is expected to actually replace a considerable
amount of experimental testing and verification in the natural sciences, according to
Shapiro et al. [
Regarding online interactive products, like websites, this strategy has been readily implemented (as AB testing) where the results of using different versions or prototypes are experimentally compared in the real-world context of use, generally even without the users being aware that they are participating in an comparison study.
The fourth source of inspiration follows from the last observation: regarding online
products and prototypes, it is relatively easy to imagine to collect usage data because
the products themselves are accessed and used over the internet. Holström-Olsson and
colleagues [
All the aforementioned theoretical reasons and practical inspirations have been
implemented in a number of areas, particularly in the advanced tech arena, like
interactive websites, smart-apps, the redesign of real-world products into online products,
and in Ubiquitous Computing and Internet of Things applications (cf. de Haan [
As an illustrative example, in their conjoint PhD thesis, van Kollenburg and Bogers
[
Within the design exploration approach, the parents and children are not merely used
to determine requirements and to evaluate the design but are continuously and
actively interacting and participating with the design team via meetings, an online app, and
behavioral data. After analyzing the project with respect to both the user-experience
and the prototype-centric perspective, they conclude that data-enabled design of
intelligent eco-systems should involve both behavioral data as well as experience data, and
that the design cycle is best portrayed as a combination of a real-life loop, in which
the prototype is used and data is collected and a design research loop in which
reflection and creative design take place in an 8-shaped form [
Regardless of the direct conclusion that the PhD candidates draw about their own research and development projects, their study excellently illustrates how a userscentered, conceptual and design-exploratory approach may proceed in a socially structured complex environment. In addition, both the development project as well as the research investigation so-to-speak prove the pudding for Cognitive Ergonomic Exploratory Design - certainly better then an average student IoT application would ever do.
This position paper argued for the further development and application of Exploratory Design in Cognitive Ergonomics. It was argued that, in the present circumstances we may equate Cognitive Ergonomics and Human-Computer Interaction since both deal with complex and socially-structured task environments.
Furthermore, the argument for Exploratory Design, as a User-Centered Design approach that focusses on design concepts and using an agile exploration of the designspace during the actual process of design derives from an extrapolation of paradigms for research in CE and HCI, as well as several mostly theoretical considerations regarding elements of Exploratory Design, and on a more personal level, a number of inspirations regarding the possibility and need for such a design approach.
Finally, the research and development project by van Kollenburg and Bogers [