<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Usability: A Cybernetics Perspective</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Mikko Rajanen</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Dorina Rajanen</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>INTERACT Research Unit, University of Oulu</institution>
          ,
          <addr-line>Oulu</addr-line>
          ,
          <country country="FI">Finland</country>
        </aff>
      </contrib-group>
      <fpage>28</fpage>
      <lpage>33</lpage>
      <abstract>
        <p>Usability as a concept is well established, encapsulating the rich interaction between different kinds of users, information technology systems, and contexts of use. In the context of sociotechnical systems, technology shapes societies and human interactions, and likewise technology itself is shaped by social, economic, and political forces. Therefore, the sociotechnical landscape is constantly evolving. Usability professionals have developed new usability methods and processes in order to address these changes in the socio-technical landscape. Likewise, the very concept of usability has been evolving to better fit into this ever changing socio-technical landscape. The evolved and adapted concept of usability has been feeding back to its socio-technical environment, thus creating a feedback loop. This position paper reflects this feedback loop and the concept of usability as a means of communication and shared language between stakeholders in the socio-technical systems development context from the cybernetics perspective.</p>
      </abstract>
      <kwd-group>
        <kwd>1 Socio-Technical Systems</kwd>
        <kwd>Usability</kwd>
        <kwd>Cybernetics</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>1. Introduction</p>
      <p>
        A socio-technical systems approach views organizations and societies as amalgamations of technical
systems and social systems, where individuals and technologies interact with each other towards a
common goal [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Therefore, the socio-technical systems approach recognizes that it is vital to facilitate
these interactions between technology and people using it [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. Furthermore, the socio-technical systems
approach recognizes that systems used in workplaces should be technically efficient and, equally
importantly, they should include social characteristics that will lead to high work satisfaction [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. The
socio-technical systems design has also been seen as an important democratizing factor, as it postulates
that users who use the developed system should be involved in shaping the quality of their future work
[
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. This human-centeredness of socio-technical systems approach is in line with the Scandinavian
information systems tradition that has been advocating for this kind of inclusive, ideal and user-centered
adaptation of technology and for design processes where all the different stakeholders and future users
are represented during technical system development for organizations and society [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. The
sociotechnical perspective has for a long time been the axis of cohesion for the information systems (IS)
discipline, providing a common language, widely accepted research orientations, and shared
assumptions and interests in form of communal knowledge, even though it is often forgotten in the IS
discourse today [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
      </p>
      <p>
        In this conceptualization of human-centered socio-technical approach, humans act as active
stakeholders in improving and contributing to their environment. Furthermore, the ideation, design, and
development of new technologies imply that a human influences these technological advances in order
that the technology fits the needs and capabilities of the human and social components. There have been
calls for new, contemporary and open perspectives for socio-technical systems to ensure that the
systems being developed are meaningful to all engaged actors [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. Furthermore, the importance of
human-centered approach and viewing the organizations as self-creating, dynamic and open systems,
have been identified in the literature as a potential path to success with emerging new concepts and
needs, such as smart innovations [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. There have also been calls for revisiting the socio-technical
perspective as the foundation for the IS discipline, to develop recommendations on how researchers,
practitioners, and other stakeholders can contribute to it, and to innovate new ways for researchers,
journals and academic units to advance the field [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
      </p>
      <p>
        In the context of socio-technical systems, the interactions between technology and human are
mutual, as both of them influence each other. Technology shapes human interactions, relations and
societies, and likewise technology is also shaped by social, economic, and political forces [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ].
Furthermore, the socio-technical perspective addresses the individuals using and developing the
technology as well as the technology itself [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. As this perspective does not favor technological aspects
over social, and focuses on efficiency and productivity as well as on human-centric values such as
individual well-being, equality and empowerment, it is in line with the core values of the usability
researchers and practitioners (see e.g. [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]).
      </p>
      <p>
        Usability research has been developing new artefacts, such as methods, technologies, and processes
for designing and evaluating the human-technology interaction, in order to answer the emerging
challenges, technological advances and developments in socio-technical contexts. The experiences
from these new artefacts have been communicated back to the researchers, who then have assessed the
fit of these artefacts and changed them where necessary. This has been in line with the socio-technical
perspective where the results have been closely monitored to establish if they have led to improvements
in technology use and quality of working life [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ].
      </p>
      <p>
        Simultaneously, usability researchers have also updated and adapted the concept of usability as the
foundation of developing these new artefacts. This evolved concept of usability has been driving and
guiding the usability research and practice. This has also been in line with the socio-technical
perspective where the theoretical concepts have been formulated and then tested against the empirical
experiences in order to understand better if the developed theoretical concepts fit the practice and help
to understand it better [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ].
      </p>
      <p>In this position paper, we look at usability as a means of communication and shared language
between stakeholders that provides a new, contemporary and open perspective on socio-technical
systems research and practice, and to this end we employ concepts from cybernetics.
2. Evolution of usability as a concept</p>
      <p>In the early days of information technology, the developers were usually also the users of the systems
that they were developing. Therefore, the designers of the new systems knew their own characteristics,
needs and the contexts of use, and as a result they could easily design these systems to be tailored to fit
their own work. However, as the organizations recognized the business potential of these information
technology systems and started to use them as integral parts of their operations, the user base of these
systems expanded considerably. As the information technology and systems became ubiquitous in
organizations, society, and later also in everyday life and work, the users could be of any age,
background, level of experience, technological skills, and knowing vital context-dependent information
about their own work. As a result, the designers no longer knew the users, nor did they have any direct
knowledge about the contexts where the systems were to be used. As a result, the developed systems
did not answer the functional needs of the users or organizations, had unnecessary features while lacking
features that were vital to the users and organizations, and these systems did not fit the established work
processes. Therefore, it was recognized that the designers of information technology and
sociotechnical systems needed to obtain information about the future users, their tasks, and the contexts of
use. Furthermore, it was realized that these practitioners needed new methods and processes to gather
information from individual users, groups and organizations, and to turn this information into designs
of easy to use and effective systems. In order to develop these new methods and processes for design
practice, the researchers needed new conceptualizations and shared language for understanding the
interactions between users and technology.</p>
      <p>
        The concept of usability emerged as one of the quality constructs in the human-computer interaction
(HCI) community in early 1980s to characterize visual displays and interactive systems from the
perspective of users [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. This new conceptualization was intended to capture the attributes of interactive
software products that would make them usable and that can be incorporated in design and further
evaluated [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. Usability was first conceptualized as a property of the system itself. As a result the early
usability research focused on finding and documenting these systemic usability properties, which were
to be taken into account in the design of the new and better systems.
      </p>
      <p>
        However, some usability researchers saw this technology-centric paradigm as being problematic and
wanted to conceptualize usability through research and documentation of the physical and cognitive
characteristic of the users, which could then be taken into account in the design of the system. This
second usability paradigm has resulted in for example cognition-based usability guidelines, such as the
basic design of graphical user interface elements we still use today (see e.g. [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]).
      </p>
      <p>A third, later paradigm conceptualized usability as the rich interaction between a particular user and
a particular system in a particular context of use. In this paradigm, the usability was incorporated in the
interaction between the user and the technology, each interaction being unique in such way that no
universal best design guidelines could be made. User-based usability evaluation methods focusing on
individual users, such as usability testing, originate from this paradigm.</p>
      <p>In addition to these major paradigm shifts, the concept of usability also evolved to adapt to the
advances in technology and other emerging needs in the socio-technical landscape. As a result, the
focus of usability research and practice has been constantly expanding. At first, the concept of usability
mainly focused on how effective the system was for the users to use, which is the degree to which the
designed interface enabled the intended task accomplishment by a user.</p>
      <p>However, soon the need to minimize the resources a user needed to expend to achieve their tasks
was identified as an important concern, so the concept of usability was expanded to include also
efficiency. As the number and complexity of the information systems increased, the need for a
standardized process to design for better usability was identified, and the process of user-centered
design was introduced. The user-centered design advocated for several small usability design and
evaluation activities spanning the entire development process instead of few larger usability evaluations
at the end of the process when the design was already finalized and the changes would be expensive.</p>
      <p>
        As the use of technology expanded from the work context into the everyday life, the need for taking
into account the more subjective pleasantness and ease of use as experienced by an individual user was
recognized, and as a result the concept of usability was further expanded in early 2000s with the user
experience aspect. The reason behind this evolution was the need to gain a better understanding on the
emotions of users before, during and after the use of the technology, as the designers wanted better
explanations for why some users would prefer one design over other. This was as a result of usability
studies showing that appraisal of technology was influenced also by aesthetic aspects of the design, as
well as user’s expectations before and reflections after the use [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ].
      </p>
      <p>
        These evolutions to the concept of usability have been encapsulated into international standards,
which were composed by committees of usability professionals. Their goal was to incorporate the
current views and best practices of the time from usability research and practice. These different
usability standards act as time capsules, having different approaches, viewpoints and conceptualizations
to usability, thus representing the views and best practices of their time (see e.g. [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]).
      </p>
      <p>Overall, practical usability work must advance through new methods, technologies, and processes
from the research to keep up with the emerging challenges and developments in socio-technical
contexts. And as a response, usability researchers must update and adjust the concept of usability to
develop better new methods, technologies, and processes for the practice. Identifying these feedback
loops between usability research and practice on social and technological levels, which has been
instrumental for evolving the concept of usability and its practices, leads us to take a look at the field
of feedback loops, the cybernetics.
3. Cybernetics perspective</p>
      <p>
        The founders of cybernetics, Stefan Odobleja [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] and Norbert Wiener [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ], argued that in addition
to a system producing a certain output in reaction to events in the environment, it is also important that
the system continuously monitor the effects of the output and reacts to these effects accordingly. This
forms a feedback loop that constantly reduces any discrepancy between the desired state and the
observed state. A thermostat that controls a radiator to keep the room at a desired temperature is an
example of a simple feedback loop. The thermostat measures the temperature of the room and switches
the heating on when the measurements show the temperature being too cold and shuts off the heating
when the room has reached the desired temperature. This first-order cybernetics, or the traditional
cybernetics, perspective focuses on aspects of the defined systems, instead of the actual situation and
context, concentrating on the local state of the system [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]. From this perspective, the system was
designed to be in isolation from its whole environment, often being portrayed as a black box with some
inputs and outputs from the perspective of its environment. However, this approach was found limited
when the technological systems became more complex and the number of interactions between humans
and technology increased. While the traditional cybernetics was very useful for engineers for designing
automated systems, it overlooked the role of and interactions with the outside observer, who might be
the designer or the user of the system, or another system [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ].
      </p>
      <p>
        Wiener also argued that the study of messages and of communication facilities is the only means of
understanding society as a whole, defining the cybernetics as “the scientific study of control and
communication in the animal and the machine” [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. Wiener predicted in 1950s that in the future,
communication between humans and technology would increase rapidly due to the technological
advances, and therefore the concepts of communication should be refined according to the increased
complexity of these interactions [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. Accordingly, cybernetics developed a hierarchical model with
different interconnected levels of abstraction, when applied to complex technologies. This was
manifested by the introduction of second-order cybernetics [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ].
      </p>
      <p>
        Second-order cybernetics was introduced to the field of cybernetics to address the increasing
complexity of interactions between humans and technology by expanding and including the outside
observations of the system, as well as the communication related to it [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]. Second-order cybernetics
argues for human-centeredness, as the systems are manifested in the form of interfaces used by humans,
and calls for human-centered design that aims at understanding how users understand and behave to
create best possibilities for the interactions and communications between users and technology [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ].
Second-order cybernetics, also known as cybernetics of cybernetics, or the recursive application of
cybernetics to itself, introduces a feedback loop, or circularity, in which the user interfaces determine
the interactions between humans and technology and where this interaction is designed based on
observations of users. Therefore, second-order cybernetics has been described as cybernetics in which
“circularity is taken seriously” [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]. As a comparison, first-order cybernetics has the engineering point
of view, studying a system as if it is a passive, objectively given thing, to be freely observed,
manipulated, removed, and taken apart, while the second-order cybernetics recognizes that a system is
an agent of its own right, as it interacts with other agents, observers, and social systems. [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ].
      </p>
      <p>Having looked at the evolution of usability as a concept as well as the theory of cybernetics, we
combine these two concepts and take a look at how usability could be conceptualized from the
cybernetics perspective.
4. Usability: a cybernetics perspective</p>
      <p>
        There are studies indicating different feedback loops in the field of human-computer interaction.
Some studies observe that the system interfaces are designed by humans, namely the designers, who
observe users in order to design the system based on these observations (see e.g. [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]). Some studies
have identified the usability work as a form of continuous feedback from the system development
process perspective (see e.g. [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ]). Further studies have had cybernetics perspective to the requirements
engineering to gather operational data for user profiling in the system development context (see e.g.
[
        <xref ref-type="bibr" rid="ref19">19</xref>
        ]). Other studies have identified the need to expand this feedback loop to include the context of use,
where the system adapts to the characteristics, needs, and states of individual users based on real-time
data before, during, and after the actual use of the system (see e.g. [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ]). When taking into account the
feedback mechanism and evolution of the usability itself as illustrated above, we can put these studies
and their different viewpoints together, and reason that usability as a concept could be viewed from the
perspective of second-order cybernetics.
      </p>
      <p>
        As we have discussed previously, usability has evolved from simplistic systemic property into a
holistic concept that covers the essential attributes of socio-technical systems in their development and
use. It allows all stakeholders to be involved in the development of the system, and it acts as a lens for
observing the socio-technical landscape (see e.g. [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ]). Furthermore, we have seen that this involvement
of stakeholders in the context of practical usability work in turn feeds back to the concept of usability.
This evolved and adapted concept of usability has in turn been feeding back to its socio-technical
ecosystem, creating a feedback loop as described in the theories of second-order cybernetics which state
that knowledge cannot be passively absorbed from the environment, but it has to be actively constructed
by the system itself through its interactions with observers, social systems and other systems [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]. We
postulate that this viewpoint has striking resemblance to how the concept of usability has been evolving
when the technological advances and changes in socio-technical landscape have rendered the existing
paradigms and conceptualizations inadequate.
      </p>
      <p>
        Therefore, we reason that usability as a concept is constantly evolving, driven by its interactions
with its ecosystem and its stakeholders through different forms of feedbacks between users, developers,
researchers, and other stakeholders and adapting to its ecosystem based on this feedback. As a result of
this feedback loop, the concept of usability has evolved over time to capture new attributes and
meanings, the observer and the system co-evolving together as theorized in second-order cybernetics.
Usability methods and processes, such as usability testing, offer constant feedback for practitioners
during their design process and for the researchers observing the use of the methods and processes they
have developed. Furthermore, there have been calls for systems that offer real-time personalization of
the system and its interface during the use (see e.g. [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ]. Therefore, we argue that having a cybernetics
perspective on the concept of usability and its evolution in socio-technical systems context would help
the researchers and practitioners to better understand and contribute to the evolution of usability and
socio-technical systems.
      </p>
      <p>In addition, we postulate that when educating future practitioners and researchers, this perspective
would be beneficial in creating a better understanding of the evolutionary history of usability and the
notion that the concept of usability is not set in stone, but is constantly evolving to fit the needs of the
socio-technical landscape. As such, the future practitioners and researchers must understand that they
have to update their knowledge base constantly in order to benefit from the latest usability concepts,
methods and processes that are fit to the current technological and social context. Furthermore, they
should understand their role in this evolution.</p>
      <p>
        One of the conundrums in the field of cybernetics is how to recognize when a concept, system or
device, which has the capacity to evolve over time to fit into its environment and arising needs, has
indeed been evolving with a new property [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]. A community of observers has been identified as a way
to verify that there is an emergent property [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]. We posit that usability professionals creating new
international standards can be seen as such community of observers, who identify new aspects and
needs from usability research and practice, and therefore the international usability standards reflect
this evolution of usability as a concept, encapsulating the views and best practices of their time.
5. Discussion and conclusions
      </p>
      <p>
        In this position paper we outlined the concept of usability from the second order cybernetics
perspective, where the rich interaction between socio-technical systems, individuals, research of this
interaction, and the design of these systems form a feedback loop as described in the theories of second
order cybernetics. We postulate that conceptualizing usability through cybernetics perspective could be
one potential answer to the calls for new, contemporary and open perspectives ensuring that the
developed socio-technical systems will be meaningful to all engaged actors, as well as potentially
leading to new ways for researchers to recommit to the IS discourse from the socio-technical perspective
(see e.g. [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ], [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]). The fields of usability, socio-technical systems, second-order cybernetics, and
Scandinavian information systems research share their focus on human-centeredness, so these fields
could develop a shared axis of cohesion and the concept of usability could act as a bridge between these
fields.
      </p>
      <p>Our aim was to outline the history of usability research and practice from the cybernetics
perspective, as this perspective allows the researchers to further conceptualize, encapsulate and analyze
the role of usability in socio-technical systems, as well as to better understand and study the evolution
of the concept of usability, and to position the concept of usability as means of communication and
shared language between different fields of research that share the human-centered approach.
Furthermore, the practitioners could adopt and utilize this perspective to better understand the
development of new socio-technical systems fitting the needs of users and organizations, and to
understand the field of usability as a constantly evolving entity. Further empirical and theoretical
research is still necessary, as the evolution and the role of the concept of usability from cybernetics
perspective should be refined further. We hope that this position paper will further revitalize the
discussion and research of the role of usability as the core concept in socio-technical systems
development.
6. References</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>Bostrom</surname>
            ,
            <given-names>R. P.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Heinen</surname>
            ,
            <given-names>J. S.</given-names>
          </string-name>
          (
          <year>1977</year>
          ).
          <article-title>MIS problems and failures: A socio-technical perspective. Part I: The causes</article-title>
          .
          <source>MIS Quarterly</source>
          ,
          <volume>17</volume>
          -
          <fpage>32</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>Mumford</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          (
          <year>1983</year>
          ).
          <article-title>Designing human systems for new technology: the ETHICS method</article-title>
          (pp.
          <fpage>127</fpage>
          -
          <lpage>57</lpage>
          ). Manchester: Manchester Business School.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>Mumford</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          (
          <year>2006</year>
          ).
          <article-title>Researching people problems: some advice to a student</article-title>
          .
          <source>Information Systems Journal</source>
          ,
          <volume>16</volume>
          (
          <issue>4</issue>
          ),
          <fpage>383</fpage>
          -
          <lpage>389</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>Greenbaum</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kyng</surname>
          </string-name>
          , M. (eds.) (
          <year>1991</year>
          ). Design at Work.
          <article-title>Cooperative Design of Computer Systems</article-title>
          . Lawrence Erlbaum Associates, New Jersey.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <surname>Sarker</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chatterjee</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Xiao</surname>
            ,
            <given-names>X.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Elbanna</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          (
          <year>2019</year>
          ).
          <article-title>The sociotechnical axis of cohesion for the IS discipline: Its historical legacy and its continued relevance</article-title>
          .
          <source>MIS Quarterly</source>
          ,
          <volume>43</volume>
          (
          <issue>3</issue>
          ),
          <fpage>695</fpage>
          -
          <lpage>720</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>Bednar</surname>
            ,
            <given-names>P. M.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Welch</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          (
          <year>2020</year>
          ).
          <article-title>Socio-technical perspectives on smart working: Creating meaningful and sustainable systems</article-title>
          .
          <source>Information Systems Frontiers</source>
          ,
          <fpage>1</fpage>
          -
          <lpage>18</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>Rip</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Kemp</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          (
          <year>1998</year>
          ).
          <article-title>Technological change</article-title>
          .
          <source>Human Choice and Climate Change</source>
          ,
          <volume>2</volume>
          (
          <issue>2</issue>
          ),
          <fpage>327</fpage>
          -
          <lpage>399</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>Briggs</surname>
            ,
            <given-names>R. O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nunamaker</surname>
            ,
            <given-names>J. F.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Sprague</surname>
            ,
            <given-names>R. H.</given-names>
          </string-name>
          (
          <year>2010</year>
          ).
          <article-title>Social aspects of sociotechnical systems</article-title>
          .
          <source>Journal of Management Information Systems</source>
          ,
          <volume>27</volume>
          (
          <issue>1</issue>
          ),
          <fpage>13</fpage>
          -
          <lpage>16</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <surname>Bevan</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Carter</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Harker</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2015</year>
          ).
          <article-title>ISO 9241-11 revised: What have we learnt about usability since 1998?</article-title>
          <source>In Proc. of Int'l Conf on HCI</source>
          <year>2015</year>
          , LNCS, Springer, vol.
          <volume>9169</volume>
          , pp.
          <fpage>143</fpage>
          -
          <lpage>151</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>Thüring</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Mahlke</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2007</year>
          ).
          <article-title>Usability, aesthetics and emotions in human-technology interaction</article-title>
          .
          <source>International Journal of Psychology</source>
          ,
          <volume>42</volume>
          (
          <issue>4</issue>
          ),
          <fpage>253</fpage>
          -
          <lpage>264</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Marghescu</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          (
          <year>2009</year>
          ).
          <article-title>Usability evaluation of information systems: A review of five international standards</article-title>
          .
          <source>In Information Systems Development</source>
          (pp.
          <fpage>131</fpage>
          -
          <lpage>142</lpage>
          ). Springer, Boston, MA.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>Odobleja</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>1938</year>
          ).
          <article-title>Psychologie consonantiste</article-title>
          (Vol.
          <volume>1</volume>
          ). Librairie Maloine.
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>Wiener</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          (
          <year>1948</year>
          ).
          <article-title>Cybernetics or Control and Communication in the Animal and the Machine</article-title>
          . Technology Press.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <surname>Heylighen</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Joslyn</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          (
          <year>2001</year>
          ).
          <article-title>Cybernetics and second-order cybernetics</article-title>
          .
          <source>Encyclopedia of Physical Science &amp; Technology, 4</source>
          ,
          <fpage>155</fpage>
          -
          <lpage>170</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <surname>Powers</surname>
            ,
            <given-names>W. T.</given-names>
          </string-name>
          (
          <year>1973</year>
          ).
          <article-title>Behavior: The control of perception</article-title>
          . Chicago: Aldine.
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <surname>Krippendorff</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          (
          <year>2019</year>
          ).
          <article-title>The cybernetics of design and the design of cybernetics</article-title>
          .
          <source>In Design Cybernetics</source>
          (pp.
          <fpage>119</fpage>
          -
          <lpage>136</lpage>
          ). Springer, Cham.
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <surname>Glanville</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          (
          <year>2004</year>
          ).
          <article-title>The purpose of second-order cybernetics</article-title>
          . Kybernetes,
          <volume>33</volume>
          (
          <issue>9</issue>
          /10),
          <fpage>1379</fpage>
          -
          <lpage>1386</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <surname>Genov</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          (
          <year>2005</year>
          ).
          <article-title>Iterative usability testing as continuous feedback: A control systems perspective</article-title>
          .
          <source>Journal of Usability Studies</source>
          ,
          <volume>1</volume>
          (
          <issue>1</issue>
          ),
          <fpage>18</fpage>
          -
          <lpage>27</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <surname>Liu</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhou</surname>
            ,
            <given-names>Q.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Liu</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Cao</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          (
          <year>2017</year>
          ).
          <article-title>Requirements cybernetics: Elicitation based on user behavioral data</article-title>
          .
          <source>Journal of Systems and Software</source>
          ,
          <volume>124</volume>
          ,
          <fpage>187</fpage>
          -
          <lpage>194</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <surname>Rajanen</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Rajanen</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          (
          <year>2017</year>
          ).
          <article-title>Personalized gamification: A model for play data profiling</article-title>
          .
          <source>In Proc. of DDGD@ MindTrek</source>
          (pp.
          <fpage>26</fpage>
          -
          <lpage>33</lpage>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <surname>Rajanen</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Rajanen</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          (
          <year>2020</year>
          ).
          <article-title>Usability as speculum mundi: A core concept in sociotechnical systems development. Complex Systems Informatics</article-title>
          and
          <string-name>
            <surname>Modeling Q.</surname>
          </string-name>
          , (
          <volume>22</volume>
          ),
          <fpage>49</fpage>
          -
          <lpage>59</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <surname>Cariani</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          (
          <year>1993</year>
          ).
          <article-title>To evolve an ear. Epistemological implications of Gordon Pask's electrochemical devices</article-title>
          .
          <source>Systems Research</source>
          ,
          <volume>10</volume>
          (
          <issue>3</issue>
          ),
          <fpage>19</fpage>
          -
          <lpage>33</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>