<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Anthony Stein</string-name>
          <email>anthony.stein@uni-hohenheim.de</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Sven Tomforde</string-name>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Jean Botev</string-name>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Peter R. Lewis</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Ontario Tech University</institution>
          ,
          <country country="CA">Canada</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>University of Hohenheim</institution>
          ,
          <country country="DE">Germany</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>University of Kiel</institution>
          ,
          <country country="DE">Germany</country>
        </aff>
        <aff id="aff3">
          <label>3</label>
          <institution>University of Luxembourg</institution>
          ,
          <country country="LU">Luxembourg</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Technology today is becoming increasingly autonomous, already comprising many features typically associated with human, animal, and even plant behaviour: for example decisionmaking, problem-solving, prediction, adaptation, and selfregulation. Drawing on and embodying techniques from artificial intelligence (AI) and bio-inspired computing, many so-called intelligent, smart, and self-adaptive systems are designed with the explicit intention of replicating such behaviours. At the same time, the study of artificial life has explored many properties of living systems, both as they are found in nature and as they can be built or conceived of by humans. This has exposed a large variety of mechanisms that produce what we call qualities typically associated with life. Examples include self-organisation, homeostasis, self-replication, evolution, learning, self-awareness, and many others. As the technological world becomes ever more complex, interconnected, fast, and invisible, there may be substantial value in these qualities also being present in the systems we build. In this paper, we take the first steps to cast light on what constitute Lifelike Computing Systems, why such systems are worth striving for in the first place, and what we need in order to pave the road for them. We bring this notion in line with existing research initiatives sharing the explicit systems engineering focus. Important research aspects are derived that serve as the basis for a working agenda towards lifelike computing systems.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>I. Why Lifelike Computing Systems?</title>
      <p>
        Technological systems have always been part of modern
human life. Historically, while initially taking the form of
passive tools, such as axes and spoons, the industrial
revolution saw the advent of powered, mechanised
technology, operating “under their own steam” without direct
human control over every action. By integrating more
complex information processing machinery, automation evolved
into autonomy, as decision-making and self-regulation
became features of modern technology. Now, so-called
“intelligent”, “smart”, and “self-adaptive systems” find,
maintain and recover suitable behaviours for changing contexts.
These are designed with the explicit intention of carrying
out ‘rational’ behaviours, leading to technology doing the
sorts of things often attributed to natural intelligence
        <xref ref-type="bibr" rid="ref8">(Boden, 2016)</xref>
        . At the same time, the study of Artificial Life
(ALife)
        <xref ref-type="bibr" rid="ref2">(Banzhaf and McMullin, 2012)</xref>
        has explored the
properties of living systems, both as they are found in
nature and as they can be built by humans, thereby pursuing a
“life-as-it-might-be” modeling philosophy
        <xref ref-type="bibr" rid="ref18">(Langton, 1992)</xref>
        .
This has exposed a large variety of mechanisms that
produce qualities typically associated with life. Examples
include self-organisation, homeostasis, self-replication,
evolution, learning, self-awareness, and many others besides. As
the technological world becomes ever more complex and
interconnected
        <xref ref-type="bibr" rid="ref41">(Weiser, 1999)</xref>
        , fast
        <xref ref-type="bibr" rid="ref37">(Tennenhouse, 2000)</xref>
        and
invisible
        <xref ref-type="bibr" rid="ref27">(Norman, 1998)</xref>
        , there may be substantial value in
these qualities also being present in the systems we build.
      </p>
      <p>The “Lifelike Computing Systems” initiative aims to
learn from the study of life and living systems to develop
new, practical, computing systems that possess ‘lifelike’
properties; a further goal is to identify when such
complex features are of particular value. The initiative’s focus
lies primarily on engineered technological systems broadly
within the domain of computing. However, this term is not
intended to separate itself from or replace previous
initiatives; in a large number of cases, there are already
technologies and research efforts that strongly lean towards what we
deem lifelike computing systems in specific aspects. Rather,
our focus is on a holistic view of these properties, their
development as a suite, and their fruitful combination. In doing
so, the initiative aims to emphasise the drawing together of
existing approaches, technologies, and systems. On that
basis, we aim to shine a light on existing research towards this
vision, and to also determine what future research is
necessary (i.e., open questions, knowledge gaps, and limitations)
in pursuing a holistic view of lifelike computing systems.</p>
      <p>
        The basic motivation to develop more ‘lifelike’ systems
also has its roots in previous initiatives: cybernetics, for
example, considered the increasing complexity of system
behaviour, adaptive control and the resulting
interrelationships and challenges
        <xref ref-type="bibr" rid="ref42">(Wiener, 2019)</xref>
        . Later, the notion of
complex adaptive systems
        <xref ref-type="bibr" rid="ref13">(Holland, 1992)</xref>
        introduced
another research perspective to capture and understand the
underlying principles of naturally existent systems such as
e.g., economies or ecology. Observing the advances and
trends in computing technology, these insights were taken
up again, for example, at the beginning of this
millennium in the context of the Proactive
        <xref ref-type="bibr" rid="ref37">(Tennenhouse, 2000)</xref>
        ,
Autonomic
        <xref ref-type="bibr" rid="ref16">(Kephart and Chess, 2003)</xref>
        , and Organic
Computing
        <xref ref-type="bibr" rid="ref26 ref33 ref40">(Mu¨ller-Schloer and Tomforde, 2017)</xref>
        initiatives; all
sharing the common understanding that the controllability
of future systems with at that time current techniques was
no longer achievable. Since then, we can observe a rapid
technological development, which on the one hand has
resulted in new types of machine intelligence, adaptive control
strategies, and accordingly more autonomous behaviour of
technical systems. On the other hand, this also created new
possibilities, in that new types of computing and
communication technologies have greatly increased performance.
      </p>
      <p>
        The difference compared to 20 years ago is now that
(besides the technological capabilities) the world is changing
towards a digitalised, data-driven, technology-mediated
environment that intertwines everything into an integrated
‘superorganism’, as e.g., visible in the context of the
Internetof-Things
        <xref ref-type="bibr" rid="ref1">(Ashton et al., 2009)</xref>
        , Internet-of-Everything
        <xref ref-type="bibr" rid="ref26 ref33">(Snyder and Byrd, 2017)</xref>
        , or social cyber-physical systems
        <xref ref-type="bibr" rid="ref32">(Sha et al., 2008)</xref>
        , fields that aim at deeply integrating
humans and technical systems. These systems are
sociotechnical, and open-ended. As a result, the need grows for
systems that automatically find solutions for needs,
challenges, and dynamics that were either simply not there, or
at least not within our sphere of awareness, during
development
        <xref ref-type="bibr" rid="ref38 ref5">(Tomforde and Mu¨ller-Schloer, 2014)</xref>
        .
      </p>
      <p>
        Building on a long and highly successful tradition in
biologically-inspired computing (cf.
        <xref ref-type="bibr" rid="ref9">Bongard (2009)</xref>
        ), the
‘lifelike’ vision not only seeks inspiration in the living
world, but also seeks to replicate its qualities explicitly in
technological systems. The envisaged agenda also goes
beyond pure ALife research, often rightly exploratory in
nature, since it focuses explicitly on building purposeful and
reliable technological systems for people, indeed based on
both AI as well as ALife principles. In this context, the
construction of lifelike computing systems will build on
several decades of previous bio-inspired initiatives. However,
the vision of explicit replication of lifelike qualities marks
a departure: indeed, we cannot claim that all bio-inspired
systems remain lifelike, nor is this in-general even always a
desirable outcome for those designing bio-inspired systems.
For example, many evolutionary algorithms, such as a
simple (1+1)-EA (cf. De Jong (2016)) are clearly biologically
inspired in origin. However, they contain very few of the
qualities that we would commonly ascribe to something
lifelike. Other examples can be found in, for example,
neuralinspired machine learning systems.
      </p>
      <p>In particular, we would like to have certain qualities
available in the systems themselves – already implemented by
design, when talking about lifelike computing systems. In this
paper, we provide an initial discussion what lifelike
computing systems might be, by focusing on an initial selection of
certain qualities possessed by natural living systems
(Section II). This initial list will include rather intuitive aspects
such as open-ended evolution or different degrees of
intelligence, involving simple reactive behavior but also more
complex capabilities such as introspection. But also beyond
these intuitive qualities, further aspects such as emergence,
resilience and social behaviour will be discussed, which
together are hypothesized to allow for more adaptive and
reliable system behaviour, ensuring socially sensitive and
compliant actions, as well as still fostering transparency even
when evolving and self-integrating into higher order system
constitutions. However, this initial list will necessarily be
both incomplete and overly prescriptive since to our
knowledge there is yet no all-agreed consent on what exact
ingredients are defining life itself. Given the basic motivation
outlined before, the fundamental properties remain indeed
almost the same as they have been and still are for e.g., the
related Autonomic and Organic Computing initiatives: We
are on a quest for highly robust, flexible, trustworthy,
reliable, and efficient solutions for technical systems designed
to act and survive under the challenging conditions the world
bears to them when they are embedded within it. Based on
our first discussion of, at least from the authors’
perspective essential, lifelike qualities, we will look at the current
state – what technology is already available from other
research initiatives concentrating on the systems engineering
aspect and to which of the delineated qualities they provide
valuable contributions (Section III). Based on this brief
assessment, we will sketch important aspects of research upon
which an initial research agenda can be build with the goal
of approaching ‘truly’ lifelike computing systems (Section
IV). Finally, the paper closes with a conclusion and outlook
(Section V).</p>
    </sec>
    <sec id="sec-2">
      <title>II. What Are Lifelike Computing Systems?</title>
      <p>The obvious question that arises here is, “Do we really need
yet another computing paradigm?”. We answer this question
with caution and want to clarify that the quest for Lifelike
Computing Systems is not to be considered an orthogonal
way of engineering complex computing systems, but an
advancing one. That is, we build on existing research
initiatives such as Organic, Autonomic, and Self-Aware
Computing (cf. Sect. III), attempting to integrate their unique and
shared perspectives, the already achieved insights, but also
still open challenges into a unifying framework. In simpler
terms, we look at the advances and ways of thinking that
these research fields have given us, and ask: what’s next?</p>
      <p>In this positioning work, we initiate this research
endeavour by first identifying five striking properties or
capabilities of living systems that we deem especially valuable to
be brought into technical systems. However, it should be
clearly noted that our first attempt is necessarily incomplete
and that we by no means strive to propose a definition of life
itself here. Therefore, here we touch upon this initial set of
‘qualities of life’, and hope to stimulate the reader in
provoking deeper thoughts and creating broader interest on this
notion.</p>
      <p>
        1. Open-ended Evolution. The first quality of living
organisms and natural systems is their continuing
evolution
        <xref ref-type="bibr" rid="ref10">(Darwin, 1859)</xref>
        . Evolution is considered a mechanism
to change and thus constitutes a building block for
continuing system adaptation. Therefore, we not only think of
typical evolutionary computation techniques
        <xref ref-type="bibr" rid="ref11">(De Jong, 2016)</xref>
        ,
but explicitly emphasise open-ended evolution
        <xref ref-type="bibr" rid="ref36">Taylor et al.
(2016)</xref>
        . In real-world settings, the objectives provided to
computing systems by humans are usually neither static nor
fixed. Objectives vary, i.e., they are subject to gradual or
abrupt changes, seasonal impacts, are often multi-modal,
and can be highly contradictory in different sub-goals. In
open systems, objectives should also be considered to be
multi-scale: present over multiple system levels in a
hierarchy of abstraction, with each level (for example) having
an impact on the lower one. This clearly raises
complexity further still
        <xref ref-type="bibr" rid="ref12">(Diaconescu et al., 2016)</xref>
        . Accordingly, and
as noted by
        <xref ref-type="bibr" rid="ref34">Stanley and Lehman (2015)</xref>
        , relying exclusively
on fitness functions, i.e., a mathematical model used to
internalise the system objectives into our engineered
‘intelligent’ systems, appears to be unnecessarily limiting for the
production of the myriad of creative ways in which living
systems evolve to behave. Much more complex mechanisms
are needed to steer these systems toward continual adaptive,
flexible, and creative behavior, such as basing fitness also
on stimuli beyond numerical utility values. Detecting
novelty and positively reinforcing novel behaviors might be one
possible path to reach this goal (cf. e.g.,
        <xref ref-type="bibr" rid="ref19">Lehman and
Stanley (2008)</xref>
        ), though more broadly, fitness may often arise
endogenously within the environment and the people and
others with which the system interacts.
      </p>
      <p>
        2. Intelligence. One of the most distinguishing
qualities between living and non-living systems is intelligence.
As is already the case for the notion of life, there is again
no universally-accepted consensus on what exactly defines
natural intelligence. However, capabilities that are
typically mentioned include, among others, rational thinking,
problem-solving, reasoning on certain (but more
fascinatingly uncertain) knowledge, the acquisition of competence,
skill, and knowledge by different forms of learning, the use
of intuition and other mental heuristics, self-awareness and
various forms of reflection, as well as emotional thinking,
and creativity. For artificial intelligence, however,
numerous attempts to delineate this scientific discipline can be
quickly spotted, especially in these days where AI is once
again perceived as probably the most promising technology
with outstanding disruptive potential. Definitions here range
from similarly listing competencies (e.g.,
        <xref ref-type="bibr" rid="ref6">Bellman (1978)</xref>
        )
to more inclusive statements concerning machines that do
things like minds do (e.g.,
        <xref ref-type="bibr" rid="ref8">Boden (2016)</xref>
        ). Following
        <xref ref-type="bibr" rid="ref31">Russell and Norvig (2020)</xref>
        , one can approach such a definition
from four directions, where an AI is thought of as (1)
Thinking Humanly, (2) Acting Humanly, (3) Thinking rationally,
and (4) Acting rationally. For the overarching purpose of
engineering and understanding complex computing systems,
we often follow the fourth angle and define intelligence in
a technical sense as being brought into systems through an
agent (software) that acts rationally, which is further defined
in the sense of a way that maximises a numeric measure of
utility shaped by external performance standards.
According to this perspective, an ‘intelligent system’ is essentially a
system that can maintain its utility under challenging
conditions (e.g., time-varying environments, emergent situations,
or disturbances) by autonomously adapting its behaviour to
changing circumstances. In line with this notion, learning
is often one of the most obvious and intuitive functions of
an intelligent system. Next to the explicit goal of improving
a system’s performance in well-defined (and therefore,
typically narrowly-defined) tasks, learning also enables a system
to extend its knowledge and, thus, to expand its concept of
known environments (cf. concept drift and generalisation).
This is possible by taking advantage of already gained
experiences and direct interaction with the environment to
experience new situations in an explorative fashion; the latter
however usually comes at the cost of trial-and-error
situations, which in the online case, can add additional risk or
cost. But beyond this specific perspcetive on ‘learning and
acting’ in response to perceived stimuli and feedback in
order to maintain a user-defined utility level, when considering
lifelike computing systems, we subsume further capabilities
of naturally intelligent organisms under this quality. For
example, creativity is a prerequisite to solving many intricate
problems, as well as for developing framings with which
to express newly discovered problems. Bringing creativity
into our systems by, e.g., computer simulations or models
that imitate mental processes detached from the purely
reactive timescale, we expect to increase the problem-solving
capabilities of systems tremendously. Another distinguishing
property of intelligent beings such as humans is their ability
for introspection. Humans are (to varying degrees at various
times) self-aware. We can reflect on our behaviour (even
if it often happens retrospectively), e.g., to assess our own
knowledge and strengths, and we often have an inner clue
that serves as a measure for their quality of work (e.g.,
perfectionism). And this this section, we have only touched on
aspects commonly associated with human intelligence; there
are many other quite different forms of intelligence found in
the natural world, with quite radically different emphases, to
learn from and emulate.
      </p>
      <p>
        3. Emergence. Beyond thinking of intelligence on an
individual scale, for lifelike computing systems, we want to
emphasize the importance of collective intelligence. In
natural living systems, often not every organism can be
considered particularly intelligent on an individual basis. However,
nature also reveals remarkable capabilities when ‘simple’
(non-intelligent) species act collectively, either in smaller
groups or in larger swarms. Often the principle of
selforganization governs in such decentralized (swarm) systems,
what results in emergent phenomena sometimes understood
as intelligence of a ‘superorganism’. Prominent examples
from nature are ant colonies or bee hives which exhibit
intriguing capabilities by far not possible to be anticipated by
looking at only one single member of the superordinate
collective. Nevertheless, emergent effects as a product of local
interaction between a large number of self-motivated
entities (self-organisation) and resulting feedback loops do not
always lead to what we might see as desired system
behavior. If we strive for evolving computing systems comprised
of intelligent system components, we must not only bear
in mind the potentially undesired side-effects, but also
develop ways to quantify and measure them (cf. e.g.,
        <xref ref-type="bibr" rid="ref25">Mnif
and Mu¨ller-Schloer (2011)</xref>
        ). In lifelike computing systems,
emergence thus needs to be detected and “controlled” (or
perhaps steered), in the sense that the system itself is
enabled to enforce positive (e.g., robustness, adaptivity) and
simultaneously dampen adverse (e.g., stalling competition
for scarce resources) emergent effects.
      </p>
      <p>4. Resilience. Another striking property of living
systems is their resilience despite disturbing events, such as
volatile climate conditions, natural disasters, and also
gradually changing conditions, where they can still successfully
recover to maintain acceptable performance, or perhaps to
simply survive. Yet, it is clear that this occurs at multiple
levels, often depending on the severity: the level of the
individual organism, at the population level, and further, at the
level of whole ecosystems. Indeed, resilience can, and needs
to be viewed on different hierarchical and temporal scales.</p>
      <p>
        Quickly recovering from unforeseen or unanticipated
disturbances to maintain a viable system operation might be
considered short-term fault-tolerance or, more generally
technical robustness. However, the capability to consider
measures on multiple system scales, e.g., as a response to
changing goals that have a strong and long-term impact on
overall system properties, or acknowleding that
reconfiguration and diverse solution perspectives on a problem exist,
might be better referred to as ecosystem flexibility. Examples
might include the necessity to form entirely new
constellations of system parts or to self-integrate with other
specialised systems
        <xref ref-type="bibr" rid="ref4">(Bellman et al., 2021)</xref>
        . Resilience such
as this can involve rapid or indeed more longer-term
exploratory learning, but equally can rely on adaptive feedback
mechanisms either simple or complex.
      </p>
      <p>The essence of this fourth quality is that, for lifelike
computing systems, mechanisms will be needed that minimize
the brittleness of technical systems in order to tackle the
inherent complexity of the world; and that this cannot be
done only by them being insulated from it. Indeed,
sociotechnical settings from infrastructure services, healthcare,
and agriculture, to manufacturing, supply chains, and many
other besides, generate dynamic, often unforeseen, and
compound environmental, legal, or societal conditions. Lifelike
computing systems present an opportunity to move beyond
the Hobson’s choice of ‘carry on regardless even though the
scope has changed’ or ‘redesign and redeploy’.</p>
      <p>This calls at least for appropriate degrees of redundancy
(cf. 3. Emergence), but also mechanisms for system
introspection (cf. 2. Intelligence), and suitable decentralized
system architectures that can quickly compensate.
‘Failures’ should instead be framed as failures in assumptions,
and be adapted to, reconfiguring into a new space of
possibilities (cf. 1. Open-Ended Evolution), and drawing on
social awareness of supporting counterparts that can help
overcome the disturbance (cf. 5. Social Awareness, below).</p>
      <p>
        5. Social Awareness. Finally, a fundamental quality of
human and animal societies is their ability to establish
social behaviour, to empathise and reason socially about
others, and to establish and follow social norms. Such
societies face challenges that increasingly occur in technical
systems: they have to interact with unknown populations,
sometimes without understanding their language and
cultural background, and have to find and maintain an inner
balance based on, for example, fairness and equality.
Transferred to our human society, this ultimately involves the
establishment and continuous consideration of ethical,
valuebased actions. Technically, this implies – especially when
humans are seen as a fundamental part of the system and
no longer just as “users” – that systems need to be socially
sensitive
        <xref ref-type="bibr" rid="ref17 ref20 ref21 ref40">(Lewis, 2017b)</xref>
        and to have a sense of the
ethical implications of their actions
        <xref ref-type="bibr" rid="ref3">(Bellman et al., 2017)</xref>
        . In
view of lifelike computing systems, mechanisms are needed
to: (1) Interact with unknown participants in open systems
(e.g., technical trust
        <xref ref-type="bibr" rid="ref28">(Reif et al., 2016)</xref>
        ), (2) to recognise
interdependencies and mutual influences, especially of a
hidden, indirect nature, e.g.,
        <xref ref-type="bibr" rid="ref30">Rudolph et al. (2019)</xref>
        , (3) to
develop and comply to norms that govern autonomous
individual behaviour in accordance with overarching common
goals (e.g.,
        <xref ref-type="bibr" rid="ref15">Kantert et al. (2016)</xref>
        ), and finally, (5) to
explain their inner reasoning to involved human
stakeholders but also other computational counterparts, requiring both
an inter-operable abstracted language and context-sensitive
human-machine interfaces.
      </p>
      <p>We reiterate that this is not intended as an exhaustive list
of characteristics. Rather, we hope it serves both to provoke
further thought on how these concepts might show up in the
behaviour of future socio-technical systems and to illustrate
how the compound nature of features often studied in
separate subdisciplines is important for a holistic view of lifelike
computing systems. Given the ambitious goals of this
initiative for these pivotal characteristics, in the next section we
focus on assessing where we are now.</p>
    </sec>
    <sec id="sec-3">
      <title>III. Where Are We Now?</title>
      <p>The vision of establishing lifelike qualities in future
technical systems has its roots in and draws upon previous
initiatives, which we briefly review in the following in
order to provide an overview of the evolution of the field.
Due to space restrictions, we however concentrate on those
with an explicit engineering perspective. Accordingly,
research fields such as artificial life, theoretical biology,
self-organization, etc., are necessarily neglected here, even
though they provide the relevant insights for capturing and
analysing the complexity of living systems upon which such
engineering efforts sit. In the future, we believe the field
would be well-served by focussed reviews concerning how
insights from these disciplines can tangibly impact the
design and operation of lifelike computing systems.</p>
      <p>
        Multi-agent Systems (MAS). MAS consist of several
interacting, intelligent entities – so-called agents
        <xref ref-type="bibr" rid="ref43">(Wooldridge,
2009)</xref>
        . In this context, the term ‘agent’ refers to a software
unit that autonomously processes tasks on behalf of a user
or administrator – but it can also refer to robots, humans,
or even heterogeneous constellations of them
        <xref ref-type="bibr" rid="ref43">(Wooldridge,
2009)</xref>
        . Agents are used to model or solve problems that
cannot be handled in a standard monolithic way due to
high degrees of parallelism and/or complexity. Usually,
a MAS forms a sort of heuristic approach for an
otherwise intractable or too complex to model problem
        <xref ref-type="bibr" rid="ref14">(Jennings,
2000)</xref>
        . In literature, the concept has been successfully
applied to several well-known tasks, e.g., modelling social
structures
        <xref ref-type="bibr" rid="ref35">(Sun and Naveh, 2004)</xref>
        , on-line trading
        <xref ref-type="bibr" rid="ref29">(Rogers
et al., 2007)</xref>
        or devising agent-based models for agricultural
systems
        <xref ref-type="bibr" rid="ref7">(Berger, 2001)</xref>
        . In an MAS, agents take their
decisions based on predefined goals and are able to interact with
each other. For our envisioned lifelike computing systems,
especially the existing technology for interaction schemes
and protocols provide valuable starting points for the
delineated qualities of ‘emergence’ and ‘social awareness’.
      </p>
      <p>
        Proactive Computing (PAC).
        <xref ref-type="bibr" rid="ref37">Tennenhouse (2000)</xref>
        stated
that – as he called it – “human-in-the-loop computing” has
its limits. Embedded computing devices became
increasingly popular, which resulted in a dramatic increase in the
number of utilised devices running information and
communication technology. The sheer number demanded a
paradigm shift in administration to further guarantee
controllability, not unlike the challenges we now face more than
twenty years later. Although PAC mainly presented a vision
and had a strong focus on hardware challenges, the
motivation still holds for lifelike computing systems and we can
draw inspiration upon PACs vision for nearly all qualities
we mentioned above.
      </p>
      <p>
        Autonomic Computing (AC). Motivated by the
increasing complexity in large data centres,
        <xref ref-type="bibr" rid="ref16">Kephart and Chess
(2003)</xref>
        argued that computing systems need an automated
backbone structure similar to the autonomic nervous
system of humans. The idea is that this autonomic structure
relieves the designer from specifying all possibly occurring
situations and configurations within the design process.
Instead, the system itself takes over the responsibility to find
appropriate reactions to perceived changes in environmental
conditions. It also relieves the administrator of
configuration and maintenance tasks, especially in finding optimised
settings for resources. Although this is still limited to
dedicated control problems, use cases, and controlled decision
freedom of the autonomic systems, the basis for certain
aspects of lifelike behaviour by means of feedback and
selfadaptation is already laid. Concerning the five qualities of
life, AC’s achievements have clearly contributed to equip
technical systems with more ‘intelligence’ and ‘resilience’.
      </p>
      <p>
        Organic Computing (OC). Based on the motivation of
mastering the ever-growing complexity in technical
systems, the OC initiative took inspiration from and bring
basic concepts from natural and biological into technical
systems
        <xref ref-type="bibr" rid="ref26 ref26 ref33 ref40 ref40">(Mu¨ ller-Schloer and Tomforde, 2017; Tomforde et al.,
2017)</xref>
        with the aim of transferring formalisations of
selfx properties to engineered technological systems. These
supported the achievement of higher-order system
characteristics, such as robustness, flexibility, and viability under
challenging real-world conditions. As a result, traditional
design-time decisions are shifted to run time, and into the
responsibilities of systems themselves. This includes
adaptation decisions based on machine learning technology,
detecting changes in the underlying processes to be controlled, or
maintaining relationships among distributed systems.
However, the concrete control problem is still narrowly defined
and dealt with within, to the best possible degree
predetermined boundaries. Already from the motivation, but also
from the obtained achievements regarding self-adaptive and
self-organising system technology, OC can be considered a
substantial basis for further developing the qualities of
‘resilience’, ‘emergence’ and also ‘intelligence’.
      </p>
      <p>
        Self-Aware Computing (SeAC). The idea of
selfawareness in computing arose over many years in a
variety of areas of computer science, artificial intelligence, and
engineering. Over the last ten years, however, drawing on
self-awareness theories in psychology, a fundamental
understanding of what self-awareness concepts can mean for
the design and operation of computing systems has been
developed (e.g.,
        <xref ref-type="bibr" rid="ref23">Lewis et al. (2011</xref>
        , 2016);
        <xref ref-type="bibr" rid="ref17">Kounev et al.
(2017)</xref>
        ). This led to a number of contributions in terms of
definitions, architectures, algorithms and case studies,
targeted at explicitly designing computational self-awareness
into technical systems. Computational self-awareness
capabilities typically reference internal state, history, social or
physical environment, goals, and even a system’s own way
of representing and reasoning about these things.
      </p>
      <p>
        A number of architectures for self-aware systems exist
(e.g.
        <xref ref-type="bibr" rid="ref22">Lewis et al. (2015)</xref>
        ;
        <xref ref-type="bibr" rid="ref17">Kounev et al. (2017)</xref>
        ), and these
typically extend the (self-)knowledge representational and
acquisition capabilities of intelligent systems (e.g., building
on the MAPE-K architecture or any of the other
knowledgebased or learning-based agents
        <xref ref-type="bibr" rid="ref31">(Russell and Norvig, 2020)</xref>
        )
with fine-grained details concerning the system itself.
      </p>
      <p>Consequently, the notion of computational self-awareness
is clearly also key to lifelike computing systems when it
comes to establishing ‘intelligence’ and the capability of
introspection. Lewis (2017a) provides a summary.</p>
      <p>
        Interwoven Systems (IwS) In contrast to traditional
system design, the rising utilisation of communication
technology and the increasing interconnectedness of systems
resulted in blurring system boundaries. Instead of following
the ‘separation of concerns’ idea by building modules that
are combined during the development process, the IwS
initiative
        <xref ref-type="bibr" rid="ref5">(Bellman et al., 2014)</xref>
        focuses on changing system
goals that define the setup and configuration of the
composition and structures at runtime. A contained component
system can in turn consist of autonomous systems itself,
modules can aggregate different groups of entities towards
a (sub-)system, and even the goal that is followed might
change continually
        <xref ref-type="bibr" rid="ref4">(Bellman et al., 2021)</xref>
        . Hence, IwS on
the one hand already considers aspects of the ‘open-ended
evolution’ quality in respect to evolving system
compositions, but also involves ‘social awareness’, both deemed key
also in lifelike computing systems.
      </p>
      <p>
        Summary
As a result of the research done in the context of these fields,
we find a wide variety of techniques, methods, and
architectural approaches that can serve as the basis for lifelike
computing systems. Several concepts for designing
individual component systems have been proposed, such as the
observer/controller pattern
        <xref ref-type="bibr" rid="ref39">(Tomforde et al., 2011)</xref>
        from OC,
the monitor-analyse-plan-execute cycle
        <xref ref-type="bibr" rid="ref16">(Kephart and Chess,
2003)</xref>
        from AC, several from self-aware computing
        <xref ref-type="bibr" rid="ref17 ref20 ref21">(Lewis,
2017a)</xref>
        , as well as those from AI and robotics
        <xref ref-type="bibr" rid="ref31">(Russell and
Norvig, 2020)</xref>
        . Essentially, however, we note that these are
simply agent patterns that each includes or emphasises a
different set of features or processes, from our list of
characteristics, over the other. One key question will be how to find
unifying metamodels (at least in the conceptual space, if not
for actual implementation) that support the holistic
consideration of lifelike systems in general.
      </p>
    </sec>
    <sec id="sec-4">
      <title>IV. Approaching Lifelike Computing Systems</title>
      <p>
        Based on the notion of lifelike qualities from Section II and
the brief overview of available technology from Section III,
we now discuss first research avenues for approaching
lifelike computing systems. It turns out that available
technology from the various fields that have emerged over the past
few decades can already be combined very well and
integrated with each other. As we have sketched in this
paper, several important aspects of potential lifelike computing
systems are partly missing in some of the existing related
research directions. On the other hand, several of these aspects
are treated deeply but in an isolated fashion in others. In an
attempt to bridge this gap, in the following, we delineate
four important aspects upon which we propose a research
agenda towards lifelike computing systems be based:
i) Framework: An integrated approach requires a
common understanding of existing activities and techniques.
Therefore, much as was done in successfully unifying the
field of evolutionary computation
        <xref ref-type="bibr" rid="ref11">(De Jong, 2016)</xref>
        , we
propose to revisit existing research and technology, integrating
it into a unifying framework, resulting in a ‘toolbox’ in the
sense of a methodological repertoire and an architectural
metamodel for lifelike computing systems. Thus, each of
these specific initiatives can then be clearly seen as
addressing a part, a perspective perhaps, on the whole.
      </p>
      <p>ii) Testbeds: It is worth noting that initiatives such as
OC and AC, despite 20 years of successful research history,
cannot provide uniform benchmarks or testbeds – as is the
case, for example, in machine learning. This is mainly due
to the challenge that, as by definition an open-ended
problem space, a use case that also draws on the reader’s intuition
as to the possibilities is always necessary to demonstrate the
technology. This makes transferability of approaches
fundamentally difficult, but neither is this complexity
necessarily something to be wished away. For example, it could be
argued that in constructing common benchmarking sets for
machine learning, the scope of what machine learning
systems are expected to do is by definition artificially narrowed.
Nevertheless, we believe that for lifelike computing systems,
it is important to consider how to establish generic testbeds,
that provide for reproducability and comparability, while not
sacrificing generality and open-endedness.</p>
      <p>iii) System quantification: Typically, the success of
technical systems is considered in relation to a specific utility
function. For lifelike computing systems, this will only
make part of the evaluation. Thus, metrics that have been
derived in part for OC/AC systems, e.g., for measuring
adaptivity, self-organisation, or robustness, need to be extended
in order to quantify the complex inner states of the
systems, as well as how these relate to broader interactions and
ecosystems. The hierarchical nature of goals and resilience,
as discussed in Section II will be an essential consideration.</p>
      <p>iv) Computational approaches: In order to realize
lifelike computing systems, novel computational processes and
their implementations will be needed to underpin and
instantiate the above. For instance, beyond conventional
approaches like evolutionary computation, mainly targeted at
optimization problems and automatic programming, novel
algorithms, computational models, architectures, and
reflective processes are required that rather aim at establishing
the more fundamental requirement for adaptive behaviour
in open-ended settings on different scales of system design.
As another example, the basic capability of the systems
must go beyond the mere data-driven building of
knowledge through experience. In particular, we note that
computational introspection and reflection, creativity, empathy,
and social intelligence are still in their infancy.
Introspection, for example, involves the systematic assessment of a
system’s own knowledge but also an intrinsic motivation for
continual self-improvement (e.g., artificial curiosity).
Further, the multi-level nature of resilience is a largely
unexplored area in computing. This requires a micro-macro
perspective, which implies cooperation with other entities on
microscopic levels, but also monitoring of emergent
macroscopic behavior. Finally, the focus of interest should also
be that lifelike computing systems have broad
compatibility: with legacy systems; with heterogeneous approaches to
problem solving; with upcoming solutions whose existence
is yet unknown but which lifelike computing systems ought
to be ‘prepared’ for from first principles; and finally,
compatible with the ‘socio-’ side of the socio-technical systems
of which they are part – with human society.</p>
      <p>As with our list of characteristics, we do not claim this
short sketch of areas of required research focus to be
exhaustive. Rather, we see this as a starting point in our pursuit of
a unifying research agenda.</p>
    </sec>
    <sec id="sec-5">
      <title>V. Conclusion and Outlook</title>
      <p>This work presents an initial step towards introducing the
vision of lifelike computing systems: technological systems,
of benefit to people, that are not only inspired by the living
world, but are explicitly intended to replicate its qualities.</p>
      <p>The capabilities of traditional technical systems no longer
suffice to master the increasing complexity within the larger
contexts they operate. Therefore, we postulated five
qualities of life that we deem essential for future computing
systems, in order to meet these heightened requirements. This
set of qualities is necessarily incomplete, and the agenda of
lifelike computing systems is intended not as any
fundamentally new paradigm, but instead to ask the question: ‘what’s
next?’ for adaptive, bio-inspired technical systems – and
further, to ask if these initiatives can be placed within a broader
unified perspective. In this paper, we sketched what such a
perspective might look like, at least some of what it ought to
draw on, and how we might answer these questions.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <surname>Ashton</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          et al. (
          <year>2009</year>
          ).
          <article-title>That 'internet of things' thing</article-title>
          .
          <source>RFID journal</source>
          ,
          <volume>22</volume>
          (
          <issue>7</issue>
          ):
          <fpage>97</fpage>
          -
          <lpage>114</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <string-name>
            <surname>Banzhaf</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          and
          <string-name>
            <surname>McMullin</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          (
          <year>2012</year>
          ).
          <article-title>Artificial life</article-title>
          . In Rozenberg, G., Ba¨ck, T., and
          <string-name>
            <surname>Kok</surname>
          </string-name>
          , J. N., editors,
          <source>Handbook of Natural Computing</source>
          , pages
          <fpage>1805</fpage>
          -
          <lpage>1834</lpage>
          . Springer.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <surname>Bellman</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Botev</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hildmann</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lewis</surname>
            ,
            <given-names>P. R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Marsh</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pitt</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Scholtes</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2017</year>
          ).
          <article-title>Socially-sensitive systems design: Exploring social potential</article-title>
          .
          <source>IEEE Technology and Society Magazine</source>
          ,
          <volume>36</volume>
          (
          <issue>3</issue>
          ):
          <fpage>72</fpage>
          -
          <lpage>80</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <surname>Bellman</surname>
            ,
            <given-names>K. L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Botev</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Diaconescu</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Esterle</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gruhl</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Landauer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lewis</surname>
            ,
            <given-names>P. R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nelson</surname>
            ,
            <given-names>P. R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pournaras</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stein</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2021</year>
          ).
          <article-title>Self-improving system integration: Mastering continuous change</article-title>
          .
          <source>Future Gener. Comput. Syst.</source>
          ,
          <volume>117</volume>
          :
          <fpage>29</fpage>
          -
          <lpage>46</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <surname>Bellman</surname>
            ,
            <given-names>K. L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , and Wu¨rtz,
          <string-name>
            <surname>R. P.</surname>
          </string-name>
          (
          <year>2014</year>
          ).
          <article-title>Interwoven systems: Self-improving systems integration</article-title>
          .
          <source>In IEEE Int. Conf. on Self-Adaptive and Self-Organizing Systems Workshops</source>
          <year>2014</year>
          , pages
          <fpage>123</fpage>
          -
          <lpage>127</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <surname>Bellman</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          (
          <year>1978</year>
          ).
          <source>Artificial Intelligence: Can Computers Think? Thomson Course Technology.</source>
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <surname>Berger</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          (
          <year>2001</year>
          ).
          <article-title>Agent-based spatial models applied to agriculture: a simulation tool for technology diffusion, resource use changes and policy analysis</article-title>
          .
          <source>Agricultural Economics</source>
          ,
          <volume>25</volume>
          (
          <issue>2-3</issue>
          ):
          <fpage>245</fpage>
          -
          <lpage>260</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <string-name>
            <surname>Boden</surname>
            ,
            <given-names>M. A.</given-names>
          </string-name>
          (
          <year>2016</year>
          ).
          <article-title>AI: Its nature and future</article-title>
          . Oxford Univ. Press.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <string-name>
            <surname>Bongard</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2009</year>
          ).
          <article-title>Biologically inspired computing</article-title>
          . IEEE computer,
          <volume>42</volume>
          (
          <issue>4</issue>
          ):
          <fpage>95</fpage>
          -
          <lpage>98</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <string-name>
            <surname>Darwin</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          (
          <year>1859</year>
          ).
          <article-title>On the Origin of Species by Means of Natural Selection. Murray, London. or the Preservation of Favored Races in the Struggle for Life</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <string-name>
            <surname>De Jong</surname>
          </string-name>
          , K. A. (
          <year>2016</year>
          ).
          <article-title>Evolutionary Computation: A Unified Approach</article-title>
          . MIT Press, Cambridge, MA, USA.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <string-name>
            <surname>Diaconescu</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Frey</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , Mu¨
          <fpage>ller</fpage>
          -Schloer,
          <string-name>
            <given-names>C.</given-names>
            ,
            <surname>Pitt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            , and
            <surname>Tomforde</surname>
          </string-name>
          ,
          <string-name>
            <surname>S.</surname>
          </string-name>
          (
          <year>2016</year>
          ).
          <article-title>Goal-oriented holonics for complex system (self-)integration: Concepts and case studies</article-title>
          .
          <source>In 2016 IEEE 10th International Conference on Self-Adaptive and Self-Organizing Systems (SASO)</source>
          , pages
          <fpage>100</fpage>
          -
          <lpage>109</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <string-name>
            <surname>Holland</surname>
            ,
            <given-names>J. H.</given-names>
          </string-name>
          (
          <year>1992</year>
          ).
          <article-title>Complex adaptive systems</article-title>
          .
          <source>Daedalus</source>
          ,
          <volume>121</volume>
          (
          <issue>1</issue>
          ):
          <fpage>17</fpage>
          -
          <lpage>30</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          <string-name>
            <surname>Jennings</surname>
            ,
            <given-names>N. R.</given-names>
          </string-name>
          (
          <year>2000</year>
          ).
          <source>Building Complex Software Systems: The Case for an Agent-Based Approach. Communications of the ACM</source>
          ,
          <volume>44</volume>
          :
          <fpage>12</fpage>
          -
          <lpage>23</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          <string-name>
            <surname>Kantert</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Edenhofer</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , Ha¨hner, J., and
          <article-title>Mu¨ller-</article-title>
          <string-name>
            <surname>Schloer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          (
          <year>2016</year>
          ).
          <article-title>Normative control: Controlling open distributed systems with autonomous entities</article-title>
          .
          <source>In Trustworthy Open Self-Organising Systems</source>
          , pages
          <fpage>89</fpage>
          -
          <lpage>126</lpage>
          . Birkha¨user.
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <string-name>
            <surname>Kephart</surname>
            ,
            <given-names>J. O.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Chess</surname>
            ,
            <given-names>D. M.</given-names>
          </string-name>
          (
          <year>2003</year>
          ).
          <article-title>The vision of autonomic computing</article-title>
          .
          <source>Computer</source>
          ,
          <volume>36</volume>
          (
          <issue>1</issue>
          ):
          <fpage>41</fpage>
          -
          <lpage>50</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          <string-name>
            <surname>Kounev</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lewis</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bellman</surname>
            ,
            <given-names>K. L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bencomo</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Camara</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Diaconescu</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Esterle</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Geihs</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Giese</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          , Go¨tz, S., et al. (
          <year>2017</year>
          ).
          <article-title>The notion of self-aware computing</article-title>
          .
          <source>In Self-Aware Computing Systems</source>
          , pages
          <fpage>3</fpage>
          -
          <lpage>16</lpage>
          . Springer.
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          <string-name>
            <surname>Langton</surname>
            ,
            <given-names>C. G.</given-names>
          </string-name>
          (
          <year>1992</year>
          ).
          <article-title>Preface</article-title>
          . In Langton,
          <string-name>
            <given-names>C. G.</given-names>
            ,
            <surname>Taylor</surname>
          </string-name>
          , C.,
          <string-name>
            <surname>Farmer</surname>
            ,
            <given-names>J. D.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Rasmussen</surname>
          </string-name>
          , S., editors,
          <source>Artificial Life II</source>
          ,
          <article-title>Volume X of SFI Studies in the Sciences of Complexity, pages xiii-xviii, Redwood City</article-title>
          , CA. AddisonWesley.
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          <string-name>
            <surname>Lehman</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Stanley</surname>
            ,
            <given-names>K. O.</given-names>
          </string-name>
          (
          <year>2008</year>
          ).
          <article-title>Exploiting openendedness to solve problems through the search for novelty</article-title>
          .
          <source>In Proc. of the Eleventh Intl. Conf. on Artificial Life (ALIFE XI)</source>
          , Cambridge, MA. MIT Press.
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          <string-name>
            <surname>Lewis</surname>
            ,
            <given-names>P. R.</given-names>
          </string-name>
          (
          <year>2017a</year>
          ).
          <article-title>Self-aware computing systems: From psychology to engineering</article-title>
          .
          <source>In Proceedings of the 2017 Design, Automation &amp; Test in Europe Conference &amp; Exhibition (DATE)</source>
          , pages
          <fpage>1044</fpage>
          -
          <lpage>1049</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          <string-name>
            <surname>Lewis</surname>
            ,
            <given-names>P. R.</given-names>
          </string-name>
          (
          <year>2017b</year>
          ).
          <article-title>Self-integrating and self-improving systems must be socially sensitive</article-title>
          .
          <source>In IEEE Int. Works.s on Foundations and Appl. of Self* Systems</source>
          , pages
          <fpage>148</fpage>
          -
          <lpage>148</lpage>
          . IEEE.
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          <string-name>
            <surname>Lewis</surname>
            ,
            <given-names>P. R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chandra</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Faniyi</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Glette</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chen</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bahsoon</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Torresen</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Yao</surname>
            ,
            <given-names>X.</given-names>
          </string-name>
          (
          <year>2015</year>
          ).
          <article-title>Architectural aspects of self-aware and self-expressive computing systems</article-title>
          .
          <source>IEEE Computer</source>
          ,
          <volume>48</volume>
          :
          <fpage>62</fpage>
          -
          <lpage>70</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          <string-name>
            <surname>Lewis</surname>
            ,
            <given-names>P. R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chandra</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Parsons</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Robinson</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Glette</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bahsoon</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Torresen</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Yao</surname>
            ,
            <given-names>X.</given-names>
          </string-name>
          (
          <year>2011</year>
          ).
          <article-title>A survey of self-awareness and its application in computing systems</article-title>
          .
          <source>In Proceedings of the Fifth IEEE International Conference on Self-Adaptive and SelfOrganizing Systems Workshops (SASOW)</source>
          , pages
          <fpage>102</fpage>
          -
          <lpage>107</lpage>
          . IEEE Computer Society Press.
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          <string-name>
            <surname>Lewis</surname>
            ,
            <given-names>P. R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Platzner</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rinner</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Torresen</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Yao</surname>
          </string-name>
          , X., editors (
          <year>2016</year>
          ).
          <source>Self-Aware Computing Systems: An Engineering Approach</source>
          . Springer.
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          <string-name>
            <surname>Mnif</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          and
          <article-title>Mu¨ller-</article-title>
          <string-name>
            <surname>Schloer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          (
          <year>2011</year>
          ).
          <source>Quantitative Emergence</source>
          , pages
          <fpage>39</fpage>
          -
          <lpage>52</lpage>
          . Springer Basel, Basel.
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          <article-title>Mu¨ller-</article-title>
          <string-name>
            <surname>Schloer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2017</year>
          ).
          <article-title>Organic Computing - Techncial Systems for Survival in the Real World. Birkha¨user.</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          <string-name>
            <surname>Norman</surname>
            ,
            <given-names>D. A.</given-names>
          </string-name>
          (
          <year>1998</year>
          ).
          <article-title>The Invisible Computer</article-title>
          . MIT Press.
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          <string-name>
            <surname>Reif</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Anders</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Seebach</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          , Stegho¨fer, J.-P., Andre´,
          <string-name>
            <surname>E.</surname>
          </string-name>
          , Ha¨hner, J.,
          <article-title>Mu¨ller-</article-title>
          <string-name>
            <surname>Schloer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Ungerer</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          (
          <year>2016</year>
          ).
          <source>Trustworthy Open Self-Organising Systems</source>
          . Springer.
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          <string-name>
            <surname>Rogers</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>David</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jennings</surname>
            ,
            <given-names>N. R.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Schiff</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2007</year>
          ).
          <article-title>The effects of proxy bidding and minimum bid increments within eBay auctions</article-title>
          .
          <source>ACM TWEB</source>
          ,
          <volume>1</volume>
          (
          <issue>2</issue>
          ):
          <fpage>9</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          <string-name>
            <surname>Rudolph</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , and Ha¨hner, J. (
          <year>2019</year>
          ).
          <article-title>Mutual influence-aware runtime learning of self-adaptation behavior</article-title>
          .
          <source>ACM Trans. Auton</source>
          . Adapt. Syst.,
          <volume>14</volume>
          (
          <issue>1</issue>
          ):4:
          <fpage>1</fpage>
          -
          <lpage>4</lpage>
          :
          <fpage>37</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          <string-name>
            <surname>Russell</surname>
            ,
            <given-names>S. J.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Norvig</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          (
          <year>2020</year>
          ).
          <article-title>Artificial Intelligence: A Modern Approach (4th Edition)</article-title>
          .
          <source>Pearson.</source>
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          <string-name>
            <surname>Sha</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gopalakrishnan</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Liu</surname>
            ,
            <given-names>X.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Wang</surname>
            ,
            <given-names>Q.</given-names>
          </string-name>
          (
          <year>2008</year>
          ).
          <article-title>Cyber-physical systems: A new frontier</article-title>
          .
          <source>In 2008 IEEE International Conference on Sensor Networks</source>
          , Ubiquitous, and Trustworthy Computing (sutc
          <year>2008</year>
          ), pages
          <fpage>1</fpage>
          -
          <lpage>9</lpage>
          . IEEE.
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          <string-name>
            <surname>Snyder</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Byrd</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          (
          <year>2017</year>
          ).
          <article-title>The internet of everything</article-title>
          .
          <source>IEEE Computer Architecture Letters</source>
          ,
          <volume>50</volume>
          (
          <issue>06</issue>
          ):
          <fpage>8</fpage>
          -
          <lpage>9</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          <string-name>
            <surname>Stanley</surname>
            ,
            <given-names>K. O.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Lehman</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2015</year>
          ).
          <article-title>Why Greatness Cannot Be Planned: The Myth of the Objective</article-title>
          . Springer.
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          <string-name>
            <surname>Sun</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Naveh</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          (
          <year>2004</year>
          ).
          <article-title>Simulating Organizational Decision-Making Using a Cognitively Realistic Agent Model</article-title>
          .
          <source>Journal of Artificial Societies and Social Simulation</source>
          ,
          <volume>7</volume>
          (
          <issue>3</issue>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          <string-name>
            <surname>Taylor</surname>
          </string-name>
          , T.,
          <string-name>
            <surname>Bedau</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Channon</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ackley</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Banzhaf</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Beslon</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dolson</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Froese</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hickinbotham</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ikegami</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>McMullin</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Packard</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rasmussen</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Virgo</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Agmon</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Clark</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>McGregor</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ofria</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ropella</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Spector</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stanley</surname>
            ,
            <given-names>K. O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stanton</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Timperley</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vostinar</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Wiser</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          (
          <year>2016</year>
          ).
          <article-title>Open-Ended Evolution: Perspectives from</article-title>
          the OEE Workshop in York. Artificial Life,
          <volume>22</volume>
          (
          <issue>3</issue>
          ):
          <fpage>408</fpage>
          -
          <lpage>423</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          <string-name>
            <surname>Tennenhouse</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          (
          <year>2000</year>
          ).
          <article-title>Proactive computing</article-title>
          .
          <source>Communications of the ACM</source>
          ,
          <volume>43</volume>
          (
          <issue>5</issue>
          ):
          <fpage>43</fpage>
          -
          <lpage>50</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref38">
        <mixed-citation>
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          and
          <article-title>Mu¨ller-</article-title>
          <string-name>
            <surname>Schloer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          (
          <year>2014</year>
          ).
          <article-title>Incremental design of adaptive systems</article-title>
          .
          <source>Journal of Ambient Intelligence and Smart Environments</source>
          ,
          <volume>6</volume>
          (
          <issue>2</issue>
          ):
          <fpage>179</fpage>
          -
          <lpage>198</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref39">
        <mixed-citation>
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Prothmann</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Branke</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , Ha¨hner, J.,
          <string-name>
            <surname>Mnif</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <article-title>Mu¨ller-</article-title>
          <string-name>
            <surname>Schloer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Richter</surname>
            ,
            <given-names>U.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Schmeck</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          (
          <year>2011</year>
          ).
          <article-title>Observation and Control of Organic Systems</article-title>
          .
          <source>In Organic Computing - A Paradigm Shift for Complex Systems</source>
          , pages
          <fpage>325</fpage>
          -
          <lpage>338</lpage>
          . Birkha¨user.
        </mixed-citation>
      </ref>
      <ref id="ref40">
        <mixed-citation>
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sick</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <article-title>and Mu¨ller-</article-title>
          <string-name>
            <surname>Schloer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          (
          <year>2017</year>
          ).
          <article-title>Organic computing in the spotlight</article-title>
          .
          <source>CoRR, abs/1701</source>
          .08125.
        </mixed-citation>
      </ref>
      <ref id="ref41">
        <mixed-citation>
          <string-name>
            <surname>Weiser</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          (
          <year>1999</year>
          ).
          <article-title>The computer for the 21st century</article-title>
          .
          <source>ACM SIGMOBILE Mob. Comput. Commun. Rev.</source>
          ,
          <volume>3</volume>
          (
          <issue>3</issue>
          ):
          <fpage>3</fpage>
          -
          <lpage>11</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref42">
        <mixed-citation>
          <string-name>
            <surname>Wiener</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          (
          <year>2019</year>
          ).
          <article-title>Cybernetics or Control and Communication in the Animal and the Machine</article-title>
          . MIT press.
        </mixed-citation>
      </ref>
      <ref id="ref43">
        <mixed-citation>
          <string-name>
            <surname>Wooldridge</surname>
            ,
            <given-names>M. J.</given-names>
          </string-name>
          (
          <year>2009</year>
          ).
          <article-title>An Introduction to MultiAgent Systems</article-title>
          . John Wiley &amp; Sons Publishers, 2nd edition.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>