<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Lifelike Systems Need Some Kind of a Skin: First Thoughts on Cyberskin Capabilities</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Kirstie L. Bellman</string-name>
          <email>st@informatik.uni.kiel.de</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Sven Tomforde</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Christian-Albrechts-Universita ̈t zu Kiel</institution>
          ,
          <addr-line>Intelligent Systems, Kiel</addr-line>
          ,
          <country country="DE">Germany</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Topcy House Consulting</institution>
          ,
          <addr-line>Thousand Oaks</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Robustness is one of the primary goals of self-adaptive and self-organizing systems, including appropriate reactions to unanticipated conditions, to disturbances, and even to attacks. In this paper, we draw from biological systems to introduce a 'Cyberskin' as a concept for introducing some potentially new strategies for robustness in complex systems. We briefly summarize characteristics of the skin of animals and humans and map this into desirable computational capabilities. We then discuss some beginning ideas for implementing some of the technical functionality to support such a Cyberskin.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>
        Openness and adaptivity are key enablers for
nextgeneration information and communication technology
(ICT). Driven by trends such as intelligent systems (e.g.,
        <xref ref-type="bibr" rid="ref6">Calma et al. (2017)</xref>
        ), cyber-physical systems (e.g.,
        <xref ref-type="bibr" rid="ref19">Rajkumar et al. (2010)</xref>
        ), and Internet of Things (e.g.,
        <xref ref-type="bibr" rid="ref2">Atzori et al.
(2010)</xref>
        ), we face constellations of typically distributed
autonomous subsystems that integrate dynamically into an
overall system constellation and self-adapt their behavior.
One of the goals of self-* (e.g. self-organization, self-
integration, or self-healing) ICT systems is that these
adaptations should not be limited to a pre-defined operational
context or a fixed repertoire of behaviors at design time. In
addition, we emphasize that self-* mechanisms are also
critically needed to make technical systems resistant against
external or internal disturbances (see
        <xref ref-type="bibr" rid="ref23">Tomforde et al. (2018)</xref>
        ).
Such self-* systems may not perform better than
conventional systems (although depending on the novelty of the
dynamic environment they may) but they return faster to a
corridor of acceptable performance in the presence of
disturbances. The ultimate goal of self-* systems is to be both
more creatively adaptive and resilient against disturbances
and attacks from the outside. We call this property
’robustness’ (
        <xref ref-type="bibr" rid="ref17">Mu¨ller-Schloer and Tomforde (2017</xref>
        )).
      </p>
      <p>In this paper, we propose a set of mechanisms which we
call a “Cyberskin” that will consolidate and augment
existing self-* capabilities. Inspired by biology, such a
standardized enclosure will allow for even higher robustness and
resilience when acting in open environments. We discuss the
biological inspiration (Section 2) and map this onto a first
concept for a ’Cyberskin’ in ICT systems (Section 3). The
paper closes with an outlook on a possible path towards the
development of such a ’Cyberskin’ (Section 4).</p>
    </sec>
    <sec id="sec-2">
      <title>Biological Inspiration</title>
      <p>ICT systems are approaching the complexity of biological
systems in their massively large number of parallel
components and subsystems, concurrent activities and goals, and
very diverse hardware and software entities. One of the
triumphs in computing has been message passing and the use
of explicit models and knowledge in many forms. With a
few basic protocols this has permitted enormous benefits
in communicating among diverse systems. However,
although biological systems use message passing they also
make use of other different ways of being architected and
integrated. These architectures are characterized by having
multiple layers and multiple tightly integrated subsystems
that allow a tremendous amount of implicit information to
be conveyed among a widely distributed and massively
parallel system. Unlike the hairs in the ear or the visual retina,
sensory systems such as the skin or the architecture of joint,
muscle and tendons are connected and integrated with both
implicit and explicit ‘signals’ and actions. In addition to the
continual reliance on networks, messages and explicit
signals, we want to explore what qualities this implicit
structuring gives to computational systems. We suggest that there
may be new characteristics in terms of response speed,
adaptations, robustness, and protection possible.</p>
      <p>
        As background to the discussion on skin, let us look at a
few of the different principles in biological systems and their
implications for computational systems.
        <xref ref-type="bibr" rid="ref4">Bellman and
Walter (1984)</xref>
        suggest that a good image for biological systems
may be that of a community rather than a single agent. That
is, the system is composed of elements that are in themselves
sophisticated living elements. As
        <xref ref-type="bibr" rid="ref16">Micheva and Smith (2007)</xref>
        state “One synapse, by itself, is more like a micro-
processor - with both memory–storage and information- processing
elements – than a mere on/off switch. In fact, one synapse
may contain on order of 1000 molecular scale switches”. In
      </p>
      <p>
        Copyright c 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
our nervous system, it is estimated that there are 100 billion
neurons, each with 7000 synapses. And that is only one type
of cell within the human body. Hence, the scale and
heterogeneity of components, subsystems and processes fits very
much into the idea of today’s Interwoven systems (see
        <xref ref-type="bibr" rid="ref3">Bellman et al. (2014)</xref>
        and
        <xref ref-type="bibr" rid="ref22 ref24">Tomforde et al. (2014)</xref>
        ), which also
have enormous diversity; systems with their own functions
and goals connected in a variety of ways and yet also
interdependent and requiring coordination.
      </p>
      <p>
        Rather than being focused on a single goal or function,
biological systems have very different ’efficiencies’ and
optimization criteria that include adaptability, robustness and
resiliency as well as performance, see
        <xref ref-type="bibr" rid="ref12">Kantert et al. (2017)</xref>
        .
      </p>
      <p>Also, they have an enormous variety of subsystems and
processes occurring simultaneously at different levels. For
example, mammalian brains consist of different regions with
distinct architectures, characteristics, and capabilities. This
means at any given moment, there may be very different
activities and processes and goals for those processes.</p>
      <p>
        Biological systems are always multi-goaled (
        <xref ref-type="bibr" rid="ref4">Bellman and
Walter (1984)</xref>
        ), and they quickly drop goals as needed, take
advantage of situations to adopt new goals, merge goals, and
take a long term view in optimizing their situation versus
any one task. Such long term optimization includes placing
oneself in a better position for later behavior and, hence,
might actually result in a temporarily lower goal
accomplishment for a given task. Biological systems make use of
‘substitutability’ (equivalent classes of actions) and
compromises among activities and goals. Like computer networks
that may have resource contentions, multiple users and
conflicting goals among those users, biological networks have a
magnitude more diverse types of processes, goals, activities
and subsequently, contentions. To handle the needed
coordination, biology has developed many more styles of
interaction and influence that includes both explicit and implicit
networks, architectures, and responses to activities. In
complex systems, one needs both the speed coming from rapid
communication and signaling, as well as the structure that
allows the system to take advantage of implicit connections
that can quickly coordinate and inform a wide variety of
distributed elements. Although there are many such examples
in biological systems, in this paper we focus on the
properties of skin which has both explicit signaling and
transmission of information and implicit and indirect methods.
      </p>
      <p>
        One of the key stabilizing principles of biological
networks is the use of active ’stabilizing’ processes that include
different strategies for ’buffering.’ By buffering, we mean
mechanisms and architectures that separate input from
output and isolate the impacts of abnormal/foreign elements
(including the detection of abnormal behavior as suggested
in
        <xref ref-type="bibr" rid="ref9">Gruhl et al. (2015)</xref>
        ). This includes nature’s version of
firewalls (e.g., skin, membranes) which are permeable,
controllable and, in some cases, sentient. Buffers in living
systems allow the separation of immediate reaction to stimuli
from response and provide time for adaptive processes and
later reasoning processes. It is one of the major methods for
’buying time’ for adaptive rather than fixed responses.
      </p>
      <p>Skin is the largest organ in your body and compared to
the other sensory systems, it does not get enough attention.</p>
      <p>Although only a subset of the skin’s functions will
immediately resonate with our computational goals, we are listing
many of skin’s diverse functions in order to convey all the
processes and activities that go beyond being a barrier.</p>
      <p>One of the major functions of skin is protection; it is an
anatomical barrier from pathogens and damage between the
internal and external environment. As such, it is key to heat
regulation and the control of evaporation. Dilated blood
vessels increase perfusion and heat loss, while constricted
vessels greatly reduce cutaneous blood flow and conserve heat.</p>
      <p>In the control of evaporation, it ensures that the skin
provides a relatively dry and semi-impermeable barrier to fluid
loss. Skin is also a key part of the adaptive immune system
(Langerhans cells) and also is one of the body’s storage and
synthesis sites; it acts as a storage center for lipids and
water, as well as a means of synthesis of Vitamin D by action
of UV on certain parts of the skin.</p>
      <p>Given the functions above, it is not surprising to find that
it plays a major role in excretion and absorption. It is good
to keep these functions in mind as we consider ideas for
controlling what goes into systems (desirable entities e.g., data,
code, sensors etc.) and what is released from computational
systems (e.g., garbage, suspicious code or data). The skin
is an important site of transport in many organisms:
oxygen, nutrients, or water. In humans, the cells comprising the
outermost 0.25-0.40 mm of the skin are supplied by oxygen.</p>
      <p>The role of the skin in sensation is of immediate
interest for our computational systems. Skin contains a variety
of nerve endings that react to heat and cold, touch,
pressure, vibration, and tissue injury. The skin includes
sensors and processing for touch (several types including
pressure and vibration,) temperature, proprioception (body
position, position of muscles, bones and joints), and
nociception (pain/tissue injury). The system reacts to diverse
stimuli using different receptors: thermo-receptors, nociceptors,
mechano-receptors and chemo-receptors. The skin is also
important in aesthetics and communication; it is used by
conspecifics to assess mood, physical state and
attractiveness. Both these internal state and external environmental
changes are rapidly available across the system because of
its intertwined and diverse architectures.</p>
      <p>
        Furthermore, these numerous types of receptors are not
evenly distributed across the skin. The density and variety
of receptors matter. This density may relate to priorities or to
coordination with other sensory systems or to the
specializations of the system to its ecosystem. For example, sensing
injury is an important role of skin; there are 200 pain
receptors for every square centimeter of the human body.
Catfish have chemo-receptors all along their body (a catfish is
like a giant tongue)
        <xref ref-type="bibr" rid="ref1">Atema (1980)</xref>
        while humans have 6-12
million olfactory receptors in the human nose alone
(bloodhounds 4 billion) , see
        <xref ref-type="bibr" rid="ref5">Bullock (1982)</xref>
        . Electric eels have
a sensory system (electrolocation) that combines some
aspects of skins and vision; the sensing of electric fields can
be used for object detection, communication, defensive and
offensive behavior such as prey stunning and warding off
predators
        <xref ref-type="bibr" rid="ref20">Shier et al. (2004)</xref>
        . This variety may give us new
ideas for how to prioritize and fuse incoming information
and how to protect key areas. One thing we would like from
a Cyberskin is quick detection of the locality, the extent, and
the type of attack. Skin can immediately locate site of
contact (within different just ’noticeable differences’):
      </p>
    </sec>
    <sec id="sec-3">
      <title>Some Thoughts on a ’Cyberskin’</title>
      <p>
        Over the last twenty years, there has been increasing work
on haptic interfaces and artificial skin. Robotics and sensors
applications such as telemedicine and manufacturing have
resulted in an interest in artificial touch receptors and
skinlike sensor meshes (for example
        <xref ref-type="bibr" rid="ref15">Lumelsky et al. (2001)</xref>
        ).
Our purpose here is different. Instead of building a skin per
se, we examine the implications of skin’s properties and
consider how to adopt them into computational systems. There
are several properties of the skin that are of particular
interest to us. First, skin is inside as well as on the external
interface to the world; the importance to us is that
potentially the same methods and architectures may be used for
different sizes of systems and for system-of-systems. Also,
there are many layers and types of skin in a given
biological system. The external skin has a large number of layered
different sensor networks and activities. The overlapping
and diverse architectures of these nets allows for
simultaneous monitoring of multiple conditions of interest and layers
reached have implicit information (e.g., injury or danger).
      </p>
      <p>One of the most important features of skin is that it is
selectively permeable, allowing for many different
implementation strategies on what can pass through – both from the
system and into the system. Skin is also sentient in terms
of ’having sense, perception’ – locality, nearness, size,
damage, and so on. This is a key property in terms of having a
distributed system capable of almost immediate processing
where it is being interacted with and with what. The
multitude of sensors noted above monitor for different things and
can be used to determine size, type of interaction, and site of
first responses to foreign objects. Given that, it is interesting
is to imagine new relevant measurands for non-biological
entities. That is, the current sensors in biological skin
depend upon our animal evolution in many physical states
(e.g., cold, hot) and threats (e.g., pushed, pained, pressured).
These may still be of importance for cyber-physical systems
as many failures are caused by temperature and dampness.
However, what properties should we sense within a
computational environment? Noisiness? Density of activity? In
addition to the external world, skin within biological systems
has a major role in self-monitoring; it informs the system of
internal changes (for example, proprioception and changes
such as hydration) and by nature to its external and internal
interfaces, helps integrate internal and external states.</p>
      <p>Lastly, the ’elasticity’ of skin refers to the property of
some solid materials that return to their original shape and
size after the forces deforming them have been removed.
This has been a key adaptive capability in biological
systems that grow and develop, but it also allows immediate
flexibility in movements. It is also clear that the
deformation (stretch and bend of skins) is signaling changes of states
such as a full stomach or the extent of voluntary and
involuntary movements (e.g., breathing) and triggering additional
actions. One of the things we would like from both rapid
locality and elasticity is perhaps a measure of how long the
interaction/attack has been ongoing. In biological skin, such
information is conveyed by the type of sensory event and
by how deeply down through skin layers the interaction has
gone. Many sensed events become pain (e.g., an alert to
danger) the farther down in the layers they go. Biological pain
(extreme pressure, heat, cold, cuts) could have new
corollaries in computational systems such as the location of a loss in
connectivity or a drop in data rate.</p>
    </sec>
    <sec id="sec-4">
      <title>A ’Cyberskin’ for Technical Systems</title>
      <p>In this section, we introduce a way of going beyond biology
and discuss how the system would actually construct an
ongoing skin due to learning and experience. Biological skin
does this a little especially in terms of size (as one gets
bigger), in terms of injury, and of course, the complex set of
adaptive sensory mechanisms. However, the ideas here
include intelligent processing and learning with some of the
aspects of skin to incorporate learning.</p>
      <p>
        The design process of traditional technical systems is
based on analysis of anticipated and predictable events,
followed by the development of countermeasures and
contingent responses (Tomforde and M u¨ller-Schloer (2014)).
However, especially in dynamic and open environments,
there will always be unpredictable events. This needs to be
covered by a concept called “spontaneous closure”
        <xref ref-type="bibr" rid="ref17">Mu¨
llerSchloer and Tomforde (2017</xref>
        ), see Figure 1). Exception
handlers and diagnosis systems are examples of implementing
spontaneous closures in technical systems. In a watchdog
timer, a timer is reset and restarted at regular intervals. If
the timer is not reset, because the monitored system is in a
malfunction, a reset is performed to a defined initial state.
      </p>
      <p>
        Research in self-adaptive and self-organizing (SASO)
systems has presented a wide variety of concepts for
establishing and persisting such spontaneous closure. Most of the
machine learning mechanisms used in SASO systems are
dedicated to finding appropriate reactions to previously
unknown or less certain con
        <xref ref-type="bibr" rid="ref7">ditions D’Angelo et al. (2019</xref>
        ) –
in this sense, learning means that recurring reactions of the
spontaneous closure become part of the designed core.
Based on the experience of machine learning applications,
we distinguish three major classes of effects that need to
be handled by spontaneous closure: a) Unanticipated
conditions, b) disturbances as a result of malfunctions, and c)
attacks. Unanticipated conditions reflect the fact that only a
fraction of environmental and internal conditions can be
anticipated at design time. Disturbances as result of
malfunctions include expected events such as wear and aging, as well
as unanticipated effects that cause failures and malfunctions
of individual components and entire systems. The system
should continue to maintain its functionality despite these
failures. Lastly, attacks are likely because an autonomous
system operates in a shared environment where interaction
with known and unknown other systems takes place. This
not only opens up potentials for continuous optimization but
also a multitude of heterogeneous attack vectors.
      </p>
      <p>
        Living systems are characterized by a powerful
spontaneous closure partly realized by the skin. In
particular, we can observe the following technical implications as
three aspects of Cyberskin. As noted above, skin can
detect and classify novel conditions extremely fast.
Technically, this means in Aspect 1 of Cyberskin we want to apply
novelty detection mechanisms (see
        <xref ref-type="bibr" rid="ref18">Pimentel et al. (2014)</xref>
        ;
        <xref ref-type="bibr" rid="ref10">Gruhl et al. (2020)</xref>
        for details) in combination with other
autonomous learning capabilities (see, e.g.,
        <xref ref-type="bibr" rid="ref14">Krupitzer et al.
(2015)</xref>
        ;
        <xref ref-type="bibr" rid="ref21">Sick et al. (2018)</xref>
        ) to improve the reaction to these
stimuli over time. Next natural skin recovers quickly from
injuries (within limits determined by the system) and raises
alerts through prioritized alert signals such as pain or
pressure. Aspect 2 of Cyberskin technically then refers to
selfhealing techniques, provisioning of backup or alternative
components, and alarm systems that are coupled to
immediate counter measures. Aspect 3 refers to the ability of
natural skin to assess contact with novel objects and
operational environments and immediately map this sensory
information to approximate reactions. Besides standard
encryption and authentication or computational trust
mechanisms (e.g.,
        <xref ref-type="bibr" rid="ref11">Kantert et al. (2015)</xref>
        ), we think that skin may
provide new inspiration for an interactive and context-based
contact establishment protocol. This protocol would be one
way to allow for some of the skin’s most subtle, rapid, and
widespread capabilities in technical systems. This may be
part of the means to process customized security questions
that need to be answered before an interaction is established.
      </p>
      <p>
        We observe that – at least for now – all the aspects
mentioned above are mostly considered in an isolated manner.
The notion of a Cyberskin is especially important as a means
to better integrate these efforts into a unified whole. Also
the basic purpose of the Cyberskin is not restricted to the
outer shell – it should also connect and protect the inner
components or subsystems. Several efforts in SASO
system engineering have proposed machine learning solutions
to handle Aspect 1 and Aspect 2. However, Aspect 3 has
mostly been considered in an attack-specific manner (i.e.,
trying to identify attacks based on abnormal behavior) and
utilizes authentication and authorization mechanisms using
standard encryption schemes. We propose to augment this
(i.e., to extend the spontaneous closure) with more
sophisticated and lifelike mechanisms. Especially, we would like to
replace the current isolated handling by a localized, efficient
and approximate computing metaphor that we call
Cyberskin. There are approaches that go in this direction – such
as the idea of a ’membrane’ introduced in
        <xref ref-type="bibr" rid="ref8">Diaconescu et al.
(2016)</xref>
        . This and related work certainly provide good
starting points for developing more flexible and powerful
spontaneous closures and finally a strong and efficient
Cyberskin. The basis for the implementation of this Cyberskin are
paradigms such as the MAPE-k cycle (
        <xref ref-type="bibr" rid="ref13">Kephart and Chess
(2003)</xref>
        ) or the Observer/Controller tandem (
        <xref ref-type="bibr" rid="ref25">Tomforde et al.
(2011)</xref>
        ,
        <xref ref-type="bibr" rid="ref26">Tomforde et al. (2016)</xref>
        ).
      </p>
    </sec>
    <sec id="sec-5">
      <title>Conclusion</title>
      <p>The trends in ICT such as openness, heterogeneity of
interaction partners, (hidden) mutual influences, and faster
response times especially in human-machine interaction
require a more ’life-like’ system design. We described our
vision of a ’Cyberskin’ that better integrates already
existing technology (e.g., self-healing techniques, learning
capabilities) and contains novel safety and security means to
dynamically realize a context-aware interaction protocol.</p>
      <p>It is also important that as a community we consider new
types of non-message-passing architectures and how we can
imitate the other manners of interactions found in
biological systems. The information conveyed by the deformation
of the skin, the immediate identification of locality, and the
interplay of different types of receptors are not conveyed by
networks alone in biology and we need creative concepts for
how we could architect and model that. Cyberskin requires
new thinking about the meaning of what is monitored for
and the implementation of new measurands and methods.</p>
      <p>Future work is on implementing aspects of ’Cyberskin’
and seeing how generalizable those implementations can
become. Looking across the commonalities among animal
skins gives us the hope that there will key principles in
implementing Cyberskin, such as permeability (intelligent
gating of traffic across membranes) or the criticality of having
different dedicated receptors.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <surname>Atema</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>1980</year>
          ).
          <article-title>Chemical senses, chemical signals and feeding behavior in fishes. Fish Behaviour and its Use in the Capture and</article-title>
          Culture of Fishes, pages
          <fpage>57</fpage>
          -
          <lpage>101</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <string-name>
            <surname>Atzori</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Iera</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Morabito</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          (
          <year>2010</year>
          ).
          <article-title>The internet of things: A survey</article-title>
          .
          <source>Computer networks</source>
          ,
          <volume>54</volume>
          (
          <issue>15</issue>
          ):
          <fpage>2787</fpage>
          -
          <lpage>2805</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <surname>Bellman</surname>
            ,
            <given-names>K. L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , and W u¨rtz,
          <string-name>
            <surname>R. P.</surname>
          </string-name>
          (
          <year>2014</year>
          ).
          <article-title>Interwoven systems: Self-improving systems integration</article-title>
          . In Eighth IEEE International Conference on Self-Adaptive and
          <article-title>Self-Organizing Systems Workshops</article-title>
          ,
          <string-name>
            <surname>SASOW</surname>
          </string-name>
          <year>2014</year>
          , London, United Kingdom,
          <source>September</source>
          <volume>8</volume>
          -
          <issue>12</issue>
          ,
          <year>2014</year>
          , pages
          <fpage>123</fpage>
          -
          <lpage>127</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <surname>Bellman</surname>
            ,
            <given-names>K. L.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Walter</surname>
            ,
            <given-names>D. O.</given-names>
          </string-name>
          (
          <year>1984</year>
          ).
          <article-title>Biological processing</article-title>
          .
          <source>American Journal of Physiology-Regulatory, Integrative and Comparative Physiology</source>
          ,
          <volume>246</volume>
          (
          <issue>6</issue>
          ):
          <fpage>R860</fpage>
          -
          <lpage>R867</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <surname>Bullock</surname>
            ,
            <given-names>T. H.</given-names>
          </string-name>
          (
          <year>1982</year>
          ).
          <source>Electroreception. Annual review of neuroscience</source>
          ,
          <volume>5</volume>
          (
          <issue>1</issue>
          ):
          <fpage>121</fpage>
          -
          <lpage>170</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <surname>Calma</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kottke</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sick</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2017</year>
          ).
          <article-title>Learning to learn: Dynamic runtime exploitation of various knowledge sources and machine learning paradigms</article-title>
          .
          <source>In 2017 IEEE 2nd International Workshops on Foundations and Applications of Self* Systems (FAS* W)</source>
          , pages
          <fpage>109</fpage>
          -
          <lpage>116</lpage>
          . IEEE.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <surname>D'Angelo</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gerasimou</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ghahremani</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Grohmann</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nunes</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pournaras</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2019</year>
          ).
          <article-title>On learning in collective self-adaptive systems: state of practice and a 3d framework</article-title>
          .
          <source>In 2019 IEEE/ACM 14th International Symposium on Software Engineering for Adaptive and SelfManaging Systems (SEAMS)</source>
          , pages
          <fpage>13</fpage>
          -
          <lpage>24</lpage>
          . IEEE.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <string-name>
            <surname>Diaconescu</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Frey</surname>
            ,
            <given-names>S.,</given-names>
          </string-name>
          <article-title>M u¨ller-</article-title>
          <string-name>
            <surname>Schloer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pitt</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2016</year>
          ).
          <article-title>Goal-oriented Holonics for Complex System (Self-)Integration: Concepts and Case Studies</article-title>
          .
          <source>In Proc. of the 10th IEEE Int. Conf. on Self-Adaptive and SelfOrganising Systems</source>
          , pages
          <fpage>100</fpage>
          -
          <lpage>109</lpage>
          . IEEE.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <string-name>
            <surname>Gruhl</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sick</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wacker</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , and Ha¨hner, J. (
          <year>2015</year>
          ).
          <article-title>A building block for awareness in technical systems: Online novelty detection and reaction with an application in intrusion detection</article-title>
          .
          <source>In IEEE 7th Inte. Conf. on Awareness Science and Technology, iCAST 2015</source>
          , pages
          <fpage>194</fpage>
          -
          <lpage>200</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <string-name>
            <surname>Gruhl</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Sick</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          (
          <year>2020</year>
          ).
          <article-title>Novelty detection in continuously changing environments</article-title>
          .
          <source>Future Generation Computing Systems</source>
          , Special Issue on '
          <source>Self-Improving Systems Integration'.</source>
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <string-name>
            <surname>Kantert</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Edenhofer</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , and
          <article-title>Mu¨ ller-</article-title>
          <string-name>
            <surname>Schloer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          (
          <year>2015</year>
          ).
          <article-title>Representation of trust and reputation in selfmanaged computing systems</article-title>
          .
          <source>In 13th IEEE International Conference on Dependable, Autonomic and Secure Computing</source>
          ,
          <string-name>
            <surname>DASC</surname>
          </string-name>
          <year>2015</year>
          , pages
          <fpage>1827</fpage>
          -
          <lpage>1834</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <string-name>
            <surname>Kantert</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.,</given-names>
          </string-name>
          <article-title>M u¨ller-</article-title>
          <string-name>
            <surname>Schloer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Edenhofer</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Sick</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          (
          <year>2017</year>
          ).
          <article-title>Quantitative robustness - A generalised approach to compare the impact of disturbances in selforganising systems</article-title>
          .
          <source>In Proceedings of the 9th International Conference on Agents and Artificial Intelligence</source>
          ,
          <source>ICAART</source>
          <year>2017</year>
          , Volume
          <volume>1</volume>
          ,
          <string-name>
            <surname>Porto</surname>
          </string-name>
          , Portugal,
          <source>February 24-26</source>
          ,
          <year>2017</year>
          , pages
          <fpage>39</fpage>
          -
          <lpage>50</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <string-name>
            <surname>Kephart</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Chess</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          (
          <year>2003</year>
          ).
          <article-title>The Vision of Autonomic Computing</article-title>
          . IEEE Computer,
          <volume>36</volume>
          (
          <issue>1</issue>
          ):
          <fpage>41</fpage>
          -
          <lpage>50</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          <string-name>
            <surname>Krupitzer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Roth</surname>
            ,
            <given-names>F. M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>VanSyckel</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Schiele</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Becker</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          (
          <year>2015</year>
          ).
          <article-title>A survey on engineering approaches for selfadaptive systems</article-title>
          .
          <source>Pervasive and Mobile Computing</source>
          ,
          <volume>17</volume>
          :
          <fpage>184</fpage>
          -
          <lpage>206</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          <string-name>
            <surname>Lumelsky</surname>
            ,
            <given-names>V. J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shur</surname>
            ,
            <given-names>M. S.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Wagner</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2001</year>
          ).
          <article-title>Sensitive skin</article-title>
          .
          <source>IEEE Sensors Journal</source>
          ,
          <volume>1</volume>
          (
          <issue>1</issue>
          ):
          <fpage>41</fpage>
          -
          <lpage>51</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <string-name>
            <surname>Micheva</surname>
            ,
            <given-names>K. D.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Smith</surname>
            ,
            <given-names>S. J.</given-names>
          </string-name>
          (
          <year>2007</year>
          ).
          <article-title>Array tomography: a new tool for imaging the molecular architecture and ultrastructure of neural circuits</article-title>
          .
          <source>Neuron</source>
          ,
          <volume>55</volume>
          (
          <issue>1</issue>
          ):
          <fpage>25</fpage>
          -
          <lpage>36</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          <article-title>Mu¨ ller-</article-title>
          <string-name>
            <surname>Schloer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2017</year>
          ).
          <source>Organic Computing - Techncial Systems for Survival in the Real World. Autonomic Systems. Birkha¨user Verlag. ISBN: 978-3-319-68476-5.</source>
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          <string-name>
            <surname>Pimentel</surname>
            ,
            <given-names>M. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Clifton</surname>
            ,
            <given-names>D. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Clifton</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Tarassenko</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          (
          <year>2014</year>
          ).
          <article-title>A review of novelty detection</article-title>
          .
          <source>Signal Processing</source>
          ,
          <volume>99</volume>
          :
          <fpage>215</fpage>
          -
          <lpage>249</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          <string-name>
            <surname>Rajkumar</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lee</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sha</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Stankovic</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2010</year>
          ).
          <article-title>Cyberphysical systems: the next computing revolution</article-title>
          .
          <source>In Design Automation Conference</source>
          , pages
          <fpage>731</fpage>
          -
          <lpage>736</lpage>
          . IEEE.
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          <string-name>
            <surname>Shier</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Butler</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Lewis</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          (
          <year>2004</year>
          ).
          <article-title>Hole's human anatomy &amp; physiology (lee, wt, lee, je, park, sh, lee. k, l</article-title>
          ., park, ka, &amp; hong, sg). Seoul: Jung Dam Publishing Co.
          <article-title>(Original work published</article-title>
          <year>2002</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          <string-name>
            <surname>Sick</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Oeste-Reiß</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Schmidt</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Zweig</surname>
            ,
            <given-names>K. A.</given-names>
          </string-name>
          (
          <year>2018</year>
          ).
          <article-title>Collaborative interactive learning</article-title>
          .
          <source>Informatik Spektrum</source>
          ,
          <volume>41</volume>
          (
          <issue>1</issue>
          ):
          <fpage>52</fpage>
          -
          <lpage>55</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , Ha¨hner, J., and
          <string-name>
            <surname>Sick</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          (
          <year>2014</year>
          ).
          <source>Interwoven Systems. Informatik-Spektrum</source>
          ,
          <volume>37</volume>
          (
          <issue>5</issue>
          ):
          <fpage>483</fpage>
          -
          <lpage>487</lpage>
          . Aktuelles Schlagwort.
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kantert</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <article-title>Mu¨ller-</article-title>
          <string-name>
            <surname>Schloer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          , Bo¨delt, S., and
          <string-name>
            <surname>Sick</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          (
          <year>2018</year>
          ).
          <article-title>Comparing the Effects of Disturbances in SelfAdaptive Systems - A Generalised Approach for the Quantification of Robustness</article-title>
          .
          <source>Springer Transactions on Computational Intelligence XXVIII</source>
          ,
          <volume>28</volume>
          :
          <fpage>193</fpage>
          -
          <lpage>220</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          and
          <article-title>Mu¨ ller-</article-title>
          <string-name>
            <surname>Schloer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          (
          <year>2014</year>
          ).
          <article-title>Incremental design of adaptive systems</article-title>
          .
          <source>J. Ambient Intell. Smart Environ</source>
          .,
          <volume>6</volume>
          (
          <issue>2</issue>
          ):
          <fpage>179</fpage>
          -
          <lpage>198</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Prothmann</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Branke</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , Ha¨hner, J.,
          <string-name>
            <surname>Mnif</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <article-title>Mu¨ ller-</article-title>
          <string-name>
            <surname>Schloer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Richter</surname>
            ,
            <given-names>U.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Schmeck</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          (
          <year>2011</year>
          ).
          <article-title>Observation and Control of Organic Systems</article-title>
          . In Mu¨ ller-Schloer,
          <string-name>
            <given-names>C.</given-names>
            ,
            <surname>Schmeck</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            , and
            <surname>Ungerer</surname>
          </string-name>
          , T., editors,
          <source>Organic Computing - A Paradigm Shift for Complex Systems, Autonomic Systems</source>
          , pages
          <fpage>325</fpage>
          -
          <lpage>338</lpage>
          . Birkha¨user Verlag.
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          <string-name>
            <surname>Tomforde</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rudolph</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bellman</surname>
            ,
            <given-names>K. L.</given-names>
          </string-name>
          , and W u¨rtz,
          <string-name>
            <surname>R. P.</surname>
          </string-name>
          (
          <year>2016</year>
          ).
          <article-title>An organic computing perspective on self-improving system interweaving at runtime</article-title>
          .
          <source>In 2016 IEEE International Conference on Autonomic Computing, ICAC</source>
          <year>2016</year>
          , Wuerzburg, Germany,
          <source>July 17-22</source>
          ,
          <year>2016</year>
          , pages
          <fpage>276</fpage>
          -
          <lpage>284</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>