=Paper=
{{Paper
|id=Vol-2287/paper25
|storemode=property
|title=Arousal and Awareness in a Humanoid Robot
|pdfUrl=https://ceur-ws.org/Vol-2287/paper25.pdf
|volume=Vol-2287
|authors=Christian Balkenius,Trond A. Tjøstheim,Birger Johansson
|dblpUrl=https://dblp.org/rec/conf/aaaiss/BalkeniusTJ19
}}
==Arousal and Awareness in a Humanoid Robot==
https://ceur-ws.org/Vol-2287/paper25.pdf
Arousal and Awareness in a Humanoid Robot
Christian Balkenius1[0000−0002−1478−6329] , Trond A. Tjøstheim1 , and Birger
Johansson1[0000−0002−9834−4279]
Lund University Cognitive Science, Sweden christian.balkenius@lucs.lu.se
http://www.lucs.lu.se
Abstract. We describe how an arousal system that controls the levels of
awareness can be implemented in a robot. The different levels of aware-
ness correspond to different states of consciousness and we argue that
an artificial arousal system modeled after its biological counterpart has
a useful function in controlling the cognitive processing of a brain-like
cognitive architecture. The level of awareness depends on arousal that
in turn is controlled by novel or emotionally charged stimuli as well as
by a circadian clock. Arousal is also modulated during cognitive tasks
to control the randomness of decision processes and to select between
exploration and exploitation.
Keywords: arousal · locus coeruleus · exploration-exploitation · gain
control.
1 Introduction
Why is the study of consciousness relevant for robotics? In our view, there are
processes that reflect the level of awareness in humans that can play an impor-
tant role in the control of a robot. These processes are commonly associated
with different conscious states. However, our interest in these processes is not
primarily that they are correlated with different conscious states, but rather that
they are useful as control mechanisms in their own right. We believe that this
gives a solid basis for studying consciousness in AI systems since the different
mechanisms are motivated by their functional role rather than by subjective
notions of awareness or conscious experience.
The present paper presents an overview of the arousal system for a humanoid
robot. This arousal mechanism, which is is roughly modeled after its biological
counterpart, determines the level of awareness and controls processing in a mem-
ory system that supports many cognitive operations. The arousal system decides
how information is processed in memory, how thoughts are focused or allowed to
wander, and sets the balance between exploration and exploitation. The level of
arousal can be considered one of several meta-parameters that control processing
in a natural or artificial cognitive system [8].
A central componenent of the robot control architecture is a biologically
motivated memory system. We have argued elsewhere that a conscious robot
needs to maintain an inner world to support various cognitive functions. This
2 C. Balkenius et al.
inner world is based on a combination of episodic, semantic and working memory
structures [3]. The basis for the model is an autoassociative network with latching
dynamics [16, 6] that binds together stimuli with locations and learns episodic
sequences. Our implemented system support tasks such as the A-not-B test,
delayed matching to sample, episodic recall, and vicarious trial and error [3].
In the memory system, memories correspond to attractor states of the net-
work and perception and recall is the process of finding the attractor state most
consistent with the input. The latching dynamics make attractors semi-stable.
This causes the memory state to jump from attractor to attractor in an episodic
manner unless the state is explicitly locked in the current attractor. As a con-
sequence, learned episodes can later be recalled as episodic memory transitions.
In addition, the memory system supports semantic transitions where a stimulus
associates to other similar stimuli [13].
A critical parameter of the memory system is its gain that controls whether
the system stays in an attractor or is allowed to transition to other memory
states. With a very high gain, the memory system will be locked in an attractor.
With lower gain, it will start to transition through learned episodic sequences.
If the gain is lowered even more, the randomness of the state transitions will
increase as a results of noise which can produce novel transitions from combina-
tion of old episodes [3]. These mechanisms supports memory paths that range
from focused recall to completely random state transitions. A central claim of
this paper is that gain control is a fundamental function of the arousal system
which goes hand in hand with different levels of awareness.
In the brain, the level of awareness is controller by the ascending reticular
activating system. It consists of a number of nuclei that can control processing
in the whole of the brain, including wakefulness and sleep. Here we will focus
on the locus coeruleus (LC), which is the primary nucleus for the control of
wakefulness though its noradrenargic projections to cortical, subcortical, cere-
bellar and brainstem circuits of the brain. A high LC activity is associated with
heightened attention. During sleep, the LC is almost completely turned off. We
have earlier developed a computational model that is able to explain how the LC
is activated by novel or emotionally charged stimuli [14]. This model forms the
basis for the system described here. The LC activation is also influenced by a
circadian oscillator in the hypothalamus that tracks the 24 hour day-night cycle
[11]. This makes the LC more susceptible to stimulation during the day than
during night.
It has been suggested that the level of noradrenaline functions as gain modu-
lation in cortical processing by controlling the randomness of decision processes
[1, 7, 9]. This can be seen as a choice between exploration and exploitation, where
a lower arousal causes a more random choice, or exploration, while a higher level
of arousal, causes exploitation of learned information. The concept of gain modu-
lation is related to Hebb’s idea of an optimal level of arousal [12] and the related
Yerkes-Dodsons law [22]. The LC also controls muscle tone. There is thus a con-
nection between mentally increasing focus or trying harder to physically using
more force. Whether this is a good strategy or not depends on the problem.
Arousal and Awareness in a Humanoid Robot 3
Simple problems benefit from an increased focus while harder problems require
the mind to evaluate different alternatives. This is parallel to physical obstacles,
where simple situations can benefit from increased force, while harder situation
require active problem solving. Interestingly, the LC also influences the size of
the pupil which makes pupil dilation a useful index of increased noradrenergic
activity [4].
2 Levels of Awareness
Different levels of awareness all have roles to play in cognitive processing, ranging
from intense focus to drowsiness and sleep. This is a continuous scale, but we
here focus on some qualitatively different levels of awareness. All these states
have useful functions in the control of a robot.
Focus At the highest level of arousal, the attentional and memory systems are a
focused on a single stimulus or memory state. During manual actions, this is the
relevant level of arousal as it allows the visual system to lock on to a stimulus
without being distracted by irrelevant stimuli [18].
Although intense focus could sound like an ideal state, this is not always the
case. The problem is that it locks the current memory state and if it is not the
correct one, it will be hard to move away from it. This can be seen, for example,
during the tip-of-the-tongue phenomenon [5] where we fail to recall a word even
though we in fact know it. Thinking more intensely about the word does not
help recall and we are better off giving up. Once the mind is allowed to wander,
the correct word will very likely be recalled. This shows that the level of arousal
must be decreased for novel memories to become activated. Interestingly, it has
been shown that when people give up on a task, arousal is lowered in this way
as can be seen in their pupils [23].
Planning and Problem Solving To find novel combinations of episodic se-
quences, as is necessary during problem solving, it is required that arousal is kept
at a high, but not too high level. Focused exploration of alternatives is combined
with unfocused mind wandering to generate new alternatives. When the robot
fails to move forward in a task, the arousal is decreased to allow novel solutions
to emerge. These solutions are either tested in the environment or evaluated in
memory. The process is similar to simulated annealing where noise is introduced
in an artificial neural network to allow its memory state to jump to a novel so-
lution [15]. Once a new solution is found, the arousal is increased again to allow
the execution of the corresponding behavior. This increase in arousal is caused
by an evaluative mechanisms that increases emotion and consequently also LC
activity.
Planning is similar to problem solving in many ways, but need not be con-
cerned with an immediate problem. When planning, the system can be viewed
as stringing together behaviours and episode-fragments in memory to compose
4 C. Balkenius et al.
a plausible path from the present context to some desired goal state. The ex-
ploration is hence goal-directed and more focused than the mind wandering,
daydreaming state described below. However, depending on the general arousal
level of the system, planning may degenerate into daydreaming, or oscillate be-
tween the two. During planning, the memory system will produce the kind of
forward sweeps seen during vicarious trial and error in animals [19]. This can be
seen as internal simulation of an action sequence that later may be used in the
real world.
Daydreaming During daydreaming or mind wandering the arousal is lower
compared to planning. The memory transitions of the robot mirror the state in
which the default network dominates access to working memory [17]. This state
is entered if the system is cognitively unoccupied, and can be viewed as a way of
exploring novel avenues through memory, not necessarily linked to specific goals.
Like planning, it has a strong episodic component, and allows internal rehearsal
of alternative behavioural sequences linked to common situations. This type of
mechanism has previously been used for example in reinforcement learning in
the DYNA-architecture, where behaviors are rehearsed from memory as well as
in the environment [21].
Drowsiness Drowsiness is associated with a very low level of arousal. In terms
of gain control, it allows the memory state to drift freely without any goal
direction or focus. This is a transitional state that can be linked to the sleep-
wake cycle, at the transition stage to sleep. However, it can also be used to
simulate the propensity to seek novelty or change task if the current task does not
yield sufficient reward. The same can happen if the task is too energy-draining
compared to the level of reward, or the level of perceived progress towards a goal.
Drowsiness might also be entered into if the robot is understimulated. That is,
the activation of its perceptual system is below some threshold. For example,
the robot may be presented with a static or blank scene or it may be required
to not move for a too long period of time. In humans and animals, a lack of
stimulation from proprioceptors and other signals from the peripheral system
associated with sitting or laying for extended periods of time tend to induce
drowsiness and lethargy [10].
Drowsiness can end up in a sleep state, but because it also motivates action,
it can also cause the robot to do something novel to increase arousal. Which
direction the arousal will take depends on contextual factors such as the avail-
ability of objects in the environment that can stimulate actions. We have earlier
modeled how novel or emotional stimuli can increase arousal through the ac-
tivation of a simulated LC [14], thus causing states associated with increased
awareness in the robot.
Sleep At the lowest level of arousal we find sleep. The obvious use for a sleep-
like state in a robot is that it serves energy conservation. However, sleep is not
Arousal and Awareness in a Humanoid Robot 5
equivalent to being turned off. The sleeping robot can quickly become aroused
and move to an active attentive state if something unexpected happens. This
implies that rudimentary sensory processing must also occur during sleep, but
perhaps at a lower resolution or at a lower rate to save energy. As in the brain,
the sleep-cycle is also controlled by a clock, which determines at what time it
is appropriate to sleep, and allows the robot to quickly go back to sleep again
after being woken up by something unexpected.
In general, monitoring of the environment during sleep depends on passive
attention to the environment. Such attention is controlled by processes of ha-
bituation that form a model of the environment that can detect unexpected
disturbances [2]. For the robot, we implement this as adaptive background mix-
ture models for each of its sensory modalities that learn the normal variation
of the background input [20]. Similar methods are used for visual, auditory and
tactile inputs.
3 Discussion
Having a system that dynamically transitions between arousal states similar to
those found in animals allows the study of aspects of consciousness that may be
hard or time consuming to study in biological systems. It may also allow insights
into the phenomenon of consciousness as such, by accessing it indirectly through
arousal, rather than attempting to tackle it directly. Thus, the infamous “hard”
problem of consciousness may perhaps be chipped away at without becoming
stuck in the traditional debates surrounding this issue.
Closer to everyday concerns, perhaps, are the requirements of artificial sys-
tems to conserve and optimize energy use when not connected to a continuous
energy supply. We argue that modelling arousal levels in general, and the LC
noradrenergic system in particular affords a biologically plausible way of man-
aging energy expenditure, as well as a highly interesting avenue for studying dy-
namic cognitive behaviour with a high level of experimental control. Moreover, a
robot with a biologically plausible arousal system may allow insight into human
maladaptive conditions associated with the noradrenergic system. This includes
depression and anxiety disorders, attention deficit disorders, schizophreniaand
narcolepsy.
To summarize, we have outlined the function of a biologically motivated
arousal system for a humanoid robot. This central system controls how the
memory system transitions from one state to the next to recall episodic memories
or to produce novel states and transitions based on earlier experiences. The level
of arousal is closely connected to level of awareness and acts as a speed control
for thought.
References
1. Aston-Jones, G., Cohen, J.D.: An integrative theory of locus coeruleus-
norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neu-
rosci. 28, 403–450 (2005)
6 C. Balkenius et al.
2. Balkenius, C.: Attention, habituation and conditioning: Toward a computational
model. Cognitive Science Quarterly 1(2), 171–214 (2000)
3. Balkenius, C., Tjøstheim, T.A., Johansson, B., Gärdenfors, P.: From focused
thought to reveries: a memory system for a conscious robot. Frontiers in Robotics
and AI 5, 29 (2018)
4. Bradley, M.M., Miccoli, L., Escrig, M.A., Lang, P.J.: The pupil as a measure of emo-
tional arousal and autonomic activation. Psychophysiology 45(4), 602–607 (2008)
5. Brown, R., McNeill, D.: The “tip of the tongue” phenomenon. Journal of verbal
learning and verbal behavior 5(4), 325–337 (1966)
6. Chossat, P., Krupa, M., Lavigne, F.: Latching dynamics in neural networks with
synaptic depression. arXiv preprint arXiv:1611.03645 (2016)
7. Donner, T.H., Nieuwenhuis, S.: Brain-wide gain modulation: the rich get richer.
Nature Neuroscience 16(8), 989–990 (2013)
8. Doya, K.: Metalearning and neuromodulation. Neural Networks 15(4), 495–506
(2002)
9. Eldar, E., Cohen, J.D., Niv, Y.: The effects of neural gain on attention and learning.
Nature neuroscience 16(8), 1146–1153 (2013)
10. Fureix, C., Meagher, R.K.: What can inactivity (in its various forms) reveal about
affective states in non-human animals? a review. Applied Animal Behaviour Science
171, 8–24 (2015)
11. González, M.M., Aston-Jones, G.: Circadian regulation of arousal: role of the no-
radrenergic locus coeruleus system and light exposure. Sleep 29(10), 1327–1336
(2006)
12. Hebb, D.O.: The Organization of Behavior. Wiley, New York (1949)
13. Herrmann, M., Ruppin, E., Usher, M.: A neural model of the dynamic activation
of memory. Biological cybernetics 68(5), 455–463 (1993)
14. Johansson, B., Balkenius, C.: A computational model of pupil dilation. Connection
Science pp. 1–15 (2016)
15. Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing.
science 220(4598), 671–680 (1983)
16. Lerner, I., Bentin, S., Shriki, O.: Automatic and controlled processes in semantic
priming: an attractor neural network model with latching dynamics. In: Proceed-
ings of the Cognitive Science Society. vol. 32, pp. 1112–1117 (2010)
17. Mason, M.F., Norton, M.I., Van Horn, J.D., Wegner, D.M., Grafton, S.T., Macrae,
C.N.: Wandering minds: the default network and stimulus-independent thought.
Science 315(5810), 393–395 (2007)
18. Oliva, M.: Attention and Learning through the Eyes of the Emotional Brain. Lund
University Cognitive Studies (2018)
19. Redish, A.D.: Vicarious trial and error. Nature reviews. Neuroscience 17(3), 147
(2016)
20. Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time
tracking. In: cvpr. p. 2246. IEEE (1999)
21. Sutton, R.S.: Dyna, an integrated architecture for learning, planning, and reacting.
ACM SIGART Bulletin 2(4), 160–163 (1991)
22. Yerkes, R.M., Dodson, J.D.: The relation of strength of stimulus to rapidity of
habit-formation. Journal of comparative neurology and psychology 18(5), 459–482
(1908)
23. Zekveld, A., Kramer, S.: Cognitive processing load across a wide range of listen-
ing conditions: Insights from pupillometry. Psychophysiology 51(3), 277–284 (Mar
2014)