-We investigate a small footprint cognitive architecture comprised of two reactive planner instances. The first interacts with the world via sensor and behaviour interfaces. The second monitors the first, and dynamically adjusts its plan in accordance with some predefined objective function. We show that this configuration produces a Darwinian mind, yet aware of its own operation and performance, and able to maintain performance as the environment changes. We identify this architecture as a second-order Darwinian mind, and discuss the philosophical implications for the study of consciousness. We use the Instinct Robot World agent based modelling environment, which in turn uses the Instinct Planner for cognition.
BIOLOGICALLY INSPIRED COGNITIVE ARCHITECTURES
From the 1950’s through to the 1980’s the study of
embodied AI assumed a cognitive symbolic planning model for
robotic systems — SMPA (Sense Model Plan Act) — the most
well known example of this being the Shakey robot project
[
In the 1990’s Rodney Brooks and others [
Following in-depth studies of animals such as gulls in their
natural environment, ideas of how animals perform action
selection were originally formulated by Nico Tinbergen and
other early ethologists [
Drive (D) has a priority and a releaser. When the Drive is released as a result of sensory input, a hierarchical plan of Competences, Action Patterns and Actions follows. POSH plans are authored, or designed, by humans alongside the design of senses and behaviour modules. An iterative approach is defined within BOD for the design of intelligent artefacts — these are known as agents, or if they are physically embodied, robots.
Daniel Dennett[
However, perhaps the simple Darwinian mind might also
be arranged to monitor itself, and in some small and limited
sense to be aware of its own performance and act to correct
it. Bryson suggests that consciousness might assist in action
selection [
The Instinct Planner [
It’s unique features are its tiny memory footprint and efficient operation, meaning that it can operate on a low powered micro-controller environment such as ARDUINO. Alternatively, as in this experiment, many planners can run within one application on a laptop PC.
The Instinct Robot World is a new agent based modelling tool, shown in Figure 1. This is an open source project and all code and configuration files are available online 1. Each virtual ‘robot’ within the Robot World uses an Instinct Planner to provide action selection. Strictly, since these virtual robots are not physically embodied, we should refer to them as agents. However, we have chosen to use ‘robot’ throughout, as intuitively these cognitive entities appear to be virtually embodied within the Robot World, and this choice of language seems more natural. In the final section of this paper we discuss
The Robot World allows many robots to be instantiated, each with the same reactive plan, or with a variety of plans.
The robots each have senses to sense the ‘walls’ of the environment, and other robots. The reactive plan invokes simple behaviours to move the robot, adjust its speed and direction, or interact with robots that it encounters within the world as it moves. Most importantly for this investigation, each robot also has a second Instinct Planner instance. This planner monitors the first, and is able to modify its parameters based on a predefined plan.
The Instinct Robot World provides statistical monitoring to report on the overall activity of the robots within the world. These include the average percentage of robots that are moving at any one time, the average number of time units (ticks) between robot interactions, and the average amount of time that the monitor planner intervenes to modify the robot plan. We use the Instinct Robot World to investigate the idea of Reflective Reactive Planning — one reactive planner driving behaviour based on sensory input and predefined drives and competences, and another reactive planner monitoring performance and intervening to modify the predefined plan of the first, in accordance with some higher level objective. This simple combination of two Darwinian minds, one monitoring the other, might also be considered to be a second-order
Darwinian mind. Reflective Reactive Planning - A 2nd Order Darwinian Mind #2
#1
WORLD ROBOT
We expect that second-order Darwinian minds will
outperform first order minds when the environment changes,
because the monitor planner is concerned with achieving
higher order objectives, and modifies the operation of the first
planner to improve its performance. We also hypothesise that
this architecture will remain stable over extended periods of
time, because by restricting ourselves to the reactive planning
paradigm we have reduced the number of degrees of freedom
within which the architecture must operate, and previous
work shows that first-order minds produce reliable control
architectures [
While the robot is in the ‘Interacting’ state it is shown as a shriek character (!) within the Robot World display. Once the robot has interacted its priority for interaction decreases, but ramps up over time. This may be likened to most natural drives, for example mating, feeding and the need for social interaction.
The Monitor Plan is designed to keep the robot exploring when it is overly diverted from social interactions. It achieves this by monitoring the time between interactions. If, over three interactions, the average time between interactions reduces below 1000 ticks, then the Monitor Planner reduces the priority of the interaction Drive. After 1000 ticks the priority is reset to its original level. We might use alternative intentional language here to say that the Monitor Planner ‘notices’ that the robot is being diverted by too many social interactions. It then reduces the priority of those interactions, so that the robot is diverted less frequently. After some time the Monitor Planner ceases to intervene until it next notices this situation re-occurring.
The Robot World is populated with varying numbers of robots (2, 3, 5, 10, 20, 50, 100, 200, 500, 1000), and for each number the experiment is run twice, once with a monitor plan, and once without. For each run, the environment is allowed to run for some time, typically about 10 minutes, until the reported statistics have settled and are seen to be no longer changing over time.
The results are most elegantly and succinctly presented as simple graphs. Firstly, the average number of robots moving at any one time within the world is shown in Figure 3. In both
Robots Moving in the World 100.00% 90.00% 80.00% 70.00% 60.00% 50.00% 40.00% 30.00% 20.00% 10.00% 0.00% 1 4000 3500 3000 2500 2000 1500 1000 500 cases, as the number of robots within the world increases, the amount of time that the robot spends moving reduces. However the Monitor Planner acts to reduce the extent of this reduction from 60% to less than 20% over the full range of two to a thousand robots within the world. Similarly, in Figure 4 we see that as more robots are introduced into the world, the average time between interactions naturally reduces. However, the action of the Monitor Planner progressively limits this reduction, so that with 1000 robots the time between interactions is almost trebled, from 310 to 885 ticks per interaction. Interestingly, in both these graphs we see smooth curves both with and without the action of the monitor plan.
Average Time Between Interactions 10 100
1000
Robots No Monitor
With Monitor
The final graph, Figure 5 also shows a smooth, sigmoid like increase in activation of the Monitor Planner as the number of robots increases, plotted on a logarithmic scale.
The Instinct Robot World was found to be a stable, reliable platform for our experiments, and the results it achieved were repeatable. The application is single threaded, and so uses only one core of the CPU on the laptop PC on which it was run. Nevertheless, it was possible to simulate 1000 robots with both reactive planners active operating in the world at the rate of 70 clock cycles (ticks) per second.
From the results we can see that by using a second Instinct instance to monitor the first, we can achieve real-time learning within a tiny-footprint yet nevertheless symbolic cognitive architecture. In addition, since this learning modifies parameters from a human designed plan, the learning can be well understood and is transparent in nature. This contrasts strongly with machine learning approaches such as neural networks that typically learn offline, are opaque, and require a much larger memory workspace. Despite the stochastic nature of the environment, the performance graphs show smooth curves over a wide range of robot populations.
This relatively simple experiment also provides further fuel
for the fire concerning the philosophical discussion of the
nature of consciousness. Critics may say that when we use
the intentional stance [
We could certainly envisage a much grander architecture, assembled at the level of reactive planners, using maybe hundreds or thousands of planners each concerned with certain objectives. Many of these planners may be homeostatic in nature, whilst others would be concerned with the achievement of higher level objectives. We must remember that planners merely coordinate action selection, and say nothing about how sensor models may be formed, nor how complex behaviours themselves may be implemented. However, all dynamic architectures need some kind of decision centric ‘glue’ to bind them together, and reactive planning seems to be a useful candidate here, as evidenced by practical experiment and biological underpinning.
Machine transparency is a core element of our research. We
have shown elsewhere [
The human brain does not run by virtue of some elegant algorithm. It is a hack, built by the unseeing forces of evolution, without foresight or consideration for modularity, transparency or any other good design practice. If we are to build intelligent systems, the brain is not a good physical model from which we should proceed. Rather, we should look at the behaviours of intelligent organisms, model the way in which these organisms react, and then scale up these models to build useful, manageable intelligent systems.
Whilst our Reflective Reactive Planner is a very simple architecture, it does share many of the characteristics cited for architectures that are worthy of evaluation, such as efficiency and scalability, reactivity and persistence, improvability, and autonomy and extended operation [19]. We hope that our work with reactive planners might strengthen the case for their consideration in situations where decision centric ‘glue’ is required.
We have shown that a second-order Darwinian mind may be constructed from two instances of the Instinct reactive planner. This architecture, which we call Reflective Reactive Planning, successfully controls the behaviour of a virtual robot within a simulated world, according to pre-defined goals and higher level objectives. We have shown how this architecture may provide both practical cognitive implementations, and inform philosophical discussion on the nature and purpose of consciousness.
The Instinct Robot World is an entirely open source platform, available online. We welcome those interested in agent based modelling, cognitive architectures generally, and reactive planning specifically, to investigate these technologies and offer suggestions for new applications and further work. One possibility might be to apply this architecture to the Small Loop Problem [20], a specific challenge for biologically inspired cognitive architectures.
We continue to develop robot applications for the Instinct Planner, together with the Instinct Robot World. We are investigating the use of a small robot swarm to build a physically embodied version of this experiment. To this end, we are currently working with the University of Manchester’s Mona robot2.
[16] P. O. A. HAIKONEN, “Consciousness and Sentient Robots,” International Journal of Machine Consciousness, vol. 05, no. 01, pp. 11–26, 2013. [Online]. Available: http://www.worldscientific.com/doi/ abs/10.1142/S1793843013400027 [17] J. H. Holland, Emergence: From Chaos to Order, ser. Popular science / Oxford University Press. Oxford University Press, 2000. [Online].
Available: https://books.google.co.uk/books?id=VjKtpujRGuAC [18] M. Boden, J. Bryson, D. Caldwell, K. Dautenhahn, L. Edwards, S. Kember, P. Newman, V. Parry, G. Pegman, T. Rodden, T. Sorell, M. Wallis, B. Whitby, and A. Winfield, “Principles of robotics,” The United Kingdom’s Engineering and Physical Sciences Research Council (EPSRC), April 2011, web publication. [19] P. Langley, J. E. Laird, and S. Rogers, “Cognitive architectures: Research issues and challenges,” Cognitive Systems Research, vol. 10, no. 2, pp. 141–160, jun 2009. [Online]. Available: http://linkinghub. elsevier.com/retrieve/pii/S1389041708000557 [20] O. L. Georgeon and J. B. Marshall, “The small loop problem: A challenge for artificial emergent cognition,” Advances in Intelligent Systems and Computing, vol. 196 AISC, pp. 137–144, 2013.