=Paper= {{Paper |id=None |storemode=property |title=Introducing Sensory-motor Apparatus in Neuropsychological Modelization |pdfUrl=https://ceur-ws.org/Vol-1100/paper7.pdf |volume=Vol-1100 |dblpUrl=https://dblp.org/rec/conf/aiia/GigliottaBM13 }} ==Introducing Sensory-motor Apparatus in Neuropsychological Modelization== https://ceur-ws.org/Vol-1100/paper7.pdf

Introducing Sensory-motor Apparatus in
Neuropsychological Modelization

Onofrio Gigliotta1 , Paolo Bartolomeo2,3 , and Orazio Miglino1
1
University of Naples Federico II, Naples, Italy
onofrio.gigliotta@unina.it orazio.miglino@unina.it
2
Centre de Recherche de l’Institut du Cerveau et de la Moelle èpiniére, Inserm
U975, UPMC-Paris6, Paris, France
paolo.bartolomeo@gmail.com
3
Department of Psychology, Catholic University, Milan, Italy

Abstract. Mainstream modeling of neuropsychological phenomena has
mainly been focused to reproduce their neural substrate whereas sensory-
motor contingencies have attracted less attention. In this study we trained
artificial embodied neural agents equipped with a pan/tilt camera, pro-
vided with different neural and motor capabilities, to solve a well known
neuropsychological test: the cancellation task. Results showed that em-
bodied agents provided with additional motor capabilities (a zooming
motor) outperformed simple pan/tilt agents, even those equipped with
more complex neural controllers. We concluded that the sole neural com-
putational power cannot explain the (artificial) cognition which emerged
throughout the adaptive process.

Keywords: Neural agents, Active Vision, Sensory motor integration,
Cancellation task

1 Introduction

Mainstream models of neuropsychological phenomena are mainly based on arti-
ficial bioinspired neural networks that explain the neural dynamics underlying
some neurocognitive functions (see for example [4]). Much less attention has
been paid to modeling the structures that allow individuals to interact with
their environment, such as the sensory-motor apparatus (see [5] for an excep-
tion). The neurally-based approach is based on the assumption that the neural
computational power and its organization is the main source of the mental life.
Alternatively, as stated by eminent theorists [8, 9, 11], cognition could be viewed
as a process that emerges from the interplay between environmental requests and
organisms’ resources (i.e. neural computational power, sensory-motor apparatus,
body features, etc.). In other words, cognition comes from the adaptive history
(phylogenetic and/or ontogenetic) in which all living organisms are immersed
and take part. This theoretical perspective leads to building up artificial models
that take into account, in embryonic form, neural structures, sensory-motor ap-
paratus, environment structure and adaptation processes (phylogenetic and/or
ontogenetic). This modelization approach is developed by the interdisciplinary
field of Artifcial Life and it is widely used in order to modelize a large spectrum of
natural phenomena[3, 10, 6, 7]. In this study we applied artificial life techniques
to building up neural-agents able to perform a well known neuropyschological
task, the cancellation task, currently used to study the neurocognitive functions
related to spatial cognition. Basically, this task is a form of visual search and it
is considered as a benchmark to detect spatially-based cognitive deficits such as
visual neglect [1].

2 Materials and Methods

2.1 The cancellation task

The cancellation task is a well known diagnostic test used to detect neuropsy-
chological deficits in human beings. The test material typically consists of a
rectangular white sheet which contains randomly scattered visual stimuli. Stim-
uli may be of two (or more) categories (for example triangles and squares, lines
and dots, A and C letters, etc.). Figure 1a shows an example of the task. Sub-
jects are asked to find and cancel by a pen stroke all the items of a given category
(e.g. open circles). Fundamentally, it is a visual search task where some items
are coded as distractors and other represent targets (the items to cancel). Brain-
damaged patients can fail to cancel targets in a sector of space, typically the left
half of the sheet after a lesion in the right hemisphere (visual neglect, see figure
1b).Here we simulated this task through a virtual sheet (a bitmap) in which a
set of targets and distractors are randomly drawn (Fig. 1c), and trained neural
agents provided with a specific sensory-motor apparatus, described in the next
section, to perform the task.

Fig. 1. a) Cancellation task in which targets are open circles and full circles are distrac-
tors; b) open circles canceled with a circular mark; c) cancellation task implemented
in our experiments: grey filled circles are targets and black ones distractors
2.2 The neural agent’s sensory-motor apparatus

A neural agent is equipped with a pan/tilt camera provided with a motorized
zoom and an actuator able to trigger the cancellation behavior (Fig.2). The
camera has a resolution of 350x350 pixels. Two motors allow the camera to
explore the visual scene by controlling rotation around x and y axes while a
third motor controls the magnification of the observed scene. Finally, the fourth
actuator triggers a cancellation movement that reproduces in a simplified fashion
the behavior shown by human individuals when asked to solve the task. The

Fig. 2. The sensory-motor apparatus: two motors control rotation around two axes,
one motor controls the zoom and a supplementary motor (not depicted) triggers the
cancellation behaviour.

behavior of the neural agents is controlled by a neural network able to control
the four actuators and to manage the camera visual input. The camera output
does not gather all the pixel data, but pre-processes visual information using
an artificial retina made up of 49 receptors (Fig. 3, right). Visual receptors are
equally distributed on the surface of the camera; each receptor has a round visual
field with a radius of 25 pixel. The activation of each receptor is computed by
averaging the luminance value of the perceived scene (Fig. 3, left)

2.3 The cancellation task on the artificial neural agent

In order to simulate a form of cancellation task in silico, we trained neural
agents endowed with different neural architectures to perform the cancellation
task. In particular, we presented a set of randomly scattered stimuli made up of
Fig. 3. Right. Neural agent’s retina. Receptors are depicted as blue filled circle, recep-
tive fields as dotted red circles. Left. Receptor activation is computed averaging the
luminance value of the perceived stimuli.

distractors (black stimuli) and targets (grey stimuli) (Fig. 4) and rewarded neural
agents for the ability to find (by putting the center of their retina over a target
stimulus) and cancel/mark correct stimuli (activating the proper actuator).

Fig. 4. Random patterns of targets (gray filled circles) and distractors (black filled
circles)

2.4 Experiments
In order to perform the cancellation task, an agent has to develop (1) the ability
to search for stimuli, and (2) to decide whether a stimulus is a target or not. To
study how these abilities emerge we used controllers which were able to learn
and self-adapt to perform the task. We provided agents with neural networks
with different architectures designed by varying the number of internal neurons,
the pattern of connections and the motor capabilities. In particular, we designed
four architectures of increasing complexity (Fig. 5). Complexity was determined
first by the number of neurons and by their connections. In this case more
complexity turns on more computational power that a controller can manage.
Second, complexity can be related to the body in terms of sensory or motor
resources that can be exploited to solve a particular task.

Fig. 5. Networks trained for the cancellation task: a) Perceptron; b)Feed forward neural
networks with a 10 neurons hidden layer; c) Network b with a recurrent connection; d)
Network c with a direct input-output connection layer.

In 8 evolutionary experiments, we trained neural agents by varying the con-
trollers’ architecture (4 conditions) and by adding the possibility to use or not
the zooming actuator (2 conditions). For each experiment 10 populations of ar-
tificial agents were trained through a standard genetic algorithm [8] for 1000
generations. For each generation neural agents were tested 20 times with ran-
dom patterns of target and distractor stimuli. Each agent was rewarded for its
ability to explore the visual scene and correctly cancel/mark target stimuli.

3 Results
For each evolutionary experiment we post-evaluated the best ten individuals for
the ability to correctly mark target stimuli. In particular, we tested each indi-
vidual with 800 different random stimuli patterns.The rationale behind the post
evaluation is twofold. First, during evolution each agent experienced a small
number of possible visual patterns (20); second, the reward function was made
up of two parts so as to avoid bootstrapping problems: one component to re-
ward exploration and the second one to reward correct cancellations. Results are
reported as proportion correct in cancellation tests. Figure 6 reports the post-
evaluation results for each architecture in each motor condition: with the ability
to operate the zoom (Fig. 6 a,b,c and d) and without this ability (a-, b-, c- and
d-).

Fig. 6. Boxplots containing the post evaluation performance for each evolutionary
experiment. Each boxplot reports the performance of the best 10 individuals.

For all the neural networks we found significant differences (p<0.001, two-
tailed Mann-Whitney U test) between the condition presence/absence of the
capacity to zoom incoming stimuli. In both groups there were significant differ-
ences between network a and the remaining networks, but no significant differ-
ence emerged between networks b, c and d. Interestingly, there were no significant
differences between a, b−, c−, and d−. This last result suggests that a greater
computational power can replace to some extent the absence of a zooming ca-
pacity. As mentioned above, neglect patients fail to process information coming
from the left side of space. However healthy individuals can also show mild signs
of spatial bias in the opposite direction (i.e., penalizing the right side of space),
a phenomenon termed pseudoneglect [12]. In order to asses if such bias could
simply have emerged as a side effect of the training process, we tested the best
evolved individuals of the network d with a set of 200 couples of target stim-
uli placed symmetrically respect to the x axes of the artificial agent. Results
(Fig. 7) show that only one individual (nr. 3 in Fig. 7) did not present a signifi-
cant left-right difference, while all the remaining had different degrees of spatial
preference.
Fig. 7. Individual proportion correct in the selection of left or right-sided targets as
first visited item.

4 Conclusion

At variance with the mainstream approach in the modeling of neuropsycholog-
ical phenomena, mainly focused on reproduction of the neural underpinnings
of cognitive mechanisms, we showed that having a proper motor actuator can
greatly improve the performance of evolved neural agents in a cancellation task.
In particular, we demonstrated that an appropriate motor actuator (able to im-
plement a sort of attentional/zooming mechanism) can overcome the limits asso-
ciated with intrinsic computational power (e.g. number of internal neurons and
neural connections in our case). Second, we showed that spatial bias in stimulus
selection in healthy neural agents can be a side effect of the training process.
In future extensions of this work we plan to test injured neural agents, eval-
uate biologically-inspired neural architectures following recent research results
on brain attentional networks[2] and to extend the range of different explored
sensory-motor capabilities.

References

1. P Azouvi, C Samuel, A Louis-Dreyfus, T Bernati, P Bartolomeo, J-M Beis,
S Chokron, M Leclercq, F Marchal, Y Martin, G de Montety, S Olivier, D Peren-
nou, P Pradat-Diehl, C Prairial, G Rode, E Siroff, L Wiart, and M Rousseaux.
Sensitivity of clinical and behavioural tests of spatial neglect after right hemisphere
stroke. Journal of Neurology, Neurosurgery & Psychiatry, 73(2):160–166, 2002.
2. Paolo Bartolomeo, Michel Thiebaut de Schotten, and Fabrizio Doricchi. Left uni-
lateral neglect as a disconnection syndrome. Cerebral Cortex, 17(11):2479–2490,
2007.
3. M. Bedau. Artificial life: organization, adaptation and complexity from the bottom
up. Trends in Cognitive Sciences, 7(11):505–512, November 2003.
4. Marco Casarotti, Matteo Lisi, Carlo Umilta, and Marco Zorzi. Paying attention
through eye movements: A computational investigation of the premotor theory of
spatial attention. J. Cognitive Neuroscience, 24(7):1519–1531, 2012.
5. Andrea Di Ferdinando, Domenico Parisi, and Paolo Bartolomeo. Modeling orient-
ing behavior and its disorders with ecological neural networks. Journal of Cognitive
Neuroscience, 19(6):1033–1049, 2007.
6. Onofrio Gigliotta and Stefano Nolfi. Formation of spatial representations in evolv-
ing autonomous robots. In Proceedings of the 2007 IEEE Symposium on Artificial
Life (CI-ALife 2007), pages 171–178, Piscataway, NJ, 2007. IEEE Press,.
7. Onofrio Gigliotta, Giovanni Pezzulo, and Stefano Nolfi. Evolution of a predic-
tive internal model in an embodied and situated agent. Theory in Biosciences,
130(4):259–276, 2011.
8. Stefano Nolfi and Dario Floreano. Evolutionary Robotics. Mit Press, 2000.
9. Rolf Pfeifer and Josh Bongard. How the body shapes the way we think. The Mit
Press, 2006.
10. Michela Ponticorvo and Orazio Miglino. Encoding geometric and non-geometric
information: a study with evolved agents. Animal Cognition, 13(1):157–174, 2010.
11. Robert F. Port and Timothy Van Gelder. Mind as motion : explorations in the
dynamics of cognition. Bradford Book, Cambridge (Mass.), London, 1995. A
Bradford books.
12. W Vingiano. Pseudoneglect on a cancellation task. International Journal of Neu-
roscience, 58(1-2):63–67, 1991.