=Paper=
{{Paper
|id=Vol-2524/paper22
|storemode=property
|title=Using technology and tangible interfaces in a visuospatial cognition task: the case of the baking tray task
|pdfUrl=https://ceur-ws.org/Vol-2524/paper22.pdf
|volume=Vol-2524
|authors=Chiara Gentile,Antonio Cerrato,Michela Ponticorvo
|dblpUrl=https://dblp.org/rec/conf/psychobit/GentileCP19
}}
==Using technology and tangible interfaces in a visuospatial cognition task: the case of the baking tray task==
https://ceur-ws.org/Vol-2524/paper22.pdf
Using technology and tangible interfaces
in a visuospatial cognition task:
the case of the Baking Tray Task
Chiara Gentile1 , Antonio Cerrato1 , and Michela Ponticorvo1
1
Department of Humanistic Studies,
University of Naples Federico II, Italy
Abstract. The Baking Tray Task (BTT) is a neuropsychological test,
aimed to assess unilateral spatial neglect (USN), a visuospatial disorder
mainly associated to right parietal lobe damage. Over the years, the BTT
has been re-proposed in di↵erent forms, other materials to be placed and
in both digital and virtual environment preserving the initial settings
and the way of administration. In this paper, we present two versions of
BTT, the E-BTT and the BTT-SCAN, improved by technology.
The aim of these tools is to present a new technological version of the
same test in order to preserve a high validity and reliability and to acquire
massive and more precise data.
Keywords: Visuospatial Cognition · Assessment · Neuropsychology ·
Baking Tray Task · Tangible Interfaces
1 Introduction
Spatial cognition is the ability in all species to visually perceive the locations of
stimuli in the space in order to navigate in the surrounding environment [14].
In particular, visuospatial abilities allow to identify visual and spatial relations
among objects in terms of spatial coordinates. Whenever the spatial cognition is
impaired, the execution of daily activities is compromised for the people and an
accurate evaluation and assessment of visuospatial abilities becomes of funda-
mental importance [10]. Neuropsychological assessment is a performance-based
procedure used to assess various cognitive functions such as memory, attention,
reasoning, judgment, problem-solving, visuospatial skills and language. Gener-
ally, neuropsychological assessment (e.g. the Hopkins Verbal Learning Battery or
the Brief Visuospatial Memory Test[23])is performed with a battery approach to
evaluate cognitive ability areas. The major part of neuropsychological tests are
administered in traditional paper-and-pencil tests, particularly long and boring
to perform.
Copyright c 2019 for this paper by its authors. Use permitted under Creative Com-
mons License Attribution 4.0 International (CC BY 4.0).
2 C. Gentile et al.
For this reason, we can use the evaluation of cognitive functions mediated
by technology to overcome some consequences of neuropsychological assessment.
For example, computerized tests can be performed in a shorter time ensuring a
high validity and reliability thanks to precision, standardization and objectivity
of the test[15]. Moreover, the computerized assessment of cognitive functions
can also minimize the ceiling e↵ect (when the performance of the individuals is
maximized in low difficult tasks) and the floor e↵ect (if the task is so difficult
that no one is able to perform it) [27]. In the next section, after a brief description
of a test adopted for the diagnose of spatial impairments, will be described two
versions of the same task that exploit the advantages derived by technology.
2 The Baking Tray Task (BTT)
In neuropsychology, the assessment of spatial cognition can be performed through
plenty of tasks, as described by Cerrato and colleagues [7].One example is rep-
resented by The Baking tray task (BTT), a relatively recent test developed by
Tham and Tegner [26] for assessing Unilateral Spatial Neglect (USN), a visu-
ospatial disorder mainly associated to right parietal lobe damage, in which the
patients show unawareness of the stimuli on the left side of their visual field.
This disorder cannot be considered a sensory (visual, acoustic and / or tactile)
deficit, indeed, these patients ignore what is placed in the controlesional space
due to an inability to orient attention and to explore space [2].
The subjects are instructed to spread, as evenly as possible, 16 cubes over a 75 x
100 cm board, as if they were buns/cookies on a baking tray. The 16 cubes have a
dimension of 3.5 cm and they are placed in a box directly in front of the subject.
There is no time limit to finish the task and each cube have to be dispose over
the board. A not uniform placement of the cubes, with a number of cubes in half
of the board lower than 6 or greater than 10, could be a sign of USN. The BTT
is an ecological, sensitive and easy test and it detects moderately severe Spatial
Neglect compared to common tests for neglect, including the barrage test [1] and
the line bisection test [25]. Furthermore, BTT requires less e↵ort and attention
than cancellation tasks, because the number of distractors and the ability of
the patients to distinguish them from the target may negatively influence the
performance [20]. In addition, the Cancellation Tasks are influenced by practice
because the patients seem to memorize the sequence of steps, while, the Baking
Tray Task is insensitive to practice and set e↵ect, there are no right or wrong
solutions and its execution is complicated to memorize despite exercise [26].
Over the years, the BTT has been re-proposed in di↵erent forms, other materi-
als to be placed and in both digital and virtual environment, while preserving
the initial settings and the way of administration. In recent years, thanks to the
support of technology, we have developed two enhanced versions of the BTT
that are described in the next subsections.
Using technology and tangible interfaces 3
2.1 E-BTT
In 2017 [5], Cerrato and Ponticorvo have realized a version of the Baking Tray
Task, named E-BTT that has been developed following the principles of Gamifi-
cation [3]. The E-BTT is a technology-enhanced version of BTT, reproduced in
a virtual environment by STELT software [16] (Smart Technologies to Enhance
Learning and Teaching) that allows to create prototypes and augmented reality
environments based on Articial Intelligence methodology (Agents Based Mod-
elling) and tangible interfaces (physical objects that can be manipulated). So,
there is a parallel and integrative use between smart technologies and physical
objects, allowing manipulative intervention by the user on the reality and the
interaction between the user and the computer, promoting multisensoriality [18].
It mainly consists in three parts/modules: Storyboarding, is the presentation of
personalized scenarios useful to provide the test instructions to participants. In
the E-BTT, the scenario is the design of a baker, who knead bread. Recording, is
to track all users data interaction and, finally, and Adaptive Tutoring, by which
the user receives on-time intelligent feedbacks from a virtual tutor. An impor-
tant aspect of the tutoring system lies on the possibility to adapt the task on
the user’s level [19, 17]. In order to perform the task, the user had to help Louis,
a cartoon baker, trying to dispose 16 small buns on the tablet surface, as evenly
as possible. The main advantage of this instrument is to diagnose spatial neglect
and other disorders related to visuospatial abilities, stimulating the participation
and involvement of users in the interaction with the computer through STELT
software.
A further development of the E-BTT is represented by the integration of an
Articial Vision module, supported by a camera, able to scan and recognize the
cubes’ disposition. Its functioning will be described in the next subsection.
2.2 BTT-SCAN
The BTT-SCAN is an ecological and technology enhanced tool to assess visual
neglect, developed by Cerrato et colleagues [6, 8].
During the administration of BTT-SCAN, the 16 cubes have to be arranged
on a surface of 48 cm x 34 cm (a tray dimension smaller compared to the one
adopted by Tham and Tegner).
Initially, we adopted 16 cubes of 3.5 cm and di↵erent colors (4 red, 4 or-
ange, 4 green and 4 blue). The cubes are automatically detected by a camera
connected to a PC through ArUco Markers [13], a kind of tags very popular in
augmented reality technology [9, 22, 21], that allow the recognition and the ac-
quisition of their spatial position (as X and Y coordinates). The ArUco Markers
are sticked on the cubes, in order to be detected, and on the corners of the board
to frame the limits of the baking tray. The BTT-SCAN also includes a software
that digitally recreates the cubes’ disposition on the screen of the PC. The in-
structions are the same of original study: subjects are asked to spread out 16
cubes on a board, as if they were buns on a baking tray. At the end of the test,
the BTT-SCAN saves the information related to the subjects performance in an
4 C. Gentile et al.
Fig. 1. A typical configuration at the BTT-SCAN
Excel file, stored in the software database for later review. The instructions, for
exporting data, are written in English on the left; in addition, BTT-SCAN, for
each experimental session, reports the following data: the name and age of the
participants, the date of the session, the start and finish time, the field height
and the field width in pixels, the test duration expressed in seconds, the cubes
on the right, the Left-Right subtraction of the cubes’ disposition, the type of
distribution and the BTT bias (in percentage). This last measure is given by the
formula 100*(right-left)/(right+centre+left) and has been developed by Facchin
[11] in order to calculate the lateralization index showed by participants during
the BTT: if it is negative, the configuration is mainly on the left, if it is positive,
the configuration is mainly on the right, if it is to 0, the configuration is optimal.
BTT-SCAN automatizes the scoring of the performance, produces automatically
and instantaneously the diagnosis and several indexes of patients performance
and helps clinicians in data collection and supervision. These data proved to
be useful for investigating some aspects related to spatial cognition of people,
highlighting, for example, the preferred starting and ending point of the cubes
configuration, and what kind of constructional strategy people adopt, with the
aim to reveal the preferred patterns showed by participants. This enhanced ver-
sion of the BTT presented some instability in collecting the data (for example
due to the processing of the shadows on the surface) and required improvements
in reliability, validity and robustness that are described in the next section.
3 Future directions and Conclusions
The aim of this paper has been to present the evolution of the BTT, from the
original version to the recent technologically advanced versions developed by our
research group, to show the advantages derived by technology. In the E-BTT,
Using technology and tangible interfaces 5
the users are involved in the interaction with the computer through STELT
software, mainly exploiting the digital environment. Instead, the BTT-SCAN,
represents a further enhanced version of the BTT integrating an Artificial Vision
module, supported by a camera, able to scan and recognize the cubes’ disposition
through ArUco Markers. In this manner, the work of clinicians is supported by
the automated diagnosis and the spatial cognition of individuals can be deepened
considering the di↵erent strategies in cubes composition showed by participants.
In spite of the advantages, the E-BTT and BTT-SCAN prototypes there have
been encountered some failures in collecting the data; we will develop the new
BTT enhanced version taking into the account the following suggestion. We will
replace the cubes with new physical objects for three main reasons: Firstly, the
cubes, in their shape, are not similar to bun and moreover we want to avoid
”blocks creations”; secondly, the color of cubes has influenced the configurations
of the participants who sometimes regrouped cubes of the same colour; thirdly,
the thickness of the cubes creates shadows on the surface that compromise the
data collection and the image processing of the artificial vision module.
Fig. 2. Two di↵erent examples of blocks creations
6 C. Gentile et al.
For this reasons, new physical objects will be thin, round and black-and-
white disks, detectable again through the ArUco Makers. In addition, we will
utilize a surface (delimited by a wood frame) with a dimension of 60x45 cm in
which individuals have to the dispose the 16 disks.
Fig. 3. New BTT-SCAN version
The STELT software will be substituted by ETAN, a platform that sup-
ports the detection of tangible user interfaces developed by Cerrato, Ponticorvo,
Gigliotta, Bartolomeo and Miglino [7, 4] to investigate visuospatial behaviors of
people in their proximal/peripersonal space, defined as the space immediately
surrounding our bodies [24].
We will implement BTT with this platform to obtain a more informative
data based on the spatial coordinates (x, y) of the objects and to track their
position. Moreover, it will be possible to store the performances of the subjects
both in a local and in an online database.
The data will be easily exported in a CSV le to access individuals perfor-
mance for further analysis on the spatial skills of the healthy and clinical samples.
ETAN has been developed for diagnostic purposes, but it would be also possible
to implement a rehabilitative module able to adapt the task on the user needs
(considering his level of abilities), starting a training and rehabilitation program
for patients a↵ected by USN and visuospatial impairments, following the princi-
ples of adaptive tutoring systems [19, 12]. Moreover, the implementation of the
BTT with ETAN will be useful to improve stability and reliability compared to
the other versions of the same test and to detect USN and di↵erent cognitive dis-
orders related to visuospatial abilities. In order to benefit to the improvements
mentioned above, it is necessary to administered the new prototype to healthy
and clinical populations and to deep evaluate any weakness. Once collected data
through ETAN, it will be possible develop also a learning analytics module able
to track individuals performances through time and compare them with the rest
Using technology and tangible interfaces 7
Fig. 4. ETAN: the software and the materials to perform BTT
of the population. In this manner we aim to design a new prototype able to better
detect the presence of spatial cognition impairments of individuals and provide
to the clinicians an useful tool able to support them during the diagnostic and
rehabilitation procedure.
References
1. Albert, M.L.: A simple test of visual neglect. Neurology (1973)
2. Bartolomeo, P., Bourgeois, A., Bourlon, C., Migliaccio, R.: Visual and motor men-
tal imagery after brain damage. In: Multisensory Imagery, pp. 249–269. Springer
(2013)
3. Cerrato, A., Ferrara, F., Ponticorvo, M., Sica, L.S., Di Ferdinando, A., Miglino, O.:
Diligo assessment tool: A smart and gamified approach for preschool children as-
sessment. In: International Conference on Smart Education and Smart E-Learning.
pp. 235–244. Springer (2017)
4. Cerrato, A., Pacella, D., Palumbo, F., Beauvais, D., Ponticorvo, M., Miglino, O.,
Bartolomeo, P.: Detecting behavioral patterns for diagnosis and rehabilitation of
visual neglect: a technology-enhanced version of the baking tray task. bioRxiv p.
849505 (2019)
5. Cerrato, A., Ponticorvo, M.: Enhancing neuropsychological testing with gami-
fication and tangible interfaces: The baking tray task. In: International Work-
Conference on the Interplay Between Natural and Artificial Computation. pp.
147–156. Springer (2017)
6. Cerrato, A., Ponticorvo, M., Bartolomeo, P., Miglino, O.: Btt-scan: An ecological
and technology enhanced tool to assess visual neglect. In: COGNITIVE PROCESS-
ING. vol. 19, pp. S36–S36. SPRINGER HEIDELBERG TIERGARTENSTRASSE
17, D-69121 HEIDELBERG, GERMANY (2018)
7. Cerrato, A., Ponticorvo, M., Gigliotta, O., Bartolomeo, P., Miglino, O.: The as-
sessment of visuospatial abilities with tangible interfaces and machine learning. In:
8 C. Gentile et al.
International Work-Conference on the Interplay Between Natural and Artificial
Computation. pp. 78–87. Springer (2019)
8. Cerrato, A., Ponticorvo, M., Gigliotta, O., Bartolomeo, P., Miglino, O.: Btt-scan:
uno strumento per la valutazione della negligenza spaziale unilaterale. Sistemi in-
telligenti 31(2), 253–270 (2019)
9. Cerrato, A., Siano, G., De Marco, A.: Augmented reality: from education and
training applications to assessment procedures. Qwerty-Open and Interdisciplinary
Journal of Technology, Culture and Education 13(1) (2018)
10. Chokron, S., Colliot, P., Bartolomeo, P.: The role of vision in spatial representation.
Cortex 40(2), 281–290 (2004)
11. Facchin, A., Beschin, N., Pisano, A., Reverberi, C.: Normative data for distal line
bisection and baking tray task. Neurological Sciences 37(9), 1531–1536 (2016)
12. Fenza, G., Orciuoli, F., Sampson, D.G.: Building adaptive tutoring model using
artificial neural networks and reinforcement learning. In: 2017 IEEE 17th Inter-
national Conference on Advanced Learning Technologies (ICALT). pp. 460–462.
IEEE (2017)
13. Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F.J., Marı́n-Jiménez, M.J.:
Automatic generation and detection of highly reliable fiducial markers under oc-
clusion. Pattern Recognition 47(6), 2280–2292 (2014)
14. Landau, B., Jackendo↵, R.: What and where in spatial language and spatial cog-
nition. Behavioral and Brain sciences 16(2), 217–238 (1993)
15. Mead, A.D., Drasgow, F.: Equivalence of computerized and paper-and-pencil cog-
nitive ability tests: A meta-analysis. Psychological bulletin 114(3), 449 (1993)
16. Miglino, O., Di Ferdinando, A., Di Fuccio, R., Rega, A., Ricci, C.: Bridging digital
and physical educational games using rfid/nfc technologies. Journal of e-Learning
and Knowledge Society 10(3) (2014)
17. Ponticorvo, M., Di Ferdinando, A., Marocco, D., Miglino, O.: Bio-inspired compu-
tational algorithms in educational and serious games: some examples. In: European
Conference on Technology Enhanced Learning. pp. 636–639. Springer (2016)
18. Ponticorvo, M., Di Fuccio, R., Ferrara, F., Rega, A., Miglino, O.: Multisensory ed-
ucational materials: Five senses to learn. In: International Conference in Method-
ologies and intelligent Systems for Techhnology Enhanced Learning. pp. 45–52.
Springer (2018)
19. Ponticorvo, M., Rega, A., Miglino, O.: Toward tutoring systems inspired by applied
behavioral analysis. In: International Conference on Intelligent Tutoring Systems.
pp. 160–169. Springer (2018)
20. Rapcsak, S.Z., Verfaellie, M., Fleet, S., Heilman, K.M.: Selective attention in hemis-
patial neglect. Archives of Neurology 46(2), 178–182 (1989)
21. Rega, A., Mennitto, A., Vita, S., Iovino, L.: New technologies and autism: Can
augmented reality (ar) increase the motivation in children with autism?
22. Rega, A., Mennitto, A.: Augmented reality as an educational and rehabilitation
support for developmental dyslexia. ICERI2017 Proceedings pp. 6969–6972 (2017)
23. Register-Mihalik, J.K., Kontos, D.L., Guskiewicz, K.M., Mihalik, J.P., Conder, R.,
Shields, E.W.: Age-related di↵erences and reliability on computerized and paper-
and-pencil neurocognitive assessment batteries. Journal of athletic training 47(3),
297–305 (2012)
24. Rizzolatti, G., Fadiga, L., Fogassi, L., Gallese, V.: The space around us. Science
277(5323), 190–191 (1997)
25. Schenkenberg, T., Bradford, D., Ajax, E.: Line bisection and unilateral visual ne-
glect in patients with neurologic impairment. Neurology 30(5), 509–509 (1980)
Using technology and tangible interfaces 9
26. Tham, K.: The baking tray task: a test of spatial neglect. Neuropsychological
Rehabilitation 6(1), 19–26 (1996)
27. Wild, K., Howieson, D., Webbe, F., Seelye, A., Kaye, J.: Status of computerized
cognitive testing in aging: a systematic review. Alzheimer’s & Dementia 4(6), 428–
437 (2008)