=Paper= {{Paper |id=Vol-3657/paper8 |storemode=property |title=Retzzles: Do Jigsaw Puzzle Actions on Interactive Display Maps Increase the Retention of Map Information? |pdfUrl=https://ceur-ws.org/Vol-3657/paper8.pdf |volume=Vol-3657 |authors=Nikola Kovačević,Jordan Aiko Deja,Maheshya Weerasinghe,Klen Čopič Pucihar,Matjaž Kljun |dblpUrl=https://dblp.org/rec/conf/hci-si/KovacevicDWPK23 }} ==Retzzles: Do Jigsaw Puzzle Actions on Interactive Display Maps Increase the Retention of Map Information?== https://ceur-ws.org/Vol-3657/paper8.pdf

Retzzles: Do Jigsaw Puzzle Actions on Interactive
Display Maps Increase the Retention of Map
Information?
Nikola Kovačević1 , Jordan Aiko Deja1,3 , Maheshya Weerasinghe1,4 , Klen Čopič
Pucihar1,2,5 and Matjaž Kljun1,5
1
University of Primorska, Faculty of Mathematics, Natural Sciences and Information Technologies, Koper, Slovenia
2
Faculty of Information Studies, Novo Mesto, Slovenia
3
De La Salle University, Manila, Philippines
4
University of Glasgow, Scotland, United Kingdom
5
Stellenbosch University, Department of Information Science, Stellenbosch, South Africa

Abstract
While maps provide upfront content, this might not always be the most effective way for users to
remember information. With the proliferation of interactive displays for tourists and visitors in public
spaces, we can create a more playful user experience with maps than just exploring them. Adding
interactions with the map could also help users retain more information as they use them. In this
paper, we investigated whether completing a jigsaw puzzle of a map supports users in retaining more
information about a specific map. The results of a between-subject study with a sample of 𝑛 = 28 indicate
that additional interaction helped improve mean scores of textual and spatial recall but not visual recall.
However, the results are not statistically significant, and the topic is subject to further investigation. Our
findings contribute to discussions on using interactive touchscreen displays in similar learning scenarios
involving memory retention.

Keywords
knowledge retention, interactive displays, interactive maps, tourism, gamification

1. Introduction and Background
Throughout history, maps have been essential resources for exploration, navigation, and com-
munication [1]. They are often used in tourism for an overview of points of interest since
they support memory and spatial cognition and display information quickly [2, 3]. However,
traditional paper-based maps frequently fall short in providing further resources as paper real
estate is always limited. Additional resources can be delivered either on extra paper-based
material or in a digital form [4]. For example, looking at a physical town map on the board,
one can explore additional digital information over the phone. Fully digital maps provide

HCI SI 2023: Human-Computer Interaction Slovenia 2023, January 26, 2024, Maribor, Slovenia
Envelope-Open kovacevicnikola@proton.me (N. Kovačević); jordan.deja@famnit.upr.si (J. A. Deja);
maheshya.weerasinghe@famnit.upr.si (M. Weerasinghe); klen.copic@famnit.upr.si (K. Čopič Pucihar);
matjaz.kljun@upr.si (M. Kljun)
Orcid 0000-0002-7913-3839 (N. Kovačević); 0000-0001-9341-6088 (J. A. Deja); 0000-0003-2691-601X (M. Weerasinghe);
0000-0002-7784-1356 (K. Čopič Pucihar); 0000-0002-6988-3046 (M. Kljun)
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)

CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
the possibility of interactivity and unlimited further resources, enabling more dynamic user
interaction with the content. Another advantage of digital maps is the possibility of showing
additional elements in the map’s context, which paper-based maps cannot achieve. It has been
observed that showing annotations linked to a target object’s actual location (that is, right on
the map) improves users’ memory skills more than showing annotations related to an unrelated
location (that is, off the map and connected to their location through other means) [5].
Digital maps are often provided on interactive displays that are common in tourist information
offices, museums, shopping malls, train stations, and even streets [6]. These displays open up
further possibilities for making interaction with maps playful and, in turn, help people remember
more information [7, 8, 9, 10, 11, 12]. Playing activity enables the notion of “learning by doing”
or experiential learning [13]. There are various ways to make map interaction playful—one
possibility is to use jigsaw puzzle pieces [14, 15]. Puzzles have been observed to help make
learning enjoyable [16, 17]. Specifically, the use of jigsaw elements, one of the most common
puzzle games, has been observed to help students acquire a deeper understanding of certain
concepts and terminologies [18] in the broader context of learning.
In this paper, we explored whether engaging with maps through playful elements such as
jigsaw puzzles can improve engagement and retention of information. We hypothesise that
users playing with and completing a jigsaw puzzle of a map will retain more information about it
compared to users who will start interacting with the complete map. In summary, this research
presents the following contributions: a) an improvement of Artifact prototype Retzzles [19]
that is presented on an interactive display, and b) Empirical findings of a user study comparing
visual, spatial and textual recall between two conditions – map interaction vs. jigsaw puzzle
interaction.

2. Retzzles: Design and Implementation
This section discusses the design and concept behind our prototype entitled Retzzles. We
designed our interaction following the theory of constructivism, according to which individuals
actively construct knowledge and create their experiences [20]. While the end goal of the puzzle
is the same for every user, the path toward achieving a solution will be unique for each user. As
the puzzle is being built, the user is piecing together the knowledge as long as they play the
game. With Retzzles, users complete a jigsaw of a tourist destination map individually; however,
the final map is the same for everyone.
Retzzles is shown on an interactive touch display where users can move the puzzle pieces
and put them in the correct locations as seen in Figure 1. We used Unity to build this prototype.
Jigsaw puzzles were created using GIMP and its pattern feature “jigsaw”. The map consisted of
20 jigsaw pieces. We opted for this number as it was shown to be not difficult and also not easy
to complete the puzzle in the pilot study. The jigsaw puzzles are all organised into one empty
parent game object. Each piece is assigned with the DragObject script, allowing it to be moved
and dragged anywhere on the screen. This script defines how the puzzle pieces snap into the
correct position. Because our pieces are treated as sprites in Unity, we used PolygonCollider2D
to handle the physical collisions. The collider’s shape is defined by a freeform edge made of
line segments, which is easily adjustable to cover any shape, in our case, any jigsaw puzzle.
Figure 1: User completing a jigsaw puzzle of a map on the interactive display. When the jigsaw is
completed the POIs on the map become interactive.

When each jigsaw piece is dragged over the correct location and dropped, it snaps automati-
cally in its spot. As the user solves the puzzle, POIs become revealed when the suitable puzzles
are pieced together. After completing the jigsaw, the POIs buttons become interactive. Tapping
on them reveals additional information about the touristic location they are placed at on the
map (Figure 3 bottom right). For this study, we put ten abstract symbols on the map as POIs, as
done in [5]. The two reasons behind this decision were to avoid the familiarity with symbols
usually used on maps and to not correlate the symbols with the information they revealed when
tapped on. The symbols were created using the online graphic design platform Canva1 . The
map was created using the Snazzy Maps2 and the “No label, Bright Colors” layout as it provided
a map with fewer distractions. For the map, we used the city of Lancaster, California, as users
were likely not familiar with its layout, as seen in Figure 2 left. For the same reason, we used 10
POIs users were likely unfamiliar with.
We have added some automated mechanisms to log data within the prototype for more
efficient data collection. We tracked the number of moves/clicks for each puzzle piece, activated
POIs, calculated the time needed to solve the puzzle, and opened and read the POIs’ information.
We also recorded and saved every movement of each piece.

1
https://www.canva.com/
2
https://snazzymaps.com/
Figure 2: The map and interact-able Points of Interest (POI) of the Retzzles: Oxford Suites Hotel,
Alderbrook Nursing Facility, Red Salmon Restaurant, U-Haul Moving Supplies, Sun Flower Caffe, Target
Grocery Store, Bowlero bowling, Sunnydale School, The Tire Store, Costco Grocery Store and Sunnydale
School.

Figure 3: Different conditions and stages of Retzzles. Top Left: the initial stage of the puzzle condition.
Top right: One piece left; when it will be placed in the map will become interactive as in bottom left.
Bottom Left: the initial stage of the map-only condition. Bottom Right: The information card opened
when tapped on the corresponding POI.

3. User Study
For this study, we proposed the following hypothesis: 𝐻0 : Puzzle interactions aid in improved
visual, spatial and textual recall of map information.
We conducted a between-subject study design to validate our hypothesis with 𝑛 = 28
Table 1
Overview of NASA-TLX Cognitive Scores
cond MD PD TD P E F
puzzle t36.1 t8.21 s24.6 t28.6 t37.5 t12.1
mean
map 49.3 8.57 19.3 29.6 45.0 14.3
puzzle 26.3 05.04 22.5 16.0 25.9 6.71
std dev
map 19.2 06.02 16.0 18.0 24.3 14.9
puzzle 0.899 0.697 0.792 0.906 0.850 0.879
Shapiro-Wilk 𝑊
map 0.958 0.672 0.785 0.915 0.962 0.697
puzzle 0.109 <.001 0.004 0.139 0.022 0.057
Shapiro-Wilk 𝑝
map 0.686 <.001 0.003 0.185 0.750 <.001
Mann-Whitney 𝑈 𝑝 **0.122 **0.758 **0.871 **0.309
𝐻𝑎 𝜇𝑝𝑢𝑧𝑧𝑙𝑒 ≠ 𝜇𝑚𝑎𝑝 ,**no sign. diff.

Legend: Cond : Condition, MD: Mental Demand, PD: Physical Demand, D: Temporal Demand, P : Performance, E:
Effort, F : Frustration

participants recruited through convenience sampling. We recruited participants who either
have experience with 2D games or have prior experience in solving puzzles. For the study, an
unfamiliar tourist place, such as the city of Lancaster, was used as the subject of the map in the
prototype across both conditions. The study involved three phases: 1) Orientation and Informed
Consent, 2) Interaction Task and 3) Wrapping phase. In 1), the moderator of the study provided
a brief introduction to the experiment and the prototype Retzzles. Next, the participants were
invited to read and sign the consent form. At the beginning of 2), the moderator explained
the task in more detail. We used two conditions, namely (1) Map condition (Figure 3 bottom
left) and (2) Puzzle condition (Figure 3) top left). Symbols and information used between both
conditions were the same. However, the interaction amount differed as the latter condition
required puzzle assembly before unlocking the touch-screen map interactions. After interacting
with the prototype, participants answered different questionnaires to measure visual, spatial
and textual recall.
The aim in both conditions was to memorise as much information as possible after engaging
with the prototype. The participants were first given a module that instructed them on utilising
the prototype. When they were ready, they proceeded to the actual task based on the conditions
set for them. Participants in both conditions were instructed to interact with the prototype while
remembering as much information as possible. Interaction for the Map condition consisted of
engaging with a map as the POIs were already interactive. Interaction with the Puzzle condition
required participants to solve the jigsaw puzzle of a map before being able to interact with the
POIs and review the information provided.
Table 2
Overview of Metrics Used. Green arrow meaning better performance compared to the map condition,
red arrow meaning worse performance. Legend: Cond : Condition, VR: Visual Recall, NS-VR: Negative
Symbol Recall, SR: Spatial Recall, N-TR: Name Textual Recall, C-TR: Category Textual Recall
cond VR NS-VR SR N-TR C-TR
puzzle t75.7 t7.14 s68.6 s50.7 s68.6
mean
map 80.7 6.43 60.7 47.1 57.9
puzzle 18.7 8.25 26.3 25.3 11.7
std dev
map 18.6 11.5 29.2 17.7 21.9
puzzle 0.915 0.767 0.880 0.956 0.936
Shapiro-Wilk 𝑊
map 0.876 0.638 0.882 0.948 0.928
puzzle 0.184 0.002 0.059 0.661 0.370
Shapiro-Wilk 𝑝
map 0.051 <.001 0.062 0.525 0.289
Mann-Whitney 𝑈 𝑝 **0.452 **0.328 **0.693 **0.116
𝐻𝑎 𝜇𝑝𝑢𝑧𝑧𝑙𝑒 ≠ 𝜇𝑚𝑎𝑝 ,**no sign. diff.

4. Results and Discussion
4.1. Information Recall
We evaluated visual, spatial, and textual recall and measured participants’ subjective cognitive
load using the NASA-TLX questionnaire. To test visual recall, participants were presented with
20 abstract symbols (10 symbols were present on the map while the others were not) and asked
to identify which 10 were on the map. The results are in the percentage of correct symbols
recalled (Visual Recall [VR]) or the percentage of incorrect symbols recalled (Negative Symbol
Recall [NS-VR]). Spatial recall was assessed by asking users to remember the positions of the
abstract symbols on the map and to place them in their correct locations on a separate blank
map [SR]. Finally, for textual recall, users were shown ten symbols on the map and asked to
provide information about each POI, including its name (Name Textual Recall [N-TR]) and
category (Category Textual Recall [N-TR). For example, they might provide the name “Costco”
and the category “grocery store”.
The main goal of the analysis was to compare map and puzzle conditions. We used the
Shapiro-Wilk test to explore the normality of data distribution. We found that the data is
normally distributed (𝑝 ≥ 0.05) for all but the visual recall. We decided to run all statistical tests
using the Mann-Whitney U test, a form of independent samples t-test used when the normal
distribution requirement is unmet.
Based on the participants’ mean scores alone (in percentages), the puzzle condition performed
better in all metrics except for the visual recall (𝑆𝑅 ∶ 68.6, 𝑁 − 𝑇 𝑅 ∶ 50.7, 𝐶 − 𝑇 𝑅 ∶ 68.6) Table 2.
Likewise, for the cognitive load, we can see that the puzzle condition was less mentally demanded
than the map-only condition (𝑀𝐷 ∶ 36.1, see Table 1). We conducted the Mann-Whitney U test,
a form of independent samples t-test, to further test the significance of the different metrics.
However, based on our sample size of 28 participants, we found no significant difference
(𝑝 ≥ 0.05) across all variables. Thus, we rejected our null hypothesis despite better results of
the puzzle condition for the spatial and textual recall.
Our sample size may not have been large enough to reach a more conclusive result based on
the participants’ mean scores in the puzzle condition, which were higher for spatial and textual
memory than the map condition. Other factors may have contributed to the effects on visual
recall as well. One is that users in the Puzzle condition were using the prototype longer than
users in the map condition, which was expected. For example, several participants mentioned
associating symbols with the colours used in the POIs, which our study did not consider a
variable.

4.2. Time and Interaction
As participants used the system, an internal logging mechanism took note of the usage times
with the prototype (from opening the scene until solving the puzzle). From here, we collected
information such as how much time they interacted with the prototype overall and per puzzle
piece. The overall duration from the start of the interaction to completing the condition varied
greatly. The shortest duration recorded in the puzzle condition was 157 seconds, while the
longest was 778 seconds, with an average of 329.21 seconds. In the map condition, the shortest
time was 110 seconds, the longest was 622 seconds, and the average time was 316.35 seconds.
Furthermore, the quantity of POI clicks varied, providing a better sense of participant engage-
ment. In the puzzle condition, the lowest number of POI clicks was 21, the maximum number
was 82, and the average was 41.31. In the map condition, the lowest number of clicks was 20,
the maximum number was 72, and the average was 35.5.

4.3. Puzzle Specific Findings
This section reports on participants’ puzzle-solving behaviour. To better understand the follow-
ing findings a clear view of starting and ending positions of puzzle is needed, which is visible in
Figure 4.

• Patterns in piece completion: We noticed that certain parts were consistently completed
by the majority of participants early on, indicating their attractiveness or perceived impor-
tance in the puzzle-solving process. These were the pieces: j23 (4 out of 14 participants),
j25 (4 out of 14 participants), and j1 (3 out of 14 participants). The parts that took the
least time to complete were j21 (24 seconds), j25 (24 seconds), j10 (26 seconds), and j1
(30 seconds). The pieces with the fewest pickups were j1, j2, j5, j10, j15, j23 and j25,
which all have 14 pickups. This indicates that each participant chose them only once
when solving the puzzle.
• Varied completion times: The fastest completion time was 98 seconds. The slowest
completion time was 255 seconds. This tells us that participants exhibited different levels
of efficiency when solving puzzles. On the other hand, the average completion time was:
133 seconds.
• Troublesome pieces: Certain pieces appeared to cause difficulty for multiple participants,
as evidenced by longer completion times or frequent backtracking. These pieces may
warrant further investigation for potential design improvements. The most often com-
pleted parts were j12 (4 out of 14 participants) and j9 (3 out of 14 participants). These
(a) The starting positions of the puzzle pieces.

(b) The ending positions of the puzzle pieces.

Figure 4: Look at puzzle positions in the prototype Retzzles.

are the centre parts and have no POI components on them. The sections that took the
longest to complete were: j14 (99 seconds), j4 (81 seconds), j19 (77 seconds), and j20
(74 seconds). Here we have border pieces j4 and j20, as well as middle parts j14 and
j19. All of them have POI elements on them. We can gain more information into this by
examining how many chunk instances of parts exist across all participants. The top three
sections in terms of the number of pickups are: j14 (24 chunks), j17 (23 pickups), and j18
(22 pickups).
• Participant-specific strategies: Each participant demonstrated distinct puzzle-solving
approaches, emphasising individuality and creativity in problem-solving behaviours.
Each participant took a different approach to solving the puzzle.
5. Conclusion and Future Work
In this paper, we explored whether engaging with maps in the form of jigsaw puzzle pieces can
help users remember more information from the map. We present our prototype, Retzzles, used
to test this. In one condition, users first had to solve jigsaw puzzles on a map before being able
to interact with points of interest displaying essential information. In another condition, the
interactive map was already present. Results of our between-subject study, involving 𝑛 = 28
participants, showed better mean scores for spatial and textual recall; however, no significant
difference between these conditions was found. Our initial analysis also found no observable
differences in the time spent interacting with the puzzle. Despite this, the puzzle condition has
shown a slight advantage. While this might be simply because users spent more time with the
puzzle, which they found playful, it adds to the knowledge that active learning has advantages.

Acknowledgments
This research was funded by the Slovenian Research Agency, grant number P1-0383, P5-0433,
IO-0035, J5-50155 and J7-50096. This work has also been supported by the research program
CogniCom (0013103) at the University of Primorska.

References
[1] M. J. Farrell, P. Arnold, S. Pettifer, J. Adams, T. Graham, M. MacManamon, Transfer of
route learning from virtual to real environments., Journal of Experimental Psychology:
Applied 9 (2003) 219.
[2] K. Jancewicz, D. Borowicz, Tourist maps-definition, types and contents, Polish Carto-
graphical Review 49 (2017).
[3] T. Rossetto, Embodying the map: Tourism practices in berlin, Tourist Studies 12 (2012)
28–51.
[4] J. Nakazawa, H. Tokuda, Phygital map: Accessing digital multimedia from physical map,
in: 21st international conference on advanced information networking and applications
workshops (AINAW’07), volume 2, IEEE, 2007, pp. 368–373.
[5] Y. Fujimoto, G. Yamamoto, H. Kato, J. Miyazaki, Relation between location of information
displayed by augmented reality and user’s memorization, in: Proceedings of the 3rd
augmented human international conference, 2012, pp. 1–8.
[6] V. Mäkelä, V. Gentile, M. Khamis, S. Sorce, Supporting tourism with public interactive
displays, in: Proceedings of the 8th ACM International Symposium on Pervasive Displays,
2019, pp. 1–2.
[7] R. Ballagas, A. Kuntze, S. P. Walz, Gaming tourism: Lessons from evaluating rexplorer, a
pervasive game for tourists, in: International conference on pervasive computing, Springer,
Springer, Sydney, Australia, 2008, pp. 244–261.
[8] M. M. Zarzuela, F. J. D. Pernas, S. M. Calzón, D. G. Ortega, M. A. Rodríguez, Educational
tourism through a virtual reality platform, Procedia Computer Science 25 (2013) 382–388.
[9] B. Rooney, E. Hennessy, Actually in the cinema: a field study comparing real 3d and
2d movie patrons’ attention, emotion, and film satisfaction, Media Psychology 16 (2013)
441–460.
[10] G. Sotlar, P. Rogelj, K. Čopič Pucihar, M. Kljun, Predmetnik: oprijemljiv uporabniški
vmesnik za informiranje turistov, Ph.D. thesis, Univerza na Primorskem, Inštitut Andrej
Marušič, 2020.
[11] G. Nodalo, J. M. Santiago, J. Valenzuela, J. A. Deja, On building design guidelines for an
interactive machine learning sandbox application, in: Proceedings of the 5th International
ACM In-Cooperation HCI and UX Conference, CHIuXiD’19, Association for Computing
Machinery, New York, NY, USA, 2019, p. 70–77. URL: https://doi.org/10.1145/3328243.
3328253. doi:10.1145/3328243.3328253 .
[12] J. M. S. III, G. Nodalo, J. Valenzuela, J. A. Deja, Explore, edit, guess: Understanding novice
programmers’ use of codeblocks for regression experiments, in: Proceedings of the 6th
HCI Slovenia Conference 2021, CEUR Workshop Proceedings, Koper, Slovenia, 2021, pp.
3–17.
[13] M. Weerasinghe, A. Quigley, J. Ducasse, K. Čopič Pucihar, M. Kljun, Educational augmented
reality games, Augmented reality games II: The gamification of education, medicine and
art (2019) 3–32.
[14] A. Cöltekin, I.-E. Lokka, A. Boér, The utilization of publicly available map types by non-
experts—a choice experiment, in: Proceedings of the 27th International Cartographic
Conference (ICC2015), Rio de Janeiro, Brazil, ICC, Rio de Janeiro, Brazil, 2015, pp. 23–28.
[15] K. N. Matthews, The Impact of the Jigsaw Technique on Improving Knowledge Retention,
Ph.D. thesis, University of South Alabama, 2021.
[16] A. S. Keshta, F. K. Al-Faleet, The effectiveness of using puzzles in developing palestinian
tenth graders’ vocabulary achievement and retention, Humanities and Social Sciences 1
(2013) 46–57.
[17] C. Linehan, G. Bellord, B. Kirman, Z. H. Morford, B. Roche, Learning curves: analysing pace
and challenge in four successful puzzle games, in: Proceedings of the first ACM SIGCHI
annual symposium on Computer-human interaction in play, Association for Computing
Machinery, New York, NY, USA, 2014, pp. 181–190.
[18] F. Khalvandi, M. Bazrafshan, Effect of the cooperative learning on studentsâ educational
performance and attitude on the basis of jigsaw puzzle pattern, Research in Curriculum
Planning 13 (2016) 113–125.
[19] N. Kovacevic, M. Weerasinghe, J. A. Deja, Retzzles: Engaging users towards retention
through touchscreen puzzles, in: Companion Proceedings of the 2022 Conference on
Interactive Surfaces and Spaces, Association for Computing Machinery, New York, NY,
USA, 2022, pp. 14–17.
[20] C. T. Fosnot, Constructivism: Theory, perspectives, and practice, Teachers College Press,
New York, NY, USA, 2013.