=Paper=
{{Paper
|id=Vol-2091/paper9
|storemode=property
|title=Visualization of Cultural-Heritage Content based on Individual Cognitive Differences
|pdfUrl=https://ceur-ws.org/Vol-2091/paper9.pdf
|volume=Vol-2091
|authors=George E. Raptis,Christina Katsini,Christos Fidas,Nikolaos Avouris
|dblpUrl=https://dblp.org/rec/conf/avi/RaptisKFA18
}}
==Visualization of Cultural-Heritage Content based on Individual Cognitive Differences==
https://ceur-ws.org/Vol-2091/paper9.pdf
Visualization of Cultural-Heritage Content based on Individual
Cognitive Differences
George E. Raptis Christina Katsini Christos Fidas Nikolaos Avouris
Human Opsis and HCI Human Opsis and HCI Dept. of Cultural Heritage HCI Group, Dept. Electrical
Group, Dept. of Electrical Group, Dept. of Electrical Management and New and Computer Engineering,
and Computer Engineering, and Computer Engineering, Technologies, University of University of Patras
University of Patras University of Patras Patras Patras, Greece
Patras, Grece Patras, Greece Patras, Greece avouris@upatras.gr
raptisg@upnet.gr katsinic@upnet.gr fidas@upatras.gr
ABSTRACT provide personalized cultural-heritage experiences to the end-users
Comprehension of visual content is linked with the visitors’ expe- (e.g., museum visitors). When designing such systems, several user-
rience within cultural heritage contexts. Considering the diversity specific and context-specific aspects [2] must be considered to
of visitors towards human cognition and the influence of individ- provide the most appropriate content in the most suitable way to
ual cognitive differences on information comprehension, current the end-users, aiming to assist them to have a more efficient and
visualization techniques could lead to unbalances regarding visi- effective comprehension of the cultural-heritage content. With re-
tors’ learning and experience gains. In this paper, we investigate gards to the user-specific aspects, the information system designers
whether the visualization of cultural-heritage content, tailored to must comply with the diversity of individuals who have different
the visitors’ individual cognitive characteristics, would improve characteristics such as personality traits [1], goals [2], and visiting
the comprehension of the cultural-heritage content. We followed a styles [5]. An aspect, which is not being considered as an important
two-step experimental approach, and we conducted two small-scale design factor by the current practices, is the human cognition, al-
between-subject eye-tracking studies (exploratory and comparative though several researchers have confirmed existing effects towards
study), in which people with different cognitive style participated content comprehension in diverse application domains, such as
in a gallery tour. The analysis of the results of the exploratory study usable security [9], gaming [20], and e-learning [28].
revealed that people with different cognitive style, differ in the Given that cultural-heritage activities often include visual con-
way they process visual information, which influences the content tent comprehension tasks (e.g., viewing a painting in an art mu-
comprehension. Based on these results we developed cognitive- seum), human cognitive characteristics related to the comprehen-
centered visualizations and we performed a comparative study, sion of visual information would be of great interest as a person-
which revealed that such visualizations could help the users to- alization factor within a cultural-heritage context. The cognitive
wards content comprehension. In this respect, individual cognitive style Visualizer-Verbalizer (V-V) is such a cognitive characteristic.
differences could be used as the basis for providing personalized ex- According to the V-V theory [16], information is processed and
periences to cultural-heritage visitors, aiming to help them towards mentally represented in two ways: verbally and visually. Hence,
content-comprehension. the individuals are distinguished to those who think either more in
pictures (visualizers) or more in words (verbalizers) [12]. Research
CCS CONCEPTS has shown that V-V influences learning and content comprehension
[10, 12] and that it is associated with visual behavior [10, 13, 28].
• Human-centered computing → Empirical studies in HCI;
Despite that there is an extensive body of research which un-
Visualization; HCI theory, concepts and models; • Computing
derpins that V-V affects users’ comprehension of visual content,
methodologies → Cognitive science;
current design approaches do not leverage on these findings and do
ACM Reference Format: not consider V-V as an important factor when designing cultural-
George E. Raptis, Christina Katsini, Christos Fidas, and Nikolaos Avouris.
heritage activities. This can be accredited to the fact that there is a
2018. Visualization of Cultural-Heritage Content based on Individual Cog-
lack in understanding the interplay among visual behavior, cultural-
nitive Differences. In Proceedings of 2nd Workshop on Advanced Visual Inter-
faces for Cultural Heritage (AVI-CH 2018). Vol. 2091. CEUR-WS.org, Article heritage activities, and human cognition factor, which have not
9. http://ceur-ws.org/Vol-2091/paper9.pdf, 7 pages. been investigated in depth. Hence, this results to an insufficient
understanding on whether and how to consider such human cog-
1 INTRODUCTION nitive factors practically within current state-of-the-art design ap-
proaches. Therefore, the research question that this paper discusses
Over the last years, cultural heritage has been a favored domain for
is whether V-V affects users’ content comprehension when per-
personalization research [2]. Stakeholders from interdisciplinary
forming a typical cultural-heritage activity, and if so, whether there
fields (e.g., computer science, user modeling, heritage sciences)
are specific visualization types, based on users’ V-V cognitive style,
have collaborated to develop adaptive information systems that
that can be used to help users towards a deeper understanding of
the visual cultural-heritage content.
AVI-CH 2018, May 29, 2018, Castiglione della Pescaia, Italy
© 2018 Copyright held by the owner/author(s).
1
AVI-CH 2018, May 29, 2018, Castiglione della Pescaia, Italy G. E. Raptis et al.
Figure 1: The paintings used in our study (from left to right): Child with rabbits (1879) by Polychronis Lembesis, Café "Neon"
at night (1965) by Yiannis Tsarouchis, The Sphinx in Cairo (n/a) by Pericles Cirigotis, In surgery (n/a) by Georgios Roilos, and
The dirge in Psara (1888) by Nikephoros Lytras.
2 STUDIES AND RESULTS Regarding the eye-tracking metrics, we focused on fixations on
To answer the research question, we followed a two-step between- the areas of interest (AOIs), following common practice [21]. Given
subject experimental approach. In the first step, we performed that each painting was accompanied with a textual description,
an exploratory study, investigating whether and how the visual two different types of AOI are identified: pictorial and textual AOIs.
behavior of individuals who have different V-V cognitive style in- For each type, we measured the: number of fixations in each AOI,
fluenced the comprehension of the cultural-heritage content. In fixation duration in each AOI, entry time in each AOI, number
the second step, based on the results of the exploratory study, we of transitions among AOIs, and fixation ratio. For each metric,
created cognitive-specific visualizations, and performed a compara- we considered the computed measures: sums, means, max, min.
tive study, aiming to evaluate the effects of the cognitive-specific To capture the participants’ eye-gaze behavior we used Tobii Pro
visualizations. Glasses 2 at 50Hz.
2.1.4 Participants. 23 adult individuals (10 females and 13 males),
2.1 Exploratory Study ranging in age between 18 and 33 years old (m = 23.3, sd = 4.9),
2.1.1 Hypotheses. To answer the first part of the research ques- took part in the study. According to VVLSR and VVQ, 12 partici-
tion, we formed the following null hypotheses: pants were classified as visualizers and 11 participants were classi-
fied as verbalizers.
H01 There is no difference between visualizers and verbalizers
2.1.5 Procedure. We recruited 23 study participants, using vary-
regarding the content comprehension.
ing methods (e.g., personal contacts, social media announcements).
H02 Visual behavior of visualizers and verbalizers is not associ-
The participants had to meet a set of minimum requirements: have
ated with the content comprehension.
never taken VVQ and VVLSR tests before; be older than 18 years;
2.1.2 Cultural heritage activity. Considering that browsing vir- know nothing about the paintings used in the study; have little
tual collections and galleries is a popular way for delivering cultural- knowledge of art history and theory. All participants were informed
heritage content [26, 30], we developed a web-based virtual-tour about the study and signed a consent form. For each participant,
application with five paintings of the National Gallery of Greece: a) we scheduled a single virtual exhibition tour of the study paintings.
Child with rabbits by Polychronis Lembesis, b) Café Neon at night by Each virtual took took place in our lab at a mutually agreed date
Yiannis Tsarouchis, c) The Sphinx in Cairo by Pericles Cirigotis, d) In and time. Before entering the tour, the participant completed the
surgery by Georgios Roilos, and e) The dirge in Psara by Nikephoros VVQ and VVLSR tests (20 minutes). Next, she/he navigated through
Lytras. The paintings are depicted in Figure 1. Each painting was the scene (20 minutes) and viewed all the paintings (no view-order
accompanied with a textual description, and thus, each painting restrictions). Then, she/he distracted with a playful activity (30
had two types of content: pictorial and textual. minutes), which was not relevant to the virtual tour. Finally, she/he
filled a form about demographics informations and answered the
2.1.3 Instruments and metrics. To classify the participants as VCC questionnaire (15 minutes).
either visualizers or verbalizers, we used a version of the Verbal- 2.1.6 Results. To investigate H01 , we performed a Mann-Whitney
Visual Learning Style Rating questionnaire (VVLSR) [12] and the U Test. The test met the required assumptions, as the distributions
Verbalizer-Visualizer Questionnaire (VVQ) [24]. Both tests have of the correct answers (i.e., VCC score) for both visualizers and ver-
been widely used in similar studies in varying contexts, such as balizers were similar, as assessed by visual inspection. Median score
e-learning [10] and comprehension of multimedia material [11] for visualizers and verbalizers was not statistically significantly dif-
To measure the visual-content comprehension (VCC), we de- ferent (Table 1). However, the analysis regarding the comprehension
signed a post-test VCC questionnaire. It consisted of ten multiple- on each type of content (i.e., VCCpic for pictorial-content compre-
choice questions (two questions for each painting: one about the hension and VCCt ex t for textual-content comprehension) revealed
pictorial content and one about the textual content), with high significant differences. In particular, visualizers had a significantly
reliability (.738) according to Kuder-Richardson-20 Test. None of better VCCpic (U = 32.000, z = −2.217, p = .027), while visualizers
the participants had seen the paintings before, thus, they had no had a significantly better VCCt ex t (U = 33.500, z = −2.287, p =
prior knowledge about their content. .022).
2
Cognition-based Cultural-Heritage Visualizations AVI-CH 2018, May 29, 2018, Castiglione della Pescaia, Italy
Table 1: Statistical analysis on content comprehension where both pictorial and textual AOIs are important to the visitor.
Therefore, we cannot exclude one type or another, but we need to
VCCtot al direct users’ attention to the AOI type that they do not inherently
prefer. In particular, we need to direct the visualizers’ attention to
Cognitive dimension N Median Mean Std.
textual AOIs and the verbalizers’ attention to pictorial AOIs.
Visualizer 12 6.000 6.082 .900
To help visualizers pay more attention on the textual AOIs and
Verbalizer 11 6.000 6.093 1.578
increase textual-content comprehension, we adopted a popular tech-
Mann-Whitney U Test U = 60.500, z = −.358, p = .721 nique found in the literature: emphasize specific key-words, that are
critical for a better comprehension [4]. Hence, the textual AOIs can
VCCpic be visualized in two ways: the default way, which is recommended
Cognitive dimension N Median Mean Std. for verbalizers, and the emphasizing way, which is recommended
Visualizer 12 3.500 3.582 .669 for visualizers. To help verbalizers pay more attention on the picto-
Verbalizer 11 3.000 2.820 .982 rial AOIs and increase pictorial-content comprehension, we applied
a saliency filter to the pictorial AOIs, which is a typical technique
Mann-Whitney U Test U = 32.000, z = −2.217, p = .027 to attract attention to specific areas of pictures [9]. Hence, the pic-
torial AOIs can be visualized in two ways: the default way, which
VCCt ex t is recommended for visualizers, and the salient way, which is rec-
Cognitive dimension N Median Mean Std. ommended for verbalizers. The simple dichotomous algorithm (in
Visualizer 12 3.000 2.521 .674 pseudo-code) to define the visualization of each painting is:
Verbalizer 11 3.000 3.272 .786
Mann-Whitney U Test U = 33.500, z = −2.287, p = .022 Algorithm 1 Simple dichotomous algorithm to set the visualization
of an AOI based on the user’s V-V cognitive dimension
1: procedure SetCognitionBasedVisualization
To investigate H02 we performed a series of Spearman’s cor- 2: if user is visualizer then
relation test between the visual behavior metrics and VCC. The 3: Set AOI → text → vis to "emphasis"
results revealed several low and moderate correlations, and a strong 4: Set AOI → pic → vis to "default"
positive correlation (r s = .883, p < .001) between VCC and the ratio 5: else
of fixation duration on pictorial and textual AOIs (Equation 1). 6: Set AOI → text → vis to "default"
7: Set AOI → pic → vis to "salient"
Fixation.durationpict or ial .aois
V Bdur −r at io = (1)
Fixation.durationt ex tual .aois
To further investigate the effect of V-V cognitive style on the
visual behavior of the users, we performed an independent-samples 2.3 Comparative study
t-test to determine whether there are differences in VBdur −r at io To investigate whether the cognition-based visualization would as-
between visualizers and verbalizers. The test met all the required sist visualizers and verbalizers to comprehend better the paintings’
assumptions. The VBdur −r at io was higher for visualizers (m = content, we conducted a between-subject comparative study.
1.890, sd = .775) than verbalizers (m = 1.238, sd = .299), sta-
tistically significant difference of (p = .017, t(21) = 2.619, d = 2.3.1 Hypotheses. To answer the second part of the research
1.110, 95%CI : [.135, .172]). The results underpin that visualizers question, we formed the following null hypotheses:
tend to perform longer fixations on the pictorial AOIs, while the H03 Cognition-based visualization does not affect significantly
verbalizers tend to perform longer fixations on the textual AOIs. the visual behavior of visualizers and verbalizers.
H04 Cognition-based visualization does not affect significantly
2.2 Visualization the comprehension of visualizers and verbalizers regarding
The results underpin the necessity of providing customized visual- paintings’ content.
izations for both visualizers and verbalizers, in order to help them
comprehend better the content of the paintings. Considering that 2.3.2 Cultural heritage activity. The activity was the same with
visualizers have an inherent preference for pictorial content, while the one discussed in the exploratory study. However, the cognition-
verbalizers have an inherent preference for textual content, we based visualization was applied for each painting, depending on
propose a cognition-based visualization that aims to trigger the vi- the V-V cognitive dimension of the user.
sualizers’ attention to textual AOIs and the verbalizers’ attention to 2.3.3 Instrument and metrics. They were identical with the in-
pictorial AOIs. Through the cognition-based visualization we expect struments and metrics that were used in the exploratory study.
visualizers to comprehend better the textual content and verbalizers
to comprehend better the pictorial content of the paintings. 2.3.4 Participants. We recruited 20 adult individuals (8 females,
A common approach to make an individual with specific cogni- 12 males) ranging in age between 20 and 31 years old (m = 25.3, sd =
tive characteristics to focus on specific AOIs is to exclude the other 3.8). According to VVLSR and VVQ, 10 participants were classified
AOIs [9]. However, this cannot be applied in a virtual gallery-tour, as visualizers and 10 participants were classified as verbalizers.
3
AVI-CH 2018, May 29, 2018, Castiglione della Pescaia, Italy G. E. Raptis et al.
Figure 2: The cognition-based visualization type helped the Figure 3: The cognition-based visualization type helped
visualizers increase their fixation duration on the textual mainly verbalizers to perform better in pictorial-content
AOIs. questions.
2.3.5 Procedure. We followed the same study procedure with
the exploratory study.
2.3.6 Results. To investigate H03 , we performed a two-way
ANOVA with V-V cognitive dimension and the type of the visu-
alization as the independent variables, and the VBdur −r at io as
the dependent variable. The test met all the required assumptions.
The results revealed a significant interaction effect (F (1, 39) =
4.835, p = .034, eta = .110). Focusing on each independent vari-
able, a significant effect was revealed both for cognitive dimension
(F (1, 39) = 6.272, p = .019, eta = .129) and the visualization type
(F (1, 39) = 4.039, p = .047, eta = 1.104). Regarding the main ef-
fects, the visualization type helped most the visualizers as they
increased the fixation duration on the textual AOIs, and thus, their
VBdur −r at io was decreased (F (1, 39) = 9.039, p = .005, eta = .188).
No main effects were revealed for the verbalizers regarding the visu- Figure 4: The cognition-based visualization type helped both
alization type. Regarding the cognitive dimension, no effects were visualizers and verbalizers to provide more correct answers
revealed for the subjects who used the cognition-based visualiza- to textual-content questions.
tion type, while there were significant effects for the subjects who
used the default visualization type, as discussed in the exploratory
study. The results are depicted in Figure 2.
To investigate H04 , we performed a two-way ANOVA with V-
V cognitive dimension and the type of the visualization as the expected, visualizers focused on the pictorial content and the ver-
independent variables, and VCC as the dependent variable. The balizers focused on the textual content in the visual exploratory
test met all the required assumptions. The results revealed no in- activity (i.e., virtual gallery tour), verifying the results of other
teraction effect. Focusing on each content-type, the analysis re- studies [10, 28] in other domains. Considering that each paint-
vealed no effects for VCCpic (Figure 3). Regarding, VCCt x t , the ing provided information both in pictorial and textual format, the
analysis revealed an effect both for the V-V cognition dimension overall content comprehension of both visualizers and verbaliz-
(F (1, 39) = 7.013, p = .012, eta = .152) and the visualization ers was not different, but it was average. The inherent preference
type (F (1, 39) = 8.940, p = .005, eta = .186). Focusing on main of visualizers for pictorial content influenced the content-related
effects, visualizers who used the cognition-based visualization pro- comprehension, as they comprehended the content of the pictorial
vided significantly more correct answers regarding the textual AOIs areas of interest, but not the content of the textual areas of interest,
(F (1, 39) = 5.520, p = .024, eta = .124), as depicted in Figure 4. as they produced shorter fixations on them, which implies diffi-
culties in memorability [29]. Likewise, the inherent preference of
3 DISCUSSION verbalizers in processing textual information, resulted in shorter
The results of the exploratory study underpin that individual cog- fixations on the pictorial areas of interest. Hence, verbalizers had
nitive differences have an impact on the users’ visual behavior and low performance regarding pictorial-context comprehension, but
content comprehension when performing a cultural activity. As they performed well regarding textual-context comprehension.
4
Cognition-based Cultural-Heritage Visualizations AVI-CH 2018, May 29, 2018, Castiglione della Pescaia, Italy
3.1 Cognition-based visualizations reported one provide the personalization rules. Following an inclu-
The aforementioned results underpin the necessity of adopting sive and open approach, the cognition-centered framework should
cognition-based visualizations to help both visualizers and verbal- support various cognitive styles and skills that have been found to
izers to comprehend better the visual information presented in affect users’ experience and/or behavior in cultural-heritage con-
cultural-heritage contexts. We proposed a simple dichotomous rule texts, such as field dependence-independence [19], visual working
(Algorithm 1) which provides a customized visualization of each memory [22], and personality traits [15].
art-exhibit based on the cognitive profile of the user. In the case
of a visualizer, the visualization type aims to direct her/his atten- 3.3 Implicit elicitation of Visualizer-Verbalizer
tion to textual areas of interest, while in the case of a verbalizer, cognitive style
the visualization type aims to direct her/his attention to pictorial The study results revealed that there is a strong correlation between
areas of interest. To evaluate the proposed visualization mecha- users’ visual behavior and content-comprehension, when consid-
nism, we performed a small-scale between-subject eye-tracking ering the Visualizer-Verbalizer cognitive dimension as the control
study. The results revealed that the cognition-based visualization factor. Given that eye-trackers become cheaper, smaller, more ro-
helped both user types to perform better regarding the compre- bust, and they are integrated in varying technological frameworks,
hension of the paintings’ content. The visualizers who used the such as mobile devices [6] and head-mounted displays [3], and
cognition-based visualization mechanism provided more correct they have already been used and evaluated within cultural-heritage
answers to the textual-content questions than the visualizers who contexts [14, 17], eye-gaze data could be the building factors of the
used the default mechanism. Likewise, the verbalizers who used cognition-centered framework, aiming to a) implicitly elicit user
the cognition-based visualization mechanism provided more cor- cognitive profile and b) provide personalized visualizations.
rect answers to the pictorial-content questions than the verbalizers Considering the recent works on eye-gaze based elicitation of
who used the default mechanism. At the same time, there were users’ cognitive characteristics [8, 23, 27] and the technological
no differences between visualizers and verbalizers regarding ei- advances in the eye-tracking industry, the development of transpar-
ther the pictorial or the textual content comprehension. Therefore, ent and in-run time elicitation modules that would model the users
they both increased the overall score of the content comprehension according to their cognitive characteristics is feasible in the near fu-
(including questions related to both pictorial and textual content). ture and in immersive contexts that are based on visual interaction,
such as mixed-reality [19]. Our recent works have revealed that the
3.2 Towards a cognition-centered approach for elicitation of the users’ cognitive style can be performed with high
presenting cultural-heritage content accuracy and in the early stages of a visual search activity when
The results of the comparative study underpin the necessity of considering task complexity [23], task segments [23], and time [8]
adopting a cognition-centered approach, such as a framework, to de- as the elicitation parameters along with the eye-gaze data.
liver personalized cultural-heritage activities, tailored to the users’ Therefore, our study findings could contribute to building a
individual cognitive preferences and needs. Such framework is ex- user-modeling module which extends the current range of cogni-
pected to benefit both cultural-heritage stakeholders and end-users. tive characteristics and increases the validity of other studies (and
Stakeholders from interdisciplinary fields (e.g., curators, educators, eventually the elicitation accuracy and performance). Based on
guides, designers) are expected to use such framework to create the transparent and run-time elicitation of users’ cognitive charac-
personalized cultural-heritage activities, tailored to the cognitive teristics, adaptation interventions can be applied in order for the
characteristics of the end-users (e.g., museum visitors). End-users cognition-centered framework to provide personalized visualiza-
are expected to be benefited towards achieving their goals (e.g., tions, tailored to the users’ individual characteristics. For example,
improve content comprehension) through cognition-effortless per- when a user is classified as visualizer in a virtual gallery tour, the
sonalized interventions, as they adapt to the end-users’ individual framework would provide her/him with default pictorial areas of
cognitive characteristics. interest along with emphasizing textual AOIs, based on the appro-
As discussed in [22], the cognition-centered framework consists priate adaptation rules, aiming to disperse her/his attention on both
of two main modules: the user-modeling module and the personal- types of areas of interest.
ization module. The user-modeling module is responsible to elicit,
store, and maintain cognition-centered user profiles. It can based 4 STUDY VALIDITY AND LIMITATIONS
on elicitation mechanisms which exploit data from various sources, This research work entails several limitations inherent to the mul-
such as eye-gaze interaction [23] and social-behavior data [5]. Re- tidimensional character and complexity of the factors investigated.
finement processes based on machine learning and computer vision Regarding internal validity the study environment and the study
techniques can be used to ensure the accuracy and the robustness procedure remained the same for all participants. The methodology
of the user-modeling module. and statistical tests used to answer the research objectives met all
The personalization module aims to adapt the cultural-heritage the required assumptions, despite the rather limited size of the
activity to the unique personalized configurations for users with sample, providing internally valid results.
specific cognitive characteristics. The personalization engine takes Regarding the ecological validity of our study, the study sessions
as an input the cognitive profile of the user, provided by the user- performed in times and days convenient for each participant. The
modeling module, and exports the personalized cognition-based desktop computer was powerful enough to support the virtual
visualizations, following a rule-based approach. Studies like the guide tour and did not affect participants’ experience in the shade
5
AVI-CH 2018, May 29, 2018, Castiglione della Pescaia, Italy G. E. Raptis et al.
of poor performance. The use of an eye-tracking technology was pictorial content respectively. Therefore, this work provides evi-
a limitation, as the individuals do not use such equipment when dence that the cognitive styles (e.g., Visualizer-Verbalizer) can be
performing computer-mediated activities. However, the fact that used to provide personalized cultural-heritage experiences, aiming
the eye-tracking technology used were wearable glasses, made to improve content comprehension and eliminate learning unbal-
the participants feel more comfortable after a while, as they could ances between users with different cognitive characteristics.
interact with the system as they would normally do. At this point
is worth-mentioning that we used an expensive and accurate eye-
tracking apparatus which could sabotage the application of such REFERENCES
[1] Angeliki Antoniou and George Lepouras. 2010. Modeling Visitors’ Profiles: A
schemes in typical real-life cultural-heritage scenarios. Therefore, Study to Investigate Adaptation Aspects for Museum Learning Technologies.
there is a need to investigate whether we would have the same Journal on Computing and Cultural Heritage (JOCCH) 3, 2, Article 7 (Oct. 2010),
results when using more conventional and cheaper eye-tracking 19 pages. https://doi.org/10.1145/1841317.1841322
[2] Liliana Ardissono, Tsvi Kuflik, and Daniela Petrelli. 2012. Personalization in
tools (e.g., based on web-camera feed) or whether simple eye-gaze Cultural Heritage: The Road Travelled and the One Ahead. User Modeling and
data that are easily detected, such as number of blinks, could provide User-Adapted Interaction 22, 1 (01 Apr 2012), 73–99. https://doi.org/10.1007/
similar results. s11257-011-9104-x
[3] Michael Barz, Florian Daiber, and Andreas Bulling. 2016. Prediction of Gaze
For the scope of the study, we focused only on visual interac- Estimation Error for Error-aware Gaze-based Interfaces. In Proceedings of the
tions. However, cultural-heritage activities also include audio-based Ninth Biennial ACM Symposium on Eye Tracking Research & Applications. ACM,
275–278. https://doi.org/10.1145/2857491.2857493
and spatial interactions, such as storytelling applications [7] and [4] Chih-Ming Chen and Sheng-Hui Huang. 2014. Web-based Reading Annotation
location-based games [25]. Hence, there is a need of investigating System with an Attention-based Self-regulated Learning Mechanism for Promot-
whether individual cognitive characteristics influence visitors’ be- ing Reading Performance. British Journal of Educational Technology 45, 5 (2014),
959–980. https://doi.org/10.1111/bjet.12119
havior and experience in such contexts. In the same line, recent [5] Eyal Dim and Tsvi Kuflik. 2014. Automatic Detection of Social Behavior of
studies in the cultural-heritage domain have raised the importance Museum Visitor Pairs. ACM Transactions on Interactive Intelligent Systems (TIIS)
of the visitors’ emotional engagement [18]; an aspect that needs to 4, 4, Article 17 (Nov. 2014), 30 pages. https://doi.org/10.1145/2662869
[6] Seongwon Han, Sungwon Yang, Jihyoung Kim, and Mario Gerla. 2012. Eye-
be investigated in relation to visitors’ cognitive characteristics. Guardian: A Framework of Eye Tracking and Blink Detection for Mobile Device
We expect that our results will be replicated for activities that are Users. In Proceedings of the Twelfth Workshop on Mobile Computing Systems &
Applications. ACM, 6. https://doi.org/10.1145/2162081.2162090
based visual search tasks which can be found in varying domains, [7] Akrivi Katifori, Manos Karvounis, Vassilis Kourtis, Marialena Kyriakidi, Maria
besides cultural-heritage, such as e-shopping, e-learning, and engi- Roussou, Manolis Tsangaris, Maria Vayanou, Yannis Ioannidis, Olivier Balet,
neering. Regarding the technological context, we expect our results Thibaut Prados, Jens Keil, Timo Engelke, and Laia Pujol. 2014. CHESS: Per-
sonalized Storytelling Experiences in Museums. In Interactive Storytelling, Alex
to be applicable for contexts which exploit the technologies across Mitchell, Clara Fernández-Vara, and David Thue (Eds.). Springer International
the virtuality continuum (AR/MR/VR), especially contexts that cre- Publishing, Cham, 232–235. https://doi.org/10.1007/978-3-319-12337-0_28
ate environments rich in visual information, such as head-mounted [8] Christina Katsini, Christos Fidas, George E. Raptis, Marios Belk, George Samaras,
and Nikolaos Avouris. 2018. Eye Gaze-driven Prediction of Cognitive Differences
displays (HMDs) and cave automatic virtual environments (CAVEs). during Graphical Password Composition. In 23rd International Conference on In-
Finally, our study increases the external validity of studies which telligent User Interfaces. ACM, 147–152. https://doi.org/10.1145/3172944.3172996
[9] Christina Katsini, Christos Fidas, George E. Raptis, Marios Belk, George Samaras,
investigate the effects of Visualizer-Verbalizer cognitive style on and Nikolaos Avouris. 2018. Influences of Human Cognition and Visual Behavior
visual-search tasks [10, 28]. on Password Strength During Picture Password Composition. In Proceedings of the
2018 CHI Conference on Human Factors in Computing Systems (CHI ’18). ACM, New
York, NY, USA, Article 87, 14 pages. https://doi.org/10.1145/3173574.3173661
[10] Marta Koć-Januchta, Tim Höffler, Gun-Brit Thoma, Helmut Prechtl, and Detlev
5 CONCLUSION Leutner. 2017. Visualizers versus Verbalizers: Effects of Cognitive Style on Learn-
ing with Texts and Pictures - An Eye-Tracking Study. Computers in Human
In this paper, we first reported the results of an eye-tracking study Behavior 68 (2017), 170–179. https://doi.org/10.1016/j.chb.2016.11.028
aiming to investigate the effects of V-V cognitive style on the com- [11] Laura J. Massa and Richard E. Mayer. 2006. Testing the ATI Hypothesis: Should
Multimedia Instruction Accommodate Verbalizer-Visualizer Cognitive Style?
prehension of the content of five paintings during a virtual gallery Learning and Individual Differences 16, 4 (2006), 321–335. https://doi.org/10.1016/
tour and explain the results considering the users’ visual behav- j.lindif.2006.10.001
ior. Significant differences were revealed between visualizers and [12] Richard E. Mayer and Laura J. Massa. 2003. Three Facets of Visual and Verbal
Learners: Cognitive Ability, Cognitive Style, and Learning Preference. Journal of
verbalizers regarding the comprehension of pictorial and textual educational psychology 95, 4 (2003), 833–846. https://doi.org/10.1037/0022-0663.
content. Their performance was also strongly related to their vi- 95.4.833
[13] Tracey J. Mehigan, Mary Barry, Aidan Kehoe, and Ian Pitt. 2011. Using Eye
sual behavior, which was different for visualizers and verbalizers. Tracking Technology to Identify Visual and Verbal Learners. In 2011 IEEE In-
Hence, this paper provides evidence that users with different V-V ternational Conference on Multimedia and Expo (ICME). IEEE, 1–6. https:
cognitive style follow different strategies when performing a vi- //doi.org/10.1109/ICME.2011.6012036
[14] Moayad Mokatren, Tsvi Kuflik, and Ilan Shimshoni. 2018. Exploring the Potential
sual exploratory cultural-heritage activity (e.g., virtual gallery tour). of a Mobile Eye Tracker as an Intuitive Indoor Pointing Device: A Case Study
These strategies are reflected on their visual behavior and they lead in Cultural Heritage. Future Generation Computer Systems 81 (2018), 528–541.
to unbalances regarding content comprehension. Triggered by the https://doi.org/10.1016/j.future.2017.07.007
[15] Yannick Naudet, Angeliki Antoniou, Ioanna Lykourentzou, Eric Tobias, Jenny
study results, we designed an assistive mechanism based on the Rompa, and George Lepouras. 2015. Museum Personalization Based on Gaming
visual behavior of the visualizers and verbalizers, which provided and Cognitive Styles: The BLUE Experiment. International Journal of Virtual
Communities and Social Networking (IJVCSN) 7, 2 (2015), 1–30. https://doi.org/
customized cognition-based visualizations of the paintings. To eval- 10.4018/IJVCSN.2015040101
uate its efficiency, we conducted a comparative eye-tracking study. [16] Allan Paivio. 1990. Mental Representations: A Dual Coding Approach. Oxford
The results revealed that the cognition-based visualizations helped University Press. https://doi.org/10.1093/acprof:oso/9780195066661.001.0001
[17] Isabel Pedersen, Nathan Gale, Pejman Mirza-Babaei, and Samantha Reid. 2017.
both visualizers and verbalizers to comprehend better textual and More than Meets the Eye: The Benefits of Augmented Reality and Holographic
6
Cognition-based Cultural-Heritage Visualizations AVI-CH 2018, May 29, 2018, Castiglione della Pescaia, Italy
Displays for Digital Cultural Heritage. Journal on Computing and Cultural Her- 25th Conference on User Modeling, Adaptation and Personalization. ACM, 164–173.
itage (JOCCH) 10, 2 (2017), 11. https://doi.org/10.1145/3051480 https://doi.org/10.1145/3079628.3079690
[18] Sara Perry, Maria Roussou, Maria Economou, Laia Pujol-Tost, and Hilary Young. [24] Alan Richardson. 1977. Verbalizer - Visualizer: A Cognitive Style Dimension.
2017. Moving Beyond the Virtual Museum: Engaging Visitors Emotionally. In Journal of Mental Imagery 1 (1977), 109–125. Issue 1.
In the Proceedings of the 23rd International Conference on Virtual Systems and [25] Christos Sintoris, Nikoleta Yiannoutsou, Soteris Demetriou, and Nikolaos M.
Multimedia (VSMM 2017). Dublin, Ireland. Avouris. 2013. Discovering the Invisible City: Location-based Games for Learning
[19] George E. Raptis, Christos Fidas, and Nikolaos Avouris. 2018. Effects of Mixed- in Smart Cities. Interaction Design and Architecture(s) Journal - IxD&A 16 (2013),
Reality on Players’ Behaviour and Immersion in a Cultural Tourism Game: A 47–64.
Cognitive Processing Perspective. International Journal of Human-Computer [26] Styliani Sylaiou, Fotis Liarokapis, Kostas Kotsakis, and Petros Patias. 2009. Virtual
Studies 114 (2018), 69–79. https://doi.org/10.1016/j.ijhcs.2018.02.003 Museums, a Survey and Some Issues for Consideration. Journal of Cultural
[20] George E. Raptis, Christos A. Fidas, and Nikolaos M. Avouris. 2016. Do Field Heritage 10, 4 (2009), 520 – 528. https://doi.org/10.1016/j.culher.2009.03.003
Dependence-Independence Differences of Game Players Affect Performance [27] Dereck Toker, Sébastien Lallé, and Cristina Conati. 2017. Pupillometry and Head
and Behaviour in Cultural Heritage Games?. In Proceedings of the 2016 Annual Distance to the Screen to Predict Skill Acquisition During Information Visualiza-
Symposium on Computer-Human Interaction in Play (CHI PLAY ’16). ACM, New tion Tasks. In Proceedings of the 22nd International Conference on Intelligent User
York, NY, USA, 38–43. https://doi.org/10.1145/2967934.2968107 Interfaces. ACM, 221–231. https://doi.org/10.1145/3025171.3025187
[21] George E. Raptis, Christos A. Fidas, and Nikolaos M. Avouris. 2016. Using [28] Nikos Tsianos, Panagiotis Germanakos, Zacharias Lekkas, Costas Mourlas, and
Eye Tracking to Identify Cognitive Differences: A Brief Literature Review. In George Samaras. 2009. Eye-Tracking Users’ Behavior in Relation to Cognitive
Proceedings of the 20th Pan-Hellenic Conference on Informatics (PCI ’16). ACM, Style within an E-learning Environment. In 2009 Ninth IEEE International Con-
New York, NY, USA, Article 21, 6 pages. https://doi.org/10.1145/3003733.3003762 ference on Advanced Learning Technologies. 329–333. https://doi.org/10.1109/
[22] George E. Raptis, Christos A. Fidas, Christina Katsini, and Nikolaos M. Avouris. ICALT.2009.110
2018. Towards a Cognition-Centered Personalization Framework for Cultural- [29] Michel Wedel and Rik Pieters. 2000. Eye Fixations on Advertisements and Memory
Heritage Content. In Extended Abstracts of the 2018 CHI Conference on Human for Brands: A Model and Findings. Marketing Science 19, 4 (2000), 297–312.
Factors in Computing Systems (CHI EA ’18). ACM, New York, NY, USA, Article https://doi.org/10.1287/mksc.19.4.297.11794
LBW011, 6 pages. https://doi.org/10.1145/3170427.3190613 [30] Rafal Wojciechowski, Krzysztof Walczak, Martin White, and Wojciech Cellary.
[23] George E. Raptis, Christina Katsini, Marios Belk, Christos Fidas, George Samaras, 2004. Building Virtual and Augmented Reality Museum Exhibitions. In Proceed-
and Nikolaos Avouris. 2017. Using Eye Gaze Data and Visual Activities to Infer ings of the Ninth International Conference on 3D Web Technology (Web3D ’04).
Human Cognitive Styles: Method and Feasibility Studies. In Proceedings of the ACM, New York, NY, USA, 135–144. https://doi.org/10.1145/985040.985060
7