=Paper=
{{Paper
|id=Vol-3160/short18
|storemode=property
|title=Addressing User Engagement With an Interactive Reading Model by Innovative Digital Expansion
|pdfUrl=https://ceur-ws.org/Vol-3160/short18.pdf
|volume=Vol-3160
|authors=Nicola Barbuti,Mauro De Bari
|dblpUrl=https://dblp.org/rec/conf/ircdl/BarbutiB22
}}
==Addressing User Engagement With an Interactive Reading Model by Innovative Digital Expansion==
https://ceur-ws.org/Vol-3160/short18.pdf
Addressing User Engagement With an Interactive Reading Model
by Innovative Digital Expansion
Nicola Barbuti, Mauro De Bari
University of Bari Aldo Moro, Umberto I, Bari, Index, Italy
Abstract
The paper presents the research results aimed to design and realize an innovative prototype of
user’s interactions method, based on the digital impact that Digital Transformation (DT) has
on society. The experimentation presented takes as an example some good practices identified
in the Victoria & Albert Museum (V&A), London: challenging, unconventional cultural goods
could attract new users. Based on this assumption, we have observed a target users’ impact on
a cultural good generally considered little appeal, transforming the user into a prosumer. The
first tests were made in 2019. The University of Bari Aldo Moro and the De Sanctis Classical
Linguistic High School of Human Sciences in Trani started a collaboration as part of a PON
project titled "Comunicare e promuovere un evento culturale". High School students have been
identified as an important target of study. Subsequently, in the paper, we examined a project
created by D.A.BI.MUS S.r.l. and Quorum Italia S.r.l, working on niche cultural goods also in
this case. In this project, an anastatic printed book became a potential endless book capable of
intercepting the interest of even large groups of digital users.
Keywords 1
Digital Transformation (DT), digital expansion, prosumer, user-centric, user perspective
1. Introduction
Recently, designing and implementing pilot experiences has been consolidated in the diverse
cultural heritage and culture sectors. Pioneering experiences can stimulate emotional impacts on users
[1]. Moreover, in this period of Digital Transformation (DT) [2], digital technology offers a wide range
of possibilities [3] with almost unlimited potential.
This trend was accentuated during the pandemic caused by COVID-19, where the closure of cultural
institutions generated a dichotomy in supply and demand. For several months, people have been
thinking about dealing with the limits imposed by the impossibility of physically enjoying cultural
experiences. The possibility of having ‘alternative’ interactive experiences available, therefore, has
been transformed into a need. However, equally, this need has manifested a reasonable opportunity to
develop new interactions with heritage [4].
The study and analysis of the digital proposals of some European leader institutions12show that
among the various solutions promoted -calibrated according to the potentially affected users- the use of
digital technologies maintains a first-level role in the creation of interactive experiences [5].
As an excellent digital innovation practice, we consider the proposals of the Victoria & Albert
Museum (V&A) [6] in London, which presents some interesting user-centric approaches [7].
IRCDL 2022: 18th Italian Research Conference on Digital Libraries, February 24–25, 2022, Padova, Italy
nicola.barbuti@uniba.it (N. Barbuti); mauro.debari@uniba.it (M. De Bari)
0000-0003-0817-4235 (N. Barbuti); 0000-0003-2730-2204 (M. De Bari)
©️ 2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
1
Some interactive proposals created by cultural institutions to promote user-centric digital experiences:
https://www.rijksmuseum.nl/en/masterpieces-up-close;
https://artsandculture.google.com/streetview/british-museum/AwEp68JO4NECkQ?sv_lng=-
0.1266024509257022&sv_lat=51.51905368906714&sv_h=129.83421644322323&sv_p=-
15.93602158329513&sv_pid=JeKwUFYAMWXNWPh3IOg3jw&sv_z=2.95691280674487;
https://3d.smb.museum/pergamonaltar/.
Katy Price, Head of Digital Media at V&A, argues that it is essential to create a compelling online
experience that does not attempt to recreate the museum experience but instead offers something indeed
distinctive [8].
It seems that the “traditional perception of heritage” could be overturned and modified towards a
touch-free approach [9]. A solution initially conceived to encourage the flow of people with visual
disabilities into museums [10] could be modified and adapted to different cultural aims, such as the
emergency determined by the pandemic. Furthermore, a touch-free approach could be an offer
comparable to the “traditional” one in the future. Nevertheless, the premise remains to generate an
intensely emotional impact in users by involving them as participants in the experience of interaction
with goods [11]. This approach is how some international cultural institutions operate more than before
[12].
However, the promising scenario does not always consider an indispensable prerequisite correctly
for activating exciting and sustainable solutions: digital as an “activator of interest” for visit-actors
increasingly oriented towards active interaction. In this, the large digital platforms have taught us a
fundamental rule: to put the user at the centre [13]. This need seems particularly crucial in the librarian
field, considering the decreasing number of people who read.
For this reason, it becomes essential to “define who the users are, including everything about work
roles, sub-roles, user class definitions, and personas” [14] to address an issue that is affecting
contemporary society.
2. State-of-the-art, how the digital affects users and reading
Some recent industry analyses have confirmed that, in the new generations, reading tends to decrease
more and more, in part, due to the DT [15].
Although, since the early 2000s, experiments have been started on printed texts mixed with digital
solutions, it seems that expanding content with graphics, animations, and digital audio in Augmented
Reality (AR) transforms printed books into a dynamic and engaging experience. This improvement
could foster a return of interest in the paper-printed book.
However, as suggested by Virginia Clinton after several studies, initial experimentations on digital
books missed something. “My findings were not fair to screens because the screens could not offer
everything they could… they were just a shiny piece of paper” [16]. Clinton, a strong supporter of
digital books, has changed her mind because many students refused to study with digital support.
Students considered those lacked suggestions and expansions. While by contrast, the introduction and
the use of AR content are shifting this interpretation, trying to improve books perception and restoring
the paper’s interest.
As McKenzie and Darnell stated, the research on applications of AR to library contents was initially
conducted mainly to design resources that fostered the learning of determinate categories of users [17].
The experiments would facilitate the understanding of two-dimensional printed contents by visualizing
three-dimensional objects that expand them. From the studies made, researchers have deduced that the
use of AR in the education system improves students’ learning by fixing abstract concepts [18]. In this
perspective, “one of the most interesting uses in recent years among the promising educational
applications are AR books” [19].
In the first decade of the new century, the first experiments of integrated texts with content in AR
were implemented in mathematical publishing and biochemical [20]. Effectively, these pioneering
experiments have initiated the research that led to the definition of the book model. Furthermore, the
aim was to motivate the experience of users positively.
The first somewhat evolved solution of this model was studied, in 2010, in the context of research
on texts usable by subjects suffering from deafness [21].
However, all these experiments were based on AR configurations for desktop PCs. Severely limited
production fallout from several critical issues: high costs, unattractive product appeal, limited user
capabilities. In fact, despite an augmented book being considered, often by opponents, like a printed
book, advocates sustain hardware and software implementations transform augmented books into
complex digital cultural entities [22].
The breakthrough came with the move to AR solutions for mobile devices, well known and widely
used by the different age groups of the new generation of digital natives.
The model AR Book that launched this new phase of the research was the “MagicBook”, realized
with the functions of Education and edutainment for children and young people of school age, which
allowed users to read the physical book as a regular text and display 3D content via handheld [23]. It
has been possible to create immersive and innovative solutions starting from this moment. The user is
progressively more involved than in the past because of the spread of increasingly accessible digital
technological solutions. Recently, the V&A has launched a project titled “Immersive Dickens” [24] that
synthesizes the aim of this paper. The museum conceived the project considering manuscripts
challenging objects. The V&A has identified a good that is generally little appreciated and usable by a
non-specialized user, a manuscript, to make it expandable and attractive through mixed reality. The
experience generated has promoted a double return for the museum in terms of user perception, creating
unconventional attractiveness and bringing a different, more expansive audience closer2.3
Subsequently, the museum has identified the user target: “We want to involve the next generation
in the development of a ‘next generation’ experience. We have chosen a demanding audience, in the
shape of teenage students. By helping them discover and understand Dickens’s creative process, we
aim to inspire young people’s self-expression” [25]. Finally, it has created the immersive experience to
shift users from watchers to proactive prosumers.
The analysis of this model and the previous research experiences are the assumptions that
substantiate the methodological approach used in the research that will be described later. In particular,
the goal is to show user interaction models, using an experiment conducted in some school libraries and
presenting an innovative platform for the marker and the interactive use of paper books. Therefore, the
methodologies of digitizing artefacts books for paper editions are integrated with the creative use of
digital technology that favours interaction with dynamic content personalized by users.
2.1. Methodology
The methodological proposal described results from some experiments that confirm the previous
assumptions. We tested our research into two different contexts, the educational and the CCIs.
2.1.1. Expanded Library project
In 2019, we started by looking for a target user to test. Finally, based on the excellent practice carried
out by the V&A, considering that “The advent of augmented reality technologies often promises great
potential for improving education”, we tested high school students as ideal users. In this orbit, the
DISUM Department of the University of Bari Aldo Moro, in the course of 2019, collaborated with the
De Sanctis Classical Linguistic High School of Human Sciences in Trani as part of the PON titled
“Comunicare e promuovere un evento Culturale”3.4
The purpose of this project was divided into two phases.
1. The first was to implement the search for digital technologies capable of expanding the
books identified by the students.
2. The second is to modernize the old school library using AR interaction modes.
As described above, it emerges that the paper book approach is “difficult” for young people. Students
expressed their perplexity in considering books as entertainment or cultural expansion elements.
Instead, the high school students have expressed their rejection of books regarded as compulsory study
tools. Hence the idea of making the school library more attractive. However, “when planning to design
2
“It can’t be handled or viewed in full; it’s full of complex handwritten revisions and not easy to read; it doesn’t have immediate visual impact
and the connection with the rich imaginative world of Dickens’s fiction isn’t always obvious. But the manuscripts are the best evidence we
have of the creative process of one of the UK’s most important and renowned authors, whose work has continuing relevance. Dickens was an
observer of urban material culture and life in a period of rapid technological innovation and widespread financial insecurity, a writer who
explored themes that still resonate today”, https://www.vam.ac.uk/blog/projects/dangerously-immersive-dickens.
3
http://www.liceodesanctis.edu.it/attachments/article/1984/1.pdf.
an Augmented Reality Learning Object (ARLO), researchers together with educators need to take into
consideration how the content will be presented to students based on what technology is available at
hand or what can be procured” [26]. 56
We have tried to bring students closer to the paper book and expand it through AR. Through co-
working, students approached a contemporary reality in the cultural debate, becoming prosumers
themselves and changing their minds on their previous beliefs. Students sampled the pages of some
volumes, then scanned and digitized them. At the end of this step, the digital expansion activities began.
These, through platforms such as BLIPPAR4 and Genial.lly5, have expanded the paper-book with digital
content, allowing a different perception and immersion of cultural contents.
The critical points of the methodology applied are summarized below:
• Identification of a target study:
Twenty-five, high school students were divided into work teams for a total of five groups of 5
students each.
• Training:
Two training sessions were organized to introduce the topic of expanded books to the students.
• Practical tasks:
1. In the school library, each workgroup has chosen one or more volumes relating to
topics chosen from the available materials, for a total of 10 volumes.
2. The selected volumes were analyzed, and the contents of potential interest for the
connection of the extensions were identified.
3. The research was carried out both online and in other paper volumes of the library to
verify the availability of the contents.
4. Pages and contents to be used have been scanned. The scanned contents and the
expansions have been located in a dedicated online repository.
5. In correspondence with the digital reproduction of the topical pages identified by the
students, the marking software for recognizing the contents of interest was applied.
6. Subsequently, the contents were expanded.
7. Finally, a prototype web link was created with everything necessary to make the
expansions usable through mobile technology.
• Test:
Students have verified that by framing the chosen book on the marked page, the display showed the
reproduction of the page with the access keys to the highlighted expansions and the types of related
contents (images, audio, video, audio-video).
In this way, the students approached little-known resources, including the “Libro Rosso della città
di Trani” (collection and transcription of ancient diplomatic documents), appreciating its contents, and
re-evaluating the book as an entity is no longer static.
7
2.1.2. EXEBook
In parallel, research was carried out to develop an innovative editorial process to produce printed
books integrated with “digital expansions” usable through mobile devices. D.A.BI.MUS. S.r.l6, (a spin-
off of the University of Bari, specialized in the design and development of digitization services and
4
https://www.blippar.com.
5
https://genial.ly.
6
https://www.dabimus.com.
digital products for the cultural heritage) and the publishing house Quorum Italia S.r.l. 7 (a company
focused on the fruition of ancient and historical books) has practically realized this aim. 8
The lack of knowledge about the Apulian authors’ book production of the past centuries has
generated interest in implementing the research. The aim is to show their importance as protagonists in
the cultural and scientific debates of the different eras. Hence, the suggestion to experiment with the
anastatic reprint of an ancient text linked to Apulia curated in the typographic design. This supposition
has based a first research question: how to make analogues qualitatively attractive and, equally, made
interactive using the innovative digital solutions on the model of ARBooks, expanding the textual
content, to intercept the interest of a young audience.
The experimental methodology was divided into three main activities.
1. The first focused on the choice of analogue materials: a collection of printed poems from
the first half of the nineteenth century was chosen. They represent the oldest known
testimony of the original Barese vernacular idiom. Next, the volume was scanned in high-
definition BITMAP format for printing. Finally, the images were post-processed by working
on the contrast between text background and the definition of the graphic stroke to favour
the best application of the markers.
2. The second activity has included the typographic design (Figure 1) and printing the paper
prototype to apply the AR markers. The aesthetic quality of the book plays a fundamental
role in the impact with the user, as a factor that helps arouse interest and stimulate physical
interaction with the object. In this regard, it was considered crucial to associate with the
classical manipulation of the physical object modes of use of the mobile device based on
Tangible user Interface (TUI), using the finger as a pointing element to interact with the
contents augmented.
Figure 1: First design of the book cover
3. The third activity was focused on the development of the application in AR. Several web
apps were analyzed, and for the development of the first version of the pilot, the open-source
Blippar2 and Unity, provided with the necessary tools to develop the application model and
usable with any system for mobile devices (iOS and Android). However, both platforms:
practical, functional, and sufficient for the students, were unstable and unsuitable for the
needs of this creation. For this reason, a platform for uploading content has been built in
house.
7
https://quorumedizioni.it.
The configuration of the AR application was based on four main components:
• the mobile device for using the content in AR;
• the mobile device camera for framing and scanning the object in real-time;
• the analogue book;
• Markers for tangible interaction and tracking targets.
The template’s design integrates the concepts of “History page” and “marker page”. The history
page covers the content of the pages of the volume. The marker page consists of the markers that
animate the AR content linked to each page. The prototype to view and interact with contents uses the
display of mobile devices that, pointed directly at the page in reading, scans it and highlights the
increased interface, allowing the user to explore the contents with dedicated tools and to enjoy them
while keeping intact the standard reading of the book (Figure 2 and 3).
Figure 2: First experimentations
Figure 3: Scanned image
Digital content was then created to expand the printed text: digital text files, audio/video files with
interactive reading, image files, avatars for interactive reading by users.
DA BI.MUS. S.r.l is implementing this proposal launching: EXEBooK, precisely. A mobile app (for
both Android and iOS operating systems) capable of taking a photo on any page (in the main screen)
of a “unique identifier” code associated with a book (among those that provide interactive content) and
to automatically “land” on the “target” resource of the platform.
The application differs because it can mark content of interest, generating a link, which allows you
to access additional digital content, both variable and augmented. Furthermore, the reader, interacting
in real-time with the marker page through the dedicated tools and having access to augmented content,
can also live the emotional experience of becoming an avatar protagonist of the text reading.
In particular, when starting the mobile app, the user will enter a code that he will find associated
with the paper book. Subsequently, the mobile app will prepare itself in “camera preview” to allow the
user to frame and photograph any page of the analogue book in his possession (obviously one enabled
to use the mobile app). Upon successful acquisition, the image will be sent to the appropriate REST
service, displayed by the server. Next, the relative alphanumeric “template” will be extracted from the
picture, which, given as input to the indexer, will allow you to determine which page (and which book)
it is, to retrieve the “target” URL address to be returned to the mobile app (as the output of the called
REST service).
The mobile app will view the digital/interactive resource, from the platform, directly in the app itself
after the user has received the “target” URL address.
The user always could repeat the action on other pages of the book or other books (by entering, in
this case, a new “unique identifier” code).
Also, in this case, users actively participate in the perception of the cultural asset, being free to enjoy
the contents of their interest. The user is not trapped in a guided choice of the expanded contents to be
enjoyed. Furthermore, this approach allows continuous and “n” expansions.
3. Conclusion
Today, cultural content cannot have the same interaction as it did in the past, especially considering
user demand and how people want to approach the heritage. For this reason, those involved in producing
cultural content could entice the user with user-centric experiences to have positive feedback.
Furthermore, the user could actively participate in the cultural production, as this is the last “evaluator”,
as demonstrated before with the case studies reported. Hence, cultural institutions seem to understand
how people are precious to the life-cycle of experience, primarily digital or phygital. Therefore, as
previously reported, choosing content that is little known and perhaps not customarily considered can
become a challenge for the supplying institutions and who is called to expand elements digitally.
Peculiar goods could be more attractive than others typically considered the praxis, becoming
generators of unique and cutting-edge experiences.
4. References
[1] E. Bonacini, “Il museo partecipativo sul web: forme di partecipazione dell’utente alla produzione
culturale e alla creazione di valore culturale”, Il capitale culturale, Studies on the Value of Cultural
Heritage, 2012, 5: 93-125.
[2] I-scoop: Digitization, digital and digital transformation: the differences; https://www.i-
scoop.eu/digital-transformation/digitization-digitalization-digital-transformation-disruption/.
[3] European Commission website: Digital cultural heritage; https://ec.europa.eu/digital-single-
market/en/digital-cultural-heritage.
[4] ICCROM: Heritage and Pandemics: Accessing heritage during a pandemic;
https://www.iccrom.org/events/heritage-and-pandemics-accessing-heritage-during-pandemic
[5] P. Callet, “3D Reconstruction from 3D Cultural Heritage models”, in: Münster, Sander / Pfarr-
Harfst, Mieke / Ioannides, Marinos (eds.): 3D Research Challenges in Cultural Heritage. Lecture
Notes in Computer Science, vol 8355. Springer, Berlin, Heidelberg, 2014: 132-145.
DOI:10.1007/978-3-662-44630-0_10136-145.
[6] Victoria and Albert Museum website: Welcome to the V&A – the world’s leading museum of art
and performance; https://www.vam.ac.uk.
[7] Victoria and Albert Museum blog: Museum interactives;
https://www.vam.ac.uk/blog/digital/museum-interactives.
[8] Artribune: Musei e digitale. L’esempio del Victoria & Albert Museum di Londra;
https://www.artribune.com/professioni-e-professionisti/who-is-who/2020/05/musei-digitale-
victoria-albert-museum-londra/.
[9] Science Museum Group Digital Lab: Is this the end of touchscreens in museums? The use of
touchless gesture-based controls; https://lab.sciencemuseum.org.uk/is-this-the-end-of-
touchscreens-in-museums-the-use-of-touchless-gesture-based-controls-ee3f3c3f37ce.
[10] A. S. Vassillos, K Charikleia, “Re-imagining the museum through “touch”: Reflections of
individuals with visual disability on their experience of museum-visiting in Greece”, Alter, 2015,
9, 2: 130-143.
[11] Victoria and Albert Museum blog, How can technology improve the museum experience?;
https://www.vam.ac.uk/blog/digital/how-can-technology-improve-the-museum-experience.
[12] I. Ten, G. Arbelaez-Garces, L. Dupont, “Designing interactive museum visitors’ experience using
a Fab Living Lab platform: the Museum-Aquarium of Nancy case”, in: 2020 IEEE International
Conference on Engineering, Technology and Innovation (ICE/ITMC), Jun 2020, Cardiff, United
Kingdom, 1-8, 10.1109/ICE/ITMC49519.2020.9198641. hal-02941618.
[13] Agenda Digitale, La digital transformation immersiva dei Beni Culturali: l’esempio di M9;
https://www.agendadigitale.eu/cultura-digitale/la-digital-transformation-immersiva-dei-beni-
culturali-lesempio-di-m9/
[14] R. Hartson, P. S. Pyla, The UX book. Process and guidelines for ensuring a quality user experience.
Morgan Kaufmann, 2016, p. 222.
[15] C. Faggiolani, A. Federici, L’affordance della biblioteca: Modalità d’uso emergenti dalle indagini
Istat, Biblioteche, 2018, 3:36, pp. 5-16.
[16] J. Barshay, “Evidence increases for reading on paper instead of screens”,
https://hechingerreport.org/evidence-increases-for-reading-on-paper-instead-of-screens/.
[17] J. McKenize, D. Darnell, The eyemagic book: A report into augmented reality storytelling on the
context of a children’s workshop. NZ: Centre for Children’s Literature, Christchurch College of
EducationAuthor, 2016. 2(5), pp. 99–110.
[18] G. Martgetis, A. Ntelidakis, X. Zabulis, S. Ntoa, P. Koutlemanis, C. Stephanidis, Augmented
physical books towards education enhancement. User-Centerd Computer Vision (UCCV), 2013
1st IEEE Workshop on, 2013, pp.43-49.
[19] H. Altinpulluk, M. Kesim, The classification of Augmented Reality books: a literature review, in
Proceedings of INTED2016 Conference 7th-9th March 2016, Valencia, Spain, 2016.
[20] H. Kaufmann, D. Schmalstieg, Mathematics and Geometry in Education with Collaborative
Augmented Reality, ACM SIGGRAPH 2002 Conference, New York, 2002, pp. 37-41.
[21] E. Medina, Y. C. Chen, S. Weghorst, Understanding biochemistry with Augmented Reality, in C.
Montogomerie & J. Seale (Eds.), Proceedings of Word Conference on Educational Multimendia,
Hypermedia and Telecommunications, 2007, pp.4235-4239.
[22] M. Billinghurst, H. Kato, I. Poupyrev, The magic book: a transitional AR interface. Computers &
graphics, 2001, 25(5), pp. 745-753.
[23] A. B.Tomia, R. Dayang, A. Rambli, An Interactive Mobile Augmented Reality Magical Playbook:
Learning Number with the Thirsty Crow, International Conference on Virtual and Augmented
Reality in Education, Procedia Computer Science 25, 2013, pp. 123-130.
[24] V&A Research Projects, Immersive Dickens,
https://www.vam.ac.uk/research/projects/immersive-dickens.
[25] K. Price, Dangerously immersive’ Dickens, https://www.vam.ac.uk/blog/projects/dangerously-
immersive-dickens.
[26] M. Kljun, G.Vladimir, K. Č. Pucihar, Augmented Reality in Education: Current Status and
Advancement of the Field, in Augmented Reality in Education, 2020.