=Paper=
{{Paper
|id=Vol-1910/paper0203
|storemode=property
|title=The Madeira Touch: Encouraging Visual-Spatial Exploration Using a Tactile Interactive Display
|pdfUrl=https://ceur-ws.org/Vol-1910/paper0203.pdf
|volume=Vol-1910
|authors=Catia Prandi,Catherine Chiodo,Ricjeareu Villaflor,Nicolas Autzen,Johannes Schöning
|dblpUrl=https://dblp.org/rec/conf/chitaly/PrandiCVAS17
}}
==The Madeira Touch: Encouraging Visual-Spatial Exploration Using a Tactile Interactive Display==
https://ceur-ws.org/Vol-1910/paper0203.pdf
The Madeira Touch: Encouraging
Visual-Spatial Exploration using a
Tactile Interactive Display
Catia Prandi Abstract
ARDITI, Madeira-ITI The current information marketplace for tourists is
Funchal 9020-105, Portugal dominated by for-profit purveyors of information.
catia.prandi@m-iti.org Potential visitors must rely on experts-for-hire or
search engine results in order to learn about a desired
Catherine Chiodo destination. In this paper, we introduce The Madeira
Ricjeareu Villaflor Touch, a multimodal display installation rooted in the
Carnegie Mellon University unique characteristics of Madeira, which allows users to
Pittsburgh PA 15213, USA explore the island by selecting a type of scenery and
catherine@chiodo.org showing the user-generated photos of that type of
rj.villaflor@gmail.com scenery in a map-based interface. To make this
pervasive display more engaging, we designed an
Nicolas Autzen exploratory tactile-input mode of interaction: users will
Johannes Schöning be able to touch a physical object, representing a type
University of Bremen of scenery (a rock for mountains, a seashell for the sea,
Bremen 28359, Germany etc.), which will then bring up suitable photos of that
n.autzen@uni-bremen.de type of scenery overlaid on a map of the island. The
schoening@uni-bremen.de display will help users to form their mental image of the
island and to plan trips that best suit their interests.
Author Keywords
Pervasive display, tactile interaction, multimodal
interaction, user-generated content, digital signage
ACM Classification Keywords
H.5.m. Information interfaces and presentation (e.g.,
Copyright is held by the author/owner(s).
HCI): Miscellaneous.
CHItaly ’17, September 18-20, 2017, Cagliari, Italy.
Introduction & Motivation The Madeira Touch, a pervasive display that allows
Scenarios of Use In an increasingly connected world, travelers seeking users to correlate photos with locations, using a
Scenario 1: While waiting at local experiences may encounter a paradox. While it is multimodal interaction that allow users to select either
the tourism office, a visitor easy to find information about destinations and places a type of scenery (touching a physical object) or a
notices a display with boxes that they may wish to visit, it is also increasingly location (using the map-based visualization). Madeira
containing different objects difficult to differentiate between the many options represents an ideal place to develop a new way for
surrounding it, in front of the available. At the same time, travelers who have visitors to explore the island, however, it is our
entrance. She touches one witnessed the homogenizing effects of globalization intention for this system to be adaptable to other
object and notices the display may place a particular premium on unique experiences tourist destinations, levering on the location’s unique
showing images from a that can only be found in certain places. offerings and characteristics.
levada walk in Madeira. She
removes her hand from the In this project, we took the island of Madeira as a
object and the display returns representative case study of a well-known touristic
to the map of the island. She destination that could be made more discoverable to
continues to touch the the island’s visitors. However, while tourists may
objects and see the images choose to visit Madeira because of its beautiful scenery
associated with them. She and outdoor activities, often related to the island’s
leaves the display with a levadas, remote canals that serve as walking paths,
better understanding of the they may not know what destinations on the island are
opportunities on the island. best suited to the kinds of sceneries they hope to see.
Currently, travelers who hope to experience natural
Scenario 2: A local Madeiran beauty can plan their trips by either starting with a
visits the tourism office to possible location and attempting to find correlating
see the new interactive photos, or by beginning with photos of destinations
display. While interacting they would like to visit and then attempting to find Figure 1: The Madeira Touch in context
with it, he notices that one of location information. Both approaches suffer
the photos is an image he breakdowns when photos are not tagged with the The Madeira Touch
took on Instagram. He feels commonly-used location names. Our solution is to provide the Madeira’s main tourism
gratified that one of his office, with our pervasive display. There, visitors to the
images is contributing to the In the field of pervasive display systems [4], we island will be able to explore a map of Madeira using
experience available on the designed our solution to investigate how the use of an two modes of interaction: 1. traditional map-based
display and to the information exploratory tactile mode of interaction to provide touchscreen interaction (Figure 2), or 2. exploratory
provided to tourists on the georeferenced visual information in the form of user- tactile-input interaction (Figure 3). With the first type of
island. generated photos of points of interest (POIs) can interaction, visitors will be able to touch the digital map
facilitate visitors in discovering locations on the touristic and see user-generated photos of that location. With
destination, enhancing their experience. The result is the second one, users will be able to touch a physical
object that corresponds to a type of scenery on the interaction with the data, engendering an experience
Scenario 3: While his island, which will then bring up user-generated photos that is less goal-oriented and more exploratory. On the
parents are in the queue, a of that type of scenery (Figure 1). By enabling both other hand, the use of tactile physical objects can raise
15-year-old notices a monitor forms of interaction, we intend to encourage the ‘affordance blindness’ issue, defined as the inability
surrounded by different exploratory scenery discovery as well as practical trip- to understand the interaction modalities of a public
natural objects in display planning in context in which multiple users, such as display [3]. Our solution aims to moderate this
boxes, in front of the family groups, can co-experience the display [5]. problematic providing visual hints to attract the user’s
entrance. He tries to touch a attention (as described in Scenario 3).
location on the monitor, Design Concepts
expecting a touch-based Traditional tourism information relies on professionally-
interaction with the system. produced content which is limited both in quantity and
In response to his touch, the in coverage. We intend for this system to serve as an
display shows a photo of that exploration of how UGC from social media can be
area. At the same time, a box curated to provide dynamic, updated and custom sets
lights up, grabbing his of information for specific audiences (i.e., visitors).
attention. He notices that the
box holds a tree branch and While interactive displays have a high potential to Figure 2: Users can touch the thumbnails on the screen.
that the photo on the screen engage passersby, they frequently go unnoticed and
is full of trees. Intrigued, he unused [7, 11], confirming the so called ‘display
decides to touch the tree blindness’ effect [8]. By situating our display in a
branch and the monitor strategic location (i.e., in front of the entrance) of a
begins to display photos of tourism office where visitors often wait to speak to
forests in different areas of someone, we intend to mitigate this issue, exposing
the island. He touches visitors to the display at a time when they will be
another object, then another. inclined to investigate. To further increase visitor
Figure 3: Users can directly touch the physical objects.
When his parents are done, engagement, we have incorporated a novel form of
he brings them over to the input in the form of physical objects that a user may
Implementation
display and together they touch to experience certain kinds of sceneries. In fact,
The Madeira Touch software architecture is composed
explore the island. studies of initial engagement with interactive displays
of three main modules (as shown in Figure 4). The
have found that physical interactions prompt greater
Geotagged Photos Retrieval Module collects geotagged
rates of engagement among passersby [6]. Moreover,
photos and paths related to the main touristic module
our solution aims to overcome the ‘interaction
first retrieves and stores georeferenced routes by
blindness’ [10] that often plagues public displays by
pedestrian walks that characterize the island. This
providing users with novel and suggestive physical
stage, we have decided to use OpenStreetMap (OSM),
objects that encourage non-linear exploration. This
an open source system that allows users to voluntarily
paradigm encourages a very different kind of
collect and share GPS tracks and georeferenced data
(i.e. Point of Interests). In Madeira, an island with an the object the user touches, showing the information in
area of 802 km², the OSM dataset includes 16000 a Google Maps based interface. Regarding the hardware
points and 24000 lines along with walking paths requirements, The Madeira Touch utilizes a touch
(levadas). There are also private datasets gathered by screen monitor and sensors to indicate when an object
companies that have collected GPS tracks specifically has been touched. Considering the design of our
related to walking paths such as Walk Me Madeira system, simple motion sensors should work well. The
(http://www.walkmeguide.com/). Based on this dataset current design also uses LED lights in each box to
of GPS tracks and routes, the module retrieves public emphasize the way in which each object corresponds to
geotagged photos from different social media platforms a certain type of photo. This way, even when the user
and photo blogs to continually integrate UGC (i.e. is interacting directly with the touchscreen, the relevant
photos) to enrich the user experience of our system object will light up, indicating the relationship between
[12]. At this stage, the system includes the Instagram object and image.
Figure 4: System architecture and Flickr platforms, both of which provide developers
with APIs for retrieving public photos based on Conclusion and Future Work
locations and/or tags. Our concept combines three characteristics in a unique
way to make an engaging pervasive display for visitors
The Classification Module is the core of our system, to the island of Madeira. By using images from social
because it enables (i) the elimination of photos which media, the system insures that the content remains
include faces and other non-nature showing images and dynamic and accurate. By allowing users to explore the
(ii) the categorization of the collected pictures based on data, either by location or by scenery type, the system
the main objects (rocks, sand, etc.) and scenery (cliff, allows for scenery exploration and practical trip
forest, etc.) in each image. Each of these categories is planning. Finally, by creating a tactile form of
correlated with a physical object with which the users interaction, the system encourages users to consider
can interact. Different kinds of machine learning the natural materials represented, allowing them to
algorithms have been developed for the recognition and form a more sensual and complete mental picture of
classification of faces/landscapes/nature elements the destination. While all three of these characteristics
represented in photos from social media platforms and are transferable and could be applied to other
photo blogs [1, 2, 9, 13]. However, this system could destinations, it is the authors’ belief that, for future
also use a crowdsourcing approach to let participating installations of this solution, the appeal and utility of
users manually check the photos. A third approach such a system relies on its ability to accurately reflect
would be a system like the Google ReCAPTCHA, which the unique character of the local environment. To
asks users to solve a puzzle by selecting all the images evaluate the effectiveness of our pervasive display in
that represent a specific element. engaging users and enhancing their visiting experience,
overcoming the display/interaction/affordance blindness
The Visualization Module is the final step in this system issues, we plan to install the system in the Madeira’s
and connects the categorized geotagged photos with main tourism office.
References Pervasive Displays (PerDis '13), 19-24.
1. Nuttapoom Amornpashara, Yutaka Arakawa, http://doi.acm.org/10.1145/2491568.2491573
Morihiko Tamai and Keiichi Yasumoto. 2015. 8. Jörg Müller, Dennis Wilmsmann, Juliane Exeler,
Landscape photo classification mechanism for Markus Buzeck, Albrecht Schmidt, Tim Jay, and
context-aware photography support system, In Antonio Krüger. 2009. Display blindness: The effect
Proceedings of Conference on Consumer Electronics of expectations on attention towards digital
(ICCE 2015), 663-666. signage. Pervasive Computing (2009): 1-8.
http://doi.acm.org/10.1109/ICCE.2015.7066570 http://dx.doi.org/10.1007/978-3-642-01516-8
2. Pu Cheng and Jie Zhou. 2011. Automatic Season 9. Mor Naaman, Susumu Harada, QianYing Wang,
Classification of Outdoor Photos, In Proceedings of Hector Garcia-Molina, and Andreas Paepcke. 2004.
the Conference on Intelligent Human-Machine Context data in geo-referenced digital photo
Systems and Cybernetics, 46-49. collections. In Proceedings of the conference on
http://doi.acm.org/10.1109/IHMSC.2011.18 Multimedia (MULTIMEDIA '04), 196-203.
3. Jorgos Coenen, Sandy Claes, and Andrew Vande http://dx.doi.org/10.1145/1027527.1027573
Moere. 2017. The concurrent use of touch and mid- 10. Gonzalo Parra, Joris Klerkx, and Erik Duval. 2014.
air gestures or floor mat interaction on a public Understanding Engagement with Interactive Public
display. In Proceedings of the Symposium on Displays: an Awareness Campaign in the Wild. In
Pervasive Displays (PerDis '17). Article 9, 9 pages. Proceedings of the Symposium on Pervasive
DOI: https://doi.org/10.1145/3078810.3078819 Displays (PerDis '14), 180-186.
4. Nigel Davies, Sarah Clinch, and Florian Alt. 2014. http://dx.doi.org/10.1145/2611009.2611020
Pervasive displays: understanding the future of 11. Peter Peltonen, Esko Kurvinen, Antti Salovaara,
digital signage. Synthesis Lectures on Mobile and Giulio Jacucci, Tommi Ilmonen, John Evans, Antti
Pervasive Computing 8.1 (2014): 1-128. Oulasvirta, Petri Saarikko. 2008. It’s mine, don't
5. Jodi Forlizzi and Katja Battarbee. 2004. touch!: interactions at a large multi-touch display
Understanding experience in interactive systems. in a city centre. In Proceedings of the Conference
In Proceedings of the conference on Designing on Human Factors in Computing Systems (CHI
interactive systems: processes, practices, methods, '08), 1285–1294.
and techniques (DIS '04), 261-268. http://dx.doi.org/10.1145/1357054.1357255
http://doi.acm.org/10.1145/1013115.1013152 12. Pavel Serdyukov, Vanessa Murdock, and Roelof van
6. Wendy Ju and David Sirkin. 2010. Animate objects: Zwol. 2009. Placing flickr photos on a map.
how physical motion encourages public interaction. In Proceedings of the conference on Research and
In Proceedings of the conference on Persuasive development in information retrieval (SIGIR '09),
Technology (PERSUASIVE'10), 40-51. 484-491.
http://doi.acm.org/10.1007/978-3-642-13226-1_6 http://doi.acm.org/10.1145/1571941.1572025
7. Kazjon Grace, Rainer Wasinger, Christopher Ackad, 13. Feng Tang, Daniel R. Tretter and Chris Willis. 2011.
Anthony Collins, Oliver Dawson, Richard Gluga, Event classification for personal photo collections.
Judy Kay, and Martin Tomitsch. 2013. Conveying In Proceedings of Conference on Acoustics, Speech
interactivity at an interactive public information and Signal Processing (ICASSP), 877-880.
display. In Proceedings of the Symposium on http://doi.acm.org/10.1109/ICASSP.2011.5946544