=Paper=
{{Paper
|id=Vol-2863/paper-09
|storemode=property
|title=Location-based Reminder for Memorizing What Visitors Learn at a Museum
|pdfUrl=https://ceur-ws.org/Vol-2863/paper-09.pdf
|volume=Vol-2863
|authors=Yoshiyuki Shoji,Kenro Aihara,Martin J. Dürst,Noriko Kando,Takuya Nakaya,Hiroaki Ohshima,Takehiro Yamamoto,Yusuke Yamamoto
|dblpUrl=https://dblp.org/rec/conf/chiir/ShojiADKNOYY21
}}
==Location-based Reminder for Memorizing What Visitors Learn at a Museum==
https://ceur-ws.org/Vol-2863/paper-09.pdf
Location-based Reminder for Memorizing What
Visitors Learn at a Museum
Yoshiyuki Shoji1 , Kenro Aihara2,3 , Martin J. Dürst1 , Noriko Kando2,3 ,
Takuya Nakaya1 , Hiroaki Ohshima4 , Takehiro Yamamoto5 and Yusuke Yamamoto6
1
Aoyama Gakuin University, 5-10-1 Fuchinobe, Chuo-ku, Sagamihara, Kanagawa 252-5258, Japan
2
National Institute of Informatics, 2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo 101-8430, Japan
3
The Graduate University for Advanced Studies, 2-1-2 Hitotsubashi, Chiyoda-ku Tokyo, 101-8430, Japan
4
University of Hyogo, 7-1-28 Minatojima-minamimachi, Chuo-ku, Kobe, Hyogo, 650-0047, Japan
5
University of Hyogo, 8-2-1 Gakuennishi-machi, Nishi-ku, Kobe, Hyogo, 651-2197, Japan
6
Shizuoka University, 3-5-1 Johoku, Naka-ku, Hamamatsu, Shizuoka 432-8011, Japan
Abstract
This paper proposes a method to promote a strongly memorable museum experience by sending re-
minder notifications when visitors approach places related to the exhibits. Memory retention studies
have reported that memory that is linked to personal experience is not easily forgotten. Our method
estimates real-world places that are related to a museum exhibit by linking characteristic terms that ap-
pear in reviews for these places with exhibit commentaries. Relevance between the characteristic terms
is calculated by their co-occurrence in documents and web search query suggestions. This enables us
to send reminders about exhibits to users’ mobile devices when they approach related locations in their
daily life. Through experiments conducted at the National Museum of Ethnology in Japan, participants
who received notifications about the exhibits remembered more exhibits one week later.
Keywords
Museum, Memory Retention, Notification, Reminder
1. Introduction
People easily forget what they have learned in books, classes, and museums. The content of
this paper will also be forgotten; however, this paper proposes a technique for remembering it.
As the saying “Example is better than precept” goes, people are more likely to remember
information that is connected to their personal experiences, rather than the knowledge that
is simply seen or heard. Human memory can be roughly divided into two types: semantic
and episodic [1, 2]. Semantic memory consists of facts and knowledge which, while long term,
WEPIR 2021: The Third Workshop on Evaluation of Personalisation in Information Retrieval at CHIIR 2021, 19 March
2021. Virtual event.
" shoji@it.aoyama.ac.jp (Y. Shoji); kenro.aihara@nii.ac.jp (K. Aihara); duerst@it.aoyama.ac.jp (M. J. Dürst);
kando@nii.ac.jp (N. Kando); nakaya@sw.it.aoyama.ac.jp (T. Nakaya); ohshima@ai.u-hyogo.ac.jp (H. Ohshima);
t.yamamoto@sis.u-hyogo.ac.jp (T. Yamamoto); yamamoto@inf.shizuoka.ac.jp (Y. Yamamoto)
~ http://shoji-lab.jp/ (Y. Shoji)
0000-0002-7405-9270 (Y. Shoji); 0000-0002-4287-2523 (K. Aihara); 0000-0001-7568-0766 (M. J. Dürst);
0000-0002-2133-0215 (N. Kando); 0000-0002-9492-2246 (H. Ohshima); 0000-0003-0601-3139 (T. Yamamoto);
0000-0001-9829-6521 (Y. Yamamoto)
© 2021 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org)
79
Name of Location
“You seem to be in front of
a certain osteopathic clinic”
Name of Exhibit
“Please think about Chikunza
right here!”
Snippet of Exhibit Description
“This is because the description refers
to a liturgical play of a dancer
wearing the mask to treat boys”
Snippet of Place Review
(Omitted due to character limit)
“And the review of this location says,
I had my body taken care of here”
Figure 1: Sample of notification when a user approaches an osteopathic clinic.
are difficult to remember. Episodic memory consists of personal experiences which are not
long-term on their own, but, because they are connected to personal experiences, are easier
to remember and recall. Such experience-based episodes are likely to be recalled over and
over again. Thus, an important episode can become a semantic memory and, finally, personal
knowledge.
To make what visitors see on-site at a museum or in a museum catalog more memorable, we
propose a method that strongly connects exhibits to their personal experience. Museums are
extraordinary and not connected to our personal lives. As a result, there is little opportunity to
recall exhibits in our daily life. Our method focuses on geographical entities (e.g., landmarks,
stores, restaurants) to connect exhibits with experiences in the real world.
Imagine that a visitor visited a museum to learn about ethnology. He or she was interested
in a morin khuur (a Mongolian fiddle) in the museum. However, the knowledge of “what is a
morin khuur and what are its characteristics” does not make concrete sense and, after a while,
maybe forgotten. Therefore, when a visitor nears an apparel store, a notification is sent to his or
her smartphone that says, “You are now visiting the shop of an apparel brand that is famous for
having a horse as its logo. The horse is actually closely related to the morin khuur! The morin
khuur is shaped like a horse’s head, and there is a founding myth about it.” By receiving such a
notification, the existence of the morin khuur is connected to the individual’s experience and,
therefore, this visitor will be more likely to remember this experience. In the future, when he or
she passes by this apparel store, they may recall and think about the morin khuur. In this way,
80
the location-based reminder notification makes the museum visit and exhibit more memorable.
This kind of attempt to fix textbook knowledge into memory by linking knowledge to places
or experiences is already common in education. One of the most typical examples is elementary
school field trips; students walk around a city with a teacher and are introduced to objects
related to what they are learning in school. For example, the meaning of foreign terms on a
signboard can be linked to language learning and the color of hydrangea flowers can be linked
to pH (power of hydrogen) in science experiments. These methods are in accordance with the
Meaningful Reception Learning theory [3].
Our method analyzes the user behavior log of a museum guide application and estimates the
exhibits that the visitor is interested in from the log. For each of these exhibits, the method
extracts keywords that characterize the exhibit from its description. It also collects review
texts from review sites for all places in a specified area. The relevance between the terms in
the exhibit’s description and the reviews are calculated, and a notification is created. Visitors
can receive notification messages on a mobile device (e.g., smartphone), such as “You are in
an apparel store right now. Actually, this store is associated with the morin khuur because the
morin khuur is shaped like a horse’s head and a review of the store mentions a horse’s head.”
Figure 1 shows an actual example of our system in which a notification about a mask was
sent to users who approached an osteopathic clinic. This mask (called a Chikunza) is worn by
a dancer of an African tribe who is in charge of supporting boys in a liturgical play. This is
because terms such as “dancer and treat” are associated with the terms “body and care” in the
osteopath’s review.
One of the aims of this research is to propose an idea that evaluates the quality of information
access in museums by measuring how much of the visiting experience is retained in the visitor’s
knowledge. For this purpose, we created an application that actually aids memory retention.
To verify the effectiveness of our method, we conducted an experiment at the Museum of
Ethnology in Japan (called “Minpaku”). We used the operation log of the “Minpaku Guide”: an
iPad guide app for the museum originally developed by us. Participants received notifications
and were asked whether they remembered the exhibits they saw at the museum one week later.
2. Related Work
This research is strongly related to studies on memory retention and museum experience, and
uses the technology of geographical information recommendations and related term extraction.
Research on memory in the field of psychology has been actively conducted for a long time.
Bower et al. [4] proposed that people remember things longer when they associate them with
their own memories. Similarly, Weinberg et al. [5] showed that when people move their bodies
during an experience, it is more likely to be remembered. The traditional mnemonics method of
loci [6] also supports the usefulness of linking a place and a topic. Memory retention has also
been studied in the field of informatics. For example, Qiu et al. [7] discuss how user interaction
in information retrieval affects the memorability of the search results. Our research also aims
to make the results of information access in museums and guide apps more memorable through
physical experience.
There has already been some research on making museum experiences more memorable.
81
Falk et al. [8] state that to make museum experiences knowledgeable, it is important to connect
the visiting experience with personal experience and to incorporate it into personal contexts.
Petrelli et al. [9] are conducting research on encouraging people to look back on their museum
experience by providing personalized online souvenirs.
The recommendation of places based on the user’s interests and preferences has also been
widely studied. Koren et al. [10] proposed an audio guide that provides personalized recom-
mendations of locations based on the user’s interests.
To link the learning content to geographic entities, the method must calculate the relevance
between two keywords. As a concept-based method, Watabe et al. [11] calculated the relevance
between two terms from the co-occurrence in the concept space. Shirakawa et al. [12] uses
Wikipedia; they proposed a method to express the relationship between two terms using the
Wikipedia Thesaurus and combining it with Web searches. In this study, our method follows
these methods to link the terms in the descriptions of exhibits with those in a place’s review.
3. Method
This section describes our method of linking exhibits and geographical entities. The method
consists of three steps:
1. extracting characteristic terms from the text of the exhibit’s commentary and the place’s
review,
2. calculating the relevancy between characteristic terms, and
3. sending reminder notifications to a smart device.
First, our method extracts characteristic terms from the exhibits and place reviews. We
used term frequency-inverse document frequency (TF-IDF) for this extraction. Sentences were
divided into terms by a morphological analyzer (terms in Japanese sentences are not set off by
spaces), and verbs and nouns were extracted. The frequency with which a term appeared in
all the descriptions and in all the reviews was counted, and the terms that characterize certain
exhibits and places were ranked by the TF-IDF score.
The method then calculates the relevance of a combination of characteristic terms of exhibits
and characteristic terms of reviews for places. We calculated the relevance based on three
hypotheses: co-occurrence, category distance, and query suggestion. The co-occurrence score
is based on the hypothesis that related terms are more likely to appear in the same document.
For this purpose, we calculated the percentage of terms that co-occur in all Wikipedia articles.
The category distance score is based on the hypothesis that terms that are semantically close
to each other are also close in taxonomy. The method uses category information taken from
Wikipedia. The score of a term contained in the title of a Wikipedia article was calculated based
on the degree of overlap of the category to which these articles belong. The query suggestion
score is intended to be slightly different from the other relevance scores; it takes into account
the amusingness and popularity of the combination. To do this, we hypothesized that popular
combinations of terms would appear in the users’ search queries. Therefore, the method sets
the score as 1 if the suggested query of the search engine for the term contained another term;
otherwise, the score is set as 0.
82
During the actual calculation, these three values are weighted and summed up. For the
experiment, the co-occurrence weight was set to 0.2 and the category distance weight was set to
0.8. However, if a term was included in the query suggestion, the final score was unconditionally
fixed at 1.0. These weights and conditions were determined empirically through preliminary
experiments. It is rare for a term to accidentally appear at the top of a query suggestion and,
when it does, the combination is meaningful.
Finally, the method actually links the place to the exhibit by aggregating the degree of
relevance between their characteristic terms. In this experiment, places were targeted for
notification if even one term with a high degree of relevance was included in the review. Each
exhibit has a ranking of which sentences in the review it is deeply related to. For the top 10
exhibits that the visitor was interested in, the system decides in advance which sentences will
be displayed when the user approaches the location.
Every review is linked to a real place and these places have location information (i.e., latitude
and longitude). Using the GPS information of the user’s mobile device, the geofence function of
the smartphone displays a notification when the user comes within a 100-meter radius of the
target location. In the application for this experiment, we used the local notification function
without accessing a server because we were able to pre-calculate where to send the notification
in advance.
As shown in Fig. 1, the title of the actual notification message includes the name of the place,
the name of the exhibit, and an image of the exhibit. The aim is to assist memorization by
using paired-association-like learning, such as when someone says “clinic”, you say “Chikunza”.
The reason why this place was recommended is presented under the title. Single sentences
containing the relevant characteristic term are extracted from the exhibit description and the
place review text, and each is displayed as a snippet. The notification text is written in a catchy,
broken tone. Tapping on the notification displays the full text of the omitted snippets and a
higher resolution image. The user can select one of two buttons, “Aha” or “Not interested”, and
they can also choose to ignore the notification.
4. Experiment
We conducted two experiments to evaluate whether reminders affect visitor memory retention.
The first was a detailed comparison to clarify whether notifications help users retain textbook-
like knowledge. Participants did not go to the museum site, but rather used the app as a
catalog in the lab. The other was a field user experiment that evaluated the degree of memory
retention and operational problems when such a system was actually used at the museum site.
In both of these experiments, participants went out in the city and received notifications (except
participants in the control groups), and one week later were asked and answered how many of
their favorite exhibits they remembered.
4.1. Dataset
For the experiments, we used the exhibit data of the National Museum of Ethnology (Minpaku),
Japan. We used Minpaku Guide; an iPad app originally developed by us for navigating in the
museum. Participants can easily search and browse detailed information on 3,031 exhibits that
83
are sampled from over 10,000 exhibits at the museum. This app can be used as a guide on-site
at the museum or as a catalog at home. The app collects user browsing logs and explicitly
lists their favorite exhibits. These features were used to determine which exhibits to use for
reminders.
As place data, we collected online reviews for locations from Google Maps: 7,393 places in
a city in the western Tokyo area (i.e., near to the first author’s laboratory) and 18,165 places
in a city in the Kobe area (i.e., near to the museum). Both areas are two kilometers square and
include the downtown area around a terminal station. All places in the dataset have one or
more text reviews, not just star ratings.
4.2. Comparison Experiment at Laboratory
To evaluate whether notification is effective for knowledge retention, we conducted a subject
experiment. We divided eight participants into three groups:
• in City: Four participants walked through the city and received notifications,
• in Lab: Two participants stayed in the laboratory and received notifications while indoors,
and
• no Reminder: Two participants received no notifications at all.
All participants operated the guide application freely for one hour in the laboratory. After that,
the top 10 exhibits with the highest number of accesses were extracted from the logs. We lent
iPhones to two groups of participants, in City and in Lab. The in City participants went to
the downtown area in front of a station and walked through the area for 60 minutes. During
this stroll, they were sent notifications when they approached places related to their favorite
exhibits. The in Lab participants spent their free time indoors. Regardless of the participant’s
behavior or current location, the participants received notifications at random times. The no
Reminder participants used the catalog app and then went home without doing anything.
One week later, we checked how many of the top 10 items the participants had viewed
were remembered. We performed this check in two stages. The first step was to answer the
question without hints; the participants responded by listing all the exhibits they remembered.
The second was with hints; the participants were first asked to “recall the location where you
received the notification.” After that, the participants provided additional exhibits that they
could recall by associating them with the location.
The results of the experiment are shown on the left side of Table 1. The participants in the in
Lab group remembered the most exhibits and there were exhibits that they were able to recall
with hints.
4.3. User Case Study Onsite at Museum
As a more realistic problem setting, we conducted an additional experiment that sent notifica-
tions while participants were walking around the city after an onsite visit to the museum. Five
participants visited the National Museum of Ethnology and appreciated its exhibits for about
two hours. They used the Minpaku Guide and selected 15 exhibits by using the app’s favorite
list function. Ten of their favorite exhibits were targeted for notification. A week later, while
84
Table 1
Results of the two memory experiments, including the average number of exhibits for each condition,
and the ratio of exhibits remembered and received notifications.
Comparison at Laboratory Museum
in City in Lab no Reminder in City
Number of notifications 8.25 7.00 - 9.80
Number of exhibits remembered without hint 5.75 5.00 5.50 5.80
Number of exhibits remembered with hint 0.25 2.50 - 0.20
Total number of exhibits remembered 6.00 7.50 5.50 6.00
Ratio of remember / notification 0.57 0.50 - 0.61
walking around the city for another event, our system sent reminders of the exhibits to their
iPhones. After an additional week, we asked what exhibits they remembered. The aim of this
experiment was to disclose problems with and improvements that can be made to the system
when it is operated in the field. For this purpose, we conducted a detailed questionnaire on the
system’s usability.
The results of the experiment are shown on the right side of Table 1. The ratio of the exhibits
that were remembered of the 10 exhibits for which notifications were sent was 0.61 and 0.31 for
the five exhibits without notification. Two weeks had passed since the visit, so the results cannot
be directly compared with the results of the previous experiment; however, most participants
remembered half of the exhibits. Similar to the comparison experiment, providing hints proved
effective for recall. As to usability, we obtained opinions such as “it is annoying when multiple
exhibits are notified in the same place” and “when notifications are displayed in rapid succession,
the previous notification became inaccessible.”
5. Discussion
Overall, reminding by notification seemed effective for memory retention. In the comparison
experiment, the number of exhibits remembered by the participants differed greatly depending
on whether they received a notification or not. On the other hand, we could not confirm the
effectiveness of linking the notification to places. The participants who received notifications
while indoors remembered the most exhibits. This may be due to the experimental setting; the
participants in the in City group were forced to walk around the city, which is more unusual
than remaining in an indoor setting. These participants may have been so busy or distracted
while walking around the city that they did not read the notifications carefully.
In the case of recall with hints, the participants commented that inappropriate or unexpected
combinations helped their recall. The degree of relevance between characteristic terms is
important, but the unexpectedness of the relationship between exhibits and place may also be
influential. The participants pointed out cases where the quality of the review text was low and
unrelated to the place. Prior cleansing is therefore needed.
Regarding the user case study, the frequency of notifications is an issue that needs to be
resolved. In this experiment, the notifications were not grouped by location, but were handled in
85
units of sentences. As a result, several exhibits were linked to one place and several notifications
were issued in close proximity. In actual operation, it is necessary to select the target place so
that notifications are sent out at equal intervals throughout the city.
6. Conclusion
In this research, we proposed a method to help memory retention by linking real places to
museum exhibits and sending out notification reminders. The experimental results showed that
the reminders increased the number of exhibits remembered by the participants one week later.
On the other hand, we were not able to prove the effect of linking it to the experience of going
out into the city.
As a future challenge, larger-scale and more sophisticated experiments are needed. We should
conduct comparative experiments at a museum site and in a more natural environment to send
notifications. Currently, the participants are forced to walk around the city in the experiment
setting. This situation is unnatural and cannot connect the exhibits to their real, daily lives.
Personalization is also an important issue. There was a bias in the areas of interest among the
participants and this affected memory retention. The calculation of the relationship between
local objects and exhibits also needs to be made more precise.
In closing, please say aloud “I read a paper about sending a notification of an exhibit
at the station!” the next time you go to the station. This will convert the content of this paper
into an experiential memory for you so that it will not be easily forgotten.
Acknowledgments
This work was supported in part by JSPS KAKENHI Grant Numbers JP18H03494, JP18H03243,
JP18K18161, JP17H00762, and JP16H01756. It was also supported by the NII Open Collaborative
Research 2020 Program, numbers 20S1001, 20AD03, and 20AD04.
We sincerely thank the Japanese National Museum of Ethnology (Minpaku) for offering the
exhibit meta-data.
References
[1] E. Tulving, Elements of episodic memory, Oxford Psychology Series (1983).
[2] E. Tulving, What is episodic memory?, Current directions in psychological science 2 (1993)
67 – 70.
[3] D. P. Ausubel, Meaningful reception learning and the acquisition of concepts, in: Analyses
of concept learning, Elsevier, 1966, pp. 157 – 175.
[4] G. H. Bower, S. G. Gilligan, Remembering information related to one’s self, Journal of
Research in Personality 13 (1979) 420 – 432.
[5] L. Weinberg, A. Hasni, M. Shinohara, A. Duarte, A single bout of resistance exercise can
enhance episodic memory performance, Acta Psychologica 153 (2014) 13 – 19.
[6] F. A. Yates, The art of memory, volume 64, Random House, 1992.
86
[7] S. Qiu, U. Gadiraju, A. Bozzon, Towards memorable information retrieval, in: Proceedings
of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval,
2020, pp. 69 – 76.
[8] J. H. Falk, L. D. Dierking, Learning from museums, Rowman & Littlefield, 2018.
[9] D. Petrelli, M. T. Marshall, S. O’Brien, P. McEntaggart, I. Gwilt, Tangible data souvenirs
as a bridge between a physical museum visit and online digital experience, Personal and
Ubiquitous Computing 21 (2017) 281 – 295.
[10] A. Koren, N. Stash, A. Andreev, A proposal for semantic recommender for outdoor audio
tour guides, in: Workshop on Personalization in Mobile Applications at the 5th ACM
Conference on Recommender Systems. Chicago, IL, USA, 2011, pp. 1 – 4.
[11] H. Watabe, N. Okumura, T. Kawaoka, The method of measuring the degree of association
between concepts using attributes of the concepts and coincidence information, Journal
of Natural Language Processing 13 (2006) 53 – 74.
[12] M. Shirakawa, K. Nakayama, E. Aramaki, T. Hara, S. Nishio, Relation extraction between
related concepts by combining wikipedia and web information for japanese language, in:
Information Retrieval Technology, Springer, 2010, pp. 310 – 319.
87