In this digital age, tourism data on the Internet grows massively. The huge amount of data can be utilized to gain value propositions in smart tourism. Japan, as a major tourism destination worldwide, has a number of organizations that actively promote tourism through various social media sites. Discovering emerging topics of trends in tourism from social media platforms is a challenging task, especially in an unsupervised manner. The presented research aims to discover important tourism topics from social media in Japan using a topic model. The data was obtained from twelve tourism agencies' Twitter accounts. 21,766 tweets and retweets were collected for the period of four years from 2016 to 2019. A topic model was built using the Latent Dirichlet Allocation (LDA) method. The popular topics obtained reveal most discussed issues posted by tourism agencies in Japan. These topics include, for example, trip guides, culinary experience, and the cherry blossom season. The topic classification from this study provides with insights of Twitter usage promoting tourism across Japan.
The growth of social media activities and user-generated content on the Internet creates an opportunity for the
tourism industry. With the support of information technology, smart tourism can be implemented by transforming
huge amounts of data into value propositions
Japan is one of the most popular tourist destinations in the world. Based on The Travel & Tourism
Competitiveness Report 2019
There are many organizations or agencies promoting tourism in Japan through various activities. The Japan
National Tourism Organization (JNTO) is one of the leading tourism agencies. Established in 1964, it was initially
a government-affiliated corporation before it was changed to become an independent agency
Analyzing tourism information data stemming from social media helps to get a better understanding about tourism trends. As one of the most popular social media networks, Twitter is a real-time microblogging service that allows users to post a status update (tweet) in 280 characters. JNTO’s official Twitter account (@Visit_Japan) has been active since 2009 and has more than 100,000 followers. In addition to JNTO, there are other tourism organizations that utilize Twitter to promote tourism in Japan. The tremendous number of tweets yields various topical engagements of tourism. Although many studies explored the social media content, there is still limited research on topics of tourism, especially in Japan. Discovering important topics would provide valuable insights for people to illustrate tourism trends and images. In this study, a topic modeling method is proposed to obtain the most important and valuable topics dealing with tourism in Japan. The Twitter data retrieved from tourism agencies in Japan was used to build a topic model. Topics generated from the data, based on the model, are evaluated and discussed through the study. The goal of this study is to discover emerging topics of trends in tourism in Japan, using reviews available on Twitter. Discovering popular topics of trends from a text document (e.g. tweet) in an unsupervised manner is a challenging task. LDA is a method applied in this study to cluster text documents in an unsupervised manner.
The rest of the paper is organized as follows. Section 2 describes the related work on social media analytics and topic modeling. The proposed method is presented in Section 3. Section 4 specifies the data used in the research. Section 5 outlines the experiment, while results obtained are given in Section 6. Finally, Section 7 briefly discusses the results and draws conclusions.
The usefulness of social media analytics to support smart tourism was discussed in several studies.
A relatively novel approach to analyze social media data is topic modeling. Topic modeling is a statistical
method applied to discover latent themes and trends in a collection of text documents. Social media data, such as
tweets, often can be characterized as text documents containing latent themes.
The concept of a system proposed to discover and analyze tourism topics from social media is illustrated in Figure 1.
First, relevant data (English texts) for is collected from Twitter accounts of Japan tourism agencies. The data is then combined into one dataset by aggregating tweets from all the relevant Twitter accounts. Before topic modeling is applied to cluster the documents, the data needs to be pre-processed to remove unnecessary elements (e.g. punctuation marks, numbers, Twitter symbols, URLs, and stop words). Non-English tweets are also removed. Each tweet is sliced into separate words (tokenization). Each tokenized word is then converted into its dictionary form (lemmatization). After finishing text preprocessing, the word dictionary is used to produce a document-term matrix. This document-term matrix is used as the input to build a topic model. 84 Tourism tweets
Preprocessing
Clusters of words by topics
Clusters of documents by topics
An LDA topic modeling method was deployed in this study. LDA builds on a generative probabilistic model
to cluster discrete data, such as a text corpus
Twitter data was collected from Twitter accounts of tourism agencies in Japan. The respective Twitter accounts were filtered, based on time and activity. Specifically, accounts that have been on Twitter for at least four years, have more than 1,000 followers, have posted more than 500 tweets over the last four years, and use English were selected. It is important to note that the selected accounts keep regularly posting tourism information, so that the analysis would reflect the current tourism trend. Some of the accounts are managed by JNTO, as it has several accounts for branch offices overseas (e.g. @visitjapanuk in London, @jntocanada in Canada). @JapanSafeTravel is JNTO’s newest Twitter account, providing foreign visitors with safety tips and latest information in the case of a natural disaster. Although the latter account exists for only one year and the number of tweets is still less than 500, it already has more than 20,000 followers, and can provide for a variety of additional topics. This account was therefore selected as well.
Since 2009 2018 2010 2010 2008 2011 2011 2014 2014 2014 2015 2012 # followers 101,696 21,358 10,311 2,446 24,789 6,093 9,734 3,830 3,014 4,741 14,500 17,853
The data was collected using the Twitter Python API for the period of four years from 2016 to 2019. The Twitter accounts used, the number of followers, and the number of tweets obtained from each account are given in Table 1. The number of tweets collected from the twelve accounts sums up to 21,766. The frequency of tweets during this period was also analyzed over time (see Figure 2). As one can see from the figure, there is a decline in the number of tweets posted after Q4 2016. In 2017 and 2018, the number of tweets each quarter was less than 1,500, with the lowest number of 998 in Q4 2017. However, there is an increase again in 2019, reaching 1,685 tweets in Q4 2019.
After finishing text preprocessing, words that occurred many times, such as Japan, visitjapan, and Japanese, are identified by exploring the word frequency. The common words tend to be dominant, and would negatively affect the topic modeling, as they would appear in every cluster. Therefore, frequent words occurring in 5% of the documents and having no distinctive meaning for a specific topic were excluded from the dictionary. Each unique term in the dictionary and its frequency were included a document-term matrix. This document-term matrix is used as an input to train the LDA model.
The LDA modeling is implemented with the Gensim package in Python. In addition to the corpus and dictionary
as inputs, the number of topics k is to be selected. This parameter was set, starting from 10 and, then, in increments
of 10, i.e. k = 10, 20, 30, …, 120 topics. As a result, there were 12 topic models built that were ranked by a
coherence score, Cv. The latter measure is calculated, based on a sliding window that uses normalized pointwise
mutual information and the cosine simila
When Cv appears to keep increasing, the model that gave the highest coherence score before flattening out is selected. With the given data, the highest coherence score was achieved for the number of topics set at 90. Such a high number of topics would however be difficult to interpret. Too few topics (e.g. k = 20 or so), on the other hand, would result in non-cohesive clusters. In an experiment with 60 topics, the topic spread appeared to be satisfactory. A model with 90 topics (the highest coherence score) was also trained, but no significant improvement in the coherence of the topics was achieved with this model, compared to the case of k = 60. LDA results with 60 topics are shown in Figure 4.
The visualization of Figure 4 was obtained, using the LDAvis interactive chart for Python. The latter system was developed by Sievart and Shirley (2014) to better understand LDA modeling. On the left-hand side, the topics 86 are presented as bubbles on a two-dimensional plane, whose centers are determined by computing semantic distances between the topics. The larger the bubble, the more prevalent the topic. An “ideal” topic model should, therefore, have only non-overlapping bubbles scattered uniformly throughout the chart. Selecting a bubble on the left panel interactively reveals the most relevant terms for the particular cluster on the right-hand side panel.
Based on the output of the LDA model obtained in the experiments, the Twitter data was clustered into 60 topics. Each document has membership degree (or weight) of each topic. Each tweet is then classified into a topic with the highest membership degree. To illustrate the output of the LDA model, the top-10 popular topics with the top5 probable words are given in Figure 5.
Topic 30 is the most popular topic of the data that includes a total of 2,135 tweets. It has five keywords in the order of their associated weights: guide, spot, thing, tokyo, and sightseeing. From these keywords, the topic would be inferred as mostly discussing the guidance to enjoy sightseeing spots in Tokyo. An example of a unique topic of Japan tourism is Topic 23: sakura, cherry blossom, spring, flower, and forecast. Japan is famous for the sakura (cherry blossom) season, which typically takes place in early April. Thus, many tourism tweets posted are related to the forecast of the full bloom of sakura.
In addition to discovering the topic keywords, an analysis of each document for what topic it contributes mostly would help interpret the topic. By reading documents within a topic, the common theme would be inferred. Three tweets were selected per topic with the highest per-topic percentage contribution. Topics 13 and 15 are mainly about the tourist guides, Topic 10 is about food and drinks, and Topic 23 is about the sakura season. Examples of tweets from the dominant topics are given in Table 2.
"Top 4 free thing to see in Tokyo: https://t.co/CueWBWngsG via @YouTube" "Stranger things: Weird ways to get festive in Japan https://t.co/FpJGILrVVf" "Visiting Roppongi, Tokyo? With so much to see and do in Roppongi here's a guide to get you started! https://t.co/iIcW5uy2gM" "Where to Eat and Drink in Kyoto | via Food & Wine" "The Best Food in Tokyo Is Eaten at the Counter https://t.co/SHCmvIzeZg via @CNTraveler" "Where to Eat and Drink in Okinawa, Japan | via Food & Wine https://t.co/OAyv3jWCaj" "Hiroyuki Ito, a photographer, spent two months documenting interesting moments across #Japan => https://t.co/fjcTcOLgIw via @nytimes https://t.co/gm1iR7uJH1" "20 easy day trips to make from Tokyo https://t.co/7FQ9ySzMBU #Japan https://t.co/3qCS5TbH3U" "Five of the best day trips from Tokyo | via Time Out Tokyo https://t.co/jE11tHK9IO" "15 Cherry Blossom Spots In Tokyo That You Just Have To See! https://t.co/SJVDFfwJCx #Tokyo #spring #sakura #travel" "Japan's Cherry Blossoms In 2019 - Forecast and Best Spots! https://t.co/8EikepUEA5 #sakura #cherryblossoms #spring" Cherry blossoms are already blooming in Atami, just south of #Tokyo! Find out more about where and when you can see these and other flowers. https://t.co/kJlenVTTZI #sakura2019 #traveling #Japan" @MATCHA_global @MATCHA_global @LIVEJAPANGuide 2 3 4
Topic 10 (food, eat, restaurant, drink, spirit)
In the presented study, topic modeling using LDA was conducted on Twitter messages from twelve Japan tourism agencies. In terms of tweet frequency within the investigated period, there is a downward trend from 2016 to 2017. However, the tweet number goes up again gradually in 2018 through 2019. This makes one wonder whether the recent increase in the number of tourism posts is related to any tourism issue. In terms of the contextual analysis, the topic model built generated 60 clusters, with three popular topics discovered being sightseeing trip guide, culinary, and sakura season. Using the clusters generated by LDA, analyzing tweets from each topic would be beneficial for business intelligence in tourism industry.
Deciding on the suitable number of topics is the most challenging part of topic modeling. If the number is set too low, some topics would be too broad, and would need splitting apart. On the other hand, if the topic number is set too high, some very similar topics that should be joined are found. Metrics, such as coherence score, can be used to help decide the optimal number of topics, but no perfect metrics exist. Research on advanced methods to select a suitable number of topics, and on semi-supervised topic modeling is left for the future.