=Paper=
{{Paper
|id=Vol-3222/paper9
|storemode=property
|title=A Cosmetic Differences Visualization System for Beauty Recommendation using the Scores of Various Evaluation Items
|pdfUrl=https://ceur-ws.org/Vol-3222/paper9.pdf
|volume=Vol-3222
|authors=Mayumi Ueda,Sayaka Yabe,Da Li,Shinsuke Nakajima
|dblpUrl=https://dblp.org/rec/conf/recsys/UedaYLN22
}}
==A Cosmetic Differences Visualization System for Beauty Recommendation using the Scores of Various Evaluation Items==
A Cosmetic Differences Visualization System for Beauty Recommendation using the Scores of Various Evaluation Items Mayumi Ueda1 , Sayaka Yabe2 , Da Li3 and Shinsuke Nakajima4 1 University of Marketing and Distribution Sciences, 3-1 Gakuen-nishimachi, Nishi-ku, Kobe, Japan 2 Kyoto Sangyo University, Kamigamo Motoyama, Kita-ku, Kyoto, Japan 3 Fukuoka University, 8 Chome-19-1 Nanakuma, Jonan Ward, Fukuoka, Japan 4 Kyoto Sangyo University, Kamigamo Motoyama, Kita-ku, Kyoto, Japan Abstract In recent years, shopping habits have changed fast during the last decade and a high percentage of consumers now shop online. Due to the impact of COVID-19, not only clothes and food, but also the demand of cosmetics online purchases is growing. However, since the evaluation of cosmetic items differs depending on the user’s skin type and preference, it is difficult to understand the feeling of a cosmetic item on one’s own skin from others’ reviews. Many people prefer to buy cosmetic items only after confirming the physical product. In this study, we developed a visualization system that enables users to easily understand the features of cosmetic items by providing a visual representation of the differences between the features of items they own and those they are considering purchasing. We are also developing a cosmetic item recommendation system based on this visualization method. Finally, we verify the validity of the developed visualization system, and report the results of evaluation experiments. Keywords Cosmetic Item, Review Analysis, Visualization System, Beauty Recommendation 1. Introduction In recent years, there have been many sites on the Internet that provide information on products, stores, etc., including not only photos and descriptions, but also reviews from purchasers [1]. As our research target of this paper, many users use online shopping review sites (e.g., @cosme1 ) when they considering to purchase cosmetic items. The product page of a conventional cosmetics website includes the customer’s overall eval- uation of the product and a review text, etc. In order to understand the detailed evaluation of the product, users have to read the whole review textual information [2]. Therefore, in the previous studies, an automatic scoring system for cosmetic product reviews by evaluation item was developed for understanding detailed evaluations without reading the reviews [3] [4]. These systems analyzed reviews of cosmetic items and automatically assigned a score for IntRS’22: Joint Workshop on Interfaces and Human Decision Making for Recommender Systems, September 22, 2022, Seattle, US (hybrid event) $ Mayumi_Ueda@red.umds.ac.jp (M. Ueda); syk.3838.yb@gmail.com (S. Yabe); lida@fukuoka-u.ac.jp (D. Li); nakajima@cc.kyoto-su.ac.jp (S. Nakajima) © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop CEUR Workshop Proceedings (CEUR-WS.org) Proceedings http://ceur-ws.org ISSN 1613-0073 1 https://www.cosme.com/ each evaluation item, such as moisture and color. However, the scores by evaluation item are subjective to the purchaser, and it cannot be said that if a user purchases a cosmetic item after seeing the score by evaluation item and actually uses it, she/he will feel that the score (feeling) is same. Therefore, this study aims to develop a system that visualizes the differences between items that users have already purchased and items they are considering purchasing, as well as a system that recommends ideal items to users using this visualization method. We describe our cosmetic differences visualization system for beauty recommendation using the scores of various evaluation items, and report on an evaluation experiment that verified the usefulness of the proposed system. The remainder of this paper is structured as follows. In Section 2, we discuss related works. Afterward, in Section 3, we describe the proposed method for a visualized comparison of the differences between cosmetic items based on their various aspects. Section 4 discusses the results attitude survey questionnaire of beauty online shopping. Then, we introduce the user interface of proposed system in Section 5. Section 6 discusses the experimental results. Finally, in Section 7, we conclude this paper and discuss future works. 2. Related Works An increasing number of websites provide reviews of products and shops [5]. Famous websites such as Rakuten2 , which lists various types of products, the restaurant-focused Tabelog3 , and the travel-focused Jarannet4 and so on. Specifically, the number of websites specializing in cosmetics and beauty increases every year. One of the most popular sites is @cosme. The management of cosme has stated that it had 6,100,000 members as of August 2020. Furthermore, e-commerce sales were twice as high as the previous year due to the coronavirus disease pandemic 2019 [6]. Many consumers use cosme to search for cosmetic information. With the increase in the number of review sites, the volume of research into product reviews has increased [7]. Our previous research was aimed at developing a system that automatically scores each evaluation item from the body of review text [3] [4] [8]. Automatic scoring requires a separate dictionary for each category of cosmetics. As it is time-consuming to build such dictionaries of all categories of evaluative expressions, Taniguchi et al. are working on a design for automatically constructing a dictionary of evaluative expressions [9]. Kanayama et al. are working on a system that will extract remarks about product reputation from an enormous database of information in a free question-and-answer format using unre- stricted, descriptive sentences such as that provided by a questionnaire to uncover the essential characteristics of the product and check the writer’s intention [10]. Yao and others conducted a survey to examine the impact of reviews on the purchasing behavior of consumers [11]. To confirm which type of reviews can be trusted, they conducted a hypothesis verification. The results of this process conflicted with their expectations, but a variance analysis shows that reviews do have an impact on purchasing behavior. 2 https://www.rakuten.co.jp 3 https://tabelog.com/ 4 https://www.jalan.net In addition, many researches on the visualization of reviews have been conducted. Fushimi et al. have developed a method to visualize evaluation documents such as reviews of products radially according to items such as the intention, topic, and evaluation viewpoint expressed by the documents, and concentric circles according to the scores of products submitted with each evaluation document, in order to intuitively grasp the overall image of product evaluation and to improve accessibility to the evaluation documents [12]. Oziom et al. are developing a map display system for shopping lists to make it easier to identify desired products and where they can be found when shopping for groceries [13]. This system displays a map of the store on a smartphone screen with the icon of the product you want to buy and its location. It lets the shopper know at a glance what items are still on their shopping list and where to find them, which reduces shopping time. Thus, studies on review analysis and review visualization have been conducted. However, few studies have been conducted on the visualization of reviews of cosmetics sites. In addition, many studies have analyzed reviews of a single product or restaurant, and there have been few studies comparing reviews of one product to others [14] [15]. Especially in the case of cosmetic items, many of them seem to be similar in color and texture just by appearance, and it is more significant to show differences by comparing them with reviews of other items rather than analyzing reviews of a single item. 3. Comparative Method based on Visualization In this section, we describe our recommendation system and summarize the method used to compare cosmetics based on the visualization of cosmetic characteristics. 3.1. Overview of the Comparative Method based on the Visualization of Cosmetic Items Figure 1: Concept of the proposed system. Recently, consumers have begun to search for useful information from review sites, social media, and YouTube before they buy cosmetic items. The photos and reviews provided through these services are helpful to decision making but, because the perception of cosmetic items is different for each user, the conventional method of accessing only basic information on the Internet is not easy or suitable for buying cosmetic items (see Figure 1). The right side of Figure 1 summarizes our proposed method. Our system can compare the features of cosmetics that users are thinking of buying with the user’s regular products. Thus, users can use the information from the Internet to understand easily a feature of cosmetics that they are considering buying. 3.2. Calculation of the Score of Each Item using the Automatic Scoring System Figure 2: Automatic scoring system. In this section, we describe a system for assigning scores for various characteristics (aspects) from cosmetic reviews; this has been outlined in our previous studies. We also explain how to calculate the score for each aspect of an item, based on previous studies. Figure 2 shows our scoring method for cosmetic reviews based on the evaluation expression dictionary. The system analyzes the text of each cosmetics review and identifies expressions for evaluation that are present in the review. Consequently, the system assigns a score to each expression according to the dictionary. If there are several expressions for a particular aspect, the system gives the average of the scores to the aspect (shown in Figure 2). For example, in Figure 2, there are two expressions for “moisturizing aspect” in the review – our system gives the average score for “moisturizing aspect.” Previous systems assigned a score to each expression in a review. This paper describes how our system scores each item. Our proposed method calculates the average of scores for each aspect for each product. 3.3. Visualizing Method based on the Differences between Owned Items and Items under Consideration In this section, we describe a visualization method based on a comparison of the features of owned cosmetics and cosmetics under consideration. Figure 3 shows the concept of our system. Figure 3: Visualizing system of differences between purchased items and items under consideration. “Owned cosmetics” in Figure 3 are the cosmetic items that buyers have purchased and whose features they are familiar with. “Cosmetics of interest” in Figure 3 is the cosmetic item that they are considering for purchasing. Our system use the automatic scoring method described in our previous study [3]. Our basic method, shown in the left side of the Figure 3, helps users be aware of the differences between potential purchases and an owned cosmetic. However, our system also provides a graph that helps a user visualize a more accurate picture of the differences, shown as a speech balloon in Figure 3. The score of various aspects of “lipstick C” is set to zero, and each bar shows the differences between the scores of “lipstick R” and “lipstick C”. For example, the score of “Long-lasting” aspect is 1.2. It is the score of “lipstick R (3.2)” minus the score of “lipstick C (2.0)”. If the score of “lipstick R” is higher than “lipstick C,” our system displays a red bar; if the score is lower, our system shows a blue bar. Using our system, a user can see the differences between their item and potential purchases at a glance. As users know their cosmetics well, they can make an informed decision about the target items by viewing this graph. 3.4. Recommendation System of Cosmetic Items Based on the Input of the Ideal Scores In this section, we describe the concept behind our recommendation system of cosmetic items based on the input of ideal scores. Figure 4 shows the basic concept of our system. First, users set the desired characteristics of their cosmetic in the upper left side of Figure 4. However, it is difficult to set ideal characteristics from scratch, so our system provides potential aspects with the option “Choice from your cosmetics” in the figure. This action shows the list of cosmetic items; then, users can select the item that they want to use as the base item for the ideal scores. Then, the users can set the ideal scores by fine-tuning the scores. As a result of the search based on the ideal scores determined by the user, our system recommends suitable items for the user as shown in the upper right side of Figure 4. The Figure 4: Recommendation system of cosmetic items based on the input of the ideal scores. recommended items are displayed in the order of their cosine similarity between the values set by the user and those of the selected item, as described in our previous study [3], in order of their degree of similarity. In addition, clicking on the name of the recommended item takes the user to the product details page, where a graph is displayed comparing the desired characteristics and those for the selected item. While browsing the recommended items, the user can see these differences. In this figure, the user finds that a slightly lighter and more matte lipstick than the recommended item would be ideal; hence, the ideal scores of the corresponding evaluation aspects are modified as shown in the lower left corner of Figure 4. Even though a user may not understand the ideal item scores and finds it difficult to set their ideal scores, the user’s true ideal scores can be determined interactively; the user can modify the ideal scores as needed while browsing. We are currently at the stage of implementing the recommendation system as described here. When the user clicks on a product name, the system goes to a web page with detailed information for the chosen lipstick and displays the comparison graph shown in Figure 3. 4. A QUESTIONNAIRE TO SURVEY THE IMPORTANT ASPECTS OF COSMETIC ITEMS 4.1. Survey Purpose In order to determine which item is the most ideal using the system introduced in Section 3, information such as color development and product description is necessary in addition to the comparison graph. In conventional systems, such information is often provided on the product detail page. In the proposed system, we have tried to make it easier to compare cosmetics by adding information for each item to the search result list page, as shown in Figure 5. Therefore, we conducted following questionnaire to understand what information users value when searching for cosmetics on the Internet. Figure 5: Lipstick ranking screen for survey. 4.2. Survey Details Google Form was used to create this survey. The questions are as follows: Q1 Age Q2 Frequency of using cosmetics review sites Q3 Information you would like to see in the blue portion of Figure 5 where the Q2 has four choices, “Usually,” “Sometimes,” “Seldom,” and “Never.” For Q3, we asked respondents to select two of “product description,” “manufacturer,” “number of color variations,” “names of colors,” “release date,” “number of reviews,” and “others.” We obtained 185 responses on the survey in one week. Table 1 [By age group] Information requested in the blue box as shown in Figure 5 by participants. All Teens 20s 30s 40s 50s Product description 60.5% 35.3% 61.5% 58.3% 71.0% 53.6% Manufacturer 43.2% 52.9% 20.5% 47.2% 45.2% 60.7% Number of color variations 17.3% 35.3% 12.8% 13.9% 14.5% 25.0% Names of colors 38.9% 35.3% 41.0% 36.1% 33.9% 46.4% Release date 5.4% 0.0% 5.1% 2.8% 6.5% 10.7% Number of reviews 38.9% 52.9% 46.2% 50.0% 33.9% 21.4% Other 9.2% 11.8% 12.8% 5.6% 8.1% 10.7% 4.3. Survey Results and Consideration Figure 6: Frequency of using cosmetics review sites. We had 185 respondents with 17 in their teens, 39 in their 20s, 36 in their 30s, 62 in their 40s, 28 in their 50s, and 3 in their 60s and older. As there were very few responses from people 60 years and older, the results of the survey for only those aged by 59 are discussed. Figure 6 shows frequency of use of cosmetics review sites, the results of Q2. According to these results, more than half of the respondents in their teens, 20s, and 30s use cosmetic review sites “Usually” or “Sometimes.” People in their 40s are also familiar with cosmetic review sites, because the percentage for those in their 40s is somewhat high, 44%. In addition, Figure 6 also shows that the percentage of the respondents in their 50s who answered “Never” is extremely high. We can say that most of those in their 50s do not research cosmetics on online review sites. The percentage of respondents in their teens who answered “Never” is slightly high because some teenagers have not yet started using make-up. Table 1 shows the results of Q3 by age group, and Table 2 shows the results by frequency of Table 2 [By the frequency of using review sites] Information requested in the blue box as shown in Figure 5 by participants. Usually Sometimes Seldom Never Product description 56.0% 63.4% 69.8% 44.4% Manufacturer 32.0% 46.5% 41.5% 47.2% Number of color variations 12.0% 19.7% 17.0% 16.7% Names of colors 40.0% 35.2% 43.4% 38.9% Release date 0.0% 7.0% 3.8% 8.3% Number of reviews 64.0% 43.7% 32.1% 22.2% Other 8.0% 8.5% 9.4% 11.1% use of cosmetic review sites. Both tables show that the product description is quite important and that the release date is less important. According to Table 1, the percentages of respondents who chose “Manufacturer” in their teens and 50s are quite high. Figure 6 indicates that the number of respondents in their teens and 50s who replied “Never” for using review sites is high. We can conclude that people who are not familiar with review sites focus on the manufacturer when they are looking for cosmetics. Table 1 also shows that the percentages of respondents who wanted to see “Number of reviews” are higher in their teens, 20s and 30s than others. According to Table 2, respondents who often use cosmetic review sites consider “The number of reviews” to be important. Figure 6 shows that more than half of the respondents in their teens and 20s referred to review sites. Thus, the results of the number of reviews in Table 1 agree with those in Table 2. In addition, we note that the percentage of respondents who want to see “A number of color variations” or “The names of colors” is not high, though we expected color information to be important in choosing lipstick. It is likely that many people want to see the color rather than read about it. Based on these results, we decided to provide “Manufacturer”, “Product description”, “Number of reviews”, “Representative color images”, and “Total number of colors” on the search results list page in Figure 5. 5. System Implementation In this section, we describe our proposed understanding support and recommendation system for unknown cosmetic items, which was developed based on the method proposed as described in Section 3 and the questionnaire survey as described in Section 4. The system screen is shown in Figure 7. The lipstick data was provided by istyle Inc5 . The method for setting the ideal score is described in Section 3.4. Items to be used as criteria for the ideal score can be searched by brand name, product name, or both. After setting the ideal score, users can click on the “View lipsticks that match the characteristics” button to see the recommended items. The page displaying recommended items includes product descriptions and representative colors to make it easier to compare items without moving to the detail page. Furthermore, by clicking on the name of the recommended lipstick, the user can move to the 5 https://www.istyle.co.jp/ Figure 7: Cosmetics recommendation system screen. detail page and view the difference comparison graph shown at the bottom of Figure 7, where the differences between the ideal score and the selected lipstick can be confirmed (See Section 3.3). Figure 8: Questionnaire results: Comparison with existing system (by the frequency of using review sites). 6. Evaluation 6.1. Experimental Objectives and Methods The proposed system has been developed with the goal of enabling users to search and view information more tailored to their preferences than conventional systems and to fully understand unknown cosmetics online. Therefore, in the evaluation experiment, we compare and evaluate the existing system and the proposed system to verify the usefulness of the proposed system. First, subjects are asked to use the existing conventional system for 5 minutes to search for lipstick. Next, they are asked to use the proposed system until they find the “ideal lipstick that they would like to purchase and use”. At that time, the time spent using the proposed system and the number of searches are measured. After they found their ideal lipstick, they were asked to answer the questionnaire shown in Figure 9 and 10. The questionnaire items were adapted from the questionnaire used in the study of Ookawara et al. [16]. In addition, we also asked respondents to indicate how often they use cosmetics review sites. Figure 9: Questionnaire results: Comparison with existing system (all subjects). A total of 45 subjects, ranging in age from teens to 50s, participated in the survey. The site for the evaluation experiment was constructed, and the subjects were asked to conduct the experiment at a time and date of their choice during a one-week period. 6.2. Experimental Results and Discussion The results of the comparison between the proposed system and the existing system are shown in Figure 8 and Figure 9, and the evaluation results of the proposed system are shown in Figure 10 and Figure 11. Figure 8 and Figure 11 show the average rating for each question, which was calculated by assigning a rating value between +2 and -2 to each of the choices. First, the results of the comparison with the existing system were analyzed. From Figure 9, it is clear that, more respondents answered that the proposed system is better overall. The proposed system is easier to search for their own preferences and to understand unknown cosmetics than the existing system. Moreover, figure 8 also shows the superiority of the proposed system, however, people who never use the review site had a lower average evaluation value than the other respondents. Next, we analyzed the results of the evaluation of the proposed system. From Figure 10, the results show that most of the respondents enjoyed understanding how to use the proposed system and would like to use it again. In addition, it is clear that many people did not feel stressed about the time spent on using or exploring the system. Furthermore, Figure 11 shows that more frequent users of the review sites rated the proposed system higher. Also, the average evaluation value of “Setting the ideal score is easy” for those Figure 10: Questionnaire results: Evaluation of the proposed system (all subjects). Figure 11: Questionnaire results: Evaluation of proposed system (by frequency of using review sites). who “never use” the review sites was close to “0”, indicating that setting the ideal score was not easy for review site novices. Therefore, we extracted opinions about the setting of the ideal score from the free descriptions to investigate the issues. The issues included: “Some ideal score items are difficult for beginners to understand”, “Insufficient selection method of items to be used as criteria for ideal score”, and “Little information on the slide bar”. We discovered that these issues need to be improved in the future, to make the setting of the ideal score easier for everyone. Based on the results of the above questionnaire and free descriptions, the good points and improvement of the proposed system are summarized as follows: [Good points] * Providing visual ideal score manipulation and difference comparison graphs * Personalized search by ideal score for each evaluation item * Lipsticks currently in use can be set as a criterion for ideal score * Easier product comparisons * Providing opportunities to discover unknown brands and products [Improvements] * Improvements to the ideal score setting UI to make it easier for beginners to use * Adding the function of specifying the evaluation items to be emphasized * Functions to enable exploration of cosmetics by image or abbreviation of criteria cosmetic * Adding search options such as color, price, etc. Thus, we confirmed that the proposed system enables user-tailored search and information presentation, which is useful for searching and understanding unknown cosmetics. Compared to the existing method, the proposed visualization method such as a sliding bar and graph comparing differences when setting the ideal score are more intuitive and easier to understand. In the future, we plan to improve the proposed system based on the improvements we have received, aiming to make the system more user-friendly for everyone. 7. Conclusion and Future Works In this paper, we proposed a cosmetic differences visualization system for beauty recommenda- tion using the scores of various evaluation items. We confirmed that using our proposed system enables the conversion of other people’s evaluations into user-specific evaluations, making it easier to understand the characteristics of the unknown cosmetic items. Moreover, we also proposed a cosmetic item recommendation method based on the input of ideal scores using a visualization method of differences between cosmetic items. Even when users themselves do not understand the characteristics of ideal items and do not know how to search for them, our system can help users find their real ideal cosmetics because they can intuitively approach the characteristics of their ideal items. From the evaluation experiments, we confirmed that the proposed system can provide user- tailored search and information presentation, and it is effective in exploring and understanding unknown cosmetics. In the future, we plan to reflect the improvements obtained from the evaluation experiments in the proposed system and to improve the accuracy of recommendations by improving the evaluation expression dictionary. Acknowledgments This work was supported by multiple JSPS KAKENHI research grants (19K12243, 20H04293, 22K12281). It is also a product of research activity of the Institute of Advanced Technology and the Center for Sciences towards Symbiosis among Human, Machine and Data which was financially supported by a Kyoto Sangyo University Research Grant (M2001). References [1] M. Kim, S. M. Lee, S. Choi, S. Y. Kim, Impact of visual information on online consumer review behavior: Evidence from a hotel booking website, Journal of Retailing and Consumer Services 60 (2021) 102494. [2] S. Zhou, Z. Qiao, Q. Du, G. A. Wang, W. Fan, X. Yan, Measuring customer agility from online reviews using big data text analytics, Journal of Management Information Systems 35 (2018) 510–539. [3] Y. Matsunami, M. Ueda, S. Nakajima, T. Hashikami, J. O’Donovan, B. Kang, Mining attribute- specific ratings from reviews of cosmetic products, in: International MultiConference of Engineers and Computer Scientists, Springer, 2016, pp. 101–114. [4] A. Okuda, Y. Matsunami, M. Ueda, S. Nakajima, Finding similar users based on their preferences against cosmetic item clusters, in: Proceedings of the 19th International Conference on Information Integration and Web-based Applications & Services, 2017, pp. 156–160. [5] H. Giao, B. Vuong, T. Quan, The influence of website quality on consumer’s e-loyalty through the mediating role of e-trust and e-satisfaction: An evidence from online shopping in vietnam, Uncertain Supply Chain Management 8 (2020) 351–370. [6] E. Gerstell, S. Marchessou, J. Schmidt, E. Spagnuolo, How covid-19 is changing the world of beauty (2020). [7] D.-H. Park, J. Lee, I. Han, The effect of on-line consumer reviews on consumer purchas- ing intention: The moderating role of involvement, International journal of electronic commerce 11 (2007) 125–148. [8] M. Ueda, Y. Taniguchi, D. Li, P. Siriaraya, S. Nakajima, A research on constructing evaluative expression dictionaries for cosmetics based on Word2Vec, in: The 23rd International Conference on Information Integration and Web Intelligence, 2021, pp. 80–86. [9] T. Yuna, O. Asami, U. Mayumi, S. N. Siriaraya Panote, Efficient method for creating evaluation expression dictionaries for each cosmetic category (in japanese), in: FSS2019, SC2-3, 2019. [10] H. Kanayama, T. Nasukawa, Textual demand analysis: Detection of users’ wants and needs from opinions, in: Proceedings of the 22nd International Conference on Computational Linguistics (COLING 2008), 2008, pp. 409–416. [11] J. Yao, H. Idota, A. Harada, The affect of the internet’s word of mouth on buying behavior: from the questionnair survey pf the cosmetics purchase of the women students, in: Proceeding of National Conference of JASMIN Spring 2014, 2014, pp. 231–232. [12] F. Takayasu, S. Tetsuji, S. Kazumi, K. Kazuhiro, Polar coodinate visualization method for review documents focusing on graph structure (in japanese), in: JSAI2016, 1E4-1, 2016. [13] M. Oziom, S. Bachl, C. Wimmer, T. Grechenig, Grofin: enhancing in-store grocery shopping with a context-aware smartphone app, in: Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia, 2019, pp. 1–11. [14] S. Puebla-Barragan, G. Reid, Probiotics in cosmetic and personal care products: Trends and challenges, Molecules 26 (2021) 1249. [15] G. Ye, L. Hudders, S. De Jans, M. De Veirman, The value of influencer marketing for business: A bibliometric analysis and managerial implications, Journal of Advertising 50 (2021) 160–178. [16] O. Kazuki, H. Hiromi, M. Soh, H. Junichi, Store visualization system for shopping mall type e-commerce (in japanese), in: Information Processing Society of Japan, 2015.