GaChat:A chat system that displays online retrieval information in dialogue text Satoshi Horiguchi Akifumi Inoue, Tohru Kenichi Okada Graduate School of Science Hoshi Faculty of Science and and Technology, Keio School of Computer and Technology, Keio University Science, Tokyo University University 3-14-l Hiyoshi, Kohoku-ku, of Technology 3-14-l Hiyoshi, Kohoku-ku, Yokohama, 223-8522 Japan 1404-1 Katakura, Hachioji, Yokohama, 223-8522 Japan horiguchi@mos.ics.keio.ac.jp Tokyo 192-0982, Japan okada@ics.keio.ac.jp akifumi, hoshi @cs.teu.ac.jp ABSTRACT to attach additional devices such as videocameras and ther- Text chat systems are popular and widely used by a lot of mometers to the system. However, a text chat system with users. However, there are sometimes redundant interactions a videocam means just a video chat system. It is not a text- between the users because of its less awareness. In this pa- chat system any longer. A video chat communication re- per, we propose a text chat system called ”GaChat”, which quires higher mental load than a text chat communication. simultaneously appends related information about the dia- Those devices also complicate the simplicity of the text chat logue text between its users. First, proper nouns are ex- system. tracted from the dialogue text by morphologic analysis. Then online images and articles related to the nouns are simulta- We believe that a text chat system should be used for casual neously displayed with the dialogue text. Such settlement of communications, which is easy to start, easy to keep, and the ambiguity helps users to reduce redundant interactions easy to quit. We propose a text chat system called “GaChat”. like searching and asking the details of the phrase. This pa- This system is developed to avoid the misunderstandings be- per describes the prototype implementation and its first eval- cause of the low- awareness without any additional devices. uation experiment. This means that the system only uses a keyboard as the input device. Instead, the system displays images and comments Author Keywords related to the dialogue of the users explicitly. Text chat communication, Instant messaging service, Web based communication The remainder of this paper is organized as follows. In Sec- tion 2, we discuss related work on communications support with text chat. In Section 3, we describe our chat system de- ACM Classification Keywords sign. In Section 4, we discuss the our prototype system. In H.5.3 Group and Organization Interfaces: CSCW Section 5, we describe example of operating the prototype. In Section 6, we discuss the current limitations ofour proto- INTRODUCTION type system. We conclude in Section 7 by discussing near Text chat systems are popular and widely used as one of the future work that we plan to explore. easy communication tools. Such systems can smoothly and quickly exchange text messages. On the other hand, a text RELATED WORK chat communication has less awareness than a face-to-face Our system displays complementary information simultane- or a video-based communication. It is difficult to infer the ously with the text messages. Several studies have take sim- vocabulary of their partner and subtle difference in nuance ilar approaches to the various situations[1, 2, 3]. Lieberman under the low-awareness condition. We often have to ex- proposed a system that assists users to take opportunities in change a lots of redundant messages to explain a trivial mat- our daily work by image retrieval and annotation[4]. ter. The purpose of those researches limits it’s use of the chat, One of the approaches to cover the lack of the awareness is and improves communication in that such limited situation. Lock-on-Chat[5] is a chat system for the communication at the place of an academic conference. Users can share the snapshot of the slide among the participants and can leave comments freely on a specific part of the snapshot. This sys- tem was actually implemented at an conference place, and a lot of participants actively discussed with it. However, our Workshop on Visual Interfaces to the Social and the Semantic Web (VISSW2009), IUI2009, goal is not for such a specific situation, but for the daily ca- Feb 8 2009, Sanibel Island, Florida, USA. sual situation. Copyright is held by the author/owner(s). 1 Message ＋ GaChat Article Message Article (1) Retrieval result Message (2) Send button Figure 2. Overview of the system Retrieval button conversation for an unknown retrieval of the word and the consensus of communications. Figure 1. Retrieval function of the existing system About online retrieval information Munemori et al[6] proposed a “emoticon” chat system. In Typical word and phrase that have different understanding this system, a user can use “emoticon” only, no plain text is the proper noun.A proper noun indicates a specific ob- messages can be used. Although this system can give uni- ject such as a name of a person or a place. It is alternative versal messages regardless of the language, the contents of whether one knows the object or not. If a user does not have the message is limited. enough knowledge about the proper nouns appeared in the exchanging message, the message might not be treated prop- While we are communicating with text chat systems, we of- erly. We also have to pay attention to the difference of the ten find unknown or unclear words in the messages. Most of nuance and connotations. us try to use the help of a web search engine to resolve the question. This search action is troublesome because it re- Proper nouns are also frequently used as search terms be- quires another web browser besides the chat system. Doing cause it is effective to make the search more specific. If we the search action or not is left to the user’s own choice. use general nouns as search terms, the result might be enor- mous and ambiguous. Such information does not help the Windows Live Messenger[7], which is most popular chat mutual understanding at text chat communication. There- system, has already integrated the web search function to the fore, our system uses proper nouns only to fetch the sup- system. A user can search keywords entered in the message plemental images and articles. If there are multiple proper area by pushing “search” button instead of “submit” button. nouns in a chat message, we choice the last one. This is be- However, the result is returned as a URL format. A retrieval cause Japanese grammar(the authors’ native language) tends example is illustrated in figure 1. To see what’s in the URL, to place more pmphasis on the last part of a sentence. a user has to run the browser again. A user also has to do chatting and searching at the sametime. From the question- We think about a presentation of the image information and naire to our colleagues, we found that the function of the formal textual information as a method of presenting infor- “retrieval(search)” button was not used positively. Even the mation on the proper noun. Supplementary information should existence was not clear for them. be strongly related to the object the user intended to explain. Our system fetch those contents from the site Wikipedia. For instance, when one tries to explain a school “Tokyo Univer- METHOD sity of Technology” to a friend who doesn’t know it, a sym- Outline bolic picture and outlined information about history and fac- ulty may be impressive and help the understanding(Figure The outline of the proposal method is shown Figure 2. A 3). Currently such well-edited contents are collected in an user inputs and sends a message as well as in the same man- encyclopaedic site. ner of the normal text chat system. Then the message is sent to the GaChat server. The server excerpts the proper noun from the message, and fetches the article about the noun Operation from wikipedia. Those additional data are automatically dis- Our proposal system consists of Chat Server and Chat Client. played on both chat windows. Chat server (GaChat server) has those functions that is man- agement of conversation, analysis of conversation text, im- If the proposal method is used, the extra activity between the age search and retrieval of article in encyclopedia. Chat chat users can be suppressed. An extra activity is an extra client (Gachat client)utilizes those additional data to talk with 2 (1). User name and chat room making The explanation about Tokyo University of Technology Image Textual information (2). User name and chat room display Established: 1947 Location: Hachioji,Tokyo,JP Website: (5). Extraction word display http://www.teu.ac.jp/ etc… (7). Outline of Wikipedia (4). Text area (6). Retrieval image Figure 3. A Image information and a formal Character information (3). Input and send of message Figure 5. Details of function ： Have you been to Tokyo University of Technology ? Remark： User A Send Proper noun Tokyo University of Technology sis machine though the proper noun was extracted from the Analysis Tokyo University of Technology conversation text of the client with the server) was used. Ya- Established: 1947 Location: Hachioji,Tokyo,JP hoo! API[9] was used for the extracted phrase. ’SimpleAPI- Image search Wikipedia retrieval Website: http://www.teu.ac.jp/ Outline of article etc… Wikipedia API[10]’ was used for the article retrieval(figure Text message Image 4). Display area: Tokyo University of Technology GaChat server Established: 1947 User A：Have you been to Tokyo University of Technology ? Location: Hachioji,Tokyo,JP The GaChat server is a text analysis besides the commu- Website: http://www.teu.ac.jp/ Tokyo University of Technology etc… nication of the message of the normal text chat, is an im- age search, and retrieves the encyclopedia. The text anal- ysis extracted the proper noun from the remark. This ex- Figure 4. Operation of the system tracted phrase is used for retrieval of image and the encyclo- pedia(Figure 5). the other one. GaChat is a chat system with the following We discuss technology’s detail on our system. Request URL functions. is made by the REST form, and it inquires of image search Web service. It is necessary the phrase to URL encode by (1)Function to acquire phrase of proper noun from content UTF-8. The image URL get from the response field of “/Re- of remark of text chat sultSet/Result/Url”. Itextract the URL of retrieved image (2)Function to do image search according to phrase acquired from Those retrieval results information of the top 12 list. by one, and to acquire pertinent site URL ”The 12 list” means the number of retrieval result in first (3)Function to do encyclopedia retrieval concerning phrase page. Because, the research company in the United States acquired by one, and to acquire content indicates that the user of 62% surfs only the first page re- (4)Function to display article in image and encyclopedia ac- sult[11]. quired by 2 and 3 in addition to remark by automatic opera- tion The encyclopedia retrieval matches to the specification of WikipediaAPI and generates request URL that adds the phrase The outline of operation extracts the proper noun by the sys- similarly encoded to UTF-8. The output form selected XML. tem when sentences are input, and retrieves the image search It is a Perth doing XML. The title of the article get from the and the encyclopedia to the retrieval word this. URL of the response field of “/results/result/title”. The digest of the ar- image is acquired in the image search, and the title and the ticle get from the response field of “/results/result/body”. text in the article are acquired in the encyclopedia retrieval. As a result, the user-name, it makes remarks to the message GaChat client area, and the title and the text are displayed in the image, the We explains each function of the GaChat client by GUI(Figure retrieval word, and the encyclopedia area in the image area. 5). IMPLEMENTATION OF GACHAT (1). We register the chat’s username and the chat room. This chapter describes chat system ”GaChat” that gives re- (2). This area displays registered name and chat room. trieval information by the conversation phrase based on the (3). Input and send of message. proposal method automatically. (4). This area displays the conversational content. (5). This area displays the proper noun and that phrase’s Environment URL. GaChat was composed of the client and the server, and im- (6). This area displays image related to the phrase. plemented by Java. It is Sen[8] as the morphological analy- (7). This area displays Wikipedia article related to the phrase. 3 A Wikipedia article I went to Tokyo Tower yesterday. related to Tokyo Tower Good. Have you been to Zojo-ji Temple A Wikipedia article A image near Tokyo Tower? related to Zojo-ji Temple related to Tokyo Tower A image related to Zojo-ji Temple Figure 6. Conversation 1 Figure 7. Conversation 2 DISCUSSION there is domination because the phrase (proper noun) is ex- EXPERIMENT tracted from the content of the conversation by the automatic We discuss The operation verification of GaChat. It is con- operation of this system, and the function to display the re- firmed that the proper noun is extracted from each a series of trieval result doesn’t request special work from the user. remark as a precondition by confirming the operation about the system of each function to mount, and the image search, Perhaps, each person might be implicitly inspecting article the display by the words and phrases, and the retrieval and information in the encyclopedia by the confirmation extent the displays of the Wikipedia article are done. To talk about when not understanding. It will be necessary to catch and to the time that hangs to the retrieval and the display of two investigate in the future. functions slightly, it is especially understood not to become it in the obstacle. The image information might be acquired from 12 high ranks at random, and neither article information nor the content of We introduce the results of the communications with GaChat. Wikipedia might match it in mounting GaChat. a actress The testee was recruited in the laboratory to which authors Ozawa pearl is displayed to retrieval ”Ozawa” in the image belonged for communications using GaChat to search for the information and politician Ichiro Ozawa is displayed in the feature, and the text communications were used to the last to wikipedia article(Figure 8). Person’s name is a problem that do. The chat software was used usually, and testee’s condi- a famous person of this family name causes in two or more tion was assumed to be a thing that the blind touch of extent situations when extracted only by the family name. ”Inter- to which the input of the message with the keyboard did not est” might be able to be offered to communications by using obstruct communications was able to be done this time. It in- such a phenomenon though it is necessary to correspond as troduces one example of feature communications as follows. trouble. The conversation is Figure 6 and 7. General information that expects everyone need not never Horiguchi: I went to Tokyo Tower yesterday. go out though the match of the image information and the Kodaira: Good. Have you been to Zojo-ji Temple near Tokyo Wikipedia article might be important. Because the concept Tower? of GaChat is fixation of the topic by conversation text and synchronous displaying related information on the topic and union, the accuracy of the retrieval result is not so important. The phrase (proper noun)extracted from Horiguchi’s remark The more important one is stability that always displays the is “Tokyo Tower”. Picture and Wikipedia article related to image information and the Wikipedia article to the conver- Tokyo Tower is displayed. The phrase (proper noun)extracted sation. There is no problem about the part under the present from Kodaira’s remark is “Zojo-ji Temple”. Picture and Wikipedia situation. article related to Zojo-ji Temple is displayed. Zojo-ji Temple is a Buddhist temple in the Shiba neighborhood of Minato- We want to analyze a further feature continuing the trial eval- ku in Tokyo, Japan. uation in the future. Moreover, the fixed quantity evaluation is necessary to judge more in detail, and to analyze the state In this case, when related information on the topic was syn- as communications. The evaluation standard that becomes a chronously presented while doing the text communications, criteria is selected therefore, and construction is important. it was able to be confirmed to be especially reactive to the image. However, this conversation example was included, Moreover, the method of displaying it is devised when on and it was not able to be confirmed that article information the other hand, there is an image information in the article in the encyclopedia influenced the topic by the present stage. on Wikipedia. It will be necessary to mount the function, 4 a quantitative evaluation in the future. I want to improve the system from the problem etc. clarified from the evaluation experiment. When communications are smoothly done, we extracted Key word investigate the role of the conversion of the topic. “Ozawa” ” It is a type of family name. REFERENCES 1. Gareth J. F. Jones and Peter J. Brown. Information access for context-aware appliances (poster session). In SIGIR ’00: Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 382–384, New York, NY, USA, 2000. ACM. 2. L. Birnbaum J. Budzik, K. J. Hammond. Information access in context. Knowledge-Based Systems, This Image is This Wikipedia article is 14(1-2):37–53, Mar. 2001. a actress named Ozawa. a politician named Ozawa. 3. B. Rhodes. Using physical context for just-in-time information retrieval. Computers, IEEE Transactions on, 52(8):1011–1014, Aug. 2003. Figure 8. ExampleDisagreement of a image information and a encyclo- pedia information 4. Henry Lieberman, Elizabeth Rozenweig, and Push Singh. Aria: An agent for annotating and retrieving and to compare it with this proposal technique in the fu- images. Computer, 34(7):57–62, 2001. ture. This time, the case where Wikipedia article informa- 5. Takeshi Nishida and Takeshi Nishida. Lock-on-chat: tion is referred frequently was not able to be confirmed, and Boosting anchored conversation and its operation at a it wants to be going to advance the examination in the future, technical conference. In INTERACT 2005, Springer and to search for the application example. LNCS, pages 970–973, 2005. CONCLUSIONS 6. Munemori Jun, Miyai Shunsuke, and Ito Junko. In this thesis, We proposed GaChat. So far we have outlined Development and application of pictograph chat the way in which our chat system is low- awareness without communicator. In IPSJ SIG Technical Reports, any additional devices. Our system aimed at the obstruction 2006-GN-61, Sep 2006(Japanese). evasion and the consensus of communications. Moreover, it 7. WindowsLiveMessenger. is thought that information displayed synchronizing with the http://messenger.live.jp/, 2008. message plays the role of the conversion of the topic when communications are smoothly done, and I want to evaluate 8. Sen. Japanese morphological analysis system. the respect. When chatting the text, it phrases it as an ap- http://ultimania.org/sen/, 2008. proach of the action assistance of the print putting of the obstruction evasion of communication and communications 9. Image search Web service. Dveloper network. together in the conversation in the text chat in this thesis. http://developer.yahoo.co.jp/search/ The technique for the union of specified meanings of the image/V1/imageSearch.html, 2008. phrase by synchronously presenting information that con- 10. WikipediaAPI. SimpleAPI vol.3. cerned, and limiting it was shown. It is thought that light- http://wikipedia.simpleapi.net/, 2008. hearted communications that were the advantages of the text chat were kept compared with the one to transmit past aware- 11. iProspect. iprospect search engine user behavior study. ness information and to attempt solving. The opinion roughly http://www.iprospect.com/index.htm, friendly was received from the testee. 2008. In the evaluation that used the prototype, it was confirmed that attention gathered in the image information conversa- tion text and synchronous displayed, and the topic was fixed. Moreover, it was confirmed that attention turned even when there was no knowledge for the presented image informa- tion. It was shown to reduce communication the obstruc- tion, and to assist the print putting of knowledge each other together. Moreover, it is thought that information displayed synchronizing with the message plays the role of the conver- sion of the topic. It will be necessary to continue the trial evaluation, and to do 5