ubiPCMM 2005 81 Personalized Information Retrieval Framework Dongpyo Hong, Yun-Kyung Park, Jeongwon Lee, Vladimir Shin, and Woontack Woo are indispensable to those who want to retrieve appropriate Abstract—In this paper, we propose a framework of information with less effort in a given time. personalized information retrieval system for a wearer (a person There have been many research activities on information who is equipped with a wearable computer) in ubiquitous retrieval to find an appropriate document from database or computing environments. In ubiquitous computing environments, personalized information retrieval is indispensable for a wearer libraries [4-6]. Over the past few years, however, it has been because desired or undesired information will be flooded around reported on the importance of information retrieval in wearable the wearer. Although there have been many research activities on computing, mobile augmented reality (AR), and in ubiquitous information retrieval in wearable computing and/or ubiquitous computing environment. For instance, S. Julier, et al. (2000) computing environments, personalization has not been recognized proposed information filtering for mobile augmented reality significantly. Accordingly, all the retrieved information is exposed because display information in AR is cluttered with too much to the wearer regardless of his/her situations or conditions. In this regard, the proposed framework enables a wearer to retrieve the information [7]. Insley (2003) figured out that one of difficult personalized information from objects. In the proposed problems for AR is effectively managing large amounts of framework, we exploit user’s context as a fundamental element in information [8]. Without a way to filter out garbage retrieving the personalized information. The proposed framework information, augmented reality would be utterly useless. Jones consists of two conceptual stages (object selection and information and Brown (2004) addressed that information retrieval may be presentation) and each stage includes several components. In facilitated by contributions from human-computer interaction order to measure the effectiveness of the proposed framework, we studies and agent technology to determine how and when to introduce a measuring method and also realize a prototyping system for personalized information retrieval. Thus, we believe deliver the information to the user or how best to act the user’s that the proposed personalized information retrieval framework behalf [9]. On the other hand, it is also an important feature to is to leverage human-computer interactions for a wearer in present the retrieved information appropriately to users ubiquitous computing environments. afterward. For example, Sinclair, et al. (2003) suggested that Adaptive Hypermedia is a solution for dealing with complex, Index Terms—Context, Personalized Information Retrieval, heavily structured information and the presentation of views of Wearable Computing, Ubiquitous Computing that structure to users [10]. Meanwhile, the relevant applications were mainly on the battle fields or tour guides. By I. INTRODUCTION way of example, S. Julier, et al. (2000) proposed that BARS (Battlefield Augmented Reality System) will ensure that only W ITH the rapid progress of technologies in the areas of computers and communications, the future computing environments will support seamless interactions from the most relevant information is displayed to the user at a particular time [11]. B. Bell, et al. (2003) proposed an AR system that could provide “information at a glance” to aid a ubiquitous computers and pervasive networking. That is, users mobile user exploring an unfamiliar environment [12]. The could do just-in-time access to any (invisible) computer at any detailed reviews are shown in Section II. Whereas the previous time and any where [1-3]. Consequently, this environment will research on information retrieval is mostly to find out require users to interact with computers through more natural appropriate documents from database regardless of user’s and comfortable interfaces. Meanwhile, due to the explosion of context, the recent research takes into account user’s context in the volume of available information for users to deal with in the development of information retrieval systems. However, this computing environment, the information retrieval systems there are few frameworks to support personalized information retrieval system based on user’s context. Manuscript received June 17, 2005. This work is supported by Seondo In this paper, we propose a framework of personalized project of MIC, Korea. Personalized Information Retrieval Framework. information retrieval system for a wearer (a person who is Dongyo Hong is with Gwangju Institute of Science and Technology (GIST), Gwangju, Korea (e-mail: dhong@gist.ac.kr). equipped with a wearable computer) in ubiquitous computing Yun-Kyung Park is with Electronics and Telecommunications Research environments. Moreover, we suggest an effective means to Institute (ETRI), Deajun, Korea (e-mail: parkyk@etri.re.kr) exploit user’s context through this framework. In the proposed Jeongwon Lee is with Department of Digital Contents, Sejong University, framework, we separate the procedure of personalized Seoul, Korea (e-mail: jwlee@sejong.ac.kr). Vladimir Shin is with Gwangju Institute of Science and Technology (GIST), information retrieval into two parts. The first part is object Gwangju, Korea (e-mail: vshin@gist.ac.kr). selection, which includes context transform and personalized Woontack Woo is with Gwangju Institute of Science and Technology object filter components. In the second part, personalized (GIST), Gwangju, Korea (corresponding author, phone: +82-62-970-2226; fax: +82-62-970-2249; e-mail: wwoo@gist.ac.kr). information filter and personalized information presentation Dongpyo Hong, Yun-Kyung Park, Jeongwon Lee, Vladimir Shin, and Woontack Woo 82 components are included. Besides these components, the mechanism and other useful utilities are implemented for proposed framework also includes context representation developers [18]. Furthermore, we deploy the implemented component. Before we introduce each component in the system for personalized information retrieval in ubiHome, a proposed framework, we assume that any information of an smart home environment test-bed, and experiment with it [2]. object as well as user’s context, namely user’s profile or This paper is organized as follows. In Section II, we review preference, can be represented as a unified context. We name the related works which address the necessity of information this process as context representation [13-17]. This basic retrieval for wearable computing, mobile AR, and ubiquitous assumption enables us easily to describe contextual information. computing environment. In Section III, we explain the Otherwise, contextual information is too complicated to utilize fundamental components for the proposed framework of in personalized information retrieval system. In addition, the personalized information retrieval with a specific example. In unified context representation allows us to access contextual Section IV, we show the experimental results and effectiveness information in a structured manner when we try to retrieve of the proposed framework through the introduced measuring information from an object. Although we have the unified method. Finally, we discuss future works in Section V. representation of contextual information, we need another transformation, context transformation, because the unified representations of context are not easily comparable. Thus, II. BACKGROUND context transformation transforms the unified context to In general, research on information retrieval has been appropriate form such as Boolean or probability for easier focused on how to retrieve the requested information from a comparison between user’s context and information of an large volume of data, particularly documents, even images in object. Owing to context representation and context the Internet (WWW), database, etc. Over the last decade, the transformation, we can compare information of an object with concept of information retrieval has been moved to mobile user’s context in a consistent way. After context transformation, computing, wearable computing, augmented reality (AR), and we are able to select the most relevant object in response to ubiquitous computing in order to retrieve only needed user’s context, where the selection criterion is defined as information because there will be too much information in such personalized object filter. In the personalized object filter, we computing paradigms. In this section, we review the most can exploit either deterministic or probabilistic selection relevant works about information retrieval, and then we extract criteria in order to filter out an object from various objects. fundamental components for personalized information retrieval After the selection of an object responding to user’s context, we system. have another filtering to present only preferable information in In [7], the authors proposed information filtering for mobile the selected object through an appropriate output device. In fact, augmented reality because display information in AR is this procedure consists of personalized information filter and cluttered with too much information. Information filtering personalized information presentation. In this framework, means to cull the information that can potentially be displayed however, we assumed that the output device of a wearer by identifying and prioritizing the information that is relevant remains the same regardless of wearers. Lastly, we also to a user at a given point in time. Thus, information can be introduce a measuring method for personalized information classified based on the user’s physical context as well as on retrieval system, i.e. recall and precision measurement. As a their current tastes and objectives. In addition, the amount of matter of fact, recall and precision measurement is widely used information shown to a user about an object is inversely in conventional information retrieval systems [4-6]. In this proportional to the distance of that object from the user. regard, we can project personalized information retrieval into a Although their information filtering framework includes searching or classification problem, and thus can adopt various well-defined components, the used functions in this framework mathematical models like Boolean matching or Bayesian are too general for developers to use them. Moreover, the decision rule according to the applications. objective properties and subjective properties in the state of Therefore, the proposed framework helps developers to user and object seem to be used as contextual information, but implement personalized information retrieval system due to the their representations are dependent on a specific domain such structured components and easier exploitation of user’s context. as battle field. Thus, the representations of user’s context and In addition, we believe that personalized information retrieval object’s context should be generalized so that the information enables a wearer to do natural interactions with computing filtering framework is applicable to other areas. resources like augmented virtual object or information in In [8], one of most difficult problems for AR is effectively ubiquitous computing environment. That is, the retrieved managing large amounts of information. The cluttering junk information is presented differently corresponding to user’s information would make it difficult to find useful information context such as user’s preference. Consequently, it leverages and even become distracting to interface with real world tasks. human-computer interactions of a wearer to retrieve Without a way to filter out garbage information, augmented personalized information from a large volume of information reality would be utterly useless. In this regard, there are around the wearer. In order to realize the proposed framework, numerous methods that allow some level of filtration. One of we exploit wear-UCAM as an empirical tool, a toolkit for the simplest techniques is to filter information based on context-aware applications, because context-awareness ubiPCMM 2005 83 location. However, local user’s context like location encountered, the user can select a filtering mode according to information is not enough to filter out appropriate information his current mission. In addition, the paradigm of an information from useless information. Thus, a set of user’s context should filter is presented by a decision mechanism that uses the user’s be applied into information retrieval. Meanwhile, the most location, the user’s current goal and the properties of objects basic type of content is labels. They can be used to display within the environment to deduce what information should be simple yet important pieces of information. However, if their displayed. The filtering performed by BARS (Battlefield numbers become excessively large, the labels will become a Augmented Reality System) will ensure that only the most problem, despite their individual simplicity. It is therefore relevant information is displayed to the user at a particular time. necessary to have a method of automatic filtering labels to keep From this, we can project personalized information retrieval to them under control, although it need not be terribly draconian. a decision problem, and thus we can employ various In [9], the author addressed that information retrieval may be mathematical approaches. facilitated by contributions from human-computer interaction In [12], the authors presented examples from prototype studies and agent technology to determine how and when to augmented reality systems that show some of the ways in which deliver the information to the user or how best to act on the information might be displayed, emphasizing the automated user’s behalf. To improve retrieval effectiveness in the future layout of overlaid graphics. In this approach, they focused on computing environments in terms of retrieval accuracy and user view management in accordance with user’s context, especially satisfaction, the key issue is the integration of technologies user’s location indoor and outdoor. However, they assumed from information retrieval and context-awareness. However, that information filtering has been done and that everything current search engines take no account of the individual user being displayed should be displayed. Thus, we believe that and their personal interests or their physical context. Although information presentation is another important feature in the author did not mention explicitly the term ‘wearable personalized information retrieval system. computer’, we believe that personal and context information is In this review, we observed that many researchers have potentially available for the retrieval process by a wearable addressed the importance of information retrieval in the future computer. Meanwhile, the information should be delivered in a computing environments. Although most researchers point out timely fashion since the information is based on the user’s the usefulness of user’s context in information retrieval, there context and the context may change. Thus, it will be often are few works to reveal how to exploit user’s context and its useless to deliver information about a situation the user has just detailed manipulation for personalized information retrieval left. In this regard, information retrieval can be extended to system in such computing environments. As we have reviewed, incorporate recommendations based on preferences or profiles there are many practical applications which show the derived from the user where long-term information needs effectiveness of information retrieval. However, there is little because the interests of the user are modified gradually over tendency to show the detailed explanations of the information time as conditions, goals and knowledge change. Lastly, the retrieval algorithms supported by mathematical derivations and personalization for retrieval is how the interests of the user are evaluations. These have inspired us to investigate on the captured. The personalization system monitors the user’s framework for personalized information retrieval system with behavior and learns profiles from this. mathematical models. In [10], Adaptive Hypermedia (AH) is a solution for dealing with complex, heavily structured information and the presentation of views of that structure to users. The AH system III. PERSONALIZED INFORMATION RETRIEVAL FRAMEWORK may maintain a user profile for each of the people interacting with it, initially based on user preferences. And the system A. Overview modifies the content that they see and the paths that they take From the previous sections, we could define that through the content accordingly updating the profiles as the personalized information retrieval is to retrieve appropriate users move through the information space. Users view information from a large volume of data (or information) with a information that is commonly presented to all users regardless wearer’s context, namely preference or profile, and also to of the context of their interaction, or any profiling, as present the retrieved information appropriately based on the non-adaptive. When the information to be displayed is adaptive, wearer’s context in ubiquitous computing environment where meaning that it changes according to the context of the viewing, any information could be augmented by anyone. As shown in most AR researchers use a dynamic approach. Thus, Figure 1, information or property of any object and wearer can information retrieval should exploit contextual information, be represented as a unified context [13-17]. In addition, especially user’s context. However, there is no relevant information retrieval should occur only if a wearer has a special explanation about a way of representing user’s context and interest on a specific object. Thus, the most relevant exploiting it in the actual interactions. information to the wearer is retrieved from the selected object. In [11], the filtering is a logical extension and refinement of However, the preferred information in response to the wearers’ the aura used with the object distribution in order not to show context within the retrieved information should be exposed to all irrelevant information to users. To help the decision process them. For example, let Wearer A and Wearer B are interested in and to take into account the variety of situations that can be the same object in the given environment. However, their Dongpyo Hong, Yun-Kyung Park, Jeongwon Lee, Vladimir Shin, and Woontack Woo 84 preferences are supposed to be different. In conventional too complicated to utilize in the information retrieval system. In information retrieval system, the retrieved result of Wearer A is addition, the unified context representation allows us to access the same as the result of Wearer B because they are interested in contextual information in a structured manner when we try to the same object, that is, Query A is equal to Query B. In the compare contexts, select an object, and retrieve information of proposed framework, however, the retrieved results of Wearer the object in the proposed framework. Generally, the unified A and Wearer B are different because the queries are based on contexts can be implemented by key-value models, markup their context, i.e. Query A and Query B are differently treated. scheme models, graphical models, object oriented models, Therefore, personalized information retrieval is achieved in the logic based models, or ontology based models [17]. Regarding proposed framework. the utilization of the unified context in the proposed framework, Unified Context we internally employ temporary data structures to manipulate INF0 INF1 INFi INFN-1 user’s context and object’s information effectively. In the framework, two temporary data structures are introduced; one Obj0 Obj1 Obji ObjN-1 is for user’s context (contextual query), and the other one is for object’s information (object database). Through the introduction of temporary data structures, we can simplify Environment Result A Result B personalized information retrieval problem as a query from a database. In Section IV, we will show an example to help Wearer understand context representation. Figure 3 shows the specific Query A Personalized Query B Information Conventional implementation of contextual query from user’s context and Info. Retrieval object database from available objects in the given Personalized Info. Retrieval environment. Wearer A Wearer B CONTEXT QUERY CONTEXTA CONTEXTB Q CONTEXT Fig. 1. Conceptual Diagram of Personalized Information Retrieval for TRANSFORMATION Wearers in Ubiquitous Computing Environment. CT(Q) Figure 2 shows the procedure of the proposed personalized information retrieval framework. We will explain the CT(Obj) Filter procedures and components of this framework in the following M(Q,Obj) subsections. Obj0 Obj1 DESIRED USER ① OBJECT SELECTION ENV ● ● ● ● ● ● OBJECT Who Personalized Obj0 Context ● ● ● ● ● ● Object Transformation Filter When Obj1 ObjN-1 •DETERMINISTIC •MATCHING CRITERIA (BOOLEAN) (SELECTING CRITERIA) •PROBABILITISTIC OBJECT DATABASE Where (BAYESIAN DECISION) Fig. 3. An Example of Temporary Data Structures. What Obji Personalized Personalized As shown in Figure 3, a user’s context can be represented as How Information Information Q and available objects can be represented as a vector of Obj Presentation Filter Why •PREFERED INFORMATION •MATCHING CRITERIA respectively. Furthermore, their elements in the set are vector ObjN-1 •ANY AVAILABLE DEVICE (SELECTING CRITERIA) types. However, this internal data structure is nothing but data Obji Context Obji Context container of the user’s context and of available objects, thereby Representation Representation we need to transform it into another form, which is explained in ② INFORMATION PRESENTATION the next subsection. Personalized Information Retrieval Framework Fig. 2. Personalized Information Retrieval Framework. C. Context Transformation In general, users would always retrieve exact and perfect B. Context Representation information to their requests from a given system. However, There are many definitions of context among researchers this is only possible when the request is exactly coincident and [13-17]. However, we exploit the unified context as the form of information in the system can be definitely identified as 5W1H (Who, When, Where, What, How, and Why) to responding to the query. For example, the most obvious exact represent user’s context and object’s information (properties) matching arises with a numerical representation of information. [15]. Due to this unified representation of user’s context and Although the context representation of 5W1H format has many object’s information as 5W1H, we are easily able to describe advantages, it is hard to be used directly in the proposed contextual information. Otherwise, contextual information is framework because the context representation is just a data ubiPCMM 2005 85 container. Therefore, user’s context and object’s information D. Personalized Object/Information Filter needs to be transformed to an appropriate format for easier Ultimately, the goal of information retrieval is to find a comparison such as Boolean or numerical format without loss match between a given query and those objects that the user of advantages in the context representation. We define this would like to retrieve in response to the query. The matching transformation as context transformation, and it can be process is complicated by the fact that the query and the objects represented as follows. may have quite different forms. Through the context transformation, however, we are able to cope with this X ′ = CT ( X ) (1) complication. On the other hand, a term in an object has been matched to a term in a query does not automatically mean that where X is an arbitrary context, X' is the transformed context, the object should be retrieved in response to the query. In the and CT represents context transformation. first place, the query probably contains more than one term, so In context transformation, any transformation method is the success or failure in matching each of the query terms must applicable if the transformed contexts are definitely be considered. In the second place, an object which contains a comparable to each other. In the proposed framework, either given query term does not mean that the object is strongly deterministic transformation or probabilistic transformation is related to the term. Therefore, matching criteria is a key factor possible. However, we exemplify Boolean transformation in personalized information retrieval. After we briefly examine method in a deterministic transformation for the sake of the types of matches that can be used, we will show a criterion simplicity. Typically the connectives in Boolean permitted are as an example of personalized object/information filter. AND, OR, and NOT. In the Boolean model of retrieval, the In general, the matching criteria in information retrieval are similarity between an object and a query is based on the categorized as follows: exact match, range match and presence of terms in both the query and the object. In spite of approximate match [4-6]. The most obvious exact match the simplicity of Boolean model, they present a number of situations arise with numerical information. Any information significant problems [4-6]. For instance, there is no good way whose term values all match the values specified in the query to weight terms for significance. Regarding weight, the vector will be retrieved; all other information will be rejected. An model based on 0-1vector can be exploited, where each immediate extension of the exact match is the range match. In component is either 0 if the corresponding term is absent or 1 if this type of matching, a range of acceptable values is given for the corresponding term is present in the object or query being each term. Range matches are possible on terms that have a represented. Whereas Boolean models have the disadvantage natural order, for example numerical or alphabetical, so that it of not incorporating term weights, vector models have the is meaningful to specify that a term be greater than some disadvantage of not being capable of expressing logical minimum value or less than some maximum value. An connectives easily. Various attempts have been made to approximate match requires some way to measure how well a mediate Boolean and vector queries by the introduction of given object matches the query. weighted or extended Boolean queries [19-21]. We focus on evaluating a match between an object (Obj') Thus, we can represent user’s context as a weighted Boolean and a query (Q'). In addition, we exploit the cosine query as follows. measurement which is widely used as a matching algorithm in information retrieval systems [6]. In this measurement, we assumed that the object and query are represented as numerical Q′ = Twwho 1 * Twwhen 2 * Twwhere 3 * Twwhat 4 * Twhow 5 * Twwhy 6 (2) vectors in t-space, that is Q' = (q0, q1, …, qt-1) and Obj' = (obj0, obj1, …, objt-1) where qi and obji are numerical weights where T5W1H are query terms, w1…6 are the weights (either 0 or associated with the keyword i. The cosine correlation is now 1), and * represents a Boolean or logical operation. Although simply defined as follows. sum of weight is equal to 1, the term of weight used in context transformation only indicate whether the transformed query t −1 includes terms or not. Thus, the total sum of weights need not ∑ q obj i i be 1. As shown in (2), user’s context can be transformed M (Q′, Objk′ ) = i =0 1/ 2 (3) weighted Boolean query format, where each term of query is ⎛ t −1 t −1 2⎞ ⎜ ∑ (qi ) ∑ (obji ) ⎟ 2 correspondent to each term of 5W1H. In case of context transformation of objects, however, we just take weight of each ⎝ i =0 i =0 ⎠ term as 1. In general, the weighted Boolean operations are relatively With the matching function (3), we can retrieve the object of complicated [5,19-21]. However, we can avoid the complicated interest as the following criterion. Suppose there are N objects weighted Boolean operations because each term of 5W1H is in the environment, then the retrieving proceeds by calculating independent value. This is one of the advantages of the unified values M(Q',Obj'k), where k is an arbitrary number in N. context representation in 5W1H format. Regarding context Through this calculation, the set of objects to be retrieved is transformation, we will explain the way of using context determined. In mathematical point of view, this function is the transformation through an example in Section IV. inner product of the query and object vectors, where 1 Dongpyo Hong, Yun-Kyung Park, Jeongwon Lee, Vladimir Shin, and Woontack Woo 86 represents the highest similarity. Thus, the function M measure the effectiveness of personalized information retrieval transforms the angular measure into a measure ranging from 1 system because it is the most widely used. for the highest similarity to 0 for the lowest. In general, there In recall and precision measurement, we can establish a 2 x 2 are two approaches to retrieve objects of interest: threshold and contingency table which shows how the information set is rank position approach. However, we adopt threshold approach divided by both relevance and retrieval. Table I shows the because it is intuitive and widely used. When the matching relationships of these two measures. function is given a suitable threshold, we can retrieve the objects above the threshold and discard the ones below. If Th is TABLE I CONTINGENCY TABLE FOR EVALUATING INFORMATION RETRIEVAL the threshold value, then the retrieved object set is represented Retrieved Not Retrieved as Ô = {Obj'k | M(Q',Obj'k) > Th}. Relevant w x n1 = w + x On the other hand, we have to keep the space or length, t, of Not Relevant y z the query and object as the same space in order to exploit the n2 = w + y N=w+x+y+z vector model. In the case of personalized object filter, this condition is kept because the user’s context and object’s In recall and precision method, precision is defined as the context are represented in 5W1H formats, i.e. t is equal to 6. proportion of retrieved information that is relevant as follows. However, it is not natural to restrict the vector space of each term in the user’s context and object’s context as the same w space. Thus, we simply count the number of query terms which P= (5) n2 an object contains when we filter out the preferred contents from the retrieved information in response to user’s context. Recall is defined as the proportion of relevant information that E. Personalized Information Presentation retrieved as follows. In general, the output from the retrieved information is rarely the exact set of information desired by the user in response to a w request [5]. Even if the retrieved information were precisely R= (6) those that the user wanted, there exist few information retrieval n1 systems that produce the personalized contents responding to the user’s preference. On the other hand, the output from the Precision and recall are both bounded above by 1 (when y = 0 retrieved information is limited by the output devices included and x = 0, respectively), and below by 0 (when w = 0). With this in the user. For example, even when the preferable information evaluation method, we can approximately evaluate the is retrieved, if there is not an available output device to display, performance of personalized information retrieval system. it may not be possible to present a satisfactory result to the user. Consequently, the information retrieval system fails apparently to present personalized information. Therefore, personalized IV. EXPERIMENTAL RESULTS information retrieval system requires that the preferred contents In this Section, we will demonstrate the effectiveness of the of the retrieved information should be reformatted based on a proposed framework with a practical example and a prototype user’s context such as transcoding technique [22,23]. In this application in ubiHome test-bed, where we use wear-UCAM as regard, we can utilize any available display device like TV in a fundamental development tool [2,18]. Figure 4 illustrates the ubiquitous computing-enabled home environment. In addition, experimental setup in ubiHome. it should support adequate output devices such as retinal display for personalized presentations [24,25]. However, we INF1 assumed that a user has or is able to acquire an appropriate INF0 output device. As explained in personalized information filter, Obj1 MRWindow ubiTV Obj0 we already have the filtered information based on the user’s context. Thus, personalized information presentation is nothing but to display the filtered contents with an adequate manner. F. Information Retrieval Measures A viable information retrieval system must be effective in returning information in response to a user’s request. While this CONTEXTA generally means that most of the retrieved information in Wearer A response to the user’s request should be judged by the user to be appropriate to the information need, such a vague statement of effectiveness provides no solid basis for determining how good wear-UCAM Simulator a given system is, or for comparing one retrieval system to Fig. 4. An Experimental Setup for Personalized Information Retrieval System. another [4-6]. Thus, we employ recall and precision method to ubiPCMM 2005 87 For the brevity of simplicity, we exploit only two objects, i.e. following strategy. If each element of terms in the 0-1 vector is ubiTV and MRWindow, in ubiHome. Furthermore, the two corresponding to 1, we assign it to a vector form of ASCII objects have their own properties or information as the unified values (string) because a string can be comparable. With this context format. Meantime, we acquire user’s context from the representation, we can transform user’s context to weighted explicit user’s inputs, which is also represented as 5W1H Boolean query form. For the brevity of simplicity, we assumed format. As shown in Figure 4, the wear-UCAM simulator is that we are interested in when and what terms, where the used on behalf of a wearable computer, and it utilizes several weights of the two terms are 1 and rest of them are 0. Another sensors to acquire user’s preference. Although we did not take assumption is that each weight of the terms is acquired from the account of actual sensors to acquire user’s context for this user’s explicit input. Therefore, the transformed user’s context experiment, it might be possible to exploit virtual sensors from is wear-UCAM toolkit. Meanwhile, a general description of user’s context can be described as “I want ubiTV service to be Q' = < 0, < “Morning” >, 0, < “ubiTV”, “Channel 39” >, 0, 0 >, turned on if I am at home in the evening and my stress is relatively high.” Regarding this description of the user’s where we present a string instead of a vector form of ASCII context, we will show how to exploit user’s context in the values. However, it must be noted that user’s context and proposed framework. Furthermore, we assumed that the user is object’s information are definitely coincident. Likewise, we interested in when and what terms only. can also achieve the transformed Obj' from the available Given the description of user’s context, we can generate objects. However, each term of objects reflects current status of contextual query. Figure 5 illustrate some parts of user’s them in order to deliver right information when the user pays context in XML and its contextual query in the temporary data attention to an object. It is often useless to deliver information structure. when the user has lost his/her attention to the object [7]. The Unified Context of User’s Contextual Query in transformed objects are Context in XML Temporary Data Structure Obj'0 = < <…>, < “Morning” >, <…>, < “ubiTV”, “Channel 37” >, : <…>, <…> >, Morning Obj'1 = < <…>, < “Morning” >, <…>, < “MRWindow”, “Flower” >, “Morning” : <…>, <…> >. ubiTV where we have only presented when and what terms because “ubiTV” : Channel 37 the rest of terms are canceled out in the computation of the matching function. : “Channel : 37” After context transformation, we are able to select an object of interest and retrieve preferred information through personalized object filter and personalized information filter, respectively. In the selection of an object, we use the matching Internal Context Representation function as shown in (3). However, if qi and obji include a set of Fig. 5. An Example of User’s Context and Contextual Query. strings, then the comparison of strings proceeds in advance. After string comparison, we just take a summation of each As shown in Figure 5, a user’s context can be described in element. For example, if q3 = < “ubiTV”, “Channel 37” > and markup scheme model, XML. This markup scheme enables us obj0,3 = < “ubiTV”, “Channel 37” >, then q3·obj0,3 = < 1, 1 > = 2. to describe the user’s context with various attributes, and it is a On the other hand, q3 = < “ubiTV”, “Channel 37” > and obj1,3 = well-defined document structure. Moreover, we can determine < “MRWindow”, “Flower” >, then q3·obj1,3 = < 0, 0> = 0. how many elements each term has in advance because of DTD Another tactic in the computation of denominator in the (Document Type Definition) in the used XML for representing matching function is to count the number of elements in each the unified context [14,26,27]. Therefore, we can keep the term as a numerical value. For example, Σ(qi)2 = framework consistent from any contextual description model as |02+12+02+22+02+02| = 5. With this auxiliary computation well as keep user’s context exchangeable among context-aware method, each value of the matching function is evaluated as applications. Similarly, we can also represent two objects as an follows. object database. Owing to the unified context representation of user’s context M(Q',Obj'0) = 0.4472, M(Q',Obj'1) = 0.1491 and object’s information as well as internal data structures for them, we can simply transform them to the weighted Boolean As a result, we can select ubiTV as the object of interest because format. Regarding weights, we acquired weight of each term it is larger than MRWindow object. However, the results of the from user’s explicit input. First of all, each term of user’s matching function are not exactly what we expected because context and object’s information is represented as a 0-1 vector. we treat the rest of terms in Obj'0 and Obj'1 as 1 in this Second of all, we convert each term to a numerical value by the computation. Thus, we need more investigation on context Dongpyo Hong, Yun-Kyung Park, Jeongwon Lee, Vladimir Shin, and Woontack Woo 88 transform, especially a numeric conversion from a string. In the to define satisfactory. In this regard, it is also hard to exploit mean time, in the case of retrieving only one object from others, precision and recall as a measurement of information retrieval we do not have to consider Th value because we take an object system without enough experiences on this metrics. Thus, we with the largest value in M(Q',Obj'k). However, the threshold briefly explain how to use it in practice. Typically, value Th is an important factor to determine the performance of precision-recall values are computed at fixed recall levels from information retrieval system. 0.1 to 1.0 in steps of 0.1 [1, 10]. With the retrieved object of interest, now we have to present Figure 7 shows similarity between query and collection of appropriate contents only. In this application, the user wants to object, where we might determine a threshold value for retrieve information of “Channel 37” in the morning among information retrieval dynamically. other properties. In this case, we simply retrieve “Channel 37” properties recursively from the object’s information in object database. Therefore, users are able to retrieve personalized information only based on their own context. Regarding time complexity of the proposed framework, we employ Big W(n) notation, especially the worst case. As we indicated, context transformation is a key feature in the proposed framework. However, it only requires at most W(mmax), where mmax is simply determined by maximum number of element in terms. Meanwhile, the worst case of personalized object filter occurs when all the elements of each term in either Q' or Obj' have a set of strings. If there are m1, m2, m3, m4, m5 and m6 elements in each term, then worse case of time complexity is still W(mmax) in the proposed framework. However, we can reduce time complexity by assigning the Fig. 7. Similarity Graph between User’s Context and Object’s Information. weights of irrelevant terms as 0. Thus, we can guarantee that Lastly, we would like to mention context transformation personalized information retrieval algorithm is linearly which transforms arbitrary contextual information to numerical dependent on the number of elements in context. values. While we were developing this framework, we found To evaluate the effectiveness of an information retrieval that if we transform context to any numerical expression, then system, it is common to compute the precision-recall values. we could apply many mathematical approaches to information Thus, we conduct experiments with virtual data sets in order to retrieval systems. This explains why the previous research on evaluate overall performance of the proposed framework. The information retrieval is mainly on a specific domain, where we experimental conditions are as follows. We randomly generate can easily employ the pre-knowledge or predefined tables for 10 sample sets which represent objects’ information as numerical references on contextual information like Ontology. numerical values. Likewise, we generate a sample query set for In this regard, we believe that context presentation scheme is user’s context. The number of relevant objects is 5. Figure 6 fundamentally required, and its numerical transformation is illustrates the plotted graph of precision and recall. also a corner stone for not only personalized information retrieval but also other context related research areas. V. DISCUSSION AND FUTURE WORKS In this paper, we have proposed a framework of personalized information retrieval system for wearers in ubiquitous computing environment with an example of deterministic approach, a weighted Boolean. The proposed framework exploits user’s context within the well defined components when the user is trying to retrieve information from objects in an environment. In particular, we transform user’s context to the contextual query statement, which is represented as a weighted Boolean. Thus, the proposed framework enables us to retrieve only preferred information to any output device in the given environment. However, we still need more investigations Fig. 6. Precision and Recall Graph of the Proposed Personalized Information Retrieval Framework. on the following issues in order to realize personalized information retrieval system in the future computing Using figures based on precision and recall requires environment. For instance, 1) how to transform contextual knowledge a priori of relevance, i.e., the answer set. However, information of users and objects to numerical values simply but this is not possible in general. Moreover, relevance is difficult ubiPCMM 2005 89 yet effectively, 2) how to deliver the retrieved information to [20] Frei H.P. and Qiu Y., “Effectiveness of Weighted Searching in an Operational IR Environment,” In Schriften zur Informationswissenschaft, the user appropriately i.e., where and when to show, and 3) how Band 12, pages 41-51, UVK, Universitätsverlag, Konstanz, 1993. to apply time factor in user’s preference because user’s [21] H. Chen, B. R. Schatz, C. Lin , “Concept Classification and Search on interests are varying on time, which apparently includes Internet Using Machine Learning and Parallel Computing Techniques,” feedbacks from either the system or the user. World Wide Web 4 Conference, Boston, MA, Pages 58-59, Poster Proceedings, December 11-14, 1995. [22] R. Han, P. Bhagwat, "Dynamic Adaptation In an Image Transcoding Proxy For Mobile Web Browsing", IEEE Personal Communications REFERENCES Magazine, Dec. 1998, pp. 8-17. [23] Claudio Pinhanez, "The Everywhere Displays Projector: A Device to [1] M. Weiser, "The Computer for the 21st Century," Scientific American, pp. Create Ubiquitous Graphical Interfaces," Proc. of Ubiquitous Computing 94-104, Sep. 1991. 2001 (Ubicomp'01), Lecture Notes In Computer Science; Vol.2201, [2] Seiie Jang, Choonsung Shin, Yoosoo Oh, and Woontack Woo, pp.315-331, Atlanta, Georgia, September 2001. "Introduction of “UbiHome” Testbed," The first Korea/Japan Joint [24] http://www.microvision.com Workshop on Ubiquitous Computing & Networking Systems 2005 [25] M. von Waldkirch, P. Lukowicz, G. Tröster, "Spectacle-Based Design of (ubiCNS2005), Proceeding CD, 2005. Wearable See-Through Display for Accommodation-Free Viewing," [3] B.J.Rhodes, P.Maes, “Just-in-time information retrieval agents,” IBM Pervasive Computing, pp.106-123, Vienna, Austria, 18.-23. April 2004. SYTEMS Journal, vol. 39, No. 3&4, pp.685-704, 2000. [26] http://www.w3c.org/XML/ [4] William B. Frakes, Ricardo Baeza-Yates, “Information Retrieval: data [27] http://java.sun.com/xml/jaxp/dist/1.1/docs/tutorial/overview/3_apis.html structures and algorithms,” Prentice Hall, 1992. [5] Korfhage, Robert R., “Information storage and retrieval,” John Dongpyo Hong received his B.S. degree in Computer Wiley&Sons, 1997. Engineering from Dong-A University, Busan, Korea, [6] C. J. van Rijsbergen, “Information Retrieval,” Butterworths, London, in 2001 and M.S. degree in Information and second edition, 1979. Communications from Gwangju Institute of Science [7] S.Julier, M.Lanzagorta, Y.Baillot, L.Rosenblum, S.Feiner, T.Höllerer, and Technolgoy (GIST), Gwangju, Korea in 2004. and S.Sestito, “Information filtering for mobile augmented reality”. In Now he is a Ph.D student in GIST since 2004. Proc. ISAR '00 (Int.Symposium on Aug-mented Reality), pages 3-11, Research Interest: HCI, Information Retrieval, Munich,Germany, October 5-6 2000. Context-awareness, Vision based User Interface, [8] Seth Insley, “Obstacle to General Purpose Augmented Reality,” Wearable computing, Ubiquitous computing, VR, and http://islab.oregonstate.edu/koc/ece399/f03/final/insley2.pdf. Entertainment Computing. [9] G.J.F. Jones, P.J. Brown., "Context-aware retrieval for ubiquitous computing environments," Invited paper in Mobile and ubiquitous Yun-Kyung Park Yun-Kyung Park received her B.S. information access, Springer Lecture Notes in Computer Science, Vol. degrees in mathematic education from Korea 2954, pp. 227-243, 2004. University, Korea, in 1987. Since 1987, she has been [10] Patrick A. S. Sinclair, Kirk Martinez, David E. Millard, and Mark J. Weal, working at Electronics and Telecommunications “Augmented Reality as an Interface to Adaptive Hypermedia Systems”, In Research Institute, and she is a principal member of New Review of Hypermedia and Multimedia, Special Issue on engineering staff of the Post-PC Platform Research Hypermedia beyond the Desktop. Vol. 9, pp.117-136, 2003. Team. Her research interests include [11] Simon Julier, Yohan Baillot, Marco Lanzagorta, Dennis Brown, context-awareness, and sensor-networking. Lawrence Rosenblum, "BARS: Battlefield Augmented Reality System", NATO Symposium on Information Proc-essing Techniques for Military Systems, 9-11 October 2000, Istanbul, Turkey. Jeongwon Lee received his B.S. in Electrical [12] B. Bell, S. Feiner, and T. Hollerer, “Information at a Glance,” IEEE Engineering, Ohio University, Ohio, USA, in 1989 Computer Graphics and Applications, Vol. 22, No. 4, pp. 6-9, July/August and M.S. in Electrical Engineering, Univ. of 2002. Wisconsin, Madison, USA in 1991. He received his [13] A.K.Dey and G.D.Abowd, “Towards a better understanding of context Ph.D. in Computer Science, Univ. of Southern and contextawareness,” In the Workshop on the What, Who, Where, California, California, USA. In 2002, as a research When and How of Context-Awareness, affiliated with the 2000 ACM associate, he worked in Integrated Media Systems Conference on Human Factors in Computer Systems. 2000. Center, California, USA. In 2002, as an Assistant [14] Albrecht Schmidt, "Implicit Human Computer Interaction Through Professor, he joined Digital Contents, Sejong Context," Personal Technologies Volume 4(2&3), June 2000, pp. University, Seoul, Korea. Research Interest: 191-199, 2000. Computer Graphics, Augmented Reality, Virtual [15] Seiie Jang, Woontack Woo, "Unified Context Describing User-Centric Reality, Computer Vision, Digital Contents, Computer Games, and Ubiquitous Situation: Who, Where, What, When, How and Why," The first Computing. Korea/Japan Joint Workshop on Ubiquitous Computing & Networking Systems 2005 (ubiCNS2005), Proceeding CD, 2005. Vladimir Shin received the B.Sc. degree and M.Sc. [16] Karen Henricksen,Jadwiga Indulska, and Andry Rakotonirainy, degree in applied mathematics from Moscow State "Modeling Context Information in Pervasive Computing Systems," Aviation Institute, Russia, in 1977 and 1979, Proceedings of the First International Conference on Pervasive respectively. In 1985 he received the Ph.D. degree in Computing, Lecture Notes In Computer Science, Vol. 2414, pp. 167-180, mathematics at the Institute of Control Science, 2002. Russian Academy of Sciences, Moscow. From 1984 to [17] T. Strang, C. Linnhoff-Popien, "A Context Modeling Survey," First 1999 he was Head of the Statistical Methods Lab. at International Workshop on Advanced Context Modelling, Reasoning And the Institute of Informatics Problems, Russian Management UbiComp 2004, Nottingham, England, September 7, 2004. Academy of Sciences, Moscow. During 1995 he was [18] Dongpyo Hong and Woontack Woo, “wear-UCAM: A Toolkit for Visiting Professor in the Mathematics Department at Wearable Computing,” The first Korea/Japan Joint Workshop on Korea Advanced Institute of Science and Technology. From 1999 to 2002 he Ubiquitous Computing & Networking Systems 2005 (ubiCNS2005), was Member of Technical Staff at the Samsung Institute of Technology. He is Proceeding CD, 2005. currently an Associate Professor at Gwangju Institute of Science and [19] J. Choi, M. Kim, and V. V. Raghavan, “Adaptive feedback methods in an Technology, South Korea. His research interests include estimation, filtering, extended boolean model,” In Proceedings of ACM SIGIR Workshop on tracking, data fusion, stochastic control, identification, and other Mathematical/Formal Methods in Information Retrieval, New Orleans, multidimensional data processing. He has authored or coauthored over 50 LA, Sept. 2001. papers in these fields. Dongpyo Hong, Yun-Kyung Park, Jeongwon Lee, Vladimir Shin, and Woontack Woo 90 Woontack Woo received his B.S. degree in EE from Kyungpook National University, Daegu, Korea, in 1989 and M.S. degree in EE from POSTECH, Pohang, Korea, in 1991. He received his Ph. D. in EE-Systems from University of Southern California, Los Angeles, USA. During 1999-2001, as an invited researcher, he worked for ATR, Kyoto, Japan. In 2001, as an Assistant Professor, he joined Gwangju Institute of Science and Technology (GIST), Gwangju, Korea and now at GIST he is leading U-VR Lab. Research Interest: 3D computer vision and its applications including attentive AR and mediated reality, HCI, affective sensing and context-aware for ubiquitous computing, etc.