-In this paper, we propose a framework of personalized information retrieval system for a wearer (a person who is equipped with a wearable computer) in ubiquitous computing environments. In ubiquitous computing environments, personalized information retrieval is indispensable for a wearer because desired or undesired information will be flooded around the wearer. Although there have been many research activities on information retrieval in wearable computing and/or ubiquitous computing environments, personalization has not been recognized significantly. Accordingly, all the retrieved information is exposed to the wearer regardless of his/her situations or conditions. In this regard, the proposed framework enables a wearer to retrieve the personalized information from objects. In the proposed framework, we exploit user's context as a fundamental element in retrieving the personalized information. The proposed framework consists of two conceptual stages (object selection and information presentation) and each stage includes several components. In order to measure the effectiveness of the proposed framework, we introduce a measuring method and also realize a prototyping system for personalized information retrieval. Thus, we believe that the proposed personalized information retrieval framework is to leverage human-computer interactions for a wearer in ubiquitous computing environments.
Wcomputers and communications, the future computing
ITH the rapid progress of technologies in the areas of
environments will support seamless interactions from
ubiquitous computers and pervasive networking. That is, users
could do just-in-time access to any (invisible) computer at any
time and any where [
There have been many research activities on information
retrieval to find an appropriate document from database or
libraries [
In this paper, we propose a framework of personalized
information retrieval system for a wearer (a person who is
equipped with a wearable computer) in ubiquitous computing
environments. Moreover, we suggest an effective means to
exploit user’s context through this framework. In the proposed
framework, we separate the procedure of personalized
information retrieval into two parts. The first part is object
selection, which includes context transform and personalized
object filter components. In the second part, personalized
information filter and personalized information presentation
components are included. Besides these components, the
proposed framework also includes context representation
component. Before we introduce each component in the
proposed framework, we assume that any information of an
object as well as user’s context, namely user’s profile or
preference, can be represented as a unified context. We name
this process as context representation [
Therefore, the proposed framework helps developers to
implement personalized information retrieval system due to the
structured components and easier exploitation of user’s context.
In addition, we believe that personalized information retrieval
enables a wearer to do natural interactions with computing
resources like augmented virtual object or information in
ubiquitous computing environment. That is, the retrieved
information is presented differently corresponding to user’s
context such as user’s preference. Consequently, it leverages
human-computer interactions of a wearer to retrieve
personalized information from a large volume of information
around the wearer. In order to realize the proposed framework,
we exploit wear-UCAM as an empirical tool, a toolkit for
context-aware applications, because context-awareness
mechanism and other useful utilities are implemented for
developers [
This paper is organized as follows. In Section II, we review the related works which address the necessity of information retrieval for wearable computing, mobile AR, and ubiquitous computing environment. In Section III, we explain the fundamental components for the proposed framework of personalized information retrieval with a specific example. In Section IV, we show the experimental results and effectiveness of the proposed framework through the introduced measuring method. Finally, we discuss future works in Section V.
In general, research on information retrieval has been focused on how to retrieve the requested information from a large volume of data, particularly documents, even images in the Internet (WWW), database, etc. Over the last decade, the concept of information retrieval has been moved to mobile computing, wearable computing, augmented reality (AR), and ubiquitous computing in order to retrieve only needed information because there will be too much information in such computing paradigms. In this section, we review the most relevant works about information retrieval, and then we extract fundamental components for personalized information retrieval system.
In [
In [
In [
In [
In [
In [
In this review, we observed that many researchers have addressed the importance of information retrieval in the future computing environments. Although most researchers point out the usefulness of user’s context in information retrieval, there are few works to reveal how to exploit user’s context and its detailed manipulation for personalized information retrieval system in such computing environments. As we have reviewed, there are many practical applications which show the effectiveness of information retrieval. However, there is little tendency to show the detailed explanations of the information retrieval algorithms supported by mathematical derivations and evaluations. These have inspired us to investigate on the framework for personalized information retrieval system with mathematical models.
III. PERSONALIZED INFORMATION RETRIEVAL FRAMEWORK
From the previous sections, we could define that
personalized information retrieval is to retrieve appropriate
information from a large volume of data (or information) with a
wearer’s context, namely preference or profile, and also to
present the retrieved information appropriately based on the
wearer’s context in ubiquitous computing environment where
any information could be augmented by anyone. As shown in
Figure 1, information or property of any object and wearer can
be represented as a unified context [
INF0
INF1 Obj0
Obj1
INFi Obji
Unified Context
INFN-1 ObjN-1
Result A
Result B Query A
Personalized Information
Query B ② INFORMATION PRESENTATION
Personalized Information Retrieval Framework Fig. 2. Personalized Information Retrieval Framework.
There are many definitions of context among researchers
[
ENV Obj0 Obj1 Obji ObjN-1
Context
Representation
too complicated to utilize in the information retrieval system. In
addition, the unified context representation allows us to access
contextual information in a structured manner when we try to
compare contexts, select an object, and retrieve information of
the object in the proposed framework. Generally, the unified
contexts can be implemented by key-value models, markup
scheme models, graphical models, object oriented models,
logic based models, or ontology based models [
CONTEXT QUERY Q <who> <when> <where> <what> <how> <why> CT(Q) <Twho> <Twhen> <Twhere> <Twhat> <Thow> <Twhy>
CONTEXT TRANSFORMATION CT(Obj) <Owho> <Owhen> <Owhere> <Owhat> <Ohow> <Owhy> Obj0 <who> <when> <where> <what> <how> Obj1 <who> <when> <where> <what> <how> ● ● ● ● ● ● ● ● ● ● ObjN-1 <who> <when> <where> <what> <how> <why>
OBJECT DATABASE Fig. 3. An Example of Temporary Data Structures. <why> <why> ● ●
Filter M(Q,Obj) DESIRED OBJECT
As shown in Figure 3, a user’s context can be represented as Q and available objects can be represented as a vector of Obj respectively. Furthermore, their elements in the set are vector types. However, this internal data structure is nothing but data container of the user’s context and of available objects, thereby we need to transform it into another form, which is explained in the next subsection.
In general, users would always retrieve exact and perfect information to their requests from a given system. However, this is only possible when the request is exactly coincident and information in the system can be definitely identified as responding to the query. For example, the most obvious exact matching arises with a numerical representation of information. Although the context representation of 5W1H format has many advantages, it is hard to be used directly in the proposed framework because the context representation is just a data container. Therefore, user’s context and object’s information needs to be transformed to an appropriate format for easier comparison such as Boolean or numerical format without loss of advantages in the context representation. We define this transformation as context transformation, and it can be represented as follows.
X ′ = C T ( X ) (1) where X is an arbitrary context, X' is the transformed context, and CT represents context transformation.
In context transformation, any transformation method is
applicable if the transformed contexts are definitely
comparable to each other. In the proposed framework, either
deterministic transformation or probabilistic transformation is
possible. However, we exemplify Boolean transformation
method in a deterministic transformation for the sake of
simplicity. Typically the connectives in Boolean permitted are
AND, OR, and NOT. In the Boolean model of retrieval, the
similarity between an object and a query is based on the
presence of terms in both the query and the object. In spite of
the simplicity of Boolean model, they present a number of
significant problems [
Thus, we can represent user’s context as a weighted Boolean query as follows.
Q′ = T who * T when * T where * T what * T how * T why w1 w2 w3 w4 w5 w6 (2) where T5W1H are query terms, w1…6 are the weights (either 0 or 1), and * represents a Boolean or logical operation. Although sum of weight is equal to 1, the term of weight used in context transformation only indicate whether the transformed query includes terms or not. Thus, the total sum of weights need not be 1. As shown in (2), user’s context can be transformed weighted Boolean query format, where each term of query is correspondent to each term of 5W1H. In case of context transformation of objects, however, we just take weight of each term as 1.
In general, the weighted Boolean operations are relatively
complicated [
Ultimately, the goal of information retrieval is to find a match between a given query and those objects that the user would like to retrieve in response to the query. The matching process is complicated by the fact that the query and the objects may have quite different forms. Through the context transformation, however, we are able to cope with this complication. On the other hand, a term in an object has been matched to a term in a query does not automatically mean that the object should be retrieved in response to the query. In the first place, the query probably contains more than one term, so the success or failure in matching each of the query terms must be considered. In the second place, an object which contains a given query term does not mean that the object is strongly related to the term. Therefore, matching criteria is a key factor in personalized information retrieval. After we briefly examine the types of matches that can be used, we will show a criterion as an example of personalized object/information filter.
In general, the matching criteria in information retrieval are
categorized as follows: exact match, range match and
approximate match [
We focus on evaluating a match between an object (Obj')
and a query (Q'). In addition, we exploit the cosine
measurement which is widely used as a matching algorithm in
information retrieval systems [
M (Q′, Objk′ ) = t−1 ∑ qiobji i=0 ⎜⎛ ∑t−1 (qi )2 ∑t−1 (obji )2 ⎟⎞ ⎝ i=0 i=0 ⎠ 1/ 2 (3)
With the matching function (3), we can retrieve the object of interest as the following criterion. Suppose there are N objects in the environment, then the retrieving proceeds by calculating values M(Q',Obj'k), where k is an arbitrary number in N. Through this calculation, the set of objects to be retrieved is determined. In mathematical point of view, this function is the inner product of the query and object vectors, where 1 represents the highest similarity. Thus, the function M transforms the angular measure into a measure ranging from 1 for the highest similarity to 0 for the lowest. In general, there are two approaches to retrieve objects of interest: threshold and rank position approach. However, we adopt threshold approach because it is intuitive and widely used. When the matching function is given a suitable threshold, we can retrieve the objects above the threshold and discard the ones below. If Th is the threshold value, then the retrieved object set is represented as Ô = {Obj'k | M(Q',Obj'k) > Th}.
On the other hand, we have to keep the space or length, t, of the query and object as the same space in order to exploit the vector model. In the case of personalized object filter, this condition is kept because the user’s context and object’s context are represented in 5W1H formats, i.e. t is equal to 6. However, it is not natural to restrict the vector space of each term in the user’s context and object’s context as the same space. Thus, we simply count the number of query terms which an object contains when we filter out the preferred contents from the retrieved information in response to user’s context.
In general, the output from the retrieved information is rarely
the exact set of information desired by the user in response to a
request [
A viable information retrieval system must be effective in
returning information in response to a user’s request. While this
generally means that most of the retrieved information in
response to the user’s request should be judged by the user to be
appropriate to the information need, such a vague statement of
effectiveness provides no solid basis for determining how good
a given system is, or for comparing one retrieval system to
another [
Fig. 4. An Experimental Setup for Personalized Information Retrieval System. measure the effectiveness of personalized information retrieval system because it is the most widely used.
In recall and precision measurement, we can establish a 2 x 2 contingency table which shows how the information set is divided by both relevance and retrieval. Table I shows the relationships of these two measures.
Not Relevant (5) (6) w P = n2 w R = n1 Recall is defined as the proportion of relevant information that retrieved as follows.
Precision and recall are both bounded above by 1 (when y = 0 and x = 0, respectively), and below by 0 (when w = 0). With this evaluation method, we can approximately evaluate the performance of personalized information retrieval system.
In this Section, we will demonstrate the effectiveness of the
proposed framework with a practical example and a prototype
application in ubiHome test-bed, where we use wear-UCAM as
a fundamental development tool [
ubiTV
Obj0
INF0
INF1 Obj1
For the brevity of simplicity, we exploit only two objects, i.e. ubiTV and MRWindow, in ubiHome. Furthermore, the two objects have their own properties or information as the unified context format. Meantime, we acquire user’s context from the explicit user’s inputs, which is also represented as 5W1H format. As shown in Figure 4, the wear-UCAM simulator is used on behalf of a wearable computer, and it utilizes several sensors to acquire user’s preference. Although we did not take account of actual sensors to acquire user’s context for this experiment, it might be possible to exploit virtual sensors from wear-UCAM toolkit. Meanwhile, a general description of user’s context can be described as “I want ubiTV service to be turned on if I am at home in the evening and my stress is relatively high.” Regarding this description of the user’s context, we will show how to exploit user’s context in the proposed framework. Furthermore, we assumed that the user is interested in when and what terms only.
Given the description of user’s context, we can generate contextual query. Figure 5 illustrate some parts of user’s context in XML and its contextual query in the temporary data structure.
Unified Context of User’s Context in XML <?xml version="1.0"?> <!DOCTYPE Context SYSTEM "Context.dtd"> <<CWohn:etenx>t> <TimeStamp Type="Symbolic">Morning</TimeStamp> <<W/Wh:haetn>> <Destination Type="Object"> <DName Type="MultiMedia">ubiTV</DName> </Destination> <<</M::MPaaannriapipmuulaelattietoiorn>n>C>hannel 37</Parameter> <<//CWo:hnatet>xt> Contextual Query in Temporary Data Structure
“Morning” <who> <when> <where> <what> <how> <why> “ubiTV” “Channel 37”
Fig. 5. An Example of User’s Context and Contextual Query.
As shown in Figure 5, a user’s context can be described in
markup scheme model, XML. This markup scheme enables us
to describe the user’s context with various attributes, and it is a
well-defined document structure. Moreover, we can determine
how many elements each term has in advance because of DTD
(Document Type Definition) in the used XML for representing
the unified context [
Owing to the unified context representation of user’s context and object’s information as well as internal data structures for them, we can simply transform them to the weighted Boolean format. Regarding weights, we acquired weight of each term from user’s explicit input. First of all, each term of user’s context and object’s information is represented as a 0-1 vector. Second of all, we convert each term to a numerical value by the following strategy. If each element of terms in the 0-1 vector is corresponding to 1, we assign it to a vector form of ASCII values (string) because a string can be comparable. With this representation, we can transform user’s context to weighted Boolean query form. For the brevity of simplicity, we assumed that we are interested in when and what terms, where the weights of the two terms are 1 and rest of them are 0. Another assumption is that each weight of the terms is acquired from the user’s explicit input. Therefore, the transformed user’s context is
Q' = < 0, < “Morning” >, 0, < “ubiTV”, “Channel 39” >, 0, 0 >,
where we present a string instead of a vector form of ASCII
values. However, it must be noted that user’s context and
object’s information are definitely coincident. Likewise, we
can also achieve the transformed Obj' from the available
objects. However, each term of objects reflects current status of
them in order to deliver right information when the user pays
attention to an object. It is often useless to deliver information
when the user has lost his/her attention to the object [
After context transformation, we are able to select an object of interest and retrieve preferred information through personalized object filter and personalized information filter, respectively. In the selection of an object, we use the matching function as shown in (3). However, if qi and obji include a set of strings, then the comparison of strings proceeds in advance. After string comparison, we just take a summation of each element. For example, if q3 = < “ubiTV”, “Channel 37” > and obj0,3 = < “ubiTV”, “Channel 37” >, then q3·obj0,3 = < 1, 1 > = 2. On the other hand, q3 = < “ubiTV”, “Channel 37” > and obj1,3 = < “MRWindow”, “Flower” >, then q3·obj1,3 = < 0, 0> = 0. Another tactic in the computation of denominator in the matching function is to count the number of elements in each term as a numerical value. For example, Σ(qi)2 = |02+12+02+22+02+02| = 5. With this auxiliary computation method, each value of the matching function is evaluated as follows.
M(Q',Obj'0) = 0.4472, M(Q',Obj'1) = 0.1491 As a result, we can select ubiTV as the object of interest because it is larger than MRWindow object. However, the results of the matching function are not exactly what we expected because we treat the rest of terms in Obj'0 and Obj'1 as 1 in this computation. Thus, we need more investigation on context transform, especially a numeric conversion from a string. In the mean time, in the case of retrieving only one object from others, we do not have to consider Th value because we take an object with the largest value in M(Q',Obj'k). However, the threshold value Th is an important factor to determine the performance of information retrieval system.
With the retrieved object of interest, now we have to present appropriate contents only. In this application, the user wants to retrieve information of “Channel 37” in the morning among other properties. In this case, we simply retrieve “Channel 37” properties recursively from the object’s information in object database. Therefore, users are able to retrieve personalized information only based on their own context.
Regarding time complexity of the proposed framework, we employ Big W(n) notation, especially the worst case. As we indicated, context transformation is a key feature in the proposed framework. However, it only requires at most W(mmax), where mmax is simply determined by maximum number of element in terms. Meanwhile, the worst case of personalized object filter occurs when all the elements of each term in either Q' or Obj' have a set of strings. If there are m1, m2, m3, m4, m5 and m6 elements in each term, then worse case of time complexity is still W(mmax) in the proposed framework. However, we can reduce time complexity by assigning the weights of irrelevant terms as 0. Thus, we can guarantee that personalized information retrieval algorithm is linearly dependent on the number of elements in context.
To evaluate the effectiveness of an information retrieval system, it is common to compute the precision-recall values. Thus, we conduct experiments with virtual data sets in order to evaluate overall performance of the proposed framework. The experimental conditions are as follows. We randomly generate 10 sample sets which represent objects’ information as numerical values. Likewise, we generate a sample query set for user’s context. The number of relevant objects is 5. Figure 6 illustrates the plotted graph of precision and recall.
Using figures based on precision and recall requires
knowledge a priori of relevance, i.e., the answer set. However,
this is not possible in general. Moreover, relevance is difficult
to define satisfactory. In this regard, it is also hard to exploit
precision and recall as a measurement of information retrieval
system without enough experiences on this metrics. Thus, we
briefly explain how to use it in practice. Typically,
precision-recall values are computed at fixed recall levels from
0.1 to 1.0 in steps of 0.1 [
Figure 7 shows similarity between query and collection of object, where we might determine a threshold value for information retrieval dynamically.
Fig. 7. Similarity Graph between User’s Context and Object’s Information.
Lastly, we would like to mention context transformation which transforms arbitrary contextual information to numerical values. While we were developing this framework, we found that if we transform context to any numerical expression, then we could apply many mathematical approaches to information retrieval systems. This explains why the previous research on information retrieval is mainly on a specific domain, where we can easily employ the pre-knowledge or predefined tables for numerical references on contextual information like Ontology. In this regard, we believe that context presentation scheme is fundamentally required, and its numerical transformation is also a corner stone for not only personalized information retrieval but also other context related research areas.
In this paper, we have proposed a framework of personalized information retrieval system for wearers in ubiquitous computing environment with an example of deterministic approach, a weighted Boolean. The proposed framework exploits user’s context within the well defined components when the user is trying to retrieve information from objects in an environment. In particular, we transform user’s context to the contextual query statement, which is represented as a weighted Boolean. Thus, the proposed framework enables us to retrieve only preferred information to any output device in the given environment. However, we still need more investigations on the following issues in order to realize personalized information retrieval system in the future computing environment. For instance, 1) how to transform contextual information of users and objects to numerical values simply but yet effectively, 2) how to deliver the retrieved information to the user appropriately i.e., where and when to show, and 3) how to apply time factor in user’s preference because user’s interests are varying on time, which apparently includes feedbacks from either the system or the user.