In an effort to improve users' search experiences during their information seeking process, providing a personalized information retrieval system is proposed to be one of the effective approaches. To personalize the search systems requires a good understanding of the users. User modeling has been approved to be a good method for learning and representing users. Therefore many user modeling studies have been carried out and some user models have been developed. The majority of the user modeling studies applies inductive approach, and only small number of studies employs deductive approach. In this paper, an EISE (Extended Information goal, Search strategy and Evaluation threshold) user model is proposed, which uses the deductive approach based on psychology theories and an existing user model. Ten users' interactive search log obtained from the real search engine is applied to validate the proposed user model. The preliminary validation results show that the EISE model can be applied to identify different types of users. The search preferences of the different user types can be applied to inform interactive search system design and development.
•Human-centered computing → methods; User models; •Information systems Personalization;
HCI design and evaluation →
Users and interactive retrieval;
Search engines e.g. (Google and Bing) are the core online
technologies, which are widely used by people for information
seeking. Information retrieval is to query the search engine with
one or multiple iterations [
This paper is divided into the following sections. Section 2 is related work to the current study, section 3 proposes the model, Section 4 describes methodology and Section 5 reports experimentation set up and results.
In this section, some related user behavior theories, psychology
theories and user models will be reviewed. The proposed model
by Wilson in 1981 [
The above user models are developed based on a deductive approach (theory -based). The following section will introduce user models developed based on the inductive approach (user data analysis based). In our view, a combination of the two is need for user model development.
Query Suggestion is one of the popular approaches for
personalization [
In this paper, a user classification model called ISE (Information
Goal, Search strategy and Evaluation threshold) [
Based on the ISE model [
This section will describe the two characteristics of
Information Goal, fixed information goals and evolving
information goals and extend these characteristics with two
mindset theory (Fixed and Growth mind-sets) [
Detail explanation of information goals are delivered in the above sections. In this section the focal point will be one of the characteristic of information goal. Correlation between fixed mind –set and fixed information goal can help us to build a resilient model established psychology theory. From analogy of the theory we can produce an operational definition to distinguish between user behaviors. Operational definitions are a set of rules to quantify user activities.
In the left column of the Table 2 presents Fixed mind-set theory of the two minds theory. The right column of Table 1 describes the search analogy of the Fixed mind-set theory.
From the above conclusion of Table 1 the following operational definitions can be obtained from the description of the theory and the search analogies:
The characteristics of growth mind set can be mapped with evolving information goals. According to two mind set theory people with growth mind can exceed more than fixed mind set but in the model will only adapt the characteristics of the growth mindset to endorse our model. The preference of one mind set has no significance on other mind. Because the study is eager to identify the variance between fixed and evolving information goals.
Fixed jumps are the types of query jumps used when there are no changes to the information goals user. If there is no changes to the user information goals and high number of jumps query are available the user has evoking information goals.
History is an example of queries that are used before in sessions and between sessions. User with evolving information goals will use more repeats to refine the result.
Search strategy is also divided into two types [
To simplify cautious behavior Table 4 can provide a detailed explanation. From the right side of the table properties of cautious behavior can be seen and left side of the table clarify the analogy to information seeking.
The operational definitions are derived from the conclusion of obtained analogy from risky behaviors.
The user will have less number of alterations to the queries.
The user will rely on first page of the results.
The user will click on the immediate link on the page for example on the first link of the page.
The user will have small number of page viewing per session.
User will spend less time per session.
The operational definitions are generated based on the search analogy to enable the application of the Cautious characteristics in user modeling are:
The user will have more alteration to the queries.
The user will move around in the session and open more and more pages beyond first page.
The user will click on many links not only relying on the first links of the page.
The user will view many pages correspond to the query to satisfy his information needs.
User will spend more time per session.
In literature there is no perfect theory to explain risky behavior in contrary to cautious behavior. So it can be appeared opposite to cautious behavior. Right column of Table 5 explains analogy from the left column of the risky behavior. number of result pages to surf all opportunities.
The last set of the characteristics of the ISE model contains
two types of result evaluation threshold, precise evaluation
threshold and weak evaluation threshold. These characteristics
involve decision making, so the study need to get some insight
knowledge of decision making to strength the characteristics of
the user model. According to Herbert A. Simon Nobel Laurent the
process of decision making depends on the available information
and understanding of the information in the required time [
Precise evaluation threshold will be validated from the behaviors of maximizers as shown in Table 6. Left column of the table illustrates the behavior of maximizers and right column shows the obtained analogy from the behaviors. The operational definitions based on the search analogy enable the application of the precise characteristics in user modeling are:
The user will have high use of repetition queries in session and between sessions.
The user will have more clicks as compare to the amount of queries.
The user will have more alteration to the queries.
The user will expend the search for long period and usage of more pages in session.
The user will perform more detail search e.g. more number of page viewing per iteration.
Weak evaluation threshold will be validated from the behaviors of satisfiers as shown in Table 7. Left column of the table explain the behavior of satisfiers and right column shows the obtained analogy from the behaviors.
The user will open less number of pages during the search process.
The user will perform a very little during search process.
View small number of p ages during the search process.
This section describes a preliminary experiment set up for
validating the EISE model. Ten user search log data from Bing
Search Engine is used, which contain a collection of 4231 queries
average of 423 per user and 40217 results average of 4021 per
user. The data log contain anonymous User id, time and date,
query name, page number of search engine page, rank in one
search engine page, URL, dwell time and click count. The Similar
search log is used by a group of researcher to classify interaction
features [
The extracted 24 key interaction features from the search log
data justify the operational definition of each characteristics
prop osed in Section 3. The 24 features include, Average clicks on
page number of search Result, Average Number of result pages
viewed per query , Average Position of each result click on
particular result page, Average Number of query per session,
Average number of result clicks per session, Average time spend
per session total number of query , total Empty result query ,
Average number of result clicks in single query , Total number of
first link clicks, Average view time first link, Total number other
link click, Average view time other links clicked, total number of
repeat, Total Subset, total Super-set, Total number of Overlap,
Total number of Back query , total number Back repeat queries
between sessions, Total number of jump query , Total number
Fixed jump query , Total number of new jump query between
session, Total number of fixed new jump between session.
The whole process of data analysis was done manually as some of
the key features require deeper analysis. From observing the data
the below 10 types of query transitions are prop osed. Five of them
where already defined in the original ISE model for CBIR
(Content based image retrieval) and the rest of other five were
discovered during our data analysis.
• Repeat: Consecutive use of the same query [
The above query iterations are divided into four group s, namely, history (repeat, back repeat, back); iteration (subset, super-set and overlap); Jumps (Jump and new jump); history plus iterations (fixed jump and fixed new jump).
This section report the data analysis results based on the methodology prop osed in the above section. Applying the model on user search data log the result will be shown one by one characteristic wise. The above characteristics of the model overlap each other due to sharing of some operational definitions. The model divides the search process into three main categories Information goals, Search strategy and Evaluation threshold. The information seeking process looks simultaneous so there is a very thin line between these categories. To avoid confusion that from where each category starts if some of the operational definitions of each category shared then maximal results can be achieved from the data.
The selection criteria of users in fixed and evolving information is based on their performance in operational definitions. To further scrutinize the users for information goals the users are examine with low, medium and high performance in there operational definitions.
After analyzing the search data log the operational definitions of each characteristic explained in the model were judged beside the results. Figure 1 is an example of fixed information goals. According to operational definitions of fixed information goals user will have less number of query iterations (subset, super set and overlaps) an example of a user with no iterations in Figure 1 can be seen. The second operational definition is less fixed jumps (fixed jumps and fixed new jumps) which can be seen in Figure.1. There are a very low number of fixed jumps. The last operational definition is low history (repeat, back and back repeat) in this case the user has very high number of histories so this operational definition is not satisfied by Figure 1. Now this user qualifies two of the operational definition so the user is considered fixed in there information goals. In Table 8 shows all of the users that qualify each of the operational definitions related to fixed information goals but only User ID 7 and User ID 3 absorb maximum characteristics of fixed information goals.
3, 7
Figure 2 is an example of a user with evolving information goals. According to the first operational definitions of evolving information goals, user will have high number of iterations (subset, super-set and overlaps). Fixed jumps (fixed jump and fixed new jump ) are also consider iterations and history due to the same information goals. So Figure 2 shows a user with high number of iterations in combination with fixed jumps. The second operational definition is already explained and high fixed jumps can be seen in Figure 2. The last operational definition is High number of history (repeat, back and back repeat). While Figure 2 shows high number of repeats but low number of back and back repeats and fixed jumps also provides history . So not only rely ing on repeats than combine both repeats and fixed jumps user will have high number of histories. The overall performance of Figure 2 show that this is user had evolving information goals. Table 9 shows the overall performance of users with respect to operational definitions related to Evolving information goals. User ID 1, 2, 4, 5, 6, 8, 9 and 10 are identified to have evolving information goals.
Now search strategy is a combination of query iterations and other interaction behaviors. To visualize the interaction behaviors of users can be seen in Figure 1, Figure 2, Figure 3, Figure 4 and Figure 5, to understand the search strategy of the users
In Table 10 User ID 1, 4, 6 and 10 had cautious search strategy. These users are selected on the basis of maximum fulfillment of operational definitions to cautious users. First operational definition is high query iteration as explained in the above section with explanation of Figure 1 and Figure 2 that how the query iteration selection process work. In that context of User ID 1, 2, 4, 5, 6, 8 and 10 have high query iterations it means they have high number subset, super-set and overlaps. Figure 5 (a) illustrate the number of clicks per page this explains our second operational definition, high clicks in other pages compare to first page. So user ID 4, 6 and 10 belongs to this operational definition. Higher position link clicked is third operational definition of this characteristic can be seen in Figure 3 (b) average position of user clicks. Where User ID 2, 3, 4, 8, 9 and 10 had more clicks in other links rather than the first link of the page. The user in Figure 3 (b) cannot be compared with each other but they will be compared with their own clicks. Large number of pages viewed is fourth operational definition shown in Figure 5 (b) users with view number of pages where all users perform on average the same. Figure 4 (a) explain time spend by the user during search iteration and quantified by the average time spend by the users. 591 seconds per session is an average below this is low and above is high on this basis user qualifies the last operational definition. User ID 1, 3, 6, 7 and 10 spend more time than average so they are consider Cautious users. On the basis of these analyses the overall selection of users and selection of users to their corresponding operational definition can be seen in Table 10.
In Table 11 User ID 5 and 9 qualify the operational definitions for risky search strategy. First operational definition is less query iteration User ID 3, 7, 9 have less query iterations it means they have less number of subset, super-set and overlaps and an example of these iteration are shown in Figure 1 and Figure 2. Our second operational definition in Figure 5 (a) can be seen with fewer clicks in other pages compare to first page. So User ID 1, 2, 3, 5, 6, 7, 8 and 9 belongs to this operational definition. The third operational definition is low position link clicked can be seen in Figure 3 (b). Where user ID 1, 5, 6 and 7 had more clicks on first link so they had less effort. Less number of pages viewed is fourth operational definition shown in Figure.5 (b) explained before in the previous section that all of the users preform same on this operational definition. Figure 4 (a) explain time spend by the user during search iteration and 591 seconds per session is an average below this is low so User ID 2, 4, 5, 8 and 9. Spend less time than average so they are considering cautious with this operational definition.
According to the operational definitions of risky search strategy below selected users falls in this category shown in Table.11.
Low query iterations less clicks in other pages compare to first page Lower links position clicked Small number of results pages viewed Spend short time
The final phase of search process, at this stage user has to make a decision to select from one from the results.
On the basis of data analysis User ID 4, 5 and 6 are precise in their selection of result to look in detail let see the data. The first operational definition can be explained with Figure 1 and Figure 2 as example and User ID 1, 2, 6, 7, and 8 are high history (Back, back repeat and also fixed jumps) users. Figure 3 (a) show the comparison of queries with clicks which is our second operational definition and according to the data User ID 4 and 5 belongs to this operational definition. The third definition can be explained with the example of Figure 1 and Figure 2 high query iterations(subset, super-set and overlaps) and User ID 1, 2, 4, 5, 6, 8, 9 and 10 falls in this operational definition. Figure 5 (a) illustrates users with higher pages clicks correspond to fourth operational definition and User ID 4, 8 and 10 belong to this operational definition. The users of last operational definition are selected on the basis of their average interaction (query and clicks) of a user in session. Average interactions is 12 clicks per session so above 12 clicks per session user belong to this definition with the User ID 4, 5, 6, 7 and 8 shown in Figure 4 (b). After analyzing the definition one by one the overall selection can be seen in Table 12.
User ID 3, 9 and 10 have weak evaluation threshold according to their operational definitions. User ID 3, 4, 5, 9 and 10 are low history users and satisfy our first operat ional definition. According to Figure 3 (a) User ID 1, 2, 3, 6, 7, 8, 9 and 10, belongs to our second operational definition low clicks compare to queries. The third definition can be explained with the example of Figure 1 and Figure 2 low query iterations (subset, super-set and overlaps) and User ID 3 and 7 falls to t his operational definition. In Figure 5 (a) User ID 1, 2, 3, 5, 6, 7 and 9 have low number of pages clicks and satisfy the forth operational definition. The average interaction (query and clicks) are 12 clicks per session. So the users perform below this interaction is low. User ID 1, 2, 3, 9 and 10 are low interaction users shown in Figure 3 (b). The users along with their operational definitions and selected users with weak evaluation threshold can be seen in table.13.
The above results explain the validation of EISE model and show the existence of behaviors in user information seeking. Although the model is validated to an extent but still some concern can be raised about the operational definition. Up to now reduced numbers of operational definitions are used from the model because of available data limitation. Further study will extend the operational definitions to fully make use the psychology theories in the model. For example in search strategy an operational definition high standards/relevance occurs but from the current data we cannot judge this operational definition. In evaluation threshold operational definition high source credibility and high resemblance cannot be extracted from the data. To solve this problem new experiment in a control environment will take place to make use of all the operational definition and build a comprehensive model.
According to the operational definition users fall in the same categories but to fully absorb the characteristics of the model the user needs to qualify maximum operational definitions. For example in Search Strategy, Cautious User ID 1, 4, 6 and 10 and Risky User ID 5 and 9 can be seen but User ID 2, 3, 7 and 8 are missing they do not qualify maximum operational definition of the characteristics.. The same result can see in evaluation threshold that User ID 1, 2, 7 and 8 missing they do not qualifies maximum operational definitions. Now a detail investigation is needed to create a new characteristic for these users if they do not fall in the above categories
The results show the existence of characteristics independently from each other. Further study will also investigate the relation between these characteristics with each other’s and the effect of information goals on search strategy and evaluation threshold and vice versa.
In this paper we present a user classification model for personalized information retrieval. In the previous studies models lack a theoretical background to personalize the search. In our study we build a model based on strong psychological theories to address human behavioral aspect and interpret those theories into information retrieval terminology. The proposed model is EISE (extended Information goal, Search strategy and Evaluation threshold) adapted from a model applied in CBIR (content based image retrieval). Enriching the model with the theories and applying on search data log we established the existing of those behaviors which we hypnotized in our model. On the basis of these behaviors we can build a personalized search to enhance user information seeking ability.
[1] Azure. DISC based personality assessment.
http://www.amazureconsulting.com/files/1/74900324 /DISCPastAndPresentAndWilliamMarston.pdf, 2011.
Accessed: 23 Dec. 2015. [2] P. N. Bennett, R. W. White, W. Chu, S. T. Dumais, P.
Bailey, F. Borisyuk, and X. Cui. M odeling the impact of short- and long-term behavior on search personalization. In Proceedings of the 35th International Conference on Research and Development in Information Retrieval (SIGIR), pages 185–194, 2012.