Web archives are a huge source of information to mine the past. However, tools to explore web archives are still in their infancy, in part due to the reduced knowledge that we have of their users. We contribute to this knowledge by presenting the first search behavior characterization of web archive users. We obtained detailed statistics about the users' sessions, queries, terms and clicks from the analysis of their search logs. The results show that users did not spend much time and effort searching the past. They prefer short sessions, composed of short queries and few clicks. Full-text search is preferred to URL search, but both are frequently used. There is a strong evidence that users prefer the oldest documents over the newest, but mostly search without any temporal restriction. We discuss all these findings and their implications on the design of future web archives.
The web has a democratic nature, where everyone can
publish all kinds of information. News, blogs, wikis,
encyclopedias, interviews and public opinions are just a few
examples. Part of this information is unique and
historically valuable. However, since the web is too dynamic, a
large amount of information is lost everyday. Ntoulas et al.
discovered that 80% of the web pages are not available after
one year [
Several initiatives of national libraries, national archives and consortia of organizations started to archive parts of the web to cope with this problem1. Some country code toplevel domains and thematic collections are being archived regularly2. Other collections related to important events, such as September 11th, are created at particular points in time3. In total, billions of web documents are already archived and their number is increasing as time passes. The Internet Archive alone collected 150 billion documents since 1996. The historic interest in the documents is also growing as they age, becoming an unique source of past information for widely diverse areas, such as sociology, history, anthropology, politics or journalism. However, to make historical analysis possible, web archives must turn from mere document repositories into living archives. The development of innovative solutions to search and explore it are required.
Current web archives are built on top of web search
engine technology. This seems like the logical solution, since
the web is the main focus of both systems. However, web
archives enable searching over multiple web snapshots of the
past, while web search engines only enable searching over
one snapshot of the close present. Users from both systems
also have different information needs [
Our results show that users of both types of systems have
similar behaviors. In general, web search technology can be
adopted to work on web archives. Nonetheless, our
identification of the users’ specificities provides insights on search
behavior, that might contribute to better support the
architectural design decisions of future web archives. Examples
include optimizing their performance [
This paper is organized as follows. In Section 2, we cover the related work. In Section 3, we describe the search environment. The methodology of analysis is explained in Section 4 and the results are detailed in Section 5. Section 6 finalizes with the discussion of results and conclusions. Copyright 2011 for the individual papers by the papers’ authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors.
TWAW 2011, March 28, 2011, Hyderabad, India. 1see http://www.nla.gov.au/padi/topics/92.html 2see http://www.archive.org/ 3see http://www.loc.gov/minerva/
There are several web archiving initiatives currently
harvesting and preserving the web heritage, but very few studies
about web archive users. The International Internet
Preservation Consortium (IIPC) reported a number of possible
user scenarios over a web archive [
The National Library of the Netherlands conducted an
usability test on the searching functionalities of its web archive
[
In a previous publication, we studied the information needs
of web archive users [
Web usage mining focuses on using data mining to
analyze search logs or other activity logs to discover interesting
patterns. Srivastava et al. pointed five applications for web
usage mining: personalization, for adjusting the results
according to the user’s profile; system improvement, for a fast
and efficient use of resources; site modification, for providing
feedback on how the site is being used; business intelligence,
for knowledge discovery aimed to increase customer sales;
and usage characterization to predict users’ behavior [
Search logs capture a large and varied amount of
interactions between users and search engines. This large number
of interactions is less susceptible to bias and enables
identifying stronger relationships among data. Additionally, search
logs can be analyzed at low cost and in a non intrusive way.
Most users are not aware that their interactions are being
logged. Users also try to fulfill their real information needs,
instead of having tasks assigned by a researcher that can
bias their behaviors. On the other hand, search logs are
limited to what can be registered. They ignore the contextual
information about users, such as their demographic
characteristics, the motivations that lead them to start searching,
and their degree of satisfaction with the system. Qualitative
studies, such as surveys and laboratory studies, can
complement log analysis with information that can explain some of
the patterns found [
Several logs from web search engines were analyzed with
the goal of understanding how these systems were used. A
common observation across these studies is that most users
conduct short sessions with only one or two queries,
composed by one or two terms each [
The PWA preserves the Portuguese web, which is considered the subset having the most interesting contents for the Portuguese community. Specifically, we define the Portuguese web as all the documents4 satisfying one the following rules: (1) hosted on a site under a .PT domain; (2) hosted on a site under another domain, but embedded in a document under the .PT domain; (3) suggested by the users and manually validated by the PWA team. Additionally, the PWA team integrated web collections from several other sources, such as the Internet Archive and the Portuguese National Library. The number of indexed documents have been growing and there are now more than 180 million accessible by full-text and URL search. As far as we know, this is the largest web archive collection searchable by full-text and over such a large time span (from 1996 to 2009). The experimental version of the PWA has been available as a service to the general public since 2010 at http://archive.pt/.
The interaction with the users and the layout of the results is similar to web search engines, such as Google. In a typical session, a user can submit a full-text query and receive a search engine results page (SERP) containing a list of 10 results matching the query. Figure 1 illustrates this case. Each result includes the title of the web page and its crawled date, a snippet of text containing the query terms and the URL. The user can then click on the results to see and navigate in the web pages as they were in the past. If the desired information is not found, the user can repeatedly modify and resubmit the query. In addition, the user can click on the navigation links to explore other SERPs or use the advanced search interface to restrict the query with advanced search operators. These operators can also be added to the query directly in the text box. 4The terms document and file are used interchangeably in this study. For instance, it can be a web page, an image or a PDF file.
This interface has some specificities. First, the text box is complemented with a date range filter to narrow the results to a time period. Second, each result has an associated link to see all versions throughout time of the respective URL. When clicked, the PWA presents the same search engine versions page (SEVP) as when a user submits that URL on the text box. A table is shown to the user, where each column contains all the versions of a year sorted by date. The user can then click on any version to see it as it was on that date. Figure 2 depicts this interface. 3.1
Our analysis is based on the logs of the PWA, covering seven months of search interactions, from June to December, 2010. By interactions, we mean all queries and clicks submitted by the users and recorded by the PWA search engine (server side). The seven month span has the advantage of being less likely to be affected by ephemeral trends.
The logs follow the Apache Common Log Format5. Each entry corresponds to an interaction with the search engine in the form of a HTTP request. It contains the user’s IP address and the user’s session identifier. Each entry contains also a timestamp indicating when the interaction occurred and the HTTP request line that came from the client.
We never used the log data to match a real identity. However, we geographically mapped the IP addresses for a better characterization of the users. We counted 72% of PWA’s users with IP addresses assigned to Portugal. Near 89% 5see http://httpd.apache.org/docs/2.0/logs.html of the interactions were submitted through the Portuguese language interface. The remaining was submitted through the English language interface. This strongly indicates that users were mostly Portuguese. 4.
The analysis focused on four dimensions: sessions, queries, terms and clicks. We define them in the following way: • A session is a set of interactions by the same user when attempting to satisfy one information need. The session is the level of analysis in determining the success or failure of a search. It is composed by one or more queries and zero or more clicks. • A query is a search request composed by a set of terms.
We define an initial query as the first query submitted in a session, while all the following queries are defined as subsequent. An identical query is a query with exactly the same terms as the previous one submitted in the same session. A unique query corresponds to one query regardless of the number of times it was logged. The set of unique queries is the set of query variations. An advanced query is a query with at least one advanced search operator. • A term is a series of characters bounded by white spaces, such as words, numbers, abbreviations, URLs, symbols or combinations between them. There are also advanced search operators, but they do not count as terms. We define a unique term as one term on the dataset regardless of the number of times it was logged.
The set of unique terms is the submitted lexicon. • A click in this context refers to the following of a hyperlink to immediately view a query result (i.e. archived web page). It can be a SERP click or a SEVP click, depending if the user clicks in a SERP or SEVP.
Next, we briefly present the methods used on the search log analysis. 4.1
We prepared the log fields for analysis through a series of
data cleansing steps. All incomplete entries, empty queries
and sessions without any query were discarded. Internal
queries submitted by the PWA monitoring system, the queries
by example displayed on the PWA entry page and sessions
conducted by clients identified as web crawlers were also
excluded. Additionally, sessions with more than 100 queries
were likely to come from crawlers, so they were removed
too. This cutoff value of 100 was used in some other
studies, thus enabling a more direct comparison with our results
[
All terms were normalized to lowercase. Extra white spaces were removed. Since the PWA did not perform stemming, all variations of a query term were considered as different terms. The set of query terms also includes misspellings. 4.2
Most studies used the users’ IP address and/or session
identifier to delimit sessions [
Statistics were computed from the logged interactions. The first pattern that we detected was that users mostly conducted two types of sessions: with only full-text queries and with only URL queries, in 59.34% and 31.10% of the times, respectively. We defined these as full-text sessions and URL sessions. In the analysis, we ignored the remaining 9.57% sessions with mixed queries for simplification.
Table 1 shows the general statistics. The users of the PWA performed 6,177 full-text sessions, averaging 2.23 queries per session. The number of query terms per query was 2.84, with 6.42 characters per term. The users saw 1.44 SERPs per query and clicked 1.06 times on their hyperlinks to view a result. They hardly clicked to see all versions of a result. This only happened in 0.06 times per query. Overall, these results mean that for each query, the users saw mostly the first and sometimes the next SERP, where they clicked once.
Sessions Queries Terms
SERPs Clicks on SERPs
Clicks on SEVPs Queries per Session Terms per Query
SERPs per Query Clicks on SERP per Query Clicks on SEVP per Query
Characters per Term
Initial Queries Subsequent Queries - Modified - Identical - Terms Swapped - New Unique Queries
Unique Terms Queries never repeated Terms never repeated
The users also submitted 3,237 URL sessions, roughly half of the full-text seassions. On average, each session had 1.54 queries with 27.27 characters. Half of the URLs submitted, 50.24%, were not found in the PWA. For the URLs found, the users clicked on 1.56 versions to see them as they were on past. Basically, a user submitted a URL and saw one or two versions of that URL. Next, we will detail our analysis and explain the remaining results. 5.1
5.1.1
The duration of a session is measured from the time the
first query is submitted until the last time the user interacted
with the PWA. We ignore if the user spent more session time
viewing the archived web pages after the last interaction or
used part of the time doing parallel tasks [
The large majority of sessions ended quickly as shown in Table 2. Around 60% of the full-text sessions lasted less than 1 minute and 89% less than 10 minutes. Only around 3% of the sessions had a longer than an half hour duration. Each session took in average 4 minutes and 8 seconds. URL sessions took even less time than full-text sessions. In average, each session took 1 minute and 14 seconds. Around 81% of the sessions lasted less than 1 minute and only 6% took longer than 5 minutes.
5.2.1
Sometimes users submit sequences of queries as a way to refine or reformulate the search in a trial and error approach. We consider that two sequential queries submitted on the same session have the same information need if they share at least one term. In this case, we called the second query a modified query. We ignored the stopwords (too common terms) in this analysis. Thus, a modified query could be a specialization of the query (adding terms), a generalization (removing terms) or both at the same time.
We counted 44.53% of modified queries from all
subsequent full-text queries. Looking at Table 4, we see that
around 71% of the modified queries are the result of a zero
or one change on the number of terms. A zero length change
means that the users modified some terms, but their
number remained the same. Users tend to add more terms in the
modified queries rather than to remove them. We counted
around 42% versus 25%. PWA’s users tend to go from broad
to narrow queries, such as in web search engines [
NOT PHRASE
SITE TYPE total % advanced queries 3.62% 78.10% 12.81% 5.48% 100.00%
A variety of reasons can lead users to repeat queries, such as a refresh of the SERP or SEVP, a back-button click or the submission of the same query more than once due to a network or search engine delay. When analyzing the fulltext queries, we counted 20.35% of identical queries, where each query has exactly the same terms as the previous one made in the same session (see Table 1). We also counted the subsequent queries with the same terms, but written in a different order. For instance, a query Web Archive followed by a query Archive Web. Only a small number of subsequent queries, 3.75%, had the order of the terms swapped. Besides the modified and identical queries, the users also submitted 31.37% of subsequent queries with only new terms. This indicates that at most this percentage of subsequent queries were the result of a new information need.
We divided the subsequent URL queries in identical and new. 78.56% of the subsequent queries were new. The remaining 21.44% were the result of the same URL submission (see Table 1).
In the PWA, users could use four advanced search operators: NOT, to exclude all results with a term in their text (e.g. -web); PHRASE, to match all results with a phrase in their text (e.g. “web archive”); SITE, to match all results from a domain name (e.g. site:wikipedia.org ); TYPE, to match all results from a media type (e.g. type:PDF ).
Table 5 presents the percentages of advanced queries (i.e.
with at least one advanced search operator). It shows that
25.86% of the queries included operators. This is a
significantly higher percentage when compared with studies over
web search engines [
The distribution of the terms per full-text query listed in
Table 6 shows that the majority of the queries had 1 or 2
terms. This is also visible by the 2.84 average of terms per
query (see Table 1). Around 87% of the queries had up to
5 terms and only 3% had 10 or more terms. These results
indicate that the users tend to submit short queries. These
values are useful, for instance, to optimize index structures
[
The users saw on average about 1.44 SERPs per full-text
query. All users saw the first SERP as expected, since the
PWA always returned it after a query. Then, the users
followed the natural order of the SERPs, but in a sharp decline
(see Table 7). For instance, the second SERP was viewed in
14.44% of the queries. This indicates that prefetching the
second SERP would not significantly improve web archive
performance. On the other hand, the close percentages
of the following SERPs indicate that prefetching them can
bring improvements as shown in other studies [
About 66% of the clicks occurred on the first SERP from
almost a click per query. The users clicked on 1.06 times per
query to access an archived web page listed on the SERPs.
We observed that users clicked on the rank of results
following a power law distribution, with a 0.88 correlation (see
Figure 3). These results are similar to web search engine
studies, which also present a discontinuity in the last
ranking position of each SERP (multiple of 10) [
We ranked the full-text unique queries by their decreasing
frequency and verified that their distribution fits the power
law with a 0.96 correlation. This finding was also observed
in web search engines [
We also ranked the URL unique queries by their
decreasing frequency and verified that their distribution, once again,
fits the power law with a 0.96 correlation. By caching around
32% of the most frequent URL queries, the PWA could
respond to 50% of the queries. Although satisfactory, the
percentage of queries cached are much superior than in
previous studies [
As a consequence of the users’ queries and clicks following a power law distribution, the archived pages seen by the users also follow a power law distribution, with a 0.94 correlation. This applies to both full-text and URL sessions.
Analogous to the query frequency distribution, we ranked
the full-text unique terms by their decreasing frequency.
Their distribution fits the power law with a 0.97 correlation.
As depicted in Figure 5, the cumulative distribution shows
that it is necessary to cache just around 6% of the most
frequent terms to handle 50% of the queries. Much less RAM is
necessary to cache terms than queries for a similar hit rate.
These results are consistent with others presented for web
search engines [
The users restricted by the end date 23.55% of the fulltext queries, while only 1.64% by the start date. The start and end dates were both changed in 12.98% of the queries. The same pattern exists in URL queries as shown in Table 8, where the start date was changed almost only when the end date also was. This indicates that users are more interested in old documents. The idea is reinforced by the distribution of the years included in the full-text queries restricted by date. As it can be seen in Figure 6, the older the years, the more likely they are of being included in queries. However, the URL queries have an almost constant rate. 5.4.2
Documents tend to have just a few years with archived versions, thus segmenting the number of clicks per year would likely bias the results. Instead, we computed for all URL queries, the percentage of clicks in each year yi with at least one version. We measured as ctilmickess((yyii)) , where the denominator represents the number of times the year yi was displayed to the user, and the numerator the number of clicks in yi. For instance, the first year y1 is 1997 if there is no archived versions for that URL in 1996. Otherwise, y1 is 1996.
In Figure 7 it is visible that users clicked much more on the first year with archived versions than on the remaining years. The first year was clicked in 55% of the times, while all the others were clicked at most 20%. With exception of the eighth year, the first three years had the higher percentages. This shows a preference for the older documents. 5.4.3
We counted the number of queries with temporal expressions, since they represent a temporal dependent intent. We started by experimenting named-entity recognition tools for Portuguese. However, queries are not grammatical, so the 2 3 4 5 6 7 8 rank of years with archived versions 9 tools presented a small precision. Instead, we used a simple match of all the queries with years, months and day patterns. Then, we classified a random subset of 1,000 queries to validate our detection patterns. Surprisingly, they worked very well. The patterns achieved a precision, recall and accuracy, of 89%, 100% and 98%, respectively. The patterns created some false positives, but unexpectedly no false negatives. This was mostly, because there were no temporal expressions in the logs without date patterns (e.g. last decade).
All matches were manually validated, from which we
excluded the false positives. In the end, we counted 3.49% of
queries with temporal expressions. Almost all are related
with past events, such as world cup 2006. This is a small
percentage in line with the 1.5% of temporal expressions
found in the logs of the AOL web search engine [
On the other hand, the single term queries, the SERPs
viewed per query and the topic most seen, are in conformity
with web search engine results [
Overall, the search patterns of the users of both types of systems show no evidence precluding the adoption of web search engine technology for web archive search. This was a surprise to us, because users from both systems have different information needs. For instance, users said they wanted to see the evolution of a page throughout time, but they tend to click on one or two versions of each URL. All information needs of the users are focused on the past, but most of the user queries are not restricted by date, neither contain temporal expressions. Users search as in web search engines. This behavior may be the consequence of we having offered a similar interface, leading them to search in a similar way.
IR system world region
name single query session queries per session single term queries terms per query advanced queries SERPs viewed per query topic most seen
U.S.
Excite [
2.3 20% - 30%
2.6 11% - 20%
1.7 Commerce, Travel web search engine Hence, new types of interfaces must be experimented, such as the temporal distribution of documents matching a query or timelines, which could create a richer perception of time for the user and eventually trigger different search behaviors.
Nevertheless, the identification of the users’ specificities might contribute to the development of better adapted web archives. We observe a strong preference in searching and seeing the oldest documents over the newest. This finding can be used in ranking results, when no other temporal data is given. The ranking should also be tuned for navigational queries when the query type is unknown. Queries, terms, clicked ranks and seen archived pages follow a power law distribution. This means that all have a small fraction that is repeated many times and can be explored to increase the performance of web archives.
The PWA is still experimental and has a much smaller user base than commercial web search engines. Still, we believe that the obtained results are general, but studies over larger datasets and from other web archives are necessary to confirm this. Our future work will use these results to improve the architecture and retrieval algorithms of the PWA.
This work could not be done without the help and infrastructure of the PWA team. We thank Michel da Corte for her review of the paper and FCT (Portuguese research funding agency) for its Multiannual Funding Programme.