In this work, we are building a cross-system recommender at the client side that uses the Wikipedia's content to derive an ontology for content and user modeling. We speculate the collaborative content of Wikipedia may cover many of the topical areas that people are generally interested in and the vocabulary may be closer to the general public users and updated sooner. Using the Wikipedia derived ontology as a shared platform to model web pages also addresses the issue of cross system recommendations, which generally requires a unified protocol or a mediator. Preliminary tests of our system may indicate that our derived ontology is a fair content model that maps an unknown webpage to its related topical categories. Once page topics can be identified, user models are formulated through analyzing usage pages. Eventually, we will formally evaluate the topicality-based user model
User modeling through content is one common solution in
recommending web pages across systems [
Conference’04, Month 1–2, 2004, City, State, Country.
Copyright 2004 ACM 1-58113-000-0/00/0004…$5.00.
Luz M. Quiroga Department of Information & Computer Science 1680 East-West Road Honolulu, HI 96822, USA
1-808-9569988
knowledge bases. Although there are only a few contributors(less
than 10% of user population) to the content of Wikipedia[
Our recommender formulates a user model based on the browsing behavior at a client side and the usage pages mapped to the derived ontology. Given the research potentials of Wikipedia’s content, we are interested in the performance of recommending web pages based on the Wikipedia derived ontology. Our research question is "Does the recommender based on the Wikipedia’s content model provide topically relevant recommendations?"
Content-based recommenders include WebWatcher[
Our recommender uses the Wikipedia’s content to derive an ontology for content and user modeling. With the ontology, our system automatically assigns the Wikipedia category(s) to a new page that pass a category’s threshold value, which formulates a “categorical” vector space model for the page. The system also captures user interests in the user model through the categories. Pages of similar topics with the profile will be recommended. There are four major components in the system -- the crawler, the Wikipedia knowledge base (WikiBase), the sensor, and the matcher. To begin with the top part of the graph, the crawler fetches those hyperlinked pages from the usage pages as well as queries search engines based on the user model managed by the sensor. Utilizing the sensor, the crawler generates a corresponding content model for each newly fetched page.
Every component in the system uses WikiBase, which stores ontologies, keywords, content models and the user model respectively. We construct WikiBase by combining the Wikipedia’s categorization system with heuristic information extraction on keywords. Heuristics include page titles, categorical labels, anchor texts, italic, bold, and TF-IDF terms. In order to associate keywords with categories, we extract heuristic keywords form pages labeled as one of the categories by the Wikipedia’s editors. Therefore, each category has a list of keywords to be utilized by the sensor.
The sensor manages the user model and maps usage pages into content models. It calculates a page's topical relevance and formulates the corresponding content model according to the WikiBase’s keyword weight and the word frequency of the web page. The user model is updated, on a frequency basis, by the sensor whenever it maps a usage page into a content model. Therefore, the user model is constantly evolved. In other words, if a user accesses a specific categorical topic in multiple times or through multiple pages, the user will score higher in the corresponding category of the user model. Keyword weighting and sensing formulas are defined below.
|Kj|, the number of keywords extracted for heuristic type j |K c|, the number of keywords in category c |Categories|, the number of categories in the knowledge base freq(Kij) is the frequency of keyword Kij for heuristic type j max(K1, … ,Km) is the maximum value among the m elements W (Kij) = ∑ aj (
freq(Kij) ) max (freq(K1j),…, freq(K|Kj| j )) 1 ≤ i ≤ |Kj|, 1 ≤ j ≤ m, aj is a weighting coefficient assigned to heuristic j.
Rc = ∑ d W(Ki c) α 1 ≤ i ≤ |K c|, 0.5 < α < 1, 1 ≤ c ≤ |Categories|
freq(Kic) d = 1, 0 < d < 0.5, partial match
full match As for the matcher, it compares the cosine similarity of those crawler-retrieved pages with the user model and then generates recommendations. In addition to cosine similarity, the matcher also relies on the ontological structure of WikiBase. With the structure, topical association among web pages can be revealed and it also helps to identify if a user is interested in particular domains or not. We define two indices (diversity and specificity) to represent the coverage of user interests. The following describes the procedure.
At the beginning, construct a minimal spanning tree that traverses all the identified categories in the user model. Identified categories are those categories with a Relevance Score over a predefined-threshold. In order to connect identified categories together, connecting nodes, such as parents or neighbors of the identified categories may be added to the tree. Definitions of the two indices are as follows: Diversity index: count the number of edges of the minimal spanning tree and normalize it by dividing the number of identified nodes, excluding connecting nodes, in the spanning tree.
Specificity index: sum the minimal distances from the root to all identified categories respectively and normalize it by dividing the number of those identified categories, excluding connecting nodes.
We plan to recruit a few participants (< 10) in the computer
science domain where we derive WikiBase. Each of them has to
rate a collection (> 300) of web pages based on topical relevance
and novelty. They have to provide certain web pages (>30) of
their interest in advance as the usage source of formulating the
user models. Afterward, they have to rate the collection. The
ratings will be divided into a training and validation set. Our
system will tune the keyword weight based on the training data.
We will compare our system performance with the SMART
system[
We have built WikiBase in the computer science domain, listed at http://www2.hawaii.edu/~pcchang/ICS699/results.html. We selected the domain due to its rich data. Two preliminary tests were conducted on two computer science professionals. In the first one, w tested the following pages for their topical relevance. http://www.algosort.com/ (A) http://tc.eserver.org (B) Considering only the top two ranking, page A is sensed as “algorithms” and “genetic algorithms” categories; page B is sensed as “human-computer interaction” and “usability” categories. In the evaluation of the classification result, both participants’ rankings are the same as the system’s ranking, considering only the top two.
In the second test, we selected fourteen pages, listed in the appendix, from four topical areas – algorithm, data mining, human computer interaction (HCI) and computer games. Both participants have to evaluate at least five categorical keywords of each page. They have to provide the degree of agreement from 1 (disagree) to 5(agree) about the following statement. "The given phrase is a topical keyword of the page." The given phrase is a categorical label generated by the sensor for each page. The following table summarizes the ratings.
Participant 1
Participant 2
Average
3.88
3.95
4.22
2.67
3.67
2.83
3.40
4.27
2.2
3.2
From the result, the ranking of both participants’ average scores
is: HCI, Data Mining, Algorithm, and Games. Except for the
game topic, the agreement score is around 4 for participant 1 and
3.5 for participant 2. We suspect that due the wide coverage of
computer games, our system performs worse in that category.
Another reason may be because of the nature of computer science
category. It reflects the common scientific techniques of theory
for producing computer games, which is different from the tested
pages that viewing computer games from a player’s perspective.
We are still in the process of tuning up the keyword weight by
utilizing the computer science pages from Open Directory Project
(DOP) [
Using the Wikipedia categories as an ontological model yields a simple user profile. This modeling approach benefits significantly in cross-system recommendations. Our recommendation engine works at the client side, which eases the privacy concern of disclosing sensitive information at web servers. Combining categories with heuristic information extractions leaves rooms for the selection of heuristics. Different domains or user groups are able to apply heuristics of interest. Given the above mentioned advantages, we are looking forward to see the results of our evaluation.
The Wikipedia content and categorization system play an important role in our method to generate recommendations. Our work emphasizes the framework to automate the ontology generation and its performance in recommendations. Nevertheless, the quality of Wikipedia content is controversial. It will be worthwhile to adopt the same framework to another Wikipedia-like platform with a different user group, such as domain experts, to ensure the content quality.
Another interesting area is to study the content statistics, such as volume or the granularity of the categories, with recommendation performance. Not every domain in Wikipedia contains rich categories and articles like computer science. Therefore, the performance of recommendations may be related to some of the statistics.
Due to limited space, only 1st page of each selected topic displays the categorical keywords.
http://www.algosort.com/ (Algorithms, Genetic algorithms, Root-finding algorithms, Networking algorithms, Disk scheduling algorithms) http://www.oopweb.com/Algorithms/Files/Algorithms.html http://cgm.cs.mcgill.ca/~godfried/teaching/algorithms-web.html Data Mining http://www.data-mining-guide.net/ (Databases, Algorithms, Knowledge representation, Natural language processing, Knowledge discovery in databases, Machine learning, Data mining) http://www.thearling.com/ http://databases.about.com/od/datamining/
Data_Mining_and_Data_Warehousing.htm http://www.ccsu.edu/datamining/resources.html HCI http://www.pcd-innovations.com/ (Human-computer interaction, Human-computer interaction researchers, Usability, Computer science organizations, Artificial intelligence, Software development)
http://www.robinlionheart.com/gamedev/genres.xhtml (Image processing, Computer programming, Demo effects Regression analysis, Computer graphics) http://open-site.org/Games/Video_Games/ http://www.literature-study-online.com/essays/alice_video.html