The Semantic Web introduces the need for semantic search engines. In this paper, we explain our vision of a catalog search engine for semantically de¯ned products. With our prototype, we address the problem of products' information retrieval over the Internet and their semantic enrichment through the mixed usage of thesauruses and ontologies. We show how we automatically build a repository of instances of ontology classes, and how we dynamically prioritize the search variables of our engine. We then introduce our prototype which, through the use of all those concepts, improves the user experience.
The Semantic Web introduces the need for semantic search engines. Although
semantic search is already available in a variety of forms such as SHOE[
Products catalogs available on the Internet all have limitations of several types. They either provide a wide range of products but have a poor search engine in terms of precision, or o®er a limited range of products with a powerful but too specialized (in terms of search variables) search engine. Whichever the catalog, the user can easily get frustrated by their poor ability to supply him/her precise results and an extensive selection of products at the same time.
One of the challenges of the semantic web is to transform the already available information into more meaningful, more usable data. A lot of the available literature tackles this problem and agrees on a fundamental problem: most of the di±culties come from the ambiguity of the human language, and from the fact that nearly all this information has been created by humans for human consumption. Products information, on the other hand, has the advantage of being, in most cases, based on an agreed vocabulary. However, the problem with products sold on the Internet is that the quality of their descriptions (in terms of availability), as well as the presentation of those descriptions (in structured tables, simple paragraphs, etc) varies greatly from a web site to another. Nonetheless, considering that the Internet is the biggest products database available, it becomes obvious that it should be the source of any search engine aspiring to be as complete and accurate as possible.
Our vision of a catalog search engine includes three distinct goals. The catalog must contain as much products as possible, its search engine must be the most accurate, and the user must be given enough tools to let him/her search e±ciently through the catalog. In this paper, we show that we answered to those three needs with the following strategies. The wideness of the catalog is insured by the gathering of existing products' information all over the Internet through the usage of dedicated parsers. The accuracy of the search engine is reached by a combination of semantic enrichment of the previously fetched information, and the automatic conversion into logic facts of every characteristic of the products. Eventually, the user's search e±ciency is enhanced by the usage of algorithms dynamically computing the usefulness of each products' characteristic. Moreover, the usage of a thesaurus during the query phase allows the user not to worry about the exactitude of the content of his/her query.
In the following sections, we ¯rst introduce the architecture of our system. Then, we focus on its usage and explain the details of its features. We continue with a short demonstration of our prototype and follow with a discussion about decisions we took during the design phase. Eventually, we take a brief look at the existing work tackling the problem of catalog search engines and conclude. 2 2.1
Overall Architecture The architecture of the catalog search engine is shown in Fig. 1. The system we suggest is a complete solution, from the server fetching data from the Internet, to the client that will let the end-user carry out his/her requests. The server is made of 4 main components; the web page fetcher, the parser, the facts creator and the pro¯les creator. The client is separated into 2 main components; a GUI with which the end-user will communicate, and a proxy which will be in charge of the client-Matchmaker communication. In between the server and the client lies the Matchmaker server which provides the search capabilities.
The main idea of this system consists in fetching information about products sold on the Internet, and publishing this information on the Matchmaker server. This information will be enriched with semantics using ontologies. On the other side, the end-user will be able to express requests to the Matchmaker. The latter will browse through all the available advertisements, try to ¯nd those which are the most closely related to the request and eventually return them to the user. We will describe the usage in the next section.
At the heart of our prototype lies the Semantic Service Matchmaker, a service
search engine based on the LARKS[
Ideally, when the requester looks for a product, the Matchmaker will retrieve a product that matches exactly the expected one. In practice, if the exact product is not available, the Matchmaker will retrieve one which capabilities are similar to those expected by the requester. Ultimately, the matching process is the result of the interaction of the products available, and the requirements of the requester. Server manager
Server client Web browser
CPU memory hdd … Results
Parameters End-user
User client Avail. items/charact.
query Web browser s u r u a s e h T / y g o lt o n O
ANN
AN
ANN
AN Record (profiles)
Matchmaking
engine
Matchmaker server Search (profile) Matchmaker client Ontology / Thesaurus
Client server tables lists
Although the Matchmaker originally provides a set of 5 ¯lters, our prototype
uses only two of them. The type ¯lter applies a set of subtype inferencing rules
mainly based on structural algorithm to determine whether an ontology class is
a subsumption of another. The constraint ¯lter has the responsibility to verify
whether the subsumption relationship for each of the constraints are logically
valid. The Matchmaker computes the logical implication among constraints by
using polynomial subsumption checking for Horn clauses. More details about the
Matchmaker's ¯lters are provided in [
Usage scenario
Server side
1. The server is initialized with a ¯le which contains information concerning the
web pages to be fetched and parsed. This ¯le connects each type of products with
a list of web pages (e.g http://somewhere/hddidetosell.html ) products
type \HDD IDE").
2. Next, web pages are fetched from selected web sites. Once done, a parser
detects relevant information from those web pages. If a new product is detected,
the server automatically creates a new instance of the ontology class which
describes the type of the product. If a new characteristic is detected, the server
updates the list of properties associated with each class of product.
3. Then, the server automatically creates a ¯le containing a list of facts written
in RDF-RuleML[
Client side 1. First the user inputs a query. This query is parsed and its content is compared with the thesaurus' words, as well as with the name of the instances of the products' types (created at step 2 of the server part). 2. The answer to the query is either a list of types of products having the best matching terms to the query, or a list of instances, or both. If the answer is a list of instances, the user can click on one of them in order to display the details of the chosen product. If the answer is a list of types of products, the user can click on one of them in order to display a list of characteristics of the chosen type. If the user wants to carry out a ¯ne search, he/she must input some values for the characteristics with which he/she wants the result to be relevant. When the user eventually clicks on the \¯ner search" button, a RDF-RuleML ¯le containing those characteristics translated into facts is automatically created. 3. The system automatically creates a \request" pro¯le. This pro¯le is then submitted to the Matchmaker, which tries to match this \request" to the \advertisements" contained in its database. If one or more matching pro¯les are found, they are sent back to the user, who sees them as individual products. He/she can then click on one of the available link to di®erent shops selling the product. 2.3
Usage of thesauruses and ontologies
Thesauruses are needed for any search engine of which search mechanism is
not exclusively based on the query's keywords. For the sake of simplicity, we
built our own thesaurus for our prototype. While the goal is, of course, to use a
rather complete thesaurus such as WordNet[
Our ontologies are written in OWL[
Catalog Search Engine C l ( a e s x s : • n a m e • c U l) R a L s s • S e t o f • S e t o f
H D D ) ( i n s t a n c e s p r o p e r t i e s P r ( o e p x e : r ( t ec yxa :c h c e a ) p a c i t y ) • n a m e • n a m e • p r o …p •e rp tr yo p ( e U r Rt Ly ) ( U R L ) • d a t a • td y a p t ea t y p e • o b j e c• to b / j e v c a t l u / e v a l u e
In our system, we have a general ontology composed of classes corresponding to various types of products (e.g. \hdd ide", \scanner", etc). We assume that this ontology has been created prior to the launch of the system. When the server of the system browses and parses the information found on various web sites, it automatically ¯lls in each instances slot for each class. For example, all the products found as being of type \hdd ide" will be automatically added to the instances slot of the product type (the class) \hdd ide". The instances growth is non-monotonic. It makes sense in that products may disappear if the catalog of the sellers is updated. In our prototype, we track each product's availability, so that if all the web sites of which we get information do not o®er a product anymore, its instance is automatically deleted.
A problem we encountered was related to the names of the products. A product has, in general, the same name wherever it is sold. Even so, some web sites on the Internet mix the name and the characteristics of the products in the same string. The instances we record being based on the name, we had to insure that, even though the name may be di®erent for a same product, new instances might not be created. To solve this problem, we ¯rst check the manufacturers of the products. If they match, we use a parser which is able to correct suspicious products' names through string comparisons. Also, as some products may have di®erent names in di®erent countries, we use a thesaurus of products'names to make sure that the products are the same. 2.4
Characteristics of the products A product is distinguished by its characteristics. A hard disk has a certain seek time, capacity, interface, etc. Translating products' characteristics into facts is quite natural, as each characteristic can be thought as a truth about the product it describes. The facts our system creates are all of the form P (x; y) where P is a predicate, and x and y two terms. The facts written by the server use two kinds of predicates, \equal" when the term y is a numerical value, and \is" when the term y is a string of characters. The facts written by the client use the predicates \is less than or equal to" and \is more than or equal to" when y is a numerical value, \is" when y is a string of characters. Each property of the products' classes of our main ontology are converted into OWL classes and used as the term x of the facts (for a discussion about this conversion, refer to section 4). The facts concerning the manufacturer, the seller and the price of a product are considered the minimum information required for a product to be taken into account.
In the table 1, we show an example of a product's characteristics converted into facts on the server and the client side. On the server side, the characteristics of a hard disk fetched from the Internet are converted into facts. On the client side, characteristics which values have been input by the user are also converted into facts. \MANUFACTURER", \COST", \SELLER" and \CAPACITY" were OWL properties converted into classes. The predicates are classes of an ontology describing predicates. We use our own thesaurus to make the connections between the words used in the description of a characteristic (e.g. \sold at") and the corresponding ontology class (e.g. \COST"). See the code at section 2.5 for an example of a fact.
Once the client has transmitted the request pro¯le (which contains links to the facts created by the client) to the Matchmaker, the latter will use its inference engine (called the constraints ¯lter) to match the facts of the advertisements to the facts of the request. 2.5
Pro¯les A pro¯le is an OWL ¯le containing a semantic description of a product, as well as a list of links to each fact present in the facts ¯les related to this same product. For each shop selling the product, a pro¯le is created (i.e. a product being sold by 10 shops will have 10 di®erent pro¯les). The information stored in those pro¯les are the ontology class of the product's type, the name of the product (only if the pro¯le is an advertisement), the list of facts and the URL to the shop selling the product. Once advertisements pro¯les have been registered to the Matchmaker, if a request pro¯le is submitted the Matchmaker applies a matching using its type ¯lter on the ontology class of the product's type and its constraint ¯lter on all the facts. The following code shows an example of a pro¯le and a fact. <product:description rdf:id="Kakaku_CPU_Athlon_64_2800_Socket754_5"> <product:name>ATHLON 64 2800 Socket754_5</productName> <product:restrictedTo rdf:resource="http://somewhere/onto.owl#cpu" /> <product:constraint rdf:resource="http://somewhere/facts.rdf#clockspeed" /> <product:constraint rdf:resource="http://somewhere/facts.rdf#cost" /> <product:constraint rdf:resource="http://somewhere/facts.rdf#manufacturer" /> ...
<product:shopURL> http://www.aShopURL.com/</product:shopURL> </product:description> <ruleml:Fact ruleml:label="cost"> <ruleml:head> <ruleml:Atom ruleml:rel="http://somewhere/predicates.owl#numericallyEqual"> <ruleml:args> <rdf:Seq> <rdf:li>
<ruleml:Var ruleml:name="http://somewhere/store.owl#COST" /> </rdf:li> <rdf:li>
<ruleml:Ind ruleml:name="20990" /> </rdf:li> </rdf:Seq> </ruleml:args> </ruleml:Atom> </ruleml:head> </ruleml:Fact> 2.6
Dynamic update and prioritization of the characteristics When the user searches for a given type of product, its related characteristics are displayed. If the user wants to carry out a ¯ne search, he/she can insert some values to the characteristics he/she wants to be respected. The list of available characteristics is updated on the server side, when fetching and parsing information from various web sites. However, the priority in which those characteristics are shown to the user is dependant of each characteristic's associated weight. The weights are updated as follows. - the more a characteristic is available for a given type of product, the greater its weight will be, - the more a characteristic has possible values, the greater its weight will be (e.g. the size of a screen can be 15", 17",19", 21", etc), - the more a characteristic is chosen by the user, the greater its weight will be.
Other conditions come also into play to determine the position of a characteristic. As products' types are ontologically de¯ned, we rely on the parent-child relationships to tell whether a product's characteristic should be shown before another. For instance, as \computer" is the parent class of \notebook computer", if the user searches for \notebook computers" its characteristics should be displayed prior to those of \computer". 3
We introduce here the prototype of the client application. Data from the Internet
has already been fetched from Kakaku[
In Fig. 3(a), the user entered \hard disk" as a query. With the thesaurus, the client found out that \hard disk" is the equivalent to \hdd". Moreover, using the ontology, the client proposes not only \hdd" but also \hdd ide" and \hdd scsi", two subclasses of \hdd". In Fig. 3(b), the user selected \hdd ide". The client now proposes a list of characteristics of the \hdd ide". The three ¯rst are always present for each product. The next ones are the characteristics available for \hdd ide", as well as \hdd", as well as any other parent class of the \hdd", back up to the root of the ontology. The user decided to input values for two characteristics, the cost and the capacity. Once done, he/she gets the result shown in Fig. 3(c). The values corresponding to the chosen characteristics in the previous step are shown in bold. A link is provided to shops selling the products. 4
Our intention was not to produce a very e±cient catalog search engine in terms of speed, but rather in terms of relevance of the results. In this regard, we reached the three goals cited in the introduction of this paper. As our system gathers data from various web sites, we get more details about the products than if we simply relied on speci¯c vendors and thus insure the wideness of the catalog. Accuracy of the search engine is provided by the combined usage of thesauruses and ontologies, allowing the system to return very precise results even if the query of the user is relatively vague. Eventually, search e±ciency is attained by handling the feedback of the user regarding the characteristics of the products. As a consequence of all this, we observed that a user needs about half the number of clicks than usually needed when accessing the same information about products on other web sites such as Kakaku.
However, as our system is still at the stage of a prototype, it is not without °aws. In fact, the parser approach to the problem of information fetching on various web sites can prove to be quite weak in the long term. It requires much more advanced techniques to be able to fetch facts or rules from web pages such as Amazon or Yahoo Shopping as those web sites do not always display information about products in a very formal way.
The reader may wonder why we chose to create facts using RDF-RuleML to describe the characteristics of each products, instead of directly using the properties of each products' classes of our ontology. The reason is that we intend to create a much more powerful search engine, which does more than giving the possibility to enter a value for each characteristic. The goal is to use a better Natural Language Processing tool during the parsing phase on the server side, so that the system becomes able to create rules such as \if the credit card is Visa or American Express, the customer can have a 5% discount ". This kind of rule can not be expressed with OWL's classes and properties.
Alternatively, we thought about expressing the facts and rules in SWRL[
The IWebS project[
Active Catalog[
Based on the Matchmaker, we developed a prototype of a catalog search engine which enables users to have more accurate results in regard to their queries. Search parameters are dynamically updated through the analysis of fetched information and the feedback from the users. We showed that the approach of fetching available products data from the Internet, adding semantic to it through the use of ontologies, and e±ciently searching through it is feasible. Using rules, our system will be able to give more expressive power to users' queries.