Web retrieval is now one of the most important issues in computer science, and we believe that applying multi-agent systems to this area is a promising approach. We introduce Kodama1 system, which is being developed and in use at Kyushu University, as a multi-agent-based approach to build a distributed Information Retrieval (IR) system that lets users retrieve relevant distributed information from the Web. We reported methods to agentify the Web, and to cluster the agentified domain into communities. In order to investigate the performance of our system, we carried out several experiments in multiple Server Agent domains and developed a smart query routing mechanism for routing the user's query. The results ensure that the idea of Web page agentification, clustering and routing techniques promise to achieve more relevant information.
With the exponentially growing amount of information available on the Internet, the task of
retrieving relevant information consistent with the user’s information need has become
increasingly difficult and the users normally face with very large hit lists with low precision.
The information gathering and retrieving processes in the traditional search engine are
independent of the User’s Preference (UP), and therefore feedback from the later process is
hardly adaptive to improve the quality of the former process. Kodama project starts in response
to the need for a new kind of agent-oriented IR system that is completely different from the
traditional search engines populated on the Internet. Researchers in Artificial Intelligence (AI)
and Information Retrieval (IR) fields have already succeeded in developing multi-agent based
techniques to automate the management of information flooding [
Cooperating intelligent Kodama agents are employed to agentify the Web servers where the
infrastructure is preexisting in the form of Web links. Kodama system uses three types of agents in
the agentification mechanism for searching the Web. A SA assigned to each Web server, a WPA
assigned to each Web page, and a User Interface Agent (UIA) assigned to each user’s machine
[
Q ← Q ∪CNqi , CNqi ← CNqi ∪ qi and CNqi ← CNqi ∪ k j for every k j ∈Tag . Where Tag is a set of keywords surrounded by specific tags in such Web pages that are in Cq j and relevant to qi .
Although a single router is scalable enough to potentially handles of thousands of SAs. In practice it is desirable to run a separate router for relevant SAs of a common topic. For instances, the SAs of AAAI, IEEE and ACM portals belong to one router agent. The router delegates the given query to the most popular and relevant SA. For each community of SAs, there is a router agent that holds a set of attributes as ontology of each SA (see Table 1). Where A11 to A1m means the set of attributes automatically determined by clustering the WPAs and adapted by the system to reflect the ontology of the SA1. W is the weight value, which assigned to each attribute and continually adapted based on the ℜ from the UIAs. Routing refers to the process of selecting the relevant SA and forwarding queries to it to retrieve the Web pages consistent with user's information need. Relevancy is used to determine the popularity of the SA for a particular type of queries. Kodama system maintains the similarity Sij between qi and the attribute fields of
SAj of the known
using the following formula. S ij = ∑i w j,i ⋅ g (k i ) . Where, g(ki ) =1 if ki ∈qi ∩SAj , otherwise g (ki ) = 0 .
SAt
There should be a single entity that controls the list of router agents. When you register a router agent, it goes through one of several dozen routers who work with Kodama to add names to the list. Kodama, in turn, keeps a central database known as the router's database that contains information about the profile of each router. Each of the routers has thousands of SAs and handles its requests. While registering the SA into a router agent, the SA sends the names of its clusters to be used for routing relevant queries into that SA. The routers are specialized agents that send your query and those of every other UIAs to their relevant SAs along thousands of pathways of SAs. When the router receives a query from the UIA, the router does the followings with it. It asks for a list of relevant SAs. If the router agent found relevant SAs, it assigns the request with specific SA because it already knows that this SA is relevant to this query. Then, it merges the results of the query and sends them back to the UIA. Otherwise, it forwards the queries to other routers if the results do not satisfy the user or the router could not find a relevant SA. It may return an error message because the router could not find any relevant SA.
We have performed several experiments to make a consistent evaluation of Kodama system performance. In the experiments, we agantified fifty Web servers by giving the portal addresses of the Web servers to the system; the number of Web pages within the agentified servers varies from 300 to 2500 pages. The system creates the hyper structure of the WPA communities based on the hyperlink structure of each Web server and creates the SA's attributes to be used in the router side for the routing. Then, the UIAs sent some queries to the system. We calculated the Precision of the retrieved URLs to user’s queries. The results (see Fig. 2) show that the idea of Web page agentification and the routing mechanisms promise to achieve more relevant information to the users and also promoted using Kodama as a PinPoint IR system. 1 0 0 567432819 000000000 0 1 2 3 4 5 6 7 8 9 1 0 1 1 1 2 1 3 1 4 1 5 1 6 1 7 1 8 1 9 2 0 2 1 2 2 2 3 2 4 2 5 2 6 2 7 2 8 2 9 3 0
Q u e r i e s
We introduced methods to agentify the Web, and to cluster the SAs and WPAs into communities. We also introduced the routing mechanism of Kodama system to select the most relevant SA to the given query. We carried out experiments to investigate the performance of Kodama system. Through these experiments, we ensure that Kodama's techniques promise to achieve more relevant information to the users. Currently, the routing of Kodama is a simple query routing that binds to two hierarchical levels of router agents. We plan to scale it by increasing the number of SAs and developing more sophisticated routing mechanism for maintaining multiple hierarchy of router agents.