=Paper=
{{Paper
|id=Vol-325/paper-2
|storemode=property
|title=Design of Web Agents Inspired by Brain Research
|pdfUrl=https://ceur-ws.org/Vol-325/paper02.pdf
|volume=Vol-325
|dblpUrl=https://dblp.org/rec/conf/adbis/DimitrovaWY07
}}
==Design of Web Agents Inspired by Brain Research==
https://ceur-ws.org/Vol-325/paper02.pdf
Design of Web Agents Inspired by Brain Research
Maya Dimitrova1, Hiroaki Wagatsuma2, and Yoko Yamaguchi2
1
Institute of Control and System Research, Bulgarian Academy of Sciences,
Acad. G. Bonchev Str., Block 2, P.O.Box 79, 1113 Sofia, Bulgaria
dimitrova@icsr.bas.bg
2
Laboratory for Dynamics of Emergent Intelligence, RIKEN BSI,
Hirosawa, Wako-shi, Saitama 351-0198, Japan
{waga, yokoy}@brain.riken.jp
Abstract. The paper presents an approach to combine knowledge from memory
and brain sciences with information retrieval research in the design of Web
agents. An information retrieval agent for classification of Web pages based on
genre features is used. In developing the agent to adapt to users’ search
preferences, a neuro-cognitive model of human episodic memory is employed.
Our studies show that neuro-realistic models, capable of abstraction of
meaningful fragments of knowledge, rather than snapshots of the retrieved Web
pages, are closer to the human way of interacting with the Web and can be used
for optimization of agent performance.
Keywords: information retrieval, Web genre, theta phase precession theory,
episodic memory.
1 Introduction
Usable Web agents have to be adaptive to the demands of the users as well as to the
constant dynamic change of Web knowledge. In a way, Web agents have to mimic the
natural flow of human memory processing - to filter useful information and remember
meaningful facts - in the context of the unceasing flow of new knowledge from the
resources on the Web. It is becoming essential that Web agents are able to predict
future knowledge needs of the diversity of Web users based on their own – the agents’
– experience [1], [2].
It has been proposed that artificial agents – as an agent subgroup – have
autobiography – in terms of the ability of “story-telling”, i.e. reconstruction of their
own past [1]. The framework is applied to autonomous robot behavior. Web agents
are similar to autonomous robots in many ways – in terms of learning, remembering,
planning, and decision-making. In this paper we outline a distinction between the
navigational (automatic) and the autobiographical (cognitive) aspects of Web
browsing on the part of the user, which makes the navigational analogy between the
artificial and the human agents’ performance incomplete.
Design of Web agents is also concerned with identifying lexical or linguistic
features of Web genre (not just topic relevance) like, for example, expertise, detail,
subjectivity, etc. that can be captured by automatically scanning/tokenizing html
Y. Ioannidis, B. Novikov and B. Rachev (Eds.): Local Proceedings of ADBIS 2007, pp. 17-26
© Technical University of Varna, 2007
Design of Web Agents Inspired by Brain Research 18
scripts and extracting meaningful information [3], [4], [5]. In order to make the
process more efficient and sufficiently fast, a lot of heuristically based knowledge is
employed. In some cases, these heuristics are grounded in long-tradition research
areas like rule-based reasoning, neural networks, linguistics and cognitive science.
Our aim is to relate the derived heuristics closer to insights from recent memory and
brain modeling studies.
In the present study we have focused on the phenomenology of episodic memory,
and on one facet of its contribution to the formation of human autobiographical
memory in Web context – the extraction of meaningful fragments from the search
episodes and its relevance to optimization of Web agent performance. The theta phase
precession theory of hippocampal memory is chosen among related models for its
account of complex cognitive phenomena like object-context integration, single-trial
learning and time-space contextualization [6], [7]. The employed neuro-realistic
model is capable of abstraction of meaningful fragments of knowledge, rather than
snapshots of the retrieved Web pages, which makes it closer to the human way of
interacting with the Web and is appropriate for information retrieval optimization.
2 Information Retrieval Studies based on Cognitive Heuristics
One application area of the information retrieval studies and information extraction
algorithms dealing with a variety of stylistic and genre features is the Semantic Web,
where software agents are built to analyze, restructure, transform and display the
information, which currently exists in human-readable form. The algorithms used for
discovering regularities in the html script that are meaningful from a user-viewer
perspective and capable of extracting structural and semantic meaning from the text
are called wrappers. The term wrapper is first proposed in [8] and, by definition, is a
set of “relatively simple information extraction procedures for semi-structured
resources like texts”. Since the amount of regularities is enormous and constantly
changing in the Web pages, a wrapper is a learner and in many cases learning is
based on exposing to examples to extract the inner data or linguistic/text model.
Structural meaning is conveyed by domain ontologies, which can be both designer-
given and user-modified. The process of wrapper construction is called wrapper
induction, which can be either handcrafted or automatically learned [9]. Semantic
meaning is not limited to semantic structure only and can account for the diversity of
user styles, preferences, attitudes, experiences, etc. [3], [10]. Being on the server side
and used for server performance optimization via collecting and processing user
requests, wrapper performance and evolution as a learning algorithm is based on the
dynamics of the regularities in the page content as well as on the multiple user
interactions with the Web.
In earlier work we designed a Java-servlet (a wrapper to a search engine) to
diagnose the level of expertise and detail of a corpus of on-line available Web pages,
employing cognitive science based heuristics [4], [11]. One application area of these
studies is search for cardiological knowledge on the Web. Currently, we have
implemented a MatLab GUI version of the wrapper to test the neurologically inspired
algorithm of remembering the significant fragments of the retrieved pages during
19 Maya Dimitrova, Hiroaki Wagatsuma and Yoko Yamaguchi
Web search. The idea is that the evolution of the meaningful fragment search for the
current user can facilitate the definition of the needed expert level of the text by the
agent.
Fig. 1. Fragment extraction and recombination of textual elements as a process of encoding
of search events
Figure 1 presents the general idea of wrapper performance (fragment extraction
and recombination of textual elements) as a process of encoding of search events. In
following the history of interaction, optimization is needed much the way natural
cognitive systems perform - learn, abstract, remember and forget. We have used the
‘autobiographical memory’ metaphor to apply to such kind of Web agents and the
theta phase precession theory to model it [6], [7], [12].
3 Theta Phase Coding in the Brain and Episodic Memory
Synaptic connections among nerve cells are widely thought to contribute to the
functional selectivity of the nerve assemblies involved in specific cognitive tasks.
Rhythmic neural activity is another collective property very frequently observed in
the brain. One focus of brain research is how the instantaneous property of rhythmic
activity benefits the programming of the brain. Theta rhythmic activity in the rat
hippocampus is observed in spatial navigation, and it has recently been also found in
cortical areas during mental calculation tasks [12]. Theta phase modulation is a
promising candidate for a neural-plasticity correlate of episodic memory tasks
Design of Web Agents Inspired by Brain Research 20
involving hippocampal-cortical relations within a global network of brain
computation.
In freely running rats, a stable theta oscillation, 8-10 Hz,1 is observed in the
hippocampus as the local field potential (LFP). According to the cognitive map theory
[13], any given set of cells that act selectively according to the rat's spatial location,
called place cells, represents an external map. O'Keefe and Recce [14] recently
discovered a phenomenon of systematic acceleration 2 of the firing phase of the cell
relative to theta rhythm as the animal traversed the cell’s receptive field, called “theta
phase precession”. In other words, place cells fire not only depending on the position,
but also depending on the behavioral context. In every theta cycle, the firing phase
shifts in advance, so that cell assemblies in the cycle appear to be a temporal sequence
representing the visiting places according to the running direction. As a result, the
duration of the behavioral sequence is compressed ten-fold and embedded in the cycle
of the theta rhythm [6], [7].
The relevance to learning attracted researchers’ interest in phase precession in the
rat hippocampus. The basic properties of the rhythm interaction system, known as the
entrainment phenomenon, appear to govern this phase precession and it is possible
that the rhythmic pattern appears in the absence of memory to cause memory
formation [6], [7]. The emergence of rhythmic patterns in the absence of memory to
cause its formation is a plausible candidate for a neural correlate of the formation of
complex and durable cognitive effects, governing future behavior i.e.
autobiographical memories.
The Theta Phase Model. Traditionally, a phase model is used as a model of a
limit cycle with a single variable of oscillation phase, assuming the amplitude
converging to a unit circle, with states either the oscillation or the resting. Sustained
oscillation is given as a rotary motion on the unit circle in the presence of the external
input. To generate the phase precession, the important assumption is a gradual
increase of the natural frequency of the phase model during the oscillation state. In the
network model of the hippocampus with the phase model description, the stable
oscillation of the theta rhythm, a LFP theta unit, and neurons with oscillatory
activities are assumed in the entrance of the hippocampus, called the entorhinal cortex
(EC). When an input is coming, the neuron starts to fire and sustains its oscillation
with the gradual increase of the natural frequency. According to the coupling with the
LFP theta unit, the firing phase regularly advances from the late phase to the earlier
phase.
Through the anatomical projection the EC to the hippocampal CA3 layer (CA3),
the temporal firing pattern generated in EC is inherited to CA3 units and it modifies
the recurrent synaptic connections among CA3 units in accordance with the Hebbian
synaptic plasticity with asymmetric time window. As a result, in the temporal coding
by using theta phase precession, the behavioral sequence is compressed into the
temporal sequence of firing phases in the theta cycle, and it makes possible the
encoding of behavioral episodes even in one-time experience, because of the time-
compression and the repetition of the firing pattern in several theta cycles. After the
formation of synaptic connections asymmetrically and locally connected, the
1 Theta rhythm is 4-10Hz in human, but 8-10 Hz in rat experiments
2 The firing phase advances from the late phase to the earlier phase in the phase precession
21 Maya Dimitrova, Hiroaki Wagatsuma and Yoko Yamaguchi
hippocampal network can replay the spatio-temporal firing pattern that represents the
animal’s past experience, as the memory retrieval. The retrieved activity may cause
further synaptic plasticity in other cortical areas for memory consolidation or memory
transfer from the hippocampus to the cortex.
Fig. 2. Model of hippocampal nerve circuit: (a) mechanism of theta phase coding. Behavioral
inputs are transferred to the firing phase in the entorhinal cortex (EC) and provide the
asymmetric connections in the hippocampus; (b) when the rat runs to the right, the phase shift
in firing within each theta rhythm cycle occurs in place cells 1 to 3, which are activated
sequentially; the phase is arranged in order of firing within one phase cycle
This phase coding has an important advantage over rate coding in storing temporal
sequences of events. The temporal compression of an input sequence into every theta
cycle ensures a highly selective synaptic plasticity over the entire time period of an
on-going experience, while in rate coding the synaptic selectivity is lost easily when
the sequence has a variety of time scales. This precession model bridges the gap
between the fixed value of a time interval (several tens of milliseconds) that is
recognized by the synapse and the experienced event (seconds). A variety of input
time series are translated by the theta rhythm, thereby giving the synapse the proper
temporal difference and enabling experienced events to be encoded on the spot.
Relevance to Episodic Memory. Episodic memory as the building element of
autobiographical memory has been the focus of intensive and elaborate studies and
modeling at various levels of memory and brain research – human, animal, cognitive,
neural-physiological, brain-imaging, and so on. In human memory research the term
‘episode’ has been used both operationally i.e. ‘the learning episode’ and
metaphorically, i.e. ‘episodic memory’. Both derive from Tulving’s theory [15], [16]
and refer, in the first case, to the situation or the environment where learning occurs,
and in the second, to the information processing system underlying the acquisition of
new knowledge - and is therefore stressing its processing complexity.
Autobiographical memory, on the other hand, is responsible for remembering the
significant events in one’s biography [17]. The distinction is also in terms of the
different kind (or scale) of information extraction to build the idea/memory of (a
fragment of) an episode or an event.
The phenomenon we have aimed to observe in our study is the structure of the
episodic trace in terms of stored page (in)completeness, search path memorizing and
insight-related experiences that may support optimized future retrieval from memory.
Design of Web Agents Inspired by Brain Research 22
4 Autobiographical Memory in Web Context
Autobiographical memory includes all that we have ever experienced, but more
importantly, all that we have ever learned. In this sense, as a significant part of one’s
daily life, the user-Web interaction is a powerful autobiographical learning
experience. We have hypothesized that the process of episodic learning in Web
environment is both navigational and autobiographical at the same time. Navigation
focuses on the optimization of the trajectory (the traversed links) to a desired goal for
subsequent reuse of the shortest path to reach the same goal. With using the Web for
educational purposes, however, the situation is different in terms of the cognitive
functions and memorizing strategies that are involved, including their
autobiographical elements. Our first assumption is that the interaction with the
present-day Web is an autobiographical experience. Our second assumption is that
memorizing events in a human-like memory system undergoes constant synthesis
principally the same way as any learning process takes place and therefore can be
implemented in autonomous agents on the Web.
We use ‘episodic memory’ in its conceptual meaning – the system of processing
episodes of Web search and transforming them into autobiographical events of
successful or unsuccessful Web navigation to reach the needed goal. Every Web page
viewed by the user is a complex perceptual and semantic stimulus. We assume that
what is stored in episodic memory from the rapid viewing of a succession of pages is
a set of fragments. The stored fragments are cues for subsequent retrieval. These cues
can be perceptual or semantic, random or meaningful to the goal. The theoretical
hypothesis is that episodic memory extracts meaningful cues from the fragments of
the page-retrieval episodes, rather than random perceptual or semantic cues.
The Search Task. The search task was to reach a significant personal goal from
incomplete initial information. We involved two participants in the study in order to
monitor the autobiographical elements of the task – a European researcher, browsing
for an art gallery (motivated) and a Japanese colleague, invited to find the gallery
based on Japanese orthography (neutral). The controlled elements were: a) personal
motivation in reaching the goal and b) emotional involvement in terms of feelings of
anticipation to reach the goal, i.e. anticipation of success. We have aimed to show that
the mechanisms of event synthesis from fragments of page-retrieval episodes in the
episodic memory system are the same for both participants, volunteering to the study.
The Search Goal. The search goal was an example of new concept learning and,
for a foreign language speaker, similar to a case of artificial concept learning. The cue
concept is composed of two Japanese words ‘kurenai-kai’ which to a non-Japanese
speaker may appear like anagrams. Kurenai-kai is related to traditional Japanese craft
arts - Japanese style embroidery. The participants did not know the concept ‘kurenai-
kai’, although it would have been the keyword, retrieving the home page of the
needed art gallery. The discovery of the new concept and its remembering was one
important element of the observed episodic memory based retrieval.
Procedure. We tested memory of the search process in two steps - each on two
subsequent days. The first step was to see if the participants remembered the newly
learned concept words to reach the goal the shortest way and the answer was
affirmative. The second step tested episodic memory of the actual search process and
the remembered search path. The participants were asked to try and recall how they
23 Maya Dimitrova, Hiroaki Wagatsuma and Yoko Yamaguchi
would have searched the same goal provided they had had forgotten the concept
words ‘kurenai-kai’. They copied and pasted their trials to reach the links leading to
the needed home page of the art gallery.
Results and Discussion. The presented in section 3 model of episodic memory has
been successful in reproducing the behavior of rats in a maze in finding the location
of a goal. In a maze behavior starts from random search until the shortest path is
found. Moreover, the model is capable of reproducing situations of single-trial
learning by composing and maintaining retrievable memories of the search-path cues
[6], [7]. Our study of Web search has replicated the mechanism of performance of the
proposed memory model based on theta phase precession, and has supported its
advantages for implementation in autobiographical Web agents. Table 1 gives a
summary of the performed searches by our participants. The Web paths resemble
random search until a meaningful path is composed.
Table 1. User performance on Web search with incomplete initial information about the search
goal
User Searches Time
Random Meaningful Approximate
Motivated 23 3-5 30 min
Neutral 19 3-5 30 min
We have observed that meaningful paths are composed, rather than encountered
that is essential to the understanding of episodic memory. The nature/character of the
search was identical in both cases. It started randomly with possible combinations of
various cue words and browsing the search engine results until the rewarding
(success) combination of cue words was found. The success combination was the
same for both users – kurenai-kai – in either English or Japanese orthography. We
assume that the ‘event’ of finding meaningful and useful information in terms of the
newly learned concept word ‘kurenai-kai’ in one’s autobiographical memory is based
on cognitive mechanisms similar to those in learning new concepts in semantic
memory [18]. Our results are an illustration to this similarity of processing of
knowledge and events in human memory in Web search context.
Table 2. The left column represents episodic memory guided retrieval based on a set of
meaningful cues and ‘insight’. The right column contains the mappings to the respective initial
cues (none of them are identical)
Steps Episodic memory Composition of the Snapshots of the
No guided retrieval retrieval path from search episodes
meaningful fragments
1 Looking for a matching
semantic cue or a useful
word phrase
/”Embroidery Center in
Atlanta”/
Design of Web Agents Inspired by Brain Research 24
2 Closest match with the
14th search step, but
picking up another
hyperlink
3 Remembering that the
insightful encounter of
‘kurenai-kai’ was in the
‘links’ section
4 Mapping the ‘link’ cue
with the 17th search step
(the ‘insight’)
5 The goal site
Table 2 shows the meaningful cues on which episodic retrieval was based for our
motivated participant. The retrieval process is composed of fragments of meaningful
information starting from a matching semantic cue, retrieving previous page,
remembering the ‘insightful’ encounter in the links section, taking the remembered
abstract cue and finding the goal (5 steps). At ‘episodic’ retrieval our motivated
participant usually started from more personalized information about the people
running the art gallery (which was her initial motivation for the search, i.e. element of
her autobiography). Our neutral, i.e. native-speaker participant started from more
general sites about the Japanese craft arts. However, the essential aspect of their
search is reflected in the aforementioned 5 steps. An interesting point to be mentioned
is that the retrieval pages never matched exactly the search pages. Our participants
mapped the meaningful cues, not the exact images of the Web pages.
Implementation. Figure 3 gives the temporal evolution of episodic events in the
task to try to find ‘kurenai-kai’ with initial keywords of ‘Embroidery Center in
Atlanta.’ Gword, TL, and Word correspond to the keywords, used in the Google
search, the titles of the search results, seen by the user in the system window, and the
words in the summary of the search results, respectively, which are automatically
stored in the system. For an autonomous agent on the Web the relevance of this study
is in the following: Users base subsequent retrieval of the search path on remembering
25 Maya Dimitrova, Hiroaki Wagatsuma and Yoko Yamaguchi
the ‘events’ of their search, which are the meaningful fragments of knowledge inside
the Web pages, not the exact ‘page-retrieval’ episodes.
Fig. 3. Temporal evolution of episodic events by the neuro-cognitive agent
We propose that the autonomous Web agent behaves in analogy with the event-
based (autobiographical) strategy that Web users are applying for retrieval of useful
information, which is some state of knowledge with anticipatory value (useful in the
future). Next steps will be tests of the agent for optimized storage of fragments of
knowledge from user searches, rather than of snapshots of sequences of Web pages.
5 Conclusions and Future Work
In this paper we presented one approach towards the design of Web agents that are
able to mimic the natural flow of human memory processing in complex
environments, which is based on new knowledge from brain research. We have tried
to demonstrate how the theoretical framework of theta phase coding endows real-time
process of memory formation of Web search experiences, and the retrieval process of
effective routes to the targets through the experienced sites. The proposed approach
focuses on bridging the gap between neuroscience and Web technologies to enlighten
the importance of the context-dependence in autobiographic memory.
Acknowledgement. This research was carried out while the first author was a visiting
researcher at the Laboratory for Dynamics of Emergent Intelligence of RIKEN BSI
on a scholarship granted by Japan Society for Promotion of Science. The
implementation is partially supported by research project contract No MI-1509/2005
Design of Web Agents Inspired by Brain Research 26
“Multimodal User and Sensor Interface for a Computer-aided System for
Cardiological Diagnosis and Intervention” of the National Research Fund of Bulgaria.
References
1. Nehaniv, Ch., Dautenhahn, K.: Embodiment and Memories: Algebras of Time and History
for Autobiographic Agents. Proc. 14th European Meeting on Cybernetics and Systems
Research, EMCSR (1998) 651-656
2. Ekdahl, B.: Agents as Anticipatory Systems. Proc. 4th World Multiconference on Systemics,
Cybernetics and Informatics (SCI 2000) and 6th International Conference on Information
Systems Analysis and Synthesis (ISAS 2000) Orlando, Florida, U.S.A. (2000)
3. Kushmerick, N.: Gleaning Answers from the Web. Proc. AAAI Spring Symp. on Mining
Answers from Texts and Knowledge Bases, Palo Alto (2002) 43-45
4. Dimitrova, M., Kushmerick, N.: Dimensions of Web Genre. WWW2003, Budapest,
Hungary, http://www2003.org/cdrom/papers/poster/p143/p143-dimitrova.htm (2003)
5. zu Eissen, S. M., Stein, B:. Genre Classification of Web Pages: User Study and Feasibility
Analysis. In: Biundo S., Fruhwirth T., Palm G. (eds.): Advances in Artificial Intelligence,
Springer (2004) 256-269
6. Yamaguchi, Y.: A Theory of Hippocampal Memory Based on Theta Phase Precession.
Biological Cybernetics, Vol.89 (2003) 1-9
7. Wagatsuma, H, Yamaguchi, Y.: Neural Dynamics of the Cognitive map in the Hippocampus.
Cognitive Neurodynamics, 1 (2007) 1871-4080
8. Kushmerick, N., Weld, D.S., Doorenbos, R.B.: Wrapper Induction for Information
Extraction. Int. Joint Conference on Artificial Intelligence IJCAI (1997) 729-737
9. Hogue, A., Karger, D.: Wrapper Induction for End-User Semantic Content Development.
Proc. First Int. Workshop on Interaction Design and the Semantic Web ISWC (2004)
10. Karlgren J.: Stylistic experiments in information retrieval. In T. Strzalkowski (Ed.) Natural
Language Information Retrieval, Kluwer (1999)
11. Dimitrova, M.: Cognitive Modelling and Web Search: Some Heuristics and Insights.
Cognition, Brain, Behavior, Vol. VII, 3 (2003) 251-258
12. Mizuhara, H., Wang, L.Q., Kobayashi, K., Yamaguchi, Y.: A Long-Range Cortical
Network Emerging with Theta Oscillation in a Mental Task. Neuroreport, Vol.15, 8 (2004)
1233-1238
13. O’Keefe, J., Nadel, L.: The Hippocampus as a Cognitive Map. Clarendon Press, Oxford
(1978)
14. O’Keefe J., Recce, M.: Phase Relationship between Hippocampal Place Units and the EEG
Theta Rhythm. Hippocampus, 3 (1993) 317-330
15. Tulving, E.: Elements of Episodic Memory. New York: Oxford University Press (1983)
16. Lepage, M., Ghaffar, O., Nyberg, L., Tulving, E.: Prefrontal Cortex and Episodic Memory
Retrieval Mode. PNAS, Vol. 97, 1 (2000) 506-511
17. Conway, M.A.: Autobiographical Knowledge and Autobiographical Memories. In: Rubin,
D.C. (Ed.): Remembering Our Past. Studies in Autobiographical Memory, Cambridge
University Press (1996) 67-93
18. Thompson-Schill, S.L.: Neuroimaging Studies of Semantic Memory: Inferring "How" from
"Where". Neuropsychologia, 41 (2003) 280-292