There is no comprehensive theory for how anticipatory thinking capabilities emerge from cognitive processes. Event cognition describes some human anticipatory thinking capabilities, but is not integrated with general theories of cognition. We use Event Segmentation Theory to motivate a theoretical account for how the Soar cognitive architecture and the Common Model of Cognition can be extended to support event cognition, and in turn account for anticipatory thinking processes and reasoning. Current cognitive architectures appear to require additional mechanisms to create computational models implementing this theoretical account.
Anticipatory thinking (AT) is an emergent cognitive
functionality. AT has been described as the ability to proactively
guide attention and take preparatory action
Event cognition research studies the human ability to
perceive, understand, and remember everyday events
While there is a neuro-physiological account for some
aspects of event cognition
Including mechanisms for human-like event cognition
in cognitive architectures can provide such an integration
to better understand how human AT functionality emerges
from cognitive processes. Cognitive architectures are
theories for the fixed computational mechanisms that underlie
cognition. While many architectures initially made
different and conflicting assumptions or described isolated aspects
of cognition, over time a consensus has emerged. This
consensus is formalized through the Common Model of
Cognition, which is a theoretical specification of the
computational processes underlying cognition
With support from observations of human behavior
The theory proposes that humans use mental models for
events. These models are divided into event models
describing specific situations and event schemas describing the
commonalities for a given type or class of event
During everyday tasks, event models predict changes to
the current situation and guide perceptual processing
The EST process model focuses on descriptions of ongoing perception for a directly-experienced event. Anticipatory thinking appears to require reasoning that includes expectations for future events beyond short-term expectations for the currently-experienced event, which motivates our account of event cognition using a cognitive architecture.
Cognitive architectures are computational models for the fixed mechanisms and processes that underlie cognition. These architectures act as theories for the functionality provided by different memory systems and cognitive processes. They also can be used to implement artificial cognitive systems. However, these architectures do not currently exhibit the event cognition functionality found in humans.
EST specifies representations of events, but does not describe how (together with other mental models) they are encoded, stored, or retrieved from memory systems, nor the reasoning processes that use them. Cognitive architectures can extend event cognition theory by including the memory systems and reasoning that EST lacks. An intriguing possi1In EST, event models are hierarchical. Thus, a single working event model describes the current situation, but it can contain nested sub-events that are event models for smaller space and/or shorter segments of time. bility is to explore the integration of EST with the Common Model of Cognition. Unfortunately, due to its abstract nature, the Common Model does not provide the level of detail necessary for implementation of running computational models. Instead, we use Soar, an architecture consistent with the Common Model, as a model for how cognitive architectures (and, more abstractly, the Common Model) can realize event cognition functionality.
Soar models cognition as a series of deliberate actions that
perform reasoning steps, retrievals from long-term
memories (episodic or semantic), or motor actions
To theoretically model event cognition phenomena using Soar, we map the different mental representations specified by EST to Soar’s memory systems. Both event schemas and event models contain relational information and lack perceptual detail. They contain entities and relations for describing an event, which are directly supported by the memory systems of Soar. We assume that event schemas and models have relations depicting changes over time, allowing for representation of future state using these relations. The association of event schemas and event models to the memory systems of Soar is depicted in Figure 3.
The working event model is grounded to ongoing action and perception. It is also used in reasoning about the current situation. To provide this functionality, it must be composed of working memory structures and also representations of perception and action. Figure 3 depicts the working event model within working memory, but specifically as including the representations for perception and control.
Retrieved Event Models
Working Event Model
In contrast, event models representing prior situations and
event schemas require long-term storage and need to be
stored within the long-term declarative memory systems in
Soar. Event models have been hypothesized as the episodes
of episodic memory in humans
By assigning the mental representations described by EST to the memory systems of Soar, we can replicate the EST process model using the mechanisms available in Soar. Perception feeds into working memory and cues for retrieval of an event schema from semantic memory. This event schema can then be grounded to perception and action to form a working event model that is compared to perception. Then, the ways in which event models within working memory can be used for reasoning depends on the reasoning strategies within procedural memory.
The contribution of this model is that reasoning is not limited to only performing the EST process model loop of maintaining a working event model to describe the present. In situations where an agent performs long-term planning (representing and reasoning about the future beyond the current event), additional event models that represent expected distant future states are retrieved into working memory to be used in reasoning. Also, previous event models can be retrieved for comparison of a specific previous situation to the present for case-based reasoning. These additional reasoning capabilities arise from the general cognitive mechanisms available for using and manipulating stored event knowledge.
The cognitive mechanisms available for manipulating event
representations in Soar’s memory systems enable modelling
of AT processes. Anticipatory thinking is associated with
three distinct processes. These processes are “recognition of
a situation based on current cues derived from previous
experience, extrapolation of a system state to a different state,
and construction of a mental model of the system based
on variable evidence”
The process of recognition uses cues from the present situation to retrieve knowledge for similar situations from the past. An example of pattern matching is the recognition of someone in the act of creating exam packets by observing them in the copy room printing papers. In our model of event cognition, there are two forms of recognition. When there is knowledge for a type of event that generalizes multiple specific events, this is stored as an event schema in semantic memory. Recognition can take the form of retrieval of an event schema from semantic memory based on the cue that someone is printing papers. However, if such knowledge is not available, there can also be knowledge of a specific similar event from the past. Recognition can thus also result from retrieval of an event model from episodic memory.
The process of extrapolation involves not only
predicting future states, but also guides action in conjunction with
predictions to realize a desired future state. An example
is catching a ball, but extrapolation also refers to
narrative understanding and prediction
Construction is the ability to reason about and create mental models for a situation. We model this as reasoning about the conditions and connections between events. When recognizing that someone is involved in a task which will require a stapler in the future, we have the ability to integrate an event model for delivery of that stapler with an event model for someone else’s future use of the stapler, allowing us to coherently model both our delivery and the satisfaction of their task. This ability to evaluate how our planned actions will impact external events in the future is an example of conditional AT which uses causal relations between event models. Soar supports construction through relations in working memory that link different event models. The model associated with preparing exams and the model for fetching a stapler can be combined to form a composite mental model within working memory.
These processes do not directly map to individual mechanisms in the architecture, but are supported by existing mechanisms and representations. In combination, these processes enable different types of AT reasoning.
In EST, event models are updated following misprediction. However, events often proceed as expected with little additional reasoning required to guide action. During these periods, proactive reasoning can be performed to prepare for future events without jeopardizing reactivity in the present. This is one case in which it is possible to perform AT.
Alternatively, an agent may have a goal, but does not have sufficient event schema knowledge for how to realize its goal. (The knowledge may not exist as an event schema in semantic memory or it may be difficult to cue for retrieval.) In this case, an agent needs additional knowledge to proceed.
In either of these cases, an agent can perform additional types of reasoning beyond the default EST behavior of retrieving a single event schema to update the current working event model. Soar provides mechanisms to account for these types of additional reasoning.
In Soar, agents can detect when their knowledge for the current situation is insufficient to select additional actions. These situations are architecturally-recognized as impasses. Note that this is distinct from misprediction. An agent could have a good model of the environment, but not have the knowledge for how to act or how the currently available actions will impact goal achievement. To resolve these impasses, additional knowledge is brought into working memory to guide action. These moments during which it is unclear which actions to perform (either in the present or in preparation for the future) provide opportunities for AT.
Geden et al. describe three types of anticipatory thinking that depend on the aforementioned AT processes: prospective branching, backcasting, and retrospective branching. Using our model of event cognition, these types of anticipatory thinking emerge from general cognitive processes in Soar (and potentially in other cognitive architectures) that support search-based planning and means-ends analysis.
Prospective branching refers to imagining potential future states, given the current state. Search-based planning is an analogous form of reasoning in which an agent imagines potential futures by simulating actions using action models. When an agent has a goal, but the agent does not have knowledge for which actions will accomplish this goal, an agent performs search-based planning to simulate how available actions would change the situation.
Backcasting is reasoning that finds ways or paths to a particular future state. Means-ends analysis performs similar reasoning in Soar. In order to determine a path to a future state, reasoning proceeds backwards from the future state, attempting to create a plan of actions that can achieve the future state, while recursively attempting to achieve the preconditions of those actions until a path is found with preconditions that are satisfied in the current state.
Retrospective branching also involves determination of the preconditions for achieving a given state. It can be implemented with means-ends analysis, but using the present state as the initial cue for retrieval instead of a future state.
Traditionally, these forms of reasoning leverage actionmodel knowledge stored in procedural memory. Action models feature preconditions and causally-related effects. However, AT includes reasoning for distant or indirectlyexperienced states while action models describe local experience. Using event schemas and event models generalizes the aforementioned forms of reasoning to perform AT2.
With each of these methods, the same underlying architecture is used and reactivity to the current situation is maintained by incremental processing. As in human AT, if an action must be taken in the moment, this reasoning may be interrupted or forgotten. Additionally, as in human AT, this reasoning can fail if there is simply insufficient knowledge available in memory. The main distinction between existing reasoning methods and the provision of AT functionality is the use of event models as the knowledge for simulating the environment. Thus, the main challenge in implementing AT in this model is learning and encoding event schema knowledge that includes causal relations and preconditions.
So far, we have only considered a theoretical specification.
Soar, and potentially other cognitive architectures,
implement the forms of reasoning described above. However,
cognitive architectures do not generally contain the necessary
mechanisms to implement event cognition. A full
implementation of event cognition includes, but is not limited to:
automatic learning of event schemas, event model
misprediction or surprise detection, memory for the past in terms of
event models, and mechanisms for retrieving event models
and event schemas based on their contents
Other cognitive architectures besides Soar have included mechanisms that partially support event cognition. These architectures include Sigma, ACT-R/e, and Icarus.
Sigma has mechanisms for detecting surprise
2This is similar to the approach taken by Cardona-Rivera et al. that used a planning-based knowledge representation for narratives.
ACT-R/e has been used to model some aspects of
event cognition explicitly
Icarus supports event cognition with a dedicated episodic
memory store
These architectures motivate extending the specification of the Common Model to provide a formal account for event cognition and anticipatory reasoning. Limitations to the Common Model include insufficient specification of how query mechanisms retrieve event models and event schemas, no event schema learning, little evaluation of error detection or misprediction mechanisms for event models, no delineation between episodic and semantic memory, and no direct specification for what generally constitutes event cognition functionality. A specification of event cognition functionality in general (including reasoning, memory, and learning) could motivate further implementation and evaluation among architectures.
Additional support for event cognition in cognitive architectures will allow for computational models of human anticipatory thinking performance. This modelling depends on integrating event representations with existing agent reasoning for achieving goals. Pursuing this specification and implementation will provide further constraint into which architectural mechanisms and agent knowledge are useful – both for modelling humans and for implementing AT functionality in artificial systems.
Special thanks to Jeffrey Zacks for comments on EST. The work described here was supported by the Office of Naval Research under Grant N00014-18-1-2010. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressly or implied, of ONR or the U.S. Government.