=Paper=
{{Paper
|id=Vol-2542/MOD-KI7
|storemode=property
|title=Towards Enhancing of Situational Awareness for Cognitive Software Agents
|pdfUrl=https://ceur-ws.org/Vol-2542/MOD-KI7.pdf
|volume=Vol-2542
|authors=Marina Tropmann-Frick,Thomas Clemen
|dblpUrl=https://dblp.org/rec/conf/modellierung/Tropmann-FrickC20
}}
==Towards Enhancing of Situational Awareness for Cognitive Software Agents==
https://ceur-ws.org/Vol-2542/MOD-KI7.pdf
Joint Proceedings of Modellierung 2020 Short, Workshop and Tools & Demo Papers
178 Workshop on Models in AI
Towards Enhancing of Situational Awareness for Cognitive
Software Agents
Marina Tropmann-Frick ,1 Thomas Clemen1
Abstract: Software agents have gained increasing attention in the field of creating digital twins of
physical, biological, and human entities. The processing of sensory inputs, individual perception,
and the selection of suitable actions are essential processes in here, and agent-based frameworks
can be utilized for supporting the design, the implementation, and the test. This paper reflects a
work-in-progress project at an early stage. However, a conceptual model is presented for an analytical
situation awareness component combining agent-based approach and data science algorithms.
Keywords: Agent-based modeling; data science; cognitive processes
1 Introduction
Decision-making is a high-level cognitive process based on cognitive processes like
perception, attention, and memory [Pr17]. Gaining a better mechanistic understanding of
the decision-making process of humans is an important research topic in psychology and
cognitive sciences, whereas the design and implementation of models that represent these
processes are where computer scientists come into play.
Cognitive architectures refer to both a theory about the structure of the human mind and to
a computational representation of such a theory used in the fields of artificial intelligence
(AI) and computational cognitive science [Li]. Here, multi-agent systems provide a capable
platform for running experiments in this field [TW12]. [KT] mentioned the connection
between cognitive architectures and deep learning and predicted that deep learning methods
would likely play an essential role in designing of cognitive architectures in the future.
The conceptual approach described in this study reflects the work-in-progress state of ideas
about how to combine agent-based modeling with data science methods.
1.1 Cognition modeling
Cognitive architectures are a part of research in general AI with the ultimate goal of
modeling the human mind, eventually enabling us to build human-level artificial intelligence
[KT]. A comprehensive review of implemented cognitive architectures has been undertaken
1 Hamburg University of Applied Sciences, Department of Computer Science, Berliner Tor 7, 20099 Hamburg,
Germany, {Marina.Tropmann-Frick,Thomas.Clemen}@haw-hamburg.de
Copyright © 2020 for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
Towards Enhancing of Situational Awareness for Cognitive Software Agents 179
by [Sa10]. More recently, [KT] provides a summary of the history of architectures, noting
55 existing architectures.
One of the most referenced models in this context is ACT-R ((Adaptive Control of Thought-
Rational) [RTO19]. ACT-R was initially designed to model the visual perception and
attention of humans [AMC97]. The ACT-R theory has a computational implementation as
an interpreter of a specific coding language. The interpreter itself is written in Common
Lisp and might be loaded into any of the Common Lisp language distributions. ACT-R
reflects a theory of mechanisms that make up cognition. That theory posits a fixed set of
mechanisms that use task knowledge to perform a task, thereby predicting and explaining
the steps of cognition that form human behavior [RTO19].
As an example, [Sa06] has created a set of declarative and procedural rules about how to
drive a car. He then added this knowledge to ACT-R that had its vision and motor systems
connected to a car simulator (example taken from [RTO19]).
Unfortunately, the structure of ACT-R and its implementation was difficult to incorporate
into an agent-based simulation system. Therefore, the C4 brain architecture model [Is01]
was evaluated and adapted for this purpose [TCK10]. C4 was originally intended to simulate
autonomous and semi-autonomous creatures in games or other virtual environments. We
enriched this architecture by an emotional component, a filter component following the
OCC model [OCC88], and applied it to a pedestrian crowd dynamics scenario similar to
what was described by [TGB09]. The results (see [TCK10] for some details) encouraged us
to choose this architecture for this study as well.
1.1.1 The MARS Framework
MARS2 is an agent-based modeling and simulation framework, which was developed at the
Hamburg University of Applied Sciences, Germany [Hü]. It incorporates a domain-specific
modeling language [Gl17], which helps scientists from a large variety of disciplines and
without significant skills in programming developing their models. Modeling of cognitive
processes and human decision-making is one of the focus research topics of the MARS
Group. [TCK10] described the impact of emotions on pedestrians at a market place, whereas
[LWC] utilized the goal-oriented action planning paradigm in MARS to evaluate adaptive
behavior.
1.2 Data science approaches
Data science is a rapidly emerging field incorporating a large amount of data-driven ap-
proaches. A combination of data science methods and agent-based modeling is a challenging
research area. Many approaches integrate, especially Reinforcement Learning, for their
purpose. In [Ch17], the proposed approach is used to find time-efficient collision-free paths
2 www.mars-group.org
180 Tropmann-Frick, Clemen
for multi-agent systems. The authors develop a value network that encodes the estimated
time to the goal given an agent’s joint configuration with its neighbors. The use of the value
network not only increases efficiency for finding a collision-free velocity vector but also
considers the uncertainty in the other agents’ motion.
Another approach using reinforcement learning focuses on a multi-agent setting for au-
tonomous driving [SSS16]. The authors analyze especially the long term driving strategies
and introduce a so-called option graph for reducing the variance of the gradient estimation.
Several other works applied recently similar methods to the multi-agent domain, e.g., the Go
[Ma14] and Altari [Ta17] games. The authors in [Su16] concentrate on the collaboration
of multiple agents. The remarkable part of this work is the application of a rather simple
neural model for a complex task of learning to communicate between the agents.
2 Modeling the cognitive process
As we know from our own experiences, humans perceive their world more holistically,
switching their attention to details if necessary. Thus, many of our everyday decisions are
made unconsciously [Ka12].
Software agents, on the other side, explore their spatial vicinity sequentially, iterating
through specific objects, e.g., other agents, points-of-interests. In some cases, it would be
preferable that agents can perceive an entire situation in one step. The reasons behind might
be a better alignment to human perception or a simple increase in processing performance.
Mainly, if software agents are representing digital twins of biological or technical entities,
both aspects could be substantial.
2.1 Sensory capture of a scene
The cognitive perception of a scene relies on the input data provided by available sensors.
These data can be used to provide a structured representation of the scene for further
processing by the next components, such as the working memory component of the C4
architecture. The structured representation of the scene is used to define the situation
awareness components of the agents.
For this purpose, we use the profile and portfolio models of the agents similar to [JT14]
and [TT10]. Analyzing the situation awareness of a particular agent, we use the profile of
this agent for processing the available information based on the abilities of the agent. A
profile defines the parameters of an agent, starting with a stereotype based on the social
system and environment of the agent. It contains information about the personality that the
agent represents with general properties, such as psychological context, education, training,
behavioral pattern, preferences, expertise, knowledge, or experience. Using the learning
component, the profile of the agent develops gaining more knowledge and experience as well
as adapting to the communities and the changing environment and new situation parameters.
The portfolio describes the involvement, tasks, and restrictions of an agent as well as their
Towards Enhancing of Situational Awareness for Cognitive Software Agents 181
operating domains. A portfolio is specified by responsibilities and combined with the
possible outcomes of the tasks. A task is an activity with an initial and a target state that
includes interfaces or channels for communication and collaboration. Tasks also contain
operating functions and are characterized by a set of parameters representing conditions
and restrictions. The refinement of those parameters at run-time determines the execution
accomplishment of the task. The result of task execution defines the transition to the target
state and should satisfy the target conditions.
2.2 Individual perception and behavior selection
Each agent constructs a predictive behavior model for potential behavior in the next step.
This model is based on calculations with possible combinations of actions of an agent.
These combinations can be constructed as graphs with edges representing transitions from
one state (starting with the current state) to a possible next state. Possible transitions are
defined by the actions from the portfolio of the agent.
The situation awareness component of each agent also contains the perception of other
agents. In a simulation environment, an agent has permanent access to the profile and
portfolio information of other involved agents. In a specific scene, an agent can observe the
environment and get information about the visible agents.
This information can be used to predict possible situation development for all possible
actions. Each possible action for the next step can be prioritized based on the evaluation of
the resulting situation with a cost function.
The following picture (Fig. 1) shows a schematic representation of the next step from the
viewpoint of the left (blue) agent. The agent’s portfolio contains (in this example) three
possible actions. There are also two more agents involved, whose actions influence the
possible outcome and are considered in the evaluation of situation development for each
action taken by the left agent. There are three possible situation developments, depicted as
colored ovals, one for each possible action of the agent. For the other agents, only the most
probable action predicted by the personal behavior model is considered in the evaluation.
The action marked with a star can be considered as the one leading to the preferable situation
development.
However, it is not sufficient to predict the actions only for the next step. A more reliable
model should be built by calculations for the next three or more possible steps, leading to
complex situation development.
A directed graph with possible action paths represents the predictive behavior model. Each
path is evaluated based on probabilities and includes a priority function for the outcomes.
The result of the evaluation is a ranking order of the paths. The next step for the agent will be
chosen from the path with the highest ranking. Due to the exponentially increasing number
of outcomes with each additional step, there are limitations for the calculated path length.
To reduce the complexity caused by the variety of possible combinations and to define a
robust priority function, a classifier will be trained based on historical data. This classifier
182 Tropmann-Frick, Clemen
Fig. 1: Analysis of the situation development from the point of view of the left agent.
calculates the probabilities of all possible actions in the portfolio of the agent. The decision
on what action to take is then made based on the highest probability provided by the
classifier.
2.3 Learning
The learning of an agent is based on data science methods.
The data science methods are responsible for the accurate situation evaluation, and adaptation
to dynamic situation changes. Data generated in previous simulations will be used to build
individual knowledge bases for the agents. This knowledge then can be used to improve the
learning of the agent behavior model.
The new goal as a result of this is the learning of interactions between the agents. The agents
communicate and collaborate in their shared environment — this collaboration influence
the behavior and negotiation abilities of the participating agents.
One of the essential tasks is to keep the balance between adaptation to the unexpected
behavior of other agents, or dynamic situation changes, and ’normal’ behavior (e.g., to be
not too defensive).
3 Conclusions
Instantaneous risk assessment of a situation in our daily lives, decision making, and learning
are essential factors for human beings. Our conceptual approach describes the first brief
idea of how to equip software agents with these capabilities by incorporating modern data
science algorithms into their cognitive architecture.
At the time of writing, concrete simulation results are still missing. However, capturing
an entire scene instead of exploring it object by object would increase the operational
Towards Enhancing of Situational Awareness for Cognitive Software Agents 183
performance of software agents. That is an essential issue if agents are considered as digital
twins of humans or autonomous cars, for example.
During the next months, we will implement the conceptual approach described above within
the MARS Framework utilizing the C4 cognitive architecture. Together with colleagues
from robotics, we will develop a digital twin for an autonomous vehicle using this concept.
Here, real-time sensory input from an IoT infrastructure replaces the software sensors of the
MARS agents.
Bibliography
[AMC97] Anderson, John R.; Matessa, Michael; Christian Lebiere: ACT-R: A Theory of Higher
Level Cognition and Its Relation to Visual Attention. Human-Computer Interaction,
12:439–462, 1997.
[Ch17] Chen, Y. F.; Liu, M.; Everett, M.; How, J. P.: Decentralized non-communicating multiagent
collision avoidance with deep reinforcement learning. In: 2017 IEEE International
Conference on Robotics and Automation (ICRA). pp. 285–292, 2017.
[Gl17] Glake, D; Weyl, J; Hüning, C; Dohmen, C; Clemen, T: Modeling through Model Transfor-
mation with MARS 2.0. In: Proceedings of the 2017 Spring Simulation Multiconference.
ADS ’17, Society for Computer Simulation International, Virginia Beach, Virginia, USA,
p. 12, 2017.
[Hü] Hüning, Christian; Adebahr, Mitja; Thiel-Clemen, Thomas; Dalski, Jan; Lenfers, Ulfia;
Grundmann, Lukas; Dybulla, Janus; Kiker, Gregory A.: Modeling & Simulation as a Service
with the Massive Multi-Agent System MARS. In: Spring Simulation Multiconference.
ADS ’16 section 2, Society for Computer Simulation International, San Diego, CA, USA,
pp. 1–8.
[Is01] Isla, Damian; Burke, Robert; Downie, Marc; Blumberg, Bruce: A layered brain architecture
for synthetic creatures. In: IJCAI’01 Proceedings of the 17th international joint conference
on Artificial intelligence - Volume 2. pp. 1051–1058, 2001.
[JT14] Jaakkola, Hannu; Thalheim, Bernhard: Multicultural Adaptive Systems. In: Information
Modelling and Knowledge Bases XXVI, 24th International Conference on Information
Modelling and Knowledge Bases (EJC 2014), Kiel, Germany, June 3-6, 2014. pp. 172–191,
2014.
[Ka12] Kahneman, Daniel: Thinking, Fast and Slow. Penguin, 2012.
[KT] Kotseruba, Iuliia; Tsotsos, John K.: 40 Years of Cognitive Architectures: Core Cognitive
Abilities and Practical Applications. Artificial Intelligence Review.
[Li] Lieto, Antonio; Bhatt, Mehul; Oltramari, Alessandro; Vernon, David: The role of cognitive
architectures in general artificial intelligence. Cognitive Systems Research, pp. 1–3.
[LWC] Lenfers, Ulfia A.; Weyl, Julius; Clemen, Thomas: Firewood Collection in South Africa:
Adaptive Behavior in Social-Ecological Models. Land, (3):97, aug.
[Ma14] Maddison, Chris J; Huang, Aja; Sutskever, Ilya; Silver, David: Move evaluation in Go
using deep convolutional neural networks. arXiv preprint arXiv:1412.6564, 2014.
184 Tropmann-Frick, Clemen
[OCC88] Ortony, Andrew; Clore, Gerald L.; Collins, Allan: The Cognitive Structure of Emotions.
Cambridge University Press, New York, NY, 1988.
[Pr17] Prezenski, Sabine; Brechmann, André; Wolff, Susann; Russwinkel, Nele: A cognitive
modeling approach to strategy formation in dynamic decision making. Frontiers in
Psychology, 8(AUG), 2017.
[RTO19] Ritter, Frank E.; Tehranchi, Farnaz; Oury, Jacob D.: ACT-R: A cognitive architecture for
modeling cognition. Wiley Interdisciplinary Reviews: Cognitive Science, 10(3):1–19,
2019.
[Sa06] Salvucci, Dario D.: Modeling driver behavior in a cognitive architecture. Human Factors,
48(2):362–380, 2006.
[Sa10] Samsonovich, Alexei V.: Toward a Unified Catalog of Implemented Cognitive Architectures.
In: BICA 2010. 2010.
[SSS16] Shalev-Shwartz, Shai; Shammah, Shaked; Shashua, Amnon: Safe, Multi-Agent, Reinforce-
ment Learning for Autonomous Driving. CoRR, abs/1610.03295, 2016.
[Su16] Sukhbaatar, Sainbayar; Fergus, Rob et al.: Learning multiagent communication with
backpropagation. In: Advances in Neural Information Processing Systems. pp. 2244–2252,
2016.
[Ta17] Tampuu, Ardi; Matiisen, Tambet; Kodelja, Dorian; Kuzovkin, Ilya; Korjus, Kristjan;
Aru, Juhan; Aru, Jaan; Vicente, Raul: Multiagent cooperation and competition with deep
reinforcement learning. PloS one, 12(4):e0172395, 2017.
[TCK10] Thiel-Clemen, Thomas; Klingenberg, Arne: Kombination von zielorientiertem Verhalten
und Emotionen in Individuen-orientierten Simulationen. In (Wittmann, Jochen; Maretis,
D K, eds): Simulation in den Umwelt- und Geowissenschaften, Workshop Osnabrück.
Gesellschaft für Informatik, Shaker, pp. 71–80, 2010.
[TGB09] Tasse, Flora Ponjou; Glass, Kevin; Bangay, Shaun: Simulating crowd phenomena in
African markets. Proceedings of AFRIGRAPH 2009: 6th International Conference on
Computer Graphics, Virtual Reality, Visualisation and Interaction in Africa, 1(212):47–52,
2009.
[TT10] Thalheim, Bernhard; Tropmann, Marina: Performance Forecasting for Performance
Critical Huge Databases. In: Information Modelling and Knowledge Bases XXII, 20th
European-Japanese Conference on Information Modelling and Knowledge Bases (EJC
2010), Jyväskylä, Finland, 31 May - 4 June 2010. pp. 206–225, 2010.
[TW12] Tuyls, Karl; Weiss, Gerhard: Multiagent Learning: Basics, Challenges, and Prospects. AI
Magazine, 33(3):41, Sep. 2012.