- This paper presents the general requirements to build a “cognitive system for decision support”, capable of simulating defensive and offensive cyber operations. We aim to identify the key processes that mediate interactions between defenders, adversaries and the public, focusing on cognitive and ontological factors. We describe a controlled experimental phase where the system performance is assessed on a multi-purpose environment, which is a critical step towards enhancing situational awareness in cyber warfare.
A cyber attack by a hostile nation-state or political organization is widely regarded as one of the most serious threats that the U.S. will face in the next decades. While greatly increased use of information systems has contributed enormously to economic growth, and has fueled a much more efficient and agile national defense, it has also made the U.S. enormously vulnerable to a variety of Internet and non-Internet cyber attacks, and to cyber espionage [1].
There are numerous factors that make cyber warfare and pure cyber defense, namely cyber security, especially problematic. The kinds of threats are diverse: destruction or theft of data, or interference with information systems and networks, across a spectrum of private and public interests. The legal and ethical status of cyber attacks or counterattacks by states are also unclear, at least when deaths or permanent destruction of physical objects does not result. It is still an open question what U.S. policy is or should be, and how cyber threats are analogous to traditional threats and policies—for example whether “first use” deterrence, and in-kind responses apply, and whether a policy of pure cyber defense does not put the far greater burden on attacked rather than attacking nations [2]. As this overview may suggest, untangling the complexity of cyber attacks becomes a key element for augmenting situational awareness in the cyber environment: in this position paper, we propose to tackle this problem from a semantic and cognitive modeling perspective, combining ontologies and
In general, time variables play an important role in the design of decision support systems [3]: temporal constraints become even more stringent when those systems have to deal with cyber attacks, where real time responses are typically hindered by the knowledge-intensive nature of cyber operations and associated tasks. Some decisions on where and when to invoke various methods of cyber defense and mitigate damage, as well as decisions to launch a cyber counterattack, need to be made quickly. Large-scale cyber attacks or counterattacks are likely going to require careful, human decision-making for some time into the future. Yet there are other responses to cyber attacks or cyber espionage that could and should be done immediately, such as revoking an employee’s access if suspicious activity is detected, blocking all remote access or from certain URLs and through certain servers, immediate assessment of likely damage and risks, and so on. What we propose in this paper is the building of a cognitive system for decision support that will emulate ideal human responses to cyber attacks. This would be accomplished through careful design of its architecture, both in terms of cognitive mechanisms and knowledge resources, and by comparing its outputs on case studies with actions of human agents. The benefits are threefold. First, by cognitive modeling we come to better understand the mechanisms underlying human decisions in the realm of cyber warfare and cyber espionage, coupling the cognitive aspects and the semantic contents of decision-making. Second, after extensive testing we could use this intelligent decision-making system to recommend steps to human decision-makers—e.g., recommendations to gather further information, or actually to act in a certain way and to assess the risks of not acting. Finally, in cases where the reliability of the system is high, and where time is of the essence or the actions have little risk (such as revoking one employee’s system access, or access to one URL), the intelligent system could act swiftly and autonomously.
Some forms of attacks, such as Distributed Denial Of Service (DDoS) and other botnet jamming of networks or servers, show signs of admitting purely technological solutions. However, human error by employees has repeatedly been cited as the most common source of vulnerability [4], [5], [6], [7]. One technique of gaining illegitimate access to an information system that still appears with remarkable frequency is spearphishing: emails to DOD employees or defense contractors with spoofed addresses from acquaintances that seem to have a harmless photograph, PDF, or other attachment1. While this exploitation might not alone gain direct access to secure systems, it may allow an attacker to gather personal information that can be used to guess passwords, answer security questions, and so on. Social networking sites and other open data and the use of analytics allow attackers to identify employers, friends, relatives, shopping and driving habits, and so on. This aids an attacker enormously in the identification of targets and gaining access: for instance, in a recent case the New York Times’ sites were brought down when a group claiming to be the Syrian Free Electronic Army used social media and spear phishing to gain access to employees' passwords to the server that handled the NY Times' Domain Network System (DNS). Likewise even if smartphones and other portable devices are not used at secure locations and do not contain classified or sensitive data, hacking into them (or intercepting cellular and WiFi communications, including with vehicles and home monitoring devices) can provide personal data that can be utilized to make direct attacks.
III.
Our approach is inspired by the notion of “sociotechnical system” [8], which emphasizes the interaction between people and technology in workplace. Ontology analysis has recently proved to be an effective tool for investigating these complex aspects [9]: nevertheless, the interactive nature of sociotechnical systems demands a broader framework, where human behavior can be studied not only in terms of action schematics, planning and rules, but also as a genuinely cognitive phenomenon, which can be properly investigated only as a dynamic system. Accordingly, the key elements of our proposed method for modeling cyber operations are: •
Cognitive architecture – design and development of
cognitive models of decision-making in cyber defense
1 Because of their prevalence and complexity in terms of kind and number of
cognitive agents, we intend to include these as paradigms of our use-cases.
•
•
based on ACT-R 2 cognitive architecture [
formal ontologies to 1) serve as a knowledge base for our cognitive models (Cyber Security Ontologies) and to 2) classify and annotate cyber security test and training data (Scenario Ontologies).
Enable the analysis of cyber defense strategies; support training for cyber security personnel; validate the cognitive models developed with an attack/mitigate/counter-attack scenarios and enhance them by leveraging learning mechanisms.
By integrating these elements in a coherent multi-purpose system, we aim at unraveling the complex structures that mediate interactions among defenders, adversaries and the public: in this respect, the overall goal is to enhance situational awareness in cyber warfare by assessing human performance in a simulated environment. The system is also meant to interact autonomously in a hybrid team, i.e. playing the role of a “teammate” sentinel in support of humans, eventually capable of prompting decisions and perform actions in more mature stages of development.
To provide a richer characterization of our approach, Section B illustrates the functional requirements of the envisioned system, while Section C and D will narrow the focus to, respectively, ACT-R cognitive architecture (the central component of the system) and the ontologies needed to frame the knowledge component of the architecture.
Modeling decision-making in the cyber security framework requires multiple factors to be investigated: (i) the size and the variety of knowledge which is necessary to classify and analyze attacks and defensive actions; (ii) the flexible behavior required by coupling alternative strategies of response to specific cyber threats, updating and revising strategies when the circumstances of the attack or the environmental conditions evolve; (iii) learning by experience how to deal with cyber attacks; (iv) interacting in a team by building a mental representation of the co-workers as well as of the enemies. These factors can be mapped to the 12 criteria distilled in [11] (from the original list compiled by Newell in [12]) that a cognitive architecture would have to satisfy in order to achieve human-level functionality. In these regards, cognition is not considered as a “tool” for optimal problem solving but, rather, as a set of limited information processing capacities (so-called ‘bounded rationality’ [13])3. In a similar fashion, Wooldridge had identified the requirements that an agent should satisfy in order to act on a rational basis [14], namely: reactivity, the capacity of properly reacting to perceptual stimuli; proactivity, the capacity of operating to pursue a goal; autonomy, implying
3 Despite the relevance of emotions in decision-making [
State-of-the-art research on cognitive architectures (SOAR,
ACT-R, CLARION, OpenCog, LIDA, etc.) has produced a
significant amount of results on specifying this extensive range
of functions4: by and large, ACT-R has accounted for the
broadest range of cognitive activities at a high level of fidelity,
reproducing aspects of human data such as learning, errors,
latencies, eye movements and patterns of brain activity [
In our context, the computational system resulting from the combination of cognitive and knowledge functionalities aims at fostering a better understanding of cyber attacks, supporting human operators in cyber warfare, eventually cooperating with 4 See [33] for a comprehensive overview of the most recent advancements in the area of cognitive architectures research. 5 Note that the distinction between ‘model’ and ‘agent’ when dealing with cognitive architectures is a blurred one. For clarity’s sake we will henceforth use ‘agent’ to avoid ambiguities with the notion of semantic model (ontology). In general, an agent is a cognitive model that dynamically interacts with the environment. them in well-defined synthetic environments. The rest of the paper presents in more detail the basic components of such a hybrid framework.
C. Replicating cognitive mechanisms with ACT-R
Cognitive architectures attempt to capture at the
computational level the invariant mechanisms of human
cognition, including those underlying the functions of control,
learning, memory, adaptivity, perception, decision-making, and
action. ACT-R [
On the basis of the first term, the more recently and frequently a chunk i has been retrieved, the higher the activation and the chances of being retrieved (tj is the time elapsed since the jth reference to chunk i and d represents the memory decay rate). In the second term of the equation, the contextual activation of a chunk i is set by the attentional weight Wk, given the element k and the strength of association Ski between k and the i. The third term states that, under partial matching, ACT-R can retrieve the chunk that matches the retrieval constraints to the greatest degree, combining the similarity Simli between l and i (a negative score that is assigned to discriminate the ‘distance’ between two terms) with the scaling mismatch penalty MP. The final factor of the equation adds a random component to the retrieval process by including Gaussian noise to make retrieval probabilistic.
The intertwined connection between declarative and
procedural knowledge, weighted by stochastic computations,
represents the necessary substrate for realizing at the
computational level the functionalities outlined in section B:
more specifically, we claim that ACT-R can successfully be
used to emulate human behavior in selecting and executing
defense strategies, matching input data from on-going cyber
attacks to deeply structured background knowledge of cyber
operations. In the past, ACT-R architecture has been
successfully used in context where integrating declarative and
procedural knowledge was also a fundamental issue, e.g. air
traffic control simulations [
The development of cyber security ontologies is a critical step in the transformation of cyber security from an art to a science. In 2010, the DOD sponsored a study to examine the theory and practice of cyber security, and evaluate whether there are underlying fundamental principles that would make it possible to adopt a more scientific approach. The study team concluded that:
The most important attributes would be the construction of
a common language and a set of basic concepts about which
the security community can develop a shared understanding.
A common language and agreed-upon experimental protocols
will facilitate the testing of hypotheses and validation of
concepts [
The need for controlled vocabularies, taxonomies, and
ontologies to make progress toward a science of cyber security
is recognized in [
The decentralization of knowledge organization and
maintenance to a variety of interconnected ontology modules
leverages a shared bridging component, i.e. BFO reference
ontology7: in this sense, BFO plays the role of the common
semantic infrastructure to define, populate and update multiple
context-driven cyber ontologies. The various modules will be
encoded in W3C language OWL8: the process of porting them
into ACT-R is managed automatically at the architecture level
by built-in LISP functions, which are able to a. read and
interpret the XML-based syntax of the semantic model and b.
convert it into ACT-R declarative format. A set of broad
schemas drives this conversion process: for instance, the direct
mapping between the “chunk-type” primitives in ACT-R and
classes in the ontologies has been designed. Further schemas
at a narrower level of granularity will be provided, as
engineered for an analogous framework presented at STIDS
2012 [
COGNITIVE SIMULATIONS OF CYBER OPERATIONS A. Experimental Design
The first objective of building an intelligent system endowed with adequate representation of cyber security knowledge is to use it in scalable synthetic environments for training human decision-makers. In addition, once the system has incorporated the necessary rational capabilities (defined in the previous section) and learned the dynamics of team interaction, we aim at testing the possibility of deploying it as an autonomous defensive agent in virtual cyber operations. In order to achieve the necessary degree of robustness and dependability, we plan simulations at different levels of complexity, as follows:
BSE — Basic Synthetic Environment: two ACT-R agents face each other playing the role of assailant and defender; HSE — Hybrid Synthetic Environment: an ACT-R agent and a human face each other playing the role of assailant and defender; HSGE — Hybrid Synthetic Group Environment: two teams, each constituted by humans and ACT-R agents face each other playing the role of assailant and defender.
In order to run these incremental simulations, we will initially collect an experimental dataset of cyber attacks, to be split into train and test set. In particular, we will focus on spear phishing attacks, as delineated in section II. The datasets will be organized to instantiate classes and properties of the defined modular ontologies. Each level of the cognitive-based simulation will be conceived as a block composed of multiple 6 For instance, exploiting material from this portal: http://militaryontology.com/cyber-security-ontology.html 7 http://ontology.buffalo.edu/bfo/ 8 http://www.w3.org/TR/owl-features/ trials9. At the BSE level, the simulation aims at assessing the soundness of the cognitive mechanisms executed by the agent, serving also as a system debugging and evaluation of experimental settings. In the HSE, the agent will have to compete against humans, whose potentially erratic behavior will be exploited by the agent as a primary source of acquisition of cyber warfare strategies and mental representation of the opponent. Finally, in HSGE the scenario will get more complex by shifting to a multi-agent framework, where each defending agent will have to learn intra-group cooperation and build mental representation of the opponent as a group (whose members act complementarily and collectively to harm the defending team).
In the delineated experimental phase we plan to expand our
previous work on applying cognitive architectures to
decisionmaking in non-zero sum games [
B. Evaluation plans
As recent studies have shown [
So far we have discussed the general requirements and described the high-level cognitive structures of an intelligent system for decision support in cyber warfare. However, a product or a solution based on these requirements and architecture will need to address specific problems in the business domain. Furthermore, the end product would likely 9 Setting to 100 the number of trials should guarantee a satisfactory level of stochasticity in the results. require integration with other technical components and frameworks. We see an opportunity to apply the concepts described in this paper for the development of an application capable of assessing and reducing information systems vulnerabilities though live, virtual, and constructive (LVC) simulations. Such an application can support a wide range of cyber defense objectives, including: (i) analysis of cyber defense strategies and identification of network; vulnerabilities through simulated attacker-defender interaction in BSE – HSE – HSGE scenarios; (ii) training for cyber security personnel with a suitable ACT-R agent simulating the attacker against human players; (iii) validation and enhancement of the cognitive models developed with an attack counter-attack scenario. To support LVC simulations, the application will need to work with existing distributed modeling and simulation infrastructures, such as the High Level Architecture (HLA)10 or Testing and Training Enabling Architecture (TENA)11. The key integration activities include: • • •
Identification and creation of reusable ‘objects’. A distributed modeling and simulation framework such as TENA encourages objects representing things such as targets and assets to be reused across simulations12. In particular, within the intelligent decision support system, we see opportunities at two levels: 1) creation of reusable objects representing attackers and defenders (these objects can be used to simulate behaviors of the actors); 2) creation of reusable objects representing IT Infrastructure components that could be under cyber attacks (these objects model the commands and instructions that can be sent to various components and their responses).
Integration of reusable objects in to the middleware layer of the modeling and simulation framework. Figure 2 shows a reusable TENA object (representing cyber attackers) plugged into the middleware layer. Implementation of runtime knowledge sharing in the modeling and simulation framework. In the example shown in figure 2, the ACT-R cognitive model (representing the defender) is integrated with knowledge sources incrementally stored in ACT-R declarative memory module: a) modular cyber security ontologies, retrieved from the TENA Repository and; b) the modular ontologies of the scenario [1] , incrementally stored in TENA Event Data Management. 10 http://standards.ieee.org/findstds/standard/1516-2010.html 11 Test and Training Enabling Architecture (TENA): https://www.tenasda.org/display/intro/Home 12 TENA object-oriented modeling features well fit our ontology-driven cognitive system.
CONCLUSION The novelty of our approach relies on grounding a decision support system in a broad spectrum of human-level cognitive functionalities blended with highly structured knowledge resources. In particular, by focusing on learning mechanism, context-driven semantic specifications and scalable simulations, the obtained computational system can serve both as a training environment for cyber personnel and as autonomous team member operating in advanced security settings. Our position paper aims at fostering the discussion within the communities of interest and can play the role of a starting platform for a scientific project proposal.
[11] John R. Anderson and Christian Lebiere, "The Newell Test for a theory of cognition," Behavioral and Brain Sciences, vol. 26, no. 5, pp. 587-637, 2003. [12] Allen Newell, Unified Theories of Cognition. Cambridge ,
Massachusetts: Harvard University Press, 1990. [13] H. Simon, "Bounded Rationality and Organizational Learning,"
Organization Science, vol. 2, no. 1, pp. 125-134. [14]
M. Wooldridge, Reasoning about Rational Agents. Cambridge, MA, United States of America: The MIT Press, 2000. [15] R. Sun, Cognition and Multi-agent Interaction, R. Sun, Ed.:
Cambridge University Press, 2006. [16] Paul Bello, "Cognitive Foundations for a Computational Theory of Mindreading," Advances in Cognitive Systems, vol. 1, pp. 59-72, 2012. [17] A. Oltramari and C. Lebiere, "Knowledge in Action: Integrating Cognitive Architectures and Ontologies," in New Trends of Research in Ontologies and Lexical Resources, Alessandro, Vossen, Piek Oltramari, Lu Qin, and Ed. Hovy, Eds.: Springer, pp. 135-154. [18] J. Ball, S. Rodgers, and K. Gluck, "Integrating ACT-R and Cyc in a large-scale model of language comprehension for use in intelligent agents," in Papers from the AAAI Workshop, Menlo Park, CA, pp. 19-25. [19] B. J. Best, N. Gerhart, and C. Lebiere, "Extracting the Ontological Structure of OpenCyc for Reuse and Portability of Cognitive Models. ," in Proceedings of the 17th Conference on Behavioral Representation in Modeling and Simulation, 2010. [20] S. Douglas, J. Ball, and S. Rodgers, "Large declarative memories in ACT-R," in Proceedings of the 9th International Conference of Cognitive Modeling, Manchester, UK. [21] H. Stuckenschmidt, C. Parent, and S. Spaccapietra, "Modular Ontologies - Concepts, Theories and Techniques for Knowledge Modularization," , 2009. [22] J. R. Anderson, How Can the Human Mind Occur in the Physical
Universe? New York: Oxford University Press. [23] A. Stocco, C. Lebiere, and J. R. Anderson, "Conditional Routing of Information to the Cortex: A Model of the Basal Ganglia's Role in Cognitive Coordination," Psychological Review, vol. 117, no. 2, pp.