Composing Tactical Agents through Contextual Storyboards Avelino J. Gonzalez(1), Rainer Knauf(2) and Klaus P. Jantke(3) (1)Intelligent Systems Laboratory (2)Faculty of Artificial Intelligence (3) Fraunhofer IDMT School of EECS Technical University of Ilmenau Children’s Media Dept. University of Central Florida PF 10 05 65 Hirschlachufer 7 Orlando, FL 32816-2362 98684 Ilmenau 99084 Erfurt USA Germany Germany ABSTRACT new training exercises could be easily custom-made for each group of trainees, but that they nevertheless This paper presents the novel use of storyboards for would guarantee an equivalent learning experience composing, organizing and visualizing tactical agents for all trainees. designed to serve as computer generated forces. This leads us to the concept of assisted scenario These tactical agents represent enemy forces that act generation for training simulations. While the and react to trainee actions and are specifically used selection and implementation of certain here to populate military training scenarios. The environmental effects such as weather, time and other tactical agents are based on the Context-based such issues is relatively easy, depending on the Reasoning human behavior representation paradigm. facilities provided by the simulation infrastructure, This application of storyboards facilitates the use and others such as the behavior and plans of the external visualization of the contextual elements that make up entities typically require much greater care. This is the composed agents. The use of the approach is because these intelligent tactical agents could exhibit described and an informal qualitative evaluation is the wide range of behaviors typically used in these conducted. scenarios, thereby resulting in large and complex models. Their large size and high complexity make 1. INTRODUCTION these agents difficult to build and possibly computationally expensive to run. Preparing a simulation for a military training session However, this is not the entire problem. The can be a time-consuming process. First of all, training external entities are the primary means through which objectives must be expressed by the instructor. the scenario designer causes the desired situations to Secondly, a mission or task to be executed by the be presented to trainees at the right moment. These trainee(s) must be specified, and the accompanying agents have to be able to react to the trainee actions environmental conditions must be defined and and still be able to present the desired educational subsequently reflected in the simulation environment. situation. In situations where the roles of the external Thirdly, if the training objectives call for the entity are quick and of a short duration, it may not trainee(s) to be faced with a specific situation, the need to be artificially intelligent. An example of this external entities with which the trainees interact must could be a distracted pedestrian crossing the street in be designed such that they present that situation to the front of the car. In such cases, the model of the trainee correctly and at the appropriate time. When pedestrian is simple, as it needs no reaction. Selection this requires the involvement of intelligent software and placement of such an external entity would be agents, these must be integrated into the simulation in rather simple. However, for other roles that require just the right manner to accomplish the desired extended contact with the trainee such simplicity may objective. Planning and organizing the simulation- not suffice. Examples of this include a driver with based training exercise to systematically include these road rage, a persistent enemy combatant, or a police three steps presents a significant problem for officer pursuing a fleeing driver. A more complex simulation-based training. process must be developed to assist the training In recent times, the widespread reuse of standard, session author in building the appropriate external reusable scenarios has led to exercises becoming entities and place them correctly within the known in advance by the trainees, thereby negating simulation. the effect of built-in surprises and diminishing the A tool that helps the session author design the effectiveness of the training session. This ultimately training session – specially the agents used in the prematurely requires that new and expensive training session would be immeasurably helpful. exercises be created. It would be ideal, therefore, if Description of such a tool is our objective here. 2. OVERALL SOLUTION APPROACH simulation-based training sessions. Therefore, we propose here to use this storyboard approach to Planning has been a core part of AI research since the represent the agent being composed for a session in a beginning. Planning is something that humans do training simulation. naturally and for the most part, effectively. Many The agents themselves are defined in the Context- tools have been built to assist planners. We based Reasoning (CxBR) modeling paradigm. CxBR investigated the feasibility of using storyboards, as specifies that agents built through CxBR be composed defined by Jantke and Knauf [3], to serve as the of several major contexts, some accompanying minor infrastructure upon which the agent models could be contexts and definition of transition criteria between planned and stored. the major contexts. While it is active, a major context, The concept of storyboards has been used together with possibly several minor contexts, successfully for many years in many applications controls the actions of the agent. When the situation such as cinematography, theater, musicals and such changes so that the context has changed, a transition time-based works. Storyboarding is a modern to a new active context is effected, with its attendant approach to planning that actually goes beyond functions and knowledge taking over the control of conventional planning. It can be said to be the “… the agent. Transition criteria determine when the organization of experience” [3]. Jantke [4] asserts that situation calls for a new major context to be made when human activity comes into play (e.g., games, active and the currently active major context to be de- war) predicting the future situations becomes difficult activated. Only one major context can be active at any because it is unknown what situation will be faced by one time. We expect here that the major contexts will the human in a conflict-based context. He maintains be defined and created a-priori and be available in that storyboards provide room for such human some repository, providing a baseline behavior for the activity by furnishing means to represent alternative agent when it finds itself in the correct context. worlds. However, the transition criteria are very application- Knauf [6] and Knauf et al [7] more recently applied dependent, and must thus be specified carefully for the storyboard concept to course design. They are each application. See Gonzalez et al [1] for details specifically used to guide the didactic process in about CxBR. traditional learning environments and in e-learning. The storyboard approach devised by Jantke & Knauf is built upon standard concepts which enjoy (1) clarity by providing a high-level modeling approach, (2) simplicity, which enables everybody to easily become a storyboard author, and (3) visual appearance as graphs. While other means of structuring the contents of the agents exist, such as state diagrams, Petri nets, etc., none meet the above three requirements as easily as does the storyboard tool described here. Jantke and Knauf define their storyboard as a nested hierarchy of directed graphs with annotated nodes and annotated edges. Nodes can be either scenes or episodes where scenes denote leaves of the nesting hierarchy and represent a non-decomposable learning activity. A scene can be (1) the presentation of a (media) document, (2) the opening of any other software tool that supports learning (e.g., an URL and/or an e-learning system) or (3) an informal description of the activity. Episodes, on the other hand, denote a sub-graph. Graphs are interpreted by the paths through which they can be traversed. Edges denote transitions between nodes. Figure 1 shows a Figure 1 – Application of Story Boarding to top-level storyboard that reflects an organization for Course Definition teaching a college-level course in Artificial We should note that the storyboard is not the agent. It Intelligence. merely helps a human to compose the agents for a The processes that are commonly represented specific scenario in a way that is clear, simple and through storyboarding are characterized by non- easily visualized. The CxBR-based agents contain determinism, involvement of human players and the the intelligence and the ability to react to events in the attempt to anticipate the behavior of these human simulation exercise. players. These characteristics also apply to The objective of the research was not to develop destroyers. Besides anti-aircraft missiles, they are a working model of the tactical agents themselves, but armed with an assortment of guns. RF1 is a mother rather to organize their definition in an easily- ship carrying three landing crafts that can be deployed visualized and manoeuvrable tool. This is what we from her hull. Each landing craft can carry a platoon- describe as composing agents from existing size unit with a light armoured vehicle or jeep with components, in our case, major and minor contexts. machine guns mounted on them. These landing craft Our software tool provides a medium for the scenario are also armed each with one 12mm machine gun. storyboard to be reflected, provides an infrastructure RF1 will seek to get close enough to the island on to store the agent models for all situations, and can its north side so that it can launch the landing craft assist the session author with customizing the and land their forces. They are not aware of the Blue transition criteria for the major contexts vis-à-vis the state convoy task force, the cargo vessel or its training session. The storyboard, however, is not an contents. The initial conditions of the developing agent representation paradigm. CxBR is the agent situation are described in Figure 2 below. Each task representation paradigm used. The storyboard merely force is not initially aware of the other. When the helps in composing the agents from previously Red task force enters the Blue state’s territorial defined major contexts and easily visualizing the waters, it is detected by an unarmed aerial resulting agent. To better describe the concept, we surveillance aircraft (not shown), that monitors the introduce an example military scenario. waters surrounding the island, and continues to monitor the movements of the Red force. Without air 3. SPECIFIC SCENARIO USED or satellite assets, the Red force later discovers the presence of the Blue task force only when the latter The training scenario used for this experiment gets within range of their ship-based radar. No other involves a fictional maritime country (Blue state) with aircraft are relevant in this scenario. a lightly defended base in an island far off its RF2 RF3 mainland coast. This island is the subject of a territorial dispute between the Blue state and a RF1 neighbouring and also fictitious Red state. In light of current situations that may lead to potential hostilities Territorial water boundary with the Red state, the Blue state seeks to reinforce the defences on the island by sending a cargo vessel with supplies and armaments needed to enhance the defences of its island base. This cargo vessel (M1) is escorted by a small task Island force composed of one anti-aircraft destroyer and flagship of the task force. This vessel is armed with BF1 SAM launchers, one torpedo tube and assorted guns. base This is the vessel to be directly controlled by the BF3 TF1 M1 trainees in this training exercise and it is labelled TF1. BF2 Three other warships make up this task force. Two anti-submarine frigates respectively labelled BF1 and BF2 come armed with anti-submarine rockets and Figure 2 – Initial Conditions of Scenario assorted guns. The fourth warship is a mine layer, In the initial scenario, the Blue force is in a major armed with mines and a 12.7 mm machine gun. It is context that calls for it to escort the cargo vessel. labelled BF3. Their mission is to escort and protect This means that the Blue task force is to sail at full the unarmed cargo vessel (M1) containing critical speed toward its destination, maintaining close supplies and weapons from the mainland port to the scrutiny of their sensors for the presence of threats, as naval base in the island in question. Their orders are the possibility of a Red force attack on the island has to protect the cargo vessel and to confront any force been considered a distinct possibility in the recent threatening it, whether air, surface or subsurface. The past. This major context in control is labelled Escort Blue state ships are at the command of the TF1 and it enforces a diamond shaped formation designed commander, who can order them to take any action in to protect the cargo ship from all directions. This accordance with the imposed rules of engagement. major context looks for the possibility of transitioning Unbeknown to the trainee Blue force, a Red state to several other contexts, such as Confront, Engage, force intends to land a heavily armed contingent in Attack, Retreat and Dock, among others. the island and capture it without a fight, given the light defences of the island base, and its long distance The Red force, on the other hand, has as its objective to the mainland. The invading Red force consists of to land undetected on the island’s north shore which three vessels, and they are labelled RF1, RF2 and has good beaches for that purpose, deploy its forces RF3. RF2 and RF3 two are AEGIS-type anti-aircraft and march overland to the base in the south end of the island and take it through sheer intimidation, scenes. Episodes are depicted by rectangles with preferably without firing any shots. Its initial major small notches at the left and right sides. As the name context, while in international waters, is simply to suggests, scenes contain more temporally short navigate to certain coordinates. This major context is actions. Scenes are depicted by simple rectangles. called Transit, and involves no special care other They intuitively equate to major contexts and minor than to maintain navigational awareness and avoid contexts respectively. collision with other objects as well as each other. Upon reaching the target coordinates, it is to transition to a more guarded form of navigation, where they get into a formation that is protective of the mother ship, and proceed in total radio silence, while at the same time in general quarters. This is the StealthTransit major context. Planning in CxBR is carried out rather informally. Unlike other AI planning languages and systems, such planning is reflected merely by a sequence of major contexts with defined transition criteria. These plans are easily visualized via the storyboarding tool described here. The major contexts that compose the agent being built can also be easily described likewise, as can the minor contexts. For example, the plan to be initially followed by the Red force agents as a unit, in terms of a sequence of major contexts is shown below and pictorially in Figure 3. Red Force: Transit Î StealthTransit Î Disembark Î Retreat Î Transit It is somewhat more complicated for the Blue force. Upon detecting the Red force, the task force splits up and different tasks are assigned by the trainee force flagship (TF1). Thus, the ships do not behave uniformly as a unit as do the Red force ships. In other words, each member of the task force has different tasks to execute. So, we describe each ship individually below: Blue Force TF1: Escort Î Confront Î Pursuit Î Transit Blue Force BF1: Escort Î Confront Î Pursuit Î Transit Figure 3 – Red Force Mission Plan Blue Force BF2: Escort Î StandBy Î Confront Î Pursuit Î Transit The storyboard tool is based on Microsoft Visio, with Blue Force BF3: Escort Î MineFieldApp Î some custom-made functions and shapes to allow the StandBy Î MineRetrieval Î Rescue free and easy movement among sheets. The main ÎTransit progression of the storyboard is reflected in the Blue Force M1: Transit Î Dock Mission sheet. This sheet is the plan for the agents that will participate in the scenario. In terms of CxBR, A full description of the scenario and the composition these represent the progression of major contexts to of the agents involved therein would exceed the page be executed by the agent being composed. These limits of this paper. The reader is referred to [2] for major contexts are represented as episodes in the the full details of the scenario and its implementation. mission sheet. The all-important transition criteria that triggers transitions between major contexts is 4. MODEL ASSEMBLY WITH TOOL found on the mission sheet, placed between the major context episodes. The storyboard tool presents the availability to create Figure 3 depicts the Mission sheet for the Red sheets, where each of these sheets contains some logic Force in this scenario. The comments shown between related to the progression of the story. The sheets can each major context represents a textual description of contain episodes, scenes or to-do boxes. An episode the transition criteria. In the case where the rule contains a longer lasting series of actions or sub- language syntax for the system being used is known, actions. It can be composed of other episodes or of this comment could include the actual code for the shown for a particular reason. One of the advantages transition rule. of CxBR is the potential for reusability of lower-level Episodes and scenes have the ability to switch to contexts by several major contexts. One of those other sheets that may contain an expansion of the predictably re-used is the Navigate sub-context. It is elements found in the episode or scene. This provides called by the Transit MC and the Retreat MC. the ability to quickly inspect a sub-context and its Conceivably, it is such an important function that it contents. should be called by all major contexts. Once the The storyboard begins with an initial condition and control passes to the Navigate sub-context, a return ends with a final condition shape. These shapes are should be executed to the major context that called it. scenes. Clicking twice on the initial conditions scene The ability to remember which major context called it will take one to the initial condition sheet, which is not intrinsic in Visio, so several return worm holes contains the same graph shown above as Figure 2. must be created, one for returning to each of the This is shown in Figure 4 below. The Initial various major contexts that may call it. While this Condition Sheet also refers to a document which puts the burden of remembering on the user, it describes the initial conditions in a narrative text. nevertheless works well. This document gives the scenario developer Lastly, an important part of a CxBR is the reactive background information on the scenario to be created. context set. These major contexts are not included in Note in Figure 3 the text between the Initial the mission plan because their use is not expected in Conditions Scene and the Transit major context the plan. However, the behaviors represented within episode in the mission sheet. This represents the these reactive contexts could be useful if the mission transition to the major context. In this case, the does not go strictly according to plan (as they rarely transition is a simple one – commencement of the ever do). Note that reactive major contexts are simulation, at t = 0.0. structurally similarly to those in the mission plan. It could be that a major context could be reactive in one mission but part of the plan in another. It just depends on the needs of the mission. R RF1 RF Transit F2 3 Island Escort BF1 base TF1 M BF3 1 BF2 Figure 4 – Initial Conditions page The funnel-looking pentagon shapes are return “worm holes”, so to speak. They represent a way to quickly return the user to the page from which the sub-sheet was called. For example, when double-clicking on the Transit MC episode on the mission page, this Figure 5 – Transit Major Context Page takes one to the page where the details of the Transit The reactive major contexts are contained in a major context are described. To return from there separate sheet called, appropriately enough, “Reactive back to the mission page, the funnel shape is clicked Major Contexts”. This sheet includes an episode for and the return is executed. Figure 5 shows the Transit each major context whose activation could be major context details. The two worm holes below the possible in the course of this mission but not sub-contexts depict the return pipe from the explicitly planned. These episodes have a link to its respective sub-contexts Navigate and respective major context description page. These AvoidCollision. The worm hole below the entire include links to the sub-contexts they call, just as was graph is the return pipe to the Mission sheet. done for those major contexts included in the mission A sub-context sheet is shown in Figure 6. This one plan. in particular is that Navigate sub-context. This one is training scenarios. The research consisted of defining a training scenario that would be typical of a military mission to teach trainees about tactics and doctrinal courses of action. Then, that scenario would be implemented in to the storyboard tool. The objective of the implementation was to gauge its applicability to simulation-based training. The results indicate that, after an informal evaluation, it does indeed satisfy the hypothesis that it would be a highly useful tool for this type of applications. While some improvements can be made to the tool vis-à-vis this type of application, it is useful as is, with only minor Figure 6 – Navigate Sub-Context Sheet with modifications made as part of this research. multiple Returns. 7. REFERENCES 5. EVALUATION AND RESULTS [1] A.J. Gonzalez, B.S. Stensrud and G. Barrett, “Formalizing Context-Based Reasoning - A Modeling The tool was used to build the scenario for the Paradigm for Representing Tactical Human intruder interception mission described above. No Behavior”, International Journal of Intelligent quantitative evaluation was done, as it is not a Systems, Vo. 23, No. 7, pp. 822-847, July 2008 performance-oriented tool. Rather, a qualitative and rather informal evaluation was deemed to be the [2] A.J Gonzalez, “Composing Tactical Agents sensible alternative. This was judged by how long it through Contextual Storyboards“, Final Report, July took to learn to use the tool. 16, 2009. Unpublished, but available upon request. As part of this research, the first author used the [3] K.P. Jantke and R. Knauf, “Didactic Design tool for the first time after only having attended a few though Storyboarding: Standard Concepts for paper presentations by the second author, totalling Standard Tools”, Proc. 4th Int’l Symp. on Information approximately two hours of lecture. These and Communication Technologies (ISICT) Workshop presentations were in the context of the latter’s on Dissemination of e-Learning Technologies and research in didactic design, and not in building Appl., Cape Town, South Africa, pp. 20-25, Jan. tactical agents for a simulation. Learning the use of 2005 the tool took approximately another two hours of working with it. This was done without [4] K.P. Jantke, “Why Storyboarding? Why not documentation of the tool, other than reading the Planning?” Computer Methods & Syst., Krakow, afore-mentioned papers. [3, 4, 5, 6, 7 and 8] Poland, Nov. 2009. However, it only took the first author a total of [5] K.P. Jantke, R Knauf and, A.J. Gonzalez, approximately 12 working hours to develop and “Storyboarding for Playful Learning”, Proc. of World organize the storyboard once he learned how to use Conf. on E-Learning in Corporate, Government, the tool. This informal and qualitative evaluation Healthcare, and Higher Education 2006 (E-Learn shows that it is indeed an extraordinarily intuitive tool 2006), Honolulu, Hawaii. to learn to use, even without formal documentation. The advantages of this tool go beyond the [6] R. Knauf, “Storyboarding - An AI Technology to organization of the agent components. It is quite Represent, Process, Evaluate, and Refine Didactic feasible to have the sheets included in the tool contain Knowledge”, Proc. of the Knowledge Media the actual source code for each component, such as Technologies. First Core-to-Core Workshop, the major contexts, the minor contexts and all Dagstuhl Castle, Germany, pp. 170-179, 2006. functions that are to be included with the CGF model [7] R. Knauf, Y. Sakurai and S. Tsuruta, “Toward for the mission in question. The ability to attach files, Making Didactics a Subject of Knowledge although not extensively used in this particular work, Engineering”, Proc. of the 7th IEEE International can serve to attach source code files to each major Conf. on Advanced Learning Technologies, Niigata context and sub-context. (Japan), pp. 788-792, 2007. 6. SUMMARY [8] S. Dohi, Y. Sakurai, S. Tsuruta and R. Knauf, “Managing Academic Education through Dynamic The research preformed here hypothesized that an Storyboarding”, Reeves, T.C. & Yamashita, S.F. existing storyboard tool, used previously for (Eds.) Proc. of the World Conf. on e-Learning in academic coursework organization and development, Corporate, Government, Healthcare, & Higher could be used to also define, organize and visualize Education 2006, October 13-17, 2006. military missions for the purposes of preparing