Context modelling is a subject that has attracted much attention in recent years. Currently most of the effort is directed in developing rather specific models for various domain applications. In this paper we take a more generic view of the problem and even though we do not present a concrete model we provide a framework as a starting point for building context models for a variety of domain applications. We discuss in some detail core issues that arise in context modelling.
Context modelling has attracted much attention in recent years mainly due to the need
to provide context aware applications in the future Ambient Intelligence
environments [
However much is still desired. For example the current approaches ignore the context in the domain of personal cognitive processes and how this influences the creation and association of context to a perceived situation. In this paper we provide an analysis of some of the core issues in context modelling so as to highlight the conceptual difficulties and provide a framework for further investigations. The goals of the paper can be summarised as follows: i. Discuss and define the general Context problem, ii. Provide a framework suitable for developing concrete models in AmI applications. In section 2 we present some examples that will help us to ground and develop the discussion in the next sections. In section 3 we discuss in some detail the issues of context definition and representation. We conclude, in section 4, with a discussion about additional issues that highlight further the points raised in section 3.
Our approach develops from and serves primarily two points of view: i. Building intelligent agents for an AmI environment so as to service user needs, and ii. Building autonomous cognitive agents (e.g. for controlling robots). The former case assumes the existence of a user, which will provide the system with clues as to the correctness of assignment of a context to the interpretation of the current situation (context here typically is in a symbolic form); the latter case lacks this evaluative feedback. The agent must develop the context concept, its contents, and mapping to a given situation by itself. In both cases, it is assumed that a ‘context variable’ would be created explicitly so as to tag agent models of interest. In both cases the application is of an open type. This contrasts with approaches such as in Knowledge-based Systems (KBS) where typically the system designer collects the new knowledge, adapts the existing one, reorganises the resulting system and defines the context structure. In effect such an approach is a closed one. We are more interested for problems that the system knowledge and the corresponding context are evolved by the means of the system alone without any external intervention.
In this section we provide some concrete application examples which will facilitate the discussion on sections 3 & 4.
This is a security type application, where a robot is patrolling an outdoors environment around a building of interest with the purpose of detecting unusual situations and consequently investigating them further. Typically the robot during the morning hours observes a number of people entering and exiting the building and cars approaching, parking in front of the building and leaving. During the evening hours, perimeter lights are switched on until the next morning. Every Friday, the perimeter lights are also switched on for 20 mins, around noon time (but not on the exact same time every Friday) for weekly testing of the equipment. Our robotic agent should develop a model for the perimeter lights’ operation that is influenced by the operation context (i.e. time of day and goals of building operators). It should flag a situation as unusual if lights are found operating during the daytime hours (except in the case of Friday noon) or if lights are switched off during night hours. The same requirement exists for people or cars during the night hours where the building is assumed empty. The agent does not possess any knowledge initially about the lighting or visiting patterns, but the extraction of the regularities and context should be achieved over time by the agent through observation and reasoning.
Assume that in the near future our cars will be equipped with intelligent agents suitable for assisting us with the driving task. An agent’s sensory data will include the information from the traffic control centre, the position from the car GPS system, the car’s state variables and some of the driver’s bio-data variables (e.g. skin temperature, blood pressure, EEG, etc). One typical output of the agent will be the calculation of the best path for travelling between two locations. The path planning is influenced by a number of factors such as the time of day, the overall traffic patterns in the city (emergent behaviour), weather conditions, constraints (such as variable speed limits in various sections of the path), conventions (such as driving slowly in a snowy day), user expediency, etc. All of the aforementioned variables constitute the context for the determination of the optimal path. The agent must learn to produce ‘good’ suggestions by using all of the aforementioned context components.
In this example assume that we have an agent that answers calls in the Customer Services centre of a company and handles routine complaints or forwards complex enquiries or frustrated costumers to a human operator. The agent makes a decision on whether to forward a case to a human operator or not based on linguistic and emotional context. The emotional context of speech indicates if the customer is frustrated, annoyed, anxious or otherwise. The linguistic context relates both to the syntactic forms used (e.g. non-standard) and to the semantic level (where the agent understands that the user sets a highly complex enquiry which cannot be serviced by the agent).
The notion of context is very complex to define precisely. It can be regarded as all the information in the universe that is complementary to the concept of a system under study. We should make an important note at this point: Usually we talk about the system and its operating environment. According to the above approach the environment could be thought, in the general case, as the complement of the system. Thus it coincides with our notion of context. However, in usual practice the notion of the environment is more restricted than that of the complement. In such a case, the system and its environment are included both in the context (i.e. the rest of the universe). From another point of view, the environment is thought as the set of external variables that have a “strong” influence in the operation of the system, while context is a set of variables that have a “weak” influence respectively. Here the environment concept includes the physical (spatiotemporal) continuum, objects in it and other agents present. It does not include the resulting emergent behaviour. The “system” concept in turn can be thought as a collection of internal variables that have a strong “influence” in the workings of the system, i.e. both in the externally perceived behaviours and the internal states. Thus we assume that the system is the collection of the internal variables and their emergent joint dynamics.
One can see the previous context idea implying a set of secondary variables in the description of our models. This view is common in statistical modelling and we include these influences as a source of error in a generic error term. However, in a level closer to the user’s notion of context, it also implies a set of social conventions and expected behaviours. This fact introduces further constraints and thus narrows the concept of the context. We will take this last view as our starting point for an operational definition. In practice context is also understood as an influence on the perception, decision-making (reasoning) and action generation capabilities of our agents. This is an essential property. This secondary set of variables always influences a model’s operation. However some of these variables may be observable, through sensors, while others may not. In the latter case the contextual set is simply what we can measure or deduce through calculation or inference based on existing knowledge. In more concrete terms the context is a sub-set of the universe’s state (at a given time) that provide a basis for local inferences. Inferences can be reasoning processes, causality models or otherwise. Local inference means the ability to make predictions about the state of affairs in the locality of some spatial location, time, or other variable. It should be contrasted to the global case, where typically our predictions produce a large error (including but not limited to the breakdown of causality relations). The only case where global predictions can be successful is in the case of constant evolution (trivial) or stationary global states (however the universe is an everchanging dynamical system). This locality also implies a near invariance (in time) in comparison with the system evolution. Thus it is possible to formulate reasonable stable inferences (and thus relations) that provide the conceptual knowledge of the system.
To summarise the above definition: Context would mean a set of influencing variables that may describe some metric space, such as location, time, etc, or may describe a choice of convention, expectations or internal cognitive processes. Note here that the contextual variables act as meta-variables (control variables). They may be even explicitly coupled with other (strong) predictive variables in a model, but in such a case it is expected that their time evolution (if any) take place in a much slower timescale than the model predictive variables. In all cases, they tag the model in question with an appropriate context label. This suggests a way in which we can think of representing context. We propose according to the above to use the following Cartesian decomposition for a Context variable (map more precisely, see comments in (2)):
Context = Spatiotemporal x Social x Cognitive x Context (1a) Spatiotemporal = Location x Time x Objects x Spatiotemporal Processes Social = Self Model x Agent Models x Emergent Situation x Constraints x
Conventions x Expectations Self Model = Goals x Beliefs x Value Systems x Emotions x Action Repertory x Knowledge Cognitive = Meaning Systems x Concept Systems x Inference Processes x
Attention Movement Definition (1) should be considered as a starting point for an operational context definition. It shows clearly the difficulty we face in defining the concept clearly. In our approach the context concept is decomposed in three main factors: 1. Spatiotemporal, including the usual notions of location and time as well the presence of other inanimate objects. Spatiotemporal processes that do not include purposeful agents as their constituents, e.g. weather, are also included here. See (1b). 2. Social, which cover the domain of multi-agent interactions. In the limiting case it covers a single agent. The constituents here are the participating agents, constraints, conventions and expectations present in the interaction relation and the overall emergent situation (which is not usually sensed in the level of an agent). See (1c). An agent is modelled, in a simple approach, as in (1d). 3. Cognitive, which includes all the internal processes that provide the agent with its Self-model and other agents’ models. Here even consciousness can also be included in principle, but due to the lack of fundamental understanding about this aspect of self, we have chosen not to do so. See (1e). 4. Context, which implies a recursive construction of the concept, moving from the coarse to fine detail and vice versa. See discussion in section 3.3.
We discuss in section 3.3 the various components in more detail. Let us observe here that definition (1) implies a concept that is subjective rather than objective. This is due to the dependence on the agent’s self-model and to the overall multi-agent emergent situation, including the agent’s view on expectations and agreed conventions. This should be contrasted with the usual operational definition of using the Location and the Time sub-sets only. Clearly the motivation is to capture not only static information but also dynamic activity due to agent activity, habits and social interactions. One agent’s context may be different from another’s given the same location, time and overall situation. What differentiate them are their preferences and expectations, i.e. the subjective utility functions of each for the given situation. For example travelling with the underground in a rush hour may be considered tolerable for one agent while for someone else is very undesirable. Note here that the above and following comments apply equally well to autonomous agents, humans as well as agents acting as proxies and servants of their human owners. In the last case, the Self-model will include not only the agent’s internal structures of (1d) but also the corresponding elements of his user. How the latter will be acquired with reasonable accuracy will not be discussed here further.
Let us discuss in more detail the various components of the context concept.
Location. Location is an obvious variable that influences the context, as it acts as a predictive variable for the set of behaviours, actions, constraints, conventions and expectations that we might associate with a given physical location. These implicit aspects are captured by other variables in definition (1). Location could be independently a predicative variable in cases where physical location informs us about available facilities, infrastructure and functionality (purpose) of a place.
Time. Time is another obvious variable that context depends upon as all agent activity takes place in time. More specifically time of day has a lot of predicative value for capturing changes in behaviours, expectations, etc.
Objects. This variable captures information about available objects, in the current location and time, and the set of possible actions that can be performed on the objects and the corresponding expectations as to their properties. The objects are considered as non-purposeful agents.
Spatiotemporal Processes. This component captures the physical processes that they are not human-originated or human-controlled and that shape the operating environment of the agent. This in turn affects the utility of a sought state (goal), the selected strategies to accomplish this goal and similar considerations. For example, seasonal changes in the weather patterns should be taken into account in most cases when planning to undertake outdoors activities. In the cases where there are processes that either originated or simply controlled by humans these can enter in the context representation under the social component as they implicitly influence multi-agent interactions.
The social component captures the multi-agent interactions as well as modelling the scope of a single agent. We will first discuss the agent interactions and then the simple representation of an agent.
Emergent Situation. In simple terms this is a set of variables that describes the overall state of a multi-agent system. It is not easy to arrive in some ‘good’ representation of such a situation. The scope of such a representation includes the description through either some system macro-variables or through ‘state’ micro-variables (i.e. of an individual agent ‘state’). In both cases what we need is to be able to differentiate among different steady states (equilibriums) of the system. Thus such a state can be described either as macro or micro depending on the ease in which we can build classifiers of states. The motivation behind this inclusion is that given all other factors unchanged, we can still conceptually distinguish among different emergent states; this affects our plan generation and ability to arrive to our current goals. Some emergent states may be desirable while others should be avoided at all costs. Typically recognising an undesirable situation would lead to the alteration of our current plan and pursuit of a different one. We should note here that the emergent situation is the result of all the agent actions, reactions, competitions, co-operations (joint plans) and evolutions. In other words it is the integrated effect of agent behaviours and individual agent evolutions through time.
A further clarification must be made at this point. Is this emergent situation different from what we observe through our perceptual input? I.e. don’t we get this information from our current sensors and agent interactions? The answer is no! The emergent situation that we describe here is usually not sensed directly from individual agents for two reasons. First, each agent only perceives partially its environment and other agents. Second, the concept of an emergent situation assumes a different level of observation in which it becomes apparent. This is a consequence of agents having both a limited observational range (in time and space) and of using an appropriate representation so as to make apparent the dynamics and equilibriums of the interactions. For example we mention the information provided by traffic control centres, through radio, to motorists so as to optimise the overall traffic flows in a city. Such higher-level information is only gathered and interpreted by a higher-level entity, the traffic centre, while each agent alone (a motorist) would have only a partial view and a ‘good’ hypothesis of what really takes place in a given time.
Constraints. As we have already discussed in the emergent case we have a set of behaviours that formulate the overall observed situation. The behaviours that the agents exhibit typically have some constraints for many reasons. They exist operational, constitutional or informal constraints on the set of ‘acceptable’ behaviours that the agents can exhibit. If a change in constraints takes place this could signal the appearance of new set of behaviours and thus the existence of new emergent states. It is then clear that when such a structural change, in the rules of the social interaction, takes place the context could be regarded as changed.
Conventions. Conventions are a special set of constraints, usually informal but
equally influential, which describe a ‘desirable’ behavioural pattern on the part of the
agents when explicit constraints would not regulate the issue of proper joint
action/interaction. As an example we mention the habit to try to be reasonably polite
when meeting people for first time. Conventions can also be seen as social norms.
Conventions can be reached and enforced by designing appropriate micro-rules for
the social game. See for example [
Expectations. Almost every individual behaviour, location, emergent situation and goal carries a set of expectations of: x What will be accomplished as a result? x What is the set of available services in a location and their constraints and conventions? x What is assumed as public knowledge regarding other agents’ beliefs, actions, etc? x That the utility function for an agent (or user) will not change (during some observation period). If it does then the following are affected: o Reward for accomplishing a given goal, o Utility from the functionality of a location, o Evaluating and readjusting the usefulness of an action, o Re-learning and using the user’s value system, x What are the rational expectations that we have of other agents in playing the social game? Thus the expectations concept carries a number of contributions arising through different considerations. These expectations are assumed constant in the scope of using a model, and thus their change indicates in the general sense a change of context because the model that the agent uses for state evaluation, reasoning or action generation is valid only for the given values of the above factors. Depending on the agent architecture one or more of these factors can be explicitly included in the decisionmaking / state evaluation models, but possibly not all. Also we note that the expectations list could include further contributions but we have stated what we considered the most important as an indication of the concept’s scope.
Self/Agent Model. This is a component that captures basic properties of complex agents. It is by no means complete, but provides a starting point to represent minimally an agent in a conceptual level. The sub-components that are included are the ones that we consider more influential. They are: Goals. Goals in (1d) represent preferred states that the agent tries to accomplish in any given time. This case includes also the user-level goals that a proxy agent tries to achieve. The existence of goals in (1) should not come as a surprise, as human beings or other purposeful agents do not operate in a goal vacuum. In contrast all the time some explicit or implicit goal of higher or lower priority always exists. Goals are the generators of meaning as the actions that we take in order to satisfy them introduce the semantics of objects, concepts and situations.
Beliefs. Beliefs represent hypotheses about the state of affairs in the world and other agents. They arise from past or present situations. These are formed by internal inferencing processes and external evidence. In the case that external evidence changes, then the current beliefs must change, so indicating a change in the interpretation of a given situation. All other things being equal, changing interpretation implies a change of semantics, which in turn may influence the assigned value of a goal, state, action, etc.
Value Systems. These systems encode the preference of an agent for various goals, states (internal or external), actions, objects’ usefulness, etc. They support the creation of meaning and encode the semantics of agent-environment interaction. Value Systems have two primary sources of information. The direct source is the agent’s experience with its environment typically through a reinforcement learning way. In a more sophisticated level, this will be represented as induced emotional reactions. The indirect way is through the belief system that formulates projections of values of entities (concepts, actions, objects, etc). Beliefs achieve this through forward (predicting) models. These hypotheses of the belief system, as to the potential utility of entities, are encoded as appropriate values. Change of these values introduces new semantics, which influence the perceived context. Note also that there is a feedback loop relation between value systems and semantics.
Emotions. Complex agents are assumed to be equipped with an emotional system. In the simplest case, this includes a ‘drive system’, which colours the world that the agent is embedded in. More sophisticated systems include a hierarchy of processing achieving more complex affective states. In any case, the emotional system encodes the world, internal states, agent actions, etc, through the values system. Change of emotions, will imply a change in the value of some entities and thus will influence the perceived context.
Actions Repertory. This includes the set of actions that an agent can perform on the given context. New actions are acquired through experience, while old ones are improved or completely substituted. If for various reasons, previous actions cannot be performed due to some exceptional reasons (or otherwise) or new ones become available this implies a context change in general. Please note that the actions concept is not limited only to physical actions, but to any generic external behaviour. Also note that constraints and conventions may restrict the set of permissible actions in a given context. Actions include in the general case two components: i. A solution strategy (in the conceptual level), ii. Its manifestation as an external behaviour. Knowledge. While all the above sub-components of the Agent / Self- model certainly constitute agent knowledge, this sub-component includes the rest of the knowledge, mainly in a conceptual level, such as concepts of objects, actions, relations, etc. In other words it represents the knowledge of the agent for itself, the other agents, its user (if this is this is the case), etc. Typically knowledge change continually inside an agent and this provides for an internal (non-observable from other agents) context. However, we consider here the more restricted case, where an agent develops new concepts either through novel experiences, communicated information from other agents or abstraction. The newly developed concepts will then be diffused in new belief, reasoning and action generation modes and finally will be manifested as external behaviour.
This component provides a vehicle to represent internal agent context (which is not
externally observable) but still influences the overall agent operation and thus
(implicitly) the social context. We propose a very minimal decomposition.
Meaning Systems. These systems provide the ability to the agent to discover patterns
in perceptual input and extract regularities in the world. The extracted patterns are
represented as concepts of various types: Objects (more concrete), actions
(spatiotemporal patterns), relations (more abstract concepts), etc. In addition, taking
additional input from the emotion system they associate an emotional reaction with the
related concepts. These systems work in conjunction with the attention system to
allow the learning of new concepts. The available concepts are then combined in
more complex concepts (beliefs) as to the most probable hypotheses encoding
regularities of the world. These systems operate in general not only to the domain of
physical sensations, but also to conceptual input, such as the one provided by
linguistic context. An agent may have a hierarchy of meaning systems, see [
Inference Processes. While meaning systems provide a point for forming interpretations of perceptual input up to the level of semantics, inference processes provide a way to formulate hypotheses as to the relations of concepts among themselves and thus establish beliefs that represent internally the external world. They also provide for solution strategies on current problems and thus facilitate action generation. Attention Movement. Attention is the highest-level controller in cognitive agents. The focus of attention provides a mechanism to serialize the access to limited computational resources for conceptual processing and thus define the internal agent context. In addition, attention is the generator of learning as it enhances the representations of the various concepts during acquisition.
One question that might exist in this sub-section of cognitive context is as to how the aforementioned components influence the overall context. All of them are assumed as fixed and what changes typically is the content of the experience. This is indeed true under normal operating conditions. One however can think also pathological cases, such as in exceptions that either will damage or in any case lead to a malfunction of the above systems. In such a case, clearly the internal processes that shape the content of the agent’s experience and its internal representation will affect this content and the agent’s knowledge. This should be signalled as an altered context. A typical example for human agents is the state of drunkenness that affects most of our high-level cognitive abilities or notorious neurological diseases. Except of the previous case, there is also the case that the design of these agent systems might include other higher-level designer parameters that effectively lead to different agent personalities if changed and thus alter their operation.
A context variable appears in the right hand side of (1a). This is not a cyclic reference but rather a recursive definition. We explain the case of the overall context variable below.
Context. Location primarily (and the other variables secondarily) allows the introduction of an important property in the overall Context variable: That of context nesting. We can imagine that a given location can be seen as an organised set of sub-contexts, where each one includes the spatiotemporal, social and cognitive components of the parent and introduces more and in finer detail. In this way we can think of a context space with points as trees; each level being more specific (in at least one variable) than previous ones regarding the variables in definition (1). An obvious example is an office building, where different expectations, rules and conventions apply in the parking zone, cafeteria, office, meeting room, etc.
The concept of nested contexts should not be mistaken as a cyclic reference as it just introduces the idea that the context is a recursive structure that can start from a fundamental level and build hierarchically more specific context variables inheriting values from the previous parent level. It should be also clear that two context variables could be treated as sets where all the normal set operations of union, intersection and complement (difference) are defined. To return to our previous point one can observe that definition (1) can be interpreted as a recursive definition defining a sequence of contexts, where each member can be written as:
Ci+1=f(STi+1,Soci+1,Cogi+1,Ci) f: Spatiotemporal x Social x Cognitive x Context -> Context where ST, Soc, Cog and C are the spatiotemporal, social, cognitive and context components respectively. f is an appropriate functional representation, such as a tree, an object in an object oriented language, a list, etc. Thus in essence we tag a given model inside the agent with an appropriately generated context variable. We write for a model M the following: (2a) (2b) (3)
M=ĭ (x,C) where ) represents the model’s map, x is an appropriate independent variables vector and C is the composite contextual variable described above in (2).
In the previous section we have developed our thesis for a generic framework that can be used as a starting point for developing more concrete context systems in various application domains. However, a number of other issues should be also discussed.
The development of the framework was based in the assumption that agents of considerable complexity will participate in the future AmI environments or future Cognitive Systems. Some assumptions were made about the internal structure of such agents, such as the existence of value, meaning, concept and other systems. This is also in agreement in treating human agents in such a fashion (even though very simplified).
The context definition of (1) has a number of nice properties: i. It can be used with the current state so as to provide an expected perceptual state in the next cycle of processing; ii. It allows sequential building of context and ignores the order in which particular features arrive to the perceptual stream; iii. There is no need to have an external supervisor to provide a context characterization as in other approaches.
We also made the assumption that the context is explicitly maintained inside the agent and acts as an additional variable in the various models present such as: meaning system, action generation, perception, etc. An alternative approach is to fuse the model and the context together. In such a case, the context variable cannot be used in an explicit manner. This leads as to the following issue.
So far we have not discussed the possible concrete representation of the various
components present. This could take the form of symbolic representations (where
typically a designer pre-defines elements of context or the complete contextual model for
the application) or the form of distributed representations. The latter are intensively
researched by cognitive neuroscience and there have been efforts to develop models
of distributed context representations in cognitive science and elsewhere (e.g. [
Also note that the constructed variable has the property that we can move from coarse to fine detail and vice versa by using forward or inverse recursion. This preserves invariant characteristics going from the finer to the coarser level and thus provides for abstraction of context.
There has been some noticeable work on context from the AI community in the
previous 15 years. For example [
In this paper we discussed a framework that can provide a basis for building concrete context models for AmI applications. One has to select appropriate representations for each of the components of definition (1). However we feel that the framework is lacking if we consider its application in the domain of autonomous agents. We envisage such systems as developing their own context model from scratch and by evolution. Our thesis is that a suitable context model and its generating mechanism must be applicable to any “space”; it should only set some requirements about the structure of the space. The space may refer to a sensory space or concept spaces of high abstraction. What is needed is a method that partitions dynamically the knowledge of the agent in context and model related sets. The knowledge can be represented by the contents of a set of variables that the agent has access to. These variables can either measured directly of inferred. They can refer to the environment, the agent or other agents. We call these variables, the variables set. According to the above they are partitioned to a model set and a context set. The set of goals provides the effective mechanism of partitioning the variables set. How such a mechanism might be concretely represented is an open question. We expect that its form will depend on the mode of representation used (sub-symbolic vs. symbolic) and the structure of the concept system that will be used in the agent to represent its knowledge.
We would like to acknowledge the support of the European Union through the IST GNOSYS project (FP6-003835) of the Cognitive Systems Initiative to our work.