Providing support to a person to manage the activities of daily life requires a framework to represent the social environment of that person. The framework introduced in this paper allows the modelling of the subjective experience of that person in specific situations, the social relationships of people playing a role in that situation, as well as general knowledge that can be used to derive additional information about specific situations.
Artificial agents that support people in various activities are becoming a reality, consider e.g., virtual coaches [Tielman et al., 2014], personal assistant agents [Myers and Yorke-Smith, 2007], and smart homes1. Simon [1972] proposes that human behaviour is shaped by the computational capabilities of the actor as well as by the structure of the task environment. This suggests that in order for artificial agents to be able to support people in their daily lives, information about their internal processes is not enough: it is also important to represent the environment. In simple words, help should be situational and therefore, we need to know what is going on around us. To provide assistance that fits the situation, an agent should be able to reason about the surrounding entities and how they relate to each other.
If we turn to sociology, the concept of definition of the situation is considered to be what people use in order to know what is expected of them in a given situation. It is a subjective understanding of the role and status of those involved in a situation. We learn how to define situations by combining our experiences with our knowledge of norms, customs, beliefs, and social expectations. The term first appeared in Park and Burgess [1921], who write: “...In fact, every single act, and eventually all moral life, is dependent upon the definition of the situation. A definition of the situation precedes and limits any possible action, and a redefinition of the situation changes the character of the action.”
Research in computer science has tackled this issue by using concepts such as situation awareness [see Endsley, 2000] and context awareness [see Akman and Surav, 1996]. Both 1http://sine.ni.com/cs/app/doc/p/id/cs-14844 terms have been used to describe attempts to enable artificial agents to better understand their surrounding environment. According to Barwise [1987] these concepts refer to the same thing, and situations represent a way of modelling contexts. Situation awareness is primarily used to model emergency situations such as rescue operations, therefore the main focus has been on modelling the physical environment. However, Kaminka [2013] argues that agent systems should incorporate general social intelligence building blocks. Dignum et al. [2014] also suggest that the next step in artificial intelligence is the ability to show social intelligent behaviour.
In this paper we focus on modelling situations for Socially
Adaptive Electronic Partners [Van Rie
In order to illustrate our approach we present an example scenario which will serve as the base for the framework in Section 4. To make the scenario appropriate for our goals of supporting people in their daily life, we include settings in which social relations influence the actions that are to be taken.
In the scenario our main character, Alice, and her two friends, Bob and Charlie, are going on a holiday trip. The first step is for them to decide when and where to go. After deciding, Alice notices that on the morning of departure she has an important meeting with her boss at work. During the holidays while on a picnic, Alice forgets to bring something to drink, so she asks Bob for a beer. Later that day when they go to a shop, Alice buys a chocolate, and then gives the chocolate to Bob. We want to model the situations in this scenario from the point of view of our main character. For this reason we need a knowledge structure based on a subjective representation of the world. The knowledge structure should explicitly represent different social relations between people, taking into account that people can have different relations in different situations. Furthermore, we need the possibility of combining the facts from the situation with more general rules in order to infer new information about the situation and possible actions. Lastly, it would be useful to have access to information from past situations during the reasoning process. 3
The approach introduced in this paper is based on existing concepts from situation awareness and sociology. In this section we present these concepts and discuss why they were chosen. Due to space restrictions, our description does not include technical details. For in-depth accounts, the reader is referred to Kokar et al. [2009] and Fiske [1992]. 3.1
Kokar et al. [2009] present a formalization of situations
based on the situation theory developed by Barwise and Perry
[1981] and extended by Devlin [1995]. This formalization is
compatible with the interpretation of situation awareness in
terms of human awareness provided by Endsley [2000]. They
formalize these concepts by using the Web Ontology
Language [
The main elements of situation theory are objects and types. Some of the basic types are IND representing the type of individuals, RELn representing n-place relations, SIT representing the types of situations etc. The main type of situation is an utterance situation, representing an expression in natural language which is linked to a real situation. These utterances can be represented as information coming from a user. The relevant part of the world mentioned in the utterance is called focal situation. Utterances might also refer to another situation, called resource situations, which is used as background information during the reasoning process.
The basic information about a situation is expressed by using infons, written as: << R, a1, ..., an, 0/1 >> where R is an arbitrary n-place relation, a1, ..., an are objects appropriate for R, and 0/1 is the polarity of the infon, showing whether the relation holds for those objects or not. For instance, the infon << workT ogether, Alice, Bob, 1 >> would express that the relation workT ogether(Alice, Bob) holds. Situations and infons are related by the support relationship ( ), which relates the situation with infons that hold under that specific situation. In this formalism, s φ means that the infon φ holds in situation s. Since situations are represented formally, it is possible to infer new facts by using OWL formalisms. Furthermore, it is possible to construct more general rules and then infer information by combining them with the facts present in the ontology.
Many of the concepts presented so far are similar to our requirements: situations are seen as a partial subjective description of the world from the point of view of a user, it is possible to refer to other situations, the framework is generic and expressive, and it is possible to add rules, which in our case can represent societal norms. Furthermore, the formalizations can be graphically represented, which gives the possibility of interactively building a model of the situation with the user. For other approaches used to model context, see Akman and Surav [1996]. However, at this point the STO framework does not support explicit relations about the social sphere. Kokar et al. [2009] acknowledge that their framework is a starting point, thus it is not complete and is open for extensions. We do so by using concepts from sociology presented below. 3.2
Fiske [1992] in his attempt towards a Unified Theory of Social Relations suggests that most kinds of social interaction are generated by using four elementary relational models. More complex social forms can then be expressed by combining these models. The relational models are not dependent on specific domains or situations, rather they can be used for any type of social relations. This feature makes the approach useful for our cause, since we aim at having a generic framework which can model any type of (social) situations.
Communal Sharing (CS) relationships represent a bounded group of people who are equivalent and undifferentiated. An example of this relation is an open buffet dinner: the guests can all take as much food as they want, so they are in some sense indistinguishable. However, these equivalence classes are not fixed: people in the same equivalence class are seen as equal only for that specific purpose. Formally, CS is an equivalence relationship in which reflexivity, symmetry, and transitivity hold.
Authority Ranking (AR) relationships represent a hierarchical ordering of people for a certain social dimension. People in higher ranks have more power, privileges, and typically also more responsibilities. An example of this relation can be found in a military setting. Formally, AR is a linear ordering, i.e., a reflexive, transitive and anti-symmetric relationship.
Equality Matching (EM) relationships represent a model of even balance, for example turn taking, tit-for-tat retaliation, or compensation by equal placement. People are mainly concerned whether there is balance in the relationships. An example can be parents taking turns to baby-sit their children. Formally, the EM relationship has the properties of a linear ordering and entails the idea of an additive identity. Furthermore, the relation obeys the associative and commutative laws. Finally, EM is order preserving.
Market Pricing (MP) relationships represent a model of proportionality in social relationships. All the features of the relation are considered under the point of view of a single utility metric that allows for comparison. Typical examples of MP relations are those which include a rational calculation of expected utility, where the usual metric of comparison is money. Formally, MP relationships have the properties of EM and some others such as the concept of multiplication as well as the fact that two entities of the same relational structure can be compared.
Dignum [2004] proposes three types of relationships which govern transactions in organizations: a hierarchical model, a market model [Williamson, 1975], and a network model [Powell, 2003]. These are similar to the AR, MP, and a combination of CS and EM relations, respectively. 4
In order to represent the example of Section 2, we split the scenario into different situation snippets: s1 Alice, Bob and Charlie have to pick a time and place for their holiday trip; s2 Alice notices she has a meeting with her boss on the departure day; s3 Alice asks Bob for a beer; s4 Alice buys a chocolate in the shop; s5 Alice gives the chocolate to Bob.
In order to model these situations, we assume that the agent has knolwedge about Alice’s social relations. How to build these knowledge structures with the users will be explored in future work. As aforementioned, we will extend the STO by adding social relations and inference rules. At the same time, we will explain how our situation snippets can be modelled using these rules and relations. Since we are in an ontology setting, the presented relations are instances of the RELn type, while all the individuals are instances of the IND type. We do not express the overall ontology due to space restrictions, however for each situation snippet we present the relevant individuals and relations, as well as the infons that are supported by the situation. Finally, we express what can be derived from that information in combination with the inference rules.
In CS relations, decisions are made by unanimous agreement by the members. This bit of knowledge is represented by the following inference rules: << member, X, G, 1 >> << member, Y, G, 1 >> << equal, X, Y, 0 >> << equal, choice(X), choice(Y ), 0 >> << disagree, G, 1 >> where G represent a group of people that X is a part of,
I f a n d a n d a n d Then s s s s s << member, X, G1, 1 >> << communalSharing, G1, 1 >> << member, X, G2, 1 >> << authorityRanking, G2, 0 >> << equal, choice(X), pref (X), 1 >> which shows that if a person is not a member of some AR group (i.e., there is no choice conflict), then the choice of the person can be the same as her preference.
[Behind the scenes of s1] In this situation the relevant individuals are Alice, Bob and Charlie, and the relevant relation is CS since we are talking about a group of friends going on holidays, so we can say that s1 s1 s1 s1 << member, Alice, f riendsGroup, 1 >> << member, Bob, f riendsGroup, 1 >> << member, Charlie, f riendsGroup, 1 >> << communalSharing, f riendsGroup, 1 >> Furthermore, at this point we do not have information about conflicting choice due to Alice being in an AR group. By using the inference rule, the system understands that in order for a decision to be made, first of all relevant individuals should be asked, and once there is no disagreement, a place (and time) can be picked.
In AR, priority goes to people holding higher positions. In this spirit, we can define the rules: I f a n d a n d Then I f s a n d s a n d s a n d s a n d s Then s a n d s s s s s << member, X, G, 1 >> << member, Y, G, 1 >> << higherAuthority, X, Y, G, 1 >> << outRanked, Y, G, 1 >> << authorityRanking, G, 1 >> << member, X, G, 1 >> << outRanked, X, G, 0 >> << member, Y, G, 1 >> << outRanked, Y, G, 1 >> << equal, select(G), choice(X), 1 >> << equal, choice(Y ), select(G), 1 >>
The last rule suggests that if a person is outranked in an AR relation, then its choice is the same as the choice of the top member of the group.
[Behind the scenes of s2] In this situation the relevant individuals are Alice and her boss. Bob and Charlie are not relevant for the situation at this point, since we are taking the point of view of Alice. The relevant situation is AR, and the situation supports the following infons: s2 s2 << member, Alice, workGr, 1 >> << member, boss, workGr, 1 >> s2
<< authorityRanking, workGr, 1 >> s2 << higherAuthority, boss, Alice, workGr, 1 >> s2 << outRanked, boss, workGr, 0 >> I f a n d a n d a n d Then I f a n d a n d Then s s s s s s s s s << communalSharing, G, 1 >> << disagree, G, 0 >> << member, X, G, 1 >> << equal, selection(G), choice(X), 1 >> this means that if there is no disagreement in the group, the choice of any member can be selected. so the systems concludes that Alice is outranked and therefore has to attend the meeting with her boss.
At this point, s1 will also change, since now Alice’s choice is different (and is not the same as her preference), therefore there is disagreement in the group. This would give rise to a new situation where the relevant individuals are Alice, Bob and Charlie, and s2 would be a resource situation, but we will not model it for space reasons.
In EM, the most important aspect is to keep some kind of balance in the relation. We can define the following rules: I f s a n d s a n d s Then s + 1 << equalityM atching, X, Y, 1 >> << owesF avor, X, Y, V, 1 >> << serves, X, Y, W, 1 >>
<< owesF avor, X, Y, val(V ) − val(W ), 1 >> Here s + 1 refers to the situation that follows from s by that actions taken in s, V and W represent the owed items, while the function val represents the value of the items. I f Then I f Then I f Then s s s s s s << owesF avor, X, Y, val(V ), 1 >> << owesF avor, Y, X, −val(V ), 1 >> << owesF avor, X, Y, 0, 1 >> << balanced, X, Y, 1 >> << balanced, X, Y, 1 >> << balanced, Y, X, 1 >> [Behind the scenes of s3] In this situation the relevant individuals are Alice and Bob, and the relevant relation is EM a. The following infons are supported: s3
<< equalityM atching, Alice, Bob, 1 >> s3
<< serves, Bob, Alice, beer, 1 >> so from now on Alice should keep in mind that she owes a favor to Bob until she has repaid him.
aThis situation can also be modelled as CS, if that is Alice’s subjective view on her relation with Bob. In that case no rule would trigger. We assume EM for illustration purposes. [Behind the scenes of s5] In this situation the relevant individuals are Alice and Bob, and s3 is a resource situation. Again, EM is the relevant relation, in which: s5 s5 << equalityM atching, Alice, Bob, 1 >> << serves, Alice, Bob, chocolate, 1 >> in s3 we inferred that Alice owes a favor to Bob. In Alice’s point of view a beer and a chocolate have a similar value, so the system can conclude that she does not owe something to Bob anymore: the balance has been reset.
For this to work in the formalism, we need to be able to refer that s5 is later than s3 and that if Alice did not serve Bob with something in the mean time, she still owes him the same as she did in the state following from s3. This inertia/persistence of (sub-)situations is known as the frame problem. In future work we will model this by formulating a meta or second-order property of predicates: persistent(P ) means that P remains true unless explicitly changed by the rules on the object level.
In an MP relation, money is the metric through which an exchange balance is kept, so we introduce the following rule: I f a n d Then s s s << marketP ricing, X, Y, 1 >> << serves, Y, X, V, 1 >> << shouldP ay, X, Y, val(V ), 1 >> [Behind the scenes of s4] In this situation the relevant individuals are Alice and the shopkeeper, and the relevant relation is MP. The following infons are supported: s4 s4 << marketP ricing, Alice, shopkeeper, 1 >> << serves, shopkeeper, Alice, chocolate, 1 >> so the system can infer that Alice should pay for the chocolate.
At this stage MP seems like a sub-case of EM, however the difference stands in the fact that EM usually implies a closer relation between the people, and the concept of paying back a favor is more subjective, while in MP the values are usually clear to all parties. 5
In order to support people in their daily activities, artificial agents should not only be able to reason about the user’s internal processes, but at the same time they should understand the user’s surrounding environment. In this paper we focus on the social environment, and we present a framework that can be used to model situations which include social elements. Furthermore, we illustrate how our framework functions through an example. As our starting point we use the Situation Theory Ontology developed by Kokar et al. [2009], and we extend it in order to enable it to represent social relations. We implement four basic social relations: communal sharing, authority ranking, equality matching and market pricing [Fiske, 1992]. The resulting framework allows us to represent the social aspects of situations, and is able to reason based on these situations. The inference rules are based on properties of the social relations. A key feature of the framework is that it represents situations from the point of view of the user that we want to support.
In future work, we will add more generic inference rules based on societal norms in order to enable the system to reason about arbitrary situations. Furthermore, the approach can be combined with activity recognition technology in order to enable encoding the infons automatically. This way the agent can use as an input explicit knowledge from the user as well as knowledge from the recognition module, and then in turn reason and act based on the preferences of the user. Moreover, we will include the temporal aspect of situations which is currently missing. Another goal is to unify our framework with a framework that accounts for the user’s behaviour, this way both the internal and external aspects are represented. Finally, we want to make it possible for users to build the representations of situations interactively with the system.
This work is part of the research programme CoreSAEP, with project number 639.022.416, which is financed by the Netherlands Organisation for Scientific Research (NWO). The authors would like to thank the anonymous reviewers for their valuable comments and suggestions.