- Explainability refers to the degree to which a software system's actions or solutions can be understood by humans. Giving humans the right amount of explanation at the right time is an important factor in maximizing the effective collaboration between an adaptive system and humans during interaction. However, explanations come with costs, such as the required time of explanation and humans' response time. Hence it is not always clear whether explanations will improve overall system utility and, if so, how the system should effectively provide explanation to humans, particularly given that different humans may benefit from different amounts and frequency of explanation. To provide a partial basis for making such decisions, this paper defines a formal framework that incorporates human personality traits as one of the important elements in guiding automated decisionmaking about the proper amount of explanation that should be given to the human to improve the overall system utility. Specifically, we use probabilistic model analysis to determine how to utilize explanations in an effective way. To illustrate our approach, Grid - a virtual human and system interaction game -- is developed to represent scenarios for human-systems collaboration and to demonstrate how a human's personality traits can be used as a factor to consider for systems in providing appropriate explanations.
As systems become more autonomous and intelligent
through the incorporation of AI techniques and self-adaptive
approaches, it becomes increasingly important for those
systems to be able to “explain” themselves to their human
users and collaborators [
While explanation is an increasingly desirable – even,
essential – capability of a system, it is not at all obvious when
and how explanation should be given, particularly since
explanation comes with a cost on human attention and delays
in system-human interaction and the fact that different humans
may need different kinds of explanation. To partially address
this problem this paper defines a formal framework, as
illustrated in Figure 1, for reasoning about the proper amount
of explanation that a system should provide to the human
based on their personality traits. Specifically, leveraging
research in the psychology of human personality, this
framework incorporates two basic personality traits
(Openness and Need for Cognition) as important elements in
a human model that can be used to guide a system in deciding
the appropriate amount of explanation that should be given to
the human in order to improve overall system utility. The
effects of given explanations (which are determined based on
personality traits of the human) affect human-system
coadaptation, represented through the
Opportunity-WillingnessCapability (OWC) model, a commonly used model for
adaptive systems’ reasoning about human-in-the-loop
behavior [
The organization of the paper is as follows: Section II describes the research problem and goals, Section III represents background information and related work, Section IV shows methodology, Section V shows the Stochastic Multi-player Games (SMG) model while Section VI shows results and analysis, Section VII represents discussion and future work and the last section focuses on the conclusion.
II.
PROBLEM STATEMENT AND
RESEARCH GOALS
A co-adaptation system is symbiotic human-in-the-loop
system where human-system cooperation is required in
achieving shared goals, and system and human actions
mutually impact each other’s behavior in accomplishing
coordinated tasks [
Given that different humans may benefit from different
amounts and frequency of explanation, in this paper we argue
that adapting the explanation to the particular human through
knowledge of their personality traits can help the system in
determining what are appropriate explanations and, therefore,
maximize the benefits of co-adaptation. In particular, given
that there are tradeoffs in determining what kind of
explanations to give, it is important to be able to tailor the
explanations to the user [
In this paper we attempt to answer these questions by defining a formal framework for reasoning about how a selfadaptive system should provide explanations based on its knowledge of a person’s personality traits. This framework uses probabilistic analysis to decide how explanations should be given, based on a formal human model that includes psychologically relevant aspects of personality. Specifically, we focus on answering the following research question: How to use knowledge about an individual’s personality traits to improve the overall system utility?
•
A formal framework that incorporates human personality traits and guides adaptive human-in-theloop systems to decide how much explanations should be given in order to improve system utility. •
An evaluation system based on a collaborative game, to simulate the effects of decision making under various scenarios.
III.
This section introduces some background on personality traits, the OWC (Opportunity-Willingness-Capability) model, model checking of stochastic multi-player games (SMG), and some of state-of-the-art studies that focus on explainability and human-system co-adaptation. Section IV will then illustrate how this background and related work are related to what we do in this research.
Psychological studies have demonstrated that human
personality traits play a strong role in determining human
behavior [
There are, of course, many differences between
individuals; however, personality traits are one of the more
important measurable characteristics that can be used to
distinguish one person from another. In the psychological
literature the Big Five (also called the Five Factor) model of
personality is one of the most widely accepted personality
taxonomies. In the Big Five model, the five dimensions of
personality include extraversion, neuroticism, openness to
experience, agreeableness, and conscientiousness [
Openness to experience is one of the personality traits that
is used to describe individual personality in the Five Factor
Model. Open people tend to be intellectually curious, creative
and imaginative. Open people have a high openness to
embrace new things, fresh ideas, and novel experiences [
In addition to the Five Factor Model, the psychological
literature also identifies Need for Cognition is an important
distinguishing characteristics of human personality trait
[
Need for Cognition (NFC) is defined as the "individual’s
tendency to engage in and enjoy effortful cognitive tasks.”
People with higher NFC levels typically prefer more detail,
while those with low levels of NFC want to quickly
understand the big picture and avoid engaging through more
detail. Based on the NFC 10-item testing instrument [
As we elaborate later, we adopt these two basic personality traits (Openness to Experience and Need for Cognition) as important elements in a human model that can be used to guide a system in deciding the proper amount of explanation that should be given to the human to improve overall system utility.
B. OWC (Opportunity-Willingness-Capability) Model
Prior research in adaptive systems has investigated various
models of humans that can be used at run time to effectively
characterize humans when deciding how best to incorporate
them into a co-adaptive system. One of the more prominent
models is the OWC (Opportunity-Willingness-Capability)
model [
OWC categorizes human attributes into: (1) Opportunity:
indicates whether a human is available to participate in a
cooperative task with the system (such as whether the human
is physically present). (2) Willingness: identifies the human’s
inclination to perform the task (affected by cognitive load,
human attention, stress level, and motivation). (3) Capability:
defines the human’s abilities and skills that are necessary to
execute the task successfully (affected by level of experience
or training, knowledge of the task, and cognitive or physical
skills) [
This model has been used effectively in a number of
papers to determine, for example, whether to involve the user
in a task or to carry it out automatically [
C. Model Checking Stochastic Multiplayer Games (SMG) and PRISM
Probabilistic model checking is used as a technique to
analyze the systems that exhibit stochastic behavior.
Stochastic Multi-player Games (SMG) is a form of
probabilistic modelling that allows us to reason quantitatively
about reward-based properties and probability such as time,
usage, and resources in a multi-agent system [
PRISM is “a probabilistic model checker, a tool for formal
modelling and analysis of systems that exhibit random or
probabilistic behavior” [
In this paper we use PRISM to dynamically determine appropriate levels of explanation to maximize expected utility (expressed as a reward).
D. Human-in-the-Loop Self-Adaptation and Explainability
Human-system integration or human-system
coadaptation is advancing the fields of human-system
interaction. Integration here means that the relationship
between system and humans has become a partnership or
symbiotic relationship in which humans (i.e., users) and
systems act with autonomy instead of the system waiting for
the user's inputs and commands to take an action.
Selfadaptation refers to a process in which an interactive system
co-adapts its behavior to a human based on its internal model
of the human, dynamic information acquired about the human,
the context of use and its surrounding environment [
Several related works have studied explainability focused
on a human-system co-adaptation perspective. In [
In [
In another related work [
While this prior work shares with our research the goal of reasoning about explanation in the context of human-system co-adaptation, and also use probabilistic reasoning to account for inherent uncertainties in our human models, none of these studies take into consideration specific personality traits of humans – the main focus of our work.
IV.
In this section, we illustrate how we use explanation as a
tactic (or action) that systems can use to improve the
efficiently and effectiveness of human-system co-adaptation
based on human personality traits. We describe also how we
utilize a probabilistic planner [
An important question is which personality traits to consider with respect to explanation? As noted earlier, the psychological literature has classified a variety of important distinguishing characteristics for human personality.
However, not all traits are relevant to explainability. In this
work we have adopted two personality traits: Need for
Cognition (NFC) and Openness to experience, since there is a
direct relationship between NFC and explainability and
between Openness and capability in OWC [
We use the “Openness to experience” trait as one factor
that affects the human’s capability to continue and complete a
task, since open people tend to be intellectually curious and
have a high level of capability to do creative tasks [
In our work we assume that the human’s personality traits
are known (for example, by using the NFC 10-item testing
instrument in [
B. Incorporating the OWC (Opportunity-Willingness
Capability) Model
We use the OWC model (described in Section III.B) to capture the co-adaptation attributes of the human. In this paper, the following indicators show the connection between our model and the OWC model and how the OWC is incorporated in the context of the collaborative Grid game: (1) time and location represent the set of variables of the Opportunity category. Is the player located at the correct location? Has the timer expired? (2) Human satisfaction represents the Willingness category. Is the human satisfied with the given explanation? That category is applied through the playerFeedback (pF) tactic. (3) Human performance represents the Capability category. The Capability category identifies the ability of the human to complete Grid task.
Giving an explanation increases the capability of the human
to successfully carry out that particular task [
C. Utlizing Model Checking Stochastic Multiplayer Games (SMG)
The probabilistic model checker (PRISM-games) is utilized to formally model our approach. PRISM-games is particularly suitable for our study because it helps us to reason quantitatively under unpredictability and uncertainty about “how much” explanations should be given. The uncertainty (or stochasticity) that is relevant in this context is about the proper amount of explanations and the impact of different amounts of explanations that should be given to the human.
We model the system (the Grid game described in Section IV.D below) as a turn-based SMG, which means exactly one player in each state of the modeled system can choose an action, where the outcome of that state will be probabilistic.
Players in a SMG may cooperate to achieve a common goal, or compete to accomplish different goals. In our examples, we model two players1, the human and the system, and we assume that they share a common goal.
We use rPATL, a probabilistic temporal logic, to express
properties of stochastic multi-player games quantitatively.
rPATL helps us to reason about the collective ability of a
group of players to achieve a goal relating to the probability
of an occurring event [
1 Note that a multiplayer here (i.e., two players) does not mean that the Grid game is a multiuser game. The concept “multiplayers” in PRISM refers to multiple agents, such as system, human, or environment. In our model, the system and the human are the only two players and they are working cooperatively (taking turns) to achieve the best possible outcome.
To illustrate our approach, we defined the Grid game– a virtual game -- as shown in Figure 2, as a game that embodies a representative scenario for human-system co-adaptation.
In the Grid game the system S instructs a player P verbally to move on a 5×5 grid from the top right corner (start) to the bottom left corner (end). The game is designed to rely on explanations, at various levels of detail, to instruct the user on what tasks to perform and how to perform them.
Follow the system instruction through a certain path within a certain maximum amount of time (60 seconds).
• • • • •
The player can move either horizontally or vertically. Game score (100 points): points are deducted for traversing extra blocks or moving into or through obstacle squares (e.g., in Figure 2 there are four obstacles: the house, a traffic light, a mountain, and a tree).
The Grid game can involve the use of five tactics for interacting with the player, as shown in Table 1. The system provides two levels of explanation to command the human to move from one point to another. The choice of level of explanation is based on the run-time calculation and explanation generation based on the probabilistic model.
In this case “less explanation (LExp)” provides an abbreviated command (e.g., “Go 2 blocks left”), while “more Commands the human to carry out an action.
The system further explains information when the human is confused and loses track.
The human requests the system to confirm information that they not entirely sure about.
Human feedback is collected about his satisfaction for Helpful, Not helpful, each given explanation instructions.
The human confirms information and follows the “Yeah”, “Thanks”
Example “Go 2 blocks left” “Move south 4 blocks” “You will go between a house and traffic light” “You go straight, and you see a car on your left side” “Should I continue above “North?” the tree?” Neutral “Okay” Acknowledgement lessExplain (lExp) moreExplain explanation (mExp)” provides an abbreviated command (e.g., “You will go between a house and traffic light”) contains additional details. The human may request clarification about a given explanation if they are not entirely sure about it (Chk) (e.g., “Should I continue above the tree?”). Or the user can confirm the information and follow the instructions (conf). The human also
gives feedback (pF) about the given explanation as to whether it was (a) helpful, (b) not helpful, or (c) neutral. (This supports explanation assessment in the framework - Figure 1).
The four utility attributes of the game are: RequiredTime (t),
(B),
LengthOfExplanations (xL), and ExplainEfficiency (xE). B and t are used for calculating the game score, and xL and xE are used as explainability attributes. Game score(s) depend on the time elapsed for completing the game (t), associated with the optimal number of the blocks (B) that the player is supposed to end the task with: • • • •
RequiredTime (t): the total elapsed time for completing the game.
Blocks (B): the number of the blocks traversed to complete the task.
LengthOfExplanations (xL): the amount of delay (or time) required to explain.
ExplainEfficiency (xE): a measurement that determines how happy the player is with the given explanations. xE is associated with the playerFeedback (pF) tactic which can be one of the following values:
2) Tactics Cost/Benefit and Utility Dimensions on utility dimensions. Different tactics cause an increase in Time (three seconds for lExp, Chk, and conf; six seconds, for mExp). The upward ↑ or downward arrow ↓ reflects utility increments and decrements, respectively. For example, the lExp tactic increases both t and xL by three seconds, which is associated with a smaller amount of costs. Human feedback is collected about the user’s satisfaction for the given explanation (lExp) which can be: a)
b) c)
Not helpful (NH) reflects utility decrements (↓), Neutral (N) reflects neither utility increments nor decrements (-).
explanation), the system we that
To compare different explainability tactics (i.e., lengths of rewards, rPATL, which enables us to analyze the utilities of use probabilistic temporal logic with explainability can influence.
rPATL
(described in Section IV.C) is used to reason about the ability
of a group of players (system and human) to collectively
achieve a specific goal [
In the formal model we define formulas that represent the accrued utility (The
∪ and the
∪
) as the maximum real immediate utility that the human can achieve along the whole task. 60), where B must be greater than or equal to
high scores to high utility derived by dividing the number of (tmax=
(the optimal number of blocks that the player is supposed to complete the task with): ∪ ( ) = (1 − ) 100 where ≥
() The ExplainEfficiency Function
∪ , as shown in function (2), maps higher levels of ExplainEfficiency (xE) derived by dividing the accumulated player Feedback (∑ )
Fig. 3 The strategy we use to model the SMG: the proper amount of explanation
For example, if a human has 75 Openness and 90 is determined based on the three personality levels of the human (represented in light
NFC. The combined human traits are 0.82 which 3b.luHeu).mTahnefteweodbeaxcpklainsactoiollnecatmedouthnatst w(leilslscoarnmbeorhee)lparfeuld, enteeurmtrainl,eodrbnyotuhsienlgpffuulnction
means he has high personality traits (by using function (represented in yellow). The human confirms information that means he moved
(3)). That means the system will explain less 18% of successfully to the next point (conf), or checks/requests the system to clarify
the time (i.e., lExp) and explain more 82% of the time information that they not entirely sure about (chk).
(i.e., mExp) during the task. As another example,
suppose a human with low personality traits has 43
openness and 49 NFC. The combined human traits are 0.46
(using function (3)). That means the system will explain less
54% of the time (i.e., lExp) and explain more 46% of the
time (i.e., mExp) while playing the Grid game.
system took 27 seconds for explanations (xL). At the end of
the task, the score of the player is 75 (by using the function
(1), where B=15 and tmax= 60), and the ExplainEfficiency
(xE) is 43 (by using the function (2), where ∑ is three and
is seven (which means seven feedbacks are
collected)).
by the total number of feedbacks ( ), where
[
()
Both personality trait variables (Openness and NFC) are initialized with some constants (as inputs) that represent the human traits. Personality traits are directly mapped to the level of explanation (the amount), and are used to calculate the probability of getting explanations in that amount. Function (3) shows combined personality traits, which will be 0 in case both traits are 0, or 1 in case the human has the highest personality traits levels (i.e., 0 → 1).
Opennessmax and NFCmax are 100, which represent the
highest personality trait levels. The values of
personality traits are determined based on the NFC
10item testing instrument in [
THE STOCHASTIC MULTI-PLAYER GAMES
(SMG) MODEL
We model the Stochastic Multi-player Games (SMG) model as two players, where the players try to collaboratively maximize accumulated reward(s): (1) Player SYS specifies the actions that are controlled by the system (i.e., it represents the Grid game). (2) Player HUMAN specifies the actions belonging to the human (i.e., it represents the game player).
The models represent the behavior of a set of agents (or
“players”) that take turns making moves, where the choice of
move is specified probabilistically or non-deterministically. A
game solver for such a system (such as PRISM-games [
The Stochastic Multi-player Games (SMG) model consists of the following four parts:
Scenario S: Can you go 2 blocks down? H: Yeah S: Then go 2 blocks left.
H: Could you repeat that? S: Go west. You will go between a house and traffic light.
H: Okay S: Go after that 2 blocks up.
H: The human is on the wrong track S: No, not south. You go north H: Okay ……… S: Go 2 blocks left S: Go south 4 blocks.
H: Okay, thanks a lot.
Tactics (lExp) (conf) (lExp) (Chk) (mExp) (conf) (lExp)
Figure 2 and Table III show an example dialogue of a scenario between the system (S) and a human (H).
The human spent 42 seconds (t) and used 15 blocks (B) to finish the task. However, the number of blocks B that the player is supposed to end the task with is 12 ( ). The () NH H
Player definition includes the declaration of the two players in the SMG and different modules that each player has control of. The two players in our game are shown in Listing 1. Player SYS (lines 1-2) specifies the actions that are controlled by the system (i.e., it represents the Grid game). Player HUMAN (lines 3-4) specifies the actions belonging to the human (i.e., it represents the game player). Our Grid game is played in turns by the two players SYS and HUMAN. Turn (line 5) is a global variable used as a controller to take turns between different players, ensuring that only one player can take an action at each state of the model execution. Tactics are executed sequentially in our model.
1. player SYS
Game, [lExpLow],[ lExpAvg],[ lExpHigh], [mExpLow],[mExpAvg],[mExpHigh] 2. endplayer 3. player HUMAN
Play, [conf], [Chk] 4. endplayer 5. global turn:[SYS..HUMAN] init SYS; 6. const SYS=1; const HUMAN=2;
Listing. 1 Player definition includes the declaration of the two players in the SMG and different modules that each player has control of
Player SYS has control of the Game model, illustrated in Listing 2. Opportunity elements are used as execution conditions of different tactics such as: the human is at the correct location ((x= 1)&(y=1)) and is not involved in a crash, and the time has not expired (t<60). The Game module is parameterized by the variables (lines 1-2), which indicate the state of tactic execution, where false means this tactic is not in use (i.e., lExp_state, and mExp_state).
During the system’s turn, the system executes these tactics sequentially: lExp (lines 5-13), and mExp (lines 1523). For the sake of clarity, we will describe only the lExpLow tactic to illustrate how tactic execution is modeled. The other explainability tactics follow the same structure. The system instructs the human with low personality traits through executing the command labeled as lExpLow (line 5). This tactic executes only if: • It is the turn of the SYS. • The human traits are low (<0.50). • The player position is on a certain block (x1,y1). • The end time of the task has not been reached yet (t<60).
If the guard is satisfied, the system will explain more by flagging mExp_state tactic true with probability human_traits (line 6). Otherwise, the system will explain less by flagging lExp_state tactic true with probability 1human_traits (line 7) and the system will: •
Commands the player to move to the position (x2,y2). • Increases the time 3 seconds (xL'=xL+3)&(t'=t+3). • Flags the lExp tactic as true (lExp_state'= true). •
Updates the value of the variable turn, changing control to the human player (turn’=HUMAN).
Similarly, the system instructs the human with average personality traits through executing the command labeled as lExpAvg (line 8-10), or the system instructs the human with high personality traits through executing the command labeled as lExpHigh (line 11-13).
The human wins by executing the command labeled as win (line 25). That means the human (turn=SYS) has arrived at the bottom left corner ((x1= 1)&(y1=1)) within the time limit (t<60). However, the human loses the game by executing the command labeled as lose (line 26) when the end time of the task has been reached (t=60). 1. global lExp_state: bool init false; 2. global mExp_state: bool init false; 3. … 4. module Game 5. [lExpLow] (turn=SYS)&(human_traits<.5)&(x1= 5) &(y1=5)&(t<60) 6. ->human_traits:(mExp_state'= true) 7. + 1-human_traits:(x2'=5) & (y2'=3)&(xL'=xL+3) & (t'=t+3)&( lExp_state'= true)&(turn'=HUMAN); 8. [lExpAvg] (turn=SYS)&(x1= 5)&(y1=5)&(t<60) 9. ->0.5:(x2'=5)&(y2'=3)&(xL'=xL+3)&(t'=t+3) &(lExp_state'= true)&(turn'=HUMAN) 10. +0.5:(mExp_state'= true); 11. [lExpHigh] (turn=SYS)&(human_traits>.8)
&(x1= 5)&(y1=5)&(t<60) 12. ->human_traits:(mExp_state'= true) 13. + 1-human_traits:(x2'=5) & (y2'=3)&(xL'=xL+3) & (t'=t+3)&( lExp_state'= true)&(turn'=HUMAN); 14. … 15. [mExpHigh] (turn=SYS)&(human_traits>.8)&
(conf_state= false)&(Chk_state= true)&(t<60) 16. ->human_traits:(mExp_state'=true)&(xL'=xL+6) &(t'=t+6)&(Chk_state'= false)&(turn'=HUMAN) 17. + 1-human_traits: (lExp_state'= true); 18. [mExpAvg] (turn=SYS)&(conf_state= false)
&(Chk_state= true)&(t<60) 19. ->0.5:(mExp_state'= true)&(xL'=xL+6)&(t'=t+6) &(Chk_state'= false)&(turn'=HUMAN) 20. +0.5:( lExp_state'= true); 21. [mExpLow] (turn=SYS)&(human_traits<.5)
&(conf_state= false)&(Chk_state= true)&(t<60) 22. ->human_traits:( lExp_state'= true) 23. + 1-human_traits:(mExp_state'=true)&(xL'=xL+6) &(t'=t+6)&(Chk_state'= false)&(turn'=HUMAN); 24. … 25. [win] (turn=SYS)&(x1= 1)&(y1=1)&(t<60)
-> (win'=true)&(turn'=0); 26. [lose] ((turn=SYS)|(turn=HUMAN))&(t=60)
-> (win'=false)&(loser'= true)&(turn'=0); 27. endmodule 28. …
Listing. 2 Game Model
Player HUMAN has control of the Play model, illustrated in Listing 3. The encodings of the HUMAN module are similar to those of the SYS module. The Play module is parameterized by variables (lines 1-2), which indicate the state of tactic execution, where false means this tactic is not in use (e.g., Chk_state, and conf_state). Personality Traits are initialized with values that represent the human’s personality (lines 3-5).
During the human’s turn, the human can execute one of these tactics: conf (line 8), and Chk (line 10). We explain only the conf tactic to illustrate how tactic execution is modeled. The human confirms (conf) and follows the system instructions (i.e., the human moves successfully from the 1st point to the second) by executing the command labeled as conf. This tactic executes only if: • It is the turn of the HUMAN. • •
The system instructs the player to move to the position (x2,y2).
The end time of the task has not been reached yet (t<60). 1. global Chk_state: bool init false; 2. global conf_state: bool init false; 3. const int INIT_OPN; const int INIT_NFC; 4. global human_Open: [1..100] init INIT_OPN; 5. global human_NFC:[1..100] init INIT_NFC; 6. … 7. module Play 8. [conf] (turn=HUMAN)&(x2= 5)&(y2=3)&(t<60) ->(x1'=5) & (y1'=3)& (t'=t+3)&(B'=B+2) &(pF'=pF+1)&(pfMAX'=pfMAX+1)&(conf_state'= true) &(lExp_state'= false)&(turn'=SYS); 9. … 10. [Chk] (turn=HUMAN)&(conf_state= false)&(x1= 5) & (y1=3)&(t<60) ->(Chk_state'= true)&(t'=t+3)&(pF'=pF1) &(pfMAX'=pfMAX+1)&(turn'=SYS); 11. [wrong] (turn=HUMAN)&(x2= 3)&(y2=5)&(loser=false) &(t<60)-> (x1'=3) & (y1'=1) &(t'=t+3) & (B'=B+2)&(pF'=pF-1)&(pfMAX'=pfMAX+1) &(conf_state'= false) &(turn'=SYS); 12. [crash] (turn=HUMAN)& ((x1=obj1x & y1=obj1y) | (x1=obj2x & y1=obj2y)| (x1=obj3x & y1=obj3y) | (x1=obj4x & y1=obj4y))->(turn'=SYS); 13. endmodule 14. …
Listing. 3 Play Model
•
• Increases the time three seconds (t'=t+3). • Increases the number of Blocks by two (B'=B+2). •
Gets the player feedback (pF= 1 means the explanation was helpful). • Increases the player feedback counter pfMAX by 1. • Flagging the conf tactic true (conf_state'= true).
Moreover, the tactic wrong (line 11) will be executed when the human moves in the wrong direction, and the tactic crash (line 12) will be executed when the human moves to one of the obstacle squares (the house, a traffic light, a mountain, or a tree in Figure 2).
Utility functions are described in Section IV.D and illustrated in Listing 4. Formulas and reward structures are used to encode the utility functions that allow us to quantify the utilities of different task states.
The Scores function, ∪ , as in lines (1-2), represents the encoded Function (1) as described in Section IV.D. ExplainEfficiency function ∪ , as in lines (3-4) , represents the encoded Function (2) described in Section IV.D. Line 5 shows the encoded combined traits function (3).
1. rewards "Scores" [win] true:(1-(B/tMax))*100; [lose] true:0; [crash] true: -5; 2. endrewards 3. rewards "ExplainEfficiency" [win] true:(pF/pfMAX)*100; [lose] true:(pF/pfMAX)*100; 4. endrewards 5. formula human_traits =
(human_Open+ human_NFC)/(Max_Open+Max_NFC);
Listing 4. Utility profile and reward structure: formulas and reward structures are used to encode the utility functions that allow us to quantify the utilities of different task states. Formulas calculate system utility of the different states.
In this section, we illustrate how our modeling framework
can produce optimal decisions with respect to how adaptive
systems should explain to the human based on their
personality traits. Specifically, we use SMG models of
explainability to determine the expected outcome utilities of
using different explainability tactics (i.e., explanation
amounts) based on the personality traits of the human. Our
modeling is done as a simulation (or set of “experiments” in
PRISM terms). We use rPATL to ask PRISM a variety of
questions such as “what is the maximum/minimum
probability a human with high/low personality traits can
guarantee to win with high/low utilities?” [
Table IV and Figure 4 show the analysis results of 44 rounds run on PRISM. All possible combinations of personality traits are taken into consideration, where high traits are (>80) (represented by orange color), average traits are (≥50 and ≤80) (represented by blue-gray color), and low traits are (<50) (represented by gray color). Plot (a) shows the 44 simulations of different personality traits and the given amounts of explanations (LengthOfExplanations (xL)) to complete the task. The average of different personality traits and the amounts of explanations (xL) is shown in Plot (b). 39% of the iterations (17 rounds) of humans with high personality traits (>80) needed more explanations to finish the task with an average of 21 seconds. 32% of the iterations (14 rounds) of humans with low personality traits (<50) needed less amount of explanations with an average of 20 seconds. The remaining 30% of the iterations (13 rounds) belongs to a human with average personality traits (≥50 and ≤80), where they use average amounts of explanation with an average of 19 seconds to complete the task. Table 5 shows the average of different utilities based on the three personality trait levels.
We can conclude from the results that a human with high
personality traits needs more detailed information (i.e.,
explanations), while a human with low personality traits needs
less detailed explanation. These conclusions are all consistent
with psychology studies (discussed in Section III. A) that
human with higher personality trait levels typically prefer
more explanations, while those with low levels of personality
trait want to quickly understand the big picture and avoid
engaging through more explanations [
In this research we presented an approach based on probabilistic model checking of SMGs to determine how much explanation should be given to the human based on their personality traits. Providing the right amount of explanation to the right human is an important factor to maximize coadaptation between the system and the human during their interaction.
There is a number of limitations of this research that future research can address based on the foundations that we have described in this paper. These explanation decisions are ideal scenarios without having actual proof of that in reality. To address this, the most important next step is to conduct an empirical study to validate these models on actual real-world systems with humans in the loop.
As we explained earlier, there are many reasons to use explainability and improving a system’s overall utility is one of the main reasons (see Section I). Explainability can help not only to improve the systems continuously through human involvement, but also to justify some information given to the human, particularly when decisions are made suddenly. Gaining more information improves the capability of the human to perform a task. Our results in this paper suggest one of the next steps of research is to go beyond the length of
Plot (a): The 44 simulations of different personality traits and the given amounts of explanations (LengthOfExplanations (xL)) to complete the task.
Plot (b): The average of different personality traits and the amounts of explanations (xL)
Fig. 4 Results of 44 rounds run on PRISM show that human with higher personality trait levels typically prefer more explanations, while those with low levels of personality trait prefer less explanations explanations, and examine in more detail questions such as how explanations should be presented: graphically, textually, verbally? A further extension of this research is to have more detailed models that allow the system to determine in a more nuanced way the ideal contents of the explanations that should be considered.
VIII. CONCLUSION
In this research we presented a formal framework that incorporates human personality traits as one of the important elements in guiding automated decision-making about the proper amount of explanation that should be given to the human to improve overall system utility. To accomplish our goal of this paper, we use probabilistic model analysis to determine how to utilize explanations in an effective way based on the difference of human’s personality traits. Grid – a virtual human and system interaction game – was developed to illustrate our approach, to represent scenarios for humansystem co-adaptation, and to demonstrate through simulation how a human’s personality traits can be used as a factor to consider for systems in providing appropriate explanations.
This research was supported in part by the NSA under Award No. H9823018D0008 and Award No. N00014172899 from the Office of Naval Research. Any views, opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSA or the Office of Naval Research.