Current trends in Artificial Intelligence are leading to the development of autonomous agents to perform critical operations in the real world. Events in real-world can endanger a wide range of discrepancies and the user should trust the agent to handle them. To achieve this the agent should be able to smartly adapt its behavior to handle the discrepancies and explain it to the human user. This thesis proposes a three-phase approach to address the above-mentioned problem. In the first phase, the agent uses case-based explanations and behavior adaptation in response to a discrepancy. This phase will not only help the agent build its knowledge about the discrepancy, but also forms a basis for its adapted behavior. In the second phase, the agent transforms the knowledge attained from the first phase to explain its behavior to the human operator. This knowledge includes both the causal understanding of the discrepancy and the reasoning behind its adapted behavior. In the final phase, the agent uses the feedback from the human counterpart to adapt its causal knowledge as well as its reasoning behind the behavior adaptation. Finally, this approach will be evaluated through the performance of the agent an underwater mine clearance domain, which is a surveillance mission to create a safe passage for ships.
Artificial Intelligence technologies made substantial progress in developing autonomous agents. Although, these agents are designed for very specific applications like driving vehicles or medical diagnosis, they are not completely trusted by their users. To bridge this gap of trust between a human user and the autonomous agent, a branch of AI called explainable Artificial Intelligence (XAI) has gained research traction. XAI focuses on explaining the behavior or decisions of the autonomous agent to the human user. Such an explainable system should develop a rich knowledge base over the time. I propose to acquire this knowledge when there is a discrepancy and transform it to explain to their human operators. Let us look at an example from an underwater mine clearance domain, If the agent finds a mine field at a location where it is not expected to be, then the agent retrieves the hypothetical causal knowledge that an enemy laid the mine. Such causal knowledge can help the agent take a smart decision to apprehend the enemy and resume its survey. Later after the mission the agent can provide the causal knowledge as a reason for its behavior to the human counterpart. Furthermore, the Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). feedback from the human counterpart helps the agent adapt its behavior as well as its causal knowledge. In this example a feedback can help the agent delegate the goal of apprehending the enemy to its counterparts and complete its survey on time.
In conclusion to the approach described earlier there are three phases involved in this process. In the first phase, when a discrepancy is identified the agent uses its causal knowledge to explain the discrepancy and adapt its behavior while in the second phase it uses the causal knowledge along with its behavior adaptation to explain it to the human operators. Finally, in the third phase it uses the feedback to adapt its causal knowledge, reasoning behind the adapted behavior or both.
Section 2 describes a representation of the explanatory cases, their retrieval, behavior adaptation and a possible approach towards explaining the agent’s behavior adaptation. Section 3 describes the underwater mine clearance domain and possible discrepancies that may occur in the domain. Related work is illustrated in Section 4 followed by Research plan in Section 5. 2
In this approach, we use case-based explanations [
A set of abstract XPs is retrieved when an unpredicted state or action observed by the agent unifies with each explains node of an XP in the case base. If the unification turns out to be successful then the pre-XP nodes of the corresponding case are unified with the observations of the corresponding states or actions, if they turn out to be successful then the specific XP is retrieved. The retrieved abstract XP is reused by binding variables in the antecedent to values found during unification of the consequent. However, if the XP-asserted nodes in the reused XP contain hypothetical information they can be revised when the new knowledge is obtained from further observations. Although, retention of the revised XP is helpful for improving the case-base it is not the scope of this paper.
In case of multiple XP’s leading to a discrepancy, weights can be associated to an XP. These weights can be based on the frequency of its retrieval in the domain or can be based on the number of evidences obtained. However, the method to calculate weights is not in the scope of this paper. 2.2
Behavior adaptation and Incorporating Human Feedback
Behavior adaptation is essential for an intelligent agent to respond to discrepancies [
The agent’s explanation to the human operator increases trust between them. As discussed in the previous sections, an XP is a data structure with the causal representation of antecedents leading to a consequent (discrepancy). A template created with a discrepancy, antecedents of the retrieved XP and the newly formulated goal will explain the agent’s adapted behavior to the human operator. Moreover, feedback from the human operator can assist the agent in giving weights to the explanations in the case base. This can be beneficial to the agent in retrieving the appropriate causal knowledge. 3
GA2 are the two octagonal areas where mines are expected to exist, while the triangular objects are the mines. The goals of the agent are to survey and clear mines in GA1 and GA2. These goals are given to the agent after a reconnaissance mission performed by a different agent across the whole sea route.
In the underwater mine clearance domain, several events often co-occur simultaneously, and many events cannot be predicted based on knowledge available to an agent. These events might affect the agent itself or the mission of the agent. Explanations help the agent to recognize these events and respond to them. We will look at several uncertain events that might happen.
Events in this domain include minelaying, sensor failure, and reconnaissance failure. Minelaying events occur when an enemy ship, aerial vehicle, or fishing vessel lays traps to hurt friendly ships. Sensor failure event indicates that the agent’s faulty sensor is responsible for a misclassification of mine, and the failure of proper reconnaissance mission indicates that an agent prior to the agent did not identify mines which in turn failed its mission. 4
Generating causal knowledge to explain a discrepancy is not novel in this approach.
However, reasoning about the causal knowledge to adapt agent’s behavior is novel.
Schank [
Roth-Berghofer et al’s [
Floyd et al. [
Plan Discrepancy Explanation and behavior adaptation Selecting an explanation case from multiple cases Explanation to external agents Behavior adaptation from human feedback Case adaptation from human feedback
Evaluation in Underwater Mine Clearance domain