experiment. As intelligent interactive technologies advance, ensuring alignment with user preferences is critical. Machine theory of mind enables systems to infer latent mental states from observed behaviors, similarly to humans. Currently, there is no formal mechanism for integrating multiple observations over time and quantifying the uncertainty of inferences as the function of accumulated evidence in a provably human-like way. This paper addresses the issue through Bayesian inference, proposing a model that maintains a posterior belief about mental states as a probability distribution, updated with observational data. The advantage of Bayesian statistics lies in the possibility of evaluating the certainty of these inferences. We validate the model's human-like mental inference capabilities through an tea. of this process.
Theory of mind, the innate human capacity to deduce others’
latent mental states from observable behavior [
Mental inference facilitates collaboration by informing
the agent and impacting its actions. The idea is that if an
intelligent machine has knowledge of the user’s goals, it
can better make decisions to help the user. However, there
is also an inherent risk in making decisions based on
inferences: because all inferences contain uncertainty [
The problem that our paper tackles is illustrated in Figure 1. The three panels depict an evolving inference by an observer of Janice’s drink preference under varying conditions in three consecutive days. Initially, Janice selects tea, but the positioning of cofee on a high shelf introduces ambiguity regarding her preference – does she favor tea, or does she simply wish to avoid climbing the kitchen ladder? This uncertainty prevents a clear inference of her preference. In the second panel, Janice uses a stool to reach the now higher-placed tea jar, while the cofee remains even further out of reach, potentially accessible with taller kitchen stairs. 11.6.2024 Vaasa, Finland ∗Corresponding author. †These authors contributed equally.
Theory of mind, or mentalizing, enables humans to infer
others’ mental states [
Models of mentalizing target the inference of mental
states such as preferences, costs [
How to create a psychologically plausible model that can
be parametrized with mental states and that then simulates
behavior? One emerging popular approach is called
computational rationality [
ceur-ws.org
Following the standard modeling pipeline in computational
rationality [
, consisting state space , action space , transition probabilities and reward function . A state ∈ encoding current information of the environment, transfers to next state ′ ∈ by performing an action ∈ according to transition probability (, , the reward = (, ) ′) = (
′|, ) , and gains . Reinforcement learning (RL) solves the optimization problem of how to choosing the action through policy () = (|)
that maximizes the expected
reward by interacting with the environment and learning
from experience. The learning process can be expressed as
the function
∗() = max[(, ) +
∑ (, ,
′∈
′
) ∗( ′)],
where ∗() is the value of a state ∈ under an optimal
policy ∗, discounting future rewards using ∈ [
The bounded optimal agent described via an MDP can be parametrized. For instance, a parameter can govern its preferences, that is, the state rewards. This permits mentalizing: given observed data, what parameters best produce predicted data that fits the observations? To this end, we utilize Bayesian inference, described by Bayes’ rule: (|) = (|) () () where represents the latent factors to be inferred, and represents observed data. The inference uses a prior () and a likelihood (|)
to calculate posterior probability
(|)
, normalized with marginal likelihood ()
. However,
the intractability of the likelihood (|)
prevents us from
deriving the posterior directly. This can be overcome with
approximation and likelihood free inference methods [
We recruited = 10 participants via the Prolific online platform. The number of participants was small, but because our experiment setup was well defined, we expected them to have a high agreement with each other. This was the case, meaning that a larger number of participants would likely not have changed the results. Their mean age was 35.6, and age range 23-56. They were required to be fluent in English, and be on a PC (no mobile devices were allowed).
The experiment consisted of eight distinct tasks, each including five stimulus images. One image shows a trajectory of a robot on a grid from a birds-eye perspective. The robot is moving from its starting position to either a blue or red circle, representing charging stations. There may also be walls, and the robot must navigate around them. Each picture is diferent, and there were a total of 8 ⋅ 5 = 40 stimuli. An example task is shown in Figure 3.
Participants were tasked with discerning the preferred charging station of a specific task’s robot, understanding that while the robot could charge at either, it had a latent preference for one. Instructed that the robot also aimed to conserve energy, possibly choosing a less favored station if it were closer, participants rated the likelihood of the robot’s preference for each station on a scale from 1 (very unlikely) to 5 (very likely). After making their likelihood assessment for the stations, they were presented with the next stimulus, with instructions to refine their inferences based on all previously shown images of the present task. Only one image was shown at any single time. Upon the task changing after ifve stimuli, participants were reminded that a new robot with diferent preferences was introduced.
For our model, we represented the tasks within a grid world that the RL agent needed to navigate. It incurred a minor negative penalty for movement and obtained positive rewards from both charging stations, determined by two specific parameters. The objective was to infer these parameters based on the observed data. We measured the discrepancy between observed and generated trajectories using Jaccard similarity. Essentially, our inference engine recreated the world as depicted in the stimulus, then ran the RL agent across varying parameters, comparing the generated trajectory against the observed one to form a posterior distribution for the two preferences. Preference likelihood ratings for the model were derived by computing the mean of the posterior distribution for preferences associated with both the blue and red charging stations.
The preference ratings of each response were first standardized so that they sum up to 1. Then, a mean rating for each stimulus in each task was computed. The model’s ratings were likewise standardized to sum up to 1, allowing comparison between human and model inferences. This comparison is shown in Figure 4. For calculating model fit, we selected only the inferences of the other color, because their values are inversions of each other after standadization. The model achieves a good fit, 2 = 0.78, = 0.1 . The most salient discrepancy between the model and human inferences is that the model is more careful in its estimates. Importantly, these results were obtained without any parameter tuning, meaning the model was not fit to the human data, but emerged similar data due to strong psychological assumptions about theory of mind.
The results exhibit the expected patterns of inference. Initially, participants faced uncertainty due to the limited evidence available. As they were exposed to additional stimuli, their inferences regarding the robot’s preferences became more definite: one station’s likelihood ratings increased, while the other’s decreased. Task 1 serves as an example of this (Figure 3): the participants’ inference that the robot prefers the red station gets stronger with each stimulus image shown. However, in tasks 3, 4, 6, 7, and 8, early stimuli suggested a certain preference, but subsequent stimuli revealed a stronger preference for the alternate station. This is similar to our motivating example in Figure 1. In these instances, the inferred preference for the more favored station shifted as the task progressed. Task 6 is an example of this (Figure 5): the participants are shown that the robot selects the red station, but it is always closer than the blue one, so there is uncertainty. Finally, in stimulus 5, it is revealed that the robot in fact prefers the blue station.
Human-AI alignment necessitates that both humans and
intelligent machines accurately interpret each other’s
intentions and actions [
Exploring decision-making under uncertainty is a large research topic. In our experiments, both humans and the model engaged in inferences and explicitly evaluated uncertainty, but they were not required to act on these inferences. A scenario where the model assists the observed actor will introduce the question of how to integrate uncertainty into decision-making. Taking the example of Janice from Figure 1, if adjusting the positions of the cofee and tea jars could aid her, the decision to do so necessitates careful consideration of potential consequences, ensuring the action truly benefits rather than hinders her. The manner in which a decision-making algorithm accounts for uncertainty during collaborative eforts is impacts the helpfulness of interventions and carries a risk of unintended obstruction.
All code, materials, and data are published online (https://version.helsinki.fi/shanz/quantifying-uncertaintyin-mtom.git) to facilitate open science.
This research has been supported by the Academy of Finland (grant 330347).