References & Citations
Computer Science > Artificial Intelligence
Title: Computing Policies That Account For The Effects Of Human Agent Uncertainty During Execution In Markov Decision Processes
(Submitted on 15 Sep 2021 (v1), last revised 3 Mar 2022 (this version, v3))
Abstract: When humans are given a policy to execute, there can be policy execution errors and deviations in policy if there is uncertainty in identifying a state. This can happen due to the human agent's cognitive limitations and/or perceptual errors. So an algorithm that computes a policy for a human to execute ought to consider these effects in its computations. An optimal Markov Decision Process (MDP) policy that is poorly executed (because of a human agent) maybe much worse than another policy that is suboptimal in the MDP, but considers the human-agent's execution behavior. In this paper we consider two problems that arise from state uncertainty; these are erroneous state-inference, and extra-sensing actions that a person might take as a result of their uncertainty. We present a framework to model the human agent's behavior with respect to state uncertainty, and can be used to compute MDP policies that accounts for these problems. This is followed by a hill climbing algorithm to search for good policies given our model of the human agent. We also present a branch and bound algorithm which can find the optimal policy for such problems. We show experimental results in a Gridworld domain, and warehouse-worker domain. Finally, we present human-subject studies that support our human model assumptions.
Submission history
From: Sriram Gopalakrishnan [view email][v1] Wed, 15 Sep 2021 17:10:46 GMT (1054kb,D)
[v2] Mon, 20 Sep 2021 21:24:20 GMT (1054kb,D)
[v3] Thu, 3 Mar 2022 22:00:30 GMT (8937kb,D)
Link back to: arXiv, form interface, contact.