We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.IT

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Electrical Engineering and Systems Science > Systems and Control

Title: Active Trajectory Estimation for Partially Observed Markov Decision Processes via Conditional Entropy

Abstract: In this paper, we consider the problem of controlling a partially observed Markov decision process (POMDP) in order to actively estimate its state trajectory over a fixed horizon with minimal uncertainty. We pose a novel active smoothing problem in which the objective is to directly minimise the smoother entropy, that is, the conditional entropy of the (joint) state trajectory distribution of concern in fixed-interval Bayesian smoothing. Our formulation contrasts with prior active approaches that minimise the sum of conditional entropies of the (marginal) state estimates provided by Bayesian filters. By establishing a novel form of the smoother entropy in terms of the POMDP belief (or information) state, we show that our active smoothing problem can be reformulated as a (fully observed) Markov decision process with a value function that is concave in the belief state. The concavity of the value function is of particular importance since it enables the approximate solution of our active smoothing problem using piecewise-linear function approximations in conjunction with standard POMDP solvers. We illustrate the approximate solution of our active smoothing problem in simulation and compare its performance to alternative approaches based on minimising marginal state estimate uncertainties.
Comments: 7 pages, 3 figures, accepted for presentation at 2021 European Control Conference
Subjects: Systems and Control (eess.SY); Information Theory (cs.IT)
Cite as: arXiv:2104.01545 [eess.SY]
  (or arXiv:2104.01545v1 [eess.SY] for this version)

Submission history

From: Timothy Molloy [view email]
[v1] Sun, 4 Apr 2021 06:16:53 GMT (819kb,D)

Link back to: arXiv, form interface, contact.