We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Time-Varying Parameters in Sequential Decision Making Problems

Abstract: In this paper we address the class of Sequential Decision Making (SDM) problems that are characterized by time-varying parameters. These parameter dynamics are either pre-specified or manipulable. At any given time instant the decision policy -- that governs the sequential decisions -- along with all the parameter values determines the cumulative cost incurred by the underlying SDM. Thus, the objective is to determine the manipulable parameter dynamics as well as the time-varying decision policy such that the associated cost gets minimized at each time instant. To this end we develop a control-theoretic framework to design the unknown parameter dynamics such that it locates and tracks the optimal values of the parameters, and simultaneously determines the time-varying optimal sequential decision policy. Our methodology builds upon a Maximum Entropy Principle (MEP) based framework that addresses the static parameterized SDMs. More precisely, we utilize the resulting smooth approximation (from the above framework) of the cumulative cost as a control Lyapunov function. We show that under the resulting control law the parameters asymptotically track the local optimal, the proposed control law is Lipschitz continuous and bounded, as well as ensure that the decision policy of the SDM is optimal for a given set of parameter values. The simulations demonstrate the efficacy of our proposed methodology.
Subjects: Optimization and Control (math.OC)
Cite as: arXiv:2201.10273 [math.OC]
  (or arXiv:2201.10273v1 [math.OC] for this version)

Submission history

From: Amber Srivastava Mr [view email]
[v1] Tue, 25 Jan 2022 12:37:12 GMT (5044kb,D)

Link back to: arXiv, form interface, contact.