Journal of Convex Analysis 30 (2023), No. 3, 897--916
Copyright Heldermann Verlag 2023
Dynamic Programming For Data Independent Decision Sets
Paul Dommel
Faculty of Mathematics, University of Technology, Chemnitz, Germany
Alois Pichler
Faculty of Mathematics, University of Technology, Chemnitz, Germany
alois.pichler@math.tu-chemnitz.de
Multistage stochastic optimization problems are oftentimes formulated informally in a pathwise way. These formulations are appealing in a discrete setting and suitable when addressing computational challenges, for example. But the pathwise problem statement does not allow an analysis with mathematical rigor and is therefore not appropriate.
R. T. Rockafellar and R. J.-B. Wets [Nonanticipativity and L1-martingales in stochastic optimization problems, Math. Programming Study 6 (1976) 170--187] address the fundamental measurability concern of the value functions in the case of convex costs and constraints. This paper resumes these foundations. The contribution is a proof that there exist measurable versions of intermediate value functions, which reveals regularity in addition. Our proof builds on the Kolmogorov continuity theorem.
It is demonstrated that verification theorems allow stating traditional problem specifications in the novel setting with mathematical rigor. Further, we provide dynamic equations for the general problem. The problem classes covered include Markov decision processes, reinforcement learning and stochastic dual dynamic programming.
Keywords: Multistage stochastic optimization, stochastic processes, measurability.
MSC: 90C08, 90C15, 60G07.
[ Fulltext-pdf (149 KB)] for subscribers only.