Constrained risk-averse Markov decision processes

From Murray Wiki
Revision as of 20:55, 4 September 2021 by Murray (talk | contribs) (Created page with "{{Paper |Title=Constrained risk-averse Markov decision processes |Authors=Mohamadreza Ahmadi, Ugo Rosolia, Michel D Ingham, Richard M Murray, Aaron D Ames |Source=35th AAAI Co...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search
Title Constrained risk-averse Markov decision processes
Authors Mohamadreza Ahmadi, Ugo Rosolia, Michel D Ingham, Richard M Murray and Aaron D Ames
Source 35th AAAI Conference on Artificial Intelligence (AAAI-21)
Abstract We consider the problem of designing policies for Markov decision processes (MDPs) with dynamic coherent risk objectives and constraints. We begin by formulating the problem in a Lagrangian framework. Under the assumption that the risk objectives and constraints can be represented by a Markov risk transition mapping, we propose an optimization-based method to synthesize Markovian policies that lower-bound the constrained risk-averse problem. We demonstrate that the formulated optimization problems are in the form of difference convex programs (DCPs) and can be solved by the disciplined convex-concave programming (DCCP) framework. We show that these results generalize linear programs for constrained MDPs with total discounted expected costs and constraints. Finally, we illustrate the effectiveness of the proposed method with numerical experiments on a rover navigation problem involving conditional-value-at-risk (CVaR) and entropic-value-at-risk (EVaR) coherent risk measures.
Type Conference paper
URL http://www.cds.caltech.edu/~murray/preprints/aaaYY-place.pdf
DOI
Tag MA+21-aaai
ID 2021e
Funding
Flags