Risk-averse control of continuous-time markov chains

Darinka Dentcheva, Andrzej Ruszczynski

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We develop an approach to time-consistent risk control of continuous-time processes in Markov systems with finite state space. Our analysis is based on time-consistent risk evaluation in continuous time, which uses the dual representation of dynamic coherent risk measures. Introducing a suitable differentiability concept for multivalued mappings and a concept of strong time consistency, we approximate the risk multikernels of the Markov system via set-valued derivatives, which gives rise to the concept of a risk multigenerator of a Markov process. We focus on risk-averse Markov decision problems with cost evaluated at the terminal state. We derive a system of ordinary differential equations for the risk evaluation of a given policy, which generalize the classical backward Kolmogorov equations for Markov processes. Furthermore, we establish optimality conditions in form of differential equation involving the support function of the risk multigenerator and we identify conditions for the existence of an optimal Markov policy.

Original languageEnglish (US)
Title of host publicationProceedings of the SIAM Conference on Control and Its Applications, CT 2017
PublisherSociety for Industrial and Applied Mathematics Publications
Pages78-85
Number of pages8
ISBN (Electronic)9781611975000
StatePublished - Jan 1 2017
Event2017 SIAM Conference on Control and Its Applications, CT 2017 - Pittsburgh, United States
Duration: Jul 10 2017Jul 12 2017

Other

Other2017 SIAM Conference on Control and Its Applications, CT 2017
CountryUnited States
CityPittsburgh
Period7/10/177/12/17

Fingerprint

Markov processes
Ordinary differential equations
Differential equations
Derivatives
Costs

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering

Cite this

Dentcheva, D., & Ruszczynski, A. (2017). Risk-averse control of continuous-time markov chains. In Proceedings of the SIAM Conference on Control and Its Applications, CT 2017 (pp. 78-85). Society for Industrial and Applied Mathematics Publications.
Dentcheva, Darinka ; Ruszczynski, Andrzej. / Risk-averse control of continuous-time markov chains. Proceedings of the SIAM Conference on Control and Its Applications, CT 2017. Society for Industrial and Applied Mathematics Publications, 2017. pp. 78-85
@inproceedings{6e3a12bb91ba4e1782ca602408269053,
title = "Risk-averse control of continuous-time markov chains",
abstract = "We develop an approach to time-consistent risk control of continuous-time processes in Markov systems with finite state space. Our analysis is based on time-consistent risk evaluation in continuous time, which uses the dual representation of dynamic coherent risk measures. Introducing a suitable differentiability concept for multivalued mappings and a concept of strong time consistency, we approximate the risk multikernels of the Markov system via set-valued derivatives, which gives rise to the concept of a risk multigenerator of a Markov process. We focus on risk-averse Markov decision problems with cost evaluated at the terminal state. We derive a system of ordinary differential equations for the risk evaluation of a given policy, which generalize the classical backward Kolmogorov equations for Markov processes. Furthermore, we establish optimality conditions in form of differential equation involving the support function of the risk multigenerator and we identify conditions for the existence of an optimal Markov policy.",
author = "Darinka Dentcheva and Andrzej Ruszczynski",
year = "2017",
month = "1",
day = "1",
language = "English (US)",
pages = "78--85",
booktitle = "Proceedings of the SIAM Conference on Control and Its Applications, CT 2017",
publisher = "Society for Industrial and Applied Mathematics Publications",
address = "United States",

}

Dentcheva, D & Ruszczynski, A 2017, Risk-averse control of continuous-time markov chains. in Proceedings of the SIAM Conference on Control and Its Applications, CT 2017. Society for Industrial and Applied Mathematics Publications, pp. 78-85, 2017 SIAM Conference on Control and Its Applications, CT 2017, Pittsburgh, United States, 7/10/17.

Risk-averse control of continuous-time markov chains. / Dentcheva, Darinka; Ruszczynski, Andrzej.

Proceedings of the SIAM Conference on Control and Its Applications, CT 2017. Society for Industrial and Applied Mathematics Publications, 2017. p. 78-85.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Risk-averse control of continuous-time markov chains

AU - Dentcheva, Darinka

AU - Ruszczynski, Andrzej

PY - 2017/1/1

Y1 - 2017/1/1

N2 - We develop an approach to time-consistent risk control of continuous-time processes in Markov systems with finite state space. Our analysis is based on time-consistent risk evaluation in continuous time, which uses the dual representation of dynamic coherent risk measures. Introducing a suitable differentiability concept for multivalued mappings and a concept of strong time consistency, we approximate the risk multikernels of the Markov system via set-valued derivatives, which gives rise to the concept of a risk multigenerator of a Markov process. We focus on risk-averse Markov decision problems with cost evaluated at the terminal state. We derive a system of ordinary differential equations for the risk evaluation of a given policy, which generalize the classical backward Kolmogorov equations for Markov processes. Furthermore, we establish optimality conditions in form of differential equation involving the support function of the risk multigenerator and we identify conditions for the existence of an optimal Markov policy.

AB - We develop an approach to time-consistent risk control of continuous-time processes in Markov systems with finite state space. Our analysis is based on time-consistent risk evaluation in continuous time, which uses the dual representation of dynamic coherent risk measures. Introducing a suitable differentiability concept for multivalued mappings and a concept of strong time consistency, we approximate the risk multikernels of the Markov system via set-valued derivatives, which gives rise to the concept of a risk multigenerator of a Markov process. We focus on risk-averse Markov decision problems with cost evaluated at the terminal state. We derive a system of ordinary differential equations for the risk evaluation of a given policy, which generalize the classical backward Kolmogorov equations for Markov processes. Furthermore, we establish optimality conditions in form of differential equation involving the support function of the risk multigenerator and we identify conditions for the existence of an optimal Markov policy.

UR - http://www.scopus.com/inward/record.url?scp=85029435609&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85029435609&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85029435609

SP - 78

EP - 85

BT - Proceedings of the SIAM Conference on Control and Its Applications, CT 2017

PB - Society for Industrial and Applied Mathematics Publications

ER -

Dentcheva D, Ruszczynski A. Risk-averse control of continuous-time markov chains. In Proceedings of the SIAM Conference on Control and Its Applications, CT 2017. Society for Industrial and Applied Mathematics Publications. 2017. p. 78-85