Efficient estimation and control for Markov processes

Apostolos N. Burnetas, Michael N. Katehakis

Research output: Contribution to journalConference articlepeer-review

2 Scopus citations


We consider the problem of sequential control for a finite state and action Markovian Decision Process with incomplete information regarding the transition probabilities P ∈ Papprox. Under suitable irreducibility assumptions for Papprox.. We construct adaptive policies that maximize the rate of convergence of realized rewards to that of the optimal (non adaptive) policy under complete information. These adaptive policies are specified via an easily computable index function, of states, controls and statistics, so that one takes a control with the largest index value in the current state in every period.

Original languageEnglish (US)
Pages (from-to)1402-1407
Number of pages6
JournalProceedings of the IEEE Conference on Decision and Control
StatePublished - 1995
Externally publishedYes
EventProceedings of the 1995 34th IEEE Conference on Decision and Control. Part 1 (of 4) - New Orleans, LA, USA
Duration: Dec 13 1995Dec 15 1995

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Modeling and Simulation
  • Control and Optimization


Dive into the research topics of 'Efficient estimation and control for Markov processes'. Together they form a unique fingerprint.

Cite this