## Markov decision processes: discrete stochastic dynamic programmingThe Wiley-Interscience Paperback Series consists of selected books that have been made more accessible to consumers in an effort to increase global appeal and general circulation. With these new unabridged softcover volumes, Wiley hopes to extend the lives of these works by making them available to future generations of statisticians, mathematicians, and scientists. "This text is unique in bringing together so many results hitherto found only in part in other texts and papers. . . . The text is fairly self-contained, inclusive of some basic mathematical results needed, and provides a rich diet of examples, applications, and exercises. The bibliographical material at the end of each chapter is excellent, not only from a historical perspective, but because it is valuable for researchers in acquiring a good perspective of the MDP research potential." -Zentralblatt fur Mathematik ". . . it is of great value to advanced-level students, researchers, and professional practitioners of this field to have now a complete volume (with more than 600 pages) devoted to this topic. . . . Markov Decision Processes: Discrete Stochastic Dynamic Programming represents an up-to-date, unified, and rigorous treatment of theoretical and computational aspects of discrete-time Markov decision processes." -Journal of the American Statistical Association |

### What people are saying - Write a review

We haven't found any reviews in the usual places.

### Other editions - View all

Markov Decision Processes: Discrete Stochastic Dynamic Programming Martin L. Puterman Limited preview - 2014 |

Markov Decision Processes: Discrete Stochastic Dynamic Programming Martin L. Puterman No preview available - 2009 |

Markov Decision Processes: Discrete Stochastic Dynamic Programming Martin L. Puterman No preview available - 2009 |

### Common terms and phrases

0-discount action elimination analysis apply assume Assumption backward induction Blackwell optimal bounded chooses action component computational Consequently contraction mapping convergence Corollary cost countable decision epoch decision maker decision problem decision rule defined denote deterministic stationary discounted models dual linear program e-optimal equals establish evaluation Example expected total reward finite finite-state formulation function holds implies induction inventory Laurent series Lemma Markov chain Markov decision problem Markov decision process matrix maximizing modified policy iteration monotone multichain n-discount optimal negative models non-negative nondecreasing Note optimal policy optimal stationary policy optimality equation period policy iteration algorithm probability distribution Proposition provides queueing randomized reward criterion satisfies sequence solving stationary policy step stochastic subset superadditive Suppose supremum Theorem transient transition probabilities unichain models value iteration vector