## Markov Chains: Gibbs Fields, Monte Carlo Simulation, and QueuesThis book discusses both the theory and applications of Markov chains. The author studies both discrete-time and continuous-time chains and connected topics such as finite Gibbs fields, non-homogeneous Markov chains, discrete time regenerative processes, Monte Carlo simulation, simulated annealing, and queueing networks are also developed in this accessible and self-contained text. The text is firstly an introduction to the theory of stochastic processes at the undergraduate or beginning graduate level. Its primary objective is to initiate the student to the art of stochastic modelling. The treatment is mathematical, with definitions, theorems, proofs and a number of classroom examples which help the student to fully grasp the content of the main results. Problems of varying difficulty are proposed at the close of each chapter. The text is motivated by significant applications and progressively brings the student to the borders of contemporary research. Students and researchers in operations research and electrical engineering as well as in physics, biology and the social sciences will find this book of interest. |

### What people are saying - Write a review

We haven't found any reviews in the usual places.

### Contents

1 | |

DiscreteTime Markov Models | 53 |

Recurrence and Ergodicity | 95 |

Long Run Behavior | 125 |

Lyapunov Functions and Martingales | 167 |

Eigenvalues and Nonhomogeneous Markov Chains | 195 |

Gibbs Fields and Monte Carlo Simulation | 253 |

Bayesian Restoration of Images | 275 |

ContinuousTime Markov Models | 323 |

Poisson Calculus and Queues | 369 |

Appendix | 417 |

433 | |

### Other editions - View all

Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues Pierre Bremaud No preview available - 2013 |

Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues Pierre Bremaud No preview available - 2010 |

### Common terms and phrases

algorithm average number balance equations birth-and-death bounded called Chapter compute configuration continuous-time HMC corresponding defined Definition denoted discrete-time eigenvalue equality event Example 1.1 exists exponential finite state space follows formula function Gibbs sampler given gives HPPs implies independent inequality infinite infinitesimal initial distribution integers invariant measure irreducible Jackson network Lemma Markov chain Markov property martingale nonnegative notation number of customers observe obtain particular phase point process Poisson process Poisson system positive recurrent probability distribution Problem Proof prove queue random field random variables random vector random walk real numbers recurrent class regular jump HMC renewal equation resp respect result reversible right-hand side satisfied semigroup Show simulated annealing simulation solution stationary distribution stochastic matrix stochastic process strong Markov property sufficient condition Suppose Theorem 2.1 theory transient transition graph transition matrix transition semigroup weakly ergodic