Adventures in Stochastic ProcessesStochastic processes are necessary ingredients for building models of a wide variety of phenomena exhibiting time varying randomness. In a lively and imaginative presentation, studded with examples, exercises, and applications, and supported by inclusion of computational procedures, the author has created a textbook that provides easy access to this fundamental topic for many students of applied sciences at many levels. With its carefully modularized discussion and crystal clear differentiation between rigorous proof and plausibility argument, it is accessible to beginners but flexible enough to serve as well those who come to the course with strong backgrounds. The prerequisite background for reading the book is a graduate level pre-measure theoretic probability course. No knowledge of measure theory is presumed and advanced notions of conditioning are scrupulously avoided until the later chapters of the book. The book can be used for either a one or two semester course as given in departments of mathematics, statistics, operation research, business and management, or a number of engineering departments. Its approach to exercises and applications is practical and serious. Some underlying principles of complex problems and computations are cleanly and quickly delineated through rich vignettes of whimsically imagined Happy Harry and his Optima Street gang’s adventures in a world whose randomness is a never-ending source of both wonder and scientific insight. The tools of applied probability---discrete spaces, Markov chains, renewal theory, point processes, branching processes, random walks, Brownian motion---are presented to the reader in illuminating discussion. Applications include such topics as queuing, storage, risk analysis, genetics, inventory, choice, economics, sociology, and other. Because of the conviction that analysts who build models should know how to build them for each class of process studied, the author has included such constructions. |
Contents
Preliminaries Discrete Index Sets andor Discrete State Spaces | 1 |
12 CONVOLUTION | 5 |
13 GENERATING FUNCTIONS | 7 |
131 DIFFERENTIATION OF GENERATING FUNCTIONS | 9 |
132 GENERATING FUNCTIONS AND MOMENTS | 10 |
133 GENERATING FUNCTIONS AND CONVOLUTION | 12 |
134 GENERATING FUNCTIONS COMPOUNDING AND RANDOM SUMS | 15 |
14 THE SIMPLE BRANCHING PROCESS | 18 |
43 TRANSFORMING POISSON PROCESSES | 308 |
431 MAXSTABLE AND STABLE RANDOM VARIABLES | 313 |
44 MORE TRANSFORMATION THEORY MARKING AND THINNING | 316 |
45 THE ORDER STATISTIC PROPERTY | 321 |
46 VARIANTS OF THE POISSON PROCESS | 327 |
47 TECHNICAL BASICS | 333 |
471 THE LAPLACE FUNCTIONAL | 336 |
48 MORE ON THE POISSON PROCESS | 337 |
15 LIMIT DISTRIBUTIONS AND THE CONTINUITY THEOREM | 27 |
151 THE LAW OF RARE EVENTS | 30 |
16 THE SIMPLE RANDOM WALK | 33 |
17 THE DISTRIBUTION OF A PROCESS | 40 |
18 STOPPING TIMES | 44 |
181 WALDS IDENTITY | 47 |
182 SPLITTING AN IID SEQUENCE AT A STOPPING TIME | 48 |
EXERCISES | 51 |
Markov Chains | 60 |
21 CONSTRUCTION AND FIRST PROPERTIES | 61 |
22 EXAMPLES | 66 |
23 HIGHER ORDER TRANSITION PROBABILITIES | 72 |
24 DECOMPOSITION OF THE STATE SPACE | 77 |
25 THE DISSECTION PRINCIPLE | 81 |
26 TRANSIENCE AND RECURRENCE | 85 |
27 PERIODICITY | 91 |
28 SOLIDARITY PROPERTIES | 92 |
29 EXAMPLES | 94 |
210 CANONICAL DECOMPOSITION | 98 |
211 ABSORPTION PROBABILITIES | 102 |
212 INVARIANT MEASURES AND STATIONARY DISTRIBUTIONS | 116 |
2121 TIME AVERAGES | 122 |
213 LIMIT DISTRIBUTIONS | 126 |
2131 MORE ON NULL RECURRENCE AND TRANSIENCE | 134 |
214 COMPUTATION OF THE STATIONARY DISTRIBUTION | 137 |
215 CLASSIFICATION TECHNIQUES | 142 |
EXERCISES | 147 |
Renewal Theory | 174 |
32 ANALYTIC INTERLUDE | 176 |
322 CONVOLUTION | 178 |
323 LAPLACE TRANSFORMS | 181 |
33 COUNTING RENEWALS | 185 |
34 RENEWAL REWARD PROCESSES | 192 |
35 THE RENEWAL EQUATION | 197 |
351 RISK PROCESSES | 205 |
36 THE POISSON PROCESS AS A RENEWAL PROCESS | 211 |
37 AN INFORMAL DISCUSSION OF RENEWAL LIMIT THEOREMS AND REGENERATIVE PROCESSES | 212 |
371 AN INFORMAL DISCUSSION OF REGENERATIVE PROCESSES | 215 |
38 DISCRETE RENEWAL THEORY | 221 |
39 STATIONARY RENEWAL PROCESSES | 224 |
310 THE BLACKWELL AND KEY RENEWAL THEOREMS | 230 |
3101 DIRECT RIEMANN INTEGRABILITY | 231 |
3102 EQUIVALENT FORMS OF THE RENEWAL THEOREM | 237 |
3103 PROOF OF THE RENEWAL THEOREM | 243 |
311 IMPROPER RENEWAL EQUATIONS | 253 |
312 MORE ON REGENERATIVE PROCESSES | 259 |
3122 THE RENEWAL EQUATION AND SMITHS THEOREM | 263 |
3123 QUEUEING EXAMPLES | 269 |
EXERCISES | 280 |
Point Processes | 300 |
42 THE POISSON PROCESS | 303 |
49 A GENERAL CONSTRUCTION OF THE POISSON PROCESS A SIMPLE DERIVATION OF THE ORDER STATISTIC PROPERTY | 341 |
410 MORE TRANSFORMATION THEORY LOCATION DEPENDENT THINNING | 343 |
411 RECORDS | 346 |
EXERCISES | 349 |
Continuous Time Markov Chains | 367 |
52 STABILITY AND EXPLOSIONS | 375 |
521 THE MARKOV PROPERTY | 377 |
53 DISSECTION | 380 |
54 THE BACKWARD EQUATION AND THE GENERATOR MATRIX | 382 |
55 STATIONARY AND LIMITING DISTRIBUTIONS | 392 |
551 MORE ON INVARIANT MEASURES | 398 |
56 LAPLACE TRANSFORM METHODS | 402 |
57 CALCULATIONS AND EXAMPLES | 406 |
571 QUEUEING NETWORKS | 415 |
58 TIME DEPENDENT SOLUTIONS | 426 |
59 REVERSIBILITY | 431 |
510 UNIFORMIZABLE CHAINS | 436 |
511 THE LINEAR BIRTH PROCESS AS A POINT PROCESS | 439 |
EXERCISES | 446 |
Brownian Motion | 482 |
62 PRELIMINARIES | 487 |
63 CONSTRUCTION OF BROWNIAN MOTION | 489 |
64 SIMPLE PROPERTIES OF STANDARD BROWNIAN MOTION | 494 |
65 THE REFLECTION PRINCIPLE AND THE DISTRIBUTION OF THE MAXIMUM | 497 |
66 THE STRONG INDEPENDENT INCREMENT PROPERTY AND REFLECTION | 504 |
67 ESCAPE FROM A STRIP | 508 |
68 BROWNIAN MOTION WITH DRIFT | 511 |
69 HEAVY TRAFFIC APPROXIMATIONS IN QUEUEING THEORY | 514 |
610 THE BROWNIAN BRIDGE AND THE KOLMOGOROVSMIRNOV STATISTIC | 524 |
611 PATH PROPERTIES | 539 |
612 QUADRATIC VARIATION | 542 |
613 KHINTCHINES LAW OF THE ITERATED LOGARITHM FOR BROWNIAN MOTION | 546 |
EXERCISES | 551 |
The General Random Walk | 558 |
71 STOPPING TIMES | 559 |
72 GLOBAL PROPERTIES | 561 |
PROBABILISTIC INTERPRETATIONS OF TRANSFORMS | 564 |
74 DUAL PAIRS OF STOPPING TIMES | 568 |
75 WIENERHOPF DECOMPOSITIONS | 573 |
76 CONSEQUENCES OF THE WIENERHOPF FACTORIZATION | 581 |
77 THE MAXIMUM OF A RANDOM WALK | 587 |
78 RANDOM WALKS AND THE GG1 QUEUE | 591 |
781 EXPONENTIAL RIGHT TAIL | 595 |
782 APPLICATIONS TO THE GM1 QUEUEING MODEL | 599 |
783 EXPONENTIAL LEFT TAIL | 602 |
784 THE MG1 QUEUE | 605 |
785 QUEUE LENGTHS | 607 |
| 613 | |
| 617 | |