Finite Mixture and Markov Switching Models: Modeling and Applications to Random Processes (Google eBook)

Front Cover
Springer Science & Business Media, Nov 24, 2006 - Mathematics - 512 pages
0 Reviews
WINNER OF THE 2007 DEGROOT PRIZE! The prominence of finite mixture modelling is greater than ever. Many important statistical topics like clustering data, outlier treatment, or dealing with unobserved heterogeneity involve finite mixture models in some way or other. The area of potential applications goes beyond simple data analysis and extends to regression analysis and to non-linear time series analysis using Markov switching models. For more than the hundred years since Karl Pearson showed in 1894 how to estimate the five parameters of a mixture of two normal distributions using the method of moments, statistical inference for finite mixture models has been a challenge to everybody who deals with them. In the past ten years, very powerful computational tools emerged for dealing with these models which combine a Bayesian approach with recent Monte simulation techniques based on Markov chains. This book reviews these techniques and covers the most recent advances in the field, among them bridge sampling techniques and reversible jump Markov chain Monte Carlo methods. It is the first time that the Bayesian perspective of finite mixture modelling is systematically presented in book form. It is argued that the Bayesian approach provides much insight in this context and is easily implemented in practice. Although the main focus is on Bayesian inference, the author reviews several frequentist techniques, especially selecting the number of components of a finite mixture model, and discusses some of their shortcomings compared to the Bayesian approach. The aim of this book is to impart the finite mixture and Markov switching approach to statistical modelling to a wide-ranging community. This includes not only statisticians, but also biologists, economists, engineers, financial agents, market researcher, medical researchers or any other frequent user of statistical models. This book should help newcomers to the field to understand how finite mixture and Markov switching models are formulated, what structures they imply on the data, what they could be used for, and how they are estimated. Researchers familiar with the subject also will profit from reading this book. The presentation is rather informal without abandoning mathematical correctness. Previous notions of Bayesian inference and Monte Carlo simulation are useful but not needed.
  

What people are saying - Write a review

We haven't found any reviews in the usual places.

Contents

Finite Mixture Modeling
1
12 Finite Mixture Distributions
3
122 Some Descriptive Features of Finite Mixture Distributions
5
123 Diagnosing Similarity of Mixture Components
9
125 Statistical Modeling Based on Finite Mixture Distributions
11
13 Identifiability of a Finite Mixture Distribution
14
131 Nonidentifiability Due to Invariance to Relabeling the Components
15
132 Nonidentifiability Due to Potential Overfitting
17
823 Statistical Modeling Based on Finite Mixture of Regression Models
246
824 Outliers in a Regression Model
249
832 Bayesian Inference When the Allocations Are Known
250
833 Choosing Prior Distributions
252
834 Bayesian Inference When the Allocations Are Unknown
253
835 Bayesian Inference Using Posterior Draws
254
836 Dealing with Model Specification Uncertainty
255
84 MixedEffects Finite Mixtures of Regression Models
256

133 Formal Identifiability Constraints
19
134 Generic Identifiability
21
Statistical Inference for a Finite Mixture Model with Known Number of Components
25
22 Classification for Known Component Parameters
26
222 The Bayes Classifier for a Whole Data Set
27
23 Parameter Estimation for Known Allocation
29
232 CompleteData Maximum Likelihood Estimation
30
233 CompleteData Bayesian Estimation of the Component Parameters
31
234 CompleteData Bayesian Estimation of the Weights
35
24 Parameter Estimation When the Allocations Are Unknown
41
241 Method of Moments
42
243 A Helicopter Tour of the Mixture Likelihood Surface for Two Examples
44
244 Maximum Likelihood Estimation
49
245 Bayesian Parameter Estimation
53
246 DistanceBased Methods
54
Practical Bayesian Inference for a Finite Mixture Model with Known Number of Components
57
32 Choosing the Prior for the Parameters of a Mixture Model
58
322 Improper Priors May Cause Improper Mixture Posteriors
59
323 Conditionally Conjugate Priors
60
325 Other Priors
62
33 Some Properties of the Mixture Posterior Density
63
332 Invariance of Seemingly ComponentSpecific Functionals
64
333 The Marginal Posterior Distribution of the Allocations
65
334 Invariance of the Posterior Distribution of the Allocations
67
34 Classification Without Parameter Estimation
68
341 SingleMove Gibbs Sampling
69
342 The MetropolisHastings Algorithm
72
35 Parameter Estimation Through Data Augmentation and MCMC 73
73
352 Data Augmentation and MCMC for a Mixture of Poisson Distributions
74
353 Data Augmentation and MCMC for General Mixtures
76
354 MCMC Sampling Under Improper Priors
78
356 Permutation MCMC Sampling
81
36 Other Monte Carlo Methods Useful for Mixture Models
83
362 Importance Sampling for the Allocations
84
363 Perfect Sampling
85
372 Using Posterior Draws for Bayesian Inference
87
373 Predictive Density Estimation
89
374 Individual Parameter Inference
91
375 Inference on the Hyperparameter of a Hierarchical Prior
92
377 Model Identification
94
Statistical Inference for Finite Mixture Models Under Model Specification Uncertainty
99
42 Parameter Estimation Under Model Specification Uncertainty
100
422 Practical Bayesian Parameter Estimation for Overfitting Finite Mixture Models
103
423 Potential Overfitting
105
43 Informal Methods for Identifying the Number of Components
107
431 Mode Hunting in the Mixture Posterior
108
432 Mode Hunting in the Sample Histogram
109
433 Diagnosing Mixtures Through the Method of Moments
110
434 Diagnosing Mixtures Through Predictive Methods
112
435 Further Approaches
114
442 AIC BIC and the Schwarz Criterion
116
443 Further Approaches
117
452 Marginal Likelihoods
118
453 Bayes Factors for Model Comparison
119
454 Formal Bayesian Model Selection
121
455 Choosing Priors for Model Selection
122
456 Further Approaches
123
Computational Tools for Bayesian Inference for Finite Mixtures Models Under Model Specification Uncertainty
124
521 ProductSpace MCMC
126
522 Reversible Jump MCMC
129
523 Birth and Death MCMC Methods
137
53 Marginal Likelihoods for Finite Mixture Models
139
532 Choosing Priors for Selecting the Number of Components
141
533 Computation of the Marginal Likelihood for Mixture Models
143
542 SamplingBased Approximations for Mixture Models
144
543 Importance Sampling
146
544 Reciprocal Importance Sampling
147
545 Harmonic Mean Estimator
148
546 Bridge Sampling Technique
150
547 Comparison of Different SimulationBased Estimators
154
548 Dealing with Hierarchical Priors
159
552 Chibs Estimator
160
553 Laplace Approximation
164
56 Reversible Jump MCMC Versus Marginal Likelihoods?
165
Finite Mixture Models with Normal Components
169
612 Parameter Estimation for Mixtures of Normals
171
614 Applications of Mixture of Normal Distributions
176
62 Bayesian Estimation of Univariate Mixtures of Normals
177
623 The Influence of the Prior on the Variance Ratio
179
624 Bayesian Estimation Using MCMC
180
626 Introducing Prior Dependence Among the Components
185
627 Further SamplingBased Approaches
187
628 Application to the Fishery Data
188
63 Bayesian Estimation of Multivariate Mixtures of Normals
190
632 Prior Distributions Standard Prior Distributions
192
633 Bayesian Parameter Estimation Using MCMC
193
634 Application to Fishers Iris Data
195
642 Model Selection Problems for Mixtures of Normals
199
Data Analysis Based on Finite Mixtures
203
712 ModelBased Clustering Using Finite Mixture Models
204
713 The Classification Likelihood and the Bayesian MAP Approach
207
714 Choosing Clustering Criteria and the Number of Components
210
715 Model Choice for the Fishery Data
216
716 Model Choice for Fishers Iris Data
218
717 Bayesian Clustering Based on Loss Functions
220
718 Clustering for Fishers Iris Data
224
722 Bayesian Inference for Outlier Models Based on Finite Mixtures
225
723 Outlier Modeling of Darwins Data
226
724 Clustering Under Outliers and Noise
227
731 Parameter Estimation
230
732 Dealing with Unknown Number of Components
233
742 Discriminant Analysis
235
743 Combining Classified and Unclassified Observations
236
744 Density Estimation Using Finite Mixtures
237
745 Finite Mixtures as an Auxiliary Computational Tool in Bayesian Analysis
238
Finite Mixtures of Regression Models
240
82 Finite Mixture of Multiple Regression Models
242
822 Identifiability
243
843 Bayesian Parameter Estimation When the Allocations Are Known
257
844 Bayesian Parameter Estimation When the Allocations Are Unknown
258
85 Finite Mixture Models for Repeated Measurements
259
851 Pooling Information Across Similar Units
260
853 Choosing the Prior for Bayesian Estimation
265
855 Practical Bayesian Estimation Using MCMC
267
856 Dealing with Model Specification Uncertainty
269
857 Application to the Marketing Data
270
86 Further Issues
273
862 Modeling the Weight Distribution
274
Finite Mixture Models with Nonnormal Components
277
912 Bayesian Inference
278
92 Finite Mixtures of Poisson Distributions
279
922 Capturing Overdispersion in Count Data
280
923 Modeling Excess Zeros
282
924 Application to the Eye Tracking Data
283
93 Finite Mixture Models for Binary and Categorical Data
286
932 Finite Mixtures of Multinomial Distributions
288
94 Finite Mixtures of Generalized Linear Models
289
941 Finite Mixture Regression Models for Count Data
290
942 Finite Mixtures of Logit and Probit Regression Models
292
943 Parameter Estimation for Finite Mixtures of GLMs
293
944 Model Selection for Finite Mixtures of GLMs
294
951 The Basic Latent Class Model
295
952 Identification and Parameter Estimation
296
953 Extensions of the Basic Latent Class Model
297
96 Further Issues
298
962 Finite Mixtures of GLMs with Random Effects
299
Finite Markov Mixture Modeling
300
1022 Irreducible Aperiodic Markov Chains
304
1023 Moments of a Markov Mixture Distribution
308
1024 The Autocorrelation Function of a Process Generated by a Markov Mixture Distribution
310
1025 The Autocorrelation Function of the Squared Process
311
1026 The Standard Finite Mixture Distribution as a Limiting Case
312
1027 Identifiability of a Finite Markov Mixture Distribution
313
103 Statistical Modeling Based on Finite Markov Mixture Distributions
314
1032 The Markov Switching Regression Model
315
1033 Nonergodic Markov Chains
316
Statistical Inference for Markov Switching Models
319
1121 Statistical Inference About the States
320
1123 Filtering for Special Cases
323
1124 Smoothing the States
324
1125 Filtering and Smoothing for More General Models
326
113 Parameter Estimation for Known States
327
1132 CompleteData Bayesian Parameter Estimation
329
114 Parameter Estimation When the States are Unknown
330
1142 Maximum Likelihood Estimation
333
1143 Bayesian Estimation
334
115 Bayesian Parameter Estimation with Known Number of States
335
1152 Some Properties of the Posterior Distribution of a Markov Switching Model
336
1153 Parameter Estimation Through Data Augmentation and MCMC
337
1154 Permutation MCMC Sampling
340
1156 Sampling Posterior Paths of the Hidden Markov Chain
342
1157 Other SamplingBased Approaches
345
116 Statistical Inference Under Model Specification Uncertainty
346
1163 Marginal Likelihoods for Markov Switching Models
347
1164 Model Space MCMC
348
1172 Capturing Overdispersion and Autocorrelation Using Poisson Markov Mixture Models
349
1173 Application to the Lamb Data
351
Nonlinear Time Series Analysis Based on Markov Switching Models
357
122 The Markov Switching Autoregressive Model
358
1222 Model Definition
360
1223 Features of the MSAR Model
362
1224 Markov Switching Models for Nonstationary Time Series
363
1225 Parameter Estimation and Model Selection
365
123 Markov Switching Dynamic Regression Models
371
124 Prediction of Time Series Based on Markov Switching Models
372
1242 Forecasting of Markov Switching Models via SamplingBased Methods
374
1252 Capturing Features of Financial Time Series Through Markov Switching Models
377
1253 Switching ARCH Models
378
1254 Statistical Inference for Switching ARCH Models
380
1255 Switching GARCH Models
383
126 Some Extensions
384
1262 Markov Switching Models for Longitudinal and Panel Data
385
1263 Markov Switching Models for Multivariate Time Series
386
Switching State Space Models
389
1312 The Linear Gaussian State Space Form
391
1313 Multiprocess Models
393
1315 The General State Space Form
394
132 Nonlinear Time Series Analysis Based on Switching State Space Models
396
1323 Capturing Sudden Changes in Time Series
398
1324 Switching Dynamic Factor Models
400
1325 Switching State Space Models as a SemiParametric Smoothing Device
401
1331 The Filtering Problem
402
1333 Filtering for the Linear Gaussian State Space Model
404
1334 Filtering for Multiprocess Models
406
134 Parameter Estimation for Switching State Space Models
410
1341 The Likelihood Function of a State Space Model
411
1342 Maximum Likelihood Estimation
412
135 Practical Bayesian Estimation Using MCMC
415
1351 Various Data Augmentation Schemes
416
1352 Sampling the Continuous State Process from the Smoother Density
417
1353 Sampling the Discrete States for a Switching State Space Model
420
136 Further Issues
421
1362 Auxiliary Mixture Sampling for Nonlinear and Nonnormal State Space Models
422
137 Illustrative Application to Modeling Exchange Rate Data
423
Appendix A1 Summary of Probability Distributions
431
A12 The Binomial Distribution
432
A14 The Exponential Distribution
433
A16 The Gamma Distribution
434
A17 The Geometric Distribution
435
A110 The Normal Distribution
436
Al Summary of Probability Distributions 437
437
438 A Appendix
438
A2 Software
439
References
441
Index
481
Copyright

Common terms and phrases

Bibliographic information