Asymptotic Theory of Statistics and Probability

Anirban DasGupta

Springer Science & Business Media, Mar 7, 2008 - Mathematics - 722 pages

This book developed out of my year-long course on asymptotic theory at Purdue University. To some extent, the topics coincide with what I cover in that course. There are already a number of well-known books on asy- totics. This book is quite different. It covers more topics in one source than areavailableinanyothersinglebookonasymptotictheory. Numeroustopics covered in this book are available in the literature in a scattered manner, and they are brought together under one umbrella in this book. Asymptotic theory is a central unifying theme in probability and statistics. My main goal in writing this book is to give its readers a feel for the incredible scope and reach of asymptotics. I have tried to write this book in a way that is accessible and to make the reader appreciate the beauty of theory and the insights that only theory can provide. Essentially every theorem in the book comes with at least one reference, preceding or following the statement of the theorem. In addition, I have p- vided a separate theorem-by-theorem reference as an entry on its own in the front of the book to make it extremely convenient for the reader to ?nd a proof that was not provided in the text. Also particularly worth mentioning is a collection of nearly 300 practically useful inequalities that I have c- lected together from numerous sources. This is appended at the very end of the book.

Contents

• Basic Convergence Concepts and Theorems: 1
• 12 Three Series Theorem and Kolmogorovs ZeroOne Law: 6
• 13 Central Limit Theorem and Law of the Iterated Logarithm: 7
• 14 Further Illustrative Examples: 10
• 15 Exercises: 12
• References: 16
• Metrics Information Theory Convergence and Poisson Approximations: 19
• 21 Some Common Metrics and Their Usefulness: 20
• 22 Convergence in Total Variation and Further Useful Formulas: 22
• 23 InformationTheoretic Distances de Bruijns Identity and Relations to Convergence: 24
• 24 Poisson Approximations: 28
• 25 Exercises: 31
• References: 33
• More General Weak and Strong Laws and the Delta Theorem: 35
• 32 Median Centering and Kestens Theorem: 38
• 33 The Ergodic Theorem: 39
• 34 Delta Theorem and Examples: 40
• 35 Approximation of Moments: 44
• 36 Exercises: 45
• References: 47
• Transformations: 49
• 41 VarianceStabilizing Transformations: 50
• 42 Examples: 51
• 43 Bias Correction of the VST: 54
• 44 Symmetrizing Transformations: 57
• 45 VST or Symmetrizing Transform?: 59
• References: 61
• More General Central Limit Theorems: 63
• 52 CLT without a Variance: 66
• 53 Combinatorial CLT: 67
• 54 CLT for Exchangeable Sequences: 68
• 55 CLT for a Random Number of Summands: 70
• 56 Infinite Divisibility and Stable Laws: 71
• 57 Exercises: 77
• References: 80
• Moment Convergence and Uniform Integrability: 83
• 62 The Moment Problem: 85
• 63 Exercises: 88
• Referencesq: 89
• Sample Percentiles and Order Statistics: 91
• 71 Asymptotic Distribution of One Order Statistic: 92
• 72 Joint Asymptotic Distribution of Several Order Statistics: 93
• 73 Bahadur Representations: 94
• 74 Confidence Intervals for Quantiles: 96
• 75 Regression Quantiles: 97
• 76 Exercises: 98
• References: 100
• Sample Extremes: 101
• 82 Characterizations: 105
• 83 Limiting Distribution of the Sample Range: 107
• 84 Multiplicative Strong Law: 108
• 85 Additive Strong Law: 109
• 86 Dependent Sequences: 111
• Sample Extremes: 114
• References: 116
• Central Limit Theorems for Dependent Sequences: 119
• 92 Sampling without Replacement: 121
• 93 Martingales and Examples: 123
• 94 The Martingale and Reverse Martingale CLTs: 126
• References: 129
• Central Limit Theorem for Markov Chains: 131
• 102 Normal Limits: 132
• 103 Nonnormal Limits: 135
• 105 Exercises: 137
• References: 139
• Accuracy of Central Limit Theorems: 141
• BerryEsseen Inequality: 142
• 112 Local Bounds: 144
• 113 The Multidimensional BerryEsseen Theorems: 145
• 114 Other Statistics: 146
• 115 Exercises: 147
• References: 149
• Invariance Principles: 151
• 121 Motivating Examples: 152
• 122 Two Relevant Gaussian Processes: 153
• 123 The ErdösKac Invariance Principle: 156
• 124 Invariance Principles Donskers Theorem and the KMT Construction: 157
• 125 Invariance Principle for Empirical Processes: 161
• 126 Extensions of Donskers Principle and VapnikChervonenkis Classes: 163
• 127 GlivenkoCantelli Theorem for VC Classes: 164
• 128 CLTs for Empirical Measures and Applications: 167
• 1281 Notation and Formulation: 168
• 1282 Entropy Bounds and Specific CLTs: 169
• Martingales Mixing and ShortRange Dependence: 172
• 1210 Weighted Empirical Processes and Approximations: 175
• 1211 Exercises: 178
• References: 180
• Edgeworth Expansions and Cumulants: 185
• 131 Expansion for Means: 186
• 132 Using the Edgeworth Expansion: 188
• 133 Edgeworth Expansion for Sample Percentiles: 189
• 134 Edgeworth Expansion for the 𝒕statistic: 190
• 135 CornishFisher Expansions: 192
• 136 Cumulants and Fishers 𝜿statistics: 194
• 137 Exercises: 198
• References: 200
• Saddlepoint Approximations: 203
• 141 Approximate Evaluation of Integrals: 204
• 142 Density of Means and Exponential Tilting: 208
• 1421 Derivation by Edgeworth Expansion and Exponential Tilting: 210
• 143 Some Examples: 211
• 144 Application to Exponential Family and the Magic Formula: 213
• 146 Edgeworth vs Saddlepoint vs Chisquare Approximation: 217
• 147 Tail Areas for Sample Percentiles: 218
• 148 Quantile Approximation and Inverting the LugannaniRice Formula: 219
• 149 The Multidimensional Case: 221
• 1410 Exercises: 222
• References: 223
• 𝑼statistics: 225
• 151 Examples: 226
• 152 Asymptotic Distribution of 𝑼statistics: 227
• 153 Moments of 𝑼statistics and the Martingale Structure: 229
• 154 Edgeworth Expansions: 230
• 155 Nonnormal Limits: 232
• References: 234
• Maximum Likelihood Estimates: 235
• 162 Inconsistent MLEs: 239
• 163 MLEs in the Exponential Family: 240
• 164 More General Cases and Asymptotic Normality: 242
• 165 Observed and Expected Fisher Information: 244
• 166 Edgeworth Expansions for MLEs: 245
• 167 Asymptotic Optimality of the MLE and Superefficiency: 247
• 168 Ha𝓳ekLeCam Convolution Theorem: 249
• 169 Loss of Information and Efrons Curvature: 251
• 1610 Exercises: 253
• References: 258
• M Estimates: 259
• 171 Examples: 260
• 172 Consistency and Asymptotic Normality: 262
• 173 Bahadur Expansion of M Estimates: 265
• 174 Exercises: 267
• References: 268
• The Trimmed Mean: 271
• 182 Lower Bounds on Efficiencies: 273
• 184 The 10203040 Rule: 275
• 185 Exercises: 277
• References: 278
• Multivariate Location Parameter and Multivariate Medians: 279
• 192 Multivariate Medians: 280
• 193 Asymptotic Theory for Multivariate Medians: 282
• 194 The Asymptotic Covariance Matrix: 283
• 195 Asymptotic Covariance Matrix of the L₁ Median: 284
• 196 Exercises: 287
• References: 288
• Bayes Procedures and Posterior Distributions: 289
• 201 Motivating Examples: 290
• 202 Bernsteinvon Mises Theorem: 291
• 203 Posterior Expansions: 294
• 204 Expansions for Posterior Mean Variance and Percentiles: 298
• 205 The TierneyKadane Approximations: 300
• 206 Frequentist Approximation of Posterior Summaries: 302
• 207 Consistency of Posteriors: 304
• 208 The Difference between Bayes Estimates and the MLE: 305
• 209 Using the Brown Identity to Obtain Bayesian Asymptotics: 306
• 2010 Testing: 311
• 2011 Interval and Set Estimation: 312
• 2012 InfiniteDimensional Problems and the DiaconisFreedman Results: 314
• 2013 Exercises: 317
• References: 320
• 1 Testing Problems: 323
• 212 Examples: 324
• 213 Asymptotic Theory of Likelihood Ratio Test Statistics: 334
• 214 Distribution under Alternatives: 336
• 215 Bartlett Correction: 338
• 216 The Wald and Rao Score Tests: 339
• 217 Likelihood Ratio Confidence Intervals: 340
• 218 Exercises: 342
• References: 344
• Asymptotic Efficiency in Testing: 347
• 221 Pitman Efficiencies: 348
• 222 Bahadur Slopes and Bahadur Efficiency: 353
• 223 Bahadur Slopes of 𝑼statistics: 361
• 224 Exercises: 362
• References: 363
• Some General LargeDeviation Results: 365
• 232 The GärtnerEllis Theorem: 367
• 233 Large Deviation for Local Limit Theorems: 370
• 234 Exercises: 374
• References: 375
• Classical Nonparametrics: 377
• 241 Some Early Illustrative Examples: 378
• 242 Sign Test: 380
• 243 Consistency of the Sign Test: 381
• 244 Wilcoxon SignedRank Test: 383
• 245 Robustness of the 𝑡 Confidence Interval: 388
• 246 The BahadurSavage Theorem: 393
• 247 KolmogorovSmirnov and Anderson Confidence Intervals: 394
• 248 HodgesLehmann Confidence Interval: 396
• 249 Power of the Wilcoxon Test: 397
• 2410 Exercises: 398
• References: 399
• TwoSample Problems: 401
• 251 BehrensFisher Problem: 402
• 252 Wilcoxon Rank Sum and MannWhitney Test: 405
• 253 TwoSample 𝑼statistics and Power Approximations: 408
• 254 Hettmanspergers Generalization: 410
• 255 The Nonparametric BehrensFisher Problem: 412
• 256 Robustness of the MannWhitney Test: 415
• 257 Exercises: 417
• References: 418
• Goodness of Fit: 421
• 261 KolmogorovSmirnov and Other Tests Based on 𝐹𝑛: 422
• 263 Some Heuristics: 423
• 264 Asymptotic Null Distributions of D𝑛 𝐶𝑛 𝐴𝑛 and 𝑉𝑛: 424
• 265 Consistency and Distributions under Alternatives: 425
• 266 Finite Sample Distributions and Other EDFBased Tests: 426
• 267 The BerkJones Procedure: 428
• 268 𝜑Divergences and the JagerWellner Tests: 429
• 269 The TwoSample Case: 431
• 2610 Tests for Normality: 434
• 2611 Exercises: 436
• References: 438
• Chisquare Tests for Goodness of Fit: 441
• 272 Asymptotic Distribution of Pearsons Chisquare: 442
• 274 Choice of 𝒌: 443
• 275 Recommendation of Mann and Wald: 445
• 277 Exercises: 448
• References: 449
• Goodness of Fit with Estimated Parameters: 451
• 281 Preliminary Analysis by Stochastic Expansion: 452
• 282 Asymptotic Distribution of EDFBased Statistics for Composite Nulls: 453
• 283 Chisquare Tests with Estimated Parameters and the ChernoffLehmann Result: 455
• 284 Chisquare Tests with Random Cells: 457
• References: 458
• The Bootstrap: 461
• 291 Bootstrap Distribution and the Meaning of Consistency: 462
• 292 Consistency in the Kolmogorov and Wasserstein Metrics: 464
• 293 Delta Theorem for the Bootstrap: 468
• 295 Other Statistics: 471
• 296 Some Numerical Examples: 473
• 297 Failure of the Bootstrap: 475
• 298 𝑚 out of 𝑛 Bootstrap: 476
• 299 Bootstrap Confidence Intervals: 478
• 2910 Some Numerical Examples: 482
• 2911 Bootstrap Confidence Intervals for Quantiles: 483
• 2913 Residual Bootstrap: 484
• 2914 Confidence Intervals 485: 485
• 2915 Distribution Estimates in Regression: 486
• 2916 Bootstrap for Dependent Data: 487
• 2917 Consistent Bootstrap for Stationary Autoregression: 488
• 2918 Block Bootstrap Methods: 489
• 2919 Optimal Block Length: 491
• 2920 Exercises: 492
• References: 495
• Jackknife: 499
• 302 Bias Correction by the Jackknife: 502
• 303 Variance Estimation: 503
• 304 Delete𝑑 Jackknife and von Mises Functionals: 504
• 305 A Numerical Example: 507
• 306 Jackknife Histogram: 508
• 307 Exercises: 511
• References: 512
• Permutation Tests: 513
• 311 General Permutation Tests and Basic Group Theory: 514
• 312 Exact Similarity of Permutation Tests: 516
• 313 Power of Permutation Tests: 519
• 314 Exercises: 520
• References: 521
• Density Estimation: 523
• 322 Measures of the Quality of Density Estimates: 526
• 324 Minimaxity Criterion: 529
• A Preview: 530
• 326 Rate of Convergence of Histograms: 531
• 327 Consistency of Kernel Estimates: 533
• 328 Order of Optimal Bandwidth and Superkernels: 535
• 329 The Epanechnikov Kernel: 538
• 3210 Choice of Bandwidth by Cross Validation: 539
• 32101 Maximum Likelihood CV: 540
• 32102 Least Squares CV: 542
• 32103 Stones Result: 544
• 3211 Comparison of Bandwidth Selectors and Recommendations: 545
• 3212 𝓛 Optimal Bandwidths: 547
• 3213 Variable Bandwidths: 548
• 3214 Strong Uniform Consistency and Confidence Bands: 550
• 3215 Multivariate Density Estimation and Curse of Dimensionality: 552
• 32151 Kernel Estimates and Optimal Bandwidths: 556
• 3216 Estimating a Unimodal Density and the Grenander Estimate: 558
• 3217 Mode Estimation and Chernoffs Distribution: 561
• 3218 Exercises: 564
• References: 568
• Mixture Models and Nonparametric Deconvolution: 571
• 331 Mixtures as Dense Families: 572
• 332 𝑧 Distributions and Other Gaussian Mixtures as Useful Models: 573
• Finite Mixtures: 577
• 3332 Minimum Distance Method: 578
• 3333 Moment Estimates: 579
• 334 Estimation in General Mixtures: 580
• 335 Strong Consistency and Weak Convergence of the MLE: 582
• 336 Convergence Rates for Finite Mixtures and Nonparametric Deconvolution: 584
• 3361 Nonparametric Deconvolution: 585
• 337 Exercises: 587
• References: 589
• HighDimensional Inference and False Discovery: 593
• 341 Chisquare Tests with Many Cells and Sparse Multinomials: 594
• The Portnoy Paradigm: 597
• Early Developments: 599
• Definitions Control and the BenjaminiHochberg Rule: 601
• 345 Distribution Theory for False Discoveries and Poisson and FirstPassage Asymptotics: 604
• 346 Newer FDR Controlling Procedures: 606
• 347 Higher Criticism and the DonohoJin Developments: 608
• 348 False Nondiscovery and Decision Theory Formulation: 611
• 3481 GenoveseWasserman Procedure: 612
• 349 Asymptotic Expansions: 614
• 3410 Lower Bounds on the Number of False Hypotheses: 616
• 34101 BühlmannMeinshausenRice Method: 617
• 3411 The Dependent Case and the HallJin Results: 620
• HallJin Results: 623
• 3412 Exercises: 625
• References: 628
• A Collection of Inequalities in Probability Linear Algebra and Analysis: 633
• 3512 Concentration Inequalities: 634
• 3513 Tail Inequalities for Specific Distributions: 639
• 3514 Inequalities under Unimodality: 641
• 3515 Moment and Monotonicity Inequalities: 643
• 3516 Inequalities in Order Statistics: 652
• 3517 Inequalities for Normal Distributions: 655
• 3518 Inequalities for Binomial and Poisson Distributions: 656
• 3519 Inequalities in the Central Limit Theorem: 658
• 35110 Martingale Inequalities: 661
• 352 Matrix Inequalities: 663
• 3522 Eigenvalue and Quadratic Form Inequalities: 667
• 353 Series and Polynomial Inequalities: 671
• 354 Integral and Derivative Inequalities: 675
• Glossary of Symbols: 689
• Index: 693

Bibliographic information

• Title: Asymptotic Theory of Statistics and Probability; Springer Texts in Statistics
• Author: Anirban DasGupta
• Edition: illustrated
• Publisher: Springer Science & Business Media, 2008
• ISBN: 0387759700, 9780387759708
• Length: 722 pages
• Subjects: Mathematics / Probability & Statistics / General; Mathematics / Probability & Statistics / Stochastic Processes