# Asymptotic Theory of Statistics and Probability

Springer Science & Business Media, Mar 7, 2008 - Mathematics - 722 pages
This book developed out of my year-long course on asymptotic theory at Purdue University. To some extent, the topics coincide with what I cover in that course. There are already a number of well-known books on asy- totics. This book is quite different. It covers more topics in one source than areavailableinanyothersinglebookonasymptotictheory. Numeroustopics covered in this book are available in the literature in a scattered manner, and they are brought together under one umbrella in this book. Asymptotic theory is a central unifying theme in probability and statistics. My main goal in writing this book is to give its readers a feel for the incredible scope and reach of asymptotics. I have tried to write this book in a way that is accessible and to make the reader appreciate the beauty of theory and the insights that only theory can provide. Essentially every theorem in the book comes with at least one reference, preceding or following the statement of the theorem. In addition, I have p- vided a separate theorem-by-theorem reference as an entry on its own in the front of the book to make it extremely convenient for the reader to ?nd a proof that was not provided in the text. Also particularly worth mentioning is a collection of nearly 300 practically useful inequalities that I have c- lected together from numerous sources. This is appended at the very end of the book.

### What people are saying -Write a review

We haven't found any reviews in the usual places.

### Contents

 Basic Convergence Concepts and Theorems 1 12 Three Series Theorem and Kolmogorovs ZeroOne Law 6 13 Central Limit Theorem and Law of the Iterated Logarithm 7 14 Further Illustrative Examples 10 15 Exercises 12 References 16 Metrics Information Theory Convergence and Poisson Approximations 19 21 Some Common Metrics and Their Usefulness 20
 215 Bartlett Correction 338 216 The Wald and Rao Score Tests 339 217 Likelihood Ratio Conﬁdence Intervals 340 218 Exercises 342 References 344 Asymptotic Efficiency in Testing 347 221 Pitman Efficiencies 348 222 Bahadur Slopes and Bahadur Efficiency 353

 22 Convergence in Total Variation and Further Useful Formulas 22 23 InformationTheoretic Distances de Bruijns Identity and Relations to Convergence 24 24 Poisson Approximations 28 25 Exercises 31 References 33 More General Weak and Strong Laws and the Delta Theorem 35 32 Median Centering and Kestens Theorem 38 33 The Ergodic Theorem 39 34 Delta Theorem and Examples 40 35 Approximation of Moments 44 36 Exercises 45 References 47 Transformations 49 41 VarianceStabilizing Transformations 50 42 Examples 51 43 Bias Correction of the VST 54 44 Symmetrizing Transformations 57 45 VST or Symmetrizing Transform? 59 References 61 More General Central Limit Theorems 63 52 CLT without a Variance 66 53 Combinatorial CLT 67 54 CLT for Exchangeable Sequences 68 55 CLT for a Random Number of Summands 70 56 Inﬁnite Divisibility and Stable Laws 71 57 Exercises 77 References 80 Moment Convergence and Uniform Integrability 83 62 The Moment Problem 85 63 Exercises 88 Referencesq 89 Sample Percentiles and Order Statistics 91 71 Asymptotic Distribution of One Order Statistic 92 72 Joint Asymptotic Distribution of Several Order Statistics 93 73 Bahadur Representations 94 74 Conﬁdence Intervals for Quantiles 96 75 Regression Quantiles 97 76 Exercises 98 References 100 Sample Extremes 101 82 Characterizations 105 83 Limiting Distribution of the Sample Range 107 84 Multiplicative Strong Law 108 85 Additive Strong Law 109 86 Dependent Sequences 111 Sample Extremes 114 References 116 Central Limit Theorems for Dependent Sequences 119 92 Sampling without Replacement 121 93 Martingales and Examples 123 94 The Martingale and Reverse Martingale CLTs 126 References 129 Central Limit Theorem for Markov Chains 131 102 Normal Limits 132 103 Nonnormal Limits 135 105 Exercises 137 References 139 Accuracy of Central Limit Theorems 141 BerryEsseen Inequality 142 112 Local Bounds 144 113 The Multidimensional BerryEsseen Theorems 145 114 Other Statistics 146 115 Exercises 147 References 149 Invariance Principles 151 121 Motivating Examples 152 122 Two Relevant Gaussian Processes 153 123 The ErdösKac Invariance Principle 156 124 Invariance Principles Donskers Theorem and the KMT Construction 157 125 Invariance Principle for Empirical Processes 161 126 Extensions of Donskers Principle and VapnikChervonenkis Classes 163 127 GlivenkoCantelli Theorem for VC Classes 164 128 CLTs for Empirical Measures and Applications 167 1281 Notation and Formulation 168 1282 Entropy Bounds and Speciﬁc CLTs 169 Martingales Mixing and ShortRange Dependence 172 1210 Weighted Empirical Processes and Approximations 175 1211 Exercises 178 References 180 Edgeworth Expansions and Cumulants 185 131 Expansion for Means 186 132 Using the Edgeworth Expansion 188 133 Edgeworth Expansion for Sample Percentiles 189 134 Edgeworth Expansion for the 𝒕statistic 190 135 CornishFisher Expansions 192 136 Cumulants and Fishers 𝜿statistics 194 137 Exercises 198 References 200 Saddlepoint Approximations 203 141 Approximate Evaluation of Integrals 204 142 Density of Means and Exponential Tilting 208 1421 Derivation by Edgeworth Expansion and Exponential Tilting 210 143 Some Examples 211 144 Application to Exponential Family and the Magic Formula 213 146 Edgeworth vs Saddlepoint vs Chisquare Approximation 217 147 Tail Areas for Sample Percentiles 218 148 Quantile Approximation and Inverting the LugannaniRice Formula 219 149 The Multidimensional Case 221 1410 Exercises 222 References 223 𝑼statistics 225 151 Examples 226 152 Asymptotic Distribution of 𝑼statistics 227 153 Moments of 𝑼statistics and the Martingale Structure 229 154 Edgeworth Expansions 230 155 Nonnormal Limits 232 References 234 Maximum Likelihood Estimates 235 162 Inconsistent MLEs 239 163 MLEs in the Exponential Family 240 164 More General Cases and Asymptotic Normality 242 165 Observed and Expected Fisher Information 244 166 Edgeworth Expansions for MLEs 245 167 Asymptotic Optimality of the MLE and Superefficiency 247 168 Ha𝓳ekLeCam Convolution Theorem 249 169 Loss of Information and Efrons Curvature 251 1610 Exercises 253 References 258 M Estimates 259 171 Examples 260 172 Consistency and Asymptotic Normality 262 173 Bahadur Expansion of M Estimates 265 174 Exercises 267 References 268 The Trimmed Mean 271 182 Lower Bounds on Efﬁciencies 273 184 The 10203040 Rule 275 185 Exercises 277 References 278 Multivariate Location Parameter and Multivariate Medians 279 192 Multivariate Medians 280 193 Asymptotic Theory for Multivariate Medians 282 194 The Asymptotic Covariance Matrix 283 195 Asymptotic Covariance Matrix of the L₁ Median 284 196 Exercises 287 References 288 Bayes Procedures and Posterior Distributions 289 201 Motivating Examples 290 202 Bernsteinvon Mises Theorem 291 203 Posterior Expansions 294 204 Expansions for Posterior Mean Variance and Percentiles 298 205 The TierneyKadane Approximations 300 206 Frequentist Approximation of Posterior Summaries 302 207 Consistency of Posteriors 304 208 The Difference between Bayes Estimates and the MLE 305 209 Using the Brown Identity to Obtain Bayesian Asymptotics 306 2010 Testing 311 2011 Interval and Set Estimation 312 2012 InﬁniteDimensional Problems and the DiaconisFreedman Results 314 2013 Exercises 317 References 320 1 Testing Problems 323 212 Examples 324 213 Asymptotic Theory of Likelihood Ratio Test Statistics 334 214 Distribution under Alternatives 336
 223 Bahadur Slopes of 𝑼statistics 361 224 Exercises 362 References 363 Some General LargeDeviation Results 365 232 The GärtnerEllis Theorem 367 233 Large Deviation for Local Limit Theorems 370 234 Exercises 374 References 375 Classical Nonparametrics 377 241 Some Early Illustrative Examples 378 242 Sign Test 380 243 Consistency of the Sign Test 381 244 Wilcoxon SignedRank Test 383 245 Robustness of the 𝑡 Confidence Interval 388 246 The BahadurSavage Theorem 393 247 KolmogorovSmirnov and Anderson Conﬁdence Intervals 394 248 HodgesLehmann Conﬁdence Interval 396 249 Power of the Wilcoxon Test 397 2410 Exercises 398 References 399 TwoSample Problems 401 251 BehrensFisher Problem 402 252 Wilcoxon Rank Sum and MannWhitney Test 405 253 TwoSample 𝑼statistics and Power Approximations 408 254 Hettmanspergers Generalization 410 255 The Nonparametric BehrensFisher Problem 412 256 Robustness of the MannWhitney Test 415 257 Exercises 417 References 418 Goodness of Fit 421 261 KolmogorovSmirnov and Other Tests Based on 𝐹𝑛 422 263 Some Heuristics 423 264 Asymptotic Null Distributions of D𝑛 𝐶𝑛 𝐴𝑛 and 𝑉𝑛 424 265 Consistency and Distributions under Alternatives 425 266 Finite Sample Distributions and Other EDFBased Tests 426 267 The BerkJones Procedure 428 268 𝜑Divergences and the JagerWellner Tests 429 269 The TwoSample Case 431 2610 Tests for Normality 434 2611 Exercises 436 References 438 Chisquare Tests for Goodness of Fit 441 272 Asymptotic Distribution of Pearsons Chisquare 442 274 Choice of 𝒌 443 275 Recommendation of Mann and Wald 445 277 Exercises 448 References 449 Goodness of Fit with Estimated Parameters 451 281 Preliminary Analysis by Stochastic Expansion 452 282 Asymptotic Distribution of EDFBased Statistics for Composite Nulls 453 283 Chisquare Tests with Estimated Parameters and the ChernoffLehmann Result 455 284 Chisquare Tests with Random Cells 457 References 458 The Bootstrap 461 291 Bootstrap Distribution and the Meaning of Consistency 462 292 Consistency in the Kolmogorov and Wasserstein Metrics 464 293 Delta Theorem for the Bootstrap 468 295 Other Statistics 471 296 Some Numerical Examples 473 297 Failure of the Bootstrap 475 298 𝑚 out of 𝑛 Bootstrap 476 299 Bootstrap Conﬁdence Intervals 478 2910 Some Numerical Examples 482 2911 Bootstrap Conﬁdence Intervals for Quantiles 483 2913 Residual Bootstrap 484 2914 Confidence Intervals 485 485 2915 Distribution Estimates in Regression 486 2916 Bootstrap for Dependent Data 487 2917 Consistent Bootstrap for Stationary Autoregression 488 2918 Block Bootstrap Methods 489 2919 Optimal Block Length 491 2920 Exercises 492 References 495 Jackknife 499 302 Bias Correction by the Jackknife 502 303 Variance Estimation 503 304 Delete𝑑 Jackknife and von Mises Functionals 504 305 A Numerical Example 507 306 Jackknife Histogram 508 307 Exercises 511 References 512 Permutation Tests 513 311 General Permutation Tests and Basic Group Theory 514 312 Exact Similarity of Permutation Tests 516 313 Power of Permutation Tests 519 314 Exercises 520 References 521 Density Estimation 523 322 Measures of the Quality of Density Estimates 526 324 Minimaxity Criterion 529 A Preview 530 326 Rate of Convergence of Histograms 531 327 Consistency of Kernel Estimates 533 328 Order of Optimal Bandwidth and Superkernels 535 329 The Epanechnikov Kernel 538 3210 Choice of Bandwidth by Cross Validation 539 32101 Maximum Likelihood CV 540 32102 Least Squares CV 542 32103 Stones Result 544 3211 Comparison of Bandwidth Selectors and Recommendations 545 3212 𝓛 Optimal Bandwidths 547 3213 Variable Bandwidths 548 3214 Strong Uniform Consistency and Conﬁdence Bands 550 3215 Multivariate Density Estimation and Curse of Dimensionality 552 32151 Kernel Estimates and Optimal Bandwidths 556 3216 Estimating a Unimodal Density and the Grenander Estimate 558 3217 Mode Estimation and Chernoffs Distribution 561 3218 Exercises 564 References 568 Mixture Models and Nonparametric Deconvolution 571 331 Mixtures as Dense Families 572 332 𝑧 Distributions and Other Gaussian Mixtures as Useful Models 573 Finite Mixtures 577 3332 Minimum Distance Method 578 3333 Moment Estimates 579 334 Estimation in General Mixtures 580 335 Strong Consistency and Weak Convergence of the MLE 582 336 Convergence Rates for Finite Mixtures and Nonparametric Deconvolution 584 3361 Nonparametric Deconvolution 585 337 Exercises 587 References 589 HighDimensional Inference and False Discovery 593 341 Chisquare Tests with Many Cells and Sparse Multinomials 594 The Portnoy Paradigm 597 Early Developments 599 Deﬁnitions Control and the BenjaminiHochberg Rule 601 345 Distribution Theory for False Discoveries and Poisson and FirstPassage Asymptotics 604 346 Newer FDR Controlling Procedures 606 347 Higher Criticism and the DonohoJin Developments 608 348 False Nondiscovery and Decision Theory Formulation 611 3481 GenoveseWasserman Procedure 612 349 Asymptotic Expansions 614 3410 Lower Bounds on the Number of False Hypotheses 616 34101 BühlmannMeinshausenRice Method 617 3411 The Dependent Case and the HallJin Results 620 HallJin Results 623 3412 Exercises 625 References 628 A Collection of Inequalities in Probability Linear Algebra and Analysis 633 3512 Concentration Inequalities 634 3513 Tail Inequalities for Specific Distributions 639 3514 Inequalities under Unimodality 641 3515 Moment and Monotonicity Inequalities 643 3516 Inequalities in Order Statistics 652 3517 Inequalities for Normal Distributions 655 3518 Inequalities for Binomial and Poisson Distributions 656 3519 Inequalities in the Central Limit Theorem 658 35110 Martingale Inequalities 661 352 Matrix Inequalities 663 3522 Eigenvalue and Quadratic Form Inequalities 667 353 Series and Polynomial Inequalities 671 354 Integral and Derivative Inequalities 675 Glossary of Symbols 689 Index 693 Copyright