## Asymptotic Theory of Statistics and ProbabilityThis book is an encyclopedic treatment of classic as well as contemporary large sample theory, dealing with both statistical problems and probabilistic issues and tools. It is written in an extremely lucid style, with an emphasis on the conceptual discussion of the importance of a problem and the impact and relevance of the theorems. The book has 34 chapters over a wide range of topics, nearly 600 exercises for practice and instruction, and another 300 worked out examples. It also includes a large compendium of 300 useful inequalities on probability, linear algebra, and analysis that are collected together from numerous sources, as an invaluable reference for researchers in statistics, probability, and mathematics. It can be used as a graduate text, as a versatile research reference, as a source for independent reading on a wide assembly of topics, and as a window to learning the latest developments in contemporary topics. The book is unique in its detailed coverage of fundamental topics such as central limit theorems in numerous setups, likelihood based methods, goodness of fit, higher order asymptotics, as well as of the most modern topics such as the bootstrap, dependent data, Bayesian asymptotics, nonparametric density estimation, mixture models, and multiple testing and false discovery. It provides extensive bibliographic references on all topics that include very recent publications. Anirban DasGupta is Professor of Statistics at Purdue University. He has also taught at the Wharton School of the University of Pennsylvania, at Cornell University, and at the University of California at San Diego. He has been on the editorial board of the Annals of Statistics since 1998 and has also served on the editorial boards of the Journal of the American Statistical Association, International Statistical Review, and the Journal of Statistical Planning and Inference. He has edited two monographs in the lecture notes monograph series of the Institute of Mathematical Statistics, is a Fellow of the Institute of Mathematical Statistics and has 70 refereed publications on theoretical statistics and probability in major journals. |

### What people are saying - Write a review

### Contents

Basic Convergence Concepts and Theorems | 1 |

12 Three Series Theorem and Kolmogorovs ZeroOne Law | 6 |

13 Central Limit Theorem and Law of the Iterated Logarithm | 7 |

14 Further Illustrative Examples | 10 |

15 Exercises | 12 |

References | 16 |

Metrics Information Theory Convergence and Poisson Approximations | 19 |

21 Some Common Metrics and Their Usefulness | 20 |

215 Bartlett Correction | 338 |

216 The Wald and Rao Score Tests | 339 |

217 Likelihood Ratio Conﬁdence Intervals | 340 |

218 Exercises | 342 |

References | 344 |

Asymptotic Efficiency in Testing | 347 |

221 Pitman Efficiencies | 348 |

222 Bahadur Slopes and Bahadur Efficiency | 353 |

22 Convergence in Total Variation and Further Useful Formulas | 22 |

23 InformationTheoretic Distances de Bruijns Identity and Relations to Convergence | 24 |

24 Poisson Approximations | 28 |

25 Exercises | 31 |

References | 33 |

More General Weak and Strong Laws and the Delta Theorem | 35 |

32 Median Centering and Kestens Theorem | 38 |

33 The Ergodic Theorem | 39 |

34 Delta Theorem and Examples | 40 |

35 Approximation of Moments | 44 |

36 Exercises | 45 |

References | 47 |

Transformations | 49 |

41 VarianceStabilizing Transformations | 50 |

42 Examples | 51 |

43 Bias Correction of the VST | 54 |

44 Symmetrizing Transformations | 57 |

45 VST or Symmetrizing Transform? | 59 |

References | 61 |

More General Central Limit Theorems | 63 |

52 CLT without a Variance | 66 |

53 Combinatorial CLT | 67 |

54 CLT for Exchangeable Sequences | 68 |

55 CLT for a Random Number of Summands | 70 |

56 Inﬁnite Divisibility and Stable Laws | 71 |

57 Exercises | 77 |

References | 80 |

Moment Convergence and Uniform Integrability | 83 |

62 The Moment Problem | 85 |

63 Exercises | 88 |

Referencesq | 89 |

Sample Percentiles and Order Statistics | 91 |

71 Asymptotic Distribution of One Order Statistic | 92 |

72 Joint Asymptotic Distribution of Several Order Statistics | 93 |

73 Bahadur Representations | 94 |

74 Conﬁdence Intervals for Quantiles | 96 |

75 Regression Quantiles | 97 |

76 Exercises | 98 |

References | 100 |

Sample Extremes | 101 |

82 Characterizations | 105 |

83 Limiting Distribution of the Sample Range | 107 |

84 Multiplicative Strong Law | 108 |

85 Additive Strong Law | 109 |

86 Dependent Sequences | 111 |

Sample Extremes | 114 |

References | 116 |

Central Limit Theorems for Dependent Sequences | 119 |

92 Sampling without Replacement | 121 |

93 Martingales and Examples | 123 |

94 The Martingale and Reverse Martingale CLTs | 126 |

References | 129 |

Central Limit Theorem for Markov Chains | 131 |

102 Normal Limits | 132 |

103 Nonnormal Limits | 135 |

105 Exercises | 137 |

References | 139 |

Accuracy of Central Limit Theorems | 141 |

BerryEsseen Inequality | 142 |

112 Local Bounds | 144 |

113 The Multidimensional BerryEsseen Theorems | 145 |

114 Other Statistics | 146 |

115 Exercises | 147 |

References | 149 |

Invariance Principles | 151 |

121 Motivating Examples | 152 |

122 Two Relevant Gaussian Processes | 153 |

123 The ErdösKac Invariance Principle | 156 |

124 Invariance Principles Donskers Theorem and the KMT Construction | 157 |

125 Invariance Principle for Empirical Processes | 161 |

126 Extensions of Donskers Principle and VapnikChervonenkis Classes | 163 |

127 GlivenkoCantelli Theorem for VC Classes | 164 |

128 CLTs for Empirical Measures and Applications | 167 |

1281 Notation and Formulation | 168 |

1282 Entropy Bounds and Speciﬁc CLTs | 169 |

Martingales Mixing and ShortRange Dependence | 172 |

1210 Weighted Empirical Processes and Approximations | 175 |

1211 Exercises | 178 |

References | 180 |

Edgeworth Expansions and Cumulants | 185 |

131 Expansion for Means | 186 |

132 Using the Edgeworth Expansion | 188 |

133 Edgeworth Expansion for Sample Percentiles | 189 |

134 Edgeworth Expansion for the 𝒕statistic | 190 |

135 CornishFisher Expansions | 192 |

136 Cumulants and Fishers 𝜿statistics | 194 |

137 Exercises | 198 |

References | 200 |

Saddlepoint Approximations | 203 |

141 Approximate Evaluation of Integrals | 204 |

142 Density of Means and Exponential Tilting | 208 |

1421 Derivation by Edgeworth Expansion and Exponential Tilting | 210 |

143 Some Examples | 211 |

144 Application to Exponential Family and the Magic Formula | 213 |

146 Edgeworth vs Saddlepoint vs Chisquare Approximation | 217 |

147 Tail Areas for Sample Percentiles | 218 |

148 Quantile Approximation and Inverting the LugannaniRice Formula | 219 |

149 The Multidimensional Case | 221 |

1410 Exercises | 222 |

References | 223 |

𝑼statistics | 225 |

151 Examples | 226 |

152 Asymptotic Distribution of 𝑼statistics | 227 |

153 Moments of 𝑼statistics and the Martingale Structure | 229 |

154 Edgeworth Expansions | 230 |

155 Nonnormal Limits | 232 |

References | 234 |

Maximum Likelihood Estimates | 235 |

162 Inconsistent MLEs | 239 |

163 MLEs in the Exponential Family | 240 |

164 More General Cases and Asymptotic Normality | 242 |

165 Observed and Expected Fisher Information | 244 |

166 Edgeworth Expansions for MLEs | 245 |

167 Asymptotic Optimality of the MLE and Superefficiency | 247 |

168 Ha𝓳ekLeCam Convolution Theorem | 249 |

169 Loss of Information and Efrons Curvature | 251 |

1610 Exercises | 253 |

References | 258 |

M Estimates | 259 |

171 Examples | 260 |

172 Consistency and Asymptotic Normality | 262 |

173 Bahadur Expansion of M Estimates | 265 |

174 Exercises | 267 |

References | 268 |

The Trimmed Mean | 271 |

182 Lower Bounds on Efﬁciencies | 273 |

184 The 10203040 Rule | 275 |

185 Exercises | 277 |

References | 278 |

Multivariate Location Parameter and Multivariate Medians | 279 |

192 Multivariate Medians | 280 |

193 Asymptotic Theory for Multivariate Medians | 282 |

194 The Asymptotic Covariance Matrix | 283 |

195 Asymptotic Covariance Matrix of the L₁ Median | 284 |

196 Exercises | 287 |

References | 288 |

Bayes Procedures and Posterior Distributions | 289 |

201 Motivating Examples | 290 |

202 Bernsteinvon Mises Theorem | 291 |

203 Posterior Expansions | 294 |

204 Expansions for Posterior Mean Variance and Percentiles | 298 |

205 The TierneyKadane Approximations | 300 |

206 Frequentist Approximation of Posterior Summaries | 302 |

207 Consistency of Posteriors | 304 |

208 The Difference between Bayes Estimates and the MLE | 305 |

209 Using the Brown Identity to Obtain Bayesian Asymptotics | 306 |

2010 Testing | 311 |

2011 Interval and Set Estimation | 312 |

2012 InﬁniteDimensional Problems and the DiaconisFreedman Results | 314 |

2013 Exercises | 317 |

References | 320 |

1 Testing Problems | 323 |

212 Examples | 324 |

213 Asymptotic Theory of Likelihood Ratio Test Statistics | 334 |

214 Distribution under Alternatives | 336 |

223 Bahadur Slopes of 𝑼statistics | 361 |

224 Exercises | 362 |

References | 363 |

Some General LargeDeviation Results | 365 |

232 The GärtnerEllis Theorem | 367 |

233 Large Deviation for Local Limit Theorems | 370 |

234 Exercises | 374 |

References | 375 |

Classical Nonparametrics | 377 |

241 Some Early Illustrative Examples | 378 |

242 Sign Test | 380 |

243 Consistency of the Sign Test | 381 |

244 Wilcoxon SignedRank Test | 383 |

245 Robustness of the 𝑡 Confidence Interval | 388 |

246 The BahadurSavage Theorem | 393 |

247 KolmogorovSmirnov and Anderson Conﬁdence Intervals | 394 |

248 HodgesLehmann Conﬁdence Interval | 396 |

249 Power of the Wilcoxon Test | 397 |

2410 Exercises | 398 |

References | 399 |

TwoSample Problems | 401 |

251 BehrensFisher Problem | 402 |

252 Wilcoxon Rank Sum and MannWhitney Test | 405 |

253 TwoSample 𝑼statistics and Power Approximations | 408 |

254 Hettmanspergers Generalization | 410 |

255 The Nonparametric BehrensFisher Problem | 412 |

256 Robustness of the MannWhitney Test | 415 |

257 Exercises | 417 |

References | 418 |

Goodness of Fit | 421 |

261 KolmogorovSmirnov and Other Tests Based on 𝐹𝑛 | 422 |

263 Some Heuristics | 423 |

264 Asymptotic Null Distributions of D𝑛 𝐶𝑛 𝐴𝑛 and 𝑉𝑛 | 424 |

265 Consistency and Distributions under Alternatives | 425 |

266 Finite Sample Distributions and Other EDFBased Tests | 426 |

267 The BerkJones Procedure | 428 |

268 𝜑Divergences and the JagerWellner Tests | 429 |

269 The TwoSample Case | 431 |

2610 Tests for Normality | 434 |

2611 Exercises | 436 |

References | 438 |

Chisquare Tests for Goodness of Fit | 441 |

272 Asymptotic Distribution of Pearsons Chisquare | 442 |

274 Choice of 𝒌 | 443 |

275 Recommendation of Mann and Wald | 445 |

277 Exercises | 448 |

References | 449 |

Goodness of Fit with Estimated Parameters | 451 |

281 Preliminary Analysis by Stochastic Expansion | 452 |

282 Asymptotic Distribution of EDFBased Statistics for Composite Nulls | 453 |

283 Chisquare Tests with Estimated Parameters and the ChernoffLehmann Result | 455 |

284 Chisquare Tests with Random Cells | 457 |

References | 458 |

The Bootstrap | 461 |

291 Bootstrap Distribution and the Meaning of Consistency | 462 |

292 Consistency in the Kolmogorov and Wasserstein Metrics | 464 |

293 Delta Theorem for the Bootstrap | 468 |

295 Other Statistics | 471 |

296 Some Numerical Examples | 473 |

297 Failure of the Bootstrap | 475 |

298 𝑚 out of 𝑛 Bootstrap | 476 |

299 Bootstrap Conﬁdence Intervals | 478 |

2910 Some Numerical Examples | 482 |

2911 Bootstrap Conﬁdence Intervals for Quantiles | 483 |

2913 Residual Bootstrap | 484 |

2914 Confidence Intervals 485 | 485 |

2915 Distribution Estimates in Regression | 486 |

2916 Bootstrap for Dependent Data | 487 |

2917 Consistent Bootstrap for Stationary Autoregression | 488 |

2918 Block Bootstrap Methods | 489 |

2919 Optimal Block Length | 491 |

2920 Exercises | 492 |

References | 495 |

Jackknife | 499 |

302 Bias Correction by the Jackknife | 502 |

303 Variance Estimation | 503 |

304 Delete𝑑 Jackknife and von Mises Functionals | 504 |

305 A Numerical Example | 507 |

306 Jackknife Histogram | 508 |

307 Exercises | 511 |

References | 512 |

Permutation Tests | 513 |

311 General Permutation Tests and Basic Group Theory | 514 |

312 Exact Similarity of Permutation Tests | 516 |

313 Power of Permutation Tests | 519 |

314 Exercises | 520 |

References | 521 |

Density Estimation | 523 |

322 Measures of the Quality of Density Estimates | 526 |

324 Minimaxity Criterion | 529 |

A Preview | 530 |

326 Rate of Convergence of Histograms | 531 |

327 Consistency of Kernel Estimates | 533 |

328 Order of Optimal Bandwidth and Superkernels | 535 |

329 The Epanechnikov Kernel | 538 |

3210 Choice of Bandwidth by Cross Validation | 539 |

32101 Maximum Likelihood CV | 540 |

32102 Least Squares CV | 542 |

32103 Stones Result | 544 |

3211 Comparison of Bandwidth Selectors and Recommendations | 545 |

3212 𝓛 Optimal Bandwidths | 547 |

3213 Variable Bandwidths | 548 |

3214 Strong Uniform Consistency and Conﬁdence Bands | 550 |

3215 Multivariate Density Estimation and Curse of Dimensionality | 552 |

32151 Kernel Estimates and Optimal Bandwidths | 556 |

3216 Estimating a Unimodal Density and the Grenander Estimate | 558 |

3217 Mode Estimation and Chernoffs Distribution | 561 |

3218 Exercises | 564 |

References | 568 |

Mixture Models and Nonparametric Deconvolution | 571 |

331 Mixtures as Dense Families | 572 |

332 𝑧 Distributions and Other Gaussian Mixtures as Useful Models | 573 |

Finite Mixtures | 577 |

3332 Minimum Distance Method | 578 |

3333 Moment Estimates | 579 |

334 Estimation in General Mixtures | 580 |

335 Strong Consistency and Weak Convergence of the MLE | 582 |

336 Convergence Rates for Finite Mixtures and Nonparametric Deconvolution | 584 |

3361 Nonparametric Deconvolution | 585 |

337 Exercises | 587 |

References | 589 |

HighDimensional Inference and False Discovery | 593 |

341 Chisquare Tests with Many Cells and Sparse Multinomials | 594 |

The Portnoy Paradigm | 597 |

Early Developments | 599 |

Deﬁnitions Control and the BenjaminiHochberg Rule | 601 |

345 Distribution Theory for False Discoveries and Poisson and FirstPassage Asymptotics | 604 |

346 Newer FDR Controlling Procedures | 606 |

347 Higher Criticism and the DonohoJin Developments | 608 |

348 False Nondiscovery and Decision Theory Formulation | 611 |

3481 GenoveseWasserman Procedure | 612 |

349 Asymptotic Expansions | 614 |

3410 Lower Bounds on the Number of False Hypotheses | 616 |

34101 BühlmannMeinshausenRice Method | 617 |

3411 The Dependent Case and the HallJin Results | 620 |

HallJin Results | 623 |

3412 Exercises | 625 |

References | 628 |

A Collection of Inequalities in Probability Linear Algebra and Analysis | 633 |

3512 Concentration Inequalities | 634 |

3513 Tail Inequalities for Specific Distributions | 639 |

3514 Inequalities under Unimodality | 641 |

3515 Moment and Monotonicity Inequalities | 643 |

3516 Inequalities in Order Statistics | 652 |

3517 Inequalities for Normal Distributions | 655 |

3518 Inequalities for Binomial and Poisson Distributions | 656 |

3519 Inequalities in the Central Limit Theorem | 658 |

35110 Martingale Inequalities | 661 |

352 Matrix Inequalities | 663 |

3522 Eigenvalue and Quadratic Form Inequalities | 667 |

353 Series and Polynomial Inequalities | 671 |

354 Integral and Derivative Inequalities | 675 |

Glossary of Symbols | 689 |

693 | |