Numerical Methods of StatisticsThis book explains how computer software is designed to perform the tasks required for sophisticated statistical analysis. For statisticians, it examines the nitty-gritty computational problems behind statistical methods. For mathematicians and computer scientists, it looks at the application of mathematical tools to statistical problems. The first half of the book offers a basic background in numerical analysis that emphasizes issues important to statisticians. The next several chapters cover a broad array of statistical tools, such as maximum likelihood and nonlinear regression. The author also treats the application of numerical tools; numerical integration and random number generation are explained in a unified manner reflecting complementary views of Monte Carlo methods. The book concludes with an examination of sorting, FFT and the application of other "fast" algorithms to statistics. Each chapter contains exercises that range in difficulty as well as examples of the methods at work. Most of the examples are accompanied by demonstration code available from the author's home page. |
Contents
Algorithms and Computers | 1 |
12 Computers | 3 |
13 Software and Computer Languages | 5 |
14 Data Structures | 8 |
15 Programming Practice | 9 |
References | 10 |
Computer Arithmetic | 11 |
22 Positional Number Systems | 12 |
Numerical Differentiation | 184 |
87 Minimization and Nonlinear Equations | 187 |
88 Condition and Scaling | 192 |
89 Implementation | 193 |
Programs and Demonstrations | 194 |
Exercises | 196 |
References | 198 |
Maximum Likelihood and Nonlinear Regression | 199 |
23 Fixed Point Arithmetic | 15 |
24 Floating Point Representations | 18 |
25 Living with Floating Point Inaccuracies | 21 |
26 The Pale and Beyond | 26 |
27 Conditioned Problems and Stable Algorithms | 30 |
Programs and Demonstrations | 32 |
Exercises | 33 |
References | 35 |
Matrices and Linear Equations | 37 |
32 Matrix Operations | 38 |
33 Solving Triangular Systems | 40 |
34 Gaussian Elimination | 41 |
35 Cholesky Decomposition | 47 |
36 Matrix Norms | 50 |
37 Accuracy and Conditioning | 52 |
Programs and Demonstrations | 56 |
Exercises | 58 |
References | 59 |
More Methods for Solving Linear Equations | 61 |
43 Banded Matrices | 65 |
44 Applications to ARMA TimeSeries Models | 67 |
45 Toeplitz Systems | 70 |
46 Sparse Matrices | 73 |
47 Iterative Methods | 76 |
Programs and Demonstrations | 78 |
References | 80 |
Regression Computations | 82 |
52 Condition of the Regression Problem | 84 |
53 Solving the Normal Equations | 87 |
54 GramSchmidt Orthogonalization | 88 |
55 Householder Transformations | 91 |
56 Householder Transformations for Least Squares | 92 |
57 Givens Transformations | 95 |
59 Regression Diagnostics | 98 |
510 Hypothesis Tests | 100 |
511 Conjugate Gradient Methods | 103 |
512 Doolittle the Sweep and AH Possible Regressions | 106 |
513 Comments | 108 |
Exercises | 109 |
References | 111 |
Eigenproblems | 114 |
63 Power Methods | 116 |
64 The Symmetric Eigenproblem and Tridiagonalization | 119 |
65 The QR Algorithm | 121 |
66 Singular Value Decomposition | 123 |
67 Applications | 126 |
68 Complex Singular Value Decomposition | 130 |
Programs and Demonstrations | 132 |
Exercises | 133 |
References | 136 |
Functions Interpolation Smoothing and Approximation | 137 |
72 Interpolation | 139 |
73 Interpolating Splines | 142 |
Smoothing and Regression | 145 |
75 Mathematical Approximation | 148 |
76 Practical Approximation Techniques | 152 |
77 Computing Probability Functions | 155 |
Programs and Demonstrations | 162 |
Exercises | 164 |
References | 168 |
Introduction to Optimization and Nonlinear Equations | 170 |
Lattice Search Golden Section and Bisection | 172 |
83 Root Finding | 175 |
Stopping and Condition | 181 |
85 Multivariate Newtons Methods | 183 |
92 Notation and Asymptotic Theory of Maximum Likelihood | 200 |
93 Information Scoring and Variance Estimates | 206 |
94 An Extended Example | 208 |
95 Concentration Iteration and the EM Algorithm | 210 |
96 Multiple Regression in the Context of Maximum Likelihood | 216 |
97 Generalized Linear Models | 217 |
98 Nonlinear Regression | 221 |
99 Parameterizations and Constraints | 225 |
Programs and Demonstrations | 229 |
Exercises | 231 |
References | 233 |
Numerical Integration and Monte Carlo Methods | 235 |
102 Motivating Problems | 236 |
103 OneDimensional Quadrature | 242 |
104 Numerical Integration in Two or More Variables | 249 |
105 Uniform Pseudorandom Variables | 256 |
106 QuasiMonte Carlo Integration | 263 |
107 Strategy and Tactics | 268 |
Programs and Demonstrations | 272 |
Exercises | 274 |
References | 276 |
Generating Random Variables from Other Distributions | 279 |
112 General Methods for Continuous Distributions | 280 |
113 Algorithms for Continuous Distributions | 284 |
114 General Methods for Discrete Distributions | 297 |
115 Algorithms for Discrete Distributions | 301 |
116 Other Randomizations | 306 |
117 Accuracy in Random Number Generation | 310 |
Programs and Demonstrations | 313 |
Exercises | 314 |
References | 317 |
Statistical Methods for Integration and Monte Carlo | 319 |
123 Distributional Tests | 326 |
124 Importance Sampling and Weighted Observations | 329 |
125 Testing Importance Sampling Weights | 335 |
126 Laplace Approximations | 337 |
127 Randomized Quadrature | 339 |
128 SphericalRadial Methods | 341 |
Programs and Demonstrations | 346 |
Exercises | 348 |
| 349 | |
Markov Chain Monte Carlo Methods | 351 |
132 Markov Chains | 353 |
133 Gibbs Sampling | 354 |
134 MetropolisHastings Algorithm | 359 |
135 TimeSeries Analysis | 362 |
136 Adaptive AcceptanceRejection | 366 |
137 Diagnostics | 370 |
Programs and Demonstrations | 374 |
References | 376 |
Sorting and Fast Algorithms | 379 |
143 Sorting Algorithms | 381 |
144 Fast Order Statistics and Related Problems | 384 |
145 Fast Fourier Transform | 385 |
146 Convolutions and the Chirpz Transform | 389 |
147 Statistical Applications of the FFT | 391 |
148 Combinatorial Problems | 401 |
Programs and Demonstrations | 405 |
Exercises | 409 |
References | 412 |
Table of Programs and Demonstrations | 415 |
Author Index | 419 |
| 425 | |
Other editions - View all
Common terms and phrases
acceptance/rejection accuracy algorithm analysis approximation arithmetic asymptotic bound Chapter characteristic function chi-square Cholesky factorization coefficients column computed condition number convergence covariance matrix demonstration density derivative digits discrete discussed distribution function double precision eigenproblem eigenvalues eigenvectors elements error estimate evaluations Example Exercise exponential floating point formula gamma Gaussian elimination Gibbs sampling given Hessian Householder transformations implementation importance sampling integration interpolation interval inverse iteration least-squares problem likelihood function linear mathematical maximum likelihood Monte Carlo multiplication multivariate Newton's method nonlinear nonlinear regression nonzero norm optimization orthogonal parameter permutation polynomial positive definite posterior QR algorithm quadratic quadrature ran¹ random variables ratio of uniforms regression requires result root secant Section sequence simple Simpson's rule solution solving spline statistical step subprogram system of equations tion triangular tridiagonal update usually variance vector weights zero