Analysis and Linear Algebra: The Singular Value Decomposition and ApplicationsThis book provides an elementary analytically inclined journey to a fundamental result of linear algebra: the Singular Value Decomposition (SVD). SVD is a workhorse in many applications of linear algebra to data science. Four important applications relevant to data science are considered throughout the book: determining the subspace that “best” approximates a given set (dimension reduction of a data set); finding the “best” lower rank approximation of a given matrix (compression and general approximation problems); the Moore-Penrose pseudo-inverse (relevant to solving least squares problems); and the orthogonal Procrustes problem (finding the orthogonal transformation that most closely transforms a given collection to a given configuration), as well as its orientation-preserving version. The point of view throughout is analytic. Readers are assumed to have had a rigorous introduction to sequences and continuity. These are generalized and applied to linear algebraic ideas. Along the way to the SVD, several important results relevant to a wide variety of fields (including random matrices and spectral graph theory) are explored: the Spectral Theorem; minimax characterizations of eigenvalues; and eigenvalue inequalities. By combining analytic and linear algebraic ideas, readers see seemingly disparate areas interacting in beautiful and applicable ways. |
Contents
Chapter 1 Introduction | 1 |
Chapter 2 Linear Algebra and Normed Vector Spaces | 13 |
Chapter 3 Main Tools | 61 |
Chapter 4 The Spectral Theorem | 99 |
Chapter 5 The Singular Value Decomposition | 123 |
Chapter 6 Applications Revisited | 171 |
Chapter 7 A Glimpse Towards Infinite Dimensions | 201 |
| 209 | |
| 213 | |
| 215 | |
Back Cover | 219 |