Matrix Manipulation
SVD decomposes a matrix into orthonormal bases, capturing data variance and aligning with maximum variance directions.
Singular Value Decomposition (SVD) is a factorization method that decomposes a rectangular matrix X into the product of three matrices: U, S, and V.
- The U matrix consists of a set of 'left' orthonormal bases
- The S matrix is a diagonal matrix
- The V matrix consists of set of 'right' orthonormal bases
The values in S are called singular values. They are non-negative, and their magnitudes indicate the importance of the corresponding bases (components). The singular values reflect the amount of data variance captured by the bases. The first basis (the one with largest singular value) lies in the direction of the greatest data variance. The second basis captures the orthogonal direction with the second greatest variance, and so on.
SVD essentially performs a coordinate rotation that aligns the transformed axes with the directions of maximum variance in the data. This is a useful procedure under the assumption that the observed data has a high signal-to-noise ratio and that a large variance corresponds to interesting data content while a lower variance corresponds to noise.
SVD makes the assumption that the underlying data is Gaussian distributed and can be well described in terms of means and covariances.