Eigen, Spectral, SVD Decompositions

 

✅ 1. Spectral Decomposition vs Eigenvalue Decomposition

These terms are often used interchangeably, especially in the context of symmetric (real) matrices.

🔷 Spectral Decomposition

• Applies specifically to diagonalizable matrices, typically symmetric real matrices or Hermitian matrices in complex space.

• For a symmetric matrix $A$:

  $$A = Q \Lambda Q^T$$
  

  where:

  • $Q$ contains the orthonormal eigenvectors of $A$
    

  • $\Lambda$ is a diagonal matrix of real eigenvalues

This decomposition represents the matrix in terms of its eigenvalues and eigenvectors.

🔷 Eigenvalue Decomposition

• More general and can apply to non-symmetric matrices.

• For diagonalizable matrix $A$:

  $$A = V \Lambda V^{-1}$$
  

  where:

  • $V$ contains the (possibly non-orthogonal) eigenvectors

  • $\Lambda$ contains the eigenvalues

✅ Key Difference:
Spectral decomposition assumes the matrix is symmetric (or Hermitian), leading to orthogonal eigenvectors and real eigenvalues.
Eigenvalue decomposition works more broadly but may involve complex eigenvalues and non-orthogonal eigenvectors.

---

✅ 2. Spectral Decomposition vs Singular Value Decomposition (SVD)

These are different decompositions and apply in different contexts.

🔷 Spectral Decomposition

• Applies only to square symmetric matrices

• Uses eigenvalues and eigenvectors

• Form:

  $$A = Q \Lambda Q^T$$
  

🔷 SVD (Singular Value Decomposition)

• Applies to any $m \times n$ matrix, not just square ones

• Uses singular values and left/right singular vectors

• Form:

  $$A = U \Sigma V^T$$
  

✅ Key Difference:

• Spectral Decomposition relies on eigenvalues of square symmetric matrices.

• SVD generalizes this to any matrix, even non-square, and always exists.

---

✅ Summary Table

Feature	Spectral Decomposition	Singular Value Decomposition (SVD)
Applies to	Square, symmetric matrices	Any matrix (square or not)
Decomposed into	Eigenvectors and eigenvalues	Singular vectors and singular values
Always exists?	Only if matrix is diagonalizable	Yes, for all matrices
Matrix form	$A = Q \Lambda Q^T$	$A = U \Sigma V^T$
Eigenvalue-based?	✅ Yes	❌ No (uses singular values)
Use case	PCA (with covariance), spectral clustering	PCA, LSA, image compression, recommender systems