Inference for Rank, Singular Values, and Eigenspaces

Conceptual map

II-10.01Exact and numerical rank
II-10.02Perturbation bounds
II-10.03Rank strata
II-10.04Risk frontiers
II-10.05Robust reporting

1. Matrix error bounds singular-value error

Exact rank is discontinuous. Singular values are Lipschitz, while singular subspaces require a positive separation from neighboring values.

Let $\hat{A}=A+E$ be real matrices of the same dimensions and order their singular values decreasingly. Weyl's perturbation inequality gives

Proposition 1 · Singular-value stability

\[|\sigma _{j}(\widehat{A})-\sigma _{j}(A)|\le \lVert E\rVert _{2} \text{for every} j.\](1)

Proof sketch. The min-max characterization expresses each singular value through extrema of $\lVert Ax\rVert$ over subspaces. The triangle inequality changes every such norm by at most $\lVert E\rVert_{2}$ ; apply the argument in both directions. ∎

If a high-probability bound $\lVert E\rVert_{2}\le \varepsilon$ is available, each population singular value lies in $[\max\{0,\hat{\sigma}_{j}-\varepsilon\},\hat{\sigma}_{j}+\varepsilon]$ . A positive lower endpoint certifies a nonzero population direction under that event.

2. A separated eigenspace has a stable angle

For symmetric matrices, take

\[A=\operatorname{diag}(3,1), \quad E=\begin{bmatrix}0 & 0.1 \\ 0.1 & 0\end{bmatrix}.\](2)

The perturbed eigenvalues are $2\pm \sqrt{1.01}$ , approximately 3.00499 and 0.99501. Its leading eigenvector rotates by $\theta=\tfrac{1}{2}\arctan(0.1)\approx0.04983$ radians, so $\sin\theta\approx0.04981$ . The original eigengap is two and $\lVert E\rVert_{2}=0.1$ ; the exact angle is below their ratio 0.05. General Davis–Kahan statements use an explicitly defined separation between the target spectral cluster and its complement.

3. Near ties destabilize orientation before values

Replace (2) by $\operatorname{diag}(1+\delta ,1)$ and keep an off-diagonal perturbation $\varepsilon$ . The rotation satisfies $\tan(2\theta)=2\varepsilon/\delta$ . For fixed $\varepsilon$ , $\theta$ approaches $\pi/4$ as $\delta$ approaches zero from above. At $\delta=0$ , the population eigenvalue is repeated and no individual leading population direction is distinguished; the full two-dimensional eigenspace is the invariant object.

Numerical rank counts singular values above a tolerance. Effective rank can instead weight the spectrum continuously. Exact rank inference needs an uncertainty statement around zero, while loss based on a discarded spectral tail may remain stable because singular values themselves obey (1).

4. A threshold is part of the reported estimand

Failure case · Positive estimated singular value labeled as a new population rank

Sampling noise generically makes a rank-deficient matrix full rank. A thresholded estimate changes with units, normalization, and noise scale unless the threshold is transformed and justified with an error bound.

Subspace bounds require a gap separating the target cluster from the complementary spectrum. At a closing gap, report the tied spectral projector, a set of admissible subspaces, or value-based functionals. Sign conventions for isolated eigenvectors remove arbitrary $\pm$ flips and still leave orientation unstable at a tie.

5. Implementation, exercises, and sources

Scale the matrix and metric first, compute residuals of the SVD or eigendecomposition, and report the error norm, confidence bound, threshold, gaps, tied clusters, and projection distances. Compare results across thresholds tied to the estimated noise scale.

Download the volume verification script →

Exercises

Reproduce the eigenvalues and angle in Section 2.
Evaluate the angle for $\delta=1$ ,0.1,0.01 with $\varepsilon=0.05$ .
Given $\hat{\sigma }$ =(2,0.3,0.04) and $\varepsilon=0.05$ , identify which nonzero singular values are certified.

Partial solutions

1. The characteristic polynomial is $(3-\lambda)(1-\lambda)-0.01$ . The rotation formula follows from diagonalizing a symmetric $2\times2$ matrix. 2. Use $\theta=\tfrac{1}{2}\arctan(0.1/\delta)$ ; the angle tends toward $\pi/4$ as $\delta$ shrinks. 3. The first two lower endpoints are positive; the third interval includes zero.

Rajendra Bhatia, Matrix Analysis, Chapter VII.Eigenvalue and singular-value perturbation.
Chandler Davis and W. M. Kahan (1970), “The Rotation of Eigenvectors by a Perturbation. III,” SIAM Journal on Numerical Analysis 7, 1–46.Invariant-subspace angle bounds.
G. W. Stewart and Ji-Guang Sun, Matrix Perturbation Theory.Separated and multiple spectral subspaces.

6. Audit checkpoint

State the population and estimated matrices, units and metrics, perturbation norm and probability event, numerical threshold, singular intervals, eigengaps, tied clusters, vector sign convention, projector distance, and value functional used downstream.

7. Scope boundary

The chapter covers deterministic finite-matrix perturbation bounds used inside statistical confidence events. Random-matrix limits and optimal rank tests require additional theory.