Target Fisher Information: Identification and Precision

Conceptual map

I-12.01Singular information matrices
I-12.02Target identification
I-12.03Precision thresholds
I-12.04Gaussian repair
I-12.05Constrained retention menus

1. Target identification is a kernel condition

A singular information matrix can identify a target that ignores every score-null direction. Rank identifies the size of the learned subspace; target precision also depends on eigenvalues and orientation.

Let $\theta \in \mathbb{R}^{p}$ index a regular local experiment with Fisher information $I\succeq0$ . Consider a differentiable target whose local derivative is $C\in \mathbb{R}^{q\times p}$ . In the linear Gaussian experiment $Y=A\theta+\varepsilon$ with $\varepsilon \sim N(0,\Sigma)$ , $\Sigma\succ0$ , information is $I=A^{\top}\Sigma^{-1}A$ .

Proposition 1 · Linear-Gaussian target estimability

In the Gaussian experiment, $C\theta$ has an unbiased linear estimator for every $\theta$ exactly when

\[\operatorname{ker}(I)\subseteq \operatorname{ker}(C) \Longleftrightarrow \operatorname{range}(C^{\top })\subseteq \operatorname{range}(I).\](1)

Under (1), the minimum covariance in the Loewner order among unbiased linear estimators is $CI^{+}C^{\top}$ .

Proof. Since $\Sigma$ is positive definite, $\operatorname{ker}(I)=\operatorname{ker}(A)$ . An estimator $$BY$$ is unbiased exactly when $$BA=C$$ , which is the factorization condition from I-02. Generalized least squares on $\operatorname{range}(A)$ yields the displayed covariance; any other unbiased linear estimator adds a component orthogonal in the $\Sigma$ -weighted observation space and therefore a positive-semidefinite covariance term. ∎

For broader regular experiments, the same kernel condition describes local pathwise estimability only after the tangent space, target derivative, and closure of the score range are specified. The Gaussian statement above is finite-dimensional and exact.

2. Equal rank can support different precision

Compare $I_{A}=\operatorname{diag}(4,1,0)$ and $I_{B}=\operatorname{diag}(1,4,0)$ . Both have rank two and the same null space. For target $c=e_{1}$ , the information bounds $c^{\top}I^{+}c$ are 1/4 and 1. For target $e_{2}$ , they are 1 and 1/4. The identification verdict agrees across the experiments, while target precision reverses.

A threshold claim such as $\operatorname{Var}(c^{\top}\hat{\theta })\le v$ therefore requires the target direction, information scale, sample-size convention, and estimator class. Matrix rank supplies none of those numerical quantities.

3. Gaussian augmentation repairs target-null directions

Add records $Z=R\theta+\eta$ , with $\eta \sim N(0,\Omega)$ , $\Omega\succ0$ , independent of the original experiment. Their information is $J=R^{\top}\Omega ^{-1}R$ , and $\operatorname{ker}(I+J)=\operatorname{ker}(I)\cap \operatorname{ker}(R)$ .

Proposition 2 · Minimum unrestricted repair rank

If rows of $$R$$ may be chosen freely, the minimum record rank that makes $C\theta$ estimable is

\[\operatorname{rank}(C|_{\operatorname{ker}(I)}).\](2)

Proof. Any valid augmentation must separate every target-relevant equivalence class in $\operatorname{ker}(I)$ , so $\operatorname{rank}(R|_{\operatorname{\operatorname{ker}}(I)})$ is at least the rank in (2). Choose that many linear functionals whose restrictions span the row space of $$C$$ on $\operatorname{ker}(I)$ . Their common kernel inside $\operatorname{ker}(I)$ lies in $\operatorname{ker}(C)$ , attaining the bound. ∎

For base information $\operatorname{diag}(4,0,1)$ and target $$C$$ =[0,1,0], (2) equals one. A scalar record of the second coordinate with noise variance one gives augmented information $\operatorname{diag}(4,1,1)$ and target variance bound one. Noise variance 1/4 raises the added information to four and lowers the bound to 1/4.

4. Prior precision and likelihood information answer different questions

With Gaussian prior precision $P_{0}$ , posterior precision is $P_{0}+I$ . A positive-definite prior produces finite posterior variance along a likelihood-null direction. That finite variance records prior concentration; repeated data from the same null experiment contribute zero information in that direction.

Failure case · Finite posterior variance labeled as data identification

Take $I=\operatorname{diag}(1,0)$ and prior precision $P_{0}=I_{2}$ . The posterior variance of $\theta _{2}$ is one, yet the likelihood is constant in $\theta _{2}$ . Changing the prior variance changes the posterior variance one-for-one. The data supply no identification of that coordinate.

5. Implementation, exercises, and sources

Symmetrize the numerical information matrix, inspect negative eigenvalues against a declared tolerance, compute its numerical null space, and test the operator norm of $$C$$ on that null space. Use the same eigenvalue threshold for the pseudoinverse and report sensitivity across plausible thresholds. Keep prior precision in a separate ledger column.

Download the volume verification script →

Exercises

Reproduce both target bounds for $I_{A}$ and $I_{B}$ .
For base information diag(4,0,1), find the minimum unrestricted repair rank for targets [0,1,0] and [[1,0,0],[0,1,1]].
Add a feasible record dictionary and compare its minimum subset size with (2).

Partial solutions

1. The pseudoinverses are $\operatorname{diag}(1/4,1,0)$ and $\operatorname{diag}(1,1/4,0)$ . 2. The base null space is $\operatorname{span}(e_{2})$ . Both target matrices have restricted rank one, so one freely chosen scalar record suffices. 3. Test $\operatorname{ker}(I)\cap \operatorname{ker}(R_{S})\subseteq \operatorname{ker}(C)$ for each subset.

C. R. Rao, Linear Statistical Inference and Its Applications, chapters on estimable functions in linear models.Estimability and generalized inverses.
A. W. van der Vaart, Asymptotic Statistics, Chapters 7 and 25.Local information and efficiency bounds.
I-02 · Linear Maps, Kernels, Quotients, and Information Dimension.Factorization and unrestricted repair rank.

6. Audit checkpoint

State the experiment, parameter and target derivative, sample-size normalization, information eigenvalues and threshold, null-space test, pseudoinverse convention, estimator class, record likelihood, feasible record set, target variance bound, and prior precision as a separate quantity.

7. Scope boundary

The formal results cover finite-dimensional Gaussian linear experiments and unrestricted linear augmentation. Semiparametric tangent closures, adaptive designs, and constrained information menus require additional analysis.