Linear Maps, Kernels, Quotients, and Information Dimension

Conceptual map

I-02.01Linear information maps
I-02.02Quotients and factorization
I-02.03Matrix tools for computation
I-02.04Weighted geometry
I-02.05Hidden directions and repair

1. Observation and target as linear maps

In a linear state model, every exact information claim can be expressed through a kernel inclusion, a row-space inclusion, or a factorization. These are three forms of the same statement.

Let $$V$$ , $$Y$$ , and $$T$$ be finite-dimensional real vector spaces. The report is $A:V\to Y$ and the target is $B:V\to T$ . A direction $v\in \operatorname{ker}(A)$ is hidden by the report. It remains target-relevant when $Bv\ne0$ .

Definition 1 · Missing target dimension

The linear information deficit is

\[d(B|A)=\operatorname{rank}(B|_{\operatorname{ker}(A)})=\operatorname{dim} B(\operatorname{ker}(A)).\](1)

2. Kernel, row space, and factorization

Proposition 1 · Linear sufficiency theorem

The following are equivalent:

$\operatorname{ker}(A)\subseteq \operatorname{ker}(B)$ ;
a unique linear map $C:\operatorname{range}(A)\to T$ satisfies $$B=CA$$ ;
after choosing bases, $\operatorname{row}(B)\subseteq \operatorname{row}(A)$ .

Proof. Under the kernel inclusion, define $$C(Av)=Bv$$ . If $$Av=Aw$$ , then $$v-w$$ lies in $\operatorname{ker}(A)$ and $$Bv=Bw$$ , so $$C$$ is well defined. Linearity and uniqueness on $\operatorname{range}(A)$ follow directly. A factorization gives the kernel inclusion. In coordinates, $$B=CA$$ makes each row of $$B$$ a row combination of $$A$$ ; row inclusion conversely supplies such a matrix $$C$$ . ∎

An extension of $$C$$ from $\operatorname{range}(A)$ to all of $$Y$$ is available in finite dimensions and is generally nonunique. Predictions at unattained reports depend on that extension.

3. Sharp dimension of an unrestricted linear repair

Proposition 2 · Missing-rank lower bound

Let $R:V\to \mathbb{R}^{\ell}$ be an added linear record. If the augmented report ( $$A,R$$ ) is sufficient for $$B$$ , then $\ell \ge d(B|A)$ . An unrestricted linear record with $\ell=d(B|A)$ exists.

Proof. Sufficiency means $\operatorname{ker}(A)\cap \operatorname{ker}(R)\subseteq \operatorname{ker}(B)$ . Restrict $$R$$ to $\operatorname{ker}(A)$ . Its kernel lies inside $\operatorname{ker}(B|\operatorname{\operatorname{ker}}(A))$ , so the induced map must separate a quotient of dimension $$d$$ . Hence $\operatorname{rank}(R|\operatorname{\operatorname{ker}}(A))\ge d$ and $\ell \ge d$ . For attainment, choose a linear coordinate map on $\operatorname{ker}(A)$ whose kernel equals $\operatorname{ker}(A)\cap \operatorname{ker}(B)$ and extend it linearly to $$V$$ . ∎

4. Three latent coordinates and two targets

Let $V=\mathbb{R}^{3}$ ,

\[A=[1,1,0], \quad B=\begin{bmatrix}1 & 0 & 1 \\ 0 & 1 & 1\end{bmatrix}.\](2)

A basis of $\operatorname{ker}(A)$ is $v_{1}$ =(1, $$-1$$ ,0) and $v_{2}$ =(0,0,1). Their target images are (1, $$-1$$ ) and (1,1), which are independent. Thus the missing dimension is two.

Record $r_{1}=x_{1}-x_{2}$ and $r_{2}=x_{3}$ . From report $s=x_{1}+x_{2}$ , the decoder is

\[Bx=((s+r_{1})/2+r_{2}, (s-r_{1})/2+r_{2})^{\top }.\](3)

Failure case · Rank of the report used as information dimension

$\operatorname{rank}(A)=1$ and $\operatorname{rank}(B)=2$ . The relevant repair number is the rank of $$B$$ on $\operatorname{ker}(A)$ , which is two here and can differ from either marginal rank in other systems.

5. Implementation, exercises, and sources

Compute an orthonormal basis $$N$$ for $\operatorname{ker}(A)$ and take the numerical rank of $$BN$$ under a declared singular-value tolerance. Verify any proposed repair by checking the null space or row span of the stacked matrix [ $$A;R$$ ].

Download the volume verification script →

Exercises

Verify all three equivalent conditions in Proposition 1 for one sufficient and one insufficient target.
Reproduce the missing dimension and decoder in (2)–(3).
Prove that $d(B|A)=\operatorname{rank}([A;B])-\operatorname{rank}(A)$ .

Partial solutions

2. The $2\times2$ matrix with columns $Bv_{1}$ and $Bv_{2}$ has determinant two, so the restricted rank is two. Substitution of $x_{1}=(s+r_{1})/2$ and $x_{2}=(s-r_{1})/2$ gives (3). 3. Restrict the quotient map from $\operatorname{row}([A;B])$ onto its quotient by $\operatorname{row}(A)$ ; its dimension equals the new independent target rows and also $\operatorname{rank}(B|\operatorname{\operatorname{ker}}(A))$ .

Sheldon Axler, Linear Algebra Done Right, sections on quotients, annihilators, and rank-nullity.Kernel factorization and quotient dimension.
Roger Horn and Charles Johnson, Matrix Analysis, sections on generalized inverses and subspaces.Row-space and null-space tests.
Gene H. Golub and Charles F. Van Loan, Matrix Computations, §5.5.Rank-revealing factorizations and numerical null spaces.

6. Audit checkpoint

State all spaces and dimensions, compute $\operatorname{ker}(A)$ , evaluate $$B$$ on that kernel, report the rank tolerance, construct the decoder on the attained image, and verify the augmented kernel after repair.

7. Scope boundary

The chapter treats exact finite-dimensional linear factorization and unrestricted linear repair. Feasible record dictionaries and nonlinear local factorization appear in I-04 and I-03.

Prerequisites

I-01 · Define the Research Object as a Contract