Constrained Policy Motion and Lifts

Conceptual map

III-03.01Report and state motion
III-03.02Implementation metrics
III-03.03Pseudoinverse lifts
III-03.04Drift and feasibility
III-03.05Minimal connection language

1. Report motion and state motion

A desired movement in reported aggregates generally permits many movements in the latent state. A lift selects one feasible state velocity by combining the report constraint with an implementation metric and any autonomous state drift.

Let $x\in\mathbb{R}^{n}$ be the state, $y=h(x)\in\mathbb{R}^{k}$ the report, and $$f(x)$$ the drift that occurs without the policy correction. At a fixed state, write $C=Dh_{x}$ . A requested report velocity $$u$$ requires a correction $$z$$ satisfying

\[Cz = u - Cf(x) \equiv r.\](1)

Let $$G$$ be a positive-definite implementation-cost matrix. The minimum-cost lift solves $\operatorname{min}_{z}$ $\frac{1}{2}z^{\top}Gz$ subject to (1). The total state velocity is $\dot{x}=f(x)+z$ .

2. Minimum-cost lift

Proposition 1 · Weighted lift

Suppose $$C$$ has full row rank and $$G$$ is positive definite. The constrained problem has the unique solution

\[z^{*} = G^{-1}C^{\top }(CG^{-1}C^{\top })^{-1}r.\](2)

Every other feasible correction equals $z^{*}+v$ for some $v\in \operatorname{ker}(C)$ , and its cost exceeds the minimum by $\frac{1}{2}v^{\top}Gv$ .

Proof. The Lagrangian is $\frac{1}{2}z^{\top}Gz-\lambda^{\top}(Cz-r)$ . Stationarity gives $z=G^{-1}C^{\top}\lambda$ . Substitution into the constraint gives $(CG^{-1}C^{\top})\lambda=r$ . Full row rank makes this matrix positive definite. For any feasible displacement $$v$$ , the cross term $v^{\top}Gz^{*}=v^{\top}C^{\top}\lambda=0$ , which gives the cost decomposition. ∎

Formula (2) is a weighted right inverse of the report derivative. The metric states which latent movements are costly. The drift determines how much correction the policy must supply. These are economic primitives of the implementation rule.

3. Worked two-state lift

Let the report be $h(x)=x_{1}+x_{2}$ , take zero drift, and request $$u=1$$ . Then $$C$$ =[1,1]. Under the Euclidean metric, (2) gives $z^{*}$ =(1/2,1/2). Both latent coordinates carry equal cost.

Now set $G=\operatorname{diag}(1,4)$ . A unit displacement in the second coordinate contributes four times as much quadratic cost as the same displacement in the first. Since $G^{-1}=\operatorname{diag}(1,1/4)$ and $CG^{-1}C^{\top}=5/4$ ,

\[z^{*} = (\frac{4}{5}, \frac{1}{5}), \quad \frac{1}{2}z^{*\top }Gz^{*} = \frac{2}{5}.\](3)

The Euclidean lift has cost 5/8 when evaluated under the second metric. The report path is identical in both cases; the hidden state path differs. With drift $$f$$ =(0.3, $$-0.1$$ ), the uncontrolled report velocity is 0.2, so the correction uses $$r=0.8$$ and becomes (0.64,0.16).

4. Paths, rank changes, and feasibility

Along a report path $$y(t)$$ , the lifted state solves an ordinary differential equation with $u=\dot{y}$ . Local existence follows from standard ODE conditions when the matrices in (2) vary smoothly and retain rank. State constraints require the vector field to remain in the tangent cone at a boundary.

Failure case · An unattainable report velocity

If $$C$$ loses row rank, a requested residual $$r$$ may lie outside $\operatorname{range}(C)$ . Then equation (1) has no solution. A pseudoinverse returns the least-squares projection of $$r$$ onto the attainable range; that calculation changes the report path. The projected path and its residual must be reported explicitly.

A second failure arises when (2) points outside a constrained state set. Algebraic feasibility of the linearized report equation does not guarantee viability of the nonlinear state path.

5. Implementation and exercises

Compute the drift-adjusted report residual $$r$$ .
Check the rank of $$C$$ and the smallest eigenvalue of $$G$$ .
Solve $GZ=C^{\top}$ , then solve $(CZ)\lambda=r$ ; set $z=Z\lambda$ .
Check $\lVert Cz-r\rVert$ and all state constraints before advancing the ODE.

Download the volume verification script →

Exercises

Prove the cost decomposition in Proposition 1 for a general feasible $$z$$ .
Replace $$G$$ in the example by a matrix with off-diagonal cost and compute the lift.
Construct a rank-deficient $$C$$ and separate attainable from unattainable report velocities.

Partial solutions

1. Write a feasible correction as $z^{*}+v$ , where $$Cv=0$$ . The cross term is $v^{\top}Gz^{*}=v^{\top}C^{\top}\lambda=0$ , leaving the nonnegative quadratic increment. 3. With $C=\begin{bmatrix}1 & 0 \\ 2 & 0\end{bmatrix}$ , its range is span{(1,2)}. A request such as (1,0) is unattainable; projection changes the intended report velocity.

6. Audit checkpoint

Record the report map, drift, metric, state constraints, rank condition, requested velocity, achieved velocity, and integration error. These objects determine the lift and its economic interpretation.

7. Scope and sources

The chapter treats smooth equality constraints and quadratic local implementation cost. Chapter III-04 studies the order dependence generated by varying lifts.

Roger Penrose (1955), “A Generalized Inverse for Matrices,” Proceedings of the Cambridge Philosophical Society 51, 406–413.Generalized inverses and minimum-norm solutions.
Stephen Boyd and Lieven Vandenberghe, Convex Optimization, §§5.5 and A.5.5.Equality-constrained quadratic minimization and Schur complements.
Jean-Pierre Aubin, Viability Theory, Chapters 2–4.Tangent conditions for state-constrained dynamics.