Comparisons Across Models · Chapter I-16

Counterfactual Frontiers Across Model Families

Model comparison requires a common response experiment: the same shock, normalization, observed response, target, and support. Family-entry and sign-reversal radii answer different questions. Coverage failure, within-response ambiguity, and held-out prediction failure provide distinct diagnoses.

Conceptual map

  1. I-16.01Common experimental contracts
  2. I-16.02Family images and fibers
  3. I-16.03Entry and reversal radii
  4. I-16.04Three forms of failure
  5. I-16.05Finite and certified support

1. Model families must share a response experiment

Distances across models acquire meaning only after the shock, response coordinates, units, horizon, normalization, target, metric, and parameter support are held fixed.

Let a finite set of model families be indexed by \(f\in \mathcal{F}\), with supports \(\Theta _{f}\). Preserve family labels by working on the disjoint union \(\bigsqcup _{f}\{f\}\times \Theta _{f}\). Each family maps a parameter into a common response space \(\mathbb{R}^{k}\) through \(r_{f}:\Theta _{f}\to \mathbb{R}^{k}\) and into a scalar counterfactual through \(\tau_{f}\). Fix a baseline response \(r_{0}\), baseline target \(\tau_{0}>0\), and a positive-definite response metric \(M\).

Definition 1 · Entry and sign-loss radii
\[\begin{aligned}\rho _{entry}=\operatorname{inf}\{\lVert r_{f}(\theta )-r_{0}\rVert _{M}: f\ne f_{0}, \theta \in \Theta _{f}\}, \\ \rho _{sign}=\operatorname{inf}\{\lVert r_{f}(\theta )-r_{0}\rVert _{M}: \tau _{f}(\theta )\le 0\}.\end{aligned}\](1)

The infimum of an empty set is +\(\infty\). The gap \(\rho _{sign}-\rho _{entry}\) compares cross-family coverage with the nearest target-sign boundary under this contract.

2. Entry can precede target disagreement

Use a scalar Euclidean response. Baseline family \(A\) has \(\theta \in[-1,1]\), \(r_{A}(\theta)=\theta\), and \(\tau_{A}(\theta)=1+\theta/2\). The baseline is \(\theta=0\), so (\(r_{0},\tau_{0}\))=(0,1). Family \(B\) has \(s\in[0,2]\), \(r_{B}(s)=2+s\), and \(\tau_{B}(s)=1-s\).

Family \(B\) first enters at \(s=0\) and response distance two. The target first reaches the sign boundary at \(s=1\) and response distance three. Family \(A\) remains positive on its support. Hence \(\rho _{entry}=2\), \(\rho _{sign}=3\), and the gap is one.

Proposition 1 · Attainment on compact supports

If every relevant \(\Theta _{f}\) is compact and \(r_{f},\tau_{f}\) are continuous, each nonempty feasible set in (1) attains its infimum.

Proof. The target-sign set is closed as the inverse image of (\(-\infty\),0]; intersecting it with a compact support remains compact. The response distance is continuous and reaches its minimum on a nonempty compact set. The entry result follows by the same argument for finitely many alternative families. ∎

3. Three failures answer different questions

Coverage failure occurs when all alternative-family response images lie outside a declared empirical or testing radius. Within-response ambiguity occurs when two labeled model points share a response and produce different targets. Held-out prediction failure occurs when a fitted model misses response coordinates excluded from estimation. Each diagnosis uses a distinct set and comparison.

Local identification inside every family only isolates nearby parameters conditional on the family label. It supplies no implication across points from different families in the disjoint union. A common response metric also requires common units and normalization; rescaling one coordinate without transforming \(M\) changes both radii.

4. Support and sign boundaries determine the claim

Failure case · Finite grids presented as continuous certificates

A finite parameter grid is a subset of the declared support, so its minimum distance is weakly above the continuous-support infimum. A grid value becomes a certificate only with a coverage bound, Lipschitz argument, interval calculation, or verified global optimization.

Open or unbounded supports can yield finite infima that are never attained. A strict reversal set {\(\tau<0\)} can have the same infimum as the weak sign boundary while lacking a minimizer. If \(\tau_{0}=0\), a reversal radius has no baseline sign; use a declared margin or directional contrast. Numerical sign checks need a tolerance tied to target units and propagated estimation error.

5. Implementation, exercises, and sources

Store the common experiment contract, family labels, supports, response and target maps, metric, sign convention, and solver bounds. Compare grid minima with refined or certified searches, retain infeasible and empty-set statuses, and report response residuals, target margins, support activity, and optimization gaps.

Download the volume verification script →

Exercises

  1. Reproduce both radii in Section 2 and change family \(B\)'s support to [0,1).
  2. Construct two locally identified families with one common response and opposite target signs.
  3. Derive a Lipschitz correction from a grid covering radius to a lower bound on the continuous frontier.
Partial solutions

1. Entry remains two. On [0,1), the sign set is empty, so the weak sign-loss radius is +\(\infty\); its limiting distance is three as \(s\) approaches one. 2. Use singleton or injectively parameterized families whose response images intersect at one point and assign targets of opposite sign there. 3. If the distance objective is \(L\)-Lipschitz and the grid has covering radius \(\delta\), the grid minimum minus \(L\delta\) is a lower bound on the continuous minimum.

  1. R. Tyrrell Rockafellar and Roger J-B Wets, Variational Analysis, chapters on set convergence and value functions.Infima over parameterized feasible sets.
  2. A. W. van der Vaart, Asymptotic Statistics, chapters on local experiments and minimum-distance estimation.Response metrics and local comparison.
  3. Chae-Yeon Xon (2026), “Between Models: Counterfactual Frontiers across Causal-Response Experiments.”Related structural-family frontier application.

6. Audit checkpoint

State the shock and normalization, response coordinates and units, horizon, target, baseline sign or margin, metric, family labels, parameter supports and endpoint inclusion, empty-set convention, strict or weak sign set, grid coverage, solver certificate, numerical tolerance, and estimation uncertainty.

7. Scope boundary

The chapter defines deterministic response-space frontiers on declared model families and supports. Statistical inference for estimated responses appears in Volume II.

Prerequisites