Extremum Estimators and Uniform Local Expansions

Conceptual map

II-06.01Population and sample criteria
II-06.02Consistency
II-06.03Scores and Hessians
II-06.04Generated first stages
II-06.05The role of high-level conditions

1. A sample optimizer needs a population reference

Numerical stationarity concerns the sample criterion. Consistency and asymptotic linearity require population identification, uniform approximation, local differentiability, and a tolerance small on the estimator's scale.

Let $\theta \in \Theta \subseteq \mathbb{R}^{p}$ and nuisance $\eta \in \mathbb{R}^{q}$ . Let $Q(\theta,\eta)$ have unique interior minimizer $\theta _{0}$ at $\eta _{0}$ , and define score $\Psi=\partial _{\theta}Q$ . Suppose $H=\partial _{\theta}\Psi(\theta _{0},\eta _{0})$ is nonsingular and $G=\partial _{\eta}\Psi(\theta _{0},\eta _{0})$ is finite.

Proposition 1 · Smooth two-step expansion

Assume $\hat{\theta}\to_{p}\theta _{0}$ , $\sqrt{n}(\hat{\eta}-\eta _{0})=O_{p}(1)$ , a uniform first-order expansion of the sample score whose joint Taylor remainder is $o_{p}(n^{-1/2})$ along $(\hat{\theta},\hat{\eta})$ , $\sqrt{n}\,\Psi_{n}(\theta _{0},\eta _{0})=n^{-1/2}\sum _{i=1}^{n}\psi _{i}+o_{p}(1)$ , and $\lVert\Psi_{n}(\hat{\theta},\hat{\eta})\rVert=o_{p}(n^{-1/2})$ . Then

\[\sqrt{n}(\hat{\theta }-\theta _{0})=-H^{-1}[n^{\frac{-1}{2}}\sum \psi _{i}+G\sqrt{n}(\hat{\eta }-\eta _{0})]+o_{p}(1).\](1)

Proof sketch. Expand the sample score at ( $\theta _{0},\eta _{0}$ ) uniformly along the random segment to $(\hat{\theta},\hat{\eta})$ . Move the approximate-stationarity remainder to the right, multiply by $\sqrt{n}$ , and use convergence of the derivative matrices to $$H$$ and $$G$$ . Nonsingularity permits inversion of $$H$$ with probability approaching one. ∎

2. A scalar criterion exposes every component

Let $Q_{n}(\theta,\eta)=1/2(\theta -\bar{X}-\eta )^{2}$ , with population values $\mathbb{E}[X]=1$ , $\eta _{0}=2$ , and $\theta _{0}=3$ . Here $$H=1$$ and $$G=-1$$ . If $\bar{X}=1.05$ and $\hat{\eta}=2.02$ , the exact optimizer is $\hat{\theta}=3.07$ :

\[\hat{\theta }-\theta _{0}=(1.05-1)+(2.02-2)=0.05+0.02=0.07.\](2)

If a solver returns score residual $r_{n}$ , its solution is $\bar{X}+\hat{\eta}+r_{n}$ in this sign convention. Root- $$n$$ inference requires $\sqrt{n}\,r_{n}\to_{p}0$ .

3. Pointwise convergence does not control a random optimizer

Consistency usually follows from uniform convergence of $Q_{n}$ to a population criterion plus separation of its minimizer. A pointwise statement at each fixed $\theta$ permits sample-dependent spikes or valleys that attract the optimizer. Local expansion then requires derivative control on a shrinking random neighborhood.

Generated regressors, simulated values, and estimated operators enter through $\hat{\eta}$ and the derivative $$G$$ . Their rate can be slower than root $$n$$ ; orthogonality corresponds to $$G=0$$ for the relevant nuisance directions and changes the required remainder rate.

4. Singular curvature and parameter boundaries change the limit

Failure case · Solver convergence used as a statistical theorem

A small sample gradient establishes approximate stationarity of the computed criterion. It supplies neither uniqueness of the population target nor uniform convergence of the sample problem.

If $$H$$ is singular, equation (1) cannot be formed and higher-order curvature or partial identification may determine the rate. If $\theta _{0}$ lies on a constraint boundary, the local solution is generally a projection onto a tangent cone; its limit can contain point masses or truncated Gaussian components. Multiple local minima require a global or basin-specific algorithmic statement.

5. Implementation, exercises, and sources

Store the population target definition, parameter constraints, initialization, objective and gradient scales, first-stage sample, Hessian eigenvalues, stationarity residual, and termination rule. Tighten solver tolerances across sample sizes and compare $\sqrt{n}$ times the residual with the sampling and first-stage components.

Download the volume verification script →

Exercises

Reproduce (2) and add score residuals $\pm0.01$ .
Replace the quadratic curvature by $h(\theta -\theta _{0})^{2}/2$ and trace the effect of $h\to0$ .
Constrain $\theta \ge0$ with $\theta _{0}=0$ and derive the projection of an unconstrained Gaussian local estimator.

Partial solutions

1. The returned estimates become 3.08 and 3.06 under the stated residual convention. 2. Sampling and first-stage score errors are multiplied by 1/ $$h$$ ; the expansion loses stability as curvature vanishes. 3. The local constrained estimate is the positive part of the unconstrained one.

Whitney K. Newey and Daniel McFadden (1994), “Large Sample Estimation and Hypothesis Testing,” in Handbook of Econometrics, Volume IV.Extremum consistency and asymptotic expansion.
A. W. van der Vaart, Asymptotic Statistics, Chapter 5.M- and Z-estimators.
Donald W. K. Andrews (1999), “Estimation When a Parameter Is on a Boundary,” Econometrica 67, 1341–1383.Boundary asymptotics.

6. Audit checkpoint

State the population criterion, identification set, parameter boundary, consistency argument, uniform neighborhood, score dimensions, Hessian spectrum, nuisance expansion, orthogonality, solver residual and scale, initialization, and local versus global optimum claim.

7. Scope boundary

The chapter covers smooth finite-dimensional two-step extremum expansions. General empirical-process conditions and nonsmooth criteria appear only through their required interfaces.