LeatherSagiKnowledgebase

Simulated-Data Recovery: Local PASS / Global FAIL

3 min read

Objective

Establish whether the constrained 54-D BOBYQA MLE pipeline is identified by recovering known DGP parameters from simulated data: (i) Exp 1.2 — local recovery from a 1% perturbation in normalized z-space; (ii) Exp 1.3 — global recovery from 5 cold Sobol starts after a feasibility screen + Hamilton scoring.

Setup

  • DGP: synthetic panel generated at theta_BOBYQA_MLE with m_g pinned, CRN seed 183, MC DGP pricing at R=1M.
  • Estimator: 6D Hamilton-Kim filter with MC cap-rate pricing (mc_H_burn=150, mc_R=5000, std=27 nats per eval).
  • Local recovery (Exp 1.2): BOBYQA from 1% perturbations, NPT=56, 400 evals, ftol=2.0.
  • Global recovery (Exp 1.3 v3): 1M-point Sobol grid screened by absorbing spectral proxy (99.1% accuracy), 3D→6D Hamilton scoring, top-5 starts to BOBYQA (NPT=109, 500 evals).
  • Comparator: oracle NLL = -5385.83 at theta_true. LL decomposition: macro 36.8%, yields 34.0%, income 23.1%, cap rates 2.5%.
  • Acceptance rule: <5% per-group RMSE locally; <20% CRE RMSE globally.

Procedure

  • Phase 0/0.5: infrastructure validation (31/32 Tier-B PASS), MC repeatability (std 0.51 nats across 3 seeds), CRN variance reduction measurement (1.4-1.6x).
  • Exp 1.2: 5 BOBYQA trials from independent 1% perturbations.
  • Exp 1.3 v1: original is_feasible_pixg_cap screen (63.3% accuracy) → 2,397 feasible from 1M Sobol → top-5 BOBYQA.
  • Exp 1.3 v3: replaced with absorbing spectral proxy (99.1% accuracy) → 6,872 feasible (2.87x improvement) → top-5 BOBYQA.

Results

  • Local (Exp 1.2): 5/5 trials FTOL_REACHED in 111-119 evals, mean gap +40 nats, best within 5 nats of oracle. CRE RMSE 2.31-3.38%, all 13 parameter groups <5% (after excluding the m_g constraint artifact).
  • Global (Exp 1.3 v3): best of 5 cold starts is rank-2 with NLL -5131.1 (gap 255 nats, CRE RMSE 52%). Other 4 starts: gaps 467-2341 nats. Absorbing proxy gives 2,481 nats improvement vs v1 (best -2,650 → -5,131) but still fails the 20% threshold.
  • Cap rate RMSE / sigma_q at truth: 2.53/3.12/2.75 (Apt/Ind/Off).
  • Audit: not yet audited (04_audit/ empty); judgment call here is that local recovery is robust because all 5 trials converge inside MC noise.

Analysis

All 54 parameters are locally identified from T=118 quarters: CRE parameters recover to 2-3% RMSE despite cap rates contributing only 2.5% of total log-likelihood — income observations (23.1% of LL) carry the heavy lifting on CRE identification. The global FAIL is consistent with the basin-finder-05-global-driver null result: from cold Sobol starts inside an extremely tight feasible region (0.24-0.69% of grid), 500 BOBYQA evals are insufficient to descend the 255-nat gap. The basin appears to be accessible (rank-2 reaches within 255 nats), so this is local-opt depth, not basin discovery.

Claim updates

  • forward-solution-unique-fundamental-ree-with-nbc: tested_by, strength moderate. Local recovery PASS validates that the forward solution is uniquely identified locally; the global FAIL is a depth artifact, not a multiplicity finding, consistent with uniqueness.
  • no-bubble-condition: tested_by, strength moderate. The absorbing spectral proxy implementing the NBC test moved best NLL from -2650 to -5131 (a 2481-nat improvement) — empirical demonstration that the production NBC gate is correctly carving out the feasible region around theta_true.

Follow-up

Graph View

Backlinks