GPB / IMM Collapsing Filters

Definition

GPB / IMM collapsing filters are a family of deterministic approximate inference algorithms for the switching-kalman-filter that fight the $M^t$ history explosion (exponential-belief-state-growth) by collapsing the mixture posterior down to a fixed number of Gaussian components at every time step using moment matching.

• GPB(r) — Generalized Pseudo-Bayesian filter of order $r$. Keeps a mixture indexed by the most recent $r$ regime values $(S_{t-r+1},\dots ,S_t)$, i.e. $M^r$ components. After each filter update, all components that share the same recent-$r$ history are merged via moment matching.
• GPB1 — $r=1$: collapse the entire posterior to a single Gaussian per step. Cheapest, least accurate; equivalent to Kim’s filter for switching state-space models with a single regime moment-matched at every step.
• GPB2 — $r=2$: collapse pairs $(S_{t-1},S_t)$ to a single Gaussian per $S_t$. Costs $M^2$ Kalman updates per step but, unlike GPB1 and IMM, can produce the cross-variance terms $V_{t,t-1|T}^j$ needed by the EM smoother.
• IMM (Interacting Multiple Models) — a cheaper $M$-filter approximation to GPB2 introduced by Blom and Bar-Shalom for target tracking. At each step, $M$ Kalman filters are run (one per regime), and their inputs are mixed before the update by a regime-conditioned weighting. IMM cannot supply the cross-variance term, so it cannot be used inside the EM smoother for SKF parameter learning.
• Moment-matching collapse — the underlying primitive: replace a mixture ${\sum }_j{P}^j\mathcal{N}({\mu }^j,V^j)$ by a single Gaussian with mean $\mu ={\sum }_j{P}^j{\mu }^j$ and covariance $V={\sum }_j{P}^j{V}^j+{\sum }_j{P}^j({\mu }^j-\mu )({\mu }^j-\mu {)}^{\prime }$. Lauritzen [Lau96] proves this is the closest single Gaussian to the original mixture in KL distance.

Intuition

When you run an SKF forward exactly, every Kalman update has to be replicated once per possible regime history $S_{1:t}$ — the bookkeeping grows as $M^t$ and is intractable beyond a handful of steps. The GPB family asks: at the end of every filter step, how much of the regime history actually matters? The answer (empirically and via Boyen-Koller’s bounded-error argument) is that recent regime history matters most, because the dynamics are mixing and older differences “spread out” and overlap. So you keep an $r$-step regime buffer and project the rest down via the optimal Gaussian moment match. IMM is the engineering shortcut: instead of running $M^2$ filters per step, mix the inputs of $M$ filters by a clever regime-conditioned weighting and recover most of the GPB2 quality at $M$-filter cost.

Formal notation

For GPB2 in the SKF setting (notation following Murphy 1998):

• Per-pair filter step — for each $(i,j)$: $({\mathbf{x}}_{t|t}^{i(j)},V_{t|t}^{i(j)},V_{t,t-1|t}^{i(j)},L_t^{i(j)})=\text{Filter}({\mathbf{x}}_{t-1|t-1}^i,V_{t-1|t-1}^i,y_t;A_j,C_j,Q_j,R_j)$
• Joint regime posterior — $M_{t-1,t|t}(i,j)\propto L_t^{i(j)}Z(i,j)M_{t-1|t-1}(i)$
• Marginal regime posterior — $M_{t|t}(j)={\sum }_i{M}_{t-1,t|t}(i,j)$
• Backward-conditional — $W_t^{i|j}=M_{t-1,t|t}(i,j)/M_{t|t}(j)$
• Collapse — $({\mathbf{x}}_{t|t}^j,V_{t|t}^j)=\text{Collapse}({\mathbf{x}}_{t|t}^{i(j)},V_{t|t}^{i(j)},W_t^{i|j})$

The GPB2 smoother uses Kim’s [Kim94] approximation $\mathrm{Pr}(S_t=j\mid S_{t+1}=k,y_{1:T})\approx \mathrm{Pr}(S_t=j\mid S_{t+1}=k,y_{1:t})$ to factor the backward pass.

Variants

• GPB1 — single Gaussian per step (cheapest)
• GPB2 — $M^2$ filters per step, smoother-capable (the standard EM-compatible choice)
• GPB(r) for $r>2$ — diminishing returns, rarely used
• IMM — cheap filter-only $M$-bank approximation to GPB2
• IMM smoother variants — published extensions exist but cannot recover the cross-variance terms needed by EM
• Collapsing inside an RBPF — particle filters that further collapse the per-particle Kalman bank back to a single Gaussian (rare; usually the particle is the collapse)

Comparison

• GPB1 vs GPB2 — GPB2 is much more accurate at $M$× the cost; GPB2 is required for EM because it produces the cross-variance term.
• GPB2 vs IMM — same per-step accuracy in well-mixed regimes; IMM is $1/M$ the cost but cannot supply cross-variances and cannot be used for smoothing-based learning.
• GPB / IMM vs RBPF — GPB / IMM are deterministic collapsing approximations; the Rao-Blackwellised particle filter is a stochastic selection approximation that keeps particles over regime histories and resamples. RBPFs avoid the moment-match bias at the cost of resampling variance; collapsing filters avoid resampling variance at the cost of a unimodal-per-regime posterior approximation.
• GPB / IMM vs variational SKF — variational filters break the $S_t\to X_t$ link and optimise a lower bound; collapsing filters keep the link but moment-match.

When to use

• Real-time target-tracking systems where determinism, latency, and predictable cost matter (the IMM is the industry standard for radar / sonar tracking)
• SKF parameter learning via EM — use GPB2 (you need the cross-variance term)
• Settings where you want a deterministic, easily-debugged alternative to Rao-Blackwellised particle filtering and the regime posterior is not sharply multimodal

Known limitations

• Unimodal-per-regime — collapsing forces a single Gaussian per regime; if the true conditional posterior is multimodal you lose information.
• IMM smoothing — IMM cannot produce the cross-variance term needed by an EM smoother (a major reason GPB2 still matters even though IMM dominates filter-only deployments).
• Error grows with $M$ — moment-matching error per step is bounded but scales unfavourably with the number of regimes when the regime posterior is diffuse.
• No native handling of regime entropy — the order $r$ in GPB($r$) is fixed in advance, not adapted to local regime uncertainty.

Open problems

• Adaptive-$r$ GPB filters that increase the kept history when the regime posterior is diffuse
• Tighter error bounds beyond the Boyen-Koller “stochasticity-induced contraction” argument
• IMM-style smoothing with a cross-variance recovery trick

Key papers

• murphy-1998-switching-kalman-filters — derives GPB2 SKF inference with cross-variance bookkeeping in modern notation, presents IMM as a cheaper approximation, and connects collapsing to EM-based parameter learning

My understanding

For this workspace’s CRE asset-pricing model, GPB2 / IMM are the deterministic sibling of the Rao-Blackwellised particle filter currently in production (SimMdlPrices/src/rbpf.jl). They are worth understanding for two reasons: (a) they are the natural fallback if RBPF resampling variance becomes problematic in tight tie-break experiments, and (b) the moment-matching collapse operator is the right mental model for any future smoothing-based MAP/MLE work where you need cross-variance terms over regime pairs. The IMM literature is also where most of the “what does the regime posterior actually look like in practice” empirical knowledge from target tracking lives.