Markov Jump Linear System

Definition

A discrete-time Markov Jump Linear System (MJLS) is a stochastic dynamical system whose state evolves according to one of N linear models, indexed by an operation mode $\theta (k)$ that itself follows a finite-state Markov chain with transition probabilities $p_{ij}$ $=P(\theta (k+1)=j$ | $\theta (k)=i$):

$x(k+1)=A_{\theta (k)}x(k)+B_{\theta (k)}u(k)+G_{\theta (k)}w(k)$ $y(k)=L_{\theta (k)}x(k)+H_{\theta (k)}w(k)$ $\theta (k)\in N=$ {1, …, N}

with {w(k)} a wide-sense white noise sequence and $(x_0,{\theta }_0)$ independent random initial conditions. The pair $(x(k),\theta (k))$ is jointly Markov even though {x(k)} alone is not.

Intuition

An MJLS is “many linear systems glued together by a Markov chain on which one they switch to next.” Crucially, the global behavior is not a simple average of the per-mode behaviors: stability of every individual mode is neither necessary nor sufficient for global mean-square stability of the MJLS. A system whose every mode is unstable can be stabilised by a sufficiently “mixing” Markov chain that prevents any mode from dominating; conversely a system whose every mode is stable can blow up if the chain spends long bursts in one mode while the other modes contract slowly. The right object to reason about is not the individual $A_i$ but the second-moment operator T(Q) acting on the augmented Hilbert space $H^n$ of N-tuples of matrices.

Formal notation

The textbook framework lives on $H^n=$ {V = (V_1, …$,V_N)$ : $V_i\in B(C^n)$}, the Hilbert space of N-tuples of complex matrices with inner product $⟨V$; $S⟩={\Sigma }_i$ tr$(V_i\ast S_i)$. Define the indicator-weighted second moment

$Q_i(k)=E[x(k)x(k)\ast 1_{\theta (k)=i}],Q(k)=(Q_1(k)$, …$,Q_N(k))\in H^{n+}$

Then for the homogeneous MJLS $(u=w=0,x(k+1)={\Gamma }_{\theta (k)}x(k))$ the recursion is

$Q(k+1)=T(Q(k)),T_j(Q)={\Sigma }_i{p}_{ij}{\Gamma }_i{Q}_i{\Gamma }_i\ast$

This single operator T (eq. 3.7 in CFM) drives the entire stability and estimation theory. With observations the textbook also defines the augmented matrix

$A=[[p_{11}{A}_1$ … $p_{N1}{A}_N]$, …$,[p_{1N}{A}_1$ … $p_{NN}$ $A_N]]\in B(R^{Nn})$

which appears in the LMMSE filter (eqs. 5.39, 5.52) and represents the same second-moment dynamics in vectorized form.

Variants

• Time-invariant MJLS — $A_i,B_i,G_i,L_i,H_i$ and $p_{ij}$ all k-independent. Most stability and CARE results require this case.
• Non-homogeneous (time-varying) MJLS — matrices and transitions allowed to depend on k. Chapter 3 §3.4 shows MSS ⇔ asymptotic wide-sense-stationary stability under second-order stationary inputs.
• Mode-known MJLS — controller / filter has access to $\theta (k)$ at each k. Standard setting for Chapters 3–5 and the partial-state Chapter 6.
• Mode-unknown MJLS — only the output y(k) is observed; the optimal nonlinear estimator has memory exponential in T, so practical work uses the linear MMSE estimator (Section 5.4) or particle / IMM approximations. This is the “mode-unknown LMMSE filter” variant: an Nn-dimensional Riccati recursion plus an extra term V(Q(k)) (eq. 5.50) that captures the cost of not observing the chain.
• Markov jump filter — the mode-known optimal linear estimator from Chapter 5, restricted to filters whose state depends only on the current $\theta (k)($not on the entire history); this restriction is what brings the number of pre-computed gains down from $O(N^{\top })($sample-path Kalman) to $N\cdot T$.
• Continuous-time MJLS — the continuous-time analogue (jump diffusion); not the focus of the CFM textbook but a parallel literature exists (e.g. Mariton, Costa–Fragoso–Todorov).
• Sub-classes: Markov-switching state-space models (econometrics), gain scheduling controllers under stochastic transitions, jump-diffusion regime models with finite states.

Comparison

Versus a single linear time-invariant system, an MJLS has the extra Markov chain dimension which (1) breaks the equivalence between per-mode stability and global stability, (2) forces the right state to be the augmented $(x,\theta )$ rather than x alone, and (3) replaces a single Lyapunov / Riccati equation with N coupled ones indexed by the mode.

Versus a hidden Markov model (HMM), an MJLS adds continuous linear dynamics on top of the discrete chain — the chain is the regime, but conditional on the regime path the continuous state evolves linearly. This makes Rao-Blackwellization possible: integrate the continuous part with a Kalman recursion conditional on each particle’s regime history.

Versus a piecewise-affine / hybrid system, an MJLS has probabilistic jumps governed by an exogenous Markov chain rather than state-triggered jumps; this admits the second-moment / operator-theoretic analysis that hybrid systems do not.

When to use

• Modeling abrupt changes due to component failures, regime shifts, or environmental switches where the transition probabilities are known.
• Asset-pricing and macro-finance models with latent regime states (monetary policy regimes, wage rigidity regimes, business-cycle phases).
• Target tracking when an object switches between maneuver modes.
• Manufacturing systems with breakdown / repair cycles.
• Any setting where the separation principle for MJLS (mode-known LQG) applies and one wants to compose a CARE-based controller with a CARE-based filter.

Known limitations

• Chapters 3–6 of CFM require finite N; the countable-state extension breaks the MSS ⇔ stochastic-stability equivalence (see Costa-Fragoso ref [67]).
• Mode-known is the comfortable case; mode-unknown forces a linear restriction (LMMSE) or a particle-filter approximation since the optimal nonlinear filter has exponential filter complexity in T (CFM §5.4.1).
• Theory is developed for additive Gaussian noise; non-Gaussian or multiplicative noise variants need separate treatment.

Open problems

• Tight bounds on the gap between LMMSE and the optimal nonlinear filter for mode-unknown MJLS.
• Sharper convergence-rate results for the coupled Riccati recursion.
• Extending the $H^n$ framework to stochastic chains with countable or continuous state spaces while preserving operator-theoretic clarity.

Key papers

• costa-fragoso-marques-mjls-textbook — canonical 2005 Springer textbook; unifies stability, optimal control, filtering, and $H_{\infty }$ via the $H^n$ operator framework.

My understanding

For the CRE asset-pricing project, the MJLS abstraction is the right cleavage plane: the macro variables (output gap, inflation, short rate) live in the “x(k)” linear part, the monetary-policy and wage-rigidity regime states live in the "$\theta (k)$" Markov chain part, and the bond / cap-rate factor loadings come out of the control CARE while the filter for the unobserved continuous state comes out of the filtering CARE (Appendix A duality). The 4-compound-state chain in the project is exactly the N=4 case of the textbook’s general N. The RBPF inherits its rigor from Theorem 5.9 and §5.4.1: the textbook formally proves that the optimal nonlinear mode-unknown filter has memory exponential in T, which is the reason regime histories must be carried by particles rather than collapsed into a sufficient statistic.