LeatherSagiKnowledgebase

Risk-Sensitive Control of Markov Jump Linear Systems

5 min read

Problem

In the classical jump-free linear-quadratic-Gaussian (LQG) setting, three optimal-control problems coincide structurally:

  1. Risk-neutral LQG — minimize expected quadratic cost with linear dynamics and Gaussian noise.
  2. Risk-sensitive LEQG — minimize the exponential of the integrated quadratic cost (Jacobson 1973). Solvable via a modified Riccati equation, recovers LQG in the small-risk limit, and obeys a known large-deviation limit.
  3. H∞ (minimax) control — solve a worst-case quadratic differential game against an adversarial disturbance. Equivalent to LEQG via a duality first noted by Jacobson and developed extensively in the late 1980s/early 1990s (Glover-Doyle, Whittle, Bensoussan-van Schuppen, Başar-Bernhard).

This equivalence chain — risk-neutral ⟸ risk-sensitive ⟸ H∞ — is the structural backbone of robust LQG. Moon and Başar ask: does this chain survive when the linear dynamics are augmented by a continuous-time finite-state Markov chain that switches the system matrices (a Markov jump linear system, MJLS)? The natural extrapolation would be a switching modified Riccati equation indexed by regime, with a corresponding switching H∞ game.

Key idea

Negative result. For Markov jump linear systems with quadratic cost, the risk-sensitive optimal control problem cannot be reduced to a coupled set of modified Riccati equations in closed form, and the risk-sensitive ⟺ H∞ duality that holds for jump-free LQG breaks down in the presence of regime jumps. Two distinct obstructions:

  1. The dynamic-programming HJB-type equation for the risk-sensitive value function under MJLS contains a cross term coupling the continuous state with the conditional jump probabilities that does not factor into a Riccati form. The exponential transformation that linearizes the LEQG HJB in the jump-free case fails to linearize the jump-augmented HJB.
  2. The H∞ small-gain / minimax differential game over MJLS yields a coupled Riccati system with a fundamentally different structure than the (failed) risk-sensitive recursion. There is no parameter substitution or sign convention that maps one to the other.

A direct consequence: no large-deviation or risk-neutral limit recovery from the would-be risk-sensitive solution to standard MJLS-LQG, because the would-be solution does not exist in closed form to take limits of.

Method

The paper proceeds in three layers:

  1. Setup. Continuous-time MJLS: dx = A(θ_t) x dt + B(θ_t) u dt + G(θ_t) dw_t, where θ_t is a finite-state Markov chain with known generator. Quadratic running cost x' Q(θ) x + u' R(θ) u. Risk-sensitive criterion: minimize (2/γ) log E[exp((γ/2) ∫ cost dt)] over admissible feedback u(x, θ).

  2. DP / HJB derivation. Apply the dynamic-programming principle conditional on (x, θ). Show the value function ansatz V(x, θ) = x' P(θ) x + q(θ) does not close: substituting it into the HJB leaves a residual term coupling P(θ) to off-diagonal regime transition expectations of exp(quadratic in x), which is not quadratic in x. Consequence: the natural Riccati ansatz fails.

  3. H∞ comparison. Independently derive the MJLS H∞ minimax controller via a switching dissipation inequality. The resulting coupled Riccati system has a different sign structure and a different coupling pattern across regimes than any candidate risk-sensitive Riccati system would. Conclude: no algebraic substitution recovers Jacobson’s equivalence.

Results

  • No closed-form risk-sensitive controller for MJLS even with linear dynamics, quadratic costs, and Gaussian process noise. The exponential transformation that works in the jump-free LEQG case fails to linearize the jump-augmented HJB.
  • No risk-sensitive ⟺ H∞ equivalence for MJLS. The two control problems have structurally distinct optimal-condition systems and cannot be related by parameter substitution.
  • No large-deviation limit / no risk-neutral limit as a corollary, because the limits would be taken on a non-existent closed-form solution.
  • The result is structural — it does not hinge on any specific cost weighting or regime persistence pattern.

Limitations

  • The paper proves non-existence of a closed-form Riccati-style solution; it does not rule out approximate, numerical, or weak-coupling solutions.
  • The argument is for continuous-time MJLS; the discrete-time MJLS case is structurally similar but not the same proof object.
  • No explicit error bound on how badly any approximate “switching modified Riccati” controller would underperform the true risk-sensitive optimum.
  • The H∞ comparison is structural rather than constructive — there is no recipe for “the closest H∞ surrogate to the risk-sensitive criterion.”

Open questions

  • Is there a tractable approximate risk-sensitive controller for MJLS with provable suboptimality bounds?
  • Does the equivalence breakdown extend to discrete-time MJLS?
  • Does any non-quadratic value function ansatz close the HJB for MJLS-LEQG?
  • Are there special parameter regimes (e.g. slow regime switching, weak coupling) where the equivalence approximately holds?

My take

For the CRE asset-pricing project this is a load-bearing negative result for any plan that wanted to use risk-sensitive / H∞ duality as a robustness lens for the Markov-switching rational-expectations (MSRE) setup. The MSRE block of the model is structurally an MJLS (regime-dependent A, B matrices with a finite-state Markov chain over compound monetary/wage regimes), and the project’s RBPF likelihood machinery is risk-neutral. If we wanted to robustify the filter against model misspecification by lifting it to a risk-sensitive LEQG formulation and then re-interpreting via H∞, Moon-Başar says: that lifting is not free. The closed-form Riccati pipeline (compute_quadratic_pricing_factors_msvar) that powers the asset-pricing block depends on quadratic value-function closure under Markov switching, and this paper rules out the analogous closure for the risk-sensitive cost. Implication for the project: any future “robust RBPF” or “worst-case asset-pricing” extension cannot be obtained as a parameter substitution in the existing Riccati system; it requires either an approximate / numerical scheme or a different mathematical formulation. This is a useful guardrail to record before someone tries the substitution and discovers the obstruction the hard way.

Graph View

Backlinks