1School of Electrical & Electronic Engineering, Hanoi University of Science and Technology (HUST), Vietnam
*Corresponding author
Working draft. Not yet public and not yet submitted for peer review — the paper and code will be posted here once ready.
Overview
Distributed Nash equilibrium (NE) seeking for players with dynamics has recently been cast as a cooperative output regulation problem, where the pseudo-gradient of the game plays the role of the regulated error. We make this fully data-driven and add what that reformulation was missing: an $H_\infty$-optimal attenuation of disturbances the model does not capture, a network-level guarantee that survives communication graphs with cycles, and a plain account of what happens when the data themselves are not exact.
Each player learns, from its own local input–state data, the saddle-point policy of a zero-sum game on its internal-model-augmented dynamics — exact rejection of exosystem disturbances plus a certified $\mathcal L_2$ attenuation of unmodelled ones, with no system identification, no initial stabilizing gain, and no communication during learning.
A state-shift homotopy builds a stabilizing gain from scratch; a second continuation in $\mu=\gamma^{-2}$ then carries the policy from the LQR solution to the $H_\infty$ solution, with every iterate certified stabilizing directly from data. As a by-product, it returns a data-based estimate of the optimal attenuation level $\gamma^\star$.
An explicit disturbance-to-pseudo-gradient $\mathcal L_2$ gain, automatic for acyclic graphs and holding under a small-gain condition for graphs with cycles — both certified from each player's own local data, with no model and no central computation.
An exact perturbation bound for the data solve, with an amplification factor computable from the same data. The estimator is errors-in-variables biased, so collecting more data does not remove the error — verified with $16\times$ more data. Averaging the learning identity over shifted start times buys about a factor $30$ in tolerable noise.
Method
Policy iteration is Newton's method on a Riccati equation and converges quadratically — but it must start from a stabilizing policy, which is itself model information. For zero-sum games a second difficulty appears: the simultaneous update is Newton's method on a Riccati equation with an indefinite quadratic term, so a stabilizing pair alone is not enough for convergence. We remove both obstacles with two nested homotopies, using only input–state data collected once, off-policy.
Theorem (convergence of the attenuation homotopy)
Under a standard excitation condition on the collected data, the continuation in $\mu=\gamma^{-2}$ accepts only finitely many steps, every policy pair it uses is certified stabilizing, and the PI at the target level converges quadratically to the true saddle point — for every $\gamma>\gamma^\star$, with no initial stabilizing policy required. As a corollary, running the continuation past the true optimum and taking the last accepted level gives a data-based estimate $\hat\gamma\to\gamma^\star$.
A network-level counterpart holds too: if every player's learned gain is $\gamma_i>\gamma_i^\star$, the closed-loop network is exponentially stable and $\|e\|_{\mathcal L_2}\le\Gamma\|d\|_{\mathcal L_2}$ for an explicit $\Gamma$ — automatically on acyclic graphs, and under a small-gain condition, certified from local data alone, on graphs with cycles.
Why the second homotopy is needed
Simultaneous zero-sum policy iteration started from a genuinely stabilizing pair need not stay stabilizing. On our benchmark, starting from the LQR gain at $\gamma=1.1\gamma^\star$ or $1.3\gamma^\star$ drives the closed loop unstable already at the first update, and the iteration diverges. This is a known initialization difficulty for the game Riccati equation's indefinite quadratic term; the attenuation homotopy sidesteps it by following the solution path in $\gamma$ instead of jumping straight to the target level.
Results
Five mobile robots with different friction coefficients play a network game on a communication graph, each with an unknown linear model, an unmodelled $\mathcal L_2$ disturbance channel, and a known exosystem to reject. Every player learns its gain from $600$ data intervals with no initial stabilizing policy, and the resulting network is compared against the data-driven LMI design of Wang et al. (2026), which stabilizes but is not optimal and needs an acyclic graph for its stability guarantee.
Learning results per player (model-based $\gamma^\star$, $K^\star$ shown only for validation — never used by the algorithm):
| Player | $\gamma^\star$ | $\hat\gamma$ (data) | $\mu$-steps | PI iters | $\|K-K^\star\|/\|K^\star\|$ |
|---|---|---|---|---|---|
| 1 | 2.177 | 2.197 | 5 | 29 | $9.5\times10^{-4}$ |
| 2 | 2.406 | 2.436 | 5 | 30 | $2.7\times10^{-3}$ |
| 3 | 2.366 | 2.391 | 5 | 29 | $1.7\times10^{-3}$ |
| 4 | 2.433 | 2.462 | 4 | 27 | $2.9\times10^{-3}$ |
| 5 | 2.389 | 2.411 | 5 | 30 | $3.3\times10^{-3}$ |
The data-based estimate $\hat\gamma$ is within about $1\%$ of the model-based $\gamma^\star$ for every player, and the learned gain $K$ matches the true GARE solution $K^\star$ to $3\times10^{-3}$ relative error — all without ever solving a model-based Riccati equation.
Network comparison with the data-driven LMI design (acyclic graph):
| Proposed | LMI baseline | |
|---|---|---|
| $\|T_{d\to e}\|_\infty$ | 3.62 | 5.15 |
| $\|T_{d\to u}\|_\infty$ | 3.05 | 3.86 |
| empirical $\|e\|_2/\|d\|_2$ (rest) | 1.77 | 2.47 |
| $\int_0^{60}\|e\|^2dt$ | 422 | 703 |
| $\int_0^{60}\|u\|^2dt$ | 2080 | 1380 |
| final $\|y-y^\star\|_\infty$ | $1.3\times10^{-7}$ | $2.0\times10^{-7}$ |
The learned $H_\infty$ gains give a $30\%$ smaller disturbance-to-error gain and a $30\%$ smaller disturbance-to-input gain than the LMI baseline. The one line where the baseline wins is total control energy over the full scenario — the learned gains are larger, which costs more effort during the initial transient.
Citation
@misc{thai2026hinfne,
title={Model-Free $H_\infty$ Nash Equilibrium Seeking in Network Games via
Double-Homotopy Policy Iteration and Output Regulation},
author={Thai, Khanh},
year={2026},
note={Working draft}
}