Safe Linear-Quadratic Dual Control with Almost Sure Performance
  Guarantee
  This paper considers the linear-quadratic dual control problem where the
system parameters need to be identified and the control objective needs to be
optimized in the meantime. Contrary to existing works on data-driven
linear-quadratic regulation, which typically provide error or regret bounds
within a certain probability, we propose an online algorithm that guarantees
the asymptotic optimality of the controller in the almost sure sense. Our dual
control strategy consists of two parts: a switched controller with
time-decaying exploration noise and Markov parameter inference based on the
cross-correlation between the exploration noise and system output. Central to
the almost sure performance guarantee is a safe switched control strategy that
falls back to a known conservative but stable controller when the actual state
deviates significantly from the target state. We prove that this switching
strategy rules out any potential destabilizing controllers from being applied,
while the performance gap between our switching strategy and the optimal linear
state feedback is exponentially small. Under our dual control scheme, the
parameter inference error scales as $O(T^{-1/4+\epsilon})$, while the
suboptimality gap of control performance scales as $O(T^{-1/2+\epsilon})$,
where $T$ is the number of time steps, and $\epsilon$ is an arbitrarily small
positive number. Simulation results on an industrial process example are
provided to illustrate the effectiveness of our proposed strategy.
