Improved Regret for Differentially Private Exploration in Linear MDP
  We study privacy-preserving exploration in sequential decision-making for
environments that rely on sensitive data such as medical records. In
particular, we focus on solving the problem of reinforcement learning (RL)
subject to the constraint of (joint) differential privacy in the linear MDP
setting, where both dynamics and rewards are given by linear functions. Prior
work on this problem due to Luyo et al. (2021) achieves a regret rate that has
a dependence of $O(K^{3/5})$ on the number of episodes $K$. We provide a
private algorithm with an improved regret rate with an optimal dependence of
$O(\sqrt{K})$ on the number of episodes. The key recipe for our stronger regret
guarantee is the adaptivity in the policy update schedule, in which an update
only occurs when sufficient changes in the data are detected. As a result, our
algorithm benefits from low switching cost and only performs $O(\log(K))$
updates, which greatly reduces the amount of privacy noise. Finally, in the
most prevalent privacy regimes where the privacy parameter $\epsilon$ is a
constant, our algorithm incurs negligible privacy cost -- in comparison with
the existing non-private regret bounds, the additional regret due to privacy
appears in lower-order terms.
