Pessimism meets VCG: Learning Dynamic Mechanism Design via Offline Reinforcement Learning
Dynamic mechanism design has garnered significant attention from both
computer scientists and economists in recent years. By allowing agents to
interact with the seller over multiple rounds, where agents' reward functions
may change with time and are state-dependent, the framework is able to model a
rich class of real-world problems. In these works, the interaction between
agents and sellers is often assumed to follow a Markov Decision Process (MDP).
We focus on the setting where the reward and transition functions of such an
MDP are not known a priori, and we are attempting to recover the optimal
mechanism using an a priori collected data set. In the setting where the
function approximation is employed to handle large state spaces, with only mild
assumptions on the expressiveness of the function class, we are able to design
a dynamic mechanism using offline reinforcement learning algorithms. Moreover,
learned mechanisms approximately have three key desiderata: efficiency,
individual rationality, and truthfulness. Our algorithm is based on the
pessimism principle and only requires a mild assumption on the coverage of the
offline data set. To the best of our knowledge, our work provides the first
offline RL algorithm for dynamic mechanism design without assuming uniform
coverage.