Robust Guarantees for Perception-Based Control
  Motivated by vision-based control of autonomous vehicles, we consider the
problem of controlling a known linear dynamical system for which partial state
information, such as vehicle position, is extracted from complex and nonlinear
data, such as a camera image. Our approach is to use a learned perception map
that predicts some linear function of the state and to design a corresponding
safe set and robust controller for the closed loop system with this sensing
scheme. We show that under suitable smoothness assumptions on both the
perception map and the generative model relating state to complex and nonlinear
data, parameters of the safe set can be learned via appropriately dense
sampling of the state space. We then prove that the resulting
perception-control loop has favorable generalization properties. We illustrate
the usefulness of our approach on a synthetic example and on the self-driving
car simulation platform CARLA.
