Time-Agnostic Prediction: Predicting Predictable Video Frames
  Prediction is arguably one of the most basic functions of an intelligent
system. In general, the problem of predicting events in the future or between
two waypoints is exceedingly difficult. However, most phenomena naturally pass
through relatively predictable bottlenecks---while we cannot predict the
precise trajectory of a robot arm between being at rest and holding an object
up, we can be certain that it must have picked the object up. To exploit this,
we decouple visual prediction from a rigid notion of time. While conventional
approaches predict frames at regularly spaced temporal intervals, our
time-agnostic predictors (TAP) are not tied to specific times so that they may
instead discover predictable "bottleneck" frames no matter when they occur. We
evaluate our approach for future and intermediate frame prediction across three
robotic manipulation tasks. Our predictions are not only of higher visual
quality, but also correspond to coherent semantic subgoals in temporally
extended tasks.
