Online Continual Learning with Natural Distribution Shifts: An Empirical
  Study with Visual Data
  Continual learning is the problem of learning and retaining knowledge through
time over multiple tasks and environments. Research has primarily focused on
the incremental classification setting, where new tasks/classes are added at
discrete time intervals. Such an "offline" setting does not evaluate the
ability of agents to learn effectively and efficiently, since an agent can
perform multiple learning epochs without any time limitation when a task is
added. We argue that "online" continual learning, where data is a single
continuous stream without task boundaries, enables evaluating both information
retention and online learning efficacy. In online continual learning, each
incoming small batch of data is first used for testing and then added to the
training set, making the problem truly online. Trained models are later
evaluated on historical data to assess information retention. We introduce a
new benchmark for online continual visual learning that exhibits large scale
and natural distribution shifts. Through a large-scale analysis, we identify
critical and previously unobserved phenomena of gradient-based optimization in
continual learning, and propose effective strategies for improving
gradient-based online continual learning with real data. The source code and
dataset are available in: https://github.com/IntelLabs/continuallearning.
