Deep Learning of Robotic Tasks without a Simulator using Strong and Weak
  Human Supervision
  We propose a scheme for training a computerized agent to perform complex
human tasks such as highway steering. The scheme is designed to follow a
natural learning process whereby a human instructor teaches a computerized
trainee. The learning process consists of five elements: (i) unsupervised
feature learning; (ii) supervised imitation learning; (iii) supervised reward
induction; (iv) supervised safety module construction; and (v) reinforcement
learning. We implemented the last four elements of the scheme using deep
convolutional networks and applied it to successfully create a computerized
agent capable of autonomous highway steering over the well-known racing game
Assetto Corsa. We demonstrate that the use of the last four elements is
essential to effectively carry out the steering task using vision alone,
without access to a driving simulator internals, and operating in wall-clock
time. This is made possible also through the introduction of a safety network,
a novel way for preventing the agent from performing catastrophic mistakes
during the reinforcement learning stage.
