Non-linear State-space Model Identification from Video Data using Deep
  Encoders
  Identifying systems with high-dimensional inputs and outputs, such as systems
measured by video streams, is a challenging problem with numerous applications
in robotics, autonomous vehicles and medical imaging. In this paper, we propose
a novel non-linear state-space identification method starting from
high-dimensional input and output data. Multiple computational and conceptual
advances are combined to handle the high-dimensional nature of the data. An
encoder function, represented by a neural network, is introduced to learn a
reconstructability map to estimate the model states from past inputs and
outputs. This encoder function is jointly learned with the dynamics.
Furthermore, multiple computational improvements, such as an improved
reformulation of multiple shooting and batch optimization, are proposed to keep
the computational time under control when dealing with high-dimensional and
large datasets. We apply the proposed method to a video stream of a simulated
environment of a controllable ball in a unit box. The study shows low
simulation error with excellent long term prediction capability of the model
obtained using the proposed method.
