Reinforcement Learning of Musculoskeletal Control from Functional
  Simulations
  To diagnose, plan, and treat musculoskeletal pathologies, understanding and
reproducing muscle recruitment for complex movements is essential. With muscle
activations for movements often being highly redundant, nonlinear, and time
dependent, machine learning can provide a solution for their modeling and
control for anatomy-specific musculoskeletal simulations. Sophisticated
biomechanical simulations often require specialized computational environments,
being numerically complex and slow, hindering their integration with typical
deep learning frameworks. In this work, a deep reinforcement learning (DRL)
based inverse dynamics controller is trained to control muscle activations of a
biomechanical model of the human shoulder. In a generalizable end-to-end
fashion, muscle activations are learned given current and desired
position-velocity pairs. A customized reward functions for trajectory control
is introduced, enabling straightforward extension to additional muscles and
higher degrees of freedom. Using the biomechanical model, multiple episodes are
simulated on a cluster simultaneously using the evolving neural models of the
DRL being trained. Results are presented for a single-axis motion control of
shoulder abduction for the task of following randomly generated angular
trajectories.
