OstrichRL: A Musculoskeletal Ostrich Simulation to Study Bio-mechanical
  Locomotion
  Muscle-actuated control is a research topic that spans multiple domains,
including biomechanics, neuroscience, reinforcement learning, robotics, and
graphics. This type of control is particularly challenging as bodies are often
overactuated and dynamics are delayed and non-linear. It is however a very well
tested and tuned actuation mechanism that has undergone millions of years of
evolution with interesting properties exploiting passive forces and efficient
energy storage of muscle-tendon units. To facilitate research on
muscle-actuated simulation, we release a 3D musculoskeletal simulation of an
ostrich based on the MuJoCo physics engine. The ostrich is one of the fastest
bipeds on earth and therefore makes an excellent model for studying
muscle-actuated bipedal locomotion. The model is based on CT scans and
dissections used to collect actual muscle data, such as insertion sites,
lengths, and pennation angles. Along with this model, we also provide a set of
reinforcement learning tasks, including reference motion tracking, running, and
neck control, used to infer muscle actuation patterns. The reference motion
data is based on motion capture clips of various behaviors that we preprocessed
and adapted to our model. This paper describes how the model was built and
iteratively improved using the tasks. We also evaluate the accuracy of the
muscle actuation patterns by comparing them to experimentally collected
electromyographic data from locomoting birds. The results demonstrate the need
for rich reward signals or regularization techniques to constrain muscle
excitations and produce realistic movements. Overall, we believe that this work
can provide a useful bridge between fields of research interested in muscle
actuation.
