Neuromechanics-based Deep Reinforcement Learning of Neurostimulation
  Control in FES cycling
  Functional Electrical Stimulation (FES) can restore motion to a paralysed
person's muscles. Yet, control stimulating many muscles to restore the
practical function of entire limbs is an unsolved problem. Current
neurostimulation engineering still relies on 20th Century control approaches
and correspondingly shows only modest results that require daily tinkering to
operate at all. Here, we present our state of the art Deep Reinforcement
Learning (RL) developed for real time adaptive neurostimulation of paralysed
legs for FES cycling. Core to our approach is the integration of a personalised
neuromechanical component into our reinforcement learning framework that allows
us to train the model efficiently without demanding extended training sessions
with the patient and working out of the box. Our neuromechanical component
includes merges musculoskeletal models of muscle and or tendon function and a
multistate model of muscle fatigue, to render the neurostimulation responsive
to a paraplegic's cyclist instantaneous muscle capacity. Our RL approach
outperforms PID and Fuzzy Logic controllers in accuracy and performance.
Crucially, our system learned to stimulate a cyclist's legs from ramping up
speed at the start to maintaining a high cadence in steady state racing as the
muscles fatigue. A part of our RL neurostimulation system has been successfully
deployed at the Cybathlon 2020 bionic Olympics in the FES discipline with our
paraplegic cyclist winning the Silver medal among 9 competing teams.
