Embodied Synaptic Plasticity with Online Reinforcement learning
  The endeavor to understand the brain involves multiple collaborating research
fields. Classically, synaptic plasticity rules derived by theoretical
neuroscientists are evaluated in isolation on pattern classification tasks.
This contrasts with the biological brain which purpose is to control a body in
closed-loop. This paper contributes to bringing the fields of computational
neuroscience and robotics closer together by integrating open-source software
components from these two fields. The resulting framework allows to evaluate
the validity of biologically-plausibe plasticity models in closed-loop robotics
environments. We demonstrate this framework to evaluate Synaptic Plasticity
with Online REinforcement learning (SPORE), a reward-learning rule based on
synaptic sampling, on two visuomotor tasks: reaching and lane following. We
show that SPORE is capable of learning to perform policies within the course of
simulated hours for both tasks. Provisional parameter explorations indicate
that the learning rate and the temperature driving the stochastic processes
that govern synaptic learning dynamics need to be regulated for performance
improvements to be retained. We conclude by discussing the recent deep
reinforcement learning techniques which would be beneficial to increase the
functionality of SPORE on visuomotor tasks.
