Reinforcement co-Learning of Deep and Spiking Neural Networks for
  Energy-Efficient Mapless Navigation with Neuromorphic Hardware
  Energy-efficient mapless navigation is crucial for mobile robots as they
explore unknown environments with limited on-board resources. Although the
recent deep reinforcement learning (DRL) approaches have been successfully
applied to navigation, their high energy consumption limits their use in
several robotic applications. Here, we propose a neuromorphic approach that
combines the energy-efficiency of spiking neural networks with the optimality
of DRL and benchmark it in learning control policies for mapless navigation.
Our hybrid framework, spiking deep deterministic policy gradient (SDDPG),
consists of a spiking actor network (SAN) and a deep critic network, where the
two networks were trained jointly using gradient descent. The co-learning
enabled synergistic information exchange between the two networks, allowing
them to overcome each other's limitations through a shared representation
learning. To evaluate our approach, we deployed the trained SAN on Intel's
Loihi neuromorphic processor. When validated on simulated and real-world
complex environments, our method on Loihi consumed 75 times less energy per
inference as compared to DDPG on Jetson TX2, and also exhibited a higher rate
of successful navigation to the goal, which ranged from 1% to 4.2% and depended
on the forward-propagation timestep size. These results reinforce our ongoing
efforts to design brain-inspired algorithms for controlling autonomous robots
with neuromorphic hardware.
