Learning to Seek: Autonomous Source Seeking with Deep Reinforcement
  Learning Onboard a Nano Drone Microcontroller
  We present fully autonomous source seeking onboard a highly constrained nano
quadcopter, by contributing application-specific system and observation feature
design to enable inference of a deep-RL policy onboard a nano quadcopter. Our
deep-RL algorithm finds a high-performance solution to a challenging problem,
even in presence of high noise levels and generalizes across real and
simulation environments with different obstacle configurations. We verify our
approach with simulation and in-field testing on a Bitcraze CrazyFlie using
only the cheap and ubiquitous Cortex-M4 microcontroller unit. The results show
that by end-to-end application-specific system design, our contribution
consumes almost three times less additional power, as compared to competing
learning-based navigation approach onboard a nano quadcopter. Thanks to our
observation space, which we carefully design within the resource constraints,
our solution achieves a 94% success rate in cluttered and randomized test
environments, as compared to the previously achieved 80%. We also compare our
strategy to a simple finite state machine (FSM), geared towards efficient
exploration, and demonstrate that our policy is more robust and resilient at
obstacle avoidance as well as up to 70% more efficient in source seeking. To
this end, we contribute a cheap and lightweight end-to-end tiny robot learning
(tinyRL) solution, running onboard a nano quadcopter, that proves to be robust
and efficient in a challenging task using limited sensory input.
