Using Deep Reinforcement Learning with Automatic Curriculum Learning for
  Mapless Navigation in Intralogistics
  We propose a deep reinforcement learning approach for solving a mapless
navigation problem in warehouse scenarios. In our approach, an automation
guided vehicle is equipped with LiDAR and frontal RGB sensors and learns to
perform a targeted navigation task. The challenges reside in the sparseness of
positive samples for learning, multi-modal sensor perception with partial
observability, the demand for accurate steering maneuvers together with long
training cycles. To address these points, we propose NavACL-Q as a method for
automatic curriculum learning in combination with a distributed version of the
soft actor-critic algorithm. The performance of the learning algorithm is
evaluated exhaustively in an unseen warehouse environment to validate both
robustness and generalizability of the learned policy. Results in NVIDIA Isaac
Sim demonstrates that our trained agent significantly outperforms a map-based
navigation pipeline provided by NVIDIA Isaac Sim with an increased agent-goal
distance of 3m and wider initial relative agent-goal rotations of 45 degree.
The ablation studies also suggests that NavACL-Q greatly facilitates the
learning process with a performance gain of roughly 40% compared to training
with random starts and that the utilization of a pre-trained feature extractor
manifestly boosts the performance by approximately 60%.
