Ultrasound-Guided Robotic Navigation with Deep Reinforcement Learning
  In this paper we introduce the first reinforcement learning (RL) based
robotic navigation method which utilizes ultrasound (US) images as an input.
Our approach combines state-of-the-art RL techniques, specifically deep
Q-networks (DQN) with memory buffers and a binary classifier for deciding when
to terminate the task.
  Our method is trained and evaluated on an in-house collected data-set of 34
volunteers and when compared to pure RL and supervised learning (SL)
techniques, it performs substantially better, which highlights the suitability
of RL navigation for US-guided procedures. When testing our proposed model, we
obtained a 82.91% chance of navigating correctly to the sacrum from 165
different starting positions on 5 different unseen simulated environments.
