Localization and Control of Magnetic Suture Needles in Cluttered
  Surgical Site with Blood and Tissue
  Real-time visual localization of needles is necessary for various surgical
applications, including surgical automation and visual feedback. In this study
we investigate localization and autonomous robotic control of needles in the
context of our magneto-suturing system. Our system holds the potential for
surgical manipulation with the benefit of minimal invasiveness and reduced
patient side effects. However, the non-linear magnetic fields produce
unintuitive forces and demand delicate position-based control that exceeds the
capabilities of direct human manipulation. This makes automatic needle
localization a necessity. Our localization method combines neural network-based
segmentation and classical techniques, and we are able to consistently locate
our needle with 0.73 mm RMS error in clean environments and 2.72 mm RMS error
in challenging environments with blood and occlusion. The average localization
RMS error is 2.16 mm for all environments we used in the experiments. We
combine this localization method with our closed-loop feedback control system
to demonstrate the further applicability of localization to autonomous control.
Our needle is able to follow a running suture path in (1) no blood, no tissue;
(2) heavy blood, no tissue; (3) no blood, with tissue; and (4) heavy blood,
with tissue environments. The tip position tracking error ranges from 2.6 mm to
3.7 mm RMS, opening the door towards autonomous suturing tasks.
