Multitask Learning of Temporal Connectionism in Convolutional Networks
  using a Joint Distribution Loss Function to Simultaneously Identify Tools and
  Phase in Surgical Videos
  Surgical workflow analysis is of importance for understanding onset and
persistence of surgical phases and individual tool usage across surgery and in
each phase. It is beneficial for clinical quality control and to hospital
administrators for understanding surgery planning. Video acquired during
surgery typically can be leveraged for this task. Currently, a combination of
convolutional neural network (CNN) and recurrent neural networks (RNN) are
popularly used for video analysis in general, not only being restricted to
surgical videos. In this paper, we propose a multi-task learning framework
using CNN followed by a bi-directional long short term memory (Bi-LSTM) to
learn to encapsulate both forward and backward temporal dependencies. Further,
the joint distribution indicating set of tools associated with a phase is used
as an additional loss during learning to correct for their co-occurrence in any
predictions. Experimental evaluation is performed using the Cholec80 dataset.
We report a mean average precision (mAP) score of 0.99 and 0.86 for tool and
phase identification respectively which are higher compared to prior-art in the
field.
