MIcro-Surgical Anastomose Workflow recognition challenge report
  The "MIcro-Surgical Anastomose Workflow recognition on training sessions"
(MISAW) challenge provided a data set of 27 sequences of micro-surgical
anastomosis on artificial blood vessels. This data set was composed of videos,
kinematics, and workflow annotations described at three different granularity
levels: phase, step, and activity. The participants were given the option to
use kinematic data and videos to develop workflow recognition models. Four
tasks were proposed to the participants: three of them were related to the
recognition of surgical workflow at three different granularity levels, while
the last one addressed the recognition of all granularity levels in the same
model. One ranking was made for each task. We used the average
application-dependent balanced accuracy (AD-Accuracy) as the evaluation metric.
This takes unbalanced classes into account and it is more clinically relevant
than a frame-by-frame score. Six teams, including a non-competing team,
participated in at least one task. All models employed deep learning models,
such as CNN or RNN. The best models achieved more than 95% AD-Accuracy for
phase recognition, 80% for step recognition, 60% for activity recognition, and
75% for all granularity levels. For high levels of granularity (i.e., phases
and steps), the best models had a recognition rate that may be sufficient for
applications such as prediction of remaining surgical time or resource
management. However, for activities, the recognition rate was still low for
applications that can be employed clinically. The MISAW data set is publicly
available to encourage further research in surgical workflow recognition. It
can be found at www.synapse.org/MISAW
