Efforts estimation of doctors annotating medical image
  Accurate annotation of medical image is the crucial step for image AI
clinical application. However, annotating medical image will incur a great deal
of annotation effort and expense due to its high complexity and needing
experienced doctors. To alleviate annotation cost, some active learning methods
are proposed. But such methods just cut the number of annotation candidates and
do not study how many efforts the doctor will exactly take, which is not enough
since even annotating a small amount of medical data will take a lot of time
for the doctor.
  In this paper, we propose a new criterion to evaluate efforts of doctors
annotating medical image. First, by coming active learning and U-shape network,
we employ a suggestive annotation strategy to choose the most effective
annotation candidates. Then we exploit a fine annotation platform to alleviate
annotating efforts on each candidate and first utilize a new criterion to
quantitatively calculate the efforts taken by doctors. In our work, we take MR
brain tissue segmentation as an example to evaluate the proposed method.
  Extensive experiments on the well-known IBSR18 dataset and MRBrainS18
Challenge dataset show that, using proposed strategy, state-of-the-art
segmentation performance can be achieved by using only 60% annotation
candidates and annotation efforts can be alleviated by at least 44%, 44%, 47%
on CSF, GM, WM separately.
