Fine tuning U-Net for ultrasound image segmentation: which layers?
  Fine-tuning a network which has been trained on a large dataset is an
alternative to full training in order to overcome the problem of scarce and
expensive data in medical applications. While the shallow layers of the network
are usually kept unchanged, deeper layers are modified according to the new
dataset. This approach may not work for ultrasound images due to their
drastically different appearance. In this study, we investigated the effect of
fine-tuning different layers of a U-Net which was trained on segmentation of
natural images in breast ultrasound image segmentation. Tuning the contracting
part and fixing the expanding part resulted in substantially better results
compared to fixing the contracting part and tuning the expanding part.
Furthermore, we showed that starting to fine-tune the U-Net from the shallow
layers and gradually including more layers will lead to a better performance
compared to fine-tuning the network from the deep layers moving back to shallow
layers. We did not observe the same results on segmentation of X-ray images,
which have different salient features compared to ultrasound, it may therefore
be more appropriate to fine-tune the shallow layers rather than deep layers.
Shallow layers learn lower level features (including speckle pattern, and
probably the noise and artifact properties) which are critical in automatic
segmentation in this modality.
