Multimodal Feature Fusion and Knowledge-Driven Learning via Experts
  Consult for Thyroid Nodule Classification
  Computer-aided diagnosis (CAD) is becoming a prominent approach to assist
clinicians spanning across multiple fields. These automated systems take
advantage of various computer vision (CV) procedures, as well as artificial
intelligence (AI) techniques, to formulate a diagnosis of a given image, e.g.,
computed tomography and ultrasound. Advances in both areas (CV and AI) are
enabling ever increasing performances of CAD systems, which can ultimately
avoid performing invasive procedures such as fine-needle aspiration. In this
study, a novel end-to-end knowledge-driven classification framework is
presented. The system focuses on multimodal data generated by thyroid
ultrasonography, and acts as a CAD system by providing a thyroid nodule
classification into the benign and malignant categories. Specifically, the
proposed system leverages cues provided by an ensemble of experts to guide the
learning phase of a densely connected convolutional network (DenseNet). The
ensemble is composed by various networks pretrained on ImageNet, including
AlexNet, ResNet, VGG, and others. The previously computed multimodal feature
parameters are used to create ultrasonography domain experts via transfer
learning, decreasing, moreover, the number of samples required for training. To
validate the proposed method, extensive experiments were performed, providing
detailed performances for both the experts ensemble and the knowledge-driven
DenseNet. As demonstrated by the results, the proposed system achieves relevant
performances in terms of qualitative metrics for the thyroid nodule
classification task, thus resulting in a great asset when formulating a
diagnosis.
