When Radiology Report Generation Meets Knowledge Graph
  Automatic radiology report generation has been an attracting research problem
towards computer-aided diagnosis to alleviate the workload of doctors in recent
years. Deep learning techniques for natural image captioning are successfully
adapted to generating radiology reports. However, radiology image reporting is
different from the natural image captioning task in two aspects: 1) the
accuracy of positive disease keyword mentions is critical in radiology image
reporting in comparison to the equivalent importance of every single word in a
natural image caption; 2) the evaluation of reporting quality should focus more
on matching the disease keywords and their associated attributes instead of
counting the occurrence of N-gram. Based on these concerns, we propose to
utilize a pre-constructed graph embedding module (modeled with a graph
convolutional neural network) on multiple disease findings to assist the
generation of reports in this work. The incorporation of knowledge graph allows
for dedicated feature learning for each disease finding and the relationship
modeling between them. In addition, we proposed a new evaluation metric for
radiology image reporting with the assistance of the same composed graph.
Experimental results demonstrate the superior performance of the methods
integrated with the proposed graph embedding module on a publicly accessible
dataset (IU-RR) of chest radiographs compared with previous approaches using
both the conventional evaluation metrics commonly adopted for image captioning
and our proposed ones.
