Reinforced Medical Report Generation with X-Linear Attention and
  Repetition Penalty
  To reduce doctors' workload, deep-learning-based automatic medical report
generation has recently attracted more and more research efforts, where
attention mechanisms and reinforcement learning are integrated with the classic
encoder-decoder architecture to enhance the performance of deep models.
However, these state-of-the-art solutions mainly suffer from two shortcomings:
(i) their attention mechanisms cannot utilize high-order feature interactions,
and (ii) due to the use of TF-IDF-based reward functions, these methods are
fragile with generating repeated terms. Therefore, in this work, we propose a
reinforced medical report generation solution with x-linear attention and
repetition penalty mechanisms (ReMRG-XR) to overcome these problems.
Specifically, x-linear attention modules are used to explore high-order feature
interactions and achieve multi-modal reasoning, while repetition penalty is
used to apply penalties to repeated terms during the model's training process.
Extensive experimental studies have been conducted on two public datasets, and
the results show that ReMRG-XR greatly outperforms the state-of-the-art
baselines in terms of all metrics.
