Establishing an Evaluation Metric to Quantify Climate Change Image
  Realism
  With success on controlled tasks, generative models are being increasingly
applied to humanitarian applications [1,2]. In this paper, we focus on the
evaluation of a conditional generative model that illustrates the consequences
of climate change-induced flooding to encourage public interest and awareness
on the issue. Because metrics for comparing the realism of different modes in a
conditional generative model do not exist, we propose several automated and
human-based methods for evaluation. To do this, we adapt several existing
metrics, and assess the automated metrics against gold standard human
evaluation. We find that using Fr\'echet Inception Distance (FID) with
embeddings from an intermediary Inception-V3 layer that precedes the auxiliary
classifier produces results most correlated with human realism. While
insufficient alone to establish a human-correlated automatic evaluation metric,
we believe this work begins to bridge the gap between human and automated
generative evaluation procedures.
