{'fraction_non_alphanumeric': 0.04538234626730202, 'fraction_numerical': 0.016252552756977536, 'mean_word_length': 4.670151174010936, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The Dialog Must Go On: Improving Visual Dialog via Generative Self-Training

Visual dialog (VisDial) is a task of answering a sequence of questions grounded in an image, using the dialog history as context. Prior work has trained the dialog agents solely on VisDial data via supervised learning or leveraged pre-training on related vision-and-language datasets. This paper presents a semi-supervised learning approach for visually-grounded dialog, called Generative Self-Training (GST), to leverage unlabeled images on the Web. Specifically, GST first retrieves in-domain images through out-of-distribution detection and generates synthetic dialogs regarding the images via multimodal conditional text generation. GST then trains a dialog agent on the synthetic and the original VisDial data. As a result, GST scales the amount of training data up to an order of magnitude that of VisDial (1.2M to 12.9M QA data). For robust training of the generated dialogs, we also propose perplexity-based data selection and multimodal consistency regularization. Evaluation on VisDial v1.0 and v0.9 datasets shows that GST achieves new state-of-the-art results on both datasets. We further observe strong performance gains in the low-data regime (up to 9.35 absolute points on NDCG).


Introduction
Recently, there has been extensive research towards developing visually-grounded dialog systems [1][2][3][4] due to their significance in many real-world applications (e.g., helping visually impaired person). Notably, Visual Dialog (VisDial) [1] has provided a testbed for studying such systems, where a dialog agent should answer a sequence of image-grounded questions. For instance, the agent is expected to answer open-ended questions like "What color is it?" and "How old does she look?". This task requires a holistic understanding of visual information, linguistic semantics in context (e.g., it and she), and most importantly, the grounding of these two.
Our research question is the following: How can the dialog agent expand its knowledge beyond what it can acquire via supervised learning or self-supervised pre-training on the provided datasets? Some recent studies have shown that semi-supervised learning and pre-training have complementary modeling capabilities in image classification [29] and text classification [30]. Inspired by the studies, we consider semi-supervised learning (SSL) as a way to address the above question. Let us assume that large amounts of unlabeled images are available. SSL for VisDial can be applied to generate synthetic conversations for the unlabeled images and train the agent with the synthetic data. However, there are two critical challenges to this approach. First, the target output for VisDial (i.e., multi-turn visual QA data) is more complex than that of the aforementioned studies [29,30]. Specifically, they have addressed the classification problems, yielding class probabilities as pseudo labels [31]. In contrast, SSL for VisDial should generate a sequence of pseudo queries (i.e., visual questions) and pseudo labels (i.e., corresponding answers) in natural language to train the answering agent. It further indicates that the target output should be generated while considering the multimodal and sequential nature of the visual dialog task. Next, even if SSL yields synthetic dialogs via text generation, there may be noise such as generating irrelevant questions or incorrect answers to given contexts. A robust training method is required to leverage such noisy synthetic dialog datasets.
To this end, we propose a new learning strategy, called Generative Self-Training (GST), that artificially generates multi-turn visual QA data and utilizes the synthetic data for training. GST first trains the teacher model (answerer) and the visual question generation model (questioner) using VisDial data. It then retrieves a set of unlabeled images from a Web image dataset, Conceptual 12M [44]. Next, the questioner and the teacher alternately generate a series of visual QA pairs for the retrieved images. Finally, the student is trained on the synthetic and the original VisDial data. We also propose perplexity-based data selection (PPL) and multimodal consistency regularization (MCR) to effectively train the student with the noisy dialog data. PPL is to selectively utilize the answers whose perplexity of the teacher is below a threshold. MCR encourages the student to yield consistent predictions when the perturbed multimodal inputs are given. Our key contributions can be summarized as follows: • We propose Generative Self-Training (GST) that generates multi-turn visual QA data to leverage unlabeled Web images effectively. To the best of our knowledge, GST is the first semi-supervised learning approach throughout a wide range of visual QA tasks, including VisDial. • We demonstrate the effectiveness of the perplexity-based data selection (PPL) and the multimodal consistency regularization (MCR) when training the noisy synthetic dialog data. • Finally, we validate GST on VisDial v1.0 and v0.9 datasets, and GST achieves new state-of-the-art results on both datasets. We also show two important results: (1) GST is indeed effective in the low-data regime, and (2) GST and self-supervised pre-training are complementary.

Preliminaries
Self-Training. We have a labeled dataset L = {(x n , y n )} N n=1 and an unlabeled dataset U = {x m } M m=1 . Typically, self-training first trains a teacher model P T on the labeled dataset L. The teacher then predicts the pseudo labelỹ for the unlabeled datax ∼ U , constructing the pseudo-labeled datasetL = Finally, a student model P S is trained on L ∪L. Many variants have been studied on this setup: (1) selecting the subset of the pseudo-labeled dataset [36,35,33], (2) adding noise to inputs [29,36,35,34,33], and (3) iterating the above setup multiple times [36,35].
Visual Dialog. The visual dialog (VisDial) dataset [1] contains an image v and a visually- where c denotes an image caption. T is the number of rounds for each dialog. At round t, a dialog agent is given a triplet (v, d <t , q t ) as an input, consisting of the image, the dialog history, and a visual question. d <t denotes all dialog rounds before the t-th round. The agent is then expected to predict a ground-truth answer a gt t . There are two broad classes of methods in VisDial: generative and discriminative. Generative models aim to generate the ground-truth answer by maximizing the log-likelihood of a gt t , whereas discriminative models are trained to retrieve the ground-truth answer from a list of answer candidates a gt t ∈ a 1 t , · · · , a 100 t . Our main focus is the generative models since they do not need pre-defined answer candidates and thus more practical to be deployed in the real-world applications.

Generative Self-Training (GST)
This subsection describes our approach, called GST, that generates multi-turn visual QA data and utilizes the generated data for training. An overview of GST is shown in Figure 1. We have a human-labeled VisDial dataset where v n is a given image, and each dialog d n = { c n dn,0 , (q n,1 , a gt n,1 ) dn,1 , · · · , (q n,T , a gt n,T ) d n,T } consists of an image caption c and T rounds of QA pairs.
In the following, we omit the superscript gt in the ground-truth answer for brevity. GST first trains a teacher P T and a questioner P Q with the labeled dataset L via supervised learning. It then retrieves unlabeled images U = {ṽ m } M m=1 from the Conceptual 12M dataset [44] using a simple outlier detection model, the multivariate normal distribution. Next, the questioner and the teacher generate the visually-grounded dialogd for the unlabeled imageṽ via multimodal conditional text generation, finally yielding a synthetic dialog datasetL = {(ṽ m ,d m )} M m=1 . We call this dataset the machine VisDial data to distinguish it from the human-labeled VisDial dataset [1] (short for the human VisDial data). Finally, a student P S is trained on a combination of the human and the machine VisDial data, while applying perplexity-based data selection (PPL) and multimodal consistency regularization (MCR) to the machine VisDial data. We describe the details of each process in the following parts.
Teacher & questioner training. As mentioned in the introduction, a series of question and answer pairs for the unlabeled images should be generated to train the answering agent. Accordingly, GST first trains the answer generator, the teacher model P T , on the human VisDial dataset. Specifically, the teacher learns to generate the ground-truth answer's word sequence a = (w 1 , · · · , w S ), given the image, the dialog history, and the question. It is optimized by minimizing the negative log-likelihood of the ground-truth answer. Formally, log P T (a n,t |v n , d n,<t , q n,t ) log P T (w s |v n , d n,<t , q n,t , w <s ) (1) where N , T , and S denote the number of data tuples in human VisDial data, dialog rounds, and the sequence length of the ground-truth answer, respectively. w <s indicates all word tokens before the s-th token in the answer sequence. Similar to the teacher, the questioner is trained to generate the question at round t, given the image and the dialog history until round t − 1 (i.e., P Q (q t |v, d <t )). The questioner is also optimized by minimizing the negative log-likelihood of the follow-up question. Note that the teacher and the questioner are trained separately to prevent possible unintended coadaptation [4]. Both the teacher and the questioner are based on encoder-decoder architecture, where an encoder aggregates the context, and a decoder generates the target sentence. We implement the models by integrating a pre-trained vision-and-language encoder, VisDial-BERT [21], with the transformer decoder [45]. We refer readers to Appendix A for a detailed architecture.
Unlabeled in-domain image retrieval. Inspired by the work [30] that highlighted the importance of using in-domain data, GST retrieves in-domain image data from the Conceptual 12M dataset [44] with an out-of-distribution (OOD) detection model. Specifically, we extract the D-dimensional feature vector for each image in the human VisDial dataset by using the Vision Transformer (ViT) [46], yielding a feature matrix for the entire images X = (X 1 , · · · , X N ) ∈ R N ×D . Based on the feature matrix, we build the multivariate normal distribution whose dimension is D, i.e., X ∼ N D (µ, Σ). We regard this normal distribution as the empirical distribution of the human VisDial images and perform the OOD detection by identifying the probability of each feature vector for the unlabeled image. Consequently, the top-M unlabeled images are retrieved out of 12 million Web images.
Visually-grounded dialog generation. This process mimics a scenario where two people have a conversation about the given images. Given the retrieved images U = {ṽ m } M m=1 , our goal is to generate the visually-grounded dialogs {d m } M m=1 where each dialogd consists of the image caption and T rounds of QA pairs. In an actual implementation, we use the image captions in the Conceptual 12M dataset [44] and thus do not generate the captions. The QA pairs are sequentially generated. Specifically, the imageṽ, the captionc, and the generated QA pairs until round t − 1 are used as inputs when the questioner generates the question at round t (i.e.,q t ). After then, the teacher produces the answerã t based on the imageṽ, the dialog historyd <t , and the questionq t . Finally, GST produces the machine VisDial datasetL = {(ṽ m ,d m )} M m=1 . Student training with noisy data. As shown in Figure 1, the student P S is trained on the combination of the machine and the human VisDial data. According to many studies [35,36,33,29] in self-training, selectively utilizing the samples in the pseudo-labeled dataset is a common approach since the confidence of the teacher model's predictions varies from sample to sample. In NLP literature, the language model-based data filtering [47] has been proposed. To this end, we introduce a simple yet effective data selection method for the sequence generation problem, perplexity-based data selection (PPL). PPL is to utilize the answers whose perplexity of the teacher is below a certain threshold. Perplexity is defined as the exponentiated average negative log-likelihood of a sequence; the lower, the better. Furthermore, inspired by the consistency regularization [34,33] widely utilized in recent SSL algorithms, we also propose the multimodal consistency regularization (MCR) to improve the generalization capability of the student. MCR encourages the student to yield predictions similar to the teacher's predictions even when the student is provided with perturbed multimodal inputs. Finally, we design a loss function for the student as: log P S (a n,t |v n , d n,<t , q n,t ) where M , 1, and τ denote the number of data tuples in machine VisDial data, indicator function, and selection threshold, respectively. The loss function is the sum of the losses for the machine and the human VisDial data. PPL and MCR are applied to compute the loss of the machine VisDial data. PPL is used in the indicator function above, selecting the synthetic answers whose perplexity of the teacher is below the threshold τ . It implies that the unselected answers are ignored during training. The teacher's perplexity of each answer is computed in the dialog generation step above. Next, M denotes the stochastic function for MCR that injects perturbations to the input space of the student. Inspired by ViLBERT [25], we implement the stochastic function by randomly masking 15% of image regions and word tokens. Specifically, masked image regions have their image features zeroed out, and the masked word tokens are replaced with a special [MASK] token. The intuition behind MCR is minimizing the distance between the perturbed (i.e., masked) predictions from the student and the unperturbed predictions (i.e.,ã m,t ) from the teacher. It indicates that MCR is not applied when the teacher generates the synthetic answers. We believe MCR makes the student robust to the input noise, and PPL encourages the student to maintain a low entropy (i.e., confident) in noisy data training. The student and the teacher have the same model capacity and leverage the same pre-trained weights of the VisDial-BERT [21] model. Evaluation protocol. We follow the standard evaluation protocol established in the work [1] for evaluating visual dialog models. The visual dialog models for both generative and discriminative tasks have been evaluated by the retrieval-based evaluation metrics: mean reciprocal rank (MRR), recall@k (R@k), mean rank (Mean), and normalized discounted cumulative gain (NDCG). Specifically, all dialogs in VisDial contain a list of 100 answer candidates for each visual question, and there is one ground-truth answer in the answer candidates. The model sorts the answer candidates by the log-likelihood scores and then is evaluated by the four different metrics. MRR, R@k, and Mean consider the rank of the single ground-truth answer, while NDCG 2 considers all relevant answers from the 100-answers list by using the densely annotated relevance scores for all answer candidates. The community regards MRR and NDCG as primary evaluation metrics.
Implementation details. As mentioned in Section 2.2, we integrate the vision-and-language encoder for a discriminative task (i.e., VisDial-BERT [21]) with the transformer decoder for sequence generation (i.e., BERT BASE [45]) to train the teacher, the questioner, and the student. The encoder architecture is based on the vision-and-language pre-training model, ViLBERT [25]. The decoder has 12 layers of transformer blocks, with each block having 12 attention heads and a hidden size of 768.
The maximum sequence length of the encoder and the decoder is 256 and 25, respectively. We extract the feature vectors of the input images by using the Faster R-CNN [49,50] pre-trained on Visual Genome [51]. The number of bounding boxes for each image is fixed to 36. We set the threshold for PPL τ to 50. We train on one A100 GPU with a batch size of 72 for 70 epochs. Training time takes about 3 days. We use the Adam optimizer [52] with an initial learning rate 1e-5. The learning rate is warmed up to 2e-5 until 10k iterations and linearly decays to 1e-5. In visually-grounded dialog generation, the questioner and the teacher decode the sequences using the top-k sampling [53][54][55] with k = 7 and the temperature of 0.7. We use the top-k sampling since its computation is cheap yielding accurate and diverse sequences. Furthermore, we apply the 4-gram penalty [56,57] when generating visual questions to ensure that no 4-gram appears twice in the questions for each dialog.
Preprocessing of synthetic data. The size of the machine VisDial data (i.e., M ) is 3.6M which is nearly 30x larger than that of the human VisDial data (N = 0.12M). Accordingly, we split the machine VisDial data into 30 chunksL = {C i } 30 i=1 where each data chunk contains ∼120k imagedialog pairs, i.e.,C = {(ṽ j ,d j )} 120k j=1 . Therefore, we regard the size of each data chunk equal to the size of the human VisDial data, i.e., |L| = |C| = N . We use the machine VisDial data in units of data chunks. Note that the machine VisDial data contains 36M QA pairs since each dialog contains 10 QA pairs. 11.7M QA pairs out of 36M (∼32%) is actually utilized after applying perplexity-based data selection when τ is 50. Consequently, the total amount of the training data is 12.9M QA pairs, combining the machine VisDial data (11.7M QA pairs) with the human VisDial data (1.2M QA pairs).

Quantitative Results & Analysis
Comparison with baselines. We compare our proposed method with the baseline models to validate the effectiveness of GST. In this experiment, the student is trained on the full human VisDial dataset with size N and a subset of the machine VisDial dataset (5 out of 30 data chunks) with size 5N (see Section 3.1). First, we introduce two baseline models: the teacher and the teacher with continued pre-training. The teacher is the encoder-decoder model trained on the human VisDial dataset. It is built on the pre-trained model, i.e., VisDial-BERT [21], by adding the decoder [45] for the multimodal conditional text generation. The teacher with continued pre-training (CPT) is the model that utilizes  a slightly different encoder. Specifically, we continue pre-training the VisDial-BERT with imagecaption pairs in the machine VisDial data. The sum of masked language modeling (MLM) loss and masked image region (MIR) loss is optimized by following the pre-training stage of VisDial-BERT. CPT is proposed to identify the effect of utilizing additional vision-and-language data (i.e., Conceptual 12M [44]) in the pre-training step. In Table 1, the student shows better performance than the teacher and the teacher with CPT on NDCG and MRR. It indicates that GST is a nontrivial approach to improving performance. Surprisingly, we observe that CPT results in a considerable drop in both evaluation metrics. We conjecture that the performance drop is due to low-precision image captions in the Conceptual 12M dataset, as mentioned in the paper. On the other hand, the student still shows competitive performance even if it also utilizes the captions in the dialog history.
Ablation study. We perform an ablation study to illustrate the effect of each component in GST.
In the same way as the experiment above, a subset of the machine VisDial data (five data chunks) is utilized along with the human VisDial data to train the student. We report the performance of ablative models:  Table 1, we observe all components play a significant role in boosting the performance.
Perplexity-based thresholding. We conduct another ablation experiment with different values of the selection threshold τ ∈ {10, 30, 50, 100, 200, ∞} in the perplexity-based data selection (PPL). We validate the models on a subset of VisDial v1.0 train split -2k dialog data -since the VisDial v1.0 validation split is utilized for the state-of-the-art model comparison in the following experiment. Note that all models in this study do not have access to the data for validation during training. As shown in Table 2, the student with τ = 50 shows the best performance on MRR. NDCG is not supported in the train split due to the absence of the densely annotated relevance scores. Nearly 30% of QA pairs in the machine VisDial data are actually utilized when the threshold τ is 50. Unless otherwise stated, the threshold is 50. Moreover, the teacher shows results comparable to the student with τ = ∞. It indicates that utilizing the unfiltered machine VisDial data is not much different from not using it.
Is GST helpful when human-labeled data is scarce? We investigate this question to identify the effect of GST in the low-data regime. We assume that only a small subset of the human VisDial data (1%, 5%, 10%, 20%, and 30%) is available. Therefore, the size of the human-labeled data is 0.01N , 0.05N , 0.1N , 0.2N , and 0.3N , respectively. We first train the teacher and the questioner on such scarce data, and then these two agents generate a new machine VisDial data for unlabeled images in the Conceptual 12M dataset [44] with size N . The student is then trained on the newly generated machine VisDial data and the small amount of the human VisDial data. Note that PPL and MCR are still applied in this experiment. In Table 3, GST yields huge improvements on both metrics, especially NDCG, boosting up to 9.35 absolute points compared with the teacher. We observe that the smaller the amount of human-labeled data, the larger the performance gap between the teacher and the student on NDCG. It implies that GST is helpful, especially when human-labeled data is scarce. We think the results in the low-data regime are particularly remarkable in other dialog-based tasks [58][59][60][61] since many tasks provide the human dialog data less than 0.3N (i.e., 360k QA pairs).  Scaling up the size of the machine VisDial data. We also conduct experiments on the high-data regime where the entire human VisDial data and the machine VisDial data with different number data chunks (1,5,10,20, and 30 out of 30 data chunks) are used to train the student. In Table 4, the student shows an increase in performance compared with the teacher except when using one data chunk. Notably, we observe significant gains in the student when increasing from one to five data chunks (i.e., 1x → 5x). NDCG seems to be saturated when using more than five data chunks, but MRR increases monotonically. The student leveraging the entire machine VisDial data (x30) shows the best performance on average.
Effect of in-domain image retrieval. To study the effect of utilizing in-domain images, we compare two student models trained on different machine VisDial data. One is the model that utilizes the machine VisDial data for in-domain (i.e., retrieved) images, and the other uses the machine VisDial data for out-of-domain images (i.e., discarded images in Figure 1). In Table 5, two models show comparable results on the metrics that consider the single ground-truth answer (i.e., MRR, R@k, and Mean), but we observe that utilizing in-domain data improves NDCG considerably. It demonstrates that the use of in-domain data helps to find correct or relevant answers more accurately.
Comparison with state-of-the-art. We compare GST with the state-of-the-art approaches on the validation set of the VisDial v1.0 and v0.9 datasets, consisting of UTC [23], MITVG [19], VD-BERT [22], LTMI [18], KBGN [17], DAM [16], ReDAN [12], DMRM [15], Primary [11], RvA [9], CorefNMN [8], CoAtt [7], HCIAE [5], and MN [1]. We decide to use the validation splits since all previous studies benchmarked the models on those splits. In Table 6 Notice that the teacher also shows competitive results compared with other pre-training methods, i.e., UTC, MITVG, and VD-BERT that utilize the pre-trained weights of ViLBERT [25], transformer [62], and BERT [24], respectively. The results of the discriminative model can be found in the Appendix B.1.  Table 6: Comparison with the state-of-the-art generative models on both VisDial v1.0 and v0.9 validation datasets. ↑ indicates higher is better. ↓ indicates lower is better. NDCG is not supported in v0.9 dataset. † denotes that the models are re-implemented by the previous work [12]. The standard deviations of our proposed models are reported ± with three different initialized models. The student model utilizes the entire machine VisDial dataset (i.e., 30x).  Figure 2: A visualization of human conversation and machine conversation on the VisDial v1.0 validation images. We excerpt the human conversation from the validation dataset, and the questioner and the student generate the machine conversation. The red-colored text denotes an incorrect answer.

Qualitative Results & Analysis
For qualitative analysis, we visualize the generated conversations from our models and the ones from human. We excerpt the human conversation from the VisDial v1.0 validation dataset, and the questioner and the student generate the machine conversation using the image and the caption in the validation data. As in Figure 2, diverse visual questions and correct answers are generated in the machine conversation. For instance, from D3 to D6 in the first example, the question generation model asks about "a cell phone," which first appears in the caption, whereas the human questioner deals with different topics. The student responds appropriately to the given questions. However, we also identify that the student sometimes fails to generate correct answers (i.e., the red-colored text in D9 of the last example), which shows the importance of more precise vision-and-language grounding.

Related Work
Visual dialog. Visual Dialog (VisDial) [1] has been proposed as an extended version of Visual Question Answering (VQA) [27,50,63], where a dialog agent should answer a series of interdependent questions using an image and the dialog history. Prior work has developed a variety attention mechanisms [5][6][7][8][9][10][11][12][13]18] considering the interactions among the image, dialog history, and question. Some studies [14,20] have attempted to discover the semantic structures of the dialog in the context of graph neural networks [64] using the soft attention mechanisms [65]. From the learning algorithm perspective, all of them have relied on supervised learning on VisDial data. More recently, a line of research [21][22][23] has employed self-supervised pre-training to leverage the knowledge of related vision-and-language datasets [26][27][28]. However, our approach is based on semi-supervised learning and produces the task-specific data (i.e., visual dialogs) for unlabeled images to train the dialog agent.
Sequence generation in vision-and-language tasks. Many studies have generated natural language for the visual inputs such as image captioning [66,50], video captioning [67,68], visual question generation (VQG) [69][70][71][72][73][74], visual dialog (VisDial) [1,12], and video dialog [59,75]. Furthermore, recent studies [76,77] have produced text data for vision-and-language pre-training. GST is similar to these studies in that the model generates the text data, but our focus is on studying the effect of semi-supervised learning (SSL) on top of such pre-training approaches. To the best of our knowledge, GST is the first approach to show the efficacy of SSL throughout a wide range of visual QA tasks.
Neural dialog generation. In NLP literature, extensive studies have been conducted regarding neural dialogue generation for both open-domain dialogue [78][79][80][81][82][83] and task-oriented dialogue [84,85]. Our approach is similar to neural dialogue generation in that the model should generate a corresponding response based on the dialog history and the current utterance. However, we aim to produce visuallygrounded dialogs, and thus the image-groundedness of the question and the semantic correctness of the answer are important. On the other hand, neural dialogue generation considers many different aspects: specificity, response-relatedness [86], interestingness, fluency [87], and diversity [80].

Discussion
Relationship between self-supervised pre-training and generative self-training. We develop the teacher, the questioner, and the student models on top of VisDial-BERT [21] which leverages visionand-language pre-training [25]. Thus, the teacher can be understood as a typical model that follows the pretrain-then-transfer learning strategy mentioned in the introduction, whereas the student leverages both pre-training and generative self-training. By comparing the three models -the teacher, the student without pre-training, and the student -in Table 1, we identify that self-supervised pre-training and GST are complementary.
Ethical considerations. Since GST generates the visually-grounded dialogs, our proposed models have the potential to produce biased and offensive language, although arguably to a lesser extent than the open-domain dialog [78][79][80][81][82][83]. We attempt to mitigate ethical concerns such as biases against people of a certain gender, race, age, and ethnicity or the use of offensive content. Our proposed method utilizes the images and the captions in the Conceptual 12M dataset [44], where several data cleansing processes (e.g., the offensive content filtering or replacing each person name with the special <PERSON> token) have been conducted. At least, we could not find any conversation violating the ethical considerations in a manual inspection by visualizing ∼100 synthetic dialogs.

Conclusion
We propose a semi-supervised learning approach for VisDial, called GST, that explicitly generates a synthetic visual dialog dataset for unlabeled Web images via multimodal conditional text generation. We also present the perplexity-based data selection and the multimodal consistency regularization to effectively leverage the generated dialogs. The experiments quantitatively support the effectiveness of our proposed method. GST achieves the new state-of-the-art performance on the VisDial v1.0 and v0.9 datasets outperforming highly competitive methods. Moreover, our methods show strong performance gains when the human-labeled dialog data is scarce. We believe the idea of GST is generally applicable to other multimodal generative domains and expect our proposed method to open the door to leveraging unlabeled images in many visual QA tasks.

A Details of Model Architecture
A detailed architecture of our proposed model is presented in Figure 3. We use the encoder-decoder model, where the encoder aggregates the multimodal context, and the decoder generates the target sentence using the hidden states of the encoder. The given image, the dialog history, and the question are the context when we train the answerer models (i.e., the student and the teacher). As shown in (a) for Figure 3, the question is concatenated to the dialog history when training the answerer models. In contrast, the questioner uses the given image and the dialog history as context.
We employ the VisDial-BERT model [21] as our encoder, which leverages the pre-trained weights of the ViLBERT model [25]. We employ the BERT BASE model [24] for sequence generation [45] as our autoregressive decoder. The decoder has 12 layers of transformer blocks, with each block having 12 attention heads and a hidden size of 768. We present a detailed view of the encoder in (b) for Model architecture. Although our main focus is the generative model, we conduct additional experiments to identify the effect of GST in the discriminative model. As mentioned in Section 3.1, our proposed models (i.e., the student, the teacher, and the questioner) are based on encoder-decoder architecture where the encoder is the vision-and-language encoder model (i.e., VisDial-BERT [21]) used for the discriminative task, and the decoder is the transformer decoder [45]. In this experiment, the student is the encoder-only model, the same model architecture as the VisDial-BERT model.
Tricks for adapting to a discriminative task. The goal of the discriminative task is to retrieve the ground-truth answer from a list of answer candidates. It implies that the human VisDial dataset [1] contains the pre-defined answer candidates for each question to train and evaluate the discriminative models. However, the machine VisDial dataset generated by our proposed models does not include   the answer candidates since the dataset is generated to train the generative models that do not need the answer candidates. To circumvent this issue, GST first trains the student model for the generative task, i.e., the encoder-decoder model, on the entire machine VisDial data (i.e., 30x). Then, we extract the trained weights of the encoder in the student and initialize the encoder-only model with the weights. Finally, the encoder-only model is trained to retrieve the ground-truth answer from the list of answer candidates using the human VisDial dataset. This trick circumvents the need for the answer candidates when training the machine VisDial data.
Results and analysis. We compare the student model with the state-of-the-art approaches in the discriminative task, consisting of VisDial-BERT [21], UTC [23], and VD-BERT [22]. As shown in Table 7, GST outperforms the SOTA approaches on all evaluation metrics. It is worth noticing that GST boosts NDCG 0.66% (64.94 → 65.60) compared with the VisDial-BERT model, whose configuration is almost the same as the student except for the use of the machine VisDial data. We further benchmark our method on the VisDial v1.0 test standard split. In Table 8, GST shows improved performance on NDCG and MRR compared with the VisDial-BERT model. Recent studies finetune the VisDial models on the densely annotated labels 3 in the validation dataset and evaluate the models on the test dataset. By following the method applied in SGL+KT [20], we also train the student model on the dense labels. In Table 8, the models like VisDial-BERT and VD-BERT show huge improvements on NDCG and counter-effect on other metrics when utilizing the dense labels. However, GST still boosts NDCG, maintaining strong performance on other metrics.

B.2 Visualization of Answer Predictions
We visualize the answer predictions from the student and the teacher models given the same context (i.e., the image, the dialog history, and the question). As shown in Figure 4, the student indeed produces correct answers compared with the teacher. Moreover, both models produce many plausible answers, although the answers are different from the ground-truth answers (see the blue-colored text). We conjecture that the ability to generate such plausible answers is evaluated as a high NDCG score; NDCG considers all relevant responses in the answer candidates. Figure 4: A visualization of answer predictions from the student and the teacher model. The redcolored text denotes an incorrect answer. The blue-colored text indicates the prediction different from the ground-truth answer, but it seems correct or plausible.