{'fraction_non_alphanumeric': 0.039776928765097364, 'fraction_numerical': 0.01136923835346315, 'mean_word_length': 4.80937891533918, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
ProQA: Structural Prompt-based Pre-training for Unified Question Answering

Question Answering (QA) is a longstanding challenge in natural language processing. Existing QA works mostly focus on specific question types, knowledge domains, or reasoning skills. The specialty in QA research hinders systems from modeling commonalities between tasks and generalization for wider applications. To address this issue, we present ProQA, a unified QA paradigm that solves various tasks through a single model. ProQA takes a unified structural prompt as the bridge and improves the QA-centric ability by structural prompt-based pre-training. Through a structurally designed prompt-based input schema, ProQA concurrently models the knowledge generalization for all QA tasks while keeping the knowledge customization for every specific QA task. Furthermore, ProQA is pre-trained with structural prompt-formatted large-scale synthesized corpus, which empowers the model with the commonly-required QA ability. Experimental results on 11 QA benchmarks demonstrate that ProQA consistently boosts performance on both full data fine-tuning, few-shot learning, and zero-shot testing scenarios. Furthermore, ProQA exhibits strong ability in both continual learning and transfer learning by taking the advantages of the structural prompt.


Introduction
Question Answering has long been an inspirational challenge in NLP research, and is viewed as the next-generation search engine and an essential tool for human beings to obtain knowledge (Etzioni, 2011). Many distinct datasets (Rajpurkar et al., 2016;Lai et al., 2017;Gao et al., 2021) have been proposed along with the research trend on QA, involving very diverse question types (e.g., extractive QA, abstractive QA, multiple-choice QA), domains (e.g., finance, daily events), and answer types (e.g., free-formed text, selected option). The majority of previous works focus on tasks with specific question types (Lai et al., 2017;Yang et al., 2018;Gao et al., 2020) or specific domains (Trischler et al., 2017;. Recent research on large pre-trained language models (Brown et al., 2020;Bommasani et al., 2021) indicates that there may be tight connections among various tasks, which sheds light on a unified paradigm that can be potentially applied to solve various QA tasks to model their commonality.
This observation motivates us to develop a unified QA model, which can model both the commonly-required QA ability and the difference between various QA tasks within a same paradigm. To achieve this goal, there are several key challenges needed to be addressed: (1) How to model commonalities and enhance transferability among different QA tasks in various domains/formats while reducing the conflict between them? (2) How to construct large-scale QA corpus as the highquality QA-centric data is scarce for pre-training?
In light of this, we conceive ProQA, a unified QA paradigm, which builds up a general model to solve different QA tasks utilizing a structural prompt and improves commonly-required QA ability via structural prompt-based pre-training. Firstly, to model the commonalities and distinguish task differences, we adopt a structural prompt to organize the inputs with a unified structurally designed input schema. As illustrated in Fig. 1, given the complex components (e.g., "Domain", "Format", "Task", "Question", "Passage") as inputs, ProQA divides components into multiple key-value pairs, in which a specific component like "Question" denotes a key, and the specific in-  stance in this component is taken as the value. In this way, the model can discriminate different input components by key indicators and model the speciality of each task via task-specific values (learnable prompts).
Secondly, to alleviate data sparsity problem and empower the model with transferability to the adaptation of new tasks, we conduct structural promptbased pre-training. We first build a large-scale synthetic QA corpus automatically from Wikipedia, utilizing only a few seed datasets as the prior supervisions for pre-training corpus construction and finally covering primary QA formats. Then we format the pre-training data with the structural prompt, and teach the model to learn the general purpose QA-centric ability and the functionality of each component in the structural prompt via pre-training.
We evaluate the effectiveness of ProQA on 11 downstream QA benchmarks, and the results show that our system achieves consistent performance boost in full data fine-tuning, few-shot learning, and zero-shot learning settings. Experiments demonstrate that ProQA can better mitigate the catastrophic forgetting issue during continual learning by restoring the task-specific soft prompts residing in the structural prompt. Further analyses illustrate that our model has better transferability as it can be more quickly adapted to a newly involved task. Ablation studies verify the effectiveness of both the soft prompt and prompt-based pre-training.
The contributions are summarized as follows: • We propose ProQA, a unified QA framework for solving various tasks within a single paradigm, taking an extensible and learnable structural prompt as the bridge. • We enhance general QA-centric capabilities via structural prompt-based pre-training. • Comprehensive experiments show that our model consistently improves the performance on 11 QA tasks especially in low-resource settings and exhibits better effectiveness in continual learning and few-shot transfer learning.

Related Work
Unifying QA formats. Despite vast diversity of current QA tasks in question type, answer type, an-swer source, and data domain (Zeng et al., 2020), there have been efforts in exploring a unified format for various QA tasks. Some pioneered to demonstrate the generalization and transferability among different QA tasks (Talmor and Berant, 2019;Dua et al., 2019a;Fisch et al., 2019). Another line of works investigate multi-task learning for QA (Mc-Cann et al., 2018;Shen et al., 2019;Deng et al., 2019) by jointly training a single encoder to promote knowledge sharing. However, these methods typically require deploying distinct prediction heads for different tasks, which lead to poor scalability and flexibility when confronted with emerging QA tasks of new types.
To this end, inspired by the success of casting multiple tasks into the same text-to-text format Raffel et al., 2020), researchers propose to learn a single model to unify various QA formats, alleviating the labor of taskspecific designs (Khashabi et al., 2020b;Tafjord and Clark, 2021). However, these models (1) do not explicitly model the task or component characteristics, thus failing to properly disentangle the difference among QA tasks; and (2) overly rely on supervised data from specific tasks, which may not be available under data-scarce scenarios.
QA-centric pre-training. Numerous efforts have been spent on improving PLMs' reasoning abilities with an intermediate pre-training stage before fine-tuning on target QA tasks, including (1) language modeling adaptation with salient span masking, which trains PLMs to recover randomly chosen (Guu et al., 2020; or machine-generated (Kang et al., 2020) masked named entities in the raw corpus; (2) training data augmentation (Zhong et al., 2022) with synthetic question-answer-context triples, such as generating (a) pseudo questions through adversarial training (Hosking and Riedel, 2019;Li et al., 2019), knowledge bases (Hu et al., 2021) or machine translation (Lewis et al., 2019;, (b) pseudo answers exploiting recurring spans (Ram et al., 2021) or rules based on heuristics (Bian et al., 2021) and (c) pseudo contexts via information retrieval (Glass et al., 2020). Nevertheless, these works largely target at improving a certain reasoning ability for PLMs, and thus cannot be easily generalized to other QA tasks.
Prompts for PLMs. To effectively stimulate the knowledge acquired through pre-training, prompt-oriented fine-tuning is receiving increasing attention Ding et al., 2021), which re-formulates the objective of downstream tasks similar to that of pre-training by inserting manually designed (Schick and Schütze, 2021a,b) or automatically searched Shin et al., 2020) hard prompt tokens into the input text. Considering that discrete prompts may not be an optimal solution in the continuous embedding space, recent works (Li and Liang, 2021;Hambardzumyan et al., 2021) proposed tunable soft prompts. It achieves satisfying performance especially when the model size grows extremely large (Lester et al., 2021). Compared with the cumbersome parameters in PLMs, soft prompts are lightweight and pluggable, which paves the way for our goal of flexible adaptation to a new QA task.

ProQA
In this section, we detailedly describe the whole framework of ProQA for general purpose QA, which solves various QA tasks within the same paradigm.

Overview
As shown in Fig. 2, we organize the inputs of various QA tasks with a unified structural prompt ( § 3.2), and adopt a unified model for question answering. Then, to enhance the model in learning the QA-centric ability and the semantics of the structural prompt, we conduct structural promptbased pre-training with synthetic pre-training corpus formatted with the structural prompt ( § 3.3).
Inspired by Khashabi et al. (2020b) and T5 (Raffel et al., 2020), we solve all downstream QA tasks with a unified text-to-text model. In this work, we mainly adopt T5 as the model backbone. Taking the structural prompt-based model input, the unified model generates the answer of the question.

Structural Prompt
Here we detailedly illustrate the design of the structural prompt and its formatted input to the model.
Definition. We organize complex QA task inputs with the structural prompt. As shown in Fig. 2, the structural prompt consists of multiple {key : value} pairs, where the key represents a specific component 2 (e.g., "Task", "Format", "Question", etc.), and the value has two possible types: (1) textual content (e.g., question, passage, options) of the data instance; (2) task attributes (e.g., format, domain) represented as the combination of a discrete hard prompt and continuous soft prompts. The hard prompt is a predefined discrete description (we adopt a special token here), and the soft prompts are lightweight learnable and pluggable continuous embeddings that are proven to be parameter-effective in task adaptation (Lester et al., 2021). The structural prompt-formatted examples are illustrated in Fig. 1. In the case of the SQuAD dataset, "⟨Format Prompt⟩", "⟨Task Prompt⟩", "⟨Domain Prompt⟩" will be "⟨Extractive QA⟩", "⟨SQuAD⟩", "⟨Wikipedia⟩", respectively. To enhance the model in discriminating the functional difference between components, we adopt a special key indicator with learnable representation to represent each key. Furthermore, to model the difference between several tasks/domains/formats, we also adopt learnable and storable specific soft prompts as the value to represent their customized characteristics, which makes the model more flexible for task adaptation.
As a result, the structural prompt can empower the model in the following aspects: (1) modeling knowledge generalization of various tasks utilizing a unified input schema; (2) discriminating different components with the special key indicator; (3) customizing the speciality of each task/format/domain with learnable and storable soft prompts as the value under corresponding keys.
Input Representation. Specifically, given a structural prompt-formatted instance, we describe the specific representation of the model input. We firstly translate k th key to a key indicator D k (a special token), which is attached by the tokens V k of the specific value to form a token sequence. It is further represented as The representation of D k is initialized and updated during training. Since we use soft prompts P task /P format /P domain as the value of the corresponding key and they are commonly required for all the tasks, we prepend them to the input for convenience and concatenate all the E k to form the final model input X: It is also worth noting that the representations D of key indicators and the soft prompts P are jointly trained with the main model parameters during pretraining for learning the semantics of the structural prompt. Moreover, after being tuned by various tasks, the soft prompts P can be stored to record the customized task-specific characteristics.

Structural Prompt-based Pre-training
In this part, we introduce how we conduct structural prompt-based pre-training to help the model in learning commonly-required QA ability and the semantics of the structural prompt during pre-training to facilitate the adaption of the structural prompt to downstream tasks.
Task Formulation. Along with the structural prompt-based paradigm, we manifest various exemplary QA format types (i.e., Extractive QA, Abstractive QA, Multiple-choice QA and Yes/No QA) for pre-training to inject the general QA-centric ability. Given the multi-format QA pre-training corpus, we transform all QA formats according to the proposed structural prompt, which enables joint pretraining while keeping the differences among various formats. Taking a structural prompt-formatted instance as the input and a free-form answer as the output, the task is further tailored to a QA task with the encoder-decoder model.
Pre-training Corpus Construction. When we prepare the QA pre-training corpus, data sparsity problem is extremely severe because (1) it is impractical and laborious to obtain a large-scale highquality annotated data for pre-training and (2) it is hard to generate QA-centric self-supervised data using rule-based methods (e.g., token masking or sentence reordering). In this work, inspired by Lewis et al. (2021), we adopt a generationfiltering based corpus construction method to synthesize a large-scale pre-training corpus, based on a large-scale unlabeled Wikipedia corpus with almost 6 million passages.
Typically, the general generation-filtering process consists of the following components: 1. A QA-pair generation model g qa (q, a|c): Given a passage c as input, g qa (q, a|c) generates q [SEP] a as the output sequence including a pair of question q and its answer a. 2. A filtering QA language model f (a|q, c) for filtering the generated QA-pairs to ensure the quality and consistency of the question and the answer. f (a|q, c) is a conditional-probabilitybased approach to filter out QA pairs softly. It scores a QA pair (q, a) with the likelihood of the answer a conditioned on the passage c and question q. The QA-pairs with scores higher than a threshold will be kept for pre-training. We adopt the same text-to-text pre-trained model T5 described in § 3.1 as the model backbone of both the generation and filtering model.
To ensure the reliability of the generation and filtering models, we inevitably select a few seed datasets (typically one for each QA format type) as the prior supervisions to train these models. It is worth mentioning that, we avoid using more supervised data for corpus construction, because we expect the whole paradigm to have better expandability. In other words, if we want to extend the paradigm for a newly-involved QA format type but with limited supervised data, we can utilize these data to automatically create a synthetic large-scale pre-training corpus.
More specifically, the construction method has little variance for different formats according to their input components. For Extractive QA and Abstractive QA, we adopt the aforementioned general method to synthesize QA-pairs. We also tried to first extract answers using rule-based method (extracted named-entities or key phrases), and only generate questions. We empirically find that this method performs much worse as it involves simple bias of the rule-based method. As the inputs for Multiple-Choice QA involve a new component "Candidate Answers", we adopt a distractor (negative options) generation model g neg (o|c, q, a) to generate three negative options o. 4 Experimental Setup

Datasets and Evaluation Metrics
We consider three formats of QA datasets in our experiments: Extractive QA, Abstractive QA and Multiple-Choice QA 3 . For each QA format, we select one seed dataset for preparing the large-scale pre-training data. The seed dataset is used to train the question-answer generation and filtering models in the process of pre-training corpus construction. In total, the experiments are conducted on 11 QA datasets with three different formats and various language understanding abilities. An overview of datasets used in the experiments and their required QA skills are summarized in Table 1.
Extractive QA. We take SQuAD 1.1 (Rajpurkar et al., 2016) as the seed dataset for extractive style QA. In addition, we consider NewsQA (Trischler et al., 2017) and Quoref (Dasigi et al., 2019) to evaluate the generalization ability of models. The EM (Exact Match) score between the extracted span and the gold answer span is used as the evaluation metric for extractive QA.
Abstractive QA. Narrative QA (NarQA) (Kočiský et al., 2018) is taken as the seed dataset for Abstractive QA. DROP (Dua et al., 2019b) and the open-domain version of NaturalQuestions (NQOpen) ) are also considered. Passages for each question in NQOpen are retrieved by the dense passage retriever (Karpukhin et al., 2020) and are concatenated into a sequence.  We use ROUGE-L (Lin, 2004) metric for NarQA and F1 score for DROP and NQOpen. The context for DREAM is in the dialogue style and we concatenate them into a sequence as the passage input. We select the option with the highest textual similarity with the generated answer as the final answer. We compute the accuracy of the correct options for all multiple choice QA datasets.

Approaches
T5 (Raffel et al., 2020) is a unified textto-text pre-training framework that covers all text-based language problems.
UnifiedQA (Khashabi et al., 2020b) crosses the format boundaries of different QA tasks by formulating them into text-to-text tasks under T5. It directly concatenates all inputs via \n into a sequence and feeds it into T5 for predicting the answer. We train our own UnifiedQA model on the combination of three aforementioned seed datasets, namely SQuAD, NarQA, and RACE.
ProQA is our proposed structural prompt-based pre-training approach. ProQA is pre-trained jointly on three formats of pre-training corpus: Extractive QA, Abstractive QA, and Multiple-Choice QA. This approach using corpus prepared from QA-pair generation-filtering model described in § 3.3 is named as ProQA (qapair). Additionally, we leverage the off-the-shelf large-scale QA pairs from Probably-Asked Questions/PAQ (Lewis et al., 2021), and replace our extractive QA pre-training corpus by a subset of PAQ (abstractive QA and multiple-choice QA corpus remains unchanged). PAQ provides a refined pipeline that introduces learned models on every step of QA pair generation, i.e., passage selection, answer identification, question generation, and filtering. We name this variant as ProQA (paq). For every downstream QA dataset, we start from the above pre-trained models and conduct experiments under full-data fine-tuning, few-shot learning, and zero-shot learning settings. For few-shot learning, we randomly sample 32 instances from the training set.

Main Results
Main results are shown in Table 2, and we have the following observations: • QA-centric pre-trained models, namely Uni-fiedQA and ProQA, outperform T5 by a large margin on both seed datasets and non-seed datasets. This is because there is some transferable knowledge across different QA tasks.  tomization ability than UnifiedQA -ProQA beats UnifiedQA by a large margin in few-shot and zero-shot settings. This is because (1) the hard and soft prompts in the structural prompt enable better knowledge customization for every QA task, especially the "Task" key-value pair that is different for every QA task; (2) structural prompt-based pretraining empowers ProQA to adapt faster ( § 5.3) and better (Table 2) to these non-seed datasets. • Comparing ProQA (qapair) and ProQA (paq), we find that ProQA (paq) performs better in most scenarios. Presumably, PAQ provides high quality pre-training corpus through its pipelined approach -there are in total four BERT-sized models to be prepared for generating PAQ corpus. Instead, our proposed QA pair generation approach is simple and can be applied to not only Extractive QA but also Abstractive QA and Multiple-choice QA in the pre-training corpus construction process.

Continual Learning via Soft Prompt
One benefit of introducing soft prompt in ProQA is that it can potentially mitigate the catastrophic forgetting issue when adapting to a new task. If ProQA is sequentially fine-tuned on task A and task B under few-shot setting, it can load task A soft prompt back when it is evaluated again on the task A. The plug-in flexibility of ProQA brings huge improvements compared with its counterpart that keeps the task B soft prompt. We conduct continual learning by setting task A and B as different combinations among datasets with formats 4 : Extractive QA (EX), Abstractive QA (AB), and Multiple-choice QA (MC). Formally, we first adapt ProQA to task A by few-shot learning to obtain the model A: f A θ with performance s A . Then we sequentially adapt f A θ to task B and receive task B model f AB θ . We evaluate performance 4 Note that we consider two tasks in continual learning because we also want to directly investigate the task adaptation to-and-fro the same format (e.g., MC → MC) or different formats (e.g., AB → EX).

EM
Step 0  of the model f AB θ on task A under two settings: (1) direct testing (task-B prompt) (2) first restoring the learned task-A prompt from f A θ to the model f AB θ and then testing. Performance of the two settings are denoted as s AB and s AB ′ , respectively. We evaluate the continual learning performance under these two settings with the percentage of the performance drop on the task A: "Task B Model"= s A −s AB s A , and "Task B Model (w/ Task A Prompt)" Table 3, the catastrophic forgetting issue does exist when evaluating task A with task B model ("Task B Model") directly. The performance drops as large as 26.2% for EX→AB. However, restoring task A prompt brings huge improvements across all task combinations ("Task B Model w/ Task A Prompt"). It is surprising to see that restoring task A prompt could sometimes even improve task A performance (MC→MC = −0.5%). Presumably, sequential learning two tasks under the same question format (MC) makes the model learn the transferable knowledge while restoring task A prompt brings task-specific knowledge. Detailed experimental results on the 33 combinations of datasets can be found in Appendix C.

Convergence Analysis
We investigate the effectiveness of pre-training by compare the step-wise performance under few-shot learning setting. The learning curves of EM scores on the validation set of the NewsQA task is shown  in Figure 3. Out of the three models, T5 convergences slowest because it does not have any QA-centric knowledge while our proposed ProQA adapts fastest and best. Moreover, we find that Uni-fiedQA EM score rapidly saturates and eventually degrades slightly, suggesting that the model overfits under the few-shot setting. On the counterpart, our ProQA continues to improve and never degrades because the hard and soft prompt inside the structural prompt balance the knowledge generalization and knowledge customization well.

Ablation Study
An ablation study is conducted to unveil the effectiveness of every component in ProQA. We consider three variants of ProQA: (1) ProQA without the soft prompt in its structural prompt; (2) ProQA further without prompt-based pre-training.
(3) Uni-fiedQA + Pre-train Corpus is the UnifiedQA model pre-trained on our prepared large-scale synthetic QA corpus. Results on three non-seed datasets under different QA formats are shown in Table 4. We find that removing the soft prompt from the model disables the task-specific knowledge learned during pre-training. Moreover, removing the promptbased pretraining drastically hurts the performance as the equivalent model (T5 + hard structural prompt) does not have any QA knowledge. Finally, UnifiedQA + Pre-train Corpus could not compete with ProQA, showing that our proposed structural prompt earns better balance between knowledge generalization and knowledge customization than UnifiedQA.

Discussion
In this section, we discuss on how to extend the ProQA to a new task even with a new schema, and sheds light on potential future directions. 1) Task Adaptation with Structural Prompt: The design of structural prompt empowers ProQA with better expandability. In our main experiments, we adopt 3 format types and 11 QA tasks. In the future, we can adapt ProQA to more tasks, formats, domains, and new input schema. Intuitively, when being adapted to a new task with unseen format/domain, ProQA can initialize the specific soft prompts and learn the characteristic of the new domain/task through model training. Moreover, if we encounter a new input schema that involves new keys (e.g., "extracted entities or commonsense knowledge"), we can add a new key-value pair in the input schema and learns the functionality of the new key indicator through training.
2) Unified QA Systems: We think further studies on unified QA systems could target on a better pre-training schema for general purpose QA, or optimizing the modeling strategy for the structural prompt to process more complex input, or output formats (e.g., adding extracted entities or retrieved knowledge).
3) Unification with Structural Prompt: The application of the structural prompt is not limited only on the QA task. Intuitively, task inputs/outputs with various formats or components can also be organized with the structural prompt, like sentiment analysis (Zhong et al., 2021), style transfer (Li et al., 2022). In this way, we can integrate multiple tasks with carefully organized structural input, and improve the uniformity and expandability of the whole paradigm.

Conclusion
We introduce ProQA, a unified QA paradigm that adopts a single model for solving various QA tasks with the bridge of a structural prompt. Structural prompt simultaneously models the common ability required for various tasks and keeps the speciality of each task, through a structurally designed learnable input schema. We further conduct structural prompt-based pre-training, seeking to empower the model with general QA-centric ability and injects the semantic knowledge of the structural prompt into the pre-training model. Experimental results on 11 QA benchmarks demonstrate that ProQA can significantly boost performance on all settings. Further analyses show that our method can better mitigate the catastrophic forgetting issue during continual learning, and our method can be adapted to a newly involved task more quickly, by taking the advantages of the structural prompt. In the future, we hope our analysis could inspire more explorations on the unified QA methods, or the unification of distinct tasks with complex inputs modeling by the structural prompt. We also hope structural prompt can be further utilized into the unification of more tasks with complex inputs.

Acknowledgments
Jian Yin is the corresponding author. Wanjun Zhong, Jiahai Wang and Jian Yin are supported by the Key-Area Research and Development Program of Guangdong Province (2020B0101100001). Yang Deng, Yuexiang Xie, Yaliang Li, Min Yang, Nan Du, Wei Fan, Kai Lei, and Ying Shen. 2019

A.1 Corpus Preparation
In this part, we describe the details of corpus construction.
The current pre-training corpus contains almost 4 million pre-training instances formulated with the structural prompt, including 1 million Multiplechoice QA instances, 2 million Extractive QA instances, and 2 million Abstractive QA instances. When generating questions and answers, we take the context as the input, and the sequence "question [SEP] answer" as the output. In order to train the filtering model, we take the context and question as the inputs, and the answer as the output. During the inference process of QA-pairs filtering, we take the context and the generated question as the model input of the QA model and set generated answer as the label. Then we compute the final soft score with the cross-entropy loss between the label and the answer generated by the QA model, Next, we rerank all the generated QA-pairs according to the soft scores in an ascending order to select the most consistent QA-pairs as the pre-training instances.
Specifically, we employ AdamW as the optimizer for model training. We adopt T5-Large as the model backbone and the seed datasets as supervisions for training both the question-answer pairs generation, and the filtering QA model. We set learning rate as 1e-5, warmup step as 0, batch size as 2 per GPU, and training epochs as 10.

A.2 Details on Pre-training and Task
Adaptation.
Pre-training. During pre-training, we jointly train the main model parameters with the representations of the special key indicators and the task/format-specific soft prompts. Initially, we don't have any specific tasks during pre-training, so we take the three pre-training corpus (i.e., "MultiChoiceQA, Extractive QA, and Abstractive QA" ) as the three initial tasks, and randomly initialize the task and format specific soft prompts.
Specifically, we use T5-Base as the model backbone, and set learning rate as 1e-4, batch size as 8 per GPU and gradient accumulation steps as 10. We adopt 8 V100 GPUs for pre-training.
Fine-tuning. During fine-tuning, we need to initialize the task/format-specific soft prompts for a specific downstream task. If the task corresponds  to a specific format participating in the pre-training stage, we use the corresponding soft prompts of this format type to initialize the soft prompts for the current tasks to transfer the learned knowledge. If the task corresponds to a new format, we can randomly initialize the task/format prompts. Specifically, we use T5-Base as the model backbone, and set learning rate as 1e-4, batch size as 2 per GPU, gradient accumulation steps as 2, and training epochs as 5. We adopt 8 V100 GPUs for fine-tuning.
Few-shot Learning. We adopt a similar way to initialize the task-specific soft prompts for fewshot learning. We use the standard setting which utilizes 32 randomly selected instances for fewshot learning. Specifically, we adopt T5-Base as the model backbone, and set learning rate as 1e-5, batch size as 1 per GPU, gradient accumulation steps as 1, and training steps as 800 for few-shot learning.
Zero-shot Learning Since zero-shot learning does not involve training stage, we just need to initialize the task-specific prompt for inference. Therefore, we initialize the task-specific prompt with the pre-trained task prompts of its corresponding format type.

B Results on Yes/No Pre-training
During our pilot study, we take the BoolQ (Clark et al., 2019) as the seed dataset to construct a largescale pre-training corpus, and test the full-data, fewshot, zero-shot on top of the pre-trained ProQA. We also take the naturally-perturbed version of this dataset BoolQ-NP (Khashabi et al., 2020a) into account for evaluation. Results are shown in Table 5. We find that the ProQA significantly outperforms T5 baseline on all settings. Note that we take a strict evaluation towards the model's output.  In other words, if the output is not any format of "yes", "no", "true", "false", that prediction will be classified as wrong. Table 6 provides the full results for the continual learning experiment. The model is firstly trained on task A under few-shot setting, and then fine-tuned on task B. Afterwards, we evaluate the trained "Task B Model" and "Task B Model (w/ Task A Prompt)" on task A to test its continual learning capability. Detailed results on every Task A/Task B combination (33 reported in total) are shown in Table 6. Note that we consider two tasks in continual learning because we also want to investigate the task adaptation to-and-fro the same format (e.g., MC → MC) or different formats (e.g., AB → EX). The results shed light on how could we arrange the order of training on tasks to achieve the best overall performance when a bunch of tasks arrive.