{'fraction_non_alphanumeric': 0.034993459166510935, 'fraction_numerical': 0.0135955896094188, 'mean_word_length': 4.984064858820241, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Attention-guided Generative Models for Extractive Question Answering

We propose a novel method for applying Transformer models to extractive question answering (QA) tasks. Recently, pretrained generative sequence-to-sequence (seq2seq) models have achieved great success in question answering. Contributing to the success of these models are internal attention mechanisms such as cross-attention. We propose a simple strategy to obtain an extractive answer span from the generative model by leveraging the decoder cross-attention patterns. Viewing cross-attention as an architectural prior, we apply joint training to further improve QA performance. Empirical results show that on open-domain question answering datasets like NaturalQuestions and TriviaQA, our method approaches state-of-the-art performance on both generative and extractive inference, all while using much fewer parameters. Furthermore, this strategy allows us to perform hallucination-free inference while conferring significant improvements to the model's ability to rerank relevant passages.


Introduction
Recently, it has been shown that pretrained generative seq2seq models can achieve great performance on question answering (QA) tasks Izacard and Grave, 2020b). In the Transformer (Vaswani et al., 2017) architecture, cross-attention plays a crucial role in extracting information from the encoder for use during generation. For the QA task, cross-attention serves as a natural architectural prior, aligning the generated answer tokens to evidence from within the context passages. We posit that this alignment can be exploited as a means for answer span extraction.
Initially, we found that training a transformer model on the task of generative question answering and then leveraging the cross-attention patterns to perform zero-shot answer span extraction works remarkably well, with results approaching the generative baseline. Consequently, we propose several additional joint training strategies that further improve both the generative and extractive QA performance.
Additionally, we point out that generative models are notoriously brittle and prone to hallucination. In particular, out-of-context predictions can cause significant degradations in QA performance. We show a few examples in Fig. 2. Generative models incur an additional risk factor in industry question answering systems, where hallucinations are especially hazardous. To address this issue, we show that our cross-attention extracted spans can also be leveraged as a fallback option to prevent hallucinations as they occur.
Finally, we demonstrate that our models, trained with the joint generative-extractive objective, offers significant improvements over previous baselines in passage ranking.
Our contributions can be summarized as follows: • We propose a simple attention-based inference strategy to extract answer spans from a seq2seq Transformer model without introducing any additional parameters.
• We propose a joint training strategy by combining the normal generative loss and a span extractive loss via enforcing the crossattention to align with answer span positions within the context passages.
• We conduct an empirical study on both closeddomain and open-domain question answering tasks. We show that our methods achieve performance similar to the state-of-the-art using much fewer parameters. Figure 1: Average cross-attention between generated tokens and input context tokens on an example from SQuAD-v1.1 dataset. Each row and column corresponds to a decoder generated token and an input token, respectively. The generated token sequence ['Stan', 'ford', 'University'] attends to the input tokens ['Stanford', 'University'].
• We propose additional ways to use the crossattention probabilities (1) to leverage the extractive predictions as a fallback to help prevent hallucination and (2) as a proxy for passage scores to help provide additional context for the generated answer, improving the interpretability of an otherwise black-box model.

Method
In this section, we first give a brief overview of the cross-attention mechanism in the Transformer models (Vaswani et al., 2017) in Section 2.1. Then we describe how to use cross attention for span extractions in Section 2.2. Finally, we introduce the joint training strategy for combining the generative and extractive losses in Section 2.3.

Cross-attention in transformer models
We briefly describe the cross-attention mechanism used in the seq2seq Transformer model (Vaswani et al., 2017). In the Transformer architecture, the encoder will produce a global contextual representation, X ∈ R n×d , for a sequence of n input tokens, and the decoder will attend on the encoder representations to generate each token of the final predicted sequence. Specifically, let h ∈ R d be the output hidden representation from the self-attention layer in the decoder at a certain position. The crossattention mechanism will perform the following operations. First, it computes the query, key and value vectors through a linear layer with weights Then, it computes the similarity scores between the query and key vectors at different positions. The scores are normalized via softmax across all the input token positions, also known as the attention weights. .
Finally it computes the updated representation by aggregating the value vectors using the attention weights, followed by a linear layer with weights A typical Transformer model usually consists of multiple cross-attention modules in parallel (multihead attention) with normalization layers and skip connections. Please refer to Vaswani et al. (2017) for additional details.

Span extraction via cross-attention pattern
In this subsection, we describe how to use crossattention patterns to perform answer span extraction. Specifically, given a question q and a context passage C = [x 1 , x 2 , . . . , x n ], the model will take [q[SEP]C] as input and autoregressively generate an answerâ = [ŷ 1 , . . . ,ŷ t ]. Intuitively, the crossattention weights illustrate the importance of each token in the source input in each decoding step.
In the case of reading comprehension, the more attended on a context token is, the more relevant that token is to the corresponding generated answer token. Under this assumption, we propose a simple strategy to extract answer spans while decoding. More concretely, we examine the cross-attention weights of the last decoder layer when the model generates the first tokenŷ 1 and the last answer tokenŷ t . We hypothesize that when the model decodes the first answer token, the most attended position is likely the start position of the extractive answer span. Similarly, when model decodes the last token, the most attended position is likely the  This example, which corresponds to the question "how many cracker barrels in the united states", presents an example of hallucinations that can arise through memorization of outdated information. Natural Questions is known to have test-set leakage (Lewis et al., 2020b) and this example occurs in the training set with an answer of '639' (extracted from the 2017 dump of Wikipedia) but the cross-attention is aligned to '645' in the new context, which was retrieved from the 2018 dump. In this example, the first 80 tokens of context are dropped for readability. end position of the answer span. We consider the cross-attention weights when decoding the first and last answer tokens as a reasonable proxy for the probabilities of the start and end positions of the answer span, i.e.
where CrossAttn() is a function that returns the cross-attention probability between two tokens. To account for the multi-head attention in Transformers, we simply take the average cross-attention probabilities over multiple heads. In addition, Transformer models can be easily extended to the class of Fusion-in-Decoder(FiD) models to deal with multiple source inputs (Izacard and Grave, 2020b). Since the cross-attention mechanism remains the same in FiD models, the methodology described above still applies. Visualization of the cross-attention alignments and examples of how they behave in various situations are shown in Fig. 1,2, 3, and 4.

Joint generative and extractive training
To further improve the cross-attention patterns for span extraction, we incorporate a span extraction loss into the training process. In the normal seq2seq model training, the generative loss is minimized through teacher-forcing, i.e.
where a = [y 1 , . . . , y t ] is the ground truth answer. Based on Eq.
(2), we can obtain the typical span extraction loss via the cross-entropy loss of the start and end position predictions, i.e. span (q, C, start, end) = CE(start, CrossAttn(y 1 , C)) where start, end are the start and end positions of the ground truth answer span, respectively, and CE() computes the cross-entropy loss. Then, the final joint training loss function is as follows: where λ ∈ [0, 1] is a hyperparameter.

Experiments and results
In this section, we apply our proposed methods on the reading comprehension and open-domain question answering tasks. Specifically, we initialize our FID models with the pretrained BART-base and BART-large parameters  in all of our experiments.

Reading comprehension
We conduct an empirical evaluation of our models on the reading comprehension dataset SQuAD-v1.1. At inference time, we use a beam size of 5. While autoregressively decoding, we also extract answer spans based on the cross-attention weights, as previously described. The results are shown in Table 1.  In Table 1, we find that BART-base trained only on the generative task has reasonable performance on extractive span prediction. This indicates that the cross-attention patterns have implicitly learned to extract answer spans through vanilla generative training with teacher-forcing. Explicitly enforcing this signal via the joint training objective Eq. (1) obtains addition improvements to the extractive answer predictions, outperforming even the generative predictions. Note that for BART-large, training only on the generative loss produces lower quality extractive predictions. We hypothesize that compared to BART-base, the increased depth (and therefore increased capacity) of BART-large makes the model less reliant on the cross-attention alignments for answer generation.

Open-domain question answering
In this section, we conduct an empirical evaluation on open-domain question answering (ODQA) tasks. A typical ODQA system consists of two steps. First, for each question, we retrieve supporting passages from a large external corpus. Then, a reader model takes the retrieved passages to predict the answer. In this work, we focus on the reader model only. Specifically, we consider the Fusionin-Decoder (FiD) architecture initialized from the BART models.
We perform experiments on the NaturalQuestions (Kwiatkowski et al., 2019) and TriviaQA (Joshi et al., 2017) datasets, using same settings as the original FiD paper (Izacard and Grave, 2020a,b). We also use the same dataset partitions; NaturalQuestions has a train/dev/test split of 79,168 / 8,757 / 3,610 queries and TriviaQA has a train/dev/test split of 78,785 / 8,837 / 11,313 queries. For fair comparison, we directly borrow the retrieved passages from (Izacard and Grave, 2020a) instead of performing retrieval ourselves. We report the standard Exact Match (EM) metric on the final answer predictions.
Training We apply distant supervision to train our reader models. Since there are no human annotated start and end positions in the training set, the ground truth answer can potentially be found in multiple retrieved passages. To account for this, we consider three strategies. (1) Multi-label spans: Consider all the occurrences and use multi-label cross-entropy loss. (2) First span: Take the first occurrence as the the start and end positions, assuming the top passages are more relevant. (3) Most-likely span: Consider the span occurrence with the highest likelihood based on the attention scores. We find that on the datasets provided by (Izacard and Grave, 2020a), the First span strategy works the best. We make detailed comparisons in Section 3.2.2.
Inference For generative answer prediction, we apply beam search decoding and choose the best beam. For extractive span prediction, we consider a simple local greedy approach. Given the start and end position probabilities P start , P end (based on cross-attention weights of the first and last decoded tokens), we consider two span prediction    candidates, ŝ tart 1 = arg max P start , end 1 = arg maxŝ tart 1 <i<ŝtart 1 +lmax P end ê nd 2 = arg max P end , start 2 = arg maxê nd 2 −lmax<i<ê nd 2 P start where l max is the maximum answer span length. The final span prediction is the span with the larger probability among these two candidates.
Technical details We utilize the retrieved results 1 from Izacard and Grave (2020a), which contain 100 retrieved passages for each question. We finetune the FiD models initialized with the pretrained BART models on each dataset independently, using Adam (Kingma and Ba, 2014) with a learning rate of 5e-5 and warmup rate of 0.1. We train the models for 10 epochs with batch size 64 and validate at each epoch. The hyperparameter λ is selected through cross validation.

Main results
We compare our models with state-of-the-art ODQA systems. The main results are shown in Table 2. First, we notice that similar to the RC task, the cross-attention pattern produces impressive zero-shot results for extractive span prediction in the open-domain and that joint training can further improve the performance. On the NQ test set, [CLS] what  our best model (BART-large with joint training) achieves 53.43 EM using the generative predictions and 50.03 EM using the extractive predictions; the generative prediction performance approaches the SOTA performance from FiD-KD, which uses a much larger model (T5-large), and the extractive prediction performance significantly outperforms the previous state-of-the-art extractive model (Col-BERT). On TriviaQA, our best model underperforms FiD-KD (T5-large) in terms of generative prediction but achieves the state-of-the-art extractive performance.

Ablation on training strategies
Here, we finetune the FiD (BART-base) model on NaturalQuestions with the three strategies described earlier. The results are shown in Fig. 5. As we can see, the Multi-label Span strategy clearly underperforms the two other approaches in both generative and extractive predictions. Increasing λ degrades the performance further, below baseline. This result suggests that using every possible answer span as a training signal is extremely noisy (for example, some answer spans could be multi-sense words or be surrounded by the wrong context). First Span slightly outperforms the Most-likely Span approach, suggesting that the ranking of the passages retrieved in Izacard and Grave (2020a) are relatively high quality. Nevertheless, we believe that the Most-likely Span is a competitive and robust approach especially when well-ranked passages are unavailable.

Reducing hallucination
To further demonstrate the advantages of our approach, we consider the issue of hallucination in generative models. For example, on the Natu-ralQuestions test set, 6.2% of generated answer predictions from a FiD (BART-large) model are not spans from within the 100 context passages. Similarly, on the TriviaQA test set, the rate of hal-  To address this, we propose and compare various strategies for removing hallucinations. Our first method uses the attention-based extractive inference as a hallucination-free baseline (ATTENTION). Secondly, we can simply drop out the out-ofcontext answers from the generative predictions (DROP).
Finally, we can perform a simple backoff which replaces hallucinations from the generative predictions with the extractive prediction (BACKOFF). To remove variations due to retrieval performance, we conduct the evaluation only on the queries where at least one of the ground truth answers exist in the retrieved passages. Under this constraint, we retain 3,236 test queries from NaturalQuestions and 9,966 test queries from TriviaQA. The results are shown in Table 3. We can see that (DROP) leads to a degradation of performance, especially for Trivi-aQA datasets (approx. 3 points EM). And using the simple BACKOFF strategy can effectively close this gap. For the models with joint training, BACKOFF achieves even better performance than the original generative predictions, indicating that BACKOFF is an effective hallucination-free inference strategy for generative QA models.

Attention score as a reranker
In this section, we demonstrate that the crossattention probabilities are a powerful proxy for the scores used in passage ranking. Particularly, in additional to answer prediction, the FiD reader model can also perform reranking over retrieved passages. Specifically, given a query q, one context passage p i among C, we define the passage score as follows: where P start and P end are the predicted start/end position probabilities of the context passages.  We conduct the passage ranking experiment based on the same passages provided by Izacard and Grave (2020a). We assume that any passage that contains a ground truth answer as the relevant passages and the rest are considered irrelevant. We rerank the top 100 passages based on Eq. (3) using our best FiD models. The results are shown in Table 4. We can see that the attention-based score defined by Eq. (3) from our FiD models produce rankings that significantly improve upon the results from the neural retrieval model in (Izacard and Grave, 2020a), which originally demonstrated the effectiveness of FiD as a passage reranker. Showing the reranked passages in additional to the answer prediction can provide additional context for the end-users or developers of ODQA systems to navigate. In practice, this can improve the interpretability and reliability of the typically black-box ODQA systems that we train and use today.

Related work
Extractive question answering Extractive question answering problems have gained wide interest under both the closed-domain and open-domain settings. In the closed-domain setting, many methods have been proposed that continuously push the envelope on datasets such as SQuAD (Rajpurkar et al., 2016), one of the most popular large-scale reading comprehension datasets. Many of the recent methods leverage pretrained language models with a simple token classification head to extract answer spans Lan et al., 2019;.
In the open-domain setting, Chen et al. (2017) introduced the retrieve-and-read framework to answer questions based on the unstructured Wikipedia corpus. Various works have been proposed to improve either or both retrieval and reader models (Wang et al., 2019;Karpukhin et al., 2020;Khattab et al., 2020;Lewis et al., 2020a;Min et al., 2019;Grave, 2021, 2020a) under this framework.
Generative models Typically, generative models are used to tackle abstractive questions answering tasks, such as NarrativeQA (Kočiský et al., 2018) and ELI5 (Fan et al., 2019), where the answer usually does not appear as a span in the context. Recently,  showed that large pretrained generative models can achieve competitive performance on SQuAD, an extractive question answering task.  proposed to use pretrained generative models to perform ODQA in a closed-book setting.
More recently Izacard and Grave (2020b) proposed the FiD models which significantly improved the end-to-end performance of ODQA. The issue of hallucination in generative models has recently gained attention in a variety of tasks including document summarization (Maynez et al., 2020), ma-chine translation , news generation (Zellers et al., 2019), and dialogue systems (Mielke et al., 2020;Shuster et al., 2021).
Our work is also inspired by the pointer network (Vinyals et al., 2015;See et al., 2017), where the cross-attention weights in a seq2seq model is used to replace or modify the distribution during decoding. In our case, we directly use the attention weights for extractive inference and provide additional useful context for generative predictions. Another closely related work is Izacard and Grave (2020a) which leverages the cross-attentions weights from FiD models to obtain weak supervision signals for training passage retrieval models, which aligns with our observation that attention scores can be used as a good proxy for passage scores (Section 3.2.4).

Conclusion
Our work introduces a novel approach for using the cross-attention patterns of a generative QA model to obtain extractive answer spans. We propose methods to jointly train our model to perform generative and extractive inference, improving the performance of both. Furthermore, our method allows us to achieve hallucination-free inference while also improving the model's passage ranking capabilities.
Our results demonstrate the effectiveness of leveraging cross-attention as a architectural prior for improving modern, state-of-the-art generative question answering systems. Our findings raise a number of questions to be explored next. In particular, while our empirical results show the effectiveness of strategies like BACKOFF for hallucinationfree generation, we foresee the potential extension of using the cross-attention aligned tokens during the decoding process.