{'fraction_non_alphanumeric': 0.030122456276256593, 'fraction_numerical': 0.008193546464857135, 'mean_word_length': 4.93527851458886, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
End-to-End Dense Video Grounding via Parallel Regression

Video grounding aims to localize the corresponding video moment in an untrimmed video given a language query. Existing methods often address this task in an indirect way, by casting it as a proposal-and-match or fusion-and-detection problem. Solving these surrogate problems often requires sophisticated label assignment during training and hand-crafted removal of near-duplicate results. Meanwhile, existing works typically focus on sparse video grounding with a single sentence as input, which could result in ambiguous localization due to its unclear description. In this paper, we tackle a new problem of dense video grounding, by simultaneously localizing multiple moments with a paragraph as input. From a perspective on video grounding as language conditioned regression, we present an end-to-end parallel decoding paradigm by re-purposing a Transformer-alike architecture (PRVG). The key design in our PRVG is to use languages as queries, and directly regress the moment boundaries based on language-modulated visual representations. Thanks to its simplicity in design, our PRVG framework can be applied in different testing schemes (sparse or dense grounding) and allows for efficient inference without any post-processing technique. In addition, we devise a robust proposal-level attention loss to guide the training of PRVG, which is invariant to moment duration and contributes to model convergence. We perform experiments on two video grounding benchmarks of ActivityNet Captions and TACoS, demonstrating that our PRVG can significantly outperform previous methods. We also perform in-depth studies to investigate the effectiveness of parallel regression paradigm on video grounding.


Introduction
Video Grounding (VG) [9,1] is to localize the temporal moment with a given language query in an untrimmed video. Different from traditional computer vision He goes to the cupboards and takes out a plate and a pineapple.
He washes the cutting board in the sink. Time He cuts and peels the pineapple and places the pieces on the plate.
He takes the pineapple pieces and chops them into small cubes. Fig. 1. An illustrative example of the dense video grounding task. Dense video grounding aims to jointly localize multiple temporally ordered moments described by a paragraph in an untrimmed video.
tasks in video understanding such as action recognition [21,24] and temporal action localization [15,20], which are limited to pre-defined actions, VG allows for an open set of activities with language as queries.
Existing methods mainly address this problem in an indirect "one-to-many" manner, i.e., generating many proposals for one language query, by pre-defining dense proposals or predicting dense proposals at all locations, which suffer from complicated label assignment and near-duplicate removal. Specifically, they can be grouped into two categories, namely proposal-based approaches [1,9,5,26,28,31,34,23,33,16] and proposal-free approaches [29,19,30,6,32,35,14]. As shown in Fig. 2(a), proposal-based methods present a propose-and-match paradigm for VG task, which first manually generate a set of moment candidates using sliding window strategy or 2D temporal map [33], and then calculate the similarity between each candidate with the language query. These proposals are usually of fixed length and lack flexibility to deal with various duration. It often requires large numbers of proposals to cover the ground-truth segments. In contrast, proposal-free approaches present a fusion-and-detection paradigm for VG task. As shown in Fig. 2(b), these methods usually first fuse two modalities of video and language and then use a dense detection framework to localize visual segment. They typically predict the offset between the current frame and boundary or the probabilities of each temporal position as the boundary, then generate dense proposals with the same or squared number of video frames. Although they can generate proposals with arbitrary length, these detection methods typically require complicated label assignment during training and removal of near-duplicate predictions.
Moreover, semantic information in a single sentence is limited, which might result in ambiguous temporal moments solely based on this description. Considering the example in Fig. 1, if we want to localize the temporal moment of "He washes the cutting board in the sink", this description might be unclear and lead to wrong localization, because the person may do "washing" several times  after cutting different ingredients. However, if we know the person cuts and peels the pineapple before it, and chops the pineapple pieces into small cubes after it, the localization will be more accurate. Therefore, more context from other sentences are required to overcome this semantic ambiguity. However, most existing methods focus on VG task with a single sentence, which fail to capture semantic relevance and temporal clues among the multiple sentences to be grounded in a video. Recently, dense VG [3] is introduced as a new task, and its goal is to jointly localize multiple temporal moments described by a paragraph in an untrimmed video. It proposes a DepNet to address this problem by selectively propagating the aggregated information to each single sentence. Yet, this process of aggregating temporal information of multiple sentences into a compact set will lose fine-grained context information for a specific language query.
In this paper, we tackle the problem of dense VG and present a simple yet effective framework for this task. Taking a different perspective on VG as a direct regression problem, we present an end-to-end parallel decoding paradigm by re-purposing a Transformer-alike architecture (PRVG). The key design in our PRVG framework is languages as queries, and using the paragraph to directly attend the contextualized visual representations for moments regression. This simple design of languages as queries in our parallel decoding paradigm shares two critical advantages. First, the cross-attention between languages and visual representations endows PRVG with the global context view, which provides the flexibility of accurate localization for temporal moments of various duration at different locations. Second, the self-attention in sentences enables PRVG to model the mutual context among different sentences, which could be leveraged to overcome the semantic ambiguity of each individual sentence. In addition, we also design a robust proposal-level attention loss to explicitly guide the attention mechanism training in PRVG, resulting in a more accurate regression model. Due to its simplicity in design, our PRVG can be deployed efficiently without any complicated label assignment and post-processing technique.
We perform extensive experiments on two benchmarks of ActivityNet Captions [13] and TACoS [18], demonstrating that our PRVG can significantly outperform previous methods. We also perform in-depth ablation studies to investigate the effectiveness of parallel regression scheme in our PRVG and try to present more insights on the task of dense VG. Our main contributions are threefold: -We cast VG as a direct regression problem and present a simple yet effective framework (PRVG) for dense VG. PRVG directly regresses the temporal boundary for each language query to the final result, leading to accurate and efficient inference. -We design a robust and scale-invariant proposal-level attention loss function to guide the training of PRVG for better performance.
-Extensive experiments demonstrate the superiority of PRVG to previous state-of-the-art methods and the effectiveness of parallel decoding paradigm on dense video grounding task.

Related Work
Video Grounding. Video grounding was introduced by [9, 1], which aims to localize a temporal moment given by a sentence description. Existing methods can be grouped into two categories, including proposal-based approaches and proposal-free approaches. Proposal-based approaches cast this task as a crossmodal matching problem and solve it in a "propose-and-match" manner. [9,1] adopted sliding window strategy to generate proposals and produced matching scores between the language query and each proposal. The proposal with the highest matching score was selected as the final prediction result. To generate more precise temporal moments, [5,26,28,34] regressed their boundaries on predefined multi-scale anchors. [31] produced proposals of different lengths by temporal pooling and 1D convolution. [33] proposed a Temporal Adjacent Network to model the temporal relations between video moments by a two-dimensional map. [16] proposed to score all pairs of start and end indices simultaneously by a biaffine mechanism.
To produce more flexible results with arbitrary length, various proposal-free approaches were proposed. [19,32,6,35] predicted the probabilities of each frame as start and end boundaries, and then got a two-dimensional score map whose element indicates the predicted probability of each moment candidate. To eliminate the problem of imbalance between positive and negative examples, [30] used the distances between the frame within the ground truth and the starting (ending) frame as dense supervisions. [29] first utilized two bidirectional LSTM networks to encode video and language and then employed co-attention to regress the start and end time points directly. [29] and our proposed PRVG both predict in a one-to-one manner, but PRVG uses self-attention to model video and sentence queries respectively, which can capture long-term temporal interactions more effectively. Instead of complicated co-attention, PRVG simply uses cross attention with languages as queries to model inter-modal interactions and achieves better performance. Dense Video Grounding. [3] introduced the task of dense video grounding, and its goal is to jointly localize all temporal moments corresponding to these sentence descriptions, given an untrimmed video and a paragraph of sentence descriptions. To extract temporal and semantic information of sentences descriptions, [3] proposed a Dense Events Propagation Network (DepNet) in an aggregation-and-propagation scheme. Comparing to our intra-modal and intermodal context modeling in our PRVG, DepNet missed fine-grained context information for a specific language query. Vision Transformer. Transformer was first presented by [22] in machine translation task. Due to its powerful global temporal modeling capability, transformer is applied in many other NLP tasks, such as named entity recognition [27] and language understanding [7]. Inspired by the recent advances in NLP tasks, transformer is applied in computer vision, such as image classification [8], object detection [4,36] and action proposal generation [20]. Our proposed PRVG model is built on transformer to capture temporal context effectively and predict temporal boundaries directly. In contrast to DETR, we use the languages as queries while they typically use hidden variables as learnable queries, which are fixed once the training is finished.

Method
We formulate the dense video grounding task as follows. Given an untrimmed video V and a paragraph of K temporally ordered sentences {S 1 , · · · , S K }. The goal of dense VG is to jointly localize multiple temporal moments {T 1 , · · · , T K } for all these sentences, where T k = (t s k , t e k ). t s k and t e k denote the start and end time points of the temporal moment corresponding to the k-th sentence, respectively. Specifically, the video is represented by a sequence of frames, i.e.
, where s ki is the i-th word in the sentence and L S k is the total number of words.
As shown in Fig. 3, the pipeline of our proposed PRVG consists two steps: feature encoding and parallel regression. First, we extract features of video and sentence descriptions in the feature encoding phrase. In the parallel regression phrase, we use Contextual Representation Encoder (CRE) to model global context and get contextual representations for both modals, then a Language Modulated Decoder (LMD) is adapted to aggregate the visual information from the entire video by using the language representations as queries. Finally, a Parallel Regression Head (PRH) directly predicts the temporal boundaries for all sentence descriptions in the paragraph simultaneously without extra post-processing such as Ranking or NMS. Our PRVG streamlines the process of dense video grounding with a direct and parallel decoding paradigm, which is composed of two steps: feature encoding and parallel regression.

Feature Encoding
For fair comparison with previous methods [33,3], we use the same video and sentence feature encoding. Specifically, for each word s ki in the input sentence S k , we first generate its word embedding vector w ki ∈ R d by the Glove word2vec [17], then feed the word embedding sequence into a three-layer LSTM network [11]. We use the last hidden state f S k ∈ R D as the feature representation of the input sentence S k , and get the sentence features F S ∈ R K×D . For an input sequence of video frames, we divide them into N v small clips without overlap, where each clip contains L frames. For each clip v i , we use a pre-trained C3D Network [21] to extract its feature. Then we feed them into a feed-forward neural network to obtain projected features f vi ∈ R D , which has the same dimension with sentence feature. For fast processing, we uniformly sample N clips [24] from each video and get video features F V ∈ R N ×D .

Parallel Regression
Our parallel regression contains a Contextualized Representation Encoder (CRE), a Language Modulated Decoder (LMD) and a customized Parallel Regression Head (PRH). Contextualized Representation Encoder. As shown in Fig. 4 (a), CRE consists a multi-head self-attention layer and feed forward network with a residual connection [10]. As attention mechanism is permutation-invariant, we add the positional embedding to the input features. The calculation can be formulated as: where LN(·) denotes layer normalization [2], MHA(·) means multihead attention and FFN(·) is the feed forward network. For sentences, the input is F S ∈ R K×D , and the output is F CRE L . For video, the input is F V ∈ R N ×D , and the output is F CRE V . This contextualized representation encoder enables our video features  Decoder (LMD). CRE is built on self-attention mechanism, whose input is the video or sentence features. CRE models long-term dependencies in video and extracts semantic relevance among the sentences in a paragraph, facilitating to contextualized representations for both modals. LMD is built on cross-attention mechanism, using language as queries and video features as keys and values. LMD aggregates visual information under the guidance of language queries for subsequent parallel regression.
to have a global view by using self-attention operations over the entire video sequence and the resulted contextualized visual representations. Besides, CRE extracts the temporal clues and semantic relevance among the sentences in the paragraph and facilitates to the contextualized language representations. It's worth noting that there is a large semantic gap between video and language representation, thus two CREs that do not share parameters are adopted to keep as much useful information as possible for each of the two modals. Language Modulated Decoder. As shown in Fig. 4 (b), LMD adopts a similar architecture to CRE, which is composed of a multi-head cross-attention layer and feed forward network with a residual connection. More specifically, we also add positional encoding to the sentence features. The input of the decoder is K sentence features f S = {f Si } K i=1 and the encoder's output E enc . The calculation can be formulated as: The output of the decoder is language-modulated visual representations X LM D ∈ R K×D .
The key design in our proposed LMD is using the language representations as queries, instead of N object queries represented by learnable positional embeddings in the transformer decoder of DETR. Compared to using hidden variables as learnable queries like DETR, using languages as queries endows our PRVG with more explainable and flexible modeling capacity to deal with the task of dense VG in an open-set setting. Detailed discussions between PRVG and DETR in the later subsection. Parallel Regression Head. Given the LMD's output F LM D , the parallel regression head consisting of a 3-layer FFN with ReLU activation function directly predicts the temporal boundary (t s , t e ) for each sentence in parallel. It is worth noting that PRVG generates only one prediction result for each language query, which is more efficient than one-to-many prediction methods. Discussion with DETR. DETR [4] views object detection as a direct set prediction problem and uses N object queries represented by learnable positional embeddings to decode detected objects, while PRVG uses languages as queries. The problem of video grounding could be transformed into a detection task and it is natural to use DETR for VG task. Based on the fused representation of video and sentence, we can propose DETR architecture to solve this grounding problem. But we argue that this straightforward application of DETR to VG has some flaws compared with our PRVG. Experimental comparisons of PRVG and DETR-based models are also provided in Section 4.4.
Flexibility with variable queries. First, the number of the queries in the decoder of PRVG is variable, but that of DETR is fixed. For dense VG task, there can be different number of sentences in different input paragraphs, which makes it hard for DETR-based models with a fixed number of queries to expand to the dense VG task. Although we can adapt DETR-based models to dense VG with post-processing methods such as beam search, the temporal relationship and semantic relevance among the temporally ordered sentences in the paragraph cannot be fully explored. However, our proposed PRVG can deal with any number of sentences by the same model and fully model the temporal interactions among the sentences by self-attention in the decoder. Second, our language queries are adjusted adaptively with different input sentences and can be generalized to unseen sentence during testing, while theirs are fixed once the training is finished and same for different inputs. This fixed query scheme lacks the flexibility to deal with an open set of activities and might lead to performance drop in case of semantics shift.
Interpretability of language queries. Each query in our decoder represents the concrete semantics of a sentence while the learned moment queries in DETRbased models are optimized in a data-driven manner without clear meaning. The clear semantics of queries makes our model easy to understand and generalize. Besides, each query in their decoder can predict both foreground and background, depending on the input. As the semantics of the queries of DETRbased models is not clear and the matching between the queries and ground truth moment is uncertain, we cannot explicitly supervise the attention weight training with ground truth region. In contrast, our proposed PRVG uses languages as queries and each query directly regresses the corresponding temporal boundary. So we can use an attention loss to guide the training of PRVG for better performance.
Retrieval framework without negative samples. DETR was introduced in object detection, which is essentially different from our video grounding in terms of task complexity. VG and dense VG both aim to localize the temporal moment for a sentence description from the whole video, and can be regarded as a retrieval task. Compared with traditional detection, the search space is bigger and the imbalance of positive and negative examples is more serious if we transform retrieval task into detection. Therefore, it is hard to train a DETRbased VG model which depends on a classification branch to distinguish between foreground and background. Thanks to our design of languages as queries and parallel regression regime, PRVG directly predicts the final result for each sentence description in an one-to-one matching between queries and ground truth without classification, and there is no need for negative samples in the training process, thus eliminating the extreme foreground-background class imbalance.

Training
To optimize the proposed model, we adopt a multi-task loss L including temporal boundary regression loss L reg and a robust proposal-level attention loss L attn : Temporal boundary regression loss. A training example consists of an input video V , a paragraph of language queries {S 1 , S 2 , · · · , S K }, and normalized temporal annotations {T 1 , T 2 , · · · , T K }. The predicted results are denoted as {T 1 ,T 2 , · · · ,T K }. We follow [4] and adapt its 2D bounding box loss to temporal 1D form. Then we get temporal boundary regression loss as follows: where λ and β are trade-off hyper-parameters. Proposal-level attention loss. To improve the accuracy of the learned sentenceto-video attention, [29] design a position-wise attention loss to encourage the video clips within the ground truth window to have higher attention values: where a j i is the attention weight between the i-th clip and j-th sentence, and a j i = 1 if the i-th clip is in the ground truth window, elseâ j i = 0. We argue that this attention loss is not robust since it considers all temporal positions in the ground truth independently rather than as a whole. First, different positions in the ground truth might not equally contribute to the final regression and we should not equally constrain the attention at all locations to be very high. Besides, Equation (5) will not be zero and gradient will continue to backpropagate, even if the model can attend to the described moment accurately due to N i=1 a j i = 1, which is easy to lead to over-fitting of the model and results in inferior results. Therefore, we devise a robust proposal-level attention loss, by relaxing the position-wise constrain. We only require the sum of attention weights in ground truth should be high. The loss function is defined as: Compared with position-wise attention loss, our proposed proposal-level attention loss regards the proposal as a whole. It not only makes the model attend to the ground truth region, but also allows the model itself to further determine the high-salient temporal locations and suppress attention values at locations less important in the ground truth region for VG. Besides, the new attention loss is scale-invariant to the moment length, which is robust for regression and helps to converge.

Datasets and Evaluation Metrics
ActivityNet Captions. [13] consists of 19,209 untrimmed videos of opendomain activities, which is originally built for dense-captioning events task and recently introduced for video grounding. For a fair comparison, we follow the experimental setting in [34], using val 1 as validation set and val 2 as testing set, which have 37,417, 17,505, and 17,031 moment-sentence pairs for training, validation, and testing, respectively. TACoS [18] consists of 127 videos of cooking activities. For a fair comparison, we follow the experimental setting in [9], using 10,146, 4,589, 4,083 momentsentence pairs for training, validation and testing respectively.
Following [9], we adopt "R@n, IoU=m" to evaluate the model's performance. It is defined as the percentage of language queries having at least one correct moment whose IoU with ground truth is larger than θ in the top-n retrieved moments. As PRVG regresses one moment for each query, we set n = 1, θ ∈ {0.3, 0.5, 0.7} for ActivityNet Captions and θ ∈ {0.1, 0.3, 0.5} for TACoS. And we also use the "mIoU" metric which calculates the average IoU with ground truth over all testing samples to compare the overall performance.

Implementation Details
We use Glove word2vec [17] to generate word embedding vector, and feed the sequence of word embedding into a three-layer LSTM network [11] to get the feature representations of sentences. For a fair comparison, we use the same visual features extracted by pre-trained C3D [21] on ActivityNet Captions and TACoS. The channel numbers of sentence feature and video feature are all set to 512. The number of sampled clips N is set to 256, 512 for ActivityNet Captions and TACoS, respectively. We use Adam [12] for optimization. The learning rate is set to 0.0001 for ActivityNet Captions and TACoS. The batch size is set to 32 for ActivityNet Captions and 16 for TACoS respectively. The trade-off parameters λ and β are set to 2 and 2 for both ActivityNet Caption and TACoS. We adopt the same split for both datasets and sampling strategy as [3] for training and inference. Specifically, we randomly sample K(2 ≤ K ≤ 8) sentence queries with temporal order for each training sample. During inference, we first split annotated paragraph queries with more than 8 sentences into multiple sub-paragraphs with length not exceeding 8, and then take the whole paragraph as input.
The results are reported in Table 1. From the results, we can find that PRVG outperforms the state-of-the-art methods with a large margin on Acitivi-tyNet Captions, e.g., more than 5 points in term of Rank1@0.3 and Rank1@0.5, and achieves better performance than other methods on TACoS except for Rank1@0.5, which demonstrate the effectiveness of our proposed PRVG model.
First, PRVG achieves superior results to proposal-based methods. We analyze that proposal-based approaches have to manually generate proposals first, which are usually with fixed length and heavily rely on prior knowledge, leading to the incapability to produce more accurate results. In contrast, our PRVG directly  predicts temporal boundaries of moments of arbitrary duration and thus can get more accurate localization results.
In addition, we compare PRVG with proposal-free methods. Experimental results show that PRVG works better than most proposal-free methods. We analyze that their inferior performance might come from the dense prediction in nature. They typically predict much more than one temporal moment for one language query, and thus need to carefully design sophisticated strategies to distinguish them. Different from these approaches, our PRVG directly predicts temporal boundaries in an one-to-one manner by using languages as queries.
Finally, we compare PRVG with other dense video grounding methods. PRVG outperforms the Beam Search method. Except for Rank1@0.5 on TACoS, PRVG achieves much better performance than 3D-TPN and DepNet on all evaluation metrics for TACoS and ActivityNet Captions. This is because 3D-TPN is incapable of modeling the interactions among sentences in a paragraph effectively, and DepNet loses fine-grained context information for a specific language query when aggregating temporal information of multiple sentence queries into a compact set. However, our PRVG effectively extracts context information without information compression by modeling pair-wise interactions with self-attention.

Ablation Studies
Study on the dense video grounding setting. To verify the effectiveness of dense video grounding setting, i.e., inputting multiple temporally ordered sentences in a paragraph as queries, we train and test PRVG in different settings: 1) one sentence or multiple sentences as input. 2) the input sentences are temporally ordered or not. The results in Table 2 (Row 1-3) demonstrates that dense video grounding setting is helpful for more accurate video grounding, since the semantic relevance and temporal cues of multiple temporally ordered sentences in a paragraph can provide more temporal context to avoid semantics ambiguity.
Meanwhile, the result in Table 2 (Row 3) shows that PRVG designed for dense video grounding task can achieve comparable performance to the stateof-the-art video grounding methods on ActivityNet Captions dataset in sparse We also test PRVG trained by multiple temporally ordered language queries with multiple temporally independent (disordered) language queries as the input of decoder. As shown in Table 2 (Row 5), the performance degradation is evident, demonstrating that PRVG can capture the temporal interactions among these temporally ordered sentences during training phase. Study on the proposal-level attention loss. Compared with ActivityNet Captions, TACoS has a longer average duration of videos and a shorter average length of temporal moments described by language queries, which is more challenging. Therefore, we conduct ablation study on our proposed proposal-level attention loss on TACoS. We re-train our model with the following settings: 1) PRVG (bbox): trained with only temporal boundary regression loss. 2) PRVG (bbox + pw): trained with temporal boundary regression loss and position-wise attention loss [29]. 3) PRVG (bbox + pl): trained with temporal boundary regression loss and our proposed proposal-level attention loss. Table 3 shows the performance comparisons. Adding these two attention loss to bbox loss both improve the performance of PRVG, but our proposed proposal-level attention loss achieves better results, which demonstrates the effectiveness of the proposallevel attention loss. More in-depth analysis on proposal-level attention loss is described in 4.5.
Comparison with DETR-based models. To compare with DETR-based models and verify the effectiveness of our design of using languages as queries, we devise two baseline models with DETR [4], marked as DETR-VG and Language DETR-VG respectively. Both baseline models adopt the same architecture as the original DETR, and have 10 learnable moment queries similar to the object queries in the original DETR. Specifically, for DETR-VG, we first perform cross-modal fusion between video and sentence features by element-wise multiplication, and then input the fused feature into the Transformer. For Language DETR-VG, the input of the encoder is only the video feature, and it uses language queries as the initial input of the decoder. To tackle the dense VG task, we use beam search as post-processing method like [3]. In particular, we first localize each sentence in the paragraph independently, then apply beam search on the top 3 grounding results of each sentence to make the final dense VG results match the temporal order. Table 4 shows that the performance of Language DETR-VG is better than DETR-VG by using language queries as initial input of the decoder. Using lan- Our proposal-level attention loss Position-wise attention loss [27]  guages as queries further, PRVG outperforms both baselines with a large margin. On the one hand, the design of languages as queries makes each query in our decoder represent the concrete semantics of a sentence and we can supervise decoder's attention distribution to guide the model to attend the ground truth moment. On the other hand, since each language query in the decoder directly predicts the moment corresponding for its sentence description without classification, there is no negative examples, thus eliminating the imbalance of positive and negative examples and preforming better. Besides, when extending from VG to dense VG, the performance of DETR-based models for dense VG is almost same as that for VG, which shows that DETR-based models with beam search as post-processing cannot capture the temporal and semantic relationship among the sentences in the paragraph. However, PRVG for dense VG fully exploits their temporal interaction and achieves much better performance than PRVG for VG.
In addition, we compare the test time of three models on ActivityNet Captions for dense VG task. As shown on Table 4, DETR-based models take more than twice the time of PRVG. This is due to the complicated and time-consuming post-processing procedure in DETR-based models. Via parallel regression, PRVG directly regresses the moment for each sentence without any post-processing technique, leading to more efficient inference.

Qualitative Analysis
We visualize an example to illustrate whether PRVG can attend on the ground truth and compare our proposal-level attention loss (pl) with position-wise attention loss (pw). As shown in Fig. 5, PRVG trained with pl assigns more attention weights to the ground truth than background, and focuses on the corresponding regions for all queries accurately. However, for PRVG trained with pw, decoder's attention is much more smooth and there is little difference between foreground and background. Moreover, our attention in the ground truth presents a multipeak shape, indicating that proposal-level attention loss can not only help PRVG attend to the region corresponding to the language description, but also de-termine the importance of different positions within this region, thus further enhance positions that are useful for boundary regression.

Conclusion
In this paper, we have proposed a simple yet effective end-to-end framework (PRVG) for dense video grounding, which directly regresses the temporal boundaries of moments by using a paragraph as input. The key in our design is to use languages as queries and directly regress one moment for each sentence without any post-processing technique. We also propose a robust and scale-invariant proposal-level attention loss to guide the training of PRVG. Extensive experiments on TACoS and ActivityNet Captions demonstrate the superior performance of PRVG and the effectiveness of parallel decoding paradigm.