{'fraction_non_alphanumeric': 0.03150714263864672, 'fraction_numerical': 0.009452142791594016, 'mean_word_length': 4.9996329601761795, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Dialogue Inspectional Summarization with Factual Inconsistency Awareness

Dialogue summarization has been extensively studied and applied, where the prior works mainly focused on exploring superior model structures to align the input dialogue and the output summary. However, for professional dialogues (e.g., legal debate and medical diagnosis), semantic/statistical alignment can hardly fill the logical/factual gap between input dialogue discourse and summary output with external knowledge. In this paper, we mainly investigate the factual inconsistency problem for Dialogue Inspectional Summarization (DIS) under non-pretraining and pretraining settings. An innovative end-to-end dialogue summary generation framework is proposed with two auxiliary tasks: Expectant Factual Aspect Regularization (EFAR) and Missing Factual Entity Discrimination (MFED). Comprehensive experiments demonstrate that the proposed model can generate a more readable summary with accurate coverage of factual aspects as well as informing the user with potential missing facts detected from the input dialogue for further human intervention.


I. INTRODUCTION
Summarization-based algorithms have enabled a broad spectrum of applications, such as auto-abbreviated news and retrieval outcomes [1], [2] to assist users in consuming lengthy document effectively. Generally, summarization methods can be divided into extractive summarization and abstractive summarization. The former selects sentences from the original document to compose the summaries [3], [4], while the latter generates the summary texts one word by word based on the understanding of the documents [5]- [7]. In recent years, with the renaissance of neural networks, deep learning based methods have achieved the state-of-the-art results on standard benchmarks for both abstractive and extractive summarization [8]- [11]. challenges for dialogue summarization. The first one is that multi-role dialogue is more complicated due to the interactions among the different parties. Enhanced representation of the atomic components (e.g., utterance and role) of the dialogue prequalifies summary generation optimization.
Second, despite the nature of multi-role dialogues that are noisy and tedious for many circumstances (e.g., a 1-2 hour judicial trial can host 10-30 thousand words), sometimes, there exists a logical/factual gap between the input dialogue and the expected output summary. For instance, in the court debate scenario [15], the judge summarizes the case narrative not only based on facts recognized from the court debate during the trial but also relying on the evidence or materials submitted by the litigants. Similarly, in the medical inquiry process between the doctor and patient, the doctor concludes the medical record based on their conversation on symptoms along with a series of medial reports (e.g., blood test, X-Ray and CT results). In such context, the current summarization framework [7], [16] by enforcing alignment between the input (with dialogue discourse) and the output (with extra knowledge from external resource) can result in the mismatch and misalignment in terms of the critical factual aspects and logics in the generated summary, i.e., semantic/statistical alignment can hardly fill the logical/factual gaps in dialogue summarization.
Motivated by such observation, in this paper, we particularly investigate the factual inconsistency problem and propose a novel framework: Dialogue Inspectional Summarization (DIS) under both non-pretraining and pretraining settings. Specifically, in the non-pretraining setting, we design a hierarchical dialogue encoder involving role information to accommodate long context and multiple turns among the multiple roles. Rather than directly aligning the input dialogue and its summary within the generation framework, we additionally propose two auxiliary tasks in the way of joint learning: Expectant Factual Aspect Regularization (EFAR) can estimate the factual aspects to be contained in the summary so as to make the model emphasize on the factual coverage of logic reasoning; and Missing Factual Entity Discrimination (MFED) predicts the missing aspects which discover/alarm the factual gap between the input and the output. DIS allows the users (e.g., judge and doctor) to further improve the summary by referring to other materials (e.g., evidence) according to the detected missing aspect-based entities. In the pretraining setting, we further investigate the factual inconsistency problem of pre-trained summarization models by equipping DIS with a pre-trained model PEGASUS [17]. Figure 1, for instance, shows a judicial trial transcript and the summary generated by our model. The predicted aspects and the missing aspects are produced by the two auxiliary tasks respectively. Such complementary results can help the professional users better understand the generated content in terms of the factual aspects coverage and the possible omissions that are not mentioned in the input dialogue but essential for writing a comprehensive summary.
To sum up, our contributions are as follows: • To the best of our knowledge, this work is the first attempt for dialogue inspectional summarization (DIS) by addressing the factual inconsistency between the input dialogue and the expected output summary. We particularly conduct experiments and validate our hypothesis on the judicial trial scenario. • We present an end-to-end DIS framework in both nonpretraining and pretraining settings, which is supervised by two auxiliary tasks: EFAR and MFED. Comprehensive experiments demonstrate that the proposed model can generate a more readable summary with high coverage of factual aspects as well as informing users with potential missing facts detected from the input dialogue. • We benchmark the DIS dataset in the judicial domain, where the input is the real civil trial debate data. The summaries come from the factfinding paragraphs extracted from the judgment documents of the corresponding cases. The factual aspect annotation is conducted on the output summary by five legal experts. To motivate other scholars to investigate this novel and essential problem, we will make the experiment dataset publicly available (while removing the sensitive information).

II. PROBLEM FORMULATION
Dialogue inspectional summarization is the task of condensing the factual information mentioned in the dialogue to a shorter version summary, along with two auxiliary tasks: Expectant Factual Aspect Regularization (EFAR) and Missing Factual Entity Discrimination (MFED).
Formally, each case contains a multi-role dialogue D and an inspectional summary F . The dialogue D = (u 1 , u 2 , ..., u L ) includes L utterances where each utterance u i is composed of a sequence of l words u i = (w i1 , w i2 , ..., w il ) and its speaker role r i . The summary of the dialogue is where N is the number of tokens.
During the annotation process, we define a set of factual aspects A = (a 1 , a 2 , ..., a K ) and a set of factual entities E = (e 1 , e 2 , ..., e M ) respectively, with the assistance of domain experts. For each sample, we annotate the expectant valueŷ a i ∈ {0, 1} for each aspect a i in set A and missing valueŷ m i ∈ {0, 1} for each entity e i in set E. To be specific, aspect expectant value indicates whether the aspect should be considered in the case. Entity missing value indicates whether the information is related to the case but was omitted in the dialogue, thus leading to factual inconsistency between dialogue and inspectional summary.
In more detail, the defined 12 factual aspects are Limitation of Action

III. METHOD
In this section, we introduce our dialogue summary generation framework, Dialogue Inspectional Summarization (DIS). DIS exploits domain knowledge to relieve the factual inconsistency problem and improve the credibility of the generated summary. We first describe it in a non-pretraining setting, which consists of Dialogue Encoder and Inspectional Decoder. Then we introduce the encoder-decoder model in a pretrained setting. Third, we introduce how to combine the nonpretraining and pretraining summarization model with two auxiliary tasks: (1) Expectant Factual Aspect Regularizer, and (2) Missing Factual Entity Discriminator. Finally, the objective function is described for parameters Optimization.

A. Dialogue Encoder
Considering the contextual structure of dialogues, we employ hierarchical encoders with role information integrated for dialogue representative learning. We first encode each utterance and then encode the entire dialogue. At utterance level, we use utterance encoder, notated as Enc U , to represent the local semantics of each utterance: where h ij are encoder hidden states corresponding to the token j in utterance i, w ij is the dense word embedding, and l is the maximum length of utterance. In general, utterances of different characters contain different angles of factual information in a multi-role dialogue scenario. For example, in a trial debate, the plaintiffs tend to state the facts while the defendants deny them. Meanwhile, the presiding judge asks questions about the factual details to advance the trial. Inspired by their different functions in the dialogue, we integrate the role information into utterance representation. To be specific, we apply the role embedding layer to learn the representation r for each type of the role (presiding judge, plaintiff, and defendant), corresponded to each utterance: where u local i represents the local semantics of utterance under the specific speaker role. The utterance hidden state h i is the last state of h ij , where j ∈ [1, l].
We then employ dialogue encoder, notated as Enc D , to learn the contextual representations u i of each utterance from the dialogue level: where L is the maximum utterance number of dialogue. Dialogue embedding d is the last state of u i , where i ∈ [1, L]. In our implementation, both Enc D and Enc U are single layer bidirectional LSTM 2 . We concatenate the forward and backward LSTM states to represent encoder hidden states:

B. Inspectional Decoder
We propose an Inspectional Decoder for generating summaries. Inspectional Decoder generates the summary via pointing mechanism, while the Expectant Factual Aspect Regularizer tries to ensure the factual consistency from aspect level.
From the perspective of bionics, humans tend to write a draft before focusing on factual aspects. We treat the Inspectional Decoder as a drafter, whose states need further regularized by the aspect-aware module. At each decoding timestep t, context vector c t is computed to represent the attended information from dialogue: where α t ij shows the corresponding attention distribution on encoder state h ij . The attended probability α t ij is calculated given the decoder state s t : The context vector, together with the decoder state, are concatenated to produce the generated probability distribution over the fixed size vocabulary: where W v and b v are learnable parameters. Following [7], we employ the pointing mechanism in the decoder to directly copy tokens from the dialogue, along with generating tokens from a fixed vocabulary. Generation probability p g is computed as: where W c , W s , W f , b g are learnable parameters and f t is the decoder input from reference summary {f 1 , f 2 , ..., f N }. N is the maximum decoding length. With the pointing mechanism, we calculate the probability of token w as follows: With the pointing mechanism integrated, the decoder can directly copy tokens from dialogue, making generated summary more accurate and relevant in factual details.

C. Pre-trained Encoder-Decoder Model for Dialogue Inspectional Summarization
In this section, we further investigate whether the proposed dialogue inspectional summarization can benefit the pretrained encoder-decoder model.
Specifically, we take PEGASUS [17] as the encoder-decoder model. PEGASUS is the state-of-the-art abstractive summarization model based on Transformer, which is pre-trained on a large-scale corpora by generating the selected important sentences as self-supervised tasks. We directly replace the multi-role dialogue encoder with PEGASUS Transformerbased encoder, and the inspectional decoder with the PEGA-SUS decoder.
Formally, given the input dialogue D = (u 1 , u 2 , ..., u L ), the PEGASUS encoder concatenates the utterances in the dialogue and encodes the dialogue into deep contextual hidden states as: We generate the decode state s t based on the dialogue hidden states h and the previous generated summary textF as follows: Based on the multi-role dialogue encoder-decoder model and pre-trained encoder-decoder models, we next introduce how to relieve the factual inconsistency problem with two auxiliary tasks.

D. Expectant Factual Aspect Regularizer
When writing formal documents like the legal verdict, people always carefully review their drafts to ensure that there are no inconsistencies in expectant aspects. Inspired by the process, we propose Expectant Factual Aspect Regularizer to verify the consistency on the aspect level.
For each aspect a i , we use aspect encoder to obtain its semantic embedding a i . The encoder Enc A is a single layer bidirectional LSTM to represent the aspect description text: We then produce a weighted sum of the decoder hidden states, known as the aspect-aware decoder state s a : where K is the number of factual aspects and score function uses additive attention: Finally, we feed s a into a 3-layer classifier to predict the expectant aspects: where F a is the notation of linear layers and y a ∈ R K×1 indicates the related probability of K aspects.

E. Missing Factual Entity Discriminator
As mentioned in section I, there always exists factual inconsistencies between the dialogue and reference summary. In the Seq2Seq framework, inconsistencies mislead the decoder to generate incorrect factual details. Missing Factual Entity Discriminator tries to detect the inconsistencies, thus mitigating the problem.
Motivated by the observation, we design the discriminator to classify whether the factual entity e i is missing in the conversation. In real applications, human summarizers can refer the predictions to complete generated text based on additional information.
Intuitively, we view inconsistency as the factual divergence between source content and target content, so we use bilinear layer as the classifier. F d and F s represent transformation by linear layers: where y m ∈ R M ×1 indicates the missing probability of M entities, W m and b m are trainable parameters.

F. Parameter Optimization
As for the summary generation task, we use negative loglikelihood error computed over all the timesteps: where P (f t ) is the generation probability of the target token f t . The two auxiliary tasks introduced in section III-D and III-E are both multi-label binary classification task, so we use binary cross entropy loss when computing L c (y a ,ŷ a ) and L c (y m ,ŷ m ): Finally, we combine the generation loss and classification loss as the final loss L to optimize the parameters: where λ a and λ m are hyperparameters to balance the main task and auxiliary tasks. We name the non-pretraining model as DIS and the pretrained based DIS as PEGASUS-DIS.

A. Dataset
For the experiment, we collected more than 300,000 court trial records of civil Private Loan Disputes (PLD) cases, paired with their corresponding verdicts. Legal experts helped us introduce 12 factual aspects and 14 types of factual entity, which are discriminative in PLD cases, and labeled 45,531 cases, i.e., for each case, we annotate the correlation of each aspect and the consistency of each entity. The aspect annotation indicates whether it will be considered in the court investigation. The entity annotation indicates whether its information is consistent between the trial and the verdict. In the inter-rater reliability measurement, the annotators achieved the Kappa coefficient as 0.9 (substantial agreement) after training.
We constructed the dataset by extracting the factfinding part from verdicts and filtered out those cases with a kind of fixed mode factfinding. The statistics of the dataset are shown in Table I. The entire dataset was then split into three subsets for training (90%, 27,432), validation (5%, 1,524) and testing (5%, 1,525). We show two complete samples 3 of our dataset to expose how the judges hear the case and summarize factfinding in a practical situation.

B. Implementation Details
We use PyTorch 1.4 and AllenNLP 1.0 for implementing our models. For DIS, the vocabulary size is constrained to 20,000 for both source and target. For the trial text, we limit the utterance number to 50 and utterance length to 30. We train the word embeddings from scratch with embedding dimensions of 300; as reported by [18], we also find no further improvement from pre-trained word embeddings. We set all the encoder hidden states to 256 dimensions. For the PEGASUS-DIS, we initialize the encoder-decoder with a PEGASUS model, which has 8 layers and 6 heads.
We use AdamW [19] to optimize our models. For DIS and PEGASUS-DIS, the learning rate is set to 0.001 and 0.0003, respectively. For training, we set the batch size to 16 and 8 for the two models and group the instances into batches according to their padding lengths. For DIS, the hyperparameters in the final objective function are tuned as λ a = 0.5, λ m = 0.35. For PEGASUS-DIS, these two hyperparameters are set to 0.1 and 0.1, respectively.
We train our models on a single NVIDIA GeForce RTX 3090 GPU and use loss on the validation set to apply early stopping. It takes the two models about 9 epochs and 50 epochs for convergence. For inference, we use the beam search with a size of 5 to generate the summary. We set the maximum decode length to 200, which outnumbers 80 percent of all summary lengths in our dataset.
We are planning to release the dataset and code for further research on the problem.

C. Baselines
To demonstrate the effectiveness of DIS, we implement the following summarization models for comprehensive comparison: • LEAD3 is a basic extractive baseline by selecting the first three utterances as the summary. • TextRank [3] proposes an unsupervised extractive method based on sentence importance ranking. In the experiment, we extract the top-4 important sentences from trial. • S2S [20] is a basic encoder-decoder model for sequence-tosequence learning. We concatenate all the utterances in the trial as the input sequence. • S2SAttn [21] modifies Seq2Seq model with the attentional decoder, which utilizes the context information given the attention distribution over encoder hidden states. • PGN [7] extends attentional Seq2Seq model with copy mechanism. The decoder calculates a generation probability  to choose between generating tokens from the fixed vocabulary and copying tokens from the source. • FastSum [22] jointly combines content selection and rewriting into a summarizer. We adjust the sentence-level content selection to utterance-level in the experiment. • DAH [18] proposes a model driven by hierarchical attention.
It is suitable for the discourse structure of judicial trials. • PEGASUS [17] is a Transformer-based abstractive summarization model which is pre-trained on large text corpora with a self-supervised objective, which removes/masks important sentences from the original documents and then generates them. To make a fair comparison, we fine-tune the pre-trained PEGASUS on the PLD dataset with exactly the same hyperparameters we used to train PEGASUS-DIS.

D. Evaluation Metrics
To validate the overall quality of the generated factfinding, we report results on three evaluation metrics for different perspectives: (1) ROUGE [23], (2) BERTScore [24] and (3) human evaluation. ROUGE is the standard metric for summarization task. We report the F 1 score for ROUGE-1, ROUGE-2, and ROUGE-L, obtaining the scores with the files2rouge 4 package. From a linguistic perspective, the indicators can reflect the informativeness and fluency of the summary.
BERTScore is recently proposed as a novel evaluation metric for text generation tasks. Instead of the exact token match in ROUGE, BERTScore computes token similarity with contextualized embeddings from pre-trained language models like BERT [25]. In the experiment, we choose 12-layer BERT BASE to get the score.
We perform human evaluation to verify whether the performance on automatic metrics correlates with human perceived quality. We hire five annotators well trained in reading legal documents and randomly select 100 cases from the test set for evaluation. For each case, we show the judicial trial, the factfinding in the verdict, and results generated by S2SAttn, DAH, and DIS. The annotators are asked to score each factfinding on three indicators under the 1 to 5 scale: Logical Completeness, Factual Consistency, and Readability.

A. Overall Performance
We evaluate the overall performance comprehensively from the following perspectives: (1) comparison against baselines on automatic metrics, (2) the effectiveness of proposed auxiliary tasks, and (3) human perceived quality of generated summaries.
Comparisons against baselines. As shown in Table II, we first report the experimental results against baselines on automatic metrics. Following the setting in section IV-C and IV-D, we have the observations based on the results: (1) Our 4 https://github.com/pltrdy/files2rouge

Judge
Now begin the court investigation, first by the plaintiff's claims.

Plaintiff
Order the defendants to repay the principal amount of $40,000 (Loan Amount) and the corresponding interest (Calculated at the monthly interest rate of 2% from Jan 1, 2015 (Repayment Period) to the date of repayment of principal). And the defendants bears the full cost of the case.

Judge
Ask the defendants to answer. Defendant 2 I am not a guarantor, I am an introducer. Plaintiff The defendant 2 is the guarantor of the loan.

Judge
The defendants have no objection to the fact that the defendant 1 borrowed $40,000 with interest. The two parties have different opinions on whether the defendant 2 is a guarantor. Does the two sides have different opinions on the controversy of this court? Plaintiff No.

Defendant 2
No. Judge Ask the plaintiff to give evidence.

Plaintiff
Provide the "Debit Note" and the plaintiff's identity certificate to prove the plaintiff's subject of litigation. Judge Ask the defendants to cross-examination.

Defendant 2
There is no objection to the evidence, however the "Debit Note" is issued by the defendant 1. Judge Plaintiff, who issued the "Debit Note"? Plaintiff The defendant 1 issued. Judge Plaintiff, how much did the defendant 1 borrow from you?

Plaintiff
The loan is $40,000, and the corresponding interest rate (Agreed Rate/Interest) is 2% per month.

Judge
How was the loan delivered?
Plaintiff I withdrew the money from the bank and delivered the loan on site (Delivery).
Judge Defendant 2, you said you are the introducer, not the guarantor. Why did you sign your name as a guarantor?
Defendant 2 I didn't sign my name on the "Debit Note". The defendant 1 wanted to borrow money, and the plaintiff had money to lend. So I introduced them to each other.

Judge
Plaintiff, did you personally see the signature of the guarantor on the loan note?

Plaintiff
The defendant signed on the loan note. When I lent the money, the defendant 2 said he would guarantee it for me.

Judge
Plaintiff, the interest had paid until December 2014 (Repayment Behavior). When will the interest be calculated afterwards?
Plaintiff Calculate subsequent interest from January 1, 2015. DIS model outperforms all the baselines without pre-training over ROUGE-1, ROUGE-2 and BERTScore, though slightly worse than DAH over ROUGE-L. (2) The fine-tuned PEGA-SUS model outperforms all the models without pre-training, which shows the advantages of large pre-trained models. (3) However, the PEGASUS-DIS model still gives better results than PEGASUS on all automatic evaluation metrics, which demonstrates that logical/factual gaps may still exist in pretrained models and the DIS framework is still usefull in this setting. (4) The two basic extractive methods, LEAD3 and TextRank, perform rather poorly on the PLD dataset because the factual details scattered in the dialogue are difficult to represent by certain utterances. DAH performs better than other abstractive baselines, indicating that the hierarchical representative learning is critical in summarizing dialogues.

Reference
Human perceived quality. As shown in table III, we report the human evaluation results. First, DIS outperforms the selected baselines without pre-training on all the indicators, especially in factual consistency, while still having a certain gap with the reference summary. It is noteworthy that S2SAttn performs poorly in factually related indicators despite having acceptable performance in terms of automatic evaluation. This finding indicates that existing evaluation metrics are insufficient to tell the differences concerning factuality. Second, The fine-tuned PEGASUS model again demonstrates its advantages in human evaluation experiments on the three indicators, which demonstrate that pre-trained models can not only improve the fluencies but also be able to relieve the logical/factual inconsistency to some extent. Third, the proposed PEGASUS-DIS still outperforms PEGASUS on the three indicators, which again shows that the DIS framework is able to benefit pre-trained abstractive summarization models. Finally, as for readability, the level of all the results is relatively high, without obvious distinction. It reflects that generating factual consistent summaries is more significant than generating fluent summaries. Fig. 4. Case Study 2. Words in red denote the factual aspects that are correctly generated by the tested methods while the words in green denote the incorrect ones. Our proposed model also generate inspectional summarization with the predicted essential aspects to be contained in the summary as well the missing entity (Loan Start Date in orange). The and × indicate whether the generated aspects and missing entities are evaluated correctly or not.
Effectiveness of auxiliary tasks. We conduct the ablation test to assess the contribution of each component in DIS framework. The fourth section in table II shows the results. We maintain the aspect embedding to compute the aspect-aware decoder state when removing EFAR task. Its removal causes 4.8% relative increase in error (RIE) for ROUGE-2 F 1 score, of which the impact is more critical than the removal of MFED task (2.0% RIE for ROUGE-2 F 1 score). The finding indicates that both the auxiliary tasks contribute to the informativeness, while EFAR presents a more significant role. Figure 3 and 4 show two case studies by comparing the generated summary of the proposed method against the best baseline and the reference summary. As shown in Figure 3, our method outperforms PEGASUS in generating high coverage of accurate factual content by predicting the expectant factual aspects correctly. For example, the proposed PEGASUS-DIS predicts the Repayment Behavior correctly and generates corresponding texts in the summary. Also, there exists one missing factual entity in the reference summary (Loan Start Date). DIS detects the inconsistency to inform the users. Similar observations can be found in Figure4. With the help of the EFAR auxiliary task, the PEGASUS-DIS generates factual contents, e.g., Repayment Behavior and Delivery, which are missed by the fine-tuned PEGASUS.

C. Error Analysis
From the sampling statistics and human evaluation feedback, we conclude the major problems occurring in DIS results as follows: (1) 25.5% of the errors come from the failure to distinguish between multiple facts. The lengthy and colloquial dialogue makes it difficult for the model to capture subtle differences and determine the correct facts. (2) 18.1% of the errors are attributed to their adverse results on auxiliary tasks. EFAR and MFED predictions are related to the summary content so that the false aspect can mislead to false facts. (3) We also find that 14.6% of the errors happen when the factual information needs to be obtained through a certain degree of reasoning. For instance, the summary may infer a specific date or calculate the total amount given fragmented information in dialogue. To improve the proposed method in the future, introducing numerical calculation and reasoning capabilities into text generation can be a promising direction.

VI. RELATED WORK
Generally, neural text summarization has two strategies: extractive summarization and abstractive summarization. Extractive summarization merges the selected utterances from dialogue to form the summary, so it fails to produce consistent results. For dialogue inspectional summarization, abstractive strategy is more suitable because it can generate novel expressions to summarize the interactions. In the past few years, neural abstractive text summarization has received much attention from the research community. Seq2Seq [20] framework enables the summarizer to generate diverse, relevant, and readable results. Since the first successful attempt by [8], many techniques have been proposed to improve the accuracy and versatility. The most influential work includes copying mechanism [7], [26], AMR parsing [27], reinforcement learning approaches [16], [22], coverage [28] and combination of extraction and abstraction [9]- [11].
Our work especially belongs to dialogue summarization, which has presented new challenges to abstractive summarization: information in the interactions are more difficult to capture than documents with sequential logic; the lack of suitable corpora restricts the relevant research progress. Recent work adapted the practice in news summarization to this field and introduced supplementary annotations. For instance, [12] incorporates dialogue acts to better recognize the interactive patterns in meeting summarization; [13] introduces key point sequence generation to improve the logic and integrity of summaries in the customer service domain. [15] pay attentions to the controversy focus space of a civil trial dialogue. Unlike their work, our proposed framework particularly focuses on the factual inconsistency problem, aimed at generating a more faithful summary.
More recently, pre-trained models based on Transformer [29]- [31] have shown their dominant advantages from language understanding tasks to language generation tasks, e.g., machine translation, dialogue generation and text summarization. For language generation pre-training, a core problem is how to construct the self-supervised task from the unlabeled corpus. UniLm [29] proposed a unified model for both language understanding and generating tasks by designing different self-attention masks for different kinds of tasks. T5 [32] converts various NLP tasks into a unified text-totext framework, and trains the tasks with a sequence-tosequence model. MASS [30] is a sequence-to-sequence pretraining model, which masks a consecutive text fragment in a sentence and predicts the masked tokens using a encoderdecoder model. BART [33] constructs the self-supervised task by corrupting the document with transformations, e.g., token deletion and text infilling, and then recovering the original document. PEGASUS [17] is the state-of-the-art abstractive summarization model, which is pre-trained on a large-scale corpora by generating the selected important sentences as a self-supervised task. Our work further investigates if such pre-trained abstractive summarization models have the factual inconsistency problem, and if the proposed DIS framework can benefit the pre-trained models.

VII. CONCLUSION
In this work, we mainly investigate Dialogue Inspectional Summarization (DIS) by solving factual inconsistency problem. We propose DIS as a novel end-to-end dialogue summarization framework under non-pretraining and pretraining settings, supervised with two auxiliary tasks, namely Expectant Factual Aspect Regularization (EFAR) and Missing Factual Entity Discrimination (MFED). The auxiliary tasks align the generated summary with dialogue on different factual granularity.
For experiments, we benchmark the DIS dataset in judical trial summarization. Based on comprehensive evaluation and analysis, we demonstrate that the DIS framework can generate a more readable summary with accurate coverage of factual aspects, as well as informing the user with potential factual inconsistencies for further human intervention.