{'fraction_non_alphanumeric': 0.032248440466503936, 'fraction_numerical': 0.009981014374830486, 'mean_word_length': 4.795504558314995, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
ClueGraphSum: Let Key Clues Guide the Cross-Lingual Abstractive Summarization

Cross-Lingual Summarization (CLS) is the task to generate a summary in one language for an article in a different language. Previous studies on CLS mainly take pipeline methods or train the end-to-end model using the translated parallel data. However, the quality of generated cross-lingual summaries needs more further efforts to improve, and the model performance has never been evaluated on the hand-written CLS dataset. Therefore, we first propose a clue-guided cross-lingual abstractive summarization method to improve the quality of cross-lingual summaries, and then construct a novel hand-written CLS dataset for evaluation. Specifically, we extract keywords, named entities, etc. of the input article as key clues for summarization and then design a clue-guided algorithm to transform an article into a graph with less noisy sentences. One Graph encoder is built to learn sentence semantics and article structures and one Clue encoder is built to encode and translate key clues, ensuring the information of important parts are reserved in the generated summary. These two encoders are connected by one decoder to directly learn cross-lingual semantics. Experimental results show that our method has stronger robustness for longer inputs and substantially improves the performance over the strong baseline, achieving an improvement of 8.55 ROUGE-1 (English-to-Chinese summarization) and 2.13 MoverScore (Chinese-to-English summarization) scores over the existing SOTA.


INTRODUCTION
With the constant acceleration of information globalization, an ever-increasing mass of information in various languages has flooded into people's views [1], [30], [43]. Whether it's scientific research or daily reading, it will always encounter the need to quickly grasp the gists of such massive articles. However, it is hard for people to accomplish this for an article written in an unfamiliar language. At this time, the need for cross-lingual summarization (CLS) has become extremely urgent. Cross-lingual summarization is the task to compress an article in a source language (e.g. English) into a summary in a different target language (e.g. Chinese). It needs to generate a concise text that contains the major points of a given article [15], [41].
With the widespread use of deep neural networks, many Transformer-based end-to-end methods [4], [5], [8], [9], [36], [48], [49] are proposed to directly understand bilingual semantics and avoid the error propagation problem. Due to lacking large-scale CLS datasets, Duan et al. [9] and Dou et al. [8] explore training end-to-end models with zeroshot learning. Then Zhu et al. [48] construct two largescale cross-lingual summarization datasets and reinforce their Transformer-based neural CLS model (TNCLS) with a multi-task framework. Subsequently, Zhu et al. [49] greatly shorten the training time of multi-task NCLS by adding attending, translating, and summarizing operations into the Transformer-based CLS model instead of using the multitask framework.
Although these methods has achieved a big improvement, there still is a challenge for models to generate a crosslingual summary that correctly translates and condenses all major points of a long text, especially when generating the multi-sentence summary. This is because a long text usually contains much noisy information like organization or author introduction, advertisements, backgrounds, etc. and the attention of models is easily distracted or misled by those.
To handle this problem, we propose a clue-guided crosslingual abstractive summarizer, ClueGraphSum. The inspiration of ClueGraphSum comes from the pattern how we write cross-lingual summaries. When writing crosslingual summaries, we find writers usually tend to select and translate keywords, named entities, etc. as key clues for summarization first, then use related sentences to write target summaries covering these key clues (As shown in Figure 1). So we first extract the keywords, named entities, Ⅰ Input (English): (CNN)A photography teacher at a Southern California high school appears to have committed suicide by hanging herself in a classroom. Jillian Jacobson, 31, was found Monday at El Dorado High School, said Lt. Eric Point, with Placentia Police. There was no indication of foul play, and every indication that she took her own life, he said. According to Steve Concialdi with the Orange County Fire Authority, students had been waiting outside the classroom to get in. Another teacher opened the door. When they went inside the classroom, they found Jacobson hanged. They quickly brought her to the ground and called 911, said Concialdi, but she was declared dead at the scene. If you're in crisis or contemplating suicide, please contact The National Suicide Prevention Lifeline at 1-800-273-8255. Students posted tributes to the teacher on social media, saying she would be greatly missed. One called her caring; another said she was a beautiful soul. Crounselors are expected to be on campus all week to support staff and students. CNN's Stella Chan and Sonya Hamasaki contributed to this report.  etc. of the input article using TextRank [29] as its key clues for summarization; then an article graph with less noisy sentences is constructed based on these clues, capturing the relations and importance of sentences; finally, article graph and key clues are sent to our ClueGraphSum model to jointly decide which clues will be translated and added to CLS summaries. In this process, key clues are used to lock all the important points of articles and article graphs are used to link key clues and supplement them into a complete and fluent summary. As shown in Figure 1, these key clues and sentences linked by them are informative enough to generate the target summary.
In conclusion, our contributions are as follows: • We propose an article graph construction method that can correctly focus on the most important sentences via the vertex weight we defined and reduce the negative effects of noise by pruning noisy sentences. It transforms an article into a graph constructed of sentences based on key clues.

•
We propose a clue-guided cross-lingual abstractive summarization method, which incorporates article graphs and key clues for summarization into a unified model, namely ClueGraphSum, to co-generate summaries. It utilizes the end-to-end framework to directly map source-to-target language semantics.
The experimental results show that our method outperforms the baselines and its performance is comparable with the SOTA. Moreover, our method has stronger robustness for the length of inputs.

•
We constructed a novel hand-written CLS dataset (namely CyEn2ZhSum) to avoid the "translationese" [14] phenomenon for the evaluation of cross-lingual summarization. It also provides extra reference information for further research.

RESEARCH OBJECTIVE
The main research objective of this work is to design an end-to-end model to improve the quality of cross-lingual summaries by enhancing the model's robustness to the article length and unifying the translating and summarizing in one model. In the previous research about cross-lingual summarization, most of them are pipeline based, which suffers from error propagation and costly inference. Thus, we chose to develop an end-to-end model to unify the bilingual semantic mapping and summarizing to avoid this problem.
In the end-to-end research about cross-lingual summarization, the overlength articles are often directly truncated and exist noisy information distracting the model's attention, resulting in the omission of some key points in the target summaries. So, we propose the clue-guided cross-lingual model to alleviate this problem. In addition, the existing CLS datasets are all obtained through translating the source texts or target summaries with different strategies, which will face the problem of "translationese". Therefore, we construct a novel handwritten CLS dataset for evaluation.

RELATED WORKS
The overwhelming information globalization forces people to handle vast quantities of information in various languages. Summarizing non-native language documents into native language abstracts provides an effective way to deal with this situation. Therefore, cross-lingual summarization is continually being researched and optimized in recent years.
Leuski et al. [24] implement the translate-thensummarize method to generate English titles from Hindi documents. Considering the translated text is not completely accurate, Ouyang et al. [32] propose to train a robust English summarizer on noisy English documents and clean English summaries. So they can translate the document in low-resource languages into English and then utilize this summarizer to generate English summaries. Lim et al. [26] adopt the summarize-then-translate method to obtain a Japanese summary by translating the Korean summary generated by their Korean summarizer. Orašan et al [31] first use the maximum Marginal Relevance (MMR) method to summarize Romanian news, and then utilize eTranslator 1 , an English-Romanian bidirectional translator, to translate the Romanian summaries into English. To prevent target summaries from becoming unreadable, Wan et al. [40] not only simply use the existing translator, they also consider the translation quality of each sentence. They utilize SVM regression to predict the translation quality of input English sentences and then choose informative ones with high translation quality to form the Chinese summaries.
All the above methods only make use of the information from one language side. However, the semantic information from both source and target languages is useful and should be considered [39]. Wan et al. [39] implement the bilingual information in their graph-based ranking framework to deal with the cross-lingual summarization task. Their experiments' results prove that the information on the target language (Chinese-side) is more helpful. Inspired by the phrase-based machine translation models [22], Yao et al. [44] simultaneously implement sentence extraction and compression based on the bilingual phrases alignment information. Zhang et al. [45] resolve source-side documents into predicate-argument structures (PAS) with targetside counterparts, and then produce summarises by fusing bilingual PAS structures. 1. http://www.etranslator.ro/

End-to-End Cross-Lingual Summarization
However, these pipeline-based methods not only result in error propagation [33] but also make the inference costly as they need to run a summarization system and a translation system sequentially [23].
Meanwhile, Zhu et al. [48] introduce a round-trip translation strategy to construct two large-scale high-quality CLS datasets, respectively En2ZhSum and Zh2EnSum. They then improve the Transformer-based neural CLS model (TNCLS) through a multi-task framework that contains one encoder and two decoders separately performing MT and MS tasks. Cao et al. [5] use two encoders and two decoders to jointly learn aligning and summarizing. MCLAS model proposed by Bai et al. [4] shares one decoder with CLS and MS tasks to establish the connections between discrete phrases in different languages. Takase et al. [36] further simplify the structure of these multi-task frameworks by attaching a special mark at the beginning of the input sentence to indicate the target task. Their model does not require additional architecture.
Nevertheless, multi-task learning still requires a large amount of extra parallel corpus and its training process is quite time-consuming. To alleviate these issues, Zhu et al. [49] integrate TNCLS with an external probabilistic bilingual lexicon through the attending, translating, and summarizing operations. Experimental results show that their method shortens the training time and enhances the model performance at the same time. To match the training objectives and evaluation objectives of CLS models, Dou et al. [8] replace the cross-entropy function with the reward mechanism of reinforcement learning (RL). They utilize RL with bilingual semantic similarity as rewards to directly produce a target language summary.
As most methods utilize Transformer as the basic model, their source articles are directly sent or truanted into the model regardless of whether source articles contain noisy information. However, Transformer cannot handle long input all [6] and cannot capture the relation between sentences of source articles and key points of target summaries. To alleviate these problems, we propose a clue-guiding cross-lingual abstractive summarization method that combines graph networks and the translation pattern into Transformer.

OUR MODEL
The overall architecture of our model namely ClueGraph-Sum, is depicted in Figure 2. Here, let V s and V t be the vocabulary of the source language and target language respectively. Given an input article as a sequence of sentences D = {s 1 , s 2 , ..., s m } where s i represents a sentence, we first extract keywords and named entities from D as key clues C = {c 1 , c 2 ..., c n } using TextRank ( [29]), where c i is a word or phrase (D, C ⊆ V s ). Then, we construct the article graph Overview of ClueGraphSum Model. The left part in the picture is Graph encoder, the right part in the picture is Clue encoder and the middle part is the decoder. s represents the embedding of a sentence. c is the key clue and w is the word in target summary. G = (V, E) based on C according to Algorithm 1 and then embed them into our model as described in Section 4.1.2. The Graph encoder in Section 4.1.2 and Clue encoder in Section 4.1.3 relatively maps G and C into hidden vectors. And the decoder generates a summary S = {w 1 , w 2 , ..., w k } in the target language from them, where w i represent a word and (w i , S ⊆ V t ).
ClueGraphSum integrates the graph network into Transformer ( [37]) using the graph attention (GAT) ( [38]) and adopts the Naive translation strategy [49] to decide which words are translated.

Graph Construction
This section introduces how to construct the article graph based on key clues. Articles extracted from online websites usually contain much noise ( [25]), like advertisements, image credits, editorial and author information, etc. Therefore, we first extract keywords, named entities, etc. of the article using TextRank as its key clues to evade these noise.
After getting key clues C of the article, we construct the article graph following Algorithm 1 (denoted as AGC). Every sentence is set as a vertex. The vertex v i and v j are connected by edge e ij if they share at least one key clue and the weight of e ij equals the number of key clues they both  8 if v i and v j contain common clues in C then 9 Add an non-self-loop edge e ij between v i and v j 10 Calculate the weight of e ij : ω ij = count(common clues) 11 end 12 end 13 for vertex v i in the vertexes of G do 14 Calculate the vertex weight: ω(v i ) = j∈neighbor(vi) ω ij + ω ii 15 end 16 Remove the vertexes whose weight equals 0 17 Return G contain. We also add a self-loop edge for every vertex to enhance attention of self information. The weight of a selfloop edge equals the number of key clues its vertex contains and the weight of every vertex equals the sum of all the connected edges' weights. The importance of an vertex is represented by its weight and the vertex with a weight of 0 is removed to reduce the impact of noisy or useless sentences.

Graph Encoder
Embedding Vertices: As stated above, a vertex is associated with a sentence which consists of a word sequence s = {w 1 , w 2 , ..., w t }. In order to capture the positional information of every word in the sentence, we use Positional Encoding (PE) ( [37]) to get its positional embedding. The final embedding of i-th word is the sum of its original word embedding w i and its positional embedding. Then we concatenate that as the sentence embedding and feed it into self-attention layer to obtain the embedding h s for each vertex v s as follows: Embedding Graph: After getting the embedding of each vertex, we implement multi-head graph attention to stabilize the learning process and capture the article graph structure as well as sentence information. The graph attention of every head is independently calculated and then concatenated together. In this process, every vertex v i updates its embedding using the attention over itself and 1-hop neighbors represented by the set N (v i ): where F denotes the number of heads, α ij denotes the attention between vertex v i and v j , and · T represents transposition. W and a are respectively learnable weight matrix and vector.
In the last layer of graph attention network, averaging is employed instead of concatenation to combine multi-head graph attention outputs: To avoid the over-smoothing problem, we employ a residual connection ( [16]) and layer normalization ( [3]) around a feed-forward network (FFN) layer to get the final outputs [z g 1 , z g 2 , ..., z g t ] of the Graph encoder as follows:

Clue Encoder
Although the Graph encoder has captured the global article structure and sentence semantics, some importance information are still omitted because a vertex can not represent different word semantics in one sentence. Therefore, we use the encoder of Transformer as Clue encoder to capture the key clues C in sentences. These key clues are concatenated as the input of Clue encoder. After embedding these clues, the hidden vector is fed into a multi-head attention block as query Q, key K and value V : where √ d k is the dimension of the key and h is the number of heads. The results of multi-head attention are passed through the rest layers of Clue encoder as shown in Figure 2 to get the outputs z c i of Clue encoder.

Decoder
The decoder shares the same architecture as Clue encoder, except that two extra cross-attention layers are added to perform multi-head attention on the outputs of Graph encoder and Clue encoder. Considering that some keywords in the cross-lingual summary can be directly translated from the key clues, we leverage the Naive strategy proposed by Zhu et al. [49] to acquire the translation probability P T , deciding which words are directly translated. The translating probability p trans∈[0,1] is computed from the decoder hidden state H dec via a dynamic gate: where b 1 and b 2 are learnable bias vectors, W 1 and W 2 are learnable matrices, and σ is the sigmoid function. The p trans determines whether to generate a word w from the neural distribution sampling P N (w) or directly select a word from the translation candidates of the source words. Thus, the final probability distribution P (w) is calculated from P N (w), P T and p trans as follows: where P T (w src ⇒ w) is the translation probability of word w src to word w.

Datasets
We evaluate our method on the benchmark CLS datasets En2ZhSum and Zh2EnSum created by Zhu et al. [48] and our human-written CLS dataset CyEn2ZhSum.
En2ZhSum is an English-to-Chinese summarization dataset converted from MSMO [47] and CNN/DailyMail [17]. Zh2EnSum is a Chinese-to-English summarization dataset constructing from LCSTS [18] dataset. The summaries of these two datasets are both translated from original monolingual summaries with a round-trip translation strategy. Since the translated texts are often "translationese", we construct a small but precise human-written dataset, namely CyEn2ZhSum, to further evaluate performance of models on human-written CLS. The input of CyEn2ZhSum is an English news article about cybersecurity, and the output is a Chinese summary which is first written by a postgraduate, and then reviewed and revised by researchers. In addition, CyEn2ZhSum also contains extra reference information like titles in Chinese and English, publish dates, etc. The statistics of CyEn2ZhSum dataset are presented in Table 1 and the division of these datasets is presented in Table 2. An example of CyEn2ZhSum dataset is provided in Appendix A.

Baselines
We compare our method with two relevant pipeline-based methods and several strong end-to-end methods: • Pipe-TS: It refers to the translate-then-summarize method. It first translates the source text into the target language by Google Translator 2 . Due to the lack of corresponding MS datasets in the target language, we then following Refs [8], [42], [48], [49] summarize the translated text by LexRank [10], a powerful unsupervised summarization method.  [48], which is an end to end automatic bilingual abstractive summarization system.

ATS-NE:
It refers to the ATS model with Naive strategy, which is an end-to-end method proposed by Ref [49]. ATS model integrates the operations of Attending, Translating and Summarizing by adding an additional translation layer to Transformer. In the translating operation, the authors propose 3 strategies, of which we use the Naive strategy.
• ATS-A: It refers to the end-to-end ATS model [49] with Adapt strategy. As the Adapt strategy performs best in its translation strategy, we choose it as our baselines as well.
We denote our ClueGraphSum model as CGS.

Evaluation Metrics
Following the previous studies [5], [9], [11], [35], [48], [49], we assessed all models with the F1 scores of standard ROUGE metrics [27] (ROUGE-1, ROUGE-2 and ROUGE-L) and METEOR metric [7] on all datasets. And the ROUGE scores of Chinese outputs are calculated on character-level [19]. Besides, we employ MoverScore 3 [46] as additional evaluation metric for English summaries, because it compares the similarity of two samples based on their semantics rather than the word repetition rate.
ROUGE: ROUGE-1/2/L evaluates the quality of generated summary by calculating the co-occurrence probability of units (unigrams, bigrams and the longest common sequence respectively) in the generated summary and reference summary. The F1 score of ROUGE metrics is defined as follows: where R R ,P R and F R are respectively the Recall, Precision and F1 score of ROUGE. For ROUGE-1/2, the length of summary equals the number of unigram/bigram in summary. Following Hu et al. [19], the ROUGE score of Chinese summaries is calculated by character. METEOR: METEOR metric expands the clue of "cooccurrence". Apart from the absolutely identical unit, it also counts words with the same stem and synonyms into the co-occurrence unit, making it more relevant to human judgments. At the same time, METEOR metric takes the word order into the assessment category and establishes a penalty mechanism based on it. The METEOR score is computed as follows:  where F M is the F1 score of METEOR; P enalty is the penalty factor for word order; α, γ and θ parameters are set according to the official settings 4 . R M and P M are the unigram Recall and unigram Precision, which are obtained in the same way as ROUGE. #chunks represents the number of overlapped chunks, #unigrams matched represents the number of overlapped unigrams in the reference summary.
MoverScore: MoverScore is a semantic-based evaluation metric for text generation tasks. It encodes the candidate and reference by contextualized word embeddings, and then leverage the Earth Mover's Distance [34] to compute the semantic similarity via comparing their embeddings.

Experimental Settings
Following configuration transf ormer base [37], the embedding dimensions d model and the inner dimension d f f are respectively set to 512 and 2048, and Clue encoder and decoder both have 6 layers of 8 heads for attention. For Graph encoder, it is set to 1 layer of 3 heads for attention. We use the batch size of 1024 for all datasets. The vocabulary setting follows Ref [48].
All the parameters are initialized via Xavier initialization method [13] and the dropout rate is 0.1. During training, we apply Adam [20] with β 1 = 0.9, β 2 = 0.998 and = 10 −9 as the optimizer to train parameters. During the evaluation stage, we decode the sequence using beam search with a width of 4 and length penalty of 0.6. For CyEn2ZhSum dataset, we carry out the experiments of several end-toend methods (including TNCLS, ATS-NE, ATS-A, CGS) on it through fine-tuning the model trained from En2ZhSum dataset.

Automatic Evaluation
Results on En2ZhSum and Zh2EnSum. We have replicated baselines on the En2ZhSum and Zh2EnSum datasets. The experimental results are provided in Table 3. As shown in Table 3, our model CGS can substantially outperform all baselines in both English-to-Chinese and Chinese-to-English summarization. It achieves an improvement of 3.39+ ROUGE-1 and 2.51+ METEOR scores compared with baselines. Compared with ATS-NE model which utilizes the same translation strategy with CGS, CGS obviously performs better in all metrics. Compared with ATS-A model 4. http://www.cs.cmu.edu/ alavie/METEOR/ which is the best model in baselines, CGS achieves an improvement of 5.60 ROUGE-2 score on the En2ZhSum dataset and 2.13 MoverScore score on the Zh2EnSum dataset. This reveals our method is superior to baselines not only in word repetition, but also in semantics.
It is clear that the performance of end-to-end methods exceeds that of pipeline methods in both directions although the pipeline methods utilize the most advanced machine translation system. This is because end-to-end methods could directly establish the mapping relation between two languages, avoiding the error propagation. Further comparing the performance of two pipeline methods, we find the summarize-then-translate method outperforms the translate-then-summarize method in both cases, which is consistent with the conclusions of research ( [5], [40]). Especially, the late-translation method far exceeds the early-translation one on the Zh2EnSum dataset with 12.37 ROUGE-1 and 13.48 MoverScore scores. We speculate this is because the Zh2EnSum corpus has higher requirements for summarization ability (as its word repetition rate of summaries and references is much lower than En2ZhSum dataset), while the early-translation method already causes the semantic deviation at the first stage.

Results on CyEn2ZhSum.
We implement end-to-end baselines and our method on the human-written CyEn2ZhSum dataset. The experimental results are shown in Table 4. It is obvious that CGS can achieve better performance in all metrics except for ROUGE-L. As the CyEn2ZhSum dataset belongs to the cybersecurity domain, it contains lots of proper nouns in the field of cybersecurity. The translation of these proper nouns are much different from daily words. Thus, we speculate that the higher ROUGE-L score of ATS-A is attributed to its Adapt strategy ( [49]) which can appropriately adjust the translation probability of the source-target pairs to the new domain. However, the Adapt strategy requires more parameters although it is better than the Native strategy we adopted.
The higher METEOR and MoverScore scores of CGS indicate the overall semantics of summaries generated by  CGS is closer to reference summaries although more the longest common sequence appear in the summaries of ATS-A. This is most likely because some reference words in the summary generated by CGS are represented by synonyms or the same morpheme with different forms.
In conclusion, our CGS model markedly outperforms baselines in terms of automatic evaluation.

Ablation study
In this section, we carry out an ablation study to analysis the importance of each key component of our method. Graph encoder and Clue encoder are respectively removed in the ablated version of CGS. Table 5 presents the experimental results of this ablation study on En2ZhSum, Zh2EnSum and CyEn2ZhSum datasets. Results reveal that both ablation operations will cause a decrease in the model performance. We find Clue encoder contributes more to our model than Graph encoder as CGS-C performs much better than CGS-G. This is because Graph encoder only captures sentence semantics while Clue encoder learns multiple key clues, capturing richer and more core semantic information.

Human Judgement
ROUGE and METEOR can evaluate the quality of summaries from the perspective of word coverage, but it is difficult to evaluate the conciseness, informativeness and fluency of the generated summaries. So we randomly selected 60 samples form the test sets of all the datasets for manual evaluation. Three researchers are firstly asked to evaluate the quality of generated summaries from three independent perspectives: conciseness, informativeness, and fluency. Each perspective is scored from 1 (worst) to 10 (best). The average scores are given in Table 6. They then compare the overall proximity of generated summaries to reference summaries using Best-Worst Scaling method ( [21]). The results are shown in Table 7 and its values range from 1 to -1. A higher value indicates a model with better performance.
In addition, we leverage Fleiss' Kappa ( [12]) to evaluate the scoring consistency between annotators.
As shown in the Table 6, our method obviously performs better than baselines in informativeness, conciseness and fluency on all datasets. What's more, the results in Table  7 indicates the summaries produced by our CGS are the closest to the reference summaries, which is the same with automatic evaluation results.

Can the AGC Algorithm focus on the importance sentences?
To further explore whether our AGC algorithm can really focus on the right important information, we compare it with two other graph-based summary extraction algorithms, respectively TextRank and LexRank. We use these three methods to extract the most important three sentences as summaries, and then evaluate their performance with the ROUGE-L, METEOR and MoverScore metrics. We conduct this experiment on En2ZhSum dataset due to the length of articles. The results in Table 8 show our AGC algorithm outperforms these two comparative methods. Thus our AGC algorithm can correctly focus on the sentences that are most important to the summary.

The Impact of Article Length
In order to explore the impact of article length on each model, we divide the articles in the En2ZhSum test set into 10 collections according to the article length. These 10 collections are shown in the legend in Figure 3, where [a, b) represents a set of articles whose number of words is bigger than or equal to a and smaller than b. We compare the changes in ROUGE-L scores of end-to-end baselines (including CGS, ATSA, ATSNE, TNCLS) on these 10 collections. Based on the results of each model on the [0, 100) collection, the score differences between other sets and this set are shown in Figure 3.  Figure 3 show that the ROUGE-L score of each model decreases with the increase of article length, but our CGS decreases much slower than other models. This indicates that CGS is more robust to the length of input texts. And the overall downward trend most likely results from the increasing redundant information distracting the attention of models. The reason why our CGS can greatly relieve the impact of the change of article length on the model is that it extracts the important information as key clues to emphatically focus on in the first step and the noisy sentences and important sentences are treated differently at the time of graph encoding. So the attention of CGS model is largely undistracted by useless information.

Case Study
We present a case study of a generated En2Zh summaries in Figure 4. It can be seen that almost all information in the reference comes from the three most important sentences chosen by AGC algorithm and only CGS correctly and concisely summarizes the source article. This reveals the effectiveness of AGC algorithm and its ability to correctly locate important sentences.
We find that the generated summaries of all baselines except for Pipe-TS more or less capture some information in the reference, but none of them are fully correct and complete. Since Pipe-TS utilizes Google translator and an extractive summarization method, the fluency and informativeness of its generated summary are obvious better than those of other baselines. But it's summary is less concise than CGSs' and is obvious "translationese".
On the other hand, the repeated information like "Gonzale, pregnant, Andrews, affair, etc." in the summaries generated by Pipe-ST, TNCL, ATS-NE and ATS-A imply that the model usually tends to generate repeated information which appears in the reference to improve the recall rate. In particular, all generated summaries omit or misrepresent the information of "Gonzale's age". Although the summary generated by CGS does not correspond to every word of the reference, it captures all the important points and expresses them in other ways.
In conclusion, our CGS can produce a summary that is more consistent with the reference than baselines.

DISCUSSION AND CONCLUSION
Cross-lingual automatic summarization is a meaningful and useful research task because it can greatly shorten the time of manual writing and can provide a tool for readers to quickly grasp the gist of an article written in an unfamiliar language. This paper has proposed an end-to-end CLS framework, called "ClueGraphSum", to implement the automatic generation of high-quality cross-lingual summaries. The main theoretical implications of this study are as follows: • Exploring the leadership role of key clues for summarization in CLS task and proposing a clue-guided cross-lingual summarization method to directly understand and summarize the bilingual semantics.
• Proposing a novel article construction algorithm (denoted as AGC) that takes sentences as nodes and links the nodes which share common key clues.
• Proposing a new node weight calculation method to locate important sentences based on AGC algorithm.
• Exploring the impact of article length to the neural net model and finding that the performance of models decreases with the increase of article length.

•
Exploring the performance of different models on translated datasets and our hand-written CLS dataset.
Our proposed method has been evaluated by both automatic evaluation metrics (ROUGE, METEOR, MoverScore) and human judgment (conciseness, informativeness and fluency). Experimental results show that our proposed method can take advantage of key clues to exceed baselines and these clues working with the constructed article graph help to achieve better robustness to the article length than strong baselines.
In terms of practical implications, our research can convert articles in various languages into summaries in the language familiar to readers, providing students, officers and researchers with quick access to the non-native information. This can help them quickly filter and locate the required part in the overwhelming global information, speeding up the process of learning, working and researching.
In the future work, we would like to supplement and reorganize these clues into meta-events to eliminate redundant information and strengthen the universal applicability in minor languages by transferring the knowledge on the cross-lingual pre-training model. Soucrce: It's a boy: Nancy Gonzalez, who became pregnant with an inmate's baby, has given birth to a son . A prison guard accused of having a sexual relationship with an inmate who killed two New York City policemen has given birth to a son. Nancy Gonzalez's attorney told the New York Daily News his client delivered the baby Thursday night on Long Island. ③Gonzalez was charged in February with having an unlawful sexual relationship with inmate Ronell Wilson while working at a federal detention center in Brooklyn. At the time her attorney said Gonzalez has had a tragic life and made poor decisions. ②Wilson was convicted in the point-blank shootings of undercover officers James Nemorin and Rodney Andrews in a 2006 gun sting. He initially was sentenced to death, but his sentence was overturned on appeal. Gonzalez was excused from a court date in New York on Wednesday due to labor contractions, her attorney Anthony Ricco told the court …… ①Illegal liaison: Gonzalez, left, was arrested after police say she had an illicit affair with convicted cop killer Ronell Wilson, right, and became pregnant with his baby to save him from execution. Explanation: Gonzalez allegedly said in a recorded phone conversation that she was 'vulnerable' when she decided to start a relationship with Wilson in a bid to give him some 'hope' …… Reference: 罗内尔· 威尔逊于2007年因在2003年用枪杀死警探詹姆斯· 内莫林和罗德尼· 安德鲁斯而被定罪。29岁的南希· 冈萨雷斯被指控去年与威 尔逊有染，以帮助他避免被判死刑。  Fig. 4: An example of generated En2Zh summaries and its human-corrected reference. The sentences marked with purple boxes are the three most important sentences determined by AGC algorithm. The sentences in summaries marked with the same color box contain similar information. The generated summaries are provided with an English translation for readers to better understand.