Running google/pegasus-xsum model on a text column with articles for summarization
Hello Everyone!I am new to using google/pegasus model and NLP, I was able to run the following code for individual text but I have a data frame which has a full column with speeches and I am trying to run the following model on that column and save the summary as another column generated by google, something like this:df[‘summary’] = df[‘final’].apply(lambda x: lex_summarizer())Google Pegasus code:model_name = ‘google/pegasus-xsum’torch_device = ‘cuda’ if torch.cuda.is_available() else ‘cpu’tokenizer = PegasusTokenizer.from_pretrained(model_name)model =PegasusForConditionalGeneration.from_pretrained(model_name).to(torch_device)text = df[‘final’]batch = tokenizer.prepare_seq2seq_batch(text, truncation=True, padding=‘longest’,return_tensors=‘pt’)translated = model.generate(**batch)pegasus_text = tokenizer.batch_decode(translated, skip_special_tokens=True)Error: ValueError: text input must of typestr(single example),List[str](batch or single pretokenized example) orList[List[str]](batch of pretokenized examples).I would appreciate if someone could tell me how to run the above code for a text column in pandas dataframe and save the summary in another column?