Quantize and Optimize summarization model (Seq2SeqLM)
I want to Quantize and Optimize thefacebook/bart-large-cnn, so i can get faster summarization results while reducing the size of the model, i have looked atOptimum with Huggingface, but they dont support summarization, and when you try to use it with seq2seq models it actually makes the model larger. I want to know if there any other ways i could optimize or quantize thefacebook/bart-large-cnn, for better speeds, while still getting similar accuracy