How to batch a multiple sentences (4000+)
I’m using finbert pretrained model for text classificantion, i have passed my 4000+ sentences through the tokenizer to create my batch, but when i pass this batch to the model to get my outputs it’s takes infinite time and can’t process, when i pass 2000-3000 sentences it’s takes time but the model can process. How can i pass 4000+ sentences ?
2023-01-30T20:51:12Z
Is this for training or inference?If training, you should follow the Hugginface training examples to learn how to use batching and a data loader.If it’s for inference, you can write a small script that splits the 4000 sentences into smaller batches that each contain several dozen sentences. Iteratively feed those into your model and accumulate the outputs.