How to get a Dataloader from a Trainer?
Hello, does anyone know how to get a train Dataloader from a Trainer? I need it to compute some metrics during training process on a training dataset but I don’t want to initialize Dataloader twice due to memory limits.Or does transformers Trainer us it’s own data storage structures? It would be great to hear some advices!
2023-07-21T16:24:18Z
Well, I found it. I`m usingon_evaluate()callback and train_dataloader parameter is contained in the**kwargs argumentof this function.
2024-03-05T19:19:27Z
Answering here as I ask myself the same question few days ago:Trainerdoes indeed provide a way to retrieve the dataloader(s), actually it even provide three methods:trainer.get_train_dataloader()trainer.get_eval_dataloader()trainer.get_test_dataloader()By utilizing these methods you can access the dataloaders without needing to initialize new ones.I hope this response (even late) proves itself helpful to someone ^^