Non shuffle training
Hi there,In order to debug something I need to make data non-shuffle. Can you please tell me how to turn off the shuffle?I am usingfrom transformers import Trainerfor training andfrom datasets import load_datasetfor data loading with default arguments.
2021-06-21T13:08:34Z
There is no option to do this natively in the Trainer, you can either make a source install and change the line that creates the training dataloader, or subclassTrainerand override theget_train_dataloadermethod.
2023-09-03T05:17:36Z
Refer to thread here:How to ensure the dataset is shuffled for each epoch using Trainer and Datasets? - #3 by lhoestq
2023-09-03T05:19:02Z
As non-shuffling is a minor demand from users, the Trainer class doesn’t provide this option to avoid careless users making mistakes.
2024-07-05T04:35:02Z
where should i change to make it non-shuffled？
2024-08-26T09:31:01Z
For the reference, the way to go was to switch train sampler. E.g. this worked for me for SFTTrainer, should be the same for vanilla Trainer. You inherit from initial class and then use your new redefined one.from torch.utils.data import SequentialSampler
from trl import SFTTrainer

class SFTTrainer2(SFTTrainer):
    def _get_train_sampler(self):
        return SequentialSampler(self.train_dataset)
2024-08-26T13:30:38Z
Hi,It would look like so:from transformers import Trainer
from torch.utils.data import DataLoader

class MyCustomTrainer(Trainer):
    def get_train_dataloader(self) -> DataLoader:
       train_dataset = self.train_dataset
       return DataLoader(train_dataset, shuffle=False, batch_size=...)Note that this is just a minimal version for demo purposes, refer to thesource codeif you want to tweak it more.