Percent slicing and rounding + Stratify
I want to incorporate 4 fold cross validation, however how do I ensure the train and validation are stratified?Codes:from datasets import load_datasetThe first 75% of datasettrain_75_25pct_ds = load_dataset(‘dataset’, split=‘train[:75%]’)train_75_25pct_dsThe last 25% of datasetvalidation_75_25pct_ds = load_dataset(‘dataset’, split=‘train[-25%:]’)validation_75_25pct_ds
2023-06-19T15:52:29Z
You can do this as follows:train_ds = load_dataset(‘dataset’, split=‘train’)
ddict  = train_ds.train_test_split(train_size=0.25, stratify_by_column="col_name")
train_ds, validation_ds = ddict["train"], ddict["test"]