How to create a new dataset from another dataset and select specific columns and the data along with the column?
How do you …from datasets import load_dataset

dataset = load_dataset("some_dataset")

# is there something like that just takes specific columns and its data ...
new_dataset = dataset.cherry_pick(["this_column", "that_column"])
2022-02-25T00:11:44Z
HelloYou can use the.remove_columnsmethod on your dataset to select the columns that youdon’twant, and this will give back a new dataset.
2022-02-25T00:43:32Z
Thanks@beneyalLet me ask this question another waySay the dataset has 35 columns. I only need a dataset with two of the columns. What’s the most efficient way to just select the two columns out of the 35?
2022-02-25T00:57:22Z
I don’t know the internals of the library, and the docs don’t mention a “cherry pick”-ish method, so the best way I see is using this:>>> dataset
Dataset({
    features: ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z'],
    num_rows: 3
})

>>> cols_to_remove = dataset.column_names
>>> cols_to_remove.remove("B")
>>> cols_to_remove.remove("Z")
>>> dataset.remove_columns(cols_to_remove)
Dataset({
    features: ['B', 'Z'],
    num_rows: 3
})
2022-08-30T18:37:46Z
For what it’s worth, I have found that operations with references to thedatasetitself, as indataset.remove_columns(cols_to_remove)withcols_to_remove = dataset.column_names, breaks the ability to cache downstreammapoperations. Better to create a variable that is a list of all features ahead of time (if you can know it) and then diff that set with the set of cols to keep.That said, I too would love adataset.select_columnsfunctionality as an addition to the API.