Do download files of dataset needed?
Hello everyone,I downloaded few datasets and quickly ran out of disk space…I was wondering whether I actually still need those download files for later usage of the dataset, or I can delete it’s content to free some space?Mydatasetsmemory layout is as follows:And I specifically asks about the./downloadsdirectory, which consist of files such as:image1104×245 15.1 KBediting:I’m mostly running therun_mlm.pyscript using those downloaded datasets.In case I use the flag--dataset_cache_directoryand specify the directory of the dataset in my files (which located under my .cache directory), do I still need the content ofdownloaddirectory?Would appreciate any help on the topic
2022-10-28T14:05:16Z
Hi! Yes, it’s safe to delete thedownloadsfiles.PS: We plan to align our caching layout withhuggingface_hub/transformersto make it easier to find/delete a (specific) dataset’s “download” files.