Understanding data of dataset_infos.json
Hi everyone,I was exploringdataset_infos.json, and I couldn’t figure out what some of thekeysrepresent in the file. Could someone please point me to a reference, which I could use as column descriptions.eg of some confusing columns: “download_size”, “dataset_size”, “size_in_bytes”, “post_processing_size” and “num_bytes”(splits).Another set of keys I couldn’t understand/interpret what they represent, were “post_processed” and “supervised_keys”.Is the structure documentation available, or is diving into the code from the commanddataset-cli testwould be the correct approach to figure this out?Example from Cifar-10 (canonical):image650×558 48.3 KB
2021-06-28T18:34:37Z
hey@dk-crazydivyou can find a description of all theDatasetInfofields in the docs:Main classes — datasets 1.8.0 documentationif something is unclear / could be improved, feel free to open a pr!
2021-06-29T04:08:28Z
Thank you. This explains it.