Load_dataset is very slow
i have a big problem load_dataset indexing is very slow ex) x[“text”][idx] or x[“audio”][idx][‘array’] this line take 30 second what should i do? already keep_in_memory is True
2024-02-27T09:34:55Z
@cogns3942Tryx[idx]["text"]instead, it should be faster.My guess is that when you requestx["text"][idx], the entire listx["text"]has to be brought into memory which makes it slower.
2024-02-27T09:39:43Z
oh my god thank you I was using it wrong
2024-02-27T21:40:21Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.