🤗 Datasets uses Arrow for its local caching system. It allows datasets to be backed by an on-disk cache, which is memory-mapped for fast lookup.
This architecture allows for large datasets to be used on machines with relatively small device memory.  
For example, loading the full English Wikipedia dataset only takes a few MB of RAM:  
```python
>>> import os; import psutil; import timeit
>>> from datasets import load_dataset