Rajat Arya's picture

10

Rajat Arya

rajatarya

·

https://rajatarya.com

AI & ML interests

None yet

Recent Activity

upvoted an article 10 days ago

From Chunks to Blocks: Accelerating Uploads and Downloads on the Hub

reacted to jsulz's post with 👀 2 months ago

Doing a lot of benchmarking and visualization work, which means I'm always searching for interesting repos in terms of file types, size, branches, and overall structure. To help, I built a Space https://huggingface.co/spaces/jsulz/repo-info that lets you search for any repo and get back: - Treemap of the repository, color coded by file/directory size - Repo branches and their size - Cumulative size of different file types (e.g., the total size of all the safetensors in the repo) And because I'm interested in how this will fit in our work to leverage content-defined chunking for versioning repos on the Hub - https://huggingface.co/blog/from-files-to-chunks - everything has the number of chunks (1 chunk = 64KB) as well as the total size in bytes. Some of the treemaps are pretty cool. Attached are https://huggingface.co/black-forest-labs/FLUX.1-dev and for fun https://huggingface.co/datasets/laion/laion-audio-preview (which has nearly 10k .tar files 🤯)

reacted to jsulz's post with 👍 2 months ago

Doing a lot of benchmarking and visualization work, which means I'm always searching for interesting repos in terms of file types, size, branches, and overall structure. To help, I built a Space https://huggingface.co/spaces/jsulz/repo-info that lets you search for any repo and get back: - Treemap of the repository, color coded by file/directory size - Repo branches and their size - Cumulative size of different file types (e.g., the total size of all the safetensors in the repo) And because I'm interested in how this will fit in our work to leverage content-defined chunking for versioning repos on the Hub - https://huggingface.co/blog/from-files-to-chunks - everything has the number of chunks (1 chunk = 64KB) as well as the total size in bytes. Some of the treemaps are pretty cool. Attached are https://huggingface.co/black-forest-labs/FLUX.1-dev and for fun https://huggingface.co/datasets/laion/laion-audio-preview (which has nearly 10k .tar files 🤯)

View all activity

Organizations

models

None public yet

datasets

None public yet