Rajat Arya's picture

10

Rajat Arya

rajatarya

·

https://rajatarya.com

AI & ML interests

None yet

Recent Activity

upvoted an article 11 days ago

From Chunks to Blocks: Accelerating Uploads and Downloads on the Hub

reacted to jsulz's post with 👀 3 months ago

Doing a lot of benchmarking and visualization work, which means I'm always searching for interesting repos in terms of file types, size, branches, and overall structure. To help, I built a Space https://huggingface.co/spaces/jsulz/repo-info that lets you search for any repo and get back: - Treemap of the repository, color coded by file/directory size - Repo branches and their size - Cumulative size of different file types (e.g., the total size of all the safetensors in the repo) And because I'm interested in how this will fit in our work to leverage content-defined chunking for versioning repos on the Hub - https://huggingface.co/blog/from-files-to-chunks - everything has the number of chunks (1 chunk = 64KB) as well as the total size in bytes. Some of the treemaps are pretty cool. Attached are https://huggingface.co/black-forest-labs/FLUX.1-dev and for fun https://huggingface.co/datasets/laion/laion-audio-preview (which has nearly 10k .tar files 🤯)

reacted to jsulz's post with 👍 3 months ago

Doing a lot of benchmarking and visualization work, which means I'm always searching for interesting repos in terms of file types, size, branches, and overall structure. To help, I built a Space https://huggingface.co/spaces/jsulz/repo-info that lets you search for any repo and get back: - Treemap of the repository, color coded by file/directory size - Repo branches and their size - Cumulative size of different file types (e.g., the total size of all the safetensors in the repo) And because I'm interested in how this will fit in our work to leverage content-defined chunking for versioning repos on the Hub - https://huggingface.co/blog/from-files-to-chunks - everything has the number of chunks (1 chunk = 64KB) as well as the total size in bytes. Some of the treemaps are pretty cool. Attached are https://huggingface.co/black-forest-labs/FLUX.1-dev and for fun https://huggingface.co/datasets/laion/laion-audio-preview (which has nearly 10k .tar files 🤯)

View all activity

Organizations

rajatarya's activity

upvoted an article 11 days ago

Article

From Chunks to Blocks: Accelerating Uploads and Downloads on the Hub

12 days ago

• 48

upvoted 2 articles 3 months ago

Article

Introducing Observers: AI Observability with Hugging Face datasets through a lightweight SDK

By

and 1 other •

Nov 21, 2024

• 35

Article

Halo: Open Source Health Tracking with Wearables

By

•

Nov 19, 2024

• 107

upvoted an article 5 months ago

Article

Improving Parquet Dedupe on Hugging Face Hub

Oct 5, 2024

• 32

upvoted an article 6 months ago

Article

The 5 Most Under-Rated Tools on Hugging Face

Aug 22, 2024

• 86

upvoted an article 7 months ago

Article

XetHub is joining Hugging Face!

Aug 8, 2024

• 82