title: Version control for machine learning data set with large amount of images?
body: <p>We starting to use <a href="https://dvc.org/" rel="noreferrer">dvc</a>  with git to control versioning of machine learning projects.
  For dvc remote storage  we use google cloud storage.</p>
  
  <p>Our data set is OCR data set with more than 100000 small images, total size is about 200 MB.
  Using  dvc to track this data set we encountered with next  problems:</p>
  
  <ol>
  <li>It took a lot of time to add data set for tracking.</li>
  <li>Very slow upload.</li>
  <li>Very slow download.</li>
  <li>Update/delete/add just one image in data set cause dvc to recompute
  a lot of things : hashes etc....</li>
  </ol>
  
  <p>From another way if we zipping our data set and track it as single file  dvc work fast enough.But the problem is in this way we can't track changes for particular file.</p>
  
  <p>The goal is to have version control for data set with large amount of files with next functionality.</p>
  
  <ol>
  <li>Tracking for each single file.</li>
  <li>Committing only changes and not whole data set.</li>
  <li>Fast checkout/pull</li>
  </ol>
  
  <p>Any suggestion for better solution acceptable.</p>