title: Large vectors and memory conservation c++
body: <p>I'm working on making a tile-map editor in C++. Right now, while a map is being edited, its attributes are stored across three vectors:</p>
  
  <pre><code>vector&lt;vector&lt;vector&lt;bool&gt;&gt;&gt; CanvasCollisionObstruction; //[collision,obstruction][map x][map y]
  vector&lt;vector&lt;vector&lt;bool&gt;&gt;&gt; CanvasZoneOverlays; //zone overlays for programmable map zones [zone type][map x][map y]
  vector&lt;vector&lt;vector&lt;canvasClip&gt;&gt;&gt; CanvasClips; //identifies which sprite occupies this tile [layer number][map x][map y]
  </code></pre>
  
  <p>In the above vectors, the 2nd and 3rd dimensions ([map x] and [map y]) refer to the actual tile coordinates on the map. These are just plain-old-square-2d maps. The type of that last vector is the following struct:</p>
  
  <pre><code>struct canvasClip
  {
      int tileset;
      int clip;
      //initialization to check if it's in-use
      canvasClip() : tileset(-1), clip(-1) {}
      bool isInitialized()
      {//a clip is only rendered if it is initialized
          return ((tileset &gt;= 0) &amp;&amp; (clip &gt;= 0));
      }
      bool operator==(const canvasClip&amp; a) const
      {//used in flood-fill to compare with target tile
          return ((tileset == a.tileset) &amp;&amp; (clip == a.clip));
      }
      bool operator!=(const canvasClip&amp; a) const
      {//used in flood-fill to compare with target tile
          return ((tileset != a.tileset) || (clip != a.clip));
      }
  };
  </code></pre>
  
  <p>For this application, I expect to eventually want to generate maps with size upwards of 50000x50000 tiles, across indefinite (but probably never more than 10) layers. There are about 12 zones total, and that number is constant.</p>
  
  <p>The map editor has some controls to change the size of the map (numeric inputs and a button). When I set the map size to a very large number, I call vector.resize() on each of those vectors, and I can watch my memory usage quickly travel upward in the task manager until my computer finally crashes.</p>
  
  <p>Can anyone offer me some advice or tips for handling very large vectors? Am I going to have to do something like compress the vector so that a single index describes a span of similar tiles? Should I store the map in a file instead of memory, and then read-back only a few chunks of it at a time, as-needed? </p>
  
  <p>How do good programmers handle this situation?</p>