title: String caching. Memory optimization and re-use
body: <p>I am currently working on a very large legacy application which handles a large amount of string data gathered from various sources (IE, names, identifiers, common codes relating to the business etc). This data alone can take up to 200 meg of ram in the application process.</p>
  
  <p>A colleague of mine mentioned one possible strategy to reduce the memory footprint (as a lot of the individual strings are duplicate across the data sets), would be to "cache" the recurring strings in a dictionary and re-use them when required. So for example…</p>
  
  <pre><code>public class StringCacher()
  {
      public readonly Dictionary&lt;string, string&gt; _stringCache;
  
      public StringCacher()
      {
          _stringCache = new Dictionary&lt;string, string&gt;();
      }   
  
      public string AddOrReuse(string stringToCache)
      {
          if (_stringCache.ContainsKey(stringToCache)
              _stringCache[stringToCache] = stringToCache;
  
          return _stringCache[stringToCache];
      }
  }
  </code></pre>
  
  <p>Then to use this caching...</p>
  
  <pre><code>public IEnumerable&lt;string&gt; IncomingData()
  {
      var stringCache = new StringCacher();
  
      var dataList = new List&lt;string&gt;();
  
      // Add the data, a fair amount of the strings will be the same.
      dataList.Add(stringCache.AddOrReuse("AAAA"));
      dataList.Add(stringCache.AddOrReuse("BBBB"));
      dataList.Add(stringCache.AddOrReuse("AAAA"));
      dataList.Add(stringCache.AddOrReuse("CCCC"));
      dataList.Add(stringCache.AddOrReuse("AAAA"));
  
      return dataList;
  }
  </code></pre>
  
  <p>As strings are immutable and a lot of internal work is done by the framework to make them work in a similar way to value types i'm half thinking that this will just create a copy of each the string into the dictionary and just double the amount of memory used rather than just pass a reference to the string stored in the dictionary (which is what my colleague is assuming).</p>
  
  <p>So taking into account that this will be run on a massive set of string data... </p>
  
  <ul>
  <li><p>Is this going to save any memory, assuming that 30% of the string values will be used twice or more?</p></li>
  <li><p>Is the assumption that this will even work correct?</p></li>
  </ul>