title: Save Large Scipy Sparse Matrix
body: <p>I am trying to cPickle a large scipy sparse matrix for later use.  I am getting this error:</p>
  
  <pre><code>  File "tfidf_scikit.py", line 44, in &lt;module&gt;
      pickle.dump([trainID, trainX, trainY], fout, protocol=-1)
  SystemError: error return without exception set
  </code></pre>
  
  <p><code>trainX</code> is the large sparse matrix, the other two are lists 6mil elements long.  </p>
  
  <pre><code>In [1]: trainX
  Out[1]:
  &lt;6034195x755258 sparse matrix of type '&lt;type 'numpy.float64'&gt;'
      with 286674296 stored elements in Compressed Sparse Row format&gt;
  </code></pre>
  
  <p>At this point, Python RAM usage is 4.6GB and I have 16GB of RAM on my laptop.</p>
  
  <p>I think I'm running into a known memory bug for cPickle where it doesn't work with objects that are too big.  I tried <code>marshal</code> as well but I don't think it works for scipy matrices.  Can someone offer a solution and preferably an example on how to load and save this?</p>
  
  <p>Python 2.7.5</p>
  
  <p>Mac OS 10.9</p>
  
  <p>Thank you.</p>