title: Using a sparse matrix versus numpy array
body: <p>I am creating some numpy arrays with word counts in Python: rows are documents, columns are counts for word X. If I have a lot of zero counts, people suggest using sparse matrices when processing these further, e.g. in a classifier. When feeding a numpy array versus a sparse matrix into the Scikit <a href="http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression.fit" rel="noreferrer">logistic regression classifier</a>, it did not seem to make much of a difference, however. So I was wondering about three things:</p>
  
  <ul>
  <li><p><a href="https://en.wikipedia.org/wiki/Sparse_matrix" rel="noreferrer">Wikipedia</a> says </p>
  
  <blockquote>
    <p>a sparse matrix is a matrix in which most of the elements are zero</p>
  </blockquote>
  
  <p>Is that an appropriate way to determine when to use a sparse matrix
  format - as soon as > 50 % of the values are zero? Or does it make
  sense to use just in case?</p></li>
  <li>How much does a sparse matrix help performance in a task like mine,
  especially compared to a numpy array or a standard list?</li>
  <li>So far, I collect my data into a numpy array, then convert into the
  csr_matrix in <a href="http://docs.scipy.org/doc/scipy/reference/sparse.html" rel="noreferrer">Scipy</a>. Is that the right way to do it? I could not
  figure out how to build a sparse matrix from the ground up, and that
  might be impossible.</li>
  </ul>
  
  <p>Any help is much appreciated!</p>