title: Python Memory error on scipy stats. Scipy linalg lstsq <> manual beta
body: <p>Not sure if this question belongs here or on crossvalidated but since the primary issue is programming language related, I am posting it here.</p>
  
  <p>Inputs:</p>
  
  <p>Y= big 2D numpy array (300000,30)</p>
  
  <p>X= 1D array (30,)</p>
  
  <p>Desired Output:</p>
  
  <p>B= 1D array (300000,) each element of which regression coefficient of regressing each row (element of length 30) of Y against X </p>
  
  <p>So <code>B[0] = scipy.stats.linregress(X,Y[0])[0]</code></p>
  
  <p>I tried this first:</p>
  
  <pre><code>B = scipy.stats.linregress(X,Y)[0] 
  </code></pre>
  
  <p>hoping that it will broadcast X according to shape of Y. Next I broadcast X myself to match the shape of Y. But on both occasions, I got this error:</p>
  
  <pre><code>    File "C:\...\scipy\stats\stats.py", line 3011, in linregress
          ssxm, ssxym, ssyxm, ssym = np.cov(x, y, bias=1).flat
      File "C:\...\numpy\lib\function_base.py", line 1766, in cov
          return (dot(X, X.T.conj()) / fact).squeeze()
    MemoryError
  </code></pre>
  
  <p>I used manual approach to calculate beta, and on Sascha's suggestion below also used scipy.linalg.lstsq as follows</p>
  
  <pre><code>B = lstsq(Y.T, X)[0] # first estimate of beta
  Y1=Y-Y.mean(1)[:,None]
  X1=X-X.mean()
  B1= np.dot(Y1,X1)/np.dot(X1,X1) # second estimate of beta
  </code></pre>
  
  <p>The two estimates of beta are very different however:</p>
  
  <pre><code>&gt;&gt;&gt; B1
  Out[10]: array([0.135623, 0.028919, -0.106278, ..., -0.467340, -0.549543, -0.498500])
  &gt;&gt;&gt; B
  Out[11]: array([0.000014, -0.000073, -0.000058, ..., 0.000002, -0.000000, 0.000001])
  </code></pre>