title: Large Dataset Polynomial Fitting Using Numpy
body: <p>I'm trying to fit a second order polynomial to raw data and output the results using Matplotlib.  There are about a million points in the data set that I'm trying to fit.  It is supposed to be simple, with many examples available around the web.  However for some reason I cannot get it right.</p>
  
  <p>I get the following warning message:</p>
  
  <blockquote>
    <p>RankWarning: Polyfit may be poorly conditioned</p>
  </blockquote>
  
  <p>This is my output:</p>
  
  <p><a href="https://i.stack.imgur.com/b3xK3.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/b3xK3.jpg" alt="Python Results"></a></p>
  
  <p>This is output using Excel:</p>
  
  <p><a href="https://i.stack.imgur.com/YTWfO.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/YTWfO.jpg" alt="Excel results"></a></p>
  
  <p>See below for my code.  What am I missing??</p>
  
  <pre><code>xData = df['X']
  yData = df['Y']
  xTitle = 'X'
  yTitle = 'Y'
  title = ''
  minX = 100
  maxX = 300
  minY = 500
  maxY = 2200
  
  title_font = {'fontname':'Arial', 'size':'30', 'color':'black', 'weight':'normal',
                'verticalalignment':'bottom'} # Bottom vertical alignment for more space
  axis_font = {'fontname':'Arial', 'size':'18'}
  
  #Poly fit
  
  # calculate polynomial
  z = np.polyfit(xData, yData, 2)
  f = np.poly1d(z)
  print(f)
  
  # calculate new x's and y's
  x_new = xData
  y_new = f(x_new)   
  
  #Plot
  plt.scatter(xData, yData,c='#002776',edgecolors='none')
  plt.plot(x_new,y_new,c='#C60C30')
  
  plt.ylim([minY,maxY])
  plt.xlim([minX,maxX])
  
  plt.xlabel(xTitle,**axis_font)
  plt.ylabel(yTitle,**axis_font)
  plt.title(title,**title_font)
  
  plt.show()      
  </code></pre>