title: Excluding data from a pandas dataframe based on percentiles
body: <p>I am having difficulties trying to use Python to remove some data outliers prior to producing a scatterplot. I have a n by 43 dataframe imported using pandas. I have figured out how to determine the thresholds for the outliers and applied this to the dataframe such that I now have some boolean values corresponding to whether the data should be included in the scatter plot or not. I am however stuck on how to use this information to exclude the appropriate data points. </p>
  
  <p>My code so far: </p>
  
  <pre><code>def identify_outliers(self,parameters_file):
      data=pandas.read_csv(parameters_file)        #import data
      header=data.keys()                           #get header
      quantiles = data.quantile([0.25,0.75],1)     #determine thresholds for all data 
      for i in range(len(header)):
          qnt_i = quantiles[header[i]].as_matrix() #get handle to quantiles 
                                                   #for specific column of data
          #identify data points that fall outside this range   
          boolean_data=data[header[i]].between(qnt_i[0],qnt_i[1])  
          for j in range(len(boolean_data)): #attempt to use boolean values to filter
                                             #data to only include 'True' (doesn't work)
              if boolean_data[j]:
                  print data[header[i]]
  </code></pre>
  
  <p>Here is a snippet of data that is imported using pandas.read_csv</p>
  
  <pre><code>(v1).Kcat   (v1).km     (v11).k1
  1.22E-02    1.20E-02    1.72E-06
  0.0122441   1.42E-02    1.61E-06
  1.04E-02    1.01E-02    1.00E-06
  0.0136581   0.0185623   5.01158
  0.0113221   0.0221445   0.0785929
  0.506949    0.01        1.35E-06
  1.16567     0.0141031   168.078
  0.01        0.0100055   1.25E-06
  0.0351003   153.682     163.082
  0.0129821   0.0164996   0.0560866
  </code></pre>
  
  <p>Does anybody have a suggestion as to how I can filter 'data' to remove all the values which do not fall within the specified range.</p>