title: Scikit-Learn Pipeline: How to Handle Preprocessing
body: <p>I'm doing some machine learning practice on Kaggle and I'm beginning to use the <code>sklearn.pipeline.Pipeline</code> class to transform my data several times and then train a model on it.</p>
  
  <p>I want to encapsulate several parts of pre-processing my data: dropping rows with 30% or more <code>NaN</code>s, dropping columns with 30% or more <code>NaN</code>s, amongst other things.</p>
  
  <p>Here's the start of my attempt at a custom <code>Transformer</code>:</p>
  
  <pre><code>class NanHandler(BaseEstimator, TransformerMixin):
      def __init__(self, target_col, row_threshold=0.7, col_threshold=0.7):
          self.target_col = target_col
          self.row_threshold = row_threshold
          self.col_threshold = col_threshold
      def transform(self, X):
          # drop rows and columns with &gt;= 30% NaN values
      def fit(self, *_):
          return self
  </code></pre>
  
  <p>However, I want to use this <code>Transformer</code> with k-fold cross-validation.  I'm concerned that if I do 3-fold cross-validation, it's unlikely (but possible) that I run into the following situation:</p>
  
  <p>Train on folds 1 and 2, test on 3</p>
  
  <p>Train on folds 2 and 3, test on 1</p>
  
  <p>Train on folds 1 and 3, test on 2</p>
  
  <p>Folds 1 and 2 combined may have over 30% <code>Nan</code>s in a specific column (call it <code>colA</code>).  So my <code>NanHandler</code> will drop this column before training.  However, folds 2 and 3 combined may have less than 30% <code>NaN</code>s and so it <em>won't</em> drop <code>colA</code>, resulting my model being trained on different columns than the first pass.</p>
  
  <p>1) How should I handle this situation?</p>
  
  <p>2) Is this also a problem if I want to drop rows that have 30% ore more NaN values (in that I'll train on a different number of rows during k-fold cross-validation)?</p>
  
  <p>Thanks!</p>