title: Feature-selection and prediction
body: <pre><code>from sklearn.feature_selection import RFECV
  from sklearn.metrics import accuracy_score
  from sklearn.model_selection import cross_val_predict, KFold
  from sklearn.preprocessing import StandardScaler
  from sklearn.ensemble import RandomForestClassifier
  from sklearn.pipeline import Pipeline
  from sklearn.datasets import load_iris
  </code></pre>
  
  <p>I have X and Y data. </p>
  
  <pre><code>data = load_iris()    
  X = data.data
  Y = data.target 
  </code></pre>
  
  <p>I would like to implement RFECV feature selection and prediction with k-fold validation approach.     </p>
  
  <h1>code corrected from the answer @ <a href="https://stackoverflow.com/users/3374996/vivek-kumar">https://stackoverflow.com/users/3374996/vivek-kumar</a></h1>
  
  <pre><code>clf = RandomForestClassifier()
  
  kf = KFold(n_splits=2, shuffle=True, random_state=0)  
  
  estimators = [('standardize' , StandardScaler()),
                ('clf', clf)]
  
  class Mypipeline(Pipeline):
      @property
      def coef_(self):
          return self._final_estimator.coef_
      @property
      def feature_importances_(self):
          return self._final_estimator.feature_importances_ 
  
  pipeline = Mypipeline(estimators)
  
  rfecv = RFECV(estimator=pipeline, cv=kf, scoring='accuracy', verbose=10)
  rfecv_data = rfecv.fit(X, Y)
  
  print ('no. of selected features =', rfecv_data.n_features_) 
  </code></pre>
  
  <h1>EDIT (for small remaining):</h1>
  
  <pre><code>X_new = rfecv.transform(X)
  print X_new.shape
  
  y_predicts = cross_val_predict(clf, X_new, Y, cv=kf)
  accuracy = accuracy_score(Y, y_predicts)
  print ('accuracy =', accuracy)
  </code></pre>