title: Cross-validation on XGBClassifier for multiclass classification in python
body: <p>I'm trying to perform cross-validation on a XGBClassifier for a multi-class classification problem using the following code adapted from <a href="http://www.analyticsvidhya.com/blog/2016/03/complete-guide-parameter-tuning-xgboost-with-codes-python/" rel="noreferrer">http://www.analyticsvidhya.com/blog/2016/03/complete-guide-parameter-tuning-xgboost-with-codes-python/</a></p>
  
  <pre><code>import numpy as np
  import pandas as pd
  import xgboost as xgb
  from xgboost.sklearn import  XGBClassifier
  from sklearn.preprocessing import LabelEncoder
  from sklearn import cross_validation, metrics
  from sklearn.grid_search import GridSearchCV
  
  
  def modelFit(alg, X, y, useTrainCV=True, cvFolds=5, early_stopping_rounds=50):
      if useTrainCV:
          xgbParams = alg.get_xgb_params()
          xgTrain = xgb.DMatrix(X, label=y)
          cvresult = xgb.cv(xgbParams,
                        xgTrain,
                        num_boost_round=alg.get_params()['n_estimators'],
                        nfold=cvFolds,
                        stratified=True,
                        metrics={'mlogloss'},
                        early_stopping_rounds=early_stopping_rounds,
                        seed=0,
                        callbacks=[xgb.callback.print_evaluation(show_stdv=False),                                                               xgb.callback.early_stop(3)])
  
          print cvresult
          alg.set_params(n_estimators=cvresult.shape[0])
  
      # Fit the algorithm
      alg.fit(X, y, eval_metric='mlogloss')
  
      # Predict
      dtrainPredictions = alg.predict(X)
      dtrainPredProb = alg.predict_proba(X)
  
      # Print model report:
      print "\nModel Report"
      print "Classification report: \n"
      print(classification_report(y_val, y_val_pred))
      print "Accuracy : %.4g" % metrics.accuracy_score(y, dtrainPredictions)
      print "Log Loss Score (Train): %f" % metrics.log_loss(y, dtrainPredProb)
      feat_imp = pd.Series(alg.booster().get_fscore()).sort_values(ascending=False)
      feat_imp.plot(kind='bar', title='Feature Importances')
      plt.ylabel('Feature Importance Score')
  
  
  # 1) Read training set
  print('&gt;&gt; Read training set')
  train = pd.read_csv(trainFile)
  
  # 2) Extract target attribute and convert to numeric
  print('&gt;&gt; Preprocessing')
  y_train = train['OutcomeType'].values
  le_y = LabelEncoder()
  y_train = le_y.fit_transform(y_train)
  train.drop('OutcomeType', axis=1, inplace=True)
  
  # 4) Extract features and target from training set
  X_train = train.values
  
  # 5) First classifier
  xgb = XGBClassifier(learning_rate =0.1,
                      n_estimators=1000,
                      max_depth=5,
                      min_child_weight=1,
                      gamma=0,
                      subsample=0.8,
                      colsample_bytree=0.8,
                      scale_pos_weight=1,
                      objective='multi:softprob',
                      seed=27)
  
  modelFit(xgb, X_train, y_train)
  </code></pre>
  
  <p>where <code>y_train</code> contains labels from 0 to 4. However, when I run this code I get the following error from the <code>xgb.cv</code> function <code>xgboost.core.XGBoostError: value 0for Parameter num_class should be greater equal to 1</code>. On the XGBoost doc I read that in the multiclass case xgb extrapolates the number of classes from the labels in the target vector, so I don't understand what is going on.  </p>