title: User Defined Metric in Caret Package
body: <p>I am looking to utilise the caret package with a metric that is not one of the default options. For the example below I use the Metrics package. I have read all of the relevant questions on StackOverflow as well as the guide on the caret website but still am receiving errors.</p>
  
  <p>In the example below I wish to use Mean Absolute Error.</p>
  
  <p>create a function:</p>
  
  <pre><code>maefunction&lt;-function(data, lev=NULL, model=NULL){
    require(Metrics)
    MAE&lt;-mae(data[, "obs"], data[, "pred"])
    out&lt;-c(MAE)
    out
  }
  </code></pre>
  
  <p>Now I insert the function into the trainControl</p>
  
  <pre><code>library(caret)
  GBM&lt;-train(train$result~., data=train, method="gbm", trControl=trainControl(summaryFunction=maefunction), metric=MAE)
  </code></pre>
  
  <p>I receive the following message</p>
  
  <pre><code>Error in list_to_dataframe(res, attr(.data, "split_labels"), .id, id_as_factor) : 
  Results must be all atomic, or all data frames
  In addition: Warning messages:
  1: In if (metric %in% c("Accuracy", "Kappa")) stop(paste("Metric",  :
    the condition has length &gt; 1 and only the first element will be used
  2: In if (metric == "ROC" &amp; !ctrl$classProbs) stop("train()'s use of ROC codes requires                class probabilities. See the classProbs option of trainControl()") :
    the condition has length &gt; 1 and only the first element will be used
  3: In if (!(metric %in% perfNames)) { :
    the condition has length &gt; 1 and only the first element will be used
  4: In train.default(x, y, weights = w, ...) :
    The metric "4" was not in the result set.  will be used instead.The metric "0.5" was    not in the result set.  will be used instead.
  </code></pre>