Understanding complex predictive models with Ghost Variables
  We propose a procedure for assigning a relevance measure to each explanatory
variable in a complex predictive model. We assume that we have a training set
to fit the model and a test set to check the out of sample performance. First,
the individual relevance of each variable is computed by comparing the
predictions in the test set, given by the model that includes all the variables
with those of another model in which the variable of interest is substituted by
its ghost variable, defined as the prediction of this variable by using the
rest of explanatory variables. Second, we check the joint effects among the
variables by using the eigenvalues of a relevance matrix that is the covariance
matrix of the vectors of individual effects. It is shown that in simple models,
as linear or additive models, the proposed measures are related to standard
measures of significance of the variables and in neural networks models (and in
other algorithmic prediction models) the procedure provides information about
the joint and individual effects of the variables that is not usually available
by other methods. The procedure is illustrated with simulated examples and the
analysis of a large real data set.
