DNN2LR: Automatic Feature Crossing for Credit Scoring
  Credit scoring is a major application of machine learning for financial
institutions to decide whether to approve or reject a credit loan. For sake of
reliability, it is necessary for credit scoring models to be both accurate and
globally interpretable. Simple classifiers, e.g., Logistic Regression (LR), are
white-box models, but not powerful enough to model complex nonlinear
interactions among features. Fortunately, automatic feature crossing is a
promising way to find cross features to make simple classifiers to be more
accurate without heavy handcrafted feature engineering. However, credit scoring
is usually based on different aspects of users, and the data usually contains
hundreds of feature fields. This makes existing automatic feature crossing
methods not efficient for credit scoring. In this work, we find local
piece-wise interpretations in Deep Neural Networks (DNNs) of a specific feature
are usually inconsistent in different samples, which is caused by feature
interactions in the hidden layers. Accordingly, we can design an automatic
feature crossing method to find feature interactions in DNN, and use them as
cross features in LR. We give definition of the interpretation inconsistency in
DNN, based on which a novel feature crossing method for credit scoring
prediction called DNN2LR is proposed. Apparently, the final model, i.e., a LR
model empowered with cross features, generated by DNN2LR is a white-box model.
Extensive experiments have been conducted on both public and business datasets
from real-world credit scoring applications. Experimental shows that, DNN2LR
can outperform the DNN model, as well as several feature crossing methods.
Moreover, comparing with the state-of-the-art feature crossing methods, i.e.,
AutoCross, DNN2LR can accelerate the speed for feature crossing by about 10 to
40 times on datasets with large numbers of feature fields.
