Feature Cross Search via Submodular Optimization
  In this paper, we study feature cross search as a fundamental primitive in
feature engineering. The importance of feature cross search especially for the
linear model has been known for a while, with well-known textbook examples. In
this problem, the goal is to select a small subset of features, combine them to
form a new feature (called the crossed feature) by considering their Cartesian
product, and find feature crosses to learn an \emph{accurate} model. In
particular, we study the problem of maximizing a normalized Area Under the
Curve (AUC) of the linear model trained on the crossed feature column.
  First, we show that it is not possible to provide an $n^{1/\log\log
n}$-approximation algorithm for this problem unless the exponential time
hypothesis fails. This result also rules out the possibility of solving this
problem in polynomial time unless $\mathsf{P}=\mathsf{NP}$. On the positive
side, by assuming the \naive\ assumption, we show that there exists a simple
greedy $(1-1/e)$-approximation algorithm for this problem. This result is
established by relating the AUC to the total variation of the commutator of two
probability measures and showing that the total variation of the commutator is
monotone and submodular. To show this, we relate the submodularity of this
function to the positive semi-definiteness of a corresponding kernel matrix.
Then, we use Bochner's theorem to prove the positive semi-definiteness by
showing that its inverse Fourier transform is non-negative everywhere. Our
techniques and structural results might be of independent interest.
