Generative Trees: Adversarial and Copycat
  While Generative Adversarial Networks (GANs) achieve spectacular results on
unstructured data like images, there is still a gap on tabular data, data for
which state of the art supervised learning still favours to a large extent
decision tree (DT)-based models. This paper proposes a new path forward for the
generation of tabular data, exploiting decades-old understanding of the
supervised task's best components for DT induction, from losses (properness),
models (tree-based) to algorithms (boosting). The \textit{properness} condition
on the supervised loss -- which postulates the optimality of Bayes rule --
leads us to a variational GAN-style loss formulation which is \textit{tight}
when discriminators meet a calibration property trivially satisfied by DTs,
and, under common assumptions about the supervised loss, yields "one loss to
train against them all" for the generator: the $\chi^2$. We then introduce
tree-based generative models, \textit{generative trees} (GTs), meant to mirror
on the generative side the good properties of DTs for classifying tabular data,
with a boosting-compliant \textit{adversarial} training algorithm for GTs. We
also introduce \textit{copycat training}, in which the generator copies at run
time the underlying tree (graph) of the discriminator DT and completes it for
the hardest discriminative task, with boosting compliant convergence. We test
our algorithms on tasks including fake/real distinction, training from fake
data and missing data imputation. Each one of these tasks displays that GTs can
provide comparatively simple -- and interpretable -- contenders to
sophisticated state of the art methods for data generation (using neural
network models) or missing data imputation (relying on multiple imputation by
chained equations with complex tree-based modeling).
