Learning High-Order Interactions via Targeted Pattern Search
  Logistic Regression (LR) is a widely used statistical method in empirical
binary classification studies. However, real-life scenarios oftentimes share
complexities that prevent from the use of the as-is LR model, and instead
highlight the need to include high-order interactions to capture data
variability. This becomes even more challenging because of: (i) datasets
growing wider, with more and more variables; (ii) studies being typically
conducted in strongly imbalanced settings; (iii) samples going from very large
to extremely small; (iv) the need of providing both predictive models and
interpretable results. In this paper we present a novel algorithm, Learning
high-order Interactions via targeted Pattern Search (LIPS), to select
interaction terms of varying order to include in a LR model for an imbalanced
binary classification task when input data are categorical. LIPS's rationale
stems from the duality between item sets and categorical interactions. The
algorithm relies on an interaction learning step based on a well-known frequent
item set mining algorithm, and a novel dissimilarity-based interaction
selection step that allows the user to specify the number of interactions to be
included in the LR model. In addition, we particularize two variants (Scores
LIPS and Clusters LIPS), that can address even more specific needs. Through a
set of experiments we validate our algorithm and prove its wide applicability
to real-life research scenarios, showing that it outperforms a benchmark
state-of-the-art algorithm.
