Summarizing Data Succinctly with the Most Informative Itemsets
  Knowledge discovery from data is an inherently iterative process. That is,
what we know about the data greatly determines our expectations, and therefore,
what results we would find interesting and/or surprising. Given new knowledge
about the data, our expectations will change. Hence, in order to avoid
redundant results, knowledge discovery algorithms ideally should follow such an
iterative updating procedure.
  With this in mind, we introduce a well-founded approach for succinctly
summarizing data with the most informative itemsets; using a probabilistic
maximum entropy model, we iteratively find the itemset that provides us the
most novel information--that is, for which the frequency in the data surprises
us the most---and in turn we update our model accordingly. As we use the
Maximum Entropy principle to obtain unbiased probabilistic models, and only
include those itemsets that are most informative with regard to the current
model, the summaries we construct are guaranteed to be both descriptive and
non-redundant.
  The algorithm that we present, called MTV, can either discover the top-$k$
most informative itemsets, or we can employ either the Bayesian Information
Criterion (BIC) or the Minimum Description Length (MDL) principle to
automatically identify the set of itemsets that together summarize the data
well. In other words, our method will `tell you what you need to know' about
the data. Importantly, it is a one-phase algorithm: rather than picking
itemsets from a user-provided candidate set, itemsets and their supports are
mined on-the-fly. To further its applicability, we provide an efficient method
to compute the maximum entropy distribution using Quick Inclusion-Exclusion.
  Experiments on our method, using synthetic, benchmark, and real data, show
that the discovered summaries are succinct, and correctly identify the key
patterns in the data.
