Complexity Measures and Concept Learning
  The nature of concept learning is a core question in cognitive science.
Theories must account for the relative difficulty of acquiring different
concepts by supervised learners. For a canonical set of six category types, two
distinct orderings of classification difficulty have been found. One ordering,
which we call paradigm-specific, occurs when adult human learners classify
objects with easily distinguishable characteristics such as size, shape, and
shading. The general order occurs in all other known cases: when adult humans
classify objects with characteristics that are not readily distinguished (e.g.,
brightness, saturation, hue); for children and monkeys; and when categorization
difficulty is extrapolated from errors in identification learning. The
paradigm-specific order was found to be predictable mathematically by measuring
the logical complexity of tasks, i.e., how concisely the solution can be
represented by logical rules.
  However, logical complexity explains only the paradigm-specific order but not
the general order. Here we propose a new difficulty measurement, information
complexity, that calculates the amount of uncertainty remaining when a subset
of the dimensions are specified. This measurement is based on Shannon entropy.
We show that, when the metric extracts minimal uncertainties, this new
measurement predicts the paradigm-specific order for the canonical six category
types, and when the metric extracts average uncertainties, this new measurement
predicts the general order. Moreover, for learning category types beyond the
canonical six, we find that the minimal-uncertainty formulation correctly
predicts the paradigm-specific order as well or better than existing metrics
(Boolean complexity and GIST) in most cases.
