Learning and Optimization with Submodular Functions
  In many naturally occurring optimization problems one needs to ensure that
the definition of the optimization problem lends itself to solutions that are
tractable to compute. In cases where exact solutions cannot be computed
tractably, it is beneficial to have strong guarantees on the tractable
approximate solutions. In order operate under these criterion most optimization
problems are cast under the umbrella of convexity or submodularity. In this
report we will study design and optimization over a common class of functions
called submodular functions. Set functions, and specifically submodular set
functions, characterize a wide variety of naturally occurring optimization
problems, and the property of submodularity of set functions has deep
theoretical consequences with wide ranging applications. Informally, the
property of submodularity of set functions concerns the intuitive "principle of
diminishing returns. This property states that adding an element to a smaller
set has more value than adding it to a larger set. Common examples of
submodular monotone functions are entropies, concave functions of cardinality,
and matroid rank functions; non-monotone examples include graph cuts, network
flows, and mutual information.
  In this paper we will review the formal definition of submodularity; the
optimization of submodular functions, both maximization and minimization; and
finally discuss some applications in relation to learning and reasoning using
submodular functions.
