Conservative set valued fields, automatic differentiation, stochastic
  gradient method and deep learning
  Modern problems in AI or in numerical analysis require nonsmooth approaches
with a flexible calculus. We introduce generalized derivatives called
conservative fields for which we develop a calculus and provide representation
formulas. Functions having a conservative field are called path differentiable:
convex, concave, Clarke regular and any semialgebraic Lipschitz continuous
functions are path differentiable. Using Whitney stratification techniques for
semialgebraic and definable sets, our model provides variational formulas for
nonsmooth automatic differentiation oracles, as for instance the famous
backpropagation algorithm in deep learning. Our differential model is applied
to establish the convergence in values of nonsmooth stochastic gradient methods
as they are implemented in practice.
