The Physical Systems Behind Optimization Algorithms
  We use differential equations based approaches to provide some {\it
\textbf{physics}} insights into analyzing the dynamics of popular optimization
algorithms in machine learning. In particular, we study gradient descent,
proximal gradient descent, coordinate gradient descent, proximal coordinate
gradient, and Newton's methods as well as their Nesterov's accelerated variants
in a unified framework motivated by a natural connection of optimization
algorithms to physical systems. Our analysis is applicable to more general
algorithms and optimization problems {\it \textbf{beyond}} convexity and strong
convexity, e.g. Polyak-\L ojasiewicz and error bound conditions (possibly
nonconvex).
