Coordinate-wise Armijo's condition
  Let $z=(x,y)$ be coordinates for the product space $\mathbb{R}^{m_1}\times
\mathbb{R}^{m_2}$. Let $f:\mathbb{R}^{m_1}\times \mathbb{R}^{m_2}\rightarrow
\mathbb{R}$ be a $C^1$ function, and $\nabla f=(\partial _xf,\partial _yf)$ its
gradient. Fix $0<\alpha <1$. For a point $(x,y) \in \mathbb{R}^{m_1}\times
\mathbb{R}^{m_2}$, a number $\delta >0$ satisfies Armijo's condition at $(x,y)$
if the following inequality holds: \begin{eqnarray*} f(x-\delta \partial
_xf,y-\delta \partial _yf)-f(x,y)\leq -\alpha \delta (||\partial
_xf||^2+||\partial _yf||^2). \end{eqnarray*}
  When $f(x,y)=f_1(x)+f_2(y)$ is a coordinate-wise sum map, we propose the
following {\bf coordinate-wise} Armijo's condition. Fix again $0<\alpha <1$. A
pair of positive numbers $\delta _1,\delta _2>0$ satisfies the coordinate-wise
variant of Armijo's condition at $(x,y)$ if the following inequality holds:
\begin{eqnarray*} [f_1(x-\delta _1\nabla f_1(x))+f_2(y-\delta _2\nabla
f_2(y))]-[f_1(x)+f_2(y)]\leq -\alpha (\delta _1||\nabla f_1(x)||^2+\delta
_2||\nabla f_2(y)||^2). \end{eqnarray*}
  We then extend results in our recent previous results, on Backtracking
Gradient Descent and some variants, to this setting. We show by an example the
advantage of using coordinate-wise Armijo's condition over the usual Armijo's
condition.
