TITLE: Orthogonal Projection of Approximate Matrix
QUESTION [1 upvotes]: Given a matrix $A$ its orthogonal projection is given by $P_A = (AA^T)^{-1}A^T$. Now, assume we know $A \in B_{\epsilon}(\bar{A})$, that is $||A - \bar{A}||_2 \leq \epsilon$. Here $\bar{A}$ is some constant known matrix.
Let $G \in B_{\epsilon}(\bar{A})$ be some approximation of the matrix $A$ that also satisfies $||G - \bar{A}||_2 \leq \epsilon$. Also denote $P_G = (GG^T)^{-1}G^T$ the orthogonal projection of $G$.
What can be said about $||P_A - P_G||_2$?

REPLY [0 votes]: Assuming that $\|X\|_2$ denotes the Frobenius norm (or some other submultiplicative norm) 
and that the projector was mis-typed and is actually 
$$P = A^T(AA^T)^{-1}A \;=\; A^+A$$
where $A^+$ denotes the Moore-Penrose inverse.
Then the differential of $P$ can be calculated as follows. 
$$\eqalign{
dP
 &= dA^T(AA^T)^{-1}A + A^T(AA^T)^{-1}dA - A^T(AA^T)^{-1}(A\,dA^T+dA\,A^T)(AA^T)^{-1}A \\
 &= dA^T(A^+)^T + A^+dA - P\,dA^T(A^+)^T - A^+dA\,P \\
}$$
Vectorization yields
$$\eqalign{
dp &= \operatorname{vec}(dP) \\
  &= \Big(I\otimes A^+ - P\otimes A^+ + (A^+\otimes I)K-(A^+\otimes P)K\Big)\,da \\
  &= Q\,da \\
\|dp\| &\le \|Q\|\cdot\|da\| \\
\|dP\| &\le \|Q\|\cdot\|dA\| \\
}$$
where $K$ is the commutation matrix associated with the Kronecker product $(\otimes)$. 
In terms of the variables in the original problem statement
$$\eqalign{
P_A &= P \\
  G &=   A + dA \\
P_G &= P + dP \\
}$$
{"set_name": "stack_exchange", "score": 1, "question_id": 3550969}