TITLE: Projection onto submatrix, projected onto matrix
QUESTION [2 upvotes]: Consider a matrix $X\equiv\left[\begin{matrix}Z & W\end{matrix}\right]$, with $Z^{\intercal}P_{W}Z$ and $W^{\intercal}P_{Z}W$ nonsingular, where $P_{\bullet}\equiv \bullet(\bullet^{\intercal}\bullet)^{-1}\bullet^\intercal$. 
Intuitively, I'd say that $P_{X}P_{Z}=P_{Z}$ since $\mathrm{col}Z\subset \mathrm{colX}$, yet I can't show that from straight arithmetics. Next is my work.
Denote $M_\bullet\equiv I-P_{\bullet}$. Then:
$ P_{X}P_{Z} =\left[\begin{matrix}Z & W\end{matrix}\right]\left[\begin{matrix}Z^{\intercal}Z & Z^{\intercal}W\\
W^{\intercal}Z & W^{\intercal}W
\end{matrix}\right]^{-1}\left[\begin{matrix}Z^{\intercal}\\
W^{\intercal}P_{Z}
\end{matrix}\right]
 =\left[\begin{matrix}Z & W\end{matrix}\right]\left[\begin{matrix}\left(Z^{\intercal}Z-Z^{\intercal}P_{W}Z\right)^{-1} & -\left(Z^{\intercal}Z\right)^{-1}Z^{\intercal}W\left(W^{\intercal}W-W^{\intercal}P_{Z}W\right)^{-1}\\
-\left(W^{\intercal}W\right)^{-1}W^{\intercal}Z\left(Z^{\intercal}Z-Z^{\intercal}P_{W}Z\right)^{-1} & \left(W^{\intercal}W-W^{\intercal}P_{Z}W\right)^{-1}
\end{matrix}\right]\left[\begin{matrix}Z^{\intercal}\\
W^{\intercal}P_{Z}
\end{matrix}\right]
 =Z\left(Z^{\intercal}Z-Z^{\intercal}P_{W}Z\right)^{-1}Z^{\intercal}-P_{Z}W\left(W^{\intercal}W-W^{\intercal}P_{Z}W\right)^{-1}W^{\intercal}P_{Z}-P_{W}Z\left(Z^{\intercal}Z-Z^{\intercal}P_{W}Z\right)^{-1}Z^{\intercal}+W\left(W^{\intercal}W-W^{\intercal}P_{Z}W\right)^{-1}W^{\intercal}P_{Z}
 =Z\left(Z^{\intercal}M_{W}Z\right)^{-1}Z^{\intercal}-P_{Z}W\left(W^{\intercal}M_{Z}W\right)^{-1}W^{\intercal}P_{Z}-P_{W}Z\left(Z^{\intercal}M_{W}Z\right)^{-1}Z^{\intercal}+W\left(W^{\intercal}M_{Z}W\right)^{-1}W^{\intercal}P_{Z}
 =M_{W}Z\left(Z^{\intercal}M_{W}Z\right)^{-1}Z^{\intercal}+M_{Z}W\left(W^{\intercal}M_{Z}W\right)^{-1}W^{\intercal}P_{Z}$
I am also thinking that the right-hand term in the last line is $0$, but I can't show that either. What am I missing?

REPLY [1 votes]: Note that $Z = X \begin{pmatrix} I \\ \mathbf{0} \end{pmatrix}$ where $I$ is an identity matrix of the same order as $Z$ and $\mathbf{0}$ is a $0$ matrix of the same order as $W$.
So 
$
\begin{align}
P_XP_Z &= X(X^\top X)^{-1}X^T Z(Z^\top Z)^{-1} Z^\top\\ &= X(X^\top X)^{-1}X^TX\begin{pmatrix} I \\ \mathbf{0} \end{pmatrix} (Z^\top Z)^{-1} Z^\top\\
&=X\begin{pmatrix} I \\ \mathbf{0} \end{pmatrix} (Z^\top Z)^{-1} Z^\top = Z(Z^\top Z)^{-1} Z^\top = P_Z\\
\end{align}
$
{"set_name": "stack_exchange", "score": 2, "question_id": 2574638}