TITLE: Showing that $D^{-1}- A'( A D A')^{-1} A$ is positive semi-definite
QUESTION [0 upvotes]: Suppose $\mathbf A$ is an $n \times r$ matrix and $\mathbf D$ a diagonal $r \times r$ matrix with all elements in the diagonal strictly positive. Also $\mathbf A\mathbf D\mathbf A'$ is invertible. I need to show that 
$$ \mathbf D^{-1}-\mathbf A'(\mathbf A\mathbf D\mathbf A')^{-1}\mathbf A $$
is positive semi-definite. I am not sure where to start from though. Any hints would be very welcome.

REPLY [2 votes]: Per hardmath's suggestion: Let $y = D^{-1/2}x$, and let $B = AD^{1/2}$.  We can rewrite 
$$
D^{-1} - A'(ADA')^{-1}A = \\
D^{-1/2}(I - D^{1/2}A'(AD^{1/2}D^{1/2}A')^{-1}AD^{1/2})D^{-1/2} =\\
D^{-1/2}(I - B'(BB')^{-1}B)D^{-1/2}
$$
To show that $I - B'(BB')^{-1}B$ is positive definite, it suffices to note that $B'(BB')^{-1}B$ defines the orthogonal projection onto the row-space of $B$.  In particular, $B'(BB')^{-1}B y$ produces $B'x$, where $x$ solves the "least squares" equation $BB'x = By$.  So, $I - B'(BB')^{-1}B$ defines the projection onto the orthogonal complement of the row-space.

For a more computational ending: $M = B'(BB')^{-1}B$ is symmetric with $M^2 = M$.  So, its eigenvalues are in $\{0,1\}$.  So, the eigenvalues of $I - M$ are also in $\{0,1\}$.  So, $I - M$ is symmetric with non-negative eigenvalues.
{"set_name": "stack_exchange", "score": 0, "question_id": 2132192}