TITLE: Understand a matrix expression
QUESTION [0 upvotes]: Here is something I read and I didn't understand :

Let $n \in \mathbb{N}^{\ast}$ and $\Gamma$ be a $n \times n$ positive definite matrix. Let $X \in \mathbb{R}^{n}$, $X \neq 0$ and $\big(X, \varepsilon_{2}, \ldots, \varepsilon_{n}\big)$ an  orthogonal basis of $\mathbb{R}^{n}$ such that, in this basis, $\Gamma$ writes :
$$\begin{pmatrix} x_{0} & 0 & \ldots & 0 \\ 0 & & & \\ \vdots & & S & \\ 0 & & & & \end{pmatrix}$$
Then, the matrix $\Gamma$ is :
$$ \Gamma = x_{0} \frac{X X^\top}{\Vert X \Vert^{2}} + \Big(\mathrm{I}_{n} - \frac{X X^\top}{\Vert X \Vert^{2}} \Big) \begin{pmatrix} \ast & \ast & \ldots & \ast \\ \ast & & & \\ \vdots & & S & \\ \ast & & & & \end{pmatrix} \Big( \mathrm{I}_{n} - \frac{X X^\top}{\Vert X \Vert^{2}} \Big)^\top \tag{$\clubsuit$}$$
  where $\ast$ represents some real coefficients.

I don't understand why the expression $(\clubsuit)$ is true. Here is what I tried :
Let $B_{\mathrm{can}}$ be the canonical basis of $\mathbb{R}^{n}$ and $B=\big(X,\varepsilon_{2},\ldots,\varepsilon_{n}\big)$. Let $P$ be the matrix of change of basis from $B_{\mathrm{can}}$ to $B$. Then :
$$ \Gamma = P \begin{pmatrix} x_{0} & 0 & \ldots & 0 \\ 0 & & & \\ \vdots & & S & \\ 0 & & & & \end{pmatrix} P^{-1} $$
and :
$$ \frac{X X^\top}{\Vert X \Vert^{2}} = P \begin{pmatrix} 1 & 0 & \ldots & 0 \\ 0 & & & \\ \vdots & & \mathbf{0} & \\ 0 & & & & \end{pmatrix} P^{-1} \quad \mathrm{and} \quad \Big(\mathrm{I}_{n} - \frac{X X^\top}{\Vert X \Vert^{2}} \Big) = P \begin{pmatrix} 0 & 0 & \ldots & 0 \\ 0 & & & \\ \vdots & & \mathrm{I}_{n-1} & \\ 0 & & & & \end{pmatrix} P^{-1} $$
Therefore, $\Big(\mathrm{I}_{n} - \frac{X X^\top}{\Vert X \Vert^{2}} \Big) \begin{pmatrix} \ast & \ast & \ldots & \ast \\ \ast & & & \\ \vdots & & S & \\ \ast & & & & \end{pmatrix} \Big( \mathrm{I}_{n} - \frac{X X^\top}{\Vert X \Vert^{2}} \Big)^\top$ is equal to :
$$
\begin{eqnarray*}
 &  & P \begin{pmatrix} 0 & 0 & \ldots & 0 \\ 0 & & & \\ \vdots & & \mathrm{I}_{n-1} & \\ 0 & & & & \end{pmatrix} P^{-1} \begin{pmatrix} \ast & \ast & \ldots & \ast \\ \ast & & & \\ \vdots & & S & \\ \ast & & & & \end{pmatrix} P \begin{pmatrix} 0 & 0 & \ldots & 0 \\ 0 & & & \\ \vdots & & \mathrm{I}_{n-1} & \\ 0 & & & & \end{pmatrix} P^{-1} \\[3mm]
 & = & P \begin{pmatrix} 0 & 0 & \ldots & 0 \\ 0 & & & \\ \vdots & & \mathrm{I}_{n-1} & \\ 0 & & & & \end{pmatrix} \begin{pmatrix} \diamond & \diamond & \ldots & \diamond \\ \diamond & & & \\ \vdots & & \tilde{S} & \\ \diamond & & & & \end{pmatrix} \begin{pmatrix} 0 & 0 & \ldots & 0 \\ 0 & & & \\ \vdots & & \mathrm{I}_{n-1} & \\ 0 & & & & \end{pmatrix} P^{-1} \\[3mm]
& = & P \begin{pmatrix} 0 & 0 & \ldots & 0 \\ 0 & & & \\ \vdots & & \tilde{S} & \\ 0 & & & & \end{pmatrix} P^{-1}
\end{eqnarray*} $$
So, in the end I get :
$$ \Gamma = P \begin{pmatrix} x_{0} & 0 & \ldots & 0 \\ 0 & & & \\ \vdots & & \tilde{S} & \\ 0 & & & & \end{pmatrix} P^{-1} $$
which is not the expected expression. Where am I wrong ?

REPLY [2 votes]: Considering that $(X,\epsilon_2,\dots,\epsilon_n)$ forms an orthogonal basis, we have direct sum decomposition of $V=\mathbb R^n$. We write
$$V=\mathrm{span}\{X\}\oplus\mathrm{span}\{\epsilon_2,\dots,\epsilon_n\}=\mathrm{span}\{X\}\oplus\mathrm{span}\{X\}^\perp$$
It's not hard to write down the projection operator $\mathcal P_X=\frac{XX^T}{||X||^2}$ on subspace $\mathrm{span}\{X\}$ and the rejection operator $\mathcal R_X=I-\mathcal P_X=I-\frac{XX^T}{||X||^2}$. Now the decomposition becomes
$$V=\mathrm{Im}\mathcal P_X\oplus\mathrm{Im}\mathcal R_X$$
Next your question is readily followed by direct computations.
$$\begin{align}&x_0\frac{XX^T}{||X||^2}=x_0\mathcal P_XI_n\\
=&x_0\begin{pmatrix} 1 & 0 \ldots & 0 \\ 0 & & \\ \vdots & \mathbf O_{n-1} \\ 0 & \end{pmatrix}=\begin{pmatrix} x_{0} & 0 & \ldots & 0 \\ 0 & & & \\ \vdots & & \mathbf O_{n-1} \\ 0 & & & \end{pmatrix}\end{align}\\$$
$$\begin{align}&\Big(I_n-\frac{X X^T}{\Vert X \Vert^{2}} \Big) \begin{pmatrix} \ast & \ast & \ldots & \ast \\ \ast & & \\ \vdots & & S \\ \ast & & & \end{pmatrix} \Big(I_n-\frac{X X^\top}{\Vert X \Vert^{2}} \Big)^T\\
=&\mathcal R_X\begin{pmatrix} \ast & \ast & \ldots & \ast \\ \ast & & \\ \vdots & & S \\ \ast & & & \end{pmatrix}\mathcal R_X^T
=\begin{pmatrix} 0 & 0 & \ldots & 0 \\ \ast & & \\ \vdots & & S  \\ \ast & & & \end{pmatrix}\mathcal R_X^T\\
=&\begin{pmatrix} 0 & 0 & \ldots & 0 \\ 0 & &  \\ \vdots & & S \\ 0 & & &  \end{pmatrix}\end{align}$$
Hence
$$\Gamma=x_0\mathcal P_XI_n+\mathcal R_X\tilde S\mathcal R_X^T$$
{"set_name": "stack_exchange", "score": 0, "question_id": 574580}