TITLE: $\det(xA-B) = 0$ has three distinct solutions
QUESTION [1 upvotes]: Let $A,B \in M_{3\times 3}(\mathbb{C})$ be two complex symmetric matrices, assume $\det(xA-B) = 0$ has three distinct solutions $a,b,c$. Show that there is an invertible $P \in M_{3\times 3}(\mathbb{C})$ such that $P^TAP = I_3$, $P^TBP = diag(a,b,c).$
I have got no clue where to start.

REPLY [1 votes]: Hint: With Takagi's factorization, we see that there exists a unitary $V_1$ such that $D=  V_1^TA V_1$ is diagonal with non-negative entries.  We then have
$$
\det(x\bar V_1 D V_1^* - B) = \det[\bar V_1 (xD - V_1^TBV_1)V_1^*] = \det(V_1)^{-2} \det(xD - [V_1BV_1^T])
$$
For convenience, I'll define $B_1 = V_1BV_1^T$.
Now, consider $p(x) = \det(x D - B_1)$. Note that in order for $p(x)$ to have 3 distinct roots, $p$ must be of degree at least 3, which means that $D$ must have positive diagonal entries.  Thus, we may write
$$
\det(xD - B_1) = \det[D^{1/2}(xI - D^{-1/2}B_1 D^{-1/2})D^{1/2}] = 
\det(D) \det(xI - [D^{-1/2}B_1 D^{-1/2}])
$$
Let $B_2 = D^{-1/2}B_1 D^{-1/2}$. It now suffices to consider the problem with $A = I$ and $B = B_2$ an arbitrary complex-symmetric matrix.
{"set_name": "stack_exchange", "score": 1, "question_id": 3092662}