TITLE: MLE of a two normal population of three variables likelihood
QUESTION [0 upvotes]: Given two samples: $X_1,X_2,\ldots,X_m$ from a normal population with mean $\mu_1$ and variance $\sigma^2$ and $Y_1,Y_2,\ldots,Y_n$ from a different normal population with mean $\mu_2$ but the same variance $\sigma^2$, find the maximum likelihood estimator for the common variance $\sigma^2$, $\mu_1$ and $\mu_2$.
The solution to this problem gives the likelihood function as $$L(\sigma^2)=(2\pi\sigma^2)^{\frac{-(m+n)}{2}}\exp\left(-\frac{1}{2\sigma^2}\left [\sum_{i=1}^m (x_i-\mu_1)^2+\sum_{i=1}^n (y_i-\mu_2)^2\right]\right)$$
And say $$\hat{u_1}=\bar{x}\text{ and } \hat{u_2}=\bar{y}$$
Then if $\ell$ is the log-likelihood function, then by setting
$$\frac{\partial \ell}{\partial \sigma^2}=0$$
we have
$$\hat{\sigma^2}=\frac{1}{m+n} \left(\displaystyle\sum_{i=1}^n (x_i-\mu_1)^2 + \displaystyle\sum_{j=1}^m (y_j-\mu_2)^2\right)$$
Then check
$$\frac{\partial^2 \ell}{\partial (\sigma^2)^2}|_{\hat{\sigma^2}}<0$$
We say we attainded maximum at $\hat{\sigma^2}, \hat{u_1}, \hat{u_2}$.
My question is why we don't use the $3 \times 3$ Hessian Matrix to check whether we attained the maximum value? I actually checked the determinant of this Hessian matrix, it is zero. Why we can simply say that $\hat{u_1}$ and $\hat{u_2}$ is the MLE here?

REPLY [0 votes]: We should indeed check the Hessian to be sure.
The hessian matrix evaluated those values that I computed is of the form of 
$$\begin{bmatrix} -\frac{n}{\hat{\sigma^2}} & 0 & 0\\
0 & -\frac{m}{\hat{\sigma^2}}  & 0 \\ 0 & 0 & 2\frac{(m+n)}{\hat{\sigma^2}}- 4 \sum_{i=1}^n \frac{(x_i-\hat{\mu}_1)^2}{\hat{\sigma^4}}- 4 \sum_{j=1}^m \frac{(y_j-\hat{\mu}_2)^2}{\hat{\sigma^4}}\end{bmatrix}.$$
We claim that this matrix is negative definite. 
Observe that $\hat{\sigma^2} < 2 \hat{\sigma^2}$ if $\hat{\sigma} > 0$.
Hence, $\hat{\sigma^2} < \frac4{2(m+n)}\left( \sum_{i=1}^n (x_i-\hat{\mu}_1)^2+ \sum_{j=1}^m(y_j-\hat{\mu}_2)^2\right)$
$$\frac{\hat{\sigma^4}}{\hat{\sigma^2}} < \frac4{2(m+n)}\left( \sum_{i=1}^n (x_i-\hat{\mu}_1)^2+ \sum_{j=1}^m(y_j-\hat{\mu}_2)^2\right)$$
$$\frac{2(m+n)}{\hat{\sigma^2}} < \frac4{\hat{\sigma^4}}\left( \sum_{i=1}^n (x_i-\hat{\mu}_1)^2+ \sum_{j=1}^m(y_j-\hat{\mu}_2)^2\right)$$
and hence the $(3,3)$-entry of the Hessian matrix is negative.
{"set_name": "stack_exchange", "score": 0, "question_id": 2528491}