TITLE: MLE of simultaneous exponential distributions
QUESTION [3 upvotes]: Given the $X_i\sim \text{exp}({\theta})$ and $Y_i\sim \text{exp}(\frac{1}{\theta})$, where $X_i$ and $Y_i$ are indpendent, with the same $\theta>0$. I have to find the MLE and its distribution.
I find the simultaneous distribution to be:
$$f_{X_i,Y_i}(x,y)=e^{-\left(\frac{x_i}{\theta }+\theta  y_i\right)}1_{(0,\infty )}(x_i)1_{(0,\infty )}(y_i),$$
and the Fisherinformation:
$$i_n(\theta)=\frac{2 n}{\theta ^2}$$
As it's convex, the maximum of the loglikelihood must be a minima, so I find the MLE:
$$\hat{\theta}=\left(\frac{X_{\bullet }}{Y_{\bullet }}\right)^{0.5},$$
where $X_{\bullet }=\sum _{i=1}^n x_i$ and $Y_{\bullet }=\sum _{i=1}^n y_i$. So the asymptotic distribution is:
$$\hat{\theta}\sim\mathcal{N}\left(\theta ,\frac{\theta^2}{2n^2}\right),$$
as $\hat{\theta}\approx\mathcal{N}\left(\theta,\frac{1}{ni_n(\theta)}\right),$
where $i_\theta$ is the Fisherinformation.
I am in doubt whether this is the right procedure.

EDIT
It is the right procedure - my professor provided the correct solution.

REPLY [3 votes]: For a sample of $n$ observations on $(X_i,Y_i)$, the likelihood function given $(x_1,y_1),\ldots,(x_n,y_n)$ is
\begin{align}
L(\theta)&=\prod_{i=1}^n e^{-\left(x_i/\theta+\theta y_i\right)}\mathbf1_{x_i>0,y_i>0}
\\&=\exp\left[{-\sum_{i=1}^n\left(\frac{x_i}{\theta}+\theta y_i\right)}\right]\mathbf1_{x_1,\ldots,x_n,y_1,\ldots,y_n>0}
\\&=\exp\left[-\frac{n\bar x}{\theta}-\theta n\bar y\right]\mathbf1_{x_{(1)},y_{(1)}>0}\quad,\,\theta>0
\end{align}
This, as you say, yields the MLE $$\hat\theta(\mathbf X,\mathbf Y)=\sqrt{\frac{\overline X}{\overline Y}}$$, where $\overline X,\overline Y$ are the sample means.
Now $X_i$'s are i.i.d $\mathsf{Exp}$ with mean $\theta$, and $Y_i$'s are i.i.d $\mathsf{Exp}$ with mean $1/\theta$, where $X_i,Y_i$ are independent.
Or, $$\frac{2}{\theta}X_i\stackrel{\text{ i.i.d }}\sim\chi^2_2\quad,\text{ independent of }\quad2\theta Y_i\stackrel{\text{ i.i.d }}\sim\chi^2_2$$
Therefore summing over the observations, we have
$$\frac{2}{\theta}n\overline X\sim\chi^2_{2n}\quad,\text{ independent of }\quad 2\theta n\overline Y\sim\chi^2_{2n}$$
Taking ratio of the statistics gives $$\frac{\overline X}{\theta^2 \overline Y}\sim F_{2n,2n}$$
So the (exact) distribution of the MLE $\hat\theta$ can be expressed as $$\frac{\hat\theta}{\theta}\stackrel{d}{=} \sqrt F\quad,\text{ where }F\sim F_{2n,2n}$$

Since by (weak) law of large numbers $\frac{1}{n}\sum\limits_{i=1}^n X_i\stackrel{P}\longrightarrow E(X_1)=\theta$ and $\frac{1}{n}\sum\limits_{i=1}^n Y_i\stackrel{P}\longrightarrow E(Y_1)=\frac{1}{\theta}$,
$$\hat\theta^2=\frac{\frac{1}{n}\sum\limits_{i=1}^n X_i}{\frac{1}{n}\sum\limits_{i=1}^n Y_i}\stackrel{P}\longrightarrow \frac{E(X_1)}{E(Y_1)}=\theta^2$$
And by continuous mapping theorem, $$\hat\theta \stackrel{P}\longrightarrow \theta$$
This further implies convergence in distribution. So as one might expect, the asymptotic distribution of $\hat\theta$ is degenerate at $\theta$. 

Asymptotic normality of MLE is a general result which holds under some  smoothness assumptions of the population density. These so called regularity conditions are sufficient conditions for verifying asymptotic normality. If these conditions hold, then indeed a non-degenerate limiting distribution of MLE is given by
$$\sqrt n\left(\hat\theta-\theta\right)\stackrel{L}\longrightarrow N\left(0,\frac1{I(\theta)}\right)\,,$$
where $I(\theta)$ is the Fisher information contained in a single observation.
For a direct proof of asymptotic normality in this particular problem, see this answer.
Alternatively, we can use the multivariate central limit theorem combined with the delta method to reach the same answer. 
By CLT we have
$$\sqrt n\left(\left(\overline X,\overline Y\right)-\left(\theta,\frac1{\theta}\right)\right)\stackrel{L}\longrightarrow N\left(\mathbf 0,\Sigma\right)\,,$$
where $$\Sigma=\begin{pmatrix}\theta^2 & 0 \\ 0 & \frac1{\theta^2}\end{pmatrix}$$
By the delta-method, the following holds for some function $g$ :
$$\sqrt n\left(g\left(\overline X,\overline Y\right)-g\left(\theta,\frac1{\theta}\right)\right)\stackrel{L}\longrightarrow N\left( 0, \nabla g\left(\theta,\frac1{\theta}\right)^T \cdot \Sigma \cdot \nabla g\left(\theta,\frac1{\theta}\right)\right)$$
Taking $g(x,y)=\sqrt{\frac xy}$ for $x,y>0$, this reduces to
$$\sqrt n\left(\hat\theta-\theta\right)\stackrel{L}\longrightarrow N\left(0,\frac{\theta^2}{2}\right)$$
{"set_name": "stack_exchange", "score": 3, "question_id": 3193796}