TITLE: Computing CDF of multivariate gaussian from eigen decomposition
QUESTION [1 upvotes]: I have a multivariate gaussian for a set of data and I'd like to compute the confidence interval for that data sample.
In hopes of finding an elegant solution, I did an eigen decomposition and transformed the data into the basis defined by the eigenvectors where the eigenvalues are the variance of that dimension. What's nice about this basis is there is 0 covariance and each axis is "independent". I compute the cdf of each gaussian independently and from that determine the confidence interval for each dimension, but now I run into the problem of how to combine them.
In some sense, if these de-correlated dimensions are independent, I ought to multiply the probabilities, but it it doesn't make sense to me to multiply the confidence intervals. I thought about averaging them, but that also doesn't make sense because if one dimension has a confidence interval of ${\sim} 1$, then averaging doesn't take into account the exponential nature of this math. The last thing I considered was an L2 norm, but that doesn't work at all.
Anyways, I'd appreciate some help figuring this out. Specifically, how can I determine the multivariate gaussian cdf from the cdf's of the the individual de-correlated gaussian dimensions? Or confidence interval instead of cdf -- basically the same information I am interested in.
Thanks
Chet

REPLY [0 votes]: Even if the components are known to be independent, it doesn't make sense just to find a confidence interval for each component separately.  That gives you a box-shaped thing, whereas you should use an ellipsoid-shaped confidence region.
I will assume you meant a confidence interval for the expected value $\mu$ of the distribution, although you should include that information in a question like this.  Suppose $X_1,\ldots,X_n\sim\text{i.i.d. } N(\mu,V)$ where $\mu\in\mathbb R^d$ and $V\in\mathbb R^{d\times d}$ is positive definite.
You can estimate $V$ by
$$
\hat V = \frac1{n-1} \sum_{k=1}^n (X_k-\bar X)(X_k-\bar X)^T \qquad\text{(This is a $d\times d$ matrix.)}
$$
where $\bar X=(X_1+\cdots+X_n)/n$, a $d\times1$ column vector.  This matrix $\hat V$ has a Wishart distribution with $n-1$ degrees of freedom.  The sample mean $\bar X$ is distributed as $N(\mu, \frac 1 n V)$.  What may not be immediately obvious is that $\bar X$ and $\hat V$ are independent.
Now the probability distribution of the random variable $(\bar X-\mu)^T \hat V^{-1} (\bar X -\mu)$ does not depend on the values of $V$ and $\mu$.  If you want a $100p\%$ confidence interval, then you in the value of $c$ for which $\Pr((\bar X-\mu)^T \hat V^{-1} (\bar X -\mu)<c)=p$.
And here I'm finding I'm a bit rusty $\ldots$ I'm thinking $(\bar X-\mu)^T \hat V^{-1} (\bar X -\mu)$ may have an $F$ distribution with $n-1$ degrees of freedom in the denominator; I'm not sure at this moment what the d.f. in the numerator is.
{"set_name": "stack_exchange", "score": 1, "question_id": 697016}