Suppose I do linear regression for data $y \in \mathbb{R}^n$ and design matrix $X \in \mathbb{R}^{n \times m}$, with $n \gg m$. I seek  $$ \hat{\beta} = \operatorname*{argmin}_{\beta \in \mathbb{R}^m} \| X\beta - y \|_2. $$  What are the ways to quantify uncertainty in $\hat{\beta}$? I considered bootstrap and maybe a Bayesian estimator that may give a prior with closed form expression for the variance of $\beta$. What are other approaches?  References are appreciated but a full derivation (with intuition) would be ideal.  Full derivation:
Background  Suppose we have an Ordinary Least Squares model where we have $k$ coefficients in our regression model, $$\mathbf{y}=\mathbf{X}\mathbf{\beta} + \mathbf{\epsilon}$$  where $\mathbf{\beta}$ is an $(k\times1)$ vector of coefficients, $\mathbf{X}$ is the design matrix defined by  $$\mathbf{X} = \begin{pmatrix} 1 &amp; x_{11} &amp; x_{12} &amp; \dots &amp; x_{1\;(k-1)} \\ 1 &amp; x_{21} &amp; \dots  &amp;       &amp;   \vdots \\ \vdots &amp;   &amp; \ddots &amp;       &amp;   \vdots \\ 1 &amp; x_{n1} &amp; \dots  &amp; \dots &amp;   x_{n\;(k-1)}     \end{pmatrix}$$ and the errors are IID normal,  $$\mathbf{\epsilon} \sim \mathcal{N}\left(\mathbf{0},\sigma^2 \mathbf{I}\right) \;.$$  We minimize the sum-of-squared-errors by setting our estimates for $\mathbf{\beta}$ to be $$\mathbf{\hat{\beta}}= (\mathbf{X^T X})^{-1}\mathbf{X}^T \mathbf{y}\;. $$   An unbiased estimator of $\sigma^2$ is  $$s^2 = \frac{\left\Vert \mathbf{y}-\mathbf{\hat{y}}\right\Vert ^2}{n-p}$$ where $\mathbf{\hat{y}} \equiv \mathbf{X} \mathbf{\hat{\beta}}$ ().   The covariance of $\mathbf{\hat{\beta}}$ is given by $$\operatorname{Cov}\left(\mathbf{\hat{\beta}}\right) = \sigma^2 \mathbf{C}$$ where $\mathbf{C}\equiv(\mathbf{X}^T\mathbf{X})^{-1}$ () .  Question  How can I prove that for $\hat\beta_i$, $$\frac{\hat{\beta}_i - \beta_i} {s_{\hat{\beta}_i}} \sim t_{n-k}$$ where $t_{n-k}$ is a t-distribution with $(n-k)$ degrees of freedom, and the standard error of $\hat{\beta}_i$ is estimated by $s_{\hat{\beta}_i} = s\sqrt{c_{ii}}$.    My attempts  I know that for $n$ random variables sampled from $x\sim\mathcal{N}\left(\mu, \sigma^2\right)$, you can show that  $$\frac{\bar{x}-\mu}{s/\sqrt{n}} \sim t_{n-1} $$ by rewriting the LHS as $$\frac{ \left(\frac{\bar x - \mu}{\sigma/\sqrt{n}}\right)  }   {\sqrt{s^2/\sigma^2}}$$ and realizing that the numertor is a standard normal distribution, and the denominator is square root of a Chi-square distribution with df=(n-1) and divided by (n-1) ().  And therefore it follows a t-distribution with df=(n-1) ().  I was unable to extend this proof to my question...  Any ideas?  I'm aware of , but they don't explicitly prove it, they just give a rule of thumb, saying "each predictor costs you a degree of freedom".
When ever I hear the music to indicate the Mysterious Stranger has arrived, I can never find him. It is almost like he is not in the vault.  What is a good way to catch the Mysterious Stranger? Where does he usually show up? Find all integral solutions of $ x^4 + y^4 + z^4 -w^4 = 1995 $ How to diagnose computer lockups and freezes? I built a desktop computer a couple years back with the following specs:  CPU: Intel  Yorkfield 2.5GHz 6 MB L2 Cache LGA 775 95W Quad-Core Processor BX80580Q9300 Motherboard:  LGA 775 NVIDIA nForce 680i SLI ATX Intel Motherboard Video Card: Two  GeForce 8600GT SCC 256 MB 128-bit GDDR3 PCI Express x16 SLI Supported Video Card PSU: SeaSonic S12 Energy Plus  ATX12V V2.3 / EPS12V V2.91 SLI Certified CrossFire Ready 80 PLUS Certified Active PFC Power Supply Memory: Two G.SKILL 4 GB (2 x 2 GB) 240-Pin DDR2 SDRAM DDR2 800 (PC2 6400) Dual Channel Kit Desktop Memory   Since its inception, the machine has periodically locked up, the regularity having varied over the years from once a day to once a month. Typically, lockups happen once every few days. By &quot;lockup&quot; I mean my computer just freezes. The screen locks up, I can't move the mouse. Hitting keys on my keyboard that normally turn LEDs on or off on the keyboard (such as Caps Lock) no longer turn the LEDs on or off. If there was music playing at the time of the lockup, noise keeps coming out of the speakers, but it's just the current frequency/note that plays indefinitely. There is no BSOD. When such a lockup occurs I have to do a hard reboot by either turning off the computer or hitting the reset button. I have the most recent version of the NVIDIA hardware drivers, and update them semi-regularly, but that hasn't seemed to help. I am currently using Windows 7 x64, but was previously using Windows Server 2003 x64 and having the same lockup issues. My guess is that it's somehow video driver or motherboard related, but I don't know how to go about diagnosing this problem to narrow down which of the two is the culprit.  Additional information re: cooling Regarding cooling... I've not installed any after-market cooling systems aside from two regular fans I scavenged from an older computer. The fan atop the CPU is the one that shipped with it. One of the two scavenged fans I added it located at the bottom tower of the corner, in an attempt to create some airflow from front to back. The second fan is pointed directly at the two video cards.  SpeedFan installation and readings Per studiohack's suggestion, I installed , which provided the following temperature readings:  GPU: 63C GPU: 65C System: 76C CPU: 64C AUX: 36C Core 0: 78C Core 1: 76C Core 2: 79C Core 3: 79C   Update #3: Another Lockup :-( Well, I had another lockup last night. :-( SpeedFan reported the CPU temp at 38 C when it happened, and there was no spike in temperature leading up to the freeze. One thing I notice is that the freeze seems more likely to happen if I am watching a video. In fact, of the last 5 freezes over the past month, 4 of them have been while watching a video on Flickr. Not necessarily the same video, but a video nevertheless. I don't know if this is just coincidence or if it means anything. (As an aside, each night before bedtime my 2 year old daughter sits on my lap and watches some home videos on Flickr and, in the last month, has learned the phrase, &quot;Uh oh, computer broke.&quot;)  Update #4: MemTest86 and 3DMark06 Test Results: Per suggestions in the comments, I ran the MemTest86 overnight and it cycled through the 8 GB of memory 5 times without error. I also ran the 3DMark06 test without a problem (see my scores at ). So... what now? :-) Any further suggestions on what to check? Is there some way to get a stack trace or something when the computer locks like that? Resolution I have never did figure out the particular problems, but based on the suggestions here and elsewhere, I'm presuming it was a motherboard issue. In any event, I recently upgraded my system, buying a new motherbeard, PSU, CPU, and RAM, and that new rig has been working splendidly the past several weeks. I am using the same graphic cards as in the old setup, so I think it's safe to reason that they weren't the cause of the problem. What does "All things X" mean? Why is Severus Snape's Patronus the same as Lily Potter's? Are Snape and Lily related somehow? Are Snape and Harry related? I recently spilled water on my computers keyboard, and as a result the left shift key seems to be stuck in the on position. This means that the system always boots into safe boot unless I hold down the option key. I have managed to change my password to one which is all caps and once I login I use keyremap4mac but it is still very annoying. Is there anyway which I can remap the keyboard during boot? Possibly with a startup bash script or something similiar? If that is not possible is there any way to use a script to bypass safe boot? I was planning on installing a new bootloader anyway in order to dual-boot linux, I don't know if this will help. Advice is appreciated. mbstring is missing for phpmyadmin in ubuntu 16.04 When I installed phpmyadmin in Ubuntu 16.04 LTS, it gives me an error mentioned below.     The mbstring extension is missing. Please check your PHP configuration   then I tried to install php-mbstring, it says mbstring is up to date, Can anyone help me to solve this? In a bash script sometimes you need the user to wait some seconds for a background process to finish.  I usually use for example:  sleep 10   How can I add a kind of progressbar to the script, so the user knows how long to wait?  I installed the command bar but I don't understand the manual. Howto wildcard SELECT * to query all fields of objects in APEX? I have created sObject and I need all the fields and I have many. Is there a way I can directly get all of them rather then typing each one of them. Integral vanishes on all intervals implies the function is a.e. zero I am having trouble with the following problem:      $f:\mathbb{R}\to \mathbb{R}$ is a measurable function such that for all $a$:   $$\int_{[0,a]}f\,dm=0.$$   Prove that $f=0$ for $m$ almost every $x$ (here $m$ is the Lebesgue measure).    I have no problem proving this for $f$ non-negative, or under the assumption that $f$ is integrable. But the question only assumes that $f$ is measurable and no more.  My idea was the usual thing; we look at the set of points where $f$ is positive and negative and assume one of these has measure greater than zero. Then I wanted to estimate one of these by an open set, look at the integral on the open set and show that it had to be greater than zero, a contradiction. But a key part of this attack is the assumption of the absolute continuity of the integral, which only holds in the case where $f$ is integrable.  Alternatively, if it were integrable one could simply estimate $f$ by a continuous function, where the result is quite obvious.  Ultimately we are going to show that $f$ is integrable, but it is not clear to me how to show this before showing it is zero a.e. So there must be a simpler way. Does anyone have suggestions? In a field that is the direct product of two fields, why isn't it a field? Suppose that we have the direct product of two fields, defined as: $\mathbb{F}^2 = \mathbb{F} \ \times\ \mathbb{F}$. Then, this is obviously not a field because we can take $(1,0)$, and this doesn't have a multiplicative inverse. However, I saw a mention to this today in a book, and the reasoning was:     "$\mathbb{F} \ \times\ \mathbb{F}$ has the zero-divisor $(1,0) \neq  (0,0)$, hence it is not a field".   (Lectures on Rings and Modules by Joachim Lambek, pg. 17).  I am not sure what they mean by zero-divisor here nor what $(1,0) \neq  (0,0)$ has anything to do with the direct product of fields not being a field. I always assumed there wasn't a multiplicative inverse, and that was it. Am I missing something deeper? Question statement is as in the title:     $\varphi:G\to H$ is a Lie Group Homomorphism, then $\ker \varphi$ has Lie algebra $\ker D_e \varphi$   Since $\varphi$ is a Lie group homomorphism, in particular it is continuous. As $H$ must be Hausdorff, every point set is closed, and hence $\{e\}$ is closed in $H$, which implies that $\ker\varphi = \varphi^{-1}(e)$ is closed as well.  So $N=\ker\varphi$ is an embedded Lie subgroup of $G$. Therefore, the inclusion $j:N\to G$ is a Lie Group homomorphism, and induces a Lie algebra homomorphism $D_e j :T_e N \to T_e G$. So I only need to show that $D_e j$ is in fact an isomorphism between $T_e N$ and $\ker D_e \varphi$.  But I'm stuck at this point. I know that for $X\in T_e N$, we have, for $f\in C^{\infty}(H)$,  $$ D_e\varphi (D_e j(X))(f)=D_e j (X)(f\circ \varphi)=X(f\circ \varphi\circ j)=0$$  So $D_ej(T_e N) \subset \ker D_e \varphi$.  At this point, I can either show inverse inclusion: that is, for every $X\in \ker D_e \varphi$, we can find $Y\in T_e N$ such that $D_e j (Y)=X$, or for all $f\in C^{\infty}(G)$, $Y(f\circ j)=X(f)$  Or I can somehow show that dimensions of the two spaces are the same.   But I can't proceed from here.  It'd be great if someone could help me Capillary action and conservation of energy When I dip a paper towel in a cup of water the water gets drawn up due to . How is this reconciled with conservation of energy, as it would seem on the surface that the potential energy of the system has gone up with no work being done? How to delete directories with path/names too long for normal delete Why shouldn't I care what model/make/year my bicycle is? We often get questions on bicycles.stackexchange about identifying a bicycles manufacturer, model, make, and year.  Often these questions are downvoted or closed -- usually with a comment attached to them that it's not necessary to know the exact model/make/year of a bicycle if you want to fix, repair, or ride your bike. Why should owners not care about the model/make/year of their bicycle? Especially when it comes to less expensive bikes (aka BSOs) and BMX bicycles? This is intended as a canonical question that we can point closed questions to. Also see      (generally not helpful for identifying a brand/model) How to connect to my web server? I have implemented a simple web server as a solution to a course assignment and I want to connect to this web server over the internet. I've set it up to listen on port 22881 and configured my router accordingly as the screenshot below shows    I'm able to connect to my web server locally using http://localhost:22881 and I can connect to the router's interface when I visit http://my.external.ip, but I can't connect to my web server using http://my.external.ip:22881 (I get a connection timeout). Any idea of what may be the problem?  Thanks in advance! I just installed a fresh install of 32-bit 12.04 LTS Pangolin today. When I tried to install a few applications, it was really slow. But when I download a .deb or .tar.gz file from on my web browser, it is a lot faster.  Computer specs:  HP Pavilion Slimline s7700n PC  [Memory]         873.7 MiB  [Processor]      AMD Athlon(tm) 64 X2 Dual Core Provessor 3800+ x 2  [Graphics]       GeForce 6105 LE/integrated/SSE2/3DNOW!  [Disk Size] 638.5 GB  I normally get 200-400 kbps.  This is the comparison. How much energy does lowering an object into a black hole generate? An object of mass m is slowly lowered into a black hole of mass 1000 m. Is the amount of braking energy larger than $0.6 mc^2$?   Now what if, after lowering the mass close to the event horizon, we wait the black hole to shrink to mass 100 m, by emitting Hawking radiation, and then we lower the mass some more? Is the braking energy of the second lowering process larger than $0.5 mc^2$?   I wonder can we extract more than $E=mc^2$ energy from mass m?  (In this thought experiment we use a wire and a winch, or anything that works) Capitalization of explanation of abbreviations If $AB = I$ then $BA = I$