system
Given the following post, respond as if you're commenting on Reddit:
human
ELI5: What's P < 0.05 mean and why does it make everybody feel so confident?
gpt
In statistics, you're usually doing something called *hypothesis testing*, which relies on something called a *null hypothesis*. For example, after flipping a coin a few times and getting heads each time, you might wonder if the coin is loaded. Your null hypothesis is the *opposite* of what you're trying to prove - if you're interested in showing whether the coin is loaded, your null hypothesis is "the coin is fair".

After you develop this hypothesis, you flip the coin 10 times, and get 7 heads. You want to know whether this gives you any evidence against the null hypothesis (and thus, *for* your original supposition that the coin is loaded). Obviously, not every 10 flips of a fair coin will come up with 5 heads, so the fact that you got more heads than tails doesn't necessarily tell you the coin is loaded.

A *p-value* - which is what the 'p' in your question is referring to - is the chance of getting the results you got *if the null hypothesis had been true*. In this example, it's the chance of getting 7 heads out of 10 flips if the coin were fair. It turns out it's a little less than 12%, which isn't *that* unlikely, so you don't have very good evidence to say the coin is loaded. On the other hand, the chance of flipping *nine* heads out of ten on a fair coin is less than 1%, so if you'd observed that instead, you'd have very good evidence to disprove your null hypothesis (i.e., evidence against the coin being fair).

Lower p-values, which correspond to lower chances of getting your given result if your null hypothesis (remember, this is the opposite of what you're trying to prove) had been true, provide stronger evidence. In academic writing, a p-value of less than 0.05 (that is, less than a 5% chance) is an arbitrary cutoff that is usually seen as the most generous threshold of evidence. Lower p-values, like those less than 1%, less than 0.1%, or even lower than that, provide progressively stronger evidence.