system
Given the following post, respond as if you're commenting on Reddit:
human
How does speculative execution in CPUs work?

"Speculative execution" in CPUs has been mentioned in the news recently due to the Intel bug.  What operations can be speculatively executed at the CPU level and how does the processor know it has a likelihood of a speedup without any higher level understanding of the problem it is solving?
gpt
Let’s start with pipelines: Traditionally you would expect a CPU to execute one command, then the next command, and so on, one command per clock cycle. Unfortunately this would need very low CPU speeds. To get faster speeds pipelines were introduced: During one clock cycle only parts of the command are done and the intermediate state is saved, during the next cycle this intermediate state is passed on to the next stage, meanwhile the next command starts already. A simple pipeline might just have 3-4 steps, modern CPUs have many more. There are two cases where this leads to problems:

* a command needs the result of a previous command that is still in the pipeline
* the program logic depends on the outcome of a command: “if x>5 do this else do that”

The first case can be reduced by software that makes sure these dependencies are not too fast behind each other. For the other case: A CPU could stop executing commands and wait for the condition to be known (here: what is x), but that would take time. Usually these conditions occur in loops, where the outcome is the same many times (x is smaller 99% of the time). A modern CPU has extra logic that makes a best estimate what the result will be (which code to execute next) and start working on the following code. If the prediction turns out to be wrong then the result of this predictive execution has to be discarded. Most elements of modern CPUs are logic to determine what depends on what, what can be done in parallel, how to clean up if the prediction was wrong and so on. A bug there is not too surprising.