system
Given the following post, respond as if you're commenting on Reddit:
human
Why can GPUs do more floating operations per second than CPUs?

I've read that modern GPU can do up to ~2 Teraflops while an Intel is ~80  Gigaflops, also GPUs have hundreds of cores while a modern CPU around 8 to 4.
Why is the calculation difference between these processors so big?
Why do we not use GPUs instead of CPUs?
Is there other types of specialized processors that give an advantage for a certain type of calculation?
gpt
A GPU is basically a big multilevel bank of computation units.  It is made for performing the same instruction many times on slightly different data inputs.  A CPU typically has one or rarely a small number of computation units per core, but is optimized for serial operations.  In practice, it is usually impossible to get the full efficiency out of a GPU because you usually can't fill up all the computation units at the same time.  A GPU is great if you want to quickly do a set of identical operation on a large dataset and you can run those operations in parallel, say when rendering graphics.  However, if you want to just execute a sequential series of operation, then a GPU is unoptimized for that task and you can't use more than a fraction of its power.  An analogy would be to describe the GPU as like a sweatshop, where a massive amount of a specific product is produce by numerous but relatively unskilled labors.  In contrast, the CPU is a more professional firm, with fewer, but better trained workers, capable of only a few jobs at a time, but is better equipped for a diverse set of jobs and the better training can often allow a specific job to finish faster.