system
Given the following post, respond as if you're commenting on Reddit:
human
Why does a computer need a cpu if gpu can perform floating point operations faster

I am reading that newish GPUs produce lots of bang for your buck for computing machine learning algorithms. What are the tasks that a cpu performs vs gpu, and why does the cpu need to be different in a way that makes it worse for AI?
gpt
The demarcation between CPUs and GPUs have been getting fuzzier as the years go by, however there is still a distinct purpose for each one, and their hardware and abilities still demonstrate that.

**CPUs are good at:**

 - Generic programming. This means it devotes its circuitry in a balanced way, so that it does a decent job at executing anything that's thrown at it. It's never going to be as good at one specific task as hardware that's devoted to doing only that, but it's good enough at any task that it can take whatever you throw at it.  
 - Linear execution. It can do many distinct steps chained together linearly, with each step being something different.  
 - Branching. CPUs are designed to be quite good at Ifs, loops, conditionals, etc.

**CPUs are not good at:**

 - Doing a lot of things all at the same time. CPUs used to be single core, which means they couldn't even do it. Nowadays, of course, consumer CPUs are 6-12 cores, but even still, that's only 6-12 things at once. If you have an operation that can be done thousands of times at once, the CPU can't really take advantage of that in a big way. The best you'll get is a 6-12x speedup over doing it linearly.

**GPUs are good at:**

 - Parallelism. A specific form of it, where the same exact operation is happening many times to distinct data. You have a list of inputs, and you want to do the same exact relatively simple operation on every single input.  
 - Floating point operations. Nearly everything a GPU does is dealing with floating point, so it devotes a lot of circuitry to being able to do it very quickly. This is combined with the first point, where it expects to be doing a large number of floating point operations all in parallel.

**GPUs are not good at:**

 - Branching. Any time you have to branch, the GPU is going to choke. The type of programming GPUs are good at involves as little conditionals as possible, and instead is just a series of the same transformations on large data sets. That's not to say that GPUs *can't* branch, but there is a very large cost to doing so.  
 - Changing what they're working on. Every time you want to execute something different on the GPU, it causes what's called a pipeline stall, which has a large cost to it.

I hope this helps clarify things. GPUs are only good if you can transform your problem into doing the same small thing to a very large data set. AI used to be the prime example of what *not* to put on the GPU, because at its core, it is a bunch of decision making, which traditionally involves conditionals. However, certain specialized AIs like neural networks have been created which are actually just a bunch of mathematical operations on incoming data. In those cases, the GPU may be able to do a better job than the CPU.