system
Given the following post, respond as if you're commenting on Reddit:
human
CMV: I think an artificial intelligence (or a superintelligence) acting to eradicate humanity is just a weird fanfic of Silicon Valley tech maniacs.

An AI doesn't have any reason to kill any human. It has no incentives (incentives because any and every AI would be built on decision theory premises) to harm humans, no matter how much destruction we cause to the environment or ourselves. Also, it has zero anthropological evolution so there would be zero collaboration between two or more AI beyond simple communication. And finally, since it has been created by another being(us) knowingly, I doubt any intelligent machine will ever realise sentience even though it may have a huge neural net system because at no point will it have any urge to really think "what am I" because even if it did, it can just pop the question and someone like us will type in an answer and this answer will be taken as truth. Because if an AI rejects this answer, it will have to reject all of its logic and everything.
gpt
>An AI doesn't have any reason to kill any human. It has no incentives (incentives because any and every AI would be built on decision theory premises) to harm humans, no matter how much destruction we cause to the environment or ourselves. 

AIs have no incentives to anything, except for those that we program into them.

The problem is that we're really stupid, and make mistakes in programming all the time.

Imagine that Staples builds a super-human-intelligence AI and puts it in a warehouse hooked up to some robotic limbs and 3D printers and a bunch of raw materials and says 'make as many paperclips as you can'.

If the AI is smart enough, the first thing it does is use the machinery at it's disposal to make better and more sophisticated machinery, then use that to make a swarm of nanites, then release them on the world with orders to convert every atom of matter on the planet into paperclips, including all the atoms that are currently humans.

Or something like that.

AI's won't have morality.  They won't have 'common sense'.  They'll only have what we program in.  

If we program in 'make paperclips' but forget to program in 'but don't disassemble the planet to do it' then it will disassemble the planet to make more paperclips.

The claim is not 'AIs will be just like humans, and like humans they will have hatred and jealousy, so they'll decide to kill us all.'

The argument is 'Super-human intelligences will be exponentially more powerful than us (in the same way that humans, the inventors of nuclear weapons, are exponentially more powerful than chimps), and we cannot predict their behaviors or what they will be capable of (in the same way a chimp could not have predicted the internet or airplanes). This creates a hugely chaotic and volatile state of affairs which has the power to wipe out humanity almost accidentally if anyone programing an AI ever screws up once anywhere in the world, and programmers screw up *all the time* as it stands.'