system
Given the following post, respond as if you're commenting on Reddit:
human
I believe that experimenting with Strong AI is unethical. CMV

Fairly recently, Google bought a company called DeepMind which specializes in AI and machine learning. While this isn't quite the same as  'Strong AI', it got me thinking about the subject. The whole field of machine learning is fascinating to me, and so are the mechanics of the human mind. I don't know how far attempts have gotten toward achieving something that resembles a conscious being, but there is no doubt that this is an open problem that people are actively working on. 

In my mind, what gives a human life value is it's ability to think and feel. This is why I don't have any problem pulling the plug on people who are brain dead even though they are still alive in a biological sense, whereas I would have a problem killing a conscious being.

In the pursuit of making a Strong AI, undoubtably many prototypes would be made and tested and ultimately halted. On early prototypes I don't really see this as a problem. It's no worse than having any other process on my computer finish. It's pretty ridiculous to think that running `kill -9` on an app would be morally equivalent to killing a person. But as the state of the art advances, and machines do start becoming more like sentient beings, deactivating the program starts becoming more like killing them. At some point, when you shut off a running AI, you'd be permanently halting the thought process of a conscious entity. And even though it would just be a process on a computer, that still feels to me like killing someone. 

On the other hand, a Strong AI wouldn't necessarily be like a human in many ways. Maybe it lacks the will to live? Does that make a difference though? Is it okay to create a thinking being that is totally fine with being killed? Maybe Strong AI is too different to human life for humanity to even be a meaningful analog? I don't know.

Like I said, I find the whole thing fascinating and the development of actual intelligent computers to be supremely cool, but I can't help shake the feeling that it's necessary to leave a trail of digital corpses in along the path of progress, and that seems a little dark to me. I'd love for someone to convince me otherwise.
gpt
I don't think it makes sense to apply current human laws and morals to something as new and unfamiliar to us as AI. What you talk about isn't going to develop overnight. Getting there is going to be a very long, very slow process, and along the way how we perceive this AI is going to evolve as well. Maybe by that time the concepts of life and death as we know them cease to exist?

Even otherwise, science and ethics haven't exactly walked hand in hand in the past, and there is a fine line that is often crossed in the name of progress. Are the means justified if the goals are noble? I'd say sometimes they are. Lots of people consider stem-cell research unethical, but look at its potential to save countless lives and change the face of medicine for the better. Having "true" artificial intelligence is definitely going to be one of the biggest technological advances this race has ever achieved.