title: WindowsError: [Error 2] The system cannot find the file specified - pyspark
body: <p>Geia!
  I am facing a FileNotFound error on pycharm when runninng this code:</p>
  
  <pre><code>import findspark
  findspark.init("C:\\Users\\user\\spark-2.3.0-bin-hadoop2.7")
  
  from pyspark import SparkConf 
  from pyspark.sql import SparkSession
  
  conf = SparkConf().setAppName('Fresh-Fish')  
  spark = SparkSession.builder.config(conf=conf).getOrCreate()
  </code></pre>
  
  <p>I've tried several proposed ways to resolve it but no good luck. I am using Windows 8.1 Pro.</p>
  
  <pre><code>Traceback (most recent call last):
    File "C:/Users/user/PycharmProjects/spark-project/spark1.py", line 8, in &lt;module&gt;
      spark = SparkSession.builder.config(conf=conf).getOrCreate()
    File "C:\Users\user\spark-2.3.0-bin-hadoop2.7\python\pyspark\sql\session.py", line 173, in getOrCreate
      sc = SparkContext.getOrCreate(sparkConf)
    File "C:\Users\user\spark-2.3.0-bin-hadoop2.7\python\pyspark\context.py", line 331, in getOrCreate
      SparkContext(conf=conf or SparkConf())
    File "C:\Users\user\spark-2.3.0-bin-hadoop2.7\python\pyspark\context.py", line 115, in __init__
      SparkContext._ensure_initialized(self, gateway=gateway, conf=conf)
    File "C:\Users\user\spark-2.3.0-bin-hadoop2.7\python\pyspark\context.py", line 280, in _ensure_initialized
      SparkContext._gateway = gateway or launch_gateway(conf)
    File "C:\Users\user\spark-2.3.0-bin-hadoop2.7\python\pyspark\java_gateway.py", line 80, in launch_gateway
      proc = Popen(command, stdin=PIPE, env=env)
    File "C:\Python27\lib\subprocess.py", line 390, in __init__
      errread, errwrite)
    File "C:\Python27\lib\subprocess.py", line 640, in _execute_child
      startupinfo)
  
  WindowsError: [Error 2] The system cannot find the file specified
  
  Process finished with exit code 1    
  </code></pre>
  
  <p>My system variables are:</p>
  
  <pre><code>SPARK_HOME = C:\Users\user\spark-2.3.0-bin-hadoop2.7
  HADOOP_HOME = C:\hadoop  #hadoop folder contains bin folder and bin folder contains winutils.exe
  PATH = C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files\Java\jre1.8.0_171\bin;C:\Python27;%SPARK_HOME%\bin;%JAVA_HOME%\bin;%HADOOP_HOME%\bin
  JAVA_HOME = C:\Program Files\Java\jdk1.8.0_161
  </code></pre>
  
  <p>I have also tried to point python.exe from a new variable on its own like:</p>
  
  <pre><code>PYSPARK_HOME = C:\Python27
  </code></pre>
  
  <p>Or even add specific paths to Pycharm Project Interpreter pointing to:</p>
  
  <pre><code>C:\Users\user\spark-2.3.0-bin-hadoop2.7\python
  "C:\Users\user\spark-2.3.0-bin-hadoop2.7\python\lib\py4j-0.10.6-src.zip"    
  </code></pre>
  
  <p>but it didn't work</p>
  
  <p>Two screenshots from Pycharm: </p>
  
  <p><a href="https://i.stack.imgur.com/eMG5x.jpg" rel="nofollow noreferrer">Project Structure Pycharm</a></p>
  
  <p><a href="https://i.stack.imgur.com/en2NX.jpg" rel="nofollow noreferrer">Project Interpreter Pycharm</a></p>
  
  <p>If you need more details I an happy to provide. I've been struggling for quite some days, any new ideas are very welcome!</p>