title: How to read a CSV file with commas within a field using pyspark?
body: <p>I have a csv file containing commas within a column value. For example,</p>
  
  <pre><code>Column1,Column2,Column3    
  123,"45,6",789  
  </code></pre>
  
  <p>The values are wrapped in double quotes when they have extra commas in the data. In the above example, the values are Column1=123, Column2=45,6 and Column3=789 But, when trying to read the data, it gives me 4 values because of extra comma in Column2 field.</p>
  
  <p>How to get the right values when reading this data in <strong>PySpark</strong>? I am using <strong>Spark 1.6.3</strong></p>
  
  <p>I am currently doing the below to create a rdd and then a data frame from rdd.</p>
  
  <pre><code>rdd = sc.textFile(input_file).map(lambda line: line.split(','))
  df = sqlContext.createDataFrame(rdd) 
  </code></pre>