title: Different types of data in a same column in Parquet partition
body: <p>I'm getting error while reading PARQUET file from S3, and the reason comes that "final_height" column got String and Double types in a same partition. For information, there are over 20 columns in parquet file. The errors I'm getting are:</p>
  
  <pre><code>ERROR 1: Failed merging schema of file ".../part1.gz.parquet":
  
  ERROR 2: Caused by: org.apache.spark.SparkException:
  Failed to merge fields 'final_height' and 'final_height'. Failed to merge incompatible data types double and string
  
  ERROR 3: com.databricks.sql.io.FileReadException:
  Error while reading file ".../part1.gz.parquet".
  Parquet column cannot be converted. Column: [final_height], Expected: StringType, Found: DOUBLE
  
  ERROR 4: com.databricks.sql.io.FileReadException:
  Error while reading file ".../part1.gz.parquet".
  Parquet column cannot be converted. Column: [final_height], Expected: DoubleType, Found: BINARY
  
  ERROR 5: org.apache.spark.sql.execution.datasources.SchemaColumnConvertNotSupportedException
  </code></pre>
  
  <p>Found some solutions when <em>"part1.gz.parquet"</em> has <strong>String</strong> data for X column, and <em>"part2.gz.parquet"</em> has <strong>Double</strong> data in a same column. But they <strong>aren't working</strong> when different types in a <strong>same column</strong> found in a same partition.</p>
  
  <p>Tried:</p>
  
  <ul>
  <li>reading with mergeSchema and inferSchema options <strong>(got ERROR 1 &amp; 2)</strong></li>
  <li>reading with predefined manual schema with type String for "final_height" column <strong>(got ERROR 3 &amp; 5)</strong></li>
  <li>reading with predefined manual schema with type Double for "final_height" column <strong>(got ERROR 4 &amp; 5)</strong></li>
  </ul>
  
  <p>Hoping for some solution :)</p>
  
  <p>Thanks!</p>