title: Problem with storing and retrieving very large numbers in parquet format
body: <p>I have a weird problem when storing number with large number of digits (18 digits) as parquet and retrieved. I get different values back. Further drilling, looks like this problem happens only when the input list is a mix of None and the actual values. The case when the list is free from None values, values are fetched back as expected.</p>
  <p>I don't think its related to display problem. Tried display with unix command like cat, vi editor etc, so it doesn't look like a display problem.</p>
  <p>There are 2 sections in the code,</p>
  <ol>
  <li><p>Creates parquet from list with combination of None and large numbers. This is where the problem is. For ex: value : 235313013750949476 is changed to 235313013750949472 as seen in the output.</p>
  </li>
  <li><p>Creates parquet from list with just large numbers and no None values. It works as expected.</p>
  </li>
  </ol>
  <p><strong>Code</strong></p>
  <pre><code>import pandas as pd
  import pyarrow as pa
  import pyarrow.parquet as pq
  
  def get_row_list():
      row_list = []
  
      row_list.append(None)
      row_list.append(235313013750949476)
      row_list.append(None)
      row_list.append(135313013750949496)
      row_list.append(935313013750949406)
      row_list.append(835313013750949456)
      row_list.append(None)
      row_list.append(None)
  
      return row_list
  
  def get_row_list_with_no_none():
      row_list = []
  
      row_list.append(235313013750949476)
      row_list.append(135313013750949496)
      row_list.append(935313013750949406)
      row_list.append(835313013750949456)
  
      return row_list
  
  def create_parquet(row_list, col_list, parquet_filename):
      df = pd.DataFrame(row_list, columns=col_list)
  
      schema_field_list = [('tree_id', pa.int64())]
      pa_schema = pa.schema(schema_field_list)
  
      table = pa.Table.from_pandas(df, pa_schema)
  
      pq_writer = pq.ParquetWriter(parquet_filename,
                                   schema=pa_schema)
  
      pq_writer.write_table(table)
      pq_writer.close()
  
      print(&quot;Parquet file [%s] created&quot; % parquet_filename)
  
  def main():
      col_list = ['tree_id']
  
      # Row list without any none
      row_list = get_row_list_with_no_none()
      print (row_list)
      create_parquet(row_list, col_list, 'without_none.parquet')
  
      # Row list with none
      row_list = get_row_list()
      print (row_list)
      create_parquet(row_list, col_list, 'with_none.parquet')
  
  # ==== Main code Execution =====
  if __name__ == '__main__':
      main()
  </code></pre>
  <p><strong>[Execution]</strong></p>
  <pre><code>python test-parquet.py
  
  [235313013750949476, 135313013750949496, 935313013750949406, 835313013750949456]
  Parquet file [without_none.parquet] created
  [None, 235313013750949476, None, 135313013750949496, 935313013750949406, 835313013750949456, None, None]
  Parquet file [with_none.parquet] created
  
  </code></pre>
  <p><strong>[Lib version]</strong></p>
  <pre><code>pyarrow                  5.0.0
  pandas                   1.1.5
  
  python -v
  Python 3.6.6
  </code></pre>
  <p><strong>[Tested by consuming parquet as spark df]</strong></p>
  <pre><code>&gt;&gt;&gt; dfwithoutnone = spark.read.parquet(&quot;s3://some-bucket/without_none.parquet/&quot;)
  &gt;&gt;&gt; dfwithoutnone.count()
  4
  &gt;&gt;&gt; dfwithoutnone.printSchema()
  root
   |-- tree_id: long (nullable = true)
  
  &gt;&gt;&gt; dfwithoutnone.show(10, False)
  +------------------+                                                            
  |tree_id           |
  +------------------+
  |235313013750949476|
  |135313013750949496|
  |935313013750949406|
  |835313013750949456|
  +------------------+
  
  </code></pre>
  <pre><code>&gt;&gt;&gt; df_with_none = spark.read.parquet(&quot;s3://some-bucket/with_none.parquet/&quot;)
  &gt;&gt;&gt; df_with_none.count()
  8                                                                               
  &gt;&gt;&gt; df_with_none.printSchema()
  root
   |-- tree_id: long (nullable = true)
  
  &gt;&gt;&gt; df_with_none.printSchema()
  root
   |-- tree_id: long (nullable = true)
  
  &gt;&gt;&gt; df_with_none.show(10, False)
  +------------------+
  |tree_id           |
  +------------------+
  |null              |
  |235313013750949472|
  |null              |
  |135313013750949504|
  |935313013750949376|
  |835313013750949504|
  |null              |
  |null              |
  +------------------+
  </code></pre>
  <p>I did search in StackOverflow, could not find anything appropriate. Can you please provide some pointers?</p>
  <p>Thanks</p>