title: when querying the same table ,spark sql return null values but hive and impaly get nomal data?
body: <p>I have a table in hive</p>
  
  <p><a href="https://i.stack.imgur.com/cleOB.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/cleOB.png" alt="enter image description here"></a></p>
  
  <p>Query the same table in two ways:</p>
  
  <ol>
  <li>hive  or impala: I get the expetcted results like this</li>
  </ol>
  
  <pre><code>0: jdbc:hive2://cdh-master3:10000/&gt; SELECT * FROM kafka_table.risk_order_user_level_info rouli WHERE rouli.month = '2019_01' AND rouli.day = '08' androuli.order_id &gt;0 limit 5;
  INFO  : OK
  +-----------------+-------------------+------------+--------------+---------------+-------------------+-----------------------+---------------+---------------------+----------------------+-------------------+--------------+------------+--+
  | rouli.order_id  | rouli.order_type  | rouli.uid  | rouli.po_id  | rouli.status  | rouli.user_level  | rouli.pre_user_level  | rouli.credit  | rouli.down_payment  | rouli.open_order_id  | rouli.createtime  | rouli.month  | rouli.day  |
  +-----------------+-------------------+------------+--------------+---------------+-------------------+-----------------------+---------------+---------------------+----------------------+-------------------+--------------+------------+--+
  | 39180235        | 2                 | 10526665   | -999         | 100           | 10                | 106                   | 27000         | 0              | -999                 | 1546887803138     | 2019_01      | 08         |
  | 39180235        | 2                 | 10526665   | -999         | 100           | 10                | 106                   | 27000         | 0              | -999                 | 1546887805302     | 2019_01      | 08         |
  | 39180235        | 2                 | 10526665   | -999         | 100           | 10                | 106                   | 27000         | 0              | -999                 | 1546887807457     | 2019_01      | 08         |
  | 39180235        | 2                 | 10526665   | -999         | 100           | 10                | 106                   | 27000         | 0              | -999                 | 1546887809610     | 2019_01      | 08         |
  | 39804907        | 2                 | 15022908   | -999         | 100           | -999              | -999                  | 0             | 85000              | -999                 | 1546887807461     | 2019_01      | 08         |
  +-----------------+-------------------+------------+--------------+---------------+-------------------+-----------------------+---------------+---------------------+----------------------+-------------------+--------------+------------+--+
  </code></pre>
  
  <ol start="2">
  <li>but usr spark  whate python or scala ,I got this,several colums are null</li>
  </ol>
  
  <pre><code>scala&gt; spark.sql("SELECT * FROM kafka_table.risk_order_user_level_info WHERE month = '2019_01' AND day = '08'  limit 5").show()
  +--------+----------+--------+-----+------+----------+--------------+-------+------------+-------------+-------------+-------+---+
  |order_id|order_type|     uid|po_id|status|user_level|pre_user_level| credit|down_payment|open_order_id|   createTime|  month|day|
  +--------+----------+--------+-----+------+----------+--------------+-------+------------+-------------+-------------+-------+---+
  |    null|      null|14057428| null|    90|      null|          null|2705000|        null|         null|1546920940672|2019_01| 08|
  |    null|      null| 5833953| null|    90|      null|          null|2197000|        null|         null|1546920941872|2019_01| 08|
  |    null|      null|10408291| null|   100|      null|          null|1386000|        null|         null|1546920941979|2019_01| 08|
  |    null|      null|  621761| null|   100|      null|          null| 100000|        null|         null|1546920942282|2019_01| 08|
  |    null|      null|10408291| null|   100|      null|          null|1386000|        null|         null|1546920942480|2019_01| 08|
  +--------+----------+--------+-----+------+----------+--------------+-------+------------+-------------+-------------+-------+---+
  </code></pre>
  
  <h3>How can I make sparksql return expected results ???</h3>
  
  <p>ps:
  I execute the flowing sql in spark and hive find different results;</p>
  
  <pre><code>SELECT * FROM kafka_table.risk_order_user_level_info rouli
  WHERE rouli.month = '2019_01' AND rouli.day = '08'
  and order_id IN (
   39906526,
   39870975,
   39832606,
   39889240,
   39836630
  )
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/1ClQ6.png" rel="nofollow noreferrer">two results</a>
  <a href="https://i.stack.imgur.com/1ClQ6.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/1ClQ6.png" alt="two results"></a></p>
  
  <p>this is where  this question posted this page hit me;</p>
  
  <p>I also check the records' number of the table  in two ways above and the counts are same</p>