title: Azure Databricks Missing entries when streaming to SQL Datawarehouse
body: <p>first I had following instructions, and when upload 20.000 files I got 20.000 records in the DB (Each file only holds 1 rec).</p>
  
  <pre><code>aTracking = sqlContext.read.format('csv').options(header='true', delimiter=';').schema(csvSchema).load("wasbs://" + blobContainer + "@" + blobStorage + ".blob.core.windows.net/rtT*.csv")
  
  aTracking.write \
      .option('user', dwUser) \
      .option('password', dwPass) \
      .jdbc('jdbc:sqlserver://' + dwServer + ':' + dwJdbcPort + ';database=' + dwDatabase, 'stg_tr_energy_xmlin.csv_in', mode = 'append' )
  </code></pre>
  
  <p>Then, for speed purposes I thought would be better to stream with Polybase ... Coded as ... But there I only got +- 17.000 entries.</p>
  
  <pre><code>aTracking = spark.readStream.format('csv').options(header='true', delimiter=';').schema(csvSchema).load("wasbs://" + blobContainer + "@" + blobStorage + ".blob.core.windows.net/rtT*.csv")
  
  aTracking.writeStream \
           .format("com.databricks.spark.sqldw") \
           .option("url", sqlDwUrl) \
           .option("tempDir", "wasbs://uploaddw@" + blobStorage + ".blob.core.windows.net/stream") \
           .option("forwardSparkAzureStorageCredentials", "true") \
           .option("dbTable", "stg_tr_energy_xmlin.csv_in") \
           .option("checkpointLocation", "/checkpoint") \
           .start()
  </code></pre>
  
  <p>Any suggestions what could cause this ?</p>