title: Snowpipe fails to read AVRO compressed by DEFLATE exported from BigQuery
body: <p>I am trying to import data exported from BigQuery as AVRO and compressed as DEFLATE. The only encoding common to both is DEFLATE besides NONE.
  I am exporting one of publicly available datasets bigquery-public-data:covid19_open_data.covid19_open_data with 13,343,598 rows. I am using the following command to export:</p>
  <pre><code>bq extract --destination_format=AVRO --compression=DEFLATE bigquery-public-data:covid19_open_data.covid19_open_data gs://staging/covid19_open_data/avro_deflate/covid19_open_data_2_*.avro
  </code></pre>
  <p>The command creates 17 files in GCP. When I query the data in the files with command:</p>
  <pre><code>SELECT count(*) FROM @shared.data_warehouse_ext_stage/covid19_open_data/avro_deflate;
  </code></pre>
  <p>I only get a count of 684,5021 rows. To troubleshoot the error in the pipe I issue the command:</p>
  <pre><code>SELECT * from table(information_schema.copy_history(table_name=&gt;'covid19_open_data', start_time=&gt; dateadd(hours, -1, current_timestamp())));
  </code></pre>
  <p>The error reported by the pipeline is as follows:</p>
  <blockquote>
  <p>Invalid data encountered during decompression for file: 'covid19_open_data_3_000000000006.avro',compression type used: 'DEFLATE', cause: 'data error'</p>
  </blockquote>
  <p>The SQL for the File Format command is:</p>
  <pre><code>CREATE OR REPLACE FILE FORMAT monitoring_blocking.dv_avro_deflate_format TYPE = AVRO COMPRESSION = DEFLATE;
  </code></pre>
  <p>I know the problem is only related to the compression being DEFLATE. There are only two compressions for AVRO that are common for both BigQuery and Snowflake NONE and DEFLATE. I also created two pipes one file format AVRO with compression NONE and the second with CSV and GZIP. They both load data into the table. The two AVRO pipelines are a mirror of each other except for the file format. Here is snippet of the SQL for the pipe:</p>
  <blockquote>
  <p>CREATE OR REPLACE PIPE covid19_open_data_avro
  AUTO_INGEST = TRUE
  INTEGRATION = 'GCS_PUBSUB_DATA_WAREHOUSE_NOTIFICATION_INT' AS
  COPY INTO covid19_open_data(
  location_key
  ,date
  ,place_id
  ,wikidata_id
  ...
  )
  FROM
  (SELECT
  $1:location_key
  ,$1:date AS date
  ,$1:place_id AS place_id
  ,$1:wikidata_id AS wikidata_id
  ...
  FROM @shared.staging/covid19_open_data/avro_deflate)
  FILE_FORMAT = monitoring_blocking.dv_avro_deflate_format;</p>
  </blockquote>