title: Spark get one row from every parquet file
body: <p>I have large amount of data (12 GB, compressed) partitioned in parquet files (about 2000 files, every partition has 10 files).</p>
  <p>I want to get one row from every parquet file and do it in parallel, not just on the driver. I do not care which row I get but it is important to get a record from every file.</p>
  <p>I have come up with this code but it is very slow.</p>
  <pre class="lang-scala prettyprint-override"><code>val df = spark.read.parquet(&quot;/path/to/parquet/&quot;)
      .groupBy(input_file_name().as(&quot;fileUri&quot;)).agg(
        first(&quot;myColumn&quot;).as(&quot;myColumnValue&quot;)
      )
      .collect()
  </code></pre>