title: Are coalesce + orderBy commutative in Spark?
body: <p>I want to run the following code:</p>
  <pre class="lang-py prettyprint-override"><code>df = df.coalesce(1).orderBy([&quot;my_col&quot;])
  </code></pre>
  <p>but its execution will obviously bottleneck on a single task doing all the sort work.</p>
  <p>I know it's possible to run the following:</p>
  <pre class="lang-py prettyprint-override"><code>df = df.orderBy([&quot;my_col&quot;]).coalesce(1)
  </code></pre>
  <p>however I am uncertain if Spark will maintain the ordering after the partitions are collapsed.  Does it?</p>
  <p>The second code will be preferred if so as the sort will be performed distributed and the results merged after, but I am worried it might not be properly preserved.</p>
  <p>If it <em>is</em> preserved, this would mean the two are commutative!</p>