title: How to change query plan before execution (possibly turning an optimization off)?
body: <p>I have a simple spark SQL query : </p>
  
  <pre><code>SELECT x, y
  FROM t1 INNER JOIN t2 ON t1.key = t2.key
  WHERE expensiveFunction(t1.key)
  </code></pre>
  
  <p>Where <code>expensiveFunction</code> is a spark UDF (User-defined function).</p>
  
  <p>When I look at the query plan generated by spark, I see that <strong>it has two <em>filter</em> operations</strong> instead of just one: it checks not only <code>expensiveFunction(t1.key)</code>, but also <code>expensiveFunction(t2.key)</code>.</p>
  
  <p><img src="https://i.imgur.com/O0oK8Iv.png" alt="image"></p>
  
  <p>In general, this optimization is not a bad thing, because it reduces the number of records to join, and joining is an expensive operation. But in my case <code>expensiveFunction(t2.key)</code> always returns true, so I would like to remove it. </p>
  
  <p>Is there a way to change the query plan before executing a query ? Is there a way to indicate to spark that I don’t want a given optimization to be applied to my query ? </p>