title: Convert long type column to calendarinterval type in Spark SQL
body: <p>Two queries:</p>
  
  <p>How to convert long type column having numbers in seconds to calendarinterval type having in Python
      Spark SQL?</p>
  
  <p>How to convert the below code to plain Spark SQL query:</p>
  
  <pre><code>from pyspark.sql.functions import unix_timestamp
  df2 = df.withColumn(
      "difference_duration",
      unix_timestamp("CAL_COMPLETION_TIME") - unix_timestamp("Prev_Time")
      )
  </code></pre>
  
  <p>Sample dataframe SS:
  <a href="https://i.stack.imgur.com/pe2EU.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/pe2EU.png" alt="enter image description here"></a></p>
  
  <p>Basically am trying to achieve this below PGSQL query in Spark SQL:</p>
  
  <pre><code>case 
      when t1.prev_time &lt;&gt; t1.prev_time_calc and t1."CAL_COMPLETION_TIME" - t1.prev_time &lt; interval '30 min' 
        then t1.next_time_calc - t1.prev_time_calc
      when (t1.next_time &lt;&gt; t1.next_time_calc and t1.next_time - t1."CAL_COMPLETION_TIME" &lt; interval '30 min') or (t1.next_time - t1."CAL_COMPLETION_TIME" &lt; interval '30 min')
        then t1.next_time_calc - t1."CAL_COMPLETION_TIME"
    else null
    end min_diff
  </code></pre>
  
  <p>But this part <em>t1."CAL_COMPLETION_TIME" - t1.prev_time &lt; interval '30 min'</em> is throwing following error:</p>
  
  <pre><code>AnalysisException: "cannot resolve '(t1.`CAL_COMPLETION_TIME` - t1.`prev_time`)' due to data type mismatch: '(t1.`CAL_COMPLETION_TIME` - t1.`prev_time`)' requires (numeric or calendarinterval) type, not timestamp;
  </code></pre>