title: window functions( lag) implementation and the use of IsNotIn in pyspark
body: <p>Below is the T-SQL code attached. I tried to convert it to pyspark using window functions which is also attached.</p>
  
  
  
  <pre><code>  case 
               when eventaction = 'OUT' and lag(eventaction,1) over (PARTITION BY barcode order by barcode,eventdate,transactionid) &lt;&gt; 'IN'  
                    then 'TYPE4'
               else ''
        end as TYPE_FLAG,
  </code></pre>
  
  <p>Pyspark code giving error using window function <code>lag</code></p>
  
  <pre class="lang-py prettyprint-override"><code>Tgt_df = Tgt_df.withColumn(
      'TYPE_FLAG',
      F.when(
          (F.col('eventaction')=='OUT')
          &amp;(F.lag('eventaction',1).over(w).isNotIn(['IN'])),
      "TYPE4"
  ).otherwise(''))  
  </code></pre>
  
  <p>But it's not working. What to do!?</p>