title: Filtering pyspark dataframe if text column includes words in specified list
body: <p>I've seen questions posted here that are similar to mine, but I'm still getting errors in my code when trying some accepted answers.  I have a dataframe with three columns--created _at, text, and words (which is just tokenized version of text).  See below:</p>
  
  <p><a href="https://i.stack.imgur.com/542tk.png" rel="noreferrer"><img src="https://i.stack.imgur.com/542tk.png" alt="enter image description here"></a></p>
  
  <p>Now, I have a list of companies <code>['Starbucks', 'Nvidia', 'IBM', 'Dell']</code>, and I only want to keep the rows where the text includes those words above.</p>
  
  <p>I've tried a few things, but with no success:</p>
  
  <pre><code>small_DF.filter(lambda x: any(word in x.text for word in test_list))
  </code></pre>
  
  <p>Returns : TypeError: condition should be string or Column</p>
  
  <p>I tried creating a function and using <code>foreach()</code>:</p>
  
  <pre><code>def filters(line):
     return(any(word in line for word in test_list))
  df = df.foreach(filters)
  </code></pre>
  
  <p>That turns df into 'Nonetype'</p>
  
  <p>And the last one I tried:</p>
  
  <pre><code>df = df.filter((col("text").isin(test_list))
  </code></pre>
  
  <p>This returns an empty dataframe, which is nice as I get no error, but obviously not what I want.</p>