title: How to remove less frequent words from pandas dataframe
body: <p>How do i remove words that appears less than x time for example words appear less than 3 times in pandas dataframe. I use nltk as non english word removal, however the result is not good. I assume that word apear less than 3 times as non english words. </p>
  
  <pre><code>input_text=["this is th text one tctst","this is text two asdf","this text will be remove"]
  def clean_non_english(text):
      text=" ".join(w for w in nltk.wordpunct_tokenize(text)if w.lower() in words or not w.isalpha())
      return text
  Dataset['text']=Dataset['text'].apply(lambda x:clean_non_english(x))
  </code></pre>
  
  <p>Desired output</p>
  
  <pre><code>input_text=["this is text ","this is text ","this is text"]
  </code></pre>
  
  <p>so the word appear in the list less than 3 times will be removed</p>