title: How to remove punctuation and numbers during TweetTokenizer step in NLP?
body: <p>I am relatively new to <code>NLP</code> so please be gentle. I 
  have a complete list of the text from Trump's tweets since taking office and I am tokenizing the text to analyze the content.</p>
  
  <p>I am using the <code>TweetTokenizer</code> from the <code>nltk</code> library in python and I'm trying to get everything tokenized except for numbers and punctuation. Problem is my code removes all the tokens except one.</p>
  
  <p>I have tried using the .<code>isalpha()</code> method but this did not work, which I thought would as should only be True for strings composed from the alphabet.</p>
  
  <pre class="lang-py prettyprint-override"><code>#Create a content from the tweets
  text= non_re['text']
  #Make all text in lowercase
  low_txt= [l.lower() for l in text]
  
  #Iteratively tokenize the tweets
  TokTweet= TweetTokenizer()
  tokens= [TokTweet.tokenize(t) for t in low_txt
          if t.isalpha()]
  </code></pre>
  
  <p>My output from this is just one token. 
  If I remove the if <code>t.isalpha()</code> statement then I get all of the tokens including numbers and punctuation, suggesting the <code>isalpha()</code> is to blame from the over-trimming.</p>
  
  <p>What I would like, is a way to get the tokens from the tweet text without punctuation and numbers. 
  Thanks for your help!</p>