title: "name 'word_tokenize' is not defined" in python word count frequency
body: <p>i am trying to figure out the word frequency from the specific word column.</p>
  
  <p>I want to remove the stopwords from the dictionary.</p>
  
  <p>Here's the code : </p>
  
  <h1>Code</h1>
  
  <pre><code>import unicodecsv as csv
  import nltk
  import pandas as pd
  import chardet
  
  from nltk.corpus import stopwords
  from nltk.tokenize import sent_tokenize
  
  with open('data.csv','rb') as f:
      result = chardet.detect(f.read())
  
  file_band = file[file['article'].str.contains("first time")]
  file.loc[:,'extracted'] = file_band['article']
  
  top_N = 200
  
  a = file_band['extracted'].str.lower().replace(r'\|', ' ').str.cat(sep=' ')
  words = nltk.tokenize.word_tokenize(a)
  word_dist = nltk.FreqDist(words)
  print (word_dist)
  
  stop_words = set(stopwords.words('english'))
  word_tokens = word_tokenize(word_dist)
  
  filtered_sentence = [w for w in word_tokens if not w in stop_words]
  filtered_sentence = []
  
  for w in word_tokens:
      if w not in stop_words:
          filtered_sentence.append(w)
  
  print(word_tokens)
  print(filtered_sentence)
  </code></pre>
  
  <h1>Error</h1>
  
  <p>The problem error is:</p>
  
  <blockquote>
    <p>NameError                                 Traceback (most recent call
    last)  in ()
         27 #filter words
         28 stop_words = set(stopwords.words('english'))
    ---> 29 word_tokens = word_tokenize(word_dist)
         30 
         31 filtered_sentence = [w for w in word_tokens if not w in stop_words]</p>
    
    <p>NameError: name 'word_tokenize' is not defined</p>
  </blockquote>