title: tokenizer errors with nltk
body: <p>I'm very new to Python, and am trying to learn in conjunction with using nltk.</p>
  
  <p>I've been following some examples and testing things out, but it seems I am very limited in what I can do due to errors being returned by python. </p>
  
  <p>I know nltk is installed and importing fine, because this code works </p>
  
  <pre>
  from nltk.sem import chat80
  print chat80.items
  </pre>
  
  <p>However, 'from nltk.tokenizer import *' returns <pre>'File "stdin", line1. </pre>
  I get similar errors when using any sort of "TOKEN=" or I'm guessing tokenization of anything. </p>
  
  <p>I've installed python many times in the last few days, hoping a different version or better install might help.</p>
  
  <p>I'm getting this error on windows7 using activePython2.6, though I've gotten similar err
  ors with python 3.1 activePython3.1 and Python 2.6.
  as well as on Mac OSx 10.5 with Python 2.5. </p>
  
  <p>The mac is giving a bit more data with "Import Error: No module named tokenizer. </p>
  
  <p>I'm just trying some of the introductory demos to nltk online, not even trying to write my own code yet, and I'm getting more errors than successes. </p>