title: Word frequency algorithm for natural language processing
body: <p>Without getting a degree in information retrieval, I'd like to know if there exists any algorithms for counting the frequency that words occur in a given body of text.  The goal is to get a "general feel" of what people are saying over a set of textual comments.  Along the lines of <a href="http://wordle.net/" rel="noreferrer">Wordle</a>.</p>
  
  <p>What I'd like:</p>
  
  <ul>
  <li>ignore articles, pronouns, etc ('a', 'an', 'the', 'him', 'them' etc)</li>
  <li>preserve proper nouns</li>
  <li>ignore hyphenation, except for soft kind</li>
  </ul>
  
  <p>Reaching for the stars, these would be peachy:</p>
  
  <ul>
  <li>handling stemming &amp; plurals (e.g. like, likes, liked, liking match the same result)</li>
  <li>grouping of adjectives (adverbs, etc) with their subjects ("great service" as opposed to "great", "service")</li>
  </ul>
  
  <p>I've attempted some basic stuff using Wordnet but I'm just tweaking things blindly and hoping it works for my specific data.  Something more generic would be great.</p>