title: General approach to count word occurrence in large number of files
body: <p>This is sort of an algorithm question. To make it clear, I'm not interested in working code but in how to approach the task generally.</p>
  
  <blockquote>
    <p>We have a server with 4 CPU's, and no databases. There are 100,000 HTML documents, stored on disk. Each document is 2MB in size. We need an efficient way to determine the count of the word "CAMERA" (case insensitive) appearing in that collection.</p>
  </blockquote>
  
  <p>My approach would be to </p>
  
  <ul>
  <li>parse the HTML document to extract only words </li>
  <li>and then sort the words, </li>
  <li>then use binary search on that collection.</li>
  </ul>
  
  <p>In other words, I would create threads to let them use all 4 CPU's to parse the HTML documents into a single, large word collection text file, then sort it, and then using binary search.</p>
  
  <p>What do you think of this?</p>