title: How accurate is PHP IntlBreakIterator for languages that don't separate words with spaces?
body: <p>I am building a word counter for a lot of languages. Languages dealing with white spaces as word separators are easy. However, the matter becomes a great deal more difficult when it comes to languages like Thai or Khmer that don't use white spaces.</p>
  
  <p>In search of a solution, I stumbled upon the IntlBreakIterator with whicih (among other things) one can count words. The problem is that I don't know how accurate this tool is with languages like the ones mentioned above and I cannot validate it myself because I've never learned any of these.</p>
  
  <p>Here is the code I am working with (the language in this example is Burmese):</p>
  
  <pre><code>$words = IntlBreakIterator::createWordInstance('my');
  $words-&gt;setText('ဗမာများသည်တရုတ် - တိဘက်ဘာသာစကား၏တောင်ပိုင်းဗမာဌာနခွဲတွင်ပါ ၀ င်သည်။ မြန်မာစကားသည်');
  
  $count = 0;
  foreach( $words as $offset ){
    if( IntlBreakIterator::WORD_NONE !== $words-&gt;getRuleStatus() ){
      $count++;
    }
  }
  echo $count;
  </code></pre>
  
  <p>Burmese only uses white spaces between phrases, at least that's what I've read.</p>
  
  <p>So, my question goes to eveyone speaking one of these languages (Khmer, Burmese, Thai, Lao or any other written language that does not use white spaces to separate words), how accurate are the following examples (example phrase with a following word count obtained by using IntlBreakIterator):</p>
  
  <ul>
  <li><p>Burmese: ဗမာများသည်တရုတ် - တိဘက်ဘာသာစကား၏တောင်ပိုင်းဗမာဌာနခွဲတွင်ပါ ၀ င်သည်။ မြန်မာစကားသည်; Word count: 19.</p></li>
  <li><p>Thai: หรือโคแฟกเตอร์ โปรตีนยังสามารถทำงานร่วมกันเพื่อบรรลุหน้าที่บางอย่าง และบ่อยครั้งที่โปรตีนมากกว่าหนึ่งชนิดรวมกันเพื่อสร้าง; Word count: 26.</p></li>
  <li><p>Lao: ລຳດັບກົດອາມີເນໃນປະໂລເຕອິນກຳໜົດໂດຍລຳດັບຂອງແຊນ ເຊິ່ງເຂົ້າລະຫັດໃນລະຫັດພັນທຸກຳ. ໂດຍທົ່ວໄປ ລະຫັດພັນທຸກຳປະກອບດ້ວຍກົດອາມີເນມາຕະຖານ 20 ຊະນິດ; Word count: 36.</p></li>
  <li><p>Khmer: ភាសាខ្មែរ គឺជាភា សារបស់ ប្រជាជាតិខ្មែរ ។ ភាសាសំស្ក្រឹត និងភាសាបាលីបាន​ជួយបង្កើតខេមរភាសា ព្រោះភាសាខ្មែរបានខ្ចីពាក្យច្រើនពីភាសាអស់នោះ; Word count: 23.</p></li>
  </ul>
  
  <p>A word is anything that you would find in a dictionary (in case of inflected words the corresponding dictionary form).</p>
  
  <p>In case you know a resource where such a test has already been conducted, please let me know. I didn't happen to find one.</p>