title: Ran out of diskspace, now elasticsearch is broken in weird ways, how to debug?
body: <p>I had elasticsearch running, another program ran out of disk space. Now it's broken in many different ways. No clue how to debug.</p>
  
  <ol>
  <li>Creating an index pattern in kibana just hangs</li>
  <li>the logs shows <code>Unexpected error while indexing monitoring document</code> every few seconds</li>
  <li>Read queries seem to be working, all indexed are green or yellow</li>
  <li>Some indexes seem to error with <code>cluster_block_exception</code> for writing</li>
  <li>Some indexes seem to write properly</li>
  </ol>
  
  <p>What's my plan of action for figuring it out? I already did docker-compose down and up many times, as well as reboot, didn't help. Googleing <code>cluster_block_exception</code> and <code>Unexpected error while indexing monitoring document</code> doesn't seem to be related to my problem.</p>