title: Blocking in scala future?
body: <p>I have a very long sequence of strings which individually need to be processed by some processing function and then collected as another sequence object. The problem seems to be ideally suited to a fork/join type attack.</p>
  
  <p>The function is a member of a class that is quite expensive to instantiate. But instantiating and sharing a single class object among the futures seemed to cause problems, so I instantiate 4 times the number of processors available and then split them up among the futures.</p>
  
  <pre><code>// instantiate processing class objects
  val processors = 1 to SomeNumber map (x=&gt; new MyProcessor)
  val processorstream = Stream.continually(processors).flatten
  // the input strings
  val input: Seq[String] = some sequence of strings
  val splitinput = input.grouped(some large number).toStream
  // create futures
  val mytask = splitinput.zip(processorstream).collect {
      case (subseq of strings, processor) =&gt; future {
          map elements of subsequence of strings with processor}}
  </code></pre>
  
  <p>Then I collect the output like so</p>
  
  <pre><code>val result = mytask.map(x =&gt; x.apply()).reduce(_++_) // or some appropriate concatenation operator
  </code></pre>
  
  <p>My problem is that this does not give me full cpu utilization even though I have 8 cores. It only utilizes one core.</p>
  
  <p>To investigate, an alternative I tried was </p>
  
  <pre><code>val input: Seq[String] = some sequence of strings
  // no stage where I split the input into subsequences
  val mytask = input.zip(processorstream).collect {
      case (string, processor) =&gt; future {
          process string with processor}}
  val result = mytask.map(x =&gt; x.apply())
  </code></pre>
  
  <p>This alternative both worked and didn't work. It achieved full cpu utilization, but several exceptions were thrown because (a hypothesis) the processor was running through each string too fast and sometimes the same processor object would be applied to different strings simultaneously.</p>
  
  <p>I'm even more sure of my hypothesis that the processors are working too fast because if I provide longer input (say, whole text documents instead of 10 word headlines) I get full cpu utilization without any exceptions thrown.</p>
  
  <p>I've also tried experimenting with akka futures and scalaz promises and they all seem to only use one cpu when I split the input sequence into subsequences.</p>
  
  <p><strong>So how do I get full cpu utilization with futures in this instance while using subsequences of strings as input?</strong></p>