title: Producer/consumer pattern with a fixed-size FIFO queue
body: <p>I need to implement the producer/consumer pattern around a fixed-size FIFO queue.  I <em>think</em> a wrapper class around a ConcurrentQueue might work for this but I'm not completely sure (and I've never worked with a ConcurrentQueue before).  The twist in this is that the queue needs to only hold a fixed number of items (strings, in my case).  My application will have one producer task/thread and one consumer task/thread.  When my consumer task runs, it needs to dequeue all of the items that exist in the queue at that moment in time and process them.</p>
  
  <p>For what it's worth, processing of the queued items by my consumer is nothing more than uploading them via SOAP to a web app that isn't 100% reliable.  If the connection can't be established or the call SOAP call fails, I'm supposed to discard those items and go back to the queue for more.  Because of the overhead of SOAP, I was trying to maximize the number of items from the queue that I could send in one SOAP call.</p>
  
  <p>At times, my producer may add items faster than my consumer is able to remove and process them.  If the queue is already full and my producer needs to add another item, I need to enqueue the new item but then dequeue the oldest item so that the size of the queue remains fixed.  Basically, I need to keep the most recent items that are produced in the queue at all time (even if it means some items don't get consumed because my consumer is currently processing previous items).</p>
  
  <p>With regard to the producer keeping the number if items in the queue fixed, I found one potential idea from this question:</p>
  
  <p><a href="https://stackoverflow.com/questions/5852863/fixed-size-queue-which-automatically-dequeues-old-values-upon-new-enques">Fixed size queue which automatically dequeues old values upon new enques</a></p>
  
  <p>I'm currently using a wrapper class (based on that answer) around a ConcurrentQueue with an Enqueue() method like this:</p>
  
  <pre><code>public class FixedSizeQueue&lt;T&gt;
  {
      readonly ConcurrentQueue&lt;T&gt; queue = new ConcurrentQueue&lt;T&gt;();
  
      public int Size { get; private set; }
  
      public FixedSizeQueue(int size)
      {
          Size = size;
      }
  
      public void Enqueue(T obj)
      {
          // add item to the queue
          queue.Enqueue(obj);
  
          lock (this) // lock queue so that queue.Count is reliable
          {
              while (queue.Count &gt; Size) // if queue count &gt; max queue size, then dequeue an item
              {
                  T objOut;
                  queue.TryDequeue(out objOut);
              }
          }
      }
  }
  </code></pre>
  
  <p>I create an instance of this class with a size limit on the queue like this:</p>
  
  <pre><code>FixedSizeQueue&lt;string&gt; incomingMessageQueue = new FixedSizeQueue&lt;string&gt;(10); // 10 item limit
  </code></pre>
  
  <p>I start up my producer task and it begins filling the queue.  The code in my Enqueue() method seems to be working properly with regard to removing the oldest item from the queue when adding an item causes the queue count to exceed the max size.  Now I need my consumer task to dequeue items and process them but here's where my brain gets confused.  What's the best way to implement a Dequeue method for my consumer that will take a snapshot of the queue at a moment in time and dequeue all items for processing (the producer may still be adding items to the queue during this process)?</p>