title: Ensure no threads are waiting on a Critical Section before destroying it
body: <p>I have an issue in my use of critical sections.  My app has a large number of threads, say 60, which all need access to a global resource.  I therefore protect that resource with a critical section.  This works perfectly during operation, however when my application shuts down, I trigger the threads to quit, and then destroy the critical section.</p>
  
  <p>The problem comes if some of those threads are waiting on the critical section at exit time, and thus are blocked from quitting themselves.</p>
  
  <p>I've written a wrapper around the windows CriticalSection calls that has an 'Initialised' flag, which I set to true when the crit is created, and set to false when I'm about to leave the crit (both cases are set when inside the crit).  This flag is checked before the 'enter crit' wrapper function tries entering the crit, bypassing the request if the flag is false.  The flag is also checked the moment any thread successfully enters the crit, making it immediately leave the crit if it's false.</p>
  
  <p>What I to do prior to deleting the crit is set the flag to false, then wait for any waiting threads to: be allowed into the crit; see the Initialised flag is false; then leave the crit (which should be a pretty quick operating for each thread).</p>
  
  <p>I check the number of threads waiting for access to the crit by checking the LockCount inside the CRITICAL_SECTION struct, and waiting until it hits 0 (in XP, that's <code>LockCount - (RecursionCount-1)</code>; in 2003 server and above, the lock count is <code>((-1) - (LockCount)) &gt;&gt; 2</code>), before I then destroy the critical section.</p>
  
  <p>This <em>should</em> be sufficient, however I'm finding that the LockCount reaches 0 when there's still one thread (always just one thread, never more) waiting to enter the crit, meaning if I delete the crit at that point, the other thread subsequently wakes up from waiting on the crit, and causes a crash as the CRITICAL_SECTION object has by that time been destroyed.</p>
  
  <p>If I keep my own internal lock count of threads waiting for access, I have the correct count; however this isn't ideal as I have to increment this count outside of the crit, meaning that value isn't protected and therefore can't be entirely relied upon at any one time.</p>
  
  <p>Does anyone know why the LockCount in the CRITICAL_SECTION struct would be out by 1?  If I use my own lock count, then check the CRITICAL_SECTION's lock count <em>after</em> that last thread has exited (and before I destroy the crit), it's still 0...</p>
  
  <p>Or, is there a better way for me to protect the global resource in my app with that many threads, other than a critical section?</p>
  
  <p>This is my wrapper struct:</p>
  
  <pre><code>typedef struct MY_CRIT {
      BOOL Initialised;
      CRITICAL_SECTION Crit;
      int MyLockCount;
  }
  </code></pre>
  
  <p>Here's my Crit init function:</p>
  
  <pre><code>BOOL InitCrit( MY_CRIT *pCrit )
  {
      if (pCrit)
      {
          InitializeCriticalSection( &amp;pCrit-&gt;Crit );          
          pCrit-&gt;Initialised = TRUE;
          pCrit-&gt;MyLockCount = 0;
          return TRUE;
      }
      // else invalid pointer
      else    
          return FALSE;
  }
  </code></pre>
  
  <p>This is my enter crit wrapper function:</p>
  
  <pre><code>BOOL EnterCrit( MY_CRIT *pCrit )
  {
      // if pointer valid, and the crit is initialised
      if (pCrit &amp;&amp; pCrit-&gt;Initialised)
      {
          pCrit-&gt;MyLockCount++;
          EnterCriticalSection( &amp;pCrit-&gt;Crit );
          pCrit-&gt;MyLockCount--;
  
          // if still initialised
          if (pCrit-&gt;Initialised)
          {
              return TRUE;
          }
          // else someone's trying to close this crit - jump out now!
          else
          {
              LeaveCriticalSection( &amp;pCrit-&gt;Crit );
              return FALSE;
          }
      }
      else // crit pointer is null
          return FALSE;
  }
  </code></pre>
  
  <p>And here's my FreeCrit wrapper function:</p>
  
  <pre><code>void FreeCrit( MY_CRIT *pCrit )
  {
      LONG    WaitingCount = 0;
  
      if (pCrit &amp;&amp; (pCrit-&gt;Initialised))
      {
          // set Initialised to FALSE to stop any more threads trying to get in from now on:
          EnterCriticalSection( &amp;pCrit-&gt;Crit );
          pCrit-&gt;Initialised = FALSE;
          LeaveCriticalSection( &amp;pCrit-&gt;Crit );
  
          // loop until all waiting threads have gained access and finished:
          do {
              EnterCriticalSection( &amp;pCrit-&gt;Crit );
  
              // check if any threads are still waiting to enter:
              // Windows XP and below:
              if (IsWindowsXPOrBelow())
              {
                  if ((pCrit-&gt;Crit.LockCount &gt; 0) &amp;&amp; ((pCrit-&gt;Crit.RecursionCount - 1) &gt;= 0))
                      WaitingCount = pCrit-&gt;Crit.LockCount - (pCrit-&gt;Crit.RecursionCount - 1);
                  else
                      WaitingCount = 0;
              }
              // Windows 2003 Server and above:
              else
              {
                  WaitingCount = ((-1) - (pCrit-&gt;Crit.LockCount)) &gt;&gt; 2;
              }
  
                          // hack: if our own lock count is higher, use that:
              WaitingCount = max( WaitingCount, pCrit-&gt;MyLockCount );
  
              // if some threads are still waiting, leave the crit and sleep a bit, to give them a chance to enter &amp; exit:
              if (WaitingCount &gt; 0)
              {
                  LeaveCriticalSection( &amp;pCrit-&gt;Crit );
                  // don't hog the processor:
                  Sleep( 1 );
              }
              // when no other threads are waiting to enter, we can safely delete the crit (and leave the loop):
              else
              {
                  DeleteCriticalSection( &amp;pCrit-&gt;Crit );
              }
          } while (WaitingCount &gt; 0);
      }
  }
  </code></pre>