title: Apache VERY high page load time
body: <p>My Drupal 6 site has been running smoothly for years but recently has experienced intermittent periods of extreme slowness (10-60 sec page loads). Several hours of slowness followed by hours of normal (4-6 sec) page loads. The page always loads with no error, just sometimes takes forever.</p>
  
  <p>My setup:</p>
  
  <ul>
  <li>Windows Server 2003</li>
  <li>Apache/2.2.15 (Win32) Jrun/4.0</li>
  <li>PHP 5</li>
  <li>MySql 5.1</li>
  <li>Drupal 6</li>
  <li>ColdFusion 9</li>
  <li>Vmware virtual environment</li>
  <li>DMZ behind a corporate firewall</li>
  <li>Traffic: 1-3 hits/sec peak</li>
  </ul>
  
  <p>Troubleshooting</p>
  
  <ul>
  <li>No applicable errors in apache error log</li>
  <li>No errors in drupal event log</li>
  <li>Drupal devel module shows 242 queries in 366.23 milliseconds,page execution time 2069.62 ms.  (So it looks like queries and php scripts are not the problem)</li>
  <li>NO unusually high CPU, memory, or disk IO</li>
  <li>Cold fusion apps, and other static pages outside of drupal also load slow</li>
  <li>webpagetest.org test shows very high time-to-first-byte </li>
  </ul>
  
  <p>The problem seems to be with Apache responding to requests, but previously I've only seen this behavior under 100% cpu load.  Judging solely by resource monitoring, it looks as though very little is going on.</p>
  
  <p>Here is the kicker - roughly half of the site's access comes from our LAN, but if I disable the firewall rule and block access from outside of our network, internal (LAN) access (1000+ devices) is speedy.  But as soon as outside access is restored the site is crippled.</p>
  
  <p>Apache config? Crawlers/bots?  Attackers? I'm at the end of my rope, where should I be looking to determine where the problem lies?</p>
  
  <p>------Edit:-----</p>
  
  <p>Attached is a waterfall chart from webpagetest.org showing a 15 second load time. I've seen times as high as several minutes.  And again, the server runs fine much of the time.  The green areas indicate that the browser has sent a request and is waiting to recieve the first byte of data back from the server.  This is certainly a back-end delay, but it is puzzling that the CPU is barely used during this slowness.</p>
  
  <p>(Not enough rep to post an image, see <a href="https://webmasters.stackexchange.com/questions/54658/apache-very-high-page-load-time">https://webmasters.stackexchange.com/questions/54658/apache-very-high-page-load-time</a></p>
  
  <p>------Edit------</p>
  
  <p>On the Apache side of things - Is this possibly a ThreadsPerChild issue?</p>