title: Sequentially chaining AJAX calls within loop using deferreds, avoiding callback hell
body: <p>I have what I would have thought was a very basic issue whose solution is for some reason completely eluding me.</p>
  
  <p>I have an HTML form <code>#sendform</code> with a lot of checkboxes each with the class <code>.userCheckbox</code>. The number of checkboxes is not predictable; it can be anywhere from just one or two to several thousand. Each box represents a user, and if it is checked, that user should receive an e-mail when the form is submitted.</p>
  
  <p>The intended outcome is that submitting the form sends an AJAX request to the PHP backend, adding a class <code>loading</code> to the parent container (which shows a little spinning wheel) when the request is sent, and then replacing that with a <code>success</code> or <code>error</code> class, as relevant, when the request completes. I don’t want to send off the entire bulk of checkboxes at once, since (correct me if I’m wrong) that would entail having to wait for all rows to be processed before getting any visual feedback on the page.</p>
  
  <p>So the very basic skeleton is something like this:</p>
  
  <pre><code>$(document).on('submit', "#sendform", function(e) {
      e.preventDefault();
      var $form = $(this);
  
      $form.find('input.userCheckbox').each(function(i) {
          var $this = $(this);
  
          if (this.checked) {
              $.ajax({
                  url: $form.prop('action'),
                  type: $form.prop('method'),
                  dataType: 'json',
                  data: { /* ... stuff here ... */ }
                  beforeSend: function() { /* Add loading class to parent */ },
                  success: function(data, textStatus, jqXHR) { /* Add success/fail class to parent depending on data returned */ },
                  error: function(jqXHR, textStatus, errorThrown) { /* Add fail class to parent */ }
              });
          }
      });
  });
  </code></pre>
  
  <p>This of course works, in theory, but in practice, all the AJAX requests being sent off pretty much at the same time wreaks havoc. The server (a bog-standard, low-end shared server) seems to handle about twenty or so more-or-less-simultaneous AJAX requests, and then it gives up and the rest of the calls fail with the <code>textStatus</code> “Error” and the <code>errorThrown</code> “Too many requests”.</p>
  
  <p>So what I need is to fire off the AJAX requests one at a time, each iteration of the loop waiting until the previous AJAX call has completed before sending off another one—but also, preferably, without locking up the entire browser, as <code>async: false</code> would do. Enter callback hell, or at least the (to me) very inelegant solution of a recursive function called in the <code>complete</code> callback of the AJAX request.</p>
  
  <p>Surely this must be an exceedingly common question to have, I thought to myself, and went searching for better alternatives. Sure enough, there is <a href="https://stackoverflow.com/search?q=jquery+chain+sequential+ajax">a plethora of StackOverflow questions</a> on variations of this topic, and several plethorata<sup>1</sup> of random Internet people offering various solutions to it as well.</p>
  
  <p>The common denominator among the vast majority of these solutions is that using deferred objects and promises is The Way To Go. Very well—I’ve never really grokked or made use of deferred objects, so I figure this is as good a time as any to figure out just what exactly they <em>are</em> and how they work. This turns out to be trickier than expected, and I’m still not sure I’ve quite got it, which may be why I’m not getting this.</p>
  
  <p>I’ve tried implementing at least five different variations of promise-based solutions found here and elsewhere now, and my problem is that <strong>none of them works</strong>. Partly because the vast majority are six or seven years old, and deferreds and promises have changed a lot since then in ways that I can’t quite keep track of.</p>
  
  <p>At the moment, I have implemented a variation on <a href="http://www.jefferydurand.com/jquery/sequential/javascript/ajax/2015/04/13/jquery-sequential-ajax-promise-deferred.html" rel="nofollow noreferrer">this solution</a>, essentially thus:</p>
  
  <pre><code>$(document).on('submit', "#sendform", function(e) {
      e.preventDefault();
      var $form = $(this);
      var promise = $.when({});
  
      $form.find('input.userCheckbox').each(function(i) {
          var $this = $(this);
  
          if (this.checked) {
              promise = promise.then(sendmail($this, $form));
          }
      });
  });
  
  function sendmail($this, $form) {
      var defer = $.Deferred();
  
      $.ajax({
          url: $form.prop('action'),
          type: $form.prop('method'),
          dataType: 'json',
          data: { /* Stuff here */ },
          beforeSend: function() {
              console.log($this.val() + " start: " + Date.now();
              /* + Add loading class to parent */
          },
          success: function(data, textStatus, jqXHR) { /* Add success/fail class to parent */ },
          error: function(jqXHR, textStatus, errorThrown) { /* Add fail class to parent */ },
          complete: function() {
              console.log($this.val() + " end: " + Date.now());
              defer.resolve();
          }
      });
  
      return defer.promise();
  }
  </code></pre>
  
  <p>This works… exactly the same way that the code at the top works. In other words, it just shoots off all the AJAX requests basically at once. No waiting for the previous call in the promise queue to finish before sprinting ahead. I log the start and completion times for each AJAX request to the console just to check; and sure enough, this is what I get, trying with ten checkboxes:<sup>2</sup></p>
  
  <pre><code>1007 start: 12:12:41.333
  1008 start: 12:12:41.341
  1009 start: 12:12:41.346
  1010 start: 12:12:41.350
  1011 start: 12:12:41.355
  1012 start: 12:12:41.359
  1013 start: 12:12:41.363
  1014 start: 12:12:41.367
  1015 start: 12:12:41.372
  1016 start: 12:12:41.375
  1007 end: 12:12:42.140
  1008 end: 12:12:42.553
  1010 end: 12:12:42.639
  1009 end: 12:12:42.772
  1011 end: 12:12:42.889
  1013 end: 12:12:43.007
  1015 end: 12:12:43.157
  1016 end: 12:12:43.289
  1012 end: 12:12:43.422
  1014 end: 12:12:43.570
  </code></pre>
  
  <p>Each AJAX request takes about a second or so to complete, but the next one gets started immediately, within a few milliseconds.</p>
  
  <p>I <strong>think</strong> I see why this is happening.</p>
  
  <p>The <code>sendform</code> function returns a promise object; but this happens immediately, before the <code>complete</code> callback to the AJAX request has resolved the underlying deferred object. So it essentially makes no difference: there is no instruction to actually <strong>wait</strong> until the deferred object is resolved before adding the next call to the queue.</p>
  
  <p>All the different approaches I’ve come across here on SO and elsewhere have had this in common—they essentially do nothing to solve the actual issue, as far as I can tell.</p>
  
  <p>So the actual question:</p>
  
  <h3>Is there no way to make use of deferred/promise object queues to make sure that an AJAX request made in a loop iteration is not made until the request made in the previous iteration has completed?</h3>
  
  <p>Am I just completely missing something blindingly obvious, or misunderstanding how deferreds and promises work?</p>
  
  <p>&nbsp;</p>
  
  <hr>
  
  <p><sup><sup>1</sup> Yes, I know <em>plethora</em> is feminine, not neuter—indulge me.</sup></p>
  
  <p><sup><sup>2</sup> Not actually using <code>Date.now()</code> since Unix timestamp milliseconds aren’t the most readable things in the world, but just a trivial wrapper function that I’ve left out here lest this answer get any longer than it already is.</sup></p>