title: Celery retry only the fail request in the loop and continue with the other
body: <p>I'm kinda new to celery as a whole and having the problem with retry case on a for loop:</p>
  
  <p>i have the following task:</p>
  
  <pre><code>@app.task(bind=True, autoretry_for=(CustomException,), retry_kwargs={'max_retries': 10,'countdown': 30})
  def call_to_apis(self):
     api_list = [api1, api2, api3, api4, api5,...]
     for api in api_list:
         try:
             response = requests.get(api)
             if response.status_code == 500:
                raise CustomException
         except CustomException:
             continue
  </code></pre>
  
  <p>From my understanding celery will retry on my <code>CustomException</code> get raised. </p>
  
  <p>In the case of retry, will it only retry for the the failed api or will it just run the whole process of every api in the api_list again? If so is there anyway for it to only retry the failed api ?</p>
  
  <p>Expected result: only retry the failed api</p>
  
  <p><strong>EDIT:</strong></p>
  
  <p>i have split it into 2 different tasks and 1 request function as follow:</p>
  
  <pre><code>@app.on_after_configure.connect
  def setup_periodic_tasks(sender, **kwargs):
      sender.add_periodic_task(300.0, call_to_apis.s())
      print("setup_periodic_tasks")
  
  def call_api(api):
      response = requests.get(api)
      if response.status_code == 500:
          raise CustomException
      elif response.status_code == 404:
          raise CustomWrongLinkException
  
  
  @app.task(default_retry_delay=30, max_retries=10)
  def send_fail_api(api):
      try:
          call_api(api)
      except NonceTooLowException:
          try:
              send_fail_api.retry()
          except MaxRetriesExceededError:
              print("reached max retry number")
              pass
      except Exception:
          pass
  
  
  @app.task()
  def call_to_apis():
     api_list = [api1, api2, api3, api4, api5,...]
     for api in api_list:
         try:
            call_api(api)
         except CustomException:
            send_fail_api.delay(api)
         except CustomWrongLinkException:
            print("wrong link")
         except Exception:
            pass
  </code></pre>
  
  <p>it worked and the other apis get to complete, with failed api it supposed to call to another task and retry for 10 time each delay 30 seconds.</p>
  
  <p>But i'm getting more than expected retries about 24 times(expected to only retry 10 times) and it also printed out <code>reached max retry number</code> at 10th retry but it still retry until 24 retries</p>
  
  <p>What am i doing wrong ?</p>