title: Pickling asyncio tasks
body: <p>For a large set of coroutines, I am trying to pickle <em>done</em> and <em>todo</em> tasks periodically. Sample code is given below:</p>
  <pre><code>import asyncio
  import pandas as pd
  import numpy as np
  import pickle
  import random
  import time
  
  from typing import Callable, Coroutine
  
  async def coro():
      dates = pd.date_range('20130101', periods=6)
      df = pd.DataFrame(np.random.randn(6, 4), index=dates, columns=list('ABCD'))
      await asyncio.sleep(random.randint(0,15))
      return df
  
  async def todo_tasks():
      for i in range(500):
          todo.add(asyncio.create_task(coro(), name = 'coro'+str(i)))
  
  async def progressAsync(algo: Callable[...,Coroutine],
                         timeout=2) -&gt; None:
      
      task = asyncio.create_task(algo(), name=algo.__name__)
      
      todo.add(task) # add task to the asyncio loop
      
      start = time.time()
      
      while len(todo):
          
          # checkpoints at timeout
          done, pending = await asyncio.wait(todo, timeout=timeout)
          
          # remove done task from todo after timeout, update result and pickle it
          todo.difference_update(done)
          result.update(done)
          
          # ...the following does not work!!!
          with open('./result.pkl', 'wb') as f:
              pickle.dump(list(result), f)
          
          with open('./todo.pkl', 'wb') as f:
              pickle.dump(list(todo), f)
              
          # report pendings
          pending_names = (t.get_name() for t in todo)
          print(f&quot;{len(todo)}: &quot;+ &quot; &quot;.join(sorted(pending_names))[-75:])
          
      # success!
      end = time.time()
      print(f&quot;Took {int(end-start)} seconds&quot;)
      
  todo = set()
  result = set()
  
  asyncio.run(progressAsync(algo=todo_tasks))
  </code></pre>
  <p>The program runs well, except for the portion in the <em>pickle.dump</em> portion. It gives: <code>TypeError: cannot pickle 'coroutine' object</code>.</p>
  <p>Is there a way to save/pickle the results temporarily? Intent is to restart progressAsync for the remaining todos and use result for additional processing separately.</p>