title: Invalid argument: TypeError: an integer is required (got type NoneType) when closing Tensorflow
body: <p>I'm implementing the IMPALA framework <a href="https://github.com/deepmind/scalable_agent" rel="nofollow noreferrer">https://github.com/deepmind/scalable_agent</a> which uses tensorflows multiprocessing. Everything seems to work fine as the experiment ran for the specified number of steps... however I start to get a type error when tensorflow is closing processes. </p>
  
  <p>Does someone know what could've contributed to this error and how to solve it? This seems to be something with tensorflow rather than the original code itself. </p>
  
  <pre><code>    INFO:tensorflow:Closing all processes.
      [750.    450.     25.125]
      2020-01-13 17:01:00.624889: W tensorflow/core/framework/op_kernel.cc:1389] Invalid argument: TypeError: an integer is required (got type NoneType)
      Traceback (most recent call last):
  
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/ops/script_ops.py", line 207, in __call__
          ret = func(*args)
  
        File "/home/haianh/anaconda3/envs/lab/scalable_agent/py_process.py", line 86, in py_call
          result = self._out.recv()
  
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 250, in recv
          buf = self._recv_bytes()
  
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 411, in _recv_bytes
          return self._recv(size)
  
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 379, in _recv
          chunk = read(handle, remaining)
  
      TypeError: an integer is required (got type NoneType)
  
  
      INFO:tensorflow:Error reported to Coordinator: &lt;class 'tensorflow.python.framework.errors_impl.InvalidArgumentError'&gt;, TypeError: an integer is required (got type NoneType)
      Traceback (most recent call last):
  
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/ops/script_ops.py", line 207, in __call__
          ret = func(*args)
  
        File "/home/haianh/anaconda3/envs/lab/scalable_agent/py_process.py", line 86, in py_call
          result = self._out.recv()
  
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 250, in recv
          buf = self._recv_bytes()
  
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 411, in _recv_bytes
          return self._recv(size)
  
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 379, in _recv
          chunk = read(handle, remaining)
  
      TypeError: an integer is required (got type NoneType)
  
  
           [[{{node scan/while/flow_environment_step/step}}]]
      INFO:tensorflow:All processes closed.
      Traceback (most recent call last):
        File "experiment.py", line 689, in &lt;module&gt;
          tf.app.run()
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
          _sys.exit(main(argv))
        File "experiment.py", line 683, in main
          train(action_set, level_names)
        File "experiment.py", line 630, in train
          session.run(enqueue_ops)
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 788, in __exit__
          self._close_internal(exception_type)
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 826, in _close_internal
          self._sess.close()
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1082, in close
          self._sess.close()
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1244, in close
          ignore_live_threads=True)
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/training/coordinator.py", line 389, in join
          six.reraise(*self._exc_info_to_raise)
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/six.py", line 696, in reraise
          raise value
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/training/queue_runner_impl.py", line 257, in _run
          enqueue_callable()
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1257, in _single_operation_run
          self._call_tf_sessionrun(None, {}, [], target_list, None)
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
          run_metadata)
      tensorflow.python.framework.errors_impl.InvalidArgumentError: TypeError: an integer is required (got type NoneType)
      Traceback (most recent call last):
  
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/site-packages/tensorflow/python/ops/script_ops.py", line 207, in __call__
          ret = func(*args)
  
        File "/home/haianh/anaconda3/envs/lab/scalable_agent/py_process.py", line 86, in py_call
          result = self._out.recv()
  
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 250, in recv
          buf = self._recv_bytes()
  
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 411, in _recv_bytes
          return self._recv(size)
  
        File "/home/haianh/anaconda3/envs/lab/lib/python3.6/multiprocessing/connection.py", line 379, in _recv
          chunk = read(handle, remaining)
  
      TypeError: an integer is required (got type NoneType)
  
  
           [[{{node scan/while/flow_environment_step/step}}]]
  </code></pre>