title: Why is the subprocess.Popen argument length limit smaller than what the OS reports?
body: <p>I am running Python 3.4.3 on Linux 3.16.0. I want to use <code>subprocess.Popen</code> to run a command with a long single argument (a complex Bash invocation), roughly 200KiB.</p>
  
  <p>According to <code>getconf</code> and <code>xargs</code>, this should be well within my limits:</p>
  
  <pre><code>$ getconf ARG_MAX
  2097152
  $ xargs --show-limits &lt; /dev/null
  Your environment variables take up 3364 bytes
  POSIX upper limit on argument length (this system): 2091740
  POSIX smallest allowable upper limit on argument length (all systems): 4096
  Maximum length of command we could actually use: 2088376
  Size of command buffer we are actually using: 131072
  </code></pre>
  
  <p>However, Python fails with quite smaller limits:</p>
  
  <pre><code>&gt;&gt;&gt; subprocess.Popen('echo %s &gt; /dev/null' % ('a' * (131072-4096)), shell=True, executable='/bin/bash')
  &lt;subprocess.Popen object at 0x7f4613b58410&gt;
  &gt;&gt;&gt; subprocess.Popen('echo %s &gt; /dev/null' % ('a' * (262144-4096)), shell=True, executable='/bin/bash')
  Traceback (most recent call last):
    [...]
  OSError: [Errno 7] Argument list too long
  </code></pre>
  
  <p>Note that the Python limit is roughly the same as the "actually using" command buffer <code>xargs</code> reports. This suggests that <code>xargs</code> is somehow smart enough to start with a smaller limit and increase it as needed, but Python is not.</p>
  
  <p>Questions:</p>
  
  <ol>
  <li>Why is the Python limit smaller than the OS limit of 2MiB?</li>
  <li>Can I increase the Python limit?</li>
  <li>If so, how?</li>
  </ol>