title: Python CGI accessing mySQL: end of script output before headers
body: <p>I am having trouble getting a Python CGI script to execute properly on an apache2 webserver that is running on a virtual Ubuntu 18 server.  The hosting provider is DreamCompute, if that matters.  I have gotten CGI scripts working under <code>var/www/html/apps/cgi-bin/</code>.  Both helloworld.py and helloworld.pl execute fine both in the SSH terminal and in browser (Microsoft Edge).</p>
  <p>The script giving me trouble is a Python script that accesses a MySQL database, reads data from it, then uses that data to fill some lists and generate some simple output at random (a magical spell with random effects for a tabletop RPG).</p>
  <p>The spell generator script also executes fine in SSH, but when I try to view it in a browser it throws a 500 internal server error.  The apache error logs tell me that the problem is <code>end of script output before headers</code>.  I looked around, but couldn't find anyone with a similar problem and configuration.</p>
  <p>EDIT: The full entry in the error log is:
  <code>[Tue Apr 20 00:20:25.324101 2021] [cgid:error] [pid 17275:tid 140105176721152] [client 2607:fea8:1d41:b800:7dca:305:b11a:447f:62505] End of script output before headers: spell_generator.py, referer: http://my.website/apps/cgi-bin/</code></p>
  <p>After adding and removing parts of the Python script to see how it behaves in a browser, I believe I have isolated the problem: the part of the script that connects to the MySQL database.  That database is also hosted on the same Ubuntu virtual machine, and it definitely has the right kind of data in it (just strings and IDs; nothing fancy).</p>
  <p>Here's the Python code.  I've removed some documentation comments but it should be pretty straightforward:</p>
  <pre><code>#!/usr/bin/python3
  print(&quot;Content-type: text/html\n\n&quot;);
  # The above two lines are boilerplate to ensure no print statements cause 
  # problems later in the program.
  
  # Import statements
  import cgi
  import cgitb
  cgitb.enable();
  from random import choice
  import mysql.connector
  import sys
  from enum import Enum
  
  
  # Initialize enums for system comparison
  class Ruleset(Enum):
      GENERIC = &quot;GENERIC&quot;
      GENERIC_DB = &quot;generic_attributes&quot;
      DND_5 = &quot;DND5e&quot;
      DND_5_DB = &quot;dnd5e_attributes&quot;
      PATHFINDER_1 = &quot;PF1e&quot;
      PATHFINDER_1_DB = &quot;pathfinder1e_attributes&quot;
  
  
  # Determine what system to use, generic by default
  spellSystem = Ruleset.GENERIC.value;
  
  if (len(sys.argv)) &gt; 1:
      if (sys.argv[1] == Ruleset.DND_5.value):
          spellSystem = Ruleset.DND_5.value;
          
      if (sys.argv[1] == Ruleset.PATHFINDER_1.value):
          spellSystem = Ruleset.PATHFINDER_1.value;
          
  
  # === PROBLEM HERE ===
  
  # Initialize SQL cursor, defaulting to generic
  if spellSystem == Ruleset.DND_5.value:
      spellDB = mysql.connector.connect(
          host = &quot;localhost&quot;,
          user = &quot;RemoteUser&quot;,
          password = &quot;password&quot;,
          database = Ruleset.DND_5_DB.value
      )
  elif spellSystem == Ruleset.PATHFINDER_1.value:
      spellDB = mysql.connector.connect(
          host = &quot;localhost&quot;,
          user = &quot;RemoteUser&quot;,
          password = &quot;password&quot;,
          database = Ruleset.PATHFINDER_1_DB.value
      )
  else:
      spellDB = mysql.connector.connect(
          host = &quot;localhost&quot;,
          user = &quot;RemoteUser&quot;,
          password = &quot;password&quot;,
          database = Ruleset.GENERIC_DB.value
      )
  spellCursor = spellDB.cursor();
  spellCursor.execute(&quot;SELECT ElementName FROM Element&quot;);
  listHolder = spellCursor.fetchall();
  
  # === CODE BELOW DOES NOT CAUSE PROBLEMS ===
  
  #
  # [logic that uses the SQL data]
  #
  
  # Output HTML page
  print(&quot;&lt;html&gt; &lt;head&gt; &lt;title&gt;TEST - Magic Spell Generator&lt;/title&gt; &lt;link rel='stylesheet' href='../../css/style.css'&gt; &lt;/head&gt;&quot;);
  print(&quot;&lt;body&gt; body contents&quot;);
  print(&quot;&lt;/body&gt;&quot;);
  print(&quot;&lt;/html&gt;&quot;);
  
  </code></pre>
  <p>The <code>RemoteUser</code> is a user account on the SQL server, which is used by &quot;external&quot; (non-root) programs to access the databases.  <code>password</code> in the actual deployed script is a cryptographically secure password.</p>
  <p>I'm not a Python expert, but the code runs with no problems when I execute it from the SSH terminal, so I don't think that bad code is to blame (although I could certainly be wrong).  Here's a list of things I've tried already:</p>
  <ul>
  <li>Checking for too many active SQL connections (there appears to be only one, the root user).</li>
  <li>Making sure the script file has the correct privileges (<code>chmod 755</code>, same as the rest).</li>
  <li>Making sure the necessary Python modules are installed (they are, and the Python-MySQL connector is the up to date one that works with the version of Python I'm using).</li>
  <li>Restarting apache2.</li>
  </ul>
  <p>I've spent most of today trying to find an answer.  Any help or potential leads are welcome.</p>