title: C/Python: Undefined behavior from C code, dir() causing a TypeError
body: <p>I am testing a module that I rewrote from C to Python and having one of the stranger bugs I've seen in my career. The offending Python code with tests is just three lines:</p>
  
  <pre><code># MATCHERS is a list of compiled regular expression objects defined as a
  # global in the top level of the module
  print dir(MATCHERS[0])
  # diff.Differ is a class implemented in C with thed Python API; zoneA/B Words are
  # lists and CompareLines, isJunk, and SetStaticLine are functions
  differ = diff.Differ(zoneAWords, zoneBWords, CompareLines, isJunk, SetStaticLine)
  # Causes a TypeError
  dir(MATCHERS[0])
  </code></pre>
  
  <p>These three lines produce the following output:</p>
  
  <pre><code>['__class__', '__copy__', '__deepcopy__', '__delattr__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'findall', 'finditer', 'flags', 'groupindex', 'groups', 'match', 'pattern', 'scanner', 'search', 'split', 'sub', 'subn']
  Traceback (most recent call last):
    File "Diff.py", line 1032, in &lt;module&gt;
      main()
    File "Diff.py", line 1019, in main
      diffs = CompareFiles(sys.argv[1], sys.argv[2])
    File "Diff.py", line 584, in CompareFiles
      print dir(MATCHERS[0])
  TypeError: eqTest must be a function
  </code></pre>
  
  <p>Printing the dir() of the compiled regular expression object works fine before the object written in C is created, but afterward simply taking the dir() causes a TypeError. As far as I know, there is no way for dir() to cause a TypeError. The TypeError and message associated with it are from my code as part of the customized setter function for the object's eqTest field, but a call to dir() does not, of course, try to set eqTest to anything; it has already been successfully set in the previous line.</p>
  
  <p>This all leads me to believe that something went wrong in the C code to produce such strange behavior. My current hypothesis is a buffer overflow; writing past the bounds of an array changed something that is causing these errors. (Asking MATCHERS[0] to match a string simply crashes Python with no error message) I've been going over the initialization code for a while and just thought I would check if there is anything else that could be causing this.</p>
  
  <p>My code is too long to include here and is on Pastebin <a href="http://pastebin.com/0Jj5xhD4" rel="nofollow">here</a>; you can ignore everything after line 434 as it hasn't been executed yet.</p>