title: Python unicode string rejected by psycopg
body: <p>I've received a unicode string from the wild that causes some of our psycopg2 statements to fail.</p>
  
  <p>I have reduced the problem down to a SSCE:</p>
  
  <pre><code>import psycopg2
  conn = psycopg2.connect(...)
  cur = conn.cursor()
  x = u'\ud837'
  cur.execute("SELECT %s", (x,))
  print cur.fetchone()
  </code></pre>
  
  <p>Running this gives the following exception:</p>
  
  <pre><code>Traceback (most recent call last):
    File ".../run.py", line 65, in &lt;module&gt;
      cur.execute("SELECT %s AS test", (x,))
  psycopg2.DataError: invalid byte sequence for encoding "UTF8": 0xed 0xa0 0xb7
  </code></pre>
  
  <p>Based on some of the comments, it has become clear that this particular character is one half of a surrogate pair, making it invalid to live on its own. </p>
  
  <p>Specifically then, I am looking for a mechanism to detect when a string contains an incomplete surrogate pair in Python 2.</p>
  
  <p>One such method I have found that leads to an exception is trying <code>x.encode('utf16').decode('utf16')</code>, however, since I don't totally understand the risks associated, I would be somewhat concerned here.</p>
  
  <p>Edit: Reduced SSCE string to single character causing the problem, added information based on comments.</p>