title: UnicodeEncodeError: 'charmap' codec can't encode - character maps to <undefined>, print function
body: <p>I am writing a Python (Python 3.3) program to send some data to a webpage using POST method.  Mostly for debugging process I am getting the page result and displaying it on the screen using <code>print()</code> function.</p>
  
  <p>The code is like this:</p>
  
  <pre><code>conn.request("POST", resource, params, headers)
  response = conn.getresponse()
  print(response.status, response.reason)
  data = response.read()
  print(data.decode('utf-8'));
  </code></pre>
  
  <p>the <code>HTTPResponse</code> <code>.read()</code> method returns a <code>bytes</code> element encoding the page (which is a well formated UTF-8 document)  It seemed okay until I stopped using IDLE GUI for Windows and used the Windows console instead.  The returned page has a U+2014 character (em-dash) which the print function translates well in the Windows GUI (I presume Code Page 1252) but does not in the Windows Console (Code Page 850).  Given the <code>strict</code> default behavior I get the following error:</p>
  
  <pre><code>UnicodeEncodeError: 'charmap' codec can't encode character '\u2014' in position 10248: character maps to &lt;undefined&gt;
  </code></pre>
  
  <p>I could fix it using this quite ugly code:</p>
  
  <pre><code>print(data.decode('utf-8').encode('cp850','replace').decode('cp850'))
  </code></pre>
  
  <p>Now it replace the offending character "—" with a <code>?</code>.  Not the ideal case (a hyphen should be a better replacement) but good enough for my purpose.</p>
  
  <p>There are several things I do not like from my solution.</p>
  
  <ol>
  <li>The code is ugly with all that decoding, encoding, and decoding.</li>
  <li>It solves the problem for just this case.  If I port the program for a system using some other encoding (latin-1, cp437, back to cp1252, etc.) it should recognize the target encoding.  It does not.  (for instance, when using again the IDLE GUI, the emdash is also lost, which didn't happen before)</li>
  <li>It would be nicer if the emdash translated to a hyphen instead of a interrogation bang.</li>
  </ol>
  
  <p>The problem is not the emdash (I can think of several ways to solve that particularly problem) but I need to write robust code.  I am feeding the page with data from a database and that data can come back.  I can anticipate many other conflicting cases: an 'Á' U+00c1 (which is possible in my database) could translate into CP-850 (DOS/Windows Console encodign for Western European Languages) but not into CP-437 (encoding for US English, which is default in many Windows instalations).</p>
  
  <p>So, the question:</p>
  
  <p>Is there a nicer solution that makes my code agnostic from the output interface encoding?</p>