title: Handling non-utf8 characters in csv in Python 3 vs Python 2
body: <p>I have the following code which reads csv files (some containing non-UTF8 characters). It works well in Python 2.7.x:</p>
  
  <pre><code>    encodings = {'ukprocessed.csv': 'utf8',
                   'usprocessed.csv': 'utf8',
                   'uyprocessed.csv': 'latin1',
                   'arprocessed.csv': 'latin1'}
  
      with codecs.open(filepath, 'r') as csvfile:
          reader = csv.reader(csvfile)
          for row in reader:
              row = [x.decode(encodings[filename]).encode('utf8') for x in row]
  </code></pre>
  
  <p>However, in Python 3.4.x the tests fail with a variety of errors:</p>
  
  <ul>
  <li>AttributeError: 'str' object has no attribute 'decode'</li>
  <li>UnicodeDecodeError: 'ascii' codec can't decode byte 0xf1 in position 1078: ordinal not in range(128)
  etc...</li>
  </ul>
  
  <p>I have played around with specifying 'encoding=' in the file open, opening as bytes with 'rb' and a number of other things but I can't find a solution which works in both Python 2 and 3.</p>
  
  <p>Does anyone have any ideas as to how I can fix this?</p>
  
  <p>Thanks</p>