title: Python JSON to CSV - bad encoding, UnicodeDecodeError: 'charmap' codec can't decode byte
body: <p>I have a problem converting nested JSON to CSV. For this i use <a href="https://github.com/vinay20045/json-to-csv" rel="noreferrer">https://github.com/vinay20045/json-to-csv</a> (forked a bit to support python 3.4), here is full json-to-csv.py file.
  Converting is working, if i set </p>
  
  <pre><code>    #Base Condition
  else:
      reduced_item[str(key)] = (str(value)).encode('utf8','ignore')
  </code></pre>
  
  <p>and </p>
  
  <pre><code>fp = open(json_file_path, 'r', encoding='utf-8')
  </code></pre>
  
  <p>but when i import csv to MS Excel i see bad cyrillic characters, for example \xe0\xf1  , english text is ok.
  Experimented with setting encode('cp1251','ignore') but then i got an error
  UnicodeDecodeError: 'charmap' codec can't decode byte X in position Y: character maps to  (as here <a href="https://stackoverflow.com/questions/9233027/unicodedecodeerror-charmap-codec-cant-decode-byte-x-in-position-y-character">UnicodeDecodeError: &#39;charmap&#39; codec can&#39;t decode byte X in position Y: character maps to &lt;undefined&gt;</a>)</p>
  
  <pre><code>import sys
  import json
  import csv
  
  ##
  # This function converts an item like 
  # {
  #   "item_1":"value_11",
  #   "item_2":"value_12",
  #   "item_3":"value_13",
  #   "item_4":["sub_value_14", "sub_value_15"],
  #   "item_5":{
  #       "sub_item_1":"sub_item_value_11",
  #       "sub_item_2":["sub_item_value_12", "sub_item_value_13"]
  #   }
  # }
  # To
  # {
  #   "node_item_1":"value_11",
  #   "node_item_2":"value_12",
  #   "node_item_3":"value_13",
  #   "node_item_4_0":"sub_value_14", 
  #   "node_item_4_1":"sub_value_15",
  #   "node_item_5_sub_item_1":"sub_item_value_11",
  #   "node_item_5_sub_item_2_0":"sub_item_value_12",
  #   "node_item_5_sub_item_2_0":"sub_item_value_13"
  # }
  ##
  def reduce_item(key, value):
      global reduced_item
  
      #Reduction Condition 1
      if type(value) is list:
          i=0
          for sub_item in value:
              reduce_item(key+'_'+str(i), sub_item)
              i=i+1
  
      #Reduction Condition 2
      elif type(value) is dict:
          sub_keys = value.keys()
          for sub_key in sub_keys:
              reduce_item(key+'_'+str(sub_key), value[sub_key])
  
      #Base Condition
      else:
          reduced_item[str(key)] = (str(value)).encode('cp1251','ignore')
  
  
  if __name__ == "__main__":
      if len(sys.argv) != 4:
          print("\nUsage: python json_to_csv.py &lt;node_name&gt; &lt;json_in_file_path&gt; &lt;csv_out_file_path&gt;\n")
      else:
          #Reading arguments
          node = sys.argv[1]
          json_file_path = sys.argv[2]
          csv_file_path = sys.argv[3]
  
          fp = open(json_file_path, 'r', encoding='cp1251')
          json_value = fp.read()
          raw_data = json.loads(json_value)
  
          processed_data = []
          header = []
          for item in raw_data[node]:
              reduced_item = {}
              reduce_item(node, item)
  
              header += reduced_item.keys()
  
              processed_data.append(reduced_item)
  
          header = list(set(header))
          header.sort()
  
          with open(csv_file_path, 'wt+') as f:#wb+ for python 2.7
              writer = csv.DictWriter(f, header, quoting=csv.QUOTE_ALL, delimiter=',')
              writer.writeheader()
              for row in processed_data:
                  writer.writerow(row)
  
          print("Just completed writing csv file with %d columns" % len(header))
  </code></pre>
  
  <p><strong>How to convert cyrillic correctly and also i want to skip bad characters?</strong></p>