title: python3 default encoding UnicodeDecodeError ascii using apache WSGI
body: <pre><code>import locale
  prefered_encoding = locale.getpreferredencoding()
  prefered_encoding 'ANSI_X3.4-1968'
  </code></pre>
  
  <p>I'm using a framework called <a href="http://inginious.readthedocs.io" rel="noreferrer">inginious</a> and it's using <a href="http://webpy.org/" rel="noreferrer">web.py</a> to render its template. </p>
  
  <pre><code>web.template.render(os.path.join(root_path, dir_path),
                                     globals=self._template_globals,
                                     base=layout_path)
  </code></pre>
  
  <p>The rendering <strong>works on my localhost</strong> but <strong>not on my staging server</strong>. </p>
  
  <p>They both run python3. I see that web.py enforces <code>utf-8</code> on </p>
  
  <p>the encoding in Python2 only (that's out of my hands)</p>
  
  <pre><code>def __str__(self):
      self._prepare_body()
      if PY2:
          return self["__body__"].encode('utf-8')
      else:
          return self["__body__"]
  </code></pre>
  
  <p>here is the stack trace</p>
  
  <pre><code>t = self._template(name),
  File "/lib/python3.5/site-packages/web/template.py", line 1028, in _template,
  self._cache[name] = self._load_template(name),
  File "/lib/python3.5/site-packages/web/template.py", line 1016, in _load_template
  return Template(open(path).read(), filename=path, **self._keywords)
  File "/lib64/python3.5/encodings/ascii.py", line 26, in decode
  return codecs.ascii_decode(input, self.errors)[0]
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 83: ordinal not in range(128),
  </code></pre>
  
  <p>My html do include hebew chars, small example </p>
  
  <pre><code>&lt;div class="modal-content"&gt;
                      &lt;div class="modal-header"&gt;
                          &lt;button type="button" class="close" data-dismiss="modal"&gt;&amp;times;&lt;/button&gt;
                          &lt;h4 class="modal-title feedback-modal-title"&gt;
                              חישוב האיברים הראשונים בסדרה של איבר ראשון חיובי ויחס שלילי:
                              &lt;span class="red-text"&gt;אי הצלחה&lt;/span&gt;
  </code></pre>
  
  <p>and I open it like so :</p>
  
  <pre><code>open('/path/to/feedback.html').read()
  </code></pre>
  
  <p>and the line where the encoding fails is where the Hebrew chars are.  </p>
  
  <p>I tried setting some environment variables in <code>~/.bashrc</code>:</p>
  
  <pre><code>export PYTHONIOENCODING=utf8
  export LC_ALL=en_US.UTF-8
  export LANG=en_US.UTF-8
  export LANGUAGE=en_US.UTF-8
  </code></pre>
  
  <p>under the user <code>centos</code></p>
  
  <p>The ingenious framework is installed as a <code>pip</code> under python3.5 site-packages. and it served by an apache server under the user <code>apache</code></p>
  
  <p>Tried setting the environment variables in the code (during the init of the app) so that the apache WSGI will be aware of them </p>
  
  <pre><code>import os 
  os.environ['LC_ALL'] = 'en_US.UTF-8'
  os.environ['LANG'] = 'en_US.UTF-8'
  os.environ['LANGUAGE'] = 'en_US.UTF-8'
  </code></pre>
  
  <p>I have edited the <code>/etc/httpd/conf/httpd.conf</code> using the <a href="https://httpd.apache.org/docs/2.4/mod/mod_env.html#setenv" rel="noreferrer">setenv</a> method:</p>
  
  <pre><code>SetEnv LC_ALL en_US.UTF-8
  SetEnv LANG en_US.UTF-8
  SetEnv LANGUAGE en_US.UTF-8
  SetEnv PYTHONIOENCODING utf8
  </code></pre>
  
  <p>and restarted using <code>sudo service httpd restart</code> and still no luck. </p>
  
  <p>My question is, what is the best practice to solve this. I understand there are hacks for this, but I want to understand what is the underline cause as well as how to solve it. </p>
  
  <p>Thanks!</p>