Most of the operating systems in common use today support filenames that contain arbitrary Unicode characters. Usually this is implemented by converting the Unicode string into some encoding that varies depending on the system. Today Python is converging on using UTF-8: Python on MacOS has used UTF-8 for several versions, and Python 3.6 switched to using UTF-8 on Windows as well. On Unix systems, there will only be a filesystem encoding if you’ve set the LANG or LC_CTYPE environment variables; if you haven’t, the default encoding is again UTF-8. The sys.getfilesystemencoding() function returns the encoding to use on your current system, in case you want to do the encoding manually, but there’s not much reason to bother. When opening a file for reading or writing, you can usually just provide the Unicode string as the filename, and it will be automatically converted to the right encoding for you: Functions in the os module such as os.stat() will also accept Unicode filenames. The$_{os.listdir()}$ function returns filenames, which raises an issue: should it return the Unicode version of filenames, or should it return bytes containing the encoded versions? os.listdir() can do both, depending on whether you provided the directory path as bytes or a Unicode string. If you pass a Unicode string as the path, filenames will be decoded using the filesystem’s encoding and a list of Unicode strings will be returned, while passing a byte path will return the filenames as bytes. For example, assuming the default filesystem encoding is UTF-8, running the following program: will produce the following output: The first list contains UTF-8-encoded filenames, and the second list contains the Unicode versions. Note that on most occasions, you should can just stick with using Unicode with these APIs. The bytes APIs should only be used on systems where undecodable file names can be present; that’s pretty much only Unix systems now. This section provides some suggestions on writing software that deals with Unicode. The most important tip is: Software should only work with Unicode strings internally, decoding the input data as soon as possible and encoding the output only at the end. If you attempt to write processing functions that accept both Unicode and byte strings, you will find your program vulnerable to bugs wherever you combine the two different kinds of strings. There is no automatic encoding or decoding: if you do e.g.$_{str + bytes}$ , a$_{TypeError}$ will be raised. When using data coming from a web browser or some other untrusted source, a common technique is to check for illegal characters in a string before using the string in a generated command line or storing it in a database. If you’re doing this, be careful to check the decoded string, not the encoded bytes data; some encodings may have interesting properties, such as not being bijective or not being fully ASCII-compatible. This is especially true if the input data 10 3.2 Tips for Writing Unicode-aware Programs 3.1 Unicode filenames filename='filename \u4500 abc' with open(filename, 'w') as f: f.write('blah ') fn ='filename \u4500 abc' f =open(fn,'w') f.close() import os print(os.listdir(b'.')) print(os.listdir('.')) $ python listdir-test.py [b'filename\xe4\x94\x80abc', ...] ['filename\u4500abc', ...]