python encoding ai_generated true

UnicodeEncodeError: 'ascii' codec can't encode characters — Korean folder in sys.path

ID: python/unicode-path-encoding-error

Also available as: JSON · Markdown
90%Fix Rate
92%Confidence
4Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
311 active — — —

Root Cause

Python process fails when project lives in a Korean-named directory (e.g. /home/user/프로젝트) and locale is not UTF-8.

generic

Workarounds

  1. 95% success Set locale to UTF-8: export LANG=en_US.UTF-8 (or ko_KR.UTF-8) and ensure locale-gen is run 
    export LANG=ko_KR.UTF-8; sudo locale-gen ko_KR.UTF-8  # fixes filesystem encoding

    Sources: https://docs.python.org/3/howto/unicode.html

  2. 92% success Set PYTHONUTF8=1 environment variable (Python 3.7+ UTF-8 mode) 
    export PYTHONUTF8=1  # forces UTF-8 for all file operations regardless of locale

    Sources: https://docs.python.org/3/using/cmdline.html#envvar-PYTHONUTF8

Dead Ends

Common approaches that don't work:

  1. Rename the folder to ASCII 50% fail

    Works but does not fix the root cause; other Korean paths will still break

  2. Set PYTHONIOENCODING=utf-8 only 70% fail

    PYTHONIOENCODING affects stdout/stderr encoding, not filesystem path encoding