python encoding_error ai_generated true

UnicodeDecodeError: 'charmap' codec can't decode byte 0x9a in position 1234: character maps to <undefined>

ID: python/pip-requirements-file-encoding-error

Also available as: JSON · Markdown · 中文
80%Fix Rate
85%Confidence
0Evidence
2024-05-12First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
3.8 active — — —
3.9 active — — —
3.10 active — — —
3.11 active — — —

Root Cause

A requirements.txt file contains non-UTF-8 encoded characters (e.g., Windows-1252), and pip attempts to decode it using the system's default encoding.

generic

中文

requirements.txt 文件包含非 UTF-8 编码的字符（例如 Windows-1252），pip 尝试使用系统默认编码解码该文件。

Workarounds

  1. 95% success Re-save requirements.txt with UTF-8 encoding using a proper editor 
    Open file in VS Code or Sublime, click 'Save with Encoding' and choose UTF-8, then re-run `pip install -r requirements.txt`
  2. 85% success Force pip to use UTF-8 encoding by setting PYTHONUTF8 environment variable 
    On Windows: `set PYTHONUTF8=1` then `pip install -r requirements.txt`. On Linux/macOS: `PYTHONUTF8=1 pip install -r requirements.txt`

Dead Ends

Common approaches that don't work:

  1. Manually editing the file in Notepad and saving as ANSI 70% fail

    Notepad's 'ANSI' encoding is often Windows-1252, which may still contain problematic bytes for other locales.

  2. Using `pip install -r requirements.txt --no-cache` 90% fail

    The --no-cache flag does not affect file decoding; the error occurs during file reading, not caching.