python encoding_error ai_generated true

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

ID: python/pip-freeze-unicode-decode-error

Also available as: JSON · Markdown · 中文
80%Fix Rate
82%Confidence
0Evidence
2024-08-04First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
3.x active — — —

Root Cause

A site-packages .dist-info METADATA or RECORD file was written with a non-UTF-8 encoding (often cp1252 or UTF-16 on Windows), and pip freeze/install tries to read it as UTF-8.

generic

中文

某个 site-packages 下的 .dist-info METADATA 或 RECORD 文件以非 UTF-8 编码写入（Windows 上常为 cp1252 或 UTF-16），而 pip freeze/install 尝试以 UTF-8 读取。

Workarounds

  1. 90% success 
    python - <<'PY'
import pathlib, sys
for p in pathlib.Path(sys.prefix).rglob('METADATA'):
    try:
        p.read_text(encoding='utf-8')
    except UnicodeDecodeError:
        print('BAD:', p)
PY
# then uninstall the owning package
python -m pip uninstall -y badpkg
  2. 85% success 
    python -m pip install --force-reinstall --no-cache-dir badpkg

Dead Ends

Common approaches that don't work:

  1. 85% fail

    Affects stdout/stderr only, not the file-reading path that raises the error.

  2. 90% fail

    The corrupt file belongs to another package's metadata, not pip itself.