python encoding_error official true

UnicodeError

ID: python/unicodeerror

Also available as: JSON · Markdown
80%Fix Rate
95%Confidence
0Evidence

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
3.x active — — —

Root Cause

A Unicode-related encoding or decoding error occurred. This is the base class for UnicodeEncodeError, UnicodeDecodeError, and UnicodeTranslateError.

generic

Official Documentation

https://docs.python.org/3/library/exceptions.html

Workarounds

  1. 90% success Always specify encoding explicitly: open(file, encoding='utf-8') 
    Never rely on locale.getpreferredencoding() which varies by OS
  2. 90% success Use errors='replace' or 'surrogateescape' for controlled handling of bad bytes 
    open(f, encoding='utf-8', errors='replace') replaces bad chars with U+FFFD

Dead Ends

Common approaches that don't work:

  1. Using errors='ignore' to skip bad characters 80% fail

    Silently loses data; may produce incorrect output