python serialization_error ai_generated partial

_pickle.UnpicklingError: invalid load key, '\x00'.

ID: python/pickle-unpickling-error

Also available as: JSON · Markdown
55%Fix Rate
74%Confidence
41Evidence
2015-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
311 active — — —

Root Cause

The error indicates the data being unpickled is not valid pickle format. Common causes: the file was saved in a different format (safetensors, HDF5, JSON) but loaded with pickle, the file was corrupted during transfer (incomplete download, text-mode FTP), or there is a pickle protocol version mismatch. Resolution depends on whether the original data is recoverable.

generic

Workarounds

  1. 78% success Identify the actual file format and use the correct loader 
    Inspect the file header: 'head -c 16 file | xxd'. Safetensors files start with '{', HDF5 with '\x89HDF', numpy .npy with '\x93NUMPY'. For PyTorch: try 'torch.load()' with weights_only=True (PyTorch 2.0+), or use safetensors.torch.load_file() for .safetensors files.

    Sources: https://huggingface.co/docs/safetensors/

  2. 82% success Use safetensors instead of pickle for ML model weights 
    If the model was saved in safetensors format, load with 'from safetensors.torch import load_file; state_dict = load_file("model.safetensors")'. For new models, save with safetensors to avoid pickle entirely: 'from safetensors.torch import save_file; save_file(model.state_dict(), "model.safetensors")'.

    Sources: https://github.com/huggingface/safetensors

  3. 45% success Ensure pickle protocol version compatibility 
    If the file is valid pickle but uses protocol 5 (Python 3.8+), ensure the loading Python version supports it. When saving, specify a lower protocol for compatibility: 'pickle.dump(obj, f, protocol=4)'. Use 'pickletools.dis(open(file, "rb"))' to inspect the protocol version.

    Sources: https://docs.python.org/3/library/pickle.html#pickle.DEFAULT_PROTOCOL

Dead Ends

Common approaches that don't work:

  1. Upgrading Python or the pickle module 95% fail

    The pickle module is part of CPython's standard library and cannot be independently upgraded. While newer Python versions support higher pickle protocols, they are backwards-compatible: Python 3.11 can read any protocol 0-5 file. The error means the data is not valid pickle at all, not that the protocol is too new.

  2. Clearing __pycache__ or .pyc files 99% fail

    .pyc files are compiled bytecode caches, unrelated to pickle data serialization. Clearing them has no effect on pickle loading. This advice likely stems from confusion between Python's bytecode format and pickle's serialization format.

  3. Re-downloading or re-generating the file without verifying the format 60% fail

    If the file was never pickle format to begin with (e.g., it is a safetensors, GGUF, or HDF5 file), re-downloading it produces the same error. The issue is using the wrong deserialization function, not a corrupt download.

Error Chain

Frequently confused with: