LLAMA-CPP-ERR-0003 llm install_error ai_generated true

llama_cpp.llama_cpp.LlamaError: Model file 'model.gguf' is not a valid GGUF file or is corrupted. Expected magic number 0x46554747, got 0x00000000.

ID: llm/llama-cpp-gguf-model-mismatch

Also available as: JSON · Markdown · 中文
92%Fix Rate
87%Confidence
1Evidence
2024-03-12First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
llama-cpp-python==0.2.60 active — — —
llama-cpp-python==0.2.70 active — — —
llama.cpp==b2775 active — — —

Root Cause

The GGUF model file is either incomplete (truncated download), corrupted during transfer, or is not actually a GGUF file (e.g., a PyTorch checkpoint or a different format renamed to .gguf).

generic

中文

GGUF 模型文件要么不完整（下载截断），在传输过程中损坏，或者实际上不是 GGUF 文件（例如，PyTorch 检查点或其他格式重命名为 .gguf）。

Official Documentation

https://github.com/ggerganov/llama.cpp/blob/master/docs/development/gguf.md

Workarounds

  1. 98% success Verify the GGUF file integrity using the sha256 checksum provided by the model source (e.g., Hugging Face). Re-download the file if the checksum doesn't match: `sha256sum model.gguf` then compare with the published hash. Example: `curl -L -o model.gguf https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGUF/resolve/main/mistral-7b-instruct-v0.2.Q4_K_M.gguf`. 
    Verify the GGUF file integrity using the sha256 checksum provided by the model source (e.g., Hugging Face). Re-download the file if the checksum doesn't match: `sha256sum model.gguf` then compare with the published hash. Example: `curl -L -o model.gguf https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGUF/resolve/main/mistral-7b-instruct-v0.2.Q4_K_M.gguf`.
  2. 85% success Use the `convert.py` script from llama.cpp to convert a PyTorch checkpoint to GGUF: `python convert.py --outfile model.gguf /path/to/pytorch/model`. 
    Use the `convert.py` script from llama.cpp to convert a PyTorch checkpoint to GGUF: `python convert.py --outfile model.gguf /path/to/pytorch/model`.
  3. 90% success Check the first few bytes of the file to confirm it's a GGUF: `head -c 4 model.gguf | od -A x -t x1z`. Expected output: `000000 47 47 55 46 >GGUF<`. If it shows zeros or other values, the file is corrupted or not GGUF. 
    Check the first few bytes of the file to confirm it's a GGUF: `head -c 4 model.gguf | od -A x -t x1z`. Expected output: `000000 47 47 55 46 >GGUF<`. If it shows zeros or other values, the file is corrupted or not GGUF.

中文步骤

  1. 使用模型源（例如 Hugging Face）提供的 sha256 校验和验证 GGUF 文件完整性。如果校验和不匹配，则重新下载文件：`sha256sum model.gguf` 然后与发布的哈希值进行比较。示例：`curl -L -o model.gguf https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGUF/resolve/main/mistral-7b-instruct-v0.2.Q4_K_M.gguf`。
  2. 使用 llama.cpp 的 `convert.py` 脚本将 PyTorch 检查点转换为 GGUF：`python convert.py --outfile model.gguf /path/to/pytorch/model`。
  3. 检查文件的前几个字节以确认它是 GGUF：`head -c 4 model.gguf | od -A x -t x1z`。预期输出：`000000 47 47 55 46 >GGUF<`。如果显示为零或其他值，则文件已损坏或不是 GGUF。

Dead Ends

Common approaches that don't work:

  1. 98% fail

    The error is in the model file, not the library. Reinstalling the package does not repair a corrupted or invalid GGUF file.

  2. 95% fail

    llama.cpp only supports GGUF format. Changing the extension doesn't convert the file; the library will still try to parse it as GGUF and fail.

  3. 85% fail

    The quantization level is embedded in the GGUF file itself. You cannot change the quantization by renaming or modifying metadata; you need the correct GGUF file for that quantization.