llm install_error ai_generated true

error: failed to load model: incompatible GGUF version: model version is 3, but llama.cpp supports version 2

ID: llm/llamacpp-gguf-version-mismatch

Also available as: JSON · Markdown · 中文

95%Fix Rate

90%Confidence

1Evidence

2023-11-20First Seen

Version Compatibility

Version	Status	Introduced	Deprecated	Notes
llama.cpp 2023-10-01	active	—	—	—
llama.cpp 2023-12-15	active	—	—	—
llama.cpp 2024-01-10	active	—	—	—
GGUF v2	active	—	—	—
GGUF v3	active	—	—	—

Root Cause

GGUF model file was created with a newer version of the GGUF format (e.g., v3) than what the installed llama.cpp build supports (e.g., v2), due to outdated llama.cpp binaries.

generic

中文

GGUF模型文件使用的GGUF格式版本（例如v3）比已安装的llama.cpp构建所支持的版本（例如v2）更新，原因是llama.cpp二进制文件过时。

Official Documentation

https://github.com/ggerganov/llama.cpp#build

Workarounds

95% success Update llama.cpp to the latest version by rebuilding from source: git clone https://github.com/ggerganov/llama.cpp cd llama.cpp make clean && make -j ./main -m model.gguf
```
Update llama.cpp to the latest version by rebuilding from source:

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make clean && make -j
./main -m model.gguf
```
90% success Use a pre-built binary from the latest release on GitHub: wget https://github.com/ggerganov/llama.cpp/releases/latest/download/llama.cpp-main-ubuntu-x64.tar.gz tar -xzf llama.cpp-main-ubuntu-x64.tar.gz ./main -m model.gguf
```
Use a pre-built binary from the latest release on GitHub:

wget https://github.com/ggerganov/llama.cpp/releases/latest/download/llama.cpp-main-ubuntu-x64.tar.gz
tar -xzf llama.cpp-main-ubuntu-x64.tar.gz
./main -m model.gguf
```

中文步骤

通过从源代码重新构建，将llama.cpp更新到最新版本：

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make clean && make -j
./main -m model.gguf

使用GitHub最新版本发布的预构建二进制文件：

wget https://github.com/ggerganov/llama.cpp/releases/latest/download/llama.cpp-main-ubuntu-x64.tar.gz
tar -xzf llama.cpp-main-ubuntu-x64.tar.gz
./main -m model.gguf

Dead Ends

Common approaches that don't work:

60% fail
Downgrading may lose metadata or quantization information; converters often produce corrupted files when going backward.
90% fail
Breaks file integrity checks and causes undefined behavior or crashes during model loading.