llm config_error ai_generated true

ValueError: Embedding dimension mismatch: query embedding dimension (384) does not match index embedding dimension (768)

ID: llm/llamaindex-embedding-model-mismatch-on-retrieval

Also available as: JSON · Markdown · 中文

90%Fix Rate

89%Confidence

1Evidence

2024-04-12First Seen

Version Compatibility

Version	Status	Introduced	Deprecated	Notes
llama-index 0.10.0	active	—	—	—
llama-index 0.10.15	active	—	—	—
llama-index 0.10.30	active	—	—	—
sentence-transformers 2.2.2	active	—	—	—
sentence-transformers 3.0.0	active	—	—	—

Root Cause

LlamaIndex retrieves a query using an embedding model with a different output dimension than the model used to build the vector index, often due to changing the embed_model config after index construction.

generic

中文

LlamaIndex使用与构建向量索引时不同的输出维度的嵌入模型来检索查询，通常是由于在索引构建后更改了embed_model配置。

Official Documentation

https://docs.llamaindex.ai/en/stable/module_guides/loading/embeddings/

Workarounds

95% success Rebuild the index with the same embedding model used for queries: from llama_index.core import VectorStoreIndex, Settings from llama_index.embeddings.huggingface import HuggingFaceEmbedding embed_model = HuggingFaceEmbedding(model_name='BAAI/bge-small-en-v1.5') Settings.embed_model = embed_model index = VectorStoreIndex.from_documents(documents, embed_model=embed_model)
```
Rebuild the index with the same embedding model used for queries:

from llama_index.core import VectorStoreIndex, Settings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

embed_model = HuggingFaceEmbedding(model_name='BAAI/bge-small-en-v1.5')
Settings.embed_model = embed_model
index = VectorStoreIndex.from_documents(documents, embed_model=embed_model)
```
90% success Persist the embedding model name in index metadata and validate on load: from llama_index.core import StorageContext, load_index_from_storage storage_context = StorageContext.from_defaults(persist_dir='./index') index = load_index_from_storage(storage_context) if index.embed_model != current_embed_model: raise ValueError(f"Embed model mismatch: index uses {index.embed_model}")
```
Persist the embedding model name in index metadata and validate on load:

from llama_index.core import StorageContext, load_index_from_storage

storage_context = StorageContext.from_defaults(persist_dir='./index')
index = load_index_from_storage(storage_context)
if index.embed_model != current_embed_model:
    raise ValueError(f"Embed model mismatch: index uses {index.embed_model}")
```

中文步骤

使用与查询相同的嵌入模型重建索引：

from llama_index.core import VectorStoreIndex, Settings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

embed_model = HuggingFaceEmbedding(model_name='BAAI/bge-small-en-v1.5')
Settings.embed_model = embed_model
index = VectorStoreIndex.from_documents(documents, embed_model=embed_model)

在索引元数据中持久化嵌入模型名称，并在加载时验证：

from llama_index.core import StorageContext, load_index_from_storage

storage_context = StorageContext.from_defaults(persist_dir='./index')
index = load_index_from_storage(storage_context)
if index.embed_model != current_embed_model:
    raise ValueError(f"嵌入模型不匹配：索引使用{index.embed_model}")

Dead Ends

Common approaches that don't work:

80% fail
Reshaping distorts the embedding space, producing semantically meaningless similarity scores and retrieval results.
50% fail
The default embedding model may still have a different dimension than the one used during indexing, unless it's the exact same model.