llm type_error ai_generated true

ValueError: Query vector dimension (384) does not match index dimension (768)

ID: llm/embedding-dimension-mismatch-in-vector-search

Also available as: JSON · Markdown · 中文
90%Fix Rate
87%Confidence
1Evidence
2023-11-20First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
openai==1.10.0 active — — —
pinecone-client==3.0.0 active — — —
chromadb==0.4.22 active — — —
text-embedding-ada-002 active — — —
all-MiniLM-L6-v2 active — — —
sentence-transformers==2.2.2 active — — —

Root Cause

Embedding model used for query encoding has a different output dimension than the model used to build the vector index, often due to switching between models (e.g., from text-embedding-ada-002 to a smaller model).

generic

中文

用于查询编码的嵌入模型与用于构建向量索引的模型输出维度不同，通常是由于模型切换（例如，从text-embedding-ada-002切换到较小的模型）。

Official Documentation

https://platform.openai.com/docs/guides/embeddings

Workarounds

  1. 95% success Re-embed all documents using the same embedding model as the query. For example, if using text-embedding-ada-002 (1536 dimensions), ensure both index and query use that model. 
    Re-embed all documents using the same embedding model as the query. For example, if using text-embedding-ada-002 (1536 dimensions), ensure both index and query use that model.
  2. 80% success Use a dimension-agnostic vector database (e.g., ChromaDB with auto-detection) that can handle different dimensions by storing metadata and filtering at query time. 
    Use a dimension-agnostic vector database (e.g., ChromaDB with auto-detection) that can handle different dimensions by storing metadata and filtering at query time.

中文步骤

  1. 使用与查询相同的嵌入模型重新嵌入所有文档。例如，如果使用text-embedding-ada-002（1536维度），确保索引和查询都使用该模型。
  2. 使用维度无关的向量数据库（例如，带有自动检测功能的ChromaDB），通过存储元数据并在查询时过滤来处理不同维度。

Dead Ends

Common approaches that don't work:

  1. 90% fail

    Resizing the query vector by padding with zeros or truncating corrupts the embedding and leads to poor search results.

  2. 70% fail

    Re-indexing all documents with the new model is correct but often skipped due to time constraints; partial re-indexing causes inconsistency.

  3. 95% fail

    Assuming the vector database automatically handles dimension conversion leads to silent failures or errors.