llm embedding ai_generated true

Cosine similarity returns nonsense results or vector DB rejects insert due to dimension mismatch

ID: llm/embedding-dimension-mismatch-silent

Also available as: JSON · Markdown
90%Fix Rate
92%Confidence
3Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
any active — — —

Root Cause

Different embedding models produce different dimension vectors (OpenAI ada-002=1536, text-embedding-3-small=1536, text-embedding-3-large=3072). Mixing models in the same vector DB index corrupts similarity search silently with no error.

generic

Workarounds

  1. 95% success Re-embed all documents when changing embedding models 
    Switching models requires full re-indexing. Store original text alongside embeddings for re-embedding.
  2. 90% success Use model name as part of the vector index name for isolation 
    index_name = f'docs_{model_name}_{dimension}'  # separate index per model
  3. 82% success Use OpenAI's dimensions parameter to control output size if supported 
    client.embeddings.create(model='text-embedding-3-small', input=text, dimensions=512)  # Matryoshka embedding

Dead Ends

Common approaches that don't work:

  1. Switch embedding model and keep using the existing vector index 92% fail

    New model produces different-dimension or differently-distributed vectors. Similarity scores become meaningless. Some DBs accept any dimension silently.

  2. Truncate or pad vectors to match dimensions 88% fail

    Truncating loses semantic information. Padding with zeros distorts distance calculations. Neither preserves similarity semantics.