llm retrieval ai_generated true

RAG retrieval misses relevant docs — cosine similarity threshold too high

ID: llm/embedding-cosine-similarity-threshold-arbitrary

Also available as: JSON · Markdown
82%Fix Rate
85%Confidence
4Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
any active — — —

Root Cause

Hardcoded cosine similarity threshold misses semantically relevant documents.

generic

Workarounds

  1. 88% success Calibrate threshold per embedding model using a validation set 
    Find the threshold that maximizes F1 score on known-relevant docs

    Sources: https://www.pinecone.io/learn/vector-similarity/

  2. 90% success Use top-K retrieval instead of threshold-based filtering 
    retriever = vectorstore.as_retriever(search_kwargs={'k': 5})

    Sources: https://python.langchain.com/docs/how_to/vectorstore_retriever/

Dead Ends

Common approaches that don't work:

  1. Set a universal threshold like 0.8 for all embedding models 85% fail

    Different models have different similarity distributions

  2. Use Euclidean distance instead of cosine for normalized embeddings 70% fail

    For normalized vectors, cosine and Euclidean are monotonically related