Embedding Strategies for RAG
Choose and optimize embedding models for semantic search and RAG — a model comparison table, the query/document prefixes that make-or-break BGE and E5, Matryoshka dimension reduction, four chunking strategies, and a retrieval-quality evaluation harness (precision@k, recall@k, MRR, nDCG@k) so you measure an embedding change instead of guessing at it.
An E5 or BGE embedding model used without its required query/document prefix quietly returns worse results than a model a third its size — the prefix isn't optional tuning, it's how the model was trained, and it's the single most common reason a self-hosted RAG retrieval setup underperforms the API it replaced.
Who it's for: developers building RAG or semantic search, ML engineers choosing between API and self-hosted embedding models, teams tuning chunking strategy or retrieval quality, anyone whose vector search returns near-misses and isn't sure whether the model, the chunking, or the preprocessing is at fault
Example
"Our RAG retrieval quality is mediocre — help me improve the embeddings" → A model picked to match the domain (code vs prose vs multilingual), the correct query/document prefix applied for that model, a chunking strategy chosen to preserve semantic boundaries, dimensions reduced via Matryoshka where storage matters, and the whole change measured against a labeled set with precision@k / recall@k / MRR / nDCG@k rather than eyeballed
CLAUDE.md Template
New here? 3-minute setup guide → | Already set up? Copy the template below.
npx ccplaybooks add embedding-strategies# Embedding Strategies
Select and optimize embedding models for semantic search and RAG applications. Use when choosing an embedding model, implementing chunking, reducing dimensions, handling multilingual content, or evaluating retrieval quality.
## Choosing a Model
| Model | Dimensions | Max Tokens | Best For |
|---|---|---|---|
| text-embedding-3-large | 3072 | 8191 | High accuracy |
| text-embedding-3-small | 1536 | 8191 | Cost-effective |
| voyage-2 | 1024 | 4000 | Code, legal |
| bge-large-en-v1.5 | 1024 | 512 | Open source |Get new playbooks like this one
One email a week with new Claude Code workflows. Free, like everything here.
No spam. Unsubscribe anytime.
What This Does
A practical guide to the embedding layer of a RAG or semantic-search system, covering the decisions that actually move retrieval quality: which model to pick (a comparison table across OpenAI, Voyage, BGE, MiniLM, and E5 with dimensions, token limits, and best-fit use cases), the model-specific query/document prefixes that several open models were trained with and silently underperform without (BGE's query prefix, E5's query:/passage: split), Matryoshka dimension reduction to cut storage and speed up search on the text-embedding-3-* models, and four chunking strategies (token, sentence, markdown-section, and recursive character splitting) chosen to preserve semantic boundaries rather than cut mid-thought.
Crucially, it closes the loop with a retrieval-quality evaluation harness — precision@k, recall@k, MRR, and nDCG@k computed against a labeled query→relevant-docs set — so an embedding or chunking change gets measured instead of guessed at, plus the normalization and batching practices that keep cosine similarity correct and API costs down. The code is runnable Python (OpenAI, sentence-transformers, tiktoken, nltk), and the Do's/Don'ts call out the traps that corrupt a vector index silently: mixing embedding models in one index, ignoring token-limit truncation, and skipping preprocessing.
Quick Start
Step 1: Create a Project Folder
mkdir rag-embeddings && cd rag-embeddings
Step 2: Download the Template
Click Download above, then:
mv ~/Downloads/CLAUDE.md ./
Step 3: Build or Tune Your Embedding Layer
claude
Describe your RAG setup and what's underperforming. Claude will help pick a model for your domain, apply the correct prefixes, choose a chunking strategy, and — most importantly — stand up the retrieval-quality metrics so you can prove a change helped.
Tips & Best Practices
- Apply the model's required query/document prefix before anything else — a BGE or E5 model used without it is the most common self-inflicted RAG retrieval problem, and no amount of chunking tuning compensates for it.
- Never mix embedding models within a single index; their vector spaces are incompatible and the similarity scores become meaningless, usually without any error to warn you.
- Measure every embedding or chunking change against a labeled set with the included metrics — "it feels better" is exactly how teams ship a change that quietly lowered recall.
Limitations
- The model comparison table is a point-in-time snapshot of a fast-moving field — treat dimensions and token limits as current-ish starting points and confirm against each provider's live docs before committing an index.
- Retrieval-quality metrics require a labeled query→relevant-docs set to compute; if you have no labeled data yet, building that set is the real first step, and the harness only pays off once it exists.
- Covers the embedding/retrieval layer specifically — the generation side of RAG (prompt construction, reranking, answer grounding) is a separate concern this playbook doesn't address.