BM25 vs Vector Search vs Hybrid Search: A Practical Retrieval Guide

Quick answer

BM25 matches exact words, vector search matches meaning, and hybrid search uses both.

Reciprocal Rank Fusion is a simple, reliable way to combine the two result lists.

Test each option on your own real questions before you choose one.

Hey, welcome back to Roy's AI Lab!

Have you ever asked a RAG chatbot a simple question and watched it retrieve a paragraph that was almost relevant, but not quite the one you needed?

Maybe you searched for an exact error code and got a general explanation. Or you asked in everyday language while the document used completely different terms.

You might blame the embedding model, change the chunk size or rewrite the prompt. Sometimes the problem is earlier in the pipeline: your retrieval strategy doesn't match the query.

That's why BM25, vector search and hybrid search matter. They aren't just terms to memorise for a GenAI interview. They are different ways of finding evidence, and each one has strengths, limitations and trade-offs.

In this guide we'll build an intuition for all three, look at realistic examples, see how hybrid search combines results, and walk through a practical way to evaluate them.

By the end, you should be able to answer two questions:

When is keyword search enough?
When should you combine keyword and semantic retrieval?

Let's get into it.

1. What does retrieval actually do in a RAG system?

Retrieval-Augmented Generation (RAG) gives a language model relevant external information before it writes an answer. A simplified pipeline looks like this:

User question
     ↓
Query processing
     ↓
Retriever searches the indexed content
     ↓
Relevant chunks are selected
     ↓
Context is passed to the LLM
     ↓
Answer is generated

The retriever's job is not to write the answer. Its job is to find useful evidence for the question. That distinction matters. If the right passage never reaches the model, a better prompt won't fix the underlying problem. The model can only work with the evidence it receives.

Three common retrieval options are:

BM25: keyword-based retrieval.
Vector search: retrieval based on similarity between vector representations.
Hybrid search: a combination of keyword and vector retrieval.

2. What is BM25?

BM25 is a ranking function used in full-text search. It scores documents by how well their terms match the query. It takes into account how often a term appears, how rare that term is across the collection, and how long the document is.

In plain English, BM25 asks: "How well do the words in this query match the words in this document?"

Imagine your knowledge base holds these three documents:

ERR_4032: Authentication failed because the access token expired.
How to renew an expired access token.
Troubleshooting common login problems.

A developer searches for ERR_4032. BM25 is well suited to this, because the exact identifier appears in the first document. It also works well for product IDs, error codes, names, technical terms and any query where exact wording matters.

Why not just count words?

Suppose the word "token" appears in nearly every document, while ERR_4032 appears in only one. The rare identifier is far more useful for picking the right document than a common word.

BM25 handles this with inverse document frequency (IDF), which rewards rare terms. It also uses a term-frequency component that levels off, so repeating a word many times doesn't increase the score without limit. And it normalises for document length, so long documents don't win just because they contain more words.

You don't need to memorise the full formula. In an interview, start by explaining what each part is trying to solve.

Where BM25 struggles

Consider a user asking, "How do I stop my subscription?" The documentation says, "Cancel your membership."

The two express the same intent, but they share no key word. A keyword retriever may miss the match unless other terms or query expansion help it.

BM25 doesn't understand that "stop my subscription" and "cancel my membership" can mean the same thing. Its strength is lexical matching. Its limitation is that matching words is not the same as understanding meaning.

3. What is vector search?

Vector search retrieves content by comparing vector representations, usually embeddings, of the query and the candidate chunks.

An embedding model turns text into numbers that are meant to capture meaning. The query is embedded, each chunk has its own embedding, and a vector index finds the chunks that sit closest to the query under a chosen similarity measure.

In plain English, vector search asks: "Which pieces of content are closest in meaning to this question, according to the embedding model?"

Back to the subscription example. A capable embedding model may place "stop my subscription" and "cancel your membership" close together, so vector search can retrieve the right passage even though the wording differs. That makes it a good fit for natural-language questions, paraphrases, and knowledge bases where users describe an idea differently from the source.

Does vector search understand meaning perfectly?

No. Embeddings reflect patterns the model learned. They don't guarantee that every important detail is preserved.

Search for ERR_4032 with vector search and you may get general passages about authentication failures or expired tokens. Those are related, but the user needs that exact error code.

Vector search can also miss useful evidence because of poor chunking, an unsuitable embedding model, indexing problems, restrictive filters, or a query the embedding model handles badly.

Semantic similarity is a useful retrieval signal. It is not proof that a passage answers the question.

4. BM25 vs vector search: the practical difference

Dimension	BM25 / keyword search	Vector search
Main signal	Matching terms and their importance	Similarity between vector representations
Exact IDs and error codes	Often a strong fit	May be less reliable for exact identifiers
Paraphrases and synonyms	May struggle without query expansion	Often handles them better
Explainability	Matching terms can be inspected directly	Similarity is less directly interpretable
Key dependencies	Tokenisation, term statistics, index configuration	Embedding model, vector index, similarity metric
Common failure	Relevant content uses different words	Related content ranks highly without containing the exact answer
Best starting point	Exact terminology, identifiers, lexical matching	Natural-language queries and semantic matching

These are tendencies, not rules. Real results depend on the corpus, the query mix, the analysers, the embedding model, the filters and how you evaluate.

A simple way to remember it

BM25: "Find documents that contain these words."
Vector search: "Find documents that mean something similar."
Hybrid search: "Use both signals, so either one can surface the evidence."

5. What is hybrid search?

Hybrid search runs more than one retrieval method, usually BM25 and vector search, and merges their results into one ranking.

User question
   ↓
BM25 search and vector search, run side by side
   ↓
Fusion (for example RRF) merges the two result lists
   ↓
Optional reranker reorders the shortlist
   ↓
Final context goes to the LLM

Why run both? Because they fail in different ways. BM25 may find the exact error code that vector search ranks too low. Vector search may find the right policy when the user's word doesn't appear in the document.

Combining them improves the chance that useful evidence makes it into the candidate set and gets ranked sensibly.

Hybrid search isn't automatically better for every dataset, though. It adds retrieval work and complexity, so measure whether it improves results on the queries your users actually ask.

6. How do you combine BM25 and vector results?

There are several fusion strategies. Two common ones are Reciprocal Rank Fusion (RRF) and score-based fusion.

Option A: Reciprocal Rank Fusion (RRF)

BM25 and vector similarity scores usually live on different scales. Adding them directly can let one method dominate simply because its numbers are bigger.

RRF avoids this by ignoring the raw scores and using each document's rank in each list. The common form is:

RRF(d) = sum over each ranked list r of 1 / (k + rank_r(d))

Where:

d is a candidate chunk.
rank_r(d) is the chunk's position in one result list. If a chunk doesn't appear in a list, it contributes nothing from that list.
k is a smoothing constant. The original RRF paper used k = 60, and many systems keep that default.

A chunk ranked highly by more than one retriever tends to get a stronger fused score.

A simple example

Suppose the two retrievers return these results:

Document	BM25 rank	Vector rank
Chunk A	1	4
Chunk B	5	1
Chunk C	2	2
Chunk D	3	Not in top results

Chunk A is the top BM25 result, Chunk B is the top vector result, and Chunk C is consistently strong in both lists. With k = 60, Chunk C scores about 0.0323, Chunk A about 0.0320 and Chunk B about 0.0318. So Chunk C comes out on top, even though it isn't first in either list.

The lesson isn't that RRF always picks the same kind of document. RRF is a practical way to combine rankings when raw scores aren't directly comparable.

Option B: Score-based fusion

Another approach is to normalise each retriever's scores so they're comparable, then combine them with a weight:

S(d) = alpha * S_bm25(d) + (1 - alpha) * S_vector(d)

Alpha controls how much the keyword signal counts. This gives you more direct control, but normalisation and weight choice matter a lot. A weighting that works on one dataset can fail on another.

Which should you start with?

For a first hybrid baseline, RRF is a sensible choice, because it avoids making raw scores comparable. Move to score-based fusion if you have a reason to want calibrated scores or explicit weighting.

The right choice is the one that performs well on your evaluation set, not the one that looks most sophisticated on a whiteboard.

7. Where does reranking fit in?

Fusion and reranking are related, but they are separate steps. A typical multi-stage system works like this:

Retrieve a broad candidate set with BM25, vector search, or both.
Combine or filter the candidates.
Reorder the shortlist with a reranker.
Pass the most useful evidence to the LLM.

A cross-encoder reranker reads the query and a candidate together, then produces one relevance score for that pair. That can be more precise than comparing two independently created embeddings.

The cost is computation. Scoring many query-document pairs is slow, so the usual pattern is to retrieve a manageable set first and rerank only those candidates.

For example, first-stage retrieval might return 50 candidates. The reranker reorders them, and the application keeps a handful for the final context. Those numbers are illustrative. Choose yours based on retrieval quality, latency, cost and the context budget of your application.

A reranker can't recover a relevant document that never made it into the candidate set. If the evidence is missing entirely, look at retrieval, indexing, chunking and query processing first.

8. When should you use each approach?

Start with BM25 when:

Users search for exact identifiers, error codes, SKUs or product names.
Exact terminology matters.
You need a simple lexical baseline.
Your content already uses the same words as your users.

Start with vector search when:

Users ask natural-language questions.
The same idea appears under different wording.
Semantic similarity is an important signal.
Users search a broad knowledge base without knowing its terminology.

Test hybrid search when:

Your queries mix exact identifiers with natural language.
Users paraphrase terms from the source documents.
Keyword-only search misses relevant passages.
Vector-only search finds related content but misses exact terms.
Your evaluation set contains both kinds of query.

Consider reranking when:

The first stage finds relevant candidates, but their order isn't good enough.
The top results contain several related but less useful chunks.
The quality gain justifies the extra latency and cost.

Don't add every technique just because it's available. Start with the simplest baseline that meets your needs, then add complexity when your evaluation shows it helps.

9. How to evaluate BM25, vector and hybrid retrieval

This is where many implementations fall short.

A typical story: a developer tries hybrid search, asks three questions, gets a better answer and assumes the system has improved. Three questions can reveal a bug, but they can't show that one strategy is consistently better.

Build a representative evaluation set instead.

Step 1: Collect realistic queries

Include a mix of:

Exact IDs and error codes.
Natural-language questions.
Paraphrases and synonyms.
Rare terminology.
Questions that need several passages.
Questions with no answer in the knowledge base.

If you can, use real user queries with sensitive information removed.

Step 2: Label the relevant evidence

For each query, mark the chunk or chunks that contain useful evidence. Some questions have several relevant passages.

Keep the labels consistent. Retrieval metrics are only as good as the definition of "relevant" behind them.

Step 3: Compare on the same test set

Evaluate at least these configurations:

BM25 only.
Vector search only.
Hybrid search.
Hybrid search plus reranking, if reranking is part of the proposed design.

Keep the corpus, query set, access filters and other conditions the same, so the comparison is fair.

Step 4: Track retrieval metrics

Metric	What it tells you
Recall@k	Whether relevant evidence appears in the top k results
Precision@k	How much of the top k results is relevant
MRR	How highly ranked the first relevant result is
nDCG	Whether graded relevance is ordered well across the list

For example, Recall@10 asks whether relevant evidence appears among the top ten results. MRR rewards placing the first relevant result high. nDCG is useful when relevance has grades rather than a simple yes or no.

No single metric tells the whole story. Pick metrics that match your task, and read the failed queries alongside the averages.

Step 5: Measure the operational trade-offs

Retrieval quality is only part of production performance. Also measure:

Latency.
Compute or API cost.
Indexing and update requirements.
Memory and infrastructure needs.
Stability across different query types.
Behaviour when no relevant evidence exists.

A hybrid retriever that lifts recall slightly but doubles latency may or may not be worth it. It depends on your application.

Step 6: Evaluate the final answer too

Better retrieval doesn't guarantee a better answer. After you evaluate retrieval, check whether the LLM uses the evidence correctly. Track answer correctness, groundedness and citation accuracy separately from the retrieval metrics.

That tells you whether a failure came from retrieval, context assembly or generation.

10. Common mistakes to avoid

Mistake 1: Adding raw BM25 and vector scores

Their scales may not be comparable. Use a fusion method such as RRF, or normalise and calibrate the scores before combining them.

Mistake 2: Assuming vector search handles every identifier

Embeddings don't guarantee exact matching. Test error codes, version numbers, SKUs and rare terms explicitly.

Mistake 3: Assuming hybrid search always wins

On some datasets it adds complexity without a meaningful gain. Compare it against strong single-retriever baselines.

Mistake 4: Reranking too many candidates

Rerankers can improve ordering, but running an expensive model over a large candidate set adds latency and cost you may not need.

Mistake 5: Evaluating only the final answer

A wrong answer can come from missing evidence, poor ranking, truncated context or the generation step. Inspect the retrieved candidates and the exact context the LLM received.

Mistake 6: Ignoring filters and index freshness

Metadata filters, inconsistent analysers, stale indexes and mismatched embedding versions can quietly hurt retrieval. Check that documents and queries are preprocessed the same way, and that filters do what you expect.

Mistake 7: Choosing parameters by feel

Candidate counts, fusion settings and reranking depth should be tested on a representative set. Keep that evaluation set fixed, so future changes can be compared fairly.

11. A practical workflow for your next RAG project

Start simple:

Prepare and inspect your documents.
Build a reliable chunking and indexing pipeline.
Build a BM25 baseline.
Build a vector search baseline.
Compare both on the same labelled queries.
Add hybrid fusion if the two approaches complement each other.
Add reranking if the candidate order still needs work.
Evaluate the final answers and the operational performance.
Keep regression tests for failures you've already fixed.
Monitor as the corpus and query patterns change.

This makes your architecture easier to explain and debug. In an interview, describe it as: set a baseline, find the failure pattern, make the smallest useful change, and measure whether it helped.

12. How to explain it in a GenAI interview

Interviewer: "What is the difference between BM25, vector search and hybrid search?"

A clear answer:

"BM25 is a lexical method. It ranks documents using query-term matches, term statistics and document-length normalisation. It works well for exact identifiers and terminology, but it can struggle with paraphrases.

Vector search retrieves content by similarity between embeddings, so it can match queries even when the wording differs. Semantic similarity doesn't guarantee an exact match, though.

Hybrid search combines both signals. I'd set up BM25 and vector baselines, evaluate them on the same labelled queries, then test a fusion method such as Reciprocal Rank Fusion. If the right chunks are retrieved but ranked poorly, I'd consider reranking. Before choosing a production design, I'd compare the relevance gains against latency and cost."

A likely follow-up

"Why not just add the BM25 score to the vector similarity score?"

The raw scores often have different distributions and scales, so one signal can dominate for numerical reasons rather than because it's more relevant. RRF combines rank positions instead. Score-based fusion is also possible, but it needs proper normalisation and evaluation.

Another follow-up

"Does hybrid search solve hallucinations?"

No. It can improve the evidence the model sees, but it doesn't guarantee the model uses that evidence correctly. You still need to evaluate grounding, citations, and what the system does when the evidence is insufficient.

Final takeaway

BM25, vector search and hybrid search aren't competing buzzwords. They are retrieval strategies with different strengths.

BM25 is useful when the exact words carry the signal. Vector search helps when meaning matters more than wording. Hybrid search uses both, and reranking can improve the order of a candidate set.

The most important lesson is this: don't choose a retrieval architecture because it sounds advanced. Choose it because your evaluation shows it finds better evidence for your users' questions.

Practise this with real questions

Reading about retrieval is the easy part. Interviewers want to see you diagnose a retriever that is failing, so practise that directly.

Free RAG practice sets: https://roy-s-ai-lab.vercel.app/topics/rag. Every answer comes with an explanation.
Free RAG interview answers: https://roy-s-ai-lab.vercel.app/interview-qa/RAG. Read one, then explain it aloud in your own words.
Deeper scenarios: the LangChain guide covers hybrid retrieval and Reciprocal Rank Fusion in more depth: https://roy-s-ai-lab.vercel.app/products/langchain-for-ai-engineers-200-scenario-based-interview-qa

Start with the free sets. If the scenario style helps you, the paid guide goes further.