Hybrid search is a retrieval method that combines keyword search and vector search, then merges their results into one ranking. It matters in AI engineering because each method misses things the other catches, and a retrieval-augmented generation (RAG) system can only answer from what retrieval finds.
For example, suppose a user searches support docs for “ERR 1042 billing retry.” Keyword search returns the error-code table first, while vector search returns a general payment failure guide first. After fusion, the retry policy that both searches found ranks first, and the error-code table and the payment failure guide follow.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Why hybrid search matters for AI retrieval
Keyword search and vector search fail in different ways. Keyword search finds exact names and codes but misses paraphrases. Vector search finds paraphrases but can miss exact strings.
Hybrid search runs both and merges the results, so a query has two chances to find the right chunk.
Hybrid search helps with:
- Finding exact identifiers: Keep error codes, product names, and function names in the results even when embeddings miss them.
- Handling paraphrased questions: Still retrieve relevant passages when the user’s words differ from the document’s words.
- Diagnosing retrieval misses: See which side found a chunk and which side missed it.
- Covering mixed traffic: Serve both code-like queries and natural-language questions from one retrieval step.
The hard part is fusion. The two methods produce scores on different scales, and the best mix often depends on the query.
How hybrid search works
Hybrid search has three stages: two retrievers run on the same query, then a fusion step merges their lists.
The keyword side
Lexical search, commonly BM25, scores documents by how well their words match the query’s words. It is strong on rare, exact terms such as “ERR 1042.”
The vector side
Semantic search converts the query and each chunk into vectors, then scores how similar their meanings are. It can match “why did my payment fail?” to a passage about declined charges with no shared words.
For more on how these vectors work, see embeddings and vector search.
The fusion step
A BM25 score and a cosine similarity live on different scales. A BM25 score has no fixed upper bound, while cosine similarity has a fixed range. You cannot add them directly and get a meaningful result.
Systems handle this in one of two ways:
- Rank-based fusion: Ignore the raw scores and combine each document’s position in each list.
- Score normalization: Rescale each side’s scores per query so they share a range, then combine them with weights.
Weighted score fusion can work when scores are well calibrated. However, a weight tuned on one set of queries often fails on another.
In a retrieval-augmented generation (RAG) system, hybrid search happens at candidate generation. That is before any reranking and before the model sees the retrieved context.
How to fuse results with reciprocal rank fusion
Reciprocal rank fusion (RRF) is a common rank-based method. Elastic’s reciprocal rank fusion reference documents it.
Each document earns points from every list it appears in. A higher position earns more points.
Formula:
RRF score(d) = Σ over each result list 1 / (k + rank(d))
Where:
- d is a document, or chunk.
- rank(d) is the document’s position in that list, starting from 1.
- k is a ranking constant. In Elasticsearch, it defaults to 60.
A document missing from a list earns nothing from that list. A document that appears in both lists collects points twice.
For more on this method, see reciprocal rank fusion.
Calculate RRF in Python
The following script fuses the top three results from each side of the example query. It uses k = 60.
lexical = ["error-code table", "retry policy", "billing FAQ"]
vector = ["payment failure guide", "retry policy", "checkout errors"]
k = 60
scores = {}
for ranking in (lexical, vector):
for rank, chunk in enumerate(ranking, start=1):
scores[chunk] = scores.get(chunk, 0.0) + 1 / (k + rank)
for chunk, score in sorted(scores.items(), key=lambda item: -item[1]):
print(f"{score:.4f} {chunk}")
Output:
0.0323 retry policy
0.0164 error-code table
0.0164 payment failure guide
0.0159 billing FAQ
0.0159 checkout errors
Worked example: an error code and a plain question
Suppose your support docs receive two kinds of queries. Here is how each side handles the first one:
query "ERR 1042 billing retry"
lexical top: [error-code table, retry policy, billing FAQ]
vector top: [payment failure guide, retry policy, checkout errors]
fused (RRF, k = 60): [retry policy, error-code table, payment failure guide]
The retry policy ranks first because both sides found it. Its score of 0.0323 is roughly double that of any chunk found by only one side.
The error-code table and the payment failure guide tie at 0.0164. Each was ranked first by one side. How a tie is broken depends on the implementation.
The result keeps the exact error-code chunk near the top, even though it was missing from the vector side’s top three. It also keeps a relevant troubleshooting passage that was missing from the keyword side’s top three.
Now consider a second query: “why did my payment fail?” Keyword search has few distinctive words to match, so vector search does most of the work.
Report retrieval quality separately for code-like queries and natural-language queries. That split shows which side is actually contributing.
How to measure and debug hybrid search
Judge the merged candidate list, not each side alone. These metrics help:
- Judging the top of the list: Precision@K measures how many of the top K results are relevant.
- Separating retrieval from generation: RAG evaluation tells you whether a bad answer came from a retrieval miss or a generation error.
If a known chunk disappears after fusion, find out which of two problems you have.
- Recall problem: The chunk was absent from both candidate lists. Fix the index, chunking, or query.
- Fusion problem: The chunk was found but ranked down during fusion. Adjust the fusion parameters.
Keep the source chunks and both original rankings inspectable. Then you can trace a bad merged ranking to one side.
Limitations of hybrid search
Hybrid search adds moving parts, and each one can fail.
- More parameters and latency: Fusion adds settings to tune and a second retriever to run.
- New ways to bury a correct result: A chunk that one side ranked well can still drop after fusion.
- Normalization traps: Min-max scaling stretches every query’s scores to the full range of 0 to 1. When one side returns nearly equal scores, tiny differences become large gaps, and that side’s ordering can outweigh the other’s.
- Duplicate chunks: A chunk returned by both sides needs deduplication, or copies crowd the context.
- Uneven benefit: An average improvement can hide regressions on the exact-term queries that support teams ask most.
Run both retrievers over well-built indexes. Combining two weak indexes still gives weak results.
Hybrid search vs. reranking and single-method search
Hybrid search decides which candidates from two retrievers enter one list. Reranking then reorders a candidate list with a slower, more expensive model. Many systems use both, with hybrid search first.
A single strong vector index with good chunking can beat a hybrid setup added on top of a poor keyword index. Treat hybrid search as a remedy for misses you have observed, not a default. Measure your single-method baseline first.
Frequently asked questions
Why not just add the keyword and vector scores together?
Adding raw scores lets whichever score happens to be larger dominate. The two scales differ, and they shift as the index and query change. Rank-based fusion or careful per-query normalization is more predictable.
Is hybrid search always better than vector search alone?
No. Hybrid search helps when you observe keyword misses, such as on codes and names. On purely paraphrased queries, the keyword side may add noise. Compare both on judged queries from your own traffic.
How many candidates should each side return?
Each side should return enough candidates that the right chunk usually appears in at least one list. Teams often retrieve more per side than the final context size, then fuse and trim. Tune this with recall measurements, not a copied number.
Where does hybrid search fit in a RAG pipeline?
Hybrid search fits at candidate generation, before any reranking and before the model sees the context. Log both source rankings so you can trace a bad final answer to retrieval, fusion, or generation.
Putting hybrid search to work
Before adding hybrid search, measure a single-method baseline on judged queries from your own traffic. Include exact identifiers and paraphrased questions, and score them separately.
If you see keyword misses, add the second retriever and start with rank-based fusion. Record the fusion method, the value of k, and how many candidates each side returns.
Log both source rankings and the fused list for every retrieval. Tracing tools such as Arize Phoenix can capture those retrieval steps, so you can see whether a missing chunk was never found or was ranked down during fusion.
