Hybrid retrieval + reranking

Production · ~9 min read · Updated 23 Jun 2026

Vector search alone often misses exact keyword matches; BM25 alone misses paraphrases. Hybrid retrieval runs both, merges the candidate lists, and a reranker then fixes the final order before your LLM sees anything.

Why hybrid?

Bi-encoder vector search excels at semantic similarity — “vehicle” matches “car”. BM25 excels at lexical overlap — product SKUs, legal clause numbers, error codes. In enterprise RAG, the right chunk often needs both signals. Hybrid retrieval widens recall; reranking supplies precision on the merged shortlist.

Hybrid finds more of the right hay. Reranking finds the needle in it.

The retrieve-merge-rerank pattern

1. BM25 top 50        ─┐
                       ├─▶ dedupe + merge ─▶ ~80 unique candidates
2. Vector top 50      ─┘
3. Cross-encoder rerank on merged list ─▶ top 5–10 for LLM

Retrieve wider than you would with a single channel — each list might return 40–60 items. After deduplication you still want 50–100 unique chunks going into the reranker. See rerank for RAG for top-k defaults.

Fusion strategies

Minimal Python example

from rank_bm25 import BM25Okapi
from sentence_transformers import CrossEncoder

reranker = CrossEncoder("BAAI/bge-reranker-base")

def hybrid_rerank(query, corpus_chunks, vector_hits, bm25_hits, top_n=5):
    # Merge by chunk id, preserve text
    seen, merged = set(), []
    for chunk_id, text in vector_hits + bm25_hits:
        if chunk_id in seen: continue
        seen.add(chunk_id)
        merged.append(corpus_chunks[chunk_id])
    pairs = [(query, c) for c in merged]
    scores = reranker.predict(pairs)
    ranked = sorted(zip(scores, merged), reverse=True)
    return [c for _, c in ranked[:top_n]]

Frameworks like Elasticsearch hybrid queries, Weaviate hybrid search, and LlamaIndex QueryFusionRetriever wrap the same idea — retrieve from multiple backends, then optionally rerank.

Tuning tips

• Over-retrieve each channel. If BM25 returns 30 and vectors return 30, overlap means fewer than 60 unique — aim higher per channel.
• Reranker is the equaliser. Fusion order matters less once a good cross-encoder sees every candidate.
• Measure on your data. Hybrid helps most when your eval set has both lexical and semantic queries — see evaluate rerankers.

Keep reading