Hybrid Retrieval | HyperKit.ai

Definition

Hybrid retrieval combines sparse (keyword-based) and dense (semantic) retrieval methods to leverage the strengths of both. This approach captures both exact term matches and semantic similarity, often outperforming either method alone.

Results from both methods are merged using techniques like reciprocal rank fusion or weighted combination.

Key Concepts

Score fusion: Combining scores from different retrieval methods
Reciprocal rank fusion (RRF): Ranking-based combination approach
Weighted combination: Tunable weights for each method
Complementary strengths: Exact matching + semantic understanding

Examples

Concept

Hybrid Retrieval Flow

Query: "Python machine learning tutorial"

SPARSE RETRIEVAL (BM25):
1. "Python ML tutorial for beginners" → score: 8.5
2. "Machine learning with Python" → score: 7.2
3. "Python programming basics" → score: 5.1

DENSE RETRIEVAL (Embeddings):
1. "Getting started with scikit-learn" → score: 0.89
2. "Python ML tutorial for beginners" → score: 0.85
3. "Deep learning fundamentals" → score: 0.78

HYBRID FUSION (RRF):
1. "Python ML tutorial for beginners"   ← Both methods agree!
2. "Getting started with scikit-learn"  ← Dense found semantic match
3. "Machine learning with Python"       ← Sparse keyword match

Best of both: exact matches + semantic discoveries

Implementation

Reciprocal Rank Fusion

def reciprocal_rank_fusion(results_list, k=60):
    """
    Combine multiple ranked result lists using RRF.
    k is a constant (typically 60) for smoothing.
    """
    fused_scores = {}

    for results in results_list:
        for rank, doc in enumerate(results):
            doc_id = doc['id']
            # RRF formula: 1 / (k + rank)
            if doc_id not in fused_scores:
                fused_scores[doc_id] = 0
            fused_scores[doc_id] += 1 / (k + rank + 1)

    # Sort by fused score
    return sorted(
        fused_scores.items(),
        key=lambda x: x[1],
        reverse=True
    )

# Usage:
sparse_results = bm25_search(query)
dense_results = vector_search(query)

hybrid_results = reciprocal_rank_fusion([
    sparse_results,
    dense_results
])

Interactive Exercise

✎

Design Hybrid Strategy

You're building a RAG system for a customer support chatbot. How would you weight sparse vs dense retrieval for these query types?

1. "Order #12345 status"
2. "How do I return a damaged item?"

Pro Tips

Start with equal weights (0.5/0.5) and tune from there
Use query classification to dynamically adjust weights
RRF is robust and doesn't require score normalization
Consider late-interaction models for best of both worlds

Definition

Key Concepts

Examples

Interactive Exercise

Related Terms