Retrieval-Augmented Generation (RAG) has become essential for building intelligent systems that provide accurate, contextually relevant answers. However, the initial retrieval step often returns many candidates, not all of equal quality. This is where reranking comes in—and choosing the right technique can dramatically improve your results.
In this comprehensive guide, we’ll compare two major reranking approaches: bi-encoders and cross-encoders, and examine popular models like BGE-Reranker and Cohere’s Rerank to help you make an informed decision.
Understanding RAG Reranking
Before diving into the comparison, let’s clarify what reranking does in a RAG pipeline:
1. Initial Retrieval: A fast retriever (like BM25 or dense embeddings) returns top-K candidates
2. Reranking: A more sophisticated model re-scores these candidates to identify the most relevant ones
3. Context Selection: Top-reranked items are passed to the LLM for answer generation
This two-stage approach balances speed and accuracy—using fast retrievers at scale, then applying expensive, accurate models to a smaller subset.
Bi-Encoders: The Traditional Approach
How Bi-Encoders Work
Bi-encoders (or dual-encoders) encode the query and documents independently, computing embeddings separately:
Query: "What is machine learning?" → Query Embedding (768-dim)
Doc 1: "Machine learning is..." → Document Embedding (768-dim)
Doc 2: "Deep learning uses..." → Document Embedding (768-dim)
Relevance Score = Cosine Similarity(Query Emb, Doc Emb)
Advantages of Bi-Encoders
✅ Highly Scalable: Pre-compute document embeddings once, reuse infinitely
✅ Fast Inference: Simple similarity computation (dot product)
✅ Efficient Storage: Store embeddings in vector databases (e.g., Pinecone, Weaviate)
✅ Cost-Effective: Minimal computational overhead during inference
✅ Easy to Implement: Straightforward pipeline architecture
Disadvantages of Bi-Encoders
❌ Limited Context Understanding: No direct interaction between query and document
❌ Weaker Fine-tuning: Symmetric embeddings may miss nuanced relevance
❌ Representation Bottleneck: 768 dimensions may compress information loss
❌ Lower Accuracy: Generally underperforms on complex ranking tasks
Example: BGE-M3-Embedding
The BGE (BAAI General Embedding) M3 is a popular bi-encoder:
– Dense embedding dimension: 1024
– Supports multiple languages
– Trained on large-scale paired data
– Good for initial retrieval and lightweight reranking
Cross-Encoders: The Power Player
How Cross-Encoders Work
Cross-encoders process the query and document together through a single encoder, allowing direct interaction:
Input: "[CLS] What is machine learning? [SEP] Machine learning is... [SEP]"
↓
BERT/RoBERTa Encoder
↓
Classification Head
↓
Relevance Score (0-1)
Advantages of Cross-Encoders
✅ Superior Accuracy: Direct query-document interaction captures nuanced relevance
✅ Contextual Understanding: The model sees both inputs simultaneously
✅ Fine-tuning Friendly: Easier to adapt to specific domains
✅ Proven Performance: Consistently outperforms bi-encoders on benchmarks
✅ Detailed Scoring: Produces calibrated probability scores
Disadvantages of Cross-Encoders
❌ Computationally Expensive: Must encode each query-doc pair individually
❌ Slow Inference: Cannot pre-compute document embeddings
❌ Poor Scalability: Linear complexity with number of candidates
❌ Cannot Use Vector DB: Requires real-time computation
❌ Higher Latency: Not suitable for ranking thousands of items
Head-to-Head: BGE-Reranker vs Cohere Rerank
Let’s examine two leading production-ready rerankers:
BGE-Reranker-Large
Type: Cross-Encoder
Architecture: RoBERTa-Large based
Parameters: ~340M
Dimension: Output is relevance score (0-1)
Training Data: BAAI’s comprehensive ranking datasets
Multilingual: Yes (supports 100+ languages)
Performance Characteristics:
– NDCG@10 on MTEB: ~63-67%
– Inference time: ~50-100ms per query-doc pair
– Best for: General-purpose reranking
Strengths:
– Open-source and free
– Excellent multilingual support
– Comparable to commercial models
– Active community support
Limitations:
– Slower than lighter models
– Requires GPU for practical deployment
– Training data may have biases
from sentence_transformers import CrossEncoder
model = CrossEncoder('BAAI/bge-reranker-large')
pairs = [
['What is machine learning?', 'ML is a subset of AI...'],
['What is machine learning?', 'The weather is sunny...']
]
scores = model.predict(pairs)
# Output: [0.95, 0.12]
Cohere Rerank
Type: Proprietary Cross-Encoder
Architecture: Proprietary Cohere model
API-based: Yes (cloud-only)
Multilingual: Yes
Cost: Pay-per-request ($0.001 per document)
Performance Characteristics:
– NDCG scores: ~65-72% (top tier)
– API latency: 200-500ms (including network)
– Batch processing: Up to 100 documents per request
Strengths:
– State-of-the-art accuracy
– No infrastructure needed (cloud API)
– Excellent documentation and support
– Built-in batch processing
– Constantly improving models
Limitations:
– Proprietary (closed-source)
– Ongoing API costs
– Network latency overhead
– Privacy concerns for sensitive data
– Vendor lock-in
import cohere
co = cohere.Client(api_key="your-key")
results = co.rerank(
model="rerank-english-v2.0",
query="What is machine learning?",
documents=[
{"text": "Machine learning is a subset of AI..."},
{"text": "The weather is sunny today..."},
{"text": "Deep learning uses neural networks..."}
],
top_n=2
)
for result in results:
print(f"Index: {result.index}, Score: {result.relevance_score}")
Comparison Matrix
| Criteria | Bi-Encoder (BGE-M3) | Cross-Encoder (BGE-Reranker) | Cohere Rerank |
| Accuracy | ⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ |
| Speed | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐⭐ |
| Scalability | ⭐⭐⭐⭐⭐ | ⭐⭐ | ⭐⭐ |
| Cost | Free | Free | Paid |
| Ease of Use | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ |
| Privacy | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ |
| Multilingual | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ |
| Setup Complexity | Low | Low-Medium | Very Low |
Practical Implementation Comparison
Using BGE-Reranker (Open-Source)
from sentence_transformers import CrossEncoder
import time
# Initialize model
model = CrossEncoder('BAAI/bge-reranker-large', device='cuda')
# Initial retrieval results (example)
query = "How does photosynthesis work?"
candidates = [
"Photosynthesis is the process where plants convert light energy...",
"The color of leaves is determined by chlorophyll levels...",
"Photosynthesis occurs in the chloroplasts of plant cells...",
]
# Create pairs
pairs = [[query, doc] for doc in candidates]
# Rerank
start = time.time()
scores = model.predict(pairs)
elapsed = time.time() - start
# Sort by score
ranked = sorted(zip(candidates, scores), key=lambda x: x[1], reverse=True)
print(f"Reranking took {elapsed:.3f}s")
for i, (doc, score) in enumerate(ranked, 1):
print(f"{i}. Score: {score:.4f} - {doc[:60]}...")
Using Cohere Rerank (API-Based)
import cohere
co = cohere.Client(api_key="your-api-key")
query = "How does photosynthesis work?"
documents = [
"Photosynthesis is the process where plants convert light energy...",
"The color of leaves is determined by chlorophyll levels...",
"Photosynthesis occurs in the chloroplasts of plant cells...",
]
# Rerank via API
results = co.rerank(
model="rerank-english-v2.0",
query=query,
documents=[{"text": doc} for doc in documents],
top_n=3
)
# Results are already sorted by relevance
for result in results:
doc = documents[result.index]
print(f"Score: {result.relevance_score:.4f} - {doc[:60]}...")
When to Use Each Approach
Choose Bi-Encoders When:
– You need to rank thousands of documents per query
– Latency is critical (real-time applications)
– You’re operating on limited infrastructure
– You need cost-effective solutions
– Pre-computing embeddings is feasible
– Slight accuracy trade-offs are acceptable
Example: E-commerce search with 1M+ products
Choose Cross-Encoders When:
– Accuracy is paramount (customer-facing, critical decisions)
– Ranking small to medium candidate sets (< 1000)
– You have GPU infrastructure available
– You can afford slightly higher latency (200-500ms)
– Your domain needs fine-tuning capability
– You need calibrated confidence scores
Example: Legal document analysis, medical question answering
Choose API-Based (Cohere) When:
– You don’t want to manage infrastructure
– You prefer always-updated models
– Privacy concerns are minimal
– You’re willing to pay per request
– You need enterprise-grade SLAs
– Vendor lock-in is acceptable
Example: Startup MVP, enterprise applications with budget
Hybrid Approach: Best of Both Worlds
Many production systems use a two-stage pipeline:
Retrieval Stage:
User Query → BM25/Vector Search → Top 100 Candidates
Reranking Stage:
Top 100 → BGE-Reranker (Cross-Encoder) → Top 10
LLM Stage:
Top 10 Documents → LLM → Final Answer
Why this works:
– Fast retrieval with bi-encoders or BM25 (handles scale)
– Accurate reranking with cross-encoders (improves quality)
– Balanced latency: ~100ms retrieval + ~500ms reranking = ~600ms total
– Cost-effective: Only rerank top candidates
Advanced Reranking Techniques
Beyond basic bi-encoders and cross-encoders, consider these innovations:
1. Colbert (Contextualized Late Interaction)
– Hybrid approach using late interaction
– Better than bi-encoders, faster than cross-encoders
– Models: `colbert-ir/colbertv2.0`
2. Learning to Rank (LTR)
– Combines multiple signals (BM25 score, embedding similarity, length, etc.)
– XGBoost-based reranking
– Most flexible approach
3. Multi-Vector Reranking
– Multiple embeddings per document
– Better representation of complex content
– Higher computational cost
Benchmarking Results
Here’s how models typically perform on MTEB (Massive Text Embedding Benchmark):
Reranking Benchmarks (NDCG@10):
Cohere Rerank 3.0 | ████████████████ 71%
BGE-Reranker-Large | ████████████████ 67%
BGE-Reranker-Base | ██████████████ 65%
jina-reranker-v1 | ███████████████ 66%
UPR (Universal Passage Ranker) | ██████████████ 64%
BAAI/bge-large-en-v1.5 (bi-encoder) | ████████████ 54%
Performance Optimization Tips
For BGE-Reranker:
# Batch processing for efficiency
from sentence_transformers import CrossEncoder
model = CrossEncoder('BAAI/bge-reranker-large', device='cuda')
pairs = [[query, doc] for doc in large_doc_list]
# Process in batches
scores = model.predict(pairs, batch_size=32, show_progress_bar=True)
# Use mixed precision for speed
model.half() # FP16 inference
For Cohere API:
# Batch multiple queries
queries_and_docs = [
{"query": q, "documents": docs_for_q}
for q, docs_for_q in zip(queries, all_docs)
]
# Process with concurrency
import asyncio
async def rerank_batch(co, requests):
results = await asyncio.gather(*[
asyncio.to_thread(co.rerank, **req)
for req in requests
])
return results
Future Trends in Reranking
1. Multimodal Reranking: Text + image + video ranking
2. Query-Specific Fine-tuning: Adapting models to specific domains in real-time
3. Efficient Transformers: Smaller models with comparable performance (DistilBERT-based rerankers)
4. Retrieval-Reranking Co-optimization: End-to-end learning of both stages
5. Generative Reranking: LLMs as rankers (promising but slower)
Conclusion
The choice between bi-encoders and cross-encoders isn’t binary—it depends on your specific constraints:
– BGE-Reranker offers the best open-source cross-encoder alternative, with excellent multilingual support and no ongoing costs
– Cohere Rerank provides unmatched accuracy and convenience if API costs fit your budget
– Bi-encoders remain ideal for large-scale retrieval and initial candidate selection
For most production RAG systems, we recommend:
1. Start with a fast bi-encoder retriever for initial candidate selection
2. Use BGE-Reranker or Cohere to rerank top-100 candidates
3. Monitor metrics and iterate based on user feedback
This balanced approach gives you the speed of bi-encoders and accuracy of cross-encoders—the best of both worlds.
Have you implemented RAG reranking in your projects? Share your experiences in the comments below!
Leave a Reply