A reranker is a machine learning model that refines the ranking of a small number of candidate items (typically 10–1,000) produced by an efficient first-stage retrieval system. In modern retrieval-augmented generation (RAG) pipelines and search systems, the retriever (e.g., BM25, dense retrieval like DPR, or a bi-encoder) quickly narrows the corpus to a manageable candidate set, trading recall for speed. The reranker then applies a more computationally expensive but more accurate scoring function—often a cross-encoder based on a Transformer architecture—to reorder these candidates by relevance to the query.
Technically, a cross-encoder reranker concatenates the query and each candidate document into a single input sequence, separated by a special token (e.g., [SEP]), and feeds it through a full Transformer model. The output [CLS] token representation is passed through a linear layer to produce a relevance score between 0 and 1. Unlike bi-encoders that encode query and document independently (allowing pre-computed document embeddings but losing interaction), cross-encoders model rich query-document interactions via self-attention across the entire sequence. This yields higher accuracy, especially for complex queries where term overlap, synonymy, and semantic nuance matter. For example, a cross-encoder can distinguish "apple fruit" from "Apple company" by attending to the full context.
Why it matters: Rerankers dramatically improve top-k precision in retrieval systems. In the 2020 MS MARCO passage ranking leaderboard, cross-encoder rerankers like those from the MonoT5 family (based on T5) achieved MRR@10 scores above 0.40, while the best bi-encoders were around 0.35. In RAG, reranking the top 20 retrieved passages before feeding them to a generator (e.g., GPT-4, Llama 3.1) has been shown to reduce hallucinations by up to 30% in knowledge-intensive tasks (as reported in the 2024 KILT benchmark). Rerankers are also used in enterprise search, legal document retrieval, and question answering.
When used vs alternatives: Rerankers are the go-to when accuracy is paramount and the candidate set is small enough for the latency budget. For real-time systems (e.g., web search with sub-100ms targets), they may be replaced by late-interaction models like ColBERTv2, which approximate cross-attention via token-level embeddings and efficient scoring. Alternatively, distilled rerankers (e.g., MiniLM-based cross-encoders from the sentence-transformers library) offer a middle ground, trading some accuracy for latency. In 2025–2026, the state of the art includes large reranker models fine-tuned with contrastive learning on diverse relevance datasets (e.g., the BGE-Reranker-v2 series from BAAI, Cohere's rerank v3). These models often use LoRA adapters for efficient fine-tuning and can be deployed on a single GPU for candidate sets up to 1,000.
Common pitfalls: (1) Using a reranker on the entire corpus—defeats the purpose, as cross-encoder inference on millions of documents is prohibitively slow. (2) Overfitting to training distribution—rerankers trained on one domain (e.g., web search) may perform poorly on specialized domains (e.g., biomedical). (3) Ignoring score calibration—reranker scores are not probabilities and may not be comparable across queries. (4) Not updating the retriever—if the retriever misses relevant documents entirely, the reranker cannot recover them. (5) Latency underestimation—for real-time applications, the reranker's inference time must be carefully budgeted; batching candidates per query helps but adds complexity.
Current state of the art (2026): The most advanced rerankers are cross-encoders based on 7B-parameter models (e.g., Qwen2.5-7B reranker variants) that have been fine-tuned with pairwise and listwise ranking losses. They incorporate techniques from reinforcement learning with human feedback (RLHF) to align relevance judgments with user preferences. The top entries on the BEIR benchmark (e.g., the Cohere rerank v3 model) achieve NDCG@10 scores above 0.60 on average across 18 datasets, compared to ~0.50 for the best bi-encoders. Efficient inference is achieved via FlashAttention-2, 4-bit quantization, and custom CUDA kernels, enabling sub-50ms reranking of 100 candidates on an A100 GPU. Research is also exploring hybrid rerankers that combine lexical (e.g., SPLADE) and semantic signals, and multi-stage cascades where multiple rerankers are applied sequentially with increasing model size and decreasing candidate set size.