Does a reranker actually help on top of my vector search?

A reranker (Cohere Rerank, Voyage rerank-2, BGE reranker) re-orders the top 20 or 50 chunks from vector search using a finer scoring model. For RAG with a few thousand documents it can make a real difference, but it is not magic and it adds latency and tokens.

Try this first

1. 1First measure your baseline: how often is the right answer in top-3 without reranker? At 90 percent recall, a reranker is unnecessary. At 60 percent, gains are likely.
2. 2Test Cohere Rerank 3 or Voyage rerank-2 on your eval set. Same top-50 from vector search, then check top-3 after rerank. If you see 15 to 25 percent more correct answers, it is worth it.
3. 3Account for cost and latency: a rerank step adds 100 to 500 ms per query plus an API call. For an internal tool that is fine, for a live customer chatbot it matters.
4. 4Combine smartly: top-3 from vector alone is cheaper, with reranker take top-50 then top-5. Do not over-engineer both.
5. 5Pin the reranker version like the embedding model. Models get deprecated.

None of the above fits?

Describe your situation below. We pass your input plus the steps you already saw to our AI and return tailored next-step advice. If it's too risky to DIY, we'll say so.