Skip to content
anythingai.space
Retrieval & RAGintermediate12 min · 9 steps

Reranking Strategies

Retrieve wide and cheap, then let a cross-encoder read each candidate with the query and put the right chunk first.

NDCG0.65
retrieval75ms
rank by cosineBI-ENCODER SCOREDoc 1 · GPU memory via CUDA0.72Doc 2 · memory mgmt in CUDA0.75 · ranked first (wrong)Doc 3 · RAM allocation for ML0.71Query“configure CUDA memory”Bi-encoderquery, doc embedded apartTop candidatescosine similarity
requestdataheavy work
metrics are illustrative
1 / 9
ProblemStep 1 of 9

The first stage gets the right documents in the wrong order

A bi-encoder embeds the query and each document separately, then compares two fixed vectors. It is fast enough to search a billion docs, but it never sees how the query's words relate to the document's.

For “How to configure CUDA memory” the three candidates score 0.72, 0.75 and 0.71 — nearly tied — and the wrong one comes first.

Lesson 6 of 37 in Retrieval & RAG