Retrieval & RAGintermediate12 min · 9 steps
Ranking Beyond Relevance
Relevance scores aren't the whole ranking. Add domain rules, recency, field weights, cluster diversity and safety labels — and measure every change with recall, precision, MRR and nDCG.
nDCG@50.41
stale in top-52
datasignal
metrics are illustrativeProblemStep 1 of 9
The most similar chunks are not the best five
Ranked by relevance alone, the top 5 for “what is our travel expense limit?” are: three near-identical paragraphs from one page, a 2021 policy that was replaced, and an old email thread that outranks the official handbook.
Every one of them is similar to the query. The ranking is still wrong, because relevance ignores authority, time, structure, diversity and safety.
Lesson 27 of 37 in Retrieval & RAG