Retrieval & RAGadvanced12 min · 8 steps
Late Interaction & ColBERT
Keep one small vector per token and match token to token at query time — near cross-encoder accuracy with documents still encoded offline.
vectors / doc1
datarequest
metrics are illustrativeProblemStep 1 of 8
Fast and blurry, or sharp and slow
A bi-encoder compresses a whole document into one vector ahead of time: fast, but a rare term like “Widget-X” gets averaged away. A cross-encoder reads query and document together: most accurate, but nothing can be precomputed, so it only works for reranking a small set.
The difference is when query and document interact: never (bi-encoder), fully (cross-encoder) — or late: encode separately, then interact cheaply token by token.
Lesson 11 of 37 in Retrieval & RAG