Module 3 · Lesson 4 of 7
Reranking the best few
A careful second look
A reranker reads the question with each top result and rescores them, so the best few reach the prompt.
Scene 1 of 4 · A rough first search
A fast first search returns the top 50 chunks in 15 ms, with a sign-in tip at number 1.
- Help app — healthy
- First search — healthy
- Reranker — healthy
| Chunk | Was | Score | Now |
|---|---|---|---|
| Sign-in tip (highlighted) | #1 | ? | – |
| Depot delays | #2 | ? | – |
| Tracking updates | #3 | ? | – |
| Return labels | #4 | ? | – |
| Lost-parcel rule | #9 | ? | – |
- A rough first search. A fast first search returns the top 50 chunks in 15 ms, with a sign-in tip at number 1.
- Why it ranked first. It shares "stuck" and "for a week" with the question, but it is about the app, not parcels.
- Read each pair. The reranker reads the question with each chunk; all 50 take about 100 ms more.
- The best few. The lost-parcel rule climbs from number 9 to number 1; only the best 2 are sent on to the model.
- Now you try. Choose how many first-search results the reranker rescores. For example: Search and rerank take 25 ms, and the reranker never sees the lost-parcel rule. The first search put it at number 9, below the cut, so no amount of care can find it.
Scene 1 of 4: A rough first search. A fast first search returns the top 50 chunks in 15 ms, with a sign-in tip at number 1.
Key points
- The first search scores the question and each chunk apart, so it is rough.
- A reranker reads the question and a chunk together.
- It is slower, so it only rescores the top few dozen.
- A chunk the first search ranked too low to rescore is never seen.
The first search is fast but rough. A (A slower, more careful model that rescores the top results of a first search, so the best few go into the prompt.) is a second, slower model that reads the question and each top (A short passage cut from a longer document, so it can be turned into an embedding and found on its own.) together, and scores how well they match.
Because it is slow, it only rescores the top 20 to 100 chunks from the first search. Then the best few go into the (Everything sent to a language model in one request: instructions, context and the question.).
This two-step shape appears in most search systems: fast and rough over everything first, then slow and careful over a few.
Try the stuck parcel question above: word search pulls in a sign-in ticket. A reranker that reads the whole text would push it down.
In system design: Reranking raises answer quality but adds delay, so count it in your (How long a user waits from sending a request until the first piece of the answer appears.).