Module 0 · Lesson 10 of 10
RAG: look it up, then answer
- FindSearch your own documents by meaning for passages that match the question.
- Build the promptPut the instructions, those passages and the question together.
- AnswerThe model writes its answer, told to use only those passages.
Runs for every question a user asks.
On: the search returns 5 chunks, and a second check picks which to keep.
- User — healthy — NOW
- Application — runs each step — healthy
- Language model — writes the answer — healthy
- Embedding model — text to vector — healthy
- Vector index — chunks and vectors — healthy
- Reranker — turned off — IDLE
- Request
- Search
- Response
Step 1 of 6 · User
Done: The user asks a question
- What happens
- The user sends a question to the application.
- Why
- The model was not trained on this shop's help pages. The application has to look the answer up first.
- What changes
- The application now holds the question: "Can I return a sale item after 20 days?"
- Next
- Turn the question into a vector
Event log
5 more to go. Press a line to go back to that step.
The chunks found, the prompt and the answer show here as the run reaches them.
Step 1 of 6: User. The user asks a question. The user sends a question to the application.
Things to try
- Keep 1 passage for the sale question. The closest chunk is the 30-day rule, so the answer is wrong.
- Now turn the reranker on. The 14-day chunk moves to place 1 and the answer is right again.
- Ask about price matching. The pages do not cover it, yet the search still returns its closest chunks.
- Set the chunk size to 4 sentences. Whole pages go into the prompt, and its size grows.
- Ask about Canada with chunks of 2 sentences and keep 1. The Canada sentence shares a chunk with the address rule, which pulls that chunk away. The chunk that says 3 to 5 days wins, so the answer is wrong.
- Now keep 2 chunks, or turn the reranker on. The Canada chunk reaches the prompt and the answer is right.
What this simplifies. The help pages, the map places and the answers are made up for teaching. A real embedding has hundreds of numbers, worked out by a model. Here a chunk's place is the average of its sentences' places, which were set by hand. The reranker uses grades set by hand, where a real one is a second model. The answer follows a fixed rule: it is right when the passage with the answer is in the prompt. A real model usually follows its passages, but not always. Token counts come from this track's teaching splitter.
(Looking up relevant documents first and putting them in the prompt, so the model answers from them instead of from memory alone.), or RAG, answers questions from your own documents instead of from the model's memory alone. It works in three steps: find, build the (Everything sent to a language model in one request: instructions, context and the question.), and answer.
First, the app searches your documents by meaning for the passages that match the question, using the (A list of numbers that stands for the meaning of a piece of text. Texts with similar meanings get similar lists.) from the last two lessons. Only the closest few are kept.
Second, the app builds a prompt from the instructions, those passages and the question. Third, the model writes its answer, told to use only those passages. It usually does, but not always, so answers still need checks.
RAG is the most common way to give a model your own data. It needs no retraining, and an answer can point back to the real passages it was built from. The answer is only as good as the search: given the wrong passage, the model answers from that.
In system design: RAG adds document search before every model call. Cutting text into (A short passage cut from a longer document, so it can be turned into an embedding and found on its own.) (short passages), embeddings, a (A database that stores vectors and finds the ones closest to a given vector, so you can search by meaning.) and prompt limits all become part of your system's design.