Module 8 · Lesson 1 of 4
Exact-match caches
Same prompt, same answer
An exact-match cache answers a repeated prompt from storage, but only when every character is the same.
Scene 1 of 4 · The first request
The first "What are your opening hours?" misses the cache, so the model answers and the answer is stored.
- Help page — healthy
- Your backend — healthy
- Answer cache — healthy — 1 answer stored
- Language model — healthy
| Prompt | Cache |
|---|---|
| What are your opening hours? (highlighted) | miss: the model answers, the answer is stored |
- Requests today
- 1
- Model calls
- 1
- The first request. The first "What are your opening hours?" misses the cache, so the model answers and the answer is stored.
- The same again. The other 4,999 identical requests that day come straight from the cache, with no model call.
- One word more. Adding one word, "today", makes a different prompt, so it misses and calls the model.
- Refresh with a TTL. Each answer expires 24 hours after it was stored; the next request after that gets a fresh one.
- Now you try. The cache holds the answer to "What are your opening hours?". Send a new prompt. For example: Cache miss: the backend calls the model. The meaning is the same, but the characters differ, and an exact cache compares characters only.
Scene 1 of 4: The first request. The first "What are your opening hours?" misses the cache, so the model answers and the answer is stored.
Key points
- The cache key is the whole prompt, character for character.
- A repeat answers from storage, with no model call.
- One extra word, a capital letter or a date makes a new prompt.
- A TTL makes old answers expire, so they refresh.
The simplest (A small, fast store that keeps copies of popular data, so most reads skip the slower database.) stores each (Everything sent to a language model in one request: instructions, context and the question.) and its answer. If exactly the same prompt arrives again, the answer comes from the cache, with no model call.
It cannot match the wrong question, since the prompt must be identical. It can still serve an old answer, so each entry needs a time limit. Hits are also rarer than you might hope: one extra word or a changed date makes a new prompt.
In system design: Use an exact cache for repeated, fixed prompts, such as a daily summary, and give each entry a (How long a cached copy is kept before it is thrown away.) so it refreshes.