Module 8 · Lesson 4 of 4
When caching goes wrong
The right answer for the wrong person
A shared cache keyed only on the question can hand one customer's answer to another, and any cached fact can go stale.
Scene 1 of 4 · Customer A asks
user:42 asks "Where is my order?", and the model answers that order 1187 is out for delivery.
- user:42 — healthy — order 1187
- user:77 — healthy
- Your backend — healthy
- Shared cache — healthy — stores order 1187
- Language model — healthy
| Customer | Cache key | Gets | Check |
|---|---|---|---|
| user:42 (highlighted) | "Where is my order?" | order 1187 | their own |
- Customer A asks. user:42 asks "Where is my order?", and the model answers that order 1187 is out for delivery.
- Customer B asks. user:77 sends the same words, and a cache keyed on the question alone returns user:42's order.
- User in the key. With the user in the key, user:77 misses the cache and gets order 2240, their own.
- Facts change too. A cached price or policy goes stale when it changes, so each entry needs a TTL or a clear.
- Now you try. Choose the cache key for "Where is my order?", and see what user:77 gets. For example: user:77 gets order 1187, which belongs to user:42; the two requests make 1 model call. The key holds only the words, so user:77 gets the answer stored for user:42: wrong, and a privacy leak.
Scene 1 of 4: Customer A asks. user:42 asks "Where is my order?", and the model answers that order 1187 is out for delivery.
Key points
- The same words can need a different answer for each user.
- Put the user in the key, or keep the question out of the cache.
- Cached facts go stale when a price or policy changes.
- Give entries a TTL, or clear them when the facts change.
A cached answer goes stale when the facts behind it change, such as a price or a policy. So entries need a (How long a cached copy is kept before it is thrown away.), or must be cleared when the facts change.
A (A small, fast store that keeps copies of popular data, so most reads skip the slower database.) shared between users must never mix up personal data: one customer's order status is not an answer for another.
Keep personal or account questions out of shared caches, or include the user in the cache key as well as the question.
In system design: Decide for each feature what may be cached, for how long and for whom, before turning a cache on.