Module 4 · Lesson 2 of 5
Retrying safely
One retry, then plan B
Retry a brief error once after a short, jittered wait, then fall back; do not repeat a step that changes something without checking first.
Scene 1 of 4 · A brief error
The call fails after 0.3 s with "busy, try again": a brief error that a retry often fixes.
One request
- Call or retry at 0 s, call
- "Busy" error at 0.3 s, busy
- A brief error. The call fails after 0.3 s with "busy, try again": a brief error that a retry often fixes.
- Wait, then retry once. The app waits about 1 s, retries once, and gets "busy" again.
- Then fall back. It stops retrying and moves on to its fallback, instead of keeping the user waiting longer.
- Spread the retries. Without jitter, every client retries at the same moment; with jitter, the retries spread out.
- Now you try. A call failed. Pick what went wrong, and see whether to retry. For example: Do not retry blindly. The email may already have gone, so a blind retry could send it twice; check first.
Scene 1 of 4: A brief error. The call fails after 0.3 s with "busy, try again": a brief error that a retry often fixes.
Key points
- A "busy" error is often gone a second later.
- Retry it once or twice, with backoff, then fall back.
- Jitter spreads many clients' retries apart.
- A step that sends or books something must never be retried blindly.
Many model errors are brief, such as a provider saying it is busy, so one retry with (Waiting longer after each failed attempt before trying again.) often works. The Retries lab shows backoff and (Randomness in how long each client waits before a retry, up to the backoff wait, so clients do not all retry at the same moment.) at work.
But a retry after a (The time a caller waits before giving up on a call and treating it as failed.) may pay for the (A piece of text that a language model reads or writes, often part of a word. Prices and limits are counted in tokens.) again, and retrying an action, such as sending an email, may do it twice. Retry only what is safe to repeat.
Cap the retries, and spread them with jitter, so many clients do not hit a struggling provider at the same moment.
Compare retry policies during an outage and a traffic spike, and measure the extra load each one creates.
Open the lab →In system design: Retry brief errors once or twice with backoff and jitter, and never blindly retry a step that changes something.