Module 9 · Lesson 6 of 6
Tracing one request
One request, every step timed
A trace gives one request an ID and records every step under it, so a slow or costly request shows which step to blame.
Scene 1 of 5 · One ID per request
The request gets a trace ID, and each step records its time and tokens under it.
- Your app — starts the trace — healthy — trace 7f3a
- Document search — healthy
- Language model — healthy
- Order system — healthy
- One ID per request. The request gets a trace ID, and each step records its time and tokens under it.
- Search, then plan. Search takes 150 ms, and the first model call takes 1.0 s to choose a tool.
- Tool, then answer. The order lookup takes 300 ms, then writing the answer takes 5.9 s.
- Where the time goes. Of 7.4 s in all, writing the answer token by token takes the most.
- A slow request. On a slow day, the same trace shows search grew from 150 ms to 1.8 s.
- Now you try. Pick a request, and read its trace to find the step that grew. For example: Total 11.1 s, up from 7.4 s. "Tool: look up the order" grew by 3.7 s. The span that grew points at the part to fix, before anyone guesses.
Scene 1 of 5: One ID per request. The request gets a trace ID, and each step records its time and tokens under it.
Key points
- Every step of one request is recorded under the same trace ID.
- Each step, or span, keeps its time, tokens and cost.
- Put a slow trace beside a normal one to see which step grew.
- Writing the answer is often the longest span, one token at a time.
One AI request can touch several parts: a search, two model calls and a tool. When it is slow or costly, averages for each part cannot say which step was to blame this time.
A (The record of one request across every part it touched, kept under one ID.) gives the request one ID and records every step under it. Each step is a (One step inside a trace, with what ran, when it started and how long it took.): what ran, when it started, how long it took, and its (A piece of text that a language model reads or writes, often part of a word. Prices and limits are counted in tokens.) and cost.
To find a problem, compare a slow trace with a normal one, and look for the span that grew. Often the longest span is writing the answer, one token at a time.
Traces hold (Everything sent to a language model in one request: instructions, context and the question.) and answers, so they need the same care as logs: limit who can read them, hide private fields and delete old ones.
In system design: Use the same tracing as the rest of your services, and add the model, tokens and cost to every model span.