Module 1 · Lesson 4 of 6
Output your code can check
Three checks before your code acts
A reply meant for code goes through three checks: does it parse, does it match the schema, and are its facts allowed.
Scene 1 of 5 · The model replies
Asked to sort an email, the model sends back text meant to hold three named fields.
- Model reply — healthy
- Parse — healthy
- Shape check — healthy
- Fact check — healthy
- Your code acts — healthy
| Step | This reply |
|---|---|
| The reply (highlighted) | {"intent":"refund","order":"A1042","urgent":false} |
| Parse the text | Pass |
| Check the shape | Pass |
| Check the facts | Pass |
- The model replies. Asked to sort an email, the model sends back text meant to hold three named fields.
- Parse the text. Code turns the text into data, and anything that is not valid JSON stops right here.
- Check the shape. Each field must be there, have the right type and use a value from the allowed list.
- Check the facts. Order A1042 belongs to this customer, which your records know and no schema can.
- Only then act. With all three passed, your code opens the refund; any failure sends it back or to a person.
- Now you try. Pick a reply the model sends, and see which check stops it. For example: Stopped at the parse check: the text is not valid JSON. Text around the JSON breaks parsing. Your code asks once more, quoting the error.
Scene 1 of 5: The model replies. Asked to sort an email, the model sends back text meant to hold three named fields.
Key points
- Parse first: a sentence wrapped around the JSON already fails.
- Then check every field, its type and its allowed values.
- Then check facts only your own data knows, such as who owns an order.
- Act only on a reply that passes all three checks.
Often your code needs data from the model, not prose: a category, an order number, a yes or no. So the (Everything sent to a language model in one request: instructions, context and the question.) asks for (A plain-text format for data, such as {"id": 42, "name": "Asha"}. Most web APIs send and receive it.) with named fields, and lists the values each field may take.
A model can still return broken JSON, wrap it in a sentence, leave a field out or invent a value. Some APIs can force the reply to match a (A description of the fields some data must have, with their types and the values they may take.) you give. That fixes most format problems, but a reply cut off by the (A piece of text that a language model reads or writes, often part of a word. Prices and limits are counted in tokens.) limit is still incomplete.
A reply in the right shape can still be wrong, such as a real-looking order number that belongs to someone else. So code checks the values and the permissions before it acts.
When a check fails, ask once more with the error included. If that fails too, fall back, such as passing the email to a person.
In system design: Treat model output like user input: parse it, check it against a schema, and check permissions before any code acts on it.