Module 9 · Lesson 1 of 6
What one request costs
Two meters on every request
Each request pays one price for the tokens it sends and a higher one for the tokens the model writes.
Scene 1 of 4 · Two prices
At example prices, input costs $3 per million tokens and output costs $15: 5 times more.
- One request
- not shown yet
- 1,000,000 requests
- not shown yet
- Two prices. At example prices, input costs $3 per million tokens and output costs $15: 5 times more.
- Count the input. The 2,000 prompt tokens cost $0.006.
- Count the output. The 300 answer tokens cost $0.0045: close to the input, from far fewer tokens.
- Add them up. One request costs $0.0105; 1,000,000 requests a month cost about $10,500.
- Now you try. Change the prompt size; the answer stays at 300 tokens. For example: $0.0285 a request, 84% of it for the prompt; 1,000,000 requests cost about $28,500. Output costs more per token, but a long enough prompt still makes input the bigger part.
Scene 1 of 4: Two prices. At example prices, input costs $3 per million tokens and output costs $15: 5 times more.
Key points
- Input and output tokens have separate prices.
- Output often costs several times more per token.
- A long prompt can still cost more than the answer.
- Tiny sums per request add up across a month of traffic.
Model providers charge by the (A piece of text that a language model reads or writes, often part of a word. Prices and limits are counted in tokens.): one price for the tokens you send, and another, usually higher, for the tokens the model writes.
So one request costs its (Everything sent to a language model in one request: instructions, context and the question.) tokens times the input price, plus its answer tokens times the output price.
In system design: Long prompts, such as many retrieved (A short passage cut from a longer document, so it can be turned into an embedding and found on its own.), raise the cost of every request, so send only the context the answer needs.