Explore

What one AI request costs, and why it's slow

Send one request with a meter running, then change one thing per step and watch the cost and the wait move. About 10 minutes.

Step 01 of 06

Tokens

Someone on a support team asks the company's policy assistant a question. That person is the user. The app is the software around the model: the chat window plus the code behind it, which builds what gets sent to the model and pays the bill. The model reads what the app sends and writes a reply. Both sides are counted in tokens, pieces of words (in English, a token is about three-quarters of a word). The app pays for every token it sends and every token the model writes back, and writing costs more: 5 times more per token on the price lists used here.

Your guess

Someone asks a one-line question and gets a five-paragraph answer. Which part costs more?

Make a guess to continue.

The answer is A. The answer has more tokens, and each token the model writes costs 5 times as much as each token it reads. Here, a 100-token question costs $0.0002 to read, and a 400-token answer costs $0.004 to write.

What can go wrong now

Long answers cost more and arrive more slowly, because the model writes one token at a time, and the user waits for every one of them.

How teams handle it

Ask for short answers in the app's instructions, and cap how many tokens a reply can use. More in Models and inference.

Next:The reply is only part of it. Everything the app sends counts too, and it's sent again with every request: The context.

Questions people ask

Who pays for the tokens: the user or the app?

The app, or rather the company that runs it: it has an account with the model provider and gets the bill. Whether the user pays anything is a product decision (a subscription, a usage fee or nothing). The tokens cost the same either way.

Why does output cost more than input?

The model reads all the input in one pass, but writes the output one token at a time, and each new token needs another pass through the model. Writing keeps the hardware busy for longer, so providers charge more for it.

Why is the first word slow but the rest fast?

Before the first word, the model reads everything the app sent and does all its thinking. After that, it writes the answer a token at a time, and the app shows each word as it arrives. So a big context and a lot of thinking show up as a long pause at the start.