Someone builds a small internal tool that summarizes customer support tickets, tests it on a handful of examples, and it costs almost nothing. They roll it out to the whole team, usage climbs, and the monthly bill arrives at ten times what the pilot suggested. Nothing about the tool changed. What changed is that nobody had actually worked out which part of the bill scales the fastest, and it usually isn’t the part people assume.
AI pricing looks deceptively simple on a vendor’s homepage: a single price “per million tokens.” The reality has more moving parts, and the parts that matter most for your actual bill are rarely the ones highlighted in the marketing.
How Does AI Token Pricing Work?
AI providers charge per token processed, split into two separate rates: an input rate for the text you send, and a higher output rate for the text the model generates back. A token is a chunk of text roughly three-quarters of a word long, and total cost is calculated as input tokens multiplied by the input rate, plus output tokens multiplied by the output rate.
That basic formula is consistent across every major provider. What varies, and what actually determines your bill, is the size of the gap between the two rates, and a handful of secondary charges that rarely show up in the headline price.
Why Output Always Costs More Than Input
Across every major AI provider, generating text costs significantly more per token than reading it – commonly several times more. This isn’t a pricing quirk. Generating each token requires the model to run a full forward pass through its network for that individual token, while processing input text can be batched and computed more efficiently in parallel. The computational cost genuinely differs between reading and writing, and the pricing reflects that.
The practical consequence catches people off guard: a request with a short prompt and a long, detailed response can cost far more than a request with a long prompt and a short answer, even though the long-prompt request “feels” like it should be more expensive. If your AI tool tends to write long explanations, essays, or detailed code, that output volume – not the length of what you asked it – is usually driving your bill.
What Actually Makes Up Your Bill
A single request rarely breaks down into just “input” and “output.” A token itself is a chunk of text, not a word – OpenAI’s own documentation puts it at roughly four characters or 0.75 words of English – and providers, including OpenAI’s documentation on token counting, describe several distinct categories that can appear on a single API call:
| Token category | What it covers | Typical cost relative to standard input |
|---|---|---|
| Standard input | The prompt, system instructions, and any documents you send fresh | Baseline rate |
| Cached input | Input tokens the provider recognizes as repeated from a recent request | Often a fraction of the standard input rate |
| Output | The visible reply the model generates | Several times the input rate |
| Reasoning tokens | Internal reasoning steps some models generate before the visible answer | Billed at the same rate as regular output, even though you don’t see them |
Reasoning tokens are the least visible line item and one of the more common sources of an unexpectedly high bill. On models with an internal reasoning step, the token count you’re billed for can be meaningfully higher than the word count of the answer you actually see, because the model’s intermediate reasoning is billed as output even though it never appears in the response.
Why Prompt Caching Can Cut Costs Significantly
Most major providers now offer a discount for input tokens that repeat across requests – a system prompt, a long reference document, or a consistent set of instructions sent with every call. Instead of charging full price to reprocess identical text every time, the provider recognizes the repeated portion and charges a reduced rate for it.
This matters most for applications that send the same large context repeatedly with only a small part of the request actually changing – a customer support tool referencing the same product documentation on every ticket, for example. Structuring a workload so the repeated, stable content stays at the front of the request, with only the varying part appended at the end, is one of the more effective ways to reduce cost without changing what the tool actually does.
A Practical Way to Estimate Cost Before You Build
Before committing to a tool or workflow, it helps to estimate roughly where your usage will land rather than discovering it from a bill:
- Estimate your typical input length in tokens, using the rough guide of about 0.75 words per token.
- Estimate your typical output length separately, and weight it more heavily than input length, since output is priced higher everywhere.
- Check whether the task involves a reasoning-enabled model, and if so, expect the effective token count to run higher than the visible answer suggests.
- Check whether your workload repeats the same large context on every call, since that’s exactly the pattern prompt caching is built to discount.
- Always verify current rates directly on the provider’s own pricing page rather than a comparison chart, since these figures change often enough that even a recent article can be out of date within weeks.
Frequently Asked Questions
Why does output cost more than input for every AI provider? Generating text requires a full computational pass for each new token produced, while reading input text can be processed more efficiently in parallel. The cost difference reflects a genuine difference in computational work, not just a pricing choice.
What’s a token, exactly? A token is a chunk of text used internally by AI models, roughly equivalent to three-quarters of an English word. Common short words are usually a single token; longer or less common words often split into two or more.
Do I get charged for a model’s internal reasoning steps? On models that use an internal reasoning process before answering, yes – those reasoning tokens are typically billed at the same rate as visible output, even though you never see them in the response.
How much can prompt caching actually save? The specific discount varies by provider and changes over time, but cached input tokens are consistently priced well below fresh input tokens across the providers that offer the feature. The savings are largest for workloads that repeatedly send the same large block of context.
Is a model with a lower advertised price always cheaper to use? Not necessarily. A model with a low input price but a high output price can end up costing more than a competitor for an output-heavy task, and vice versa. The only reliable way to compare is to estimate your actual input-to-output ratio and apply both rates, rather than comparing a single headline number.
Where can I find current, accurate pricing for a specific model? Directly on the provider’s own pricing documentation. Third-party comparison pages are useful for understanding the mechanics, but exact figures change frequently enough that a provider’s own current page is the only reliably up-to-date source.
The Practical Takeaway
Token-based pricing is simple in structure and easy to misjudge in practice, because the two numbers that matter most – your output volume and whether reasoning tokens are involved – are usually the ones people estimate least carefully before they build something. Before scaling any AI-powered tool, work out your actual input-to-output ratio rather than trusting a single advertised rate. Cost efficiency is one of the five categories in our own UB Score methodology, precisely because a tool that performs well but prices unpredictably isn’t actually a good recommendation – see our full reviews for how that plays out model by model. For the related concept that also affects both cost and quality, see our guide to context windows, since a longer context directly increases the input side of your bill regardless of whether the model uses all of it.