Cost per token
Cost per token is what a language model charges for each token it reads or writes, usually quoted per million tokens, with input, cached input and output tokens priced separately. Output tokens typically cost several times more than input, so the mix of token types, not just the volume, decides the bill.
In plain English
Think of a print shop that charges a little to read your document and a lot more for every page it prints. Bring the same reference pack every day, and it gives you a discount for keeping a copy on file.
Formula and worked example
Blended cost per 1M tokens = token cost ÷ tokens processed × 1,000,000
- Input tokens
- = tokens sent to the model: instructions, context and conversation history
- Output tokens
- = tokens the model generates, usually priced several times higher than input
- Token cost
- = each type of token × its price, added up, with cached input at its lower rate
- Tokens processed
- = input tokens + output tokens
| Cached input: 300M tokens at $0.20 per 1M | $60 |
|---|---|
| Other input: 100M tokens at $2 per 1M | $200 |
| Output: 40M tokens at $10 per 1M | $400 |
| Token cost for March | $660 |
| Blended cost per 1M tokens: $660 ÷ 440M tokens | $1.50 |
Output is 9% of the tokens but 61% of the cost, and caching saves $540 against paying full price for every input token. Per conversation, the assistant costs a third of a cent: the number the product team plans with.
Why cost per token matters
AI spend doesn't follow the cloud's usual units. There are no instances to rightsize; the bill is tokens × price, and both move with every prompt change, model upgrade and new feature. Cost per token is the rate side of that equation, and it varies more than most teams expect: between models, between token types, and between providers selling the same model.
On Anthropic's API, for example, output tokens cost five times input on current models, cache reads cost about a tenth of the input price or less, and the Message Batches API halves the price of work that can wait. Amazon Bedrock, Azure and Google Cloud have their own batch, caching and provisioned-capacity prices.
Common mistakes
- Comparing models on input price alone. Output is usually the bigger line. Compare models on your real mix of input, cached and output tokens.
- Breaking the cache. Caching works on an identical prompt prefix, so a timestamp or a reordered tool list at the start makes every request pay full input price. Track cache hit rate next to cost.
- Watching price per token, not cost per task. A cheaper model that needs more retries or longer answers can cost more per resolved conversation. Measure cost per completed task.
- Assuming tokens are words. Tokenizers differ between models, so the same text can be a different number of tokens after a model switch. Re-measure when you change models.
Where to find it in AWS, Azure and Google Cloud
| Cloud | What it is called | Where to find it |
|---|---|---|
 AWS | Amazon Bedrock | Priced per input and output token for each model, with batch inference at about half the on-demand price for select models, discounted cached input with prompt caching, and Provisioned Throughput billed by the hour for reserved capacity. |
 Azure | Azure OpenAI in Microsoft Foundry | Standard deployments are priced per token, and batch at a discounted per-token rate. Provisioned throughput units (PTUs) are billed hourly for each unit deployed, whether or not they serve requests, with reservations for a 1-month or 1-year commitment. |
 Google Cloud | Gemini API and Vertex AI | Priced per million input and output tokens, with a higher tier for very long prompts. Cached tokens cost 10% of the input price on Gemini 2.5 and later, batch runs at half the standard rate, and Provisioned Throughput covers steady workloads. |
How CloudYali helps
CloudYali's Price per 1M Tokens views show what each model actually costs you per million tokens on Anthropic, Amazon Bedrock, Gemini and Cursor, with cost by token type and cache hit rate by model, so a model switch or a broken cache shows up in the trend.
Related terms
AI, data & SaaS costs
AI cost tracking
AI cost tracking is collecting what an organization spends on AI, from model APIs such as Anthropic and OpenAI, cloud AI services such as Amazon Bedrock, and AI tools such as coding assistants, in one place, and attributing it to the teams, products and models behind it. It applies FinOps to AI spend.
FinOps foundations
Unit economics
Unit economics is measuring cloud cost per unit of business value, such as cost per order, per customer or per API call, by dividing what a product costs by how much it delivered. It shows whether spend is growing because the business is growing or because the product is becoming less efficient.
Protocols & tooling
Model Context Protocol (MCP)
The Model Context Protocol (MCP) is an open standard for connecting AI applications to external tools and data. An MCP server exposes tools, resources and prompts over JSON-RPC, and any MCP client, such as Claude, ChatGPT or Cursor, can use them, so one integration works across many AI assistants.
Billing & cost types
On-demand price
On-demand price is the public, pay-as-you-go price for cloud usage with no commitment: you pay per second, hour, gigabyte or request for what you use. It is the baseline that every commitment discount, and metrics such as coverage and effective savings rate, are measured against.
Governance & planning
Cost anomaly
A cost anomaly is spend well outside what a service, account or team normally costs, such as a day costing several times its recent average. Anomaly detection compares each day with a baseline learned from history and alerts on the difference, so a runaway job is caught in days, not at the end of the month.
Frequently asked questions
Why are output tokens more expensive than input tokens?
Generating a token costs a provider more than reading one, so output is priced higher, often around five times input. Long answers and reasoning-heavy work therefore dominate many AI bills.
What are cached tokens?
Input the provider has kept from an earlier request with the same prefix, such as a long system prompt. Reading it from the cache costs a fraction of the normal input price; writing it can cost a little more.
How do I lower cost per token?
Cache stable prompt prefixes, send work that can wait through a batch API at about half price, use the smallest model that holds quality, and cut unneeded output. Then check cost per task, not just per token.
