Token

Tokens are the smallest units a language model splits text into before it computes with it. Not words, not characters — fragments in between. Every bill, every limit and every context window is measured in tokens.

What a token is

A token is a frequently occurring fragment of characters. Short, common words often form a single token; longer or rarer words break into several. Spaces and punctuation count too.

As a planning rule of thumb, one English token is roughly four characters or about 0.75 words. The only reliable figure comes from the provider’s own tokenizer — every model splits differently.

Why German costs more than English

Tokenizers are trained predominantly on English text. German compounds, umlauts and inflectional endings therefore break into more fragments than their English equivalents. The same content costs noticeably more tokens in German than in English.

For operations this matters: anyone planning volume should calculate in the language the work actually happens in, not against English benchmarks.

Input, output and the price gap

Providers bill separately. Input tokens are everything going into the model: the instruction, the context, attached documents and, in chats, the conversation so far. Output tokens are the generated response.

Output typically costs a multiple of input. So the most expensive habit is not a long prompt but an unnecessarily verbose answer. How this turns into running cost is covered in the entry on LLM inference.

Context window: the second limit

The context window is the maximum number of tokens a model can process in one pass — input and output combined. Once exhausted, context falls away: long conversations lose their beginning, large documents lose sections.

A large context window is therefore no licence to fill it. Every token carried along is paid for and crowds out other content. Precise input beats complete input.

What this means for operations

Tokens are the only lever through which AI cost is directly controllable. Three of them work immediately:

• Limit the context. Attach only the documents that matter for the task. A whole folder as an attachment is expensive and makes answers worse, not better.
• Specify response length. Where an output should be structured, the format belongs in the instruction. Unbounded answers are the most common avoidable expense.
• Match the model to the task. Classification, extraction and routine work need no frontier model. The price gap between model classes is larger than most prompt-level optimizations.

How this becomes a systematic approach is described in Token-Smart. One principle applies throughout: clients keep their own AI accounts. We do not earn from their consumption — we help reduce it.

Last updated: August 2026