The Complete Guide to AI Tokens: What They Are and How They Work
An AI token is the unit a language model actually reads and writes: a chunk of text, usually about four characters or three quarters of a word in English. Models never see words or letters, only tokens, and both AI pricing and context limits are counted in them.
Every AI platform bills in tokens and measures its limits in tokens, yet most explanations stop at "pieces of words". This guide covers what a token is, how your text turns into them, what they cost and how to spend fewer of them, with numbers from the Rewind.ai catalogue you can verify on the model pages.
What is an AI token?
Before a model processes your text, a tokenizer splits it into pieces from a fixed vocabulary. Common words survive as one piece; rarer ones get cut up. A few examples of how this plays out in English:
- "hello" is a single token.
- "extraordinary" splits into several, along the lines of "extra" + "ordinary".
- Punctuation, spaces and numbers all consume tokens of their own.
- A rough rule of thumb: one token per 4 characters, or 3 tokens per 4 words.
The ratio shifts by language. Chinese and Japanese text typically costs more tokens per character than English, so the same message can be cheaper or dearer depending on the language it is written in.
Why prices are quoted per token
AI services bill both directions: the prompt you send consumes input tokens and the reply consumes output tokens. That is why model prices read like utility rates, and why they vary enormously. Four models from the Rewind.ai catalogue make the spread visible:
| Model | Price per 1M input tokens |
|---|---|
| gpt-oss-20b (OpenAI) | $0.03 |
| Qwen2.5 7B Instruct (Alibaba) | $0.10 |
| GPT-5 (OpenAI) | $1.25 |
| Claude Sonnet 4.6 (Anthropic) | $3.00 |
The same thousand-word question costs a hundred times more on the last row than on the first. On Rewind.ai you get 2,500 free tokens a day without an account and 5,000 with a free one, so the practical question is not whether tokens cost money but how far your daily allowance stretches on the model you picked.
Context windows are token budgets
A model's context window is the total number of tokens it can hold in one conversation: every earlier message, your new prompt and the reply it is about to write, all counted together. Windows differ as much as prices do. In the Rewind.ai catalogue, Qwen2.5 7B holds 32,768 tokens while Claude Sonnet 4.6 holds 1,000,000, enough for a full-length book with room left over.
When a conversation outgrows the window, the oldest content stops fitting and the model loses track of it. That is the usual reason a long chat starts contradicting its own earlier answers: the beginning has literally fallen out of the budget.
What everyday operations cost in tokens
Different tools meter differently. On Rewind.ai the working numbers are:
- A chat exchange: typically 100 to 2,000 tokens, depending on message length.
- Image generation: a fixed amount per image, around 1,000 tokens.
- Text-to-speech: roughly 1 token per 4 characters of input.
- Transcription: about 4 tokens per second of audio.
Token costs show in real time as you use each tool, and the account dashboard breaks usage down by tool and by day.
How to make tokens go further
- Write focused prompts. A clear 50-token question routinely beats a rambling 500-token one, and costs a tenth as much.
- Extract before you paste. Sending the relevant two pages of a document instead of all forty keeps both the bill and the model's attention on the point.
- Match the model to the task. Simple rewrites do not need a $3.00-per-million model when a $0.03 one answers just as well.
- Split big jobs. Several small, specific requests usually cost fewer total tokens than one sprawling request that needs two follow-ups to fix.