IT lexicon › AI & ML › Token cost

Token cost

AI & ML På svenska → Updated: 2026-07-28

The LLM pricing unit: 1 token ≈ 0.75 English words ≈ 4 bytes. Swedish costs more (token/word ratio ~1.5 vs English 0.75) because the BPE vocabulary is English-heavy.

Tokenizer matters: GPT-4 cl100k vs Claude vs Llama have different splits. "Hello world" = 2 tokens in GPT-4. "Hej världen" = 4 tokens. Code: one line of Python ~5-10 tokens. Non-Latin scripts (Japanese, Arabic) used to be much more expensive before cl100k_base — old GPT-3 davinci could cost 5x on Japanese text. Context window price: a 200K-context call isn't 200K-tokens price — you only pay for what you send. Caching (Anthropic, Google): cached prefixes 5-10x cheaper on re-read.

← Back to the lexicon