It’s easy to hear people talking about “tokens” in a generative AI context and infer that they are like a weird new kind of money. After all, the kind of tokens used in cryptocurrencies were like a weird new kind of money, and those tokens were tangled up with GPUs, energy usage, data centers, economic exploitation, and so on, too. But I want you to think about generative AI tokens as words, a unit of measure for text, not a holder of value. This shift matters because people already know how to reason about reading, writing, and rereading. Treating tokens primarily as an accounting unit distracts from conversations we could be having about literacy, turning them into debates about the value of content subscriptions.
Some part of me needs to clarify up front that the tokens processed by LLMs aren’t literally words, just word-like. I think you should accept this, not just so we can get on with the blog post, but because even literal English words are only a bit like words in other languages anyway. A linguist might step in to distinguish lexemes from morphemes from graphemes, but let’s just say that sometimes the natural chunks of text don’t align with apparent word boundaries. You can decompose “precaution” into “pre-” and “caution” or acknowledge that “put up” (as in “to put up with”) isn’t just putting something in an upward direction.
For now, let’s assume we have enough of a shared sense of words to talk about word counts. I can talk about how I read a twelve-thousand-word research paper or how I need to write a new paper’s abstract in no more than 250 words. Already, counts of words read versus written are distinct. Further, reading twelve thousand words of unfamiliar text is quite a different task from reading twelve thousand words of familiar text, and maybe the text is familiar because I just wrote it myself or at recently reviewed it for another purpose. These intuitive distinctions between kinds of word counts will come in handy in just a moment.
To get an LLM hosted on a commercial inference service to generate text for you, you need to pay. So what does a token cost? Let’s look up the pricing for a currently leading open-weight LLM, Kimi K3, on the OpenRouter inference marketplace. As I write this, there’s a headline price of $13 per million tokens. So, that’s it, right? Wait, this page is covered with slightly different prices, all measured in the same dollars-per-million-token units, and the numbers are changing over time. Some of these prices seem to distinguish “input” tokens from “output” tokens and even from “cached read” input tokens. Let’s map it out using the idea of tokens as words.
$13/M is the current output-token price for the cheapest provider of K3. This is like saying there are many writers for hire, all writing in the same style, but maybe they write at different speeds and are more or less busy with work from other clients. The tables show that different K3 providers have different recent throughput rates (some have recently been generating about 10 tokens per second, whereas others have been running around 70 tokens per second). At the same time, the tables show different uptime percentages. This tells us how often the different providers are turning customers away because they are unavailable or overloaded. These writers-for-hire are moody! It’s not always the case that the more expensive providers are the fastest or most reliable ones, so it is justifiable that OpenRouter takes a cut to route requests among providers according to preferences for price, throughput, latency, and availability.
Okay, we’ve figured out how to map the idea of written words onto output tokens, and we’ve accepted that prices can vary because there are many different writing teams out there. But what about reading? We haven’t even explained half the prices on that page.
The next headline number on the OpenRouter K3 page is the “in” or “input token” cost, currently $2.60/M. This tells us something about the cost for models to process input text before deciding how they are going to generate the outputs. But what even are the inputs when you are using a chatbot or coding agent?
Imagine you are ten turns deep into a conversation with a chatbot. You’ve just sent your latest question, and you want the harnessed model (an agent) to reply. To generate the reply, your harness sends your ten previous user messages, the ten previous agent replies, and your most recent user message. This is the text the model needs to read before writing a reply. Even though reading is generally easier than writing (and this is reflected in the prices we saw, $2.60/M in versus $13/M out), keeping a conversation going requires a lot of reading. In particular, it requires a lot of rereading text that should be familiar, assuming our request is routed to the same provider endpoint that handled the earlier responses. Familiar reading has a different market price from unfamiliar reading, and for K3 that’s currently about $0.30/M (varying by provider, of course).
In transformer-architecture LLMs, this notion of familiarity with text has a technical interpretation. In most LLMs, the in-context meaning of each word (sorry, token) is influenced by all of the words that came before it in the text. Consider “the cat sat on the mat.” Specifically, consider being partway through reading it: “the cat sat on the _.” The second “the” isn’t just an isolated definite article; it’s the definite article of the as-yet-unspecified place where the cat is sitting. This contextualized, thickened representation is part of why the model is more likely to say “the cat sat on the mat” than “the cat sat on the cat,” even if “the cat” was more common than “the mat” in the model’s training data. It requires compute to build up these thickened representations, so we’d like to save them to avoid recomputing them for each new request that shares an identical input text prefix. At the same time, these thickened representations take extra storage space to keep around, so providers won’t keep them around forever. Providers give you a discount on reading text that is familiar in the technical sense that its previously computed representation remains available through prompt caching.
At this point, we have three different dollars-per-million-token prices for a specific model from a specific provider at a specific point in time in a competitive marketplace. We can trace these to the distinct efforts needed to write versus read, or to read familiar versus unfamiliar text, all measured in word counts. But we’re not done interpreting that OpenRouter page! If you click the “Pricing” tab, you get yet another price: “Weighted Avg Input Price,” currently $0.79/M (somewhere between the cached and uncached input prices). What’s this?
People use different models for different purposes, so the typical ratio of cached to uncached inputs (of requests containing familiar text to those containing unfamiliar text) can vary. Further, providers with different amounts of cache storage or different cache-eviction (forgetting? defamiliarizing?) strategies can influence this balance. “Weighted Avg Input Price” is trying to abstract over a bunch of details to give you one simple input price. The “Weighted Avg Output Price” (currently $14.84/M for K3) is trying to do the same on the output side. Even though a lot of interesting technical and economic detail is getting erased here, these prices are starting to become useful.
Suppose I want to ask my assistant to generate a 250-word summary of a 12,000-word research paper. I’ve got input and output word counts. We can apply the rough rule of thumb that an English word maps to about 1.3 LLM tokens. From there, we can plug in these weighted average input and output prices to find that our summarization request might cost, uh, about 1.7 cents. (The true cost is likely to be higher because this paper is likely unfamiliar and uncached, and newer models will likely write additional internal reasoning text to themselves before writing the final summary. Nevertheless, it’ll be cheap.)
When a new model comes out or an existing provider adds support for an old model, we sometimes want to summarize the usage cost of the model with a single number, without reference to a specific task. Artificial Analysis calculates its blended prices using a “7:2:1” ratio: assume that requests involve 70% cached inputs, 20% uncached inputs, and 10% outputs, by token. If we’re considering blended cost ratios, the units of text cancel out: it doesn’t matter that tokens aren’t exactly words so long as they are typically proportional. Because the blended cost of inputs is cheaper than outputs by about the same ratio that inputs are more common than outputs, we sometimes just read the raw dollars-per-million output token market price as the single summary figure for the model as a whole.
Even though tokens aren’t money, there is a justifiable chain of abstractions and approximations that lets us say “GLM-5.2 costs $3.60/M” (this is the model powering the Basic agent on BayLeaf Chat) and “Opus-5 costs $25/M” (this is the model I often use for maintenance of BayLeaf’s infrastructure). When a new model comes out that scores perhaps “82% as smart as Opus-5” by the ratio of their Artificial Analysis Intelligence Index scores, at a blended cost of $0.18/M, hmm, maybe we should switch to that one. (DeepSeek V4 Flash 0731 is the one I’m thinking about today, and it is already marked as the recommended model on the BayLeaf API.)
Even if there’s an analytically useful way to treat LLM tokens as if they were money (or simply cost money), this immediately erases the rich distinction between reading and writing and the importance of familiarity when reading. If you want to control the US-dollar cost of your LLM usage, simply asking your assistant to be concise in replies isn’t going to matter much. Getting your assistant to read less, or to read in ways that put much more of the text into the familiar category, matters much more.
Perhaps I should write another post specifically about these responsible cost-control techniques. The key concept you’d need to learn is how to identify the longest identical prefix shared from one request to the next. But before getting into techniques, we should ask why users need to recover these distinctions from provider accounting categories in the first place.
Most people using LLMs access them via pay-per-month subscription services that offer them a quantitatively unspecified amount of service. Some are a bit more transparent, promising that your subscription covers a certain number of requests within rate-limiting windows of a certain duration. If you are only thinking in terms of atomic requests (each of which involves some specific amount of familiar and unfamiliar reading before some hidden reasoning and visible writing), nothing is nudging you to learn even about token counts, let alone the distinctions between cached and uncached inputs.
Until the last month, I had gone about two years without monthly subscriptions to LLM inference services. I probably paid a lot more as a result. Flat-rate individual subscriptions can provide far more inference than the same payment would buy at API rates, while enterprise services increasingly expose metered consumption. Whatever the providers’ motives for that subsidy, I think paying per token taught me things I couldn’t have learned as easily as a subscription customer.
On BayLeaf Chat, under each agent reply, there’s a little info button you can click to get some usage and cost data about that reply. The display is a little too technical, but what’s important is that it has these unpacked token counts. The cached_tokens figure is less than the prompt_tokens figure, but only by a bit (because most of the text in the conversation is recycled from one reply to the next), and the total_tokens figure is only a little higher than that (because the replies are typically small compared with the inputs that prompted them).
One of the things you can learn from examining these usage cards is that the actual size (token or word count) of a conversation is often not easy to predict from the words you see on the screen. When you ask the agent something that causes it to look things up on the web or interact with other external systems, the agent needs to write some additional words to do those searches and read a whole lot more additional words to make sense of the search results before it writes back to you. This isn’t an argument for us to turn off web search to optimize costs. This is an argument for people to demand visibility into token counts so they can understand what’s actually driving the system’s individual and socialized costs.
The everyday interfaces of ChatGPT, Claude, and Gemini generally hide token counts from users as if they were a distracting technical detail mostly only relevant to system integrators. But if you can allow the slight fudge that tokens are words, every user already has some intuition for interpreting word counts and making responsible choices on that basis. The “token” terminology makes you think about accounting, whereas “word” sets you up to think about reading versus writing, how meaning thickens in context, how familiarity degrades when things fall out of working memory, etc. It’s easy to look at a chatbot’s apparently lengthy reply and wonder if the output was worth the cost. I want you to think about all of the words that got read before writing that reply, think about which human wrote them, and how we can plug in more relevant context sources while encoding them compactly.
Tokens are words, not money. Hopefully I’ve thickened your representation for “tokens” in a generative AI context so that you’re ready to make choices differently in the future.