Counting Tokens: The Hidden Meter Behind Every AI Tool
Brian Long
In the first article in this series, we landed on a mental model that cuts through most of the confusion about AI: a large language model is a language processor. It takes language in and gives you transformed language back, the way a food processor takes in vegetables and hands you back something chopped or blended to spec.
That framing answers what the tool is for. This article answers a question that arrives the moment you start using it for real work: what does it cost to run, and how do I tell a good deal from a bad one?
The answer comes down to a single unit you’ve probably seen on your bill or your plan’s fine print without quite knowing what it meant: the token. Tokens are the meter AI runs on. Once you understand them, the whole economics of AI — pricing, speed, and the looming question of electricity — stops being mysterious and starts being something you can actually manage.
What Is a Token?
A token is a chunk of text — usually a little smaller than a word. AI models don’t read letter by letter or word by word; they break text into these chunks and process them one at a time.
A handy rule of thumb: one token is about three-quarters of a word. A thousand words of English runs to roughly 1,300 to 1,500 tokens (Microsoft, Fullstack). Common words like “the” or “house” are a single token. Longer or unusual words get split: “unbelievable” might become three tokens — “un,” “believ,” and “able.” Punctuation and spaces count too.
Why bother chopping words into pieces? Because it lets the model handle any word it encounters — including ones it has never seen, names, typos, technical jargon — by assembling them from familiar parts, the way you can sound out a word you’ve never read before. The practical upshot for you: everything you type in, and everything the AI types back, is measured and billed in tokens. The meter is always running, in both directions.
Prompt In, Response Out: Two Sides of the Meter
Here’s the part that surprises most people: the tokens you send and the tokens you get back are counted separately — and they usually aren’t priced the same.
The tokens in your request are called input (or “prompt”) tokens. The tokens in the AI’s reply are output (or “response”) tokens. Output almost always costs more — often four or five times more per token than input.
That sounds arbitrary until you remember the food-processor model. Reading your request is the easy part; the AI ingests your whole prompt in essentially one pass. Generating the response is the expensive part — the model produces it one token at a time, each new word depending on all the words before it. You’re paying for the labor of creation, not the labor of reading. Think of a translator who charges a small fee to read your document but bills by the page for what they write back.
This has a direct, money-saving implication. A long, detailed prompt — exactly the kind of thorough briefing we recommended in the last article — is cheap. Loading the AI with context and examples costs you very little, because input tokens are the bargain side of the meter. What you want to be deliberate about is asking for enormous outputs you don’t need. “Summarize this in three bullet points” is not just clearer than “tell me everything about this” — it’s cheaper.
One more wrinkle worth knowing about, because it’s reshaping AI bills across the industry. The newest “reasoning” models do something extra: before answering, they generate a hidden internal monologue — working through the problem step by step. Those thinking tokens get billed just like any other output, even though you never see them. A reply that shows you 200 words might have quietly burned through tens of thousands of tokens of private deliberation first (CodeAnt). That’s powerful for genuinely hard problems and wasteful for simple ones — and knowing the difference is becoming a real skill, which we’ll come back to.
How Fast Is Fast Enough? Token Speed
Tokens also measure speed. When you watch an AI’s reply stream onto the screen word by word, you’re watching tokens arrive, and the rate is measured in tokens per second.
The useful benchmark here is wonderfully human: about six to seven tokens per second is ordinary reading speed (Artificial Analysis). If a model produces text at least that fast, it’s keeping up with you as you read along — which is all you need for chatting, drafting, or summarizing. Anything faster than your eyes is, in a sense, wasted on a human reader.
So why does anyone care about the models that crank out 200, 300, even hundreds more tokens a second? Two reasons. First, when the AI is doing work you’re not reading in real time — churning through a hundred customer emails, processing a long document — faster means you wait less for the finished result. Second, the newer agentic tools chain many AI steps together behind the scenes, and a slow model makes the whole chain crawl. For the everyday chat-window work most small businesses start with, though, today’s mainstream models are all comfortably faster than you can read. Speed is rarely your bottleneck. Cost and judgment are.
Token Efficiency: The Number That Will Define the Next Few Years
Once you see the world in tokens, a more important question comes into focus — not “what does a token cost?” but “how much useful work do I get per token?” That’s efficiency, and it comes in two flavors that matter to a business owner: tokens per dollar, and tokens per watt.
Tokens per Dollar: The Cost Illusion
The good news first: raw token prices have been falling off a cliff. The cost to buy a given level of AI capability has dropped severalfold year over year, and the competition is fierce. As of mid-2026, budget-tier models run around $0.10 per million input tokens; mainstream workhorses like OpenAI’s GPT-5 series or Anthropic’s Claude Sonnet sit near $2.50 to $3 per million input tokens (with output four to five times higher); and frontier models climb toward $5 to $30 (CloudZero, BenchLM). A million tokens is roughly 750,000 words — most of a thick novel — for the price of a coffee. By that math, AI looks nearly free.
Here’s the catch, and it’s the single most important thing on your invoice to understand: per-token prices are falling while total AI bills are rising. Industry analysts call this the “token cost illusion” (Artefact, Indexnine). The price per unit dropped about 75% in a year — but the number of units a modern task consumes exploded. Those hidden thinking tokens, plus agentic workflows that loop through dozens of steps, mean a single complex request can quietly chew through 50,000 to 500,000 tokens before it hands you an answer. Cheaper units, far more of them, bigger bill.
The discipline this rewards is exactly the one we’ve been preaching: match the tool to the task. Using a heavyweight reasoning model to write a thank-you email is like running a backhoe to plant a tulip — it works, and it’s wildly inefficient. The same simple query can cost dozens of times more on a “thinking” model than on a right-sized one for no better result. The frontier matters here too: Anthropic’s newly released Claude Fable 5, the public version of its top-tier Mythos-class model, launched in June 2026 at roughly twice the price of the previous flagship (Anthropic, CNBC). It’s a remarkable tool for genuinely hard, high-stakes reasoning — and complete overkill for rewriting a job posting. Efficiency isn’t about always buying the cheapest model or the smartest one. It’s about never paying frontier prices for routine work.
Tokens per Watt: The Question Behind the Question
There’s a second meter, and almost nobody outside the industry is watching it yet: energy. Every token, in the end, is produced by a chip drawing electricity. Researchers now measure AI efficiency in tokens per watt — or tokens per joule — for exactly this reason (John Snow Labs).
This is not a fringe concern. By some estimates the world’s data centers were on track to consume around 1,050 terawatt-hours of electricity in 2026 — enough that, if they were a country, they’d rank as the fifth-largest power consumer on the planet, somewhere between Japan and Russia (TTMS). Grid operators are now treating “speed to power” as the make-or-break factor for whether new AI facilities can even be built (World Economic Forum). Electricity, not chips, is becoming the real bottleneck.
For a small business this seems abstract — until you connect it to your bill. The astonishingly cheap token prices of today are propped up by enormous investment and, in places, by selling compute below its true cost to win market share. Power is the bill that can’t be discounted away. Energy efficiency, not raw capability, is what will determine the long-term, real-world price of AI — and the commercial viability of the companies providing it. The model that does your job for the fewest watts is the model that stays affordable when the subsidies fade.
Why Local and Hybrid Is the Road Ahead
This is where the two meters — dollars and watts — point to the same destination: you don’t need a frontier supercomputer for most of what a business actually does.
The energy math is stark. A model’s appetite for power grows faster than its size: going from a 7-billion-parameter model to a 70-billion one can multiply the energy per token by roughly a hundredfold (arXiv). Most business language tasks — drafting, summarizing, rewriting, extracting — don’t need the hundred-times-hungrier model. They need a competent, right-sized one. And those small models have gotten remarkably good. IBM’s open-source Granite family now ships versions small enough to run directly inside a web browser, built deliberately for efficient, predictable, low-cost enterprise work (VentureBeat, IBM Research). A capable open model can produce useful text on a modern laptop or phone chip drawing just a handful of watts — about what a lightbulb uses (arXiv).
That opens up a hybrid approach, and it’s where the smart money is heading. Run the routine, high-volume, privacy-sensitive work on small, efficient models close to home — on your own hardware, where the marginal cost per token approaches zero and your data never leaves the building. Reach for the expensive cloud frontier only for the occasional genuinely hard problem that earns its keep. You get the best of both: the cost and power efficiency of local AI for the bulk of the work, and frontier muscle on tap for the rare case that needs it. The whole industry — OpenAI, Google, IBM, and the fast-moving open-source world alike — is racing toward smaller, cheaper, more efficient models precisely because that’s where the sustainable economics live.
The Quiet Advantage, Continued
Back to the hype cycle we opened the series with. During the boom, the story was about ever-bigger models and ever-more-amazing demos. That phase is ending. The durable advantage in the next phase won’t be built by whoever spends the most on the flashiest model — it’ll be built by whoever uses AI efficiently: the right-sized tool, briefed well, run for the fewest dollars and the fewest watts to get the job done.
That’s good news for small and medium businesses, and it’s the same good news as before. You were never going to win by outspending the giants. You win by being deliberate — by knowing that a token is a unit of work you’re paying for in money and in energy, and by spending both only where they buy you something real. The tools are getting cheaper, smaller, and more efficient by the month. The businesses that learn to count their tokens will quietly build an advantage while everyone else is still staring at the frontier.
At Synapses Technologies, helping small and medium businesses find that practical, efficient fit — the right tool for the real job — is what we do. If you’re wondering which tasks belong on a lean local model and which are worth the frontier, that’s exactly the conversation we love to have.
Next in the series: where to find the recurring language tasks hiding in your own operation, and how to verify AI output without losing the time you just saved.