The “intelligence tax” is real, and businesses are increasingly paying it in money, data, and control. This three-part series explores the compounding costs of vendor AI dependency and the growing shift toward enterprise AI sovereignty.
TL;DR
Despite falling AI token prices, the increased use of token-hungry large language models is driving up enterprise AI costs at an alarming rate. The reasons: “rented” frontier LLMs reset with every API call and consume vastly more tokens than necessary for most enterprise applications. As an alternative, organizations are turning to domain-tuned small language models that match frontier APIs at a fraction of the cost—and compound in value over time.
The Proof is in the Numbers
In my previous post, we discussed how overreliance on frontier AI models can expose businesses to vendor blackouts, override enterprise data control, and drive up computational costs. In this post, we’ll explore how AI token economics — which connect AI compute work with real-world power use and energy consumption — and LLM market dominance are forcing enterprises to pay for more AI than they need.
Let’s start with a reasonable counterargument: token prices have been falling with each model generation, and OpenAI priced GPT-5 deliberately low. But the volume of AI tokens consumed per agent interaction is rising faster than prices are dropping. An agentic workload generates 10 to 100 times more tokens than a chatbot query — tool calls, reasoning chains, retries, multi-step execution. Cheaper per token, vastly more tokens. The total bill grows.
The immediate objection is capability. A fine-tuned 7B model is not GPT-5.5. It is not Opus. The comparison is only honest if the smaller model is good enough for the task.
A year ago, that was a theoretical argument. It is not anymore.
Most enterprises don’t need AI token-hungry models
In June 2026, researchers fine-tuned a LLaMA 3.1 8B model on just 219 examples for compliance evaluation of conversational transcripts — an 18-field structured prediction task in a regulated industry. The result: 100% accuracy on the most critical classification field in blind evaluation on production data. 83% overall accuracy. Inference in two seconds on a single A100 GPU — 2 to 5 times faster than frontier APIs — at $0.013 per evaluation versus $0.025 to $0.055 for GPT-4o and Opus. A 46 to 76 percent cost reduction, with matching or better accuracy, using a model with only 2% of its parameters adjusted through fine-tuning.
The finding is domain-scoped, and that is exactly the point. Enterprises do not need a model that can do everything. They need a model that does their thing. They need to perform compliance reviews, claims processing, customer routing, and contract analysis better, cheaper, and without sending proprietary transcripts to a third-party API. A global enterprise, for example, might still use frontier APIs for exploratory work but move high-volume post-call compliance scoring, routing, and workflow execution onto fine-tuned small language models (SLMs) where latency, privacy, and unit economics actually determine whether the system survives procurement.
The broader market confirms this at a different scale. OpenRouter publishes real-time usage rankings — a real-time measure of what developers actually choose to run. As of June 2026, the five most-used models on the platform are all open-weight. DeepSeek V4 Flash leads at 4.51 trillion AI tokens served. MiniMax M3 is third and growing 103% month over month. The first US closed model, Claude Opus 4.7, appears at number six.
The benchmark data explains why. SWE-bench Verified is the industry’s most widely cited test for AI on real software engineering — 1,865 actual code changes across 41 open-source repositories. DeepSeek V4 Pro Max scores 80.6% on it, tied with Gemini 3.1 Pro, at $0.87 per million output tokens. Claude Opus 4.8 costs $25. GPT-5.5 costs $30. MiniMax M3 scores 80.5% at $1.20 per million output. On the benchmarks that matter most for production agents — real code, real repositories, real engineering tasks — the open-weight models are matching the frontier at a fraction of the price.
The Pony Alpha Story
Before publicly revealing GLM-5, Zhipu AI and Tsinghua University released their 744B-parameter model anonymously on OpenRouter under the name “Pony Alpha.” On web browsing tasks, GLM-5 scored 75.9 versus Claude Opus 4.5’s 57.8. On tool-augmented reasoning, 50.4 versus 43.4. On math, on terminal tasks, on long-context work it matched or beat the frontier across the board.
Users speculated Pony Alpha was Claude Sonnet 5, DeepSeek, or Grok. Twenty-five percent guessed it was an Anthropic model. It was later revealed to be an open-weight model from China.
That does not mean enterprises should anchor their future to any single open model vendor. It means the source of advantage is shifting. Raw model intelligence is becoming more available, more competitive, and harder to monopolize. The durable value is moving up the stack: model selection, orchestration, tuning, governance, and deployment.
When the people closest to the models cannot tell the difference between the rented product and the open alternative, the intelligence tax stops buying intelligence. It buys only convenience — and a set of dependencies most enterprises have not fully priced.
The compounding disadvantage
The intelligence tax is not a one-time premium. It is a compounding disadvantage.
A February 2026 NBER study of roughly 6,000 executives across four countries found that nine in ten firms reported no measurable impact of AI on employment or productivity. In May, Gartner surveyed 350 executives at companies with over $1 billion in revenue: approximately 80% had reduced headcount after deploying autonomous AI, but there was no correlation between layoffs and AI ROI. Companies with strong returns cut at the same rate as companies with negative returns.
These two data points — 90% no impact, and no link between layoffs and results — together tell a story neither tells alone. The firms seeing nothing are not failing at AI. They are failing at ownership. They rent a model, bolt it onto existing processes, cut headcount to show the board a number, and wonder why nothing compounds.
The difference between renting and owning AI is the difference between a tool and a system. A rented model answers today’s question at today’s price. An owned model — fine-tuned on your data, deployed in your environment, improved with every production interaction — learns your business. The rental resets with every API call. The owned system compounds.
Creating a flywheel of intelligence
That ability to learn — from enterprise-owned data and contextual knowledge — continuously and autonomously transforms the moat into a flywheel of intelligence.
The process is simple: Curate domain knowledge. Train or fine-tune a model. Deploy on your infrastructure. Collect production feedback — which interactions succeeded, which failed, which edge cases the model had never seen. Improve the model with that data. Redeploy.
Every quarter, domain-tuned models get measurably better on the tasks that matter to that specific customer, while the frontier API they replaced charges the same per-token rate it charged on day one. Each cycle makes the owned model more capable and the cost advantage wider.
Stop paying for AI tokens you don’t need
Sarah Guo, the investor behind Conviction Capital, recently articulated the other side of this argument in an essay called “The Untrainable.” Her framework: anything you can put on a leaderboard, you can train against. Anything measurable is on its way to commodity. The valuable work is illegible by construction — private correctness that exists only inside someone’s data.
“A company that brings the translation is tough to copy, and the translation never ends.”
– Sara Guo, Founder and Partner
Conviction Capital
Guo writes as an investor: where does durable value accrue? The operator’s corollary is: how do you build the layer that cannot be copied? You cannot rent your way into domain expertise. The “translation” she describes — arranging a company’s reality, so a model can act on it — must live in something you own. A fine-tuned model that has absorbed your edge cases, your compliance rules, your customer patterns is not something a competitor replicates by subscribing to the same API. A rented frontier model starts from zero with every customer. An owned model remembers.
Frequently Asked Questions (FAQs)
AI tokens are the smallest units of data used for AI computation. A single agentic workflow execution can consume 15,000 to 80,000 tokens as the model retrieves context, reasons through steps, calls external tools, and validates its outputs. While flagship LLMs run at nearly $5 per million output tokens for standard models, and up to $60 for advanced reasoning models, self-hosted, fine-tuned SLMs consume significantly fewer tokens (and are exponentially cheaper) without sacrificing performance.
Enterprise AI costs are climbing because token consumption is outpacing falling per-token prices. Frontier large language models (LLMs) are particularly token-hungry and reset with every API call. As AI use expands across workflows and queries become more complex, enterprises that “rent” frontier models are routinely seeing higher-than-expected AI bills.
Yes. Open-weight small language models (SLMs) have been shown to match or beat LLM performance for domain-specific tasks at a fraction of the AI token cost. Enterprises that replace frontier LLMs with SLMs that are fine-tuned to specific tasks (i.e. compliance reviews, claims processing, customer routing, contract analysis) could potentially save thousands to hundreds of thousands in AI costs.