Three steps to mastering AI token economics
To experienced IT leaders (and with full credit to Yogi Berra), managing AI often feels like déjà vu all over again.
Most of us remember the challenge of wrangling shadow IT, and the hard work of reclaiming financial control over cloud computing. AI is repeating those patterns at a speed and scale we haven’t seen before, as evidenced by the fast-circulating story about an enterprise that “accidentally” spent $500 million on Claude AI in a single month.* A bill that size could only happen at a massive corporation, but the pattern behind it hits mid-market companies proportionally harder. Leaner budgets and smaller FinOps teams leave far less room to absorb a surprise.
This phenomenon came into sharper focus for us recently at Dell Technologies World, where the economics of AI was a recurring topic on stage and in informal conversations. It got us thinking about the financial implications of the AI infrastructure decisions our clients are facing.
Why it’s time to think about AI token economics
AI and agentic initiatives may launch below IT’s radar, but the resulting costs quickly become IT’s problem. As the recent Dell event reinforced, it’s worth getting familiar with token economics — the practice of measuring and managing what AI is consuming.
Every AI request is processed in units called tokens. When those requests happen in the cloud, every token brings an associated cost. AI token economics is the discipline of managing activity, so you can see where it’s happening and, in the case of cloud-based requests, what it costs. With these insights, you can start to manage spend and determine the right environments for all your AI workloads.
Until recently, the cloud was the only practical place to run AI, so organizations typically brought their data to the AI. Now AI can run close to your data — even on hardware you own and control. As token usage further escalates cloud costs, we’re helping more organizations explore options for supporting AI workloads on premises.
AI with the right fit, at the right cost
Not every AI request needs the same expensive resources. Some requests should be routed to a premium cloud model that will incur cloud token costs. However, many can run smaller models on hardware inside your enterprise.
Sending every prompt to the highest-cost resource is like buying a high-end sports car just to pick up your groceries. It gets the job done, but you’re squandering dollars on horsepower you don’t need.
Hardware innovations keep expanding the options. At one end are compact, self-contained AI devices small enough to sit on a desk, such as the Dell Pro Max with GB10 line of workstations. At the other end of the spectrum are on-premises servers and large AI factories built for the heaviest workloads. As a vendor-agnostic integrator, we help organizations match each workload to the right system, regardless of which provider makes it.
Three steps to mastering AI token economics
Having spent 30 years managing the data centers, storage, networks, and security that organizations run on, we recognize that AI hasn’t altered what “good” infrastructure looks like. AI has raised the stakes for getting infrastructure rights.
As you’re getting your arms around the economics of AI infrastructure and workloads, consider three critical steps:
- Assess where AI is already running in your environment, including surfacing any “shadow” activity. Our AI & Data Strategy practice can help identify your current AI footprint and spend before you commit further.
- Map that activity to the right mix of cloud and on-premises resources, matching each workload to the most cost-effective place to run it. Our AI Infrastructure and Infrastructure Modernization teams can help you place each workload on the right system — from a premium cloud model to hardware you own.
- Implement governance and monitoring so spend stays visible over time and AI bill shock doesn’t become a regular occurrence. A combination of our Data Governance & Management and Managed Services can keep that visibility in place well beyond the initial project.
Many mid-market organizations lack the internal bandwidth to tackle these steps alone. That's where we come in. CBTS specializes in helping companies like yours design, build, and operate technology foundations for the AI era, bringing enterprise-grade AI economics within reach without an enterprise-sized FinOps team.
Ready to see where your AI spend stands? Let’s talk.