AI/ML 
Software Development 
Why Do 80% of Companies Burn Through Their AI Budgets in Four Months? The Hidden Token Trap and the Evolving Role of the CTO
Chief Technology Officers and executive teams at technology companies are facing an unprecedented economic dilemma. On the one hand, investing in coding agents such as Claude Code, GitHub Copilot, and Cursor has become essential for maintaining competitiveness. On the other hand, traditional and predictable SaaS budgets have been replaced by uncontrolled, variable token consumption.
Does this sound familiar?
- Have you deployed AI tools, optimized team staffing, and then discovered two months later that your LLM token and credit costs had more than offset the expected savings?
- Did you see strong developer enthusiasm during the first month, only to find that key architects are now hitting a $1,500 per-session limit by Wednesday afternoon?
- Is the board asking for hard ROI data while your analytics show an 18-fold increase in token usage, with no noticeable improvement in the number of merged features?
Uber’s case, in which the company reportedly exhausted its entire annual AI tooling budget within four months after rolling out coding agents to 5,000 engineers in April 2026, is no longer being viewed as an isolated anomaly. It has become a textbook example of what Gartner analysts describe as a “token consumption crisis.”
The real question facing every CTO today is not: “How do we stop developers from using AI?” The real question is this: Why are most companies wasting as much as 60% of their token spend, and why does this affect not only budgets, but also software quality?
Table of Contents
The Power-Law Trap: 5% of Engineers Consume 80% of the Budget
An analysis of transaction data from technology companies, including 2026 research by Ramp, reveals a counterintuitive pattern. AI spending within software engineering teams follows an extreme power-law distribution.
While median expenditure per average developer is only $11 to $15 per month, the top 5% of so-called power users generate costs ranging from $600 to as much as $7,500 per person per month.
How does this happen? Unlike traditional chatbots, autonomous coding agents execute complex decision-making loops, often based on ReAct-style reasoning and tool use. A single uncontrolled session, in which an agent repeatedly attempts to resolve a defect in tightly coupled legacy code, can consume between 400,000 and 2 million tokens within two hours, generating costs of $1,000 or more.
Where Is the Money Actually Going? Three Hidden Sources of Waste
Based on VM.PL engineering audits, token waste is rarely caused by developers writing “bad prompts.” More often, it results from systemic deficiencies in the organization’s knowledge infrastructure.
- Context Overload and Context Rot
To ensure that AI “understands the problem,” developers often load entire repository directories into a session. However, research by Chroma indicates that model accuracy and reasoning performance can decline by more than 30% as contextual noise increases. The agent effectively becomes overwhelmed by its own context, begins to hallucinate, and produces code that requires extensive manual correction. - Protocol Overhead and Tool-Generated Noise
SWE-bench-style analyses indicate that between 39% and nearly 60% of the tokens sent to an agent may consist of raw, unprocessed tool output, including screenshots, logs, and complete JSON schemas generated by MCP servers. Connecting only three MCP servers can consume as much as 70% of a model’s context window before the developer enters the first instruction. - Using Frontier Models for Routine Tasks
The absence of task segmentation causes the most expensive flagship models to be used for simple unit tests, documentation templates, and other routine activities. These tasks could often be completed successfully by standard-tier models at a fraction of the cost.
The Mirror Effect: AI Will Not Fix Poor Architecture
Google’s 2025 DORA report, State of AI-Assisted Software Development, delivers a critical warning for technology leaders. AI acts as an amplifier.
- In teams working with transparent, modular architectures, AI can significantly increase feature delivery speed.
- In teams operating on monolithic legacy code, AI can sharply increase deployment failure rates and amplify operational chaos.
The sophistication of the coding agent is ultimately irrelevant if it is deployed against a disorganized monolith without clear skills, repository-level instructions, or well-defined boundaries. The organization pays twice. First, it pays for wasted tokens. Second, it pays for the engineering hours required to repair production issues.
Moving from Uncontrolled Consumption to Managed Efficiency: The VM.PL Three-Step Approach
At VM.PL, we help organizations escape the token trap not by imposing severe restrictions, but by introducing engineering discipline into context management.
Step 1: Deploy Telemetry and Protect the Tail of the Distribution, Week 1
Native OpenTelemetry-based monitoring makes it possible to immediately detect abnormal agent loops, assess cache hit rates, and identify the top 5% of inefficient sessions.
Step 2: Clean Up Context and Build Skills, Weeks 2 to 3
Instead of sending hundreds of files into each session, we implement precise repository instructions, prompt caching mechanisms that can reduce repeat-read costs by up to 90%, and task templates with human-in-the-loop checkpoints.
Step 3: Conduct an AI Readiness Audit and Modernize Legacy Code, Month 2 and Beyond
The real breakthrough occurs when the codebase is adapted for agent-assisted development. Transforming tightly coupled modules into well-defined interfaces enables AI agents to work faster, more reliably, and at a fraction of the previous cost.
The Limits of Ad Hoc Solutions: Why Superficial Optimization Creates Hidden Technical Debt
After reading introductory guidance, many IT Directors attempt to implement individual mechanisms independently. In complex enterprise environments, these ad hoc efforts quickly reach their limits.
VM.PL’s advantage does not lie in providing a simple checklist of optimization techniques. It lies in implementing a mature, scalable operating model built around three core pillars.
- Durable Cache Architecture
Standard industry publications often describe caching as a simple configuration flag. In reality, without properly structured workflows and prompt-state control, caching efficiency deteriorates rapidly, resulting in unpredictable and duplicated operating costs. VM.PL designs a stable context architecture that sustains high cost-recovery rates over the long term, regardless of team size. - Governed AI Telemetry and Observability
Installing dashboards alone produces raw informational noise. A token consumption chart without a direct connection to business logic does not explain the causes of waste. We transform raw telemetry into a management system by linking technical metrics directly to value indicators, including cost per completed task, suggested-code acceptance rate, and automated session circuit breakers. This gives CTOs a clear basis for forecasting quarterly AI budgets. - Dynamic Context Disclosure and Skill Architecture
Adding more static rules to a repository intensifies model attention fragmentation and Context Rot. The agent consumes budget processing its own instructions instead of executing the actual engineering task. We implement a dynamic context-delivery architecture in which dedicated instructions and skills are provided only at the point of execution for a specific class of tasks. This protects the model from unnecessary information overload.
Conclusion
Reducing token costs by 60% is not the result of mechanical cost cutting. It is a natural by-product of engineering maturity. Organizations that move first from unstructured AI adoption to managed AI architecture will gain more than lower operating costs. They will also achieve a substantial advantage in software quality, delivery speed, and time to market.
Would you like to identify where tokens are being wasted within your engineering organization? Schedule a 45-minute strategic session with VM.PL experts.
Categories:
Design, Development, DevOps or Cloud – which team do you need to speed up work on your projects? Chat with your consultation partners to see if we are a good match.
Jakub Orczyk
Member of the Management Board/ Sales Director VM.PL
Book a free consultation