When tokens cost more than people

For most of the current AI cycle, the starting premise has been that automation is cheaper than headcount. Now that assumption is being tested against real invoices.

In April 2026, Axios reported that some companies are spending more on AI than on salaries. Nvidia's vice president of applied deep learning, Bryan Catanzaro, said the cost of compute for his team runs far beyond the cost of the people on it. Uber's CTO reportedly used up the company's entire 2026 AI budget on token costs alone. Global IT spend in general is on the rise and it's projected to reach 6.31 trillion dollars in 2026, up 13.5 percent year on year, driven largely by AI. The FinOps Foundation's 2026 State of FinOps survey found that 73 percent of enterprises saw their AI costs exceed original projections.

Of course, AI is not inherently more expensive than people nor does the comparison even make sense for most cases. But automating narrow, high-volume tasks with AI is cost-wise a no-brainer. Hence the cost problem is not a property of AI. It is a property of how AI is applied.

The waste is in the re-reading

AI agents do not make one call and stop. They loop, retry, and call again, and on each step they push context back into the model: files, prior output, documentation, source code. Because models charge by the token, the same context gets paid for again and again.

One FinOps vendor's audit found that re-sent context accounted for roughly 62 percent of the agent bill, the single largest line to optimise. This figure should be treated as indicative rather than universal, since it comes from a vendor with a commercial interest in the answer. The direction is not in dispute. When an agent re-reads what it has already read, most of the spend buys nothing new.

People are no different when they first meet a system. Learning something new is slow and effortful. You read the code, trace the logic, ask questions, and make mistakes. But that cost is paid once. After that the knowledge becomes intuitive, and the same task takes a fraction of the effort. An engineer who has worked on a system for a year does not relearn it every morning. In other words, the expensive part is learning and the cheap part is knowing. An agent that re-finds, and re-reads its context from a huge amount of material on every query pays the full cost of learning again and again, and never arrives at knowing.

Monolithic legacy code makes the problem worse

A large legacy code base is the hardest possible case for the re-reading pattern. On mainframes especially, the codebase is enormous, there are many people working on them at the same time, the dependencies are far-reaching, and the questions people ask about it are repetitive. What calls this program? Where does this field come from? Which DB2 access paths are redundant? Which code is never executed?

Give every employee in a bank access to an LLM and let all of them point that at the whole estate, and fire a way. Hundreds of people querying hundreds of millions of lines of code, thousands of documents and what not. From all of that compute, only a fraction is actual "new information" and the majority is just re-iterating something that has already been answered by someone else. Needless to say, the cost compounds, the latency compounds, and because the model is inferring structure rather than reading it deterministically, two runs of the same question can disagree. So, instead of creating understanding, the LLM is creating misunderstanding as different team members get different information. For a system where mistakes are expensive, that's far from optimal.

There is also a budget context specific to banks and insurers. These organisations already watch mainframe operating cost closely, because much of the core software is licensed on MSU consumption and billed on the peak rolling four-hour average each month. Adding an AI tool that re-reads the estate on every query does not sit alongside that discipline. It introduces a new, growing cost line on top of one that is already scrutinised.

An AST-based knowledge graph is a deterministic memory

When we started building Nomain in 2024, LLM context windows were tiny compared to today. That forced a discipline: feed the model only what it strictly needs. So instead of re-reading the codebase on every query, we built a knowledge layer, where an analysis of the code is stored in a hierarchical data structure, which is then used to retrieve what ever information is needed to get the job done. That decision matters now for an additional purpose. The same design that kept us inside a small context window now keeps token costs down by minimizing the need for re-learning and re-reading. With the recent addition of a memory system, Nomain goes even further from storing a pre-calculated analysis to actually creating institutional knowledge, while saving even more tokens.

At the heart of Nomain, there is an AST-based knowledge graph. That means, it's derived directly from source code and enriched with other available data, like documents or even interview transcripts. That graph is constituting the memory of Nomain, keeping the code as the source of the truth and filling the gaps with other data sources. The knowledge layer now contains pre-calculated analyses for things like:

• Pre-calculated business processes
• Program call graphs and data lineage
• Redundant and repeated DB2 calls
• Write-without-read data flows

When someone asks about any of this later, we do not re-analyse the estate. We surface the pre-calculated result. The expensive work happens once and every question after that reads an answers instead of paying to recompute it.

The difference from a learned memory is the point of the whole design. Nomain's memory is deterministic and traceable back to the exact program and line that produced each result. That is what makes it usable not just for speed, but for audit, for DORA transparency obligations, and for ISAE 3000 evidence. A memory you can verify is worth more than a memory you have to trust.

The takeaway

The token bills making headlines are a symptom of architecture, not of AI itself. Systems that re-read the same context on every query will keep getting more expensive as models get more capable. Systems that build understanding once, hold it in a deterministic memory, and surface it on demand do not.

For a mainframe estate, that is the practical question worth asking of any AI tool: does it understand your system once, or does it pay to relearn it every time you ask.

‍