To the Max: Value trumps tokens when it comes to AI ROI

Realisation is now dawning that AI is not yielding the results anticipated. Tokenmaxxing, the practice of ploughing through tokens, the currency used to buy and consume units of data processed by AI models, to generate outputs faster has now fallen out of favour. Instead, the pendulum has swung back the other way, with token minimisation now being used to rein in spend. But limiting training data and outputs is often a false economy.

While it makes sense to avoid the runaway use of tokens, allowing the numbers to dictate spend can prove detrimental in the long run by, for example, influencing model selection. Selecting a lesser model could well hamstring the capabilities of AI over time. Similarly, reducing the data the model consumes could negatively impact learning and inference, leading to poorer output.

It’s this need to marry financial operations with business value that has seen the emergence of a discipline referred to as ‘tokenomics’. The Tokenomics Foundation defines this as converting energy and capital into AI capabilities and efficiently consuming that intelligence to realise measurable business value. In other words, tokenmaxxing is out and valuemaxxing is in.

Tailored consumption

In a bid to tie cost and value together, we’re now seeing a host of different price plans offered to the end user/organisation. These can be based on input and output, tokens-per-minute, and time-to-first token, which sees token latency ie the time it takes the AI model to respond, used as the determining factor.

Each of these will depend on what is perceived of value to the business. For an organisation that requires speedy decisions, time-to-first token matters, whereas a research organisation that prizes the thoroughness of the research but is not time sensitive would tolerate higher latency and put the emphasis on output. So what these price plans reveal is that it’s no longer simply enough to optimise token spend; it’s also necessary to tailor that consumption to the use case in order to maximise value.

For the enterprise, that means a dual approach is needed. A financial lens to evaluate the cost, consumption, ownership and unit economics of AI and to make it defensible and an enterprise lens to measure and govern the business value. Combining the two makes it possible to see when value is realised at the lowest cost. This then prevents the ‘rush to the bottom’ we’ve seen with token minimisation which can sacrifice the quality of output.

The AI Journey

It’s an approach that requires visibility of workloads, ownership and the drivers of consumption, the financial understanding of those costs and how they have been forecast and budgeted for, as well as any optimisation opportunities, and what decisions have been based upon, be that objectives, risks or adoption rates. The financial and enterprise outputs are then made part of a recurring review cycle that allows the performance of the AI to be assessed.

However, while that sounds fairly straightforward, moving from visibility to value is not an easy task. What we’re looking at here is AI transformation that can take three to five years of planning as the business strives to improve its maturity. It’s not a linear journey either, with changes to AI workloads, technologies, risks and business priorities requiring the organisation go back to certain stages to refine the operating model.

That operating model features five key stages. It begins with visibility, as the organisation assesses AI usage, cost and ownership before considering attribution which connects spend to workloads and outcomes. These then facilitate effective governance and informed decisions to be made to proceed, pause or retire with the project. The next stage is optimisation. It’s here where the business can look at ways to improve the economics and outputs without eroding trust such as through model right sizing to the data and the infrastructure to avoid token wastage. And the final stage is value, where outcomes can be demonstrated and proven and spend redirected accordingly.

Assigning value will again be business drive. In addition to financial growth against TCO, it might see the measurement of any boost to performance attributed to customer experience (CX) metrics, the effectiveness of operational processes, employee use versus productivity, or whether the AI is performing optimally and within established guard rails. These are all areas in which AI can prove itself through real measurable results - not through activity alone. Heavy AI usage, token consumption or request volumes with no corresponding uptick in business outcome mean you are measuring activity, not value.

It’s for this reason that valuemaxxing must see a dual approach that combines the financial and enterprise lens. Only then can we move beyond the rudimentary measurement of cost per token to AI spend against business outcomes and make AI prove its worth.