Despite declining token costs, businesses continue to see their AI tab climb.
The price of a token — the chunks of text that large language models (LLMs) use to generate responses to prompts and that AI companies use to bill their customers — has collapsed, noted Jeff Barrington, managing director of Windsor Drake, an investment banking and M&A advisory firm in Toronto.
"They're down about 98% since early 2024, yet enterprise AI bills keep rising," he told TechNewsWorld. "Cheaper compute just gets consumed faster, and 73% of enterprises blew past their original AI budgets last year."
"Uber handed 5,000 engineers a coding assistant in December and burned its entire annual AI budget by April," he added.
One reason for the growing consumption: Companies are moving from pilots to production, and from chatbots to agents. "An agent does not answer one prompt," Barrington explained. "It fans that prompt out into planning, tool calls, retrieval and verification, which can turn a single instruction into hundreds of calls and five to 30 times the tokens."
