Mistral's AI Summit at Paris's Le Carrousel du Louvre.
STEPHANE DE SAKUTIN / AFP via Getty Images
Silicon Valley has spent the past year turning AI usage into a scoreboard.But after asking four executives about how they measure AI return on investment at Mistral AI's summit in Paris last month, I noticed something striking: none of them started by talking about how many AI tokens employees were using.Charles Holive, chief AI officer at BNP Paribas CIB, said his team focuses on outcomes rather than what he called "vanity metrics.""We try to go away from vanity metrics — billions of tokens per day," he said. "We try to make sure that what we track is an outcome, not a vanity metric."Instead of asking how many AI tokens employees have used, Holive said he asks: "What did you do, you didn't do before? How much faster did you do it?"Antoine Pichot, director of innovation, digital and data at La Banque Postale, told Business Insider the bank measures AI by whether it makes employees more efficient, improves customer service, and delivers value for money.Amit Kapur, chief AI and transformation officer at Tata Consultancy Services, one of the world's largest IT services and consulting firms, said his focus is on whether AI is improving business performance rather than just how many tokens employees consume.
