As AI technology continues to expand in enterprise operations, the business world is closely watching how to evaluate the return on investment (ROI) of AI projects. This focus stems from the increasing spending by companies to support the deployment of AI tools. According toa Deloitte Insights survey, more than half of organizations allocate between 21% and 50% of their digital program budgets to AI. The firm estimates that for a company with $13 billion in annual revenue, this amounts to approximately $700 million.

However, before companies can measure the success of their AI efforts, they must test solutions and scale them into production—a tricky step for most organizations. The willingness of CEOs and their boards to make significant investments is a prerequisite for such efforts. Strong partners from human resources, sales, marketing, and other business units are also crucial.

The 'magic' of AI in HR and payment leasing

When fintech companyProg Holdingsmigrated to Workday's HR software, its leaders wondered whether they could use AI to help employees get answers about benefits and IT issues. CTO Sridhar Nallani collaborated with leaders from HR, R&D, and other business units to build Piper—a chatbot that uses large language models to provide access to information.

According to Nallani, the software has resolved over 18,000 employee questions, with 58% of questions answered correctly on the first interaction. Nallani told CIO Dive that adoption has spread like wildfire as a result. Despite the rapid adoption, it was not an overnight success. To assess the chatbot's capabilities, Nallani's team first built a proof of concept based on ChatGPT, testing it with a small group of users before rolling it out to the broader business.

Piper's success inspired Nallani to create an application that helps Prog set lease terms, pricing, and eligibility, thereby generating instant quotes for customers. Nallani said the software reduced decision time by 75% while increasing direct-to-consumer conversion rates by 10%. Prog also built a generative AI tool to help customers pay for products, as well as software that generates content for marketing campaigns. Since then, the company has migrated most of itschat-oriented AI servicesto Microsoft Copilot.

Although Nallani is not oblivious to the global obsession with ROI, he said that not every AI implementation needs to deliver a calculable return. Building digital services that reduce friction and time can create happier employees and more loyal customers. Nallani said, "You don't need to put a number in a spreadsheet and say, 'I'm going to get $10 million or $12 million from this,' for it to create value."

AI agents keep freezers stocked

The cost-cutting potential of AI agents quickly caught the attention of Arctic Glacier Premium Ice, its CIO Doug Saunders told CIO Dive. The packaged ice supplier deployed agents to automate inquiries about ice deliveries, saving tens of millions of dollars by reducing customer call centers and transportation operations. Saunders, who joined the company in 2023, said these operations were the result of trial and error, including switching platforms before going into production.

Arctic Glacier initially built an agentic AI system based on Microsoft Copilot to help answer customer requests for ordering ice or tracking shipment locations. With promising early returns, Saunders expanded the program. The company eventually migrated toSalesforce's Agentforce platform, which connects to an IoT system with lidar sensors in ice product bins at each customer site.

When the system senses that ice is about to run out, it sends an order to the AI agent, which begins dispatching trucks to fulfill the order. The system also considers historical sales trends, weather analysis, and other factors to predict ice demand for each route. Saunders said automated ice delivery represents a competitive advantage—at least for now. The company is exploring dynamic pricing for businesses in warmer climates that may run out of ice sooner than expected.

Iteration is guiding technology strategy. "You start with a tiny business problem, prove it, get a very good ROI, and then you see different ways to scale it that make EBITDA grow even more," Saunders said.

The path to value is rarely linear

Despite positive outcomes, CIOs should remember that AI is still a tool, not magical fairy dust. That's the view of Mihai Strusievici, who founded Axsion Digital Evolution, which provides IT strategy consulting to small and medium-sized businesses. Strusievici said some companies and consultancies' relentless pursuit of ROI overlooks a more important point: AI can help users do tasks better, faster, and possibly cheaper. "Did they calculate the ROI of word processors or spreadsheets?" Strusievici asked. "I think it's a fantastic tool, but it's still a tool."

What is clear is that AI pilots are moving from exploration to production. While only 25% of senior leaders say their organizations have moved 40% or more of AI experiments into production so far,Deloittesaid in a recent report tracking AI adoption and impact that 54% expect to reach that level in the coming months.

In this era of AI exploration, IT leaders must look for solutions that deliver real value to the business, said Jason James, CIO of retail technology provider Aptos Retail. The path is rarely linear. "We're in an R&D era," said James, who is experimenting with AI agents. "We need to figure out what works. We'll spend money on things that ultimately prove ineffective and of low value."