Scaling AI Pilot Projects: Success Paths and Key Practices
Enterprises are undergoing agentic AI transformation, but pilot failure rates remain high. Boomi's Senior Vice President of Products, Mani Gill, and AI Field CTO, Patricia Bradby Moore, pointed out at the AWS Summit in New York that the key to success lies in building trust, measuring impact, showcasing results, and maintaining governance. They recommend starting with low-risk use cases and scaling gradually, emphasizing the importance of data foundations, ROI definition, and human oversight.
Enterprises are actively engaging in agentic AI transformation, pursuing productivity gains and cost savings despite thehigh failure rateof AI pilot projects. Mani Gill, Senior Vice President of Products at Boomi, stated that companies that can learn to build trust, measure impact, demonstrate results, and maintain governance as projects progress during early AI initiatives tend to achieve greater success. Gill and her colleague Patricia Bradby Moore, CTO and Head of Innovation for AI in the application integration platform, shared the most successful case studies from their customers at the AWS Summit held in New York on Wednesday.
"It's not just about connecting data itself; the key is that agents need to understand the meaning of the data and how to use it," Bradby Moore pointed out.
Gill emphasized that teams need a solid data foundation and proper onboarding of the AI tools they intend to use to gradually build trust. She suggested that teams start with low-risk AI use cases to test tools, as this makes it easier for leaders to establish a foundation, identify changes, and scale in this manner.
"Everyone tends to choose the use case that seems coolest and has the most impact," Gill said, "but the reality is that the coolest and most impactful use cases are often the most complex."
After teams build trust, they can then focus on measuring impact and proving return on investment (ROI). Although definitions of ROI mayvary across departmentswithin an organization, ROI cannot be discussed without clarifying the business impact of specific applications. Gill stated that teams should also clarify the specific metrics they need to measure.
Gill added that ROI cannot be determined without considering risk. Teams need to weigh whether the productivity gains achieved are worth the potential risks introduced.
Identifying AI Value
Leaders can guide adoption plans toward success by explicitly encouraging employees to experiment. Gill noted that showcasing AI project results within the organization helps everyone understand which methods work and which do not. He observed that some teams feel ashamed of using AI outputs rather than completing tasks or writing code themselves.
"We need to reverse this mindset and clearly state: 'We are not using tools to produce AI garbage, but to create AI value,' and we must be able to demonstrate that value," Gill emphasized.
For AI projects that pass the pilot stage, Gill believes maintainingguardrails and governanceis crucial. This means establishing rules and training for employee use and output monitoring, while companies should track the number of agents in operation and their access permissions.
As teams expand use cases from the simplest automation to more complex tasks, such as agentic workflows that participate in organizational decision-making, human-in-the-loop protocols are indispensable.
"We are not only driving technological change, but also process andcultural change." Gill concluded.