Quick Overview

  • Five major technology vendors jointly supported the adoption of cross-industry data provenance standards on Thursday. Cisco, IBM, Intel, Microsoft, and Red Hat co-sponsored the establishment of the OASIS Data Provenance Standards Technical Committee, as announced by the nonprofit organization OASIS Open.
  • The group will meet on April 8, with the mission to refine the data provenance standards developed by members of the Data & Trust Alliance and promote widespread adoption.
  • Nineteen Data & Trust Alliance affiliates, including American Express, Humana, IBM, Mastercard, Pfizer, UPS, and Walmart, participated in the development of version 1.0.0 of the standard, which was released in July 2024.

Deep Insight

Data preparation is an old problem that has reached new levels of urgency, as data-hungry generative AI applications accelerate the pace and scale of data consumption. The path to AI adoption is fraught with privacy, compliance, and integration hurdles that remain difficult to bypass.

"Data governance is not new—people have been working on this for a long time," Kristina Podnar, senior policy director at the Data & Trust Alliance, told CIO Dive. "But never have companies come together to drive it forward."

Standardizing provenance protocols across vendors and industries could pave the way for better data quality management and tools for automated verification processes.

The alliance's framework defines a common metadata classification system, helping organizations efficiently verify the quality and reliability of data sets for traditional analytics and AI business applications.

"We want to ensure that any enterprise in any industry can have visibility and transparency into the data they use," Podnar said.

In an unstable regulatory environment, broad participation is needed to establish trust benchmarks for third-party data sources. Policy professionals hope to set stronger guardrails for AI data, but most expect effective government intervention to take years.

"There has been much activity around AI adoption and regulation, but we still lack standard definitions for its key elements," Saira Jesani, executive director of the Data & Trust Alliance, said in a post in July when the organization released its standards. "From copyright infringement to privacy to authenticity, the consequences could impact the commercial value of the technology and its acceptance."

IBM tested these standards internally last year, aligning the committee's recommendations with its governance policies. The company reported a "consistent and quantifiable impact" on the overall efficiency of its data due diligence and management processes, according to Christina Montgomery, IBM's chief privacy and trust officer, in a November post.

The technical committee will seek to improve existing standards while injecting momentum into implementation tools. Podnar believes the current version covers most, but not all, of the standardization issues that should be addressed.

"It's about 80% complete," she said. "We know we haven't gotten it 100% right, and the standards must continue to evolve."

After the technical committee convenes, the organization intends to bring in more members. According to Podnar, the sponsoring companies aim to have widely usable metadata quality metrics ready within 12 to 18 months.

"They're not interested in dragging this process out too long," she said. "They want tools available in the market."