Some winds of change are stronger than others

Source: i-Stockr/iStock via Getty images.

We have written extensively about the simultaneous existence of both data management for AI and AI for data management. Data management for AI represents the data control, stewardship and staging of enterprise data required to use and support the adoption of AI tooling successfully within the organization. AI for data management, alternately, represents the embedded automation and AI within these data management ecosystems necessary to apply the correct data controls at massive, dynamic scale. Organizations increasingly depend on both approaches.

Firms are eager to adopt AI-enabled technology but often encounter barriers to success. These barriers can either be technical or organizational, but many pain points share common threads. Efforts to use AI are rarely successful without trusted business data. DataOps personas and teams sit at this critical junction. A survey conducted by 451 Research by S&P Global targeted respondents involved in data management and DataOps within their organizations. This report presents primary findings, which often vary based on a business’s self-reported DataOps maturity and other variables.

The Take

The more some things change, the less others do. Data management practices might as well be categorized in the latter category. While DataOps and data management technology have largely been overhauled to take advantage of the newest automated and AI-enabled features and functionality (i.e., AI for data management), the underlying need for data management has steadily increased amid the rollout of more enterprise AI models, agents and users. AI technology is now consistently acting at machine speed and scale. Data management and DataOps practices provide the correct data on which these AI models and tools thrive. Full visibility into underlying enterprise data sources also provides an important glimpse into AI lineage and auditability.

It is difficult in a survey to ask about self-reported maturity, as many participants tend to inflate or overestimate their stats. But what we have seen in this study is that self-reported maturity in DataOps, in fact, does closely correlate with many other measures of DataOps activity and progress. As AI and agents have taken center stage in the enterprise, DataOps professionals remain largely behind the scenes: less often seen but always critical enablers of business success.

Summary of findings

Most organizations are beyond the initial adoption and implementation phase of DataOps. A majority of respondents report their organization’s DataOps journey is in the “developing” (35%) or “established” (32%) stages, which places most in the middle of the DataOps maturity curve. It appears only 17% report still being at an “early” adoption stage, while another 16% — the smallest group — are confident in in their “advanced” status. This curve suggests that most organizations are taking data management and DataOps seriously, yet there is still ample room to develop and establish more advanced practices and processes.

Integrated commercial platforms reign supreme, yet data management tech remains highly varied. When it comes to technologies for data management or DataOps initiatives, organizations have a demonstrated preference for larger, multipurpose commercial platforms, with 52% of respondents reporting their use within the organization. This parallels findings from our DataOps survey last year, where multipurpose platforms were also the top — although less dominant — response. Yet the technology ecosystem is not homogenous, with multiple non-exclusive options and product models to choose from.

Automation is widespread in DataOps, but most organizations remain in the middle of the maturity curve. The use of automation in DataOps mirrors aforementioned maturity trends, leaving few surprises, with many populating the middle of the curve. Yet while 34% report “developing usage” and another 38% report “established usage” for methods of automation in DataOps, only 18% have achieved “advanced usage” whereby automation is considered fundamental to DataOps methodology and has widespread use across the organization. The term “DataOps” should be synonymous with automation via its definition, yet businesses are still pragmatic in assessing their progress.

Many commercial technologies support DataOps, but governance and security tech is nearly universal. Most reported adoption rates for categories of commercial DataOps-related technologies are relatively high, hovering at 50% to 70%. However, the most widely adopted DataOps practices and technologies reflect the need to control and secure data.

While this is a broad category of functionality, the voracious adoption rate suggests that these functions are critical, even among smaller organizations. Core DataOps functions like “data impact analysis” (71%) and “data transformation and integration” (70%) are also extremely common to support with commercial tooling.

Organizations prefer to buy technology, but they still build models when appropriate. When organizations incorporate AI/ML models into their DataOps efforts, a “buy over build” strategy appears to be most common. Commercial models are reportedly used by 57% of organizations in DataOps initiatives. Yet this is not mutually exclusive with the use of other types of models. The reliance on in-house models (45%) and open-source models (43%) remains notable, pointing to a hybrid or blended approach whereby organizations buy off-the-shelf when they can, but build when they must for specific use cases.

DataOps efforts do not exist in a vacuum; most organizations track the impact of data management. One of the most persistent challenges in data management and DataOps has always been connecting efforts and spending to measurable business outcomes. Today, more than three-quarters of organizations (76%) formally monitor the business impact of their DataOps initiatives. When they do, they are most likely to track technical performance indicators such as “data quality metrics” (52%) and “data availability/outage metrics” (45%). However, a significant number are connecting DataOps to business value by tracking high-level performance related to “financial outcome metrics” (42%) and “operational KPIs” (37%). Such precise tracking can demonstrate the financial impact of DataOps efforts and help secure future resources or budget.

A maturity gap is evident in the way organizations measure DataOps impact on the business. Even amid rudimentary self-assessment, organizations that report more advanced maturity on several measures tend to score much higher in their initiatives to measure the material business impacts of their efforts. Those that rank themselves as having more advanced digital transformation efforts and those that are earlier to adopt enterprise technology tend to most closely monitor DataOps effects on business performance at an either “moderate” or “great” extent. Considering business models, those with a mixed B2B and B2C approach tend to be the most mature in measurement methods. Among these organizations, an impressive 86% report that they monitor and measure the impact of DataOps initiatives on business outcomes at a high level.

DataOps problems are most often technical, yet cultural and resource problems persist. When respondents are forced to rank the top three categories of significant barriers in DataOps for their organization, “technical issues” top the list with 42% reporting this at the No. 1 challenge. Technical issues can include challenges associated with data silos, technology functionality and data quality (among others). On the other hand, “resource issues” (e.g., budget, talent) and “people issues” (e.g. strategy, communication) were nearly tied for a distant second in this ranking at roughly 29% and 28%, respectively.

DataOps is a team sport, but key collaborators frequently lean toward technical roles. Those involved in DataOps roles generally report high rates of organizational interaction with adjacent technical functions. The strongest levels of collaboration are with information security roles, where 74% report “high” rates of interaction, as opposed to “low” or “no” rates of interaction. The same holds true with operations or operational technology roles, where 72% report high DataOps interaction and dependency. Other prominent roles that collaborate highly with DataOps include data storage (68%) and cloud/platform infrastructure (65%). Collaboration with executive leadership is also quite common — and necessary — with 55% of DataOps respondents reporting “high” interaction with this faction.

Agentic AI practices and technology are now common in DataOps. A majority of organizational respondents (60%) are reporting that their firm already uses agentic AI to support DataOps tasks. Among these adopters, the primary use cases are to improve core data management functions. Among these users, top data management tasks being tackled include “data quality or data observability,” (52%), “data discovery or classification” (44%), and “process optimization” (43%).

IT and non-IT roles often perceive the execution of DataOps quite differently. The survey here reveals a consistent perception gap between the IT roles guiding the DataOps work process and the line-of-business staff tasked with consuming data. Non-IT roles, for instance, are less likely than designated IT roles to report that any form of agentic AI is being used for DataOps purposes within the business: 51% versus 64%. Similarly, non-IT roles are also less likely to report specific use cases for agentic AI in DataOps. This may suggest that IT teams are more intimate with the deployment and rollout of AI, but also that non-IT business users may have some of the complexity of AI abstracted away from them.

What is an AI Datacenter?

Read Report

Want insights on data analytics trends delivered to your inbox? Join the 451 Alliance.