Who is leading the way with DataOps? Regulated industries

3 years ago 351

A new survey finds that companies subject to GDPR and other data rules have the most advanced strategies and are more likely to be cloud ambitious.

Turns out you can follow data privacy and security regulations and still win with your data strategy, according to a new survey.

S&P Global Market Intelligence and Immuta asked 525 data experts about how well DataOps is going. The survey found that companies in regulated industries are leading the way with fewer challenges with data access and use. These companies that are subject to data privacy and protection regulations such as GDPR are also more likely to have a dedicated data engineering team and to provide self-service analytics.

Regulated companies are also more likely to be cloud-first or cloud forward as compared to the majority of companies in nonregulated industries that were more likely to be "cloud conservative." Paige Bartley, a senior analyst at 451 Research, part of S&P Global Market Intelligence and author of the report, concludes that: The assumption that regulated industries or firms tend to shy away from cloud technology is outdated at best.

Companies in banking, retail and healthcare also are more likely to have a dedicated data engineering team, which means that these firms are more likely to invest resources in the data supply side, according to Bartley.

The new report, "DataOps Dilemma: Survey Reveals Gap in the Data Supply Chain" looks at gaps, points of friction and mismatches in resources. The survey includes responses from both data consumers and data suppliers. 451 Research surveyed 525 enterprise practitioners in the U.S., Canada, U.K., Germany, and France who had a hands-on data job or who directly influenced data-centric functions within their organization. Survey respondents worked at companies with 1,000 or more employees and used a cloud data platform. The survey was completed in May.

Bartley also identified these trends in how data is gathered and analyzed:

More people are consuming more data in more ways
Quality is becoming more important than volume
Large organizations report more data sharing outside company walls
More chief data officers join the executive team

Data experts said that the number of data consumers in their organizations is increasing steadily and 73% said that even more humans and machines will need access to data over the next 24 months.

SEE: The state of data scientists: Overwhelmed and underfunded (TechRepublic)

People are using a wider variety of tools to consume data as well, according to the survey, including:

Self-service visualization
Self-service data prep
Data science tools and platforms
Internal data marketplaces

Also, 90% of survey respondents said data quality and trust would become more important than data quantity over the next 24 months.

About a year ago, 451 Research conducted a similar survey and found that only 41% of survey respondents had a chief data officer on the team. That number is now 60% with most CDOs reporting to the CEO (50%) followed by the chief technology officer (30%) and chief information officer (18%).

Weaknesses in the data supply chain

For people supplying the data to feed the ravenous enterprise demands, 29% list "not enough automation" as a key pain point. For people consuming the data, 37% reported that technology-based bottlenecks were due to lack of product automation.

The increase in streaming and real-time data sources is another challenge to the data supply chain. The survey authors found that data is typically available at "point-in-time" rather than in real-time, which limits its usefulness in agile decision-making.

Just over half of all survey respondents either somewhat or completely agree that data is often stale or out-of-date by the time it is consumed or analyzed. The report authors conclude that lack of automation in the management and synthesis of data sources can hinder the availability of timely datasets.

DataOps is still a teenager

DataOps is in the early days for a majority of companies, according to the survey:

Emerging: 42%
Accelerated: 31%
Nascent: 12%
Optimized: 10%
No strategy: 5%

Companies in the emerging group have a DataOps strategy defined but not fully operationalized while the accelerated group's data strategy is delivering value.

Bartley recommends that companies prioritize automation and scalability in addition to investing in people, process and technology to speed up the DataOps transformation.