Bigeye raises $17M to algorithmically monitor data quality

Bigeye, a data quality engineering platform, today announced it has raised $17 million in a series A round led by Sequoia Capital. The company says the funds will be used to improve its platform and help make it available to more data teams.

Data is increasingly critical to enterprises and is woven into the products and services that directly affect customers. To keep pace, data engineering has increased in scale, complexity, and automation, leading to a number of significant workflow challenges. A clear majority of employees (87%) peg data quality issues as the reason their organizations failed to successfully implement AI and machine learning, according to a recent Alation report.

San Francisco, California-based Bigeye, previously called Toro Data Labs, employs machine learning to enable companies to instrument data lakes and warehouses with thousands of data quality metrics. Founded in 2020, the platform automatically instruments datasets and pipelines with metrics, creating alerts driven by anomaly detection techniques.

How it works

Bigeye uses connectors and read-only accounts to connect to data sources and record health metrics. Available in fully managed software-as-a-service form or as an on-premises app for enterprises, Bigeye samples objects like tables and generates recommended metrics based on data profiling and semantic analysis. By default, all metrics have automatic thresholds enabled — within 5 to 10 days, Bigeye learns the behavior of the metrics and begins to make adjustments. When those thresholds are reached, the platform sends alerts via email, Slack, and other channels and optionally triggers remediation steps.

For one company, Bigeye identified that its customer data had a number of rows in which the values had been written into the wrong columns. The percentage of rows affected was small enough that analysts might not have spotted it, but at the scale that the company was working, it could have led to hundreds of customer support tickets that would have needed to be resolved.

Bigeye can draw from Snowflake, Redshift, BigQuery, and other popular sources, and its no-code interface allows teams to create, edit, and read configuration and metric histories. The company says that as a part of its efforts to improve the platform, it recently increased support for service-level agreements, which can help engineers build trust through transparency with users.

Data quality

As processes around data remain a hurdle in adopting AI — 34% of respondents to a 2021 Rackspace survey stated poor data quality as the reason for AI R&D failure — observability solutions like Bigeye are attracting investments. There’s Aporia, Monte Carlo, and WhyLabs, a startup developing a solution for model monitoring and troubleshooting. Another competitor is Domino Data Lab, a company that claims to prevent AI models from mistakenly exhibiting bias or degrading.

“Right now, modern data teams are held up by the heroics of data engineers, analysts, and data scientists trying to triage data quality incidents after something has already gone wrong. We’ve been the people who have to stay up until 3 a.m. on a Saturday trying to backfill a pipeline — and it doesn’t feel heroic,” cofounder and CEO Kyle Kirwan told VentureBeat via email. “For companies to realize the value of their data, it needs to be effortless for data teams to measure, improve, and communicate data quality for their organizations.”

But Bigeye has already successfully courted large customers, including Instacart, Crux Informatics, and Lambda School.

In addition to Sequoia, Costanoa Ventures also participated in Bigeye’s latest funding round. The three-year-old company has 11 employees, and the funds bring its total raised to $21 million.

By VentureBeat Source Link