Data flow automation engine Prefect raises $32M

Dataflow automation startup Prefect today announced it has raised $32 million in a series B funding round led by Tiger Global. The company says it will use the capital to further develop its platform, attract talent, and support its growing community of users.

When McKinsey surveyed 1,500 executives across industries and regions in 2018, 66% said addressing skills gaps related to automation and digitization was a “top 10” priority. Salesforce’s recent Trends in Workflow Automation report found that 95% of IT leaders are prioritizing automation and 70% of execs are seeing the equivalent of over four hours saved per employee each week. Moreover, according to market research firm Fact.MR, the adoption of business workflow automation at scale could create a market opportunity worth over $1.6 billion between 2017 and 2026.

Prefect, which was founded in 2018, offers a platform that can build, run, and monitor up to millions of data workflows and pipelines. The company’s hybrid execution model keeps code and data private while taking advantage of a managed orchestration service. Customers can use Prefect for scheduling, error handling, data serialization, and parameterization, leveraging a Python framework to combine tasks into workflows and then deploy and monitor their execution through a dashboard or API.

Prefect customers can design their workflows with a framework called Core, which sends metadata to Prefect’s cloud in order to register a flow for scheduling. Flow updates are asynchronously sent to the cloud as metadata, ensuring Prefect can’t view the content of the flows themselves.

Hybrid engine

Founder and CEO Jeremiah Lowin says the key to Prefect’s cloud hybrid execution model lies in agents — small open source programs that can launch flows into any environment. Agents can stream real-time state updates and kick off new runs, with an API to query data as well as join, filter, sort, and transform it.

Prefect’s scheduler service offers options that allow for per-run changes, and it lets users label flows so they’re picked up by agents with matching labels to support multiple environments. Prefect can start, pause, and resume tasks at any time, allowing manual steps like review and approval, and it can give flows access to sensitive information at runtime, including API keys or passwords.

If a task crashes unexpectedly, Prefect can restart it autonomously via special “Lazarus” agents. And the platform alerts stakeholders when an agent goes offline, exposing logs for streaming, filtering, and searching.

Above: Prefect’s online dashboard.

Image Credit: Prefect

Lowin says Prefect had a banner year in 2020, with 130% quarter-over-quarter usage growth since February 2020. The company recently announced a relationship with Microsoft for Startups to advance dataflow automation, and it claims its platform is now processing 25 million tasks per month and 2 million workflows per month.

Prefect’s success aligns with broader industry trends in digital transformation. According to McKinsey, data flows have raised the global gross domestic product by at least 10% over a decade. The value totaled $7.8 trillion in 2014 alone, contributing to economic growth primarily by raising productivity.

Market Research Future predicts the global data analytics market alone will be valued at over $132 billion by 2026. A range of organizations can use data to boost their marketing strategies, increase their bottom line, personalize their content, and better understand their customers. In fact, businesses that use big data increase their profits by an average of 8%, according to a survey conducted by BARC research.

Bessemer Venture Partners also participated in Prefect’s most recent funding round. This brings the Washington, D.C.-based company’s total raised to date to over $57 million.

Source Link