Snowflake previews data marketplace, governance tools, management features

Snowflake today announced a marketplace which would allow organizations to sell data based on usage. The marketplace, now generally available, was one of several announcements regarding new data monetization, governance, and performance features for the cloud platform.

Announced during the online Snowflake Summit, the new capabilities will make the Snowflake Data Marketplace more accessible to a wider range of organizations that don’t today typically monetize their data, said Christian Kleinerman, senior vice president of product for Snowflake.

There are now more than 160 providers of data services participating in that marketplace. The bulk of those organizations sell data commercially as a core part of their business model. However, roughly 20 to 30% of organizations might be able to monetize their data if it was simple enough for them to sell, added Kleinerman. The biggest obstacle those organizations face is they don’t have the time or resources required to set up and maintain a data marketplace of their own, he noted. They can only sell data if it doesn’t distract them from their primary mission, he added. “It’s not their primary business,” said Kleinerman.

Snowflake today also announced forthcoming editions to its platform, including a Snowpark applications development environment available via a private preview. That development platform currently supports Java and Scala programming languages. Support for additional languages is planned for future release.

The company also previewed Java user-defined-functions (UDFs) which would enable IT teams to deploy custom code and business logic they have developed on the Snowflake platform, as well as a Snowflake SQL application programming interface (API) through which applications will be able to directly invoke Snowflake.

Snowflake also discussed new governance tools for managing unstructured data stored on a Snowflake platform that is based on a relational database engine. There is also a classification tool that automatically detects personally identifiable information (PII) in a table and uses the tagging framework to annotate the data in a way that makes it possible for IT teams to enforce compliance policies.

The company committed to adding support for Anonymized views that will better protect the privacy and identity as a dataset by automatically redacting sensitive data.

In terms of daily operations of the platform, Snowflake previewed a usage dashboard to enable customers to more easily track consumption of resources and revealed it has been employing a new compression algorithm to reduce the size of data sets anywhere from nine to 30% depending on the type of data, said Kleinerman. Snowflake also previewed updates that can result in up to a six-fold improvement on query throughput on a single compute cluster and an up to eight-fold improvement in average query duration.

Finally, Snowflake said its platform can now also be used as a data source for Amazon SageMaker Data Wrangler, a tool the cloud service provider makes available to collect data for training AI models, and that it is rolling out a Powered by Snowflake branding initiative for partners that build applications on top of its cloud platform.

Snowflake as a cloud service running on the Amazon Web Services cloud has gained a significant amount of traction despite the fact AWS offers rival services. In addition, competitors are making a case for data lakes based on object-storage systems that eliminate the need for a separate platform to query a database. The core capabilities that differentiate Snowflake are how tightly coupled data sets are made available and the overall governance capabilities that capability enables Snowflake to provide, said Kleinerman.

It’s not clear how the battle between data platforms in the cloud will play out. The line between what constitutes a data warehouse versus a data lake is getting blurry. The one thing that is clear is that as more data gets stored in the cloud the more challenging organization will find it to bring order to a lot of potential chaos.

Source Link