Amazon launches S3 Object Lambda to transform data across applications

Apps in the cloud each have their own requirements and sometimes need a different view of data in a dataset. For example, an ecommerce  database might include personally identifiable information that isn’t required when the data is processed for analytics. On the other side, if the ecommerce dataset is used for a marketing campaign, it might need to be enriched with additional details like information from a customer loyalty database.

To address this modality challenge, Amazon today launch S3 Object Lambda, a new capability that allows developers to add their own code to process data retrieved from Amazon Web Services (AWS) Simple Storage Service (S3) before returning it to an app. Amazon says that S3 Object Lambda works with existing apps, leveraging AWS Lambda functions to automatically transform data as it’s being retrieved from an S3 bucket.

Prior to the rollout of S3 Object Lambda, AWS offered two primary ways to provide different views of data to multiple applications. Enterprises could create, store, and maintain additional derivative copies of the data, so that each app had its own custom dataset tailored to its needs. Alternatively, they could build and manage infrastructure as a proxy layer in front of S3 to intercept and process data as it was requested.

But both options added complexity and costs.

With S3 Object Lambda, Amazon says that companies can more easily automate tasks like converting across data formats, compressing or decompressing files as they’re being downloaded, resizing and watermarking images, and implementing custom authorization rules to access databases. Using S3 Object Lambda, developers can present multiple views from the same dataset and update the necessary Lambda functions to modify these views at any time.

“When retrieving an object using S3 Object Lambda, there is no need for an object with the same name to exist in the S3 bucket. The Lambda function can use information in the name of the file or in the HTTP headers to generate a custom object,” AWS chief evangelist Danilo Poccia explained in a blog post. “This new capability makes it much easier to share and convert data across multiple applications.”

S3 Object Lambda is available today in all AWS Regions excepting Asia Pacific (Osaka), AWS GovCloud (US-East), AWS GovCloud (US-West), China (Beijing), and China (Ningxia) Regions. Customers can use S3 Object Lambda with the AWS Management Console, AWS Command Line Interface, and AWS SDKs. They pay for the AWS Lambda compute and request charges required to process the data, the data S3 Object Lambda returns to their applications, and S3 requests invoked by Lambda functions.

To begin using S3 Object Lambda, follow these steps suggested by Amazon:

1. Create a Lambda Function to transform data for your use case.
2. Create an S3 Object Lambda Access Point from the S3 Management Console.
3. Select the Lambda function that you created above.
4. Provide a supporting S3 Access Point to give S3 Object Lambda access to the original object.
5. Update your application configuration to use the new S3 Object Lambda Access Point to retrieve data from S3.

By VentureBeat Source Link