The moment I learnt that S3 Object Lambda was out, I knew I’d want to experiment with it. Why? For two reasons really – at work we have quite a few scenarios when the same objects in S3 need to be presented in different shapes or forms, data extracted, or content transformed. So I’ve volunteered to speak about it and its use cases. The second reason is to practise using AWS CDK for C# more – I’ve mentioned a few times that there are very few examples in C#, and I thought it’s be a good idea to provide one more and hopefully make someone’s life easier.
In a nutshell, S3 Object Lambda allows you to amend the data that you usually get by using S3 Get requests. The main characters in the story are:
An S3 Bucket where we drop files we want to transform:
A Lambda function which will do the transformation. My Lambda function does a simple XML transformation:
What is worth noting here is that:
- The request that is sent from the Lambda contains getObjectContext property that contains inputs3Url which you use to get the original object
- You need to use a new WriteGetObjectResponseRequest method that is used to include the transformed content. WriteGetObjectResponseAsync sends the transformed object when Object Lambda Access Points are called.
- RequestToken allows the Lambda to connect the response with the caller.
Before we start looking at the main part of the CDK stack, I’m going to use namespace aliases to save some typing:
We then define the resources in the stack:
We also need to give the Lambda execution role appropriate permissions like so (if you don’t, ERROR: Forbidden will be returned from WriteGetObjectResponseAsync) :
Now the fun part starts. In CDK we have to use L1 constructs to create Access Points. First we create a S3 Access Point (a way of managing access to certain objects). It’s used to return original S3 objects.
And finally, an S3 Object Lambda Access Point. This is the access point that should be used in the application when making a GetObject request.
A few things here:
- the Actions array will always have only one element “GetObject” since it’s the only operation supported with the Object Lambda.
- with CDK in C# you need to use a Dictionary when there are Javascript arrays or untyped objects. I must say this bit is so much simpler in Typescript!
Here are some other findings:
- This error occurred when I changed the name of the Access Point to contain any capital letters or hyphens. So I just left it in lowercase.
2. In the ContentTransformation container you can also send FunctionPayload, and customize the behaviour of the function based on that payload.
3. To use the Object Lambda Access Point all you need to do is to replace the BucketName value of GetObjectRequest with the ARN of the Object Lambda Access Point:
It took me a while to build the stack since I’ve not worked with L1 Constructs much before. A huge thanks to Petra Novandi and the CDK team for giving me a hand, helping me to learn how S3 Object Lambda works, improving my knowledge of the CDK and enabling me to share my learnings with the world.
Useful resources: