We are all aware that in the AWS-cloud world of today, immutable infrastructure and deployments are preferrable. It is also a fact that if we use immutable deployments, it means we often create multiple Amazon Machine Images (AMIs). To reduce storage costs we might want to delete (or deregister, in AWS speak) these AMIs and associated storage volumes.
In this blog post I will describe how to write a Lambda AMI cleaner function in Python to search and delete unused Amazon Machine Images.
The main part is a Lambda function. It checks the images and deletes them and accompanying EBS snapshots. The function is written in Python, and it uses Boto3, an AWS SDK for Python. It also relies on JMESPath, the query language of the AWS CLI for querying JSON (more on it here). The function takes the following in the “event” argument:
- regions (list of strings): in what region you’d like to run the cleaner
- max_ami_age_to_prevent_deletion (number): if an AMI is older than the specified value, it can safely be deleted
- ami_tags (a map of strings where each object has a tag key and tag value): if an image has the specified tags, it could be a candidate for deletion
Let’s have a look at the helper methods that are used in the Lambda:
1) A method to find AMIs used in autoscaling groups:
Here, by using boto3 we paginate through autoscaling groups in a region. And then we use an equivalent of AWS CLI query to get the details of the autoscaling groups that are most interesting for us:
filtered_asgs = page_iterator.search(f"AutoScalingGroups[*].[InstanceId, LaunchTemplate.LaunchTemplateId,LaunchTemplate.Version]]")
The result we get is a string, and by using this regex: "'(.+?)'" we break down the string into separate variables.
After that we use boto3 ec2 client to extract the AMI Id used in autoscaling groups, and save this value into an array.
2) The next function will get AMI Ids that are used in running EC2s, including those that were not launched using autoscaling:
3) A method that creates AMI filters in the correct format. We pass in values as a map(string) in Terraform, and we need to convert these values into JMESPath format, which is the following:
{
'Name': 'tag:CatName',
'Values': [ 'Boris' ]
}
The method itself looks like this:
4) A function that sends a message to an SNS topic:
The main function, or the handler looks like this:
SUMMARY
Hopefully, this post exemplifies how to do AMI cleanup based on tags using Python. I have learnt a lot from this piece of work, and I hope someone will learn something new about AWS or Python too.