January 28, 2018 Vignesh Sathiyanantham 29Comment

AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume – there is no charge when your code is not running.

With Lambda, you can run code for virtually any type of application or backend service – all with zero administration. Just upload your code and Lambda takes care of everything required to run and scale your code with high availability. You can set up your code to automatically trigger from other AWS services or call it directly from any web or mobile app.

In my last post, i wrote about Uses and event triggers

We are working on a Cloud data protection and DR related produce, usually, the developer in my team does unit testing with create_snapshot API and they forgot to delete the snapshot or they test the scheduler with 5-minute interval and leaves it. End of the day we will be having tons of useless snapshots from different Amazon EBS volumes in different regions which add more to the monthly bill.

I created a simple AWS Lambda function which deletes the EBS snapshots older than one day

Create a Lambda Service

  • Log in to the AWS console
  • Click on Lambda
  • Skip the blueprint and create a new function
    • Set some name, description
    • Runtime to Python 3.6
  • Paste the code, save and Test.

In delete_snapshots days value is hardcoded to 1 that can be replaced using events. Events are the key-value pair that can be accessed in function using event[‘key’].

In the above code I set the filter description “Created by Nimesa” , you can set filter by tags or any other parameter

Setup Execution role

  • You can choose the existing role or create new
  • When accessing EBS snapshots Lambda should have EC2 access to list and delete snapshots

Timeout Settings

In basic settings setup the timeout value and memory, these setting allows you to control the code execution performance and costs for your Lambda function. Changing your resource settings (by selecting memory) or changing the timeout may impact your function cost.

Add Triggers

delete_snapshots_lambda_events

  • Create a cloud watch event
  • Rule: Fixed rate of 1 Days
    • The function will be triggered by the schedule of 1 day
  • You can add API Gateway also to call like web API

29 thoughts on “AWS Lambda : Delete old EBS snapshots using Boto3

  1. Hi Vignesh

    It’s very useful post, i have one requirement, I’m making daily backup snapshots for production servers, i want to delete more than 15 days old snapshots from particular EBS Volumes, how to mention that particular volume id’s in the above code?

    Thanks in advance!

  2. Great write up! I have a few questions…As you can see in my code, I look for the description “Created by Lambda backup function ebs-snapshots”. But when I test this, I see no snapshots have been removed.

    from datetime import datetime, timedelta, timezone

    import boto3

    class Ec2Instances(object):

    def __init__(self, region):
    print(“region “+ region)
    self.ec2 = boto3.client(‘ec2’, region_name=region)

    def delete_snapshots(self, older_days=0):
    delete_snapshots_num = 0
    snapshots = self.get_lambda_created_snapshots()
    for snapshot in snapshots[‘Snapshots’]:
    fmt_start_time = snapshot[‘StartTime’]
    if (fmt_start_time < self.get_delete_data(older_days)):
    self.delete_snapshot(snapshot['SnapshotId'])
    delete_snapshots_num+1
    return delete_snapshots_num

    def get_lambda_created_snapshots(self):
    snapshots = self.ec2.describe_snapshots(Filters=[{'Name': 'description', 'Values': ['Created by Lambda backup function ebs-snapshots']}])
    return snapshots

    def get_delete_data(self, older_days):
    delete_time = datetime.now(tz=timezone.utc) – timedelta(days=older_days)
    return delete_time;

    def delete_snapshot(self, snapshot_id):
    self.ec2.delete_snapshot(SnapshotId=snapshot_id)

    def lambda_handler(event, context):
    print("event " + str(event))
    print("context " + str(context))
    ec2_reg = boto3.client('ec2')
    regions = ec2_reg.describe_regions()
    for region in regions['Regions']:
    region_name = region['RegionName']
    instances = Ec2Instances(region_name)
    deleted_counts = instances.delete_snapshots(1)
    print("deleted_counts for region "+ str(region_name) +" is " + str(deleted_counts))
    return 'completed'
    ___
    START RequestId: 2c29ab3d-8113-11e8-bf58-f5dde8cf91a3 Version: $LATEST
    event {'key3': 'value3', 'key2': 'value2', 'key1': 'value1'}
    context
    region ap-south-1
    deleted_counts for region ap-south-1 is 0
    region eu-west-3
    deleted_counts for region eu-west-3 is 0
    region eu-west-2
    deleted_counts for region eu-west-2 is 0
    region eu-west-1
    deleted_counts for region eu-west-1 is 0
    region ap-northeast-2
    deleted_counts for region ap-northeast-2 is 0
    region ap-northeast-1
    deleted_counts for region ap-northeast-1 is 0
    region sa-east-1
    deleted_counts for region sa-east-1 is 0
    region ca-central-1
    deleted_counts for region ca-central-1 is 0
    region ap-southeast-1
    deleted_counts for region ap-southeast-1 is 0
    region ap-southeast-2
    deleted_counts for region ap-southeast-2 is 0
    region eu-central-1
    deleted_counts for region eu-central-1 is 0
    region us-east-1
    deleted_counts for region us-east-1 is 0
    region us-east-2
    deleted_counts for region us-east-2 is 0
    region us-west-1
    deleted_counts for region us-west-1 is 0
    region us-west-2
    deleted_counts for region us-west-2 is 0
    END RequestId: 2c29ab3d-8113-11e8-bf58-f5dde8cf91a3
    REPORT RequestId: 2c29ab3d-8113-11e8-bf58-f5dde8cf91a3 Duration: 33069.49 ms Billed Duration: 33100 ms Memory Size: 128 MB Max Memory Used: 39 MB

  3. Hello-

    I am filtering by description: “Created by Lambda backup function ebs-snapshots”, however, when I test the function, I can see nothing gets removed.

    Here is my code and results of the test:

    START RequestId: cb7982ed-8113-11e8-9ad5-25207a29664d Version: $LATEST
    event {‘key3’: ‘value3’, ‘key2’: ‘value2’, ‘key1’: ‘value1’}
    context
    region ap-south-1
    deleted_counts for region ap-south-1 is 0
    region eu-west-3
    deleted_counts for region eu-west-3 is 0
    region eu-west-2
    deleted_counts for region eu-west-2 is 0
    region eu-west-1
    deleted_counts for region eu-west-1 is 0
    region ap-northeast-2
    deleted_counts for region ap-northeast-2 is 0
    region ap-northeast-1
    deleted_counts for region ap-northeast-1 is 0
    region sa-east-1
    deleted_counts for region sa-east-1 is 0
    region ca-central-1
    deleted_counts for region ca-central-1 is 0
    region ap-southeast-1
    deleted_counts for region ap-southeast-1 is 0
    region ap-southeast-2
    deleted_counts for region ap-southeast-2 is 0
    region eu-central-1
    deleted_counts for region eu-central-1 is 0
    region us-east-1
    deleted_counts for region us-east-1 is 0
    region us-east-2
    deleted_counts for region us-east-2 is 0
    region us-west-1
    deleted_counts for region us-west-1 is 0
    region us-west-2
    deleted_counts for region us-west-2 is 0
    END RequestId: cb7982ed-8113-11e8-9ad5-25207a29664d
    REPORT RequestId: cb7982ed-8113-11e8-9ad5-25207a29664d Duration: 32700.97 ms Billed Duration: 32800 ms Memory Size: 128 MB Max Memory Used: 39 MB

  4. Hi asvignesh ,

    I added volime-id to delete snaps by volume id but it didnot delete as per
    def delete_snapshots(self, older_days=10):
    snapshots = self.ec2.describe_snapshots(Filters=[{‘Name’: ‘volume-id’, ‘Values’: [‘vol-06a4cddddb626fbd6df5’]}])

    It deleted all except one snapshot.

    Nikhil

      1. Thanks Vignesh for getting back quickly on this. Sorry i am new to python .

        Do you mean this way ,
        snapshots = self.ec2.describe_snapshots(Filters=[{‘volume-id key’: ‘volume-id’, ‘Values’: [‘vol-06a4cddddbfbd6df5’]}])

        And for retention to keep for 10days
        def delete_snapshots(self, older_days=10)

        is that write ?

        1. Added – snapshots = self.ec2.describe_snapshots(Filters=[{‘volume-id key’: ‘volume-id’, ‘Values’: [‘vol-06a4cdb626fbd6df5’]}])

          but run came with error

          {
          “errorMessage”: “Parameter validation failed:\nUnknown parameter in Filters[0]: \”volume-id key\”, must be one of: Name, Values”,
          “errorType”: “ParamValidationError”,
          “stackTrace”: [
          [
          “/var/task/lambda_function.py”,
          44,
          “lambda_handler”,
          “deleted_counts = instances.delete_snapshots(1)”
          ],
          [
          “/var/task/lambda_function.py”,
          13,
          “delete_snapshots”,
          “snapshots = self.get_nimesa_created_snapshots()”
          ],
          [
          “/var/task/lambda_function.py”,
          25,
          “get_nimesa_created_snapshots”,
          “snapshots = self.ec2.describe_snapshots(Filters=[{‘volume-id key’: ‘volume-id’, ‘Values’: [‘vol-06a4cdb626fbd6df5’]}])”
          ],
          [
          “/var/runtime/botocore/client.py”,
          314,
          “_api_call”,
          “return self._make_api_call(operation_name, kwargs)”
          ],
          [
          “/var/runtime/botocore/client.py”,
          586,
          “_make_api_call”,
          “api_params, operation_model, context=request_context)”
          ],
          [
          “/var/runtime/botocore/client.py”,
          621,
          “_convert_to_request_dict”,
          “api_params, operation_model)”
          ],
          [
          “/var/runtime/botocore/validate.py”,
          291,
          “serialize_to_request”,
          “raise ParamValidationError(report=report.generate_report())”
          ]
          ]
          }

          1. use snapshots = self.ec2.describe_snapshots(
            Filters=[{‘Name’: ‘volume-id’, ‘Values’: [‘vol-242342342343’]}])

            Replace vol-242342342343 with your volume id

  5. Hello Vignesh!
    Thanks for writing this awesome blog, I have followed the steps given but somehow it’s not deleting the snapshot volume (though I am even used the same tags)
    I’m not getting any error but as you can see below no snapshot volumes are deleted , i do have more than 1 older volumes.
    “`
    START RequestId: fdnh343-d2dc-11e8-82d8-6fddcfffa078 Version: $LATEST
    event {‘key1’: ‘value1’, ‘key2’: ‘value2’, ‘key3’: ‘value3’}
    context
    region ap-south-1
    deleted_counts for region ap-south-1 is 0
    region eu-west-3
    deleted_counts for region eu-west-3 is 0
    region eu-west-2
    deleted_counts for region eu-west-2 is 0
    region eu-west-1
    deleted_counts for region eu-west-1 is 0
    region ap-northeast-2
    deleted_counts for region ap-northeast-2 is 0
    region ap-northeast-1
    deleted_counts for region ap-northeast-1 is 0
    region sa-east-1
    deleted_counts for region sa-east-1 is 0
    region ca-central-1
    deleted_counts for region ca-central-1 is 0
    region ap-southeast-1
    deleted_counts for region ap-southeast-1 is 0
    region ap-southeast-2
    deleted_counts for region ap-southeast-2 is 0
    region eu-central-1
    deleted_counts for region eu-central-1 is 0
    region us-east-1
    deleted_counts for region us-east-1 is 0
    region us-east-2
    deleted_counts for region us-east-2 is 0
    region us-west-1
    deleted_counts for region us-west-1 is 0
    region us-west-2
    deleted_counts for region us-west-2 is 0
    END RequestId: fdnh343-d2dc-11e8-82d8-6fddcfffa078
    REPORT RequestId: fdnh343-d2dc-11e8-82d8-6fddcfffa078 Duration: 30503.52 ms Billed Duration: 30600 ms Memory Size: 128 MB Max Memory Used: 51 MB
    “`

    1. Hi Minnu,

      use ec2.create_snapshot(VolumeId=, Description=) to create a snapshot for the volume, have the proper naming convention ( or store in DB, I feel it’s not much needed) and create another Cloudwatch Schedule expression to trigger

      Have some logic in Tag to identify and delete the oldest snapshot and run it on another schedule expression.

  6. Hello Vignesh,

    I have a requirement to delete the snapshots which are older than 7 days from the current date. What piece of code should I modify from the above provided code.

  7. Thanks.
    Do i also need to change the same value in def delete_snapshots(self, older_days=1): ?

    The above code is working fine, only issue is while triggering lambda status is showing as failed with below error:
    Task timed out after 3.00 seconds.
    When i am increasing the timeout to 4 or 5 sec, it is still giving the error as : Task timed out after 4.00 or 5.00 seconds.
    Any idea how can i fix this?

  8. How can i hard-code the region value to only one region?
    Currently it is trying to get the snapshot details from all the regions which is not my requirement.

    1. for region in regions[‘Regions’]:
      region_name = region[‘RegionName’]
      instances = Ec2Instances(region_name)

      Remove the for loop, and hardcode the region like instances = Ec2Instances(‘us-east-1’)

  9. HI Vignesh,

    I want delete all the snapshots in the account older than 30 days, and there is no Name or Description been added to the snapshots being created right now.

    What changes should I make to the above Python script? Sorry I do not know Python. 🙂

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.