How To Set Time-To-Live(TTL) Attribute For Amazon DynamoDB entries?

Learn how to make Amazon DynamoDB(DDB) tables clean up itself.

You can make Amazon DynamoDB table entries expire automatically by setting a time-to-live timestamp attribute. This timestamp is in Unix Epoch time format which is number of seconds that have elapsed since 1 January 1970 00:00:00. Your DDB entries will be scheduled to be deleted if current Unix Epoch time is greater than its time-to-live value.

In case you wonder what epoch means, the definition of Epoch on Google Dictionary is a period of time in history or a person's life, typically one marked by notable events or particular characteristics.

Possible use cases

1. Keep only relatively new data for debugging or analytics.
2. Expire users’ periodic usage data so that usage quota can be reset at a specific time.
3. Keep data temporary for pattern searching and usage tracking.

Setup

You need to configure your DDB tables to set up time-to-live attribute using AWS CLI or AWS DynamoDB Console.

1 AWS CLI

You can use AWS DynamoDB CLI update-time-to-live command. The following is the command synopsis.

update-time-to-live
--table-name <value>
--time-to-live-specification <value>
[--cli-input-json <value>]
[--generate-cli-skeleton <value>]

Example

aws dynamodb update-time-to-live
--table-name demoTable
--time-to-live-specification Enabled=true,AttributeName=ttl

2 AWS DynamoDB Console

1. Open up your AWS console and navigate to DynamoDB.

2. Select a table that you want to set up time-to-live.

3. In the table details, there is a ‘Time to live attribute’ item. Click Manage TTL button next to it.

4. The following modal will show up, you can write your attribute name in the TTL attribute input.

Python Code to set ttl

You can refer to AWS Boto3 DynamoDB document for AWS DynamoDB APIs

Write to DDB

import datetime
import time
import boto3

def write_to_ddb(key, data):
  dynamodbClient = boto3.client('dynamodb')
  # expires after one week  
  expiryTimestamp = int(time.time() + 24*3600*7)

  # OR
  # week = datetime.datetime.today() + \
  #  datetime.timedelta(days=7)
  # expiryTimestamp = int(time.mktime(week.timetuple()))
  try:
      dynamodbClient.put_item(
          TableName = os.environ['DEMO_TABLE'],
          Item = {
              'id': {
                  'S': key
              },
              'data': {
                  'S': data
              }
              'ttl': {
                  'N': str(expiryTimestamp) 
              }
          }
      )
      return True
  except Exception as e:
      print('Exception: ', e)
      return False

Read from DDB

Be cautious that DynamoDB does not delete expired items immediately. On Aws DynamoDB update-time-to-live command document, it states that expired items are removed within 2 days or 48 hours from expiration time. And, these supposedly expired items will still show up in read, query and scan operations.

You need to have FilterExpression = '#t > :ttl' to make sure the retrieved entries aren’t expired.

import datetime
import time
import boto3

def get_from_ddb(key):
    dynamodbClient = boto3.client('dynamodb')
    epochTimeNow = int(time.time())
    try:
        res = dynamodbClient.query(
            TableName = os.environ['DEMO_TABLE'],
            KeyConditionExpression = '#id = :id',
            FilterExpression = '#t > :ttl',
            ExpressionAttributeNames = {
                '#t': 'ttl',
                '#id': 'id'
            },
            ExpressionAttributeValues = {
                ':ttl': {
                    'N': str(epochTimeNow),
                },
                ':id': {
                    'S': key
                }
            }
        )
                
        if 'Items' in res:
            if len(res['Items']) >= 1:
                return res['Items']
        return None
    except Exception as e:
        print('Exception: ', e)
        return None

Caveat

1. Note that your expiry timestamp(number of seconds since Epoch time) should be cast into Integer (no decimals).
2. The attribute type for ttl should be Number. However, the value in the argument should be a String.

   expiryTimestamp = int(time.time() + 24*3600*7)
   ExpressionAttributeValues = {
     'ttl': {   
       'N': str(expiryTimestamp) 
     }
   }
   

With this, you can have your DynamoDB tables remove irrelevant entries automatically.

Further Reading

To learn how to verify if TTL is configured properly for your DynamoDB tables, check out How to Verify TTL Configuration For Amazon DynamoDB Table article.

Support Jun

Thank you for reading!    Support Jun

Support Jun on Amazon US

Support Jun on Amazon Canada

If you are preparing for Software Engineer interviews, I suggest Elements of Programming Interviews in Java for algorithm practice. Good luck!

You can also support me by following me on Medium or Twitter.

Feel free to contact me if you have any questions.