AWS Lambda PII Handling in Production: DynamoDB Field Encryption with KMS

Introduction

In the modern cloud-native world, AWS Lambda and DynamoDB are widely used to build serverless applications. However, handling Personally Identifiable Information (PII) in such systems can be challenging due to the default security posture of these services. While AWS provides encryption at rest for DynamoDB, it does not protect application data from unauthorized access. This article presents a production-tested pattern for encrypting PII (e.g., user's home address, email, name, surname, date of birth) in AWS Lambda before storing it in DynamoDB using customer-managed KMS keys. The goal is to ensure that sensitive data is encrypted and only authorized components can decrypt it if necessary.

The full code for this pattern, which includes CRUD operations handled by Lambda, is available on GitHub here.

Why This Pattern?

There are numerous serverless systems built on AWS Lambda and DynamoDB that store PII in plaintext. This pattern addresses the need to protect such data by encrypting it at the application layer. The infrastructure is defined using AWS CDK in Python, with a Lambda runtime also in Python. Encryption keys are stored in AWS Key Management Service (KMS) with explicit key policies and a strictly defined Lambda execution role.

This design does not aim to solve encrypted search or complex querying over sensitive fields, as these require different trade-offs. The focus is on limiting access to PII fields that could lead to legal issues in case of a data breach.

DynamoDB server-side encryption is mandatory in any environment but insufficient for PII protection. It protects against physical and low-level threats but does not prevent plaintext access through IAM. Any resource with IAM access to the table and the ability to scan or query data can become a vulnerability.

The solution is to ensure sensitive fields are never stored in plaintext. Encryption in the application layer transforms any database into a persistence layer for ciphertext. Data decryption becomes an explicit operation, accessible only to authorized compute resources with the correct IAM permissions.

The responsibility for key rotation, correct IAM scoping, and preventing plaintext leaks lies with the application developer.

Architecture Overview

The architecture consists of three main components:

Lambda Function: Acts as the entry point for user payloads from API Gateway.
AWS KMS: Manages encryption keys for PII fields.
DynamoDB: Stores encrypted data.

The flow is straightforward:

User payload enters a Lambda function via API Gateway.
Sensitive fields are encrypted using AWS KMS.
Encrypted data is written to DynamoDB.

When reading and interpreting data, the Lambda function explicitly decrypts the fields using the same KMS key from the encryption context.

Implementation

Step 1: Set Up AWS CDK and Lambda

First, create an AWS CDK project and define the Lambda function. The Lambda runtime is Python, and it will handle all CRUD operations with DynamoDB.

python

from aws_cdk import (
    aws_lambda as lambda_,
    aws_dynamodb as dynamodb,
    aws_iam as iam
)

class PiiLambdaStack(stack.Stack):
    def __init__(self, scope: construct.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        # Define the Lambda function
        self.lambda_fn = lambda_.Function(
            self, "PiiLambda",
            runtime=lambda_.Runtime.PYTHON_3_9,
            handler="handler.lambda_handler",
            code=lambda_.Code.from_asset("lambda"),
            environment={
                "DYNAMODB_TABLE": dynamodb.Table.from_table_name(
                    self, "PiiTable", "pii-table"
                ).table_name
            }
        )

        # Define the DynamoDB table
        self.dynamodb_table = dynamodb.Table(
            self, "PiiTable",
            partition_key={
                "name": dynamodb.AttributeType.STRING
            },
            encryption_enabled=True,
            removal_policy=core.RemovalPolicy.DESTROY
        )

        # Define the IAM role for Lambda
        self.lambda_role = iam.Role(
            self, "PiiLambdaRole",
            assumed_by=iam.ServicePrincipal("lambda.amazonaws.com"),
            managed_policies=[iam.ManagedPolicy.from_aws_managed(
                "service-role/AWSLambdaBasicExecutionRole"
            )],
            additional_policies=[
                iam.Policy.from_json_string(
                    json.dumps({
                        "Version": "2012-10-17",
                        "Statement": [
                            {
                                "Effect": "Allow",
                                "Action": [
                                    "dynamodb:PutItem",
                                    "dynamodb:GetItem",
                                    "dynamodb:Query",
                                    "dynamodb:Scan"
                                ],
                                "Resource": self.dynamodb_table.table_arn
                            },
                            {
                                "Effect": "Allow",
                                "Action": [
                                    "kms:Encrypt",
                                    "kms:Decrypt",
                                    "kms:ReEncrypt*",
                                    "kms:GenerateDataKey*",
                                    "kms:DescribeKey"
                                ],
                                "Resource": "*"
                            }
                        ]
                    })
                )
            ]
        )

        # Attach the IAM role to the Lambda function
        self.lambda_fn.add_to_role_policy(self.lambda_role)

Step 2: Encrypt and Decrypt PII Fields

In the Lambda function, use the AWS SDK for Python (Boto3) to interact with KMS for encryption and decryption.

python

import boto3
from botocore.exceptions import ClientError

kms = boto3.client('kms')

def encrypt_pii(data, key_id):
    try:
        response = kms.encrypt(
            KeyId=key_id,
            Plaintext=data.encode()
        )
        return response['CiphertextBlob']
    except ClientError as e:
        raise Exception(f"Error encrypting PII: {e}")

def decrypt_pii(ciphertext_blob, key_id):
    try:
        response = kms.decrypt(
            CiphertextBlob=ciphertext_blob,
            KeyId=key_id
        )
        return response['Plaintext'].decode()
    except ClientError as e:
        raise Exception(f"Error decrypting PII: {e}")

Step 3: Handle CRUD Operations

Modify the Lambda handler to encrypt PII fields before storing them in DynamoDB and decrypt them when reading.

python

def lambda_handler(event, context):
    # Example event payload
    payload = {
        "name": "John Doe",
        "email": "john.doe@example.com",
        "dob": "1980-01-01"
    }

    # Encrypt PII fields
    encrypted_name = encrypt_pii(payload['name'], 'your-kms-key-id')
    encrypted_email = encrypt_pii(payload['email'], 'your-kms-key-id')
    encrypted_dob = encrypt_pii(payload['dob'], 'your-kms-key-id')

    # Store encrypted data in DynamoDB
    table = boto3.resource('dynamodb').Table(os.environ['DYNAMODB_TABLE'])
    table.put_item(
        Item={
            'name': encrypted_name,
            'email': encrypted_email,
            'dob': encrypted_dob
        }
    )

    # Retrieve and decrypt data
    response = table.get_item(
        Key={
            'name': encrypted_name
       

(Note: This article was automatically expanded from news sources.)

AWS Lambda PII Handling in Production: DynamoDB Field Encryption with KMS

AWS Lambda PII Handling in Production: DynamoDB Field Encryption with KMS

Introduction

Why This Pattern?

Architecture Overview

Implementation

Step 1: Set Up AWS CDK and Lambda

Step 2: Encrypt and Decrypt PII Fields

Step 3: Handle CRUD Operations

Related Reading

What Predicts a Hit? I Trained 3 ML Models to Find Out

I Love Detailed Releases. I Hate Doing Them.

Multichannel AI Agent: Shared Memory Across Messaging Platforms