Encryption Key Storage for AI Applications with Vault AWS

Question

Pulumi · Accepted Answer

To securely manage encryption keys for AI applications on AWS with HashiCorp Vault, we'll take advantage of a configuration that involves creating an AWS KMS (Key Management Service) to generate and manage cryptographic keys, an AWS S3 bucket to store application data, and Vault's AWS Secret Backend to manage the access to AWS resources.

Here's how the integration typically works:

1. **AWS KMS**: Create a customer master key (CMK) that will be used to encrypt and decrypt the application data.

2. **AWS S3 Bucket**: Create an S3 bucket that will serve as the storage for the AI application data. This S3 bucket will be encrypted using the key generated by AWS KMS.

3. **Vault AWS Secret Backend**: Configure Vault's AWS Secret Backend to generate dynamic, short-lived AWS credentials with permissions to access the S3 bucket. Vault will use these credentials to access the application's data securely, and since Vault controls the creation and revocation of these credentials, you'll have a secure method of key management.

Let's walk through the Pulumi Python program to set this up step-by-step:

```python
import pulumi
import pulumi_aws as aws
import pulumi_vault as vault

# Step 1: Create an AWS KMS Key for encrypting your S3 bucket data.
kms_key = aws.kms.Key("ai-app-kms-key",
                      description="KMS key for AI application data")

# Step 2: Create an S3 Bucket for your AI application encrypted with the KMS key.
s3_bucket = aws.s3.Bucket("ai-app-s3-bucket",
                          server_side_encryption_configuration=aws.s3.BucketServerSideEncryptionConfigurationArgs(
                              rule=aws.s3.BucketServerSideEncryptionConfigurationRuleArgs(
                                  apply_server_side_encryption_by_default=aws.s3.BucketServerSideEncryptionConfigurationRuleApplyServerSideEncryptionByDefaultArgs(
                                      sse_algorithm="aws:kms",
                                      kms_master_key_id=kms_key.id))))

# Step 3: Configure Vault's AWS Secret Backend.
# Enables AWS Secret Backend for Vault.
aws_secrets_backend = vault.aws.SecretBackend("ai-app-backend",
                                              description="AWS secrets backend for AI application",
                                              default_lease_ttl_seconds=3600,
                                              max_lease_ttl_seconds=86400)

# Configure Vault's AWS Secret Backend role that maps IAM policies to credentials.
vault.aws.SecretBackendRole("ai-app-role",
                            backend=aws_secrets_backend.name,
                            credential_type="iam_user",
                            policy_document=pulumi.Output.all(s3_bucket.arn, kms_key.arn).apply(lambda args: json.dumps({
                                "Version": "2012-10-17",
                                "Statement": [{
                                    "Effect": "Allow",
                                    "Action": [
                                        "s3:GetObject",
                                        "s3:PutObject",
                                        "s3:ListBucket"
                                    ],
                                    "Resource": [
                                        f"{args[0]}/*",  # S3 bucket ARN
                                        f"{args[1]}"     # KMS key ARN
                                    ]
                                }]
                            }))
                            )

# Export the bucket name and Vault backend configuration.
pulumi.export("bucket_name", s3_bucket.bucket)
pulumi.export("vault_backend", aws_secrets_backend.path)
```

### Detailed Explanation

**AWS KMS Key**: We create a master key with KMS, ensuring that all data at rest in our S3 bucket can only be accessed by entities with the correct permissions, adding a layer of security at the data level.

**AWS S3 Bucket**: An S3 bucket is configured for the AI application's data. It is set up to use server-side encryption with the KMS key created earlier. This ensures our AI application data is encrypted at rest using a key from KMS, benefiting from the security and management features of KMS.

**Vault AWS Secret Backend**: We set up a Vault backend that will manage AWS-specific secrets. This backend generates AWS credentials dynamically, which means that applications using these credentials can only perform actions for the duration of the credentials' TTL, minimizing the risks associated with long-lived credentials.

**Vault AWS Secret Backend Role**: We create a role in Vault associated with a policy that allows for actions `GetObject`, `PutObject`, and `ListBucket` on the S3 bucket. Also, it allows for key management operations, as specified by the ARN of the KMS key. This role generates IAM users with credentials to access the S3 bucket using policies that reference the bucket and KMS key ARNs.

The variables exported at the end, `bucket_name` and `vault_backend`, provide us with the ability to know the identifiers of the created resources, which could be useful for debugging, further automation, or integration with other processes or applications.

By using Pulumi to define both your infrastructure and Vault secret management in code, you create a single source of truth that is version-controlled, allowing your infrastructure to be reproducible, auditable, and easily managed.