Build a Storage Solution

By Pulumi Team
Published
Updated

The Challenge

Applications need durable, secure file storage that protects data while controlling costs as storage grows. S3 provides the storage layer, but getting encryption, access controls, versioning, and lifecycle management configured correctly requires coordinating multiple settings.

What You'll Build

  • S3 bucket with server-side encryption enabled
  • Versioning configured for data protection and recovery
  • Lifecycle policies for automatic cost optimization
  • IAM role with least-privilege access for applications

Neo Try This Prompt in Pulumi Neo

Run this prompt in Neo to deploy your infrastructure, or edit it to customize.

Best For

Use this prompt when you need to set up production-ready file storage on AWS with proper security and cost management. Applicable to application data, user uploads, log archives, backups, or any workload that stores files in S3 and needs to follow operational best practices.

Architecture Overview

This deployment creates an S3 bucket with the security and operational features that production workloads require. Encryption protects data at rest, versioning preserves previous copies of objects so accidental overwrites or deletions can be reversed, and lifecycle policies move aging data to cheaper storage classes automatically. An IAM role provides scoped access so applications can interact with the bucket without using overly broad permissions.

These features work together as layers of protection and cost management. Encryption ensures that data stored on disk is unreadable without the correct key, even if someone gains access to the underlying storage hardware. Versioning creates a recoverable history of every object, acting as a lightweight backup mechanism. Lifecycle policies prevent storage costs from growing unboundedly by transitioning data that is no longer actively accessed to archival tiers. The IAM role limits the blast radius if application credentials are compromised, because the role can only access this specific bucket.

This pattern applies broadly. Whether you are storing user uploads, application logs, database backups, or static assets, the combination of encryption, versioning, lifecycle management, and scoped access represents the baseline configuration that most organizations require for any S3 bucket holding non-trivial data.

Server-Side Encryption

S3 server-side encryption (SSE) encrypts objects when they are written to disk and decrypts them when they are read. AES-256 encryption (SSE-S3) uses Amazon-managed keys, which requires no key management on your part. For workloads with stricter compliance requirements, you can use SSE-KMS with a customer-managed KMS key to gain control over key rotation, access policies, and audit trails through CloudTrail.

Bucket Versioning

Versioning keeps every version of every object stored in the bucket. When an object is overwritten, S3 retains the previous version. When an object is deleted, S3 adds a delete marker rather than removing the data. You can restore previous versions at any time, which protects against application bugs that corrupt data, accidental deletions by users, and ransomware that encrypts files. Versioning pairs well with lifecycle policies that expire old versions after a retention period to control costs.

Lifecycle Policies

Lifecycle rules automate storage class transitions based on object age. A common pattern moves objects to S3 Standard-IA (Infrequent Access) after 30 days, then to S3 Glacier after 90 days for long-term archival. You can also configure rules to expire (delete) objects or old versions after a specified period. These transitions happen automatically, so storage costs decrease over time without manual intervention or cleanup scripts.

IAM Role with Least-Privilege Access

The IAM role grants an application exactly the S3 permissions it needs and nothing more. A typical policy allows s3:GetObject, s3:PutObject, and s3:ListBucket on the specific bucket ARN. The role does not grant permissions to other buckets, other AWS services, or administrative actions like deleting the bucket or changing its configuration. Applications assume this role through instance profiles (EC2), task roles (ECS), or execution roles (Lambda).

Common Customizations

  • Add cross-region replication: Enable replication to a bucket in another AWS region for disaster recovery or to serve data closer to users in multiple geographies.
  • Use KMS encryption: Replace SSE-S3 with SSE-KMS for customer-managed encryption keys, giving you control over key rotation and the ability to audit key usage through CloudTrail.
  • Add event notifications: Configure S3 event notifications to trigger a Lambda function, SQS queue, or SNS topic when objects are created, deleted, or restored.
  • Enable access logging: Turn on S3 server access logging to record every request made to the bucket, providing an audit trail for compliance and security analysis.