Serverless App to Copy and Zip Objects Between Amazon S3 Buckets

This example lives in the pulumi/examples repository. Check out just this directory to use it:

$ git clone --filter=blob:none --sparse https://github.com/pulumi/examples pulumi-examples
$ git -C pulumi-examples sparse-checkout set aws-ts-s3-lambda-copyzip
$ cd pulumi-examples/aws-ts-s3-lambda-copyzip

This example sets up two AWS S3 Buckets and a single Lambda that listens to one and, upon each new object arriving in it, zips it up and copies it to the second bucket. Its architecture looks like this:

This example is also featured in the blog post Easy Serverless Apps and Infrastructure — Real Events, Real Code.

Deploying the App#

To deploy your new serverless application, follow the below steps.

Prerequisites#

0. Ensure you have Node.js
1. Install Pulumi
2. Configure AWS Credentials

Steps#

After cloning this repo, from this working directory, run these commands:

0. Install Node.js dependencies, either using NPM or Yarn:

   $ npm install

1. Create a new Pulumi stack, which is an isolated environment for this example:

   $ pulumi stack init

   This will ask you to give your stack a name; dev is a fine name to begin with.

2. Configure the AWS region for this program — any valid AWS region will do:

   $ pulumi config set aws:region us-east-1

3. Deploy the application:

   $ pulumi up

4. After about 20 seconds, your buckets and lambda will have been deployed. Their names are printed:

   Outputs:
       tpsReportsBucket: "tpsreports-21b7b7a"
       tpsZipsBucket   : "tpszips-c869600"

5. Now copy a file to the tpsReportsBucket using the AWS CLI:

   $ aws s3 cp ./myTpsReport001.txt s3://$(pulumi stack output tpsReportsBucket)

6. Tail the logs to see evidence the Lambda ran:

   $ pulumi logs -f
   Collecting logs for stack dev since 2019-03-10T10:09:56.000-07:00...
   2019-03-10T11:10:48.617-07:00[zipTpsReports] Zipping
       tpsreports-96458ef/tps001.txt into tpszips-edfde11/tps001.txt.zip

7. ^C out of pulumi logs -f, and then download your new zipfile!

   $ aws s3 cp s3://$(pulumi stack output tpsZipsBucket)/myTpsReport001.txt.zip .

8. Once you’re done, destroy your stack and remove it — eliminating all traces of running:

   # First, delete files so we can destroy the buckets (by default, bucket content isn't auto-deleted):
   $ aws s3 rm s3://$(pulumi stack output tpsReportsBucket)/myTpsReport001.txt
   $ aws s3 rm s3://$(pulumi stack output tpsZipsBucket)/myTpsReport001.txt.zip
   $ pulumi destroy --yes
   $ pulumi stack rm --yes