Pulumi Blog: Lambda

AWS Lambda Response Streaming with Pulumi

Isaac Harris — Thu, 06 Apr 2023 00:00:00 +0000

Since its introduction in 2014, AWS Lambda has rapidly expanded its capabilities from simple “functions in the cloud” at launch to a comprehensive serverless platform with support for containerized functions and public per-function URL endpoints.

As serverless applications have increased in sophistication, developers have used functions-as-a-service as a first-class tool in their microservices strategy. As organizations increasingly look to break up their monolithic applications into services, adoption of AWS Lambda has not been a viable option for applications that return payloads larger than the 6 MiB Lambda service limit. In addition to the payload response limit, AWS Lambda has been limited to returning the entire response once it has been fully buffered in memory within the Lambda service. This historical limitation meant that end users would not see any response until the entire response was available. Both of these limitations made Lambda difficult or impossible to use for data-intensive or latency-sensitive use cases.

Introducing Response Streaming

Today, AWS announced Lambda Response Streaming: a new feature that enables response payloads beyond the 6MiB limit, supports binary content in responses and reduces response times for latency-sensitive applications. Response Streaming reduces the time to first byte (TTFB) by enabling your functions to send partial responses to the client as soon as they are ready instead of waiting for a fully-generated and buffered response. This reduces multi-second TTFB to milliseconds, and, you can stream payloads that are gigabytes in size with a soft 20MiB limit for streamed payloads. The maximum throughput for a streamed response is 2MiB/s.

Enabling Response Streaming requires a new Lambda function handler signature that provides a stream object that the function can write to. When your function writes data to this stream, it is sent immediately to the client. To help clients understand the contents of the stream, you can optionally set the Content-Type header for a response.

This feature supports NodeJS v14.x, v16.x and v18.x at launch and comes with an updated billing model that charges for bytes processed and streamed from your Lambda Function.

Authoring a Lambda Function for Response Streaming

For an example of how response streaming works in practice, we’ve built a Lambda function that will stream the Gettysburg Address one sentence at a time with a one second pause between sentences. For brevity, each sentence is abbreviated below (but the code in GitHub, which demonstrates the same function with and without streaming, contains the full text of the speech):

const handleInternal = async (_event, responseStream, _context) => {
 responseStream.setContentType("text/plain");

 const sentences = [
 "Fourscore and seven years ago...",
 "Now we are engaged in a great civil war...",
 "We have come to dedicate...",
 "It is altogether fitting...",
 "But, in a larger sense...",
 "The brave men...",
 "The world will little note...",
 "It is for us the living...",
 ]

 const timer = ms => new Promise( res => setTimeout(res, ms));

 for(let i = 0; i < sentences.length; i++) {
 responseStream.write(`${sentences[i]}\n`);
 await timer(1000);
 }

 responseStream.end();
}

exports.handler = awslambda.streamifyResponse(handleInternal);

The notable changes to the Lambda function code that support response streaming are:

The function must be wrapped in the awslambda.streamifyResponse() middleware. (The streamifyResponse() function is defined in the Node runtime that’s supplied by the Lambda service.)
Handler methods require an additional parameter to the usual (event, context) signature to become (event, responseStream, context).
The responseStream parameter implements Node’s Writable interface to give it a familiar API with other streams that Node developers may be familiar with.
Responses to the client must be written via responseStream.write(), the method that writes to the stream.
The function must call responseStream.end() to signal that no more data is expected.
Note that functions authored for streaming can still be seamlessly invoked with buffered responses.

Streaming Performance Considerations

For this simple text-based example, there is little need to worry about overall performance. However, if the response is large or the client is expected to perform a lot of processing on the response data, AWS suggests using Node’s pipeline() in place of the built-in write() method in case data is produced faster than it can be consumed by the client. pipeline() effectively helps you to pipe between streams, forward errors and properly clean up – providing a callback when the pipeline is complete. You can learn more about this approach in Backpressuring in streams in the Node docs.

Packaging and deploying your Lambda Function for streaming with Pulumi

We can package our Lambda function for streaming with just a few dozen lines of code using Pulumi. First, we’ll need to import some libraries that will allow us to package our Lambda and create the necessary resources in AWS. For this exercise, we’ll be using both the AWS Cloud Control provider and the AWS provider. The AWS Cloud Control provider works with the AWS Cloud Control API to enable same-day support for new AWS features like Lambda response streaming.

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as awsNative from "@pulumi/aws-native";

Next, we’ll create an IAM role for our Lambda function to assume that has the standard required permissions for a Lambda function:

const role = new aws.iam.Role("role", {
 assumeRolePolicy: JSON.stringify({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Principal": {
 "Service": "lambda.amazonaws.com",
 },
 "Action": "sts:AssumeRole",
 }],
 }),
});

new aws.iam.RolePolicyAttachment("role-policy-attachment", {
 role: role.name,
 policyArn: "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole",
});

Streamed responses are not currently supported with API Gateway nor with Application Load Balancers, so they must be invoked via Lambda Function URLs or invoked and streamed through the InvokeWithResponseStream API using the AWS SDK. Let’s create our function, add a function URL, and the necessary permissions to allow unauthenticated access to the function URL. Note that we set the invokeMode input of the function URL to RESPONSE_STREAM. This is what allows us to invoke the function with streaming enabled:

const streamingFunc = new aws.lambda.Function("streaming-func", {
 code: new pulumi.asset.FileArchive("../../lambda"),
 role: role.arn,
 handler: "index.handler",
 runtime: "nodejs14.x",
 timeout: 30,
});

new aws.lambda.Permission("streaming-permission", {
 action: "lambda:InvokeFunctionUrl",
 "function": streamingFunc.arn,
 principal: "*",
 functionUrlAuthType: "NONE",
});

const streamingUrl = new awsNative.lambda.Url("streaming-url", {
 authType: "NONE",
 targetFunctionArn: streamingFunc.arn,
 invokeMode: "RESPONSE_STREAM",
});

Finally, we’ll add a stack export so we can access the generated function URL from outside of our Pulumi program:

exports.streamingUrl = streamingUrl.functionUrl;

To deploy our resources, we run pulumi up -y:

$ pulumi up -y
Previewing update (dev)

View Live: https://app.pulumi.com/jkodrofftest/aws-lambda-streaming/dev/previews/d2e04257-222e-4704-b163-98e6bf4ff704

 Type Name Plan Info
 + pulumi:pulumi:Stack aws-lambda-streaming-dev create
 + ├─ aws:iam:Role role create
 + ├─ aws:iam:RolePolicyAttachment role-policy-attachment create
 + ├─ aws:lambda:Function streaming-func create
 + ├─ aws:lambda:Permission streaming-permission create
 + └─ aws-native:lambda:Url streaming-url create


Outputs:
 streamingUrl: output<string>


Updating (dev)

View Live: https://app.pulumi.com/jkodrofftest/aws-lambda-streaming/dev/updates/25

 Type Name Status Info
 + pulumi:pulumi:Stack aws-lambda-streaming-dev creating (43s)...
 + ├─ aws:iam:Role role created (1s)
 + ├─ aws:iam:RolePolicyAttachment role-policy-attachment created (0.33s)
 + ├─ aws:lambda:Function streaming-func created (22s)
 + ├─ aws-native:lambda:Url streaming-url created (2s)
 + └─ aws:lambda:Permission streaming-permission created (0.58s)

Outputs:
 streamingUrl: "https://s5a3pi4zzsufh5cgaigdxq3wva0psaib.lambda-url.eu-west-1.on.aws/"

Resources:
 + 6 created

Duration: 46s

Reading a Response Stream

Now that we’ve deployed our streaming function, we can read a response stream from the deployed Lambda URL. Per AWS, we can use any client that supports HTTP/1.1 chunked transfer coding. In cURL, streamed responses can be read via the -N flag. We can also add output that displays our time to connect, time to first byte (TTFB), and the total time to return the response:

curl -N -w "Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} \n" $(pulumi stack output streamingUrl)

When successful, your output will look like this:

This example visually shows the advantage of response streaming because the first message appears shortly (approximately 1 second) after invocation. If we disable streaming, the fully buffered response takes several seconds to arrive. You can see the buffered response example in action. This is precisely why TFFB is important: it’s the difference between an app or service that appears to hang while awaiting a response and one that feels much more responsive to an end user.

Wrapping up

AWS Lambda Response Streaming continues the Lambda team’s steady drumbeat of innovation that adds new capabilities and use cases to serverless functions. Grab the sample code and give the feature a try. Then, let us know what you think in the Pulumi Community Slack.

AWS Lambda SnapStart with Pulumi

Josh Kodroff — Mon, 28 Nov 2022 14:00:00 -0800

As AWS Lambda has matured as a serverless platform, there are two key ways the service has evolved:

New capabilities that extend the platform to support new use cases like Lambda Container support, Lambda URLs and attribute-based access control support.
Performance enhancements that enable Lambda functions to be more responsive and cost-effective such as Tiered compilation, and Graviton2 support are just a few examples of the investments AWS made in this space.

With each advancement, the reasons to deploy a full VM or cluster to support your application get fewer and the time to deliver value in the cloud gets shorter.

One challenge that many teams have faced when adapting their services to take advantage of AWS Lambda’s simple deployment model and attractive economics is cold starts: the latency incurred when the service prepares a new execution environment for your function. There is good analysis on the source of this latency as well as some available optimizations here.

Functions using Java runtimes typically experience significantly longer initialization times than functions using other runtimes.

Lambda creates a fresh execution environment when a function is invoked for the first time or when the function gets scaled to handle a spike in traffic. Initialization requires downloading function code, initializing the runtime, and running any pre-invocation initialization code in your function code. (Examples of pre-invocation code are retrieving secrets from AWS Secrets Manager or initializing database or other network connections.) This initialization process can add seconds of latency to the first invocation of your function.

Reducing Function Latency with AWS Lambda SnapStart

Today, AWS released a major improvement for Java-based functions that specifically addresses the latency incurred from cold starts in Java functions: AWS Lambda SnapStart. SnapStart improves startup performance for latency-sensitive Java applications by up to 10x at no extra cost, and typically with no changes to your function code. With SnapStart, instead of initializing function code on invoke, initialization happens when a function version is published. The way AWS describes this new feature is that the Lambda service will take a Firecracker MicroVM snapshot of the memory and disk state of the function’s fully-initialized execution environment, persist the encrypted snapshot and place it in a cache it for rapid access. On a first invocation or on scaling the function, Lambda resumes new environments using this snapshot.

AWS Lambda further improves invoke times by dividing snapshots into chunks. This means only the chunks required for your function are downloaded from a distributed cache on invoke. In real terms, this process removes several seconds of latency from function execution and scaling times.

The new Lambda Telemetry API works with SnapStart so you can easily track platform metrics related to this new capability.

All of the benefits of SnapStart are a great reason to give this feature a try, and there is no additional cost for enabling SnapStart for your functions. However, there are some considerations to keep in mind before you get started:

SnapStart improvements require the Amazon Corretto 11 runtime - Amazon’s production-ready distribution of OpenJDK.
After 14 days without an invocation, your function version becomes inactive and Lambda will delete the snapshot. If you invoke an inactive function version, Lambda will initialize a new snapshot.
Network connections may not persist when a function is invoked from a snapshot, so, your code should assume connections need to be restored.
Ephemeral data such as temporary credentials created during initialization may expire between the time the snapshot is created and later resumed.
Since snapshots are reused many times to resume execution environments, there are additional requirements to ensure random numbers generated by your function remain unique.

Lambda functions that need to persist network connections, rely on ephemeral data created during initialization, or have strict requirements for randomness will require extra attention. The SnapStart documentation and AWS blog are helpful for understanding the code changes required to support these scenarios. For example, a common mistake is using a Pseudorandom Number Generator (PRNG) such as java.lang.Math.random() when a Cryptographically Secure Pseudorandom Number Generator (CPRNG) such as java.security.SecureRandom() will ensure your application code generates random values as expected with SnapStart.

Configuring Lambda Functions to work with SnapStart in Pulumi

To take advantage of faster cold starts, you must create your function resource with the option to use SnapStart. Note that beyond the previously described requirements to make your function code SnapSafe, no changes need to be made to the function code itself to use SnapStart. In the following example, we use the AWS Cloud Control provider to provision our function with SnapStart. The AWS Cloud Control provider uses the new Cloud Control API behind the scenes and has same-day support for new resources included in Cloud Control API - like SnapStart. This example also uses the AWS provider and the Pulumi Command Provider.

First, we add the necessary imports, create a role for our function to assume, and an S3 bucket to hold our Lambda code. Then we add our function’s zipped code as an object in the bucket. For this example, the Lambda function code is the AWS pet store example in Spring Boot:

import * as pulumi from "@pulumi/pulumi";
import * as awsNative from "@pulumi/aws-native";
import * as aws from "@pulumi/aws";
import * as command from "@pulumi/command";

const role = new aws.iam.Role("role", {
 assumeRolePolicy: JSON.stringify({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Principal": {
 "Service": "lambda.amazonaws.com",
 },
 "Action": "sts:AssumeRole",
 }],
 }),
});

new aws.iam.RolePolicyAttachment("role-policy-attachment", {
 role: role.name,
 policyArn: "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole",
});

const bucket = new aws.s3.Bucket("snapstart-bucket", {
 versioning: {
 enabled: true,
 }
});

const functionCode = new aws.s3.BucketObject("function-code", {
 bucket: bucket.bucket,
 source: new pulumi.asset.FileArchive("../petstore.zip"),
});

Next, we create our function with SnapStart enabled using the AWS Cloud Control provider, publish a version of our function (a requirement to use SnapStart) via the AWS CLI, and create a function alias for the version we created:

const func = new awsNative.lambda.Function("snapstart-func", {
 code: {
 s3Bucket: functionCode.bucket,
 s3Key: functionCode.key,
 s3ObjectVersion: functionCode.versionId,
 },
 role: role.arn,
 runtime: "java11",

 handler: "com.amazonaws.serverless.sample.springboot2.StreamLambdaHandler::handleRequest",
 memorySize: 1512,
 timeout: 60,
 snapStart: {
 applyOn: "PublishedVersions",
 },
});

const publishVersion = new command.local.Command("publish-version", {
 create: func.arn.apply(arn => `aws lambda publish-version --function-name ${arn}`),
 triggers: [func],
}, {
 dependsOn: func,
});

const aliasName = "v1";

const alias = new aws.lambda.Alias("alias", {
 functionName: func.arn,
 functionVersion: "1",
 name: aliasName,
}, {
 dependsOn: publishVersion,
});

Finally, we create the necessary resources to integrate our function with API Gateway and add our test URL as a stack output. This allows us to access it from outside of our Pulumi program:

const api = new aws.apigatewayv2.Api("snapstart-api", {
 protocolType: "HTTP",
});

const integration = new aws.apigatewayv2.Integration("lambdaIntegration", {
 apiId: api.id,
 integrationType: "AWS_PROXY",
 integrationUri: alias.arn,
 integrationMethod: "GET",
 payloadFormatVersion: "1.0",
 passthroughBehavior: "WHEN_NO_MATCH",
 connectionType: "INTERNET"
});

const route = new aws.apigatewayv2.Route("apiRoute", {
 apiId: api.id,
 routeKey: "$default",
 target: pulumi.interpolate`integrations/${integration.id}`,
});

new aws.apigatewayv2.Stage("apiStage", {
 apiId: api.id,
 name: "$default",
 routeSettings: [
 {
 routeKey: route.routeKey,
 throttlingBurstLimit: 5000,
 throttlingRateLimit: 10000,
 },
 ],
 autoDeploy: true,
}, { dependsOn: [route] });

new aws.lambda.Permission("permission", {
 action: "lambda:InvokeFunction",
 principal: "apigateway.amazonaws.com",
 function: alias.arn,
 sourceArn: api.executionArn.apply(x => `${x}/*/*`),
});

export const apiUrl = pulumi.concat(api.apiEndpoint, "/pets");

Deployment and Performance Testing

To deploy our function, we simply run the pulumi up command:

pulumi up -y

As the command completes, you’ll see output like the following:

Updating (dev)

View Live: https://app.pulumi.com/jkodroff/blog-lambda-snapshots-springboot-snapstart/dev/updates/9

 Type Name Status Info
 + pulumi:pulumi:Stack blog-lambda-snapshots-springboot-snapstart-dev created (4s) 1 warning
 + ├─ aws:iam:Role role created (0.82s)
 + ├─ aws:s3:Bucket snapstart-bucket created (2s)
 + ├─ aws:apigatewayv2:Api snapstart-api created (1s)
 + ├─ aws:iam:RolePolicyAttachment role-policy-attachment created (0.24s)
 + ├─ aws:s3:BucketObject function-code created (13s)
 + ├─ aws-native:lambda:Function snapstart-func created (22s)
 + ├─ command:local:Command publish-version created (3s)
 + ├─ aws:lambda:Alias alias created (0.66s)
 + ├─ aws:lambda:Permission permission created (0.40s)
 + ├─ aws:apigatewayv2:Integration lambdaIntegration created (0.74s)
 + ├─ aws:apigatewayv2:Route apiRoute created (0.54s)
 + └─ aws:apigatewayv2:Stage apiStage created (1s)


Outputs:
 apiUrl: "https://56r9255ey4.execute-api.us-west-2.amazonaws.com/pets"

Resources:
 + 13 created

Duration: 52s

We invoke our function by issuing an HTTP GET request against our stack output:

$ curl $(pulumi stack output apiUrl)
[{"id":"24ecd2bc-5106-46f8-95f2-af7de19cba57","breed":"Bloodhound","name":"Cooper","dateOfBirth":1639148877751},{"id":"caf6e8fa-fc97-4b9c-bfe7-996f14dc7995","breed":"Bloodhound","name":"Zoe","dateOfBirth":1438614477751},{"id":"b3d28309-8c92-4ae8-b7bd-7b656a5ea0bf","breed":"Beagle","name":"Murphy","dateOfBirth":1282489677751},{"id":"3c572757-298a-4343-9ca7-5c0865f9fce1","breed":"Beagle","name":"Winston","dateOfBirth":1410188877751},{"id":"4f8ab29a-760d-4b29-801d-6e35a98c538b","breed":"Dalmatian","name":"Max","dateOfBirth":1619968077751},{"id":"31fe978f-5eb6-4116-9933-ba8d9edc89a8","breed":"Dalmatian","name":"Sasha","dateOfBirth":1615475277751},{"id":"772218c3-cc56-44b7-87be-1db1b46730a3","breed":"Afghan Hound","name":"Lucky","dateOfBirth":1579187277751},{"id":"e994816e-8a6a-448a-bc48-ed404b463e46","breed":"Dalmatian","name":"Roxy","dateOfBirth":1408806477752},{"id":"3987cae6-6db2-48f9-96cb-dcb9812b1ab1","breed":"Bernese Mountain Dog","name":"Bailey","dateOfBirth":1329059277752},{"id":"514c8db6-225e-4fab-8c5b-63acdc88da56","breed":"Bloodhound","name":"Maggie","dateOfBirth":1218812877752}]%

To benchmark performance, use the Apache ab utility which is installed by default on macOS. The following command will run 1000 GET requests against our function:

ab -n 1000 -c 50 $(pulumi stack output apiUrl)

Comparing performance before and after SnapStart

During extensive benchmarking, the Pulumi team tried both a simple “Hello World” function as well as the more complex Spring Boot example above. Invoking this example Lambda function with and without SnapStart over many iterations, the Pulumi team found cold start times for a simple “Hello World” were 40% faster with SnapStart. The team found a much larger performance improvement when deploying the SpringBoot example. Cold starts were more than 10x faster. In real-world usage, this will translate into more responsive applications and services.

Conclusion

Addressing the challenges of cold starts for AWS Lambda functions is a major performance improvement that will enable Lambda functions to be useful in more scenarios that were previously the domain of dedicated VMs and container services. A 10x improvement in latency with common frameworks is a generational leap that we expect many teams to find valuable as they seek to modernize their infrastructure while keeping services responsive for end users.

Check out our GitHub repo with examples to try out Java-based functions with and without SnapStart enabled and see for yourself how these improvements can improve performance.

Happy coding!

Secure AWS Lambda with IAM ABAC Policies

Paul Stack — Tue, 19 Jul 2022 00:00:00 +0000

Event-driven, serverless functions have become a defining feature of many modern cloud architectures. With recent capabilities such as AWS Lambda URLs and AWS Lambda Containers, AWS has made it clear that Lambda Functions are a platform that teams can use to deliver increasingly sophisticated services without worrying about managing underlying compute resources.

Today, AWS announced another advancement for their Lambda Functions platform: Attribute-Based Access Control (ABAC). At its core, ABAC support brings more granular permissions that are automatically applied based on IAM role tags, Lambda tags, or both. This update builds on well-established Role-Based Access Control (RBAC) principles while making it possible to implement granular controls without permissions updates for every new user and resource.

What are Attributes?

Attributes are a key or key/value pair - in AWS, these attributes are called tags. Tags can be applied to multiple roles – for example identifying members of a team.

Across many teams, your organization may share a lot of common roles, and, in order to enforce the principle of least access, you may wish to restrict access across teams such that Developer Team members cannot access Ops Team resources and vice versa.

Using ABAC for AWS Lambda Functions

For this example, we’ll define a new role ‘abac-test’ as well as a team attribute that must have the value of either ‘developers’ or ‘ops’ to enable creating a Lambda function.

To get started, let’s first install the Pulumi AWS provider:

$ npm install @pulumi/aws

$ dotnet add package Pulumi.Aws

$ pip3 install pulumi_aws

$ go get github.com/pulumi/pulumi-aws/sdk/v5

Now, create a new IAM Role that a user can assume:

const role = new aws.iam.Role("abac-test", {
 assumeRolePolicy: JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Sid: "",
 Effect: "Allow",
 Principal: {
 AWS: "arn:aws:iam::MYACCOUNTID:root",
 },
 Action: "sts:AssumeRole",
 },
 ],
 }),
});

var role = new Aws.Iam.Role("abac-test", new Aws.Iam.RoleArgs
{
 AssumeRolePolicy = JsonSerializer.Serialize(new Dictionary<string, object?>
 {
 { "Version", "2012-10-17" },
 { "Statement", new[]
 {
 new Dictionary<string, object?>
 {
 { "Action", "sts:AssumeRole" },
 { "Effect", "Allow" },
 { "Principal", new Dictionary<string, object?>
 {
 { "AWS", "arn:aws:iam::MYACCOUNTID:root" },
 } },
 },
 }
 },
 }),
});

role = aws.iam.Role("abac-test",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "AWS": "arn:aws:iam::MYACCOUNTID:root",
 },
 }],
 }),
 tags={
 "Team": "developers",
 })

role, err := iam.NewRole(ctx, "abac-test", &iam.RoleArgs{
 AssumeRolePolicy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Sid": "",
 "Effect": "Allow",
 "Principal": {
 "AWS": "arn:aws:iam::MYACCOUNTID:root"
 },
 "Action": "sts:AssumeRole"
 }]
 }`),
})
if err != nil {
 return err
}

 abacTest:
 type: aws:iam:Role
 properties:
 assumeRolePolicy:
 Fn::ToJSON:
 Version: 2012-10-17
 Statement:
 - Action: sts:AssumeRole
 Effect: Allow
 Principal:
 AWS: arn:aws:iam::MYACCOUNTID:root

Let’s now create a new IAM Policy that allows a user to create Lambda functions:

const createPolicy = new aws.iam.Policy(
 "createLambda",
 {
 policy: JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Effect: "Allow",
 Action: ["lambda:CreateFunction", "lambda:TagResource"],
 Resource: "arn:aws:lambda:*:*:function:*",
 Condition: {
 // the requesting resource must have a tag with
 // team = developers/ops
 StringEquals: {
 "aws:RequestTag/Team": ["developers", "ops"], // must match
 },
 // your request must contain these tags
 // eg. project, but you might not have this defined yet
 "ForAllValues:StringEquals": {
 "aws:TagKeys": "Team", // must exist
 },
 },
 },
 {
 // Pulumi needs to check what functions exist and what versions exist to create updates and new versions
 Effect: "Allow",
 Action: [
 "lambda:GetFunction",
 "lambda:ListVersionsByFunction",
 "lambda:GetFunctionCodeSigningConfig",
 ],
 Resource: "*",
 },
 {
 Effect: "Allow",
 // These are assume role policies
 Action: ["iam:ListRoles", "iam:PassRole"],
 Resource: "*",
 },
 ],
 }),
 },
);

var createPolicy = new Aws.Iam.Policy("createPolicy", new Aws.Iam.PolicyArgs
{
 PolicyDocument = JsonSerializer.Serialize(new Dictionary<string, object?>
 {
 { "Version", "2012-10-17" },
 { "Statement", new[]
 {
 new Dictionary<string, object?>
 {
 { "Effect", "Allow" },
 { "Action", new[]
 {
 "lambda:CreateFunction",
 "lambda:TagResource",
 }
 },
 { "Resource", "arn:aws:lambda:*:*:function:*" },
 { "Condition", new Dictionary<string, object?>
 {
 // the requesting resource must have a tag with
 // team = developers/ops
 { "StringEquals", new Dictionary<string, object?>
 {
 { "aws:RequestTag/Team", new[]
 {
 "developers",
 "ops",
 }
 },
 } },
 // your request must contain these tags
 // eg. project, but you might not have this defined yet
 { "ForAllValues:StringEquals", new Dictionary<string, object?>
 {
 { "aws:TagKeys", new[]
 {
 "Team",
 }
 },
 } },
 } },
 },
 new Dictionary<string, object?>
 {
 // Pulumi needs to check what functions exist and what versions exist to create updates and new versions
 { "Effect", "Allow" },
 { "Action", new[]
 {
 "lambda:GetFunction",
 "lambda:ListVersionsByFunction",
 "lambda:GetFunctionCodeSigningConfig",
 }
 },
 { "Resource", "*" },
 },
 new Dictionary<string, object?>
 {
 // These are assume role policies
 { "Effect", "Allow" },
 { "Action", new[]
 {
 "iam:ListRoles",
 "iam:PassRole",
 }
 },
 { "Resource", "*" },
 },
 }
 },
 }),
});

create_policy = aws.iam.Policy("createPolicy", policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [
 {
 "Effect": "Allow",
 "Action": [
 "lambda:CreateFunction",
 "lambda:TagResource",
 ],
 "Resource": "arn:aws:lambda:*:*:function:*",
 "Condition": {
 #the requesting resource must have a tag with
 #team = developers/ops
 "StringEquals": {
 "aws:RequestTag/Team": [
 "developers",
 "ops",
 ],
 },
 # your request must contain these tags
 # eg. project, but you might not have this defined yet
 "ForAllValues:StringEquals": {
 "aws:TagKeys": ["Team"],
 },
 },
 },
 {
 # Pulumi needs to check what functions exist and what versions exist to create updates and new versions
 "Effect": "Allow",
 "Action": [
 "lambda:GetFunction",
 "lambda:ListVersionsByFunction",
 "lambda:GetFunctionCodeSigningConfig",
 ],
 "Resource": "*",
 },
 {
 # These are assume role policies
 "Effect": "Allow",
 "Action": [
 "iam:ListRoles",
 "iam:PassRole",
 ],
 "Resource": "*",
 },
 ],
}))

createPolicy, err := iam.NewRolePolicy(ctx, "createPolicy", &iam.RoleArgs{
 Policy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Sid": "",
 "Effect": "Allow",
 "Condition": map[string]interface{}{
 "StringEquals": map[string]interface{}{
 "aws:RequestTag/Team": []string{
 "developers",
 "ops",
 },
 },
 "ForAllValues:StringEquals": map[string]interface{}{
 "aws:TagKeys": []string{
 "Team",
 },
 },
 },
 "Resource": "arn:aws:lambda:*:*:function:*",
 "Action": []string{
 "lambda:CreateFunction",
 "lambda:TagResource",
 },
 },
 {
 "Effect": "Allow",
 "Resource": "*",
 "Action": []string{
 "lambda:GetFunction",
 "lambda:ListVersionsByFunction",
 "lambda:GetFunctionCodeSigningConfig",
 },
 },
 {
 {
 "Effect": "Allow",
 "Resource": "*",
 "Action": []string{
 "lambda:CreateFunction",
 "lambda:TagResource",
 },
 }
 }]
 }`),
})
if err != nil {
 return err
}

 createPolicy:
 type: aws:iam:Policy
 properties:
 policy:
 Fn::ToJSON:
 Version: 2012-10-17
 Statement:
 - Effect: Allow
 Action:
 - lambda:CreateFunction
 - lambda:TagResource
 Resource: 'arn:aws:lambda:*:*:function:*'
 Condition:
 StringEquals:
 aws:RequestTag/Team:
 - developers
 - ops
 ForAllValues:StringEquals:
 aws:TagKeys:
 - Team
 - Effect: Allow
 Action:
 - lambda:GetFunction
 - lambda:ListVersionsByFunction
 - lambda:GetFunctionCodeSigningConfig
 Resource: '*'
 - Effect: Allow
 Action:
 - iam:ListRoles
 - iam:PassRole
 Resource: '*'

Notice, as part of the policy condition, we ensure that the requesting resource must have a tag with team: developers or team: ops.

Let’s ensure that this policy is attached to the Role we initially created:

const rpa = new aws.iam.RolePolicyAttachment("rpa", {
 policyArn: createPolicy.arn,
 role: role,
})

var rpa = new Aws.Iam.RolePolicyAttachment("rpa", new Aws.Iam.RolePolicyAttachmentArgs
{
 PolicyArn = createPolicy.Arn,
 Role = role.Name,
});

rpa = aws.iam.RolePolicyAttachment("rpa",
 policy_arn=create_policy.arn,
 role=role.name)

rpa, err := iam.NewRolePolicyAttachment(ctx, "rpa", &iam.RolePolicyAttachmentArgs{
 PolicyArn: createPolicy.Arn,
 Role: role.Name,
})
if err != nil {
 return err
}

 rpa:
 type: aws:iam:RolePolicyAttachment
 properties:
 policyArn: ${createPolicy.arn}
 role: ${abacTest.name}

We can deploy these resources using the command pulumi update:

pulumi up
Previewing update (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/previews/c6fb4580-2706-47ee-8a56-b8942631254e

 Type Name Plan
 + pulumi:pulumi:Stack aws-abac-test-abac-testing create
 + ├─ aws:iam:Policy createLambda create
 + ├─ aws:iam:Role abac-test create
 + └─ aws:iam:RolePolicyAttachment createPolicy create

Resources:
 + 4 to create

Do you want to perform this update? yes
Updating (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/updates/1

 Type Name Status
 + pulumi:pulumi:Stack aws-abac-test-abac-testing created
 + ├─ aws:iam:Role abac-test created
 + ├─ aws:iam:Policy createLambda created
 + └─ aws:iam:RolePolicyAttachment createPolicy created

Resources:
 + 4 created

Duration: 7s

For the purposes of testing, we are now going to create a specific AWS Provider resource that we can pass the IAM Role details to assume.

const abacUserProvider = new aws.Provider(
 "assumeRole",
 {
 // assume the previously created role with the ABAC configured policy into our provider
 assumeRole: {
 roleArn: role.arn,
 sessionName: "abacTesting",
 },
 region: "us-east-1",
 },
 {
 // we want to depend on the role policy attachment being in place before we create the provider
 dependsOn: [rpa],
 }
);

var abacUserProvider = new Aws.Provider("abacUserProvider", new Aws.ProviderArgs
{
 AssumeRole = new Aws.Config.Inputs.AssumeRoleArgs
 {
 RoleArn = role.Arn,
 SessionName = "abacTesting",
 },
 Region = "us-east-1",
}, new CustomResourceOptions
{
 DependsOn =
 {
 rpa,
 },
});

abac_user_provider = pulumi.providers.Aws("abacUserProvider",
 assume_role=aws.config.AssumeRoleArgs(
 role_arn=role.arn,
 session_name="abacTesting",
 ),
 region="us-east-1",
 opts=pulumi.ResourceOptions(depends_on=[rpa]))

abacUserProvider, err := providers.Newaws(ctx, "abacUserProvider", &providers.awsArgs{
 AssumeRole: config.AssumeRole{
 RoleArn: role.Arn,
 SessionName: "abacTesting",
 },
 Region: "us-east-1",
}, pulumi.DependsOn([]pulumi.Resource{
 rpa,
}))
if err != nil {
 return err
}

 abacUserProvider:
 type: pulumi:providers:aws
 properties:
 assumeRole:
 roleArn: ${role.arn}
 sessionName: "abacTesting"
 region: "us-east-1"
 options:
 dependsOn:
 - ${rpa}

Now we can create an AWS Lambda with Pulumi as follows:

const lambdaRole = new aws.iam.Role("iamRole", {
 assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal({
 Service: "lambda.amazonaws.com",
 }),
});

const deleteLambdaPolicy = new aws.iam.Policy(
 "deleteLambdaPolicy",
 {
 policy: JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Effect: "Allow",
 Action: ["lambda:DeleteFunction"],
 Resource: "*",
 },
 ],
 }),
 },
 { parent: lambdaRole }
);

new aws.iam.RolePolicyAttachment(
 `deletePolicy`,
 {
 policyArn: deletePolicy.arn,
 role: role,
 },
 { parent: deletePolicy })

const lambda = new aws.lambda.Function(
 "lambdaFunction",
 {
 code: new pulumi.asset.AssetArchive({
 ".": new pulumi.asset.FileArchive("./app"),
 }),
 runtime: "nodejs12.x",
 role: lambdaRole.arn,
 handler: "index.handler",
 },
 { provider: abacUserProvider }
);

var lambdaRole = new Aws.Iam.Role("lambdaRole", new Aws.Iam.RoleArgs
{
 AssumeRolePolicy = JsonSerializer.Serialize(new Dictionary<string, object?>
 {
 { "Version", "2012-10-17" },
 { "Statement", new[]
 {
 new Dictionary<string, object?>
 {
 { "Action", "sts:AssumeRole" },
 { "Effect", "Allow" },
 { "Principal", new Dictionary<string, object?>
 {
 { "Service", "lambda.amazonaws.com" },
 } },
 },
 }
 },
 }),
});
var deletePolicy = new Aws.Iam.Policy("deletePolicy", new Aws.Iam.PolicyArgs
{
 PolicyDocument = JsonSerializer.Serialize(new Dictionary<string, object?>
 {
 { "Version", "2012-10-17" },
 { "Statement", new[]
 {
 new Dictionary<string, object?>
 {
 { "Action", "lambda:DeleteFunction" },
 { "Effect", "Allow" },
 { "Resource", "*" },
 },
 }
 },
 }),
});
var deletePolicyAttachment = new Aws.Iam.RolePolicyAttachment("deletePolicyAttachment", new Aws.Iam.RolePolicyAttachmentArgs
{
 Role = role.Name,
 PolicyArn = deletePolicy.Arn,
});
var lambdaFunction = new Aws.Lambda.Function("lambdaFunction", new Aws.Lambda.FunctionArgs
{
 Code = new FileArchive("./app"),
 Role = lambdaRole.Arn,
 Handler = "index.handler",
 Runtime = "nodejs14.x",
}, new CustomResourceOptions
{
 Provider = abacUserProvider,
});

lambda_role = aws.iam.Role("lambdaRole", assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": "lambda.amazonaws.com",
 },
 }],
}))
delete_policy = aws.iam.Policy("deletePolicy", policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": "lambda:DeleteFunction",
 "Effect": "Allow",
 "Resource": "*",
 }],
}))
delete_policy_attachment = aws.iam.RolePolicyAttachment("deletePolicyAttachment",
 role=role.name,
 policy_arn=delete_policy.arn)
lambda_function = aws.lambda_.Function("lambdaFunction",
 code=pulumi.FileArchive("./app"),
 role=lambda_role.arn,
 handler="index.handler",
 runtime="nodejs14.x",
 opts=pulumi.ResourceOptions(provider=abac_user_provider))

lambdaRole, err := iam.NewRole(ctx, "lambdaRole", &iam.RoleArgs{
 AssumeRolePolicy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Principal": {
 "Service": "lambda.amazonaws.com"
 },
 "Action": "sts:AssumeRole"
 }]
 }`),
})
if err != nil {
 return err
}

deletePolicy, err := iam.NewRolePolicy(ctx, "deletePoicy", &iam.RoleArgs{
 Policy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Resource": "*",
 "Action": []string{
 "lambda:DeleteFunction",
 },
 }]
 }`),
})
if err != nil {
 return err
}

_, err = iam.NewRolePolicyAttachment(ctx, "deletePolicyAttachment", &iam.RolePolicyAttachmentArgs{
 Role: role.Name,
 PolicyArn: deletePolicy.Arn,
})
if err != nil {
 return err
}

_, err = lambda.NewFunction(ctx, "lambdaFunction", &lambda.FunctionArgs{
 Code: pulumi.NewFileArchive("./app"),
 Role: lambdaRole.Arn,
 Handler: pulumi.String("index.handler"),
 Runtime: pulumi.String("nodejs14.x"),
}, pulumi.Provider(abacUserProvider))
if err != nil {
 return err
}

 lambdaRole:
 type: aws:iam:Role
 properties:
 assumeRolePolicy:
 Fn::ToJSON:
 Version: 2012-10-17
 Statement:
 - Action: sts:AssumeRole
 Effect: Allow
 Principal:
 Service: lambda.amazonaws.com
 deletePolicy:
 type: aws:iam:Policy
 properties:
 policy:
 Fn::ToJSON:
 Version: 2012-10-17
 Statement:
 - Action: lambda:DeleteFunction
 Effect: Allow
 Resource: '*'
 deletePolicyAttachment:
 type: aws:iam:RolePolicyAttachment
 properties:
 role: ${abacRole.name}
 policyArn: ${deletePolicy.arn}
 lambdaFunction:
 type: aws:lambda:Function
 properties:
 code:
 Fn::FileArchive: ./app
 role: ${lambdaRole.arn}
 handler: index.handler
 runtime: nodejs14.x

If we try and deploy the lambda, we will get the following using the role we are going to assume then we will get an error as the lambda function has no Team tag.

pulumi up
Previewing update (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/previews/7fdc10b2-5e44-497e-a067-f2e8b257496c

 Type Name Plan
 pulumi:pulumi:Stack aws-abac-test-abac-testing
 + ├─ aws:iam:Role iamRole create
 ├─ aws:iam:Role abac-test
 + │ └─ aws:iam:Policy deletePolicy create
 + │ └─ aws:iam:RolePolicyAttachment deletePolicy create
 + ├─ pulumi:providers:aws assumeRole create
 + └─ aws:lambda:Function lambdaFunction create

Resources:
 + 5 to create
 4 unchanged

Do you want to perform this update? yes
Updating (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/updates/2

 Type Name Status Info
 pulumi:pulumi:Stack aws-abac-test-abac-testing **failed** 1 error
 + ├─ aws:iam:Role iamRole created
 ├─ aws:iam:Role abac-test
 + │ └─ aws:iam:Policy deletePolicy created
 + │ └─ aws:iam:RolePolicyAttachment deletePolicy created
 + ├─ pulumi:providers:aws assumeRole created
 + └─ aws:lambda:Function lambdaFunction **creating failed** 1 error

Diagnostics:
 pulumi:pulumi:Stack (aws-abac-test-abac-testing):
 error: update failed

 aws:lambda:Function (lambdaFunction):
 error: 1 error occurred:
 * error creating Lambda Function (1): AccessDeniedException:
 status code: 403, request id: 4a8676f2-2b68-4d42-ad4b-f0e10ca31afd

Resources:
 + 4 created
 4 unchanged

Duration: 6s

If we add the correct Team tag to the Lambda function then the deploy will proceed as expected:

pulumi up
Previewing update (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/previews/0afa267f-5f9f-43f5-9e00-d3ee19dbf2b8

 Type Name Plan
 pulumi:pulumi:Stack aws-abac-test-abac-testing
 + └─ aws:lambda:Function lambdaFunction create

Resources:
 + 1 to create
 8 unchanged

Do you want to perform this update? yes
Updating (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/updates/3

 Type Name Status
 pulumi:pulumi:Stack aws-abac-test-abac-testing
 + └─ aws:lambda:Function lambdaFunction created

Resources:
 + 1 created
 8 unchanged

Duration: 9s

Try It Out!

Lambda ABAC capabilities will help you to simplify governance by helping you to standardize roles and maintain security boundaries between teams. A common use case is using tags to restrict/enable the invoke action for a function in addition to the create action demonstrated above. Give Lambda ABAC functionality a try and let us know what you think in Community Slack.

Test-Driven Infrastructure Development with Pulumi and Jest

Christian Nunciato — Mon, 13 Jun 2022 00:00:00 +0000

When I was a kid growing up in Southern California, there was a phone number you could call to find out what time it was. It was a local number, 853-1212 (easy to remember as the arrangement of the numbers on the keypad made a capital T), and I used it all the time, to set my watch, adjust the alarm clock, fix the display on the VCR. I don’t recall the last time I used it, probably sometime in the mid ’90s, but I do remember clearly the sound of the voice at the other end of the line.

Services like these were called speaking clocks (although I never called them that; to me, it was just “calling Time”), and they date back to the early 1930s or so. In the U.S., most were powered by a machine called an Audichron that used mechanical drums to play back the time of day, often with a little advertisement or the current temperature to go along with it.

By the mid-2000s, though, most of these locally run services had been shut down — by then, we had clocks baked into our phones — and today, only a handful of Audichrons survive. Thanks to the National Institute of Standards and Technology, however, you can still call a phone number to get the time, and while the voice might not be the same, and long distance rates will apply, it’s definitely there, and you can use it. So on the off chance you happen to find yourself with no idea what time it is and only an analog phone line in reach, fear not — old-school telephone tech has your back. For now. Assuming you remember the number.

Recalling all this stuff did make me wonder, though, what a more modern version of a speaking clock might look like. So in this post, we’re going to build one ourselves. We won’t use an actual phone number, but we will use Pulumi and AWS — and because we want to do it right, we’ll take a test-driven approach to developing the infrastructure with Jest, the JavaScript testing framework. We’ll use TypeScript and Node.js for everything, focus on unit tests, and when we’re done, we’ll have a single, serverless, browser-friendly HTTPS endpoint that returns an MP3 audio stream that speaks the current time.

Let’s get started.

Sketching it out

The first thing we’ll need is a runtime environment — someplace to run some server-side JavaScript that can render and deliver an audio file. Until recently, the easiest way to get an HTTP endpoint up and running on AWS has generally been with AWS Lambda and API Gateway, using Lambda to run the requisite code and API Gateway to expose the Lambda to the internet. Pulumi Crosswalk actually makes this really easy, too — but with the release of AWS Lambda Function URLs this April, we now have another option, one that doesn’t need API Gateway at all.

A Lambda Function URL is just what it sounds like: a URL that exposes a Lambda function. Specifically, it’s an AWS cloud resource that consists of a few properties that tell AWS whether to allow anonymous access to the function or to protect it with AWS IAM, and optionally, you can also provide a few cross-origin resource-sharing (CORS) rules to provide (or restrict) access by websites running on other domains.

For an app like this one, a Function URL is a good fit, as it’s simple, quick to deploy (and update), and it gives us all the control we need to get the job done. Architecturally, then, our little app will look something like this:

That about covers the infrastructure — but what about the sound?

As it happens, Google Translate offers a publicly accessible text-to-speech API that can, as long as you don’t overuse it, render a string to MP3 audio. We can use that API to generate the time-of-day message, tack the sound of a beep onto the end of it (for maximum nostalgic effect), and voilà: instant, infinitely scalable, cloud-native speaking clock.

Plan in place, we can kick things off by creating a new Pulumi project.

Create a new TypeScript project

Start by creating a new AWS TypeScript project in the usual way:

$ mkdir audichron-2022 && cd audichron-2022
$ pulumi new aws-typescript

Step through the prompts to create a new stack (you’ll only need one stack for this project), and when the new-project wizard completes, clear out the contents of index.ts entirely, as we’ll be building this program entirely from the ground up.

Install and configure Jest

Since we’re working with TypeScript, we can use ts-jest, which conveniently brings Jest along for the ride:

$ npm install ts-jest @types/jest --save-dev

Jest requires a bit of configuration, so add a new file at the root of the project, called jest.config.ts, with the following code:

import type { Config } from "@jest/types";

const config: Config.InitialOptions = {
 preset: "ts-jest",
};

export default config;

With Jest now installed and configured, and its helpers (like describe and expect) declared globally, you can create a new spec file to hold your tests. Add another file at the root of the project called index.spec.ts, and give it the following temporary content:

import "jest";

describe("My speaking clock", () => {
 it ("works", () => {
 expect(true).toBeTruthy;
 });
});

Lastly, in package.json, add a scripts block with a helper to make running the tests a little easier:

{
 "name": "audichron-2022",
 "devDependencies": {
 "@types/node": "^14",
 "ts-jest": "^28.0.3"
 },
 ...
 "scripts": {
 "test": "jest"
 }
}

Now run the tests, just to make sure you’ve got everything wired up correctly. If all goes well, you should see that the temporary spec we just added happily passes:

$ npm test

PASS ./index.spec.ts
 My speaking clock
 ✓ works

With that, it’s time to get to start writing some real tests.

Start with some failing tests

As devoted practitioners of test-driven development, we’re going to start by writing some unit tests — specifically, some failing unit tests that we can fix by writing the Pulumi code to make them pass.

Recall that our design requires just two cloud resources: a Lambda function and a Lambda function URL. For the function itself, we’ll use the high-level aws.lambda.CallbackFunction resource, one of my favorites for managing for Lambdas because it requires only one property: an inline JavaScript function to handle the event that triggers the Lambda.

For the URL resource — the eventual triggerer of that event — you’ll use an aws.lambda.FunctionURL configured to make the Lambda publicly accessible (i.e., available to anyone on the internet) and embeddable on any domain, making it easy, for example, to embed the URL as the src attribute of an HTML5 audio element.

So to get things going, we’ll need to:

Configure Pulumi to mock AWS resources. We’re writing unit tests, after all, and we want them to be fast, so we’ll need to prevent those tests from provisioning any real cloud infrastructure.
Import your Pulumi resource declarations — the function and the function URL — into index.spec.ts so you can reference them in tests.

In that light, replace the contents of index.spec.ts with the following code, which, after some setup to switch Pulumi into unit-testing mode, contains just one test: a check to make sure that the function URL is configured with the right authorizationType. I’ve included some comments to help explain what each line is doing:

import * as pulumi from "@pulumi/pulumi";
import "jest";

describe("My speaking clock", () => {

 // Define the infra variable as a type whose shape matches that of the
 // to-be-defined resources module.
 // https://www.typescriptlang.org/docs/handbook/2/typeof-types.html
 let infra: typeof import("./resources");

 beforeAll(() => {

 // Put Pulumi in unit-test mode, mocking all calls to cloud-provider APIs.
 pulumi.runtime.setMocks({

 // Mock requests to provision cloud resources and return a canned response.
 newResource: (args: pulumi.runtime.MockResourceArgs): {id: string, state: any} => {

 // Here, we're returning a same-shaped object for all resource types.
 // We could, however, use the arguments passed into this function to
 // customize the mocked-out properties of a particular resource based
 // on its type. See the unit-testing docs for details:
 // https://www.pulumi.com/docs/iac/concepts/testing/unit
 return {
 id: `${args.name}-id`,
 state: args.inputs,
 };
 },

 // Mock function calls and return whatever input properties were provided.
 call: (args: pulumi.runtime.MockCallArgs) => {
 return args.inputs;
 },
 });
 });

 beforeEach(async () => {

 // Dynamically import the resources module.
 // https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import#dynamic_imports
 infra = await import("./resources");
 });

 describe("function URL", () => {

 it("is publicly accessible", (done) => {
 infra.timeURL.authorizationType.apply(authType => {
 try {
 expect(authType).toBe("NONE");
 done();
 } catch (err) {
 done(err);
 }
 });
 });
 });
});

We can’t quite run these tests just yet, though, as they rely on a module called resources that we haven’t yet defined. Let’s do that now by creating a new file, resources.ts, containing some stubs for the cloud resources we’ve elected to use:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

export const timeFunction = new aws.lambda.CallbackFunction("time-function", {
 callback: () => {
 // Leave the callback implementation empty for now.
 },
});

export const timeURL = new aws.lambda.FunctionUrl("time-url", {
 functionName: timeFunction.name,
 authorizationType: "", // Leave this empty, too. (We want our first test to fail.)
});

Now run the tests, and you should see that the public-accessibility test fails — which shouldn’t be surprising, as we haven’t yet supplied an acceptable value:

$ npm test

FAIL ./index.spec.ts
 My speaking clock
 function URL
 ✕ is publicly accessible

 ● My speaking clock › function URL › is publicly accessible

 expect(received).toBe(expected) // Object.is equality

 Expected: "NONE"
 Received: ""

Now let’s turn that sad-looking ❌ into a ✅.

Make the tests pass

Update the FunctionUrl resource to change its authorizationType input to NONE (meaning for anonymous access, no authorization required):

export const timeURL = new aws.lambda.FunctionUrl("time-url", {
 functionName: timeFunction.name,
- authorizationType: "",
+ authorizationType: "NONE",
});

Once you do that, you should be able to run the test suite again, and this time, see it pass:

$ npm test

PASS ./index.spec.ts
 My speaking clock
 function URL
 ✓ is publicly accessible

Excellent: you’re now well on your way to writing more tests with Jest — and we’ll do that in a moment. But first, let’s digress momentarily to explore a few other ways of writing Jest specs with Pulumi.

Tidy up the specs

If you’ve written JavaScript tests before, particularly with tools like Jest, Jasmine, and Mocha, there’s a good chance you’ve gotten used to writing those tests in a particular way. And if you’re like me, you might’ve raised a bit of an eyebrow when you saw how that first test was written. Here it is again, this time with comments that capture the questions I might’ve had myself if I were seeing this code for the first time:

it("is publicly accessible", (done) => {
 infra.timeURL.authorizationType.apply(auth => { // Why do I have to use .apply() here?
 try { // What happens if I don't try/catch?
 expect(auth).toBe("NONE"); // Is it possible to use expect() by itself?
 done(); // Do I have to call done()? What if I don't?
 } catch (err) { // How can I make these tests more readable?
 done(err); // Is all of this code necessary?
 }
 });
});

This is a perfectly valid test, of course; we know because we just ran it. Still, you might wonder, especially since we’re only checking one property, whether this test might be written a bit more concisely — something more like this, maybe:

it("is publicly accessible", () => {
 expect(infra.timeURL.authorizationType).toBe("NONE"); // Alas, nope.
});

The main reason is the asynchronous nature of the objects we’re testing. An aws.lambda.FunctionUrl, after all, isn’t just a plain ol’ JavaScript object — it’s a Pulumi CustomResource, a representation of an eventual resource running in the cloud, with properties that may not be known until sometime after the resource has been deployed. While it’s true we provided authorizationType as a regular JavaScript string, by the time we attempt to read it in the test, it’s been transformed into a pulumi.Output<string> — an asynchronous value, like a promise, that has to be unwrapped with a call to apply() before it can be read.

However, just adding apply and calling expect from within its callback doesn’t quite work, either:

it("is publicly accessible", () => {
 infra.timeURL.authorizationType.apply(authType => {
 expect(authType).toBe("NONE"); // Also nope.
 });
});

Although that might not be obvious at first, since on the surface, the test appears to pass:

$ npm test

PASS ./index.spec.ts
 My speaking clock
 function URL
 ✓ is publicly accessible

It isn’t until you force the test to fail that it becomes clearer something’s gone sideways. Try changing the expected authorizationType to something other than NONE and you’ll see what I mean:

$ npm test

PASS ./index.spec.ts
 My time app
 function URL
 ✓ is publicly accessible

Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total

[UnhandledPromiseRejection: This error originated either by throwing inside of an async function
without a catch block, or by rejecting a promise which was not handled with .catch(). The promise
rejected with the reason "Error: expect(received).toBe(expected) // Object.is equality

Expected: "AWS_IAM"
Received: "NONE"".] {
 code: 'ERR_UNHANDLED_REJECTION'
}

All is not lost, though. We do have other options, and that UnhandledPromiseRejection offers a bit of a clue.

Turn outputs into promises

Jest actually has built-in support for writing asynchronous tests with promises, and there are a couple of ways you can write specs that use them:

Have the function passed to it (or its alias, test) return a promise containing your expectation
Pass an async function to it and await the promise in the usual way

There’s only one problem: pulumi.Outputs may indeed be promise-like, but they aren’t actual promises.

They can, however, be wrapped in promises pretty easily. I like the way Pulumi community member @aviflax handled this with a little utility function that wraps the Output and preserves the type of its underlying value. Here’s that function, followed by a couple of examples that use it:

// Convert a pulumi.Output to a promise of the same type.
function promiseOf<T>(output: pulumi.Output<T>): Promise<T> {
 return new Promise(resolve => output.apply(resolve));
}

// Our test, rewritten to have it() return a Promise<string>.
it("is publicly accessible", () => {
 return promiseOf(infra.timeURL.authorizationType).then(authType => {
 expect(authType).toBe("NONE");
 });
});

// The same test, rewritten to use async/await.
it("is publicly accessible", async () => { // <-- Don't miss that async!
 const authType = await promiseOf(infra.timeURL.authorizationType);
 expect(authType).toBe("NONE");
});

Either approach works equally well, but I’m slightly more partial to the latter for its conciseness. So for now, update index.spec.ts to add the promiseOf helper and adjust the test accordingly, then run the suite one more time to make sure all’s well:

// ...

// Convert a pulumi.Output to a promise of the same type.
function promiseOf<T>(output: pulumi.Output<T>): Promise<T> {
 return new Promise(resolve => output.apply(resolve));
}

describe("My speaking clock", () => {
 // ...

 describe("function URL", () => {

 it("is publicly accessible", async () => {
 const authType = await promiseOf(infra.timeURL.authorizationType);
 expect(authType).toBe("NONE");
 });
 });
});

Assuming you’re back in ✅ territory, let’s move on so we can finish things off.

Finish the app

We’re in good shape: we’ve got scaffolding for our Jest-powered unit tests and we’re generally happy with how they look, so it’s time to round out the rest of the app and its tests so we can deploy. We’ll go through this last part briskly, as we’ve covered most of the important stuff at this point, and pause only briefly to point out what’s important as we go.

Here’s the plan:

We’ll add one more test to make sure the function URL is CORS-friendly
We’ll add another test to make sure the URL resource is bound to the right Lambda function
We’ll add the Lambda code to generate and return the audio file
We’ll deploy, and bask in the warm glow of nostalgia

Let’s get to it.

Add the remaining tests

To index.spec.ts, add two more tests, one to ensure the function is accessible to browsers running on any domain, the other to verify the function URL resource is exposing the right function (if we didn’t, it’d be easy to change that reference by mistake and accidentally expose the wrong function):

describe("My speaking clock", () => {
 // ...

 describe("function URL", () => {
 // ...

 it("is CORS-friendly", async () => {
 const authType = await promiseOf(infra.timeURL.cors);
 expect(authType).toEqual({
 allowOrigins: ["*"],
 allowMethods: ["GET"],
 });
 });

 it("is bound to the right Lambda function", async () => {
 const timeFuncName = await promiseOf(infra.timeFunction.name);
 const timeURLFunc = await promiseOf(infra.timeURL.functionName);
 expect(timeFuncName).toEqual(timeURLFunc);
 });
 });
});

Run the tests, and you should see one failure:

$ npm test

FAIL ./index.spec.ts (14.256 s)
 My speaking clock
 function URL
 ✓ is publicly accessible (3022 ms)
 ✕ is CORS-friendly (3 ms)
 ✓ is bound to the right Lambda function (1 ms)

 Expected: {"allowMethods": ["GET"], "allowOrigins": ["*"]}
 Received: undefined

Tests: 1 failed, 2 passed, 3 total

A quick addition to the FunctionUrl in resources.ts should get you to green:

// ...
export const timeURL = new aws.lambda.FunctionUrl("time-url", {
 functionName: timeFunction.name,
 authorizationType: "NONE",
+ cors: {
+ allowOrigins: ["*"],
+ allowMethods: ["GET"],
+ },
});

$ npm test

PASS ./index.spec.ts
 My speaking clock
 function URL
 ✓ is publicly accessible
 ✓ is CORS-friendly
 ✓ is bound to the right Lambda function

Tests: 3 passed, 3 total

That’s it for the tests! Now for the final additions.

Produce the audio message

The last thing we’ll need to do is render some audio. For this, we’ll need to communicate with Google Translate, our text-to-speech provider of choice, and that means we’ll need an HTTP client. I generally like node-fetch, so let’s go with that:

npm install --save node-fetch@2

Now, finish the program by importing node-fetch into resources.ts, adding a function to produce a speech-friendly string indicating the current time and updating the CallbackFunction’s callback property to send the string to Google Translate to produce the desired result. Here’s the complete file, with comments, for reference:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const fetch = require("node-fetch");

// Convert the current time into a speech-friendly string.
export function getSpeechText() {
 const now = new Date();
 now.setSeconds(now.getSeconds() + 6);

 const local = new Date(now.toLocaleString("en-US", { timeZone: "America/Los_Angeles" }));
 const localHour = local.getHours();
 const localMinute = local.getMinutes();
 const h = localHour > 12 ? localHour - 12 : localHour;
 const m = localMinute < 10 ? `oh ${localMinute}` : localMinute; // So 2:03 -> "two-oh-three"
 const s = local.getSeconds();

 return `At the tone, the time will be ${h} ${m}. And ${s} seconds.`;
}

export const timeFunction = new aws.lambda.CallbackFunction("time-function", {

 // Update the Lambda callback body to convert the current time into an MP3 file.
 callback: async () => {
 const text = getSpeechText();
 const speechURL = `https://translate.google.com/translate_tts?ie=UTF-8&q=${encodeURIComponent(text)}&textLen=${text.length}&tl=en&client=tw-ob`;
 const beepURL = "https://www.pulumi.com/uploads/beep.mp3";
 const ttsResponse = await fetch(speechURL);
 const beepResponse = await fetch(beepURL);
 const speech = await ttsResponse.arrayBuffer();
 const beep = await beepResponse.arrayBuffer();

 // Tack a beep onto the end of the audio returned from Google Translate, then
 // render the whole thing as a base-64 encoded string.
 const body = Buffer.concat([Buffer.from(speech), Buffer.from(beep)]).toString("base64");

 // Return an appropriately shaped HTTP response.
 return {
 body,
 headers: { "Content-Type": "audio/mpeg" },
 isBase64Encoded: true,
 statusCode: 200,
 };
 },
});

export const timeURL = new aws.lambda.FunctionUrl("time-url", {
 functionName: timeFunction.name,
 authorizationType: "NONE",
 cors: {
 allowOrigins: ["*"],
 allowMethods: ["GET"],
 },
});

// Export the public URL of our shiny new service.
export const audioURL = timeURL.functionUrl;

Finally, open index.ts (which should still be empty) and add a couple of lines to import the resources module and export the function URL as a Pulumi stack output:

import "./resources";

export { audioURL } from "./resources";

And that’s it. We’re ready to deploy.

Deploy the service

Doublecheck your AWS credentials are configured, then deploy the new service with a single pulumi up:

$ pulumi up

...
Updating (dev)

View Live: https://app.pulumi.com/cnunciato/audichron-2022/dev/updates/1

 Type Name Status
 + pulumi:pulumi:Stack audichron-2022-dev created
 + ├─ aws:iam:Role time-function created
 + ├─ aws:lambda:Function time-function created
 + ├─ aws:iam:RolePolicyAttachment time-function-7cd09230 created
 + ├─ aws:iam:RolePolicyAttachment time-function-a1de8170 created
 + ├─ aws:iam:RolePolicyAttachment time-function-1b4caae3 created
 + ├─ aws:iam:RolePolicyAttachment time-function-019020e7 created
 + ├─ aws:iam:RolePolicyAttachment time-function-4aaabb8e created
 + ├─ aws:iam:RolePolicyAttachment time-function-74d12784 created
 + ├─ aws:iam:RolePolicyAttachment time-function-e1a3786d created
 + ├─ aws:iam:RolePolicyAttachment time-function-6c156834 created
 + ├─ aws:iam:RolePolicyAttachment time-function-b5aeb6b6 created
 + └─ aws:lambda:FunctionUrl time-url created

Outputs:
 audioURL: "https://52rjmmybwfwcud3nxxhosznpum0cycbw.lambda-url.us-west-2.on.aws/"

Resources:
 + 13 created

Duration: 20s

When the update completes, open the URL in your browser of choice, and you should hear the message you’ve been waiting for — one that might even make Pat Fleet proud:

$ open $(pulumi stack output audioURL)

When you’re ready, you can tear everything down with a pulumi destroy.

Wrapping up

Beyond just having a nifty (and admittedly rather silly) new way to tell time, you should have a much better sense at this point of how you can use Pulumi with Jest to write better, safer infrastructure code.

From here, there’s a bunch more you might think about next: writing more tests to cover the code we just added, exploring some additional flavors of testing in the docs, or having a look at a few examples. You’ll find the full source for this walkthrough up on GitHub as well.

Happy testing!

API Gateway to EventBridge with Pulumi

Christian Nunciato — Wed, 12 Jan 2022 13:00:00 -0700

If you’re familiar with Amazon API Gateway, you know it’s all about making it easier to provision and manage a web API. Maybe you’ve used it, as I have, with Crosswalk, our AWS extension library, to stand up a REST API and handle requests with AWS Lambda functions:

import * as awsx from "@pulumi/awsx";

// Create a new API Gateway instance.
const api = new awsx.apigateway.API("my-api", {
 routes: [
 {
 // Define an HTTP endpoint.
 method: "GET",
 path: "/things",

 // Handle requests with an AWS Lambda function.
 eventHandler: async (apiGatewayEvent) => {
 return {
 statusCode: 200,
 body: JSON.stringify(["thingOne", "thingTwo"]),
 };
 },
 },
 ],
});

// Export the API's public URL. 🎉
export const apiUrl = api.url;

I love this abstraction, and I use it all the time; it’s an incredibly convenient way to solve a really common problem. But if you examine the code, you’ll notice that it’s making a fairly strong assumption about how you’ll be handling HTTP requests—namely, that you’ll do so with a single Lambda function, and that that function will always return a JavaScript object of a particular shape.

Indeed, this arrangement is the API contract of a Lambda proxy integration—API Gateway integrations come in many shapes and sizes; Lambda integrations just happen to be one of the more popular—and much of the time, an approach like this one works out just fine. But depending on the needs of the application, it might not always be the best fit.

Imagine you were building a print-on-demand service, for example, and you wanted to expose an API to let your customers upload documents and have them converted into orders. On AWS, you might reach for API Gateway, as above, to define an HTTP method and route (POST /uploads, say), wire it up to an AWS Lambda, and have the Lambda parse the upload, write the order to a database, and return a response. Visually, such a design might look something like this:

It’d definitely work, and again, it’s quite common. But at some point, you might find this tight coupling between API Gateway and Lambda too limiting. Say you wanted to be notified whenever a new order was received, with a Slack message, maybe, in one of your team’s shared workspace channels. Under the current design, you’d probably add a few lines of code to the Lambda function to import an HTTP library and make a call to the Slack API to post the message:

That’d work, too—but it’d be less than ideal for a number of reasons. For one, that Lambda would now have two jobs: taking orders and sending Slack notifications. That might be fine for today (it’s only a few lines of code, after all), but over time, those two jobs could easily become three, and then four, and soon enough, that poor Lambda could wind up becoming a lot more difficult to maintain. And given the importance of its main job—capturing orders—it’s not something you’d want to risk failing at runtime because of a random Slack outage or other transient internet mishap. Moreover, with every bit of extra work you tack on to that function, you take a tiny step closer to hitting API Gateway’s 30-second limit.

What you really need, then, is to be able to take multiple independent, possibly long-running actions based on a single API Gateway request. And one way to do that is with Amazon EventBridge.

Hello, EventBridge

Amazon EventBridge (formerly CloudWatch Events) is as a serverless event bus whose job is to receive structured event data—from your own applications, from other AWS Services—and use that data to notify other applications or services using event-handling rules that you specify. With EventBridge, you can build loosely coupled, event-driven systems that take advantage of AWS’s rich support for serverless architectures and that scale gracefully as the needs of those systems change over time.

For this particular application, EventBridge would let you address the multiple-handler problem in a more scalable and easily maintainable way. Rather than have API Gateway invoke Lambda directly, leaving one Lambda responsible for handling multiple tasks, you could have API Gateway publish to EventBridge instead, and let EventBridge invoke as many Lambdas (or other AWS services) as you like, all serverlessly and in parallel—and of course, all easily managed with Pulumi.

Let’s see how. Here’s a revised architecture diagram showing how you might approach building an application like this one with EventBridge positioned between API Gateway and Lambda:

Now let’s have a look at what it’d be like to build it with Pulumi. We won’t build everything in this diagram—things like writing to the database or messaging Slack are left for you to explore—but we will build enough to give you clear picture and a working example of how to connect all of these parts into a working application. Specifically:

an API Gateway instance to act as a container for your public API, along with a stage and a route to handle inbound HTTP requests;
an EventBridge integration (comprised of an event bus and event rule) to handle notifications from API Gateway; and finally,
one or more Lambda functions to be invoked in response to event-rule matches.

Let’s get started.

Create a new project and stack

As always, it’s good practice to begin with a new project and stack:

$ pulumi new aws-typescript

$ pulumi new aws-python

Make sure you configure your AWS credentials as well, and when prompted, choose whatever stack name and AWS region work best for you.

In this post, we’ll be using API Gateway V2 as it’s newer and has a few features (like auto-deploy) that make it easier to work with in this context. The same ideas apply to both V1 and V2, though, so while V2 still lacks full feature parity with V1, the underlying resources we’ll be using—methods, routes, integrations—are the same. You’ll find links to examples of both at the end of this post.

Create the API gateway and stage

Start by replacing the contents of index.js index.ts __main__.py main.go Program.cs Program.fs Program.vb App.java Pulumi.yaml main.tf with the following code to declare a new API Gateway API:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Create an HTTP API.
const api = new aws.apigatewayv2.Api("api", {
 protocolType: "HTTP",
});

import json
import pulumi
import pulumi_aws as aws

# Create an HTTP API.
api = aws.apigatewayv2.Api("example",
 protocol_type="HTTP"
)

Next, add a stage (feel free to name it whatever you like; I usually use the current stack name for convenience), and set it to deploy automatically whenever a change is made to the API:

// ...

// Create a stage and set it to deploy automatically.
const stage = new aws.apigatewayv2.Stage("stage", {
 apiId: api.id,
 name: pulumi.getStack(),
 autoDeploy: true,
});

# Create a stage and set it to deploy automatically.
stage = aws.apigatewayv2.Stage("stage",
 api_id=api.id,
 name=pulumi.get_stack(),
 auto_deploy=True
);

The next thing to do is register a route on the gateway to give your users a publicly accessible endpoint to upload to. But in order to do that, you’ll need to tell API Gateway what to do when an upload happens. Since the plan of record is to notify EventBridge (using API Gateway’s built-in support for it), you’ll need to declare a few EventBridge things first.

Add an event bus and event rule

Every AWS account gets an event bus by default (one that’s aptly named default), but given how easy it is to create one, we might as well do that for this application. You’ll also need to define an event rule—a resource that watches a specific event bus for events that conform to a particular pattern or shape, and then routes those events to one or more targets (e.g., Lambda functions). Add the following lines to your program for both:

// ...

// Create an event bus.
const bus = new aws.cloudwatch.EventBus("bus");

// Create an event rule to watch for events.
const rule = new aws.cloudwatch.EventRule("rule", {
 eventBusName: bus.name,

 // Specify the event pattern to watch for.
 eventPattern: JSON.stringify({
 source: ["my-event-source"],
 }),
});

# ...

# Create an event bus.
bus = aws.cloudwatch.EventBus("bus")

# Create an event rule to watch for events.
rule = aws.cloudwatch.EventRule("rule",
 event_bus_name=bus.name,
 event_pattern=json.dumps({"source": ["my-event-source"]})
)

The most noteworthy property of the EventRule resource is probably the eventPattern. EventBridge events all conform to a certain schema, and in this case, we’re expressing that this particular event rule should take action on any event that originates from my-event-source. (The source property is just a free-form string that by convention identifies the application or service responsible for the event.)

With the event bus and event rule in place, you’re ready to define the integration itself—the resource responsible for connecting the gateway route (which we’ll get back to shortly) to your newly created event bus. As I mentioned earlier, there are several types of API Gateway integration to choose from, each one suited to a particular purpose. For this example, the AWS_PROXY type is good fit, as it’s simple and requires very little code; it doesn’t give you quite as much control over the API Gateway response as you might like—as a proxy, it just returns to the caller whatever is returned by the backend, in this case EventBridge—but it’s more than enough for the task at hand.

Add the following lines for the integration and route. The comments should explain what each block is doing:

// ...

// Define a policy granting API Gateway permission to publish to EventBridge.
const apiGatewayRole = new aws.iam.Role("api-gateway-role", {
 assumeRolePolicy: {
 Version: "2012-10-17",
 Statement: [
 {
 Action: "sts:AssumeRole",
 Effect: "Allow",
 Principal: {
 Service: "apigateway.amazonaws.com",
 },
 },
 ],
 },
 managedPolicyArns: ["arn:aws:iam::aws:policy/AmazonEventBridgeFullAccess"],
});

// Create an API Gateway integration to forward requests to EventBridge.
const integration = new aws.apigatewayv2.Integration("integration", {
 apiId: api.id,

 // The integration type and subtype.
 integrationType: "AWS_PROXY",
 integrationSubtype: "EventBridge-PutEvents",
 credentialsArn: apiGatewayRole.arn,

 // The body of the request to be sent to EventBridge. Note the
 // event source matches pattern defined on the EventRule, and the
 // Detail expression, which just forwards the body of the original
 // API Gateway request (i.e., the uploaded document).
 requestParameters: {
 EventBusName: bus.name,
 Source: "my-event-source",
 DetailType: "my-detail-type",
 Detail: "$request.body",
 },
});

// Finally, define the route.
const route = new aws.apigatewayv2.Route("route", {
 apiId: api.id,
 routeKey: "POST /uploads",
 target: pulumi.interpolate`integrations/${integration.id}`,
});

# ...

# Define a policy granting API Gateway permission to publish to EventBridge.
api_gateway_role = aws.iam.Role("api-gateway-role",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": "apigateway.amazonaws.com",
 },
 },
 ],
 }),
 managed_policy_arns=[
 "arn:aws:iam::aws:policy/AmazonEventBridgeFullAccess",
 ],
);

# Create an API Gateway integration to forward requests to EventBridge.
integration = aws.apigatewayv2.Integration("integration",
 api_id=api.id,

 # The integration type and subtype.
 integration_type="AWS_PROXY",
 integration_subtype="EventBridge-PutEvents",
 credentials_arn=api_gateway_role.arn,

 # The body of the request to be sent to EventBridge. Note the
 # event source matches pattern defined on the EventRule, and the
 # Detail expression, which just forwards the body of the original
 # API Gateway request (i.e., the uploaded document).
 request_parameters={
 "EventBusName": bus.name.apply(lambda name: name),
 "Source": "my-event-source",
 "DetailType": "my-detail-type",
 "Detail": "$request.body",
 },
);

# Finally, define the route.
route = aws.apigatewayv2.Route("route",
 api_id=api.id,
 route_key="POST /uploads",
 target=integration.id.apply(lambda id: f"integrations/{id}"),
);

With that, you’re ready to configure the Lambda.

Add a Lambda function handler

The hard part’s done: you’ve declared an API and route, mapped that route to an integration, configured the integration to publish events to an event bus, and defined an event rule to respond to those events. All that’s left now is to tell the rule how to respond.

So to finish things off, you need:

a Lambda function to handle uploads,
an EventBridge target to bind your event rule to that function, and
an IAM policy granting EventBridge permission to invoke the function.

Add the following lines to your program to complete it:

First, make a new folder and file alongside your Pulumi program, at ./api/handlers.py, and add a Python function to act as your function handler:

// ...

// Create a Lambda function handler with permission to log to CloudWatch.
const lambda = new aws.lambda.CallbackFunction("lambda", {
 policies: [aws.iam.ManagedPolicies.CloudWatchLogsFullAccess],
 callback: async (event: any) => {
 // For now, just log the event, including the uploaded document.
 // That'll be enough to verify everything's working.
 console.log({ source: event.source, detail: event.detail });
 },
});

// Create an EventBridge target associating the event rule with the function.
const lambdaTarget = new aws.cloudwatch.EventTarget("lambda-target", {
 arn: lambda.arn,
 rule: rule.name,
 eventBusName: bus.name,
});

// Give EventBridge permission to invoke the function.
const lambdaPermission = new aws.lambda.Permission("lambda-permission", {
 action: "lambda:InvokeFunction",
 principal: "events.amazonaws.com",
 function: lambda.arn,
 sourceArn: rule.arn,
});

// Export the API Gateway URL to give us something to POST to.
export const url = pulumi.interpolate`${api.apiEndpoint}/${stage.name}`;

# In ./api/handlers.py:
import json

def capture_order(event, context):
 # For now, just log the event, including the uploaded document.
 # That'll be enough to verify everything's working.
 print(json.dumps({ "source": event["source"], "detail": event["detail"] }))

Now, back in index.js index.ts main.py main.go Program.cs Program.fs Program.vb App.java Pulumi.yaml main.tf , add the following lines to your program to complete it:

# ...

# Define a role and policy allowing Lambda functions to log to CloudWatch.
lambda_role = aws.iam.Role("lambda-role",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Principal": {
 "Service": "lambda.amazonaws.com"
 },
 "Effect": "Allow"
 }
 ]
 })
)
lambda_role_policy = aws.iam.RolePolicy("lambda-role-policy",
 role=lambda_role.id,
 policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Action": [
 "logs:CreateLogGroup",
 "logs:CreateLogStream",
 "logs:PutLogEvents"
 ],
 "Resource": "arn:aws:logs:*:*:*"
 }]
 })
)

# Create a Lambda function handler.
lambda_function = aws.lambda_.Function("lambda",
 role=lambda_role.arn,
 runtime="python3.7",
 handler="handlers.capture_order",
 code=pulumi.AssetArchive({
 ".": pulumi.FileArchive('./api')
 })
)

# Create an EventBridge target associating the event rule with the function.
lambda_target = aws.cloudwatch.EventTarget("lambda-target",
 arn=lambda_function.arn,
 rule=rule.name,
 event_bus_name=bus.name,
);

# Give EventBridge permission to invoke the function.
lambda_permission = aws.lambda_.Permission("lambda-permission",
 action="lambda:InvokeFunction",
 principal="events.amazonaws.com",
 function=lambda_function.arn,
 source_arn=rule.arn,
);

# Export the API Gateway URL to give us something to POST to.
pulumi.export("url", pulumi.Output.concat(api.api_endpoint, "/", stage.name))

All Together, Now

Now that the program’s complete, you can run Pulumi to bring it to life:

$ pulumi up
...

Updating (dev)
...

 Type Name Status
 + pulumi:pulumi:Stack eventbridge-v2-dev created
 + ├─ aws:apigatewayv2:Api api created
 + ├─ aws:apigatewayv2:Stage stage created
 + ├─ aws:cloudwatch:EventBus bus created
 ...

Outputs:
 apiURL: "https://geqfietbcl.execute-api.us-west-2.amazonaws.com/dev"

Resources:
 + 15 created

Duration: 31s

When the update finishes, you’ll have a fully functioning API Gateway-EventBridge integration that you can verify with curl:

$ curl --data '{"some-key": "some-value"}' --header "Content-Type: application/json" \
 "$(pulumi stack output url)/uploads"

{"Entries":[{"EventId":"cdc44763-6976-286c-9378-7cce674dff81"}],"FailedEntryCount":0}

Note the response, which comes directly from EventBridge (courtesy of the AWS_PROXY integration), confirming the event was received and written to the event bus.

The final step is to confirm that the request made it all the way to Lambda, which you can easily do by tailing its output with pulumi logs:

$ pulumi logs --follow

Collecting logs for stack dev since 2022-01-06T16:18:48.000-08:00.
...

{
 source: 'my-event-source',
 detail: { 'some-key': 'some-value' }
}

When you’re happy, be sure to tidy up with a pulumi destroy.

What’s next?

There’s a lot more you can do with integrations like this that we didn’t cover: add more Lambda function handlers, have EventBridge target other AWS services (Step Functions might be a good one to try next), validate HTTP request bodies (with API Gateway models, to keep bad data from ever reaching EventBridge), and more. Hopefully this gives you sense of what’s possible, though. And as promised, you’ll find examples that use both versions of API Gateway in our examples repository on GitHub:

API Gateway V2 to EventBridge in TypeScript
API Gateway V2 to EventBridge in Python
API Gateway V1 to EventBridge in TypeScript, with request validation and custom HTTP response mapping

AWS Lambda Functions Powered by AWS Graviton2 Processors

Paul Stack — Wed, 29 Sep 2021 00:00:00 +0000

In late 2018, AWS launched their first EC2 instances powered by ARM-based AWS Graviton Processors. These instances had been optimized for performance and cost. Since that initial launch, Amazon has continued to innovate in the Graviton space. In June 2021, they launched the Graviton Challenge for users to move their applications to AWS Graviton2. AWS Graviton2 processor instance types are up to 20% lower cost than x86 based instance types and see up to 40% better price performance.

Today, AWS launched Graviton2 support for AWS Lambda Functions, which means that your Lambda Functions can now execute on Graviton2 processors and take advantage of the improved price performance of that architecture.

Deploying a Lambda Function with Graviton2 Support

Let’s use Pulumi to deploy a new AWS Lambda Function using a Graviton2 architecture. First, let’s create the IAM role and policies required to create a Lambda Function:

import * as aws from "@pulumi/aws";

const role = new aws.iam.Role("lambda-role", {
 assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal({
 Service: "lambda.amazonaws.com"
 }),
});

const fullAccess = new aws.iam.RolePolicyAttachment("mylambda-access", {
 role: role,
 policyArn: aws.iam.ManagedPolicy.LambdaFullAccess,
});

import pulumi
import json
import pulumi_aws as aws

test_role = aws.iam.Role("lambda-role",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Sid": "",
 "Principal": {
 "Service": "lambda.amazonaws.com",
 },
 }],
 }))

full_access = aws.iam.RolePolicyAttachment("mylambda-access",
 role=role.name,
 policy_arn=aws.iam.ManagedPolicy.LAMBDA_FULL_ACCESS)

package main

import (
 "encoding/json"

 "github.com/pulumi/pulumi-aws/sdk/v4/go/aws/iam"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // create IAM role and policy wiring
 assumeRolePolicyJSON, err := json.Marshal(map[string]interface{}{
 "Version": "2012-10-17",
 "Statement": []interface{}{
 map[string]interface{}{
 "Action": "sts:AssumeRole",
 "Principal": map[string]interface{}{
 "Service": "lambda.amazonaws.com",
 },
 "Effect": "Allow",
 },
 },
 })
 if err != nil {
 return err
 }
 role, err := iam.NewRole(ctx, "lambda-role", &iam.RoleArgs{
 AssumeRolePolicy: pulumi.String(assumeRolePolicyJSON),
 })
 if err != nil {
 return err
 }

 _, err = iam.NewRolePolicyAttachment(ctx, "mylambda-access", &iam.RolePolicyAttachmentArgs{
 Role: role.Name,
 PolicyArn: pulumi.String("arn:aws:iam::aws:policy/AWSLambda_FullAccess"),
 })
 if err != nil {
 return err
 }

 return nil
 })
}

using Pulumi;
using Pulumi.Aws.Iam;
using Pulumi.Aws.Lambda;

class LambdaStack : Stack
{
 public LambdaStack()
 {
 var role = new Role("lambda-role", new RoleArgs
 {
 AssumeRolePolicy =
 @"{
 ""Version"": ""2012-10-17"",
 ""Statement"": [
 {
 ""Action"": ""sts:AssumeRole"",
 ""Principal"": {
 ""Service"": ""lambda.amazonaws.com""
 },
 ""Effect"": ""Allow"",
 ""Sid"": """"
 }
 ]
 }"
 });

 var fullAccess = new Aws.Iam.RolePolicyAttachment("mylambda-access", new Aws.Iam.RolePolicyAttachmentArgs
 {
 Role = role.Name,
 PolicyArn = "arn:aws:iam::aws:policy/AWSLambda_FullAccess",
 });
 }
}

This IAM role can then be used when deploying the Lambda Function:

const lambda = new aws.lambda.Function("mylambda", {
 code: new pulumi.asset.AssetArchive({
 ".": new pulumi.asset.FileArchive("./app"),
 }),
 role: role.arn,
 handler: "index.handler",
 runtime: aws.lambda.NodeJS12dXRuntime,
 architectures: ["arm64"],
});

function = aws.lambda_.Function("mylambda",
 role=role.arn,
 runtime="nodejs12.x",
 handler="index.handler",
 code=pulumi.AssetArchive({
 '.': pulumi.FileArchive('./app')
 }),
 architectures=["arm64"],
 )

function, err := lambda.NewFunction(ctx, "mylambda", &lambda.FunctionArgs{
 Handler: pulumi.String("index.handler"),
 Role: role.Arn,
 Runtime: pulumi.String("nodejs12.x"),
 Code: pulumi.NewFileArchive("./app"),
 Architectures: pulumi.StringArray{pulumi.String("arm64")},
})
if err != nil {
 return nil
}

 var lambda = new Function("mylambda", new FunctionArgs
 {
 Runtime = "nodejs12.x",
 Code = new FileArchive("./app"),
 Handler = "index.handler",
 Role = role.Arn
 Architectures = {"arm64"}
 });

The only new thing we need to do to deploy our Lambda function to Graviton2-based infrastructure is to provide the architectures input, specifying a value of arm64. With that small change, our Lambda Function will execute on an Arm-based instance.

When we deploy this via Pulumi, we can see that the Function has been deployed with the correct architecture in AWS:

$ pulumi up
Previewing update (dev)

View Live: https://app.pulumi.com/stack72/lambda-graviton-function/dev/previews/ed6a2b10-d0a8-4c08-ba2d-c749f216db67

 Type Name Plan
 + pulumi:pulumi:Stack lambda-graviton-function-dev create
 + ├─ aws:iam:Role lambda-role create
 + ├─ aws:iam:RolePolicyAttachment mylambda-access create
 + └─ aws:lambda:Function mylambda create

Resources:
 + 4 to create

Do you want to perform this update? yes
Updating (dev)

View Live: https://app.pulumi.com/stack72/lambda-graviton-function/dev/updates/1

 Type Name Status
 + pulumi:pulumi:Stack lambda-graviton-function-dev created
 + ├─ aws:iam:Role lambda-role created
 + ├─ aws:lambda:Function mylambda created
 + └─ aws:iam:RolePolicyAttachment mylambda-access created

Resources:
 + 4 created

Duration: 25s

If the architectures input is not specified, AWS will set the value by default to x86_64.

Since we are running our application code on ARM, we need to ensure that the code in the archive we use for our Lambda function can run natively on the ARM-based infrastructure. For interpreted languages like Node.js and Python, most applications will work as is. For compiled languages like Go, we need to ensure that our Go code is cross-compiled for GOARCH=arm64, and that the resulting ARM64 binary is included in our Lambda source archive.

Creating a Lambda Layer with Graviton2 Support

To create a Lambda Layer with Graviton2 support, we can extend our example above. Firstly, let’s create a lambda layer:

const layer = new aws.lambda.LayerVersion("my-layer-version", {
 code: new pulumi.asset.AssetArchive({
 ".": new pulumi.asset.FileArchive("./base-layer"),
 }),
 compatibleRuntimes: [
 aws.lambda.Runtime.NodeJS14dX,
 ],
 layerName: "my-base-layer",
 compatibleArchitectures: ["arm64", "x86_64"]
});

layer = aws.lambda_.LayerVersion("my-layer-version",
 compatible_runtimes=["nodejs12.x"],
 code=pulumi.FileArchive("./base-layer"),
 layer_name="my-base-layer",
 compatible_architectures=["arm64", "x86_64"])

layer, err := lambda.NewLayerVersion(ctx, "my-layer-version", &lambda.LayerVersionArgs{
 LayerName: pulumi.String("my-base-layer"),
 Code: pulumi.NewFileArchive("./base-layer"),
 CompatibleRuntimes: pulumi.StringArray{pulumi.String("nodejs12.x")},
 CompatibleArchitectures: pulumi.StringArray{pulumi.String("nodejs12.x")},
})
if err != nil {
 return nil
}

 var layer = new LayerVersion("my-layer-version", new LayerVersionArgs
 {
 CompatibleRuntimes =
 {
 "nodejs12.x",
 },
 CompatibleArchitectures =
 {
 "arm64",
 "x86_64",
 },
 Code = new FileArchive("./base-layer"),
 LayerName = "my-base-layer",
 });

This will create a Lambda layer that is compatible with NodeJS 12.x and ARM64 & X86_64. Layers can be created that are compatible with ARM64, X86_64 or both architectures. We can then build a new lambda based on this Lambda layer:

const lambda = new aws.lambda.Function("mylambda", {
 code: new pulumi.asset.AssetArchive({
 ".": new pulumi.asset.FileArchive("./app"),
 }),
 role: role.arn,
 handler: "index.handler",
 runtime: aws.lambda.NodeJS12dXRuntime,
 architectures: ["arm64"],
 layers: [layer.arn]
});

function = aws.lambda_.Function("mylambda",
 role=role.arn,
 runtime="nodejs12.x",
 handler="index.handler",
 code=pulumi.AssetArchive({
 '.': pulumi.FileArchive('./app')
 }),
 architectures=["arm64"],
 layers=[layer.arn]
 )

function, err := lambda.NewFunction(ctx, "mylambda", &lambda.FunctionArgs{
 Handler: pulumi.String("index.handler"),
 Role: role.Arn,
 Runtime: pulumi.String("nodejs12.x"),
 Code: pulumi.NewFileArchive("./app"),
 Architectures: pulumi.StringArray{pulumi.String("arm64")},
 Layers: pulumi.StringArray{layer.Arn},
})
if err != nil {
 return nil
}

 var lambda = new Function("mylambda", new FunctionArgs
 {
 Runtime = "nodejs12.x",
 Code = new FileArchive("./app"),
 Handler = "index.handler",
 Role = role.Arn
 Architectures = {"arm64"}
 Layers =
 {
 lambdaLayer.Arn,
 },
 });

This will deploy a function built on top of the Lambda layer targeting the Graviton2 architecture and NodeJS 12.x runtime.

Summary

AWS support for Lambda Functions powered by AWS Graviton2 offers users the ability to get better price performance for their workloads. We’ve shown how you can build your Functions using this ability. You can read more about AWS Lambda functions on the AWS Documentation website and getting started with AWS Graviton on GitHub.

Deploy AWS Lambda with Elastic File System using Pulumi

Luke Hoban — Tue, 16 Jun 2020 00:00:00 +0000

Some of the code in this post is out of date. See the AWS guides for an updated overview and examples.

Ever since AWS Lambda was released in 2015, users have wanted persistent file storage beyond the small 512MB /tmp disk allocated to each Lambda function. The following year, Amazon launched EFS, offering a simple managed file system service for AWS, but initially only available to mount onto Amazon EC2 instances. Over the last few months, AWS has been extending access to EFS to all of the modern compute offerings. First EKS for Kubernetes, then ECS and Fargate for containers. Today, AWS announced that EFS is now also supported in Lambda, providing easy access to network file systems from your serverless functions.

In this post, we’ll show you how to get started using EFS with Lambda and Pulumi.

Why would you want to use EFS from Lambda? There are a few common reasons:

You need more local disk than the 512MB of /tmp space available to a single Lambda Function.
You want to share persistent storage across various functions and would prefer to use file system storage instead of blob storage or a database.
You want to share persistent storage between Lambda and other compute (like containers or ECS), and would prefer to use file storage over blob storage or a database.

Storing data to the file system is particularly natural for the kind of small functions common in Lambda - no need to use the AWS SDK - just call fs.writeFileSync() (or equivalent for your language of choice).

EFS + Lambda Concepts

To use an EFS file system from Lambda, we need a few things:

A VPC
An EFS File System in the VPC
EFS mount points in each of the subnets of our VPC
An EFS Access Point specifying how Lambda should mount the EFS volume
An AWS Lambda Function running in the same VPC and private subnets and referencing the EFS Access Point

Let’s walk through how to build the required infrastructure. We’ll use Pulumi to provision the necessary resources. You can check out the full source code in the Pulumi Examples.

Because EFS is tied to a VPC, we can only access it from a Lambda running inside a VPC. We should create a VPC with both public and private subnets to ensure the Lambda function can reach the internet (since Lambda functions inside VPCs are not assigned public IPs). We could use an existing VPC or create a new one for our application.

const vpc = new awsx.ec2.Vpc("vpc", { subnets: [{ type: "private" }, { type: "public" }] });
const subnetIds = await vpc.publicSubnetIds;

An EFS FileSystem is a regional service, but accessing it requires adding mount points to a VPC. We can add those to the private subnets where our Functions will run. Each mount target has a security group, and we can use the default security group to allow access only from other compute resources in the same security group in our VPC. We could also use an existing FileSystem and mount targets if they have already been provisioned for another application.

const filesystem = new aws.efs.FileSystem("filesystem");
const targets = [];
for (let i = 0; i < subnetIds.length; i++) {
 targets.push(new aws.efs.MountTarget(`fs-mount-${i}`, {
 fileSystemId: filesystem.id,
 subnetId: subnetIds[i],
 securityGroups: [vpc.vpc.defaultSecurityGroupId],
 }));
}

To use EFS from our Lambda Function, we need to tell Lambda how to mount the NFS volume. Specifically, we must specify the Unix user and group to use to enable access, and the location and permission scheme to use as the root folder of the mount. This is done using an EFS AccessPoint.

const ap = new aws.efs.AccessPoint("ap", {
 fileSystemId: filesystem.id,
 posixUser: { uid: 1000, gid: 1000 },
 rootDirectory: { path: "/www", creationInfo: { ownerGid: 1000, ownerUid: 1000, permissions: "755" } },
}, { dependsOn: targets });

Finally, each Lambda Function we want to run will need to be in our VPC, use our default security group, and mount the EFS file system via our AccessPoint. We can write a small function that creates Lambda Functions with this configuration and we can specify where we want to mount this into the Lambda Function, though we must mount it underneath /mnt/.

function efsvpcCallback(name: string, f: aws.lambda.Callback<awsx.apigateway.Request, awsx.apigateway.Response>) {
 return new aws.lambda.CallbackFunction(name, {
 policies: [aws.iam.ManagedPolicy.AWSLambdaVPCAccessExecutionRole, aws.iam.ManagedPolicy.LambdaFullAccess],
 vpcConfig: {
 subnetIds: vpc.privateSubnetIds,
 securityGroupIds: [vpc.vpc.defaultSecurityGroupId],
 },
 fileSystemConfig: { arn: ap.arn, localMountPath: "/mnt/storage" },
 callback: f,
 });
}

We now have our infrastructure all set up to start using Lambda with EFS. Let’s build an app!

Interact with EFS from an HTTP API

For a first simple Lambda application, we’ll expose an HTTP API that GETs files from, and POSTs files to, our EFS file system. We’ll implement this with API Gateway and two routes. Using the helper we defined above, each of these is just a few lines of code.

Note that we can use our normal Node.js fs.readFileSync and fs.writeFileSync APIs to interact with the EFS file system - no need for an AWS SDK to talk to EFS. This makes it easy to take existing software and libraries that can interact with the filesystem and run them inside Lambda.

const api = new awsx.apigateway.API("api", {
 routes: [
 {
 method: "GET", path: "/files/{filename+}", eventHandler: efsvpcCallback("getHandler", async (ev, ctx) => {
 try {
 const f = "/mnt/storage/" + ev.pathParameters!.filename;
 const data = fs.readFileSync(f)
 return {
 statusCode: 200,
 body: data.toString(),
 };
 } catch {
 return { statusCode: 500, body: "" }
 }
 }),
 },
 {
 method: "POST", path: "/files/{filename+}", eventHandler: efsvpcCallback("uploadHandler", async (ev, ctx) => {
 try {
 const f = "/mnt/storage/" + ev.pathParameters!.filename;
 const data = new Buffer(ev.body!, 'base64');
 fs.writeFileSync(f, data)
 return {
 statusCode: 200,
 body: "",
 };
 } catch {
 return { statusCode: 500, body: "" }
 }
 }),
 },
 ]
});

Once this is deployed, we can interact with the API to add files to our file system:

$ curl -X POST -d 'Hello world' $(pulumi stack output url)files/file.txt
$ curl -X GET $(pulumi stack output url)files/file.txt
Hello, world

Easy as that - our two Lambda functions are sharing data via EFS!

We can get more insight into what the file system looks like inside Lambda by exposing a REST endpoint to run commands inside the a Lambda function. This will allow us to run curl to run commands to populate and read the contents of the file system.

...
 {
 method: "POST", path: "/", eventHandler: efsvpcCallback("postHandler", async (ev, ctx) => {
 const cmd = new Buffer(ev.body!, 'base64').toString()
 const buf = cp.execSync(cmd);
 return {
 statusCode: 200,
 body: buf.toString(),
 };
 }),
 },
...

We can now interact with our EFS file system by just running shell commands. Since it’s a normal file system mount, all our standard Unix tools work as we expect. Note that the files are owned by uid/gid 1000/1000, and have the 755 permissions specified by our Access Point.

$ curl -X POST -d 'ls -la /mnt/storage' $(pulumi stack output url)
total 8
drwxr-xr-x 2 1000 1000 6144 Jun 10 05:37 .
drwxr-xr-x 3 root root 4096 Jun 10 05:35 ..
-rw-rw-r-- 1 1000 1000 21 Jun 10 05:37 file.txt

$ curl -X POST -d 'echo "<h1>Hello World</h1>" > /mnt/storage/index.html' $(pulumi stack output url)
$ curl -X POST -d 'ls -la /mnt/storage' $(pulumi stack output url)
total 12
drwxr-xr-x 2 1000 1000 6144 Jun 10 05:38 .
drwxr-xr-x 3 root root 4096 Jun 10 05:35 ..
-rw-rw-r-- 1 1000 1000 21 Jun 10 05:37 file.txt
-rw-rw-r-- 1 1000 1000 21 Jun 10 05:38 index.html

Now that we can read and write to our file system from Lambda, what about sharing it between Lambda and other compute services? We could mount our volume to EC2, EKS, ECS, or Fargate. Let’s try out Fargate by running a small nginx server that serves from the same file system we just used.

First, we create an EFS volume configuration that describes the volume we want to mount in our Fargate Task. This is similar to Lambda - we use the same AccessPoint and root directory.

const efsVolumeConfiguration: aws.types.input.ecs.TaskDefinitionVolumeEfsVolumeConfiguration = {
 fileSystemId: filesystem.id,
 authorizationConfig: { accessPointId: ap.id, },
 rootDirectory: "/www",
};

Then we can launch our nginx task. We use our EFS volume configuration to define a volume named efs in the task, and then mount that volume into the /usr/share/nginx/html path in the nginx container so it can serve content directly from the EFS file system. Note that we also have to explicitly set the platformVersion to 1.4.0 so that we can use EFS since 1.4.0 is not yet the default. We ensure that we are using the same security group that can access the file system and that we’ve opened up access to port 80 for our nginx server.

const nginx = new awsx.ecs.FargateService("nginx", {
 cluster: cluster,
 taskDefinitionArgs: {
 container: {
 image: "nginx",
 memory: 128,
 portMappings: [{ containerPort: 80, hostPort: 80, protocol: "tcp" }],
 mountPoints: [{ containerPath: "/usr/share/nginx/html", sourceVolume: "efs" }],
 },
 volumes: [{ name: "efs", efsVolumeConfiguration }],
 },
 securityGroups: [vpc.vpc.defaultSecurityGroupId, ...cluster.securityGroups],
 subnets: vpc.publicSubnetIds,
 platformVersion: "1.4.0",
});

Navigating to our Task’s public IP - we see the contents of the file we stored in our file system.

Conclusion

Support for EFS in Lambda opens up a lot of new options for managing persistent data inside serverless functions. And even more importantly, makes it easy to share data in a format that existing applications in VM instances or containers can natively work with.

In this post, we’ve shown how you can use Pulumi to manage Lambda + EFS. Pulumi makes it easy to provision and manage cloud infrastructure on any cloud, using programming languages you are familiar with, including TypeScript, Python, Go and .NET. This is a great fit for EFS and Lambda, which combine managed infrastructure with small serverless function code.

Check out the full Lambda + EFS example above in the Pulumi Examples, learn more about Lambda support for EFS in the AWS docs, and get started with Pulumi for AWS today.

Controlling AWS Costs with Pulumi and AWS Lambda

Paul Stack — Thu, 09 Apr 2020 00:00:00 +0000

Due to the nature of the product we build, the Pulumi team needs to have access to several cloud providers to develop and test the product. An increasing number of cloud providers comes with an associated ever-increasing cost.

Giving a development team the ability to launch the resources they need in the cloud is both the best and most scary thing that happens in a company. The flexibility for them to be able to achieve their goal is fantastic, but with this great freedom can come problems. We depend on developers to clean up after themselves, but sometimes that doesn’t happen. Throw in automated testing failures that leak cloud resources, and we can very quickly get ourselves into a situation where costs spiral out of control.

There are multiple ways that Pulumi can help you deal with this situation. Joe Duffy wrote a blog post on Automatically Enforcing Tagging Policies to demonstrate how Policy as Code can ensure that tags are automatically added to any Pulumi managed infrastructure. If your company doesn’t use Pulumi to build and manage their infrastructure, then you can still use Pulumi in collaboration with AWS Lambda to build and deploy an application that can monitor your cloud costs.

To keep our cloud costs down, Pulumi uses an AWS Lambda Go application that removes wasteful resources from our AWS accounts daily. If a developer needs to protect their resources from being deleted, they need to add a tag to their infrastructure of “Owner: ”. This ensures the owner of a resource can be tracked at any time.

Cloud Cost Lambda

Here’s our application in it’s simplest form.

package main

import (
 "log"
 "os"
 "strconv"

 "github.com/aws/aws-lambda-go/lambda"
 "github.com/aws/aws-sdk-go-v2/aws/external"
)

var DeleteEnabled bool

func main() {
 lambda.Start(HandleRequest)
}

func HandleRequest() {
 destroyOperations, _ := os.LookupEnv("DESTROY_ENABLED")
 deleteEnabled, err := strconv.ParseBool(destroyOperations)
 if err != nil {
 log.Print("loading DESTROY_ENABLED envvar failed")
 log.Fatal(err)
 } else {
 DeleteEnabled = deleteEnabled
 }

 cfg, err := external.LoadDefaultAWSConfig()
 if err != nil {
 log.Print("loading AWS config failed")
 log.Fatal(err)
 }

 log.Printf("Running cleanup operations in %q", cfg.Region)
 log.Printf("Are destroy operations enabled? %q", destroyOperations)

 log.Print("Finished cleanup operations")
}

To deploy this application, we need to build and package the lambda. Packaging a Go lambda requires compiling it for Linux and then zipping the lambda up. We can do this as follows:

$ GOOS=linux GOARCH=amd64 go build -o main main.go && zip deployment.zip main

After creating a new Pulumi application, we can write the following code to deploy this lambda for us:

import * as aws from "@pulumi/aws";
import * as pulumi from "@pulumi/pulumi";

const lambdaRole = new aws.iam.Role("my-lambda-role", {
 name: "sample-role",
 assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal({ Service: "lambda.amazonaws.com" }),
});

const lambda = new aws.lambda.Function("my-lambda-function", {
 name: "sample-lambda",
 runtime: aws.lambda.Go1dxRuntime,
 timeout: 900,
 role: lambdaRole,
 handler: "main",
 memorySize: 1024,
 code: new pulumi.asset.FileArchive("deployment.zip"),
 environment: {
 variables: {
 "DESTROY_ENABLED": "true"
 }
 },
});

We can deploy the application using pulumi up:

$ pulumi up
Previewing update (dev):
 Type Name Plan
 + pulumi:pulumi:Stack lambda-post-dev create
 + ├─ aws:iam:Role my-lambda-role create
 + └─ aws:lambda:Function my-lambda-function create

Resources:
 + 3 to create

Do you want to perform this update? yes
Updating (dev):
 Type Name Status
 + pulumi:pulumi:Stack lambda-post-dev created
 + ├─ aws:iam:Role my-lambda-role created
 + └─ aws:lambda:Function my-lambda-function created

Resources:
 + 3 created

Duration: 27s

We can go to the AWS Lambda console and trigger the application. To do this, we must trigger a test event. Our application doesn’t accept any inputs, so the test event appears as follows:

When we trigger the lambda, we should get an output as follows:

START RequestId: 0910c342-4c43-4946-aeca-cf57683aa36b Version: $LATEST
2020/04/07 21:37:01 Running cleanup operations in "us-east-1"
2020/04/07 21:37:01 Are destroy operations enabled? "true"
2020/04/07 21:37:01 Finished cleanup operations
END RequestId: 0910c342-4c43-4946-aeca-cf57683aa36b
REPORT RequestId: 0910c342-4c43-4946-aeca-cf57683aa36b Duration: 1.96 ms Billed Duration: 100 ms Memory Size: 1024 MB Max Memory Used: 38 MB Init Duration: 99.32 ms

Our lambda application works!! But, it’s not going to delete anything because we haven’t told it what to delete. We can start very small. Let’s tell AWS to cleanup any EBS disks that are currently unattached.

func handleUnattachedVolumes(cfg aws.Config) {
 log.Print("Running cleanup operations for unattached ebs volumes")
 svc := ec2.New(cfg)

 input := &ec2.DescribeVolumesInput{
 Filters: []ec2.Filter{
 {
 Name: aws.String("status"),
 Values: []string{"available"},
 },
 },
 }
 volumes := make([]ec2.Volume, 0)

 for {
 req := svc.DescribeVolumesRequest(input)
 res, err := req.Send(context.Background())
 if err != nil {
 log.Fatal(err)
 }

 volumes = append(volumes, res.Volumes...)

 if res.NextToken == nil {
 break
 }

 input.NextToken = res.NextToken
 }

 for _, volume := range volumes {
 log.Printf("Killing EBS Volume %q as it's unattached", *volume.VolumeId)
 if DeleteEnabled {
 delReq := svc.DeleteVolumeRequest(&ec2.DeleteVolumeInput{
 VolumeId: volume.VolumeId,
 })
 _, err := delReq.Send(context.Background())
 if err != nil {
 log.Print(err)
 log.Printf("error deleting volume %q", *volume.VolumeId)
 }
 }
 }
}

We can then update our lambda handler to include this new EBS cleanup function:

func HandleRequest() {
 destroyOperations, _ := os.LookupEnv("DESTROY_ENABLED")
 deleteEnabled, err := strconv.ParseBool(destroyOperations)
 if err != nil {
 log.Print("loading DESTROY_ENABLED envvar failed")
 log.Fatal(err)
 } else {
 DeleteEnabled = deleteEnabled
 }

 cfg, err := external.LoadDefaultAWSConfig()
 if err != nil {
 log.Print("loading AWS config failed")
 log.Fatal(err)
 }

 log.Printf("Running cleanup operations in %q", cfg.Region)
 log.Printf("Are destroy operations enabled? %q", destroyOperations)

 handleUnattachedVolumes(cfg)

 log.Print("Finished cleanup operations")
}

When we run pulumi up, Pulumi will understand that the source code of the lambda has changed:

$ pulumi up
Previewing update (dev):
 Type Name Plan Info
 pulumi:pulumi:Stack lambda-post-dev
 ~ └─ aws:lambda:Function my-lambda-function update [diff: ~code]

Resources:
 ~ 1 to update
 2 unchanged

Do you want to perform this update? yes
Updating (dev):
 Type Name Status Info
 pulumi:pulumi:Stack lambda-post-dev
 ~ └─ aws:lambda:Function my-lambda-function updated [diff: ~code]

Resources:
 ~ 1 updated
 2 unchanged

Duration: 22s

When we execute the lambda code, we will receive an error now:

START RequestId: dc34a4f0-55f3-434b-a7e4-7832132865d1 Version: $LATEST
2020/04/07 21:45:01 Running cleanup operations in "us-east-1"
2020/04/07 21:45:01 Are destroy operations enabled? "true"
2020/04/07 21:45:01 Running cleanup operations for unattached ebs volumes
2020/04/07 21:45:01 UnauthorizedOperation: You are not authorized to perform this operation.
status code: 403, request id: c03efbf0-a3b4-4dd3-805b-02cec3523e0b
END RequestId: dc34a4f0-55f3-434b-a7e4-7832132865d1
REPORT RequestId: dc34a4f0-55f3-434b-a7e4-7832132865d1 Duration: 172.66 ms Billed Duration: 200 ms Memory Size: 1024 MB Max Memory Used: 49 MB Init Duration: 121.26 ms
RequestId: dc34a4f0-55f3-434b-a7e4-7832132865d1 Process exited before completing request

We haven’t given our IAM Role permission to interact with EBS. Let’s attach a specific policy to the IAM Role that allows us to interact with EBS:

const lambdaRolePolicy = new aws.iam.Policy("my-policy", {
 name: "sample-lambda-plicy",
 policy: {
 Version: "2012-10-17",
 Statement: [{
 Action: [
 "ec2:DescribeVolumes",
 "ec2:DeleteVolume",
 ],
 Resource: "*",
 Effect: "Allow",
 }],
 },
});

const policyAttachment = new aws.iam.PolicyAttachment("my-attachment", {
 policyArn: lambdaRolePolicy.arn,
 roles: [lambdaRole],
});

After deploying this updated IAM role, we can then see the execution will be allowed to proceed:

START RequestId: 7ca595aa-0a0f-4662-ab64-835ab0b02258 Version: $LATEST
2020/04/07 21:49:28 Running cleanup operations in "us-east-1"
2020/04/07 21:49:28 Are destroy operations enabled? "true"
2020/04/07 21:49:28 Running cleanup operations for unattached ebs volumes
2020/04/07 21:49:28 Finished cleanup operations
END RequestId: 7ca595aa-0a0f-4662-ab64-835ab0b02258
REPORT RequestId: 7ca595aa-0a0f-4662-ab64-835ab0b02258 Duration: 182.97 ms Billed Duration: 200 ms Memory Size: 1024 MB Max Memory Used: 26 MB

You can continue adding new segments to the cleanup lambda to help cleanup your AWS account.

Deploying To Multiple Regions

We want to ensure that all of our regions are cleaned up; therefore, we need to change the structure of the code. We can take advantage of explicit provider configuration to allow us to target the deployment of a resource to a specific region.

Let’s assume we want to deploy our application to us-east-1, us-east-2, eu-west-1, eu-west-2 and us-west-2. We can create a map of Providers to do this as follows:

const providers: {[key: string]: aws.Provider} = {
 "us-east-1": new aws.Provider("us-east-1", {region: "us-east-1"}),
 "us-east-2": new aws.Provider("us-east-2", {region: "us-east-2"}),
 "us-west-2": new aws.Provider("us-west-2", {region: "us-west-2"}),
 "eu-west-1": new aws.Provider("eu-west-1", {region: "eu-west-1"}),
 "eu-west-2": new aws.Provider("eu-west-2", {region: "eu-west-2"}),
};

We can iterate through the map and deploy our lambda to each of the configured regions as follows:

const providers: {[key: string]: aws.Provider} = {
 "us-east-1": new aws.Provider("us-east-1", {region: "us-east-1"}),
 "us-east-2": new aws.Provider("us-east-2", {region: "us-east-2"}),
 "us-west-2": new aws.Provider("us-west-2", {region: "us-west-2"}),
 "eu-west-1": new aws.Provider("eu-west-1", {region: "eu-west-1"}),
 "eu-west-2": new aws.Provider("eu-west-2", {region: "eu-west-2"}),
};

for (const providerKey of Object.keys(providers)) {
 const provider = providers[providerKey];

 const lambda = new aws.lambda.Function(`my-lambda-function-${providerKey}`, {
 name: "sample-lambda",
 runtime: aws.lambda.Go1dxRuntime,
 timeout: 900,
 role: lambdaRole.arn,
 handler: "main",
 memorySize: 1024,
 code: new pulumi.asset.FileArchive("deployment.zip"),
 environment: {
 variables: {
 "DESTROY_ENABLED": "true"
 }
 },
 }, {provider});
}

Pulumi can then deploy the lambdas to all of the providers in the loop. If a new region requires cleaning, then it needs to be added to the map of providers.

In Conclusion

The Go application we built continues to evolve. There are new areas of our account that have started to accumulate costs. After a few iterations of the application, we were able to drop 64% of our daily AWS costs.

We look forward to hearing the results of your work in this area!

Scheduling Serverless

Cyrus Najmabadi — Tue, 25 Feb 2020 00:00:00 +0000

Scheduling events has long been an essential part of automation; many tasks need to run at specific times or intervals. You could be checking StackOverflow for new questions every 20 minutes or compiling a report that is emailed every other Friday at 4:00 pm. Today, many of these tasks can be efficiently accomplished in the cloud. While each cloud has its flavor of scheduled functions, this post steps you through an example using AWS CloudWatch with the help of Pulumi.

Taking out the trash

Let’s assume we have an S3 Bucket similar to the Recycle Bin on your desktop. It contains discarded items that we might not be ready to delete yet, but because we are paying for storage, we want to empty this bucket every Friday at 6:00 pm EST.

import * as aws from "@pulumi/aws";
import { ObjectIdentifier } from "aws-sdk/clients/s3";

// Create an AWS resource (S3 Bucket)
const trashBucket = new aws.s3.Bucket("trash");

We’re now ready to create our Lambda function and schedule it. Pulumi simplifies creating Lambda functions by allowing us to provide a handler function to an event such as cloudwatch.onSchedule. To learn more, check out our Lambdas as Lambdas post. Let’s create a cloudwatch.EventRuleEventHandler async function that will get all the items from the trash bucket and delete them.

// A handler function that will list objects in the bucket and bulk delete them
const emptyTrash: aws.cloudwatch.EventRuleEventHandler = async (
 event: aws.cloudwatch.EventRuleEvent
) => {
 const s3Client = new aws.sdk.S3(); //creates interface to service
 const bucket = trashBucket.id.get();

 const { Contents = [] } = await s3Client //get list of objects in bucket
 .listObjects({ Bucket: bucket })
 .promise();
 const objects: ObjectIdentifier[] = Contents.map(object => {
 return { Key: object.Key! };
 });

 await s3Client //delete objects
 .deleteObjects({
 Bucket: bucket,
 Delete: { Objects: objects, Quiet: false }
 })
 .promise()
 .catch(error => console.log(error));
 console.log(
 `Deleted ${Contents.length} item${
 Contents.length === 1 ? "" : "s"
 } from ${bucket}.`
 );
};

Now that we have our handler function, we can create a CloudWatch event that fires based on the specified schedule, which is every Friday at 6:00 pm EST. We’ll need to represent that as a Schedule Expression. CloudWatch events support cron and rate expressions. If we wanted our function to run on a set interval (i.e., every 20 minutes), we would choose a rate expression. Since we want fine-grained schedule control, we’ll be using a cron expression. You can learn more at the Schedule Expressions documentation. Because time for cron jobs uses UTC, we’ll need to schedule our event to fire on Friday at 11:00 pm UTC.

// Schedule the function to run every Friday at 11:00pm UTC (6:00pm EST)
// More info on Schedule Expressions at
// https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html
const emptyTrashSchedule: aws.cloudwatch.EventRuleEventSubscription = aws.cloudwatch.onSchedule(
 "emptyTrash",
 "cron(0 23 ? * FRI *)",
 emptyTrash);

// Export the name of the bucket
export const bucketName = trashBucket.id;

Now, run pulumi up to deploy your new scheduled function.

$ pulumi up
Previewing update (dev):
…

Updating (dev):

 Type Name Status
 pulumi:pulumi:Stack aws-ts-scheduled-function-dev
 + └─ aws:cloudwatch:EventRuleEventSubscription emptyTrash created
 + ├─ aws:cloudwatch:EventRule emptyTrash created
 + ├─ aws:iam:Role emptyTrash created
 + ├─ aws:iam:RolePolicyAttachment emptyTrash-32be53a2 created
 + ├─ aws:lambda:Function emptyTrash created
 + ├─ aws:lambda:Permission emptyTrash created
 + └─ aws:cloudwatch:EventTarget emptyTrash created

Outputs:
 bucketName: "trash-1cde441"

Resources:
 + 7 created
 2 unchanged

Duration: 14s

Test it out by adding some items to your bucket and see if they’re deleted. You can use pulumi logs to view the console.log from the handler function. If you can’t wait until Friday, experiment with changing the cron expression and rerun pulumi up. You can find the full code in our GitHub repository.

Want to know more?

This simple example demonstrates what you can do with serverless functions, such as AWS Lambda, without having to set up your infrastructure. Check out more serverless examples in the Pulumi Examples to see what you can build.

Managing AWS Containers with Pulumi

Sophia Parafina — Mon, 06 Jan 2020 00:00:00 +0000

Some of the code in this post is out of date. See the AWS guides for an updated overview and examples.

The Amazon Web Services (AWS) Cloud ecosystem is large and vibrant, so vast and vibrant that at times, it can be challenging to know where best to start! In the case of containers, Abby Fuller tweeted a descriptive summary about using AWS container services.

It looks straightforward, but it’s easy to get lost in the details when configuring multiple services to get your application running on AWS. Although AWS has extensive documentation and a point and click interface to these services, replicating then can be challenging, and it isn’t conducive to understanding the architecture. However, using infrastructure as code lets you see the details and results in a reproducible deployment that you can expand on. Let’s implement the diagram using Pulumi to deploy your container infrastructure with code in AWS.

Store your images

The AWS Elastic Container Registry (ECR) is a container registry that supports private container registries. ECR makes it easy to store, build, manage, and deploy container images and eliminates the need to operate your registry or use public registries, all in a highly available and scalable architecture.

In this example, we use Pulumi’s Crosswalk for AWS. Crosswalk for AWS is a collection of frequent tasks and best practices that simplify deploying infrastructure on AWS. We first declare a repository; this creates the AWS Container Repository resource. The last line of the code exports the URL so that we can access the repository after we have updated our Pulumi stack. After we’ve run pulumi up, the repository is ready to go, and you can use the Docker CLI to push, pull, and manage images.

import * as awsx from "@pulumi/awsx";

// Create a repository, as before.
const repo = new awsx.ecr.Repository("my-repo", {
 forceDelete: true,
});

// And publish its URL, so we can push to it if we'd like.
export const url = repo.repository.repositoryUrl;

Choose your scheduler

There are two services for running containers in the AWS Cloud: Elastic Cloud Service (ECS), or Elastic Kubernetes Service (EKS). ECS is a proprietary service for running containers in the AWS Cloud. In contrast, EKS is a managed Kubernetes service for containers using a Kubernetes control plane. It has three master nodes distributed across three availability zones to provide high availability.

ECS is designed to work with other AWS services and provides more straightforward configuration and integration with them while providing high availability. In contrast, EKS is a managed Kubernetes service, and because Kubernetes is open-source, your infrastructure is portable to other cloud providers. Also, Kubernetes can provide fine-grain control over deployed services. The choice of which scheduler to use depends on your requirements.

You can create either an ECS or EKS cluster using Pulumi. If you wish to use ECS, @pulumi/aws provides all the primitives needed to build infrastructure on AWS.

import * as awsx from "@pulumi/aws";

// Create an ECS cluster explicitly, and give it a name tag.
const cluster = new aws.ecs.Cluster("custom", {
 tags: {
 "Name": "my-custom-ecs-cluster",
 },
});

If you wish to use EKS, use the Pulumi EKS library (@pulumi/eks) for Kubernetes primitives on AWS.

import * as eks from "@pulumi/eks";

// Create an EKS cluster with the default configuration.
const cluster = new eks.Cluster("my-cluster");

Choose where/how to run, Fargate or EC2

Fargate is an AWS service that runs containers. It is suited to running small workloads or batch workloads with occasional bursts that require scaling quickly. AWS EC2 is ideal for large workloads requiring many CPU cores and gigabytes of memory.

Fargate

In this example, we create a load balancer open on port 80, spin up two instances of our container, and publish the endpoint URL.

import * as pulumi from "@pulumi/pulumi";

const lb = new awsx.lb.ApplicationLoadBalancer("nginx");

const service = new awsx.ecs.FargateService("service", {
 cluster: cluster.arn,
 assignPublicIp: true,
 taskDefinitionArgs: {
 containers: {
 nginx: {
 image: image.imageUri,
 essential: true,
 portMappings: [{
 containerPort: 80,
 targetGroup: lb.defaultTargetGroup,
 }],
 }
 }
 },
 desiredCount: 2,
});

// Export the load balancer's address so that it's easy to access.
export const appURL = pulumi.interpolate`http://${loadbalancer.loadBalancer.dnsName}`;

EC2 with Kubernetes

With Kubernetes, we create a deployment for the application and a service to make the application accessible via port 80. We publish both application endpoint to make it accessible and cluster config for use with kubectl CLI.

// Deploy a small canary service (NGINX), to test that the cluster is working.
const appName = "my-app";
const appLabels = { appClass: appName };
const deployment = new k8s.apps.v1.Deployment(`${appName}-dep`, {
 metadata: { labels: appLabels },
 spec: {
 replicas: 2,
 selector: { matchLabels: appLabels },
 template: {
 metadata: { labels: appLabels },
 spec: {
 containers: [{
 name: appName,
 image: "nginx",
 ports: [{ name: "http", containerPort: 80 }]
 }],
 }
 }
 },
}, { provider: cluster.provider });
const service = new k8s.core.v1.Service(`${appName}-svc`, {
 metadata: { labels: appLabels },
 spec: {
 type: "LoadBalancer",
 ports: [{ port: 80, targetPort: "http" }],
 selector: appLabels,
 },
}, { provider: cluster.provider });

// Export the URL for the load balanced service.
export const url = service.status.loadBalancer.ingress[0].hostname;

// Export the cluster's kubeconfig.
export const kubeconfig = cluster.kubeconfig;

Build!

Let’s put it all together for both ECS and EKS. To get started, follow the Pulumi AWS documentation to install Pulumi, install AWS Client set your AWS environment variables, and create a new project in Typescript.

$ mkdir get-started && cd get-started
$ pulumi new aws-typescript

In this example, we build the application in code using the awsx.ecr.Image resource. This resource uses Docker to build the image locally and push it to our repository. Make sure that you have Docker installed and running.

The container application for this example is an HTML page in NGINX. Make a ./app directory in your Pulumi project and add the Dockerfile below.

Dockerfile:

FROM nginx
COPY content /usr/share/nginx/html

Create a ./content director in the ./app directory and add the HTML file below.

<html>
 <head><meta charset="UTF-8">
 <title>Hello, Pulumi!</title>
 </head>
 <body>
 <p>Hello, containers!</p>
 <p>Made with ❤️ with <a href="https://pulumi.com">Pulumi</a></p>
 </body>
</html>

Now that setup is complete you project directory should look similar to this:

get-started/
├── Pulumi.dev.yaml
├── Pulumi.yaml
├── app
│   ├── Dockerfile
│   └── content
│   └── index.html
├── index.ts
├── node_modules
├── package-lock.json
├── package.json
└── tsconfig.json

ECS

Now we’re ready to start building our infrastructure. Replace the generated index.ts file with the example below. The example shows how to deploy containers on ECS using Fargate.

import * as awsx from "@pulumi/awsx";

// Create a repository
const repo = new awsx.ecr.Repository("my-repo", {
 forceDelete: true,
});

// Build an image from the "./app" directory
// and publish it to our ECR repository.
export const image = new awsx.ecr.Image("image", {
 repositoryUrl: repo.url,
 path: "./app",
});

// Create an ECS cluster explicitly, and give it a name tag.
const cluster = new aws.ecs.Cluster("custom", {
 tags: {
 "Name": "my-custom-ecs-cluster",
 },
});

// load balancer on port 80
const lb = new awsx.lb.ApplicationListener("nginx", { port: 80 });
const service = new awsx.ecs.FargateService("nginx", {
 cluster,
 taskDefinitionArgs: {
 containers: {
 nginx: {
 image: image.imageUri,
 memory: 512,
 portMappings: [ lb ],
 },
 },
 },
 desiredCount: 2,
});

// Publish the load balancer address so it's accessible.
export const appURL = lb.endpoint.hostname;

// Publish the repository URL, so we can push to it.
export const repoURL = repo.repository.repositoryUrl;

To deploy the application, run pulumi up. You’ll see a preview of the resources to be deployed. Accept the deployment by selecting yes, and when finished, your program lists the URLs for the application and the repository.

Outputs:
 appURL : "nginx-0296a2a-996897041.us-east-1.elb.amazonaws.com"
 image : "153052954103.dkr.ecr.us-east-1.amazonaws.com/my-repo-41f69f4:37c8add0fbd01474806c9c2319e62a857737de9e1460b09f935133860a069f4e"
 repoURL: "153052954103.dkr.ecr.us-east-1.amazonaws.com/my-repo-41f69f4"

Test to see if your application is working using either a browser or curl to see the web page.

$ curl nginx-0296a2a-996897041.us-east-1.elb.amazonaws.com
<html>
 <head><meta charset="UTF-8">
 <title>Hello, Pulumi!</title>
 </head>
 <body>
 <p>Hello, containers!</p>
 <p>Made with ❤️ with <a href="https://pulumi.com">Pulumi</a></p>
 </body>
</html>

EKS

The EKS example uses the @pulumi/eks and @pulumi/kubernetes node modules. Add the eks and kubernetes modules to the project.

$ node install “@plulumi/eks”
$ node install “@plulumi/kubernetes”

Like the ECS example, the EKS example builds the application image locally and pushes it into our custom repository. The application deployment has two replicas in the configuration. The service uses the EKS cluster, my-cluster, we declared to create a load balancer with port 80 open.

import * as awsx from "@pulumi/awsx";
import * as eks from "@pulumi/eks";
import * as k8s from "@pulumi/kubernetes";

// Create a repository, as before.
const repo = new awsx.ecr.Repository("my-repo", {
 forceDelete: true,
});

// Build an image from the "./app" directory. and publish it to our ECR repository.
export const image = new awsx.ecr.Image("image", {
 repositoryUrl: repo.url,
 path: "./app",
});

// Create an EKS cluster with the default configuration.
const cluster = new eks.Cluster("my-cluster")

// Deploy a small canary service (NGINX), to test that the cluster is working.
const appName = "my-app";
const appLabels = { appClass: appName };
const deployment = new k8s.apps.v1.Deployment(`${appName}-dep`, {
 metadata: { labels: appLabels },
 spec: {
 replicas: 2,
 selector: { matchLabels: appLabels },
 template: {
 metadata: { labels: appLabels },
 spec: {
 containers: [{
 name: appName,
 image: image.imageUri,
 ports: [{ name: "http", containerPort: 80 }]
 }],
 }
 }
 },
}, { provider: cluster.provider });
const service = new k8s.core.v1.Service(`${appName}-svc`, {
 metadata: { labels: appLabels },
 spec: {
 type: "LoadBalancer",
 ports: [{ port: 80, targetPort: "http" }],
 selector: appLabels,
 },
}, { provider: cluster.provider });

// Export the cluster's kubeconfig.
export const kubeconfig = cluster.kubeconfig;

// Publish the repository URL.
export const repoURL = repo.repository.repositoryUrl;

// Publish the URL for the load balanced service.
export const appURL = service.status.loadBalancer.ingress[0].hostname;

As with the ECS example, run pulumi up to deploy the application. To check the deployment, use either curl or a browser to see the page.

Want to know more?

Although these examples are simple, they demonstrate the basic building blocks for building, storing, and managing AWS Containers. They also show how to create ECS or EKS clusters for deploying apps. To get started with AWS and Pulumi check out the AWS Guide for core services and the Crosswalk for AWS guide for convenience APIs that simplify deploying infrastructure as code. For a deeper dive into managing containers on AWS, check out our blogs on ECS vs Fargate vs EKS: The Lowdown on Containers in AWS and How to Scale Your Amazon EKS Cluster: EC2, Managed Node Groups, and Fargate.

Introducing Pulumi Crosswalk for AWS: The Easiest Way to AWS

Luke Hoban — Mon, 10 Jun 2019 00:00:00 +0000

Some of the code in this post is out of date. See the AWS guides for an updated overview and examples.

Amazon Web Services provides an incredible platform for developers to build cloud-native applications, and is used by millions of customers of all sizes. The building block services that AWS offers enable teams to offload undifferentiated heavy-lifting to AWS. To maximally benefit from these services though, cloud engineering teams must learn how to compose all of these building blocks together to build and deliver their own applications. Today, this is still too hard. Getting from your laptop to a production-ready AWS deployment frequently takes days or weeks instead of minutes or hours. And AWS building block services frequently leave you to re-implement (and re-discover) best-practices instead of providing these as smart defaults.

Pulumi Crosswalk for AWS is a new open source library of infrastructure-as-code components that make it easier to get from zero to production on AWS, easier to adopt AWS best practices by default, and easier to evolve your AWS infrastructure as your application needs mature.

Crosswalk for AWS offers higher-level components for a wide range of the AWS platform - from serverless (Lambda, API Gateway and more) to containers (ECS, Fargate) to Kubernetes (EKS) and across networking (VPCs, SecurityGroups), monitoring (Cloudwatch Dashboards and Alarms) and load balancing (ALB and NLB). Because Crosswalk builds on the primitive building blocks of the AWS platform, you can use 100% of what AWS offers, and can seamlessly move between high-level components and low-level platform primitives, mixing-and-matching as needed.

In some of these domains, there have been successful tools that have helped developers more easily work with one specific aspect of AWS - Serverless Framework for serverless, Docker Compose for ECS, eksctl for EKS. But these tools have traditionally been limited to one specific kind of compute, limiting what users can accomplish, fragmenting development of complete cloud solutions, and leading to sharp cliffs when application needs evolve. Crosswalk for AWS offers the same kind of simpler and more productive developer interface as these tools, but applied uniformly across the entire AWS platform. Simple things are simple, and arbitrarily complex things are possible.

Whether you are starting a new SaaS project at a startup, migrating an enterprise application to cloud-native infrastructure, or delivering a new service as part of an established cloud application, Crosswalk for AWS provides the easiest way to get from zero to production, and to then evolve with your projects’ needs. You can get started with Crosswalk for AWS today!

Note: Crosswalk for AWS projects are authored using the Pulumi infrastructure-as-code tools. Pulumi allows you to define your infrastructure using code. As your infrastructure code evolves, Pulumi makes the minimal required changes to your cloud resources, ensuring that your infrastructure is updated without downtime (if possible).

From Zero to Production And Beyond with Crosswalk for AWS

Serverless (Lambda, API Gateway)

When getting started with a new cloud project, the starting point is often just serving a simple API on the internet. With just a few lines of code and tens of seconds to deploy, Crosswalk for AWS gives us our API.

import * as awsx from "@pulumi/awsx";

const api = new awsx.apigateway.API("api", {
 routes: [{
 method: "GET",
 path: "/",
 eventHandler: async (ev, ctx) => {
 return {
 statusCode: 200,
 body: JSON.stringify(ev),
 };
 }
 }],
})

export const endpoint = api.url;

By building on top of great AWS building blocks like Lambda, API Gateway, IAM and more, we avoid needing to worry about infrastructure, pay nearly zero fixed costs, and gain the ability to iterate quickly. With just a few more lines of code, we can wire up to our own domain, add authorization, provision a database, or chain together a more complex event-driven application.

Containers (ECS, Fargate)

As our needs grow, we might find we need to deploy containers - either custom applications or existing Docker images. Again, just a few lines of code gets our container running in production in AWS.

import * as awsx from "@pulumi/awsx";

// Create the cluster
const cluster = new awsx.ecs.Cluster("cluster");

// Configure a Load Balancer
const alb = new awsx.lb.ApplicationLoadBalancer("lb", {
 securityGroups: cluster.securityGroups,
});
const web = alb.createListener("web", { port: 80, external: true });

// Define the scaled-out Fargate service
const appService = new awsx.ecs.FargateService("app-svc", {
 cluster,
 taskDefinitionArgs: {
 container: {
 image: awsx.ecs.Image.fromPath("app", "./app"),
 portMappings: [ web ],
 },
 },
 desiredCount: 5,
});

export const url = web.endpoint.hostname;

Using ECS, Fargate, ECR and ELB, we get a robust production-ready container deployment - horizontally scaled out, load-balanced, and integrated with a private image repository. With just a few more lines of code, we can add autoscaling, customize our ECS cluster, or wire through advanced container configuration (volumes, environment variables, and more).

Networking (VPC)

To adopt best practices security posture, we may want to move our cluster into a custom VPC, and run our containers in private subnets. We can define and configure our VPC, and then redeploy our containers into this new VPC with ease:

import * as awsx from "@pulumi/awsx";

const vpc = new awsx.ec2.Vpc("vpc", {
 numberOfAvailabilityZones: 3,
});

const cluster = new awsx.ecs.Cluster("cluster", {
 vpc: vpc,
});

// ... same as before ...

The Vpc component builds on best-practices Virtual Private Cloud design patterns based on AWS guidance and documentation. By building in simple defaults for routing, subnet structure, NATs and multi-AZ you can get started quickly without having to re-discover these best practices. And as your needs grow, you can deeply customize and evolve your VPC structure - defining custom CIDR blocks, customizing Internet and NAT Gateways or additional additional private subnets.

Kubernetes (EKS)

As our project continues to evolve, we may want to move our container workloads into Kubernetes using EKS. We can stand up an EKS cluster with just a few lines of code, then customize our cluster and deploy Kubernetes workloads into our cluster.

import * as awsx from "@pulumi/awsx";
import * as eks from "@pulumi/eks";

// Create a VPC for our cluster.
const vpc = new awsx.ec2.Vpc("vpc");

// Create the EKS cluster itself.
const cluster = new eks.Cluster("cluster", {
 vpcId: vpc.id,
 subnetIds: vpc.privateSubnetIds,
 instanceType: "m5.large",
 desiredCapacity: 2,
 minSize: 1,
 maxSize: 2,
 deployDashboard: true,
});

// Export the cluster's kubeconfig.
export const kubeconfig = cluster.kubeconfig;

Building on EKS we get all the benefits of managed Kubernetes paired with the platform capabilities of AWS. With Crosswalk for AWS, we can get started quickly, and then evolve to take advantage of all of these platform features, like customized node groups, private networking, and even deploying Kubernetes YAML and Helm charts to our cluster.

Monitoring (CloudWatch)

Getting our infrastructure up and running in AWS is great, but we also want to be able to monitor it. We can add a few lines to any of the solutions above to define metrics, dashboards, alerting and more to monitor our AWS infrastructure.

import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";

// Create a bucket and a function to log all new object created events
const bucket = new aws.s3.Bucket("b");
const subscription = bucket.onObjectCreated("newObject", async (ev) => {
 console.log(JSON.stringify(ev));
});

// Create a CloudWatch Dashobard for our functions invocations
const dashboardName = "funcDashboard";
const dashboard = new awsx.cloudwatch.Dashboard(dashboardName, {
 widgets: [
 new awsx.cloudwatch.LineGraphMetricWidget({
 title: "Lambda invocations",
 width: 14,
 metrics: awsx.lambda.metrics.invocations({
 function: subscription.func, statistic: "Sum", period: 60, }), }), ], });

// Export the URL of the dashboard in the AWS console
export const dashboardUrl =
 `https://${aws.config.region}.console.aws.amazon.com/cloudwatch/home?` +
 `region=${aws.config.region}#dashboards:name=${dashboardName}`;

With CloudWatch being deeply integrated into all AWS services, we can easily build up robust logging, alerting and dashboarding solutions across our AWS infrastructure. With just a few more lines of code we can get logs for our functions and containers, create alarms when we cross critical thresholds, and create rich dashboard layouts all of which will version along with our infrastructure!

Early Feedback

We’ve been working on pieces of Crosswalk for AWS since before we launched Pulumi, and we’re excited to be launching Crosswalk for AWS today to bring these together for all AWS users. Over the last few months, users have built on Crosswalk for AWS for all of the scenarios above and more.

Tableau has used Crosswalk for AWS to accelerate their Kubernetes migration on AWS, building on the networking and EKS capabilities of Crosswalk for AWS.

“We’ve been happily using Pulumi’s EKS support for more than three months now. Our team was looking for an end-to-end solution to tame the complexity of Kubernetes on AWS and ensure we adhere to AWS best practices. Pulumi’s Crosswalk effort has equipped our team to scale far better, our delivery is now automated, and we can now deliver new application and infrastructure features with much faster turn-around, sometimes in just a few hours. Pulumi is a key contributor to our team’s improvement in productivity.” - Pankaj Dhingra, Sr Director, Cloud Engineering, Tableau

Compute Software has used Crosswalk for AWS to go from zero to production, and then to evolve their infrastructure in AWS, all without needing a dedicated cloud infrastructure engineering team.

“We use Crosswalk for AWS with Fargate. It’s been instrumental in getting us into production. Deploying to the cloud is necessarily complex. Crosswalk streamlines the process by exposing the controls you care about, always allowing you to dive deeper when needed.” - Kenny Williams, Co-Founder, Compute Software

We’re excited by the success these teams and many more have already had with Crosswalk for AWS, but even more excited about the potential to truly democratize access to the AWS platform and unlock highly productive cloud development for all kinds of engineering organizations.

What’s Next?

We’re still just scratching the surface of what’s possible with Crosswalk. We’re continuing to evolve the open source projects to simplify additional features of the AWS platform. And we’re also working to bring the same approach to other cloud platforms, including Azure, GCP and Kubernetes.

We believe that making it easier for cloud engineers to develop and deploy natively in the cloud will unlock creativity and empower developers to take advantage of more of the rich capabilities of AWS. We can’t wait to see what you build with Crosswalk!

Crosswalk for AWS is free and open source, and you can get started today.

For more on Crosswalk for AWS see:

Secure APIs with Auth0 Lambda Authorizers Using Pulumi & AWS

Erin Krengel — Wed, 24 Apr 2019 00:00:00 +0000

Creating serverless applications just got even easier! You can now protect your application APIs in just three easy steps. We’ve already posted about how easy it is to create serverless apps in Pulumi. Now, we’re helping you simplify protecting those apps with API Gateway and Lambda authorizers.

With Pulumi’s new AWSX package, you can quickly define a Lambda and an AWS Lambda authorizer to protect it. We’re once again harnessing the power of Lambdas as Lambdas to allow developers to focus on writing code.

Today, we will walkthrough creating a simple serverless app using AWS and Pulumi. We will simplify implementing the OAuth protocol by using Auth0 and AWS Lambda authorizers to authorize users. Auth0 provides a universal authentication and authorization platform for applications. It has become an extremely popular platform for user management because Auth0 makes OAuth easy.

Concepts

Before we get started, let’s go over some concepts we’ll be using.

API Gateway

An API Gateway takes clients requests and then re-routes them to the appropriate backend service. Typically, a gateway serves as the “front door” for an API and is used for traffic management as well as flexible security controls.

Using AWS API Gateway, you can build a fully managed collection of resources which integrate with AWS Lambda and various other AWS resources. To control who can call your API, you can use IAM permissions, an Amazon Cognito User Pool or set up custom logic using a Lambda authorizer.

OAuth

OAuth is an industry-standard for token-based authorization. Using OAuth, users can grant third-party applications or websites access to their information on another website without disclosing their password. Google and Facebook are two very common examples of sites that permit users to share their personal information with websites or applications using OAuth.

Lambda Authorizers

An AWS Lambda authorizer is a Lambda that provides access control to an API. It contains custom logic for authorizing requests to an endpoint. If a Lambda authorizer is configured, API Gateway routes a client’s call to the Lambda first. The Lambda authorizer runs its custom logic and returns a Policy and principal ID, which are used by API Gateway to determine if the call to the backend is allowed.

Step-by-Step Guide To Creating a Lambda Authorizer

Using a Lambda authorizer, we can implement the authorization flow using Auth0 to handle our Access Tokens.

API Gateway can be used to protect many different resources, and in the following example we will front an AWS Lambda. Users authenticate with Auth0 and then use the token they’ve obtained to make a request to our Lambda via our API Gateway, which authorizes the request by validating the token with Auth0. Some of the code examples have been stubbed or shortened for this blog post. The full code example can be found in our Pulumi examples repo.

Below is a diagram of the architecture we will set up. In Step 1, we will create our API Gateway and Hello World Lambda. In Step 2, we will set up the logic for our Lambda authorizer. And finally in Step 3, we will bring it all together by telling API Gateway to use our authorizer.

1 - Define Your Routes

Using Pulumi’s AWSX package, you can easily define routes to serverless apps as well as routes to static resources. We will create a very simple Hello world! backend Lambda, but in practice you can have this do much more interesting things.

import * as awsx from "@pulumi/awsx";

// Create our API and reference the Lambda authorizer
const api = new awsx.apigateway.API("myapi", {
    routes: [{
        path: "/hello",
        method: "GET",
        eventHandler: async () => {
            return {
                statusCode: 200,
                body: "<h1>Hello world!</h1>",
            };
        },
    }],
});

2 - Write Your Custom Auth Logic

Just as Pulumi allows you to define your Lambda inline, you can define your custom authorization code. We will define a small wrapper for our custom logic and write the rest of the code as a function called authenticate. Pulumi will package up all our runtime code and create an AWS Lambda for us.

import * as awsx from "@pulumi/awsx";
import * as pulumi from '@pulumi/pulumi';

import * as jwksClient from 'jwks-rsa';
import * as jwt from 'jsonwebtoken';
import * as util from 'util';

const config = new pulumi.Config();
const jwksUri = config.require("jwksUri");
const audience = config.require("audience");
const issuer = config.require("issuer");

const authorizerLambda = async (event: awsx.apigateway.AuthorizerEvent) => {
    try {
        return await authenticate(event);
    }
    catch (err) {
        console.log(err);
        // Tells API Gateway to return a 401 Unauthorized response
        throw new Error("Unauthorized");
    }
}

/**
 * Below is all code that gets added to the Authorizer Lambda. The code was copied and
 * converted to TypeScript from
 * [Auth0's GitHub Example](https://github.com/auth0-samples/jwt-rsa-aws-custom-authorizer)
 */

function getToken(event: awsx.apigateway.AuthorizerEvent): string {
    // Stubbed function to extract and return the Bearer Token from the Lambda event parameter
}

// Check if the Token is valid with Auth0
async function authenticate(event: awsx.apigateway.AuthorizerEvent): Promise<awsx.apigateway.AuthorizerResponse> {
    const token = getToken(event);

    const decoded = jwt.decode(token, { complete: true });
    if (!decoded || typeof decoded === "string" || !decoded.header || !decoded.header.kid) {
        throw new Error('invalid token');
    }

    const client = jwksClient({
        cache: true,
        rateLimit: true,
        jwksRequestsPerMinute: 10, // Default value
        jwksUri: jwksUri
    });

    const key = await util.promisify(client.getSigningKey)(decoded.header.kid);
    const signingKey = key.publicKey || key.rsaPublicKey;
    if (!signingKey) {
        throw new Error('could not get signing key');
    }

    const verifiedJWT = await jwt.verify(token, signingKey, { audience, issuer });
    if (!verifiedJWT || typeof verifiedJWT === "string" || !isVerifiedJWT(verifiedJWT)) {
        throw new Error('could not verify JWT');
    }
    return awsx.apigateway.authorizerResponse(verifiedJWT.sub, 'Allow', event.methodArn);
}

3 - All Together Now

Now let’s tie it all together! We will modify the route we created in Step 1 to include our authorizer.

import * as awsx from "@pulumi/awsx";

// Create our API and reference the Lambda authorizer
const api = new awsx.apigateway.API("myapi", {
    routes: [{
        path: "/hello",
        ...
        authorizers: awsx.apigateway.getTokenLambdaAuthorizer({
            authorizerName: "jwt-rsa-custom-authorizer",
            header: "Authorization",
            handler: authorizerLambda,
            identityValidationExpression: "^Bearer [-0-9a-zA-Z._]*$",
            authorizerResultTtlInSeconds: 3600,
        }),
    }],
});

The Finished Product

When we run pulumi up, Pulumi will create our resources and provide us with our protected endpoint.

$ pulumi up
...
     Type Name Status Info
 + pulumi:pulumi:Stack lambda-authorizer-dev created 1 message
 + ├─ aws:apigateway:x:API myapi created
 + │ ├─ aws:iam:Role myapifc45ff03 created
 + │ ├─ aws:iam:RolePolicyAttachment myapifc45ff03-32be53a2 created
 + │ ├─ aws:lambda:Function myapifc45ff03 created
 + │ ├─ aws:apigateway:RestApi myapi created
 + │ ├─ aws:apigateway:Deployment myapi created
 + │ ├─ aws:lambda:Permission myapi-62a1b306 created
 + │ └─ aws:apigateway:Stage myapi created
 + ├─ aws:iam:Role jwt-rsa-custom-authorizer-authorizer-role created
 + ├─ aws:iam:Role jwt-rsa-custom-authorizer created
 + ├─ aws:iam:RolePolicyAttachment jwt-rsa-custom-authorizer-32be53a2 created
 + ├─ aws:lambda:Function jwt-rsa-custom-authorizer created
 + └─ aws:iam:RolePolicy jwt-rsa-custom-authorizer-invocation-policy created
Outputs:
    url: "https://XXXXX.execute-api.us-west-2.amazonaws.com/stage/"
Resources:
    + 14 created
Duration: 28s

You’ll notice there’s a ton of resources we didn’t explicitly define. AWSX provisions the appropriate Lambdas, roles, and more for you. If required, all of these resources can be overridden in your Pulumi program by explicitly defining them.

Let’s give try out our new API.

We can now curl our endpoint without a token and should get a 401 Unauthorized response.

$ curl $(pulumi stack output url)hello
{"message":"Unauthorized"}

We can curl our endpoint with an invalid token and should once again get a 401 Unauthorized response.

$ curl $(pulumi stack output url)hello -H "Authorization: Bearer invalid"
{"message":"Unauthorized"}

Finally, we expect a 200 response when we obtain a token from Auth0 and use it to call our API. We can get a token by visiting the API Details page for our API and clicking the Test tab. This provides us with a valid token for testing purposes only. The way tokens are acquired by users or client apps will depend on the type of application (i.e. native mobile app, single-page app, web app, etc.) being developed. For more information on authenticating users, visit Auth0’s documentation.

Using the test access token, the API will return a 200 response: Hello world!

$ curl $(pulumi stack output url)hello -H "Authorization: Bearer <VALID_TOKEN>"
<h1>Hello world!</h1>

For the complete code example, visit our examples repo.

Keep It Going

While we used OAuth and Auth0 in this blog post, AWS Lambda authorizers are an extremely flexible tool that allow you to implement whatever custom authorization logic you require. AWS also supports Cognito Authorizers, which use AWS Cognito User Pools, and we will share even more examples soon.

There are a vast amount of resources for deploying your code into the cloud, and we want to make it easier for developers to make the most of those resources. Pulumi’s AWSX package provides reusable abstractions to enable developers to get their code production ready faster.

Get going with these resources:

Pulumi Blog: Lambda

AWS Lambda Response Streaming with Pulumi

Introducing Response Streaming

Authoring a Lambda Function for Response Streaming

Streaming Performance Considerations

Packaging and deploying your Lambda Function for streaming with Pulumi

Reading a Response Stream

Wrapping up

AWS Lambda SnapStart with Pulumi

Reducing Function Latency with AWS Lambda SnapStart

Configuring Lambda Functions to work with SnapStart in Pulumi

Deployment and Performance Testing

Comparing performance before and after SnapStart

Conclusion

Secure AWS Lambda with IAM ABAC Policies

What are Attributes?

Using ABAC for AWS Lambda Functions

Try It Out!

Test-Driven Infrastructure Development with Pulumi and Jest

Sketching it out

Create a new TypeScript project

Install and configure Jest

Start with some failing tests

Make the tests pass

Tidy up the specs

Turn outputs into promises

Finish the app

Add the remaining tests

Produce the audio message

Deploy the service

Wrapping up

API Gateway to EventBridge with Pulumi

Hello, EventBridge

Create a new project and stack

Create the API gateway and stage

Add an event bus and event rule

Add a Lambda function handler

All Together, Now

What’s next?

AWS Lambda Functions Powered by AWS Graviton2 Processors

Deploying a Lambda Function with Graviton2 Support

Creating a Lambda Layer with Graviton2 Support

Summary

Deploy AWS Lambda with Elastic File System using Pulumi

EFS + Lambda Concepts

Interact with EFS from an HTTP API

Sharing a File System between Fargate and Lambda

Conclusion

Controlling AWS Costs with Pulumi and AWS Lambda

Cloud Cost Lambda

Deploying To Multiple Regions

In Conclusion

Scheduling Serverless

Taking out the trash

Want to know more?

Managing AWS Containers with Pulumi

Store your images

Choose your scheduler

Choose where/how to run, Fargate or EC2

Fargate

EC2 with Kubernetes

Build!

ECS

EKS

Want to know more?

Introducing Pulumi Crosswalk for AWS: The Easiest Way to AWS

From Zero to Production And Beyond with Crosswalk for AWS

Serverless (Lambda, API Gateway)

Containers (ECS, Fargate)

Networking (VPC)

Kubernetes (EKS)

Monitoring (CloudWatch)

Early Feedback

What’s Next?

Secure APIs with Auth0 Lambda Authorizers Using Pulumi & AWS

Concepts

API Gateway

OAuth

Lambda Authorizers

Step-by-Step Guide To Creating a Lambda Authorizer