Pulumi Blog: API gateway

API Gateway to EventBridge with Pulumi

Christian Nunciato — Wed, 12 Jan 2022 13:00:00 -0700

If you’re familiar with Amazon API Gateway, you know it’s all about making it easier to provision and manage a web API. Maybe you’ve used it, as I have, with Crosswalk, our AWS extension library, to stand up a REST API and handle requests with AWS Lambda functions:

import * as awsx from "@pulumi/awsx";

// Create a new API Gateway instance.
const api = new awsx.apigateway.API("my-api", {
 routes: [
 {
 // Define an HTTP endpoint.
 method: "GET",
 path: "/things",

 // Handle requests with an AWS Lambda function.
 eventHandler: async (apiGatewayEvent) => {
 return {
 statusCode: 200,
 body: JSON.stringify(["thingOne", "thingTwo"]),
 };
 },
 },
 ],
});

// Export the API's public URL. 🎉
export const apiUrl = api.url;

I love this abstraction, and I use it all the time; it’s an incredibly convenient way to solve a really common problem. But if you examine the code, you’ll notice that it’s making a fairly strong assumption about how you’ll be handling HTTP requests—namely, that you’ll do so with a single Lambda function, and that that function will always return a JavaScript object of a particular shape.

Indeed, this arrangement is the API contract of a Lambda proxy integration—API Gateway integrations come in many shapes and sizes; Lambda integrations just happen to be one of the more popular—and much of the time, an approach like this one works out just fine. But depending on the needs of the application, it might not always be the best fit.

Imagine you were building a print-on-demand service, for example, and you wanted to expose an API to let your customers upload documents and have them converted into orders. On AWS, you might reach for API Gateway, as above, to define an HTTP method and route (POST /uploads, say), wire it up to an AWS Lambda, and have the Lambda parse the upload, write the order to a database, and return a response. Visually, such a design might look something like this:

It’d definitely work, and again, it’s quite common. But at some point, you might find this tight coupling between API Gateway and Lambda too limiting. Say you wanted to be notified whenever a new order was received, with a Slack message, maybe, in one of your team’s shared workspace channels. Under the current design, you’d probably add a few lines of code to the Lambda function to import an HTTP library and make a call to the Slack API to post the message:

That’d work, too—but it’d be less than ideal for a number of reasons. For one, that Lambda would now have two jobs: taking orders and sending Slack notifications. That might be fine for today (it’s only a few lines of code, after all), but over time, those two jobs could easily become three, and then four, and soon enough, that poor Lambda could wind up becoming a lot more difficult to maintain. And given the importance of its main job—capturing orders—it’s not something you’d want to risk failing at runtime because of a random Slack outage or other transient internet mishap. Moreover, with every bit of extra work you tack on to that function, you take a tiny step closer to hitting API Gateway’s 30-second limit.

What you really need, then, is to be able to take multiple independent, possibly long-running actions based on a single API Gateway request. And one way to do that is with Amazon EventBridge.

Hello, EventBridge

Amazon EventBridge (formerly CloudWatch Events) is as a serverless event bus whose job is to receive structured event data—from your own applications, from other AWS Services—and use that data to notify other applications or services using event-handling rules that you specify. With EventBridge, you can build loosely coupled, event-driven systems that take advantage of AWS’s rich support for serverless architectures and that scale gracefully as the needs of those systems change over time.

For this particular application, EventBridge would let you address the multiple-handler problem in a more scalable and easily maintainable way. Rather than have API Gateway invoke Lambda directly, leaving one Lambda responsible for handling multiple tasks, you could have API Gateway publish to EventBridge instead, and let EventBridge invoke as many Lambdas (or other AWS services) as you like, all serverlessly and in parallel—and of course, all easily managed with Pulumi.

Let’s see how. Here’s a revised architecture diagram showing how you might approach building an application like this one with EventBridge positioned between API Gateway and Lambda:

Now let’s have a look at what it’d be like to build it with Pulumi. We won’t build everything in this diagram—things like writing to the database or messaging Slack are left for you to explore—but we will build enough to give you clear picture and a working example of how to connect all of these parts into a working application. Specifically:

an API Gateway instance to act as a container for your public API, along with a stage and a route to handle inbound HTTP requests;
an EventBridge integration (comprised of an event bus and event rule) to handle notifications from API Gateway; and finally,
one or more Lambda functions to be invoked in response to event-rule matches.

Let’s get started.

Create a new project and stack

As always, it’s good practice to begin with a new project and stack:

$ pulumi new aws-typescript

$ pulumi new aws-python

Make sure you configure your AWS credentials as well, and when prompted, choose whatever stack name and AWS region work best for you.

In this post, we’ll be using API Gateway V2 as it’s newer and has a few features (like auto-deploy) that make it easier to work with in this context. The same ideas apply to both V1 and V2, though, so while V2 still lacks full feature parity with V1, the underlying resources we’ll be using—methods, routes, integrations—are the same. You’ll find links to examples of both at the end of this post.

Create the API gateway and stage

Start by replacing the contents of index.js index.ts __main__.py main.go Program.cs Program.fs Program.vb App.java Pulumi.yaml main.tf with the following code to declare a new API Gateway API:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Create an HTTP API.
const api = new aws.apigatewayv2.Api("api", {
 protocolType: "HTTP",
});

import json
import pulumi
import pulumi_aws as aws

# Create an HTTP API.
api = aws.apigatewayv2.Api("example",
 protocol_type="HTTP"
)

Next, add a stage (feel free to name it whatever you like; I usually use the current stack name for convenience), and set it to deploy automatically whenever a change is made to the API:

// ...

// Create a stage and set it to deploy automatically.
const stage = new aws.apigatewayv2.Stage("stage", {
 apiId: api.id,
 name: pulumi.getStack(),
 autoDeploy: true,
});

# Create a stage and set it to deploy automatically.
stage = aws.apigatewayv2.Stage("stage",
 api_id=api.id,
 name=pulumi.get_stack(),
 auto_deploy=True
);

The next thing to do is register a route on the gateway to give your users a publicly accessible endpoint to upload to. But in order to do that, you’ll need to tell API Gateway what to do when an upload happens. Since the plan of record is to notify EventBridge (using API Gateway’s built-in support for it), you’ll need to declare a few EventBridge things first.

Add an event bus and event rule

Every AWS account gets an event bus by default (one that’s aptly named default), but given how easy it is to create one, we might as well do that for this application. You’ll also need to define an event rule—a resource that watches a specific event bus for events that conform to a particular pattern or shape, and then routes those events to one or more targets (e.g., Lambda functions). Add the following lines to your program for both:

// ...

// Create an event bus.
const bus = new aws.cloudwatch.EventBus("bus");

// Create an event rule to watch for events.
const rule = new aws.cloudwatch.EventRule("rule", {
 eventBusName: bus.name,

 // Specify the event pattern to watch for.
 eventPattern: JSON.stringify({
 source: ["my-event-source"],
 }),
});

# ...

# Create an event bus.
bus = aws.cloudwatch.EventBus("bus")

# Create an event rule to watch for events.
rule = aws.cloudwatch.EventRule("rule",
 event_bus_name=bus.name,
 event_pattern=json.dumps({"source": ["my-event-source"]})
)

The most noteworthy property of the EventRule resource is probably the eventPattern. EventBridge events all conform to a certain schema, and in this case, we’re expressing that this particular event rule should take action on any event that originates from my-event-source. (The source property is just a free-form string that by convention identifies the application or service responsible for the event.)

With the event bus and event rule in place, you’re ready to define the integration itself—the resource responsible for connecting the gateway route (which we’ll get back to shortly) to your newly created event bus. As I mentioned earlier, there are several types of API Gateway integration to choose from, each one suited to a particular purpose. For this example, the AWS_PROXY type is good fit, as it’s simple and requires very little code; it doesn’t give you quite as much control over the API Gateway response as you might like—as a proxy, it just returns to the caller whatever is returned by the backend, in this case EventBridge—but it’s more than enough for the task at hand.

Add the following lines for the integration and route. The comments should explain what each block is doing:

// ...

// Define a policy granting API Gateway permission to publish to EventBridge.
const apiGatewayRole = new aws.iam.Role("api-gateway-role", {
 assumeRolePolicy: {
 Version: "2012-10-17",
 Statement: [
 {
 Action: "sts:AssumeRole",
 Effect: "Allow",
 Principal: {
 Service: "apigateway.amazonaws.com",
 },
 },
 ],
 },
 managedPolicyArns: ["arn:aws:iam::aws:policy/AmazonEventBridgeFullAccess"],
});

// Create an API Gateway integration to forward requests to EventBridge.
const integration = new aws.apigatewayv2.Integration("integration", {
 apiId: api.id,

 // The integration type and subtype.
 integrationType: "AWS_PROXY",
 integrationSubtype: "EventBridge-PutEvents",
 credentialsArn: apiGatewayRole.arn,

 // The body of the request to be sent to EventBridge. Note the
 // event source matches pattern defined on the EventRule, and the
 // Detail expression, which just forwards the body of the original
 // API Gateway request (i.e., the uploaded document).
 requestParameters: {
 EventBusName: bus.name,
 Source: "my-event-source",
 DetailType: "my-detail-type",
 Detail: "$request.body",
 },
});

// Finally, define the route.
const route = new aws.apigatewayv2.Route("route", {
 apiId: api.id,
 routeKey: "POST /uploads",
 target: pulumi.interpolate`integrations/${integration.id}`,
});

# ...

# Define a policy granting API Gateway permission to publish to EventBridge.
api_gateway_role = aws.iam.Role("api-gateway-role",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": "apigateway.amazonaws.com",
 },
 },
 ],
 }),
 managed_policy_arns=[
 "arn:aws:iam::aws:policy/AmazonEventBridgeFullAccess",
 ],
);

# Create an API Gateway integration to forward requests to EventBridge.
integration = aws.apigatewayv2.Integration("integration",
 api_id=api.id,

 # The integration type and subtype.
 integration_type="AWS_PROXY",
 integration_subtype="EventBridge-PutEvents",
 credentials_arn=api_gateway_role.arn,

 # The body of the request to be sent to EventBridge. Note the
 # event source matches pattern defined on the EventRule, and the
 # Detail expression, which just forwards the body of the original
 # API Gateway request (i.e., the uploaded document).
 request_parameters={
 "EventBusName": bus.name.apply(lambda name: name),
 "Source": "my-event-source",
 "DetailType": "my-detail-type",
 "Detail": "$request.body",
 },
);

# Finally, define the route.
route = aws.apigatewayv2.Route("route",
 api_id=api.id,
 route_key="POST /uploads",
 target=integration.id.apply(lambda id: f"integrations/{id}"),
);

With that, you’re ready to configure the Lambda.

Add a Lambda function handler

The hard part’s done: you’ve declared an API and route, mapped that route to an integration, configured the integration to publish events to an event bus, and defined an event rule to respond to those events. All that’s left now is to tell the rule how to respond.

So to finish things off, you need:

a Lambda function to handle uploads,
an EventBridge target to bind your event rule to that function, and
an IAM policy granting EventBridge permission to invoke the function.

Add the following lines to your program to complete it:

First, make a new folder and file alongside your Pulumi program, at ./api/handlers.py, and add a Python function to act as your function handler:

// ...

// Create a Lambda function handler with permission to log to CloudWatch.
const lambda = new aws.lambda.CallbackFunction("lambda", {
 policies: [aws.iam.ManagedPolicies.CloudWatchLogsFullAccess],
 callback: async (event: any) => {
 // For now, just log the event, including the uploaded document.
 // That'll be enough to verify everything's working.
 console.log({ source: event.source, detail: event.detail });
 },
});

// Create an EventBridge target associating the event rule with the function.
const lambdaTarget = new aws.cloudwatch.EventTarget("lambda-target", {
 arn: lambda.arn,
 rule: rule.name,
 eventBusName: bus.name,
});

// Give EventBridge permission to invoke the function.
const lambdaPermission = new aws.lambda.Permission("lambda-permission", {
 action: "lambda:InvokeFunction",
 principal: "events.amazonaws.com",
 function: lambda.arn,
 sourceArn: rule.arn,
});

// Export the API Gateway URL to give us something to POST to.
export const url = pulumi.interpolate`${api.apiEndpoint}/${stage.name}`;

# In ./api/handlers.py:
import json

def capture_order(event, context):
 # For now, just log the event, including the uploaded document.
 # That'll be enough to verify everything's working.
 print(json.dumps({ "source": event["source"], "detail": event["detail"] }))

Now, back in index.js index.ts main.py main.go Program.cs Program.fs Program.vb App.java Pulumi.yaml main.tf , add the following lines to your program to complete it:

# ...

# Define a role and policy allowing Lambda functions to log to CloudWatch.
lambda_role = aws.iam.Role("lambda-role",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Principal": {
 "Service": "lambda.amazonaws.com"
 },
 "Effect": "Allow"
 }
 ]
 })
)
lambda_role_policy = aws.iam.RolePolicy("lambda-role-policy",
 role=lambda_role.id,
 policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Action": [
 "logs:CreateLogGroup",
 "logs:CreateLogStream",
 "logs:PutLogEvents"
 ],
 "Resource": "arn:aws:logs:*:*:*"
 }]
 })
)

# Create a Lambda function handler.
lambda_function = aws.lambda_.Function("lambda",
 role=lambda_role.arn,
 runtime="python3.7",
 handler="handlers.capture_order",
 code=pulumi.AssetArchive({
 ".": pulumi.FileArchive('./api')
 })
)

# Create an EventBridge target associating the event rule with the function.
lambda_target = aws.cloudwatch.EventTarget("lambda-target",
 arn=lambda_function.arn,
 rule=rule.name,
 event_bus_name=bus.name,
);

# Give EventBridge permission to invoke the function.
lambda_permission = aws.lambda_.Permission("lambda-permission",
 action="lambda:InvokeFunction",
 principal="events.amazonaws.com",
 function=lambda_function.arn,
 source_arn=rule.arn,
);

# Export the API Gateway URL to give us something to POST to.
pulumi.export("url", pulumi.Output.concat(api.api_endpoint, "/", stage.name))

All Together, Now

Now that the program’s complete, you can run Pulumi to bring it to life:

$ pulumi up
...

Updating (dev)
...

 Type Name Status
 + pulumi:pulumi:Stack eventbridge-v2-dev created
 + ├─ aws:apigatewayv2:Api api created
 + ├─ aws:apigatewayv2:Stage stage created
 + ├─ aws:cloudwatch:EventBus bus created
 ...

Outputs:
 apiURL: "https://geqfietbcl.execute-api.us-west-2.amazonaws.com/dev"

Resources:
 + 15 created

Duration: 31s

When the update finishes, you’ll have a fully functioning API Gateway-EventBridge integration that you can verify with curl:

$ curl --data '{"some-key": "some-value"}' --header "Content-Type: application/json" \
 "$(pulumi stack output url)/uploads"

{"Entries":[{"EventId":"cdc44763-6976-286c-9378-7cce674dff81"}],"FailedEntryCount":0}

Note the response, which comes directly from EventBridge (courtesy of the AWS_PROXY integration), confirming the event was received and written to the event bus.

The final step is to confirm that the request made it all the way to Lambda, which you can easily do by tailing its output with pulumi logs:

$ pulumi logs --follow

Collecting logs for stack dev since 2022-01-06T16:18:48.000-08:00.
...

{
 source: 'my-event-source',
 detail: { 'some-key': 'some-value' }
}

When you’re happy, be sure to tidy up with a pulumi destroy.

What’s next?

There’s a lot more you can do with integrations like this that we didn’t cover: add more Lambda function handlers, have EventBridge target other AWS services (Step Functions might be a good one to try next), validate HTTP request bodies (with API Gateway models, to keep bad data from ever reaching EventBridge), and more. Hopefully this gives you sense of what’s possible, though. And as promised, you’ll find examples that use both versions of API Gateway in our examples repository on GitHub:

API Gateway V2 to EventBridge in TypeScript
API Gateway V2 to EventBridge in Python
API Gateway V1 to EventBridge in TypeScript, with request validation and custom HTTP response mapping

Introducing Pulumi Crosswalk for AWS: The Easiest Way to AWS

Luke Hoban — Mon, 10 Jun 2019 00:00:00 +0000

Some of the code in this post is out of date. See the AWS guides for an updated overview and examples.

Amazon Web Services provides an incredible platform for developers to build cloud-native applications, and is used by millions of customers of all sizes. The building block services that AWS offers enable teams to offload undifferentiated heavy-lifting to AWS. To maximally benefit from these services though, cloud engineering teams must learn how to compose all of these building blocks together to build and deliver their own applications. Today, this is still too hard. Getting from your laptop to a production-ready AWS deployment frequently takes days or weeks instead of minutes or hours. And AWS building block services frequently leave you to re-implement (and re-discover) best-practices instead of providing these as smart defaults.

Pulumi Crosswalk for AWS is a new open source library of infrastructure-as-code components that make it easier to get from zero to production on AWS, easier to adopt AWS best practices by default, and easier to evolve your AWS infrastructure as your application needs mature.

Crosswalk for AWS offers higher-level components for a wide range of the AWS platform - from serverless (Lambda, API Gateway and more) to containers (ECS, Fargate) to Kubernetes (EKS) and across networking (VPCs, SecurityGroups), monitoring (Cloudwatch Dashboards and Alarms) and load balancing (ALB and NLB). Because Crosswalk builds on the primitive building blocks of the AWS platform, you can use 100% of what AWS offers, and can seamlessly move between high-level components and low-level platform primitives, mixing-and-matching as needed.

In some of these domains, there have been successful tools that have helped developers more easily work with one specific aspect of AWS - Serverless Framework for serverless, Docker Compose for ECS, eksctl for EKS. But these tools have traditionally been limited to one specific kind of compute, limiting what users can accomplish, fragmenting development of complete cloud solutions, and leading to sharp cliffs when application needs evolve. Crosswalk for AWS offers the same kind of simpler and more productive developer interface as these tools, but applied uniformly across the entire AWS platform. Simple things are simple, and arbitrarily complex things are possible.

Whether you are starting a new SaaS project at a startup, migrating an enterprise application to cloud-native infrastructure, or delivering a new service as part of an established cloud application, Crosswalk for AWS provides the easiest way to get from zero to production, and to then evolve with your projects’ needs. You can get started with Crosswalk for AWS today!

Note: Crosswalk for AWS projects are authored using the Pulumi infrastructure-as-code tools. Pulumi allows you to define your infrastructure using code. As your infrastructure code evolves, Pulumi makes the minimal required changes to your cloud resources, ensuring that your infrastructure is updated without downtime (if possible).

From Zero to Production And Beyond with Crosswalk for AWS

Serverless (Lambda, API Gateway)

When getting started with a new cloud project, the starting point is often just serving a simple API on the internet. With just a few lines of code and tens of seconds to deploy, Crosswalk for AWS gives us our API.

import * as awsx from "@pulumi/awsx";

const api = new awsx.apigateway.API("api", {
 routes: [{
 method: "GET",
 path: "/",
 eventHandler: async (ev, ctx) => {
 return {
 statusCode: 200,
 body: JSON.stringify(ev),
 };
 }
 }],
})

export const endpoint = api.url;

By building on top of great AWS building blocks like Lambda, API Gateway, IAM and more, we avoid needing to worry about infrastructure, pay nearly zero fixed costs, and gain the ability to iterate quickly. With just a few more lines of code, we can wire up to our own domain, add authorization, provision a database, or chain together a more complex event-driven application.

Containers (ECS, Fargate)

As our needs grow, we might find we need to deploy containers - either custom applications or existing Docker images. Again, just a few lines of code gets our container running in production in AWS.

import * as awsx from "@pulumi/awsx";

// Create the cluster
const cluster = new awsx.ecs.Cluster("cluster");

// Configure a Load Balancer
const alb = new awsx.lb.ApplicationLoadBalancer("lb", {
 securityGroups: cluster.securityGroups,
});
const web = alb.createListener("web", { port: 80, external: true });

// Define the scaled-out Fargate service
const appService = new awsx.ecs.FargateService("app-svc", {
 cluster,
 taskDefinitionArgs: {
 container: {
 image: awsx.ecs.Image.fromPath("app", "./app"),
 portMappings: [ web ],
 },
 },
 desiredCount: 5,
});

export const url = web.endpoint.hostname;

Using ECS, Fargate, ECR and ELB, we get a robust production-ready container deployment - horizontally scaled out, load-balanced, and integrated with a private image repository. With just a few more lines of code, we can add autoscaling, customize our ECS cluster, or wire through advanced container configuration (volumes, environment variables, and more).

Networking (VPC)

To adopt best practices security posture, we may want to move our cluster into a custom VPC, and run our containers in private subnets. We can define and configure our VPC, and then redeploy our containers into this new VPC with ease:

import * as awsx from "@pulumi/awsx";

const vpc = new awsx.ec2.Vpc("vpc", {
 numberOfAvailabilityZones: 3,
});

const cluster = new awsx.ecs.Cluster("cluster", {
 vpc: vpc,
});

// ... same as before ...

The Vpc component builds on best-practices Virtual Private Cloud design patterns based on AWS guidance and documentation. By building in simple defaults for routing, subnet structure, NATs and multi-AZ you can get started quickly without having to re-discover these best practices. And as your needs grow, you can deeply customize and evolve your VPC structure - defining custom CIDR blocks, customizing Internet and NAT Gateways or additional additional private subnets.

Kubernetes (EKS)

As our project continues to evolve, we may want to move our container workloads into Kubernetes using EKS. We can stand up an EKS cluster with just a few lines of code, then customize our cluster and deploy Kubernetes workloads into our cluster.

import * as awsx from "@pulumi/awsx";
import * as eks from "@pulumi/eks";

// Create a VPC for our cluster.
const vpc = new awsx.ec2.Vpc("vpc");

// Create the EKS cluster itself.
const cluster = new eks.Cluster("cluster", {
 vpcId: vpc.id,
 subnetIds: vpc.privateSubnetIds,
 instanceType: "m5.large",
 desiredCapacity: 2,
 minSize: 1,
 maxSize: 2,
 deployDashboard: true,
});

// Export the cluster's kubeconfig.
export const kubeconfig = cluster.kubeconfig;

Building on EKS we get all the benefits of managed Kubernetes paired with the platform capabilities of AWS. With Crosswalk for AWS, we can get started quickly, and then evolve to take advantage of all of these platform features, like customized node groups, private networking, and even deploying Kubernetes YAML and Helm charts to our cluster.

Monitoring (CloudWatch)

Getting our infrastructure up and running in AWS is great, but we also want to be able to monitor it. We can add a few lines to any of the solutions above to define metrics, dashboards, alerting and more to monitor our AWS infrastructure.

import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";

// Create a bucket and a function to log all new object created events
const bucket = new aws.s3.Bucket("b");
const subscription = bucket.onObjectCreated("newObject", async (ev) => {
 console.log(JSON.stringify(ev));
});

// Create a CloudWatch Dashobard for our functions invocations
const dashboardName = "funcDashboard";
const dashboard = new awsx.cloudwatch.Dashboard(dashboardName, {
 widgets: [
 new awsx.cloudwatch.LineGraphMetricWidget({
 title: "Lambda invocations",
 width: 14,
 metrics: awsx.lambda.metrics.invocations({
 function: subscription.func, statistic: "Sum", period: 60, }), }), ], });

// Export the URL of the dashboard in the AWS console
export const dashboardUrl =
 `https://${aws.config.region}.console.aws.amazon.com/cloudwatch/home?` +
 `region=${aws.config.region}#dashboards:name=${dashboardName}`;

With CloudWatch being deeply integrated into all AWS services, we can easily build up robust logging, alerting and dashboarding solutions across our AWS infrastructure. With just a few more lines of code we can get logs for our functions and containers, create alarms when we cross critical thresholds, and create rich dashboard layouts all of which will version along with our infrastructure!

Early Feedback

We’ve been working on pieces of Crosswalk for AWS since before we launched Pulumi, and we’re excited to be launching Crosswalk for AWS today to bring these together for all AWS users. Over the last few months, users have built on Crosswalk for AWS for all of the scenarios above and more.

Tableau has used Crosswalk for AWS to accelerate their Kubernetes migration on AWS, building on the networking and EKS capabilities of Crosswalk for AWS.

“We’ve been happily using Pulumi’s EKS support for more than three months now. Our team was looking for an end-to-end solution to tame the complexity of Kubernetes on AWS and ensure we adhere to AWS best practices. Pulumi’s Crosswalk effort has equipped our team to scale far better, our delivery is now automated, and we can now deliver new application and infrastructure features with much faster turn-around, sometimes in just a few hours. Pulumi is a key contributor to our team’s improvement in productivity.” - Pankaj Dhingra, Sr Director, Cloud Engineering, Tableau

Compute Software has used Crosswalk for AWS to go from zero to production, and then to evolve their infrastructure in AWS, all without needing a dedicated cloud infrastructure engineering team.

“We use Crosswalk for AWS with Fargate. It’s been instrumental in getting us into production. Deploying to the cloud is necessarily complex. Crosswalk streamlines the process by exposing the controls you care about, always allowing you to dive deeper when needed.” - Kenny Williams, Co-Founder, Compute Software

We’re excited by the success these teams and many more have already had with Crosswalk for AWS, but even more excited about the potential to truly democratize access to the AWS platform and unlock highly productive cloud development for all kinds of engineering organizations.

What’s Next?

We’re still just scratching the surface of what’s possible with Crosswalk. We’re continuing to evolve the open source projects to simplify additional features of the AWS platform. And we’re also working to bring the same approach to other cloud platforms, including Azure, GCP and Kubernetes.

We believe that making it easier for cloud engineers to develop and deploy natively in the cloud will unlock creativity and empower developers to take advantage of more of the rich capabilities of AWS. We can’t wait to see what you build with Crosswalk!

Crosswalk for AWS is free and open source, and you can get started today.

For more on Crosswalk for AWS see:

Secure APIs with Auth0 Lambda Authorizers Using Pulumi & AWS

Erin Krengel — Wed, 24 Apr 2019 00:00:00 +0000

Creating serverless applications just got even easier! You can now protect your application APIs in just three easy steps. We’ve already posted about how easy it is to create serverless apps in Pulumi. Now, we’re helping you simplify protecting those apps with API Gateway and Lambda authorizers.

With Pulumi’s new AWSX package, you can quickly define a Lambda and an AWS Lambda authorizer to protect it. We’re once again harnessing the power of Lambdas as Lambdas to allow developers to focus on writing code.

Today, we will walkthrough creating a simple serverless app using AWS and Pulumi. We will simplify implementing the OAuth protocol by using Auth0 and AWS Lambda authorizers to authorize users. Auth0 provides a universal authentication and authorization platform for applications. It has become an extremely popular platform for user management because Auth0 makes OAuth easy.

Concepts

Before we get started, let’s go over some concepts we’ll be using.

API Gateway

An API Gateway takes clients requests and then re-routes them to the appropriate backend service. Typically, a gateway serves as the “front door” for an API and is used for traffic management as well as flexible security controls.

Using AWS API Gateway, you can build a fully managed collection of resources which integrate with AWS Lambda and various other AWS resources. To control who can call your API, you can use IAM permissions, an Amazon Cognito User Pool or set up custom logic using a Lambda authorizer.

OAuth

OAuth is an industry-standard for token-based authorization. Using OAuth, users can grant third-party applications or websites access to their information on another website without disclosing their password. Google and Facebook are two very common examples of sites that permit users to share their personal information with websites or applications using OAuth.

Lambda Authorizers

An AWS Lambda authorizer is a Lambda that provides access control to an API. It contains custom logic for authorizing requests to an endpoint. If a Lambda authorizer is configured, API Gateway routes a client’s call to the Lambda first. The Lambda authorizer runs its custom logic and returns a Policy and principal ID, which are used by API Gateway to determine if the call to the backend is allowed.

Step-by-Step Guide To Creating a Lambda Authorizer

Using a Lambda authorizer, we can implement the authorization flow using Auth0 to handle our Access Tokens.

API Gateway can be used to protect many different resources, and in the following example we will front an AWS Lambda. Users authenticate with Auth0 and then use the token they’ve obtained to make a request to our Lambda via our API Gateway, which authorizes the request by validating the token with Auth0. Some of the code examples have been stubbed or shortened for this blog post. The full code example can be found in our Pulumi examples repo.

Below is a diagram of the architecture we will set up. In Step 1, we will create our API Gateway and Hello World Lambda. In Step 2, we will set up the logic for our Lambda authorizer. And finally in Step 3, we will bring it all together by telling API Gateway to use our authorizer.

1 - Define Your Routes

Using Pulumi’s AWSX package, you can easily define routes to serverless apps as well as routes to static resources. We will create a very simple Hello world! backend Lambda, but in practice you can have this do much more interesting things.

import * as awsx from "@pulumi/awsx";

// Create our API and reference the Lambda authorizer
const api = new awsx.apigateway.API("myapi", {
    routes: [{
        path: "/hello",
        method: "GET",
        eventHandler: async () => {
            return {
                statusCode: 200,
                body: "<h1>Hello world!</h1>",
            };
        },
    }],
});

2 - Write Your Custom Auth Logic

Just as Pulumi allows you to define your Lambda inline, you can define your custom authorization code. We will define a small wrapper for our custom logic and write the rest of the code as a function called authenticate. Pulumi will package up all our runtime code and create an AWS Lambda for us.

import * as awsx from "@pulumi/awsx";
import * as pulumi from '@pulumi/pulumi';

import * as jwksClient from 'jwks-rsa';
import * as jwt from 'jsonwebtoken';
import * as util from 'util';

const config = new pulumi.Config();
const jwksUri = config.require("jwksUri");
const audience = config.require("audience");
const issuer = config.require("issuer");

const authorizerLambda = async (event: awsx.apigateway.AuthorizerEvent) => {
    try {
        return await authenticate(event);
    }
    catch (err) {
        console.log(err);
        // Tells API Gateway to return a 401 Unauthorized response
        throw new Error("Unauthorized");
    }
}

/**
 * Below is all code that gets added to the Authorizer Lambda. The code was copied and
 * converted to TypeScript from
 * [Auth0's GitHub Example](https://github.com/auth0-samples/jwt-rsa-aws-custom-authorizer)
 */

function getToken(event: awsx.apigateway.AuthorizerEvent): string {
    // Stubbed function to extract and return the Bearer Token from the Lambda event parameter
}

// Check if the Token is valid with Auth0
async function authenticate(event: awsx.apigateway.AuthorizerEvent): Promise<awsx.apigateway.AuthorizerResponse> {
    const token = getToken(event);

    const decoded = jwt.decode(token, { complete: true });
    if (!decoded || typeof decoded === "string" || !decoded.header || !decoded.header.kid) {
        throw new Error('invalid token');
    }

    const client = jwksClient({
        cache: true,
        rateLimit: true,
        jwksRequestsPerMinute: 10, // Default value
        jwksUri: jwksUri
    });

    const key = await util.promisify(client.getSigningKey)(decoded.header.kid);
    const signingKey = key.publicKey || key.rsaPublicKey;
    if (!signingKey) {
        throw new Error('could not get signing key');
    }

    const verifiedJWT = await jwt.verify(token, signingKey, { audience, issuer });
    if (!verifiedJWT || typeof verifiedJWT === "string" || !isVerifiedJWT(verifiedJWT)) {
        throw new Error('could not verify JWT');
    }
    return awsx.apigateway.authorizerResponse(verifiedJWT.sub, 'Allow', event.methodArn);
}

3 - All Together Now

Now let’s tie it all together! We will modify the route we created in Step 1 to include our authorizer.

import * as awsx from "@pulumi/awsx";

// Create our API and reference the Lambda authorizer
const api = new awsx.apigateway.API("myapi", {
    routes: [{
        path: "/hello",
        ...
        authorizers: awsx.apigateway.getTokenLambdaAuthorizer({
            authorizerName: "jwt-rsa-custom-authorizer",
            header: "Authorization",
            handler: authorizerLambda,
            identityValidationExpression: "^Bearer [-0-9a-zA-Z._]*$",
            authorizerResultTtlInSeconds: 3600,
        }),
    }],
});

The Finished Product

When we run pulumi up, Pulumi will create our resources and provide us with our protected endpoint.

$ pulumi up
...
     Type Name Status Info
 + pulumi:pulumi:Stack lambda-authorizer-dev created 1 message
 + ├─ aws:apigateway:x:API myapi created
 + │ ├─ aws:iam:Role myapifc45ff03 created
 + │ ├─ aws:iam:RolePolicyAttachment myapifc45ff03-32be53a2 created
 + │ ├─ aws:lambda:Function myapifc45ff03 created
 + │ ├─ aws:apigateway:RestApi myapi created
 + │ ├─ aws:apigateway:Deployment myapi created
 + │ ├─ aws:lambda:Permission myapi-62a1b306 created
 + │ └─ aws:apigateway:Stage myapi created
 + ├─ aws:iam:Role jwt-rsa-custom-authorizer-authorizer-role created
 + ├─ aws:iam:Role jwt-rsa-custom-authorizer created
 + ├─ aws:iam:RolePolicyAttachment jwt-rsa-custom-authorizer-32be53a2 created
 + ├─ aws:lambda:Function jwt-rsa-custom-authorizer created
 + └─ aws:iam:RolePolicy jwt-rsa-custom-authorizer-invocation-policy created
Outputs:
    url: "https://XXXXX.execute-api.us-west-2.amazonaws.com/stage/"
Resources:
    + 14 created
Duration: 28s

You’ll notice there’s a ton of resources we didn’t explicitly define. AWSX provisions the appropriate Lambdas, roles, and more for you. If required, all of these resources can be overridden in your Pulumi program by explicitly defining them.

Let’s give try out our new API.

We can now curl our endpoint without a token and should get a 401 Unauthorized response.

$ curl $(pulumi stack output url)hello
{"message":"Unauthorized"}

We can curl our endpoint with an invalid token and should once again get a 401 Unauthorized response.

$ curl $(pulumi stack output url)hello -H "Authorization: Bearer invalid"
{"message":"Unauthorized"}

Finally, we expect a 200 response when we obtain a token from Auth0 and use it to call our API. We can get a token by visiting the API Details page for our API and clicking the Test tab. This provides us with a valid token for testing purposes only. The way tokens are acquired by users or client apps will depend on the type of application (i.e. native mobile app, single-page app, web app, etc.) being developed. For more information on authenticating users, visit Auth0’s documentation.

Using the test access token, the API will return a 200 response: Hello world!

$ curl $(pulumi stack output url)hello -H "Authorization: Bearer <VALID_TOKEN>"
<h1>Hello world!</h1>

For the complete code example, visit our examples repo.

Keep It Going

While we used OAuth and Auth0 in this blog post, AWS Lambda authorizers are an extremely flexible tool that allow you to implement whatever custom authorization logic you require. AWS also supports Cognito Authorizers, which use AWS Cognito User Pools, and we will share even more examples soon.

There are a vast amount of resources for deploying your code into the cloud, and we want to make it easier for developers to make the most of those resources. Pulumi’s AWSX package provides reusable abstractions to enable developers to get their code production ready faster.

Get going with these resources: