Pulumi Blog: Paul Stack

Secure AWS Lambda with IAM ABAC Policies

Paul Stack — Tue, 19 Jul 2022 00:00:00 +0000

Event-driven, serverless functions have become a defining feature of many modern cloud architectures. With recent capabilities such as AWS Lambda URLs and AWS Lambda Containers, AWS has made it clear that Lambda Functions are a platform that teams can use to deliver increasingly sophisticated services without worrying about managing underlying compute resources.

Today, AWS announced another advancement for their Lambda Functions platform: Attribute-Based Access Control (ABAC). At its core, ABAC support brings more granular permissions that are automatically applied based on IAM role tags, Lambda tags, or both. This update builds on well-established Role-Based Access Control (RBAC) principles while making it possible to implement granular controls without permissions updates for every new user and resource.

What are Attributes?

Attributes are a key or key/value pair - in AWS, these attributes are called tags. Tags can be applied to multiple roles – for example identifying members of a team.

Across many teams, your organization may share a lot of common roles, and, in order to enforce the principle of least access, you may wish to restrict access across teams such that Developer Team members cannot access Ops Team resources and vice versa.

Using ABAC for AWS Lambda Functions

For this example, we’ll define a new role ‘abac-test’ as well as a team attribute that must have the value of either ‘developers’ or ‘ops’ to enable creating a Lambda function.

To get started, let’s first install the Pulumi AWS provider:

$ npm install @pulumi/aws

$ dotnet add package Pulumi.Aws

$ pip3 install pulumi_aws

$ go get github.com/pulumi/pulumi-aws/sdk/v5

Now, create a new IAM Role that a user can assume:

const role = new aws.iam.Role("abac-test", {
 assumeRolePolicy: JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Sid: "",
 Effect: "Allow",
 Principal: {
 AWS: "arn:aws:iam::MYACCOUNTID:root",
 },
 Action: "sts:AssumeRole",
 },
 ],
 }),
});

var role = new Aws.Iam.Role("abac-test", new Aws.Iam.RoleArgs
{
 AssumeRolePolicy = JsonSerializer.Serialize(new Dictionary<string, object?>
 {
 { "Version", "2012-10-17" },
 { "Statement", new[]
 {
 new Dictionary<string, object?>
 {
 { "Action", "sts:AssumeRole" },
 { "Effect", "Allow" },
 { "Principal", new Dictionary<string, object?>
 {
 { "AWS", "arn:aws:iam::MYACCOUNTID:root" },
 } },
 },
 }
 },
 }),
});

role = aws.iam.Role("abac-test",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "AWS": "arn:aws:iam::MYACCOUNTID:root",
 },
 }],
 }),
 tags={
 "Team": "developers",
 })

role, err := iam.NewRole(ctx, "abac-test", &iam.RoleArgs{
 AssumeRolePolicy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Sid": "",
 "Effect": "Allow",
 "Principal": {
 "AWS": "arn:aws:iam::MYACCOUNTID:root"
 },
 "Action": "sts:AssumeRole"
 }]
 }`),
})
if err != nil {
 return err
}

 abacTest:
 type: aws:iam:Role
 properties:
 assumeRolePolicy:
 Fn::ToJSON:
 Version: 2012-10-17
 Statement:
 - Action: sts:AssumeRole
 Effect: Allow
 Principal:
 AWS: arn:aws:iam::MYACCOUNTID:root

Let’s now create a new IAM Policy that allows a user to create Lambda functions:

const createPolicy = new aws.iam.Policy(
 "createLambda",
 {
 policy: JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Effect: "Allow",
 Action: ["lambda:CreateFunction", "lambda:TagResource"],
 Resource: "arn:aws:lambda:*:*:function:*",
 Condition: {
 // the requesting resource must have a tag with
 // team = developers/ops
 StringEquals: {
 "aws:RequestTag/Team": ["developers", "ops"], // must match
 },
 // your request must contain these tags
 // eg. project, but you might not have this defined yet
 "ForAllValues:StringEquals": {
 "aws:TagKeys": "Team", // must exist
 },
 },
 },
 {
 // Pulumi needs to check what functions exist and what versions exist to create updates and new versions
 Effect: "Allow",
 Action: [
 "lambda:GetFunction",
 "lambda:ListVersionsByFunction",
 "lambda:GetFunctionCodeSigningConfig",
 ],
 Resource: "*",
 },
 {
 Effect: "Allow",
 // These are assume role policies
 Action: ["iam:ListRoles", "iam:PassRole"],
 Resource: "*",
 },
 ],
 }),
 },
);

var createPolicy = new Aws.Iam.Policy("createPolicy", new Aws.Iam.PolicyArgs
{
 PolicyDocument = JsonSerializer.Serialize(new Dictionary<string, object?>
 {
 { "Version", "2012-10-17" },
 { "Statement", new[]
 {
 new Dictionary<string, object?>
 {
 { "Effect", "Allow" },
 { "Action", new[]
 {
 "lambda:CreateFunction",
 "lambda:TagResource",
 }
 },
 { "Resource", "arn:aws:lambda:*:*:function:*" },
 { "Condition", new Dictionary<string, object?>
 {
 // the requesting resource must have a tag with
 // team = developers/ops
 { "StringEquals", new Dictionary<string, object?>
 {
 { "aws:RequestTag/Team", new[]
 {
 "developers",
 "ops",
 }
 },
 } },
 // your request must contain these tags
 // eg. project, but you might not have this defined yet
 { "ForAllValues:StringEquals", new Dictionary<string, object?>
 {
 { "aws:TagKeys", new[]
 {
 "Team",
 }
 },
 } },
 } },
 },
 new Dictionary<string, object?>
 {
 // Pulumi needs to check what functions exist and what versions exist to create updates and new versions
 { "Effect", "Allow" },
 { "Action", new[]
 {
 "lambda:GetFunction",
 "lambda:ListVersionsByFunction",
 "lambda:GetFunctionCodeSigningConfig",
 }
 },
 { "Resource", "*" },
 },
 new Dictionary<string, object?>
 {
 // These are assume role policies
 { "Effect", "Allow" },
 { "Action", new[]
 {
 "iam:ListRoles",
 "iam:PassRole",
 }
 },
 { "Resource", "*" },
 },
 }
 },
 }),
});

create_policy = aws.iam.Policy("createPolicy", policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [
 {
 "Effect": "Allow",
 "Action": [
 "lambda:CreateFunction",
 "lambda:TagResource",
 ],
 "Resource": "arn:aws:lambda:*:*:function:*",
 "Condition": {
 #the requesting resource must have a tag with
 #team = developers/ops
 "StringEquals": {
 "aws:RequestTag/Team": [
 "developers",
 "ops",
 ],
 },
 # your request must contain these tags
 # eg. project, but you might not have this defined yet
 "ForAllValues:StringEquals": {
 "aws:TagKeys": ["Team"],
 },
 },
 },
 {
 # Pulumi needs to check what functions exist and what versions exist to create updates and new versions
 "Effect": "Allow",
 "Action": [
 "lambda:GetFunction",
 "lambda:ListVersionsByFunction",
 "lambda:GetFunctionCodeSigningConfig",
 ],
 "Resource": "*",
 },
 {
 # These are assume role policies
 "Effect": "Allow",
 "Action": [
 "iam:ListRoles",
 "iam:PassRole",
 ],
 "Resource": "*",
 },
 ],
}))

createPolicy, err := iam.NewRolePolicy(ctx, "createPolicy", &iam.RoleArgs{
 Policy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Sid": "",
 "Effect": "Allow",
 "Condition": map[string]interface{}{
 "StringEquals": map[string]interface{}{
 "aws:RequestTag/Team": []string{
 "developers",
 "ops",
 },
 },
 "ForAllValues:StringEquals": map[string]interface{}{
 "aws:TagKeys": []string{
 "Team",
 },
 },
 },
 "Resource": "arn:aws:lambda:*:*:function:*",
 "Action": []string{
 "lambda:CreateFunction",
 "lambda:TagResource",
 },
 },
 {
 "Effect": "Allow",
 "Resource": "*",
 "Action": []string{
 "lambda:GetFunction",
 "lambda:ListVersionsByFunction",
 "lambda:GetFunctionCodeSigningConfig",
 },
 },
 {
 {
 "Effect": "Allow",
 "Resource": "*",
 "Action": []string{
 "lambda:CreateFunction",
 "lambda:TagResource",
 },
 }
 }]
 }`),
})
if err != nil {
 return err
}

 createPolicy:
 type: aws:iam:Policy
 properties:
 policy:
 Fn::ToJSON:
 Version: 2012-10-17
 Statement:
 - Effect: Allow
 Action:
 - lambda:CreateFunction
 - lambda:TagResource
 Resource: 'arn:aws:lambda:*:*:function:*'
 Condition:
 StringEquals:
 aws:RequestTag/Team:
 - developers
 - ops
 ForAllValues:StringEquals:
 aws:TagKeys:
 - Team
 - Effect: Allow
 Action:
 - lambda:GetFunction
 - lambda:ListVersionsByFunction
 - lambda:GetFunctionCodeSigningConfig
 Resource: '*'
 - Effect: Allow
 Action:
 - iam:ListRoles
 - iam:PassRole
 Resource: '*'

Notice, as part of the policy condition, we ensure that the requesting resource must have a tag with team: developers or team: ops.

Let’s ensure that this policy is attached to the Role we initially created:

const rpa = new aws.iam.RolePolicyAttachment("rpa", {
 policyArn: createPolicy.arn,
 role: role,
})

var rpa = new Aws.Iam.RolePolicyAttachment("rpa", new Aws.Iam.RolePolicyAttachmentArgs
{
 PolicyArn = createPolicy.Arn,
 Role = role.Name,
});

rpa = aws.iam.RolePolicyAttachment("rpa",
 policy_arn=create_policy.arn,
 role=role.name)

rpa, err := iam.NewRolePolicyAttachment(ctx, "rpa", &iam.RolePolicyAttachmentArgs{
 PolicyArn: createPolicy.Arn,
 Role: role.Name,
})
if err != nil {
 return err
}

 rpa:
 type: aws:iam:RolePolicyAttachment
 properties:
 policyArn: ${createPolicy.arn}
 role: ${abacTest.name}

We can deploy these resources using the command pulumi update:

pulumi up
Previewing update (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/previews/c6fb4580-2706-47ee-8a56-b8942631254e

 Type Name Plan
 + pulumi:pulumi:Stack aws-abac-test-abac-testing create
 + ├─ aws:iam:Policy createLambda create
 + ├─ aws:iam:Role abac-test create
 + └─ aws:iam:RolePolicyAttachment createPolicy create

Resources:
 + 4 to create

Do you want to perform this update? yes
Updating (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/updates/1

 Type Name Status
 + pulumi:pulumi:Stack aws-abac-test-abac-testing created
 + ├─ aws:iam:Role abac-test created
 + ├─ aws:iam:Policy createLambda created
 + └─ aws:iam:RolePolicyAttachment createPolicy created

Resources:
 + 4 created

Duration: 7s

For the purposes of testing, we are now going to create a specific AWS Provider resource that we can pass the IAM Role details to assume.

const abacUserProvider = new aws.Provider(
 "assumeRole",
 {
 // assume the previously created role with the ABAC configured policy into our provider
 assumeRole: {
 roleArn: role.arn,
 sessionName: "abacTesting",
 },
 region: "us-east-1",
 },
 {
 // we want to depend on the role policy attachment being in place before we create the provider
 dependsOn: [rpa],
 }
);

var abacUserProvider = new Aws.Provider("abacUserProvider", new Aws.ProviderArgs
{
 AssumeRole = new Aws.Config.Inputs.AssumeRoleArgs
 {
 RoleArn = role.Arn,
 SessionName = "abacTesting",
 },
 Region = "us-east-1",
}, new CustomResourceOptions
{
 DependsOn =
 {
 rpa,
 },
});

abac_user_provider = pulumi.providers.Aws("abacUserProvider",
 assume_role=aws.config.AssumeRoleArgs(
 role_arn=role.arn,
 session_name="abacTesting",
 ),
 region="us-east-1",
 opts=pulumi.ResourceOptions(depends_on=[rpa]))

abacUserProvider, err := providers.Newaws(ctx, "abacUserProvider", &providers.awsArgs{
 AssumeRole: config.AssumeRole{
 RoleArn: role.Arn,
 SessionName: "abacTesting",
 },
 Region: "us-east-1",
}, pulumi.DependsOn([]pulumi.Resource{
 rpa,
}))
if err != nil {
 return err
}

 abacUserProvider:
 type: pulumi:providers:aws
 properties:
 assumeRole:
 roleArn: ${role.arn}
 sessionName: "abacTesting"
 region: "us-east-1"
 options:
 dependsOn:
 - ${rpa}

Now we can create an AWS Lambda with Pulumi as follows:

const lambdaRole = new aws.iam.Role("iamRole", {
 assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal({
 Service: "lambda.amazonaws.com",
 }),
});

const deleteLambdaPolicy = new aws.iam.Policy(
 "deleteLambdaPolicy",
 {
 policy: JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Effect: "Allow",
 Action: ["lambda:DeleteFunction"],
 Resource: "*",
 },
 ],
 }),
 },
 { parent: lambdaRole }
);

new aws.iam.RolePolicyAttachment(
 `deletePolicy`,
 {
 policyArn: deletePolicy.arn,
 role: role,
 },
 { parent: deletePolicy })

const lambda = new aws.lambda.Function(
 "lambdaFunction",
 {
 code: new pulumi.asset.AssetArchive({
 ".": new pulumi.asset.FileArchive("./app"),
 }),
 runtime: "nodejs12.x",
 role: lambdaRole.arn,
 handler: "index.handler",
 },
 { provider: abacUserProvider }
);

var lambdaRole = new Aws.Iam.Role("lambdaRole", new Aws.Iam.RoleArgs
{
 AssumeRolePolicy = JsonSerializer.Serialize(new Dictionary<string, object?>
 {
 { "Version", "2012-10-17" },
 { "Statement", new[]
 {
 new Dictionary<string, object?>
 {
 { "Action", "sts:AssumeRole" },
 { "Effect", "Allow" },
 { "Principal", new Dictionary<string, object?>
 {
 { "Service", "lambda.amazonaws.com" },
 } },
 },
 }
 },
 }),
});
var deletePolicy = new Aws.Iam.Policy("deletePolicy", new Aws.Iam.PolicyArgs
{
 PolicyDocument = JsonSerializer.Serialize(new Dictionary<string, object?>
 {
 { "Version", "2012-10-17" },
 { "Statement", new[]
 {
 new Dictionary<string, object?>
 {
 { "Action", "lambda:DeleteFunction" },
 { "Effect", "Allow" },
 { "Resource", "*" },
 },
 }
 },
 }),
});
var deletePolicyAttachment = new Aws.Iam.RolePolicyAttachment("deletePolicyAttachment", new Aws.Iam.RolePolicyAttachmentArgs
{
 Role = role.Name,
 PolicyArn = deletePolicy.Arn,
});
var lambdaFunction = new Aws.Lambda.Function("lambdaFunction", new Aws.Lambda.FunctionArgs
{
 Code = new FileArchive("./app"),
 Role = lambdaRole.Arn,
 Handler = "index.handler",
 Runtime = "nodejs14.x",
}, new CustomResourceOptions
{
 Provider = abacUserProvider,
});

lambda_role = aws.iam.Role("lambdaRole", assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": "lambda.amazonaws.com",
 },
 }],
}))
delete_policy = aws.iam.Policy("deletePolicy", policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": "lambda:DeleteFunction",
 "Effect": "Allow",
 "Resource": "*",
 }],
}))
delete_policy_attachment = aws.iam.RolePolicyAttachment("deletePolicyAttachment",
 role=role.name,
 policy_arn=delete_policy.arn)
lambda_function = aws.lambda_.Function("lambdaFunction",
 code=pulumi.FileArchive("./app"),
 role=lambda_role.arn,
 handler="index.handler",
 runtime="nodejs14.x",
 opts=pulumi.ResourceOptions(provider=abac_user_provider))

lambdaRole, err := iam.NewRole(ctx, "lambdaRole", &iam.RoleArgs{
 AssumeRolePolicy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Principal": {
 "Service": "lambda.amazonaws.com"
 },
 "Action": "sts:AssumeRole"
 }]
 }`),
})
if err != nil {
 return err
}

deletePolicy, err := iam.NewRolePolicy(ctx, "deletePoicy", &iam.RoleArgs{
 Policy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Resource": "*",
 "Action": []string{
 "lambda:DeleteFunction",
 },
 }]
 }`),
})
if err != nil {
 return err
}

_, err = iam.NewRolePolicyAttachment(ctx, "deletePolicyAttachment", &iam.RolePolicyAttachmentArgs{
 Role: role.Name,
 PolicyArn: deletePolicy.Arn,
})
if err != nil {
 return err
}

_, err = lambda.NewFunction(ctx, "lambdaFunction", &lambda.FunctionArgs{
 Code: pulumi.NewFileArchive("./app"),
 Role: lambdaRole.Arn,
 Handler: pulumi.String("index.handler"),
 Runtime: pulumi.String("nodejs14.x"),
}, pulumi.Provider(abacUserProvider))
if err != nil {
 return err
}

 lambdaRole:
 type: aws:iam:Role
 properties:
 assumeRolePolicy:
 Fn::ToJSON:
 Version: 2012-10-17
 Statement:
 - Action: sts:AssumeRole
 Effect: Allow
 Principal:
 Service: lambda.amazonaws.com
 deletePolicy:
 type: aws:iam:Policy
 properties:
 policy:
 Fn::ToJSON:
 Version: 2012-10-17
 Statement:
 - Action: lambda:DeleteFunction
 Effect: Allow
 Resource: '*'
 deletePolicyAttachment:
 type: aws:iam:RolePolicyAttachment
 properties:
 role: ${abacRole.name}
 policyArn: ${deletePolicy.arn}
 lambdaFunction:
 type: aws:lambda:Function
 properties:
 code:
 Fn::FileArchive: ./app
 role: ${lambdaRole.arn}
 handler: index.handler
 runtime: nodejs14.x

If we try and deploy the lambda, we will get the following using the role we are going to assume then we will get an error as the lambda function has no Team tag.

pulumi up
Previewing update (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/previews/7fdc10b2-5e44-497e-a067-f2e8b257496c

 Type Name Plan
 pulumi:pulumi:Stack aws-abac-test-abac-testing
 + ├─ aws:iam:Role iamRole create
 ├─ aws:iam:Role abac-test
 + │ └─ aws:iam:Policy deletePolicy create
 + │ └─ aws:iam:RolePolicyAttachment deletePolicy create
 + ├─ pulumi:providers:aws assumeRole create
 + └─ aws:lambda:Function lambdaFunction create

Resources:
 + 5 to create
 4 unchanged

Do you want to perform this update? yes
Updating (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/updates/2

 Type Name Status Info
 pulumi:pulumi:Stack aws-abac-test-abac-testing **failed** 1 error
 + ├─ aws:iam:Role iamRole created
 ├─ aws:iam:Role abac-test
 + │ └─ aws:iam:Policy deletePolicy created
 + │ └─ aws:iam:RolePolicyAttachment deletePolicy created
 + ├─ pulumi:providers:aws assumeRole created
 + └─ aws:lambda:Function lambdaFunction **creating failed** 1 error

Diagnostics:
 pulumi:pulumi:Stack (aws-abac-test-abac-testing):
 error: update failed

 aws:lambda:Function (lambdaFunction):
 error: 1 error occurred:
 * error creating Lambda Function (1): AccessDeniedException:
 status code: 403, request id: 4a8676f2-2b68-4d42-ad4b-f0e10ca31afd

Resources:
 + 4 created
 4 unchanged

Duration: 6s

If we add the correct Team tag to the Lambda function then the deploy will proceed as expected:

pulumi up
Previewing update (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/previews/0afa267f-5f9f-43f5-9e00-d3ee19dbf2b8

 Type Name Plan
 pulumi:pulumi:Stack aws-abac-test-abac-testing
 + └─ aws:lambda:Function lambdaFunction create

Resources:
 + 1 to create
 8 unchanged

Do you want to perform this update? yes
Updating (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/updates/3

 Type Name Status
 pulumi:pulumi:Stack aws-abac-test-abac-testing
 + └─ aws:lambda:Function lambdaFunction created

Resources:
 + 1 created
 8 unchanged

Duration: 9s

Try It Out!

Lambda ABAC capabilities will help you to simplify governance by helping you to standardize roles and maintain security boundaries between teams. A common use case is using tags to restrict/enable the invoke action for a function in addition to the create action demonstrated above. Give Lambda ABAC functionality a try and let us know what you think in Community Slack.

Crosswalk for AWS in all Pulumi Languages

Paul Stack — Wed, 04 May 2022 06:57:00 -0700

Portions of this blog post are out of date. See the AWS guides for an updated overview and examples.

Crosswalk for AWS is a collection of libraries that make it easy to work with AWS using Pulumi Infrastructure as Code. The Crosswalk for AWS libraries are some of the most widely used higher-level components in the Pulumi ecosystem, with hundreds of organizations building their infrastructure on the simple abstractions over key AWS services like ECS, API Gateway, VPC, Load Balancing, CloudTrail, EC2, ECR, and more.

Crosswalk for AWS is the easiest way to get started with core compute and application primitives on AWS. Running a containerized or serverless application on AWS requires just a handful of lines of code. This enables application and service development teams to directly own and manage the infrastructure for their applications. As Crosswalk builds on the primitive building blocks of the AWS platform, you can use 100% of what AWS offers, and can seamlessly move between high-level components and low-level platform primitives, mixing-and-matching as needed.

These libraries use automatic well-architected best practices to make common infrastructure as code tasks in AWS easier and more secure.

Crosswalk for AWS is available to users in all Pulumi languages, TypeScript, Python, .NET, Go, Java and YAML. The AWSx package now joins the existing API Gateway and EKS packages in the Pulumi Registry, available in all Pulumi languages.

From Zero to Production And Beyond with Crosswalk for AWS

Let’s walk through some examples of using Crosswalk for AWS for common use cases.

Containers (ECS, Fargate)

Many Pulumi customers deploy containerized workloads on AWS with ECS or Fargate. AWSx significantly simplifies deploying your application to these managed container services. With just a few lines of code and tens of seconds to deploy, our container is running in production in AWS.

import * as awsx from "@pulumi/awsx";
import * as aws from "@pulumi/aws";

// Create a repository.
const repo = new awsx.ecr.Repository("my-repo");

// Build an image from the "./app" directory (relative to our project and containing Dockerfile),
// and publish it to our ECR repository provisioned above.
const image = new awsx.ecr.Image("image", {
 repositoryUrl: repo.url,
 path: "./app",
})

// Create an ECS Cluster
const cluster = new aws.ecs.Cluster("default-cluster");

// // Create a load balancer on port 80 and spin up two instances of Nginx.
const lb = new awsx.lb.ApplicationLoadBalancer("nginx-lb");

const service = new awsx.ecs.FargateService("my-service", {
 cluster: cluster.arn,
 taskDefinitionArgs: {
 container: {
 image: image.imageUri,
 cpu: 512,
 memory: 128,
 essential: true,
 portMappings: [
 {
 containerPort: 80,
 targetGroup: lb.defaultTargetGroup,
 },
 ],
 },
 },
});

// Export the load balancer's address so that it's easy to access.
export const url = lb.loadBalancer.dnsName;

import pulumi
import pulumi_aws as aws
import pulumi_awsx as awsx

repo = awsx.ecr.Repository("my-repo");

image = awsx.ecr.Image("image",
 repository_url=repo.url,
 path="./app")


cluster = aws.ecs.Cluster("default-cluster")

lb = awsx.lb.ApplicationLoadBalancer("nginx-lb")

service = awsx.ecs.FargateService("service",
 cluster=cluster.arn,
 task_definition_args=awsx.ecs.FargateServiceTaskDefinitionArgs(
 containers={
 "nginx": awsx.ecs.TaskDefinitionContainerDefinitionArgs(
 image=image.image_uri,
 memory=128,
 port_mappings=[awsx.ecs.TaskDefinitionPortMappingArgs(
 container_port=80,
 target_group=lb.default_target_group,
 )]
 )
 }
 ))

pulumi.export("url", lb.load_balancer.dns_name)

using System.Collections.Generic;
using System.Threading.Tasks;
using Pulumi;
using Pulumi.Awsx.Ecs.Inputs;
using Aws = Pulumi.Aws;
using Ecr = Pulumi.Awsx.Ecr;
using Ecs = Pulumi.Awsx.Ecs;
using Lb = Pulumi.Awsx.Lb;

class MyStack : Stack
{
 public MyStack()
 {
 var repo = new Ecr.Repository("my-repo");

 var image = new Ecr.Image("image", new Ecr.ImageArgs
 {
 RepositoryUrl = repo.Url,
 Path = "./app",
 });

 var cluster = new Aws.Ecs.Cluster("demo-cluster");

 var lb = new Lb.ApplicationLoadBalancer("nginx-lb");

 var service = new Ecs.FargateService("my-service", new Ecs.FargateServiceArgs
 {
 Cluster = cluster.Arn,
 TaskDefinitionArgs = new FargateServiceTaskDefinitionArgs
 {
 Container = new TaskDefinitionContainerDefinitionArgs
 {
 Memory = 128,
 Cpu = 512,
 Image = “
 Essential = true,
 PortMappings = new TaskDefinitionPortMappingArgs
 {
 ContainerPort = 80,
 TargetGroup = lb.DefaultTargetGroup,
 }
 }
 }
 });
 }
}

class Program
{
 static Task<int> Main(string[] args) => Deployment.RunAsync<MyStack>();
}

import pulumi
import pulumi_aws as aws
import pulumi_awsx as awsx

repo = awsx.ecr.Repository("my-repo");

image = awsx.ecr.Image("image",
 repository_url=repo.url,
 path="./app")


cluster = aws.ecs.Cluster("default-cluster")

lb = awsx.lb.ApplicationLoadBalancer("nginx-lb")

service = awsx.ecs.FargateService("service",
 cluster=cluster.arn,
 task_definition_args=awsx.ecs.FargateServiceTaskDefinitionArgs(
 containers={
 "nginx": awsx.ecs.TaskDefinitionContainerDefinitionArgs(
 image=image.image_uri,
 memory=128,
 port_mappings=[awsx.ecs.TaskDefinitionPortMappingArgs(
 container_port=80,
 target_group=lb.default_target_group,
 )]
 )
 }
 ))

pulumi.export("url", lb.load_balancer.dns_name)

package myproject;

import com.pulumi.Pulumi;
import com.pulumi.aws.ecs.Cluster;
import com.pulumi.aws.lb.LoadBalancer;
import com.pulumi.awsx.ecr.ImageArgs;
import com.pulumi.awsx.ecr.Repository;
import com.pulumi.awsx.ecr.Image;
import com.pulumi.awsx.ecs.FargateService;
import com.pulumi.awsx.ecs.FargateServiceArgs;
import com.pulumi.awsx.ecs.inputs.FargateServiceTaskDefinitionArgs;
import com.pulumi.awsx.ecs.inputs.TaskDefinitionContainerDefinitionArgs;
import com.pulumi.awsx.ecs.inputs.TaskDefinitionPortMappingArgs;
import com.pulumi.awsx.lb.ApplicationLoadBalancer;

public class App {
 public static void main(String[] args) {
 Pulumi.run(ctx -> {
 // Create a repository.
 var repo = new Repository("my-repo");

 // Build an image from the "./app" directory (relative to our project and containing Dockerfile),
 // and publish it to our ECR repository provisioned above.
 var image = new Image("image", ImageArgs.builder()
 .repositoryUrl(repo.url())
 .path("./app")
 .build()
 );

 // Create an ECS Cluster
 var cluster = new Cluster("default-cluster");

 // // Create a load balancer on port 80 and spin up two instances of Nginx.
 var lb = new ApplicationLoadBalancer("nginx-lb");

 new FargateService("my-service", FargateServiceArgs.builder()
 .cluster(cluster.arn())
 .taskDefinitionArgs(FargateServiceTaskDefinitionArgs.builder()
 .container(TaskDefinitionContainerDefinitionArgs.builder()
 .image(image.imageUri())
 .cpu(521)
 .memory(128)
 .essential(true)
 .portMappings(TaskDefinitionPortMappingArgs.builder()
 .containerPort(80)
 .targetGroup(lb.defaultTargetGroup())
 .build()
 )
 .build())
 .build())
 .build()
 );

 // Export the load balancer's address so that it's easy to access.
 ctx.export("load-balancer-dns-name", lb.loadBalancer().applyValue(LoadBalancer::dnsName));
 });
 }
}

name: awsx-containers-yaml
runtime: yaml
description: A minimal AWS Pulumi YAML program

resources:
 # Create an AWS resource (S3 Bucket)
 bucket:
 type: aws:s3:Bucket
 # Create a repository
 repo:
 type: awsx:ecr:Repository
 # Build an image from the "./app" directory (relative to our project and
 # containing Dockerfile), and publish it to our ECR repository provisioned above.
 image:
 type: awsx:ecr:Image
 properties:
 repositoryUrl: ${repo.url}
 path: "./app"
 # Create an ECS Cluster
 cluster:
 type: aws:ecs:Cluster
 # Create a load balancer on port 80 and spin up two instances of Nginx.
 lb:
 type: awsx:lb:ApplicationLoadBalancer
 service:
 type: awsx:ecs:FargateService
 properties:
 cluster: ${cluster.arn}
 taskDefinitionArgs:
 container:
 image: ${image.imageUri}
 cpu: 512
 memory: 128
 essential: true
 portMappings:
 - containerPort: 80
 targetGroup: ${lb.defaultTargetGroup}

outputs:
 # Export the name of the bucket
 url: ${lb.loadBalancer.dnsName}

Using ECS, Fargate, ECR and ALB, we get a robust production-ready container deployment - horizontally scaled out, load-balanced, and integrated with a private image repository. With just a few more lines of code, we can add autoscaling, customize our ECS cluster, or wire through advanced container configuration (volumes, environment variables, and more).

Networking (VPC)

To adopt a security posture using best practices, we may want to move our cluster into a custom VPC, and run the containers in private subnets. We can define and configure our VPC:

import * as awsx from "@pulumi/awsx";

// Allocate a new VPC with the default settings:
const vpc = new awsx.ec2.Vpc("custom");

// Export a few resulting fields to make them easy to use:
export const vpcId = vpc.vpcId;
export const privateSubnetIds = vpc.privateSubnetIds;
export const publicSubnetIds = vpc.publicSubnetIds;

import pulumi
import pulumi_awsx as awsx

vpc = awsx.ec2.Vpc("custom")

pulumi.export("vpcId", vpc.vpc_id)
pulumi.export("publicSubnetIds", vpc.public_subnet_ids)
pulumi.export("privateSubnetIds", vpc.private_subnet_ids)

using Pulumi;
using Ec2 = Pulumi.Awsx.Ec2;

class MyStack : Stack
{
 public MyStack()
 {
 var vpc = new Ec2.Vpc("custom");

 this.VpcId = vpc.VpcId;
 }


 [Output] public Output<string> VpcId { get; set; }
}

class Program
{
 static Task<int> Main(string[] args) => Deployment.RunAsync<MyStack>();
}

package myproject;

import com.pulumi.Pulumi;
import com.pulumi.awsx.ec2.Vpc;

public class App {
 public static void main(String[] args) {
 Pulumi.run(ctx -> {
 var vpc = new Vpc("custom");

 ctx.export("vpcId", vpc.vpcId());
 ctx.export("privateSubnetIds", vpc.privateSubnetIds());
 ctx.export("publicSubnetIds", vpc.publicSubnetIds());
 });
 }
}

name: awsx-networking-yaml
runtime: yaml
description: A minimal AWS Pulumi YAML program

resources:
 # Allocate a new VPC with the default settings:
 vpc:
 type: awsx:ec2:Vpc

outputs:
 vpcId: ${vpc.VpcId}
 privateSubnetIds: ${vpc.privateSubnetIds}
 publicSubnetIds: ${vpc.publicSubnetIds}

The Vpc component builds on best-practices Virtual Private Cloud (VPC) design patterns based on AWS guidance and documentation. By building in simple defaults for routing, subnet structure, NATs and multi-AZ we can get started quickly without having to re-discover these best practices. And as your needs grow, we can deeply customize and evolve our VPC structure - defining custom CIDR blocks, customizing Internet and NAT Gateways or additional private subnets.

Serverless (Lambda, API Gateway)

It’s common to start with serving a simple REST API on the internet when beginning a new cloud project. With just a few lines of code and tens of seconds to deploy, Crosswalk for AWS makes it easy to build and deploy our API.

import * as apigateway from "@pulumi/aws-apigateway";
import * as aws from "@pulumi/aws";

const f = new aws.lambda.CallbackFunction("f", {
 callback: async (ev, ctx) => {
 console.log(JSON.stringify(ev));
 return {
 statusCode: 200,
 body: "goodbye",
 };
 },
});

const api = new apigateway.RestAPI("api", {
 routes: [{
 path: "/",
 method: "GET",
 eventHandler: f,
 }],
});

export const url = api.url;

import json
import pulumi
import pulumi_aws as aws
import pulumi_aws_apigateway as apigateway

role = aws.iam.Role("mylambda-role",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Principal": { "Service": "lambda.amazonaws.com" },
 "Action": "sts:AssumeRole"
 }]
 })
)

policy = aws.iam.RolePolicy("mylambda-policy",
 role=role.id,
 policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": ["logs:*", "cloudwatch:*"],
 "Resource": "*",
 "Effect": "Allow",
 }],
 }))

# Closure serialization is not supported in multi-lang components
# so we need to provide a handler function explicitly from the file-system.
# Refer to https://github.com/pulumi/pulumi-aws-apigateway/tree/main/examples/simple-py/handler
# for an example handler.
f = aws.lambda_.Function("mylambda",
 runtime=aws.lambda_.Runtime.PYTHON3D8,
 code=pulumi.AssetArchive({
 ".": pulumi.FileArchive("./handler"),
 }),
 timeout=300,
 handler="handler.handler",
 role=role.arn,
 opts=pulumi.ResourceOptions(depends_on=[policy]),
)

api = apigateway.RestAPI('api', routes=[
 apigateway.RouteArgs(path="/", method="GET", event_handler=f),
])

pulumi.export('url', api.url)

using Pulumi;
using System.Collections.Generic;
using ApiGW = Pulumi.AwsApiGateway;
using Lambda = Pulumi.Aws.Lambda;
using Iam = Pulumi.Aws.Iam;

class MyStack : Stack
{
 public MyStack()
 {
 var lambdaRole = new Iam.Role("mylambda-role", new Iam.RoleArgs
 {
 AssumeRolePolicy =
 @"{
 ""Version"": ""2012-10-17"",
 ""Statement"": [{
 ""Effect"": ""Allow"",
 ""Principal"": { ""Service"": ""lambda.amazonaws.com"" },
 ""Action"": ""sts:AssumeRole""
 }]
 }"
 });

 var rolePolicy = new Iam.RolePolicy("mylambda-policy", new Iam.RolePolicyArgs
 {
 Role = lambdaRole.Id,
 Policy =
 @"{
 ""Version"": ""2012-10-17"",
 ""Statement"": [{
 ""Action"": [""logs:*"", ""cloudwatch:*""],
 ""Resource"": ""*"",
 ""Effect"": ""Allow""
 }]
 }"
 });

 // Closure serialization is not supported in multi-lang components
 // so we need to provide a handler function explicitly from the file-system.
 // Refer to https://github.com/pulumi/pulumi-aws-apigateway/tree/main/examples/simple-cs/handler
 // for an example handler.
 var lambda = new Lambda.Function("lambda", new Lambda.FunctionArgs
 {
 Runtime = Lambda.Runtime.Python3d8,
 Code = new AssetArchive(new Dictionary<string, AssetOrArchive>{
 ["."] = new FileArchive("./handler"),
 }),
 Timeout = 300,
 Handler = "handler.handler",
 Role = lambdaRole.Arn
 }, new Pulumi.CustomResourceOptions { DependsOn = { rolePolicy } });

 var restAPI = new ApiGW.RestAPI("api", new ApiGW.RestAPIArgs
 {
 Routes = new List<ApiGW.Inputs.RouteArgs>{
 new ApiGW.Inputs.RouteArgs{Path="/", Method=ApiGW.Method.GET, EventHandler=lambda}}
 });

 this.Url = restAPI.Url;
 }

 [Output]
 public Output<string> Url { get; set; }
}

package main

import (
 apigateway "github.com/pulumi/pulumi-aws-apigateway/sdk/go/apigateway"
 "github.com/pulumi/pulumi-aws/sdk/v4/go/aws/iam"
 "github.com/pulumi/pulumi-aws/sdk/v4/go/aws/lambda"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {

 role, err := iam.NewRole(ctx, "lambda-role", &iam.RoleArgs{
 AssumeRolePolicy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Principal": { "Service": "lambda.amazonaws.com" },
 "Action": "sts:AssumeRole"
 }]
 }`),
 })
 if err != nil {
 return err
 }

 policy, err := iam.NewRolePolicy(ctx, "lambda-policy", &iam.RolePolicyArgs{
 Role: role.ID(),
 Policy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Action": ["logs:*", "cloudwatch:*"],
 "Resource": "*",
 "Effect": "Allow"
 }]
 }`),
 })
 if err != nil {
 return err
 }

 // Closure serialization is not supported in multi-lang components
 // so we need to provide a handler function explicitly from the file-system.
 // Refer to https://github.com/pulumi/pulumi-aws-apigateway/tree/main/examples/simple-go/handler
 // for an example handler.
 f, err := lambda.NewFunction(ctx, "lambda", &lambda.FunctionArgs{
 Runtime: lambda.RuntimePython3d8,
 Code: pulumi.NewAssetArchive(map[string]interface{}{
 ".": pulumi.NewFileArchive("./handler"),
 }),
 Timeout: pulumi.Int(300),
 Handler: pulumi.String("handler.handler"),
 Role: role.Arn,
 }, pulumi.DependsOn([]pulumi.Resource{policy}))
 if err != nil {
 return err
 }

 getMethod := apigateway.MethodGET
 restAPI, err := apigateway.NewRestAPI(ctx, "api", &apigateway.RestAPIArgs{
 Routes: []apigateway.RouteArgs{
 apigateway.RouteArgs{
 Path: "/",
 Method: &getMethod,
 EventHandler: f,
 },
 },
 })
 if err != nil {
 return err
 }

 ctx.Export("url", restAPI.Url)
 return nil
 })
}

By building on top of great AWS building blocks like Lambda, API Gateway, IAM and more, you avoid needing to worry about infrastructure, pay nearly zero fixed costs, and gain the ability to iterate quickly. With just a few more lines of code, you can wire up your own domain, add authorization, provision a database, or chain together a more complex event-driven application.

Kubernetes (EKS)

As your project continues to evolve, you may want to move your container workloads into Kubernetes using EKS. We can stand up an EKS cluster with just a few lines of code, then customize the cluster and deploy Kubernetes workloads into the cluster.

import * as pulumi from "@pulumi/pulumi";
import * as eks from "@pulumi/eks";

const projectName = pulumi.getProject();

// Create an EKS cluster with the default configuration.
const cluster1 = new eks.Cluster(`${projectName}-1`);


// Export the clusters' kubeconfig.
export const kubeconfig1 = cluster1.kubeconfig;

import pulumi
import pulumi_eks as eks

project_name = pulumi.get_project()

# Create an EKS cluster with the default configuration.
cluster1 = eks.Cluster(f"{project_name}-1")

pulumi.export("kubeconfig1", cluster1.kubeconfig)

using Pulumi;
using Pulumi.Eks;

class MyStack : Stack
{
 public MyStack()
 {
 string projectName = Deployment.Instance.ProjectName;

 var cluster1 = new Cluster($"{projectName}-1");

 Kubeconfig1 = cluster1.Kubeconfig;
 }


 [Output("kubeconfig1")]
 public Output<object> Kubeconfig1 { get; set; }
}

class Program
{
 static Task<int> Main(string[] args) => Deployment.RunAsync<MyStack>();
}

package main

import (
 "github.com/pulumi/pulumi-eks/sdk/go/eks"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // Create cluster with default settings
 cluster1, err := eks.NewCluster(ctx, "example-cluster-go-1", nil)
 if err != nil {
 return err
 }

 // Export the kubeconfig for clusters
 ctx.Export("kubeconfig1", cluster1.Kubeconfig)
 return nil
 })
}

By building on EKS, we get all the benefits of managed Kubernetes paired with the platform capabilities of AWS. With Crosswalk for AWS, we can get started quickly, and then evolve to take advantage of all of these platform features, like customized node groups, private networking, and even deploying Kubernetes YAML and Helm charts to our cluster.

Upgrading from an existing version of AWSx

Users of the existing TypeScript version of the AWSx package can migrate to the new multi-language implementations of the core AWSx features, or can continue using the existing TypeScript-specific implementations, now available in the classic namespace. This means that you are not required to use the new code when you update to the new version of the library. In order to use the existing version of the library, you simply need to use the classic namespace:

import * as awsx from “@pulumi/awsx/classic”;

const vpc = new awsx.ec2.Vpc(“my-classic-vpc”, {})

export const vpcId = vpc.Id;

What’s Next?

Crosswalk for AWS is free and open source, and you can get started today.

For more on Crosswalk for AWS see:

Announcing v5.0.0 of the Pulumi AWS Provider

Paul Stack — Mon, 28 Mar 2022 00:00:00 +0000

We are excited to announce v5.0.0 of the Pulumi AWS provider. The AWS provider is one of the most heavily used providers across the Pulumi ecosystem, and offers access to the full surface area of the upstream Terraform AWS Provider from within Pulumi projects in all supported Pulumi languages. The v5.0.0 release brings a substantial set of fixes and improvements to the provider, including a number of breaking changes as part of the major version release.

At Pulumi, we take compatibility seriously, and are conscious of the impact that major version updates can have on developer adoption. To that end, it is always our goal to minimize the extent of breaking changes where possible, and to provide easy paths for adoption of new versions. Among the breaking changes that come along with the upstream provider update, one in particular has potential to be disruptive to common usage of the Pulumi AWS provider - a change to the API for the AWS S3 Bucket resource which removes many features from the Bucket resource and introduces new resources to represent these capabilities.

In order to ensure that Pulumi AWS users are not adversely impacted by this breaking change, we have extended the Pulumi AWS provider to offer both the old implementation and the new implementation of the S3 Bucket resource. This means that all existing usages of S3 Bucket will continue to work as normal. We also now offer the ability to create a v2 variants of the S3 Bucket supported in the new upstream AWS provider.

Let’s consider an existing implementation of an S3 Bucket that is used to create a static website:

import * as aws from "@pulumi/aws";

const siteBucket = new aws.s3.Bucket("s3-website-bucket", {
 website: {
 indexDocument: "index.html",
 },
});

from pulumi_aws import s3

web_bucket = s3.Bucket('s3-website-bucket',
 website=s3.BucketWebsiteArgs(
 index_document="index.html",
 ))

using Pulumi;
using Pulumi.Aws.S3;
using Pulumi.Aws.S3.Inputs;

class WebsiteStack : Stack
{
 public WebsiteStack()
 {
 var bucket = new Bucket("my-bucket", new BucketArgs
 {
 Website = new BucketWebsiteArgs
 {
 IndexDocument = "index.html"
 }
 });
 }
}

package main

import (
 "github.com/pulumi/pulumi-aws/sdk/v4/go/aws/s3"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err := s3.NewBucket(ctx, "s3-website-bucket", &s3.BucketArgs{
 Website: s3.BucketWebsiteArgs{
 IndexDocument: pulumi.String("index.html"),
 },
 })
 if err != nil {
 return err
 }

 return nil
 })
}

In the upstream provider, the S3 Bucket resource has been broken into multiple pieces meaning that you would need to move the website part of the bucket out to be it’s own resource. We have ensured that this will continue to work in Pulumi.

What if I want to migrate to use the new implementation of an S3 Bucket?

The Pulumi AWS Provider, has introduced a new resource, aws.s3.BucketV2 that users can use to create new variants of how an S3 bucket resource would look. To create a static website using BucketV2, we can see that the code gets more targeted toward the resource that we are writing:

import * as aws from "@pulumi/aws";

const siteBucket = new aws.s3.BucketV2("s3-website-bucket");

const bucketWebsiteConfig = new aws.s3.BucketWebsiteConfigurationV2("website-config", {
 bucket: siteBucket.bucket,
 indexDocument: {
 suffix: "index.html"
 },
});

from pulumi_aws import s3

site_bucket = s3.BucketV2('s3-website-bucket')

website_config = aws.s3.BucketWebsiteConfigurationV2("website-config",
 bucket=site_bucket.bucket,
 index_document=aws.s3.BucketWebsiteConfigurationV2IndexDocumentArgs(
 prefix="index.html",
 ))

using Pulumi;
using Pulumi.Aws.S3;
using Pulumi.Aws.S3.Inputs;

class WebsiteStack : Stack
{
 public WebsiteStack()
 {
 var siteBucket = new BucketV2("s3-website-bucket", new BucketArgs{});

 var websiteConfig = new Aws.S3.BucketWebsiteConfigurationV2("website-config", new Aws.S3.BucketWebsiteConfigurationV2Args
 {
 Bucket = siteBucket.Bucket,
 IndexDocument = new Aws.S3.Inputs.BucketWebsiteConfigurationV2IndexDocumentArgs
 {
 Suffix = "index.html",
 },
 });
 }
}

package main

import (
 "github.com/pulumi/pulumi-aws/sdk/v4/go/aws/s3"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 siteBucket, err := s3.NewBucketV2(ctx, "s3-website-bucket", nil)
 if err != nil {
 return err
 }

 _, err := s3.NewBucketWebsiteConfigurationV2(ctx, "website-config", &s3.BucketWebsiteConfigurationV2Args{
 Bucket: siteBucket.bucket,
 IndexDocument: &s3.BucketWebsiteConfigurationV2IndexDocumentArgs{
 Suffix: pulumi.String("index.html"),
 },
 })
 if err != nil {
 return err
 }

 return nil
 })
}

Can I migrate from `aws.s3.Bucket` to `aws.s3.BucketV2`?

Yes! As part of this major version, we have taken advantage of using aliases in Pulumi to be able to allow users to change to aws.s3.BucketV2 easily. An alias was added to the new aws.s3.BucketV2 resource as follows:

 "aws_s3_bucket": {
 Tok: awsResource(s3Mod, "BucketV2"),
 Aliases: []tfbridge.AliasInfo{
 {
 Type: stringRef("aws:s3/bucket:Bucket"),
 },
 },
 },

This ensures that when a user changes from aws.s3.Bucket to aws.s3.BucketV2 that the Pulumi engine understands that the bucket is not a new bucket but is an existing bucket that has changed. Therefore, the Pulumi engine will not destroy the bucket and recreate it. This means that to change from 1 codebase to the other, we just need to keep the names of the resources the same (e.g s3-website-bucket in the code above) and then continue to add the new code to satisfy the way the resources need to be handed (e.g. BucketWebsiteConfigurationV2 in the code above).

You can see the steps on how this works as follows:

Notice that the bucket is not recreate - the only change is the addition of the new aws s3 resource for the website configuration. Your existing infrastructure is not going to change and thus we can easily migrate between the different resource types.

Get Started Today!

You can browse our API reference docs with inline examples or explore the Pulumi AWS SDKs repository to get started today!

AWS Lambda Functions Powered by AWS Graviton2 Processors

Paul Stack — Wed, 29 Sep 2021 00:00:00 +0000

In late 2018, AWS launched their first EC2 instances powered by ARM-based AWS Graviton Processors. These instances had been optimized for performance and cost. Since that initial launch, Amazon has continued to innovate in the Graviton space. In June 2021, they launched the Graviton Challenge for users to move their applications to AWS Graviton2. AWS Graviton2 processor instance types are up to 20% lower cost than x86 based instance types and see up to 40% better price performance.

Today, AWS launched Graviton2 support for AWS Lambda Functions, which means that your Lambda Functions can now execute on Graviton2 processors and take advantage of the improved price performance of that architecture.

Deploying a Lambda Function with Graviton2 Support

Let’s use Pulumi to deploy a new AWS Lambda Function using a Graviton2 architecture. First, let’s create the IAM role and policies required to create a Lambda Function:

import * as aws from "@pulumi/aws";

const role = new aws.iam.Role("lambda-role", {
 assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal({
 Service: "lambda.amazonaws.com"
 }),
});

const fullAccess = new aws.iam.RolePolicyAttachment("mylambda-access", {
 role: role,
 policyArn: aws.iam.ManagedPolicy.LambdaFullAccess,
});

import pulumi
import json
import pulumi_aws as aws

test_role = aws.iam.Role("lambda-role",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Sid": "",
 "Principal": {
 "Service": "lambda.amazonaws.com",
 },
 }],
 }))

full_access = aws.iam.RolePolicyAttachment("mylambda-access",
 role=role.name,
 policy_arn=aws.iam.ManagedPolicy.LAMBDA_FULL_ACCESS)

package main

import (
 "encoding/json"

 "github.com/pulumi/pulumi-aws/sdk/v4/go/aws/iam"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // create IAM role and policy wiring
 assumeRolePolicyJSON, err := json.Marshal(map[string]interface{}{
 "Version": "2012-10-17",
 "Statement": []interface{}{
 map[string]interface{}{
 "Action": "sts:AssumeRole",
 "Principal": map[string]interface{}{
 "Service": "lambda.amazonaws.com",
 },
 "Effect": "Allow",
 },
 },
 })
 if err != nil {
 return err
 }
 role, err := iam.NewRole(ctx, "lambda-role", &iam.RoleArgs{
 AssumeRolePolicy: pulumi.String(assumeRolePolicyJSON),
 })
 if err != nil {
 return err
 }

 _, err = iam.NewRolePolicyAttachment(ctx, "mylambda-access", &iam.RolePolicyAttachmentArgs{
 Role: role.Name,
 PolicyArn: pulumi.String("arn:aws:iam::aws:policy/AWSLambda_FullAccess"),
 })
 if err != nil {
 return err
 }

 return nil
 })
}

using Pulumi;
using Pulumi.Aws.Iam;
using Pulumi.Aws.Lambda;

class LambdaStack : Stack
{
 public LambdaStack()
 {
 var role = new Role("lambda-role", new RoleArgs
 {
 AssumeRolePolicy =
 @"{
 ""Version"": ""2012-10-17"",
 ""Statement"": [
 {
 ""Action"": ""sts:AssumeRole"",
 ""Principal"": {
 ""Service"": ""lambda.amazonaws.com""
 },
 ""Effect"": ""Allow"",
 ""Sid"": """"
 }
 ]
 }"
 });

 var fullAccess = new Aws.Iam.RolePolicyAttachment("mylambda-access", new Aws.Iam.RolePolicyAttachmentArgs
 {
 Role = role.Name,
 PolicyArn = "arn:aws:iam::aws:policy/AWSLambda_FullAccess",
 });
 }
}

This IAM role can then be used when deploying the Lambda Function:

const lambda = new aws.lambda.Function("mylambda", {
 code: new pulumi.asset.AssetArchive({
 ".": new pulumi.asset.FileArchive("./app"),
 }),
 role: role.arn,
 handler: "index.handler",
 runtime: aws.lambda.NodeJS12dXRuntime,
 architectures: ["arm64"],
});

function = aws.lambda_.Function("mylambda",
 role=role.arn,
 runtime="nodejs12.x",
 handler="index.handler",
 code=pulumi.AssetArchive({
 '.': pulumi.FileArchive('./app')
 }),
 architectures=["arm64"],
 )

function, err := lambda.NewFunction(ctx, "mylambda", &lambda.FunctionArgs{
 Handler: pulumi.String("index.handler"),
 Role: role.Arn,
 Runtime: pulumi.String("nodejs12.x"),
 Code: pulumi.NewFileArchive("./app"),
 Architectures: pulumi.StringArray{pulumi.String("arm64")},
})
if err != nil {
 return nil
}

 var lambda = new Function("mylambda", new FunctionArgs
 {
 Runtime = "nodejs12.x",
 Code = new FileArchive("./app"),
 Handler = "index.handler",
 Role = role.Arn
 Architectures = {"arm64"}
 });

The only new thing we need to do to deploy our Lambda function to Graviton2-based infrastructure is to provide the architectures input, specifying a value of arm64. With that small change, our Lambda Function will execute on an Arm-based instance.

When we deploy this via Pulumi, we can see that the Function has been deployed with the correct architecture in AWS:

$ pulumi up
Previewing update (dev)

View Live: https://app.pulumi.com/stack72/lambda-graviton-function/dev/previews/ed6a2b10-d0a8-4c08-ba2d-c749f216db67

 Type Name Plan
 + pulumi:pulumi:Stack lambda-graviton-function-dev create
 + ├─ aws:iam:Role lambda-role create
 + ├─ aws:iam:RolePolicyAttachment mylambda-access create
 + └─ aws:lambda:Function mylambda create

Resources:
 + 4 to create

Do you want to perform this update? yes
Updating (dev)

View Live: https://app.pulumi.com/stack72/lambda-graviton-function/dev/updates/1

 Type Name Status
 + pulumi:pulumi:Stack lambda-graviton-function-dev created
 + ├─ aws:iam:Role lambda-role created
 + ├─ aws:lambda:Function mylambda created
 + └─ aws:iam:RolePolicyAttachment mylambda-access created

Resources:
 + 4 created

Duration: 25s

If the architectures input is not specified, AWS will set the value by default to x86_64.

Since we are running our application code on ARM, we need to ensure that the code in the archive we use for our Lambda function can run natively on the ARM-based infrastructure. For interpreted languages like Node.js and Python, most applications will work as is. For compiled languages like Go, we need to ensure that our Go code is cross-compiled for GOARCH=arm64, and that the resulting ARM64 binary is included in our Lambda source archive.

Creating a Lambda Layer with Graviton2 Support

To create a Lambda Layer with Graviton2 support, we can extend our example above. Firstly, let’s create a lambda layer:

const layer = new aws.lambda.LayerVersion("my-layer-version", {
 code: new pulumi.asset.AssetArchive({
 ".": new pulumi.asset.FileArchive("./base-layer"),
 }),
 compatibleRuntimes: [
 aws.lambda.Runtime.NodeJS14dX,
 ],
 layerName: "my-base-layer",
 compatibleArchitectures: ["arm64", "x86_64"]
});

layer = aws.lambda_.LayerVersion("my-layer-version",
 compatible_runtimes=["nodejs12.x"],
 code=pulumi.FileArchive("./base-layer"),
 layer_name="my-base-layer",
 compatible_architectures=["arm64", "x86_64"])

layer, err := lambda.NewLayerVersion(ctx, "my-layer-version", &lambda.LayerVersionArgs{
 LayerName: pulumi.String("my-base-layer"),
 Code: pulumi.NewFileArchive("./base-layer"),
 CompatibleRuntimes: pulumi.StringArray{pulumi.String("nodejs12.x")},
 CompatibleArchitectures: pulumi.StringArray{pulumi.String("nodejs12.x")},
})
if err != nil {
 return nil
}

 var layer = new LayerVersion("my-layer-version", new LayerVersionArgs
 {
 CompatibleRuntimes =
 {
 "nodejs12.x",
 },
 CompatibleArchitectures =
 {
 "arm64",
 "x86_64",
 },
 Code = new FileArchive("./base-layer"),
 LayerName = "my-base-layer",
 });

This will create a Lambda layer that is compatible with NodeJS 12.x and ARM64 & X86_64. Layers can be created that are compatible with ARM64, X86_64 or both architectures. We can then build a new lambda based on this Lambda layer:

const lambda = new aws.lambda.Function("mylambda", {
 code: new pulumi.asset.AssetArchive({
 ".": new pulumi.asset.FileArchive("./app"),
 }),
 role: role.arn,
 handler: "index.handler",
 runtime: aws.lambda.NodeJS12dXRuntime,
 architectures: ["arm64"],
 layers: [layer.arn]
});

function = aws.lambda_.Function("mylambda",
 role=role.arn,
 runtime="nodejs12.x",
 handler="index.handler",
 code=pulumi.AssetArchive({
 '.': pulumi.FileArchive('./app')
 }),
 architectures=["arm64"],
 layers=[layer.arn]
 )

function, err := lambda.NewFunction(ctx, "mylambda", &lambda.FunctionArgs{
 Handler: pulumi.String("index.handler"),
 Role: role.Arn,
 Runtime: pulumi.String("nodejs12.x"),
 Code: pulumi.NewFileArchive("./app"),
 Architectures: pulumi.StringArray{pulumi.String("arm64")},
 Layers: pulumi.StringArray{layer.Arn},
})
if err != nil {
 return nil
}

 var lambda = new Function("mylambda", new FunctionArgs
 {
 Runtime = "nodejs12.x",
 Code = new FileArchive("./app"),
 Handler = "index.handler",
 Role = role.Arn
 Architectures = {"arm64"}
 Layers =
 {
 lambdaLayer.Arn,
 },
 });

This will deploy a function built on top of the Lambda layer targeting the Graviton2 architecture and NodeJS 12.x runtime.

Summary

AWS support for Lambda Functions powered by AWS Graviton2 offers users the ability to get better price performance for their workloads. We’ve shown how you can build your Functions using this ability. You can read more about AWS Lambda functions on the AWS Documentation website and getting started with AWS Graviton on GitHub.

Supercharging our GitHub Action with the Automation API

Paul Stack — Mon, 01 Mar 2021 00:00:00 +0000

Guest Article: Simen A. W. Olsen is a Software Architect and Manager at Bjerk, a software development agency based in Oslo, Norway. He joins Paul Stack to talk about the new GitHub Action powered by the Pulumi Automation API.

In this article, we’ll take a look at how our GitHub Action has been re-architected to be built on top of our Automation API.

As a GitHub Actions launch partner in 2018, we released our original GitHub Action, built as a Docker container action. This Action packaged all the prerequisites with the Pulumi CLI to let users run Pulumi commands. As the Pulumi ecosystem has grown, so too did the container run in GitHub Actions.

Since then, GitHub released an alternative way to create a GitHub Action by writing JavaScript. At Pulumi, and in our community, we love expressing everything we possibly can in code, so the new JavaScript support got us quite excited about the “better together” possibilities! Enter Pulumi’s new Automation API, a new feature we launched in public preview in October 2020. This new capability lets you build infrastructure as code—in pure code—without even needing to run a CLI. This sounded like a perfect match for a JavaScript Action. Creating a JavaScript Action allows us to programmatically invoke the right Pulumi commands in response to actions, without needing to awkwardly script a CLI. Instead, we were able to leverage the Automation API to fundamentally rethink our GitHub Action.

Under the Hood

Let’s take a look at the new JavaScript-based GitHub Action and see how it uses Automation API to execute Pulumi:

const main = async () => {
 const config = await makeConfig();

 if (environmentVariables.PULUMI_ACCESS_TOKEN !== '') {
 await pulumiCli.run('login');
 } else if (config.cloudUrl) {
 await pulumiCli.run('login', config.cloudUrl);
 }

 const workDir = resolve(
 environmentVariables.GITHUB_WORKSPACE,
 config.workDir,
 );

 const stack = await LocalWorkspace.selectStack({
 stackName: config.stackName,
 workDir: workDir,
 });

 const onOutput = (msg: string) => {
 .....
 };

 const actions: Record<Commands, () => Promise<string>> = {
 up: () => stack.up({ onOutput }).then((r) => r.stdout),
 refresh: () => stack.refresh({ onOutput }).then((r) => r.stdout),
 destroy: () => stack.destroy({ onOutput }).then((r) => r.stdout),
 preview: async () => {
 const { stdout, stderr } = await stack.preview();
 onOutput(stdout);
 onOutput(stderr);
 return stdout;
 },
 };

 const output = await actions[config.command]();

 const outputs = await stack.outputs();

 for (const [outKey, outExport] of Object.entries(outputs)) {
 core.setOutput(outKey, outExport.value);
 if (outExport.secret) {
 core.setSecret(outExport.value);
 }
 }
};

The Action code is clean and concise because the power lies in the Automation API itself. The Action uses the Automation API to drive Pulumi as part of your GitHub Actions workflows. This means that your workflows can take advantage of changes to Pulumi and the Automation API as soon as they’re released! And, of course, you can always write your own custom logic using the Automation API like our Action does.

Customizing Your Environment

The new Action puts you in control of the environment that it runs on. By comparison, the previous Docker-based Action bundled specific tool versions, which left your CI pipeline at the mercy of our opinionated container. Now, if you want to pin to specific versions of NodeJS / Python / Go / .NET or even Pulumi, you have that power!

You can even include a matrix strategy to test your Pulumi code against different language runtime versions and different operating systems:

name: 'Sample Pulumi NodeJS Application'
on:
 pull_request:

jobs:
 test-workflow:
 runs-on: ${{ matrix.os }}
 name: Testing Pulumi Preview on ${{ matrix.os }} with NodeJS ${{ matrix.node_version }}
 strategy:
 matrix:
 os: [ubuntu-latest, macos-latest, windows-latest]
 node_version: [10.x, 12.x, 14.x]
 fail-fast: false
 steps:
 - uses: actions/checkout@v2
 - uses: actions/setup-node@v1
 with:
 node-version: ${{ matrix.node_version }}
 - name: Install pulumi
 uses: pulumi/action-install-pulumi-cli@v1.0.1
 - run: npm install
 - uses: pulumi/actions@v2
 env:
 PULUMI_CONFIG_PASSPHRASE: not-a-secret
 with:
 command: preview
 cloud-url: file://~
 stack-name: dev

Cloud Interaction

The previous Docker-based Action would look for the GOOGLE_CREDENTIALS environment variable; if your workflow set that variable, the Action would run the following commands on your behalf:

gcloud auth activate-service-account --key-file=$GOOGLE_APPLICATION_CREDENTIALS
gcloud --quiet auth configure-docker $GOOGLE_DOCKER_HOSTNAME_LIST

With the new Action, you’re fully in control of authentication to cloud providers. To authenticate with AWS / Azure / Google Cloud, you can now use their official Actions:

 - name: Configure AWS Credentials
 uses: aws-actions/configure-aws-credentials@v1
 with:
 aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
 aws-region: ${{ secrets.AWS_REGION }}
 aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

 - uses: azure/login@v1
 with:
 creds: ${{ secrets.AZURE_CREDENTIALS }}

 - uses: google-github-actions/setup-gcloud@v0
 with:
 service_account_key: ${{ secrets.GCP_KEY }}
 project_id: ${{ env.PROJECT_ID }}
 export_default_credentials: true

We recommend that you store all sensitive environment variables, like cloud credentials, in the Actions secret storage and consume them using the secrets attribute on your workflow’s action.

Faster

The new GitHub Action is much smaller and no longer needs to download a (very!) large Docker container. In the Docker-based Action, we frequently saw workflows take up to 3 minutes to download the container. The new Action is much faster; let’s take a look:

Old Docker-based Action (v1)

New JavaScript-based Action, powered by Automation API (v2)

The new Pulumi GitHub Action is ready for you to use. Here’s how to get started:

Already use the v1 Pulumi GitHub Action? Migrate to the new one with our migration guide.
New to Pulumi and GitHub Actions? Use our getting started guide to set up your first Pulumi workflow.

We’re eager to hear your feedback on this new Action. If you haven’t already signed up for our Community Slack, it’s quick and easy! You can join in on conversations you like and get help from other community members, as well as the Pulumi Team. That’s it for now!

Infrastructure CI/CD With Github Actions and Pulumi

Paul Stack — Mon, 14 Dec 2020 00:00:00 +0000

GitHub Actions help automate tasks within your software development life cycle. At Pulumi, we use GitHub Actions internally as part of every build/release cycle, and we run these tasks many, many times per day. This helps us to automate our CI/CD process and eliminate manual steps. Pulumi and many of our customers also deliver infrastructure resources as part of a CI/CD process: delivering infrastructure and applications in a single, integrated pipeline.

Better Automation with GitHub Actions and Pulumi

Today, we’re pleased to announce that Pulumi is available as a verified GitHub Action in the GitHub Marketplace. It’s now easier than ever to start automating your infrastructure deployments with Pulumi and GitHub Actions. Our new guide shows you how to get started with GitHub Actions and Pulumi. Here are just a few great examples of how to use the Pulumi GitHub Action that will simplify your cloud engineering tasks:

Master Builds

name: Pulumi
on:
 push:
 branches:
 - master
jobs:
 up:
 name: Update
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v1
 - uses: pulumi/actions@v1
 with:
 command: up
 env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}

Pull Request Builds

name: Pulumi
on:
 - pull_request
jobs:
 preview:
 name: Preview
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v2
 - uses: pulumi/actions@v1
 with:
 command: preview
 env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
 GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 IS_PR_WORKFLOW: true
 COMMENT_ON_PR: 1

Announcing the New Pulumi CI/CD Integration Assistant

We often hear from developers who want to setup CI/CD automation for their stacks but don’t have the time to figure out how to configure a working pipeline. Today, we are happy to announce the launch of the CI/CD Integration Assistant to make setting up CI/CD automation more straightforward. You can easily get started using popular CI/CD services without the hassle of the initial setup.

The integration assistant guides you with instructions based on your preferred CI/CD service and provides the customized pipeline configuration for CI/CD services that you can add to your Version Control System (VCS) repository.

We are super excited to offer GitHub Actions support as part of the CI/CD Integration Assistant and other popular tools like Azure Pipelines and more. If you’d like to request support for more CI/CD services, please use this convenient template on the Pulumi GitHub repo.

Managing Your Infrastructure with a VCS

You may already be familiar with Git-based services such as GitHub, Bitbucket, and GitLab. Adding your code to a central VCS allows you to secure your codebase against failures and boost team collaboration. When you add an automated CI/CD pipeline, your team will deliver software safely and efficiently by ensuring that automated tests are passed prior to every deployment. This process is valuable for application code and infrastructure code.

The Assistant also goes beyond helping you get started with an automated pipeline. It works throughout the Pulumi Service providing hints for configuring a VCS for projects that are not already configured. It can also help identify updates to stacks that were not run in an automated pipeline (in the activity tab). These additions will help you quickly identify the stacks where best practices are not followed and take corrective actions. You can access the Assistant in the Integrations option of your stack’s settings page to help you through the whole process of configuring a VCS to setting up a working automated pipeline.

Learn more about the CI/CD Integration Assistant in our user guide.

We are eager to hear your feedback on how we can improve your experience with this feature. If you haven’t already signed up for our Community Slack, it’s quick and easy! You can join in on conversations you like and get help from other community members, as well as the Pulumi Team. That’s it for now! 👋

Pulumi container images now available on Amazon ECR Public

Paul Stack — Tue, 01 Dec 2020 00:00:00 +0000

At re:Invent, the AWS team unveiled the new Amazon Elastic Container Registry Public (Amazon ECR Public), creating a new option for users in publishing and pulling public container images. Pulumi fully supports Amazon ECR Public in two ways:

Official Pulumi container images are available today on Amazon ECR Public.
Pulumi is the easiest way to package and publish your container images, and we’ll support publishing your container images to Amazon ECR Public very soon.

If you want to learn more about Pulumi and building resources in AWS, join one of our upcoming workshops.

Pulumi publishes official container images for running Pulumi along with the desired Pulumi programming language runtimes. These containers are used directly by developers as the foundation for process automation, such as GitHub Actions; in custom container environments with Pulumi’s Automation API; and in Kubernetes-based solutions, like the Pulumi Kubernetes Operator. These images are publicly available via Docker Hub and will continue to be available there.

With the unveiling of Amazon ECR Public, we want to make sure Pulumi’s official container images are always available to ensure that our users can ultimately choose the right container image that suits their needs. Going forward, Pulumi’s official container images will be available on Amazon ECR Public.

We’ve published our Pulumi container images in this new repository, and we plan to publish to both the Docker Hub and Amazon ECR Public repositories for all new Pulumi releases. You can browse Pulumi images in the Amazon ECR gallery here: https://gallery.ecr.aws/?searchTerm=pulumi.

Try it out!

Here’s how easy it is to grab the latest Pulumi container images from the Amazon ECR Public repository:

$ docker pull public.ecr.aws/pulumi/pulumi:latest

To grab the latest language runtime specific containers, run the following commands:

Python:

$ docker pull public.ecr.aws/pulumi/pulumi-python:latest

Go:

$ docker pull public.ecr.aws/pulumi/pulumi-go:latest

TypeScript / JavaScript:

$ docker pull public.ecr.aws/pulumi/pulumi-nodejs:latest

DotNet:

$ docker pull public.ecr.aws/pulumi/pulumi-dotnet:latest

Finally, you can grab the latest official image for the Pulumi Kubernetes Operator with this command:

$ docker pull public.ecr.aws/pulumi/pulumi-kubernetes-operator:latest

Pulumi Import: Generate IaC for Existing Cloud Resources

Paul Stack — Tue, 24 Nov 2020 00:00:00 +0000

Most infrastructure projects require working with existing cloud resources, either by building on top of existing resources or adopting existing resources under management with a new and more robust infrastructure provisioning solution.

In June 2019, Pulumi introduced the ability to import existing infrastructure resources to be under Pulumi management no matter how you’ve provisioned these resources — manually in your cloud provider’s console or CLI, using an infrastructure as code tool like Terraform or AWS CloudFormation. Today, we are happy to announce a richer resource import experience.

As of v2.12.0, Pulumi has introduced a pulumi import command. This command will import the cloud resource into the Pulumi state and generate the code for the user’s Pulumi program in the appropriate language.

The Anatomy of the Import Command

No matter how resources have been provisioned in the cloud, Pulumi enables you to adopt and manage those resources as part of your infrastructure project. The pulumi import command is made of the following:

pulumi import [type] [name] [id]

When Pulumi performs an import, the resource is added to the Pulumi state, it is marked as a protected resource (by default), and it will emit the generated code that the user can add to their program before running a pulumi up. The resources are marked as protected to ensure that imported infrastructure is not accidentally deleted if the user forgets to include the code for the resource in their program before doing a deployment.

Importing an S3 Bucket

A resource needs to be imported into an existing Pulumi project. Let’s consider that a user wants to import an S3 bucket manually created in their AWS Account. To do this, we can execute the following command:

pulumi import aws:s3/bucket:Bucket infra-logs company-infra-logs

This command instructs Pulumi to import the S3 bucket called company-infra-logs with the Pulumi resource name infra-logs. The import command will output as follows:

The user can then see the generated code they need to use as part of their application. I imported this resource into a TypeScript application:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const demo = new aws.s3.Bucket("infra-logs", {
 acl: "private",
 bucket: "company-infra-logs",
 forceDestroy: false,
}, {
 protect: true,
});

import pulumi
import pulumi_aws as aws

demo = aws.s3.Bucket("infra-logs",
 acl="private",
 bucket="company-infra-logs",
 force_destroy=False,
 opts=ResourceOptions(protect=True))

package main

import (
 "github.com/pulumi/pulumi-aws/sdk/v3/go/aws/s3"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err := s3.NewBucket(ctx, "infra-logs", &s3.BucketArgs{
 Acl: pulumi.String("private"),
 Bucket: pulumi.String("company-infra-logs"),
 ForceDestroy: pulumi.Bool(false),
 }, pulumi.Protect(true))
 if err != nil {
 return err
 }
 return nil
 })
}

using Pulumi;
using Aws = Pulumi.Aws;

class MyStack : Stack
{
 public MyStack()
 {
 var demo = new Aws.S3.Bucket("infra-logs", new Aws.S3.BucketArgs
 {
 Acl = "private",
 Bucket = "company-infra-logs",
 ForceDestroy = false,
 }, new CustomResourceOptions
 {
 Protect = true,
 });
 }

}

This is valid Pulumi application code and can be added to your Pulumi application. Notice that the Pulumi resource is marked as protect: true. This means Pulumi will not delete that resource unless this protection attribute is removed from the resource. If this code is not added to your Pulumi application, then a Pulumi update will return the following:

Diagnostics:
 pulumi:pulumi:Stack (import-example-dev):
 error: preview failed

 aws:s3:Bucket (demo):
 error: Preview failed: refusing to delete protected resource 'urn:pulumi:dev::import-example::aws:s3/bucket:Bucket::infra-logs'

Importing Multiple Resources

Should a user need to import multiple resources to be managed by Pulumi, the CLI import command can handle this scenario for us. This is helpful when using the pulumi import command as part of scripting larger bulk imports of cloud resource:

{
 "resources": [{
 "type": "aws:ec2/vpc:Vpc",
 "name": "application-vpc",
 "id": "vpc-0ad77710973388316"
 },
 {
 "type": "aws:ec2/subnet:Subnet",
 "name": "public-1",
 "id": "subnet-0fb5fdff92b9e5a3b"
 },
 {
 "type": "aws:ec2/subnet:Subnet",
 "name": "private-1",
 "id": "subnet-0a39d25dd9f7b7808"
 }
 ]
}

We can then run the command:

pulumi import -f resources.json

We can see the resultant code generation:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const application_vpc = new aws.ec2.Vpc("application-vpc", {
 assignGeneratedIpv6CidrBlock: false,
 cidrBlock: "172.16.0.0/16",
 enableDnsSupport: true,
 instanceTenancy: "default",
 tags: {
 Name: "pulumi-vpc",
 Owner: "pulumi",
 Project: "pulumi-k8s-aws-cluster",
 },
}, {
 protect: true,
});
const public_1 = new aws.ec2.Subnet("public-1", {
 assignIpv6AddressOnCreation: false,
 cidrBlock: "172.16.32.0/19",
 mapPublicIpOnLaunch: true,
 tags: {
 Name: "pulumi-vpc-public-1",
 Owner: "pulumi",
 Project: "pulumi-k8s-aws-cluster",
 "kubernetes.io/role/elb": "1",
 type: "public",
 },
 vpcId: "vpc-0ad77710973388316",
}, {
 protect: true,
});
const private_1 = new aws.ec2.Subnet("private-1", {
 assignIpv6AddressOnCreation: false,
 cidrBlock: "172.16.160.0/19",
 mapPublicIpOnLaunch: false,
 tags: {
 Name: "pulumi-vpc-private-1",
 Owner: "pulumi",
 Project: "pulumi-k8s-aws-cluster",
 "kubernetes.io/role/internal-elb": "1",
 type: "private",
 },
 vpcId: "vpc-0ad77710973388316",
}, {
 protect: true,
});

import pulumi
import pulumi_aws as aws

application_vpc = aws.ec2.Vpc("application-vpc",
 assign_generated_ipv6_cidr_block=False,
 cidr_block="172.16.0.0/16",
 enable_dns_support=True,
 instance_tenancy="default",
 tags={
 "Name": "pulumi-vpc",
 "Owner": "pulumi",
 "Project": "pulumi-k8s-aws-cluster",
 },
 opts=ResourceOptions(protect=True))
public_1 = aws.ec2.Subnet("public-1",
 assign_ipv6_address_on_creation=False,
 cidr_block="172.16.32.0/19",
 map_public_ip_on_launch=True,
 tags={
 "Name": "pulumi-vpc-public-1",
 "Owner": "pulumi",
 "Project": "pulumi-k8s-aws-cluster",
 "kubernetes.io/role/elb": "1",
 "type": "public",
 },
 vpc_id="vpc-0ad77710973388316",
 opts=ResourceOptions(protect=True))
private_1 = aws.ec2.Subnet("private-1",
 assign_ipv6_address_on_creation=False,
 cidr_block="172.16.160.0/19",
 map_public_ip_on_launch=False,
 tags={
 "Name": "pulumi-vpc-private-1",
 "Owner": "pulumi",
 "Project": "pulumi-k8s-aws-cluster",
 "kubernetes.io/role/internal-elb": "1",
 "type": "private",
 },
 vpc_id="vpc-0ad77710973388316",
 opts=ResourceOptions(protect=True))

package main

import (
 "github.com/pulumi/pulumi-aws/sdk/v3/go/aws/ec2"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err := ec2.NewVpc(ctx, "application_vpc", &ec2.VpcArgs{
 AssignGeneratedIpv6CidrBlock: pulumi.Bool(false),
 CidrBlock: pulumi.String("172.16.0.0/16"),
 EnableDnsSupport: pulumi.Bool(true),
 InstanceTenancy: pulumi.String("default"),
 Tags: pulumi.StringMap{
 "Name": pulumi.String("pulumi-vpc"),
 "Owner": pulumi.String("pulumi"),
 "Project": pulumi.String("pulumi-k8s-aws-cluster"),
 },
 }, pulumi.Protect(true))
 if err != nil {
 return err
 }
 _, err = ec2.NewSubnet(ctx, "public_1", &ec2.SubnetArgs{
 AssignIpv6AddressOnCreation: pulumi.Bool(false),
 CidrBlock: pulumi.String("172.16.32.0/19"),
 MapPublicIpOnLaunch: pulumi.Bool(true),
 Tags: pulumi.StringMap{
 "Name": pulumi.String("pulumi-vpc-public-1"),
 "Owner": pulumi.String("pulumi"),
 "Project": pulumi.String("pulumi-k8s-aws-cluster"),
 "kubernetes.io/role/elb": pulumi.String("1"),
 "type": pulumi.String("public"),
 },
 VpcId: pulumi.String("vpc-0ad77710973388316"),
 }, pulumi.Protect(true))
 if err != nil {
 return err
 }
 _, err = ec2.NewSubnet(ctx, "private_1", &ec2.SubnetArgs{
 AssignIpv6AddressOnCreation: pulumi.Bool(false),
 CidrBlock: pulumi.String("172.16.160.0/19"),
 MapPublicIpOnLaunch: pulumi.Bool(false),
 Tags: pulumi.StringMap{
 "Name": pulumi.String("pulumi-vpc-private-1"),
 "Owner": pulumi.String("pulumi"),
 "Project": pulumi.String("pulumi-k8s-aws-cluster"),
 "kubernetes.io/role/internal-elb": pulumi.String("1"),
 "type": pulumi.String("private"),
 },
 VpcId: pulumi.String("vpc-0ad77710973388316"),
 }, pulumi.Protect(true))
 if err != nil {
 return err
 }
 return nil
 })
}

using Pulumi;
using Aws = Pulumi.Aws;

class MyStack : Stack
{
 public MyStack()
 {
 var application_vpc = new Aws.Ec2.Vpc("application-vpc", new Aws.Ec2.VpcArgs
 {
 AssignGeneratedIpv6CidrBlock = false,
 CidrBlock = "172.16.0.0/16",
 EnableDnsSupport = true,
 InstanceTenancy = "default",
 Tags =
 {
 { "Name", "pulumi-vpc" },
 { "Owner", "pulumi" },
 { "Project", "pulumi-k8s-aws-cluster" },
 },
 }, new CustomResourceOptions
 {
 Protect = true,
 });
 var public_1 = new Aws.Ec2.Subnet("public-1", new Aws.Ec2.SubnetArgs
 {
 AssignIpv6AddressOnCreation = false,
 CidrBlock = "172.16.32.0/19",
 MapPublicIpOnLaunch = true,
 Tags =
 {
 { "Name", "pulumi-vpc-public-1" },
 { "Owner", "pulumi" },
 { "Project", "pulumi-k8s-aws-cluster" },
 { "kubernetes.io/role/elb", "1" },
 { "type", "public" },
 },
 VpcId = "vpc-0ad77710973388316",
 }, new CustomResourceOptions
 {
 Protect = true,
 });
 var private_1 = new Aws.Ec2.Subnet("private-1", new Aws.Ec2.SubnetArgs
 {
 AssignIpv6AddressOnCreation = false,
 CidrBlock = "172.16.160.0/19",
 MapPublicIpOnLaunch = false,
 Tags =
 {
 { "Name", "pulumi-vpc-private-1" },
 { "Owner", "pulumi" },
 { "Project", "pulumi-k8s-aws-cluster" },
 { "kubernetes.io/role/internal-elb", "1" },
 { "type", "private" },
 },
 VpcId = "vpc-0ad77710973388316",
 }, new CustomResourceOptions
 {
 Protect = true,
 });
 }

}

Check out the video clip below for a demo.

Documentation

Full CLI Documentation is available, showing all of the different import command flags. The flags include writing directly to a file and the ability to ensure a resource is not protected on import. Examples of importing resources can be found on their specific resource documentation pages. Such an example for importing a VPC can be found in the VPC import documentation.

Improved Pulumi Previews

Paul Stack — Thu, 29 Oct 2020 00:00:00 +0000

Today we are announcing a minor but significant improvement to the Pulumi preview experience.

Change in Behaviour

We can look at a sample Pulumi application that creates a Google GKE cluster. Pulumi then deploys an application into the Kubernetes cluster.

import * as gcp from "@pulumi/gcp";
import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";

const name = "helloworld";

const config = new pulumi.Config();
export const masterVersion = config.get("masterVersion") ||
 gcp.container.getEngineVersions().then(it => it.latestMasterVersion);

// Create a GKE cluster
const cluster = new gcp.container.Cluster(name, {
 // We can't create a cluster with no node pool defined, but we want to only use
 // separately managed node pools. So we create the smallest possible default
 // node pool and immediately delete it.
 initialNodeCount: 1,
 removeDefaultNodePool: true,

 minMasterVersion: masterVersion,
 resourceLabels: {
 "stack": pulumi.getStack()
 },
});

const nodePool = new gcp.container.NodePool(`primary-node-pool`, {
 cluster: cluster.name,
 initialNodeCount: 2,
 location: cluster.location,
 nodeConfig: {
 preemptible: true,
 machineType: "n1-standard-1",
 oauthScopes: [
 "https://www.googleapis.com/auth/compute",
 "https://www.googleapis.com/auth/devstorage.read_only",
 "https://www.googleapis.com/auth/logging.write",
 "https://www.googleapis.com/auth/monitoring",
 ],
 },
 version: masterVersion,
 management: {
 autoRepair: true,
 },
}, {
 dependsOn: [cluster],
});

const clusterName = cluster.name;

// Manufacture a GKE-style kubeconfig. Note that this is slightly "different"
// because of the way GKE requires gcloud to be in the picture for cluster
// authentication (rather than using the client cert/key directly).
export const kubeconfig = pulumi.
 all([ cluster.name, cluster.endpoint, cluster.masterAuth ]).
 apply(([ name, endpoint, masterAuth ]) => {
 const context = `${gcp.config.project}_${gcp.config.zone}_${name}`;
 return `apiVersion: v1
clusters:
- cluster:
 certificate-authority-data: ${masterAuth.clusterCaCertificate}
 server: https://${endpoint}
 name: ${context}
contexts:
- context:
 cluster: ${context}
 user: ${context}
 name: ${context}
current-context: ${context}
kind: Config
preferences: {}
users:
- name: ${context}
 user:
 auth-provider:
 config:
 cmd-args: config config-helper --format=json
 cmd-path: gcloud
 expiry-key: '{.credential.token_expiry}'
 token-key: '{.credential.access_token}'
 name: gcp
`;
 });

// Create a Kubernetes provider instance that uses our cluster from above.
const clusterProvider = new k8s.Provider(name, {
 kubeconfig: kubeconfig,
}, {
 dependsOn: [nodePool],
});

// Create a Kubernetes Namespace
const ns = new k8s.core.v1.Namespace(name, {}, { provider: clusterProvider });
const namespaceName = ns.metadata.name;

// Create a NGINX Deployment
const appLabels = { appClass: name };
const deployment = new k8s.apps.v1.Deployment(name,
 {
 metadata: {
 namespace: namespaceName,
 labels: appLabels,
 },
 spec: {
 replicas: 1,
 selector: { matchLabels: appLabels },
 template: {
 metadata: {
 labels: appLabels,
 },
 spec: {
 containers: [
 {
 name: name,
 image: "nginx:latest",
 ports: [{ name: "http", containerPort: 80 }],
 },
 ],
 },
 },
 },
 },
 {
 provider: clusterProvider,
 },
);

// Create a LoadBalancer Service for the NGINX Deployment
const service = new k8s.core.v1.Service(name,
 {
 metadata: {
 labels: appLabels,
 namespace: namespaceName,
 },
 spec: {
 type: "LoadBalancer",
 ports: [{ port: 80, targetPort: "http" }],
 selector: appLabels,
 },
 },
 {
 provider: clusterProvider,
 },
);

A simple change to the labels for the GKE cluster led to a preview that suggested that the dependent resources would need to be replaced.

 Type Name Plan Info
pulumi:pulumi:Stack gcp-ts-gke-hello-world-dev2
~ ├─ gcp:container:Cluster helloworld update [diff: ~resourceLabels]
+- ├─ gcp:container:NodePool primary-node-pool replace [diff: ~location]
+- ├─ pulumi:providers:kubernetes helloworld replace [diff: ~kubeconfig]
+- ├─ kubernetes:core/v1:Namespace helloworld replace [diff: -metadata~provider]
+- ├─ kubernetes:core/v1:Service helloworld replace [diff: ~metadata,provider]
+- └─ kubernetes:apps/v1:Deployment helloworld replace [diff: ~metadata,provider]
Resources:
~ 1 to update
+-5 to replace
6 changes. 1 unchanged

The result of the Pulumi apply showed that only the gcp:container:Cluster changed, and no other Kubernetes resources were replaced. This is because Pulumi was trying to be conservative and let you know what “could” happen. It was calculating that changes to the cluster could potentially lead to the outputs changing which other resource rely on. In practice, that wasn’t going to happen. Pulumi previously had to be conservative for correctness because it didn’t have a way to check.

 Type Name Status Info
pulumi:pulumi:Stack gcp-ts-gke-hello-world-dev2
~ └─ gcp:container:Cluster helloworld updated [diff: ~resourceLabels]
Resources:
~ 1 updated
6 unchanged

Pulumi now supports asking the individual resource providers about what outputs will change as part of an update to a resource. This allows the providers (like the GCP provider in this case) to give more specific information, which allows Pulumi to provide more accurate previews:

 Type Name Plan Info
pulumi:pulumi:Stack gcp-ts-gke-hello-world-dev2
~ └─ gcp:container:Cluster helloworld update [diff: ~resourceLabels]
Resources:
~ 1 to update
6 unchanged

Conclusion

The improved preview is available in Pulumi v2.12.0 and is supported in all new provider releases. If you would like to disable this new behaviour and preserve the previous, more conservative behaviour, you can set an environment variable PULUMI_DISABLE_PROVIDER_PREVIEW to a truthy value (1 or true).

Rotating Secret Providers

Paul Stack — Wed, 05 Aug 2020 00:00:00 +0000

Customers and users have asked for the ability to change the secrets manager associated with their stacks. This would allow a user to rotate their secrets providers when people leave their organization or even to be able to migrate to another secret manager of their choice. The v2.8.0 release of Pulumi adds support for this specific feature.

Let’s have a look at how to change a secrets provider for an existing stack:

pulumi stack change-secrets-provider "awskms://alias/ChangeSecretsProvider?region=us-west-2"

This command will change the secrets provider for the currently selected stack. It will ensure that the configuration values in Pulumi.<stack>.yaml are re-encrypted with the new secrets provider and that the latest checkpoint (state file) also has any secrets held within are migrated with the new secrets provider.

Creating New Stacks based on existing stacks

When creating a new stack, we added the ability to create that new stack based on the configuration of an existing stack. This means that when you need to create a new developer stack or a new stack for a different environment, then you can create that stack as follows:

pulumi stack init test --copy-config-from dev

This flag creates a new test stack and initiates the default secrets provider for the stack. It will then get the existing configuration from the dev stack and re-encrypt it to the test stack’s default secret provider. We can, of course, pass a secrets provider to this new stack:

pulumi stack init test --secrets-provider="awskms://alias/ChangeSecretsProvider?region=us-west-2" --copy-config-from dev

Copying Config Between Stacks

When working on a Pulumi project that has multiple stacks, there are times that we create a configuration in one of the stacks that need to be copied to the other stacks in the project. Rather than selecting each stack individually, we can now programmatically copy the configuration between stacks. Let’s take an example where a new database password was added to the dev stack. We can copy that password to the production stack as follows:

pulumi config cp MyDatabasePassword --dest production

This ensures that the MyDatabasePassword configuration will be re-encrypted into the production stack. If there’s a situation where we want to copy all of the configuration between the dev and production stacks, then we omit the key (MyDatabasePassword):

pulumi config cp --dest production

We can iterate over all of the stacks that we need to copy the configuration to and the new configuration values will be encrypted as per the new stack.

Learn More

If you’d like to learn about Pulumi and manage your infrastructure through code, get started today. Pulumi is open source and free to use.

As always, you can check out our code on GitHub, follow us on Twitter, subscribe to our YouTube channel, or join our Community Slack channel if you have any questions, need support, or just want to say hello.

Controlling AWS Costs with Pulumi and AWS Lambda

Paul Stack — Thu, 09 Apr 2020 00:00:00 +0000

Due to the nature of the product we build, the Pulumi team needs to have access to several cloud providers to develop and test the product. An increasing number of cloud providers comes with an associated ever-increasing cost.

Giving a development team the ability to launch the resources they need in the cloud is both the best and most scary thing that happens in a company. The flexibility for them to be able to achieve their goal is fantastic, but with this great freedom can come problems. We depend on developers to clean up after themselves, but sometimes that doesn’t happen. Throw in automated testing failures that leak cloud resources, and we can very quickly get ourselves into a situation where costs spiral out of control.

There are multiple ways that Pulumi can help you deal with this situation. Joe Duffy wrote a blog post on Automatically Enforcing Tagging Policies to demonstrate how Policy as Code can ensure that tags are automatically added to any Pulumi managed infrastructure. If your company doesn’t use Pulumi to build and manage their infrastructure, then you can still use Pulumi in collaboration with AWS Lambda to build and deploy an application that can monitor your cloud costs.

To keep our cloud costs down, Pulumi uses an AWS Lambda Go application that removes wasteful resources from our AWS accounts daily. If a developer needs to protect their resources from being deleted, they need to add a tag to their infrastructure of “Owner: ”. This ensures the owner of a resource can be tracked at any time.

Cloud Cost Lambda

Here’s our application in it’s simplest form.

package main

import (
 "log"
 "os"
 "strconv"

 "github.com/aws/aws-lambda-go/lambda"
 "github.com/aws/aws-sdk-go-v2/aws/external"
)

var DeleteEnabled bool

func main() {
 lambda.Start(HandleRequest)
}

func HandleRequest() {
 destroyOperations, _ := os.LookupEnv("DESTROY_ENABLED")
 deleteEnabled, err := strconv.ParseBool(destroyOperations)
 if err != nil {
 log.Print("loading DESTROY_ENABLED envvar failed")
 log.Fatal(err)
 } else {
 DeleteEnabled = deleteEnabled
 }

 cfg, err := external.LoadDefaultAWSConfig()
 if err != nil {
 log.Print("loading AWS config failed")
 log.Fatal(err)
 }

 log.Printf("Running cleanup operations in %q", cfg.Region)
 log.Printf("Are destroy operations enabled? %q", destroyOperations)

 log.Print("Finished cleanup operations")
}

To deploy this application, we need to build and package the lambda. Packaging a Go lambda requires compiling it for Linux and then zipping the lambda up. We can do this as follows:

$ GOOS=linux GOARCH=amd64 go build -o main main.go && zip deployment.zip main

After creating a new Pulumi application, we can write the following code to deploy this lambda for us:

import * as aws from "@pulumi/aws";
import * as pulumi from "@pulumi/pulumi";

const lambdaRole = new aws.iam.Role("my-lambda-role", {
 name: "sample-role",
 assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal({ Service: "lambda.amazonaws.com" }),
});

const lambda = new aws.lambda.Function("my-lambda-function", {
 name: "sample-lambda",
 runtime: aws.lambda.Go1dxRuntime,
 timeout: 900,
 role: lambdaRole,
 handler: "main",
 memorySize: 1024,
 code: new pulumi.asset.FileArchive("deployment.zip"),
 environment: {
 variables: {
 "DESTROY_ENABLED": "true"
 }
 },
});

We can deploy the application using pulumi up:

$ pulumi up
Previewing update (dev):
 Type Name Plan
 + pulumi:pulumi:Stack lambda-post-dev create
 + ├─ aws:iam:Role my-lambda-role create
 + └─ aws:lambda:Function my-lambda-function create

Resources:
 + 3 to create

Do you want to perform this update? yes
Updating (dev):
 Type Name Status
 + pulumi:pulumi:Stack lambda-post-dev created
 + ├─ aws:iam:Role my-lambda-role created
 + └─ aws:lambda:Function my-lambda-function created

Resources:
 + 3 created

Duration: 27s

We can go to the AWS Lambda console and trigger the application. To do this, we must trigger a test event. Our application doesn’t accept any inputs, so the test event appears as follows:

When we trigger the lambda, we should get an output as follows:

START RequestId: 0910c342-4c43-4946-aeca-cf57683aa36b Version: $LATEST
2020/04/07 21:37:01 Running cleanup operations in "us-east-1"
2020/04/07 21:37:01 Are destroy operations enabled? "true"
2020/04/07 21:37:01 Finished cleanup operations
END RequestId: 0910c342-4c43-4946-aeca-cf57683aa36b
REPORT RequestId: 0910c342-4c43-4946-aeca-cf57683aa36b Duration: 1.96 ms Billed Duration: 100 ms Memory Size: 1024 MB Max Memory Used: 38 MB Init Duration: 99.32 ms

Our lambda application works!! But, it’s not going to delete anything because we haven’t told it what to delete. We can start very small. Let’s tell AWS to cleanup any EBS disks that are currently unattached.

func handleUnattachedVolumes(cfg aws.Config) {
 log.Print("Running cleanup operations for unattached ebs volumes")
 svc := ec2.New(cfg)

 input := &ec2.DescribeVolumesInput{
 Filters: []ec2.Filter{
 {
 Name: aws.String("status"),
 Values: []string{"available"},
 },
 },
 }
 volumes := make([]ec2.Volume, 0)

 for {
 req := svc.DescribeVolumesRequest(input)
 res, err := req.Send(context.Background())
 if err != nil {
 log.Fatal(err)
 }

 volumes = append(volumes, res.Volumes...)

 if res.NextToken == nil {
 break
 }

 input.NextToken = res.NextToken
 }

 for _, volume := range volumes {
 log.Printf("Killing EBS Volume %q as it's unattached", *volume.VolumeId)
 if DeleteEnabled {
 delReq := svc.DeleteVolumeRequest(&ec2.DeleteVolumeInput{
 VolumeId: volume.VolumeId,
 })
 _, err := delReq.Send(context.Background())
 if err != nil {
 log.Print(err)
 log.Printf("error deleting volume %q", *volume.VolumeId)
 }
 }
 }
}

We can then update our lambda handler to include this new EBS cleanup function:

func HandleRequest() {
 destroyOperations, _ := os.LookupEnv("DESTROY_ENABLED")
 deleteEnabled, err := strconv.ParseBool(destroyOperations)
 if err != nil {
 log.Print("loading DESTROY_ENABLED envvar failed")
 log.Fatal(err)
 } else {
 DeleteEnabled = deleteEnabled
 }

 cfg, err := external.LoadDefaultAWSConfig()
 if err != nil {
 log.Print("loading AWS config failed")
 log.Fatal(err)
 }

 log.Printf("Running cleanup operations in %q", cfg.Region)
 log.Printf("Are destroy operations enabled? %q", destroyOperations)

 handleUnattachedVolumes(cfg)

 log.Print("Finished cleanup operations")
}

When we run pulumi up, Pulumi will understand that the source code of the lambda has changed:

$ pulumi up
Previewing update (dev):
 Type Name Plan Info
 pulumi:pulumi:Stack lambda-post-dev
 ~ └─ aws:lambda:Function my-lambda-function update [diff: ~code]

Resources:
 ~ 1 to update
 2 unchanged

Do you want to perform this update? yes
Updating (dev):
 Type Name Status Info
 pulumi:pulumi:Stack lambda-post-dev
 ~ └─ aws:lambda:Function my-lambda-function updated [diff: ~code]

Resources:
 ~ 1 updated
 2 unchanged

Duration: 22s

When we execute the lambda code, we will receive an error now:

START RequestId: dc34a4f0-55f3-434b-a7e4-7832132865d1 Version: $LATEST
2020/04/07 21:45:01 Running cleanup operations in "us-east-1"
2020/04/07 21:45:01 Are destroy operations enabled? "true"
2020/04/07 21:45:01 Running cleanup operations for unattached ebs volumes
2020/04/07 21:45:01 UnauthorizedOperation: You are not authorized to perform this operation.
status code: 403, request id: c03efbf0-a3b4-4dd3-805b-02cec3523e0b
END RequestId: dc34a4f0-55f3-434b-a7e4-7832132865d1
REPORT RequestId: dc34a4f0-55f3-434b-a7e4-7832132865d1 Duration: 172.66 ms Billed Duration: 200 ms Memory Size: 1024 MB Max Memory Used: 49 MB Init Duration: 121.26 ms
RequestId: dc34a4f0-55f3-434b-a7e4-7832132865d1 Process exited before completing request

We haven’t given our IAM Role permission to interact with EBS. Let’s attach a specific policy to the IAM Role that allows us to interact with EBS:

const lambdaRolePolicy = new aws.iam.Policy("my-policy", {
 name: "sample-lambda-plicy",
 policy: {
 Version: "2012-10-17",
 Statement: [{
 Action: [
 "ec2:DescribeVolumes",
 "ec2:DeleteVolume",
 ],
 Resource: "*",
 Effect: "Allow",
 }],
 },
});

const policyAttachment = new aws.iam.PolicyAttachment("my-attachment", {
 policyArn: lambdaRolePolicy.arn,
 roles: [lambdaRole],
});

After deploying this updated IAM role, we can then see the execution will be allowed to proceed:

START RequestId: 7ca595aa-0a0f-4662-ab64-835ab0b02258 Version: $LATEST
2020/04/07 21:49:28 Running cleanup operations in "us-east-1"
2020/04/07 21:49:28 Are destroy operations enabled? "true"
2020/04/07 21:49:28 Running cleanup operations for unattached ebs volumes
2020/04/07 21:49:28 Finished cleanup operations
END RequestId: 7ca595aa-0a0f-4662-ab64-835ab0b02258
REPORT RequestId: 7ca595aa-0a0f-4662-ab64-835ab0b02258 Duration: 182.97 ms Billed Duration: 200 ms Memory Size: 1024 MB Max Memory Used: 26 MB

You can continue adding new segments to the cleanup lambda to help cleanup your AWS account.

Deploying To Multiple Regions

We want to ensure that all of our regions are cleaned up; therefore, we need to change the structure of the code. We can take advantage of explicit provider configuration to allow us to target the deployment of a resource to a specific region.

Let’s assume we want to deploy our application to us-east-1, us-east-2, eu-west-1, eu-west-2 and us-west-2. We can create a map of Providers to do this as follows:

const providers: {[key: string]: aws.Provider} = {
 "us-east-1": new aws.Provider("us-east-1", {region: "us-east-1"}),
 "us-east-2": new aws.Provider("us-east-2", {region: "us-east-2"}),
 "us-west-2": new aws.Provider("us-west-2", {region: "us-west-2"}),
 "eu-west-1": new aws.Provider("eu-west-1", {region: "eu-west-1"}),
 "eu-west-2": new aws.Provider("eu-west-2", {region: "eu-west-2"}),
};

We can iterate through the map and deploy our lambda to each of the configured regions as follows:

const providers: {[key: string]: aws.Provider} = {
 "us-east-1": new aws.Provider("us-east-1", {region: "us-east-1"}),
 "us-east-2": new aws.Provider("us-east-2", {region: "us-east-2"}),
 "us-west-2": new aws.Provider("us-west-2", {region: "us-west-2"}),
 "eu-west-1": new aws.Provider("eu-west-1", {region: "eu-west-1"}),
 "eu-west-2": new aws.Provider("eu-west-2", {region: "eu-west-2"}),
};

for (const providerKey of Object.keys(providers)) {
 const provider = providers[providerKey];

 const lambda = new aws.lambda.Function(`my-lambda-function-${providerKey}`, {
 name: "sample-lambda",
 runtime: aws.lambda.Go1dxRuntime,
 timeout: 900,
 role: lambdaRole.arn,
 handler: "main",
 memorySize: 1024,
 code: new pulumi.asset.FileArchive("deployment.zip"),
 environment: {
 variables: {
 "DESTROY_ENABLED": "true"
 }
 },
 }, {provider});
}

Pulumi can then deploy the lambdas to all of the providers in the loop. If a new region requires cleaning, then it needs to be added to the map of providers.

In Conclusion

The Go application we built continues to evolve. There are new areas of our account that have started to accumulate costs. After a few iterations of the application, we were able to drop 64% of our daily AWS costs.

We look forward to hearing the results of your work in this area!

Architect AWS Application Infrastructure

Paul Stack — Thu, 17 Oct 2019 00:00:00 +0000

In this post, we will talk about the best way to architect your Pulumi applications. We are going to build out the following infrastructure in AWS:

AWS Fargate service that does not serve traffic directly
AWS ALB as the entry point to the Fargate Service
AWS RDS Instance that is stored in a separate network from the Application and does not service traffic directly from the internet

To do this, we are going to split the infrastructure into two AWS VPCs. One VPC will be used for the application based resources (e.g Fargate Service, ALB) and the other VPC will be for the database resources. The VPCs will be peered to allow communication between them so that the application can communicate with the database securely.

Layer it up

When we think of infrastructure, we tend to think of layers. A layer is a logical set of components / resources that can be grouped together. In our case, we can think of the networking as a layer upon which our database and application can be built. For the purposes of this post, an infrastructure layer will be represented as a project in Pulumi. So let’s break our application into separate Pulumi projects:

$ tree -d -L 1
.
├── application
├── database
└── networking

This means all of the work required for the networking components can be in the network project. We can use Stack References in Pulumi to expose any required information from the networking layer, to the database and application layers.

Let’s create the networking project.

Building our Networking Project

$ cd networking && pulumi new aws-typescript
$ pulumi config set aws:region us-east-2

This gives us the shell of our project we need to create our resources. In thinking of how we build our networking resources, we will create two copies of each resource due to separate VPCs for application and database. Of course, we can use a loop to do this, but the more elegant way would be to create a class that we could pass parameters into to make life easy. We want an API as follows:

const availabilityZones = aws.getAvailabilityZones({
 state: "available",
});

const myVpc = new Vpc("my-vpc", {
 description,
 baseTags,

 baseCidr,
 availabilityZoneNames,
 enableFlowLogs,
});

We can see the power of Pulumi when we build our infrastructure with TypeScript. We create a TypeScript class that extends Pulumi ComponentResource. ComponentResource is a resource that aggregates one or more other child resources into a higher level abstraction. The component resource itself is a resource, but does not require custom create, read, update and delete operations for provisioning.

The outline of the class will look as follows:

export class Vpc extends ComponentResource {
 // ...
}

We want to pass a name and some custom parameters to the class, so we create an interface for the parameters:

export interface VpcArgs {
 // ...
}

We update our class with a constructor that allows us to create a new instance of the VPC class:

export class Vpc extends ComponentResource {
 constructor(name: string, args: VpcArgs, opts?: ComponentResourceOptions) {
 super("vpc", name, {}, opts);

 // ... resources here ...
 }
}

After we implement all of the resources needed for our VPC, our Pulumi application looks like the following:

async function main() {
 const config = new Config();
 const azCount = config.getNumber("azCount") || 2;

 const availabilityZones = aws.getAvailabilityZones({
 state: "available",
 });

 const appVpc = new Vpc("app-vpc", {
 baseCidr: "172.28.0.0/16",
 availabilityZoneNames: availabilityZones.names.slice(0, azCount),
 enableFlowLogs: true,
 });

 const dataVpc = new Vpc("data-vpc", {
 baseCidr: "172.18.0.0/16",
 availabilityZoneNames: availabilityZones.names.slice(0, azCount),
 enableFlowLogs: true,
 });

 return {
 appVpcId: appVpc.vpcId(),
 dataVpcId: dataVpc.vpcId(),
 }
}

module.exports = main();

This application will create two instances of the VPC class containing all of the resources needed for a VPC. This reusable VPC class` is a more efficient way of building out infrastructure than to simply loop over the resources. We can take advantage of relating the resources to the correct parent to build out graph.

We take it one step further, and be able to peer the VPCs. TypeScript, again, gives us the ability to do this in a concise manner by using a member function. The member function of the VPC class looks as follows:

public configurePeering(args: PeerToArgs) {
 // ...
}

This gives us the ability to peer the VPCs:

appVpc.configurePeering({
 peerVpc: dataVpc,
 nameTag: `${baseTags.ManagedBy} Peer App to Data`,
 routeSubnets: "private",
});

After creating a similar member function for the peering security group, we instruct Pulumi to create the resources by running the command pulumi up. This displays a preview of changes first and the list of AWS resources it will create:

$ pulumi up

After being prompted and selecting “yes”, your deployment will begin. After completion, you will see:

...
 + │ ├─ aws:ec2:VpcEndpoint data-vpc-s3-endpoint created
 + │ └─ aws:ec2:VpcEndpoint data-vpc-dynamodb-endpoint created
 + └─ aws:ec2:SecurityGroup data-vpc-app-vpc-sg created

Outputs:
 appVpcId : "vpc-07420f4b22cce694a"
 appVpcPrivateSubnetIds : [
 [0]: "subnet-07ebb7bb1894f76ea"
 [1]: "subnet-0cc6c64ffce7fa74a"
 ]
 appVpcPublicSubnetIds : [
 [0]: "subnet-07eec9d474411d47c"
 [1]: "subnet-093c9e201b5bb3975"
 ]
 dataVpcId : "vpc-04d28ded952454899"
 dataVpcPrivateSubnetIds: [
 [0]: "subnet-048e68b5d12d6189d"
 [1]: "subnet-088fbb5acf3d0a58c"
 ]
 dataVpcPublicSubnetIds : [
 [0]: "subnet-0711e414ccc7458a3"
 [1]: "subnet-0b8d54dfec34b286b"
 ]
 peeredSecurityGroupId : "sg-08b58260a840a971b"

Resources:
 + 61 created

Duration: 3m8s

Building our Database Project

$ cd database && pulumi new aws-typescript
$ pulumi config set aws:region us-east-2

Our database project follows the same principals as the networking project - it has a RDS class that extends the Pulumi ComponentResource.

export interface RdsArgs {
 // ...
}
export class RdsInstance extends ComponentResource {
 constructor(name: string, args: RdsArgs, opts?: ComponentResourceOptions) {
 super("db", name, {}, opts);

 // ... resources here ...
 }
}

This project needs other configuration options set:

pulumi config set database-layer:dbUsername MyDBUser
pulumi config set database-layer:dbName initialDb
pulumi config set database-layer:dbPassword MyDBPassword1234! --secret

The database project needs to access the details of the resources creating in the networking layer. To do this, we use a Pulumi StackReference. We pass the correct stack name that the database project needs to talk to, so we can pass that as a configuration option. A stack reference in Pulumi will follow the structure org/project/stack. In our case it will be stack72/networking-layer/dev and we can pass that as a configuration option to the database project:

$ pulumi config set networkingStack stack72/networking-layer/dev

Inside our database project, we can access any of the exported values from the networking project. The first thing we need access to is the database private subnet Ids. To access those subnetIds, we would do the following:

import {Config, StackReference} from "@pulumi/pulumi";

const config = new Config();
const networkingStack = new StackReference(config.require("networkingStack"))

const subnetIds = networkingStack.getOutput("dataVpcPrivateSubnetIds")

Please Note: We need to pass the exact name of the output as specified in the networking stack!

Our database project looks like this:

const rds = new RdsInstance("db-instance", {
 description: `${baseTags.Project} DB Instance`,
 baseTags: baseTags,

 subnetIds: networkingStack.getOutput("dataVpcPrivateSubnetIds"),

 username: dbUsername,
 password: dbPassword,
 initalDbName: dbName,

 allocatedStorage: 40,
 engineVersion: "11.4",
 instanceClass: aws.rds.InstanceTypes.R3_Large,
 storageType: "gp2",

 finalSnapshotIdentifier: finalSnapshotIdentifier,

 sendEnhancedLogsToCloudwatch: true,
 monitoringInterval: 10,

 securityGroupIds: [networkingStack.getOutput("peeredSecurityGroupId")],
});

We can then instruct Pulumi to create our resources with a pulumi up command, as before. This will show you a preview of changes first and the list of AWS resources it will create:

$ pulumi up

After being prompted and selecting “yes”, your deployment will begin. After completion, you will see:

Updating (dev):

 Type Name Status
 + pulumi:pulumi:Stack database-layer-dev created
 + ├─ db db-instance created
 + │ ├─ aws:rds:SubnetGroup db-instance-subnet-group created
 + │ ├─ aws:iam:Role db-instance-enhanced-monitoring-role created
 + │ ├─ aws:iam:RolePolicyAttachment enhanced-policy-attachment created
 + │ └─ aws:rds:Instance db-instance-rds created
 + └─ random:index:RandomString my-random-string created

Outputs:
 dbAddress : "db-instance-rdsd86e208.cep6renxwgtk.us-west-2.rds.amazonaws.com"
 dbEndpoint: "db-instance-rdsd86e208.cep6renxwgtk.us-west-2.rds.amazonaws.com:5432"
 dbName : "initialDb"
 dbPassword: "[secret]"
 dbPort : "5432"
 dbUsername: "MyDBUser"

Resources:
 + 7 created

Duration: 7m49s

Building the Application Project

$ cd application && pulumi new aws-typescript
$ pulumi config set aws:region us-east-2

It will come as no surprise to you that we will build our application project in the same manner as that of networking and database. So we will have an Application class that extends Pulumi ComponentResource.

export interface ApplicationArgs {
 // ....
}
export class Application extends ComponentResource {
 constructor(name: string, args: ApplicationArgs, opts?: ComponentResourceOptions) {
 super("application", name, {}, opts);

 // ... resources here ...
 }
}

The application project differs from the database project in that it needs stack references to the networking and database projects. Let’s specify those via configuration as we did in the database project:

$ pulumi config set networkingStack stack72/networking-layer/dev
$ pulumi config set databaseStack stack72/database-layer/dev

We can then access details like, database endpoint, database user, application Vpc Id within our application project:

import * as awsx from "@pulumi/awsx";
import { Config, getStack, StackReference } from "@pulumi/pulumi";
import {Application} from "./application";

const config = new Config();

const networkingStack = new StackReference(config.require("networkingStack"));
const databaseStack = new StackReference(config.require("databaseStack"));

const app = new Application("app", {

 vpcId: networkingStack.getOutput("appVpcId"),

 // ALB in public subnets
 albSubnetIds: networkingStack.getOutput("appVpcPublicSubnetIds"),

 // App resources in private subnets
 appSubnetIds: networkingStack.getOutput("appVpcPrivateSubnetIds"),

 appImage: awsx.ecs.Image.fromPath("app", "./src/backend"),
 appPort: 80,

 dbName: databaseStack.getOutput("dbName"),
 dbUsername: databaseStack.getOutput("dbUsername"),
 dbPassword: databaseStack.getOutput("dbPassword"),
 dbPort: databaseStack.getOutput("dbPort"),
 dbHost: databaseStack.getOutput("dbAddress"),
});

export const albAddress = app.albAddress();

Notice that we are taking advantage of the Pulumi AWS Extensions. Pulumi AWSX is a set of component wrappers around many AWS operations. In our case, we are taking advantage of being able to create an AWS ECS container image from a known path to a dockerfile. This is a concise API that removes us from the need to run docker build as a separate process. It will happen as part of our Pulumi application.

We can, again, instruct Pulumi to create our resources for us by issuing the pulumi up command. In the interactive output, you will see how Pulumi interacts with docker and also see how it reads both of the stack references we told it to read information from. It will also show you a preview of the changes and the list of AWS resources it will create:

$ pulumi up

Testing our application

The application in this repository has a number of endpoints that we can use to test that the AWS ECS Fargate service is working as expected. We can test them either from a browser or from the command line. We can access the service using the address of the AWS ALB that was created to serve traffic to the application. Let’s test the endpoints from the CLI:

$ curl $(pulumi stack output albAddress)
{
 "hash": "59846138362875439341244157139878939483",
 "message": "hello world",
 "postgres": "please use the /postgres endpoint"
}

$ curl $(pulumi stack output albAddress)/postgres
{
 "hash": "59846138362875439341244157139878939483",
 "postgres": [
 [
 "PostgreSQL 11.4 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.3 20140911 (Red Hat 4.8.3-9), 64-bit"
 ]
 ]
}

$ curl $(pulumi stack output albAddress)/internet
I CAN connect to google.com%

$ curl $(pulumi stack output albAddress)/public-ip
<p>my external IP is: 52.24.188.63 (source: jsonip.com) </p>%

Our application is up and running and we can connect to it!

Cleaning up

To cleanup the resources we created, we need to go into each project directory and run the command

$ pulumi destroy --yes && pulumi stack rm

Once the resources have been deleted, then we can follow the interactive prompt to delete the stack.

Wrapping Up

In this post, we have discussed how we would break our infrastructure into logically grouped projects and how we share information between these projects. You can find a runable version of this infrastructure in our examples repository.

This demonstrates the suggested way of architecting Pulumi applications both for operability and collaboration.

You can try this code today get started with Pulumi today at https://www.pulumi.com and following the Getting Started for managing AWS resources with Pulumi.

Getting Started on DigitalOcean with Pulumi

Paul Stack — Thu, 18 Jul 2019 00:00:00 +0000

Pulumi recently added support for managing DigitalOcean resources. This article will show you how to deploy some load balanced Droplets on DigitalOcean using Pulumi.

To get started, let’s create a new Pulumi program written in TypeScript:

 $ mkdir infra && cd infra
 $ pulumi new typescript

Now we should install the latest Pulumi DigitalOcean provider:

 $ npm install @pulumi/digitalocean

At the top of our application, we can add the resource imports:

import * as pulumi from "@pulumi/pulumi";
import * as digitalocean from "@pulumi/digitalocean";

We can now start to build the infrastructure. To get started, we will declare two constants:

const dropletCount = 3;
const region = digitalocean.Regions.NYC3;

Now, let’s create our web servers. Since our Pulumi application is written in TypeScript, we can use a loop, as follows:

const dropletTypeTag = new digitalocean.Tag("demo-app");
const userData =
 `#!/bin/bash
 sudo apt-get update
 sudo apt-get install -y nginx`;

const droplets = [];
for (let i = 0; i < dropletCount; i++) {
 let nameTag = new digitalocean.Tag(`web-${i}`);

 droplets.push(new digitalocean.Droplet(`web-${i}`, {
 image: "ubuntu-18-04-x64",
 region: region,
 privateNetworking: true,
 size: digitalocean.DropletSlugs.Droplet512mb,
 tags: [nameTag.id, dropletTypeTag.id],
 userData: userData,
 }));
}

Let’s take a closer look at what we’ve programmed. First, we declared a dropletTypeTag; we’ll come back to what this tag signifies later. Next, we have a userData constant that we can pass to all the instances, so that we can see they are running a web server. Then, we are declaring a nameTag constant. This allows us to use the interpolation in the loop to ensure that each of our Droplets is tagged with the appropriate name. We can also see that the region constant is being used in the Droplet declaration and we are adding the meta-information back to the Droplets array for later use.

Our next piece of infrastructure is our load balancer. We are going to create a public load balancer that will accept connections on port 80 and forward those connections to the Droplets that are attached on port 80. The infrastructure block looks like this:

const lb = new digitalocean.LoadBalancer("public", {
 dropletTag: dropletTypeTag.name,
 forwardingRules: [{
 entryPort: 80,
 entryProtocol: digitalocean.Protocols.HTTP,
 targetPort: 80,
 targetProtocol: digitalocean.Protocols.HTTP,
 }],
 healthcheck: {
 port: 80,
 protocol: digitalocean.Protocols.TCP,
 },
 region: region,
});

export const endpoint = lb.ip;

The configuration is made up of an array of forwarding rules. Here, we forward traffic from the load balancer to the Droplets. Then we have a health check that makes sure that port 80 is actually available and we deploy the load balancer into the region that we declared as a constant earlier. The interesting thing about load balancers in DigitalOcean is that you can attach instances by Droplet tag without knowing the Droplet IDs. Here, we configure all Droplets that have the tag demo-app, in this region, to be attached to the load balancer.

Running this gives us:

Updating (dev):

 Type Name Status
 + pulumi:pulumi:Stack do-ref-architecture-dev created
 + ├─ digitalocean:index:Tag web-0 created
 + ├─ digitalocean:index:Tag web-1 created
 + ├─ digitalocean:index:Tag web-2 created
 + ├─ digitalocean:index:Tag demo-app created
 + ├─ digitalocean:index:Droplet web-0 created
 + ├─ digitalocean:index:Droplet web-1 created
 + ├─ digitalocean:index:LoadBalancer public created
 + └─ digitalocean:index:Droplet web-2 created

Outputs:
 endpoint: "45.55.120.64"

Resources:
 + 9 created

Duration: 3m5s

You should be able to take the endpoint address and put it into the browser or you should be able to curl it as follows:

curl "$(pulumi stack output endpoint)"

We are constantly working on ways to make Pulumi more productive to use with DigitalOcean. Pulumi is free and open source. You can get started with Pulumi today at https://www.pulumi.com.

Using Terraform Remote State with Pulumi

Paul Stack — Fri, 07 Jun 2019 00:00:00 +0000

While some people coming to Pulumi are entirely new to Infrastructure as Code, increasingly teams are moving from other tools - whether cloud-specific in the form of CloudFormation or ARM Templates, or cloud-agnostic tools such as Terraform. In these organizations, new infrastructure provisioned with Pulumi must co-exist with existing resources provisioned with other tools, and often by different teams. For example, it’s common to see an application team deploying into a VPC owned and managed by a network operations team.

Pulumi supports this kind of workflow natively using the StackReference type from the Pulumi SDK. Integration with the most popular cloud-specific tools have been supported by Pulumi since the earliest days:

The aws.cloudformation.getStack() function can be used to obtain the outputs from a CloudFormation Stack.
The get method of the azure.core.SubscriptionTemplateDeployment class can be used to obtain the outputs of an ARM Template Deployment.

We recently added similar support for reading the outputs of a Terraform state file - both from local .tfstate files, and from all of the remote state backends supported by Terraform. This is exposed via the terraform.state.RemoteStateReference type in the @pulumi/terraform NPM package.

Example - Terraform Enterprise Backend

To demonstrate the use of the RemoteStateReference type, let’s imagine we want to use the IDs of subnets in a simple AWS VPC was defined by another team using Terraform 0.12, with remote state stored in Terraform Enterprise, using the following HCL:

 terraform {
required_version = ">= 0.12"
backend "remote" {
organization = "acme"
workspaces {
name = "production"
}
}
}
provider "aws" {
region = "us-west-2"
}
resource "aws_vpc" "vpc" {
cidr_block = "172.21.0.0/16"
tags = {
Name = "VPC"
}
}
resource "aws_internet_gateway" "gw" {
vpc_id = aws_vpc.vpc.id
tags = {
Name = "VPC IG"
}
}
resource "aws_subnet" "public" {
count = 2
cidr_block = "172.21.${count.index}.0/24"
vpc_id = aws_vpc.vpc.id
map_public_ip_on_launch = true
tags = {
Name = "Public Subnet ${count.index +1}"
}
}
resource "aws_route_table" "rt_public" {
vpc_id = aws_vpc.vpc.id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.gw.id
}
tags = {
Name = "VPC Public"
}
}
output "vpc_id" {
value = aws_vpc.vpc.id
}
output "public_subnet_ids" {
value = aws_subnet.public.*.id
}

To consume the outputs of this Terraform state in our Pulumi program we can do the following:

Create a new Pulumi program written in TypeScript
```
 $ pulumi new --yes typescript
```
Install the @pulumi/terraform package from NPM:
```
 $ npm install @pulumi/terraform
```

In the index.ts file, create a terraform.state.RemoteStateReference resource to access the state. Note that we can use Pulumi secrets to ensure that our Terraform Enterprise token is encrypted and never stored in plaintext by Pulumi:

import * as pulumi from "@pulumi/pulumi";
import * as terraform from "@pulumi/terraform";

const config = new pulumi.Config();
const tfeToken = config.requireSecret("tfeToken");

const networkState = new terraform.state.RemoteStateReference("network", {
 backendType: "remote",
 token: tfeToken,
 organization: "acme",
 workspaces: {
 name: "production-network"
 },
});

We can now use either the outputs property or the getOutput() function on networkState to obtain individual outputs:

const vpcId = networkState.getOutput("vpc_id");
const publicSubnetIds = networkState.outputs["public_subnet_ids"] as pulumi.Output<string[]>;

// Create our webservers in each subnet
for (let i = 0; i < 2; i++) {
 new aws.ec2.Instance(`instance${i}`, {
 ami: nginxAmi,
 instanceType: "t2.medium",
 subnetId: publicSubnetIds[i],
 })
}

Using Pulumi to read the outputs of other deployment tools provides a great deal of flexibility for adopting Pulumi into existing environments. Resources which were provisioned by CloudFormation, ARM or Terraform can remain in place, while still allowing those values to be dynamically consumed by a Pulumi program.

Pulumi is free and open-source, and you can get started today. To learn more about migrating from Terraform to Pulumi, check out Converting Full Terraform Programs to Pulumi and the Terraform comparison documentation, or join us in the Pulumi Community Slack to discuss with the Pulumi community.