Pulumi Blog: Lee Briggs

Secure AWS Lambda with IAM ABAC Policies

Paul Stack — Tue, 19 Jul 2022 00:00:00 +0000

Event-driven, serverless functions have become a defining feature of many modern cloud architectures. With recent capabilities such as AWS Lambda URLs and AWS Lambda Containers, AWS has made it clear that Lambda Functions are a platform that teams can use to deliver increasingly sophisticated services without worrying about managing underlying compute resources.

Today, AWS announced another advancement for their Lambda Functions platform: Attribute-Based Access Control (ABAC). At its core, ABAC support brings more granular permissions that are automatically applied based on IAM role tags, Lambda tags, or both. This update builds on well-established Role-Based Access Control (RBAC) principles while making it possible to implement granular controls without permissions updates for every new user and resource.

What are Attributes?

Attributes are a key or key/value pair - in AWS, these attributes are called tags. Tags can be applied to multiple roles – for example identifying members of a team.

Across many teams, your organization may share a lot of common roles, and, in order to enforce the principle of least access, you may wish to restrict access across teams such that Developer Team members cannot access Ops Team resources and vice versa.

Using ABAC for AWS Lambda Functions

For this example, we’ll define a new role ‘abac-test’ as well as a team attribute that must have the value of either ‘developers’ or ‘ops’ to enable creating a Lambda function.

To get started, let’s first install the Pulumi AWS provider:

$ npm install @pulumi/aws

$ dotnet add package Pulumi.Aws

$ pip3 install pulumi_aws

$ go get github.com/pulumi/pulumi-aws/sdk/v5

Now, create a new IAM Role that a user can assume:

const role = new aws.iam.Role("abac-test", {
 assumeRolePolicy: JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Sid: "",
 Effect: "Allow",
 Principal: {
 AWS: "arn:aws:iam::MYACCOUNTID:root",
 },
 Action: "sts:AssumeRole",
 },
 ],
 }),
});

var role = new Aws.Iam.Role("abac-test", new Aws.Iam.RoleArgs
{
 AssumeRolePolicy = JsonSerializer.Serialize(new Dictionary<string, object?>
 {
 { "Version", "2012-10-17" },
 { "Statement", new[]
 {
 new Dictionary<string, object?>
 {
 { "Action", "sts:AssumeRole" },
 { "Effect", "Allow" },
 { "Principal", new Dictionary<string, object?>
 {
 { "AWS", "arn:aws:iam::MYACCOUNTID:root" },
 } },
 },
 }
 },
 }),
});

role = aws.iam.Role("abac-test",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "AWS": "arn:aws:iam::MYACCOUNTID:root",
 },
 }],
 }),
 tags={
 "Team": "developers",
 })

role, err := iam.NewRole(ctx, "abac-test", &iam.RoleArgs{
 AssumeRolePolicy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Sid": "",
 "Effect": "Allow",
 "Principal": {
 "AWS": "arn:aws:iam::MYACCOUNTID:root"
 },
 "Action": "sts:AssumeRole"
 }]
 }`),
})
if err != nil {
 return err
}

 abacTest:
 type: aws:iam:Role
 properties:
 assumeRolePolicy:
 Fn::ToJSON:
 Version: 2012-10-17
 Statement:
 - Action: sts:AssumeRole
 Effect: Allow
 Principal:
 AWS: arn:aws:iam::MYACCOUNTID:root

Let’s now create a new IAM Policy that allows a user to create Lambda functions:

const createPolicy = new aws.iam.Policy(
 "createLambda",
 {
 policy: JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Effect: "Allow",
 Action: ["lambda:CreateFunction", "lambda:TagResource"],
 Resource: "arn:aws:lambda:*:*:function:*",
 Condition: {
 // the requesting resource must have a tag with
 // team = developers/ops
 StringEquals: {
 "aws:RequestTag/Team": ["developers", "ops"], // must match
 },
 // your request must contain these tags
 // eg. project, but you might not have this defined yet
 "ForAllValues:StringEquals": {
 "aws:TagKeys": "Team", // must exist
 },
 },
 },
 {
 // Pulumi needs to check what functions exist and what versions exist to create updates and new versions
 Effect: "Allow",
 Action: [
 "lambda:GetFunction",
 "lambda:ListVersionsByFunction",
 "lambda:GetFunctionCodeSigningConfig",
 ],
 Resource: "*",
 },
 {
 Effect: "Allow",
 // These are assume role policies
 Action: ["iam:ListRoles", "iam:PassRole"],
 Resource: "*",
 },
 ],
 }),
 },
);

var createPolicy = new Aws.Iam.Policy("createPolicy", new Aws.Iam.PolicyArgs
{
 PolicyDocument = JsonSerializer.Serialize(new Dictionary<string, object?>
 {
 { "Version", "2012-10-17" },
 { "Statement", new[]
 {
 new Dictionary<string, object?>
 {
 { "Effect", "Allow" },
 { "Action", new[]
 {
 "lambda:CreateFunction",
 "lambda:TagResource",
 }
 },
 { "Resource", "arn:aws:lambda:*:*:function:*" },
 { "Condition", new Dictionary<string, object?>
 {
 // the requesting resource must have a tag with
 // team = developers/ops
 { "StringEquals", new Dictionary<string, object?>
 {
 { "aws:RequestTag/Team", new[]
 {
 "developers",
 "ops",
 }
 },
 } },
 // your request must contain these tags
 // eg. project, but you might not have this defined yet
 { "ForAllValues:StringEquals", new Dictionary<string, object?>
 {
 { "aws:TagKeys", new[]
 {
 "Team",
 }
 },
 } },
 } },
 },
 new Dictionary<string, object?>
 {
 // Pulumi needs to check what functions exist and what versions exist to create updates and new versions
 { "Effect", "Allow" },
 { "Action", new[]
 {
 "lambda:GetFunction",
 "lambda:ListVersionsByFunction",
 "lambda:GetFunctionCodeSigningConfig",
 }
 },
 { "Resource", "*" },
 },
 new Dictionary<string, object?>
 {
 // These are assume role policies
 { "Effect", "Allow" },
 { "Action", new[]
 {
 "iam:ListRoles",
 "iam:PassRole",
 }
 },
 { "Resource", "*" },
 },
 }
 },
 }),
});

create_policy = aws.iam.Policy("createPolicy", policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [
 {
 "Effect": "Allow",
 "Action": [
 "lambda:CreateFunction",
 "lambda:TagResource",
 ],
 "Resource": "arn:aws:lambda:*:*:function:*",
 "Condition": {
 #the requesting resource must have a tag with
 #team = developers/ops
 "StringEquals": {
 "aws:RequestTag/Team": [
 "developers",
 "ops",
 ],
 },
 # your request must contain these tags
 # eg. project, but you might not have this defined yet
 "ForAllValues:StringEquals": {
 "aws:TagKeys": ["Team"],
 },
 },
 },
 {
 # Pulumi needs to check what functions exist and what versions exist to create updates and new versions
 "Effect": "Allow",
 "Action": [
 "lambda:GetFunction",
 "lambda:ListVersionsByFunction",
 "lambda:GetFunctionCodeSigningConfig",
 ],
 "Resource": "*",
 },
 {
 # These are assume role policies
 "Effect": "Allow",
 "Action": [
 "iam:ListRoles",
 "iam:PassRole",
 ],
 "Resource": "*",
 },
 ],
}))

createPolicy, err := iam.NewRolePolicy(ctx, "createPolicy", &iam.RoleArgs{
 Policy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Sid": "",
 "Effect": "Allow",
 "Condition": map[string]interface{}{
 "StringEquals": map[string]interface{}{
 "aws:RequestTag/Team": []string{
 "developers",
 "ops",
 },
 },
 "ForAllValues:StringEquals": map[string]interface{}{
 "aws:TagKeys": []string{
 "Team",
 },
 },
 },
 "Resource": "arn:aws:lambda:*:*:function:*",
 "Action": []string{
 "lambda:CreateFunction",
 "lambda:TagResource",
 },
 },
 {
 "Effect": "Allow",
 "Resource": "*",
 "Action": []string{
 "lambda:GetFunction",
 "lambda:ListVersionsByFunction",
 "lambda:GetFunctionCodeSigningConfig",
 },
 },
 {
 {
 "Effect": "Allow",
 "Resource": "*",
 "Action": []string{
 "lambda:CreateFunction",
 "lambda:TagResource",
 },
 }
 }]
 }`),
})
if err != nil {
 return err
}

 createPolicy:
 type: aws:iam:Policy
 properties:
 policy:
 Fn::ToJSON:
 Version: 2012-10-17
 Statement:
 - Effect: Allow
 Action:
 - lambda:CreateFunction
 - lambda:TagResource
 Resource: 'arn:aws:lambda:*:*:function:*'
 Condition:
 StringEquals:
 aws:RequestTag/Team:
 - developers
 - ops
 ForAllValues:StringEquals:
 aws:TagKeys:
 - Team
 - Effect: Allow
 Action:
 - lambda:GetFunction
 - lambda:ListVersionsByFunction
 - lambda:GetFunctionCodeSigningConfig
 Resource: '*'
 - Effect: Allow
 Action:
 - iam:ListRoles
 - iam:PassRole
 Resource: '*'

Notice, as part of the policy condition, we ensure that the requesting resource must have a tag with team: developers or team: ops.

Let’s ensure that this policy is attached to the Role we initially created:

const rpa = new aws.iam.RolePolicyAttachment("rpa", {
 policyArn: createPolicy.arn,
 role: role,
})

var rpa = new Aws.Iam.RolePolicyAttachment("rpa", new Aws.Iam.RolePolicyAttachmentArgs
{
 PolicyArn = createPolicy.Arn,
 Role = role.Name,
});

rpa = aws.iam.RolePolicyAttachment("rpa",
 policy_arn=create_policy.arn,
 role=role.name)

rpa, err := iam.NewRolePolicyAttachment(ctx, "rpa", &iam.RolePolicyAttachmentArgs{
 PolicyArn: createPolicy.Arn,
 Role: role.Name,
})
if err != nil {
 return err
}

 rpa:
 type: aws:iam:RolePolicyAttachment
 properties:
 policyArn: ${createPolicy.arn}
 role: ${abacTest.name}

We can deploy these resources using the command pulumi update:

pulumi up
Previewing update (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/previews/c6fb4580-2706-47ee-8a56-b8942631254e

 Type Name Plan
 + pulumi:pulumi:Stack aws-abac-test-abac-testing create
 + ├─ aws:iam:Policy createLambda create
 + ├─ aws:iam:Role abac-test create
 + └─ aws:iam:RolePolicyAttachment createPolicy create

Resources:
 + 4 to create

Do you want to perform this update? yes
Updating (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/updates/1

 Type Name Status
 + pulumi:pulumi:Stack aws-abac-test-abac-testing created
 + ├─ aws:iam:Role abac-test created
 + ├─ aws:iam:Policy createLambda created
 + └─ aws:iam:RolePolicyAttachment createPolicy created

Resources:
 + 4 created

Duration: 7s

For the purposes of testing, we are now going to create a specific AWS Provider resource that we can pass the IAM Role details to assume.

const abacUserProvider = new aws.Provider(
 "assumeRole",
 {
 // assume the previously created role with the ABAC configured policy into our provider
 assumeRole: {
 roleArn: role.arn,
 sessionName: "abacTesting",
 },
 region: "us-east-1",
 },
 {
 // we want to depend on the role policy attachment being in place before we create the provider
 dependsOn: [rpa],
 }
);

var abacUserProvider = new Aws.Provider("abacUserProvider", new Aws.ProviderArgs
{
 AssumeRole = new Aws.Config.Inputs.AssumeRoleArgs
 {
 RoleArn = role.Arn,
 SessionName = "abacTesting",
 },
 Region = "us-east-1",
}, new CustomResourceOptions
{
 DependsOn =
 {
 rpa,
 },
});

abac_user_provider = pulumi.providers.Aws("abacUserProvider",
 assume_role=aws.config.AssumeRoleArgs(
 role_arn=role.arn,
 session_name="abacTesting",
 ),
 region="us-east-1",
 opts=pulumi.ResourceOptions(depends_on=[rpa]))

abacUserProvider, err := providers.Newaws(ctx, "abacUserProvider", &providers.awsArgs{
 AssumeRole: config.AssumeRole{
 RoleArn: role.Arn,
 SessionName: "abacTesting",
 },
 Region: "us-east-1",
}, pulumi.DependsOn([]pulumi.Resource{
 rpa,
}))
if err != nil {
 return err
}

 abacUserProvider:
 type: pulumi:providers:aws
 properties:
 assumeRole:
 roleArn: ${role.arn}
 sessionName: "abacTesting"
 region: "us-east-1"
 options:
 dependsOn:
 - ${rpa}

Now we can create an AWS Lambda with Pulumi as follows:

const lambdaRole = new aws.iam.Role("iamRole", {
 assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal({
 Service: "lambda.amazonaws.com",
 }),
});

const deleteLambdaPolicy = new aws.iam.Policy(
 "deleteLambdaPolicy",
 {
 policy: JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Effect: "Allow",
 Action: ["lambda:DeleteFunction"],
 Resource: "*",
 },
 ],
 }),
 },
 { parent: lambdaRole }
);

new aws.iam.RolePolicyAttachment(
 `deletePolicy`,
 {
 policyArn: deletePolicy.arn,
 role: role,
 },
 { parent: deletePolicy })

const lambda = new aws.lambda.Function(
 "lambdaFunction",
 {
 code: new pulumi.asset.AssetArchive({
 ".": new pulumi.asset.FileArchive("./app"),
 }),
 runtime: "nodejs12.x",
 role: lambdaRole.arn,
 handler: "index.handler",
 },
 { provider: abacUserProvider }
);

var lambdaRole = new Aws.Iam.Role("lambdaRole", new Aws.Iam.RoleArgs
{
 AssumeRolePolicy = JsonSerializer.Serialize(new Dictionary<string, object?>
 {
 { "Version", "2012-10-17" },
 { "Statement", new[]
 {
 new Dictionary<string, object?>
 {
 { "Action", "sts:AssumeRole" },
 { "Effect", "Allow" },
 { "Principal", new Dictionary<string, object?>
 {
 { "Service", "lambda.amazonaws.com" },
 } },
 },
 }
 },
 }),
});
var deletePolicy = new Aws.Iam.Policy("deletePolicy", new Aws.Iam.PolicyArgs
{
 PolicyDocument = JsonSerializer.Serialize(new Dictionary<string, object?>
 {
 { "Version", "2012-10-17" },
 { "Statement", new[]
 {
 new Dictionary<string, object?>
 {
 { "Action", "lambda:DeleteFunction" },
 { "Effect", "Allow" },
 { "Resource", "*" },
 },
 }
 },
 }),
});
var deletePolicyAttachment = new Aws.Iam.RolePolicyAttachment("deletePolicyAttachment", new Aws.Iam.RolePolicyAttachmentArgs
{
 Role = role.Name,
 PolicyArn = deletePolicy.Arn,
});
var lambdaFunction = new Aws.Lambda.Function("lambdaFunction", new Aws.Lambda.FunctionArgs
{
 Code = new FileArchive("./app"),
 Role = lambdaRole.Arn,
 Handler = "index.handler",
 Runtime = "nodejs14.x",
}, new CustomResourceOptions
{
 Provider = abacUserProvider,
});

lambda_role = aws.iam.Role("lambdaRole", assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": "lambda.amazonaws.com",
 },
 }],
}))
delete_policy = aws.iam.Policy("deletePolicy", policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": "lambda:DeleteFunction",
 "Effect": "Allow",
 "Resource": "*",
 }],
}))
delete_policy_attachment = aws.iam.RolePolicyAttachment("deletePolicyAttachment",
 role=role.name,
 policy_arn=delete_policy.arn)
lambda_function = aws.lambda_.Function("lambdaFunction",
 code=pulumi.FileArchive("./app"),
 role=lambda_role.arn,
 handler="index.handler",
 runtime="nodejs14.x",
 opts=pulumi.ResourceOptions(provider=abac_user_provider))

lambdaRole, err := iam.NewRole(ctx, "lambdaRole", &iam.RoleArgs{
 AssumeRolePolicy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Principal": {
 "Service": "lambda.amazonaws.com"
 },
 "Action": "sts:AssumeRole"
 }]
 }`),
})
if err != nil {
 return err
}

deletePolicy, err := iam.NewRolePolicy(ctx, "deletePoicy", &iam.RoleArgs{
 Policy: pulumi.String(`{
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Resource": "*",
 "Action": []string{
 "lambda:DeleteFunction",
 },
 }]
 }`),
})
if err != nil {
 return err
}

_, err = iam.NewRolePolicyAttachment(ctx, "deletePolicyAttachment", &iam.RolePolicyAttachmentArgs{
 Role: role.Name,
 PolicyArn: deletePolicy.Arn,
})
if err != nil {
 return err
}

_, err = lambda.NewFunction(ctx, "lambdaFunction", &lambda.FunctionArgs{
 Code: pulumi.NewFileArchive("./app"),
 Role: lambdaRole.Arn,
 Handler: pulumi.String("index.handler"),
 Runtime: pulumi.String("nodejs14.x"),
}, pulumi.Provider(abacUserProvider))
if err != nil {
 return err
}

 lambdaRole:
 type: aws:iam:Role
 properties:
 assumeRolePolicy:
 Fn::ToJSON:
 Version: 2012-10-17
 Statement:
 - Action: sts:AssumeRole
 Effect: Allow
 Principal:
 Service: lambda.amazonaws.com
 deletePolicy:
 type: aws:iam:Policy
 properties:
 policy:
 Fn::ToJSON:
 Version: 2012-10-17
 Statement:
 - Action: lambda:DeleteFunction
 Effect: Allow
 Resource: '*'
 deletePolicyAttachment:
 type: aws:iam:RolePolicyAttachment
 properties:
 role: ${abacRole.name}
 policyArn: ${deletePolicy.arn}
 lambdaFunction:
 type: aws:lambda:Function
 properties:
 code:
 Fn::FileArchive: ./app
 role: ${lambdaRole.arn}
 handler: index.handler
 runtime: nodejs14.x

If we try and deploy the lambda, we will get the following using the role we are going to assume then we will get an error as the lambda function has no Team tag.

pulumi up
Previewing update (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/previews/7fdc10b2-5e44-497e-a067-f2e8b257496c

 Type Name Plan
 pulumi:pulumi:Stack aws-abac-test-abac-testing
 + ├─ aws:iam:Role iamRole create
 ├─ aws:iam:Role abac-test
 + │ └─ aws:iam:Policy deletePolicy create
 + │ └─ aws:iam:RolePolicyAttachment deletePolicy create
 + ├─ pulumi:providers:aws assumeRole create
 + └─ aws:lambda:Function lambdaFunction create

Resources:
 + 5 to create
 4 unchanged

Do you want to perform this update? yes
Updating (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/updates/2

 Type Name Status Info
 pulumi:pulumi:Stack aws-abac-test-abac-testing **failed** 1 error
 + ├─ aws:iam:Role iamRole created
 ├─ aws:iam:Role abac-test
 + │ └─ aws:iam:Policy deletePolicy created
 + │ └─ aws:iam:RolePolicyAttachment deletePolicy created
 + ├─ pulumi:providers:aws assumeRole created
 + └─ aws:lambda:Function lambdaFunction **creating failed** 1 error

Diagnostics:
 pulumi:pulumi:Stack (aws-abac-test-abac-testing):
 error: update failed

 aws:lambda:Function (lambdaFunction):
 error: 1 error occurred:
 * error creating Lambda Function (1): AccessDeniedException:
 status code: 403, request id: 4a8676f2-2b68-4d42-ad4b-f0e10ca31afd

Resources:
 + 4 created
 4 unchanged

Duration: 6s

If we add the correct Team tag to the Lambda function then the deploy will proceed as expected:

pulumi up
Previewing update (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/previews/0afa267f-5f9f-43f5-9e00-d3ee19dbf2b8

 Type Name Plan
 pulumi:pulumi:Stack aws-abac-test-abac-testing
 + └─ aws:lambda:Function lambdaFunction create

Resources:
 + 1 to create
 8 unchanged

Do you want to perform this update? yes
Updating (abac-testing)

View Live: https://app.pulumi.com/stack72/aws-abac-test/abac-testing/updates/3

 Type Name Status
 pulumi:pulumi:Stack aws-abac-test-abac-testing
 + └─ aws:lambda:Function lambdaFunction created

Resources:
 + 1 created
 8 unchanged

Duration: 9s

Try It Out!

Lambda ABAC capabilities will help you to simplify governance by helping you to standardize roles and maintain security boundaries between teams. A common use case is using tags to restrict/enable the invoke action for a function in addition to the create action demonstrated above. Give Lambda ABAC functionality a try and let us know what you think in Community Slack.

Announcing Pulumi Community Slack Archive

Lee Briggs — Mon, 02 May 2022 12:00:00 -0700

Creating a place for the Pulumi community to gather, ask questions, get help in real-time, and share successes has been an important part of the explosive growth we’ve seen in both users and customers. The Pulumi community slack has grown to over 7000 members and well over 200,000 messages.

Within those 200,000 messages are years of information kept behind a “walled garden” that is undiscoverable outside Slack’s search capabilities.

Today we’re announcing that our community slack is now publicly available via https://archive.pulumi.com thanks to Linen

For many, Slack has become the de-facto collaboration tool. Adding Slack workspaces is relatively straightforward and means you can communicate with various different communities and your colleagues in real-time.

This ability to use the tools you’re using in your everyday work to also quickly get help with Pulumi has proven to be extremely successful. Our users often tell us that having access to Pulumi staff and other community members helps them accelerate their cloud engineering journey. In addition to providing value to our users, the Pulumi team enjoys having a mechanism to engage with users to get feedback and inform them about the Pulumi ecosystem. It’s clear to us that the Pulumi community slack has proven invaluable as Pulumi has grown.

The use of Slack for our community meeting place isn’t without its problems. Often, the first port of call for users that need answers is to use search engines, and all of the information stored in Slack isn’t readily available by default with Slack. The Pulumi team regularly answers questions on StackOverflow and via GitHub Discussions which are searchable and indexed by search engines, but adding the treasure trove of information from Slack will improve the experience for eager Pulumi users.

Until now, this has been exceptionally difficult. With the recent launch of Linen we are able to bring our community chat outside of Slack’s boundaries in an easily searchable, indexable way.

We’re excited to provide this to the community and encourage you to try it out at https://archive.pulumi.com.

Azure Container Services Compared: ACI vs App Service vs AKS

Lee Briggs — Tue, 12 Oct 2021 08:00:00 -0700

Containers have emerged as one of the de facto standards for running software. When adopted with the right mindset, they can drastically improve the development lifecycle and help to close the loop between local development and running your applications in the cloud.

If you’re at the stage of trying to run your application in Microsoft Azure, the choices can be overwhelming. The Azure Container product page lists 7 different products on their landing page, and for new users it can often be difficult to decide which of the myriad products is right for their use case. What can make it even more confusing is that often these container services can be interoperable, meaning you can use one container product from another!

In this post, we’re going to examine each of the main container services offered in Azure and then examine what they’re good for and what they might not be so good for. Let’s take a look!

Azure Container Instances

Azure Container Instances (ACI) bills itself as an offering that lets you “easily run containers on Azure without managing servers.”

Launched in July 2017, it was arguably the first “serverless” container offering from the major cloud providers.

What’s good about it?

ACI is a Platform-as-a-Service (PaaS) type of product that is designed to remove the burden of launching and running containers.

Simplicity

Azure Container Instances has been designed from the ground up to be straightforward to use. It’s designed to get your container launched as quickly as possible, even including integrations with many of the tools you’re probably already familiar with if you’re using containers. One example is the integration with the Docker CLI.

In many cases, the only configuration options you need to pass to your ACI provisioning mechanism is the image you’re running and some sizing decisions, like how many CPUs and how much memory you need. There is some flexibility in this decision process, but having fewer required options means you’re not bogged down with complex architectural decisions when trying to get your container running.

This lends itself particularly well to web server projects, especially at smaller scales. You can launch applications very quickly and easily by specifying the image you’ve built, and your application can be running in very short order.

Start-Up Times

ACI Containers start fast—often in a matter of seconds. This speed really helps the developer lifecycle and helps to create productivity when you’re developing your application.

Pricing

ACI will only bill you for the time your applications are running. It offers no start-up costs (which goes great with the fast start times!) and per-second billing, meaning you only pay for every second the application is actually up. The pricing is also extremely competitive against other cloud providers.

The Other Stuff

From a security perspective, ACI separates your container from others running in the same environment, meaning your application will have the same level of security as if it was running on a different virtual machine. This separation really helps to provide peace of mind and is yet another benefit of the serverless model—less infrastructure means less worries.

As expected from Microsoft, ACI has excellent support for Windows containers and .NET platform applications, which is lacking around other cloud providers. If you need to run Windows containers, this support is a huge benefit.

What’s not great about it?

Scaling

ACI supports customizing the amount of resources you can assign to a container, but it will only run one copy. There’s no horizontal scaling available, so if you need more resources, you only have one option: scaling vertically. It is possible to run N number of ACI instances and configuring load balancers across them, but it’s a very manual process and can be awkward to manage and scale successfully.

If you’re running a webserver application, horizontal scaling behind a load balancer might be a better option, so this lack of scaling flexibility and manual processing can be a real shortcoming.

In addition to this opinionated scaling mechanism, there’s also no auto scaling option for ACI. If you’re finding that you’re hitting the limits of your ACI deployment, you’ll need to manually scale the service.

Configuration

ACI has some configuration options missing, which you’d expect would be there from the beginning. One glaring omission is port mapping (for example mapping public port 80 to container port 8080). This omission can mean modifying your container just to get it running in ACI.

ACI’s opinionated architecture seems to have been designed for ease-of-use, which often means more complex workloads just aren’t possible. If you have some unique configuration options, like needing access to privileged container options or complex network requirements, ACI might not be able to support you.

What we’d use it for

The lack of flexibility in ACI means that some workloads just aren’t able to run there. If you have full control over the container images you’re building and if you just want to get off the ground quickly with minimal fuss and configuration, ACI is a great choice.

Applications with smaller and more predictable usage patterns are a great use case for ACI. The lack of horizontal scaling could be seen as a negative, but from another perspective, it means the use case is straightforward and simple. If you just want to get your application into the hands of your users and think you can accurately predict your usage patterns, choose ACI.

Azure App Service

Azure App Service is a more fully-featured offering for hosting applications, focusing more on WebApps and API services. It started life as several differing offerings (Azure Websites, Azure Mobile Service, and Azure BizTalk) before being bundled into a single service in March 2015. Its origins as multiple solutions means it has developed tight integrations with other Azure solutions and offers many value-add features depending on your use case.

Because of the myriad features Azure App Service offers, running containers is only a small part of what you can do with it.

With that in mind, let’s take a look at where it shines with regard to running containers.

What’s it good for?

Azure App Services is designed to be a one-stop shop for your web application needs, so it’s highly opinionated and pushes you in a certain direction.

Integrations

Azure App Service has many integrations with Azure. As an example, if you need to add a CDN to your web service, it’s extremely straightforward to do so with Azure CDN.

This tight integration with Azure’s many other services makes it likely to meet the needs of your application and extend it as needed, introducing common architecture components like message queues and databases. This is particularly useful when it comes to the integration with Azure’s networking model. You can run an Azure App Service inside an Azure Virtual Network which is extremely useful in many scenarios where network separation is needed, like enterprise applications.

Scaling & Availability

Azure App Service has a myriad of available scaling capabilities built out of the box. Not only can you scale the actual container you’ve deployed (while also simultaneously placing pricing caps on your service), but you can also scale related resources if necessary, which fits well with the tight integration we mentioned previously.

App service scaling for your container can either scale up (i.e., increase the amount of resources associated with your container) or scale out (ie: have more containers behind your load balancer).

This flexibility in scaling means you can be safe to assume your application will meet spiking traffic requirements and service your users’ needs.

What’s not great about it?

Opinionated

Azure App Service is highly opinionated. Its tight integrations with other Azure services mean that you’re expected to follow established patterns for achieving your goals.

This opinionated implementation means that the initial learning curve can be steep. The origins of app services means the product has grown in complexity along with its opinionated implementation, and as a result it can be overwhelming when getting started.

In addition to the learning curve, App Service is primarily focused towards web app workloads, and support for running other applications is limited at best.

What we’d use it for

The scaling options available to App Service means that it’s extremely well positioned to handle your large scale, fluctuating web applications. If you expect to have traffic spikes during peak hours, you can configure App Service to scale relatively easily in a manner that suits your budget and needs.

Azure Kubernetes Service

Azure Kubernetes Service (AKS) is Azure’s managed Kubernetes offering, which has been available since October 2017.

Azure was very quick to market with its Kubernetes offering and hired several prominent Kubernetes engineers like Brendan Burns to head up the engineering departments building the service.

Kubernetes is a very fully featured product, and AKS is a very conformant implementation of Kubernetes. Therefore, a lot of our good and bad considerations next will be an evaluation of Kubernetes, as well as AKS.

What’s it good for?

Extensibility

Kubernetes is incredibly feature rich. Out of the box with AKS, you’ll get application self-healing capabilities via the mechanism of reconcile loops built into Kubernetes, the ability to run different types of workload via native Kubernetes resources such as StatefulSets, Deployments and Jobs, and a large amount of extensibility via Kubernetes’ Custom Resource Definitions capability.

Familiarity

Kubernetes also offers a familiar API regardless of the cloud provider you’re working with. If you’re new to AKS, but have used a managed Kubernetes service from another cloud provider, you might find it relatively easy to get your container up and running in AKS.

What’s not great about it?

Complexity

Kubernetes has a steep learning curve, and AKS is no exception. A lot of the knowledge needed to run and operate Kubernetes is handled by AKS itself, but you still have to have an in-depth understanding of how Kubernetes works to get your container running.

In addition, the specification for a simple Kubernetes job can often be hundreds of lines of code. You have to learn not only how to run the container but also understand how Kubernetes exposes services to the world.

What we’d use it for

The flexibility and extensibility offered by Kubernetes as a platform means it’s possible to run many types of workloads. Large companies have battle-tested Kubernetes (and by extension, AKS), running everything from web applications to machine learning.

If you’re able to invest time into learning and understanding the ecosystem and can dedicate people to maintaining and managing your Kubernetes implementation, AKS is a fantastic option for almost any workload you might consider.

Everything Else

Azure Red Hat OpenShift

Azure Red Hat OpenShift is an enterprise grade implementation of Kubernetes developed by Red Hat. OpenShift builds on Kubernetes by adding support, enterprise-grade security, and Platform-as-a-Service integrations.

If the idea of using Kubernetes for your container solutions seems appealing but you need service level agreements (SLAs) around support and integrations, Azure’s Red Hat OpenShift solution is a great option.

Azure Service Fabric

Azure Service Fabric is a container service focused on distributed applications, like microservice architectures or complex distributed compute environments.

If you’re planning on running a complex architecture, Azure Service Fabric can be a great fit. It provides more orchestration capabilities than Azure App Service. Service Fabric can also extend outside Azure’s cloud into your on-premise data center if necessary.

If your application has needs related to low latency, especially across distributed regions, Service Fabric is a great fit.

However, Azure Service Fabric can be extremely difficult to use. It is only targeted towards distributed, complex systems - if you’re not planning on building one of those, it’s probably not the correct choice.

Azure Batch

Azure Batch is a service focused on High Performance Compute (HPC) workloads. These workloads can often scale horizontally to thousands of processes and can be difficult to orchestrate.

Azure Batch makes this process simpler by providing a managed service to handle the difficult processes.

If you’re planning on passing large amounts of data to your application via a queuing mechanism and having that data processed very quickly, Azure Batch is definitely a service you should consider.

Azure Functions

Azure Functions is Azure’s Function-as-a-Service implementation. It supports launching “function apps” inside containers as a deployment mechanism.

Function-as-a-Service products are often designed for short-lived, event-driven workloads. If you need to run a script to react to a worker queue or other event, Azure Functions are a great option. It’s important to remember that Functions are often billed based on how long they run, so short-lived workloads are often better suited.

You might also like:

Wrap-Up

There are a number of options for running containers in Azure, and choosing the right one for your workload might be difficult. We’ve tried to give an overview of the current state of the world, but Microsoft continues to add options and capabilities to all these products.

If you’ve settled on a container solution, and are ready to get started, Pulumi’s Azure Native provider supports all of the methods of running containers and can quickly get you started. Check out our Azure getting started guide, or join one of our upcoming Azure Workshops!

It's Time to Embrace Kubernetes: Part 2

Lee Briggs — Fri, 11 Dec 2020 00:00:00 +0000

In the first article in this series, we gave you some questions to help you and others at your company decide if Kubernetes is right for you. In this post, we’ll give you an example of where Kubernetes can be a good fit.

When is Kubernetes a Good Choice?

If you’ve decided your company can make the large investment of time and resources required to adopt Kubernetes, when is it the right choice? A well designed, well implemented Kubernetes platform can certainly deliver value to your customers. It does a fantastic job of reducing complexity from the application developers’ jobs. Just remember that the payoff, although dramatic, will be many months away and, before that, things will be slower than you’re used to.

One example of where Kubernetes can add a lot of value is when the application side of the engineering organization has a low level of maturity, and the operations side has a high degree of maturity. Kubernetes can make the applications developers jobs easier because it abstracts away deployment details. In turn, the operations team must be willing to take on the burden of shipping software and let the application developers concentrate on creating new products and features. This division implies a good relationship between them, and fostering cooperation is always beneficial. If trust and good communication exist, Kubernetes is a great way for engineers to work together.

Here’s one example of how Kubernetes can help. Developers spend large amounts of time writing code that’s resilient to failure. Kubernetes, on the other hand, almost encourages you not to spend time writing fault-tolerant software and let the Kubernetes handle failure. Application developers don’t need to write resiliency into their applications. They can simply let the programs fail quickly and assume that Kubernetes will take care of it.

Are There Alternatives?

Asking if there are alternatives to Kubernetes makes sense, but it’s unlikely you’ll find a simple solution. Large-scale, complex problems won’t be solved easily. Most open-source alternatives can’t compete against it because Kubernetes is generic and has many building blocks available.

Another approach is to build your own solution. One example is to rely on raw compute power and spinning VMs in and out very quickly. In this case, you’ll need to decide if you’re getting value for your money because you may not be using the entire VM or sizing it correctly, which Kubernetes does for you.

You’ll also need to build the basic building blocks, as well as the glue that ties everything together. That glue can be anything from a set of shell scripts to a very, very complex orchestration framework. The industry was in this mode for at least 10 years and there were many problems with this approach, particularly at scale. So, even if you think that the Kubernetes tooling is insufficient, you’ll run into similar problems with the DIY approach.

Picking a First Project

Most medium to large organizations run multiple applications, and some of them are only used internally. A good way to start with Kubernetes is to pick an internal application that’s not mission-critical. Migrating that application to Kubernetes will give you a feel for what the experience will look like. You’ll be able to identify the gaps, the things that you like, and the potential problems.

So, start small and give yourself the time to learn. Once you feel comfortable, evaluate if Kubernetes is right for your company. If it is, then you can start to move things over gradually

Summary

To summarize, Kubernetes isn’t for everyone but, where it’s a good fit, it has a lot to offer. Keep these questions in mind:

[ ] Do you understand your current and future requirements? Kubernetes is a popular solution but does adopting it serve the needs of your organization and customers?

[ ] What’s the size of your organization? Kubernetes is a better fit for medium and large organizations. Startups and smaller companies should take a look at off-the-shelf options.

[ ] Where does your organization sit in its adoption of DevOps? Your organization must be well along in its adoption of DevOps principles such as collaboration, good communication, trust, and the ability to work for the good of the entire company.

[ ] Do you really need to change your infrastructure? Does your current solution allow you to deliver features to your customers quickly, easily, and reliably? Ask yourself the questions, “Is what we have truly broken? Do we really need to fix it?”

[ ] Do you have the resources? Changing technology stacks is a time and resource intensive task. Do you have the budget, the buy-in from leadership, and discipline to switch to a new technology stack?

[ ] Do you have the people? A robust Kubernetes implementation demands a deep understanding of its multi-layer architecture. Do you have the engineers and subject matter experts to maintain and fix problems?

[ ] Can you afford the technical debt? Can your organization redirect engineers who maintain your current systems to a Kubernetes implementation that can take a year or more before it’s operational?

[ ] Can you afford the extra infrastructure? Implementing a Kubernetes solution is not an overnight change. It’s common to run the existing solution alongside Kubernetes before fully adopting Kubernetes. Does your organization have the budget and support people to manage this change smoothly? Remember, you are, at minimum, doubling the size of your infrastructure.

Learn More

If you’ve looked at Get Started with Kubernetes and want to learn more about how Pulumi can help you with your implementation, take a look at Kubernetes Overview and Getting Started with Kubernetes: Application Basics.

It's Time to Embrace Kubernetes: Part 1

Lee Briggs — Wed, 09 Dec 2020 00:00:00 +0000

When you’re considering whether or not to implement Kubernetes, perhaps the first question to ask yourself is do you need it at all?

The point of any technology isn’t the technology itself. When done right, Kubernetes can reduce the barrier of entry for application developers so they can get features from their machines to your customers as quickly and easily as possible. But do you already have a solution that works well? If you do, why do you want to change it? Making such a radical change in your technology is potentially quite dangerous so what’s your motivation?

It very well might be that sticking with and improving the solution you already have offers a better cost/benefit tradeoff. It’s easy to fall into the trap of believing that simply adopting a new technology like Kubernetes will instantly solve your hard organizational or technical problems, however, we know that is seldom true.

In this blog post we’ll share some tips and tricks for evaluating your own situation to see if Kubernetes is a good fit. We’ve learned these from helping hundreds of customers adopt Kubernetes — in addition to not, when there was a better solution available. We’ll see that the question isn’t that simple to answer and there are a lot of variables to consider. In the next blog post, we’ll talk about a situation where Kubernetes can be a good fit and how to start your first Kubernetes project.

Note that these blog posts assume you already have some familiarity with Kubernetes. If you are just starting to learn, our Getting Started with Kubernetes blog series is a good place to start.

Why is Kubernetes So Popular?

Kubernetes, an open-source system for deploying, scaling, and managing containerized applications, is currently the last step in an evolutionary chain that began with private data centers and, from there, went to cloud VMs, then to containers and then to container orchestrators. If you are looking to ship applications to the cloud continuously, containers are a great way to do that, and Kubernetes is the technology responsible for running them. Kubernetes was initially developed by Google and has since had an enormous community grow up around it well beyond its Google roots. Version 1.0 was released in 2015 and the latest version was released in October 2020.

Kubernetes has become increasingly possible over the last few years. In the ZDNet article, “Kubernetes jumps in popularity”, published on March 6, 2020, Steven J. Vaughan-Nichols cites a Cloud Native Computing Foundation (CNCF) survey about containers and Kubernetes. Of the companies that responded, 84% were using containers and, of those companies, 78% were using Kubernetes to manage them. The survey also found that there were over 109 tools to manage containers and 89% of them use various versions of Kubernetes.

There are several reasons why Kubernetes is so widely adopted. Here are some of them:

Extensibility. While Kubernetes comes with a large set of existing resources such as Pods and Secrets, developers can also create their own with custom resource definitions (CRD). They can also write their own Operators, with which they can automate the management of CRDs.
Community support. The Kubernetes community is large and there are many special interest groups. Also, Kubernetes is housed on a vendor-neutral platform run by the CNCF. The CNCF sponsors CloudNativeCon/KubeCon, which is one of the largest open-source events in the world.
Third-party vendor support. There are also many third-party vendors that repackage Kubernetes. These are called Kubernetes distributions. They provide opinionated implementations of production-ready Kubernetes. Examples include Openshift, Rancher and VMWare Tanzu.
Frequent releases. Kubernetes releases often, with a new significant release every three or four months. Each release adds many new features and innovations.

Is Kubernetes the Right Choice for You?

Given its widespread popularity, its many features, and its community support, it seems that selecting Kubernetes is a given. “Everyone else is using it so our company should use it, too.” But is that the case? In the rest of this article, we’re going to propose some questions that you, your peers and management can use as a basis for discussion. These questions are meant to help your organization come to an informed choice about Kubernetes.

Is Your Organization Ready for Kubernetes?

The first question to ask yourself is if your organization has the DNA required to support a broad-scale Kubernetes implementation. By “DNA,” we don’t mean only technical know-how. Bringing Kubernetes into an organization is a major effort that can affect many groups. For that effort to succeed, there must be good communication between those groups, trust, and a willingness to take on new challenges together. In particular, the relationship between the applications teams and the operations teams needs to be healthy. People must be willing to set aside rivalries and politics so they can collaborate with each other towards a common business goal.

As you can imagine, this can become a heated topic.

Another way to look at this question is to ask yourself where your organization sits in terms of its adoption of DevOps. To quote the 2020 State of DevOps report:

We still see that most organizations are struggling to expand DevOps beyond a few pockets of success. One reason DevOps often fails to expand further is that most enterprises are structured in ways that create misaligned incentives and lack of accountability or ownership over the outcomes they’re supposed to be driving.

We have seen a greater likelihood of success within our own customers that have experience practicing “Cloud Engineering”, “DevOps”, “Site Reliability Engineering”, and/or other more collaborative and engineering-oriented approaches to infrastructure. You might want to evaluate your own organization. A good guide is DevOps Maturity Model.

What Are Your Motivations?

We often make decisions for personally motivated reasons and that’s as true of technologies as it is of what car you buy. Kubernetes has a lot of appeal right now. It’s well publicized and companies with a lot of cloud engineering credibility, such as Netflix, use it. That alone might persuade some managers to make a top-down adoption decision, thinking that if it’s right for Netflix it must be right for their own companies. There are tutorials, YouTube videos and conferences that all say they’ll make it easy for you to adopt Kubernetes, and expertise in Kubernetes is a great way to build your resume. There is no question that success stories from leading edge companies and a fast-growing community are significant benefits — however, this alone is insufficient. As we’ll see, there’s no easy road to success with Kubernetes so, to make the right decision, you need to ask yourself some tough questions and evaluate more deeply whether it’s in the best interests of your company to use it.

Do You Have the Resources?

The next question to ask is if your company has the engineering resources to support a Kubernetes initiative. Implementing Kubernetes is complex and takes a long time. Companies such as Netflix have very large operations departments but most other companies don’t. It is typically better to start by identifying an isolated project to prove out the expected benefits of Kubernetes before betting the entire farm on it. That said, don’t underestimate how much effort even these smaller projects will take, especially when it comes to getting up to speed.

Do you have the people?

Kubernetes will demand a team of people who are dedicated to its implementation and support. The implementation itself is quite complex. The Kubernetes architecture is complex with many core concepts, including execution, security, and data models, distributed components to install and manage like etcd, and many additional layers on top. A robust implementation demands a deep understanding of that multi-layer architecture. This complexity is somewhat mitigated by the managed services many cloud providers now offer, but it’s still important to understand what’s going on.

Secondly, the actual interface required to deploy something with Kubernetes is also complex. It can be many thousands of lines of YAML, which often needs to be templated and manipulated with mechanisms that YAML was never designed to use. Take a look at Why the **** are we templating YAML? for a strong opinion by the author on this matter. Furthermore, as an infrastructure team, you are almost certainly going to want a better interface for your application developers to ship code without needing to learn the complex parts of Kubernetes including writing so much YAML. Eventually you will want to set up CI/CD to ship continuously.

Finally, when something goes wrong, you need a subject matter expert to fix it. Because Kubernetes gives you such a high degree of abstraction, it can be difficult to debug and most people will be unfamiliar with what’s going on under the hood. You need someone who’s really knowledgeable when there’s trouble. In fact, you probably need two people. You’re too vulnerable if that person leaves the company and you’re completely unreasonable if you can’t even let that person take a vacation. If you aren’t yet practicing observability, you will quickly want to.

Can you afford the technical debt?

Because you’re dedicating several people to Kubernetes, you may be increasing your technical debt in other areas. Do you still have people available to handle bugs and other support issues? The costs here can be seriously mitigated by adopting a managed Kubernetes service from a major cloud vendor, but the opportunity cost is real either way.

Can you afford the migration costs?

As with any large migration, moving from your current system to Kubernetes needs to happen cautiously and gradually over time. You’re not simply going to destroy your existing infrastructure. Instead, you’ll probably want to adopt a side-by-side approach and slowly move your applications across. In other words, for quite a while, you’ll need to support two infrastructures rather than one. This can cost real dollars in addition to engineering time.

What Are We Trying to Achieve, Again?

At the end of the day, it is worth taking a step back to ask ourselves, what were we trying to accomplish and did Kubernetes help us get there. Most likely the motivation for even asking this question included things like: we want to accelerate public cloud adoption (or adopt more public cloud-like technologies on-premises), we want to ship faster, and we want to empower our developers. It is true that, done well and done right, Kubernetes can achieve all of these things. But Kubernetes alone will not — it takes so much more than just that — and these can all very well be accomplished without Kubernetes, sometimes quicker and more cheaply. Many companies, and startups in particular, have big dreams. They want an infrastructure, such as Kubernetes, that can scale infinitely to support their fast growing user base of their future. It is good to dream big and plan for the future, but it is also often better to start with a simpler solution today and evolve more incrementally over time.

For instance, maybe a simpler, native container orchestrator like Amazon’s Elastic Container Service (ECS) will be easier to adopt and manage. Or perhaps a PaaS like Heroku, Google AppEngine, or Azure Service Fabric, that is more opinionated and less extensible, but far simpler to get up and running on, is a better, cheaper first step. If you have an event-driven architecture, then one of the many serverless compute technologies may be a better fit.

Architectures have to be shaped over time. It’s never the case that was just create our infrastructure once and then we’re done. Whatever infrastructure you adopt, you need to be able to change and adapt as your requirements change and evolve. In five years time, who knows what the world will look like? Betting the farm on Kubernetes too soon and too big could turn out to be an unwise decision in hindsight after you’ve spent many precious dollars deploying it.

Learn More

This blog post talked about the many issues to consider before you dive headlong into a Kubernetes project. However, there are just as many reasons to choose Kubernetes for deploying your modern applications. Stay tuned for part 2 where we’ll dive deeper into that topic. If you’re curious about just what a small project might look like (using your favorite programming language, whatever that is), check out Pulumi’s Get Started with Kubernetes and Pulumi Crosswalk for Kubernetes, playbooks with built in best practices that make Kubernetes accessible to everyone.

Getting Started with Amazon EKS Distro & Pulumi

Luke Hoban — Tue, 01 Dec 2020 11:30:00 -0800

As Kubernetes grows in popularity, the number of options for Kubernetes users continues to increase. Providers of managed Kubernetes offerings will often learn lessons about operating large numbers of clusters at scale; it’s increasingly common that they will contribute this knowledge back to the ecosystem, allowing those organizations who need more control and flexibility to reap the benefits.

With the announcement of the Amazon EKS Distro during AWS re:Invent, the Amazon EKS team has contributed back to the cloud-native community in a big way. In this post, we’ll take a brief look at what the Amazon EKS Distro is, explore why you might choose this over current managed service offerings and finally, explore how you can get started with the Amazon EKS Distro on day 1 using Pulumi.

Kubernetes is a complex, distributed system that consists of many components that operate together to create the Kubernetes experience. If you’re using a managed Kubernetes service, often these components are hidden away from you as a user to reduce the operational complexity and ease the usage. The managed service provider will take care of tricky problems like upgrades and taking care of patching and compliance.

The Amazon EKS Distro is the packaging of many of the components needed to run a Kubernetes cluster distributed in an opinionated way by the Amazon EKS team. Parts of a working Kubernetes cluster like the scheduler, API server and the backing database (etcd) have been built into Docker images based on Amazon Linux. The images have backported security fixes for each Kubernetes version and are drop-in replacements for the upstream Kubernetes distribution, meaning you get all the benefits of Kubernetes, with the added support of knowing you’re running a verified and supported version from the Amazon EKS team.

Why use the Amazon EKS Distro?

If you’re already a user of Amazon EKS, you’re already getting all of the benefits provided by the Amazon EKS Distro. However, many users aren’t utilizing the managed Kubernetes cluster provided by the Amazon EKS team, either through choice or because of external requirements. The release of the Amazon EKS Distro means that those organizations can get the benefits provided by EKS, such as secure Docker images, backported security fixes and a single upstream vendor, without having to operate with the constraints a managed service in AWS might provide.

Provisioning an Amazon EKS Distro Cluster with Pulumi

Installing an Amazon EKS Distro cluster involves bootstrapping a Kubernetes cluster but with the images provided by the Amazon EKS team. The simplest way to provision a Kubernetes cluster is using kops and the Amazon EKS Distro team provides instructions on how to replace the default Kubernetes components with the Amazon EKS Distro built parts.

It’s possible to get all of the benefits of provisioning a Kubernetes cluster with kops while also taking part in the Pulumi resource lifecycle. Pulumi offers support for running arbitrary commands using dynamic providers (available in the JavaScript, Typescript and Python SDKs) and you can register the kops commands used to create and destroy clusters as dynamic provider callbacks. Once your Kubernetes cluster has been created, you can provision Kubernetes resources using Pulumi’s Kubernetes provider, using the Pulumi resource model to set dependencies on the created cluster.

Creating an Amazon EKS Distro cluster with Pulumi

Using a Pulumi dynamic provider, you can define an Amazon EKS Distro cluster using just a few lines of code:

import * as eksdistro from "./eksdistro";

const store = new aws.s3.Bucket("kops-state-store");
const cluster = new eksdistro.Cluster("cluster", {
 name: "luke.cluster.pulumi-demos.net",
 state: pulumi.interpolate`s3://${store.id}`,

Behind the scenes, we’ve defined a Typescript dynamic provider, which renders a kops configuration file and renders it to disk using mustache. This kops configuration file references the Amazon EKS Distro images.

This dynamic provider registers callbacks for creating, updating and deleting the cluster and also registers the created cluster’s kubeconfig as a Pulumi output, which can be passed to Kubernetes resources in Pulumi.

import * as pulumi from "@pulumi/pulumi";
import * as cp from "child_process";
import * as path from "path";
import * as fs from "fs";
import * as mustache from "mustache";
import * as tmp from "tmp";

interface ClusterProviderArgs {
 // KOPS_STATE_STORE
 state: string;
 // KOPS_CLUSTER_NAME
 name: string;
}

const clusterYamlTemplate = fs.readFileSync(path.join(__dirname, "cluster.yaml")).toString();
const authenticatorYamlTemplate = fs.readFileSync(path.join(__dirname, "aws-iam-authenticator.yaml")).toString();

const clusterprovider: pulumi.dynamic.ResourceProvider = {
 async create(inputs: ClusterProviderArgs): Promise<pulumi.dynamic.CreateResult> {
 const clusterYaml = mustache.render(clusterYamlTemplate, {
 CLUSTER_NAME: inputs.name,
 STATE_STORE: inputs.state,
 });
 console.log("kops create")
 const createOut = cp.execSync(`kops create --name ${inputs.name} --state ${inputs.state} -f -`, { input: clusterYaml });
 console.log(createOut.toString());
 console.log("kops create secret")
 const createSecretOut = cp.execSync(`kops create secret --name ${inputs.name} --state ${inputs.state} sshpublickey admin -i ~/.ssh/id_rsa.pub`);
 console.log(createSecretOut.toString());

 let outs = { ...inputs };
 try {
 const updateRes = await this.update!(inputs.name, {}, inputs);
 outs = { ...outs, ...updateRes.outs};
 } catch (err) {
 console.log(err);
 }

 return {
 id: inputs.name,
 outs,
 };
 },
 async update(id: pulumi.ID, olds: any, inputs: any): Promise<pulumi.dynamic.UpdateResult> {
 const clusterYaml = mustache.render(clusterYamlTemplate, {
 CLUSTER_NAME: inputs.name,
 STATE_STORE: inputs.state,
 });
 const authenticatorYaml = mustache.render(authenticatorYamlTemplate, {
 CLUSTER_NAME: inputs.name,
 });

 console.log("kops update cluster")
 const updateOut = cp.execSync(`kops update cluster --name ${inputs.name} --state ${inputs.state} --yes`);
 console.log(updateOut.toString());

 console.log("kops export kubecfg")
 const kubeConfigName = tmp.tmpNameSync();
 const exportKubeconfigOut = cp.execSync(`kops export kubecfg --name ${inputs.name} --state ${inputs.state} --kubeconfig ${kubeConfigName}`);
 const kubeconfig = fs.readFileSync(kubeConfigName).toString();
 console.log(exportKubeconfigOut.toString());

 // Needed to allow cluster to come available and DNS to propagate
 try {
 console.log("kops validate cluster")
 const validateOut = cp.execSync(`kops validate cluster --wait 2m --name ${inputs.name} --state ${inputs.state}`);
 console.log(validateOut.toString());

 console.log("kubectl apply -f aws-iam-authenticator.yaml");
 const authApplyOut = cp.execSync(`kubectl apply -f -`, { input: authenticatorYaml });
 console.log(authApplyOut.toString());
 } catch (err) {
 console.log(err);
 }

 return {
 outs: {
 ...inputs,
 kubeconfig,
 },
 }
 },
 async delete(id: pulumi.ID, inputs: any) {
 const clusterYaml = mustache.render(clusterYamlTemplate, {
 CLUSTER_NAME: inputs.name,
 STATE_STORE: inputs.state,
 });
 console.log("kops delete")
 const deleteOut = cp.execSync(`kops delete --name ${inputs.name} --state ${inputs.state} --yes -f -`, { input: clusterYaml });
 console.log(deleteOut.toString());
 },
}

export interface ClusterArgs {
 // KOPS_STATE_STORE
 state: pulumi.Input<string>;
 // KOPS_CLUSTER_NAME
 name: pulumi.Input<string>;
 keeper?: pulumi.Input<number>;
}

export class Cluster extends pulumi.dynamic.Resource {
 kubeconfig!: pulumi.Output<string>;
 constructor(name: string, args: ClusterArgs, opts?: pulumi.CustomResourceOptions) {
 super(clusterprovider, name, {
 ... args,
 kubeconfig: undefined,
 }, opts);
 }
}

Once the cluster is created, we can build a Kubernetes provider and register resources like any other Kubernetes cluster:


const k8sProvider = new k8s.Provider("provider", { kubeconfig: cluster.kubeconfig });
const pod = new k8s.core.v1.Pod("mypod", {
 spec: {
 containers: [{ name: "echo", image: "k8s.gcr.io/echoserver:1.4" }],
 }
}, { provider: k8sProvider });
export const kubeconfig = cluster.kubeconfig;

The full code for defining an Amazon EKS Distro cluster using Pulumi can be found in our examples repo

Wrap up

The Amazon EKS Distro opens lots of avenues for users needing more flexibility and control over their Kubernetes clusters while still providing the security, compliance and reliability required to feel confident in the clusters you’re building. With Pulumi, you can bootstrap Amazon EKS Distro clusters and leverage Pulumi’s best-in-class Kubernetes resource management while still benefiting from the power of Pulumi’s modern programming languages. Give it a try!

Introducing New Slimmer Docker Images

Lee Briggs — Thu, 25 Jun 2020 00:00:00 +0000

One of the most exciting aspects of using Pulumi can also present some interesting engineering challenges. Pulumi supports three operating systems, multiple programming languages, and almost 40 different providers. This means creating tooling that works effortlessly across all possible user scenarios can often throw unexpected challenges our way.

Nowhere are these challenges more prevalent than in the Pulumi Docker containers.

The pulumi/pulumi Docker container is almost 3Gb uncompressed, which is generally considered large for a Docker image. In this post, I’ll examine why this container has grown to the size that it is, and talk about how we hope to solve it.

Anatomy of a Pulumi Program

Pulumi has the unique ability to interact with cloud providers across many boundaries. As an example, you might want to provision a Kubernetes cluster and install a Helm chart onto that cluster. Pulumi lets you do that from a single “Pulumi program”, here’s an example:

import * as digitalocean from "@pulumi/digitalocean";
import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";

const config = new pulumi.Config();
const nodeCount = config.getNumber("nodeCount") || 1;

// Create a Kubernetes Cluster in DigitalOcean
const cluster = new digitalocean.KubernetesCluster("do-cluster", {
 region: digitalocean.Regions.SFO2,
 version: digitalocean.getKubernetesVersions({versionPrefix: "1.16"}).then(p => p.latestVersion),
 nodePool: {
 name: "default",
 size: "s-1vcpu-2gb",
 nodeCount: nodeCount,
 },
});

// configure a kubernetes provider
const provider = new k8s.Provider("k8s", { kubeconfig: cluster.kubeConfigs[0].rawConfig })

const namespace = new k8s.core.v1.Namespace("ns", {
 metadata: {
 name: "nginx-ingress",
 }
}, { provider: provider });

// deploy nginx-ingress with helm
const nginx = new k8s.helm.v2.Chart("nginx-ingress",
 {
 namespace: namespace.metadata.name,
 chart: "nginx-ingress",
 version: "1.33.5",
 fetchOpts: { repo: "https://charts.helm.sh/stable/" },
 values: {
 controller: {
 replicaCount: 1,
 service: {
 type: "LoadBalancer",
 },
 publishService: {
 enabled: true,
 },
 },
 }
 },
 { providers: { kubernetes: provider } },
)

"""A DigitalOcean Python Pulumi program"""

import pulumi
import pulumi_kubernetes as k8s
import pulumi_digitalocean as do
import pulumi_kubernetes.helm.v3 as helm
from pulumi_kubernetes.core.v1 import Namespace

config = pulumi.Config()
node_count = config.get_float("nodeCount") or 1


# Create a Kubernetes Cluster in DigitalOcean
cluster = do.KubernetesCluster(
 "do-cluster",
 region="sfo2",
 version="1.16",
 node_pool={
 "name": "default",
 "size": "s-1vcpu-2gb",
 "nodeCount": node_count
 }

)

# Use the Kubeconfig output to create a k8s provider
k8s_provider = k8s.Provider("k8s", kubeconfig=cluster.kube_configs[0]["rawConfig"])

# Create a namespace using the previously retrieved kubernetes provider
namespace = Namespace(
 "ns",
 metadata={
 "name": "nginx-ingress"
 }, opts=pulumi.ResourceOptions(provider=k8s_provider))

chart = helm.Chart(
 "nginx-ingress",
 helm.ChartOpts(namespace="nginx-ingress",
 chart="nginx-ingress",
 version="1.33.5",
 fetch_opts=k8s.helm.v3.FetchOpts(
 repo="https://charts.helm.sh/stable/"
 ),
 values={
 "controller": {
 "replicaCount": 1,
 "service": {
 "type": "LoadBalancer"
 },
 "publishService": {
 "enabled": True,
 }
 }
 }), pulumi.ResourceOptions(provider=k8s_provider)
)

package main

import (
 "fmt"
 "github.com/pulumi/pulumi-digitalocean/sdk/v2/go/digitalocean"
 "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes"
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/core/v1"
 "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/helm/v2"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/meta/v1"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {

 // Create a Kubernetes cluster in DigitalOcean
 cluster, err := digitalocean.NewKubernetesCluster(ctx, "do-cluster", &digitalocean.KubernetesClusterArgs{
 Region: pulumi.String("sfo2"),
 Version: pulumi.String("1.16"),
 NodePool: &digitalocean.KubernetesClusterNodePoolArgs{
 Name: pulumi.String("default"),
 Size: pulumi.String("s-1vcpu-2gb"),
 },
 })

 if err != nil {
 return fmt.Errorf("Error creating cluster: ", err)
 }

 // Instantiate a Kubernetes provider
 provider, err := kubernetes.NewProvider(ctx, "k8s", &kubernetes.ProviderArgs{
 Kubeconfig: cluster.KubeConfigs.Index(pulumi.Int(0)).RawConfig(),
 })

 if err != nil {
 return fmt.Errorf("Error instantating Kubernetes provider: ", err)
 }

 // Create a namespace
 ns, err := corev1.NewNamespace(ctx, "nginx-ingress", &corev1.NamespaceArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("nginx-ingress"),
 },
 }, pulumi.Provider(provider))

 if err != nil {
 return fmt.Errorf("Error creating Kubernetes namespace: ", err)
 }

 // Deploy nginx-ingress using the helm chart
 _, err = helm.NewChart(ctx, "nginx-ingress", helm.ChartArgs{
 Chart: pulumi.String("nginx-ingress"),
 Version: pulumi.String("1.33.5"),
 Namespace: ns.Metadata.Name().Elem(),
 FetchArgs: &helm.FetchArgs{
 Repo: pulumi.String("https://charts.helm.sh/stable/"),
 },
 Values: pulumi.Map{
 "controller": pulumi.Map{
 "replicaCount": pulumi.Int(1),
 "service": pulumi.Map{
 "type": pulumi.String("LoadBalancer"),
 },
 "publishService": pulumi.Map{
 "enabled": pulumi.Bool(true),
 },
 },
 },
 }, pulumi.Provider(provider))

 return nil
 })
}

To achieve this level of interoperability, however, we need to have some dependencies installed locally. These are:

the Pulumi binary
the Pulumi language runtime for our chosen language
the language binary (e.g., node, python, DotNet or Go)
the languages’ dependency management tool (e.g., npm, yarn, PipEnv, etc.)
the Helm CLI for rendering the Helm chart to a template that Pulumi can then install

This list of requirements is already getting lengthy, and we’re only provisioning one piece of infrastructure. Alongside this, if you’re using this container in your CI/CD pipeline, you might also need to acquire credentials during the pulumi up process. When you consider that Pulumi’s users could be using any of our featured programming languages as well as one or many of our supported cloud providers, you’ll begin to imagine how many dependencies are needed.

To provide a seamless experience for our users, our existing pulumi/pulumi Docker image contains every possible dependency a user might need when provisioning cloud infrastructure. We install everything from the Helm CLI to the Azure CLI to ensure you can use the container for any Pulumi program. Unfortunately, as we began to support more cloud providers and more languages, the size of the image has grown considerably.

We’ve received fantastic feedback from the community that while this Docker image meets their needs, providing smaller images is desirable. Not only do our users want to see images that are smaller in size, but they also want a less opinionated image that provides the flexibility and customization needed to suit the complex workflows people can create with Pulumi.

Introducing new Language Specific Images

Starting from Pulumi version 2.4.0, we’ll be publishing new, slimmer images in the DockerHub that contain fewer dependencies and are focused on individual specific languages. We’ve built these images to be usable from day 1, and many of our pioneering users have been using them successfully. Each image contains everything you need to use Pulumi for your programming language of choice, including the Pulumi CLI, the Pulumi language runtime, and the language binary and dependency manager).

Reduced Size

Removing many dependencies and only installing a single language SDK means we’ve seen a dramatic decrease in the uncompressed size of the images:

As you can see below, the uncompressed images are up to six times smaller than the pulumi/pulumi container, which means a much smaller surface area and considerably quicker pull times.

Image Name	Compressed Size	Uncompressed Size
pulumi/pulumi:v2.4.0	856.33 MB	2.97GB
pulumi/pulumi-go:2.4.0	208.7MB	592MB
pulumi/pulumi-go:2.4.0-ubi	255.07MB	701MB
pulumi/pulumi-nodejs:2.4.0	121MB	341MB
pulumi/pulumi-nodejs:2.4.0-ubi	149.37MB	439MB
pulumi/pulumi-dotnet:2.4.0	213.08MB	598MB
pulumi/pulumi-dotnet:2.4.0-ubi	241.39MB	677MB
pulumi/pulumi-python:2.4.0	106.48MB	308MB
pulumi/pulumi-python:2.4.0-ubi	150.18MB	385MB

Using the Images

Using the new Pulumi images looks very similar to how you might use the CLI locally. Here’s an example of running Pulumi for a typescript project, and the Pulumi SaaS backend:

docker run -e PULUMI_ACCESS_TOKEN=<PULUMI_ACCESS_TOKEN> -v "$(pwd)":/pulumi/projects pulumi-nodejs:latest /bin/bash -c "npm ci && pulumi preview -s dev"

If you’re using one of our cloud backends, you’ll need to specify your cloud provider credentials and login to your selected backend:

docker run -e AWS_REGION=<AWS_REGION> -e AWS_ACCESS_KEY_ID=<AWS_ACCESS_KEY> -e AWS_SECRET_ACCESS_KEY=<AWS_ACCESS_SECRET_ACCESS_KEY> -v "$(pwd)":/pulumi/projects pulumi/pulumi-nodejs:2.4.0 /bin/bash -c "pulumi login s3://<s3-bucket> && npm ci && pulumi preview -s <stack-name>"

Pulumi’s intelligent plugin acquisition should find and detect the required resource plugins so you can run your Pulumi program. If you need additional CLI tools like Helm as part of your workflow, we recommend installing these tools as part of your CI/CD pipeline or using these images as a base image to build your own Docker images. Here’s an example of installing Helm in our Go-based Docker image.

# Build container
FROM debian:latest AS build

RUN apt-get update && apt-get install curl -y && \
 curl -L https://get.helm.sh/helm-v3.2.4-linux-amd64.tar.gz |tar xvz && \
 mv linux-amd64/helm /usr/bin/helm && \
 chmod +x /usr/bin/helm

# Runtime container
FROM pulumi/pulumi-go:latest

COPY --from=build /usr/bin/helm /usr/bin/helm

Multiple Base Images

The default operating system for these new images is Debian Buster, but we also publish Images based on RedHat’s Universal Base Image. These images are small and lightweight but also benefit from RedHat’s exceptional security focus. We scanned these images with synk.io and found zero vulnerabilities, which should provide peace of mind when running these Pulumi images.

A Note about Alpine Linux

Many users are fans of Alpine Linux’s small, lightweight footprint and its Docker focused design; however, we’ve made the decision not to publish Alpine based images at this moment. Alpine’s usage of musl libc rather than glibc means that Pulumi doesn’t run without adding additional dependencies, which made it hard to justify introducing this extra source of potential bugs into our already broad surface area. If you’re interested in building and maintaining an Alpine based Pulumi image, we have skeleton Docker files available in the Pulumi GitHub repo.

Give them a spin!

The new images are available from DockerHub; we’d love for you to give them a try! As always, if you find any issues, don’t hesitate to open an issue!

Peace of Mind with Cloud Secret Providers

Lee Briggs — Thu, 07 May 2020 00:00:00 +0000

The secrets in your infrastructure are a vital part of your security model, and provisioning infrastructure is an inherently privileged process. Previously we introduced secret encryption and started encrypting secret configuration values inside the Pulumi state so that users could be confident their passwords, tokens, and other secret values were viewable only by them while managing their infrastructure. Our first iteration of the encryption used either a passphrase for encrypting the secret or encryption via the Pulumi service backend. However, these options didn’t meet the needs of our users who needed more control over their data. That’s why we also added support for “Cloud Secret Providers,” giving users full confidence that their sensitive values are for their eyes only.

Pulumi supports encryption via the Pulumi service, AWS KMS, Azure KeyVault, Google Cloud KMS and HashiCorp Vault. This post shows you one example of using a cloud secret provider in a Pulumi stack using AWS KMS.

Create a KMS Key

First, create a KMS key. We also can set an alias on the key to make it easier to reference later:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Create a new KMS key
const key = new aws.kms.Key("stack-encryption-key", {
 deletionWindowInDays: 10,
 description: "KMS key for encrypting Pulumi secret values",
});

// Create a new alias to the key
const alias = new aws.kms.Alias("alias/stack-encryption-key", {
 targetKeyId: key.keyId,
});

// Export the arns
export const keyArn = key.arn
export const aliasArn = alias.arn

"use strict";
const pulumi = require("@pulumi/pulumi");
const aws = require("@pulumi/aws");

// Create a nw KMS Key
const key = new aws.kms.Key("stack-encryption-key", {
 deletionWindowInDays: 10,
 description: "KMS key for encrypting Pulumi secret values"
});

// Create an alias to the key
const alias = new aws.kms.Alias("alias/stack-encryption-key", {
 targetKeyId: key.keyId
});

// Export the arns
exports.keyArn = key.arn;
exports.aliasArn = alias.arn;

import pulumi
from pulumi_aws import kms

# Create a new KMS key
key = kms.Key("stack-encryption-key",
 deletion_window_in_days=10,
 description="KMS key for encrypting Pulumi secret values"
)

// Create an alias to the key
alias = kms.Alias("alias/stack-encryption-key",
 target_key_id=key.key_id
)

# Export the arns
pulumi.export('key_arn', key.arn)

package main

import (
 "github.com/pulumi/pulumi-aws/sdk/v2/go/aws/kms"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {

 // Create a new KMS key
 key, err := kms.NewKey(ctx, "stack-encryption-key", &kms.KeyArgs{
 Description: pulumi.String("KMS key for encrypting Pulumi secret values"),
 DeletionWindowInDays: pulumi.Int(10),
 })
 if err != nil {
 return err
 }

 // Create an alias to the key
 alias, err := kms.NewAlias(ctx, "alias/stack-encryption-key", &kms.AliasArgs{
 TargetKeyId: key.KeyId,
 })
 if err != nil {
 return err
 }

 // Export the arns
 ctx.Export("keyArn", key.Arn)
 ctx.Export("keyAlias", alias.Arn)
 return nil
 })
}

using Pulumi;
using Kms = Pulumi.Aws.Kms;

class KeyStack : Stack
{
 public KeyStack()
 {
 // Create a new KMS key
 var key = new Kms.Key("stack-encryption-key", new Kms.KeyArgs
 {
 DeletionWindowInDays = 10,
 Description = "KMS key for encrypting Pulumi secret values"
 });

 // Create a new alias to the key
 var alias = new Kms.Alias("alias/stack-encryption-key", new Kms.AliasArgs
 {
 TargetKeyId = key.KeyId
 });

 this.KeyArn = key.Arn;
 this.AliasArn = alias.Arn;
 }

 [Output("keyArn")] public Output<string> KeyArn { get; set; }
 [Output("aliasArn")] public Output<string> AliasArn { get; set; }
}

Creating the key is enough to allow us to start using it for our encryption provider. However, we need to consider who has access to the key before we start encrypting our sensitive values with it in our Pulumi programs.

Scoping Permission to our Key

Generally, in AWS, you scope access to resources using IAM roles. However, for sensitive values like KMS keys, IAM roles alone aren’t enough to provide the security you might need. As an example, if someone in your AWS account has an IAM role with the following policy attached:

{
 "Version": "2012-10-17",
 "Statement": [
 {
 "Sid": "Stmt1588215924595",
 "Action": "kms:*",
 "Effect": "Allow",
 "Resource": "*"
 }
 ]
}

That user gets access to every KMS key in your account, which would also mean they could decrypt any secret in your Pulumi stack.

To rectify this, we need to attach a Key Policy to the key. We can do this by updating our previous code:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const config = new pulumi.Config();
const iamRole = config.require("iamRole");
const accountID = config.require("accountID");


const keyPolicy = {
 Version: "2012-10-17",
 Id: "policy",
 Statement: [
 // This statement allows all users to view the key in the console
 {
 Sid: "AllowGetKeys",
 Effect: "Allow",
 Action: ["kms:Describe*", "kms:Get*", "kms:List*"],
 Principal: {
 "AWS": [`arn:aws:iam::${accountID}:root`]
 },
 Resource: "*",
 },
 // This is a configurable statement that we can use to allow access to an IAM arn
 {
 Sid: "AllowIAMUserAccessKeys",
 Effect: "Allow",
 Action: ["kms:*"],
 Principal: {
 "AWS": iamRole,
 },
 Resource: "*",
 }
 ]
}

// Create a new KMS key
const key = new aws.kms.Key("stack-encryption-key", {
 deletionWindowInDays: 10,
 description: "KMS key for encrypting Pulumi secret values",
 policy: JSON.stringify(keyPolicy),
});

// Create a new alias to the key
const alias = new aws.kms.Alias("alias/stack-encryption-key", {
 targetKeyId: key.keyId,
});

// Export the arns
export const keyArn = key.arn
export const aliasArn = alias.arn

"use strict";
const pulumi = require("@pulumi/pulumi");
const aws = require("@pulumi/aws");

let config = new pulumi.Config();
let iamRole = config.require("iamRole");
let accountID = config.require("accountID");

const keyPolicy = {
 Version: "2012-10-17",
 Id: "policy",
 Statement: [
 // This statement allows all users to view the key in the console
 {
 Sid: "AllowGetKeys",
 Effect: "Allow",
 Action: ["kms:Describe*", "kms:Get*", "kms:List*"],
 Principal: {
 "AWS": [`arn:aws:iam::${accountID}:root`]
 },
 Resource: "*",
 },
 // This is a configurable statement that we can use to allow access to an IAM arn
 {
 Sid: "AllowIAMUserAccessKeys",
 Effect: "Allow",
 Action: ["kms:*"],
 Principal: {
 "AWS": iamRole,
 },
 Resource: "*",
 }
 ]
}

// Create a nw KMS Key
const key = new aws.kms.Key("stack-encryption-key", {
 deletionWindowInDays: 10,
 description: "KMS key for encrypting Pulumi secret values",
 policy: JSON.stringify(keyPolicy),
});


// Create an alias to the key
const alias = new aws.kms.Alias("alias/stack-encryption-key", {
 targetKeyId: key.keyId
});


// Export the arns
exports.keyArn = key.arn;
exports.aliasArn = alias.arn;

});


// Export the arns
exports.keyArn = key.arn;
exports.aliasArn = alias.arn;

import pulumi
import json
from pulumi_aws import kms

config = pulumi.Config()
iam_role = config.require("iamRole")
account_id = config.require("accountID")


key_policy = {
 "Version": "2012-10-17",
 "Id": "policy",
 "Statement": [
 # This statement allows all users to view the key in the console
 {
 "Sid": "AllowGetKeys",
 "Effect": "Allow",
 "Action": ["kms:Describe*", "kms:Get*", "kms:List*"],
 "Principal": {
 "AWS": ["arn:aws:iam::{}:root".format(account_id)]
 },
 "Resource": "*",
 },
 # This is a configurable statement that we can use to allow access to an IAM arn
 {
 "Sid": "AllowIAMUserAccessKeys",
 "Effect": "Allow",
 "Action": ["kms:*"],
 "Principal": {
 "AWS": iam_role,
 },
 "Resource": "*",
 }
 ]
}
# Create an AWS KMS key
key = kms.Key("stack-encryption-key",
 deletion_window_in_days=10,
 description="KMS key for encrypting Pulumi secret values",
 policy=json.dumps(key_policy)
)

alias = kms.Alias("alias/stack-encryption-key",
 target_key_id=key.key_id
)

# Export the name of the bucket
pulumi.export('key_arn', key.arn)
pulumi.export('alias_arn', alias.arn)

package main

import (
 "encoding/json"
 "fmt"

 "github.com/pulumi/pulumi-aws/sdk/v2/go/aws/kms"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi/config"
)

func main() {

 type Principal struct {
 AWS string `json:"AWS"`
 }
 type Statement struct {
 Sid string `json:"Sid"`
 Effect string `json:"Effect"`
 Principal Principal `json:"Principal"`
 Action []string `json:"Action"`
 Resource string `json:"Resource"`
 }

 type KeyPolicy struct {
 Version string `json:"Version"`
 ID string `json:"Id"`
 Statements []Statement `json:"Statement"`
 }

 pulumi.Run(func(ctx *pulumi.Context) error {

 config := config.New(ctx, "")

 iamRole := config.Require("iamRole")
 accountID := config.Require("accountID")

 rootIamRole := fmt.Sprintf("arn:aws:iam::%s:root", accountID)

 rawKeyPolicy := &KeyPolicy{
 Version: "2012-10-17",
 ID: "policy",
 Statements: []Statement{
 {
 Sid: "AllowGetKeys",
 Effect: "Allow",
 Action: []string{
 "kms:Describe*", "kms:Get*", "kms:List*",
 },
 Resource: "*",
 Principal: Principal{
 AWS: rootIamRole,
 },
 },
 {
 Sid: "AllowIAMUserAccessKeys",
 Effect: "Allow",
 Action: []string{"kms:*"},
 Resource: "*",
 Principal: Principal{
 AWS: iamRole,
 },
 },
 },
 }

 keyPolicy, err := json.Marshal(rawKeyPolicy)

 if err != nil {
 panic("Error formatting keypolicy")
 }

 // Create an AWS KMS key
 key, err := kms.NewKey(ctx, "stack-encryption-key", &kms.KeyArgs{
 Description: pulumi.String("KMS key for encrypting Pulumi secret values"),
 DeletionWindowInDays: pulumi.Int(10),
 Policy: pulumi.String(string(keyPolicy)),
 })
 if err != nil {
 return err
 }

 // Create an alias to the key
 alias, err := kms.NewAlias(ctx, "alias/stack-encryption-key", &kms.AliasArgs{
 TargetKeyId: key.KeyId,
 })
 if err != nil {
 return err
 }

 // Export the name of the bucket
 ctx.Export("keyArn", key.Arn)
 ctx.Export("keyAlias", alias.Arn)
 return nil
 })
}

using Pulumi;
using Kms = Pulumi.Aws.Kms;

class KeyStack : Stack
{
 public KeyStack()
 {
 var config = new Config();
 var iamRole = config.Require("iamRole");
 var accountID = config.Require("accountID");

 var keyPolicy = $@"{{
 ""Version"": ""2012-10-17"",
 ""Id"": ""policy"",
 ""Statement"": [
 {{
 ""Sid"": ""AllowGetKeys"",
 ""Effect"": ""Allow"",
 ""Action"": [""kms:Describe*"", ""kms:Get*"", ""kms:List*""],
 ""Principal"": {{
 ""AWS"": [""arn:aws:iam::{accountID}:root""]
 }},
 ""Resource"": ""*""
 }},
 {{
 ""Sid"": ""AllowIAMUserAccessKeys"",
 ""Effect"": ""Allow"",
 ""Action"": [""kms:*""],
 ""Principal"": {{
 ""AWS"": ""{iamRole}""
 }},
 ""Resource"": ""*""
 }}
 ]
 }}";

 // Create a new KMS key
 var key = new Kms.Key("stack-encryption-key", new Kms.KeyArgs
 {
 DeletionWindowInDays = 10,
 Description = "KMS key for encrypting Pulumi secret values",
 Policy = keyPolicy
 });

 // Create a new alias to the key
 var alias = new Kms.Alias("alias/stack-encryption-key", new Kms.AliasArgs
 {
 TargetKeyId = key.KeyId
 });

 this.KeyArn = key.Arn;
 this.AliasArn = alias.Arn;
 }

 [Output("keyArn")] public Output<string> KeyArn { get; set; }
 [Output("aliasArn")] public Output<string> AliasArn { get; set; }
}

In this example, we’ve created a key policy which allows full access to a defined IAM role, and we’ve also given read permissions to the key to everyone in this AWS account. We’ve made the IAM role configurable, so let’s set the IAM role now so we can be sure we can use this key for our next Pulumi stack:

# Get the current AWS account ID and set it as a config variable
aws sts get-caller-identity | jq .Account -r | pulumi config set accountID
# Get the current IAM role we're using as set it as a config variable
aws sts get-caller-identity | jq .Arn -r | pulumi config set iamRole

Initialize a New Stack

Now our key has been created and is adequately scoped; we can create a new stack and use the secrets-provider flag on creation to specify the KMS key to encrypt our secrets.

# We need to first retrieve the stack encryption key from our previous stack
KEY_ALIAS=$(pulumi stack output aliasArn | cut -d/ -f2)
# Note, we need to set the region we're deploying to
pulumi new aws-<language> -n <projectname> -s <stackname> -d "An example stack encrypted with AWS KMS" --secrets-provider="awskms://alias/${KEY_ALIAS}?region=us-west-2" --config aws:region=us-west-2 --dir $HOME/git/new-stack

You can check inside your Pulumi.<stackname>.yaml for which key you’re using to encrypt your secrets:

secretsprovider: awskms://alias/<stack-encryption-key-alias>?region=us-west-2
encryptedkey: AQICAHjqW3rb5Hw5Vpxi0c1sayz52VXj7yn20WVwsVILJSBU8wFDjvGuox3wDCJX99TxZFzAAAAAfjB8BgkqhkiG9w0BBwagbzBtAgEAMGgGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMWNeFRZIg8kVXMxrUAgEQgDtz6zV0aqegeAbmaQUNllMp8PQJa1qbjBH813I/XH6LbfynxZO9NE3sYPG89G0u/ltYsADiUAFS0bnadQ==
config:
 aws:region: us-west-2

It’s worth noting here that the “encryptedkey” field here is an encrypted version of the data key in AWS KMS. You can read more about this here.

Use your Encrypted Secret

Now we’ve initialized our stack we can add a secret configuration value using the secret flag and see the encrypted config value in the stack configuration:

pulumi config set --secret supersecret correct-horse-battery-stable

Verify the encryption

To verify that the secret is indeed only accessible to the KMS key we created earlier, we can remove access to the key temporarily and try to perform a Pulumi operation. There are a few ways to remove your access, but first, let’s use the secret in our stack:

import * as pulumi from "@pulumi/pulumi";

const config = new pulumi.Config();
export const superSecret = config.requireSecret("supersecret");

"use strict";
const pulumi = require("@pulumi/pulumi");

const config = new pulumi.Config();
exports.superSecret = config.requireSecret("supersecret");

import pulumi
config = pulumi.Config()

pulumi.export('superSecret', config.require_secret("supersecret"))

package main

import (
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi/config"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 config := config.New(ctx, "")
 superSecret := config.RequireSecret("supersecret")

 ctx.Export("superSecret", pulumi.Sprintf("%s", superSecret))
 return nil
 })
}

using Pulumi;

class AnotherStack : Stack
{
 public AnotherStack()
 {
 var config = new Config();
 this.SuperSecret = config.RequireSecret("supersecret");
 }

 [Output("superSecret")] public Output<string> SuperSecret { get; set; }
}

Now we need to verify if the value is actually encrypted. An easy way to do that is to try and export the secret value without access to the key. How this is done depends on your AWS configuration, however in my setup, I use the AWS_PROFILE environment variable which refers to a named profile to configure access to AWS. If I unset this environment variable, I will no longer be using the AWS credentials that have access to this KMS key. Let’s unset the AWS_PROFILE environment variable and then rerun pulumi up:

unset AWS_PROFILE
pulumi up
error: getting secrets manager: secrets (code=Unknown): InvalidSignatureException: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
 status code: 400, request id: 9cf7508d-a7d5-40bb-b40f-98f68e82ac74

Excellent! We can’t read these values without access to this KMS key. We can be safe in the knowledge our secret values are only readable by us.

Examine the Pulumi State

The final part is to make sure our values are stored encrypted inside the Pulumi state. To do this, we need to use our secret value from earlier and use it in a resource. Let’s create an s3 bucket, and write out super secret value to a file in the bucket. Update the Pulumi program you used before like so:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const config = new pulumi.Config();

const superSecret = config.requireSecret("supersecret");

// Create a private bucket
const bucket = new aws.s3.Bucket("bucket", {
 acl: "private",
});

// Create an object from the secret value
const superSecretObject = new aws.s3.BucketObject("secret", {
 bucket: bucket.id,
 key: "secret",
 content: superSecret, // use our secret value as the content
})

"use strict";
const pulumi = require("@pulumi/pulumi");
const aws = require("@pulumi/aws");

const config = new pulumi.Config();

const superSecret = config.requireSecret("supersecret");

// Create a private bucket
const bucket = new aws.s3.Bucket("bucket", {
 acl: "private",
});

const superSecretObject = new aws.s3.BucketObject("secret", {
 bucket: bucket.id,
 key: "secret",
 content: superSecret, // use our secret value as the content
});

import pulumi
from pulumi_aws import s3

config = pulumi.Config()

superSecret = config.require_secret("supersecret")

# Create a private bucket
bucket = s3.Bucket('bucket', acl="private")

# Create an object from the secret value
bucketObject = s3.BucketObject("secret", bucket=bucket.id, key="secret", content=superSecret)


package main

import (
 "github.com/pulumi/pulumi-aws/sdk/v2/go/aws/s3"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi/config"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {

 config := config.New(ctx, "")

 superSecret := config.RequireSecret("supersecret")

 // Create a a private bucket
 bucket, err := s3.NewBucket(ctx, "bucket", &s3.BucketArgs{Acl: pulumi.String("private")})
 if err != nil {
 return err
 }

 _, err = s3.NewBucketObject(ctx, "secret", &s3.BucketObjectArgs{
 Bucket: bucket.ID(),
 Key: pulumi.String("secret"),
 Content: pulumi.Sprintf("%s", superSecret),
 })

 ctx.Export("superSecret", pulumi.String(superSecret))
 return nil
 })
}

using Pulumi;
using Pulumi.Aws.S3;

class AnotherStack : Stack
{
 public AnotherStack()
 {
 var config = new Config();
 this.SuperSecret = config.RequireSecret("supersecret");

 var bucket = new Bucket("bucket", new BucketArgs {
 Acl = "private",
 });

 var bucketObject = new BucketObject("secret", new BucketObjectArgs {
 Bucket = bucket.Id,
 Key = "secret",
 Content = SuperSecret

 });
 }
 [Output("superSecret")] public Output<string> SuperSecret { get; set; }
}

We now need to look inside our Pulumi statefile to verify the value is encoded there. We can do this with pulumi stack export and some JSON manipulation magic using jq:

pulumi stack export | jq '.deployment.resources[].outputs | select(.content).content'

The result should look like this:

{
 "4dabf18193072939515e22adb298388d": "1b47061264138c4ac30d75fd1eb44270",
 "ciphertext": "v1:cQ5qr21hTdb2PTM5:ZwEW8pN1kC6fUlppi1eS84D/lodoe54wV2dgEsqu0csu2VyTQg0wTf8Qv7axCQ=="
}

Wrap up

This example showed how to use client-side encryption with AWS KMS. Pulumi, as mentioned before, has support for Azure KeyVault, Google Cloud KMS, and HashiCorp Vault for storing your keys. You can find examples of how to use these encryption methods in our examples repo and take a look at our secrets provider documentation.

We hope your next compliance audit is more relaxed with this feature available!