Pulumi Blog: Praneet Loke

Organizing AWS Accounts With Pulumi

Praneet Loke — Mon, 06 Feb 2023 00:00:00 +0000

In an enterprise organization, an IT self-service “vending machine” allows employees to quickly and easily request and receive access to pre-approved cloud resources. Behind the scenes, Pulumi programs may orchestrate any of the requisite resources. We will look at an example of using Pulumi to create an AWS child account, within an AWS Organization.

One might envision a developer requesting a new AWS account to be added entirely through a Pull Request in a git repository either for their own use or on behalf of a team. This scenario is not just for enterprises, however. Teams of any size should organize their AWS accounts using the least-privileged access principle.

Here’s the Organization structure we are looking to build with Pulumi. It just has one organizational unit (OU) with one member account under it to demonstrate how teams can construct the structure that works best for them or follow AWS’ recommendations on this topic. Depending on your team size you may want to keep this simple or you may need something that can handle multiple teams each with multiple accounts. The decision is yours.

For the above organization structure, we want the following operational model:

The user accounts for developers will be provisioned (either manually or automatically through SSO depending on your HR systems) in the root account. Developers will only be able to assume a specific role in the target accounts that have a trust boundary with the root account.
Developers will be added to a group and the group should be granted access to assume a role.
An IAM user meant for automation (eg. CI/CD pipelines) in the root account should have access to assume a role in the target account in order to manage resources. No developer should have access to assume the role meant for the automation user.
An org unit-level (OU) backup policy that backs-up resources tagged with a specific key/value pair.
An org unit-level (OU) tagging policy that enforces tag-casing and the default tags that resources should have.

The component resources mentioned in this post are for demonstration and to show you how you can use the expressiveness found only in the Pulumi programming model. Some details are omitted in this post for brevity. Please see the source repository for the complete source code. In order to modify the organization resources you’ll need credentials to an admin IAM user from your management account. Create an IAM user specifically to run this infrastructure app. These elevated credentials should not be used for anything else.

Once the member accounts have been created, in order to operate in those accounts using your preferred CI/CD service, we’ll create an automation user that will specifically have the permissions to assume a role in the member account. So you should use the credentials of that user for any other resources the team would want to deploy. For example, the team might want to deploy a new ECS cluster, along with Managed Redis instances and perhaps an RDS cluster too. These are resources specific to the team and what they want to use the account for.

If you are using one of the supported CI/CD services you might be able to use federated credentials using OIDC. So if you are going to use that auth mechanism, you can skip creation of the fixed automation user account since you will not need its credentials.

Organizational Unit (OU)

An org unit can represent anything. In our example it represents the development team. You could also have multiple OUs and even nest them to create sub-hierarchies, depending on your use-case.

The organizations module in the AWS provider package contains the resources we need in order to create an OU and an account.

// If you've already created an organization in your root account
// and don't want it to be managed by Pulumi then simply retrieve
// it using the `getOrganization` function.
const organization = aws.organizations.getOrganization({});

// Otherwise, create an organization and let Pulumi manage it.
// If you are going to create the organization using the following
// construction, you should remove the call to getOrganization above.
//
// const organization = new aws.organizations.Organization("");

const devOrgUnit = new aws.organizations.OrganizationalUnit("orgUnit", {
 parentId: organization.then((o) => o.roots[0].id),
 name: "Development",
});

const devAccount = new aws.organizations.Account(
 "devAccount",
 {
 name: "DeveloperAccount",
 parentId: devOrgUnit.id,
 email: devAccountEmailContact,
 // This role can be used by this Pulumi app to execute
 // an AssumeRole action on this new member account.
 // This role is automatically created in every new
 // account created in an Organization. The name
 // `OrganizationalAccountAccessRole` is the default
 // name. You may choose a different name.
 // https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_accounts_access.html
 roleName: "OrganizationalAccountAccessRole",
 // IMPORTANT! Set this to `false` if you do not wish to have
 // accounts closed when the account resource is removed from
 // your Pulumi app.
 closeOnDeletion: true,
 },
 // This flag tells Pulumi to protect this resource from being deleted
 // during a `pulumi destroy` operation. It also means you need to
 // first unprotect it if you do wish to delete it using Pulumi.
 // https://www.pulumi.com/docs/concepts/options/protect/
 { protect: true }
);

An important note about the Account resource and destroying them. Notice how we set the closeOnDeletion to true. Closing accounts on deletion will only succeed if you have not exceeded the quotas set by AWS for account closure. So set that flag accordingly. If you expect to be closing many accounts frequently (perhaps for an ephemeral use-case), you may want to set the flag to false so you can close them manually or have some other sort of asynchronous automation do it for you. Otherwise, your Pulumi execution will fail when AWS returns an error code due to the quota being exceeded.

Developers’ Access

We’ll create an IAM group for developers and assign a group policy that allows its members to perform the sts:AssumeRole on any AWS account but only using a specific role.

const devGroup = new aws.iam.Group("developers", {
 name: "developers",
});

new aws.iam.GroupPolicy("developersGroupPolicy", {
 group: devGroup.name,
 policy: {
 Statement: [
 {
 Effect: "Allow",
 Action: "sts:AssumeRole",
 // This role will be created by the AccountPermissions component resource
 // introduced in the next section of this post.
 Resource: `arn:aws:iam::*:role/DeveloperRole`,
 },
 ],
 Version: "2012-10-17",
 },
});

Next, we’ll create the IAM user that we’ll use for automation purposes. Again, you don’t need to do this if you plan to use OIDC federation for your CI/CD service.

const automationUser = new aws.iam.User("automationUser", {
 name: "cicd-automation",
});

new aws.iam.UserPolicy("automationUserPolicy", {
 policy: {
 Statement: [
 {
 Effect: "Allow",
 Action: "sts:AssumeRole",
 // This role will be created by the AccountPermissions component resource
 // introduced in the next section of this post.
 Resource: `arn:aws:iam::*:role/AutomationRole`,
 },
 ],
 Version: "2012-10-17",
 },
 user: automationUser.name,
});

Member/Target Account Permissions

We’ve created the base infrastructure in our root (aka management) account. We now need to create roles in the target account that allows users in the management account to assume.

We’ll use a component resource to set up the necessary roles and policies in the target account. If you recall, when the target account was created, we specified a role name. That role name is the initial role that AWS creates on our behalf in the brand new account with admin privileges. The credentials we are using currently to execute our Pulumi app is assumed to be an admin user in the root account since we are dealing with highly-privileged operations here. We will need to create an explicit provider for our target account that can assume that automatically-created role.

const devAccountProvider = new aws.Provider("devAccountProvider", {
 allowedAccountIds: [devAccount.id],
 assumeRole: {
 roleArn: pulumi.interpolate`arn:aws:iam::${devAccount.id}:role/OrganizationalAccountAccessRole`,
 },
});

Now, whenever we want to modify resources in the target account in this app, we’ll use that provider. We’ll pass this provider to the component mentioned above, so that the roles and policies created by that component are performed using this provider instance. The component resource code is not included in this post for brevity. It provides a nice way to encapsulate the roles and policies we want to bootstrap our brand new account with. Of course, you can customize those roles and what permissions they get based on your use-case.

const devAccountPermissions = new AccountPermissions(
 "devAccountPermissions",
 {
 automationUser,
 managementAccountId: pulumi.Output.create(
 aws.getCallerIdentity().then((i) => i.accountId)
 ),
 // The prefix that will be used to name the child resources created by this component
 // since we expect to create a new instance of this component for every new member
 // account.
 resourceNamesPrefix: "dev"
 },
 { provider: devAccountProvider }
);

Backup Policies

Backup policies are strictly not required in order to create new member accounts in an OU. However, many teams have a requirement to have a backup policy for certain resources. Aside from the compliance requirement, it’s a good idea to ensure that your DBs, EBS volumes etc. are properly backed-up. AWS Backup is a managed backup service that allows you to define backup plans and create vaults in primary (where your resources reside) and a secondary (backup/copy region). AWS will execute your backup plans and create backups in the vaults.

We’ll once again use a component resource to encapsulate the creation of a backup vault in each of our member accounts. AWS recommends that we attach the backup policy to the organizational unit so that all member accounts inherit the policy implicitly.

AWS needs to know which resources to target to perform the backup. We provide that information using the selections property in the backup policy. The selections property allows you to specify tags that can be used as backup targets. In other words, compatible resources that have that tag will be targeted by this backup policy. In our example, the tag key is BackupType and the value is MONTHLY. This tag is optional, of course, since we don’t need every resource to have this tag; just the ones that need to be backed-up. You can enforce required tags too, which is a nice segue to our next topic.

const vault = new aws.backup.Vault(
 `backupVault-${accountName}`,
 // We rely on the vault name staying consistent across AWS accounts
 // for the purposes of the backup policy that is attached to the
 // organizational unit.
 { name: "Default" },
 { provider: accountProvider, parent: this }
);

// Create the IAM role in the target account that allows AWS Backup to execute backup plans.
const backupPolicyRole = new aws.iam.Role(
 `${accountName}BackupPolicyRole`,
 {
 // This name has to stay consistent across all member accounts
 // since we will reference this role in the actual backup policy.
 name: "MyMonthlyBackupIamRole",
 assumeRolePolicy: {
 Statement: [
 {
 Effect: "Allow",
 Action: "sts:AssumeRole",
 Principal: {
 Service: "backup.amazonaws.com",
 },
 },
 ],
 Version: "2012-10-17",
 },
 managedPolicyArns: [
 "arn:aws:iam::aws:policy/service-role/AWSBackupServiceRolePolicyForBackup",
 ],
 },
 { provider: accountProvider, parent: this }
);

The actual backup policy is a JSON document with a special syntax. In our example, the monthly plan looks like this:

Monthly_Backup_Plan: {
 regions: {
 "@@assign": this.primaryRegions,
 },
 rules: {
 Monthly: {
 schedule_expression: {
 "@@assign": "cron(0 5 1 * ? *)",
 },
 start_backup_window_minutes: { "@@assign": "480" },
 target_backup_vault_name: { "@@assign": "Default" },
 lifecycle: {
 move_to_cold_storage_after_days: {
 "@@assign": "30",
 },
 delete_after_days: { "@@assign": "365" },
 },
 copy_actions: backupVaultCopyAction,
 },
 },
 selections: {
 tags: {
 Backup_Assignment: {
 iam_role_arn: {
 "@@assign": `arn:aws:iam::$account:role/${this.monthlyBackupPolicyIamRoleName}`,
 },
 tag_key: { "@@assign": "BackupType" },
 tag_value: { "@@assign": ["MONTHLY"] },
 },
 },
 },
},

Tagging Policies

Similar to backup policies, tag policies are also not required to create member accounts in an OU. But it’s a good idea to define one. In keeping with our general theme here, we’ll encapsulate our tagging policy with yet another component resource so that we can easily include future org units. Similar to the backup policy syntax, the tag policy syntax uses JSON with some special placeholder variables.

Let’s take a look at the consumer point-of-view for this component. We want users of this component to pass in a list of allowed cost centers for a certain org unit. Say, if a team wants to have multiple org units and therefore different cost centers for each of them, this model allows those teams to use the TagPolicies component for each org unit separately and assign the allowed cost center values accordingly.

const tagPolicies = new TagPolicies("tagPolicies", {
 costCenters: [
 {
 // The component accepts an array of allowed cost center
 // values that can be used as values in tags.
 // This is just an example of how you can restrict what values
 // tags can be assigned.
 allowedCostCenters: ["Development", "Testing"],
 ou: devOrgUnit,
 },
 ],
 orgId: organization.then((o) => o.roots[0].id),
});

The tag policy for our example looks like this:

{
 type: "TAG_POLICY",
 content: JSON.stringify({
 tags: {
 CostCenter: {
 tag_value: {
 // The allowed cost center values that we passed
 // to the TagPolicy component as an arg.
 "@@assign": costCenter.allowedCostCenters,
 },
 },

 // Add tags that are enforceable only to the OU.
 Owner: {
 tag_key: {
 "@@assign": "Owner",
 // Don't allow any child policies from changing the key of this tag.
 "@@operators_allowed_for_child_policies": [
 "@@none",
 ],
 },
 },
 },
 }),
}

And that’s about it as far as setting up the simple org unit and member account structure shown at the beginning of this post. View the full source-code for the component resources in the pulumi/examples repository.

Here’s how the stack looks once the organizational unit and the development AWS account in it have been created.

Suppose there is a need to add a new AWS account for a test account in addition to the development account. We just need a new Account resource, the backup policy and the permissions for it.

const testAccount = new aws.organizations.Account(
 "testAccount",
 {
 name: "SharedTestAccount",
 parentId: devOrgUnit.id,
 email: testAccountEmailContact,
 roleName: initialRoleName,
 closeOnDeletion: true,
 },
 { protect: true }
);

const testAccountProvider = new aws.Provider("testAccountProvider", {
 allowedAccountIds: [testAccount.id],
 assumeRole: {
 roleArn: pulumi.interpolate`arn:aws:iam::${devAccount.id}:role/${initialRoleName}`,
 },
});

const testAccountPermissions = new AccountPermissions(
 "testAccountPermissions",
 {
 automationUser,
 managementAccountId: pulumi.Output.create(
 aws.getCallerIdentity().then((i) => i.accountId)
 ),
 resourceNamePrefix: "test",
 },
 { provider: testAccountProvider }
);

And with those additions, we get this preview:

This post demonstrates how you could use the Pulumi programming model to encapsulate and account for the hierarchical organization structure that your team desires or is required to have for any number of reasons. While an enterprise organization might be interested in this sort of a setup to ensure multiple teams in the organization operate in a similar way, smaller teams should also strongly consider organizing their AWS accounts in a way that enforces the principle of least privileged access. Especially, if you are looking to get SOC2 certified and need to ensure strict isolation of resources that hold any kind of real-world customer data.

Manage AWS accounts as code

Use Pulumi component resources to provision AWS Organizations units, member accounts, and least-privilege roles in a repeatable way. Build a self-service account structure your team can request and review in a pull request.

Learn more

Next Steps

There’s so much more you can do with AWS Organizations and Pulumi.

Learn about service control policies and how you can centrally control the maximum available permissions in a member account, for example.
These Pulumi features could be combined to provide a powerful self-service mechanism for your organization: Automation API, Policy-as-Code (similar in its principle to AWS SCPs but operates at the Pulumi resource-level so you can validate every resource created using Pulumi)
Best practices for your management account
Best practices for your member accounts

Infrastructure CI/CD With Github Actions and Pulumi

Paul Stack — Mon, 14 Dec 2020 00:00:00 +0000

GitHub Actions help automate tasks within your software development life cycle. At Pulumi, we use GitHub Actions internally as part of every build/release cycle, and we run these tasks many, many times per day. This helps us to automate our CI/CD process and eliminate manual steps. Pulumi and many of our customers also deliver infrastructure resources as part of a CI/CD process: delivering infrastructure and applications in a single, integrated pipeline.

Better Automation with GitHub Actions and Pulumi

Today, we’re pleased to announce that Pulumi is available as a verified GitHub Action in the GitHub Marketplace. It’s now easier than ever to start automating your infrastructure deployments with Pulumi and GitHub Actions. Our new guide shows you how to get started with GitHub Actions and Pulumi. Here are just a few great examples of how to use the Pulumi GitHub Action that will simplify your cloud engineering tasks:

Master Builds

name: Pulumi
on:
 push:
 branches:
 - master
jobs:
 up:
 name: Update
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v1
 - uses: pulumi/actions@v1
 with:
 command: up
 env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}

Pull Request Builds

name: Pulumi
on:
 - pull_request
jobs:
 preview:
 name: Preview
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v2
 - uses: pulumi/actions@v1
 with:
 command: preview
 env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
 GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 IS_PR_WORKFLOW: true
 COMMENT_ON_PR: 1

Announcing the New Pulumi CI/CD Integration Assistant

We often hear from developers who want to setup CI/CD automation for their stacks but don’t have the time to figure out how to configure a working pipeline. Today, we are happy to announce the launch of the CI/CD Integration Assistant to make setting up CI/CD automation more straightforward. You can easily get started using popular CI/CD services without the hassle of the initial setup.

The integration assistant guides you with instructions based on your preferred CI/CD service and provides the customized pipeline configuration for CI/CD services that you can add to your Version Control System (VCS) repository.

We are super excited to offer GitHub Actions support as part of the CI/CD Integration Assistant and other popular tools like Azure Pipelines and more. If you’d like to request support for more CI/CD services, please use this convenient template on the Pulumi GitHub repo.

Managing Your Infrastructure with a VCS

You may already be familiar with Git-based services such as GitHub, Bitbucket, and GitLab. Adding your code to a central VCS allows you to secure your codebase against failures and boost team collaboration. When you add an automated CI/CD pipeline, your team will deliver software safely and efficiently by ensuring that automated tests are passed prior to every deployment. This process is valuable for application code and infrastructure code.

The Assistant also goes beyond helping you get started with an automated pipeline. It works throughout the Pulumi Service providing hints for configuring a VCS for projects that are not already configured. It can also help identify updates to stacks that were not run in an automated pipeline (in the activity tab). These additions will help you quickly identify the stacks where best practices are not followed and take corrective actions. You can access the Assistant in the Integrations option of your stack’s settings page to help you through the whole process of configuring a VCS to setting up a working automated pipeline.

Learn more about the CI/CD Integration Assistant in our user guide.

We are eager to hear your feedback on how we can improve your experience with this feature. If you haven’t already signed up for our Community Slack, it’s quick and easy! You can join in on conversations you like and get help from other community members, as well as the Pulumi Team. That’s it for now! 👋

Amp-up GitLab Merge Requests With Pulumi

Praneet Loke — Wed, 26 Aug 2020 00:00:00 +0000

We are excited to announce the launch of native support for integrating GitLab Merge Requests with Pulumi. By integrating your GitLab Projects directly with Pulumi, you can now approve your merge requests with confidence.

New To Pulumi and GitLab CI/CD?

GitLab’s merge request pipelines runs a CI/CD workflow for building and testing your software. Pulumi makes a great addition to it, by allowing you to preview any infrastructure changes as part of your merge request flow.

Using the merge request pipeline feature with Pulumi can validate the changes to your infrastructure before they are applied. Once validated and approved through a peer-review process, you can apply your changes quickly and continuously with confidence while doing so, a process commonly referred to as GitOps for infrastructure.

To leverage GitOps for your infrastructure, the Pulumi CLI offers two commands: preview and up.

Concept	Pulumi command	Role
Merge requests pipeline	`pulumi preview`	Preview infrastructure changes to be merged.
Master branch pipeline	`pulumi up`	Update your infrastructure with latest changes

Learn how to use Pulumi in GitLab CI/CD with merge request pipelines and how to configure the GitLab project integration with Pulumi.

Already Using Both?

If you are already a user of either Pulumi and GitLab CI/CD, you know the importance of merge request pipelines and running Pulumi within them. With today’s announcement, you can now make any changes to your infrastructure in your merge request front and center, so you don’t miss any of those changes.

Here’s a preview of what that looks like 😉

Head over to our docs to learn how to configure the GitLab integration, whether for a single project or all of the projects under a GitLab Group.

GitLab provides lots of tools for streamlining your application delivery, and the new integration with Pulumi makes things even better. As always, we’d love to hear your feedback or suggestions for other improvements. Check out the Pulumi Community Slack.

Unlocking Spinnaker With Pulumi

Praneet Loke — Thu, 18 Jun 2020 00:00:00 +0000

Spinnaker is an open source, multi-cloud continuous delivery platform for releasing software changes with high velocity and confidence. This makes it a perfect fit for Pulumi to unleash the power of Spinnaker and its continuous delivery capabilities to all of the providers that Pulumi supports today and in the future.

We have written a developer guide showing you how to install the plugin in your Spinnaker instance and start using it right away.

Here’s a brief overview of the plugin.

Create a new Spinnaker pipeline with the Pulumi stage.

When you run a pipeline that has a Pulumi stage in it, you’ll be able to see the logs of the Pulumi CLI within your pipeline execution details window. Click the Console Output to view the logs.

Here’s a demo of the plugin:

If you have any questions or feedback about the plugin, please feel free to open an issue in the repo or join the conversation on our Community Slack as always.

Dynamic Providers

Praneet Loke — Thu, 16 Jan 2020 00:00:00 +0000

Pulumi has many resource providers that allow you to interact with your favorite cloud or resource. There are times when a provider may not deliver on the specific task that you want to accomplish. Dynamic Providers can be a powerful tool to help accomplish your infrastructure tasks.

A provider manages the CRUD (Create, Read, Update, Delete) life-cycle of a resource against some underlying service. The underlying service can be one of the public clouds or an on-premise service. For example, AWS, Azure, GCP, and Kubernetes have resource providers, which can manage resources by request.

Resource Provider

A resource provider is made up of two different pieces:

A resource plugin, which is the binary used by the deployment engine to manage a resource. These plugins are stored in the plugin cache (located in ~/.pulumi/plugins) and can be managed using the pulumi plugin set of commands.
An SDK which provides bindings for each type of resource the provider can manage.

A resource provider is an extension of the API exposed by the respective cloud providers. As such, it can be the limiting factor. Thankfully, there is a solution to this. Since the Pulumi engine works with life-cycle callbacks handing control to the provider at appropriate times, it is easy to write a provider that implements custom logic in those callbacks.

Learn more about how Pulumi works here.

Dynamic Resource Provider

Note: Dynamic providers are currently only supported in JavaScript, TypeScript, and Python.

Pulumi provides appropriate life-cycle callbacks through the pulumi.dynamic.ResourceProvider abstract class. While Pulumi does not know what sort of resources a Dynamic Provider will create, the life-cycle hooks must return data in specific formats, which Pulumi uses to call the appropriate callbacks.

For example, if the user changes an input property to your dynamic resource, the diff hook will be called with both the old (if applicable) and new inputs. You can compare then to see which input properties have changed and give hints to Pulumi as to whether delete should be called or whether to call the update hook.

The check life-cycle hook will be called before any other hook to let you validate the inputs and populate defaults, if necessary.

When a dynamic resource instance updates, the delete hook will be called to allow you to perform any necessary cleanups before the dynamic resource is removed from your stack’s state.

Provider Credentials

If you need to use the credentials used by the rest of your Pulumi app, also known as the ambient provider credentials, you can access the relevant credentials (and other configuration) through the <provider>.config where provider is the imported provider module.

Sample Use Cases

Dynamic providers can be used for almost any task. Since Pulumi uses general-purpose programming languages, you can do almost anything you would outside of your infrastructure app. You can even use libraries that are standard to the respective programming language.

Here are some use cases:

Run a command-line tool when a dependent resource is created.
Copy files to a VM once the VM is created.
Call REST APIs in each life-cycle hook (CRUD) to adopt an external resource under the cloud resource model.

Examples

Enable Azure Storage’s Static Websites Feature

The Azure resource provider does not support enabling the static website feature for a storage account. However, there is a REST API that can be called to enable the feature, so we can easily call the API from within the dynamic provider.

Here’s the source for this example. The following is an excerpt from the example where the dynamic provider is actually used. Note that the Azure Storage Account is created before using this dynamic provider.

...
...
// Create a Storage Account for our static website
const storageAccount = new azure.storage.Account("websitesa", {
 resourceGroupName: resourceGroup.name,
 accountReplicationType: "LRS",
 accountTier: "Standard",
 accountKind: "StorageV2",
});

const staticWebsite = new StorageStaticWebsite("website-static", {
 accountName: storageAccount.name,
});

Add A Custom Domain To An Azure CDN endpoint

Similar to the previous example, this is another example of a shortcoming of the regular Azure resource provider available in Pulumi. However, due to the availability of a REST API, we can easily add a custom domain to an Azure CDN resource using a dynamic provider.

Along with adding a custom domain to the CDN endpoint, this dynamic provider also enables HTTPS provided by Azure’s one-click HTTPS enablement.

As before, details such as the creation of the CDN profile and its endpoint are omitted for clarity. You can check out the full example here.

...
...
const cdnEndpoint = new azure.cdn.Endpoint("cdnEndpoint", {
 /**
 * Specify a well-known name for the endpoint name,
 * so you can add a CNAME record for your custom domain
 * pointing to this CDN endpoint to it.
 *
 * For example, the URL for this CDN endpoint when it is created
 * would be `my-cdn-endpoint.azureedge.net`.
 */
 name: "my-cdn-endpoint",
 resourceGroupName: resourceGroup.name,
 profileName: cdnProfile.name,
 isHttpsAllowed: true,
 isHttpAllowed: false,
 isCompressionEnabled: true,
 originHostHeader: storageAccount.primaryBlobHost,
 contentTypesToCompresses: [
 "text/plain",
 "text/html",
 "text/css",
 "text/javascript",
 "application/x-javascript",
 "application/javascript",
 "application/json",
 "application/xml",
 "image/png",
 "image/jpeg",
 ],
 origins: [
 {
 name: "cdn-origin",
 hostName: storageAccount.primaryBlobHost,
 httpsPort: 443,
 },
 ],
});

export const cdnCustomDomainResource = new CDNCustomDomainResource("cdnCustomDomain", {
 resourceGroupName: resourceGroupName,
 // Ensure that there is a CNAME record for mycompany.com
 // pointing to my-cdn-endpoint.azureedge.net.
 // You would do that in your domain registrar's portal.
 customDomainHostName: "mycompany.com",
 profileName: cdnProfileName,
 endpointName: cdnEndpointName,
 // This will enable HTTPS through Azure's one-click
 // automated certificate deployment.
 // The certificate is fully managed by Azure from provisioning
 // to automatic renewal at no additional cost to you.
 httpsEnabled: true,
 }, { parent: cdnEndpoint });

Dynamic Providers As Provisioners

Provisioning a VM after it is created is a common problem. Developers have the option to run user-supplied scripts while creating the VM itself. For example, the AWS EC2 resource has a userData parameter, that allows you to specify an inline script, which EC2 will run at instance startup.

Scripts provided through the userData parameter can be configured to run every time the instance is restarted. But what if you wanted to run the script every time the script changes? To do that you could use SCP to transfer the file and then execute the script on the VM. This example shows you how you can do that. Although the example uses TypeScript and is specific to AWS EC2, you can easily adapt this to other cloud providers.

This snippet of the code shows how, the dynamic resources for copying a file and executing it, are used:

...
...
const conn = {
 host: server.publicIp,
 username: "ec2-user",
 // The EC2 instance is created with the corresponding SSH public key.
 // Read more here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html.
 privateKey,
 privateKeyPassphrase,
};

// Copy a config file from the local filesystem to the instance.
const cpConfig = new provisioners.CopyFile("config", {
 conn,
 src: "myapp.conf",
 dest: "myapp.conf",
}, { dependsOn: server });

// Execute a basic command on the instance.
const catConfig = new provisioners.RemoteExec("cat-config", {
 conn,
 command: "cat myapp.conf",
}, { dependsOn: cpConfig });
…
...

Read more about dynamic providers in our docs. Let us know what cool dynamic providers you create by dropping us a note on our community Slack!

GitHub & Pulumi Join Forces To Ensure Pulumi Tokens Are Safe

Praneet Loke — Mon, 19 Aug 2019 00:00:00 +0000

We are very excited to announce that we have partnered with GitHub to offer our users better protection for their Pulumi Access Tokens.

By default, Pulumi users manage the state of their cloud infrastructure deployments using https://app.pulumi.com. This service provides state storage, concurrency control, audit history and access controls for both individuals and teams working with Pulumi. Each user and service account can generate one or more Pulumi Access Tokens to be used to authenticate with this service. These access tokens can be used on both local development machines, as well as in CI/CD systems for automated infrastructure deployments. These access tokens are sensitive secrets which should never be shared publicly, and in particular should never be committed to source control.

How GitHub Token Scanning Works for Pulumi Users

With the recent changes to integrate Pulumi with GitHub Token Scanning, GitHub now scans each commit you push to a public repo on github.com for any files containing these tokens. When GitHub’s Token Scanning Service finds a potential match for our tokens, the service will call ours to have us verify the token. Only Pulumi Access Tokens generated after June 28, 2019 that contain the prefix pul- will be matched by the scanning service.

During the processing of the requests from the Token Scanning Service, if we find that the token belongs to a Pulumi account, we will use the primary email on the account to notify the user. We will not, however, revoke the token automatically. So ensure that your account’s email address is up-to-date by going to your profile in the Pulumi Service.

The whole process, from the check-in to the email notification, typically takes just a few seconds, minimizing the window of an accidental breach. Of course, best practice is to avoid checking in the token at all!

Avoid Checking-In The Token In The First Place

Pulumi Access Tokens grant access to your stacks. They allow you to login into the CLI non-interactively and thus allow the CLI to access a stack’s configuration, which may in-turn contain other secrets. The token is typically used inside CI/CD environments. So you should never have to check-in a token to your source tree (public or otherwise.)

Most CI/CD services have a way for you to specify secret environment variables that allow you to safely inject them into your build environment. We have written-up quite a few guides on how to integrate Pulumi into your CI/CD. If you don’t see a guide for a CI/CD system that you are using, feel free to create an issue here, or even better, raise a PR for it. We love community contributions! :)

CI/CD Made Easy with Pulumi and Azure Pipelines

Praneet Loke — Mon, 06 May 2019 00:00:00 +0000

Azure DevOps is very popular among teams that want a single place to manage their development pipelines, Git repositories, builds, releases, and test plans. Pulumi’s open-source tools are a great choice for developers and operators deploying infrastructure as code on Azure. With these two tools at hand, adopting CI and CD for your Azure infrastructure is just a few steps away for you and your teams.

To make it easy to use Pulumi with Azure, we are announcing an open-source task extension for Azure Pipelines! The task extension will manage the installation of the Pulumi CLI, and run the Pulumi commands you specify against your stack.

You can install the task extension directly from the Visual Studio Marketplace. Click the “Get It Free” button to install the extension into your Azure DevOps organization. If you do not have the permissions to add it to your organization, please contact your org admin to have the extension installed. The source code for the open source task extension can be found on GitHub.

To use the Pulumi task extension in Azure Pipelines, there are two options. The full example is in the source repo. For contrast, without this task extension, here’s how you would achieve the same, but with scripts.

OPTION 1: Using the Classic Editor Console

Pasted below is a typical view of the Editor User Interface. Click on the "+" button next to the job. As visible in the screenshot, the job is called “Agent job 1”. The agent pool you choose does not matter. The Pulumi task can run on both Linux as well as Windows images.

Search for the Pulumi task by typing in the filter input on the right. Click the Add button.

Fill in the required information. You can learn about creating service connections for Azure here.

If you want to understand all the commands available with the Pulumi CLI, click on the information icon (i) next to Pulumi Command, and click on the help link to go to our quick reference page.

Option 2: Using a YAML File

Here’s an example of the Pulumi task extension defined in YAML configuration.

- task: Pulumi@0
 inputs:
 azureSubscription: '<your-service-connection-name>'
 command: "up"
 args: "--yes --skip-preview"
 cwd: "./"
 # Replace this with your org's name.
 stack: "<org-name>/pulumi-az-pipelines-task-example/dev"

Moving to Production

Once you’ve configured your Pulumi task extensions for your Azure Pipelines, you can now more easily integrate Pulumi into your CI/CD pipeline, and take advantage of previews to infrastructure changes in pull requests, push-to-deploy, and ultimately removing the friction for your DevOps.

As always, we’d love to hear what you think. Say 👋 in the Pulumi Community Slack or drop us a line.

Want to learn more? Check out this post from Mikhail Shilkov for a detailed look at deploying applications on Azure with Pulumi. Also, keep an eye on our blog posts this week for more posts about Azure.

Announcing Support for Email-based Identities

Praneet Loke — Thu, 21 Mar 2019 00:00:00 +0000

We have been hard at work the past few months providing our users with more ways to connect to Pulumi. Here are some our past announcements related to identities:

Today, we are pleased to announce that we are launching support for email-based identities. You no longer need to use a social identity to sign-up for an account on Pulumi. Just fill out the signup form, and you are ready to go.

Quickstart

Get started quickly by creating a new template-based project. This creates a project, and a stack ready to be used by the Pulumi CLI. The wizard also shows you the commands you need to run in the CLI.

Haven’t setup your CLI for a specific cloud yet? Check out this docs page on how to do that.

Once you’ve deployed your first stack, the next step is up to you. You can refer to our documentation on integrating Pulumi into your CI/CD pipeline, setting up chat-ops workflows via Webhooks, or taking your Kubernetes solutions to the next level!

We’d love to hear what you’re up to. Say 👋 in the Pulumi Community Slack or drop us a line.

Pulumi now supports Atlassian Identity

Praneet Loke — Wed, 30 Jan 2019 00:00:00 +0000

Today we added support for yet another developer favorite product, Atlassian Bitbucket. You can now sign-up for a Pulumi account with an Atlassian identity. This also means you can connect your Atlassian identity with an existing Pulumi account.

This work follows on from the support for GitLab identity and also the ability to connect identities, eliminating the need for users to create multiple accounts on Pulumi.

This helps users with repos across the major version control systems to seamlessly import their GitHub Organizations and GitLab Groups - and now Atlassian Bitbucket Teams - into a single Pulumi account. Of course, you don’t need to connect identities. You can always create separate account for each of your identities, if that’s what you want to do.

Creating a new Pulumi account using your Atlassian identity

To get started using your Atlassian identity, you can navigate to app.pulumi.com and select the Atlassian button to authenticate. If you are already logged-in to Pulumi, you should first logout and then head back to the Pulumi Service.

Connecting an existing Pulumi account to your Atlassian identity

If you want to connect your Atlassian identity to an existing Pulumi account, then navigate to app.pulumi.com and once logged in, head to Account Settings to choose the option to connect your Atlassian identity.

Bitbucket Team-backed Organizations

A Bitbucket Team helps you organize users into user groups and therefore, assign each of them a permission to your repositories. By adding a Bitbucket Team-backed organization, you can invite the same members to collaborate on Pulumi.

This allows you to set permissions on Stacks owned by your organization. You can learn more about Teams in the Pulumi and how those permissions get applied too.

CI Integration

Did you also know that the Pulumi CLI can be used in most, if not all, CI/CD workflows? Check out our documentation for integrating with several popular CI/CD systems. Get up and running with Pulumi in CI on GitLab, Travis, Azure DevOps and more. Don’t see docs for a particular CI system? Let us know or better yet file an issue here.

Having trouble? Questions? Join our community Slack or send us an email.

Connecting multiple identities to Pulumi

Praneet Loke — Fri, 14 Dec 2018 00:00:00 +0000

Hot on the heels of our GitLab sign-in support, we’ve just released support for multiple identities for a single Pulumi account in the Pulumi Service. Previously, you could only sign-up for a new Pulumi account using a GitHub or GitLab identity. Starting today, you can connect your Pulumi account with additional identities, beyond what you first signed-up with.

For example, if you signed-in to Pulumi with your GitHub account, you could only add GitHub organizations to the Pulumi Service, or collaborate with other GitHub users. With multiple identities, you can now also connect your Pulumi account to a GitLab identity too!

To connect an identity, first sign-in as usual on https://app.pulumi.com, then go to your Profile page as shown below.

Based on the example, on the Profile page, you would see a new button to connect a GitLab account. Conversely, if you signed-up originally using your GitLab account onto Pulumi, then you would see a button to connect your GitHub account.

Connecting multiple identities to Pulumi has advantages:

You can then add your GitHub organizations or GitLab groups, to collaborate with the respective members of those organizations using Pulumi.
A single Pulumi account connected to additional identities allows for more ways to sign into the Pulumi Service.

However, you don’t have to associate multiple identities with the same Pulumi account. Whenever you sign-in with a new identity for the first time on app.pulumi.com, a new Pulumi account is created. You can then choose to add additional identities at a later time.

Welcoming GitLab users to Pulumi

Praneet Loke — Mon, 03 Dec 2018 00:00:00 +0000

We are very pleased to announce that Pulumi now supports signing-in with your GitLab account. Don’t have an account on GitLab yet? Head on over to https://gitlab.com and sign-up for an account.

Why GitLab?

GitLab is a fully-integrated DevOps service that not only provides git repository management but also allows you to collaborate using issue boards, setup build and deployment pipelines. It provides an end-to-end solution that starts from your code and ends with your customers, all without ever having to leave GitLab.

And with Pulumi on GitLab, you truly never have to leave GitLab, not even to provision your infrastructure to run your services on the cloud.

Learn more about GitLab.

Log into app.pulumi.com using the GitLab Sign-in.
Click on the organizations dropdown menu and select Add an organization.
On the next page, you may select your GitLab group, based on which a Pulumi organization will be created.
Once the organization is created, you can create stacks in your newly created organization, so that other members of your organization can collaborate with you.

Inviting members in your GitLab group to use Pulumi

Select the organization you created from the organization dropdown menu. You should see a People tab. Select the People tab.

You will see the People tab if you added the organization on Pulumi or if you are an Owner of the GitLab group on gitlab.com.

Members must have an active and valid role in your GitLab group, so that you may invite them.

From the People tab, you can invite additional members in your group to also use Pulumi. By inviting members in your group to join Pulumi, you can collaborate with them more easily and share stacks, just like you would collaborate on a project on GitLab.

Running Pulumi on GitLab

Pulumi can be run in many CI/CD environments. Pulumi can be easily integrated in your CI pipeline on GitLab, too.

Deep-linking to your GitLab projects, branches and commits

When you run Pulumi in a GitLab-based repo, the pulumi CLI recognizes the correct metadata based on your repository’s remote configuration and uses that information to enhance your experience on app.pulumi.com.

Here’s an example of a project hosted on gitlab.com.

Here’s the same project with some activity having links to the commit-hash, and the branch in which this stack was updated.

FAQ

Click the GitLab sign-in button on <app.pulumi.com>, Pulumi will redirect you to gitlab.com where GitLab will present you with the sign-in options to login into your GitLab account. At that point, you may choose any of the sign-in options GitLab provides to sign-in.

I already have an account on Pulumi. Will signing-in with my GitLab identity create a new account?

Yes. Signing-in with a GitLab account will create a new account. That means, your stacks and activity will stay with the other account. You can migrate them by performing a pulumi stack export from your source stack and then importing it using pulumi stack import in a new stack in your GitLab-based account.

If you would like to add your GitLab identity to your existing Pulumi account, you can do so by connecting your GitLab identity from your Pulumi account’s profile page.

One of the benefits of using <app.pulumi.com> is to track the state of your stacks. When you are running the pulumi CLI on your machine, you can login into your account by typing pulumi login. There are two options for you to complete the login process. You can either create an Access Token on <app.pulumi.com> or simply press ENTER to let the CLI launch the browser.

If you would like to let the CLI launch the browser, ensure that you are already signed-in using GitLab at <app.pulumi.com> using your machine’s default browser. This way, when the browser is launched by the CLI, your Pulumi account based on your GitLab identity would be automatically used.