Pulumi Blog: Guinevere Saenger

Introducing envVarMappings for Provider Credentials

Guinevere Saenger — Thu, 12 Feb 2026 00:00:00 +0000

Running multiple providers with different credentials in the same Pulumi program has always been tricky. Providers expect fixed environment variable names like AWS_ACCESS_KEY_ID or ARM_CLIENT_SECRET, so if you need two AWS providers targeting different accounts, you couldn’t configure them both via environment variables.

Pulumi v3.220.0 introduces envVarMappings, a new resource option that solves this problem by letting you remap provider environment variables to custom keys.

When configuring a Pulumi provider to authenticate against a cloud provider, there are two main options available. You can set authentication values as secrets in your Pulumi.yaml config:

$ pulumi config set azure-native:clientSecret <clientSecret> --secret

Alternately, you can also use the terminal environment of where you’re running your Pulumi commands:

$ export ARM_CLIENT_SECRET=1234567

Using environment variables in this manner is especially useful in CI environments, or when you’d rather not write that auth token to state, even encrypted. But there’s currently several use cases where this breaks down, due to the hard-coded nature of the environment variables that a given provider expects.

Multiple providers or provider instances that expect different authentication values but have the same variable key

For example, if you are using multiple explicit providers targeting different Azure accounts, you were not able to set these separate configurations via environment variable.

Instead, users would have to set these values in the provider config, which may not be desirable for all use cases. Not only does the provider config write secrets to state (albeit of course encrypted), but it can also result in a noisy diff on an otherwise no-op upgrade when token rotation is used.

Remapping environment variables

For this and similar scenarios, we have a new solution for you: setting mappings of environment variable keys on your provider. The concept is as follows:

“For any environment variable that my Pulumi provider expects, I want to be able to tell the provider to use the value of a custom-defined environment variable instead.”

Example

Let’s say your provider expects ARM_CLIENT_SECRET, but you want it to use a different value than the one set in your shell. First, define a custom environment variable with your desired value:

$ export CUSTOM_ARM_CLIENT_SECRET=7654321

Then, use envVarMappings to tell the provider: “When you look for ARM_CLIENT_SECRET, read from CUSTOM_ARM_CLIENT_SECRET instead.” The mapping format is { "SOURCE_VAR": "TARGET_VAR" }:

const provider = new command.Provider("command-provider", {}, {
 envVarMappings: {
 // If CUSTOM_ARM_CLIENT_SECRET exists, provider sees the value of ARM_CLIENT_SECRET
 "CUSTOM_ARM_CLIENT_SECRET": "ARM_CLIENT_SECRET",
 },
});

provider = command.Provider("command-provider",
 opts=pulumi.ResourceOptions(
 env_var_mappings={
 # If CUSTOM_ARM_CLIENT_SECRET exists, provider sees the value of ARM_CLIENT_SECRET
 "CUSTOM_ARM_CLIENT_SECRET": "ARM_CLIENT_SECRET",
 }
 )
)

provider, err := command.NewProvider(
 ctx,
 "command-provider",
 &command.ProviderArgs{},
 pulumi.EnvVarMappings(map[string]string{
 // If CUSTOM_ARM_CLIENT_SECRET exists, provider sees the value of ARM_CLIENT_SECRET
 "CUSTOM_ARM_CLIENT_SECRET": "ARM_CLIENT_SECRET",
 }),
)

var provider = new Command.Provider("command-provider", new Command.ProviderArgs(), new CustomResourceOptions
{
 EnvVarMappings = new Dictionary<string, string>
 {
 // If CUSTOM_ARM_CLIENT_SECRET exists, provider sees the value of ARM_CLIENT_SECRET
 { "CUSTOM_ARM_CLIENT_SECRET", "ARM_CLIENT_SECRET" }
 }
});

var provider = new Provider("command-provider", ProviderArgs.Empty, CustomResourceOptions.builder()
 .envVarMappings(Map.of(
 // If CUSTOM_ARM_CLIENT_SECRET exists, provider sees the value of ARM_CLIENT_SECRET
 "CUSTOM_ARM_CLIENT_SECRET", "ARM_CLIENT_SECRET"
 ))
 .build());

resources:
 command-provider:
 type: pulumi:providers:command
 options:
 envVarMappings:
 # If CUSTOM_ARM_CLIENT_SECRET exists, provider sees the value of ARM_CLIENT_SECRET
 CUSTOM_ARM_CLIENT_SECRET: ARM_CLIENT_SECRET

You can now customize each environment variable value your provider sees by defining a new environment variable, and then mapping your provider’s defined variable to yours. Try it out with Pulumi v3.220.0 today!

For full details, see the envVarMappings documentation.

Happy coding!

Pulumi Google Cloud Provider Version 9.0.0

Guinevere Saenger — Wed, 24 Sep 2025 13:33:50 -0700

We’re excited to announce the v9 release of the Pulumi Google Cloud Provider! This major release contains important updates to Google Cloud resources and functions, and keeps you up to date with what’s new from Pulumi.

The Pulumi Google Cloud provider can be used to provision any of the Google Cloud resources available in the upstream provider. The provider is open source and available on GitHub so you can be part of the community - issues and pull requests are always welcome!

Here are a few links to help you get started if you are new to Pulumi:

Getting Started - A guided walkthrough for creating your first project
Setup & Install - Instructions on installing the Google Cloud provider
How-to guides - Learn how to use the Google Cloud provider to provision specific resources
Templates - Use a quickstart template to create a new project
Pulumi Neo - Ask Pulumi Neo to help you with your project

In case you missed it - new modules

Throughout the past year, we have worked to keep this provider in sync with the upstream provider. Notable new modules are those supporting AI and Google Gemini. New modules added since the last major release include:

What’s New in 9.0

Upstream Improvements

This release includes the latest improvements and changes from the recent major version in the upstream provider. Some themes are:

Improved Import Validation

Multiple resources throughout the provider have improved validation and error messages to ensure the supplied GCP resource ID matches import formats specified in the documentation.

Increased Field Validation

Several resource fields have been updated to reflect requirements of Google Cloud and will fail validation if not configured correctly, to avoid failures later on. Refer to the migration guide for details.

Documentation Improvements

We are always working on improving and updating our documentation. Among other improvements, this release adds 100 new top-level overview descriptions for resources and functions.

Upgrading

You can find the migration guide on the Pulumi registry.

NEW! Use Pulumi Neo to assist with your upgrade

You can use our new automation agent, Neo, to help you with your provider upgrade. In the Pulumi service, ask Neo to help you with your upgrade by providing the migration guide:

Neo will look at the provided migration guide and your stack, and determine a plan of action. Here, Neo has discovered there were some breaking changes, needing updates to your program.

And finally, Neo can open a pull request with the proposed changes to your code.

Pulumi Google Cloud Provider Version 8.0.0

Guinevere Saenger — Thu, 19 Sep 2024 10:08:42 -0700

The latest major release of the Pulumi Google Cloud Provider is available now! Our 8.0 release contains the latest upstream changes to keep you up-to-date along with the latest features and improvements from Pulumi.

The Pulumi Google Cloud provider can be used to provision any of the Google Cloud resources available in the upstream provider. The provider is open source and available on GitHub so you can always follow along with current issues and developments, or even open your first pull request.

Here are a few links to help you get started if you are new to Pulumi:

Getting Started - A guided walkthrough for creating your first project
Setup & Install - Instructions on installing the Google Cloud provider
How-to guides - Learn how to use the Google Cloud provider to provision specific resources
Templates - Use a quickstart template to create a new project
Pulumi Neo - Ask Pulumi Neo to create a new project

Looking Back

Since the last major release of this provider, we have continuously shipped improvements to our ecosystem, bringing the latest Pulumi features to your production stack. We have rolled out an improved diffing strategy and fixed state upgrades in the Pulumi Terraform Bridge, removing spurious diffs on preview and increasing confidence when deploying. Additionally we have improved accuracy and coverage for registry documentation, via better example conversion and general docs improvements for bridged providers.

New Modules

Over the last year, we have added support for several new modules. Among these are:

API Coverage Growth

The below chart shows the growth of this provider by resource, function, and supporting types since Version 7.

What’s New in 8.0

Added Deletion Protections

Several resources have new deletionProtection fields:

gcp.cloudrunv2.Service
gcp.cloudrunv2.Job
gcp.activedirectory.Domain
gcp.organizations.Folder
gcp.organizations.Project

Find the documentation for any of these resources in our registry

New default provisioning label

A new default label, goog-pulumi-provisioned lets you discover your Pulumi-provisioned resources in the GCP console to help you track resources and how they were created. This label is available as an Output only and can be disabled in your provider configuration.

Upgrading

You can find our v7 -> v8 Migration Guide on the Pulumi Registry.

Fast Docker Image Builds with Pulumi

Monica Rodriguez — Wed, 08 Mar 2023 14:00:00 -0800

How do I speed up Docker image builds with Pulumi? Use BuildKit (the default since Docker 23), enable a registry or layer cache so repeated builds reuse work, write a multi-stage Dockerfile so production images skip build-time dependencies, and reach for the dedicated Docker Build provider when you need buildx features like multi-platform images, build secrets, or Docker Build Cloud. With these techniques together, repeat builds in a Pulumi program commonly drop from minutes to seconds.

TL;DR

BuildKit is the default in current Docker releases and in the Pulumi Docker providers — you generally do not need to opt in.
Cache your layers somewhere persistent. A registry cache (cacheFrom / cacheTo) is what makes CI builds fast across runners and contributors.
Multi-stage Dockerfiles are the single biggest win for image size and rebuild speed when you have a compile or bundle step.
Use pulumi/docker-build for new projects. It exposes the full buildx surface (multi-platform, secrets, multiple cache backends, Docker Build Cloud).
Use pulumi/docker v4+ if you already depend on it — its docker.Image resource also rebuilds only on context change and supports BuildKit and registry caching.

What actually moves the needle

The numbers below are representative of what teams report from a typical Node.js or Python web app (single Dockerfile, ~300 MB final image). Your mileage varies with project size, network, and runner specs — treat this as a directional comparison, not a benchmark suite.

Configuration	Cold build	Repeat build (no source change)	Repeat build (small source change)
Legacy builder, no caching	minutes	minutes	minutes
BuildKit, local layer cache only	minutes	seconds (cache hit)	tens of seconds
BuildKit + multi-stage Dockerfile	minutes	seconds (cache hit)	tens of seconds
BuildKit + multi-stage + registry cache (`cacheFrom`/`cacheTo`)	minutes (first run)	seconds (cache hit)	seconds–tens of seconds, even on a fresh CI runner
`docker-build` provider with buildx + Docker Build Cloud	under 2 min	seconds (cache hit)	seconds

The headline pattern: each row past the first is dominated by what the build can skip, not by raw CPU. Pulumi’s job is to wire those caches up declaratively so they keep working across every pulumi up.

How does Pulumi build Docker images?

Pulumi has two providers for image builds, and which one you should use depends on how new your project is.

The Docker provider (pulumi/docker) is the original. Its docker.Image resource can build a local context and push to any registry as part of a Pulumi deployment, alongside resources for containers, networks, and volumes. Since v4 it rebuilds only when the build context changes and uses BuildKit by default.
The Docker Build provider (pulumi/docker-build) is a newer, dedicated provider focused exclusively on image builds. It exposes Docker’s buildx interface, so you get multi-platform builds, multiple cache backends, build secrets, and Docker Build Cloud as first-class features.

For new projects, prefer docker-build. For existing programs that already use docker.Image, you do not have to migrate — docker.Image continues to receive the speed and caching improvements covered below.

Why are my Docker builds slow?

Three things make Docker builds slow in practice, and Pulumi addresses each one declaratively.

No cache, or the wrong cache. A fresh CI runner has no local Docker layers. Without a remote cache, every build reinstalls every dependency.
Cache busting on noise. If your Dockerfile copies package.json and the rest of your source in the same step, every source change invalidates the dependency-install layer. Order layers from least-changing to most-changing.
Building dev tooling into the production image. Compilers, linters, and test frameworks have no business in a runtime image. Multi-stage builds let you discard them.

How do I enable BuildKit and registry caching in Pulumi?

BuildKit has been the default builder since Docker 23 and is the default in both Pulumi providers. The piece you usually have to configure is where the cache lives — local disk on a developer laptop is fine, but for CI you almost always want a registry cache so layers persist across runners.

With the Docker Build provider (recommended)

import * as docker_build from "@pulumi/docker-build";

const image = new docker_build.Image("my-image", {
 tags: ["docker.io/pulumibot/demo-image:latest"],
 context: { location: "./app" },
 dockerfile: { location: "./app/Dockerfile" },
 platforms: [
 "linux/amd64",
 "linux/arm64",
 ],
 cacheFrom: [{ registry: { ref: "docker.io/pulumibot/demo-image:cache" } }],
 cacheTo: [{ registry: { ref: "docker.io/pulumibot/demo-image:cache", mode: "max" } }],
 push: true,
});

cacheFrom and cacheTo together are the important pair: the first run populates the cache, every later run on any machine pulls it. mode: "max" exports cache for every layer, not just the final ones.

With the Docker provider (`docker.Image`)

import * as docker from "@pulumi/docker";

const image = new docker.Image("my-image", {
 imageName: "docker.io/pulumibot/demo-image:latest",
 build: {
 context: "app",
 dockerfile: "Dockerfile",
 platform: "linux/amd64",
 cacheFrom: {
 images: ["docker.io/pulumibot/demo-image:cache-base"],
 },
 },
});

The docker.Image resource accepts a list of cache images via cacheFrom.images. Tag a previous build with a stable cache tag (for example, :cache-base) and reference it here so subsequent builds reuse its layers.

How do I write a Dockerfile that builds fast?

The Dockerfile matters at least as much as the Pulumi code wrapping it. Two patterns do most of the work.

Order layers least-volatile first. Copy lockfiles and run dependency installs before copying source code. That way changing a .ts file does not bust the npm ci layer.

# Dependencies — only invalidates when lockfiles change
COPY package.json package-lock.json ./
RUN npm ci

# Source — changes constantly, but the dependency layer above survives
COPY . .
RUN npm run build

Use multi-stage builds to drop tooling. Compile in a fat image, copy the artifact into a slim one. The runtime image stays small and the build stage gets cached independently.

FROM node:20 AS build
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:20-slim AS runtime
WORKDIR /app
COPY --from=build /app/dist ./dist
COPY --from=build /app/node_modules ./node_modules
CMD ["node", "dist/server.js"]

When you combine multi-stage with a registry cache (cacheTo mode max), CI runs on fresh machines reuse both stages — the row in the table above where small source changes finish in 10–30 seconds.

How do I build multi-platform images with Pulumi?

Use the docker-build provider and pass a list to platforms. BuildKit will build all variants in parallel and push a manifest list, so a single tag serves whatever architecture the puller needs.

const image = new docker_build.Image("api", {
 tags: ["ghcr.io/example/api:latest"],
 context: { location: "./api" },
 platforms: [
 "linux/amd64",
 "linux/arm64",
 ],
 push: true,
});

If you are still on docker.Image, the build.platform field accepts a single target (for example, "linux/amd64"); for true multi-arch manifests, prefer docker-build.

How do I see Docker build output in Pulumi?

Filtered Docker build and push logs render directly in pulumi up output for both providers, so you can debug a failing layer without leaving your Pulumi run. With the Docker Build provider, structured progress shows per-stage timing — useful when you are trying to figure out which layer is the slow one.

How do I migrate an older `docker.Image` program?

If you are upgrading from a pre-v4 program, pulumi-docker v4 reshaped the docker.Image resource and renamed several supporting types, but does not manage backend state, so updating the package version and adjusting types is generally enough. The full migration guide lives in the v4.0.0 release notes.

If you are starting fresh or want the full BuildKit feature set, install pulumi-docker-build instead and follow the Docker Build provider getting started guide.

Getting started

Docker Build provider — recommended for new projects.
Docker provider — the original, including docker.Image, containers, and networks.
Introducing the Docker Build provider — deeper dive on buildx, secrets, and Docker Build Cloud.

The cheapest performance win in most Pulumi programs is wiring up a registry cache and a multi-stage Dockerfile. Start there, then reach for buildx features as the project grows.

Multicloud with Kubernetes and Pulumi

Guinevere Saenger — Thu, 09 Jun 2022 11:40:47 -0700

In this article we’ll show you how to use Pulumi Components and the Pulumi Automation API to make golden path decisions which will both support your customers on multiple different clouds, and enable infrastructure teams and frontend service teams to more easily own their respective parts of your codebase.

Let’s say you provide a hosted service for your customers. You would like to offer customizable Kubernetes cluster provisioning, on multiple different clouds, as well as your product service itself.

There’s a lot of configuration fine-tuning and manual repetition that is required when bringing up Kubernetes clusters on different clouds.

This article is a written summary of our KubeCrash live demo, see it below. Please find the repository with the full version of the code here.

Part One: Create a component

A Component Resource is an abstraction on top of other Pulumi packages, often combining a few different providers into something that fits your infrastructure needs precisely.

Prerequisites

In our example, we are using three separate clouds, one of these being a local KinD cluster.

Nodejs
Pulumi CLI
Have the Pulumi Registry documentation handy for some Pulumi providers:
- Pulumi Civo Provider
- Pulumi Linode Provider
- Pulumi Kubernetes Provider
- Pulumi KinD Provider (Kubernetes-in-Docker) Note: As this is a community package, follow installation instructions from the README. This provider only supports Go and Nodejs.

Gather access tokens for Linode and Civo, if using, and set them in the environment or as secrets.

Start by writing the code for each cloud

The below code, when run against pulumi up, will create a single cloud stack with three separate Kubernetes clusters. The Stack output will be the cluster’s name and kubeconfig.

import * as pulumi from "@pulumi/pulumi";
import * as civo from '@pulumi/civo';
import * as linode from "@pulumi/linode";
import * as kind from "@pulumi/kind";


//Civo

// Create a firewall
const fw = new civo.Firewall("guin-fw", {
 region: "nyc1",
 createDefaultRules: true,
});

const civoCluster = new civo.KubernetesCluster("guin-civo-demo", {
 name: "guin-demo",
 region: "nyc1",
 firewallId: fw.id,
 pools: {
 nodeCount: 3,
 size: "g4s.kube.xsmall",
 }
});

export const clusterNameCivo = civoCluster.name
export const kcCivo = civoCluster.kubeconfig

//Linode

const linodeCluster = new linode.LkeCluster("guin-linode-demo", {
 k8sVersion: "1.22",
 label: "guin-demo",
 pools: [{
 count: 2,
 type: "g6-standard-2",
 }],
 region: "us-central",
 tags: ["guin-demo"],
});

export const clusterLabelLinode = linodeCluster.label
export const kcLinode = linodeCluster.kubeconfig

// Kind

const kindCluster = new kind.cluster.Cluster(
 'guin-kind',
 {
 name: pulumi.interpolate`guin-kind`,
 nodes: [
 {
 role: kind.node.RoleType.ControlPlane,
 extraPortMappings: [
 {
 containerPort: args.nodePort,
 hostPort: args.nodePort,
 },
 ],
 },
 {
 role: kind.node.RoleType.Worker,
 },
 ],
 },
)

export const clusterNameKind = kindCluster.name
export const kcKind = kindCluster.kubeconfig

We could stop here, export the kubeconfigs to a file, and start interacting with our clusters via kubectl. But that would require our cluster operators to be deeply familiar with each cloud provider’s Kubernetes implementation. Let’s see if we can do better.

Create a custom component for the Kubernetes cluster

Our goal today is to bring up a “generic” Kubernetes cluster that fits a single use case for multiple clouds. To do so, we create a default implementation as a wrapper around each cloud provider. We implement the following features:

Choice of cloud
Cluster Naming
Kubeconfig Output for later use

The Pulumi SDK allows us to wrap resources into a ComponentResource, which in our Typescript example means we create a new class that extends pulumi.ComponentResource.

export class ShinyCluster extends pulumi.ComponentResource {
 constructor(
 name: string,
 args: {},
 opts: {},
 ) {
 # ...
 }
}

Because we like all things shiny, we’ll call it a ShinyCluster, and pass pkg:index:ShinyCluster as the unique resource name.

We’ll also register the name and kubeconfig for each ShinyCluster as Outputs:

import { Output } from '@pulumi/pulumi'; // add this to dependencies

#...

export class ShinyCluster extends pulumi.ComponentResource {
 constructor(
 name: string,
 args: {},
 opts: {},
 ) {
 super('pkg:index:ShinyCluster', name, {}, opts);
 }
 name: Output<string>;
 kubeConfig: Output<string>;
}

Write the implementation for each cloud provider:

export class ShinyCluster extends pulumi.ComponentResource {
 constructor(
 name: string,
 args: {
 cloud: 'linode' | 'civo' | 'kind';
 nodePort?: number; // this is a KinD-specific add-on so we can connect to our workload
 },
 opts: {},
 ) {
 super('pkg:index:ShinyCluster', name, {}, opts);
 switch (args.cloud) {
 case 'linode':
 const linodeCluster = new linode.LkeCluster(
 'my-cluster',
 {
 label: 'guin-linode',
 k8sVersion: '1.22',
 pools: [
 {
 count: 2,
 type: 'g6-standard-2',
 },
 ],
 region: 'us-central',
 tags: ['guin'],
 },
 { parent: this },
 );
 this.name = linodeCluster.label;
 this.kubeConfig = linodeCluster.kubeconfig.apply((x) =>
 Buffer.from(x, 'base64').toString(),
 );
 break;
 case 'civo':
 const fw = new civo.Firewall(
 'guin-fw',
 {
 name: 'guin-civo',
 region: 'nyc1',
 createDefaultRules: true,
 },
 { parent: this },
 );

 const civoCluster = new civo.KubernetesCluster(
 'guin-civo',
 {
 region: 'nyc1',
 name: 'guin-civo',
 firewallId: fw.id,
 pools: {
 nodeCount: 3,
 size: 'g4s.kube.xsmall',
 },
 },
 { parent: this },
 );
 this.name = civoCluster.name;
 this.kubeConfig = civoCluster.kubeconfig;
 break;
 case 'kind':
 const kindCluster = new kind.cluster.Cluster(
 'guin-kind',
 {
 name: 'guin-kind',
 nodes: [
 {
 role: kind.node.RoleType.ControlPlane,
 extraPortMappings: [
 {
 containerPort: args.nodePort,
 hostPort: args.nodePort,
 },
 ],
 },
 {
 role: kind.node.RoleType.Worker,
 },
 ],
 },
 { parent: this },
 );
 this.name = kindCluster.name;
 this.kubeConfig = kindCluster.kubeconfig;
 }
 }
 name: Output<string>;
 kubeConfig: Output<string>;
}

We are now ready to use our component.

Read in the configuration from the config file and create a cluster:

const config = new pulumi.Config();

const shinyCluster = new ShinyCluster(
 'guin',
 {
 cloud: config.require('cloud'),
 nodePort: config.getNumber('nodePort'),
 },
 {},
);

export const shinyName = shinyCluster.name;
export const shinyConfig = pulumi.secret(shinyCluster.kubeConfig);

In our terminal, we can now create a stack called dev, set the necessary inputs in the Pulumi config, and run pulumi up:

$ pulumi stack init dev
Created stack 'dev'
$ pulumi config set shinycluster:cloud civo
$ pulumi up
Previewing update (dev)

View Live: https://app.pulumi.com/guinevere/shinycluster/dev/previews/cf968a89-dadc-461f-badd-b062601c18b4

 Type Name Plan
 + pulumi:pulumi:Stack shinycluster-dev create

At this point, we have created a Pulumi Component Resource that we can use to scaffold Kubernetes infrastructure.

Side Quest: Ship an app!

The purpose of this demo is not focused on the details of deploying a web app on Kubernetes with Pulumi, so we won’t go into too much detail here. We created a shinyapp folder to hold a Pulumi program (written in yaml!) We could now follow the Kubernetes provider configuration steps to set up and deploy our shinyapp to each of our clusters.

But if we have multiple Kubernetes backends, and would like to deploy our shinyapp to each of them, this would get tedious soon. Here is where we automate deploying our frontend as well.

Part 2: Leverage the Automation API

The Pulumi Automation API allows us to run Pulumi commands without using the CLI manually.

Let’s review our basic project file structure first, for context. We place both our shinycluster backend and shinyapp frontend Pulumi projects in their own folders, to separate concerns, and for visual clarity.

.
├── shinyapp
│ ├── Main.yaml # our web app, declared in yaml
│ └── Pulumi.yaml # Pulumi project config for `shinyapp`, with yaml runtime
├── shinycluster
│ ├── index.ts # our showcased Pulumi Component Resource for a Kubernetes cluster
│ └── Pulumi.yaml # Pulumi project config for `shinycluster`, with nodejs (Typescript) runtime

The neat thing here is that both our Kubernetes clusters and our application service are in their own contained folder, and in fact each can be deployed independently using Pulumi. Let’s go ahead and implement the functionality that allows us to automate it all by adding the following files to the root of the project:

├── Pulumi.yaml # Pulumi project config for the entire demo
├── index.ts # Input and output logic for our frontend and backend stacks
├── outputFormatter.ts # Prettify output
├── runPulumiProject.ts # Our specific automation implementation, using Automation API
└── stackModule.ts # Wrapper logic to show which types are expected by each part of our project

Not shown: dependency and package files.

The Stack Module

First, we define a generic Pulumi stack, with variable working directories and outputs, as expected by the automation API.

// stackModule.ts
import * as pulumi from '@pulumi/pulumi';

export type Unwrapped<T> = NonNullable<pulumi.Unwrap<Awaited<T>>>;

export type OutputMap<T> = {
 [K in keyof T]: {
 value: T[K];
 secret: boolean;
 };
};

export type StackModule<T> = {
 workDir: string;
 projectName: string;
 stack(): Promise<T>;
};

type StackOutputs<T> = T extends StackModule<infer O> ? O : never;

/**
 * The type of the outputs, as returned by the automation API as an "OutputMap".
 */
export type StackOutputMap<T> = OutputMap<Unwrapped<StackOutputs<T>>>;

/**
 * The type of the outputs, as if executed and run in a Pulumi program:
 */
export type StackOutputValues<T> = Unwrapped<StackOutputs<T>>;

/**
 * The type of a function that gets values by name from the outputs of a stack.
 *
 * See `stackOutputConfig`.
 */
export type StackOutputGetter<T> = <
 K extends keyof StackOutputValues<T> & string,
>(
 key: K,
) => StackOutputValues<T>[K];

Wrap the Automation API

We have a few functionalities that we can expand on and implement in just the way we like, so we put them in a file called runPulumiProject.ts.

// runPulumiProject.ts
import {
 ConfigMap,
 LocalProgramArgs,
 LocalWorkspace,
 OpMap,
 PreviewResult,
 Stack,
 UpResult
} from '@pulumi/pulumi/automation';

For the purposes of this demo repo, we added a formatter as well, but it’s not necessary for the functionality:

import chalk from 'chalk';
import { Formatter } from './outputFormatter';

And we import the output mappings we abstracted away in the stack module, above.

import { OutputMap, Unwrapped } from './stackModule';

We can now decide how we want to implement our automation options.

Let’s define the options we’d like this program to be able to run with. We need:

Directory name
Project name
Stack name
Pulumi operation to run
Extra configuration (so we can pass in that kubeconfig)
The formatter
Stack outputs

// runPulumiProject.ts
interface PulumiRunOptions {
 dir: string;
 project: string;
 stackName: string;
 operation: 'up' | 'preview' | 'destroy';
 additionalConfig: ConfigMap;
 formatter: Formatter;
}
interface PulumiRunResult<T> {
 stack: Stack;
 projectName: string;
 outputs: OutputMap<Unwrapped<T>>;
}

In this example, the pulumi commands we automate are:

Preview
Up
Destroy

export async function runPulumiProject<T extends object>({
 dir, project, stackName, operation, additionalConfig, formatter,
}: PulumiRunOptions): Promise<PulumiRunResult<T> | undefined> {
 const localProgramArgs: LocalProgramArgs = {
 stackName,
 workDir: dir,
 };
 const stack = await LocalWorkspace.createOrSelectStack(localProgramArgs, {});

 formatter(`Spinning up stack ${project}/${stackName}`);
 await stack.setAllConfig({
 ...additionalConfig,
 });

 formatter('Refreshing');
 await stack.refresh();

 let result: UpResult | PreviewResult;
 let status: string = 'succeeded';
 let outputs: OutputMap<Unwrapped<T>>;
 let summaryMessage: string | undefined;
 let operations: OpMap;
 switch (operation) {
 case 'preview':
 formatter('Previewing');

 const previewResult = await stack.preview({ onOutput: formatter });
 operations = previewResult.changeSummary;
 outputs = (await stack.outputs()) as OutputMap<Unwrapped<T>>;
 break;
 case 'up':
 formatter('Deploying');

 const upResult = await stack.up({ onOutput: formatter });
 operations = upResult.summary.resourceChanges;
 status = upResult.summary.result;
 summaryMessage = upResult.summary.message;
 outputs = upResult.outputs as OutputMap<Unwrapped<T>>;
 break;
 case 'destroy':
 formatter('Destroying');

 const destroyResult = await stack.destroy({ onOutput: formatter });
 operations = destroyResult.summary.resourceChanges;
 status = destroyResult.summary.result;
 summaryMessage = destroyResult.summary.message;
 break;
 }

 if (status !== 'succeeded') {
 formatter(result.stderr);
 formatter(summaryMessage);
 throw new Error();
 }

 if (operations) {
 formatter('Succeeded! Resource summary:');
 const fmtNum = (num?: number) => `${num}`.padStart(3);
 if (operations?.create) {
 formatter(`${fmtNum(operations?.create)} ${chalk.green('created')}`);
 }
 if (operations?.replace) {
 formatter(`${fmtNum(operations?.replace)} ${chalk.magenta('replaced')}`);
 }
 if (operations?.update) {
 formatter(`${fmtNum(operations?.update)} ${chalk.yellow('updated')}`);
 }
 if (operations?.same) {
 formatter(`${fmtNum(operations?.same)} ${chalk.bold('unchanged')}`);
 }
 }

 return {
 stack,
 projectName: project,
 outputs: outputs as OutputMap<Unwrapped<T>>,
 };
}

In addition, this code gives us the ability to take a stack’s Output and apply it to another stack, which is what we will do next, in our top-level index.js file.

Putting it all together

// index.js
import { chooseColor, outputFormatter } from './outputFormatter';
import { runPulumiProject } from './runPulumiProject';

interface ClusterOutputs {
 shinyName: string;
 shinyConfig: string;
}

ClusterOutputs defines the information we need to pass on the kubeconfig to our Kubernetes web app, and the cluster name to associate each frontend stack with its backend Kubernetes cluster.

Here, we’re deploying our shinyapp to a local KinD cluster, but we have the ability to add any cluster to this list:

const clusters = [
 {name: 'kind-local', cloud: 'kind', nodePort: 32001},
];

First, we wait for our clusters to be created (note the operation: 'up' being passed):

await Promise.allSettled(
 clusters.map(async (clusterDefinition) => {
 let theme = chooseColor();

 const serviceType = clusterDefinition.nodePort ? 'NodePort' : 'LoadBalancer';
 const serviceNodePort = `${clusterDefinition.nodePort ?? 0}`;
 const cluster = await runPulumiProject<ClusterOutputs>({
 dir: './shinycluster',
 project: 'shinycluster',
 stackName: clusterDefinition.name,
 operation: 'up',
 additionalConfig: {
 cloud: { value: clusterDefinition.cloud },
 nodePort: { value: serviceNodePort },
 },
 formatter: outputFormatter(`cluster ${clusterDefinition.name}`, theme)
 });
 // here we will call runPulumiProject, this time on the web app
 }),
 );

And then deploy our web app:

runPulumiProject({
 dir: './shinyapp',
 project: 'shinyapp',
 stackName: clusterDefinition.name,
 operation: 'up',
 additionalConfig: {
 kubeconfig: cluster.outputs.shinyConfig,
 serviceType: { value: serviceType },
 serviceNodePort: { value: serviceNodePort },
 },
 formatter: outputFormatter(`app ${clusterDefinition.name}`, theme)
});

Putting it all together:

// index.js
import { chooseColor, outputFormatter } from './outputFormatter';
import { runPulumiProject } from './runPulumiProject';

interface ClusterOutputs {
 shinyName: string;
 shinyConfig: string;
}

async function main() {
 const clusters = [
 { name: 'kind-local', cloud: 'kind', nodePort: 32001 },
 ];

 await Promise.allSettled(
 clusters.map(async (clusterDefinition) => {
 let theme = chooseColor();

 const serviceType = clusterDefinition.nodePort ? 'NodePort' : 'LoadBalancer';
 const serviceNodePort = `${clusterDefinition.nodePort ?? 0}`;
 const cluster = await runPulumiProject<ClusterOutputs>({
 dir: './shinycluster',
 project: 'shinycluster',
 stackName: clusterDefinition.name,
 operation: 'up',
 additionalConfig: {
 cloud: { value: clusterDefinition.cloud },
 nodePort: { value: serviceNodePort },
 },
 formatter: outputFormatter(`cluster ${clusterDefinition.name}`, theme)
 });

 runPulumiProject({
 dir: './shinyapp',
 project: 'shinyapp',
 stackName: clusterDefinition.name,
 operation: 'up',
 additionalConfig: {
 kubeconfig: cluster.outputs.shinyConfig,
 serviceType: { value: serviceType },
 serviceNodePort: { value: serviceNodePort },
 },
 formatter: outputFormatter(`app ${clusterDefinition.name}`, theme)
 });
 }),
 );
}

main();

Run the program

All that is left to do is run our top-level program:

$ ts-node ./index.js

Once that’s done running, go visit your webapp appear on localhost with the specified port:

$ curl localhost:32002
<!DOCTYPE html>
<html>

<head>
 <meta http-equiv="Content-Type" content="text/html" charset="utf-8">
</head>

<body>
 <h1>Made with love by Pulumi!</h1>
</body>

</html>

Summary

We chose some cloud providers for bringing up Kubernetes clusters
We created a ShinyCluster custom resource that abstracts away any cloud provider implementation details, giving us a small set of defaults we care about
We programmatically deployed infrastructure using the Automation API
We deployed an app to this infrastructure, also using Automation API.

We hope you have gotten a little bit of inspiration from our whirlwind tour through these Pulumi features. Happy coding!

How we manage GitHub at Pulumi with Pulumi

Guinevere Saenger — Tue, 22 Mar 2022 11:29:33 -0700

We recently updated this article to reflect that parentTeamId of the Team resource accepts a string instead of an int since v5.9.1 of the Pulumi Github package.

Ah, GitHub. The home of all developers. The place where we share code. The world’s most awkward social media site. The secret LinkedIn for techies. The tool we use for company org structure, work planning, code ownership, and permissions…

Wait.

That’s quite a lot.

GitHub is good at many things, but a full-on organization management tool it is not.

Have you ever needed your manager to manually enable admin permission on a repo for you? Or have you needed to page the CEO to add you to a team, because your manager was out that day? Have you ever wondered who is on what team? Or which team owns a repo? What if you change teams, or a team changes names? A reorg happens, and the “platform-integrations” team is no more, but we still need to call it that on GitHub because it is the team with all the repository accesses?

When I joined Pulumi in 2021, all of the above happened to me within my first few weeks.

We at Pulumi wanted to reduce this kind of management friction, and we decided to solve it the Pulumi way: with declarative infrastructure using the Pulumi GitHub provider.

The Goal

First, we set a few ground rules. We decided that administrative privileges over Pulumi’s GitHub repositories should flow as follows:

An employee is part of one or more GitHub teams.
Teams have appropriately scoped permissions over each repository they own.
Any Pulumi employee should be able to request code repository access in an official way.

Additionally, any current team membership and repository access should not be disrupted during development of this tool.

Getting Started

Create a new Pulumi program:

$ pulumi new go
This command will walk you through creating a new Pulumi project.

Enter a value or leave blank to accept the (default), and press <ENTER>.
Press ^C at any time to quit.

project name: (team-management)
project description: (A minimal Go Pulumi program)
Created project 'team-management'

Please enter your desired stack name.
To create a stack in an organization, use the format <org-name>/<stack-name> (e.g. `acmecorp/dev`).
stack name: (dev) pulumi/prod
Created stack 'prod'

Installing dependencies...

go: downloading github.com/pulumi/pulumi/sdk/v3 v3.26.1
Finished installing dependencies

Your new project is ready to go! ✨

To perform an initial deployment, run 'pulumi up'

$ ls
Pulumi.yaml go.mod go.sum main.go

We now have a pulumi project YAML configuration file and the beginnings of a small Go program all set up.

According to the provider configuration documentation, we need to add a properly scoped token, as well as set “pulumi” as our GitHub organization owner.

$ export GITHUB_TOKEN=YYYYYYYYYYYYYY
$ export GITHUB_OWNER=pulumi

Capturing Current State of Resources

Now, Pulumi is great at creating new infrastructure from scratch via code. But this wasn’t a from-scratch situation. We had to migrate existing resources—GitHub teams—to Pulumi, without disrupting anyone’s access.

Enter Pulumi Import.

What Pulumi Import does, in a nutshell, is find existing infrastructure by unique ID (in the GitHub provider’s case, the team ID), and add them to a Pulumi Stack. You can find the specific import instructions on the registry documentation for each resource. In this case we want the GitHub import instructions.

$ pulumi import github:index/team:Team animals 1234567
Previewing import (pulumi/prod)

 Type Name Plan
 pulumi:pulumi:Stack team-mgmt-prod
 = └─ github:index:Team animals import

Resources:
 = 1 to import
 12 unchanged

Do you want to perform this import? yes
Importing (pulumi/prod)

 Type Name Status
 pulumi:pulumi:Stack team-mgmt-prod
 = └─ github:index:Team animals imported

Resources:
 = 1 imported
 1 unchanged

Now that we have imported the Team resource, it is part of our Pulumi Stack. But it is not part of our main.go program yet. Helpfully, Pulumi Import replies with sample code on how to add the imported resource to your Pulumi program, in the language you selected for your project:

Please copy the following code into your Pulumi application. Not doing so will cause Pulumi to report that an update will happen on the next update command.

Please note that the imported resources are marked as protected. To destroy them you will need to remove the `protect` option and run `pulumi update` *before* the destroy will take effect.


package main

import (
 "github.com/pulumi/pulumi-github/sdk/v4/go/github"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err := github.NewTeam(ctx, "animals", &github.TeamArgs{
 CreateDefaultMaintainer: pulumi.Bool(false),
 Description: pulumi.String("should get imported"),
 Name: pulumi.String("animals"),
 Privacy: pulumi.String("closed"),
 }, pulumi.Protect(true))
 if err != nil {
 return err
 }
 return nil
 })
}

After adding the team to main.go as suggested, this is the output of pulumi preview:

Previewing update (pulumi/prod)

 Type Name Plan Info
 pulumi:pulumi:Stack team-mgmt-prod

Resources:
 2 unchanged

Since pulumi preview shows no changes, we now know that our code reflects the existing infrastructure. It’s a bit funny to think about your program working well when it does nothing, but this was a huge first step in preserving existing infrastructure and ensuring all of our coworkers could continue their daily work uninterrupted!

To finish up, we unprotect the resource:

$ pulumi state unprotect 'urn:pulumi:prod::team-mgmt::github:index/team:Team::animals'

Creating Configuration

Next, we import the remaining teams and refactor the code to handle multiple teams at once. This is also the time to write our organization structure into a YAML configuration file.

---
org: "pulumi-demo--org"
teams:
 - name: "animals"
 slug: "animals"
 description: "A team in charge of animals"
 - name: "newteam"
 slug: "newteam"
 description: "I am a new team"

We then read this information into a Go struct with the necessary properties to pass into the Pulumi provider.

type Team struct {
 Name string `yaml:"name"`
 Description string `yaml:"description"`
 Slug string `yaml:"slug"`
}
type Organization struct {
 Org string `yaml:"org"`
 Teams []Team `yaml:"teams"`
}

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {

//import accurate team info from yaml
orgFilePath, err := filepath.Abs("./org.yaml")
if err != nil {
 return err
}
yamlFile, err := ioutil.ReadFile(orgFilePath)
if err != nil {
 return err
}
var org Organization
err = yaml.Unmarshal(yamlFile, &org)
if err != nil {
 return err
}

for _, team := range org.Teams {
 ghTeam, err := github.NewTeam(ctx, team.Slug, &github.TeamArgs{
 Description: pulumi.String(team.Description),
 Name: pulumi.String(team.Name),
 Privacy: pulumi.String("closed"),
}, pulumi.Protect(false))
if err != nil {
fmt.Println("encountered error creating new Pulumi GitHub team: ", team.Name)
 return err
}
 return nil
 })
}

Notice how the GitHub provider allows us to use team names to create teams. In contrast, team IDs are only needed for importing to the Stack, not for maintaining state.

Relationships Are Hard

The next step involved a lot of thinking about team memberships and team nesting. GitHub allows teams to be nested. This groups teams by area of responsibility but also allows for certain properties, such as permissions, to be inherited by subteams.

Parent Team Parent
 |
 -------------------------
 | |
Child Teams Child1 Child2
(inherit all permissions)

To establish this relationship, we set a ParentTeamId on child teams. First off, we add a new field to our Teams struct:

type Team struct {
 Name string `yaml:"name"`
 Description string `yaml:"description"`
 Slug string `yaml:"slug"`
 ParentTeamId int
 Teams []Team `yaml:"teams"`
}

Here is where things get a little tricky. Any GitHub Team can have subteams, but not every Team has a parent Team. We also did not want to have to hardcode GitHub Team IDs into our configuration files. We needed to meet the following requirements:

Create a parent team, using a team name.
Obtain its team ID.
Create any child teams, also using a team name.
Write the parent team’s ID into the ParentTeamId field of each child team.
Do all of the above in a single pulumi up.

This is where maintaining infrastructure with Pulumi truly shines. In our code, we can use Pulumi Apply to hold on to the promise of a parent team ID, and pass this promise into the appropriate field:

func setupTeams(ctx *pulumi.Context, parentTeam *Team) error {

// set up parent team, i.e. engineering
 ghParentTeam, err := github.NewTeam(ctx, parentTeam.Slug, &github.TeamArgs{
 Description: pulumi.String(parentTeam.Description),
 Name: pulumi.String(parentTeam.Name),
 Privacy: pulumi.String("closed"),
 }, pulumi.Protect(false))
 if err != nil {
 fmt.Println("encountered error creating new Pulumi github team: ", parentTeam.Name)
 return err
 }

//set up nested teams
 for _, childTeam := range parentTeam.Teams {
 // set each child team's parent team ID to the current team ID
 ghChildTeam, err := github.NewTeam(ctx, childTeam.Slug, &github.TeamArgs{
 Description: pulumi.String(childTeam.Description),
 Name: pulumi.String(childTeam.Name),
 Privacy: pulumi.String("closed"),
 ParentTeamId: ghParentTeam.ID(),
 }, pulumi.Protect(false))
 if err != nil {
 fmt.Println("encountered error creating new Pulumi github team: ", childTeam.Name)
 return err
 }
 }
 return nil
}

Running this as part of main.go will result in beautifully nested teams on the GitHub UI. But with Pulumi, we can do even better. We can set pulumi.Parent() on the child teams:

for _, childTeam := range parentTeam.Teams {
 ghChildTeam, err := github.NewTeam(ctx, childTeam.Slug, &github.TeamArgs{#...},
 pulumi.Protect(false),
 pulumi.Parent(ghParentTeam))
 #...
}

With this, we can see the relationship between the resources on pulumi up:

Updating (prod)

 Type Name Status
 pulumi:pulumi:Stack team-mgmt-prod
 + └─ github:index:Team newteam created
 + └─ github:index:Team newchildteam created
Resources:
 + 2 created
2 unchanged

Which will be reflected on the Pulumi Stack’s Graph View in the Pulumi Service:

Adding this relationship to Pulumi is mostly a visual nicety in our case; however, certain Pulumi properties get inherited in a helpful way. Note that on GitHub, it is possible to create multiple layers of teams, whereas the example code in this blog post only shows a single layer of subteam structure. We are small enough at Pulumi to not need more, but you may want to expand on the solution if you do.

Adding People

Next, teams should have members! Let’s add them to the YAML config:

org: "pulumi-demo-org"
teams:
 - name: "animals"
 slug: "animals"
 description: "A team in charge of animals"
 members:
 - username: "platypus"
 - username: "owlcat"

A TeamMembership in GitHub is a cross reference between a Team and a User. Fortunately these do not need to be explicitly imported, as they are merely establishing relationships between GitHub Users and Teams. We can add TeamMemberships to the Stack with a Members struct, a new Members[] field on the Team struct, and an extra function. Again, Pulumi lets us use the promised output of the team ID to set the TeamId field in the TeamMembership:

type Team struct {
 Name string `yaml:"name"`
 Description string `yaml:"description"`
 ParentTeamId int
 Teams []Team `yaml:"teams"`
 Slug string `yaml:"slug"`
 Members []Member `yaml:"members"`
}

type Member struct {
 UserName string `yaml:"username"`
 Role string `yaml:"role"`
}

#...

func addMembers(ctx *pulumi.Context, members []Member, team *github.Team, teamName string) error {
 for _, member := range members {
 // unique name for TeamMembership
 utmName := teamName + "-" + member.UserName
 _, err := github.NewTeamMembership(ctx, utmName, &github.TeamMembershipArgs{
 TeamId: team.ID(),
 Username: pulumi.String(member.UserName),
 })
 if err != nil {
 return err
 }
 }
 return nil
}

Running pulumi up:

Updating (prod)

 Type Name Status
 pulumi:pulumi:Stack team-mgmt-prod
 + ├─ github:index:TeamMembership animals-owlcat created
 + └─ github:index:TeamMembership animals-platypus created

Resources:
 + 2 created
 4 unchanged

Alright! So now we have

Imported existing teams to a Pulumi Stack to track and organize all GitHub teams from a central repository, and
Added team members to teams.

That’s pretty great so far! While there are many org chart tools, what makes this one useful to us is that we can declare the desired org state in a config file, and then let Pulumi figure out what changes should be applied. Let’s automate that!

Add CI

In order for this tool to be used by everyone, we keep code and configuration in a GitHub repository. We can use Pulumi’s GitHub Action to run a pulumi preview on a pull request, and a pulumi up on merge to main.

Here’s what that looks like on the pull request:

name: Pull Request preview
on:
 pull_request:
 workflow_dispatch:

env:
 AWS_REGION: us-west-2
 GITHUB_TOKEN: ${{ secrets.PULUMI_GITHUB_TOKEN }}
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
 GITHUB_OWNER: pulumi

jobs:
 pull-request:
 runs-on: ubuntu-latest
 name: Preview
 steps:
 - uses: actions/checkout@v2
 - name: Install Go
 uses: actions/setup-go@v2
 with:
 go-version: 1.17
 - name: Build Binary
 run: |
 go mod tidy
 go build main.go
 - name: Pulumi Preview
 uses: pulumi/actions@v3
 with:
 command: preview
 refresh: true
 stack-name: pulumi/prod
 comment-on-pr: true

And the pulumi up on merge to main:

name: Update
on:
 push:
 branches:
 - "main"
 workflow_dispatch:

env:
 AWS_REGION: us-west-2
 GITHUB_TOKEN: ${{ secrets.PULUMI_GITHUB_TOKEN }}
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
 GITHUB_OWNER: pulumi

jobs:
 main:
 runs-on: ubuntu-latest
 name: Update
 steps:
 - uses: actions/checkout@v2
 - name: Install Go
 uses: actions/setup-go@v2
 with:
 go-version: 1.17
 - name: Build Binary
 run: |
 go mod tidy
 go build main.go
 - name: Pulumi update
 uses: pulumi/actions@v3
 with:
 command: up
 refresh: true
 stack-name: pulumi/prod

Note that we are calling refresh: true in both Workflows, which uses Pulumi Refresh to make sure that the existing GitHub resources are aligned with the resource state in our Stack.

Now, anyone with access to the GitHub management repo can:

Create, re-parent, delete, or rename teams and re-create any memberships via pull request to the config file.
Add and remove team members via pull request.
Audit and explicitly maintain org structure via git history and review processes.

But of course… there’s more!

Managing Permissions

The next step involved a lot of thinking about repository permissions and permission access.

Shortly after I joined Pulumi, my team was combined with another team. They brought along all of their repositories—none of which I had access to. Moreover, all of our team names were outdated. I could rename the teams using pulumi-github… but I could not transfer all of the repository access we all needed to our new team. Yet.

On GitHub, parent teams pass permissions down to child teams. But what if we wanted permissions to be more granular?

Here’s where TeamRepositories come in. A TeamRepository has three fields: a team ID, a repository name, and a permission. Just like TeamMemberships, this resource is a link between a Team and a Repository and does not need to be imported.

We gave each team their own config file with permission levels listing repository names:

team: "animals"
permissions:
 - role: "admin"
 repos:
 - "capybaras"
 - "owls"
 - role: "maintain"
 repos:
 - role: "push"
 repos:
 - role: "read"
 repos:

This can be read into our Pulumi program as follows:

// TeamPermissions describes a github team and the levels of permissions available (i.e. admin, maintain)
type TeamPermissions struct {
 TeamName string `yaml:"team-name"`
 Permissions []Permission `yaml:"permissions"`
}

// Permission is a single permission and a list of repos with that permission.
type Permission struct {
 Role string `yaml:"role"`
 Repos []string `yaml:"repos"`
}

#...

func setupTeamRepos(ctx *pulumi.Context, team *github.Team, teamName string) error {

 repoFilePath, err := filepath.Abs("./team-repos/" + teamName + ".yaml")
 if err != nil {
 fmt.Println(err.Error())
 return err
 }
 yamlFile, err := ioutil.ReadFile(repoFilePath)
 if err != nil {
 return err
 }

 var teamPerms TeamPermissions
 err = yaml.Unmarshal(yamlFile, &teamPerms)
 if err != nil {
 return err
 }
 for _, permission := range teamPerms.Permissions {
 for _, repo := range permission.Repos {
 _, err := github.NewTeamRepository(ctx, fmt.Sprintf("%s-%s-%s", repo, teamName, permission.Role), &github.TeamRepositoryArgs{
 Permission: pulumi.String(permission.Role),
 Repository: pulumi.String(repo),
 TeamId: team.ID(),
 })

 if err != nil {
 return err
 }
 }
 }
 return nil
}

And the output of running pulumi up:

Updating (prod)

 Type Name Status
 pulumi:pulumi:Stack team-mgmt-prod
 + ├─ github:index:TeamRepository owls-animals-admin created
 + ├─ github:index:TeamRepository capybaras-animals-admin created


Resources:
 + 2 created
 2 changes. 4 unchanged

With this addition, we can now combine teams with zero access disruptions as follows:

Create the new team in the org config.
Create a TeamRepository config for the new team with desired permission levels.
Open a pull request and let CI and Pulumi do the rest.

Now we can develop standards around repository permissions based on teams’ roles and areas of responsibility, and we have tooling in place that can maintain these standards for everyone in the org to see. In fact, we can standardize the meaning of “code ownership” via access levels in this way—one centralized management location rather than asking your grandboss to dig through the UI for you.

Future challenges

We want to automate some of these steps: adding new teammates to the org should also automatically add them to this repository so they can self-serve their onboarding and see Pulumi work in action!

To prevent state drift, we will run a regular reconciliation job in CI.

Of course, we will add other Pulumi providers on top of this org structure so we can sync our other tooling.

Pulumi Blog: Guinevere Saenger

Introducing envVarMappings for Provider Credentials

Multiple providers or provider instances that expect different authentication values but have the same variable key

Remapping environment variables

Example

Pulumi Google Cloud Provider Version 9.0.0

In case you missed it - new modules

What’s New in 9.0

Upstream Improvements

Improved Import Validation

Increased Field Validation

Documentation Improvements

Upgrading

NEW! Use Pulumi Neo to assist with your upgrade

Pulumi Google Cloud Provider Version 8.0.0

Looking Back

New Modules

API Coverage Growth

What’s New in 8.0

Added Deletion Protections

New default provisioning label

Upgrading

Fast Docker Image Builds with Pulumi

TL;DR

What actually moves the needle

How does Pulumi build Docker images?

Why are my Docker builds slow?

How do I enable BuildKit and registry caching in Pulumi?

With the Docker Build provider (recommended)

With the Docker provider (docker.Image)

How do I write a Dockerfile that builds fast?

How do I build multi-platform images with Pulumi?

How do I see Docker build output in Pulumi?

How do I migrate an older docker.Image program?

Getting started

Multicloud with Kubernetes and Pulumi

Part One: Create a component

Prerequisites

Start by writing the code for each cloud

Create a custom component for the Kubernetes cluster

Side Quest: Ship an app!

Part 2: Leverage the Automation API

The Stack Module

Wrap the Automation API

Putting it all together

Run the program

Summary

How we manage GitHub at Pulumi with Pulumi

The Goal

Getting Started

Capturing Current State of Resources

Creating Configuration

Relationships Are Hard

Adding People

Add CI

Managing Permissions

Future challenges

With the Docker provider (`docker.Image`)

How do I migrate an older `docker.Image` program?