Pulumi Blog: Bryce Lampe

Improved Kubernetes Await Logic

Bryce Lampe — Wed, 04 Sep 2024 09:00:00 -0700

Kubernetes’ eventual consistency model is incredibly powerful but can also pose a challenge for workflows that provision complex applications and infrastructure. Sometimes you really just need to know that a step has succeeded before being able to proceed.

One of the advantages of using Pulumi to manage Kubernetes resources is that it natively and intuitively handles this problem of readiness and dependencies, giving you an easy way to express complex rollout relationships in the programming language of your choice.

The latest v4.18.0 release of the Pulumi Kubernetes provider includes a number of bug fixes and enhancements to the way Pulumi Kubernetes handles resource readiness:

More helpful errors are surfaced when a resource doesn’t become ready in time.
A new pulumi.com/deletionPropagationPolicy annotation enables faster deletions.
A new pulumi.com/waitFor annotation allows for custom readiness criteria.
And a new experimental feature enables readiness criteria for all resources.

Better error reporting

Our goal is that Pulumi Kubernetes users should never need to manually query the cluster to figure out why an update failed, so we try to surface as much information as possible when something fails.

To that end we’ve made three improvements to the information Pulumi surfaces when something goes wrong while it’s waiting for a resource to become ready (or be deleted):

Any Kubernetes Events involving the resource with type: Warning are now surfaced by Pulumi as warnings. This can help you see if a Service had trouble provisioning a LoadBalancer, for example.
Pulumi will now warn you if it looks like a resource’s deletion might be blocked by stuck finalizers. This is a common gotcha that is not always obvious.
Error messages from resources that actuate Pods (i.e. Deployments, StatefulSets, etc.) now include status codes and termination messages if their Pods failed to start. A Pod’s termination message can be helpful in CrashLoopBackOff situations and is read from /dev/termination-log by default.

(You can configure this path with terminationMessagePath, or use terminationMessagePolicy: FallbackToLogsOnError to use a Pod’s logs as its termination message.)

Faster deletions with “pulumi.com/deletionPropagationPolicy”

Pulumi uses foreground cascading deletion by default, which deletes a resource and all of its dependents. This is the most robust form of deletion but it is also the slowest, and until now there was no way to safely customize this behavior.

The new pulumi.com/deletionPropagationPolicy annotation allows you to customize delete behavior while still allowing Pulumi to await readiness when creating or updating the resource.

The annotation accepts three propagation policies, corresponding to the options supported by Kubernetes (see “Cascading Deletion”).

“background”: delete the owner object and leave dependent objects to be asynchronously garbage collected. This is faster than “foreground” deletion propagation, but dependent objects can remain temporarily or even indefinitely if they are not finalized.
“orphan”: delete the owner object and leave dependent objects untouched. This can be useful if you want to keep objects around for migration or debugging purposes.
“foreground”: the provider’s default behavior of deleting an object and all of its dependents. This is slower but guarantees all dependents have been cleaned up if it succeeds.

Using skipAwait to speed up deletion is not recommended when server-side apply is enabled because it is not respected by all resources and it can lead to race conditions during replacement. The current behavior is considered buggy, and future releases might change the behavior of the skipAwait annotation to have no effect during deletion.

The pulumi.com/deletionPropagationPolicy annotation is the recommended way to delete something quickly and safely if it has children that are slow to cleanup.

Experimental: Custom readiness with “pulumi.com/waitFor”

The pulumi.com/waitFor annotation is considered experimental and its syntax may change in the future.

There is no one-size-fits-all definition for what it means for a Kubernetes resource to be “ready,” so we have also introduced a new annotation, pulumi.com/waitFor, which allows you to specify custom readiness criteria on a per-resource basis. This is functionally similar to kubectl wait.

The new pulumi.com/waitFor annotation can be applied to any resource with metadata, and it accepts three possible formats:

A kubectl JSONPath expression – a string prefixed with “jsonpath=” followed by a path expression and an optional value.

For example this resource expects its “phase” status field to have a value of “Running”:
```
 "pulumi.com/waitFor": "jsonpath={.status.phase}=Running"
```
If a value isn’t provided, the resource is considered ready when any value exists at the given path. This resource will wait until it has a web hook configured with a CA bundle:
```
 "pulumi.com/waitFor": "jsonpath={.webhooks[*].clientConfig.caBundle}"
```
A string prefixed with “condition=” followed by the type of the condition and an optional status. This matches the behavior of kubectl wait --for=condition=... and will wait until the resource has a matching status condition. The expected status defaults to “True” if not specified. For example this will wait until the resource has a “Synced=True” condition:
```
"pulumi.com/waitFor": "condition=Synced"
```
and this will wait until it has a “Reconciling=False” condition:
```
"pulumi.com/waitFor": "condition=Reconciling=False"
```
Lastly, the pulumi.com/waitFor annotation also accepts a JSON-encoded string containing an array of one or more “jsonpath=” or “condition=” expressions. For example, this resource will be considered ready when it simultaneously has a “Bar=True” condition and a populated “foo” field.
```
"pulumi.com/waitFor": '["jsonpath={.foo}", "condition=Bar"]'
```

Custom readiness example: cert-manager

cert-manager is a popular X.509 certificate management solution with some installation nuances. If you follow the default instructions with a goal of also creating a new Certificate as part of your Pulumi stack, you might end up with something like this:

import * as kubernetes from "@pulumi/kubernetes";

const install = new kubernetes.yaml.v2.ConfigFile("install",
 {
 file: "https://github.com/cert-manager/cert-manager/releases/download/v1.15.2/cert-manager.yaml",
 },
);

const issuer = new kubernetes.apiextensions.CustomResource("issuer",
 {
 apiVersion: "cert-manager.io/v1",
 kind: "ClusterIssuer",
 metadata: {
 name: "selfsigned-issuer",
 },
 spec: {
 selfSigned: {},
 },
 },
 { dependsOn: [install] }
);

new kubernetes.apiextensions.CustomResource("certificate", {
 apiVersion: "cert-manager.io/v1",
 kind: "Certificate",
 metadata: {
 name: "selfsigned-cert",
 },
 spec: {
 isCA: true,
 commonName: "my-ca",
 secretName: "root-secret",
 privateKey: {
 algorithm: "ECDSA",
 size: 256,
 },
 issuerRef: {
 name: issuer.metadata.name,
 kind: issuer.kind,
 group: "cert-manager.io",
 },
 },
});

This uses a v2 ConfigFile to kubectl apply the cert-manager custom resource definitions (CRDs), and then it creates a self-signed Issuer and Certificate using those CRDs.

The dependencies between resources appear reasonable, however this program will likely fail when it is first run:

error: resource “kubernetes:cert-manager.io/v1:ClusterIssuer::issuer” was not successfully created by the Kubernetes API server: Internal error occurred: failed calling webhook “webhook.cert-manager.io”: failed to call webhook: Post “https://cert-manager-webhook.cert-manager.svc:443/validate?timeout=30s”: tls: failed to verify certificate: x509: certificate signed by unknown authority

We see this “certificate signed by unknown authority” error because a few things need to happen before the cert-manager webhooks are operational:

At least one cainjector pod needs to be running.
The cainjector pod(s) need to have a leader elected.
The cainjector leader needs to inject certificate authority data into the cert-manager webhooks.

Previously, Pulumi had no way to know whether those pre-requisite steps had taken place. However, now we can use the pulumi.com/waitFor annotation to instruct Pulumi to wait until web hooks have CA data injected before proceeding to create Certificates. One way to achieve this is by using the new transforms API to annotate the webhooks appropriately:

const install = new kubernetes.yaml.v2.ConfigFile("install",
 {
 file: "https://github.com/cert-manager/cert-manager/releases/download/v1.15.2/cert-manager.yaml",
 },
 {
 // Add a "waitFor" annotation which waits for the cainjector pod to come
 // online and modify our webhooks with valid CA bundles. Webhooks
 // needed by our Certificate will fail until this happens.
 transforms: [
 (args) => {
 if (
 args.type ===
 "kubernetes:admissionregistration.k8s.io/v1:ValidatingWebhookConfiguration" ||
 args.type ===
 "kubernetes:admissionregistration.k8s.io/v1:MutatingWebhookConfiguration"
 ) {
 args.props.metadata.annotations["pulumi.com/waitFor"] =
 "jsonpath={.webhooks[*].clientConfig.caBundle}";
 return {
 props: args.props,
 opts: args.opts,
 };
 }
 return undefined;
 },
 ],
 },
);

Here we’ve given the pulumi.com/waitFor annotation a value of jsonpath={.webhooks[*].clientConfig.caBundle}, which is a JSONPath expression that will wait until a webhook in the webhooks array has a clientConfig with a caBundle populated – indicating that the webhook is fully operational.

Now the program will deploy successfully with a Missing {.webhooks[*].clientConfig.caBundle} message while Pulumi waits for the field to be configured.

Experimental: Generic readiness

Being able to specify custom readiness is powerful, but a lot of resources follow well-known conventions like the Ready=True condition. Rather than requiring you to specify "pulumi.com/waitFor": "condition=Ready" on every resource that needs it, there’s now an opt-in feature to automatically apply well-known readiness criteria to all resources managed by Pulumi.

Running your program with the environment variable PULUMI_K8S_AWAIT_ALL=true will cause Pulumi to automatically wait for all resources to become ready when creating or updating them – including custom resources.

If a resource appears to follow well-known conventions (like the “Ready” condition, among others) Pulumi will wait for it to become ready. If the resource doesn’t appear to follow well-known conventions, Pulumi will maintain its existing behavior and assume the resource is ready immediately.

(Generic readiness criteria are determined by cli-utils (a collection of Go libraries maintained by the SIG CLI), and behavior is unchanged for any resources that Pulumi already waits for readiness.)

Please don’t hesitate to reach out on GitHub or Slack if you’re excited about this feature; if you experience any issues with it; or if you have any feedback to share!

Manage Kubernetes readiness with Pulumi

Express complex rollout and readiness relationships in the language you already use, and let Pulumi wait for your cluster resources to become ready before it continues.

Get started

Additional improvements

Several other bug fixes related to the Pulumi Kubernetes provider’s await logic are also worth mentioning.

PersistentVolumeClaims with a bind mode of WaitForFirstConsumer are now assumed to be ready as soon as they are created. Previously, these PVCs could cause deadlocks by preventing their consumer from being provisioned.

Services are now considered ready as long as they have no non-ready Endpoints. Previously, Services would block forever if they matched Deployments that had been scaled down to zero replicas, or if they had manually provisioned endpoints.

Pulumi will now wait for DaemonSets to become ready instead of always using skipAwait semantics.

Conclusion

We hope you will enjoy the increased flexibility of the pulumi.com/deletionPropagationPolicy annotation; the extensibility of the pulumi.com/waitFor annotation; and the improved visibility into errors and events Pulumi Kubernetes now provides while managing your cluster resources.

Check out the following links to learn more about Pulumi Kubernetes today!

Introducing The New Docker Build Provider

Bryce Lampe — Thu, 25 Apr 2024 09:00:00 -0700

Deploying and managing containerized workloads is one of Pulumi’s fastest-growing areas. Standing up managed container services and Kubernetes clusters is a common area for automation, and many of our customers use Pulumi to automate building and publishing images to their registry of choice.

Given this critical use case, we are thrilled to introduce the latest addition to the Pulumi ecosystem: the new Docker Build provider, designed to streamline and modernize Docker image builds from your Pulumi programs.

This new addition expands upon the solid foundation of our highly utilized Docker provider. It elevates the docker.Image resource, one of the most heavily used and most powerful resources in the Pulumi ecosystem, into its very own dedicated package. By fully embracing the power of BuildKit, the Docker Build provider brings many new features to support the best-in-class Docker capabilities developers love.

The new provider exposes Docker’s next-gen buildx interface, so now even your most complex Docker images can be handled by Pulumi. This is a remarkably flexible and versatile tool for building images and includes key features such as,

Multi-platform image support: Build images that run seamlessly across different hardware architectures.
Advanced caching mechanisms: Speed up builds and reduce resource consumption by utilizing cache backends like S3, GitHub Actions, local disk, and more.
Support for build secrets: Unlike building images with the docker CLI, which requires juggling environment variables or files on disk, Pulumi’s first-class support for secrets means you can now safely and efficiently incorporate sensitive information into your builds. With Pulumi ESC (Environments, Secrets, and Configuration), it’s easy to share build secrets with other developers and teams.
Support for multiple export types: Export your images to registries, disk, or blob storage to increase the power and flexibility of your workflows.
Support for Docker Build Cloud: Use Docker Build Cloud to offload your builds and caches to the cloud, enhancing productivity and performance.

A brief history

Pulumi first introduced the Docker provider in 2018, and it has been widely adopted since. Over the years, Pulumi has invested in the Docker provider, notably introducing version 4.0 last year and additional improvements like build-on-preview behavior.

At the same time, the Docker build ecosystem has seen tremendous advancements in how images can be built and distributed. These changes were initially experimental, but they became official when BuildKit graduated to become the default builder in Docker version 23.

We have heard many requests from users to expose more BuildKit functionality in the Docker provider. After some consideration, we decided that a new, standalone provider focused exclusively on building images would provide the best overall user experience. In addition, it allows Pulumi to stay current with the evolving Docker build landscape.

Getting started

Add the docker-build package to your Pulumi program to take full advantage of modern image builds.

import * as docker_build from "@pulumi/docker-build";

import pulumi_docker_build as docker_build

using DockerBuild = Pulumi.DockerBuild;

import "github.com/pulumi/pulumi-docker-build/sdk/go/dockerbuild"

resources:
 my-image:
 type: docker-build:Image

import com.pulumi.dockerbuild.Image

Suppose your Pulumi program has a subfolder called ./app that contains a file named Dockerfile (its contents can be as simple as FROM alpine). In that case, the example below shows:

how to build a multi-platform image (linux/amd64 and linux/arm64),
publish the image to a remote AWS ECR registry, and;
use an inline cache to speed up subsequent builds.

import * as pulumi from "@pulumi/pulumi"
import * as aws from "@pulumi/aws";
import * as docker_build from "@pulumi/docker-build";

// Create an ECR repository for pushing.
const ecrRepository = new aws.ecr.Repository("ecr-repository", {});

// Grab auth credentials for ECR.
const authToken = aws.ecr.getAuthorizationTokenOutput({
 registryId: ecrRepository.registryId,
});

// Build and push an image to ECR with inline caching.
const myImage = new docker_build.Image("my-image", {
 // Tag our image with our ECR repository's address.
 tags: [pulumi.interpolate`${ecrRepository.repositoryUrl}:latest`],
 context: {
 location: "./app",
 },
 // Use the pushed image as a cache source.
 cacheFrom: [{
 registry: {
 ref: pulumi.interpolate`${ecrRepository.repositoryUrl}:latest`,
 },
 }],
 // Include an inline cache with our pushed image.
 cacheTo: [{
 inline: {},
 }],
 // Build a multi-platform image manifest for ARM and AMD.
 platforms: [
 "linux/amd64",
 "linux/arm64",
 ],
 // Push the final result to ECR.
 push: true,
 // Provide our ECR credentials.
 registries: [{
 address: ecrRepository.repositoryUrl,
 password: authToken.password,
 username: authToken.userName,
 }],
});

// Export a ref for the pushed images so we can deploy it.
export const ref = myImage.ref;

import pulumi
import pulumi_aws as aws
import pulumi_docker_build as docker_build

# Create an ECR repository for pushing.
ecr_repository = aws.ecr.Repository("ecr-repository")

# Grab auth credentials for ECR.
auth_token = aws.ecr.get_authorization_token_output(registry_id=ecr_repository.registry_id)

# Build and push an image to ECR with inline caching.
my_image = docker_build.Image("my-image",
 # Tag our image with our ECR repository's address.
 tags=[ecr_repository.repository_url.apply(lambda repository_url: f"{repository_url}:latest")],
 context=docker_build.BuildContextArgs(
 location="./app",
 ),
 # Use the pushed image as a cache source.
 cache_from=[docker_build.CacheFromArgs(
 registry=docker_build.CacheFromRegistryArgs(
 ref=ecr_repository.repository_url.apply(lambda repository_url: f"{repository_url}:latest"),
 ),
 )],
 # Include an inline cache with our pushed image.
 cache_to=[docker_build.CacheToArgs(
 inline=docker_build.CacheToInlineArgs(),
 )],
 # Build a multi-platform image manifest for ARM and AMD.
 platforms=[
 docker_build.Platform.LINUX_AMD64,
 docker_build.Platform.LINUX_ARM64,
 ],
 # Push the final result to ECR.
 push=True,
 # Provide our ECR credentials.
 registries=[docker_build.RegistryArgs(
 address=ecr_repository.repository_url,
 password=auth_token.password,
 username=auth_token.user_name,
 )],
)

# Export a ref for the pushed images so we can deploy it.
pulumi.export("ref", my_image.ref)

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Aws = Pulumi.Aws;
using DockerBuild = Pulumi.DockerBuild;

return await Deployment.RunAsync(() =>
{
 // Create an ECR repository for pushing.
 var ecrRepository = new Aws.Ecr.Repository("ecr-repository");

 // Grab auth credentials for ECR.
 var authToken = Aws.Ecr.GetAuthorizationToken.Invoke(new()
 {
 RegistryId = ecrRepository.RegistryId,
 });

 // Build and push an image to ECR with inline caching.
 var myImage = new DockerBuild.Image("my-image", new()
 {
 // Tag our image with our ECR repository's address.
 Tags = new[]
 {
 ecrRepository.RepositoryUrl.Apply(repositoryUrl => $"{repositoryUrl}:latest"),
 },
 Context = new DockerBuild.Inputs.BuildContextArgs
 {
 Location = "./app",
 },
 // Use the pushed image as a cache source.
 CacheFrom = new[]
 {
 new DockerBuild.Inputs.CacheFromArgs
 {
 Registry = new DockerBuild.Inputs.CacheFromRegistryArgs
 {
 Ref = ecrRepository.RepositoryUrl.Apply(repositoryUrl => $"{repositoryUrl}:latest"),
 },
 },
 },
 // Include an inline cache with our pushed image.
 CacheTo = new[]
 {
 new DockerBuild.Inputs.CacheToArgs
 {
 Inline = null,
 },
 },
 // Build a multi-platform image manifest for ARM and AMD.
 Platforms = new[]
 {
 DockerBuild.Platform.Linux_amd64,
 DockerBuild.Platform.Linux_arm64,
 },
 // Push the final result to ECR.
 Push = true,
 // Provide our ECR credentials.
 Registries = new[]
 {
 new DockerBuild.Inputs.RegistryArgs
 {
 Address = ecrRepository.RepositoryUrl,
 Password = authToken.Apply(getAuthorizationTokenResult => getAuthorizationTokenResult.Password),
 Username = authToken.Apply(getAuthorizationTokenResult => getAuthorizationTokenResult.UserName),
 },
 },
 });

 // Export a ref for the pushed images so we can deploy it.
 return new Dictionary<string, object?>
 {
 ["ref"] = myImage.Ref,
 };
});

package main

import (
 "fmt"

 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ecr"
 "github.com/pulumi/pulumi-docker-build/sdk/go/dockerbuild"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // Create an ECR repository for pushing.
 ecrRepository, err := ecr.NewRepository(ctx, "ecr-repository", nil)
 if err != nil {
 return err
 }

 // Grab auth credentials for ECR.
 authToken := ecr.GetAuthorizationTokenOutput(ctx, ecr.GetAuthorizationTokenOutputArgs{
 RegistryId: ecrRepository.RegistryId,
 }, nil)

 // Build and push an image to ECR with inline caching.
 myImage, err := dockerbuild.NewImage(ctx, "my-image", &dockerbuild.ImageArgs{
 // Tag our image with our ECR repository's address.
 Tags: pulumi.StringArray{
 ecrRepository.RepositoryUrl.ApplyT(func(repositoryUrl string) (string, error) {
 return fmt.Sprintf("%v:latest", repositoryUrl), nil
 }).(pulumi.StringOutput),
 },
 Context: &dockerbuild.BuildContextArgs{
 Location: pulumi.String("./app"),
 },
 // Use the pushed image as a cache source.
 CacheFrom: dockerbuild.CacheFromArray{
 &dockerbuild.CacheFromArgs{
 Registry: &dockerbuild.CacheFromRegistryArgs{
 Ref: ecrRepository.RepositoryUrl.ApplyT(func(repositoryUrl string) (string, error) {
 return fmt.Sprintf("%v:latest", repositoryUrl), nil
 }).(pulumi.StringOutput),
 },
 },
 },
 // Include an inline cache with our pushed image.
 CacheTo: dockerbuild.CacheToArray{
 &dockerbuild.CacheToArgs{
 Inline: nil,
 },
 },
 // Build a multi-platform image manifest for ARM and AMD.
 Platforms: dockerbuild.PlatformArray{
 dockerbuild.Platform_Linux_amd64,
 dockerbuild.Platform_Linux_arm64,
 },
 // Push the final result to ECR.
 Push: pulumi.Bool(true),
 // Provide our ECR credentials.
 Registries: dockerbuild.RegistryArray{
 &dockerbuild.RegistryArgs{
 Address: ecrRepository.RepositoryUrl,
 Password: authToken.ApplyT(func(authToken ecr.GetAuthorizationTokenResult) (*string, error) {
 return &authToken.Password, nil
 }).(pulumi.StringPtrOutput),
 Username: authToken.ApplyT(func(authToken ecr.GetAuthorizationTokenResult) (*string, error) {
 return &authToken.UserName, nil
 }).(pulumi.StringPtrOutput),
 },
 },
 })
 if err != nil {
 return err
 }

 // Export a ref for the pushed images so we can deploy it.
 ctx.Export("ref", myImage.Ref)
 return nil
 })
}

description: Push to AWS ECR with caching
name: ecr
outputs:
 ref: ${my-image.ref}
resources:
 # Create an ECR repository for pushing.
 ecr-repository:
 type: aws:ecr:Repository

 # Build and push an image to ECR with inline caching.
 my-image:
 type: docker-build:Image
 properties:
 # Tag our image with our ECR repository's address.
 tags:
 - ${ecr-repository.repositoryUrl}:latest
 context:
 location: ./app
 # Use the pushed image as a cache source.
 cacheFrom:
 - registry:
 ref: ${ecr-repository.repositoryUrl}:latest
 # Include an inline cache with our pushed image.
 cacheTo:
 - inline: {}
 # Build a multi-platform image manifest for ARM and AMD.
 platforms:
 - linux/amd64
 - linux/arm64
 # Push the final result to ECR.
 push: true
 # Provide our ECR credentials.
 registries:
 - address: ${ecr-repository.repositoryUrl}
 password: ${auth-token.password}
 username: ${auth-token.userName}

runtime: yaml
variables:
 auth-token:
 # Grab auth credentials for ECR.
 fn::aws:ecr:getAuthorizationToken:
 registryId: ${ecr-repository.registryId}

package myapp;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.aws.ecr.Repository;
import com.pulumi.aws.ecr.EcrFunctions;
import com.pulumi.aws.ecr.inputs.GetAuthorizationTokenArgs;
import com.pulumi.dockerbuild.Image;
import com.pulumi.dockerbuild.ImageArgs;
import com.pulumi.dockerbuild.inputs.CacheFromArgs;
import com.pulumi.dockerbuild.inputs.CacheFromRegistryArgs;
import com.pulumi.dockerbuild.inputs.CacheToArgs;
import com.pulumi.dockerbuild.inputs.CacheToInlineArgs;
import com.pulumi.dockerbuild.inputs.BuildContextArgs;
import com.pulumi.dockerbuild.inputs.RegistryArgs;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 // Create an ECR repository for pushing.
 var ecrRepository = new Repository("ecrRepository");

 // Grab auth credentials for ECR.
 final var authToken = EcrFunctions.getAuthorizationToken(GetAuthorizationTokenArgs.builder()
 .registryId(ecrRepository.registryId())
 .build());

 // Build and push an image to ECR with inline caching.
 var myImage = new Image("myImage", ImageArgs.builder()
 // Tag our image with our ECR repository's address.
 .tags(ecrRepository.repositoryUrl().applyValue(repositoryUrl -> String.format("%s:latest", repositoryUrl)))
 .context(BuildContextArgs.builder()
 .location("./app")
 .build())
 // Use the pushed image as a cache source.
 .cacheFrom(CacheFromArgs.builder()
 .registry(CacheFromRegistryArgs.builder()
 .ref(ecrRepository.repositoryUrl().applyValue(repositoryUrl -> String.format("%s:latest", repositoryUrl)))
 .build())
 .build())
 // Include an inline cache with our pushed image.
 .cacheTo(CacheToArgs.builder()
 .inline()
 .build())
 // Build a multi-platform image manifest for ARM and AMD.
 .platforms(
 "linux/amd64",
 "linux/arm64")
 // Push the final result to ECR.
 .push(true)
 // Provide our ECR credentials.
 .registries(RegistryArgs.builder()
 .address(ecrRepository.repositoryUrl())
 .password(authToken.applyValue(getAuthorizationTokenResult -> getAuthorizationTokenResult).applyValue(authToken -> authToken.applyValue(getAuthorizationTokenResult -> getAuthorizationTokenResult.password())))
 .username(authToken.applyValue(getAuthorizationTokenResult -> getAuthorizationTokenResult).applyValue(authToken -> authToken.applyValue(getAuthorizationTokenResult -> getAuthorizationTokenResult.userName())))
 .build())
 .build());

 ctx.export("ref", myImage.ref());
 }
}

The Pulumi output, ref, provides a convenient way to reference the pushed image in downstream Pulumi resources like ECS TaskDefinitions or Kubernetes Deployments.

Build container images with Pulumi

Use the Docker Build provider to build multi-platform images, cache layers across builds, and push to your registry of choice as part of your Pulumi program.

Get started

Migrating from docker.Image to docker_build.Image

The new docker_build.Image replaces the old docker.Image resource. It adds new features and addresses many bugs with the previous resource by fully aligning with Docker’s underlying buildx commands. From now on, we will continue to invest in Docker Build and the docker_build.Image resource.

We do not expect to make further changes to docker.Image. It will remain available for the foreseeable future, so you can continue using it if it meets your needs. However, we recommend you migrate your images to docker_build.Image to get the best possible support, features, and performance. We expect to deprecate the docker.Image resource in the future.

The migration process is straightforward and detailed in the API documentation. Docker Build provides a superset functionality over the previous docker.Image resource; thus, you can migrate existing resources without issue. The new Docker Build Image options will look very familiar if you use the Docker command-line tool.

The new Docker Build provider matches the Docker CLI behavior. Thus, it does not push images by default. If you want to push to a registry, include push: true in the code just as you would include --push on the command line.

The new Docker Build provider builds images by default during previews to reduce the risk of merging broken images as part of CI pipelines. You can change the default behavior by specifying a buildOnPreview flag.

Jump over to the applicable migration guide to learn how to update your existing Pulumi program:

Migrating from Docker provider v3
Migrating from Docker provider v4

Migrating from Docker provider v3

Version 3.x of the Docker provider is still widely used because it exposes BuildKit functionality through raw command-line arguments. However, the new Docker Build provider exposes those arguments as top-level fields on the resource.

Reference the below TypeScript code to compare a v3 definition of a docker.Image with the new dockerbuild.Image. In particular, note the extraOptions in v3 are now top-level fields, e.g., cacheFrom, cacheTo, platforms, etc.

// v3 Image
const v3 = new docker.Image("v3-image", {
 imageName: "myregistry.com/user/repo:latest",
 localImageName: "local-tag",
 skipPush: false,
 build: {
 dockerfile: "./Dockerfile",
 context: "../app",
 target: "mytarget",
 args: {
 MY_BUILD_ARG: "foo",
 },
 env: {
 DOCKER_BUILDKIT: "1",
 },
 extraOptions: [
 "--cache-from",
 "type=registry,myregistry.com/user/repo:cache",
 "--cache-to",
 "type=registry,myregistry.com/user/repo:cache",
 "--add-host",
 "metadata.google.internal:169.254.169.254",
 "--secret",
 "id=mysecret,src=https://www.pulumi.com/local/secret",
 "--ssh",
 "default=/home/runner/.ssh/id_ed25519",
 "--network",
 "host",
 "--platform",
 "linux/amd64",
 ],
 },
 registry: {
 server: "myregistry.com",
 username: "username",
 password: pulumi.secret("password"),
 },
});

// v3 Image after migrating to docker-build.Image
const v3Migrated = new dockerbuild.Image("v3-to-buildx", {
 tags: ["myregistry.com/user/repo:latest", "local-tag"],
 push: true,
 dockerfile: {
 location: "./Dockerfile",
 },
 context: {
 location: "../app",
 },
 target: "mytarget",
 buildArgs: {
 MY_BUILD_ARG: "foo",
 },
 cacheFrom: [{ registry: { ref: "myregistry.com/user/repo:cache" } }],
 cacheTo: [{ registry: { ref: "myregistry.com/user/repo:cache" } }],
 secrets: {
 mysecret: "value",
 },
 addHosts: ["metadata.google.internal:169.254.169.254"],
 ssh: {
 default: ["/home/runner/.ssh/id_ed25519"],
 },
 network: "host",
 platforms: ["linux/amd64"],
 registries: [{
 address: "myregistry.com",
 username: "username",
 password: pulumi.secret("password"),
 }],
});

Migrating from Docker provider v4

The new Docker Build provider largely exposes the same fields as v4, but the fields are pluralized or renamed to better align with the Docker CLI.

Reference the below TypeScript code to compare a v4 definition of a docker.Image with the new dockerbuild.Image. In particular, note the top-level, pluralized names, e.g., registries, platforms, etc.

// v4 Image
const v4 = new docker.Image("v4-image", {
 imageName: "myregistry.com/user/repo:latest",
 skipPush: false,
 build: {
 dockerfile: "./Dockerfile",
 context: "../app",
 target: "mytarget",
 args: {
 MY_BUILD_ARG: "foo",
 },
 cacheFrom: {
 images: ["myregistry.com/user/repo:cache"],
 },
 addHosts: ["metadata.google.internal:169.254.169.254"],
 network: "host",
 platform: "linux/amd64",
 },
 buildOnPreview: true,
 registry: {
 server: "myregistry.com",
 username: "username",
 password: pulumi.secret("password"),
 },
});

// v4 Image after migrating to docker-build.Image
const v4Migrated = new dockerbuild.Image("v4-to-buildx", {
 tags: ["myregistry.com/user/repo:latest"],
 push: true,
 dockerfile: {
 location: "./Dockerfile",
 },
 context: {
 location: "../app",
 },
 target: "mytarget",
 buildArgs: {
 MY_BUILD_ARG: "foo",
 },
 cacheFrom: [{ registry: { ref: "myregistry.com/user/repo:cache" } }],
 cacheTo: [{ registry: { ref: "myregistry.com/user/repo:cache" } }],
 addHosts: ["metadata.google.internal:169.254.169.254"],
 network: "host",
 platforms: ["linux/amd64"],
 registries: [{
 address: "myregistry.com",
 username: "username",
 password: pulumi.secret("password"),
 }],
});

Conclusion

Launching the new Docker Build provider marks a significant milestone in modernizing container management for developers. The new package, which contains the docker_build.Image resource reflects our commitment to innovation and user-centric development. By incorporating the latest BuildKit technology, the Docker Build provider supports the newest features, enhances multi-architecture builds, and improves caching strategies. Moreover, its integration with Docker Build Cloud underscores our dedication to providing a cutting-edge toolset for developers. This launch is a testament to our ongoing efforts to modernize and improve how developers build images.

Check out the Pulumi Docker Build documentation for more details on utilizing the Docker Build provider.

If you are still wondering, “Which provider should I use?” the answer depends on whether you are doing anything related to docker build.

Provider	Use cases
`@pulumi/docker-build`	Use this provider to build and push Docker images. Ideal for scenarios requiring automation of image creation with Docker, including setting up CI/CD pipelines for Docker image generation.
`@pulumi/docker`	Use this provider for everything else, including managing local Docker containers, networks, and volumes on the host machine.

Property Search: Enhanced Resource Search in Pulumi Cloud

Meagan Cojocar — Tue, 13 Jun 2023 00:00:00 +0000

Two months ago we launched Resource Search for Pulumi Cloud to offer multi-cloud resource search and analytics across every cloud environment in your organization. Today, we’re excited to announce that we are taking this a step further and exposing a much larger set of data to search over: resource properties.

Including property names and values in your resource searches enables you to search on an even richer variety of dimensions, everything from instance type, to database version, to region and availability zone, to whether resources are open to the internet. By searching on property data, in addition to overall resource metadata, you can craft expressive queries that serve key compliance, operational and diagnostic use cases.

Pulumi Resource Search, including the ability to search on property data, is applicable across all cloud resources, whether they reside in AWS, Google Cloud, Azure, or any of Pulumi’s 100+ cloud providers. Support for querying on property data is available in the Enterprise and Business Critical editions of Pulumi Cloud.

Feature Overview

Cloud resources come with various properties that define their behavior, performance, location, and more. You can now use Pulumi Resource Search to search based on these properties in addition to other resource metadata across 100+ clouds. Whether you’re trying to find resources running in a specific region, instances with a particular tag, or databases running a certain engine version, searching over properties with Pulumi Cloud streamlines and simplifies your tasks.

Using the new feature is simple and straightforward. Go to the Pulumi console, navigate to the search bar, and start typing the property you’re interested in. For example, if you’re searching for all AWS RDS instances running MySQL with a particular engine version, simply type .engineVersion:5.7 and press enter.

The ability to search for resources based on their properties is a powerful tool for optimizing cloud resource management. It provides deeper insights into your infrastructure and aids in identifying resources that are misconfigured, out-of-date, or not adhering to your organization’s best practices. Essentially, it’s a significant step towards more proactive, informed, and efficient management of your cloud environment.Customers like Alkira, a company that reinvents networking for the cloud era, have found it to be a significant step towards more proactive, informed, and efficient management of the cloud.

“I’m making developers at Alkira significantly more productive while also making my job easier using Pulumi’s IaC platform and features like Pulumi Insights and Deployments,” said Santosh Dornal, Head of Software Test & DevOps, Alkira. “I can get developers using IaC immediately with Pulumi Deployments and its GitHub integration, while Pulumi Insights makes it really easy to find idle developer environments that need to be shut down, which reduces our cloud costs.”

Example queries

Before Property Search, using just Resource Search, you were able to find resources by their name, type, URN, parent, the team they are in, and so on… but today, any property on your resource will be indexed by Pulumi Cloud and searchable after each stack update. Let’s walk through some examples of the types of queries you can now perform.

AI Assist & properties

Pulumi AI Assist allows you to use natural language to search for resources. AI Assist works by converting natural language to a query in our search syntax. This becomes even more valuable with property search. Resources can have dozens of properties, and remembering exactly how to search for property values can be challenging, especially when they change depending on the provider. AI Assist is going to step in so that you don’t have to remember everything.

Here is a list of example natural language queries we have seen customers use and the AI Assist output:

Natural language	Query result
resources tagged with stack production	`.tags.stack:production`
resources without a tag in production	`-tags: stack:production`
aurora engines running MySQL engine version 5	`type:"aws:rds/cluster:Cluster" .engine:aurora .engineVersion:5`
s3 buckets read write in production	`type:"aws:s3/bucket:Bucket" stack:production .acl:public-read-write`
buckets objects in all clouds	`type:"aws:s3/bucket:Bucket" OR type:"google:storage/bucket:Bucket" OR type:"azure:storage:BlobContainer"`
all resources in us east 1	`region:"us-east-1"`
resources with a private ip	`.privateIpAddress:`

Search syntax

Properties can be used in your queries by using a . prefix. For example, if you want to see all resources with an Instance Type property, you type in the search bar .instanceType:. If you want to only look at a specific value, you can search for the value behind the colon. In this case our query would become .instanceType:t3.large.

Here are some query examples we have seen customers use:

project:pulumi-service stack:staging .instanceType:
type:aws:rds/cluster:Cluster .engine:aurora .engineVersion:8
type:aws:s3/bucket:Bucket stack:production .acl:public-read
team:data package:snowflake .availabilityZone:us-west-2a
vpc created:2023-06-12 .tags.Environment:production

Use the REST API

In addition to searching for resources you can also export them to perform analytics: whether that is something simple like opening it in a spreadsheet or something more sophisticated, like if you want to regularly ingest it to your data warehouse. You can download a CSV from the console or get the data using the Data Export API.

But the value of Resource Search is not limited to what you do in the console and for analytics. You can also leverage the Resource Search API to create automation and workflows off of search results. Find below details on how to use the API.

const headers = {
 Accept: "application/json",
 Authorization: "token pul-abc123",
};

const body = await (
 await fetch("https://api.pulumi.com/api/orgs/{org}/search/resources?properties=true?query=.instanceType", {
 method: "GET",
 headers: headers,
 })
).json();

console.log(body);

const headers = {
 Accept: "application/json",
 Authorization: "token pul-abc123",
};

const body = await(
 await fetch("https://api.pulumi.com/api/orgs/{org}/search/resources?properties=true?query=.instanceType", {
 method: "GET",
 headers: headers,
 }),
).json();

console.log(body);

import requests
headers = {
 'Accept': 'application/json',
 'Authorization': 'token pul-abc123'
}

r = requests.get('https://api.pulumi.com/api/orgs/{org}/search/resources?properties=true?query=.instanceType', headers = headers)

print(r.json())

package main

import (
 "bytes"
 "net/http"
)

func main() {
 headers := map[string][]string{
 "Accept": []string{"application/json"},
 "Authorization": []string{"token pul-abc123"},
 }

 data := bytes.NewBuffer([]byte{jsonReq})
 req, err := http.NewRequest("GET", "https://api.pulumi.com/api/orgs/{org}/search/resources?properties=true?query=.instanceType", data)
 req.Header = headers

 client := &http.Client{}
 resp, err := client.Do(req)
 // ...
}

using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Threading.Tasks;
using Newtonsoft.Json;

public class HttpExample
{
 private HttpClient Client { get; set; }

 public HttpExample()
 {
 Client = new HttpClient();
 }

 public async Task MakeGetRequest()
 {
 string url = "https://api.pulumi.com/api/orgs/{org}/search/resources?properties=true?query=.instanceType";
 var result = await GetAsync(url);
 }

 /// Performs a GET Request
 public async Task GetAsync(string url)
 {
 //Start the request
 HttpResponseMessage response = await Client.GetAsync(url);

 //Validate result
 response.EnsureSuccessStatusCode();

 }

 /// Deserialize object from request response
 private async Task DeserializeObject(HttpResponseMessage response)
 {
 //Read body
 string responseBody = await response.Content.ReadAsStringAsync();

 //Deserialize Body to object
 var result = JsonConvert.DeserializeObject(responseBody);
 }
}

URL obj = new URL("https://api.pulumi.com/api/orgs/{org}/search/resources?properties=true?query=.instanceType");
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
con.setRequestMethod("GET");
int responseCode = con.getResponseCode();
BufferedReader in = new BufferedReader(
 new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
 response.append(inputLine);
}
in.close();
System.out.println(response.toString());

Wrapping it all up

We are excited to be extending Pulumi Resource Search with access to search over every property of your resources. We can’t wait to see what new capabilities it unlocks for your organization. As always, submit any feedback on the feature in the Pulumi Cloud Requests repository.

Happy searching!