Pulumi Blog: Monica Rodriguez

Pulumi Google Cloud Classic 7.0

Monica Rodriguez — Fri, 10 Nov 2023 09:00:00 -0400

The latest major release of the Pulumi Google Cloud Classic Provider is available now! This 7.0 release contains the latest upstream changes to keep you up-to-date along with a highly requested bug fix, keeping your journey in managing Google Cloud resources fresh and smooth.

The Pulumi Google Cloud Classic provider can be used to provision any of the cloud resources available in the upstream provider. It is part of the suite of Pulumi official providers, which means that it is officially maintained and supported by Pulumi. The provider is also open source and available on GitHub for you to contribute and grow.

Join Pulumi on Thursday, November 30, 2023 for a live workshop on getting started with serverless apps on Google Cloud Run, which will feature the new GCP 7.0 provider.

If you are new to Pulumi and would like to get started with the Google Cloud provider, visit any of these pages to jump right in!

Getting Started - A guided walkthrough for creating your first project
Setup & Install - Instructions on installing the Google Cloud provider
How-to guides - Learn how to use the Google Cloud provider to provision specific resources
Templates - Use a quickstart template to create a new project
Pulumi Neo - Ask Pulumi Neo to create a new project

Looking back

Since the initial launch of 6.0 almost two years ago, we’ve shipped over 70 releases to deliver the latest upstream enhancements while also adding many of our own. Here are some of the highlights from these releases:

Coverage and Growth

The Google Cloud provider now supports 250+ functions, 800+ resources, 10,000+ input properties. It continues to grow in coverage and features while staying up-to-date with the latest upstream changes. The chart below shows the growth of the provider in the last 2 years.

New Modules

Version 6.0 of the Google Cloud provider introduced support for several new modules, including:

If you haven’t used these modules yet, now is a great time to try them out! Check out the Pulumi Registry for more information on how to get started or ask Pulumi Neo to create a new project for you.

What’s New in 7.0

Service Account Rename

This release includes a fix for the naming of the Service Account resource. For consistency and ease of use, serviceAccount has been renamed to serviceaccount. Node.js, Python, and .Net programs do not require any updates as the resource name has been aliased. However updates are required for Go, Java, and YAML to adjust the import paths and resource type.

- import "github.com/pulumi/pulumi-gcp/sdk/v6/go/gcp/serviceAccount"
+ import "github.com/pulumi/pulumi-gcp/sdk/v7/go/gcp/serviceaccount"

- import com.pulumi.gcp.serviceAccount.*;
+ import com.pulumi.gcp.serviceaccount.*;

- type: gcp:serviceAccount:Account
+ type: gcp:serviceaccount:Account

Labels Rework

Upstream changes for provider, resource, and data source labels are included in this release. Global Google Cloud Platform labels configured on the provider are supported through the new defaultLabels field. Default labels will be applied to all resources with a labels field. Resources that previously contained a single labels field will now contain three fields:

The labels field is now non-authoritative and only manages the label keys defined in your configuration for the resource.
The output-only pulumiLabels field merges the labels defined in the resource’s labels field and the global defaultLabels. If the same label key exists on both the resource level and provider level, the value on the resource will override the provider-level default.
The output-only effectiveLabels will list all the labels present on the resource in GCP, including labels configured through Pulumi.

Due to the two new output fields, a resource that has a labels field may show a diff when upgrading from pulumi-gcp v6 to v7. This diff reflects adding the new resource Output fields and is safe to do.

You can find more information about this change in the upstream documentation.

Upgrading

Upgrading from the previous version of the Google Cloud provider is straightforward. Simply update the version in your dependency manifest file, install the new package based on your language and package manager, then run pulumi up to migrate your existing stack. For Go programs, you will need to update specific imports within your program as well. This information is also available in our migration guide in the Pulumi Registry.

- "@pulumi/gcp": "^6.0.0"
+ "@pulumi/gcp": "^7.0.0"

- pulumi_gcp>=6.0.0,<7.0.0
+ pulumi_gcp>=7.0.0,<8.0.0

- require github.com/pulumi/pulumi-gcp/sdk/v6/go/pulumi v6.0.0
+ require github.com/pulumi/pulumi-gcp/sdk/v7/go/pulumi v7.0.0

- <PackageReference Include="Pulumi.Gcp" Version="6.*" />
+ <PackageReference Include="Pulumi.Gcp" Version="7.*" />

- com.pulumi:pulumi-gcp:6.0.0
+ com.pulumi:pulumi-gcp:7.0.0

Wrapping Up

As always, we welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is easy and free. Get started today!

Happy coding!

Pulumi Release Notes: AI in the CLI, Go generics, AWS 6.0, Review Stacks and so much more!

Monica Rodriguez — Fri, 29 Sep 2023 09:00:00 -0400

We’ve had an exciting quarter at Pulumi, shipping all kinds of improvements from our providers to our Cloud service. To stay up-to-date on all the details and additional improvements between release blogs, be sure to check out the pulumi/pulumi repo changelog for CLI enhancements and the Pulumi Cloud features in the new features blogs for updates on Pulumi Service features. We have a lot to cover, so let’s get started!

AI
- Resource Search - AI Assist
- Pulumi Insights and AI in the CLI
Pulumi Cloud
Core
Providers and Packages
Wrap Up

AI

Resource Search - AI Assist

Pulumi Cloud Resource Search AI assist is now generally available to all organizations, offering natural language search capabilities. This feature allows you to query their infrastructure with plain language requests. Suggested queries and an “I’m Feeling Lucky” button are now available to simplify the search process and provide insights into Resource Search functionality.

Pulumi Insights and AI in the CLI

We’ve integrated Pulumi Insights, including AI and Resource Search, into the pulumi CLI. Now, you can harness the power of AI and resource search directly in your terminal. This update also includes features like AI Assist for resource search and fine-grained resource property search, making these tools easily accessible via the pulumi CLI.

pulumi org search ai -q "all Kinesis streams modified since September 1st 2023"

pulumi ai web "deploy metabase on AWS with a managed postgres database"

Pulumi Cloud

Slack and Deployment Notifications

We’ve made several improvements to our Pulumi Cloud Webhooks to enhance deployment notifications and response times. Now, you can easily integrate Slack with Pulumi Cloud for organization or stack notifications without hosting their infrastructure. This integration provides centralized visibility, allowing teams to quickly access stack and update details, operation information, and user activity, streamlining collaboration and reducing context switching. Event notifications and filtering options have been expanded for both Pulumi Cloud Webhooks and the Slack integration, making deployment monitoring more efficient and accessible. These enhancements are available to all Pulumi Cloud organizations.

Microsoft Teams Webhook Integration

Setting up Microsoft Teams notifications is now easier than ever with support for Microsoft Teams formatted webhooks. You can quickly enable notifications about your Pulumi stacks and organizations in your Microsoft Teams workspace by providing an incoming webhook URL. New REST API endpoints and a Service Provider are available, allowing for efficient webhook creation at scale. These improvements enhance ChatOps workflows and provide better visibility into infrastructure changes while maintaining support for generic webhooks. For more details and how to get started, check out this blog post.

Admin Organization Access Tokens in Pulumi Cloud

The introduction of Admin permission scope for Organization Access Tokens provides elevated privileges for users who need them. These tokens enable actions like transferring stacks, modifying user access, team management, and more, previously restricted with regular Organization Access Tokens. Admin Organization Access Tokens cater to programmatic use cases that require higher-level permissions while maintaining a member’s usual permissions. This enhancement expands the utility of Organization Access Tokens for Enterprise and Business Critical customers, and is available during the 14-day trial period as well.

Restore Stacks

Pulumi Cloud now offers the capability to restore previously deleted stacks, a feature requested by many users. This addition enables users to recover stacks that were either mistakenly deleted or intentionally removed while retaining their activity history. This self-service recovery option can be conveniently accessed via the Pulumi Cloud console by navigating to the Stacks page, locating the “restore stacks” function, and selecting the stacks they wish to recover. Up to the last 10 deleted stacks can be restored, and older ones can be retrieved through support.

Review Stacks

New to Pulumi Deployments are Review Stacks, ephemeral cloud environments created automatically when a pull request is opened. These environments enable developers to make changes to both application and infrastructure code simultaneously, streamlining the development process. Review Stacks comment with the outputs from the deployment, making it easier to review code changes and collaborate with teammates. Review Stacks help optimize cloud costs by only existing as long as necessary during feature development. Configuring Review Stacks is a simple process that enhances development efficiency and collaboration among teams. Find more information in the Review Stacks documentation.

Property Search

Resource Search for Pulumi Cloud capabilities have been enhanced by allowing users to search over resource properties in addition to metadata, across over 100 cloud providers. The Property Search feature provides deeper insights into cloud infrastructure, enabling users to search based on specific property names and values, such as instance type, region, or database version. This new feature is available in the Enterprise and Business Critical editions of Pulumi Cloud and aims to streamline resource management, compliance, and diagnostics, ultimately enhancing efficiency and collaboration in cloud environments.

Natural language	Query result
resources tagged with stack production	`.tags.stack:production`
resources without a tag in production	`-tags: stack:production`
aurora engines running MySQL engine version 5	`type:"aws:rds/cluster:Cluster" .engine:aurora .engineVersion:5`
s3 buckets read write in production	`type:"aws:s3/bucket:Bucket" stack:production .acl:public-read-write`
buckets objects in all clouds	`type:"aws:s3/bucket:Bucket" OR type:"google:storage/bucket:Bucket" OR type:"azure:storage:BlobContainer"`
all resources in us east 1	`region:"us-east-1"`
resources with a private ip	`.privateIpAddress:`

Dependent Stack Updates with Pulumi Deployments

Pulumi Deployments has two new features to help manage complex infrastructure projects more easily. The Deployment Webhook Destinations allow automatic triggering of updates in dependent stacks when changes occur in the parent stack. We’ve also added the Pulumi Auto Deploy Package to simplify the way you express stack dependencies, automating the creation and updating of Deployment Webhooks. For all the details and to get started, visit the blog post, auto deploy package registry page, and Pulumi Deployments documentation.

Core

Go Generics

We’ve implemented Go generics in our SDKs, enabling type safety and reducing excessive code generation. This means you can write more reliable and maintainable infrastructure code with reduced chances of panics, leading to more robust cloud infrastructure deployments. For a deeper dive into this feature and how to get started, explore this blog post.

// Given
var a pulumi.IntOutput
var b pulumi.StringOutput

// Before (could panic at runtime if you got something wrong)
o := pulumi.All(a, b).ApplyT(func(vs []interface{}) string { // could panic
 a := vs[0].(int) // could panic
 b := vs[1].(string) // could panic
 return strconv.Itoa(a) + b
}).(pulumi.StringOutput) // could panic

// After (compile-time type-safety)
o := pulumix.Apply2(a, b, func(a int, b string) string {
 return strconv.Itoa(a) + b
})

Pulumi Convert

The introduction of pulumi convert in the Pulumi CLI simplifies the migration process for teams transitioning from Terraform to Pulumi. This feature allows you to convert entire Terraform projects into Pulumi, including support for modules, core Terraform 1.4 features, and most Terraform built-in functions. Starting from Pulumi CLI version 3.71.0, you can directly access this converter, streamlining the conversion of full Terraform programs. This enhancement builds upon our CrossCode foundations, enabling universal infrastructure as code support and introducing the concept of converter plugins for future integrations. This feature supports various use cases, such as migrating existing Terraform projects and modules to Pulumi, enhancing our commitment to providing versatile solutions for infrastructure as code.

$ pulumi convert --from terraform --language typescript --out pulumi

$ pulumi convert --from terraform --language python --out pulumi

$ pulumi convert --from terraform --language go --out pulumi

$ pulumi convert --from terraform --language csharp --out pulumi

Pulumi New support for SSH-style Git URLs

In the v3.77.0 release of the Pulumi CLI, we added support for pulumi new to use SSH-style Git URLs which enables accessing templates from private Git repos, authenticated via normal Git SSH support. Many teams have been building libraries of their own private templates, and this enables those templates to be used much more naturally via pulumi new and the Deploy with Pulumi button.

$ pulumi new git@github.com:acmecorp/templates/website

New Community Supported Pulumi Language - Scala

The team at Virtus Lab released support for Scala in Pulumi as part of their Besom project at https://github.com/VirtusLab/besom. As they highlight in their README:

Besom Scala SDK for Pulumi lets you leverage the full power of Pulumi Infrastructure as Code Platform using the Scala programming language. Scala support is currently in Public Beta.

The Scala language support builds on Pulumi’s existing Java and JVM support, but with an SDK custom designed to target idiomatic Scala API design patterns.

 // For each file in the directory, create an S3 object stored in `siteBucket`
 val uploads: Array[Output[s3.BucketObject]] = File(siteDir).listFiles().map { file =>
 val name = NonEmptyString(file.getName) match
 case Some(name) => name
 case None => throw new RuntimeException("Unexpected empty file name")
 s3.BucketObject(
 name,
 s3.BucketObjectArgs(
 bucket = siteBucket.id, // reference the s3.Bucket object
 source = FileAsset(file.getAbsolutePath), // use FileAsset to point to a file
 contentType = Files.probeContentType(file.toPath) // set the MIME type of the file
 ),
 CustomResourceOptions(
 dependsOn = siteBucket.map(List(_))
 )
 )
 }

Providers and Packages

AWS 6.0

The 6.0 release of the Pulumi AWS provider introduces essential enhancements and updates to one of the most extensively used providers in the Pulumi ecosystem. These improvements encompass TF Plugin Framework support, the inclusion of new resources and functions, significant SDK size optimizations for faster downloads and improved developer experiences, resolutions to top-upvoted issues, and removal of deprecated resources. To dig deeper into these changes, check out the migration guide and blog post.

Azure Native 2.0

The Pulumi Azure Native provider 2.0 has been shipped with enhanced performance and usability, making it easier to manage Azure resources efficiently. Key features include a significant reduction in SDK size for faster development, refreshed default module versions for access to the latest Azure features, expanded top-level default module coverage, title case consistency, simplified user-assigned identity inputs, and more. Visit the blog post and migration guide to learn more.

Kubernetes 4.0

Pulumi Kubernetes 4.0 brings new enhancements in managing Kubernetes resources. It embraces Kubernetes Server-Side Apply as the default mode, enabling features like upserting and Patch resources. Improved diffs remove the dependence on kubectl annotations, and provide simpler access to outputs in the Python, Go and Java SDKs, supporting any self-hosted or managed Kubernetes cluster with version 1.13 or newer. You can migrate existing projects or start new ones with Kubernetes 4.0 for superior Kubernetes resource management. Find more information in the blog post and migration guide.

OIDC Support for Azure

The Azure Native and Azure Classic providers now fully support Azure OIDC authentication. OIDC, or OpenID Connect. OIDC authentication allows you to authenticate your Pulumi program with Azure using federated identity, reducing the need for secrets management. You can establish the trust relationship between Azure and other services like GitHub and configure your Pulumi program to use OIDC for authentication, making your cloud infrastructure management more secure and efficient.

New provider resources

In addition to releasing major versions of 3 of our most popular providers, we’re always keeping our providers up-to-date to bring you the latest functionality in your Pulumi programs. We’ve added 793 resources and 762 functions across our most popular providers, along with many more throughout our ecosystem.

New Community Providers

Welcome to the 17 new community providers in the registry!

Wrap Up

We’ve had an incredible couple months introducing so many exciting features and enhancements across our entire platform. From Slack and Microsoft Teams integrations to Review Stacks, major provider release to Go Generics, we’ve been focused on making your cloud development experience more efficient and powerful. Don’t miss out on all these exciting developments—check out our changelog for CLI updates and our new features blogs for the latest on Pulumi Cloud. Explore these new features today and see the difference they can make in your projects. Happy coding!

Announcing 6.0 of the Pulumi AWS Provider

Mikhail Shilkov — Tue, 29 Aug 2023 09:00:00 -0400

We are excited to announce 6.0 of the Pulumi AWS provider. The AWS provider is the most heavily used provider across the entire Pulumi ecosystem, and offers access to the full surface area of the upstream Terraform AWS Provider in Pulumi projects in all supported languages. The 6.0 release brings a substantial set of fixes and improvements to the provider, including a number of breaking changes as part of the major version release.

This blog post walks you through the list of notable changes in the new major version.

Join Pulumi on Thursday, September 7, 2023 for a live workshop on getting started with infrastructure as code on AWS, which will feature the new AWS 6.0 provider.

Terraform Plugin Framework Support

More and more Terraform resources are migrating to the Terraform Plugin Framework. We are excited to announce that 6.0 supports all resources implemented with both the new Terraform Plugin SDK and the legacy Terraform Plugin Framework technologies. This means that you get access to the entire API of the upstream provider.

New Resources and Functions

Since the previous major release of 5.0, we have shipped 57 additional minor and patch releases, enabling you to keep up with the latest and greatest enhancements. The new major version of the AWS provider brings 56 new resources and 23 new functions. Additions include resources for Audit Manager, Chime, Fin Space, Global Accelerator, Open Search, Quicksight, Resource Explorer, Route53, Transfer, and many other AWS services.

You can see a full list of new resources and functions in Release Notes.

SDK Size Optimization

The AWS provider contains more than 1,200 resources with over 12,000 properties between them. As a result, it’s expected that the SDK size of the provider has grown to be fairly large. With that in mind, we invested in some key areas to reduce the size of generated files. Depending on the runtime, the SDK is now up to 90% smaller than it used to be in v5 of the provider!

One particular optimization is related to wafv2 resources, that were particularly difficult to use in Pulumi AWS v5 (issue one, two, three). The new release includes a refactored wafv2 module which uses recursive types. Using recursive types allowed us to remove a lot of excess generated code.

Smaller SDK sizes mean faster downloads, snappier experience with editors and code completion tools, less time needed for pulumi preview and pulumi up operations, and smaller binaries, all of which contribute to a streamlined developer experience for Pulumi users.

Upstream Breaking Changes

Pulumi AWS 6.0 ships all improvements and bug fixes of upstream versions from 5.0.0 to 5.11.0. It also contains a number of upstream breaking changes. Please refer to the changelog to navigate the entire list.

Fixes for Highly Upvoted Issues

The new release ships fixes for many highly upvoted issues along with Pulumi-specific enhancements. This includes resolving constant diff from wafv2 with no changes and unexpected behavior of defaultTags that have been fixed and are available in the new release.

MaxItemsOne and Deprecations

The 6.0 release removes several deprecated resources. All of those resources were replaced with newer versions containing a more logical organization of modules and resource names. The removed resources have been deprecated for years, so it’s unlikely that you were still using them. But in case you do, we applied automatic aliases to make migration as straightforward as possible.

We also shipped a few, small breaking changes to properties that may allow one or many items. We aligned those with the capabilities of the upstream provider and the AWS services ensure Pulumi users can access the full capabilities of the AWS platform.

Migration Guide

You can see a full list of changes and learn more about migrating your existing programs in our Migration Guide.

Get Started Today

You can browse our API reference docs with inline examples or explore the Pulumi AWS Provider repository to get started today! And don’t forget to register for the live workshop on September 7, 2023 to see AWS 6.0 in action!

Pulumi Kubernetes 4.0: Even More Kubernetes-Native

Levi Blackstone — Wed, 19 Jul 2023 16:00:00 -0400

Since the very earliest days of the Pulumi project, Kubernetes has been a core part of the Pulumi platform. The initial Pulumi Kubernetes provider supported the entire API surface area of the Kubernetes platform, derived directly and automatically from the Kubernetes OpenAPI specifications, and available to all of Pulumi’s familiar programming languages. Since then, we have offered day one support for every new Kubernetes version, added support for Helm, YAML, Kustomize and CRDs, added tools for converting to Pulumi (kube2pulumi and crd2pulumi) and delivered the Pulumi Kubernetes Operator. During that same time, Kubernetes usage has continued to expand within the ecosystem and among Pulumi users, with the Kubernetes provider growing from the fourth most used to the second most used provider on the platform.

We are excited to release the next major version of our Kubernetes provider - Pulumi Kubernetes 4.0.

Join us for a free, on-demand webinar with Pulumi and the CNCF to learn how to use Pulumi’s Kubernetes Provider and other open-source tools to build internal developer platforms. Register →

Pulumi Kubernetes 4.0 uses Kubernetes Server-Side Apply by default, supports upserting on all resources, and enables new Patch resources for every resource type in the provider. It also brings improved diffs, removes dependence on kubectl annotations, and provides simpler access to outputs in the Python, Go and Java SDKs, supporting any self-hosted or managed Kubernetes cluster with version 1.13 or newer.

Together, these additions make Pulumi’s support for Kubernetes even more Kubernetes-native, allowing the best of both worlds between Kubernetes’ flexible resource model and Pulumi’s robust desired state infrastructure as code model, all while using your favorite languages (TypeScript, Python, Go, .NET and Java) to define and manage your Kubernetes Infrastructure. Pulumi programs that need to manage Kubernetes resources that are also modified by other systems (external controllers, managed Kubernetes service control planes, kubectl or other IaC solutions) can faithfully represent the modifications they want to make to existing resources within the Pulumi resource lifecycle. Pulumi accurately detects any differences between the current state of resources in the cluster, and the desired state specified by Pulumi IaC.

Server-Side Apply, Upserts and Patch Resources

Kubernetes Server-side Apply (SSA) mode is now default in the v4 provider. SSA mode enables several useful new features: Kubernetes resources can be safely managed by more than one controller. Any conflicting fields will produce a detailed error message which avoids accidentally overriding changes made by other controllers. Resources can be “upserted”; in other words, create the resource if it does not exist, or apply the specified configuration to an existing resource. Patch resources add a new way to manage shared Kubernetes resources. Patch resources can be used to make targeted changes, and do not require the entire resource to be specified. This is a convenient way to update shared resources; a common example is resources that are provisioned by the cloud provider rather than as part of the Pulumi stack. Deleting a Patch resource undoes the patched change, but does not remove the shared Kubernetes resource. See the SSA how-to guide for more details and usage examples for SSA mode.

Note: Users may opt into the previous default Client-side Apply (CSA) mode by setting the enableServerSideApply provider configuration to “false”.

In this example, we define a ConfigMap as an upsert which will create the resource if it doesn’t exist.

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";

const example = new kubernetes.core.v1.ConfigMap("example", {
 metadata: {
 annotations: {
 "pulumi.com/patchForce": "true",
 },
 name: "example",
 },
 data: {
 foo: "bar",
 },
});

import pulumi
import pulumi_kubernetes as kubernetes

example = kubernetes.core.v1.ConfigMap(
 "example",
 metadata=kubernetes.meta.v1.ObjectMetaArgs(
 annotations={
 "pulumi.com/patchForce": "true",
 },
 name="example",
 ),
 data={
 "foo": "bar",
 })

package main

import (
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/core/v1"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/meta/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err = corev1.NewConfigMap(ctx, "example", &corev1.ConfigMapArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Annotations: pulumi.StringMap{
 "pulumi.com/patchForce": pulumi.String("true"),
 },
 Name: pulumi.String("example"),
 },
 Data: pulumi.StringMap{
 "foo": pulumi.String("bar"),
 },
 })
 if err != nil {
 return err
 }
 return nil
 })
}

using System.Collections.Generic;
using Pulumi;
using Kubernetes = Pulumi.Kubernetes;

return await Deployment.RunAsync(() =>
{
 var example = new Kubernetes.Core.V1.ConfigMap("example", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Annotations =
 {
 { "pulumi.com/patchForce", "true" },
 },
 Name = "example",
 },
 Data =
 {
 { "foo", "bar" },
 },
 });
});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.kubernetes;
import com.pulumi.kubernetes.core_v1.ConfigMap;
import com.pulumi.kubernetes.core_v1.ConfigMapArgs;
import com.pulumi.kubernetes.meta_v1.inputs.ObjectMetaArgs;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 var example = new ConfigMap("example", ConfigMapArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .annotations(Map.of("pulumi.com/patchForce", "true"))
 .name("example")
 .build())
 .data(Map.of("foo", "bar"))
 .build());

 }
}

resources:
 example:
 type: kubernetes:core/v1:ConfigMap
 properties:
 metadata:
 annotations:
 pulumi.com/patchForce: "true"
 name: example
 data:
 foo: bar

Improved diff experience

The process for calculating what changes a provider needs to make to achieve the desired state is called “diffing”. This is a complex process that accounts for the current and previous state of the Pulumi program (the “inputs”), the stack’s state/checkpoint, and the actual current state of any resources managed by the provider. A major challenge of this process is detecting “drift”, where resources have been changed by something other than the Pulumi stack so that these changes are not reflected in Pulumi’s view of the world. The Server-side Apply (SSA) support available in v3.x versions does detect resource drift, but the implementation was slow for larger stacks. The v4 provider now detects resource drift in both Client-side Apply and Server-side Apply by running a refresh operation. This brings the behavior in line with other providers, and offers consistent performance that scales well to larger stacks.

As part of this work, we removed the use of the kubectl.kubernetes.io/last-applied-configuration annotation, which was previously used by the provider to track input values for Kubernetes resources. This change addresses many longstanding issues caused by the use of this annotation, including challenges working with resources that were modified by tools other than kubectl.

Migrating to Kubernetes 4.0

You can migrate your existing Pulumi Kubernetes projects to 4.0 today, or start a new project targeting 4.0 with any of the Pulumi templates.

For users migrating to 4.0, check out the v4 Provider Migration Guide with full details of the additional changes in this release.

Getting Started

Try Infrastructure as Code for Kubernetes by following this quick getting started guide.

Join us for a free, on-demand webinar with Pulumi and the CNCF to learn how to use Pulumi’s Kubernetes Provider and other open-source tools to build internal developer platforms.Register →

Azure Native Provider 2.0: Streamlined, Expanded, and More Powerful than Ever

Monica Rodriguez — Wed, 14 Jun 2023 14:00:00 -0400

Update: this content was updated on July 11 2023 to reflect the general availability of the 2.0 release.

We are thrilled to announce the release of the Pulumi Azure Native Provider 2.0, a significant upgrade to Pulumi’s native provider for Microsoft Azure. The Azure Native provider offers the most complete support for Azure possible - with same day access to the entire surface area of the Azure features from Azure Resource Manager. Every property of each module is always represented in the SDKs. The 2.0 release brings a host of exciting features and improvements for performance and usability that will enhance your experience with managing Azure resources and empower you to build robust and scalable cloud infrastructure more efficiently.

At Pulumi, we understand the importance of keeping up with the ever-evolving cloud landscape. The Pulumi Azure Native Provider 2.0 represents our commitment to providing you with the best tools to harness the full power of Microsoft Azure. With automatic generation from Azure API specs, we are able to provide you with same-day access to the entire Azure API surface including all available properties. Let’s dive into some of the key enhancements in this release!

Join Pulumi, Azure, and GitHub on Tuesday, September 19, 2023 for a live workshop on getting started with infrastructure as code on azure. You’ll learn how to use the Azure Native Provider and set up an automated deployment workflow through GitHub.

Reduced SDK Size for Faster Development

We’ve listened to your feedback and have made significant optimizations to the Azure Native Provider SDKs. With careful fine-tuning and restructuring, we have achieved a significant decrease in size of over 50% for SDKs with multi-versioned modules. This improvement translates into faster development times, reduced resource consumption, and an overall smoother experience when working with the provider.

So, how did we accomplish this? For starters, we’ve removed all deprecated Azure API modules up to the 2.0 release date. The bigger impact was made by identifying and removing all forward-compatible module versions. That is to say we’ve excluded any redundant versions and kept only the versions that introduce breaking changes. Where these versions are not the current latest and new default, they remain available as explicit versions.

You may be wondering about the significance between explicit and default versions. Well, when Azure releases new features they often create a new version of a service API. These versions are named using dates such as “2021-01-02” or “2022-05-08-preview”. There’s many versions of each module in the SDK, so we pick a version of each module for the top-level default. To learn more about module versions, visit the Azure Native Version Guide.

Refreshed Default Module Versions

In the latest release, we have refreshed the default module versions to align with the latest Azure API versions from Microsoft. With our continued commitment to same-day access to Azure API surface, you always have access to the most up-to-date features and capabilities offered by Azure services. By leveraging the latest module versions, you can take advantage of the latest and most desired features directly from Azure while building your infrastructure with Pulumi.

If you would like to upgrade to 2.x, but continue using the same Azure API version that was used in your 1.x, you can do that by specifying the compatible explicit version available in the 2.x provider. Shown below are examples of how that change might look.

- import { EventSubscription } from "@pulumi/azure-native/eventgrid";
+ import { EventSubscription } from "@pulumi/azure-native/eventgrid/v20200601";

- from pulumi_azure_native import eventgrid
+ from pulumi_azure_native.eventgrid import v20200601 as eventgrid20200601

- using EventGrid = Pulumi.AzureNative.EventGrid;
+ using EventGrid20200601 = Pulumi.AzureNative.EventGrid.V20200601;

- import eventgrid "github.com/pulumi/pulumi-azure-native-sdk/eventgrid/v2"
+ import eventgrid20200601 "github.com/pulumi/pulumi-azure-native-sdk/eventgrid/v2/v20200601”

- type: azure-native:eventgrid:Topic
+ type: azure-native:eventgrid/v20200601:Topic

Expanded Top-Level Default Module Coverage

We’re excited to expand our top-level default version coverage by 226 resources, 274 functions, and 5526 input properties in this release, bringing the total to 1,637, 2,015, and 30,652 respectively. With this extensive collection of modules at your disposal, you can easily provision and manage the full range of Azure services using Pulumi. Whether you’re working with virtual machines, databases, networking, or any other Azure service, the Azure Native Provider has you covered.

Resolved Title Case Inconsistency

For a consistent and predictable experience across the provider, we have resolved an issue with title case inconsistencies across resources and functions. All previous resource names have been aliased, and do not require any changes to the Pulumi program. However, functions that have a corrected name will need to be updated. This enhancement ensures a unified and predictable experience while working across resource types, making it easier for you to navigate and understand the provider’s capabilities.

Simplified User Assigned Identity Inputs

We have simplified the representation of user assigned identity inputs by recognizing them as string arrays. This change provides a more intuitive and clear approach when configuring user assigned identities for your Azure modules. You can now reference these as UserAssignedIdentities: [exampleUserAssignedIdentity.id] without jumping through any hoops.

import * as resources from "@pulumi/azure-native/resources";
import * as managedidentity from "@pulumi/azure-native/managedidentity";
import * as storage from "@pulumi/azure-native/storage";

// Create a resource group
const resourceGroup = new resources.ResourceGroup("my-resource-group");

// Create a user-assigned managed identity
const userAssignedIdentity = new managedidentity.UserAssignedIdentity("my-user-assigned-identity", {
 resourceGroupName: resourceGroup.name,
});

// Create a storage account that references the user-assigned managed identity
const storageAccount = new storage.StorageAccount("mystorageaccount", {
 resourceGroupName: resourceGroup.name,
 kind: "StorageV2",
 location: resourceGroup.location,
 identity: {
 type: "UserAssigned",
 userAssignedIdentities: [userAssignedIdentity.id],
 },
 sku: {
 name: "Standard_LRS",
 },
});

Before this change, the identity block would need to use an apply as follows:

user_assigned_identity.id.apply(lambda id: {id: {}})

Migrating from 1.x to 2.x

We recommend you update to the latest version of Azure Native 1.x prior to moving to 2.x. This will enable you to address existing deprecations and see the recommended substitutions ahead of the major version upgrade. Head over to the migration guide which outlines the necessary steps to update your existing Pulumi projects and ensuring a smooth migration experience.

Current users of the Pulumi Azure Classic Provider can follow the classic to native migration guide to take advantage of all the latest and greatest features available in Pulumi the Azure Native Provider.

While you can immediately start leveraging the improved performance, you may need to make some changes in order to continue using specific Azure API module versions or user assigned identity inputs. Use the Pulumi Azure Native Provider documentation within your IDE to determine the explicit API version that is available and fully compatible with your existing deployment.

The Pulumi Azure Native Provider 2.0 is available today for Pulumi programs using TypeScript, Python, .NET, and Go. To learn more about the Pulumi Azure Native Provider 2.0 and explore its capabilities, check out our updated documentation and resources. We value your feedback and encourage you to reach out in community slack or open an issue for any questions or suggestions.

Try out the Pulumi Azure Native Provider 2.0 today and take the next step in your journey with infrastructure as code on Microsoft Azure!

Getting Started

Try Infrastructure as Code on Azure by following this quick getting started guide.

Fast Docker Image Builds with Pulumi

Monica Rodriguez — Wed, 08 Mar 2023 14:00:00 -0800

How do I speed up Docker image builds with Pulumi? Use BuildKit (the default since Docker 23), enable a registry or layer cache so repeated builds reuse work, write a multi-stage Dockerfile so production images skip build-time dependencies, and reach for the dedicated Docker Build provider when you need buildx features like multi-platform images, build secrets, or Docker Build Cloud. With these techniques together, repeat builds in a Pulumi program commonly drop from minutes to seconds.

TL;DR

BuildKit is the default in current Docker releases and in the Pulumi Docker providers — you generally do not need to opt in.
Cache your layers somewhere persistent. A registry cache (cacheFrom / cacheTo) is what makes CI builds fast across runners and contributors.
Multi-stage Dockerfiles are the single biggest win for image size and rebuild speed when you have a compile or bundle step.
Use pulumi/docker-build for new projects. It exposes the full buildx surface (multi-platform, secrets, multiple cache backends, Docker Build Cloud).
Use pulumi/docker v4+ if you already depend on it — its docker.Image resource also rebuilds only on context change and supports BuildKit and registry caching.

What actually moves the needle

The numbers below are representative of what teams report from a typical Node.js or Python web app (single Dockerfile, ~300 MB final image). Your mileage varies with project size, network, and runner specs — treat this as a directional comparison, not a benchmark suite.

Configuration	Cold build	Repeat build (no source change)	Repeat build (small source change)
Legacy builder, no caching	minutes	minutes	minutes
BuildKit, local layer cache only	minutes	seconds (cache hit)	tens of seconds
BuildKit + multi-stage Dockerfile	minutes	seconds (cache hit)	tens of seconds
BuildKit + multi-stage + registry cache (`cacheFrom`/`cacheTo`)	minutes (first run)	seconds (cache hit)	seconds–tens of seconds, even on a fresh CI runner
`docker-build` provider with buildx + Docker Build Cloud	under 2 min	seconds (cache hit)	seconds

The headline pattern: each row past the first is dominated by what the build can skip, not by raw CPU. Pulumi’s job is to wire those caches up declaratively so they keep working across every pulumi up.

How does Pulumi build Docker images?

Pulumi has two providers for image builds, and which one you should use depends on how new your project is.

The Docker provider (pulumi/docker) is the original. Its docker.Image resource can build a local context and push to any registry as part of a Pulumi deployment, alongside resources for containers, networks, and volumes. Since v4 it rebuilds only when the build context changes and uses BuildKit by default.
The Docker Build provider (pulumi/docker-build) is a newer, dedicated provider focused exclusively on image builds. It exposes Docker’s buildx interface, so you get multi-platform builds, multiple cache backends, build secrets, and Docker Build Cloud as first-class features.

For new projects, prefer docker-build. For existing programs that already use docker.Image, you do not have to migrate — docker.Image continues to receive the speed and caching improvements covered below.

Why are my Docker builds slow?

Three things make Docker builds slow in practice, and Pulumi addresses each one declaratively.

No cache, or the wrong cache. A fresh CI runner has no local Docker layers. Without a remote cache, every build reinstalls every dependency.
Cache busting on noise. If your Dockerfile copies package.json and the rest of your source in the same step, every source change invalidates the dependency-install layer. Order layers from least-changing to most-changing.
Building dev tooling into the production image. Compilers, linters, and test frameworks have no business in a runtime image. Multi-stage builds let you discard them.

How do I enable BuildKit and registry caching in Pulumi?

BuildKit has been the default builder since Docker 23 and is the default in both Pulumi providers. The piece you usually have to configure is where the cache lives — local disk on a developer laptop is fine, but for CI you almost always want a registry cache so layers persist across runners.

With the Docker Build provider (recommended)

import * as docker_build from "@pulumi/docker-build";

const image = new docker_build.Image("my-image", {
 tags: ["docker.io/pulumibot/demo-image:latest"],
 context: { location: "./app" },
 dockerfile: { location: "./app/Dockerfile" },
 platforms: [
 "linux/amd64",
 "linux/arm64",
 ],
 cacheFrom: [{ registry: { ref: "docker.io/pulumibot/demo-image:cache" } }],
 cacheTo: [{ registry: { ref: "docker.io/pulumibot/demo-image:cache", mode: "max" } }],
 push: true,
});

cacheFrom and cacheTo together are the important pair: the first run populates the cache, every later run on any machine pulls it. mode: "max" exports cache for every layer, not just the final ones.

With the Docker provider (`docker.Image`)

import * as docker from "@pulumi/docker";

const image = new docker.Image("my-image", {
 imageName: "docker.io/pulumibot/demo-image:latest",
 build: {
 context: "app",
 dockerfile: "Dockerfile",
 platform: "linux/amd64",
 cacheFrom: {
 images: ["docker.io/pulumibot/demo-image:cache-base"],
 },
 },
});

The docker.Image resource accepts a list of cache images via cacheFrom.images. Tag a previous build with a stable cache tag (for example, :cache-base) and reference it here so subsequent builds reuse its layers.

How do I write a Dockerfile that builds fast?

The Dockerfile matters at least as much as the Pulumi code wrapping it. Two patterns do most of the work.

Order layers least-volatile first. Copy lockfiles and run dependency installs before copying source code. That way changing a .ts file does not bust the npm ci layer.

# Dependencies — only invalidates when lockfiles change
COPY package.json package-lock.json ./
RUN npm ci

# Source — changes constantly, but the dependency layer above survives
COPY . .
RUN npm run build

Use multi-stage builds to drop tooling. Compile in a fat image, copy the artifact into a slim one. The runtime image stays small and the build stage gets cached independently.

FROM node:20 AS build
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:20-slim AS runtime
WORKDIR /app
COPY --from=build /app/dist ./dist
COPY --from=build /app/node_modules ./node_modules
CMD ["node", "dist/server.js"]

When you combine multi-stage with a registry cache (cacheTo mode max), CI runs on fresh machines reuse both stages — the row in the table above where small source changes finish in 10–30 seconds.

How do I build multi-platform images with Pulumi?

Use the docker-build provider and pass a list to platforms. BuildKit will build all variants in parallel and push a manifest list, so a single tag serves whatever architecture the puller needs.

const image = new docker_build.Image("api", {
 tags: ["ghcr.io/example/api:latest"],
 context: { location: "./api" },
 platforms: [
 "linux/amd64",
 "linux/arm64",
 ],
 push: true,
});

If you are still on docker.Image, the build.platform field accepts a single target (for example, "linux/amd64"); for true multi-arch manifests, prefer docker-build.

How do I see Docker build output in Pulumi?

Filtered Docker build and push logs render directly in pulumi up output for both providers, so you can debug a failing layer without leaving your Pulumi run. With the Docker Build provider, structured progress shows per-stage timing — useful when you are trying to figure out which layer is the slow one.

How do I migrate an older `docker.Image` program?

If you are upgrading from a pre-v4 program, pulumi-docker v4 reshaped the docker.Image resource and renamed several supporting types, but does not manage backend state, so updating the package version and adjusting types is generally enough. The full migration guide lives in the v4.0.0 release notes.

If you are starting fresh or want the full BuildKit feature set, install pulumi-docker-build instead and follow the Docker Build provider getting started guide.

Getting started

Docker Build provider — recommended for new projects.
Docker provider — the original, including docker.Image, containers, and networks.
Introducing the Docker Build provider — deeper dive on buildx, secrets, and Docker Build Cloud.

The cheapest performance win in most Pulumi programs is wiring up a registry cache and a multi-stage Dockerfile. Start there, then reach for buildx features as the project grows.

Pulumi Blog: Monica Rodriguez

Pulumi Google Cloud Classic 7.0

Looking back

Coverage and Growth

New Modules

What’s New in 7.0

Service Account Rename

Labels Rework

Upgrading

Wrapping Up

Pulumi Release Notes: AI in the CLI, Go generics, AWS 6.0, Review Stacks and so much more!

AI

Resource Search - AI Assist

Pulumi Insights and AI in the CLI

Pulumi Cloud

Slack and Deployment Notifications

Microsoft Teams Webhook Integration

Admin Organization Access Tokens in Pulumi Cloud

Restore Stacks

Review Stacks

Property Search

Dependent Stack Updates with Pulumi Deployments

Core

Go Generics

Pulumi Convert

Pulumi New support for SSH-style Git URLs

New Community Supported Pulumi Language - Scala

Providers and Packages

AWS 6.0

Azure Native 2.0

Kubernetes 4.0

OIDC Support for Azure

New provider resources

New Community Providers

Wrap Up

Announcing 6.0 of the Pulumi AWS Provider

Terraform Plugin Framework Support

New Resources and Functions

SDK Size Optimization

Upstream Breaking Changes

Fixes for Highly Upvoted Issues

MaxItemsOne and Deprecations

Migration Guide

Get Started Today

Pulumi Kubernetes 4.0: Even More Kubernetes-Native

Server-Side Apply, Upserts and Patch Resources

Improved diff experience

Migrating to Kubernetes 4.0

Getting Started

Azure Native Provider 2.0: Streamlined, Expanded, and More Powerful than Ever

Reduced SDK Size for Faster Development

Refreshed Default Module Versions

Expanded Top-Level Default Module Coverage

Resolved Title Case Inconsistency

Simplified User Assigned Identity Inputs

Migrating from 1.x to 2.x

Getting Started

Fast Docker Image Builds with Pulumi

TL;DR

What actually moves the needle

How does Pulumi build Docker images?

Why are my Docker builds slow?

How do I enable BuildKit and registry caching in Pulumi?

With the Docker Build provider (recommended)

With the Docker provider (docker.Image)

How do I write a Dockerfile that builds fast?

How do I build multi-platform images with Pulumi?

How do I see Docker build output in Pulumi?

How do I migrate an older docker.Image program?

Getting started

With the Docker provider (`docker.Image`)

How do I migrate an older `docker.Image` program?