Pulumi Blog: Releases

Now in Public Preview: Store Terraform State in Pulumi Cloud

Claire Gaestel — Thu, 05 Mar 2026 00:00:00 +0000

Platform engineering teams managing infrastructure across Terraform and Pulumi now have a way to unify state management without rewriting a single line of HCL. Starting today, Pulumi Cloud can serve as a Terraform state backend, letting you store and manage Terraform state alongside your Pulumi stacks. Your team continues using the Terraform or OpenTofu CLI for day-to-day operations while gaining the benefits of Pulumi Cloud: AI-powered infrastructure management with Pulumi Neo — our infrastructure agent — encrypted state storage, update history, state locking, role-based access control, audit policies, and unified resource visibility through Insights.

This feature is now available in public preview.

Why this matters

Most organizations adopting Pulumi are not starting from scratch. They have years of Terraform deployments spread across teams, and migrating everything to a new IaC tool overnight is not realistic. We have heard from customers who are excited about the power of Pulumi Cloud but have had to manage migration projects before they can fully benefit from centralized visibility and governance.

The Terraform state backend in Pulumi Cloud changes that equation. Instead of requiring a full code conversion before teams see value, you can migrate your state in minutes and immediately unlock Pulumi Cloud capabilities for your existing Terraform infrastructure — including Neo, Pulumi’s AI infrastructure agent. Once your Terraform state is in Pulumi Cloud, Neo can reason about those resources the same way it does for Pulumi IaC stacks: finding resources, troubleshooting issues, understanding dependencies, and writing infrastructure code PRs. Teams that prefer Terraform can keep using it, while platform engineers get a single AI-powered control plane across the entire infrastructure estate.

What you get

When you store Terraform state in Pulumi Cloud, your Terraform-managed resources get the following added functionality:

Agentic infrastructure with Neo. Neo, Pulumi’s AI infrastructure agent, works across your entire cloud footprint — Terraform and Pulumi IaC alike. Once your Terraform state is in Pulumi Cloud, you can ask Neo to find resources across both tools, trace dependencies that span Terraform and Pulumi stacks, troubleshoot configuration issues, and generate new infrastructure code informed by your existing resources. This means platform teams get a single AI-powered interface regardless of which IaC tool manages each piece of infrastructure.

Encrypted state with update history. State is encrypted in transit and at rest. Every change is tracked as a versioned checkpoint visible in the stack activity tab, giving you full rollback capability. This is a common concern for teams currently storing state in S3 buckets.

Automatic state locking. Pulumi Cloud prevents concurrent Terraform operations from corrupting state, without requiring you to configure DynamoDB tables or other external locking mechanisms.

Role-based access control. Control who can read or modify each stack using teams and RBAC, applying the same access policies you use for Pulumi stacks.

Unified resource visibility. View Terraform-managed resources alongside Pulumi-managed resources in Resource Search. Each Terraform resource appears in the console using a pulumi:terraform:<tf-type> naming convention, so you can search and filter using the attribute names you already know.

Audit policies. Run audit (detective) policy packs against your Terraform-managed stacks, including Pulumi’s pre-built compliance packs for CIS, PCI, and more. Pulumi Cloud performs a best-effort schema mapping from Terraform resource shapes to Pulumi provider equivalents, so existing policy packs work without modification in most cases.

Stack outputs and references. Terraform root module outputs are automatically mapped to Pulumi stack outputs, making them available via stack references and the pulumi-stacks ESC provider. This is useful for sharing foundational infrastructure like VPC IDs or DNS zones between Terraform and Pulumi stacks, and for incremental migrations where legacy infrastructure stays in Terraform while new stacks are written in Pulumi.

How it works

Pulumi Cloud implements the Terraform remote backend API. You point the Terraform CLI at Pulumi Cloud using the standard backend "remote" configuration block, and no changes to your Terraform code or workflow are required.

Each Terraform workspace maps to a Pulumi stack. The workspace name follows the convention <project>_<stack>. For example, networking_prod creates a stack named prod in the networking project.

Get started

Migration from S3, Azure Blob, GCS, local backends, or HCP Terraform (Terraform Cloud) takes minutes and is documented in the Terraform state backend guide. From S3, Azure Blob, GCS, or local state, back up your state, update your backend block to point to Pulumi Cloud, set TF_TOKEN_api_pulumi_com, and run terraform init -migrate-state. From HCP Terraform, export state manually and push it to Pulumi Cloud.

Each Terraform resource stored in Pulumi Cloud counts as a resource under management, the same as a Pulumi-managed resource. See the pricing page for details.

Store Terraform State in Pulumi Cloud

If you have questions or feedback, join us in the Pulumi Community Slack or open an issue on GitHub.

Now GA: Up to 20x Faster Pulumi Operations for Everyone

Thomas Gummerer — Thu, 05 Mar 2026 00:00:00 +0000

In January, we introduced a major performance enhancement for Pulumi Cloud through a fundamental change to how Pulumi manages state that speeds up operations by up to 20x. After a staged rollout across many organizations, it is now enabled by default for every Pulumi Cloud operation. No opt-in required—just use Pulumi CLI v3.225.0+ with Pulumi Cloud. The improvement applies to pulumi up, pulumi destroy, and pulumi refresh; pulumi preview does not modify state, so it is unchanged.

What this means for you

First and foremost, nothing about how you work with pulumi needs to change. Your updates now benefit from better parallelism and should thus complete faster. Before this change, pulumi always saved a full snapshot to the cloud, so the current state could always be recovered if something goes wrong. With journaling, we now only send the state of each operation, which allows us to send these updates in parallel, as long as resources are not related to each other. For the full deep dive, see the blog post linked above.

Production results

Since January, we’ve had many early adopters of journaling. This helped us shake out one final bug on the server side, and journaling has been stable since then. With that we feel confident in rolling this out to all our users.

We’ve also gathered some real-world data on how journaling is performing. The data from the preview period shows some significant improvements for update times. For stacks with fewer than 100 resources, the median improvement is 25.3%, while the p90 improvement is 75.2%, and we’ve seen a p99 improvement of up to 92.6% Meanwhile, for larger stacks, the median improvement is 60.2%. We need more data for stacks with more than 100 resources, we will update this blog once that comes in.

This data already shows the expected significant improvement in update times, especially for larger stacks, though the improvements strongly depend on the shape and type of resources that are being set up. Stacks with many resources, that are quick to update benefit more than smaller stacks with slower to set up resources. For more numbers see also the Benchmarks section in the previous blog post

What you need to do

While this was an opt-in process using the PULUMI_ENABLE_JOURNALING environment variable, this opt-in is no longer required. Just upgrade your Pulumi CLI to v3.225.0+ and use the Pulumi Cloud backend, and journaling will automatically speed up your updates.

If you encounter any issues, reach out on the Pulumi Community Slack or through Pulumi Support. You can also set the PULUMI_DISABLE_JOURNALING=true env variable to opt out of journaling.

New in Pulumi IaC: `onError` Resource Hook

Tom Harding — Mon, 23 Feb 2026 00:00:00 +0000

You can now control what happens when a resource fails during create, update, or delete—retry with backoff, fail fast, or handle errors in custom code. Last year, Pulumi IaC introduced the resource hooks feature, allowing you to run custom code at different points in the lifecycle of resources. Today we’re adding the onError hook so you can react when operations fail.

Recovering from errors

When a Pulumi program encounters an error while creating, updating, or deleting a resource, the operation halts and the error is reported back. Sometimes that’s not what we want—errors can be intermittent or temporary. If you’ve hit transient failures or resource-not-ready errors, the onError hook can help.

A common case is resource readiness: creating resources that depend on DNS propagation or the readiness of other servers. The program can fail simply because it ran too soon. Instead of failing, we can wait and retry. The example below shows how:

import * as pulumi from "@pulumi/pulumi";

const notStartedRetryHook = new pulumi.ErrorHook(
 "retry-when-not-started",
 async (args) => {
 const latestError = args.errors[0] ?? "";

 if (!latestError.includes("resource has not yet started")) {
 return false; // do not retry, this is another type of error
 }

 await new Promise((resolve) => setTimeout(resolve, 5000));
 return true; // retry
 },
);

const res = new MyResource("res", {}, {
 hooks: {
 onError: [notStartedRetryHook],
 },
});

import time

import pulumi


def retry_when_not_started(args: pulumi.ErrorHookArgs) -> bool:
 latest_error = args.errors[0] if args.errors else ""

 if "resource has not yet started" not in latest_error:
 return False # do not retry, this is another type of error

 time.sleep(5)
 return True # retry


not_started_retry_hook = pulumi.ErrorHook(
 "retry-when-not-started",
 retry_when_not_started,
)

res = MyResource(
 "res",
 opts=pulumi.ResourceOptions(
 hooks=pulumi.ResourceHookBinding(
 on_error=[not_started_retry_hook],
 ),
 ),
)

package main

import (
 "strings"
 "time"

 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 hook, err := ctx.RegisterErrorHook(
 "retry-when-not-started",
 func(args *pulumi.ErrorHookArgs) (bool, error) {
 latest := ""
 if len(args.Errors) > 0 {
 latest = args.Errors[0]
 }

 if !strings.Contains(latest, "resource has not yet started") {
 return false, nil // do not retry, this is another type of error
 }

 time.Sleep(5 * time.Second)
 return true, nil // retry
 },
 )
 if err != nil {
 return err
 }

 _, err = NewMyResource(ctx, "res", &MyResourceArgs{}, pulumi.ResourceHooks(&pulumi.ResourceHookBinding{
 OnError: []*pulumi.ErrorHook{hook},
 }))
 if err != nil {
 return err
 }

 return nil
 })
}

using System;
using System.Threading;
using System.Threading.Tasks;
using Pulumi;

class ErrorHookStack : Stack
{
 public ErrorHookStack()
 {
 var retryHook = new ErrorHook(
 "retry-when-not-started",
 async (args, cancellationToken) =>
 {
 var latestError = args.Errors.Count > 0 ? args.Errors[0] : "";

 if (!latestError.Contains("resource has not yet started"))
 {
 return false; // do not retry, this is another type of error
 }

 await Task.Delay(TimeSpan.FromSeconds(5), cancellationToken);
 return true; // retry
 });

 var res = new MyResource("res", new MyResourceArgs(), new CustomResourceOptions
 {
 Hooks = new ResourceHookBinding
 {
 OnError = { retryHook },
 },
 });
 }
}

Each time the operation fails, the hook receives the new error plus all previous attempts’ errors (newest first). The hook returns true or false to tell Pulumi whether to retry. If you return false, the program fails as normal with the most recent error. With that information, you can implement many failure models:

Use the number of errors to implement backoff—for example, wait one second on the first failure, two seconds on the second, and so on.
For known-intermittent resources, always retry once before failing.
Inspect error text to retry only for specific conditions (as in the example) and fail fast for others.

The callback runs in your language of choice, so you have full control over how failures are handled.

Next steps

This feature is fully supported in our Node, Python, Go, and .NET SDKs as of v3.219.0. For more information, see the hooks documentation.

Java and YAML do not support resource hooks.

Thanks for reading, and feel free to reach out with any questions via GitHub, X, or our Community Slack.

New in Pulumi IaC: `replacementTrigger` Resource Option

Tom Harding — Thu, 22 Jan 2026 00:00:00 +0000

Pulumi IaC gives us a declarative interface to updates. When we perform an update, Pulumi calculates the difference between your currently deployed infrastructure and what is being proposed, then deploys only what is required to migrate from the old state to the new state. Normally, this is exactly what we want: we minimize the amount of work required to perform the update, and don’t recreate anything unnecessarily. However, every now and then, we want to override this behavior.

Recently, we talked about the new replaceWith option, which allows us to tell Pulumi that any replacement of one resource should always trigger a replacement of another (for example, if the database is replaced, we should replace our application server). Today, we’re going to take this idea one step further and talk about another new feature that gives us even more control over this process: replacement triggers.

Forcing replacements

Let’s imagine we have a resource that, upon creation, generates some private keys. If that’s all the resource does, it probably isn’t ever going to change as far as Pulumi’s concerned: we wanted the resource before, and we want it after, so there is no change and no need to replace the resource that is already live. Now, let’s imagine that we want to cycle these private keys every month. How do we tell Pulumi that, if this update happens more than two weeks after the last time this resource was created, we want to recreate it?

As another example, perhaps we want to replace a resource every time some external version number is bumped. Let’s imagine a documentation server may need to be replaced every time a new version of an API is made available. We could use the --replace flag each time, but this process is error-prone to do manually, and would incur a maintenance burden to automate.

Defining replacement triggers

In essence, a replacement trigger is just a value attached to the resource as metadata. However, whenever this value changes between updates, it will trigger a replace operation on the given resource, regardless of whether anything else has changed.

Let’s take our previous example: we want something to be replaced every month. With replacement triggers, we can solve this by representing the current year and month as a string:

...

const today = new Date()

const keyManager = new KeyManagerResource("key-manager", {}, {
 replacementTrigger: today.getMonth() + '-' + today.getFullYear()
});

...

...
from datetime import datetime

today = datetime.now()
trigger = f"{today.month}-{today.year}"

key_manager = KeyManagerResource("key-manager", {},
 opts=pulumi.ResourceOptions(replacement_trigger=trigger))

...

...

today := time.Now()
trigger := fmt.Sprintf("%d-%d", int(today.Month()), today.Year())

keyManager, err := NewKeyManagerResource(ctx, "key-manager", &KeyManagerResourceArgs{},
 pulumi.ReplacementTrigger(pulumi.Any(trigger)))

...

...

var today = LocalDate.now();
var trigger = String.format("%d-%d", today.getMonthValue(), today.getYear())

var keyManager = new KeyManagerResource("key-manager",
 KeyManagerResourceArgs.Empty,
 CustomResourceOptions.builder()
 .replacementTrigger(Output.of(trigger))
 .build());

...

...

var today = DateTime.Now;
var trigger = $"{today.Month}-{today.Year}";

var keyManager = new KeyManagerResource("key-manager", new KeyManagerResourceArgs(),
 new CustomResourceOptions
 {
 ReplacementTrigger = Output.Create(trigger)
 });

...

When we run this update for the first time, the replacement trigger will be persisted to the Pulumi state. Each time we run an update, we’ll re-calculate the date string, and compare it against the current string. Finally, when we run the update again next month, and the date string no longer matches, our key-manager will be replaced and our new keys will be generated!

Next steps

This feature is fully supported across all our SDKs as of v3.215.0. For more information about resource options, see the resource options documentation.

Thanks for reading, and feel free to reach out with any questions via GitHub, X, or our Community Slack.

Introducing the Stash Resource in Pulumi IaC

Fraser Waters — Mon, 12 Jan 2026 00:00:00 +0000

We’re excited to announce the Stash resource, a new built-in Pulumi resource that lets you save arbitrary values directly to your stack’s state. Whether you need to capture a computed result, record who first deployed your infrastructure, or persist configuration that should remain stable across updates, Stash provides a simpler and more ergonomic solution.

Why Stash?

Infrastructure code often produces values that need to persist beyond a single deployment. Maybe you’re generating a random identifier that should stay consistent, tracking which team member initially set up a stack, or recording a timestamp from the first deployment. Previously, you’d need workarounds like external storage, custom resources, or careful state manipulation.

The Stash resource helps with that. It takes an input value, stores it in your stack’s state, and makes it available as an output property. The output property preserves the original value even when the input changes in subsequent deployments, making Stash perfect for “first-run” scenarios where you want to capture and preserve a value from the initial deployment.

Using Stash

Creating a Stash is straightforward. Here’s how you’d capture the username of whoever first deploys the stack (using Node.js):

import * as pulumi from "@pulumi/pulumi";
import * as os from "os"; // Node.js built-in module

const firstDeployer = new pulumi.Stash("firstDeployer", {
 input: os.userInfo().username,
});

export const originalDeployer = firstDeployer.output;

The first time this runs, both input and output will show the current user. On subsequent deployments by different users, input will update to show the new user, but output will continue returning the original deployer’s name.

Stash supports any value type—strings, numbers, objects, arrays, and nested structures. It also respects secret annotations, so if you stash a secret value, it stays encrypted in your state.

When to replace

Since Stash preserves the original value by design, updating the stored value requires a replacement. You have several options:

Use --target-replace during pulumi up:

pulumi up --target-replace urn:pulumi:dev::my-project::pulumi:index:Stash::firstDeployer

Run pulumi state taint to mark the resource for replacement:

pulumi state taint urn:pulumi:dev::my-project::pulumi:index:Stash::firstDeployer

Use the replacementTrigger resource option to automate replacements based on value changes:

import * as pulumi from "@pulumi/pulumi";
import * as os from "os";

const remoteConfig = fetch("https://example.com/my-service").then(response => response.json())

const myStash = new pulumi.Stash("myStash",
 {
 input: os.userInfo().username,
 }, {
 replacementTrigger: remoteConfig.someValue,
 }
);

export const stashedValue = myStash.output;

With replacementTrigger, when remoteConfig.someValue changes, the Stash resource will be replaced and the new input value will be captured.

Learn more

The Stash resource is available now in Pulumi v3.208.0 and later across all supported languages. Check out the Stash documentation for detailed examples in TypeScript, Python, Go, C#, and YAML.

We’d love to hear how you’re using Stash and any feedback you have on it! Share your use cases in our Community Slack or open an issue if you see any on pulumi/pulumi.

Happy hacking!

New in Pulumi IaC: `replaceWith` Resource Option

Tom Harding — Tue, 09 Dec 2025 00:00:00 +0000

The magic of Pulumi is that we rarely have to worry about the fine details of how our deployment and infrastructure management works, allowing us to focus instead on what we want. If our program declares an S3 bucket, Pulumi handles creation, updates, and deletion automatically.

Most of the time, this is exactly what we want. However, some use cases require finer-grained control over resource dependencies. Today, we’re introducing the replaceWith resource option, a new feature that gives you explicit control over replacement dependencies between resources.

Custom resource dependencies

When replacing a resource, the Pulumi engine walks the dependency graph to determine what else needs replacement. However, the answer isn’t always obvious:

Some dependencies aren’t obvious at the infrastructure level. For example, restarting a database server may require restarting its consumers to reconnect.
Provider-specific challenges exist. You might not be able to update a subnet’s properties while resources are using it, even without an observable dependency. Any subnet change must be accompanied by changes to all its consumers to avoid errors.
Application-level dependencies may exist beyond infrastructure. If two services maintain open connections in the application layer, replacing one should trigger replacement of the other.

Declaring our own dependencies

The replaceWith resource option makes these dependencies explicit in your code. Similar to deletedWith, passing references to other resources via replaceWith defines a relationship: when any target resource is replaced, the source is also replaced. This has some interesting implications:

We can define a number of resources that all replaceWith each other, and so any change to any member of the group will trigger an update to all the others.
Relationships are transitive: organizing resources hierarchically where children replaceWith their parents means any ancestor change triggers replacement of all descendants.

In code

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const database = new aws.rds.Instance("app-db", { /* ... */ });

// The database location is provided to this instance
// by other means, so there is not an explicit dependency
// linking the two.
const app = new aws.ec2.Instance("app-service", { /* ... */ }, {
 // Here, we explicitly tie the two together: if the
 // database is upgraded, for example, we will need to
 // restart the application instance to ensure that
 // a connection is re-established.
 replaceWith: [database],
});

import pulumi
import pulumi_aws as aws

database = aws.rds.Instance('app-db', {})

# The database location is provided to this instance
# by other means, so there is not an explicit dependency
# linking the two.
application = aws.ec2.Instance('app-service', {},
 # Here, we explicitly tie the two together: if the
 # database is upgraded, for example, we will need to
 # restart the application instance to ensure that
 # a connection is re-established.
 opts=pulumi.ResourceOptions(replace_with=[database])
)

import (
 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/rds"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

database, err := rds.NewInstance(ctx, "app-db", &rds.InstanceArgs{ ... })
if err != nil {
 return err
}

// The database location is provided to this instance
// by other means, so there is not an explicit dependency
// linking the two.
application, err := ec2.NewInstance(ctx, "app-service", &ec2.InstanceArgs{ ... },
 // Here, we explicitly tie the two together; if the
 // database is upgraded, for example, we will need to
 // restart the application instance to ensure that
 // a connection is re-established.
 pulumi.ReplaceWith([]pulumi.Resource{database}))
if err != nil {
 return err
}

using Pulumi;
using Aws = Pulumi.Aws;

var database = new Aws.Rds.Instance("app-db", new() { ... });

// The database location is provided to this instance
// by other means, so there is not an explicit dependency
// linking the two.
var application = new Aws.Ec2.Instance("app-service", new() { ... }, new CustomResourceOptions
{
 // Here, we explicitly tie the two together; if the
 // database is upgraded, for example, we will need to
 // restart the application instance to ensure that
 // a connection is re-established.
 ReplaceWith = new[]
 {
 database,
 },
});

import com.pulumi.aws.ec2.Instance;
import com.pulumi.aws.ec2.InstanceArgs;
import com.pulumi.resources.CustomResourceOptions;

var database = new com.pulumi.aws.rds.Instance("app-db",
 com.pulumi.aws.rds.InstanceArgs.builder()
 .build());

// The database location is provided to this instance
// by other means, so there is not an explicit dependency
// linking the two.
var application = new Instance("app-service",
 InstanceArgs.builder()
 .build(),
 // Here, we explicitly tie the two together; if the
 // database is upgraded, for example, we will need to
 // restart the application instance to ensure that
 // a connection is re-established.
 CustomResourceOptions.builder()
 .replaceWith(database)
 .build());

resources:
 database:
 type: aws:rds:Instance
 properties:
 # ...
 # The database location is provided to this instance
 # by other means, so there is not an explicit dependency
 # linking the two.
 application:
 type: aws:ec2:Instance
 properties:
 # ...
 options:
 # Here, we explicitly tie the two together: if the
 # database is upgraded, for example, we will need to
 # restart the application instance to ensure that
 # a connection is re-established.
 replaceWith:
 - ${database}

Of course, this is a simple example, but hopefully it illustrates the point: we now have a way to make implicit dependencies between resources more explicit within our programs, and provide hints to Pulumi as to which services depend on others.

Next steps

This feature is supported as of Pulumi v3.207.0, and is now available within the Go, Python, NodeJS, and Java SDKs, with C# and YAML coming soon. Thanks for reading, and feel free to reach out with any questions via GitHub, X, or our Community Slack.

All Pulumi CLI flags are now supported as environment variables

Tom Harding — Fri, 21 Nov 2025 00:00:00 +0000

With the release of Pulumi v3.208.0, all CLI flags can now be configured as environment variables. This addresses a common friction point of having to remember the same flags across multiple commands and of ensuring that your entire team uses consistent CLI options.

Use cases

An example use case might be that you want to always refresh state before operations, or re-run programs during refresh and destroy (using --run-program). But remembering to add --refresh every time is tedious.

Another example is you’re working on a specific subset of your infrastructure and need to pass the same --target or --exclude flags repeatedly across multiple operations.

You’re working in a CI environment where you want to skip interactive prompts and previews. That means adding --yes and --skip-preview to every single command in your pipeline.

The usual workaround, if there is no env var already, is writing wrapper scripts or shell aliases, but these become another thing to maintain and share across your team.

How it works

Any CLI flag can now be set as an environment variable. The naming convention is straightforward: prefix the flag name with PULUMI_OPTION_, and convert dashes to underscores.

For example:

export PULUMI_OPTION_REFRESH=true

Target specific resources without repeating flags:

export PULUMI_OPTION_TARGET=foo,bar
# Equivalent to: --target foo --target bar

This works for any flag the CLI accepts. The environment variable takes precedence if you don’t explicitly pass the flag on the command line.

Using with direnv for project-level defaults

This becomes particularly useful when combined with tools like direnv. You can define project-specific CLI defaults in a .envrc file at the root of your project:

# Enable very verbose logging for debugging purposes
export PULUMI_OPTION_VERBOSE=3

# Always refresh before any Pulumi operation
export PULUMI_OPTION_REFRESH=true

# Skip previews and any dialogs or user prompts
export PULUMI_OPTION_YES=true
export PULUMI_OPTION_SKIP_PREVIEW=true

Now, running direnv allow . in this directory means that, whenever we access this directory, these options will be automatically applied to all commands that we run until we leave the directory!

Wrapping up

We hope that this change makes Pulumi much easier to configure across a range of different environments, and gives you another tool for managing operation defaults across your team. Thanks for reading, and feel free to share any feedback on GitHub, X, or our Community Slack.

Pulumi Google Cloud Provider Version 9.0.0

Guinevere Saenger — Wed, 24 Sep 2025 13:33:50 -0700

We’re excited to announce the v9 release of the Pulumi Google Cloud Provider! This major release contains important updates to Google Cloud resources and functions, and keeps you up to date with what’s new from Pulumi.

The Pulumi Google Cloud provider can be used to provision any of the Google Cloud resources available in the upstream provider. The provider is open source and available on GitHub so you can be part of the community - issues and pull requests are always welcome!

Here are a few links to help you get started if you are new to Pulumi:

Getting Started - A guided walkthrough for creating your first project
Setup & Install - Instructions on installing the Google Cloud provider
How-to guides - Learn how to use the Google Cloud provider to provision specific resources
Templates - Use a quickstart template to create a new project
Pulumi Neo - Ask Pulumi Neo to help you with your project

In case you missed it - new modules

Throughout the past year, we have worked to keep this provider in sync with the upstream provider. Notable new modules are those supporting AI and Google Gemini. New modules added since the last major release include:

What’s New in 9.0

Upstream Improvements

This release includes the latest improvements and changes from the recent major version in the upstream provider. Some themes are:

Improved Import Validation

Multiple resources throughout the provider have improved validation and error messages to ensure the supplied GCP resource ID matches import formats specified in the documentation.

Increased Field Validation

Several resource fields have been updated to reflect requirements of Google Cloud and will fail validation if not configured correctly, to avoid failures later on. Refer to the migration guide for details.

Documentation Improvements

We are always working on improving and updating our documentation. Among other improvements, this release adds 100 new top-level overview descriptions for resources and functions.

Upgrading

You can find the migration guide on the Pulumi registry.

NEW! Use Pulumi Neo to assist with your upgrade

You can use our new automation agent, Neo, to help you with your provider upgrade. In the Pulumi service, ask Neo to help you with your upgrade by providing the migration guide:

Neo will look at the provided migration guide and your stack, and determine a plan of action. Here, Neo has discovered there were some breaking changes, needing updates to your program.

And finally, Neo can open a pull request with the proposed changes to your code.

Azure Native 3.8: Unified Credentials and Private Clouds

Eron Wright — Thu, 04 Sep 2025 12:00:00 -0700

Today we’re excited to announce Azure Native Provider v3.8, featuring several enhancements that simplify authentication and extend support to private Azure environments. These updates make it easier than ever to manage Azure infrastructure using credentials provided by the hosting environment, such as in Azure Kubernetes Service (AKS), Azure VM, and Azure Cloud Shell.

Simplified Authentication Across Environments

The highlight of this release is a new authentication mode based on DefaultAzureCredential, a feature of the Azure SDK that unifies authentication-related settings across deployment environments.

What’s New

DefaultAzureCredential automatically discovers and uses the best available authentication method for your environment, eliminating the need for an environment-specific configuration. It follows the Azure SDK’s standard credential chain:

Order	Credential	Description
1	Environment	Reads environment variables (e.g., `AZURE_TENANT_ID`, `AZURE_CLIENT_ID`) to authenticate as a service principal.
2	Workload Identity	For programs run on an Azure Kubernetes Service (AKS) cluster.
3	Managed Identity	For programs deployed to an Azure compute resource (e.g., Azure Virtual Machines) or App hosting platform.
4	Azure CLI	For local development using Azure CLI’s `az login` command.
5	Azure Developer CLI	For local development using Azure Developer CLI’s `azd auth login` command.

Real-World Benefits

This means your Pulumi programs can now run seamlessly across:

Local development machines using Azure CLI credentials
CI/CD pipelines with service principals via environment variables
Azure Kubernetes Service with Workload Identity
Azure VMs and App Services with Managed Identity

All without changing a single line of configuration code.

Getting Started

Enable DefaultAzureCredential using Pulumi configuration:

pulumi config set azure-native:useDefaultAzureCredential true
pulumi config set azure-native:subscriptionId <your-subscription-id>

Note that subscriptionId is a required configuration setting in this (and most) authentication modes; it ensures your resources are deployed to the correct Azure subscription.

Private Azure Cloud Support

This release brings improved support for Azure private clouds. A private cloud is a dedicated cloud computing environment used by a single organization.

The provider can now automatically discover, and configure itself for, any Azure cloud based on the ARM_METADATA_HOSTNAME environment variable. Note that this takes precedence over the ARM_ENVIRONMENT variable.

How To Use

The provider can query the Azure metadata service to automatically configure itself for your specific Azure environment:

# Using environment variable
export ARM_METADATA_HOSTNAME=management.azure.example

# Or via Pulumi configuration
pulumi config set azure-native:metadataHost management.azure.example

The provider expects the 2022-09-01 metadata schema, which resembles:

{
 "authentication": {
 "loginEndpoint": "https://login.microsoftonline.com",
 "audiences": [
 "https://management.core.windows.net/",
 "https://management.azure.com/"
 ]
 },
 "name": "AzureCloud",
 "suffixes": {
 "keyVaultDns": "vault.azure.net",
 "storage": "core.windows.net"
 },
 "resourceManager": "https://management.azure.com/",
 "microsoftGraphResourceId": "https://graph.microsoft.com/"
}

The provider automatically retrieves the correct endpoints for authentication, resource management, and other services.

Disabling Instance Discovery

Also in this release is a new setting, disableInstanceDiscovery, to determine whether the provider requests Microsoft Entra instance metadata from the login endpoint (https://login.microsoftonline.com) before authenticating. This setting is for Pulumi programs authenticating in disconnected clouds or private clouds.

Setting disableInstanceDiscovery to true will completely disable both instance discovery and authority validation. Please ensure that the configured authority host is valid and trustworthy.

How To Use

The provider can be configured to disable instance discovery:

# Using environment variable
export ARM_DISABLE_INSTANCE_DISCOVERY=true

# Or via Pulumi configuration
pulumi config set azure-native:disableInstanceDiscovery true

Authentication in AKS with Workload Identity

For programs run in a pod on an Azure Kubernetes Service (AKS) cluster, DefaultAzureCredential automatically uses the workload identity of the pod’s service account. This workload identity could then be granted roles in Azure to deploy stack resources.

Ensure that the application pods using workload identity include the label azure.workload.identity/use: "true" in the pod spec.

Walkthrough

Let’s use Pulumi Kubernetes Operator (PKO) to demonstrate a use case where you’d run Pulumi deployment operations in a pod and could benefit from workload identity.

The operator allocates a dedicated pod for each Pulumi stack under its management, to serve as the execution environment for stack operations. Each stack may use the same or a different service account. With AKS, that service account has a workload identity that providers may use to authenticate to Azure cloud and even to Pulumi cloud.

Create an AKS Cluster

Please follow the steps in “Deploy and configure workload identity on an Azure Kubernetes Service (AKS) cluster” to create an AKS cluster, managed identity, and Kubernetes service account. Those steps are:

You now have an Azure managed identity and associated Kubernetes service account that DefaultAzureCredential can use to authenticate to Azure cloud.

Take note of the clientId that represents the managed identity:

az identity show --name "${USER_ASSIGNED_IDENTITY_NAME}" --resource-group "${RESOURCE_GROUP}" --query clientId
"c2bbe0f5-6349-480a-9f6f-3a5cb3e4ecf9"

Install the Pulumi Kubernetes Operator

Install the operator into the pulumi-kubernetes-operator namespace:

kubectl apply -f https://raw.githubusercontent.com/pulumi/pulumi-kubernetes-operator/refs/tags/v2.0.0/deploy/quickstart/install.yaml

Configure a Pulumi Cloud Access Token

By default, the operator uses Pulumi Cloud as the state backend for your stacks. Please create a Secret containing a Pulumi access token to be used to authenticate to Pulumi Cloud. Follow these instructions to create a personal, organization, or team access token.

kubectl create secret generic -n ${SERVICE_ACCOUNT_NAMESPACE} pulumi-api-secret --from-literal=accessToken=${PULUMI_ACCESS_TOKEN}

Update the Kubernetes Service Account

To use the service account that was created earlier, we need to grant it the system:auth-delegator role:

cat <<EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
 name: "${SERVICE_ACCOUNT_NAMESPACE}-${SERVICE_ACCOUNT_NAME}:system:auth-delegator"
roleRef:
 apiGroup: rbac.authorization.k8s.io
 kind: ClusterRole
 name: system:auth-delegator
subjects:
- kind: ServiceAccount
 name: "${SERVICE_ACCOUNT_NAME}"
 namespace: "${SERVICE_ACCOUNT_NAMESPACE}"
EOF

Define an Example Program

Let’s define a simple Pulumi program that uses Azure Native 3.8 and calls the “getClientConfig” function to access the current identity of the provider.

cat <<"EOF" | kubectl apply -n ${SERVICE_ACCOUNT_NAMESPACE} -f -
apiVersion: pulumi.com/v1
kind: Program
metadata:
 name: sample-workload-identity
program:
 variables:
 clientConfig:
 fn::invoke:
 function: azure-native:authorization:getClientConfig
 clientToken:
 fn::secret:
 fn::invoke:
 function: azure-native:authorization:getClientToken
 outputs:
 clientConfig: ${clientConfig}
 clientToken: ${clientToken}
EOF

Run the Program using Workload Identity

Create a Stack object to run the Pulumi program using your service account and with azure-native:useDefaultAzureCredential enabled.

cat <<EOF | kubectl apply -f -
apiVersion: pulumi.com/v1
kind: Stack
metadata:
 name: sample-workload-identity
 namespace: "${SERVICE_ACCOUNT_NAMESPACE}"
spec:
 programRef:
 name: sample-workload-identity
 stack: sample-workload-identity
 envRefs:
 PULUMI_ACCESS_TOKEN:
 type: Secret
 secret:
 name: pulumi-api-secret
 key: accessToken
 serviceAccountName: "${SERVICE_ACCOUNT_NAME}"
 config:
 azure-native:useDefaultAzureCredential: "true"
 azure-native:subscriptionId: "${SUBSCRIPTION}"
 workspaceTemplate:
 spec:
 podTemplate:
 metadata:
 labels:
 azure.workload.identity/use: "true"
EOF

Checking the stack outputs, we see a clientId matching that of the managed identity!

kubectl get stack/sample-workload-identity -oyaml

apiVersion: pulumi.com/v1
kind: Stack
metadata:
 name: sample-workload-identity
 namespace: default
status:
 conditions:
 - lastTransitionTime: "2025-09-04T23:32:11Z"
 message: the stack has been processed and is up to date
 reason: ProcessingCompleted
 status: "True"
 type: Ready
 outputs:
 clientConfig:
 clientId: c2bbe0f5-6349-480a-9f6f-3a5cb3e4ecf9
 objectId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
 subscriptionId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
 tenantId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
 clientToken: '[secret]'

Conclusion

Azure Native Provider v3.8 represents our commitment to making Azure infrastructure management work seamlessly across all deployment scenarios, from local development to production, from public cloud to private cloud.

Have questions or feedback? Open an issue on GitHub or join the conversation in our Community Slack (#azure channel).

Precise Resource Replacement with Pulumi State Taint

Meagan Cojocar — Sun, 31 Aug 2025 00:00:00 +0000

Sometimes infrastructure needs a clean slate. A VM with a corrupted disk, an expired certificate, or a stuck Kubernetes object. Pulumi CLI v3.192.0 introduces pulumi state taint and pulumi state untaint commands that let you mark resources for replacement—especially valuable when you have state access but restricted cloud permissions.

The New Commands

pulumi state taint <resource-urn> - Mark a resource for replacement
pulumi state untaint <resource-urn> - Cancel the replacement (use --all to untaint all resources)

Note: This provides the same functionality as pulumi up --replace <urn>, but lets you mark resources upfront and preview changes before the update. This can be preferred when the pulumi up happens later in CI/CD, where arranging for --replace flags would be tedious.

Why This Matters

The taint and untaint commands solve several infrastructure management challenges. For example, many organizations restrict direct cloud access to production environments, allowing changes only through CI/CD pipelines. In these cases, you can access Pulumi state but not cloud APIs directly—taint lets you mark problematic resources for replacement and let the next automated deployment handle the actual changes, no emergency access requests required.

Workflow

# 1. Find the resource URN
pulumi stack --show-urns

# 2. Mark for replacement
pulumi state taint urn:pulumi:production::webapp::aws:ec2/instance:Instance::web-server

# 3. Preview the changes
pulumi preview

# 4. Apply the replacement
pulumi up

# Optional: Cancel the taint
pulumi state untaint urn:pulumi:production::webapp::aws:ec2/instance:Instance::web-server

Get Started

Available now in Pulumi CLI v3.192.0. Try it in a development stack first to get familiar with the workflow.

This feature originated from feedback in issue #11657. See the v3.192.0 release notes for full details. Have feedback or ideas? Join the discussion in the Pulumi CLI repository.

Resource hooks

Will Jones — Mon, 30 Jun 2025 00:00:00 +0000

Pulumi programs are declarative, allowing you to specify the desired state of your infrastructure while Pulumi figures out the rest. But what about the times where you want to be more involved in what Pulumi is doing? Resource hooks are one of our most requested features, and from Pulumi 3.182.0 we’re excited to announce that you’ll be able to use them to run arbitrary code at any point in Pulumi’s resource lifecycle!

When you run pulumi up, Pulumi runs your program and works out the set of create, update and delete operations required to realise the state it describes. Hooks allow you to attach custom callbacks to these operations, enabling you to execute custom behaviour before or after they occur. You might want to set up an SSH tunnel to a bastion host before Pulumi attempts to create or update a virtual machine or database; or you may wish to send metrics to your data warehouse whenever Pulumi updates certain resources or properties. Hooks allow you to do all this and more – as with everything else in Pulumi, you are free to make use of the full power of your favourite programming language.

Hooks in action

Let’s check out the use case of opening a tunnel to a remote host before running a command. For this we’ll use a contrived example where we use the socat command to open a local TCP port that forwards to a remote host. Here we’ll open localhost:1234 and forward it to example.com:80. We’ll then use Pulumi’s command provider to invoke curl to make a request to that local port, which will be forwarded to the remote host. To start with, let’s write the program without hooks:

import * as command from "@pulumi/command"

export = async () => {
 const curl = "curl -H'Host: example.com' localhost:1234"

 const cmd = new command.local.Command("curl", {
 create: curl,
 update: curl,
 delete: curl,
 triggers: [new Date().toString()],
 })

 return {
 stdout: cmd.stdout,
 }
}

import * as command from "@pulumi/command"

export = async () => {
 const curl = "curl -H'Host: example.com' localhost:1234"

 const cmd = new command.local.Command("curl", {
 create: curl,
 update: curl,
 delete: curl,
 triggers: [new Date().toString()],
 })

 return {
 stdout: cmd.stdout,
 }
}

import datetime
import pulumi
import pulumi_command

curl = "curl -H'Host: example.com' localhost:1234"

cmd = pulumi_command.local.Command("curl",
 create=curl,
 update=curl,
 delete=curl,
 triggers=[str(datetime.datetime.now())],
)

pulumi.export("stdout", cmd.stdout)

package main

import (
 "time"

 "github.com/pulumi/pulumi-command/sdk/go/command/local"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 curl := "curl -H'Host: example.com' localhost:1234"

 cmd, err := local.NewCommand(ctx, "curl", &local.CommandArgs{
 Create: pulumi.String(curl),
 Update: pulumi.String(curl),
 Delete: pulumi.String(curl),
 Triggers: pulumi.StringArray{pulumi.String(time.Now().String())},
 })
 if err != nil {
 return err
 }

 ctx.Export("stdout", cmd.Stdout)

 return nil
 })
}

using System.Threading.Tasks;

using Pulumi;
using Pulumi.Command.Local;

class Program
{
 static Task<int> Main() => Deployment.RunAsync<MyStack>();
}

class MyStack : Stack
{
 public MyStack()
 {
 var curl = "curl -H'Host: example.com' localhost:1234";

 var cmd = new Command("curl", new CommandArgs
 {
 Create = curl,
 Update = curl,
 Delete = curl,
 Triggers = { System.DateTime.Now.ToString() },
 });

 return new Dictionary<string, object?>
 {
 ["stdout"] = cmd.Stdout,
 };
 }
}

We set up a local Command that will run the same curl command whether it is being created, updated, or deleted. For the purposes of illustration, we also set the triggers property to the current date and time, so that the command will always be run when we run pulumi up. Let’s now do just that:

pulumi up

Assuming we have nothing running on port 1234, we’ll get an error such as the following:

Updating (...)
View in Browser (Ctrl+O): https://app.pulumi.com/...
Type Name Status Info
pulumi:pulumi:Stack resource-hooks **failed** 1 error; 9 messages
++ └─ command:local:Command curl **creating failed** [diff: ]; 1 error
Diagnostics:
command:local:Command (curl):
error: exit status 7: running "curl -H'Host: example.com' localhost:1234":
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (7) Failed to connect to localhost port 1234 after 0 ms: Could not connect to server

As expected, nothing is listening on port 1234, so curl fails with an error. Previously, we might have fixed this by using a wrapper around our Pulumi program, such as the Automation API. With resource hooks, however, the solution is at our fingertips! We’ll hook into the Command resource’s lifecycle to open a tunnel before any operation is run, and to close it afterwards:

import * as child_process from "child_process"
import * as command from "@pulumi/command"
import * as pulumi from "@pulumi/pulumi"

export = async () => {
 let tunnel

 const beforeHook = new pulumi.ResourceHook("before", async args => {
 console.log("Opening tunnel")

 tunnel = child_process.spawn(
 "socat",
 [
 "TCP-LISTEN:1234,fork",
 "TCP:example.com:80",
 ],
 { detached: true }
 )

 console.log(`Tunnel opened: ${tunnel?.pid}`)
 })

 const afterHook = new pulumi.ResourceHook("after", async args => {
 console.log(`Closing tunnel: ${tunnel?.pid}`)
 tunnel?.kill("SIGKILL")
 })

 const curl = "curl -H'Host: example.com' localhost:1234"

 const cmd = new command.local.Command("curl", {
 create: curl,
 update: curl,
 delete: curl,
 triggers: [new Date().toString()],
 }, {
 hooks: {
 beforeCreate: [beforeHook],
 afterCreate: [afterHook],
 beforeUpdate: [beforeHook],
 afterUpdate: [afterHook],
 beforeDelete: [beforeHook],
 afterDelete: [afterHook],
 },
 })

 return {
 stdout: cmd.stdout,
 }
}

import * as child_process from "child_process"
import * as command from "@pulumi/command"
import * as pulumi from "@pulumi/pulumi"

export = async () => {
 let tunnel: child_process.ChildProcessWithoutNullStreams | undefined

 const beforeHook = new pulumi.ResourceHook("before", async args => {
 console.log("Opening tunnel")

 tunnel = child_process.spawn(
 "socat",
 [
 "TCP-LISTEN:1234,fork",
 "TCP:example.com:80",
 ],
 { detached: true }
 )

 console.log(`Tunnel opened: ${tunnel?.pid}`)
 })

 const afterHook = new pulumi.ResourceHook("after", async args => {
 console.log(`Closing tunnel: ${tunnel?.pid}`)
 tunnel?.kill("SIGKILL")
 })

 const curl = "curl -H'Host: example.com' localhost:1234"

 const cmd = new command.local.Command("curl", {
 create: curl,
 update: curl,
 delete: curl,
 triggers: [new Date().toString()],
 }, {
 hooks: {
 beforeCreate: [beforeHook],
 afterCreate: [afterHook],
 beforeUpdate: [beforeHook],
 afterUpdate: [afterHook],
 beforeDelete: [beforeHook],
 afterDelete: [afterHook],
 },
 })

 return {
 stdout: cmd.stdout,
 }
}

import datetime
import subprocess
import pulumi
import pulumi_command

tunnel = None

def before(args: pulumi.ResourceHookArgs):
 global tunnel
 pulumi.log.info("Opening tunnel")
 tunnel = subprocess.Popen(
 ["socat", "TCP-LISTEN:1234,fork", "TCP:example.com:80"],
 )
 pulumi.log.info(f"Tunnel opened: {tunnel.pid}")


before_hook = pulumi.ResourceHook("before", before)

def after(args: pulumi.ResourceHookArgs):
 global tunnel
 if tunnel:
 pulumi.log.info(f"Closing tunnel: {tunnel.pid}")
 tunnel.terminate()
 tunnel.wait()
 tunnel = None

after_hook = pulumi.ResourceHook("after", after)

curl = "curl -H'Host: example.com' localhost:1234"

cmd = pulumi_command.local.Command(
 "curl",
 create=curl,
 update=curl,
 delete=curl,
 triggers=[str(datetime.datetime.now())],
 opts=pulumi.ResourceOptions(
 hooks=pulumi.ResourceHookBinding(
 before_create=[before_hook],
 after_create=[after_hook],
 before_update=[before_hook],
 after_update=[after_hook],
 before_delete=[before_hook],
 after_delete=[after_hook],
 )
 ),
)

pulumi.export("stdout", cmd.stdout)

package main

import (
 "fmt"
 "os/exec"
 "time"

 "github.com/pulumi/pulumi-command/sdk/go/command/local"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

var tunnel *exec.Cmd

func before(args *pulumi.ResourceHookArgs) error {
 fmt.Println("Opening tunnel")
 tunnel = exec.Command("socat", "TCP-LISTEN:1234,fork", "TCP:example.com:80")
 err := tunnel.Start()
 if err != nil {
 return fmt.Errorf("failed to start tunnel: %w", err)
 }
 fmt.Printf("Tunnel opened: %d\n", tunnel.Process.Pid)
 return nil
}

func after(args *pulumi.ResourceHookArgs) error {
 if tunnel != nil && tunnel.Process != nil {
 fmt.Printf("Closing tunnel: %d", tunnel.Process.Pid)
 if err := tunnel.Process.Kill(); err != nil {
 return err
 }
 tunnel.Wait()
 tunnel = nil
 }
 return nil
}

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 beforeHook, err := ctx.RegisterResourceHook("before", before, nil)
 if err != nil {
 return err
 }
 afterHook, err := ctx.RegisterResourceHook("after", after, nil)
 if err != nil {
 return err
 }

 curlCmd := "curl -H'Host: example.com' localhost:1234"

 cmd, err := local.NewCommand(ctx, "curl", &local.CommandArgs{
 Create: pulumi.String(curlCmd),
 Update: pulumi.String(curlCmd),
 Delete: pulumi.String(curlCmd),
 Triggers: pulumi.Array{pulumi.String(fmt.Sprintf("%d", time.Now().Unix()))},
 }, pulumi.ResourceHooks(&pulumi.ResourceHookBinding{
 BeforeCreate: []*pulumi.ResourceHook{beforeHook},
 AfterCreate: []*pulumi.ResourceHook{afterHook},
 BeforeUpdate: []*pulumi.ResourceHook{beforeHook},
 AfterUpdate: []*pulumi.ResourceHook{afterHook},
 BeforeDelete: []*pulumi.ResourceHook{beforeHook},
 AfterDelete: []*pulumi.ResourceHook{afterHook},
 }),
 )
 if err != nil {
 return err
 }

 ctx.Export("stdout", cmd.Stdout)

 return nil
 })
}

using System.Diagnostics;
using System.Threading.Tasks;

using Pulumi;
using Pulumi.Command.Local;

class Program
{
 static Task<int> Main() => Deployment.RunAsync<MyStack>();
}

class MyStack : Stack
{
 public MyStack()
 {
 Process? tunnel = null;

 var beforeHook = new ResourceHook("before", async (args, cancellationToken) =>
 {
 System.Console.WriteLine("Opening tunnel");

 tunnel = Process.Start(new ProcessStartInfo
 {
 FileName = "socat",
 Arguments = "TCP-LISTEN:1234,fork TCP:example.com:80",
 UseShellExecute = false,
 CreateNoWindow = true,
 });

 System.Console.WriteLine($"Tunnel opened: {tunnel?.Id}");
 });

 var afterHook = new ResourceHook("after", async (args, cancellationToken) =>
 {
 System.Console.WriteLine($"Closing tunnel: {tunnel?.Id}");
 tunnel?.Kill();
 });

 var curl = "curl -H'Host: example.com' localhost:1234";

 var cmd = new Command("curl", new CommandArgs
 {
 Create = curl,
 Update = curl,
 Delete = curl,
 Triggers = { System.DateTime.Now.ToString() },
 }, new CustomResourceOptions
 {
 Hooks =
 {
 BeforeCreate = { beforeHook },
 AfterCreate = { afterHook },
 BeforeUpdate = { beforeHook },
 AfterUpdate = { afterHook },
 BeforeDelete = { beforeHook },
 AfterDelete = { afterHook },
 }
 });

 return new Dictionary<string, object?>
 {
 ["stdout"] = cmd.Stdout,
 };
 }
}

When we run pulumi up now, the hooks will be invoked before and after the relevant operation. Since our previous operation failed, we should expect to see a hooked create on our first run:

pulumi up

Updating (...)
View in Browser (Ctrl+O): https://app.pulumi.com/...
Type Name Status Info
+ pulumi:pulumi:Stack resource-hooks created (3s) 3 messages
+ └─ command:local:Command curl created (0.66s)
Diagnostics:
pulumi:pulumi:Stack (resource-hooks):
Opening tunnel
Tunnel opened: 2131167
Closing tunnel: 2131167
Outputs:
stdout: "<!doctype html>\n<html>\n<head>\n <title>Example Domain</title>\n\n <meta charset=\"utf-8\" />\n <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\n <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\n <style type=\"text/css\">\n body {\n background-color: #f0f0f2;\n margin: 0;\n padding: 0;\n font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n \n }\n div {\n width: 600px;\n margin: 5em auto;\n padding: 2em;\n background-color: #fdfdff;\n border-radius: 0.5em;\n box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n }\n a:link, a:visited {\n color: #38488f;\n text-decoration: none;\n }\n @media (max-width: 700px) {\n div {\n margin: 0 auto;\n width: auto;\n }\n }\n </style> \n</head>\n\n<body>\n<div>\n <h1>Example Domain</h1>\n <p>This domain is for use in illustrative examples in documents. You may use this\n domain in literature without prior coordination or asking for permission.</p>\n <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\n</div>\n</body>\n</html>"
Resources:
+ 2 created
Duration: 5s

Success! The tunnel was opened before our create operation and our stack output contains the HTML of the page at example.com.

Another example: application health checks

Let’s tackle a more realistic scenario where hooks can be useful. We’ll take the how-to guide for creating an EC2 web server and add resource hooks to health check the web server before allowing Pulumi to mark the deployment as complete. In this way, we know that when Pulumi is finished, the application is up and ready to go!

import * as aws from "@pulumi/aws"
import * as pulumi from "@pulumi/pulumi"

export = async () => {
 // Define the instance's user data script to set up a simple HTTP server that
 // serves some static content. We'll try to fetch the health.json in a
 // lifecycle hook later on to check whether or not the server is up and running.
 const userData = `#!/bin/bash
echo "Hello, World!" > index.html
echo '{"ok": true}' > health.json
nohup python -m SimpleHTTPServer 80 &`

 // Define a resource hook that will run after create and update and not return
 // until the health check passes.
 const afterHook = new pulumi.ResourceHook("after", async args => {
 // Since this is an after hook, we'll have access to the new outputs of the
 // resource.
 const outputs = args.newOutputs

 // Attempt to fetch health.json from the instance's public endpoint, backing
 // off linearly if it is not yet available.
 const maxRetries = 30
 for (let i = 0; i < maxRetries; i++) {
 try {
 const response = await fetch(`http://${outputs.publicDns}/health.json`)
 if (response.ok) {
 const data = await response.json()
 console.log(`Health check passed: ${JSON.stringify(data)}`)
 return
 }
 } catch (error) {
 console.log(`Health check attempt ${i + 1} failed`)
 }

 await new Promise(resolve => setTimeout(resolve, (i + 1) * 1000))
 }
 })

 // Set up the resources needed to run the web server, as outlined in the
 // how-to guide linked above.

 const instanceType = "t2.micro"
 const ami = aws.ec2.getAmiOutput({
 filters: [{
 name: "name",
 values: ["amzn2-ami-hvm-*"],
 }],
 // Amazon owns this AMI so we'll use their owner ID.
 owners: ["137112412989"],
 mostRecent: true,
 })

 const group = new aws.ec2.SecurityGroup("webserver-secgrp", {
 ingress: [
 { protocol: "tcp", fromPort: 22, toPort: 22, cidrBlocks: ["0.0.0.0/0"] },
 { protocol: "tcp", fromPort: 80, toPort: 80, cidrBlocks: ["0.0.0.0/0"] },
 ],
 })

 const server = new aws.ec2.Instance("webserver-www", {
 instanceType,
 vpcSecurityGroupIds: [group.id],
 ami: ami.id,
 userData,
 }, {
 hooks: {
 afterCreate: [afterHook],
 afterUpdate: [afterHook],
 },
 })

 return {
 publicDns: server.publicDns,
 publicIp: server.publicIp,
 }
}

import * as aws from "@pulumi/aws"
import * as pulumi from "@pulumi/pulumi"

export = async () => {
 // Define the instance's user data script to set up a simple HTTP server that
 // serves some static content. We'll try to fetch the health.json in a
 // lifecycle hook later on to check whether or not the server is up and running.
 const userData = `#!/bin/bash
echo "Hello, World!" > index.html
echo '{"ok": true}' > health.json
nohup python -m SimpleHTTPServer 80 &`

 // Define a resource hook that will run after create and update and not return
 // until the health check passes.
 const afterHook = new pulumi.ResourceHook("after", async args => {
 // Since this is an after hook, we'll have access to the new outputs of the
 // resource.
 const outputs = args.newOutputs as aws.ec2.InstanceState

 // Attempt to fetch health.json from the instance's public endpoint, backing
 // off linearly if it is not yet available.
 const maxRetries = 30
 for (let i = 0; i < maxRetries; i++) {
 try {
 const response = await fetch(`http://${outputs.publicDns}/health.json`)
 if (response.ok) {
 const data = await response.json()
 console.log(`Health check passed: ${JSON.stringify(data)}`)
 return
 }
 } catch (error) {
 console.log(`Health check attempt ${i + 1} failed`)
 }

 await new Promise(resolve => setTimeout(resolve, (i + 1) * 1000))
 }
 })

 // Set up the resources needed to run the web server, as outlined in the
 // how-to guide linked above.

 const instanceType = "t2.micro"
 const ami = aws.ec2.getAmiOutput({
 filters: [{
 name: "name",
 values: ["amzn2-ami-hvm-*"],
 }],
 // Amazon owns this AMI so we'll use their owner ID.
 owners: ["137112412989"],
 mostRecent: true,
 })

 const group = new aws.ec2.SecurityGroup("webserver-secgrp", {
 ingress: [
 { protocol: "tcp", fromPort: 22, toPort: 22, cidrBlocks: ["0.0.0.0/0"] },
 { protocol: "tcp", fromPort: 80, toPort: 80, cidrBlocks: ["0.0.0.0/0"] },
 ],
 })

 const server = new aws.ec2.Instance("webserver-www", {
 instanceType,
 vpcSecurityGroupIds: [group.id],
 ami: ami.id,
 userData,
 }, {
 hooks: {
 afterCreate: [afterHook],
 afterUpdate: [afterHook],
 },
 })

 return {
 publicDns: server.publicDns,
 publicIp: server.publicIp,
 }
}

import json
import time

import pulumi
import requests
from pulumi_aws import ec2

# Define the instance's user data script to set up a simple HTTP server that
# serves some static content. We'll try to fetch the health.json in a
# lifecycle hook later on to check whether or not the server is up and running.
user_data = """#!/bin/bash
echo "Hello, World!" > index.html
echo '{"ok": true}' > health.json
nohup python -m SimpleHTTPServer 80 &"""


# Define a resource hook that will run after create and update and not return
# until the health check passes.
def health_check(args: pulumi.ResourceHookArgs):
 # Since this is an after hook, we'll have access to the new outputs of the
 # resource.
 outputs = args.new_outputs

 # Attempt to fetch health.json from the instance's public endpoint, backing
 # off linearly if it is not yet available.
 max_retries = 30
 for i in range(max_retries):
 try:
 response = requests.get(
 f"http://{outputs['publicDns']}/health.json", timeout=10
 )
 if response.status_code == 200:
 data = response.json()
 print(f"Health check passed: {json.dumps(data)}")
 return
 except Exception as error:
 print(f"Health check attempt {i + 1} failed: {error}")

 # Linear backoff - wait (i + 1) seconds before next attempt
 time.sleep(i + 1)


instance_type = "t2.micro"

ami = ec2.get_ami_output(
 filters=[
 {
 "name": "name",
 "values": ["amzn2-ami-hvm-*"],
 }
 ],
 # Amazon owns this AMI so we'll use their owner ID.
 owners=["137112412989"],
 most_recent=True,
)

group = ec2.SecurityGroup(
 "webserver-secgrp",
 ingress=[
 {
 "protocol": "tcp",
 "from_port": 22,
 "to_port": 22,
 "cidr_blocks": ["0.0.0.0/0"],
 },
 {
 "protocol": "tcp",
 "from_port": 80,
 "to_port": 80,
 "cidr_blocks": ["0.0.0.0/0"],
 },
 ],
)

server = ec2.Instance(
 "webserver-www",
 instance_type=instance_type,
 vpc_security_group_ids=[group.id],
 ami=ami.id,
 user_data=user_data,
 opts=pulumi.ResourceOptions(
 hooks=pulumi.ResourceHookBinding(
 after_create=[health_check],
 after_update=[health_check],
 ),
 ),
)

package main

import (
 "encoding/json"
 "fmt"
 "net/http"
 "time"

 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

// Define a resource hook that will run after create and update and not return
// until the health check passes.
func healthCheck(args *pulumi.ResourceHookArgs) error {
 // Since this is an after hook, we'll have access to the new outputs of the
 // resource.
 publicDns := args.NewOutputs["publicDns"].StringValue()

 // Attempt to fetch health.json from the instance's public endpoint, backing
 // off linearly if it is not yet available.
 maxRetries := 30
 for i := 0; i < maxRetries; i++ {
 url := fmt.Sprintf("http://%s/health.json", publicDns)

 client := &http.Client{
 Timeout: 10 * time.Second,
 }

 resp, err := client.Get(url)
 if err == nil && resp.StatusCode == 200 {
 var data map[string]interface{}
 if err := json.NewDecoder(resp.Body).Decode(&data); err == nil {
 resp.Body.Close()
 dataJSON, _ := json.Marshal(data)
 fmt.Printf("Health check passed: %s\n", string(dataJSON))
 return nil
 }
 resp.Body.Close()
 }

 if resp != nil {
 resp.Body.Close()
 }

 fmt.Printf("Health check attempt %d failed: %v\n", i+1, err)

 // Linear backoff - wait (i + 1) seconds before next attempt
 time.Sleep(time.Duration(i+1) * time.Second)
 }

 return fmt.Errorf("health check failed after %d attempts", maxRetries)
}

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // Define the instance's user data script to set up a simple HTTP server
 // that serves some static content. We'll try to fetch the health.json
 // in a lifecycle hook later on to check whether or not the server is up
 // and running.
 userData := `#!/bin/bash
echo "Hello, World!" > index.html
echo '{"ok": true}' > health.json
nohup python -m SimpleHTTPServer 80 &`

 instanceType := "t2.micro"

 ami, err := ec2.LookupAmi(ctx, &ec2.LookupAmiArgs{
 Filters: []ec2.GetAmiFilter{
 {
 Name: "name",
 Values: []string{"amzn2-ami-hvm-*"},
 },
 },
 // Amazon owns this AMI so we'll use their owner ID.
 Owners: []string{"137112412989"},
 MostRecent: pulumi.BoolRef(true),
 })
 if err != nil {
 return err
 }

 group, err := ec2.NewSecurityGroup(ctx, "webserver-secgrp", &ec2.SecurityGroupArgs{
 Ingress: ec2.SecurityGroupIngressArray{
 &ec2.SecurityGroupIngressArgs{
 Protocol: pulumi.String("tcp"),
 FromPort: pulumi.Int(22),
 ToPort: pulumi.Int(22),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 &ec2.SecurityGroupIngressArgs{
 Protocol: pulumi.String("tcp"),
 FromPort: pulumi.Int(80),
 ToPort: pulumi.Int(80),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 },
 })
 if err != nil {
 return err
 }

 hook, err := ctx.RegisterResourceHook("health-check", healthCheck, nil)
 if err != nil {
 return err
 }

 server, err := ec2.NewInstance(ctx, "webserver-www", &ec2.InstanceArgs{
 InstanceType: pulumi.String(instanceType),
 VpcSecurityGroupIds: pulumi.StringArray{group.ID()},
 Ami: pulumi.String(ami.Id),
 UserData: pulumi.String(userData),
 }, pulumi.ResourceHooks(&pulumi.ResourceHookBinding{
 AfterCreate: []*pulumi.ResourceHook{hook},
 AfterUpdate: []*pulumi.ResourceHook{hook},
 }))
 if err != nil {
 return err
 }

 ctx.Export("publicDns", server.PublicDns)
 ctx.Export("publicIp", server.PublicIp)

 return nil
 })
}

using System;

using System.Threading.Tasks;
using Pulumi;

using Pulumi.Aws.Ec2;
using Pulumi.Aws.Ec2.Inputs;
using Pulumi.Command.Local;
using System.Diagnostics;
using System.Collections.Generic;

class Program
{
 static Task<int> Main() => Deployment.RunAsync<MyStack>();
}

class MyStack : Stack
{
 public MyStack()
 {
 // Define the instance's user data script to set up a simple HTTP server that
 // serves some static content. We'll try to fetch the health.json in a
 // lifecycle hook later on to check whether or not the server is up and running.
 var userData = @"#!/bin/bash
echo ""Hello, World!"" > index.html
echo '{""ok"": true}' > health.json
nohup python -m SimpleHTTPServer 80 &";

 // Define a resource hook that will run after create and update and not return
 // until the health check passes.
 var afterHook = new ResourceHook("after", async (args, cancellationToken) =>
 {
 // Since this is an after hook, we'll have access to the new outputs of the
 // resource.
 var outputs = args.NewOutputs

 // Attempt to fetch health.json from the instance's public endpoint, backing
 // off linearly if it is not yet available.
 const int maxRetries = 30;
 for (var i = 0; i < maxRetries; i++)
 {
 try
 {
 using var client = new HttpClient();
 var response = await client.GetAsync($"http://{outputs?["publicDns"]}/health.json");
 if (response.IsSuccessStatusCode)
 {
 var data = await response.Content.ReadAsStringAsync();
 Console.WriteLine($"Health check passed: {data}");
 return;
 }
 }
 catch (Exception)
 {
 Console.WriteLine($"Health check attempt {i + 1} failed");
 }

 await Task.Delay((i + 1) * 1000);
 }
 });

 // Set up the resources needed to run the web server, as outlined in the
 // how-to guide linked above.

 var instanceType = "t2.micro";
 var ami = GetAmi.Invoke(new GetAmiArgs
 {
 Filters =
 {
 new GetAmiFilterArgs
 {
 Name = "name",
 Values = { "amzn2-ami-hvm-*" },
 },
 },
 Owners = { "137112412989" }, // Amazon owns this AMI so we'll use their owner ID.
 MostRecent = true,
 });

 var group = new SecurityGroup("webserver-secgrp", new SecurityGroupArgs
 {
 Ingress =
 {
 new SecurityGroupIngressArgs
 {
 Protocol = "tcp",
 FromPort = 22,
 ToPort = 22,
 CidrBlocks = { "0.0.0.0/0" },
 },
 new SecurityGroupIngressArgs
 {
 Protocol = "tcp",
 FromPort = 80,
 ToPort = 80,
 CidrBlocks = { "0.0.0.0/0" },
 },
 },
 });

 var server = new Instance("webserver-www", new InstanceArgs
 {
 InstanceType = instanceType,
 VpcSecurityGroupIds = { group.Id },
 Ami = ami.Apply(a => a.Id),
 UserData = userData,
 }, new CustomResourceOptions
 {
 Hooks =
 {
 AfterCreate = { afterHook },
 AfterUpdate = { afterHook },
 }
 });

 // Export the public DNS and IP of the server.
 this.PublicDns = server.PublicDns;
 this.PublicIp = server.PublicIp;
 }

 [Output]
 public Output<string> PublicDns { get; private set; }

 [Output]
 public Output<string> PublicIp { get; private set; }
}

Thanks to our hooks, pulumi up will now wait until the web server is up and running before marking the deployment as complete. We can see this in action thanks to our logging output:

pulumi up

Updating (...)

View in Browser (Ctrl+O): https://app.pulumi.com/...

 Type Name Status Info
 + pulumi:pulumi:Stack resource-hooks created (79s) 6 messages
 + ├─ aws:ec2:SecurityGroup webserver-secgrp created (3s)
 + └─ aws:ec2:Instance webserver-www created (67s)

Diagnostics:
 pulumi:pulumi:Stack (resource-hooks):
 Health check attempt 1 failed
 Health check attempt 2 failed
 Health check attempt 3 failed
 Health check attempt 4 failed
 Health check attempt 5 failed
 Health check passed: {"ok":true}

Outputs:
 publicHostName: "ec2-XXX-YYY-ZZZ-WWW.us-west-2.compute.amazonaws.com"
 publicIp : "XXX.YYY.ZZZ.WWW"

Resources:
 + 3 created

Duration: 1m21s

And indeed, a command such as curl http://ec2-XXX-YYY-ZZZ-WWW.us-west-2.compute.amazonaws.com/health.json returns {"ok":true} immediately, without us having to wait for the web server to start up after the pulumi up command completes!

Where can we go from here?

Resource hooks are a powerful new feature that allow you to run custom code at any point in the resource lifecycle. This opens up a wide range of possibilities, such as:

Custom application health checks after creating or updating resources
Collecting logs or metrics when your Pulumi resources change
Running custom scripts such as database migrations as part of your Pulumi workflow.

Resource hooks require Pulumi to run your program. For refresh and destroy operations, this means you must use the --run-program flag for hooks to work. Read the full documentation on resource hooks to learn more.

Share any issues with your experience with us on GitHub, X, or our Community Slack.

The Pulumi Go Provider SDK is Now Generally Available

Eron Wright — Tue, 20 May 2025 00:00:00 +0000

At Pulumi, we are committed to accelerating your cloud infrastructure journey by eliminating barriers between your teams and the tools they need. Today, we’re thrilled to announce a game-changing milestone that puts unprecedented power in your hands: the v1.0.0 release of the Pulumi Go Provider SDK!

Unlock the Full Potential of Your Cloud Ecosystem

The Pulumi Go Provider SDK v1.0 is a transformative framework that enables you to integrate any service, tool, or API into your infrastructure as code workflows. After extensive development since 2022 and battle-testing with enterprise customers, this framework is now stable and production-ready for your most critical infrastructure needs.

Pulumi has worked diligently to bring this SDK to maturity, incorporating feedback from enterprise users and leveraging it internally to build production providers like the Docker Build Provider. With v1.0.0, building robust, multi-language Pulumi providers is more accessible than ever.

Key Enhancements for Your Team

The Pulumi Go Provider SDK delivers capabilities that benefit everyone from platform engineers to DevOps professionals:

Go 1.24 Support: Utilize the latest language features and performance improvements
Intuitive Provider Builder API: A fluent, developer-friendly API that eliminates boilerplate and accelerates provider development
Functional-Style Components: Create reusable infrastructure building blocks with a natural, functional programming style
Enterprise-Grade Testing Framework: Ensure reliability with comprehensive testing capabilities for both resources and components

For a complete technical overview, see our comprehensive Pulumi Go Provider SDK documentation.

Defining and Testing a Custom Resource

Creating a Custom Resource in Minutes, Not Days

Let’s see how the SDK transforms the provider development experience. Using the infer package, you define your resource types using straightforward Go structs:

func main() {
 p.RunProvider("greetings", "0.1.0", provider())
}

func provider() p.Provider {
 p, _ := infer.NewProviderBuilder().
 WithResources(
 infer.Resource(&HelloWorld{}),
 ).
 Build()
 return p
}

// Each resource has a controlling struct.
type HelloWorld struct{}

// Each resource has in input struct, defining what arguments it accepts.
type HelloWorldArgs struct {
 Name string `pulumi:"name"`
 Loud *bool `pulumi:"loud,optional"`
}

// Each resource has a state, describing the fields that exist on the created resource.
type HelloWorldState struct {
 HelloWorldArgs
 Message string `pulumi:"message"`
}

func (l *HelloWorldState) Annotate(a infer.Annotator) {
 a.Describe(&l, "A Hello World resource")
 a.Describe(&l.Name, "Who to say hello to.")
 a.SetDefault(&l.Name, "user-")
 a.Describe(&l.Loud, "Indicates whether to give a LOUD greeting.")
 a.Describe(&l.Message, "The greeting message.")
}

You then implement the necessary methods for the resource’s lifecycle with surprising simplicity:

// All resources must implement Create at a minimum.
func (HelloWorld) Create(
 ctx context.Context, name string, input HelloWorldArgs, preview bool,
) (string, HelloWorldState, error) {
 state := HelloWorldState{HelloWorldArgs: input}
 if preview {
 return name, state, nil
 }
 state.Message = fmt.Sprintf("Hello, %s", input.Name)
 if input.Loud != nil && *input.Loud {
 state.Message = strings.ToUpper(state.Message)
 }
 return name, state, nil
}

This minimal code creates a fully-functional resource that your teams can use in their infrastructure code! The SDK handles all the complex gRPC communication with the Pulumi engine and automatically generates the necessary schema for multi-language support.

For detailed step-by-step instructions on building your own provider, check out our Build a Provider guide.

Confidence Through Comprehensive Testing

One of the biggest challenges with custom infrastructure providers is ensuring they’re reliable. The SDK’s built-in testing framework solves this with targeted capabilities:

Lifecycle Testing: Verify your resources’ CRUD operations function correctly
Component Testing: Validate complex resource compositions with mock capabilities
Integration Testing: Ensure your provider works seamlessly with the Pulumi ecosystem

Here’s how easy it is to test your custom resource:

func TestHelloWorldLifecycle(t *testing.T) {
 server, err := integration.NewServer(t.Context(), "greetings", semver.Version{Minor: 1},
 integration.WithProvider(provider()),
 )
 require.NoError(t, err)

 // Test the lifecycle methods of the HelloWorld resource.
 integration.LifeCycleTest{
 Resource: "pkg:index:HelloWorld",
 Create: integration.Operation{
 Inputs: property.NewMap(map[string]property.Value{
 "name": property.New("tester"),
 "loud", property.New(true),
 }),
 Hook: func(inputs, output property.Map) {
 message := output.Get("message").AsString()
 assert.Equal(t, "Hello, tester", message)
 },
 },
 }.Run(t, server)
}

For more information, see Pulumi Go Provider SDK.

Standardize Infrastructure with Powerful Component Resources

Beyond simple resources, providers can also define Component Resources. Components are higher-level abstractions that encapsulate multiple child resources working together to implement a specific capability. They are perfect for codifying organizational standards and best practices into reusable (cross-language!) building blocks.

The Pulumi Go Provider SDK introduces support for defining components using a functional style, similar to how you might define a local component within a single program. This provides a streamlined way to build components. Here’s an example using the functional style:

func provider() p.Provider {
 p, _ := infer.NewProviderBuilder().
 WithComponents(
 infer.ComponentF(NewRandomLogin),
 ).
 Build()
 return p
}

type RandomLoginArgs struct {
 Prefix pulumi.StringInput `pulumi:"prefix"`
}

type RandomLogin struct {
 pulumi.ResourceState
 RandomLoginArgs

 Username pulumi.StringOutput `pulumi:"username"`
 Password pulumi.StringOutput `pulumi:"password"`
}

func NewRandomLogin(ctx *pulumi.Context, name string, args RandomLoginArgs, opts ...pulumi.ResourceOption) (*RandomLogin, error) {
 comp := &RandomLogin{}
 err := ctx.RegisterComponentResource(p.GetTypeToken(ctx), name, comp, opts...)
 if err != nil {
 return nil, err
 }

 username, err := random.NewRandomPet(ctx, name+"-username", &random.RandomPetArgs{
 Prefix: args.Prefix,
 }, pulumi.Parent(comp))
 if err != nil {
 return nil, err
 }
 comp.Username = username.ID().ToStringOutput()

 password, err := random.NewRandomPassword(ctx, name+"-password", &random.RandomPasswordArgs{
 Length: pulumi.Int(12),
 }, pulumi.Parent(comp))
 if err != nil {
 return nil, err
 }
 comp.Password = password.Result

 return comp, nil
}

func (l *RandomLogin) Annotate(a infer.Annotator) {
 a.Describe(&l, "Generate a random login credential (a username and password).")
 a.Describe(&l.Prefix, "An optional prefix for the generated username.")
 a.SetDefault(&l.Prefix, "user-")
}

For a deep dive into creating component resources, see our comprehensive Build a Component guide.

And testing components is equally straightforward:

func TestRandomLogin(t *testing.T) {
 provider, err := provider()
 require.NoError(t, err)
 server, err := integration.NewServer(t.Context(),
 "random-login",
 semver.Version{Minor: 1},
 integration.WithProvider(provider),
 integration.WithMocks(&integration.MockResourceMonitor{
 NewResourceF: func(args integration.MockResourceArgs) (string, property.Map, error) {
 switch {
 case args.TypeToken == "random:index/randomPet:RandomPet" && args.Name == "login-username":
 assert.Equal(t, "foo", args.Inputs.Get("prefix").AsString())
 return "user", property.Map{}, nil

 case args.TypeToken == "random:index/randomPassword:RandomPassword" && args.Name == "login-password":
 assert.Equal(t, 12.0, args.Inputs.Get("length").AsNumber())
 return args.Name, property.NewMap(map[string]property.Value{
 "result": property.New("12345").WithSecret(true),
 }), nil

 return "", property.Map{}, nil
 },
 }),
 )
 require.NoError(t, err)

 // test the "random-login:RandomLogin" component
 resp, err := server.Construct(p.ConstructRequest{
 Urn: "urn:pulumi:stack::project::random-login:index:RandomLogin::login",
 Inputs: property.NewMap(map[string]property.Value{
 "prefix": property.New("foo"),
 }),
 })
 require.NoError(t, err)

 require.Equal(t, property.NewMap(map[string]property.Value{
 "username": property.New("user"),
 "password": property.New("12345").WithSecret(true),
 }), resp.State)
}

For more information about component resources, see the blog post Announcing the Next-Generation of Pulumi Components: Enabling Infrastructure Abstractions.

A major advantage of using the Pulumi Go Provider SDK is that your provider, once written in Go, can be used in any Pulumi program, in any supported language (TypeScript, Python, Go, .NET, Java, or YAML). The SDK’s automatic schema generation takes care of creating the necessary language-specific SDKs, enabling cross-language consumption without manual effort.

Packaging and publishing your provider for others to use, whether within your organization or publicly, is a key step. Follow our Publishing Packages guide for detailed instructions on distributing your custom provider.

Understanding Provider Schema

The SDK automatically generates a schema for your provider based on the Go types and annotations you define. This schema describes the resources, their inputs and outputs, and documentation. For more details on how schema generation works and how to customize it, check out our Schema documentation.

Get Started Today!

The release of the Pulumi Go Provider SDK represents a significant step forward in simplifying and accelerating the development of Pulumi providers. By focusing on a code-first approach, automatic schema generation, and providing a robust testing framework, we are empowering engineers to integrate Pulumi with virtually any service or tool.

We encourage you to explore the Pulumi Go Provider SDK, build your own integrations, and contribute to the growing ecosystem of Pulumi providers. Start with our Extending Pulumi overview documentation to get a complete picture of the possibilities.

Explore our documentation on Extending Pulumi
Review examples in the GitHub repository
Learn how to build a provider with our step-by-step guide
Identify your first integration - what custom service would provide the most value to your team?
Join our community in the Pulumi Community Slack to learn from others building custom providers
Share your feedback with us on the in Slack or by opening an issue in the pulumi/pulumi-go-provider GitHub repository.

Happy building!

New in Pulumi IaC: Support for skipping a resource

Tom Harding — Wed, 14 May 2025 00:00:00 +0000

Managing large-scale infrastructure can be challenging, especially when you need to perform operations on specific subsets of your resources. Pulumi’s stack operations like pulumi up and pulumi destroy are powerful for deploying and tearing down environments, but sometimes you need more fine-grained control over which resources are affected.

Today, we’re excited to announce a highly requested feature that will save you time and reduce complexity in your workflows: the ability to exclude specific resources from stack operations using the new --exclude and --exclude-dependents flags.

These new flags complement the existing --target functionality, giving you powerful options whether you want to focus on a small subset of resources or exclude just a few from larger operations. No more workarounds or custom scripts to achieve selective deployments!

The challenge: partial operations on large stacks

When managing infrastructure at scale, you often want to operate on most—but not all—resources in your stack. For example:

Deploying all resources except a database that requires a maintenance window
Refreshing most resources while skipping those with known differences
Updating production infrastructure while leaving critical services untouched
Testing changes to most components while preserving test data in development

Pulumi already has the --target flag to specify which resources to include in an operation. This works well when you want to target a small number of resources, but becomes unwieldy when you want to operate on most of your stack while excluding only a few resources.

The solution: introducing `--exclude` and `--exclude-dependents`

Our new --exclude flag solves this problem by letting you specify which resources to omit from stack operations. When paired with --exclude-dependents, you can also exclude all child resources of the specified resources, making it easy to exclude entire branches of your resource tree.

These flags are now available for all major stack operations:

pulumi up --exclude <URN>::resource-to-skip
pulumi preview --exclude <URN>::resource-to-skip
pulumi refresh --exclude <URN>::resource-to-skip
pulumi destroy --exclude <URN>::resource-to-skip

Each of these commands can also use the --exclude-dependents flag to exclude child resources.

An example: selective deployment of blog content

Let’s imagine you’re managing a static blog website with Pulumi. As part of your deployment, you have multiple HTML pages you’d like to deploy:

...

for (const file in await glob('posts/**/*.html')) {
 new aws.s3.BucketObject(`post-${file}`, {
 source: new pulumi.asset.FileAsset(file),
 ...
 })
}

...

...

for (const file in await glob('posts/**/*.html')) {
 new aws.s3.BucketObject(`post-${file}`, {
 source: new pulumi.asset.FileAsset(file),
 ...
 })
}

...

...

for file in glob("posts/**/*.html"):
 aws.s3.BucketObject(file,
 source = pulumi.FileAsset(file),
 ....
 )

...

...

files, err := filepath.Glob("posts/**/*.html")
if err != nil { return err }

for _, file := range files {
 _, err := s3.NewBucketObject(ctx, file, &s3.BucketObjectArgs{
 Source: pulumi.FileAsset(file),
 ...
 })

 if err != nil { return err }
}

...

...

var files = Directory.GetFiles("posts", "*.html");

foreach (var file in files)
{
 var bucketObject = new BucketObjectv2(file, new BucketObjectv2Args
 {
 Source = new FileAsset(file),
 ...
 });
}

...

This works well, but what if we have a list of draft articles that we don’t want to include in the deployment? We can optimistically assume we’ve finished more articles than we’ve started, so using --target to specify every article, as well as supporting resources (CSS, JavaScript, ownership controls, et cetera), would quickly become unmanageable.

pulumi up --target <URN>::style.css --target <URN>::post-hello.html ...

With the --exclude flag, this becomes much more manageable:

pulumi up --exclude <URN>::post-draft-1.html --exclude <URN>::post-draft-2.html ...

With this command, everything not specified using an --exclude tag will be included in the up operation, and thus we can avoid the hassle of naming every resource that isn’t a draft.

Next step: a draft group

This is fine for a personal blog site, but can still become unmanageable when we’re dealing with multiple authors, each with multiple drafts. In this case, we might want to group our drafts under a common parent:

...

// A parent component for all drafts
const drafts = new pulumi.ComponentResource('ComponentResource', 'drafts')

for (const file in await glob('drafts/**/*.html')) {
 new aws.s3.BucketObject(`draft-${file}`, {
 source: new pulumi.asset.FileAsset(`drafts/${file}`),
 ...
 }, { parent: drafts })
}

...

...

// A parent component for all drafts
const drafts: pulumi.ComponentResource =
 new pulumi.ComponentResource('ComponentResource', 'drafts')

for (const file in await glob('drafts/**/*.html')) {
 new aws.s3.BucketObject(`draft-${file}`, {
 source: new pulumi.asset.FileAsset(`drafts/${file}`),
 ...
 }, { parent: drafts })
}

...

...

# A parent component for all drafts
drafts = pulumi.ComponentResource(t="ComponentResource", name="drafts")

for file_path in glob("posts/**/*.html"):
 aws.s3.BucketObject(file_path,
 source = pulumi.FileAsset(file_path),
 opts = pulumi.ResourceOptions(parent = drafts),
 ....
 )

...


drafts := &DraftGroupComponent{}
err = ctx.RegisterComponentResource("ComponentResource", "drafts", drafts)
if err != nil { return err }

...

files, err := filepath.Glob("posts/*.html")
if err != nil { return err }

for _, file := range files {
 _, err := s3.NewBucketObject(ctx, file, &s3.BucketObjectArgs{
 Key: pulumi.String(file),
 ...
}, pulumi.Parent(drafts))

 if err != nil { return err }
}

...


public class MyComponentResource : ComponentResource { ... }

...

var files = Directory.GetFiles("posts", "*.html");
var drafts = new MyComponentResource("drafts");

foreach (var file in files)
{
 var bucketObject = new BucketObjectv2(file, new BucketObjectv2Args
 {
 Source = new FileAsset(file),
 ...
 }, new CustomResourceOptions
 {
 Parent = drafts
 });
}

...

In this setup, we now have a parent resource for all drafts. Using --exclude-dependents, we can now exclude everything under this parent resource without having to enumerate all of them individually:

pulumi up --exclude <URN>::ComponentResource::drafts --exclude-dependents

This command will exclude all drafts from the up operation, regardless of how many we have or how they’re named. We now have a nice, scalable way to manage our drafts across production and development environments!

Next steps

With these flags now available in Pulumi CLI v3.158.0., expect to see them introduced in the automation APIs and GitHub action soon. Thanks for reading, and feel free to share any feedback on GitHub, X, or our Community Slack.

Pulumi Private Registry: The Source of Truth for Golden Paths

Pulumi IDP Team — Wed, 07 May 2025 00:00:00 +0000

As part of the Pulumi IDP announcement at PulumiUP, we introduced the Pulumi Private Registry. For years, we’ve worked with organizations that have built their own internal developer platforms on top of Pulumi. During that time, we identified what we believe is the best method for creating flexible golden paths – a bottom-up approach that utilizes a central source of truth to drive golden paths. Thanks to Pulumi Private Registry, this approach has never been simpler.

Codified Security and Compliance

Platform teams use Pulumi’s components and templates to standardize how resources and programs are shaped and provisioned. They use Pulumi ESC to automatically import secret dependencies and lease short-term credentials. They incorporate Pulumi CrossGuard policy as code to ensure that provisioned infrastructure remains compliant even when customized by developers. Publishing and consuming packages through the private registry guarantees platform teams can achieve the consistency, security, and compliance their business needs without sacrificing the flexibility their developers need.

Streamlined Publishing

The CLI publishing workflow prioritizes ergonomics, making it easy to use. Publishing a package is as simple as running a single CLI command, ensuring seamless integration in local or automated workflows. It supports publishing packages from public or private git repositories, package versioning, multiple organizations, and custom README paths.

Simplified Discovery

Once published, developers can discover standardized and compliant components and templates from a developer-friendly, centralized location – helping teams streamline infrastructure provisioning and avoid shadow workflows. Developers also get immediate context thanks to automatically generated API docs and READMEs.

Free Day 2 Context

Day 2 operations – maintaining, extending, and decommissioning infrastructure – comprise the bulk of the infrastructure lifecycle effort. Yet these tasks often feel like flying blind. Pulumi IDP takes a bottom-up approach, driving building block publishing, discovery, and consumption through the Private Registry. This also provides usage insights out of the box, enabling platform engineers to see where components are used and which versions are in play, making it easier to plan upgrades and safely decommission outdated packages.

Get Started Today

Pulumi IDP, including the Pulumi Private Registry, is now available in Public Preview for all users. Sign up for a free account to get started, learn more about Pulumi IDP, or start publishing packages to your private registry.

Announcing Pulumi IDP: Platform Engineering Accelerated

Pulumi IDP Team — Tue, 06 May 2025 00:00:00 +0000

Today, we’re excited to introduce Pulumi IDP (Internal Developer Platform), the latest evolution of the Pulumi Cloud Platform, designed to help organizations automate, secure, and manage everything they run in the cloud.

Get Started with Pulumi IDP

For the past eight years, we’ve helped organizations simplify the deployment and management of their infrastructure. Pulumi launched at the height of DevOps, bringing general-purpose programming languages to infrastructure as code (IaC) at a time when application teams were often manually deploying their own infrastructure. Users were instantly hooked on the ability to leverage familiar programming paradigms to manage their infrastructure.

As we evolved, we introduced additional products to meet the needs of an expanding user base. We launched Pulumi Components to increase standardization and reuse across any programming language. We launched Pulumi CrossGuard to codify policy as code and ensure compliance at scale. And last fall, we made Pulumi ESC and Pulumi Insights generally available to help tame secret sprawl and better manage all your cloud infrastructure.

At the same time that Pulumi was evolving, so too was the industry. DevOps ushered in a new way of shipping and maintaining software, but for many organizations, it brought about tool sprawl and Day 2 pain. These lessons learned, along with others, have led to the emergence of platform engineering and platform teams, which build tools and workflows that enable their internal users to provision infrastructure and deploy software.

Platform Teams

The goal and promise of platform teams are to understand the needs of their users, creating scalable workflows to provision and maintain secure, compliant, and reliable infrastructure through Internal Developer Platforms without being blocked by other teams or having to maintain it themselves.

However, Platform teams have faced a harsh reality – offering flexible golden paths that account for security, compliance, and maintainability is inherently difficult. Most platform teams – especially in enterprises – must also build for multiple personas with varying consumption methods, from writing their own IaC code to point-and-click deployments.

Without establishing a foundation of standardized, reusable primitives that can be consumed in multiple ways, platform teams will stay trapped in a cycle of trying to build everything for everyone.

Pulumi IDP

Pulumi IDP is a bottom-up framework that integrates with the entire Pulumi product suite, enabling platform teams to build and manage self-service workflows based on codified golden paths. We have launched a suite of new features and invested in existing ones to make the Day 0 through Day 2 experience of operating infrastructure seamless.

Day 0: Establish a source of truth for standardized infrastructure building blocks that can be consumed in any workflow.
Day 1: Provision infrastructure through flexible workflows based on standardized building blocks.
Day 2: Establish context, gain insights, and perform operational tasks.

Private Registry as the Source of Truth

The foundation of Pulumi IDP is the Pulumi Private Registry, the source of truth for private components, templates, providers, and policies.

Pulumi has long focused on enabling organizations to author reusable infrastructure building blocks. We recently introduced the latest generation of Pulumi components for encapsulating and reusing resources across programming languages. Reusable packages offer tremendous benefits, but significant pain points have also emerged:

Discoverability: Once authored, teams often store packages in Git, but in most organizations, especially enterprises, they quickly become invisible. If no one can find packages, they won’t reuse them, and teams will write one-off solutions instead.
Lifecycle Management: Organizations that are able to drive discovery and adoption encounter Day 2 issues, such as a lack of usage insights, performing updates, and more.

The Pulumi Private Registry was built to solve these issues and more. With a single pulumi publish command, platform engineers can ensure their standardized building blocks are discoverable from a central location. Their end-users can discover the packages available, explore README files, consume automatically generated API documentation, and learn about installing and using the package.

Driving standardized component and template utilization through the registry also provides valuable usage insights for lifecycle management. Platform engineers can see where each package or template is used and at which version – critical for assessing the impact of changes and tracking version drift.

Flexible Self-Service Workflows

To ensure that users can consume packages and templates ergonomically, we’re introducing new flexible workflow tools to meet the needs of any persona.

Platform teams can exhaust themselves with designing, codifying, and publishing standardized patterns, but if they are not easily consumable, they won’t be used. Platform teams are often tasked with accommodating a spectrum of personas, ranging from users who want to scaffold infrastructure through components to non-technical users who want to point and click to deploy from a template. Each of their needs is valid, but the adage “designing for everyone means designing for no one” can quickly take effect.

With Pulumi IDP’s bottom-up approach, this isn’t an issue. The same standardized, secure, and compliant packages and templates published in a private registry are the same ones that power self-service workflows.

No-Code

In no-code workflows, users can instantly create and deploy Pulumi programs without worrying about the underlying IaC code or stack configuration. The no-code workflow is built on Pulumi organization templates and stores stack config in Pulumi ESC, eliminating the need to write to a VCS like GitHub or GitLab. By storing config in Pulumi ESC, developers can also edit config on-demand and instantly redeploy.

Low-Code

Users often need to compose their own Pulumi programs, but platform engineering teams want to ensure they use approved components. With the new Pulumi Private registry and the latest generation of Pulumi components, that’s never been easier.

Platform engineers can author and publish components in any language. Users can then discover the available components through the private registry, add them as dependencies in a Pulumi YAML program, and deploy the infrastructure.

Full-Code

In some organizations, platform engineers want to offer application teams a starting point without dictating how they deploy infrastructure. Templates are now available from the Pulumi CLI to facilitate a local workflow. Application developers can scaffold infrastructure using the same templates available from the private registry directly from the convenience of their terminal.

For instances where a Pulumi program is written from scratch, the suite of Pulumi IaC languages is always nearby, with the ability to easily incorporate components to accelerate development.

Organizational Context through Services

After you define and deploy infrastructure, you need to manage it, and that’s easier when it reflects your organization’s context. To make this possible, we’re introducing Pulumi Services.

Services enable teams to logically group stacks, environments, and resources, creating representations that are familiar within their organization. The service’s entities can live anywhere in Pulumi – across IaC and ESC projects or in different Insights Accounts.

Services also support properties – metadata that adds important context to a service, such as links to observability dashboards, Slack channels, and more.

A New Era for Platform Teams

Ready to accelerate your platform engineering journey? Get started with Pulumi IDP today and explore our comprehensive documentation to learn how to build self-service infrastructure workflows, establish golden paths, and boost developer productivity. Join the growing community of platform engineers who are transforming how their organizations deliver and manage cloud infrastructure at scale.

Azure Native V3: A Leaner, Faster SDK for Microsoft Azure

Thomas Kappler — Thu, 03 Apr 2025 00:00:00 +0000

Pulumi Azure Native V3 is the most comprehensive infrastructure as code (IaC) solution for Microsoft Azure, combining full resource coverage with dramatic improvements in performance and developer experience.

With V3, the Pulumi Azure Native SDK is 75% smaller while maintaining 100% coverage of the Azure Resource Manager (ARM) API. This makes it the most efficient and complete way to define and manage Azure infrastructure with real programming languages.

Pulumi Azure Native provides direct access to the Azure Resource Manager API, making it hands-down the best infrastructure as code solution for Microsoft Azure. Unlike traditional providers that abstract away cloud APIs, Azure Native gives you more resource and property coverage compared to Terraform by directly mapping to Azure’s native REST API. This means:

Complete API coverage: Access every Azure resource available in ARM
Same-day updates: New Azure services and properties are available immediately
Advanced property support: Work with every resource property, not just a subset
Latest API versions: Always up to date with the newest stable Azure APIs

And with V3’s slimmer SDK footprint, developers benefit from faster IDE performance, reduced memory usage, quicker installs, and improved CI/CD pipeline efficiency.

These improvements came about through our RFC process, where we explored various options to balance SDK size with the flexibility needed by advanced users.

A Slimmer, More Efficient SDK

The most significant improvement in Azure Native V3 is the dramatic 75% reduction in SDK size across all languages. By optimizing the SDK to include only the default API versions, we’ve solved one of the main pain points reported by our users, the size of the package:

Language	Before	After	Reduction
.NET	77 MB	15 MB	80.5%
JS/TS	494 MB	87 MB	82.4%
Python	104.6 MB	25 MB	76.3%
Go (source)	910 MB	167 MB	81.6%
Java (explicit versions were not supported in v2)	68 MB	71 MB	+4.4%
YAML	no SDK	no SDK	-

This transformation delivers immediate benefits that every developer will notice: faster IDE performance with reduced load times, more responsive language servers for better autocompletion, reduced memory consumption, significantly quicker package installation, and improved CI/CD pipeline performance.

Additional Enhancements in Azure Native V3

Azure Native V3 introduces more than just SDK size optimization. Other improvements include:

Reorganized module structure better aligned with Azure SDK conventions
Improved resource naming for clearer distinction between similar resources
Enhanced handling of nested properties to avoid accidental overwriting
Better state management with the removal of unnecessary __inputs fields (most noticeable with large stacks)

Understanding Azure Versioning

The Pulumi Azure Native provider needs to manage two different versioning systems:

Pulumi’s Semantic Versioning: Our SDKs follow the traditional major.minor.patch format (e.g., 3.0.0)
- Patch versions (3.0.1) contain bug fixes only
- Minor versions (3.1.0) add new resources and non-breaking features
- Major versions (3.0.0) may contain breaking changes
Azure’s Date-based API Versions: Azure resources use date-based versions like “2024-01-01” or “2023-05-15-preview”
- Each Azure service (resource provider) has its own set of API versions
- Breaking changes can occur between different API versions
- Preview versions (with “-preview” suffix) contain newer features that may not be stable yet

Default API Versions

Azure Native V3 includes refreshed default API versions across nearly all resources, ensuring you’re automatically working with the latest stable Azure features and capabilities without any extra configuration. The new default versions unlock powerful new Azure functionality out of the box, such as:

EventGrid (now 2025-02-15): Support for advanced filtering, dead-lettering, and custom event schemas
MachineLearningServices (now 2024-10-01): Enhanced compute instance capabilities and new model deployment options
Storage (now 2024-01-01): Improved performance tiers and expanded security features
KeyVault (now 2024-03-01): Enhanced certificate management and expanded access policy options

Using Non-Default API Versions

Most users (approximately 95% based on our Pulumi Cloud data) will be perfectly served by these default API versions. However, there are situations where you might need to use a specific API version.

For these cases, Azure Native V3 provides two flexible approaches:

Option 1: Accessing Explicit API Versions via Local Packages

The most type-safe way to use explicit API versions is through local packages. This approach gives you the full SDK experience with strong typing, auto-completion, and documentation.

Add the specific API version package

Use the Pulumi CLI to add just the packages and versions you need:

pulumi package add azure-native storage v20240101

For more details on the package add command, see the CLI documentation.

Step 2: Import and use the versioned package

Once added, you can import and use the specific API version in your code:

import * as resources from "@pulumi/azure-native/resources";
import * as storage_v20240101 from "@pulumi/azure-native_storage_v20240101";

const resourceGroup = new resources.ResourceGroup("resourceGroup");

const storageAccount = new storage_v20240101.storage.StorageAccount("sa", {
 resourceGroupName: resourceGroup.name,
 sku: {
 name: storage_v20240101.storage.SkuName.Standard_LRS,
 },
 kind: storage_v20240101.storage.Kind.StorageV2,
});

import pulumi
from pulumi_azure_native import resources
import pulumi_azure_native_storage_v20240101 as storage_v20240101

resource_group = resources.ResourceGroup("resource_group")

storage_account = storage_v20240101.storage.StorageAccount("sa",
 resource_group_name=resource_group.name,
 sku=storage_v20240101.storage.SkuArgs(
 name=storage_v20240101.storage.SkuName.STANDARD_LRS,
 ),
 kind=storage_v20240101.storage.Kind.STORAGE_V2)

using Pulumi;
using Pulumi.AzureNative.Resources;
using Storage_v20240101 = Pulumi.AzureNative_Storage_v20240101;

class MyStack : Stack
{
 public MyStack()
 {
 var resourceGroup = new ResourceGroup("resourceGroup");

 var storageAccount = new Storage_v20240101.Storage.StorageAccount("sa", new()
 {
 ResourceGroupName = resourceGroup.Name,
 Sku = new Storage_v20240101.Storage.SkuArgs
 {
 Name = Storage_v20240101.Storage.SkuName.Standard_LRS,
 },
 Kind = Storage_v20240101.Storage.Kind.StorageV2,
 });
 }
}

package main

import (
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
 "github.com/pulumi/pulumi-azure-native/sdk/go/azure/resources"
 storage_v20240101 "github.com/pulumi-azure-native-storage-v20240101/sdk/go/azure/storage"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 resourceGroup, err := resources.NewResourceGroup(ctx, "resourceGroup", nil)
 if err != nil {
 return err
 }

 storageAccount, err := storage_v20240101.NewStorageAccount(ctx, "sa", &storage_v20240101.StorageAccountArgs{
 ResourceGroupName: resourceGroup.Name,
 Sku: &storage_v20240101.SkuArgs{
 Name: storage_v20240101.SkuNameStandard_LRS,
 },
 Kind: storage_v20240101.KindStorageV2,
 })
 if err != nil {
 return err
 }

 return nil
 })
}

name: azure-native-example
runtime: yaml
description: A minimal Azure Native Pulumi YAML program
resources:
 resourceGroup:
 type: azure-native:resources:ResourceGroup
 properties:
 location: westus2

 # Using specific API version package
 storageAccount:
 type: azure-native_storage_v20240101:storage:StorageAccount
 properties:
 resourceGroupName: ${resourceGroup.name}
 sku:
 name: Standard_LRS
 kind: StorageV2

package myproject;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.azurenative.resources.ResourceGroup;
import com.pulumi.azurenative.resources.ResourceGroupArgs;
import com.pulumi.azurenative_storage_v20240101.storage.StorageAccount;
import com.pulumi.azurenative_storage_v20240101.storage.StorageAccountArgs;
import com.pulumi.azurenative_storage_v20240101.storage.enums.Kind;
import com.pulumi.azurenative_storage_v20240101.storage.enums.SkuName;
import com.pulumi.azurenative_storage_v20240101.storage.inputs.SkuArgs;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 private static void stack(Context ctx) {
 var resourceGroup = new ResourceGroup("resourceGroup");

 var storageAccount = new StorageAccount("sa", StorageAccountArgs.builder()
 .resourceGroupName(resourceGroup.name())
 .sku(SkuArgs.builder()
 .name(SkuName.Standard_LRS)
 .build())
 .kind(Kind.StorageV2)
 .build());
 }
}

Option 2: Using the Generic Resource

An alternative approach is to use the generic azure-native.resources.Resource. This special resource allows you to access any Azure resource at any API version, including older versions that might not be available through local packages.

While not as type-safe as the local package approach, the generic resource offers maximum flexibility and is useful when you need to access a resource that is not yet supported by the Pulumi provider or when an issue prevents you from using a specific resource.

In most cases, we recommend using the local packages approach when you need a specific API version, and only falling back to the generic resource for special cases or resources not yet supported by the Pulumi provider.

For more details, see the Azure Native version guide.

Getting Started with Azure Native V3

If You’re Already Using Pulumi Azure Native

Upgrading ensures you benefit from the smaller SDK, refreshed defaults, and the best performance when managing Azure at scale.

Upgrading from v2: Follow the v2 to v3 migration guide.
Upgrading from v1: First move to v2 using our v1 to v2 upgrade guide, the follow the v2 to v3 migration guide. This two-step process resolves deprecation warnings and ensures a smoother migration.

For language-specific installation instructions, see our Azure Native setup guide.

If You’re New to Pulumi

Pulumi makes it easy to define, deploy, and manage Azure infrastructure using your favorite programming languages. To get started:

Sign up for Pulumi Cloud - free for individuals and teams.
Explore the Azure Native installation guide.
Try a quickstart project to provision real Azure resources in minutes.

Conclusion

Pulumi Azure Native V3 is the most complete and efficient solution for managing Azure infrastructure as code. With a dramatically smaller SDK, updated defaults, flexible API versioning, and enhanced developer experience, it offers the best combination of performance, coverage, and control for Microsoft Azure users.

We’d like to thank our community for their valuable feedback throughout the development of V3. Your input helped shape this release, and we’re excited to see what you’ll build with it.

Have questions or suggestions? Open an issue on GitHub, join our Community Slack.

Improved refresh and destroy experience for Pulumi IaC

Will Jones — Wed, 02 Apr 2025 00:00:00 +0000

Pulumi enables teams to manage their infrastructure using the programming languages and tools they are already familiar with, supporting use cases such as complex authentication workflows, dynamically configured resources, and more.

In this post we’re excited to announce an improvement to the pulumi refresh and pulumi destroy commands: the --run-program flag! This new feature makes Pulumi even more powerful for teams with complex infrastructure workflows.

This enhancement is particularly valuable for teams working with short-lived credentials, dynamic resources, or any workflow where your code needs to run to establish the right context. Whether you’re using OIDC-based authentication, dynamically fetching credentials from a secrets manager, or working with dynamic providers, the --run-program flag ensures your infrastructure operations have the context they need to succeed.

The pulumi refresh and pulumi destroy commands sit alongside pulumi up and support refreshing and destroying resources in your stack. While refresh and destroy take into account updated configuration (such as that specified in your Pulumi.<stack>.yaml), they have historically not taken into account changes in your code. For stacks which depend on code to update credentials for a provider, or to determine which resources to create, this can break or hamper the use of refresh and destroy. Well, no more! As of the latest release of Pulumi (v3.160.0), the pulumi refresh and pulumi destroy commands now support the --run-program flag, which allows you to run your program before refreshing or destroying your stack.

We plan to make running your program the default behavior for refresh and destroy operations eventually. We’re introducing this as an opt-in flag first to ensure a smooth transition. For more details and to provide feedback, please see our RFC on making program execution the default behavior.

Let’s see it in action

We are writing a Pulumi program to deploy infrastructure to AWS. Our organization’s platform team provides us with a library that we can use to dynamically fetch appropriate AWS credentials for our stack. We use this to retrieve some values and set up an explicit AWS provider (1, 2, 3):

import * as orgConfig from "@org/config";
import * as aws from "@pulumi/aws";

export = async () => {
 const awsConfig = await orgConfig.getAWSConfig({
 project: pulumi.getProject(),
 stack: pulumi.getStack(),
 });

 const provider = new aws.Provider("provider", {
 accessKey: awsConfig.accessKey,
 secretKey: awsConfig.secretKey,
 token: awsConfig.securityToken,
 region: awsConfig.region,
 });

 const bucket = new aws.s3.Bucket("bucket", { ... }, { provider });
};

import * as orgConfig from "@org/config";
import * as aws from "@pulumi/aws";

export = async () => {
 const awsConfig = await orgConfig.getAWSConfig({
 project: pulumi.getProject(),
 stack: pulumi.getStack(),
 });

 const provider = new aws.Provider("provider", {
 accessKey: awsConfig.accessKey,
 secretKey: awsConfig.secretKey,
 token: awsConfig.securityToken,
 region: awsConfig.region,
 });

 const bucket = new aws.s3.Bucket("bucket", { ... }, { provider });
};

import org_config
import pulumi
import pulumi_aws as aws

aws_config = org_config.get_aws_config(
 project=pulumi.get_project(),
 stack=pulumi.get_stack(),
)

provider = aws.Provider("provider",
 access_key=aws_config.access_key,
 secret_key=aws_config.secret_key,
 token=aws_config.security_token,
 region=aws_config.region,
)

bucket = aws.s3.Bucket("bucket", ... , opts=pulumi.ResourceOptions(provider=provider))

package main

import (
 "github.com/org/config"
 "github.com/pulumi/pulumi-aws/sdk/v7/go/aws/s3"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 awsConfig, err := config.GetAWSConfig(ctx)
 if err != nil {
 return err
 }

 provider, err := aws.NewProvider(ctx, "provider", &s3.ProviderArgs{
 AccessKey: awsConfig.AccessKey,
 SecretKey: awsConfig.SecretKey,
 Token: awsConfig.SecurityToken,
 Region: awsConfig.Region,
 })
 if err != nil {
 return err
 }

 bucket, err := s3.NewBucket(ctx, "bucket", &s3.BucketArgs{ ... }, pulumi.Provider(provider))
 if err != nil {
 return err
 }

 return nil
 })
}

package app;

import com.org.AwsConfig;
import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.aws.Provider;
import com.pulumi.aws.ProviderArgs;
import com.pulumi.aws.s3.Bucket;
import com.pulumi.aws.s3.BucketArgs;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 final var awsConfig = AwsConfig.getAWSConfig(ctx.projectName(), ctx.stackName());

 final var provider = new Provider("provider", ProviderArgs.builder()
 .accessKey(awsConfig.accessKey())
 .secretKey(awsConfig.secretKey())
 .token(awsConfig.securityToken())
 .region(awsConfig.region())
 .build());

 final var bucket = new Bucket("bucket", BucketArgs.builder().....build(),
 pulumi.CustomResourceOptions.builder().provider(provider).build());
 }
}

using System.Threading.Tasks;

using Org.Config;
using Pulumi;

class Program
{
 static Task<int> Main() => Deployment.RunAsync<MyStack>();
}

class MyStack : Stack
{
 public MyStack()
 {
 var awsConfig = GetAwsConfig(
 project: Deployment.Instance.ProjectName,
 stack: Deployment.Instance.StackName
 );

 var provider = new Pulumi.Aws.Provider("provider", new Pulumi.Aws.ProviderArgs
 {
 AccessKey = awsConfig.AccessKey,
 SecretKey = awsConfig.SecretKey,
 Token = awsConfig.SecurityToken,
 Region = awsConfig.Region,
 });

 var bucket = new Pulumi.Aws.S3.Bucket("bucket", new Pulumi.Aws.S3.BucketArgs {
 ...
 }, new Pulumi.ResourceOptions {
 Provider = provider
 });
 }
}

We run pulumi up and everything works as we expect – our program grabs the appropriate tokens and sets up our S3 bucket.

A few days, weeks, months, or even years later, we find that some tweaks have been made to the bucket in the AWS console. Before we update and re-run our program, we want to refresh Pulumi’s state so that it picks up the current properties of the bucket. We run pulumi refresh and, after waiting a while… it fails! What has gone wrong?

The issue is that our program needs to run to fetch the latest credentials using our platform team’s library. However, pulumi refresh doesn’t run our program by default. To fix this, we can use the new --run-program flag:

pulumi refresh --run-program

This time, Pulumi will run our program before refreshing the state of our stack. As a result, our program will fetch the latest credentials using the library provided by our platform team, and everything will work as expected! If in a few months we need to clean up our stack, pulumi destroy also accepts the new option:

pulumi destroy --run-program

What’s next?

Running the program for all Pulumi operations paves the way for several other highly-requested features. Top of our list is lifecycle hooks – the ability to run arbitrary program code at various points in the lifecycle of a Pulumi resource.

As mentioned earlier, we plan to make the behavior of --run-program the default in a future release. This means that Pulumi will automatically run your program before refresh and destroy operations, without requiring the flag. For more details and to provide feedback, please see our RFC on making program execution the default behavior for refresh and destroy.

Share any issues with your experience with us on GitHub, X, or our Community Slack.

Announcing the Next-Generation of Pulumi Components: Enabling Infrastructure Abstractions

Mikhail Shilkov — Thu, 27 Mar 2025 00:00:00 +0000

Pulumi Components enable you to create, share, and consume reusable infrastructure building blocks across your organization and the broader community. Today, we’re excited to announce significant enhancements to Pulumi Components that make them more powerful, accessible, and easier to use than ever before.

With this release, we’ve made it possible to author components in one language and consume them in any other Pulumi language—including Pulumi YAML. This breakthrough enables platform teams to build sophisticated infrastructure abstractions in their preferred programming language while application developers can easily consume these components using simpler interfaces or even YAML, all without sacrificing type safety or functionality.

For platform teams, the new Pulumi Components offer a more intuitive, more secure way to codify organizational standards. For developers, it’s a straightforward way to pull in platform-approved abstractions without having to worry about all the underlying details. It’s a win-win that drastically accelerates innovation and drives productivity across your teams.

Understanding Pulumi Components

At their core, Pulumi Components are logical groupings of resources that encapsulate infrastructure patterns and best practices. Unlike standard resources that map directly to cloud provider resources, components are higher-level abstractions that can contain multiple child resources working together to implement a specific capability. While similar in concept to Terraform modules, Pulumi Components offer more power through full programming language capabilities, stronger typing, and now, cross-language support.

For example, a SecureBucket component might include a bucket, versioning configuration, encryption settings, and tagging policies—all bundled together as a single, reusable unit that enforces your organization’s security and compliance standards.

Pulumi has always supported creating components within a single language - you can simply create a class that extends ComponentResource in your language of choice and use it within programs written in that same language. This approach works well for teams working within a single language ecosystem, requiring no additional setup beyond standard programming patterns. However, these traditional single-language components can’t be consumed from other Pulumi languages, limiting their shareability across teams with different language preferences.

What’s New with Pulumi Components

Our latest enhancements focus on making components more accessible and easier to share:

Cross-language consumption: Components authored in one language can now be consumed in any Pulumi language, now including Pulumi YAML.
Simplified sharing: Share components by simply pushing code to a git repository - no need to manually publish SDKs.
Streamlined consumption: Use the pulumi package add command with any git URL (like github.com/org/repo) to easily incorporate any components into your projects.
YAML authoring support: For the first time, you can now author components directly in YAML, making it easier for teams with limited programming experience to create and maintain reusable infrastructure patterns.

How It Works

Creating Components

The code for creating a cross-language component starts similarly to a single-language component: define a class that extends ComponentResource:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

export interface SecureBucketArgs {
 bucketName?: pulumi.Input<string>;
 versioning?: pulumi.Input<boolean>;
 encryption?: pulumi.Input<boolean>;
 tags?: { [key: string]: pulumi.Input<string> };
}

export class SecureBucket extends pulumi.ComponentResource {
 public readonly bucket: aws.s3.BucketV2;
 public readonly bucketName: pulumi.Output<string>;

 constructor(name: string, args: SecureBucketArgs = {}, opts?: pulumi.ComponentResourceOptions) {
 super("mycomponents:index:SecureBucket", name, {}, opts);

 // Create an S3 bucket with best practices by default
 this.bucket = new aws.s3.BucketV2(`${name}`, {
 bucket: args.bucketName,
 tags: {
 ManagedBy: "Pulumi",
 ...args.tags,
 },
 }, { parent: this });

 // Conditionally enable versioning
 if (args.versioning !== false) {
 new aws.s3.BucketVersioningV2(`${name}-versioning`, {
 bucket: this.bucket.id,
 versioningConfiguration: {
 status: "Enabled",
 },
 }, { parent: this });
 }

 // Conditionally enable encryption
 if (args.encryption !== false) {
 new aws.s3.BucketServerSideEncryptionConfigurationV2(`${name}-encryption`, {
 bucket: this.bucket.id,
 rules: [{
 applyServerSideEncryptionByDefault: {
 sseAlgorithm: "AES256",
 },
 }],
 }, { parent: this });
 }

 this.bucketName = this.bucket.id;

 this.registerOutputs({
 bucket: this.bucket,
 bucketName: this.bucketName,
 });
 }
}

import pulumi
import pulumi_aws as aws
from typing import Optional, Dict, Any, Mapping

class SecureBucketArgs(TypedDict):
 bucket_name: Optional[pulumi.Input[str]]
 versioning: Optional[bool]
 encryption: Optional[bool]
 tags: Optional[Mapping[str, pulumi.Input[str]]]

class SecureBucket(pulumi.ComponentResource):
 bucket_name: pulumi.Output[str]
 def __init__(self, name: str, args: SecureBucketArgs, opts: Optional[pulumi.ResourceOptions] = None):
 super().__init__('mycomponents:index:SecureBucket', name, {}, opts)

 args = args or SecureBucketArgs()

 # Create an S3 bucket with best practices by default
 bucket = aws.s3.BucketV2(
 f"{name}",
 bucket=args.get("bucket_name"),
 tags={
 "ManagedBy": "Pulumi",
 **args.get("tags", {})
 },
 opts=pulumi.ResourceOptions(parent=self)
 )

 # Conditionally enable versioning
 if args.get("versioning", True):
 aws.s3.BucketVersioningV2(
 f"{name}-versioning",
 bucket=bucket.id,
 versioning_configuration=aws.s3.BucketVersioningV2VersioningConfigurationArgs(
 status="Enabled"
 ),
 opts=pulumi.ResourceOptions(parent=self)
 )

 # Conditionally enable encryption
 if args.get("encryption", True):
 aws.s3.BucketServerSideEncryptionConfigurationV2(
 f"{name}-encryption",
 bucket=bucket.id,
 rules=[aws.s3.BucketServerSideEncryptionConfigurationV2RuleArgs(
 apply_server_side_encryption_by_default=aws.s3.BucketServerSideEncryptionConfigurationV2RuleApplyServerSideEncryptionByDefaultArgs(
 sse_algorithm="AES256"
 )
 )],
 opts=pulumi.ResourceOptions(parent=self)
 )

 self.bucket_name = bucket.id

 self.register_outputs({
 "bucketName": self.bucket_name
 })

using System.Collections.Generic;
using Pulumi;
using Pulumi.Aws.S3;
using Pulumi.Aws.S3.Inputs;

namespace MyComponents
{
 public class SecureBucketArgs : ResourceArgs
 {
 [Input("bucketName")]
 public Input<string>? BucketName { get; set; }

 [Input("versioning")]
 public bool? Versioning { get; set; }

 [Input("encryption")]
 public bool? Encryption { get; set; }

 [Input("tags")]
 public Dictionary<string, Input<string>>? Tags { get; set; }
 }

 public class SecureBucket : ComponentResource
 {
 [Output("bucketName")]
 public Output<string> BucketName { get; private set; }

 public SecureBucket(string name, SecureBucketArgs? args, ComponentResourceOptions? opts = null)
 : base("mycomponents:index:SecureBucket", name, opts)
 {
 args ??= new SecureBucketArgs();

 // Create an S3 bucket with best practices by default
 var tags = new InputMap<string>
 {
 { "ManagedBy", "Pulumi" }
 };

 if (args.Tags != null)
 {
 foreach (var tag in args.Tags)
 {
 tags[tag.Key] = tag.Value;
 }
 }

 var bucket = new BucketV2($"{name}", new BucketV2Args
 {
 Bucket = args.BucketName,
 Tags = tags
 }, new CustomResourceOptions { Parent = this });

 // Conditionally enable versioning
 if (args.Versioning != false)
 {
 new BucketVersioningV2($"{name}-versioning", new BucketVersioningV2Args
 {
 Bucket = bucket.Id,
 VersioningConfiguration = new BucketVersioningV2VersioningConfigurationArgs
 {
 Status = "Enabled"
 }
 }, new CustomResourceOptions { Parent = this });
 }

 // Conditionally enable encryption
 if (args.Encryption != false)
 {
 new BucketServerSideEncryptionConfigurationV2($"{name}-encryption", new BucketServerSideEncryptionConfigurationV2Args
 {
 Bucket = bucket.Id,
 Rules = new[]
 {
 new BucketServerSideEncryptionConfigurationV2RuleArgs
 {
 ApplyServerSideEncryptionByDefault = new BucketServerSideEncryptionConfigurationV2RuleApplyServerSideEncryptionByDefaultArgs
 {
 SseAlgorithm = "AES256"
 }
 }
 }
 }, new CustomResourceOptions { Parent = this });
 }

 BucketName = bucket.Id;

 RegisterOutputs(new Dictionary<string, object?>
 {
 { "BucketName", BucketName }
 });
 }
 }
}

package components

import (
 "github.com/pulumi/pulumi-aws/sdk/v7/go/aws/s3"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

// SecureBucketArgs contains the arguments for creating a SecureBucket
type SecureBucketArgs struct {
 BucketName pulumi.StringInput `pulumi:"bucketName,optional"`
 Versioning *bool `pulumi:"versioning,optional"`
 Encryption *bool `pulumi:"encryption,optional"`
 Tags map[string]pulumi.StringInput `pulumi:"tags,optional"`
}

// SecureBucket is a component that creates an S3 bucket with security best practices
type SecureBucket struct {
 pulumi.ResourceState

 BucketName pulumi.StringOutput `pulumi:"bucketName"`
}

// NewSecureBucket creates a new SecureBucket component
func NewSecureBucket(ctx *pulumi.Context, name string, args SecureBucketArgs, opts ...pulumi.ResourceOption) (*SecureBucket, error) {
 component := &SecureBucket{}
 err := ctx.RegisterComponentResource("mycomponents:index:SecureBucket", name, component, opts...)
 if err != nil {
 return nil, err
 }

 // Create an S3 bucket with best practices by default
 tags := pulumi.StringMap{
 "ManagedBy": pulumi.String("Pulumi"),
 }
 if args.Tags != nil {
 for k, v := range args.Tags {
 tags[k] = v
 }
 }

 bucket, err := s3.NewBucketV2(ctx, name, &s3.BucketV2Args{
 Bucket: args.BucketName,
 Tags: tags,
 }, pulumi.Parent(component))
 if err != nil {
 return nil, err
 }

 // Conditionally enable versioning
 if args.Versioning != nil && !*args.Versioning {
 // Skip versioning
 } else {
 _, err = s3.NewBucketVersioningV2(ctx, name+"-versioning", &s3.BucketVersioningV2Args{
 Bucket: bucket.ID(),
 VersioningConfiguration: &s3.BucketVersioningV2VersioningConfigurationArgs{
 Status: pulumi.String("Enabled"),
 },
 }, pulumi.Parent(component))
 if err != nil {
 return nil, err
 }
 }

 // Conditionally enable encryption
 if args.Encryption != nil && !*args.Encryption {
 // Skip encryption
 } else {
 _, err = s3.NewBucketServerSideEncryptionConfigurationV2(ctx, name+"-encryption", &s3.BucketServerSideEncryptionConfigurationV2Args{
 Bucket: bucket.ID(),
 Rules: s3.BucketServerSideEncryptionConfigurationV2RuleArray{
 &s3.BucketServerSideEncryptionConfigurationV2RuleArgs{
 ApplyServerSideEncryptionByDefault: &s3.BucketServerSideEncryptionConfigurationV2RuleApplyServerSideEncryptionByDefaultArgs{
 SseAlgorithm: pulumi.String("AES256"),
 },
 },
 },
 }, pulumi.Parent(component))
 if err != nil {
 return nil, err
 }
 }

 component.BucketName = bucket.ID().ToStringOutput()
 return component, nil
}

package io.mikhail.components;

import com.pulumi.core.Output;
import com.pulumi.aws.s3.BucketV2;
import com.pulumi.aws.s3.BucketV2Args;
import com.pulumi.aws.s3.BucketVersioningV2;
import com.pulumi.aws.s3.BucketVersioningV2Args;
import com.pulumi.aws.s3.inputs.BucketVersioningV2VersioningConfigurationArgs;
import com.pulumi.aws.s3.BucketServerSideEncryptionConfigurationV2;
import com.pulumi.aws.s3.BucketServerSideEncryptionConfigurationV2Args;
import com.pulumi.aws.s3.inputs.BucketServerSideEncryptionConfigurationV2RuleArgs;
import com.pulumi.aws.s3.inputs.BucketServerSideEncryptionConfigurationV2RuleApplyServerSideEncryptionByDefaultArgs;
import com.pulumi.resources.ComponentResource;
import com.pulumi.resources.ComponentResourceOptions;
import com.pulumi.resources.CustomResourceOptions;
import com.pulumi.core.annotations.Export;
import java.util.Map;
import java.util.HashMap;
import java.util.List;

public class SecureBucket extends ComponentResource {
 @Export
 public final Output<String> bucketName;

 public SecureBucket(String name, SecureBucketArgs args, ComponentResourceOptions options) {
 super("mycomponents:index:SecureBucket", name, options);

 // Create an S3 bucket with best practices by default
 Map<String, Output<String>> tags = new HashMap<>();
 tags.put("ManagedBy", Output.of("Pulumi"));
 if (args.tags() != null && !args.tags().isEmpty()) {
 tags.putAll(args.tags());
 }

 var bucket = new BucketV2(name, BucketV2Args.builder()
 .bucket(args.bucketName())
 //.tags(tags)
 .build(),
 CustomResourceOptions.builder().parent(this).build());

 // Conditionally enable versioning
 if (args.versioning() != Boolean.FALSE) {
 new BucketVersioningV2(name + "-versioning", BucketVersioningV2Args.builder()
 .bucket(bucket.id())
 .versioningConfiguration(BucketVersioningV2VersioningConfigurationArgs.builder()
 .status("Enabled")
 .build())
 .build(),
 CustomResourceOptions.builder().parent(this).build());
 }

 // Conditionally enable encryption
 if (args.encryption() != Boolean.FALSE) {
 new BucketServerSideEncryptionConfigurationV2(name + "-encryption",
 BucketServerSideEncryptionConfigurationV2Args.builder()
 .bucket(bucket.id())
 .rules(List.of(BucketServerSideEncryptionConfigurationV2RuleArgs.builder()
 .applyServerSideEncryptionByDefault(
 BucketServerSideEncryptionConfigurationV2RuleApplyServerSideEncryptionByDefaultArgs.builder()
 .sseAlgorithm("AES256")
 .build())
 .build()))
 .build(),
 CustomResourceOptions.builder().parent(this).build());
 }

 this.bucketName = bucket.id();

 this.registerOutputs(Map.of(
 "bucketName", this.bucketName
 ));
 }
}

package io.mikhail.components;

import com.pulumi.core.Output;
import com.pulumi.resources.ResourceArgs;
import com.pulumi.core.annotations.Import;

import java.util.Map;
import java.util.HashMap;

public class SecureBucketArgs extends ResourceArgs {
 @Import(name="bucketName")
 private Output<String> bucketName;

 public Output<String> bucketName() {
 return this.bucketName;
 }

 @Import(name="versioning")
 private Boolean versioning;

 public Boolean versioning() {
 return this.versioning;
 }

 @Import(name="encryption")
 private Boolean encryption;

 public Boolean encryption() {
 return this.encryption;
 }

 @Import(name="tags")
 private Map<String, Output<String>> tags = new HashMap<>();

 public Map<String, Output<String>> tags() {
 return this.tags;
 }

 private SecureBucketArgs() {}

 public SecureBucketArgs(Output<String> bucketName, Boolean versioning, Boolean encryption, Map<String, Output<String>> tags) {
 this.bucketName = bucketName;
 this.versioning = versioning;
 this.encryption = encryption;
 this.tags = tags;
 }
}

# YAML doesn't directly support authoring complex components
# For a complete component, you'd typically write it in another language
# This is a simplified component without conditionals or tags merges
runtime: yaml
name: my-components
components:
 SecureBucket:
 inputs:
 bucketName:
 type: string
 resources:
 bucket:
 type: aws:s3/bucketV2:BucketV2
 properties:
 bucket: ${bucketName}
 tags:
 ManagedBy: Pulumi

 bucketVersioning:
 type: aws:s3/bucketVersioningV2:BucketVersioningV2
 properties:
 bucket: ${bucket.id}
 versioningConfiguration:
 status: Enabled

 bucketEncryption:
 type: aws:s3/bucketServerSideEncryptionConfigurationV2:BucketServerSideEncryptionConfigurationV2
 properties:
 bucket: ${bucket.id}
 rules:
 - applyServerSideEncryptionByDefault:
 sseAlgorithm: AES256

 outputs:
 bucketName: ${bucket.id}

The PulumiPlugin.yaml File

The PulumiPlugin.yaml file tells Pulumi that this directory is a component package that can be consumed across languages (it is not required for a single language component), rather than a standard Pulumi program. In it, you define the language runtime needed to load the plugin:

runtime: nodejs

runtime: python

runtime: dotnet

runtime: go

runtime: java

runtime: yaml

For other languages, you’d use the appropriate runtime value. This file is the key that enables Pulumi to generate SDKs in other languages from your component code.

Define an Entry Point

The entrypoint analyzes components to automatically build a schema, and interact with the Pulumi engine to manage the component lifecycle.

No separate entry point is required for TypeScript. All exported component classes will automatically be exposed to be consumed from other languages.

Create a _main_.py file in your component directory
In the main function, add a call to component_provider_host, specifying a list of components for the components argument

from pulumi.provider.experimental import component_provider_host
from staticpage import MyComponent

if __name__ == "__main__":
 component_provider_host(name="python-components", components=[MyComponent])

Define a main.go file
Declare an instance of NewProviderBuilder, passing in a name, namespace and the components being built

The code below uses the new pulumi-go-provider v1 APIs. Make sure you are using the latest version of github.com/pulumi/pulumi-go-provider.

package main

import (
 "context"

 "github.com/pulumi/pulumi-go-provider/infer"
)

func main() {
 prov, err := infer.NewProviderBuilder().
 WithNamespace("your-org-name").
 WithComponents(
 infer.ComponentF(NewSecureBucket),
 ).
 Build()
 if err != nil {
 panic(err)
 }

 _ = prov.Run(context.Background(), "go-components", "v0.0.1")
}

Create a Program.cs file
Add an entry point that calls the ComponentProviderHost

using System.Threading.Tasks;

class Program
{
 public static Task Main(string []args) =>
 Pulumi.Experimental.Provider.ComponentProviderHost.Serve(args);
}

Create an App.java file
Create a new instance of ComponentProviderHost in the entry point

package com.example.components;

import java.io.IOException;
import com.pulumi.provider.internal.Metadata;
import com.pulumi.provider.internal.ComponentProviderHost;

public class App {
 public static void main(String[] args) throws IOException, InterruptedException {
 new ComponentProviderHost("java-components", App.class.getPackage()).start(args);
 }
}

No separate entry point is required for YAML.

Pulumi Components new capability is designed to make sharing infrastructure building blocks as easy as possible, whether you’re publishing them to a Git repository or referencing them from a local folder in a monorepo.

Publishing to Git

Once you’ve authored your component, sharing it is as simple as pushing your code to a Git repository and optionally tagging a release. From there, anyone (including you) can pull it down and use it in their own Pulumi projects.

For example, after pushing your SecureBucket to GitHub and tagging it as v1.0.0, others can consume it by running:

pulumi package add github.com/myorg/secure-s3-component@v1.0.0

Under the hood, Pulumi:

Fetches your code from GitHub
Auto-generates a local SDK from the component’s schema
Makes it available to your Pulumi program in your chosen language

Referencing Components Locally

For scenarios like monorepos, rapid development iterations, or when you’re working with components that don’t need to be published to a repository, you can reference local source code directly:

pulumi package add ./components/secure-s3-component

Pulumi will identify the folder as a Pulumi component project, generate a local SDK, and make it available for import in your program—even if your consumer program is in a different language. Both relative paths (as shown above) and absolute paths (like /path/to/local/secure-s3-component) are supported, giving you flexibility for different project layouts.

Consuming Components Across Languages

The real power comes in how these components can now be consumed in any Pulumi language:

import * as pulumi from "@pulumi/pulumi";
import { SecureBucket } from "@myorg/secure-s3-component";

const secureBucket = new SecureBucket("my-secure-bucket", {
 bucketName: "my-company-secure-assets",
 tags: {
 Environment: "production",
 Department: "engineering",
 },
});

export const bucketName = secureBucket.bucketName;

import pulumi
from myorg_secure_s3_component import SecureBucket

secure_bucket = SecureBucket("my-secure-bucket",
 bucket_name="my-company-secure-assets",
 tags={
 "Environment": "production",
 "Department": "engineering",
 })

pulumi.export("bucket_name", secure_bucket.bucket_name)

using System.Collections.Generic;
using Pulumi;
using MyOrg.SecureS3Component;

return await Deployment.RunAsync(() =>
{
 var tags = new Dictionary<string, string>
 {
 { "Environment", "production" },
 { "Department", "engineering" }
 };

 var secureBucket = new SecureBucket("my-secure-bucket", new SecureBucketArgs
 {
 bucketName = "my-company-secure-assets",
 Tags = tags
 });

 // Export the bucket name
 return new Dictionary<string, object?>
 {
 ["BucketName"] = secureBucket.BucketName
 };
});

package main

import (
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
 secure "github.com/myorg/secure-s3-component/sdk/go/secure-s3-component"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 tags := make(map[string]string)
 tags["Environment"] = "production"
 tags["Department"] = "engineering"

 secureBucket, err := secure.NewSecureBucket(ctx, "my-secure-bucket", &secure.SecureBucketArgs{
 BucketName: pulumi.String("my-company-secure-assets"),
 Tags: pulumi.ToStringMap(tags),
 })
 if err != nil {
 return err
 }

 ctx.Export("bucketName", secureBucket.BucketName)
 return nil
 })
}

package mypackage;

import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.myorg.secures3component.SecureBucket;
import com.myorg.secures3component.SecureBucketArgs;

import java.util.Map;

public class App {
 public static void main(String[] args) {
 Pulumi.run(ctx -> {
 var tags = Map.of(
 "Environment", "production",
 "Department", "engineering"
 );

 var bucketArgs = SecureBucketArgs.builder()
 .bucketName("my-company-secure-assets")
 .tags(tags)
 .build();

 var secureBucket = new SecureBucket("my-secure-bucket", bucketArgs);

 ctx.export("bucketName", secureBucket.bucketName());
 });
 }
}

name: my-infrastructure
runtime: yaml
description: A stack that uses SecureBucket component

packages:
 secure-s3-component: github.com/myorg/secure-s3-component@v1.0.0

resources:
 secureBucket:
 type: secure-s3-component:SecureBucket
 properties:
 bucketName: my-company-secure-assets
 tags:
 Environment: production
 Department: engineering

outputs:
 bucketName: ${secureBucket.bucketName}

See It In Action

Check out this video of authoring components in Java and consuming them in YAML to see Pulumi Components in action:

Pro Tips

Tagging Releases: If you’re working with Git-based components, you can tag version releases (both stable and preview versions). This ensures downstream users can easily choose specific versions without chasing a moving target.
Managing Dependencies: Once you’ve added an entry to the packages section of your Pulumi.yaml file, you can run pulumi install to generate a local SDK in your project. This command will process all packages listed in your Pulumi.yaml and create the necessary SDK files. Check in these files if you want fully reproducible builds; or add them to .gitignore if you prefer to regenerate them on each checkout. When using .gitignore, team members will need to run pulumi install after checkout to regenerate the SDK.
Leveraging Private Repos: If you need to keep your components private, you can still point to a Git source in a private repository. Just make sure the correct access tokens are set in the environment as described in our package management documentation.

The Spectrum of Pulumi Components You Can Build

You can use Pulumi Components with more flexibility and control depending on your use case. This table shows the variety of use cases:

Feature	Single language	Multi-language with Auto-Generated SDKs	Manual Schema and SDKs
Best for	Quick development within a single language ecosystem	Cross-language teams needing to share components	More flexibility and control needed or strict API requirements
Cross-language consumption	No - limited to original language	Yes - consume in any Pulumi language	Yes - consume in any Pulumi language
Setup complexity	Minimal - standard programming patterns	Minimal - just requires package management	High - requires schema authoring and validation
Development workflow	Fast iteration with direct code changes	Fast, requires SDK regeneration when component changes	Complex with schema updates and SDK publishing
API control	Full control within your language	Moderate - auto-generated from source	Full - ship the same interface to all consumers
Version management	Simple - standard code versioning	Moderate - requires careful API changes	Complex - strict semantic versioning needed
Typical user	Individual developers or same-language teams	Platform teams sharing with developers	Enterprise teams with strict requirements or package publishers
Ideal use cases	• Rapid prototyping • Single team projects • Simple components	• Organization-wide libraries • Platform engineering • Multi-language environments	• Published packages • Complex validation needs
Limitations	• Single language only • No additional setup required	• SDK regeneration overhead • Runtime dependencies • Some translation limitations	• Complex setup • Steep learning curve • Slower iteration

Conclusion

The ability to author components in one language and consume them in any other Pulumi language—including YAML—provides unprecedented flexibility and collaboration opportunities between platform teams and application developers.

To learn more about Pulumi Components, check out our documentation on components, building a component, packaging and publishing components, and component schemas.

We’re excited to see what you build with Pulumi Components! Share your feedback with us on the Pulumi Community Slack or open an issue in the pulumi/pulumi GitHub repository.

Expanding the Pulumi Registry: 27 New Providers to Expand the Ecosystem

Meagan Cojocar — Tue, 18 Mar 2025 00:00:00 +0000

Pulumi is excited to announce the addition of 27 new providers to the Pulumi Registry, significantly expanding our ecosystem and empowering developers with more choices for infrastructure as code. This expansion includes providers like Temporal, Vantage, and Honeycomb, among many others, making it even easier to manage diverse cloud and SaaS services with Pulumi.

This milestone is made possible by Pulumi’s new ability to use any Terraform provider (bridging existing Terraform providers for use in Pulumi), as detailed in our recent blog post. Since that announcement, Pulumi users have already begun to use many of the new infrastructure and SaaS providers that are now accessible within the familiar Pulumi programming model. To ensure the best developer experience, we’ve also added comprehensive documentation for 27 of these new providers directly in the Pulumi Registry. This means you can find everything you need right where you’re already spending time.

“Developers thrive on tools that seamlessly integrate into their existing build, test, and deployment workflows. That’s why we’re excited to have Temporal included in the Pulumi Registry—making it effortless to create and manage Temporal resources alongside other critical infrastructure using the programming languages developers already know and love.” - Jay Sivachelvan, VP of Partnerships - Temporal

This release represents just the first wave of our expanded provider documentation initiative. We’ve prioritized these initial 27 providers based on direct user and customer feedback. We’re committed to continuing this effort, with plans to add documentation for many more providers to the Pulumi Registry in the coming months. Our goal is to create the most comprehensive and user-friendly infrastructure as code ecosystem, making it even easier for you to manage your entire cloud infrastructure from a single platform.

This update is designed with developers in mind and features enriched documentation directly in the Registry, ensuring you have all the resources you need right at your fingertips. We’re also spotlighting a few partner providers:

Temporal – Manage Temporal Cloud as code.
Vantage – Gain cost visibility and optimize cloud spending.
Honeycomb – Seamlessly integrate observability best practices.
Timescale – Time series and real-time analytics on PostgreSQL.

“Timescale Cloud brings high-performance PostgreSQL to time-series, real-time analytics, and AI workloads, now with a seamless Pulumi integration. Developers can deploy and manage their infrastructure using their preferred language, making it easier than ever to scale PostgreSQL without resource-management headaches.” - Mike Freedman, CTO - Timescale

Full List of New Providers

Explore the complete list of new providers now available in the Pulumi Registry:

Dive into the comprehensive docs in the Registry, experiment with the new capabilities, and let us know what you build next.

Happy coding!

Pulumi + Gitlab: Better Than Ever

Meagan Cojocar — Fri, 14 Mar 2025 00:00:00 +0000

Pulumi’s integration with GitLab has reached new heights with enhancements designed to streamline your infrastructure as code workflows. Today, we’re excited to announce several significant improvements to our GitLab integration that make managing cloud infrastructure with Pulumi and GitLab more seamless than ever before: GitLab as a first-class VCS in Pulumi Cloud, enhanced merge request comments, organizational templates in GitLab, and later this year, Pulumi Deployments for GitLab.

Pulumi and GitLab have long enjoyed a productive partnership. From supporting GitLab sign-in to enabling GitLab CI/CD pipelines for infrastructure deployments, we’ve consistently worked to ensure GitLab users can leverage Pulumi’s powerful infrastructure as code capabilities within their existing workflows.

Our GitLab Merge Request integration allows teams to visualize infrastructure changes directly within merge requests, making code reviews more effective and infrastructure changes more transparent.

What’s New in Our GitLab Integration

Enhanced Merge Request Comments

We’ve revamped how Pulumi interacts with GitLab by building a Gitlab application to handle the authentication. Previously, only customers using Gitlab as an identity provider to Pulumi Cloud could leverage it. Now, Pulumi provides even more detailed and actionable comments on your merge requests, giving reviewers comprehensive insights into proposed infrastructure changes.

These comments include:

Streamlined discoverability of infrastructure changes
Resource-by-resource breakdown of changes
Direct links to the Pulumi Console for deeper analysis

To configure the GitLab integration:

Navigate to Organization Settings → Integrations in Pulumi Cloud
Select GitLab and enter your access token
Choose which projects or groups should receive Pulumi comments

GitLab as a First-Class VCS in Pulumi Cloud

Version control systems are a first-class, configurable concept in Pulumi Cloud. When you select GitLab as your VCS, the Pulumi Cloud interface adapts to show only relevant options and terminology, creating a more intuitive experience for GitLab users.

This integration enables:

Direct linking to GitLab repositories, branches, and commits
Automatic detection of Pulumi projects in GitLab repositories
Streamlined setup for GitLab-based CI/CD workflows, using the CI/CD wizard on a stack’s page

Organizational Templates in GitLab

Pulumi Cloud now fully supports organizational templates stored in GitLab repositories. This powerful feature allows platform teams to define standardized infrastructure templates that developers across the organization can easily discover and use to create new projects.

Once a GitLab repository is registered as a template source, Pulumi automatically detects pulumi.yaml files and surfaces them in the template gallery. These templates can be:

Public or private repositories
Subject to Pulumi’s role-based access control (RBAC)
Available in both the UI and CLI via pulumi new

This integration makes it easier than ever to standardize infrastructure patterns across your organization while leveraging GitLab for version control and collaboration.

Coming Soon: Pulumi Deployments for GitLab

Pulumi Deployments will also soon support GitLab, enabling fully automated infrastructure provisioning directly from your GitLab pipelines. This integration will provide:

Fully managed compute for running Pulumi operations
Click to deploy self service workflows in the Pulumi Cloud UI
Rich new out of the box functionality like Time to Live Stacks, Drift Detection, Click to Deploy, Git Push to Deploy, Review Stacks and Scheduled Deployments.

All this functionality is available today by specifying the raw git credentials, but we look forward to making the integration frictionless in the future. To learn more about Pulumi Deployments read our Pulumi Deployments documentation.

Getting Started

Ready to try these new GitLab integrations? Here’s how to get started:

If you’re new to Pulumi, sign up for a free account
For existing users, navigate to your organization settings to configure the GitLab integration
Check out our updated documentation for detailed setup instructions

Conclusion

With these enhancements, Pulumi’s GitLab integration is better than ever, providing a seamless experience for managing infrastructure as code within your GitLab workflows. Learn more in the Pulumi GitLab documentation. We’re committed to continuing to improve this integration based on your feedback.

Have questions or feedback? Join our Community Slack to connect with the Pulumi team and other users, or open an issue on our GitHub repository.

Faster Secrets in Pulumi IaC

Daniel Bradley — Thu, 13 Mar 2025 01:00:00 -0700

Pulumi now handles secrets more efficiently through optimized encryption and decryption processes, reducing deployment times while maintaining security standards. Users of Pulumi Cloud for state management will notice the most improvement due to new batch API capabilities.

Secrets Management in Pulumi

Pulumi Infrastructure as Code (IaC)’s built-in secrets management encrypts individual secrets within the stack for fine-grained protection. Pulumi Cloud transmits and stores stack state securely, ensuring that sensitive information—such as database passwords, API keys, and access tokens—remains protected even if someone gains access to your stack file. For organizations with more complex secrets management needs, Pulumi also offers Pulumi ESC (Environments, Secrets, and Configuration), which provides centralized secrets management with hierarchical environments and dynamic credential generation.

Beyond simple encryption, Pulumi tracks the transitive use of secrets to prevent accidental exposure of sensitive values. This tracking works across your entire infrastructure definition, ensuring secrets remain protected:

Within the stack state
In CLI output
In the Pulumi Cloud console

When using Pulumi Cloud as your secrets provider, all encryption and decryption operations happen server-side without transmitting the encryption key to the Pulumi program. Combined with Pulumi Cloud’s robust user authorization and SSO integration, this creates a comprehensive security framework that ensures only authorized team members can access sensitive information.

Performance Challenges with Secrets

While security is paramount, performance is equally crucial for developer productivity. Every second spent waiting for encryption or decryption operations is time not spent building and improving your infrastructure.

Previously, when working with stacks containing numerous secrets, users could experience noticeable delays during operations that required encryption or decryption. This was primarily due to:

Individual network requests for each secret operation
Network latency compounding with each separate request

For teams with complex infrastructure containing dozens or hundreds of secrets, these delays could add up to significant wait times.

Our Performance Improvements

We’ve reduced network overhead by batching encryption work into a single request instead of making separate encryption requests for each secret. This batching approach:

Reduces the total number of HTTP requests
Minimizes the impact of network latency
Allows for more efficient server-side processing

This is combined with identifying when no secrets have changed between intermediate deployment steps to bypass the work of re-encrypting all secrets.

Measurable Performance Gains

The performance improvements are particularly noticeable for stacks with many secrets or in environments with higher network latency:

Secrets Count	Network Latency	Typical Time Savings
10	100ms	~1 second
25	150ms	~4 seconds
50+	150ms	10+ seconds
100+	150ms	20+ seconds

The performance improvements are noticeable for stacks with many secrets or in environments with higher network latency. For teams running CI/CD pipelines with frequent deployments, these savings compound throughout the day, potentially saving significant cumulative wait time across your engineering organization.

Getting Started

To benefit from these performance improvements:

Update to Pulumi IaC version 3.155.0 or later
No configuration changes are needed—the optimizations work automatically
Ensure you’re using Pulumi Cloud as your secrets provider for additional speed gains

These improvements maintain complete backward compatibility with existing stacks and secret values. Your existing encrypted secrets will continue to work without any migration steps required.

Conclusion

At Pulumi, we’re committed to providing both robust security and exceptional performance. These secrets management optimizations represent our ongoing effort to improve the developer experience without compromising on security.

By reducing wait times during encryption and decryption operations, we’re helping teams be more productive while maintaining the same high level of protection for sensitive information. This update exemplifies our philosophy that security and performance should complement rather than compete with each other.

Beyond Stack Secrets: Pulumi ESC

While these performance improvements enhance Pulumi’s built-in secrets management, some organizations require more advanced capabilities. If you need secrets to be centrally managed, composable, and reusable across teams and applications, Pulumi ESC (Environments, Secrets, and Configuration) may be a better fit. Pulumi ESC keeps sensitive information out of source control, supports hierarchical environments, and enables dynamic credential generation. It’s designed for organizations that need to share secrets between multiple teams, applications, and infrastructure deployments while maintaining strict access controls and auditability.

Get Started Today

Update to Pulumi IaC 3.155.0 today to experience these performance improvements and let us know what you think!

Pulumi Kubernetes Operator 2.0 is Now Generally Available!

Eron Wright — Thu, 20 Feb 2025 00:00:00 +0000

Today marks an exciting milestone as we announce the General Availability (GA) release of the Pulumi Kubernetes Operator 2.0! This release builds upon the foundation we laid during the beta phase, delivering a production-ready solution that transforms how teams manage their cloud infrastructure.

Learn more about the Pulumi Kubernetes Operator in our documentation

What Is the Pulumi Kubernetes Operator?

The Pulumi Kubernetes Operator brings infrastructure automation directly to your Kubernetes cluster. By treating your infrastructure as native Kubernetes resources, it enables teams to manage their entire cloud environment—from databases to DNS records—using familiar Kubernetes workflows and tools.

Imagine having a dedicated infrastructure automation engine running within your cluster, continuously ensuring your cloud resources stay in sync with your desired state. That’s exactly what the Pulumi Kubernetes Operator delivers, complete with the observability and control you need for production environments.

The Evolution to 2.0

Pulumi Kubernetes Operator 2.0 introduces a completely new architecture for running Pulumi programs. The key changes include:

Each Stack now runs in its own dedicated Workspace pod, rather than within the Operator pod
A new Workspace custom resource manages the Workspace pod lifecycle
Stack operations are coordinated through a new Update custom resource
The Stack custom resource API remains the primary interface, maintaining backward compatibility

This architecture delivers improved resource isolation, better secrets management, and flexible Workspace customization options. The pod-per-stack model ensures reliable resource allocation and clear operational boundaries for each deployment.

What’s New in GA?

The GA release introduces three powerful capabilities that give you more control over your deployments:

Enhanced Logging Control

You can now set the log verbosity level of the Pulumi CLI for any given stack or Workspace. This feature offers granular control over logging output, making debugging and fine-tuning your deployment process more efficient.

New Controller Events for Better Instrumentation

Additional controller events have been introduced to provide richer instrumentation and observability. These events offer improved insights into the operator’s behavior, enabling more effective monitoring of your infrastructure.

Workspace Reclaim Policy

Managing Workspace lifecycles is now even more flexible with the new workspaceReclaimPolicy field in the Stack specification. This enhancement allows you to define how workspace resources should be handled after stack deployment. The supported policies are:

Retain: Keeps the Workspace and its resources after deployment (default)
Delete: Automatically removes the Workspace once the stack is fully deployed

This gives you fine-grained control over resource cleanup and helps prevent workspace sprawl in your cluster.

See It in Action

Here’s an example that demonstrates how to deploy a Pulumi stack using the operator. This configuration creates a service account with appropriate permissions and defines a Stack resource that deploys a sample project from the Pulumi examples repository:

import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";
import * as pulumiservice from "@pulumi/pulumiservice";

// Create a Kubernetes ServiceAccount for the Pulumi Workspace pod
const sa = new k8s.core.v1.ServiceAccount("random-yaml", {});

// Grant system:auth-delegator to the ServiceAccount
const crb = new k8s.rbac.v1.ClusterRoleBinding("random-yaml:system:auth-delegator", {
 roleRef: {
 apiGroup: "rbac.authorization.k8s.io",
 kind: "ClusterRole",
 name: "system:auth-delegator",
 },
 subjects: [{
 kind: "ServiceAccount",
 name: sa.metadata.name,
 namespace: sa.metadata.namespace,
 }],
});

// Provision a Pulumi Cloud access token and store it in a Kubernetes Secret
const accessToken = new pulumiservice.AccessToken("random-yaml", {
 description: `For stack "${pulumi.runtime.getOrganization()}/${pulumi.runtime.getProject()}/${pulumi.runtime.getStack()}"`,
});
const apiSecret = new k8s.core.v1.Secret("random-yaml", {
 stringData: {
 "accessToken": accessToken.value,
 }
});

// Deploy the "random-yaml" program from the github.com/pulumi/examples repository.
const stack = new k8s.apiextensions.CustomResource("random-yaml", {
 apiVersion: 'pulumi.com/v1',
 kind: 'Stack',
 spec: {
 serviceAccountName: sa.metadata.name,
 projectRepo: "https://github.com/pulumi/examples",
 repoDir: "random-yaml/",
 branch: "master",
 shallow: true,
 stack: "pulumi-ts",
 refresh: true,
 destroyOnFinalize: true,
 envRefs: {
 PULUMI_ACCESS_TOKEN: {
 type: "Secret",
 secret: {
 name: apiSecret.metadata.name,
 key: "accessToken",
 },
 },
 },
 workspaceTemplate: {
 spec: {
 image: "pulumi/pulumi:3.147.0-nonroot",
 },
 },
 },
}, {dependsOn: [sa, crb, apiSecret]});

Here’s an example of the enhanced event stream that shows the lifecycle of a stack deployment, from workspace creation to successful completion:

88s Normal SuccessfulCreate statefulset/random-yaml-09aa1aad-workspace create Pod random-yaml-09aa1aad-workspace-0 in StatefulSet random-yaml-09aa1aad-workspace successful
88s Normal StackUpdateDetected stack/random-yaml-09aa1aad Initial stack update
0s Normal Pulled pod/random-yaml-09aa1aad-workspace-0 Container image "pulumi/pulumi:3.147.0-nonroot" already present on machine
0s Normal Created pod/random-yaml-09aa1aad-workspace-0 Created container pulumi
0s Normal Started pod/random-yaml-09aa1aad-workspace-0 Started container pulumi
0s Normal Initialized workspace/random-yaml-09aa1aad Initialized workspace pod "random-yaml-09aa1aad-workspace-0"
0s Normal StackUpdateDetected stack/random-yaml-09aa1aad Initial stack update
0s Normal UpdateSucceeded update/random-yaml-09aa1aad-195200641a9 Updated stack "pulumi-ts"
0s Normal StackCreated stack/random-yaml-09aa1aad Successfully updated stack

Additional Resources

To help you get the most out of Pulumi Kubernetes Operator 2.0, we’ve prepared comprehensive documentation:

Migration Guide: Step-by-step instructions for upgrading from v1.x to v2.0
Troubleshooting Guide: Common issues and their solutions
Release Notes: Full details of what’s new in version 2.0

Start Using It Today

We are excited to see how you will use these new features to transform your cloud operations. Share your experiences, questions, and insights with us through our community forums or Slack channel.

Get started with the Pulumi Kubernetes Operator in our documentation

Happy deploying!

Pulumi Java is Now Generally Available

Mark Huber — Mon, 10 Feb 2025 16:15:00 -0800

One of Pulumi’s core Infrastructure as Code (IaC) features is the ability to model infrastructure using well-traveled, familiar general-purpose programming languages. Today, we’re thrilled to announce that Java, one of the world’s most popular programming languages, is now generally available in Pulumi. This release joins our existing first-class support for TypeScript, Python, Go, YAML, and C#, enabling Java developers to manage cloud infrastructure using the language they know and trust.

We’ve grown our Java support based on feedback from the expanding set of companies that have adopted Pulumi Java in production workloads. Let’s look at some examples of using Java to power your Pulumi programs.

Familiar Composability

Using Java with Pulumi lets you model your infrastructure using familiar patterns. Paired with rich abstractions, you can efficiently build Pulumi programs. It only seems appropriate that we take a look at a Java example using Oracle Cloud:

We start our program by grabbing our configuration values that describe which distro and AD to use.

var config = ctx.config();
var compartmentId = config.require("compartmentId");
var availabilityDomain = config.require("availabilityDomain");
var ubuntuId = config.require("ubuntuId");

Next, we create a set of abstractions to describe the networking and OS.

var vcn = new VirtualNetwork("virtualNetworkResource", VirtualNetworkArgs.builder()
 .compartmentId(compartmentId)
 .cidrBlock("10.0.0.0/16")
 .build());

var subnet = new Subnet("testSubnet", SubnetArgs.builder()
 .cidrBlock("10.0.0.0/24")
 .compartmentId(compartmentId)
 .vcnId(vcn.id())
 .availabilityDomain(availabilityDomain)
 .build());

var vnicDetails = InstanceCreateVnicDetailsArgs.builder()
 .subnetId(subnet.id())
 .assignPublicIp("true")
 .build();

var sourceDetails = InstanceSourceDetailsArgs.builder()
 .sourceType("image")
 .sourceId(ubuntuId)
 .build();

Finally, we compose our instance.

var instance = new Instance("instanceResource", InstanceArgs.builder()
 .availabilityDomain(availabilityDomain)
 .compartmentId(compartmentId)
 .shape("VM.Standard.E2.1.Micro")
 .preserveBootVolume(false)
 .createVnicDetails(vnicDetails)
 .sourceDetails(sourceDetails)
 .build());

With just a few lines of code (especially by Java standards), we composed a set of strongly typed, easy-to-reason about resources using the builder pattern (a Java favorite) and without introducing any new language paradigms.

New: Java Automation API

With the general availability of our Java language support, we’re also excited to announce that the Automation API is now supported in Java. The Automation API is a fully typed SDK that allows you to interact with Pulumi programs outside the Pulumi CLI. You can directly access and orchestrate your Pulumi projects and stacks with the SDK, allowing you to supercharge your Infrastructure as Code.

In our Automation API examples repo, we have an example demonstrating how you could construct a Pulumi program with the Automation API to perform a database migration. Let’s take a look at a few of the interesting aspects:

Like regular Pulumi programs, we’re able to obtain the current operating context and use the details to construct new resources, inline with the rest of the Automation API program.

Consumer<Context> program = ctx -> {
 ...

 var subnetGroup = new SubnetGroup("db-subnet", ...);

 // Make a public security group for our cluster for the migration
 var securityGroup = new SecurityGroup("public-security-group", ...);

 // Example values for our db
 var dbName = "hellosql";
 var dbUser = "hellosql";
 var dbPassword = "hellosql";

 // Provision our db
 var cluster = new Cluster("db", ClusterArgs.builder()
 .engine(EngineType.AuroraMysql)
 .engineVersion("8.0.mysql_aurora.3.08.0")
 .databaseName(dbName)
 .masterUsername(dbUser)
 .masterPassword(dbPassword)
 .skipFinalSnapshot(true)
 .dbSubnetGroupName(subnetGroup.name())
 .vpcSecurityGroupIds(securityGroup.id().applyValue(List::of))
 .build());

 var clusterInstance = new ClusterInstance("db-instance", ClusterInstanceArgs.builder()
 .clusterIdentifier(cluster.clusterIdentifier())
 .instanceClass("db.t3.medium")
 .engine(EngineType.AuroraMysql.getValue())
 .engineVersion("8.0.mysql_aurora.3.08.0")
 .publiclyAccessible(true)
 .dbSubnetGroupName(subnetGroup.name())
 .build());

 ctx.export("host", cluster.endpoint());
 ctx.export("db_name", dbName);
 ctx.export("db_user", dbUser);
 ctx.export("db_pass", dbPassword);
};

With the Automation API, however, we’re also able to directly interact with project and stack objects, giving us the opportunity to orchestrate workflows. This could include scaffolding projects, working across stacks, or in our case, completing a one-time workflow to migrate a database.

var projectName = "database_migration_project";
var stackName = "dev";

try (var stack = LocalWorkspace.createOrSelectStack(projectName, stackName, program)) {
 ...

 stack.workspace().installPlugin("aws", "v6.68.0");

 stack.setConfig("aws:region", new ConfigValue("us-west-2"));

 System.out.println("updating stack...");
 var result = stack.up(UpOptions.builder()
 .onStandardOutput(System.out::println)
 .build());

 var changes = result.summary().resourceChanges();
 if (!changes.isEmpty()) {
 System.out.println("update summary:");
 changes.forEach((key, value) -> {
 System.out.printf(" %s: %d%n", key, value);
 });
 }

 System.out.printf("db host url: %s%n", result.outputs().get("host").value());
 configureDatabase(
 ...
 );
}

The full example, along with others, is available in our Automation API examples repo. We’re excited to bring the Automation API to the Java SDK for the first time and we’re looking forward to your feedback.

Get Started with Pulumi Java

To learn more about and to get started with Pulumi Java, you can check out the following resources:

Java SDK Repo: The Java SDK is fully open-source, and we’re excited to get your feedback or review any pull requests.
Examples Repo: The Pulumi examples repo has many examples of Pulumi Java programs to help you get up and running with Pulumi Java.
Pulumi Java Converter: The Pulumi converter tool efficiently migrates existing IaC applications, such as Terraform, ARM, and Bicep, to Pulumi Java.

Enforcing Policy as Code on Discovered Resources with Pulumi

Tyler D — Tue, 04 Feb 2025 00:00:00 +0000

In this post, we’re introducing a powerful new capability in Pulumi Insights that extends policy as code (PaC) beyond infrastructure as code to automatically govern all cloud resources in your environment. By unifying policy enforcement across both IaC and discovered resources, you can now write policies once and apply them universally - dramatically simplifying how organizations maintain security and compliance standards at scale.

What is Policy as Code (PaC)?

Policy as Code (PaC) is a way to define and enforce infrastructure policies using code. Instead of relying on manual processes or static configuration files, PaC enables automated policy enforcement in a consistent, versioned way. Pulumi’s CrossGuard feature allows you to write policies in general purpose programming languages, applying them across various cloud platforms such as AWS, Azure, GCP, and Kubernetes.

You can read more about CrossGuard in Pulumi’s CrossGuard documentation.

Accounts and Insights

Before diving into how you can configure PaC for discovered resources, it’s important to understand how Pulumi discovers resources within your cloud environment regardless of how they were created. Pulumi Insights provides visibility into the state of your cloud resources by using read only access to your cloud environment in order to catalog the resources it contains. This is done through the Accounts page in Pulumi, which shows all the accounts for discovering resources you have added to your Pulumi organization. On this page, you can add new accounts which you can then add to policy groups to begin evaluating all of your cloud resources using your organization’s policy as code.

After you have added an account, you can connect it with one or multiple Policy Groups to enforce the policies within those groups across your stacks and Insights accounts. Once linked to a Policy Group, your resources will be automatically evaluated against the policies you define whenever those policies or resources are updated.

Setting Up Policy Groups for Discovered Resources

To apply policies to your discovered resources, you first need to set up a Policy Group. A Policy Group in Pulumi enforces policies across a group of stacks and accounts in your organization. Each Policy Group can contain multiple stacks and accounts, and you can assign multiple Policy Packs to these groups.

Here’s an example of the Policy Group configuration page, where you can create and manage policy groups:

Once your Policy Group is set up, you can add Insights Accounts to the group, which will ensure that the resources in those accounts are evaluated against the policies in the group.

Running Policies on Discovered Resources

Now that your Policy Group is set up, you can start running policies against your discovered resources. Currently, Pulumi Insights supports running resource policies against all resource types supported by Pulumi Insights with support for stack policies coming soon. Pulumi Insights will evaluate each resource against the policies defined in your Policy Pack. If the resources violate any of the defined policies, those violations will show up on the Policy Violations page, which gives you a detailed view of any non-compliant resources.

Each violation includes details about the resource and the reason for the violation, helping you quickly identify and address issues.

Key Takeaways

With Pulumi’s new PaC over discovered resources feature, you can now apply the same policies to both IaC and discovered resources across your cloud environment. This enables you to write a policy once and have it apply across all of your cloud resources. This results in consistent, compliant infrastructure no matter how resources are provisioned.

To begin securing your cloud infrastructure, here are the steps to take:

Set up Policy Groups to enforce policies across stacks and accounts.
Link Insights Accounts to your Policy Groups.
View violations on the Policy Violations page and take action to resolve them.

This new capability is available for AWS, Azure, OCI, and Kubernetes. Stay tuned for more updates on policy enforcement and discover how you can leverage Pulumi for cloud security and compliance.

Try it yourself: Start enforcing policies across your discovered resources in minutes with Pulumi Insights. Check out our getting started guide to begin securing your entire cloud environment with policy as code.

For more information on getting started with Policy as Code, check out the following resources:

Pulumi Copilot is Now Available in VS Code

Meagan Cojocar — Mon, 03 Feb 2025 00:00:00 +0000

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

Programming languages offer dozens of advantages for writing Infrastructure as Code (IaC). One of them is that Large Language Models are effective at using general-purpose programming languages, thanks to the vast amount of high-quality training data available. Building on this advantage, we introduced Pulumi AI and Pulumi Copilot last year to enhance Infrastructure-as-Code development with generative AI capabilities. These tools have significantly streamlined infrastructure deployment for tens of thousands of developers.

Today, we are thrilled to announce that Pulumi Copilot is now available directly within Pulumi Copilot Chat Extension. By simply typing @pulumi in Copilot Chat, developers can now access the power of Pulumi Copilot right within their IDE, saving them time on writing IaC and getting infrastructure deployed.

Video Walk Through

Hear Adam and Eron from Pulumi walk through this new feature in our PulumiTV introduction to the Visual Studio Code extension.

What is Pulumi Copilot?

Pulumi Copilot is an AI-powered conversational assistant that seamlessly integrates with Pulumi Cloud, helping users:

Explore and manage cloud infrastructure
Gain insights into resources, policies, and deployments
Troubleshoot errors and optimize configurations
Author and deploy Pulumi IaC more effectively

Pulumi Copilot combines industry-leading AI models with Pulumi specific knowledge like the latest versions of providers, IaC best practices, and context from Pulumi Cloud on your environment.

The Pulumi Copilot REST API

The Pulumi Tools Visual Studio Code extension is powered by the recently announced Pulumi Copilot REST API, which enables developers to integrate Pulumi Copilot’s capabilities into their own tools and platforms. Just as we’ve built this extension to bring Pulumi Copilot directly into your development environment, you can use the same REST API to create your own innovative integrations, whether that’s a custom CLI tool, a chat bot, or another IDE extension. The API’s support for multi-turn conversations and contextual understanding makes it possible to build rich, interactive experiences like the one we’re delivering today.

How to Use Pulumi Copilot in Visual Studio Code

Using Pulumi Copilot in Visual Studio Code (VS Code) is as simple as 1-2-3:

Install the Pulumi VS Code extension
Open VS Code and navigate to Copilot Chat (icon to the right of the search bar)
Type @pulumi to activate Pulumi Copilot

You can then ask questions about your infrastructure, generate code, check deployment status, debug Pulumi errors, and more.

This seamless integration brings Pulumi’s intelligence directly into the developer workflow.

Pulumi Copilot respects all role-based access control (RBAC) settings in Pulumi Cloud, ensuring that AI-generated responses are scoped only to authorized users within an organization.

Key Use Cases

Let’s explore four powerful ways to use Pulumi Copilot in VS Code:

Infrastructure Code Generation

With Pulumi Copilot in VS Code, developers can generate Pulumi programs for new use cases, solving the blank page problem and getting you from 0 to 0.9.

Simply describe what you want to build and Pulumi Copilot will generate the infrastructure code for you. For example, ask “Create an AWS S3 bucket with versioning enabled and encrypted with a KMS key” and Copilot will generate the complete Pulumi program. Or ask Pulumi Copilot to extend an existing project such as adding a ManagedNodeGroup to your AWS EKS cluster.

Terraform to Pulumi Migration

Migrating from Terraform to Pulumi? Just point Copilot at your Terraform files and ask it to convert them. For example:

Open your main.tf file in VS Code
Type @pulumi convert this Terraform configuration to TypeScript
Copilot will analyze your Terraform code and generate equivalent Pulumi code in index.ts

The conversion maintains your existing resource configurations while taking advantage of Pulumi’s native programming language features.

Resource Discovery

Pulumi Copilot can search across your entire infrastructure to find resource usage patterns. Ask questions like “Do we use AWS Lambda functions in any other stacks?” or “Show me all S3 buckets with public access enabled.” Copilot will search your codebase and Pulumi Cloud to provide comprehensive answers about your infrastructure.

Error Resolution

When you encounter errors in the terminal during Pulumi operations, Copilot can help explain and resolve them. Simply copy the error message and ask Copilot to explain it. For example:

Error: Error creating S3 Bucket (my-bucket): BucketAlreadyExists: The requested bucket name is not available

Copilot will explain the error in plain language, suggest potential solutions, provide code examples for fixing the issue, and reference similar issues from your organization’s history.

Getting Started

Pulumi Copilot is free to use for all organizations in Pulumi Cloud. Organization administrators can enable Pulumi Copilot by navigating to: Settings > Access Management > Pulumi Copilot in the Pulumi Cloud console.

Try it now in VS Code and experience AI-driven cloud infrastructure management right in your IDE.

👉 Start Using Pulumi Copilot 👉 Start Using Pulumi Copilot in Visual Studio Code

We can’t wait to see what you build with Pulumi Copilot! 🚀

Introducing Customizable Resource Auto-naming in Pulumi

Mikhail Shilkov — Thu, 16 Jan 2025 00:00:00 +0000

I’m thrilled to announce that you can now customize how Pulumi names your cloud resources! Our default auto-naming feature has helped thousands of customers successfully manage cloud resources at scale by automatically ensuring unique, conflict-free resource names across their cloud deployments. This robust naming system has been particularly valuable for teams managing multiple environments, handling zero-downtime deployments, and maintaining clear resource organization. Today, we’re taking it to the next level by giving you control over how these names are generated.

Over the years, we’ve heard from many teams using Pulumi that while they love the power and convenience of our auto-naming system, they need it to work with their organization’s naming standards - whether that’s adding cost center identifiers, following compliance rules, or matching existing naming patterns. The GitHub issue tracking this feature has gathered quite some attention (50 thumbs up!) and lots of great input from the community.

Today, I’m excited to introduce resource auto-naming configuration. Now you can have the best of both worlds: keep the robustness of Pulumi’s auto-naming while making it follow your team’s naming conventions. Want your resources to include environment tags? Project prefixes? Random suffixes of specific length? Disable auto-naming entirely? It’s all possible now, and it works across all major cloud providers.

The Road to Better Resource Naming

The original feature request I opened in June 2018 when I was a Pulumi customer. It has generated extensive discussion, with users sharing various use cases and requirements. Today, I’m happy to finally close that issue with a solution that addresses the community’s needs while maintaining Pulumi’s robust resource management capabilities.

Check out the full story:

Introducing Auto-naming Configuration

With the new auto-naming configuration feature, you now have full control over how Pulumi generates resource names. Here are some common scenarios you can achieve:

Disable Auto-naming

If you want complete control over your resource names, you can disable Pulumi auto-naming entirely:

pulumi:autonaming:
 mode: disabled

In this mode, Pulumi will require you to provide explicit physical names for all resources.

Use Logical Names As-Is

For scenarios where you want Pulumi to copy exactly the logical names to become the physical names, you can use the verbatim mode:

pulumi:autonaming:
 mode: verbatim

No random suffixes will be added to the resource names.

Note, when an update requires replacing the resource, Pulumi’s default behavior is to create the new resource and then delete the old resource. However, when using verbatim names or patterns without random components, resources that need to be replaced will be deleted before creating the new resource. This can lead to downtime.

Custom Naming Patterns

Create your own naming patterns that combine static text, resource information, and random elements:

pulumi:autonaming:
 pattern: ${project}-${stack}-${name}${alphanum(6)}

See the auto-naming configuration documentation to see the full list of available expressions.

See It In Action

Let’s look at a practical example. Say you’re creating an S3 bucket and a DynamoDB table in your Pulumi program:

import * as aws from "@pulumi/aws";

// Create an S3 bucket
const bucket = new aws.s3.Bucket("uploads");

// Create a DynamoDB table
const table = new aws.dynamodb.Table("users", {
 hashKey: "id",
 attributes: [{
 name: "id",
 type: "S",
 }],
 billingMode: "PAY_PER_REQUEST",
});

import pulumi_aws as aws

# Create an S3 bucket
bucket = aws.s3.Bucket("uploads")

# Create a DynamoDB table
table = aws.dynamodb.Table("users",
 hash_key="id",
 attributes=[aws.dynamodb.TableAttributeArgs(
 name="id",
 type="S"
 )],
 billing_mode="PAY_PER_REQUEST"
)

using Pulumi;
using Pulumi.Aws.S3;
using Pulumi.Aws.DynamoDB;
using Pulumi.Aws.DynamoDB.Inputs;

return await Deployment.RunAsync(() =>
{
 // Create an S3 bucket
 var bucket = new Bucket("uploads");

 // Create a DynamoDB table
 var table = new Table("users", new TableArgs
 {
 HashKey = "id",
 Attributes = new[]
 {
 new TableAttributeArgs
 {
 Name = "id",
 Type = "S"
 }
 },
 BillingMode = "PAY_PER_REQUEST"
 });
});

package main

import (
 "github.com/pulumi/pulumi-aws/sdk/v5/go/aws/dynamodb"
 "github.com/pulumi/pulumi-aws/sdk/v5/go/aws/s3"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // Create an S3 bucket
 bucket, err := s3.NewBucket(ctx, "uploads", nil)
 if err != nil {
 return err
 }

 // Create a DynamoDB table
 table, err := dynamodb.NewTable(ctx, "users", &dynamodb.TableArgs{
 HashKey: pulumi.String("id"),
 Attributes: dynamodb.TableAttributeArray{
 &dynamodb.TableAttributeArgs{
 Name: pulumi.String("id"),
 Type: pulumi.String("S"),
 },
 },
 BillingMode: pulumi.String("PAY_PER_REQUEST"),
 })
 if err != nil {
 return err
 }

 return nil
 })
}

package myproject;

import com.pulumi.Pulumi;
import com.pulumi.aws.s3.Bucket;
import com.pulumi.aws.dynamodb.Table;
import com.pulumi.aws.dynamodb.TableArgs;
import com.pulumi.aws.dynamodb.inputs.TableAttributeArgs;

public class App {
 public static void main(String[] args) {
 Pulumi.run(ctx -> {
 // Create an S3 bucket
 var bucket = new Bucket("uploads");

 // Create a DynamoDB table
 var table = new Table("users", TableArgs.builder()
 .hashKey("id")
 .attributes(TableAttributeArgs.builder()
 .name("id")
 .type("S")
 .build())
 .billingMode("PAY_PER_REQUEST")
 .build());
 });
 }
}

resources:
 # Create an S3 bucket
 uploads:
 type: aws:s3:Bucket

 # Create a DynamoDB table
 users:
 type: aws:dynamodb:Table
 properties:
 hashKey: id
 attributes:
 - name: id
 type: S
 billingMode: PAY_PER_REQUEST

By default, Pulumi would generate names like uploads-ae26f3b and users-4c2dd09. But let’s say you want your resources to follow a pattern that includes your project and stack name. You can configure this in your stack configuration file (Pulumi.<stack-name>.yaml):

config:
 pulumi:autonaming:
 pattern: ${project}-${stack}-${name}

Now when you run pulumi up, your resources will be created with predictable names:

S3 bucket: myproject-dev-uploads
DynamoDB table: myproject-dev-users

You can also set different patterns for specific providers or resource types:

config:
 pulumi:autonaming:
 pattern: ${project}-${stack}-${name}
 providers:
 aws:
 resources:
 "aws:s3/bucket:Bucket":
 pattern: ${name}-${stack}-${alphanum(6)}

With this configuration, you’ll get:

S3 bucket: uploads-dev-x7yz9n (with a random suffix for global uniqueness)
DynamoDB table: myproject-dev-users (following the default pattern)

Configuration Syntax

The configuration syntax differs slightly depending on where you define it:

In your project file Pulumi.yaml:

config:
 pulumi:autonaming:
 value:
 mode: verbatim

In your stack configuration file Pulumi.<stack-name>.yaml:

config:
 pulumi:autonaming:
 mode: verbatim

The same applies to other configuration patterns shown above - use the value: key in project-level configuration, but omit it in stack-level configuration.

Getting Started

To use the auto-naming configuration feature, you’ll need:

Pulumi CLI 3.146.0 or later
The following minimum provider versions (as applicable):
- AWS provider 6.66.0 or later
- Azure Native provider 2.78.0 or later
- Azure Classic provider 6.14.0 or later
- Google Cloud Platform provider 8.11.0 or later
- Kubernetes provider 4.20.0 or later
- AWS Cloud Control provider 1.21.0 or later

Once you have the required versions installed, simply add your desired auto-naming configuration to your Pulumi configuration file.

For complete documentation and advanced usage scenarios, visit our resource auto-naming documentation.

General Availability

We’re excited to announce that the auto-naming feature is now generally available across our major cloud providers. This release marks an important milestone in Pulumi’s evolution, delivering a robust and flexible solution for resource naming.

Thank you to everyone who participated in the RFC discussion and the preview period and for providing valuable feedback. Your input has been invaluable in creating a solution that works for diverse use cases while maintaining Pulumi’s core strengths.

If you have any questions or feedback about the resource auto-naming feature, please don’t hesitate to reach out to us on GitHub or in the Pulumi Community Slack.

The Past 6 Months of Pulumi Releases

Meagan Cojocar — Tue, 24 Dec 2024 00:00:00 +0000

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

As we wrap up 2024, let’s look back at the significant features and improvements Pulumi has delivered in the last half of the year. Some key highlights:

Pulumi Visual Studio Code Extension
Pulumi ESC General Availability
Pulumi Insights 2.0 with Resource Explorer
Enhanced Kubernetes support

Pulumi Infrastructure as Code

Kubernetes Enhancements

Pulumi Kubernetes Operator 2.0 Major update introducing a horizontally scalable architecture, enhanced security features, and improved customization for the Pulumi Kubernetes Operator.
EKS Auto Mode A groundbreaking feature that simplifies EKS cluster management by automatically handling compute, storage, and networking configurations based on AWS’s operational expertise.
Pulumi EKS 3.0 Release Update introducing support for Amazon Linux 2023, Access Entries for IAM integration, EKS managed addons, and enhanced security features.
Pulumi Kubernetes Await Logic Better resource dependency tracking, improved error reporting, faster deletions with propagation policies, and custom readiness criteria support.

Provider Updates

Use Any Terraform Provider Automatic generation of Pulumi providers from any Terraform provider, dramatically expanding the ecosystem of available resources.
AWS CDK On Pulumi 1.0 Expanded compatibility with AWS CDK features and construct hub integration, allowing developers to continue using familiar AWS CDK patterns within Pulumi.
Azure v6 Release Major update ensuring compatibility with the latest Azure services, improved error messages, and new resource types.
Google Cloud Provider v8.0.0 Comprehensive support for new Google Cloud services and improved resource management capabilities.
AWS Cloud Control Provider GA Native AWS API support with day-zero coverage for new AWS features and improved consistency in resource management.

Core Platform Features

Pulumi Visual Studio Code Extension Enhanced IaC debugging in the IDE, Pulumi YAML support, and ESC management within VS Code.
Pulumi Docker Image Improvements Versioned images with pre-installed tools for Python, Node.js, and .NET. Pulumi Deployments now supports setting the version of the Node.js and Python runtimes used in the Deployment environment.
Python UV Support Built-in support for uv, an extremely fast Python package manager that speeds up dependency installation by up to 100x compared to traditional tools.
DependsOn for Provider Functions Manage explicit dependencies between provider functions for more accurate execution in complex deployments.

Pulumi Environments, Secrets and Configuration (Pulumi ESC)

General Availability & Key Enhancements

ESC General Availability GA of Pulumi ESC - redefining secrets management with features like dynamic credentials, hierarchical environments, a secrets broker with external stores like AWS Secrets Manager, and Kubernetes operators for runtime integration.
ESC Projects & Environment Tags Organize secrets and configurations at scale with new grouping capabilities.
ESC Webhooks Real-time notifications and automated actions based on environment changes.
AWS Parameter Store Integration Native integration with AWS Systems Manager Parameter Store for seamless import of parameters and secrets into ESC environments.
Environment Import Discovery Enhanced visibility into environment dependencies with new features to track downstream usage, version tags, and impact analysis for configuration changes.

Deployment Improvements

Dependency Caching Speed up deployments by up to 80% by caching dependencies between runs, with support for npm, pip, and go modules.
Kubernetes-native Agent Purpose-built agent for Kubernetes environments, enabling self-hosted deployment agents within your Kubernetes clusters without requiring Docker in Docker or privileged execution.

Pulumi AI and Insights

Pulumi Insights 2.0

Resource Explorer Enhanced visual interface for exploring cloud resources with new features like customizable columns, nested grouping capabilities, advanced filtering, and shareable saved views across your organization.
Account Discovery Preview New capability to discover and manage all cloud resources regardless of how they were created, with AI-powered search and comprehensive resource visibility across your infrastructure.

Pulumi Copilot Enhancements

REST API Integration: Enabling programmatic access to Pulumi Copilot’s AI capabilities through a REST API, supporting multi-user interactions, automation of queries, and integration with workplace collaboration platforms.
System Prompts: New organization-level customization allowing teams to set default preferences for programming languages, cloud providers, compliance guidelines, and infrastructure standards.
One-Click Update Analysis: Debug failed updates instantly with Copilot by analyzing logs and suggesting fixes directly from the update page or from the CLI.
Documentation Integration: Integrated throughout Pulumi’s website and documentation, providing contextual assistance and documentation search capabilities to help users find relevant resources and answers to their questions.

Pulumi Cloud Console Improvements

Organization-wide Policies New centralized Policy Violations page providing comprehensive visibility of compliance across projects, with the ability to group violations by Policy Pack, Project, Stack, and Resource, plus API access for automation.
Stacks Page Redesign Major redesign offering improved performance with thousands of stacks, flexible grouping up to three levels, enhanced sorting options, shareable views via URL parameters, and personalized preferences that persist between sessions.

That’s a wrap for July through December 2024! From major IaC updates to advanced Pulumi ESC features to the launch of Pulumi Insights 2.0 and powerful AI integrations, Pulumi continues to innovate across the cloud engineering space. Be sure to check out the linked blog posts and docs for more details on each release. Happy building!

Pulumi ESC: Discovering Environment Imports

Sean Yeh — Fri, 13 Dec 2024 00:00:00 -0700

Managing secrets and configuration across multiple environments and stacks can easily become complex, leading to duplicated values, inconsistencies, and security risks. Pulumi ESC solves this with composable environments via imports, allowing you to define configuration once and reuse it organization-wide. Now, with the new capabilities to discover environment imports, you gain unprecedented visibility and control, simplifying the management of even the most complex infrastructure and applications.

Understanding Pulumi ESC Imports

Pulumi ESC lets you define secrets and configuration as code collections called environments. These environments can import other environments, creating a hierarchical structure that promotes reusability, and reduces duplication and sprawl. You can centralize secrets and configuration within ESC and use them across Pulumi IaC Stacks, Pulumi Insights Accounts, and various other platforms through our integrations.

By using version tags, imports can target specific versions of an environment. This locks down the configuration used by dependents, ensuring stability and preventing unintended changes from rolling out to your critical environments.

What’s New?

We’ve added powerful discoverability features to give you clear insight into your environment dependencies:

New “Imported By” tab: Easily understand downstream dependencies by clicking the “Imported By” tab for each environment. This tab shows all environments and stacks that import a specific environment, including the specific tagged version used. Filter and sort by imported versions and downstream resources to narrow in on the resources that will be affected by a change you are trying to make.
Warnings for deletions and tag movements: Pulumi ESC now warns you about the potential impact of deleting an environment or modifying version tags before you make the change, helping you avoid disruptions.
Enhanced environment page: The Environments page now clearly indicates if an environment is used by other environments, stacks, or insights accounts, providing valuable relationship context at a glance.
Pulumi IaC stack visibility: Environment import information is also available on the Stacks page and within individual stack overview pages.

Conclusion

Pulumi ESC’s import feature is a paradigm shift in how you manage your secrets and configuration. The new suite of import discoverability features gives you the confidence to manage complex deployments with ease. Explore the new features and share your feedback through our Community Slack or GitHub Repo.

Pulumi Copilot is Now Integrated with Pulumi Docs: A New Way to Learn and Explore

Meagan Cojocar — Thu, 24 Oct 2024 23:59:00 -0700

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

Pulumi Copilot has been making our customers’ day-to-day tasks easier since its release, and today we’re excited to expand its capabilities—Pulumi Copilot is now available across Pulumi Documentation and pulumi.com, and comes equipped with a powerful new Documentation Skill!

These additions make learning about Pulumi more interactive and intuitive by allowing users to explore cloud infrastructure concepts directly within the documentation through a conversational interface. Whether you’re a seasoned Pulumi user or just starting out, Pulumi Copilot is here to help you when you need it most, where you need it most.

New Enhancements in Pulumi Copilot

Expanded Website Integration

Pulumi Copilot is now embedded throughout the Pulumi website, providing contextual assistance no matter where you are. Whether you’re exploring documentation, reading a blog post, browsing the Pulumi registry, or reviewing case studies, Pulumi Copilot offers helpful suggestions, explanations, and links to guide you to relevant resources.

Pulumi Documentation Skill

The new Documentation Skill is designed to make Pulumi’s docs more accessible from wherever. You can ask questions like “How do I use the dependsOn resource option?” or “How do I use SAML in my organization?" or “How can I store and retrieve secrets using Pulumi ESC?” and Pulumi Copilot will answer them by searching through our documentation on your behalf. This new skill is available everywhere Pulumi Copilot is: in Pulumi Cloud, in Pulumi documentation, across the Pulumi website, and the Registry.

Try Pulumi Copilot Today

Pulumi Copilot’s new Documentation Skill and website integration are available now! These features are free to use today. You can unlock more messages by being logged in to Pulumi Cloud.

To get started, visit the Pulumi Cloud console and enable Pulumi Copilot in your settings under Access Management.

Pulumi Copilot is already making waves, and with these latest enhancements, it’s now easier than ever to manage and learn about your cloud infrastructure. We can’t wait for you to experience it firsthand!

Unveiling the New Stacks Page

Germán Lena — Thu, 24 Oct 2024 00:00:00 -0700

We’re thrilled to unveil the redesigned Pulumi Infrastructure as Code (IaC) Stacks page within Pulumi Cloud console, addressing one of the most upvoted issues in our Pulumi Cloud Requests repository! This revamped experience delivers a significant boost in performance and usability, empowering even the most complex organizations to manage their infrastructure with ease.

Why a New Stacks Page?

As our customers scaled, the previous Stacks page faced some limitations that impacted its effectiveness. Here were some of the primary challenges:

Performance Issues: The page became sluggish with large numbers of stacks, making it difficult to scroll smoothly when you had thousands of stacks.
Limited Grouping and Sorting Options: Users had limited customization options for organizing their stacks, with minimal control over grouping and sorting.
Tag-Based Grouping Limitations: Grouping by tags wasn’t as reliable as we wanted—it sometimes failed to load completely.

What’s New?

The new Stacks page provides more information at a glance, offering a streamlined and powerful experience:

Effortless Scalability: Manage thousands of stacks with ease. The new page is optimized for performance, ensuring smooth scrolling and interaction regardless of scale.
Flexible Grouping: Group your stacks by repository, project, and tags simultaneously—up to three levels of grouping. Or opt for no grouping at all! The choice is yours. Expand and collapse groups as needed for a more organized and focused browsing experience.
Enhanced Sorting: Sort your stacks by last update or group name (ascending/descending) to quickly find what you need.
Shareable Views: The page now includes query parameters in the URL, allowing you to save and share your exact view with team members effortlessly.
Personalized Experience: The page remembers your last selected groups and sorting preferences, so you can pick up right where you left off.
Dynamic Loading: Enjoy instantaneous loading as you scroll, eliminating wait times and boosting efficiency.
Time-Saving Shortcuts: Use quick option to make group and sort selections or to reset them.

Classic View: Still Available

For users who prefer the familiar grouping by repository and project, and sorting by last updated, we’ve got you covered! Simply click the “Classic View” shortcut in the top right-corner to switch to the classic grouping and sorting option instantaneously.

Your Feedback Matters

We’re excited for you to experience the power and flexibility of the new Pulumi Stacks page! Share your feedback and suggestions through our Pulumi Cloud requests GitHub repository—we’re always listening and striving to improve.

Pulumi EKS Provider Version 3.0.0

Florian Stadler — Thu, 17 Oct 2024 17:32:03 +0200

The latest major release of the Pulumi EKS Provider is available now!

This release delivers significant improvements in flexibility, security and introduces new features to enhance your Kubernetes experience on AWS. AWS recently announced the deprecation of two features used by default in Pulumi EKS: the aws-auth ConfigMap and the AL2 operating system. Pulumi EKS v3 addresses these deprecations, enhances the maintainability of the provider, and aligns it with EKS best practices.

Here are a few links to help you get started if you are new to Pulumi:

EKS Setup & Installation - Instructions on installing and configuring the Pulumi EKS provider
EKS How-to Guides - Learn how to use the EKS provider to provision and manage Kubernetes clusters on AWS
Pulumi Neo - Ask Pulumi Neo to help you create a new EKS project

Key Highlights of EKS V3

Support for Amazon Linux 2023 (AL2023) and Bottlerocket Operating Systems: Enhanced operating system options for node groups, allowing you to choose the OS that best fits your workloads and compliance needs. This addresses the upcoming deprecation of Amazon Linux 2 (AL2).
Access Entries for IAM Integration: Enables replacement of the deprecated aws-auth ConfigMap with Access Entries for managing Kubernetes authentication.
EKS Managed Addons: Simplified management of vpc-cni, coredns, and kube-proxy as EKS managed addons.
EKS Security Groups for Pods and Network Policies: Enhanced network security and control within EKS clusters.

New Features and Improvements

Support for Amazon Linux 2023 and Bottlerocket

We have expanded the operating system options for node groups in EKS v3 to address the upcoming deprecation of Amazon Linux 2 (AL2). You can now choose between Amazon Linux 2 (deprecated), Amazon Linux 2023 and Bottlerocket for your EKS nodes. This flexibility allows you to select the OS that best fits your workloads, security requirements, and compliance needs, while ensuring you are using a supported and actively maintained operating system. We’ve introduced a new operatingSystem property for node groups to facilitate this choice.

Access Entries for IAM Integration

AWS has introduced Access Entries as a new method for granting IAM principals access to Kubernetes resources. This approach relies solely on AWS resources for managing Kubernetes auth, replacing the deprecated aws-auth ConfigMap. You can now leverage Access Entries by setting the authenticationMode to API in your cluster configuration.

EKS Managed Addons

The EKS cluster components vpc-cni, coredns, and kube-proxy are now configured as EKS managed addons. This change simplifies management, especially for clusters with private API endpoints, and ensures that these critical components stay up to date automatically. Additionally it removes the dependency on kubectl, allowing pulumi-native management of clusters.

Cluster Autoscaler Integration

Pulumi EKS v3 introduces better support for the Kubernetes Cluster Autoscaler. A new ignoreScalingChanges parameter for node groups allows Pulumi to ignore external scaling changes, facilitating seamless integration with dynamic scaling solutions.

EKS Security Groups for Pods and Network Policies

We’ve added support for EKS security groups for pods (example) and EKS Network Policies (example), providing more granular control over pod-to-pod and pod-to-external network communication within your EKS clusters.

Migration Guide

To help you transition smoothly, we’ve prepared a migration guide with these key steps:

Update node groups to use AL2023 or explicitly configure AL2 if needed.
Replace the deprecated NodeGroup component with NodeGroupV2.
Update your code to handle new output types for certain properties.
Review and update your use of default security groups, which can now be disabled.

Please refer to our EKS v3 Migration Documentation for a detailed guide.

Conclusion

Pulumi EKS v3 represents a significant step forward in managing Kubernetes clusters on AWS, empowering you to build and manage more robust and efficient EKS clusters.

We encourage all users to upgrade to this latest version to take advantage of these improvements and ensure your EKS deployments remain secure and up-to-date.

For more information on getting started with Pulumi EKS v3, check out our documentation or join our community Slack for support and discussions.

Happy clustering with Pulumi EKS v3!

Enhancing Pulumi Copilot: Introducing System Prompts for Your Organization

Meagan Cojocar — Thu, 10 Oct 2024 00:00:00 -0700

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

We are excited to announce a new feature for Pulumi Copilot: System Prompts. This enhancement empowers organizations to customize Pulumi Copilot’s responses for your organization, making your interactions with our AI assistant even more personalized to save you even more time.

Pulumi Copilot is a conversational chat interface integrated throughout Pulumi Cloud, enabling users to quickly accomplish a variety of cloud infrastructure management tasks by leveraging the power of large language models plus the rich capabilities of Pulumi Cloud. We released Pulumi Copilot in June this year, and have seen remarkable uptake across our customer base. We are excited to be announcing enhancements on the Pulumi Copilot experience- keep an eye out for more to come in the near future.

System prompts allow organization administrators to set default preferences and guidelines for Pulumi Copilot. By configuring these prompts, you can tailor Pulumi Copilot’s behavior to better suit your team’s needs and policies. Here are some ways you might use organization system prompts:

Set a Default Programming Language: Ensure that all code snippets and examples provided by Pulumi Copilot are in your team’s preferred language.
Specify a Default Cloud Provider: Streamline resource searches and code generation by focusing on your primary cloud environment. Specify providers your organization prefers to use, such as using Azure Native instead of Azure Classic.
Enforce Compliance Guidelines: Instruct Copilot to generate code and provide guidance that adheres to specific compliance standards, such as SOC 2.
Standardize Infrastructure Components: Provide example code templates for resources to ensure consistency across your organization’s projects, such as “here is how we create VPCs”

How to Set Organization System Prompts

Setting up system prompts is straightforward:

Log in to Pulumi Cloud and go to your organization’s settings in the left hand navigation
Under the settings menu, select “Access Management.”
In the Copilot section, you’ll find the option to set your system prompts.

Though the system prompt supports 10k characters of context, we recommend keeping it concise to ensure optimal performance. See more information on setting system prompts in our Pulumi Copilot documentation.

Wrapping it up

This is just the beginning. We’re dedicated to making Pulumi Copilot an indispensable part of your cloud infrastructure management toolkit. Stay tuned for more updates and enhancements based on your feedback.

If you have suggestions or encounter any issues, please let us know through our Pulumi Cloud requests GitHub repository.

Pulumi Azure Classic Provider Version 6.0.0

Daniel Bradley — Fri, 27 Sep 2024 12:04:00 +0000

The latest major release of the Azure Classic Pulumi Provider is available now! Our 6.0 release contains the latest upstream changes to keep you up-to-date along with the latest features and improvements from Pulumi.

The Azure Classic provider is derived from the upstream Terraform AzureRM provider and can be used to provision most resources on Azure. The Azure Classic provider can be used to complement the Pulumi Azure Native provider, our recommended provider, which has amazing resource coverage due to being derived directly from the official specifications. The provider is open source and available on GitHub so you can always follow along with current issues and developments, or even open your first pull request.

Here are a few links to help you get started if you are new to Pulumi:

Getting Started - A guided walkthrough for creating your first project
Setup & Install - Instructions on installing the Azure Classic provider
How-to guides - Learn how to use the Azure Classic provider to provision specific resources
Pulumi Neo - Ask Pulumi Neo to create a new project

Looking Back

Since the last major release of this provider, we have continuously shipped improvements to our ecosystem, bringing the latest Pulumi features to your production stack. We have rolled out an improved diffing strategy and fixed state upgrades in the Pulumi Terraform Bridge, removing spurious diffs on preview and increasing confidence when deploying. Additionally we have improved accuracy and coverage for registry documentation, via better example conversion and general docs improvements for bridged providers.

New Modules

Over the last two years, we have added support for several new modules. Among these are:

API Coverage Growth

The below chart shows the growth of this provider by resource, function, and supporting types since Version 5.

Upgrading

You can find our v5 to v6 Migration Guide on the Pulumi Registry.

Pulumi Google Cloud Provider Version 8.0.0

Guinevere Saenger — Thu, 19 Sep 2024 10:08:42 -0700

The latest major release of the Pulumi Google Cloud Provider is available now! Our 8.0 release contains the latest upstream changes to keep you up-to-date along with the latest features and improvements from Pulumi.

The Pulumi Google Cloud provider can be used to provision any of the Google Cloud resources available in the upstream provider. The provider is open source and available on GitHub so you can always follow along with current issues and developments, or even open your first pull request.

Here are a few links to help you get started if you are new to Pulumi:

Getting Started - A guided walkthrough for creating your first project
Setup & Install - Instructions on installing the Google Cloud provider
How-to guides - Learn how to use the Google Cloud provider to provision specific resources
Templates - Use a quickstart template to create a new project
Pulumi Neo - Ask Pulumi Neo to create a new project

Looking Back

New Modules

Over the last year, we have added support for several new modules. Among these are:

API Coverage Growth

The below chart shows the growth of this provider by resource, function, and supporting types since Version 7.

What’s New in 8.0

Added Deletion Protections

Several resources have new deletionProtection fields:

gcp.cloudrunv2.Service
gcp.cloudrunv2.Job
gcp.activedirectory.Domain
gcp.organizations.Folder
gcp.organizations.Project

Find the documentation for any of these resources in our registry

New default provisioning label

A new default label, goog-pulumi-provisioned lets you discover your Pulumi-provisioned resources in the GCP console to help you track resources and how they were created. This label is available as an Output only and can be disabled in your provider configuration.

Upgrading

You can find our v7 -> v8 Migration Guide on the Pulumi Registry.

Announcing the Pulumi Visual Studio Code Extension

Meagan Cojocar — Wed, 18 Sep 2024 08:56:00 +0000

At the heart of Pulumi’s approach to cloud infrastructure and secrets management is a belief in empowering engineers to use the best software engineering tools to manage complexity at scale and to be maximally productive building cloud infrastructure and applications for their businesses.

Today, we’re excited to announce a next big step in delivering great software engineering tools for Pulumi users, with the launch of a new Pulumi Visual Studio Code (VS Code) Extension. The Pulumi VS Code Extension brings Pulumi key features for Pulumi IaC and Pulumi ESC directly into the IDE environment that many Pulumi users work in every day. Pulumi IaC users can now debug their applications and get Pulumi YAML language support directly in VS Code. And Pulumi ESC users can now create and manage environments, secrets and configuration directly within the IDE with rich IDE features.

The following are a few examples of new capabilities available for Pulumi users in VS Code today:

Launch your Pulumi program under a debugger
Automatically generate a launch configuration for a Pulumi project
Create and explore Pulumi ESC (Environments, Secrets, and Configuration) environments
Get contextual warnings and errors directly within the editor for Pulumi YAML

Getting Started

Install the Software

Install Pulumi 3.132.0 (or greater) using these instructions.
Install the Pulumi VS Code extension from the Visual Studio Marketplace.

Debugging Pulumi IaC Programs

For simple Pulumi programs, it’s easy enough to see what will happen when you look at the IaC code. But as teams build more complex cloud infrastructure components, fully taking advantage of Pulumi’s support for all of the rich componentization and control flow features in their language of choice, it can become more challenging to diagnose why a particular input value is being used for a cloud infrastructure resource, and what set of decisions were made by the infrastructure components to arrive at that. The pulumi CLI recently added support for a new --attach-debugger flag, which allows attaching a debugger to any language processes that are run as part of a Pulumi operation - regardless of the language. We’ve now added native VS Code debugger extension support for it as well, so that you can debug your Pulumi programs in any language with just a couple of clicks.

Start Debugging

The Pulumi VS Code Extension allows you to run and debug Pulumi programs with ease. Open a new or existing Pulumi project in VS Code and use an automatic or customized launch configuration.

Select the Run and Debug icon.
Choose Show all automatic debug configurations.
Select “Pulumi…” then “pulumi preview” or “pulumi up”.

Pulumi automatically generates a debug configuration for the current stack, making it simple to get started. You can also set breakpoints in your program and use the full functionality of the VS Code debugger.

You can now step through your code, set breakpoints on resource constructors or inside apply callbacks, and see the callstack and the values of variables within the VS Code debugger.

Create a Launch Configuration

For more control, you can create a customized launch configuration:

Click the gear icon when selecting an automatic debug configuration to customize it.
Alternatively, create a launch.json file and use a configuration template.

You can run without debugging by selecting “Run Without Debugging” from the Run menu or by adding "noDebug": "true" to your configuration.

Using Pulumi ESC

Pulumi ESC manages secrets and configuration for your applications and infrastructure within a centralized and composable secrets store. The environments are stored securely within Pulumi Cloud, and can be edited and opened from the Pulumi Cloud console or from the pulumi CLI or REST API.

For developers working with both their application infrastructure code as well as their Pulumi ESC environments, it is convenient to be able to edit ESC environments and the secrets they contain directly within the IDE without switching contexts.

The Pulumi ESC integration in the Pulumi VS Code extensions makes this easy. You can navigate through your environments, see all the projects and versions, and directly edit the source of the environments within VS Code. When you save, the edit to the environment is saved directly back into Pulumi ESC as a new version. And while you have it open in VS Code, you get all the amazing software engineering productivity feature of your IDE - colorization, completion lists, error checking, goto definition, find references and much more!

Open the ESC Explorer

From the primary sidebar, open the “Pulumi ESC Explorer” view and click “Login” to authenticate to Pulumi Cloud. Once logged in, you’ll see a tree of your organizations and environments.

Key Capabilities

Easily create or edit environments by clicking the plus sign on an organization or project node. The extension opens an editor where you can define and save new environment revisions. The IntelliSense functionality will guide you on various values, such as completion for setting up provider configuration.

You can also:

Run your application in the context of your environment by hovering over it and selecting the terminal icon

Delete environments by hovering over them and selecting the delete icon
Decrypt environments for cloning or moving
View environment versions over time
Compare different environments or environment versions
Use the search icon to quickly find environments
Tag specific environment revisions for easy reference.

Pulumi YAML Support

Along with support for many popular programming languages including TypeScript, Python, Go, .NET and Java - Pulumi also supports a simple declarative YAML language for defining and managing infrastructure. VS Code already has great support for all the other languages Pulumi supports, but for Pulumi YAML, the Pulumi VS Code Extension offers great support for working in this YAML dialect, including support for type checking, completion lists, colorization and more.

Building on a Pulumi YAML Language Server Protocol (LSP) available for use in any IDE environment, the Pulumi YAML IDE support providers a wide range of editor integrations.

Type Checking

The Pulumi YAML LSP provides contextual warnings and errors directly within the editor.

Warnings: Alerts when a variable is defined but never referenced.
Errors:
- Invalid YAML document detection.
- Reference errors for non-existent variables.
- Duplicate names for variables/resources.

Hover Tooltips

Hover over a resource type token or function type token to see a popup with descriptions, helping you understand your resources and functions better.

Completion Lists

Experience semantic completion for:

Predefined Pulumi YAML keys such as “resources” or “properties”.
Resource properties or function arguments.
Type tokens for resources or functions.
Structured variables (e.g., referencing properties within complex objects).

Wrapping up

This extension is currently in public beta, and we’re eager to hear from the Pulumi community. Your feedback is crucial in helping us shape the future of the extension. If you have suggestions for new features or encounter any bugs, please open an issue.

What’s Next

We’re committed to making Pulumi development as efficient and enjoyable as possible. In the coming months, expect even more features and improvements, all aimed at helping you work smarter and faster. Stay tuned for updates as we continue to build out this powerful extension and bring new capabilities to the Pulumi platform.

Pulumi ESC Table Editor Now Supports Dynamic Credential and Secret Integrations

Kimberley Mackenzie — Tue, 10 Sep 2024 00:00:00 -0700

We are pleased to announce a major update to the Pulumi ESC’s Table Editor: full support for provider configurations is now live! When we first introduced the ESC Table view, our goal was to offer a user-friendly interface for managing complex configuration and secrets for your applications and infrastructure. This has empowered teams to focus on what they do best—building and deploying software.

In our initial launch, the Table Editor allowed you to perform CRUD operations on your secrets and plaintext values, decrypt secrets, import environments, and manage variables—all within a visual interface. Today, we’re taking the next step by enabling full read-write support for provider configurations within the Table Editor. This new capability allows you to add, edit, and manage your provider configurations directly from the table view, offering a more integrated and seamless experience.

Primer on ESC Provider Configuration

Pulumi ESC’s provider ecosystem is one of its most powerful features. It supports two categories of providers:

Dynamic Cloud Provider Credentials: Dynamically acquire credentials from cloud providers like AWS, Azure, GCP, and Vault at runtime, eliminating the need for static, long-lived credentials. This enhances security by removing the reliance on dotenv files, locally stored secrets, or long-lived credentials in CI/CD environments, strengthening your workflows’ overall security.
Dynamic Secret Retrieval: ESC retrieves secrets from external secrets managers, such as AWS Secrets Manager, 1Password, and HashiCorp Vault, securely pulling them directly into your environments. Whether you are migrating or utilizing Pulumi ESC’s rich toolset, you can retrieve sensitive information at runtime and apply it seamlessly to your applications.

Enhancing the Table Editor: Provider Configuration Support

With the addition of provider configuration support in the Table view, incorporating Pulumi ESC provider capabilities into your workflow is now more straightforward. A clean, user-friendly form provides step-by-step guidance, promoting best practices like OIDC adoption and the secure handling of sensitive data. This streamlined approach reduces the need for external documentation, making the entire process more intuitive and efficient.

Conclusion

The new provider configuration support in the ESC Table view marks another step in our efforts to simplify and enhance the developer experience. We’re eager to see how you’ll leverage this new capability to streamline your workflows and strengthen your infrastructure’s security. As always, your feedback is invaluable to our development process, and we’re committed to continually improving the Table Editor and other Pulumi tools based on your needs. Please share your feedback with us here.

Check out our documentation to learn more about Pulumi ESC.

Introducing the Centralized Policy Violations Page: Streamlined Compliance Management

Meagan Cojocar — Mon, 26 Aug 2024 09:00:00 -0700

We’re excited to introduce a new centralized Policy Violations destination in Pulumi Cloud to simplify policy management and compliance. This powerful addition provides a comprehensive view of all policy violations across your organization, helping you maintain visibility and control over your infrastructure.

Pulumi’s Policy as Code (PaC) feature, CrossGuard, allows you to define and enforce policies for your infrastructure using familiar programming languages. By integrating these policies directly into your infrastructure code, you can automate compliance checks, gate deployments to ensure security best practices, and maintain governance across all your providers.

Note: Pulumi CrossGuard is available in the Pulumi Business Critical edition.

Centralized Policy Violations: Why It Matters

Ensuring compliance across multiple projects and stacks can be challenging in large and complex environments. The new Policy Violations page addresses this challenge by consolidating all violations into one place. This allows you to quickly assess the overall compliance of your infrastructure and separate signal from noise. This streamlined view saves time and reduces the risk of missing critical issues.

By having all violations on a single page, organization admins can efficiently prioritize remediation efforts and identify patterns in non-compliance. They can triage the violations and send the critical ones to teams to resolve.

Grouping Violations: Enhanced Insights Across Multiple Dimensions

One of the standout features of the new page is the ability to group violations by various fields, including Policy Pack, Policy, Project, Stack, Resource, and more! This flexibility allows you to gain deeper insights into your infrastructure’s compliance landscape.

Grouping violations based on these fields enables you to answer critical questions such as:

Policy Pack and Policy: Which Policy Packs and Policies have the most frequent violations?
Project and Stack: Are there specific projects or stacks that are more prone to compliance issues?
Resource: Which resources have failed policies?
Reason and Enforcement Level: What are the common reasons for violations, and how critical are they?
Violation Date: When are violations most likely to occur, and are there patterns over time?

These insights empower you to refine your policies, address the most problematic areas, and improve your overall compliance strategy.

Accessing Policy Violations via API

To further enhance your compliance management, you can leverage the Pulumi Cloud REST API to programmatically access policy violations, trigger automated workflows, and import data into your analytics systems. The API can automate the monitoring of your infrastructure’s compliance status, ensuring that policy violations are promptly detected and addressed. This capability enables you to integrate Pulumi’s compliance data with your existing tools, creating a seamless and efficient compliance management process.

For example, you can use the following command to retrieve a list of policy violations:

curl \
 -H "Accept: application/vnd.pulumi+8" \
 -H "Content-Type: application/json" \
 -H "Authorization: token $PULUMI_ACCESS_TOKEN" \
 https://api.pulumi.com/api/orgs/{organization}/policyresults/violations

For more details on how to integrate this API into your workflow, visit the Pulumi API documentation.

Streamlining Compliance with Future Enhancements

Centralizing and surfacing policy violations is just the beginning. Future updates will include additional capabilities like advanced search and filtering, making it even easier to manage violations. We’re committed to providing the tools you need to keep your infrastructure secure, compliant, and running smoothly.

Wrapping It Up

The Centralized Policy Violations Page to Pulumi is a significant step forward in simplifying policy management and enhancing compliance. By bringing all violations into a single, easily accessible view, we’re making it easier for admins to maintain control and ensure their infrastructure remains secure and compliant.

Try out this new feature today and experience the benefits of streamlined compliance management. For more information, visit our Policy Violations documentation. To use this feature, start a trial or contact sales to get a demo or to trial Policy as Code.

Kubernetes-native Option for Pulumi Deployments Customer-Managed Agents

Levi Blackstone — Wed, 14 Aug 2024 08:08:31 -0700

We are excited to announce the availability of Kubernetes-native support for Pulumi Deployments Customer-Managed Agents, further enhancing the flexibility and control over your infrastructure deployments. This new feature allows you to self-host deployment agents within your Kubernetes environment, bringing the same power and flexibility of Pulumi-hosted deployments to isolated environments.

Customer Managed Agents: Flexibility and Control

Customer Managed Agents, announced earlier this year, allow you to self-host deployment agents, delivering the power and flexibility of Pulumi Deployments within your own infrastructure. Here’s how you can benefit:

Host Anywhere: Deploy Customer Managed Agents on any hardware or environment of your choice, including fully private VPCs.
Compliance and Security: Keep your cloud provider credentials within your private network, meeting stringent compliance requirements.
Scalability: Set up multiple agent pools and scale dynamically, supporting up to 150 concurrent deployments.
Versatility: Mix and match Pulumi-hosted and self-hosted deployments to suit different infrastructure needs.

Key Features of Pulumi Deployments

Customer-Managed agents support all the capabilities of Pulumi Deployments including:

Multiple Deployment Triggers: Supports all Pulumi Deployment triggers such as Drift Detection, Scheduled Deployments, click to deploy, REST API, Git push to deploy, Review Stacks, and remote Automation API.
OpenID Authentication: Fetch tokens dynamically for increased security.
Flexible Configuration: Configure through YAML files or environment variables, adaptable to various deployment environments.

Docker and Kubernetes Support

By default, Customer-Managed Agents are deployed on Virtual Machines using Docker. However, with this new release, you can now leverage Kubernetes to run these agents, enabling you to deploy them within your existing infrastructure. This integrates natively with Kubernetes pods and does not require Docker in Docker or privileged execution.

Getting Started with Kubernetes

To use Kubernetes for running Customer-Managed Agents:

Install the Pulumi GitHub App: Install the Pulumi GitHub App and ensure that it is integrated with your source control.
Set Up Deployment Pools:
- Navigate to Deployment Runners under Organization Settings in the left hand navigation bar.
- Create a new pool by clicking ‘Add new pool’
- You will get a new access token, copy and save the token.
Install Agents:
- Follow the instructions provided on the Deployment Runners page to install agents within your Kubernetes environment.
Configure Agents:
- All configuration for Customer-Managed Agents is done through the pulumi-deployment-agent.yamlfile. This file can be created manually or with the customer-managed-deployment-agent configure command.
- Or you can leverage the Pulumi program example repo.
- Set environment variables directly in your Kubernetes deployment.
Configure Stack:
- Go to Deploy settings, found under the Settings tab on a stack. Scroll down and select the pool you created under the Deployment Runner pool drop-down.

Verification

After setting up, verify the agent by doing a pulumi refresh through the Deploy Actions drop-down on your stack page.

To increase deployment concurrency, use the same token to set up multiple agents and they will be assigned to the same pool.

Wrapping It Up

Customer-Managed Agents with Kubernetes support empower you to manage deployments according to your specific needs. Whether it’s for security, compliance, or operational efficiency, this feature provides the flexibility and control necessary for modern infrastructure management.

For more details, check out our documentation or contact our sales team to get started with Customer-Managed Agents today.

Feel free to reach out with feedback or questions. We’re here to support you every step of the way.

Self-Hosted Pulumi Cloud Support for Resource Search and Pulumi Deployments

Meagan Cojocar — Wed, 03 Jul 2024 09:08:31 -0700

We are excited to introduce two powerful new features for our Self-Hosted Pulumi Cloud offering: Resource Search and Pulumi Deployments. These two additions are the most significant features added to Pulumi Cloud in the last year and we are thrilled to bring them to our self-hosted customers. Resource Search and Pulumi Deployments both improve the way you can manage and deploy your cloud infrastructure, providing greater visibility, control, and automation.

Alongside these features, we are also introducing today a Self-Hosted Changelog, a log of all relevant changes that self-hosted customers should be aware of.

Self-Hosted Pulumi Cloud Overview

Self-Hosting is available in the Pulumi Business Critical edition of Pulumi Cloud, you can try it out with our 30 day trial. This allows organizations to run the Pulumi Cloud within their own environments, ensuring data control, compliance, and customizability. While Pulumi manages all aspects of its SaaS offering (Pulumi Cloud), Self-Hosted Pulumi Cloud requires customers to handle data backups, service maintenance, and updates.

Here’s a deep dive into what the new features offer and how they can benefit your organization:

Resource Search: Find and Analyze Your Cloud Resources with Ease

Pulumi Resource Search offers multi-cloud search and analytics across every resource in your organization. You can issue queries that find all of your AWS VPCs, or all of your VPCs in AWS and VNets in Azure, or all resources with the “production” tag across all cloud environments. You also have access to statistics about cloud usage, including a breakdown by cloud provider, resource type, and department. Resource search enables you to find the needle within your cloud haystack and visualize cloud consumption trends.

Key Features

Multi-Cloud Search: Query resources across various cloud environments.
Advanced Filtering: See top stacks, projects, providers, teams, and types by resource count.
Resource Search API: Integrate search functionality into your internal systems using the Search API.
Search from the CLI: Issue resource search queries from within the terminal without needing to change contexts.
Data Export: Export resource data for further analysis in your data warehouse.

Get more details on these features in our Resource Search documentation. For more information on how to enable it in your self-hosted instance read the self-hosted Resource Search documentation.

Pulumi Deployments: Automate and Manage Your Infrastructure with Ease

Pulumi Deployments is engineered to streamline infrastructure management tasks with a focus on scalability, automation, and control. This platform allows cloud engineers and platform teams to execute infrastructure actions such as previews, updates, destroys, and refreshes directly within Pulumi’s managed service. By bundling deployment settings like source code, cloud credentials, and environment variables into a single unit, Pulumi Deployments eliminates the complexities of stack management. Users can trigger deployments via Git events, through the Pulumi Cloud UI or leverage REST APIs for custom workflows, providing the flexibility to efficiently transition from code to cloud. In addition, customers using Pulumi Deployments can set up Drift Detection and Remediation, the ability to get notified when the state of your resources changes unexpectedly and optionally remediate it, as well as Time to Live Stacks, the ability to set a expiration on a stack when it will be destroyed.

Key Features

Git Push to Deploy: Automatically deploy infrastructure changes with each git push.
Click to Deploy: Deploy infrastructure with a single click from the Pulumi console.
REST API: Integrate Pulumi deployments with your applications using the REST API.
Automation API: Automate infrastructure provisioning and management with Pulumi’s Automation API.
Drift Detection: Identify and rectify configuration drift in your deployed stacks.
Time to Live Stacks: Automatically destroy stacks after a specified time period.
Scheduled Deployments: Schedule deployments to occur at specific times.
Review Stacks: Review and approve stack changes before deployment.

Get more details on these features in our Pulumi Deployments documentation. For more information on how to enable it in your self-hosted instance read the self-hosted Pulumi Deployments documentation.

Self-Hosted Changelog

The Self-Hosted Changelog is a newly introduced page that provides a comprehensive log of updates and changes specifically relevant to our self-hosted customers. This changelog ensures customers are fully informed about the latest improvements, bug fixes, and new features in the codebase and associated infrastructure. By keeping up-to-date of these updates, self-hosted customers can better manage their systems, anticipate and resolve potential issues, and take advantage of new functionalities as soon as they become available. Some examples of the types of recent updates on the changelog are adding support for expiration dates on access tokens or adding the ability to enable OTel metrics. This transparency not only helps in maintaining an optimal system but also empowers customers with the knowledge needed to leverage the full potential of our platform.

Wrapping It Up

The addition of Resource Search and Pulumi Deployments to our Self-Hosted Pulumi Cloud offering empowers you to manage your cloud resources more effectively and ensure the stability of your infrastructure. Whether you are looking to gain insights into your cloud usage or automate the detection and correction of configuration drift, these features provide the tools you need to succeed.

As of today, all Pulumi Cloud features that have been made generally available are supported in the self-hosted version. We intend to continue to have parity with Pulumi Cloud features in the self-hosted version within 6 months of making the feature generally available.

Try these new features today and experience the enhanced capabilities of the Self-Hosted Pulumi Cloud. For more information or to start a 30-day trial, contact us or visit our self-hosting documentation.

Announcing Short Lived Access Tokens in Pulumi Cloud

Meagan Cojocar — Tue, 25 Jun 2024 09:08:31 -0700

We are excited to introduce a new feature that our users have been eagerly awaiting: Short-Lived Access Tokens! This enhancement allows you to set an optional expiry date on Personal Access Token, Team Access Token, and Organization Access Token, making them automatically invalid after a specified date. This feature is now available in the Pulumi Cloud console and the Pulumi Cloud REST API, providing enhanced security and control over your access tokens.

Why Short-Lived Access Tokens?

Security and flexibility are paramount in managing infrastructure as code (IaC) and Environments, Secrets and Config (ESC), and our users have consistently requested the ability to create access tokens with a limited lifespan. With short-lived access tokens, you can:

Enhance Security: By limiting the lifespan of your access tokens, you reduce the risk of tokens being misused if they are inadvertently exposed.
Improve Automation: Automate token management in your CI/CD pipelines, ensuring that tokens expire as needed and can then be rotated.
Simplify Audits: Easily track and manage token expirations, ensuring compliance with your organization’s security policies.

How It Works

Creating Short-Lived Tokens in the console

When creating a new access token in the Pulumi Console, you will now see an optional expiry date field. Simply set the date when you want the token to expire, and it will automatically become invalid on that date. This feature is available for Personal Access Tokens, Team Access Tokens, and Organization Access Tokens (note: Team Access Tokens and Organization Access Tokens are only available in the Enterprise and Business Critical editions of Pulumi Cloud).

Learn more by reviewing our access token documentation.

OIDC Trust Relationships

A few months ago we released OpenID Connect (OIDC) Trust Relationships for Pulumi Cloud, allowing you to generate short-lived Pulumi Cloud tokens using OIDC from within your CI/CD pipelines, such as GitHub Actions. Now we are exposing the optional expiration field in the console and in the REST API. Your OIDC generated tokens will also show in the UI alongside those created with other methods.

REST API Support

We have also exposed this functionality in the Pulumi REST API, allowing you to programmatically create and manage short-lived tokens. This makes it easier to integrate token management into your automation scripts and tools.

# Example API request to create a short-lived token
curl \
 -H "Accept: application/vnd.pulumi+8" \
 -H "Content-Type: application/json" \
 -H "Authorization: token $PULUMI_ACCESS_TOKEN" \
 --request POST \
 --data '{"description": "{description}", "name": "{unique_name}", "expires": 0}' \
 https://api.pulumi.com/api/orgs/{org}/teams/{team}/tokens

Try It Today

We invite you to try out the new short lived access tokens in Pulumi Cloud. As always, we value your feedback and look forward to hearing how this feature helps streamline your workflows and enhances security.

For more details, check out our documentation and API reference docs. If you have any questions or need assistance, our support team is here to help.

Pulumi Google Cloud Classic 7.0

Monica Rodriguez — Fri, 10 Nov 2023 09:00:00 -0400

The latest major release of the Pulumi Google Cloud Classic Provider is available now! This 7.0 release contains the latest upstream changes to keep you up-to-date along with a highly requested bug fix, keeping your journey in managing Google Cloud resources fresh and smooth.

The Pulumi Google Cloud Classic provider can be used to provision any of the cloud resources available in the upstream provider. It is part of the suite of Pulumi official providers, which means that it is officially maintained and supported by Pulumi. The provider is also open source and available on GitHub for you to contribute and grow.

Join Pulumi on Thursday, November 30, 2023 for a live workshop on getting started with serverless apps on Google Cloud Run, which will feature the new GCP 7.0 provider.

If you are new to Pulumi and would like to get started with the Google Cloud provider, visit any of these pages to jump right in!

Getting Started - A guided walkthrough for creating your first project
Setup & Install - Instructions on installing the Google Cloud provider
How-to guides - Learn how to use the Google Cloud provider to provision specific resources
Templates - Use a quickstart template to create a new project
Pulumi Neo - Ask Pulumi Neo to create a new project

Looking back

Since the initial launch of 6.0 almost two years ago, we’ve shipped over 70 releases to deliver the latest upstream enhancements while also adding many of our own. Here are some of the highlights from these releases:

Coverage and Growth

The Google Cloud provider now supports 250+ functions, 800+ resources, 10,000+ input properties. It continues to grow in coverage and features while staying up-to-date with the latest upstream changes. The chart below shows the growth of the provider in the last 2 years.

New Modules

Version 6.0 of the Google Cloud provider introduced support for several new modules, including:

If you haven’t used these modules yet, now is a great time to try them out! Check out the Pulumi Registry for more information on how to get started or ask Pulumi Neo to create a new project for you.

What’s New in 7.0

Service Account Rename

This release includes a fix for the naming of the Service Account resource. For consistency and ease of use, serviceAccount has been renamed to serviceaccount. Node.js, Python, and .Net programs do not require any updates as the resource name has been aliased. However updates are required for Go, Java, and YAML to adjust the import paths and resource type.

- import "github.com/pulumi/pulumi-gcp/sdk/v6/go/gcp/serviceAccount"
+ import "github.com/pulumi/pulumi-gcp/sdk/v7/go/gcp/serviceaccount"

- import com.pulumi.gcp.serviceAccount.*;
+ import com.pulumi.gcp.serviceaccount.*;

- type: gcp:serviceAccount:Account
+ type: gcp:serviceaccount:Account

Labels Rework

Upstream changes for provider, resource, and data source labels are included in this release. Global Google Cloud Platform labels configured on the provider are supported through the new defaultLabels field. Default labels will be applied to all resources with a labels field. Resources that previously contained a single labels field will now contain three fields:

The labels field is now non-authoritative and only manages the label keys defined in your configuration for the resource.
The output-only pulumiLabels field merges the labels defined in the resource’s labels field and the global defaultLabels. If the same label key exists on both the resource level and provider level, the value on the resource will override the provider-level default.
The output-only effectiveLabels will list all the labels present on the resource in GCP, including labels configured through Pulumi.

Due to the two new output fields, a resource that has a labels field may show a diff when upgrading from pulumi-gcp v6 to v7. This diff reflects adding the new resource Output fields and is safe to do.

You can find more information about this change in the upstream documentation.

Upgrading

Upgrading from the previous version of the Google Cloud provider is straightforward. Simply update the version in your dependency manifest file, install the new package based on your language and package manager, then run pulumi up to migrate your existing stack. For Go programs, you will need to update specific imports within your program as well. This information is also available in our migration guide in the Pulumi Registry.

- "@pulumi/gcp": "^6.0.0"
+ "@pulumi/gcp": "^7.0.0"

- pulumi_gcp>=6.0.0,<7.0.0
+ pulumi_gcp>=7.0.0,<8.0.0

- require github.com/pulumi/pulumi-gcp/sdk/v6/go/pulumi v6.0.0
+ require github.com/pulumi/pulumi-gcp/sdk/v7/go/pulumi v7.0.0

- <PackageReference Include="Pulumi.Gcp" Version="6.*" />
+ <PackageReference Include="Pulumi.Gcp" Version="7.*" />

- com.pulumi:pulumi-gcp:6.0.0
+ com.pulumi:pulumi-gcp:7.0.0

Wrapping Up

As always, we welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is easy and free. Get started today!

Happy coding!

Pulumi Release Notes: AI in the CLI, Go generics, AWS 6.0, Review Stacks and so much more!

Monica Rodriguez — Fri, 29 Sep 2023 09:00:00 -0400

We’ve had an exciting quarter at Pulumi, shipping all kinds of improvements from our providers to our Cloud service. To stay up-to-date on all the details and additional improvements between release blogs, be sure to check out the pulumi/pulumi repo changelog for CLI enhancements and the Pulumi Cloud features in the new features blogs for updates on Pulumi Service features. We have a lot to cover, so let’s get started!

AI
- Resource Search - AI Assist
- Pulumi Insights and AI in the CLI
Pulumi Cloud
Core
Providers and Packages
Wrap Up

AI

Resource Search - AI Assist

Pulumi Cloud Resource Search AI assist is now generally available to all organizations, offering natural language search capabilities. This feature allows you to query their infrastructure with plain language requests. Suggested queries and an “I’m Feeling Lucky” button are now available to simplify the search process and provide insights into Resource Search functionality.

Pulumi Insights and AI in the CLI

We’ve integrated Pulumi Insights, including AI and Resource Search, into the pulumi CLI. Now, you can harness the power of AI and resource search directly in your terminal. This update also includes features like AI Assist for resource search and fine-grained resource property search, making these tools easily accessible via the pulumi CLI.

pulumi org search ai -q "all Kinesis streams modified since September 1st 2023"

pulumi ai web "deploy metabase on AWS with a managed postgres database"

Pulumi Cloud

Slack and Deployment Notifications

We’ve made several improvements to our Pulumi Cloud Webhooks to enhance deployment notifications and response times. Now, you can easily integrate Slack with Pulumi Cloud for organization or stack notifications without hosting their infrastructure. This integration provides centralized visibility, allowing teams to quickly access stack and update details, operation information, and user activity, streamlining collaboration and reducing context switching. Event notifications and filtering options have been expanded for both Pulumi Cloud Webhooks and the Slack integration, making deployment monitoring more efficient and accessible. These enhancements are available to all Pulumi Cloud organizations.

Microsoft Teams Webhook Integration

Setting up Microsoft Teams notifications is now easier than ever with support for Microsoft Teams formatted webhooks. You can quickly enable notifications about your Pulumi stacks and organizations in your Microsoft Teams workspace by providing an incoming webhook URL. New REST API endpoints and a Service Provider are available, allowing for efficient webhook creation at scale. These improvements enhance ChatOps workflows and provide better visibility into infrastructure changes while maintaining support for generic webhooks. For more details and how to get started, check out this blog post.

Admin Organization Access Tokens in Pulumi Cloud

The introduction of Admin permission scope for Organization Access Tokens provides elevated privileges for users who need them. These tokens enable actions like transferring stacks, modifying user access, team management, and more, previously restricted with regular Organization Access Tokens. Admin Organization Access Tokens cater to programmatic use cases that require higher-level permissions while maintaining a member’s usual permissions. This enhancement expands the utility of Organization Access Tokens for Enterprise and Business Critical customers, and is available during the 14-day trial period as well.

Restore Stacks

Pulumi Cloud now offers the capability to restore previously deleted stacks, a feature requested by many users. This addition enables users to recover stacks that were either mistakenly deleted or intentionally removed while retaining their activity history. This self-service recovery option can be conveniently accessed via the Pulumi Cloud console by navigating to the Stacks page, locating the “restore stacks” function, and selecting the stacks they wish to recover. Up to the last 10 deleted stacks can be restored, and older ones can be retrieved through support.

Review Stacks

New to Pulumi Deployments are Review Stacks, ephemeral cloud environments created automatically when a pull request is opened. These environments enable developers to make changes to both application and infrastructure code simultaneously, streamlining the development process. Review Stacks comment with the outputs from the deployment, making it easier to review code changes and collaborate with teammates. Review Stacks help optimize cloud costs by only existing as long as necessary during feature development. Configuring Review Stacks is a simple process that enhances development efficiency and collaboration among teams. Find more information in the Review Stacks documentation.

Property Search

Resource Search for Pulumi Cloud capabilities have been enhanced by allowing users to search over resource properties in addition to metadata, across over 100 cloud providers. The Property Search feature provides deeper insights into cloud infrastructure, enabling users to search based on specific property names and values, such as instance type, region, or database version. This new feature is available in the Enterprise and Business Critical editions of Pulumi Cloud and aims to streamline resource management, compliance, and diagnostics, ultimately enhancing efficiency and collaboration in cloud environments.

Natural language	Query result
resources tagged with stack production	`.tags.stack:production`
resources without a tag in production	`-tags: stack:production`
aurora engines running MySQL engine version 5	`type:"aws:rds/cluster:Cluster" .engine:aurora .engineVersion:5`
s3 buckets read write in production	`type:"aws:s3/bucket:Bucket" stack:production .acl:public-read-write`
buckets objects in all clouds	`type:"aws:s3/bucket:Bucket" OR type:"google:storage/bucket:Bucket" OR type:"azure:storage:BlobContainer"`
all resources in us east 1	`region:"us-east-1"`
resources with a private ip	`.privateIpAddress:`

Dependent Stack Updates with Pulumi Deployments

Pulumi Deployments has two new features to help manage complex infrastructure projects more easily. The Deployment Webhook Destinations allow automatic triggering of updates in dependent stacks when changes occur in the parent stack. We’ve also added the Pulumi Auto Deploy Package to simplify the way you express stack dependencies, automating the creation and updating of Deployment Webhooks. For all the details and to get started, visit the blog post, auto deploy package registry page, and Pulumi Deployments documentation.

Core

Go Generics

We’ve implemented Go generics in our SDKs, enabling type safety and reducing excessive code generation. This means you can write more reliable and maintainable infrastructure code with reduced chances of panics, leading to more robust cloud infrastructure deployments. For a deeper dive into this feature and how to get started, explore this blog post.

// Given
var a pulumi.IntOutput
var b pulumi.StringOutput

// Before (could panic at runtime if you got something wrong)
o := pulumi.All(a, b).ApplyT(func(vs []interface{}) string { // could panic
 a := vs[0].(int) // could panic
 b := vs[1].(string) // could panic
 return strconv.Itoa(a) + b
}).(pulumi.StringOutput) // could panic

// After (compile-time type-safety)
o := pulumix.Apply2(a, b, func(a int, b string) string {
 return strconv.Itoa(a) + b
})

Pulumi Convert

The introduction of pulumi convert in the Pulumi CLI simplifies the migration process for teams transitioning from Terraform to Pulumi. This feature allows you to convert entire Terraform projects into Pulumi, including support for modules, core Terraform 1.4 features, and most Terraform built-in functions. Starting from Pulumi CLI version 3.71.0, you can directly access this converter, streamlining the conversion of full Terraform programs. This enhancement builds upon our CrossCode foundations, enabling universal infrastructure as code support and introducing the concept of converter plugins for future integrations. This feature supports various use cases, such as migrating existing Terraform projects and modules to Pulumi, enhancing our commitment to providing versatile solutions for infrastructure as code.

$ pulumi convert --from terraform --language typescript --out pulumi

$ pulumi convert --from terraform --language python --out pulumi

$ pulumi convert --from terraform --language go --out pulumi

$ pulumi convert --from terraform --language csharp --out pulumi

Pulumi New support for SSH-style Git URLs

In the v3.77.0 release of the Pulumi CLI, we added support for pulumi new to use SSH-style Git URLs which enables accessing templates from private Git repos, authenticated via normal Git SSH support. Many teams have been building libraries of their own private templates, and this enables those templates to be used much more naturally via pulumi new and the Deploy with Pulumi button.

$ pulumi new git@github.com:acmecorp/templates/website

New Community Supported Pulumi Language - Scala

The team at Virtus Lab released support for Scala in Pulumi as part of their Besom project at https://github.com/VirtusLab/besom. As they highlight in their README:

Besom Scala SDK for Pulumi lets you leverage the full power of Pulumi Infrastructure as Code Platform using the Scala programming language. Scala support is currently in Public Beta.

The Scala language support builds on Pulumi’s existing Java and JVM support, but with an SDK custom designed to target idiomatic Scala API design patterns.

 // For each file in the directory, create an S3 object stored in `siteBucket`
 val uploads: Array[Output[s3.BucketObject]] = File(siteDir).listFiles().map { file =>
 val name = NonEmptyString(file.getName) match
 case Some(name) => name
 case None => throw new RuntimeException("Unexpected empty file name")
 s3.BucketObject(
 name,
 s3.BucketObjectArgs(
 bucket = siteBucket.id, // reference the s3.Bucket object
 source = FileAsset(file.getAbsolutePath), // use FileAsset to point to a file
 contentType = Files.probeContentType(file.toPath) // set the MIME type of the file
 ),
 CustomResourceOptions(
 dependsOn = siteBucket.map(List(_))
 )
 )
 }

Providers and Packages

AWS 6.0

The 6.0 release of the Pulumi AWS provider introduces essential enhancements and updates to one of the most extensively used providers in the Pulumi ecosystem. These improvements encompass TF Plugin Framework support, the inclusion of new resources and functions, significant SDK size optimizations for faster downloads and improved developer experiences, resolutions to top-upvoted issues, and removal of deprecated resources. To dig deeper into these changes, check out the migration guide and blog post.

Azure Native 2.0

The Pulumi Azure Native provider 2.0 has been shipped with enhanced performance and usability, making it easier to manage Azure resources efficiently. Key features include a significant reduction in SDK size for faster development, refreshed default module versions for access to the latest Azure features, expanded top-level default module coverage, title case consistency, simplified user-assigned identity inputs, and more. Visit the blog post and migration guide to learn more.

Kubernetes 4.0

Pulumi Kubernetes 4.0 brings new enhancements in managing Kubernetes resources. It embraces Kubernetes Server-Side Apply as the default mode, enabling features like upserting and Patch resources. Improved diffs remove the dependence on kubectl annotations, and provide simpler access to outputs in the Python, Go and Java SDKs, supporting any self-hosted or managed Kubernetes cluster with version 1.13 or newer. You can migrate existing projects or start new ones with Kubernetes 4.0 for superior Kubernetes resource management. Find more information in the blog post and migration guide.

OIDC Support for Azure

The Azure Native and Azure Classic providers now fully support Azure OIDC authentication. OIDC, or OpenID Connect. OIDC authentication allows you to authenticate your Pulumi program with Azure using federated identity, reducing the need for secrets management. You can establish the trust relationship between Azure and other services like GitHub and configure your Pulumi program to use OIDC for authentication, making your cloud infrastructure management more secure and efficient.

New provider resources

In addition to releasing major versions of 3 of our most popular providers, we’re always keeping our providers up-to-date to bring you the latest functionality in your Pulumi programs. We’ve added 793 resources and 762 functions across our most popular providers, along with many more throughout our ecosystem.

New Community Providers

Welcome to the 17 new community providers in the registry!

Wrap Up

We’ve had an incredible couple months introducing so many exciting features and enhancements across our entire platform. From Slack and Microsoft Teams integrations to Review Stacks, major provider release to Go Generics, we’ve been focused on making your cloud development experience more efficient and powerful. Don’t miss out on all these exciting developments—check out our changelog for CLI updates and our new features blogs for the latest on Pulumi Cloud. Explore these new features today and see the difference they can make in your projects. Happy coding!

Enhanced search & Navigation: The new Pulumi Docs experience

susan evans — Mon, 12 Jun 2023 00:00:00 +0000

Engineers spend a lot of their valuable time searching documentation for answers. At Pulumi, we believe in exceptional documentation experiences that help people using Pulumi find what they need quickly and use it successfully. Today, we are announcing a set of improved Pulumi documentation experiences that collectively make it easier than ever to discover, learn and build cloud infrastructure with Pulumi.

The improvements we’ve introduced to the Pulumi documentation experience focus on three areas - search, navigation and discoverability. Together, they help every user of the Pulumi documentation find the answers they are looking for quickly, so they can solve their infrastructure as code use case and leverage Pulumi effectively in their projects. These changes also introduce new design patterns and user experience elements across previously independent parts of the Pulumi documentation experience, creating a more cohesive developer experience.

Search

As of today, you can quickly and easily search for documentation across all of these types, and we will lead you to the documentation you are looking for to solve your problem. The new search clearly groups documentation by Docs and Packages with clear breadcrumbs to quickly get you to the right place.

We have unified navigation across both docs and packages and expect to add more content types to Pulumi Docs over the rest of the year. Enjoy easily switching between documentation types while having your left navigation and table of contents navigation remain consistent.

Discoverability

In addition to search and navigation, folks must be able to discover new content that will help them even if they don’t know that content exists. To that end, let’s showcase a few types of Pulumi documentation:

Quickstart experience

The Pulumi quickstart experience is for people new to Pulumi to get up and running quickly.

Cloud get started tutorials

The Pulumi cloud get started tutorials are for people who want a more detailed walkthrough when using Pulumi for the first time.

Concept docs

The Pulumi concept docs are where you can dive into Pulumi concepts. Learn how they work together and how to effectively use them to ship infrastructure.

Packages

The package documentation is where to find configuration and API documentation for Pulumi supported clouds, version control systems, databases, infrastructure monitoring, and networking tools. We support 119 providers today, and we add new providers every week!

Head to Pulumi docs, learn something new, and send us your feedback!

Pulumi Release Notes: Resource Search, Deployment Actions, Projects in Self-Managed Backends, and more

Mikhail Shilkov — Wed, 24 May 2023 00:00:00 +0000

We have been busy shipping improvements in the last 2 months. Let’s walk through the release highlights across Pulumi engineering areas from March and April. If you want to learn more between release blogs, follow the CLI improvements in the pulumi/pulumi repo changelog and Pulumi Cloud features in the new features blogs.

Quick shortcuts to jump to your favorite area:

Pulumi CLI
Pulumi Automation API
- Manage Stack Tags
- Full “Who Am I” command support
Cloud Providers and Packages
Pulumi Cloud & Pulumi.com

Pulumi CLI

Aligning Projects between Service and Self-Managed Backends

While the majority of Pulumi users do choose to use Pulumi Cloud, we also know that there are good reasons why some organizations would prefer to use Pulumi IaC alone without Pulumi Cloud. The self-managed backends have historically stored all stacks in a single namespace across all projects.

With the 3.61.0 release of Pulumi, we’ve added support for project-scoped stacks to the Self-Managed backend. Stacks created in new or empty self-managed backends will automatically use the new project-scoped layout.

👉 Learn more in the Aligning Projects between Service and Self-Managed Backends blog post.

Improved Interactive Scrolling Experience

We’ve improved the user experience of deploying a large number of resources interactively in the CLI. The recent changes make two improvements to scrolling through a long list of action:

The interactive display can now be scrolled using the page-down and page-up keys. These keys scroll the display N lines at a time, where N is the height of the terminal.
If the display is scrolled to the bottom, it will now autoscroll it as new lines are added to the output.

👉 Learn more in the Improve interactive scrolling pull request.

Shortcut to See the Current Update in Web Browser

While a CLI preview or update operation is running, you can now use the Ctrl-O keyboard shortcut to open the same update in Pulumi Cloud using your web browser.

The shortcut and the Pulumi Cloud URL are shown at the top of the update operation:

$ pulumi up
Previewing update (dev)
View in Browser (Ctrl+O): https://app.pulumi.com/mikhailshilkov/dev-ts/dev/previews/01234567-ec03-4ccc-9b44-02ec864e14ee
Type Name Plan
+ pulumi:pulumi:Stack ts-dev create...

👉 Learn more in the Open in Web Browser shortcut pull request.

New Flag to Specify Teams for a Newly Created Stack

We’ve added a --teams flag to pulumi stack init command. The flag accepts a string and can be provided multiple times. Each team that is provided is assigned read/write permissions on the stack after it has been initialized.

$ pulumi stack init dev-stack --teams devs

👉 Learn more in the Add –teams flag to assign team name to stack pull request.

Visual Indication of Retained Resources

You can use the RetainOnDelete resource option to let Pulumi retain the actual cloud resource while deleting it from your Pulumi stack. This option is useful, for instance, when you need to start managing the same resource from another stack or project.

Now, the CLI will explicitly mark deleted or replaced resources as retained in the cloud. See the Plan column of the preview view:

👉 Learn more in the Retain display pull request.

View More Details in Stack History

Sometimes, to troubleshoot a past action, you would want to see more details about the context of a past CLI operation. We’ve added more details to pulumi stack history command output, so that you could see the names of CLI parameters, the names of environment variables, the version of the Pulumi CLI, and OS architecture:

No sensitive information like parameter or environment variable values is captured or displayed.

👉 Learn more in the Add more details to pulumi stack history pull request.

Pulumi Automation API

Manage Stack Tags

We’ve added support to set, get, list, and remove tags on Pulumi stacks. Shoutout to Matthew Rodrigues for contributions in this area!

👉 Learn more in the Ensure Parity for Stack Tags issue.

Full “Who Am I” command support

The pulumi whoami CLI command now added a --json flag. The same flag is consumed from the Automation API to return user’s organization and URL in addition to the user name.

Another shoutout to Matthew Rodrigues for the contribution!

👉 Learn more in the WhoAmI JSON flag pull request.

Cloud Providers and Packages

Every week we keep shipping provider updates. A few highlights on our native providers:

Kubernetes shipped support for v1.27.0 and all its resources (changelog)
Azure Native shipped six new resources for containerstorage, guestconfiguration, and hybridcloud (releases)
AWS Native shipped 33 new resources in apigatewayv2, appconfig, chatbot, comprehend, connect, datasync, devopsguru, ec2, frauddetector, lambda, msk, neptune, quicksight, ram, sagemaker, ssmcontacts, and vpclattice modules (changelog)

Two new community packages were added to the Registry:

AzAPI - shoutout to Engin Diri
StrongDM - shoutout to Piers Karsenbarg

Pulumi Cloud & Pulumi.com

Resource Search Improvements

More Advanced Filtering

Based on customer feedback we have extended the number of advanced filtering options (Stack, Project, Package and Type) to show the top 10 results by resource count instead of top 3. We know our customers are getting a ton of value from these aggregates to inform and refine their searches and expect the 7 additional values to improve this further.

Pulumi Teams support

Customers who use Pulumi Teams for role based access control on Pulumi Cloud can now filter resources by team access. We have added a new aggregate on the Resource Search page for Teams, allowing you to see at a glance the teams in your organization with the most resources. You can query which teams have access to what resources, track resource usage by team by exporting your resource data and see in the search results table which teams have access to that resource.

Events Dashboard card

We have added a new card to the organizations dashboard which shows upcoming Pulumi events and Pulumi workshops to give users a central place to see events of interest.

Stacks Page Icons

We have added icons on the Stacks page to enable you to see at a glance information about your Pulumi Stacks, such as if Policies are being applied, if Pulumi Deployments is configured for that stack and the resource count. In addition, there is now a quick action menu on each stack to jump to configuring Pulumi Deployments and adding it to a Policy.

Deployment Actions

You can now view the Pulumi Deployments Click-to-Deploy menu on all stack pages, even if not configured to Pulumi Deployments. This allows users to quickly navigate to Deployments Settings to connect the stack.

SAML Admin

For customers on the Enterprise or Business Critical Editions of Pulumi Cloud there is a new role for a SAML admin in your organization. SAML Admins can add another identity to authenticate when SAML is the only permitting authentication method. This could be an email and password or another authentication method, like GitHub. We are slowly rolling this out across our customer base, if it is not yet in your organization you should expect it in the coming weeks.

Pulumi Insights: Intelligence for Cloud Infrastructure

Luke Hoban — Thu, 13 Apr 2023 06:00:00 -0800

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

You can now get Insights capabilities on all your cloud infrastructure, even resources not managed by Pulumi IaC. Get started with Insights 2.0

There are new intelligent cloud management capabilities available in Pulumi Copilot. Learn More

We’ve seen incredible acceleration of cloud adoption over the past 5 years. Pulumi’s flagship open source IaC solution gives engineers great tools to scale up their cloud infrastructure using the same programming languages and tools they already know and love. As a result, thousands of companies of every size and scale have adopted Pulumi as a lynchpin of their cloud infrastructure strategy.

Today we’re excited to announce Pulumi Insights, the next major productivity enhancement for infrastructure as code. Pulumi Insights provides intelligence, search, and analytics over any infrastructure, in any cloud across your organization, leveraging the latest advances in generative AI and Large Language Models (LLMs). Whether you have an AWS VPC, a Kubernetes CRD, or a DataDog alarm definition, Pulumi Insights enables you to intelligently find and interact with all of your resources from within the Pulumi Cloud.

Pulumi Insights was born from partnering with our community and observing their struggles to make sense of millions of cloud resources and resource updates per month, across hundreds of clouds, regions, and accounts. With Pulumi Insights, we now help teams manage cloud footprints of any size with the following key components:

🧠 Intelligence: New AI-powered tool for learning, discovering and building cloud infrastructure with Pulumi, using the power of generative AI and Large Language Models (LLMs).
🔎 Search: Multi-cloud resource search, powered by the Pulumi resource supergraph, which surfaces rich metadata and relationships between cloud infrastructure.
📊 Analytics: Visualize Pulumi resource data with your favorite BI tools, including Excel, Tableau and others, using your own data warehouse for reporting on cost, compliance, and other operational use cases.

All of these features work across Pulumi’s 130+ supported cloud providers, including AWS, Azure, Google Cloud, Cloudflare, and Kubernetes, and can be defined in any programming language, including TypeScript, Python, .NET, Go, Java and YAML.

Pulumi Insights is available for all Pulumi Cloud users today.

Intelligence - Productivity with the Power of AI

The scale of Cloud APIs is often intimidating for new and even seasoned cloud engineers. At Pulumi, we provide access to hundreds of clouds, tens-of-thousands of resources and hundreds-of-thousands of resource properties. Pulumi has many thousands of examples showing how to solve specific problems, and yet developers still sometimes struggle to find the perfect code snippet that addresses their exact set of requirements.

Our new Pulumi Neo is an assistant purpose-built for creating cloud infrastructure using Pulumi. It builds on the power of Large Language Models and GPT to dramatically reduce the time it takes to discover, learn and use new cloud infrastructure APIs. Because Pulumi uses popular programming languages that GPT has already been trained on, Pulumi AI understands how to solve complex Cloud Infrastructure problems.

Using natural language, you can ask to accomplish a specific Cloud Infrastructure goal, and receive a Pulumi program in your favorite language. In addition, Pulumi AI supports iterating on your cloud infrastructure allowing the addition of new features, improving security and performance, fixing correctness issues and clarifying requirements.

The open source pulumi-ai CLI takes these ideas even further, by interactively deploying any cloud infrastructure you request using natural language prompts. You can build arbitrarily complex cloud infrastructure, without requiring detailed knowledge of specific APIs! Of course, since we’re using Pulumi’s Automation API under the hood, you can go to the Pulumi Cloud console to see everything that has been deployed, and even jump straight into your Cloud Console to inspect the results.

Pulumi AI will enhance your team’s ability to go from concept to product. This demo shows Pulumi AI taking natural language input and deploying cloud infrastructure and services which are exposed live on the internet, constantly updating as you refine your request.

Check out Pulumi Neo today at pulumi.com/neo.

Search - Find Anything in Any Cloud

Teams don’t use just a single cloud or a single cloud account. Indeed, many teams manage resources across dozens of clouds and hundreds of accounts! It can be almost impossible to aggregate and search your infrastructure across these accounts, since every cloud and every account is (by design!) a walled garden.

Pulumi Resource Search offers multi-cloud search and analytics across every environment in your organization. You can issue queries that find all of your AWS VPCs, or all of your VPCs in AWS and Azure, or all resources with the “production” tag across all cloud environments. You also have access to statistics about cloud usage, including a breakdown by cloud provider, resource type, and department. Resource search enables you to find the needle within your cloud haystack and visualize cloud consumption trends.

Infrastructure can be discovered interactively with a rich, structured query language, for example:

package:snowflake
data-warehouse stack:production package:aws modified:>=2023-03-21
stack:production type:”github:index/issuelabel:IssueLabel”
package:aws project:pulumi-cloud-import-aws stack:-staging-us-west-2

Search is powered by the Pulumi resource supergraph, which exposes rich metadata around cloud infrastructure resources and models relationships between resources, such as the project and stack they are deployed from, the components they are part of, the provider used to manage them. Pulumi enables faceted search, which allows users to specify criteria by applying multiple filters based on classification of resources.

AI Assist for Search

We are also making available today a private preview of AI Assist for resource search that enables natural language queries. You can provide a plain English (or Spanish, or French, or Japanese!) request, to express queries where you might not know the exact syntax, type tokens, or package names. AI Assist makes it easier to gain insight over infrastructure. You can join the waitlist now for access to this additional functionality for search.

Cloud Import

While all resources managed by Pulumi are part of the resource supergraph, you can also bring resources defined and managed outside of Pulumi into the graph to make them available for Insights. We’re also opening up a private preview of Pulumi Cloud Import, a feature that lets you apply Search and Insights to resources created outside of Pulumi, whether via ClickOps or through other tools like Terraform, CloudFormation, ARM or Kubernetes YAML. You can join the waitlist for access to these new features today.

Analytics - Deeper Insights for Cloud Infrastructure

We are also making the data in the Pulumi supergraph available for export into other Business Intelligence tools for deeper analysis in combination with other key business data. You can visualize resource data with your favorite BI tools, including Excel, Tableau, Looker, PowerBI and others, using your own data warehouse, such as Snowflake, Redshift, Databricks and more, and create reporting on cost, compliance, and many other operational use cases.

The ability to join Pulumi resource data with other key business data in your data warehouse enables users to address a broad range of key business needs - answering questions like:

What infrastructure is stale and possibly orphaned / contributing to our cost?
Where are we spending the most, broken down by department or environment?
Are all my primary data stores marked as protected to avoid unintentional deletion?
What are my most commonly used clouds? What are the most common resource types?
Who has performed the most updates across my organization?

For example, here is a dashboard in Metabase, created by combining Pulumi resource data with cost and department data in a Snowflake data warehouse.

Pulumi resource Data Export is available today both interactively in the console and via REST API . Learn more about these features in the Pulumi Data Export documentation.

Conclusion

We’d like to express our thanks to the entire community that helped us design, develop, test and ship this incredible release. From Intelligence to Search to Analytics, the new features in Pulumi Insights will make all of us more productive. We’re incredibly excited by the use cases we’ve already seen our customers build, and we can’t wait to see what you build on top of these foundations!

To get started with Pulumi Insights today:

Sign up or log in to Pulumi Cloud
Try out Pulumi Neo
Watch the videos for Pulumi Insights, Pulumi AI and Pulumi Search
Check out the Pulumi Insights Documentation
Join the Pulumi Community Slack

Aligning Projects between Service and DIY Backend

Justin Van Patten — Mon, 03 Apr 2023 00:00:00 +0000

At Pulumi, our goal is to offer the best Infrastructure as Code experience for all cloud developers. From the very beginning, we’ve believed that the best IaC experience is made possible by combining a great open source SDK and CLI with a great backend management service. This is why we built Pulumi Cloud, a rich management platform for your Infrastructure as Code, which includes a forever free option for individuals, a generous free tier for teams, and critical tools for enterprises to manage IaC at scale.

Over the last few years, we’ve continued to expand the features of the Pulumi Service - with Deployments, Audit Logs, SAML SSO and SCIM, Teams, Stack Transfers, Favorites, Organization and Team Access Tokens and much more.

While the majority of Pulumi users do choose to use the Pulumi Service, we also know that there are good reasons why some organizations would prefer to use Pulumi IaC alone without the Pulumi Service. And so we support and continue to invest in enabling a variety of additional backends that allow the Pulumi CLI to be used with state stored in the local filesystem or in cloud storage like S3, Azure Blob Storage, or Google Cloud Storage.

Historically the Pulumi Service backend and the DIY backend have differed in their handling of “projects”. The Pulumi Service stores state for a Pulumi stack in a separate namespace per project. The DIY backends have historically stored all stacks in a single namespace across all projects. This inconsistency has been a common source of confusion for users getting started with Pulumi when using the file storage backends.

Today, we are aligning how projects are managed across all backends, adding Project-Scoped Stacks support to DIY backends.

Background

This section goes over some of the background necessary to understand project-scoping and why it’s important. Feel free to skip ahead to the next section if you’re already familiar with this.

Projects and stacks

Pulumi projects are a unit of organization. A project holds the code and configuration for your infrastructure, and can be used to deploy multiple stacks. A stack is a single, isolated instance of your infrastructure.

For example, you might have an “app” project for your application, deploying to the “test” and “prod” stacks.

What’s a state backend?

Each stack tracks metadata about your infrastructure so it can manage your cloud resources. This metadata is called state and it is stored in a backend of your choosing: Service or DIY.

The Service backend is hosted at app.pulumi.com or on your own server. It provides the best combination of usability, safety, and security for most users.

The DIY backend stores state in a JSON file, deferring to you to manage persistence of this information. You can store it in Amazon S3, Azure Blob Store, Google Cloud Storage, a system such as Minio or Ceph with an API compatible with Amazon S3, or on your local filesystem. Using this backend trades reliability and convenience of the Service backend for additional control over where the information is stored.

If you’re unsure of which backend to use, see Deciding On a State Backend.

Project-scoped stacks

Depending on your needs, you can have several Pulumi projects. For example, you might have a “net” project to set up an Amazon VPC, a “k8s” project to set up a Kubernetes cluster deployed to it, and an “app” project to build and deploy your application to the cluster. Each of these projects will have their own stacks, e.g. “test” and “prod”.

The Pulumi Service backend scopes stacks to the project they belong to, so we can re-use the names “test” and “prod” across different projects.

This was not the case for the DIY backend, where stack names were global across all projects. To work around this, users resorted to qualifying stack names in the form <project>-<stack> when using the DIY backend.

We’re pleased to announce that such workarounds are no longer necessary!

Project-scoped stacks in DIY backends

With the latest release of Pulumi, we’ve added support for project-scoped stacks to the DIY backend. Stacks created in new or empty DIY backends will automatically use the new project-scoped layout.

With this change, you can now use stack names like “test” and “prod” across your projects without risk of conflict—the same way you do with the Service backend.

Upgrading existing backends

Only stacks created in new or empty DIY backends will scope stacks to projects. Existing backends will continue to operate as they do today.

If you have an existing DIY backend that you’d like to upgrade to use project-scoped stacks, you can do so with the new pulumi state upgrade command, which upgrades all stacks in the backend to use the new project-scoped layout.

$ pulumi state upgrade
This will upgrade the current backend to the latest supported version.
Older versions of Pulumi will not be able to read the new format.
Are you sure you want to proceed?
Please confirm that this is what you'd like to do by typing `yes`:

Once stacks in a DIY backend are upgraded, older versions of the Pulumi CLI will be unable to access those stacks. This change cannot be reverted.

Cleaning up old stack names

If you previously qualified stack names with the project name to avoid conflicts, you can use the pulumi stack rename command to clean up these names after the upgrade.

$ pulumi stack rename --stack app-test test

Referencing stacks in the CLI

Many commands in the Pulumi CLI expect a stack name to be specified. These commands accept a fully-qualified stack name in the form:

<org>/<project>/<stack>

Where, <org> is the organization name, <project> the project name, and <stack> the stack name.

They also accept one of the following shorthand forms if we’re inside the project directory:

<org>/<stack>
<stack>

For instance, given a stack “prod” inside the “app” project, the following are all valid ways to refer to it:

my-org/app/prod
my-org/prod // if inside the app directory
prod // if inside the app directory

With the latest release of Pulumi, the above is true for both the Service backend and the DIY backend.

Referencing DIY project-scoped stacks

All projects in a DIY backend are placed under a virtual organization named “organization”. This value is constant and cannot be changed.

Repeating the above examples with a DIY backend, we can use any of the following forms to refer to the same stack:

organization/app/prod
organization/prod // if inside the app directory
prod // if inside the app directory

Availability

Support for project-scoped stacks in DIY backends is now available in Pulumi v3.61.0. Give it a try and share your thoughts with us on the Pulumi Community Slack!

Pulumi Release Notes: Colorized Stack Traces, Output Methods for JSON, OIDC integration, and much more!

Mikhail Shilkov — Wed, 22 Mar 2023 00:00:00 +0000

We have been busy shipping improvements in the last 2 months. Let’s walk through the release highlights across Pulumi engineering areas from January and February. If you want to learn more between release blogs, follow the CLI improvements in the pulumi/pulumi repo changelog and Pulumi Service features in the new features blogs.

Quick shortcuts to jump to your favorite area:

Pulumi CLI
Pulumi Language SDKs
Pulumi Automation API
- Adding tags to stacks through Automation API
- Support cloning from Azure DevOps in Automation API
Cloud Providers and Packages
Pulumi Service & Pulumi.com
- OIDC integration
- Pulumi Deployments

Pulumi CLI

Colorized stack traces for Node.js errors

When an error happens in a Pulumi program, stack traces can be instrumental to understand the reasons behind the failure. Stack traces can often be large and contain frames unrelated to user code, which makes debugging harder.

We have now changed the CLI to display Node.js stack traces with colors, where the important bits are highlighted and less important parts faded out. The new display technique helps draw attention to relevant details and is similar to running vanilla node programs.

👉 Learn more in the Emit colorized stack traces pull request.

New –shell option for `pulumi stack output`

We added a new --shell option to the pulumi stack output command that writes the outputs for a stack as a shell script:

% pulumi stack output --shell
bucketName=mybucket-1234
websiteURL="http://mybucket-1234.example.com"

On Windows, it will produce a Powershell script:

% pulumi stack output --shell
$bucketName = 'mybucket-1234'
$websiteURL = 'http://mybucket-1234.example.com'

As a result, Pulumi users are now able to write shell scripts like the one below:

eval "$(pulumi stack output --shell)"
echo "Created bucket $bucketName"
# => Created bucket mybucket-1234

echo "Visit at $websiteURL"
# => Visit at http://mybucket-1234.example.com

👉 Learn more in the Add –shell option pull request.

`pulumi new` in directories with source control

By default, pulumi new won’t run unless the current directory is empty to avoid overriding any existing files. This behavior meant that the command failed in directories with initialized source control.

We shipped an improvement where Pulumi CLI will ignore .git, .hg, and .bzr files and directories when considering whether a directory is empty.

👉 Learn more in the Ignore VCS directories pull request.

Support plugins hosted in GitLab

Pulumi comes with a wealth of packages but users can also create and use custom providers and components. By default, Pulumi expects the binaries to be hosted on GitHub. On top of that, the CLI now supports GitLab as a first-class source of package downloads.

If your package binary is hosted on GitLab, you can now set its pluginDownloadURL property (see Publish your package) to point to your GitLab project, e.g. to gitlab://gitlab.com/43429536.

👉 Learn more in the Support custom plugins from other sources issue.

Use access tokens to authenticate to Google Cloud backend

When storing the state files in Google Cloud backend, you can now authenticate to the Google Cloud Storage using Google access tokens. If the environment variable GOOGLE_OAUTH_ACCESS_TOKEN is set during a pulumi login gs://..., Pulumi will utilize the provided token to login to the bucket, assuming this has not been preempted by another authentication type.

👉 Learn more in the Pulumi login to support google oauth access tokens for GCS buckets pull request.

Delete temporary files from plugin downloads

While a plugin is being installed, the CLI downloads the necessary archive and stores the archive in /tmp/pulumi-plugin-tar*. The CLI is now more mindful of cleaning the temporary file after plugin installation completes to save disk space.

👉 Learn more in the Delete temporary files from plugin downloads pull request.

“Yes, using Update Plans” prompt removed

Update Plans is a preview feature to constrain the update only to the operations that were planned during preview. To get extra feedback from our users, we experimented with showing an extra option “yes, using Update Plans” at the update prompt. The experiment is now complete, so we removed the option.

Lean more in the Remove the “yes, using Update Plans” prompt issue.

Pulumi Language SDKs

Output methods to serialize and deserialize JSON

Building JSON from values and parsing strings into JSON are two of the most common operations in Pulumi programs. All programming languages have built-in libraries to do so. However, quite often JSON operations need to be combined with Pulumi Inputs and Outputs, which required writing a cumbersome combination of Apply and serialization method calls.

To streamline this experience, Pulumi SDKs shipped helper serialization and deserialization functions that understand the type Output<T> natively. Here is a simple usage example in Python, where table.arn is an output coming from another resource:

policy = iam.Policy(
 "iam-policy",
 policy= Output.json_dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": [
 "dynamodb:PutItem",
 "dynamodb:GetItem"
 ],
 "Effect": "Allow",
 "Resource": [table.arn]
 }]
 }))

👉 Learn more in the Output methods issue that also tracks a number of further additions to the toolset of helper methods.

Retrieve stack reference outputs as plain values

You can read outputs from one stack in another stack’s program using Stack References. In the past, the resulting values were always wrapped inside an Output<T> container in order to preserve secret values (when needed).

With recent SDKs, we shipped an alternative method StackReference.getOutputDetails. This method returns a plain object with two fields: value and secretValue. At most one of these fields is set, depending on whether the stack reference output is a secret or not.

If you know that the output is not a secret, you can write a program to use the plain value immediately, for example, to iterate though a list in a loop and create resources in each iteration.

👉 Learn more for each language:

Native arm64 binaries in Python SDK

Pulumi Python SDK has been upgraded to use the grpcio library version 1.51.3, which comes with a wheel with native arm64 binaries (universal2). The change eases the installation of the Pulumi Python SDK on Apple M1/M2 machines.

👉 Learn more in the Python grpc 1.51.3 contains universal2 binaries pull request.

Go version raised to 1.18

The minimum version of Go required by Pulumi was raised to 1.18. Raising this directive allows us to use language features like generics in the codbase. The change is applied to both pkg and sdk.

👉 Learn more in the go.mod: Raise to ‘go 1.18 pull request.

Delegate alias computation to the engine

Sometimes, two classes in Pulumi SDKs represent the same cloud resource. To enable migration from one to another, Pulumi sets aliases for those classes and declares them to be compatible.

In the past, each SDK was responsible for calculating all possible alias combinations for a given resource and its inherited parent. We recently added support in the engine for this computation along with an optimization to reduce complexity from a multiplicative factor to a linear one.

As a follow-up step, we updated our SDKs to only provide the base aliases, and allow the engine to do this computation. This enables smaller SDKs and faster runtime performance, especially for longer chains of parented resources. Other than that, the user experience hasn’t changed.

👉 Learn more in the Delegate alias computation from SDKs to Engine issue.

Pulumi Automation API

Adding tags to stacks through Automation API

You can now use Automation API to get, set, list, or remove tags on stacks programmatically. Here is a quick example in TypeScript:

const workspace = await LocalWorkspace.create({
 projectSettings: projectSettings,
});
await workspace.createStack(stackName);
await workspace.setTag(stackName, "foo", "bar");
const tags = await workspace.listTags(stackName);
// tags will contain tags for `pulumi:project`, `pulumi:runtime`, and `foo` keys.

👉 Learn more in the Add support for adding tags to stacks through Automation API issue.

Support cloning from Azure DevOps in Automation API

We implemented a workaround to support Azure DevOps repositories when driving Pulumi from Automation API in Go. This is now also utilized in the Pulumi Kubernetes Operator.

👉 Learn more in the Support clones from Azure DevOps pull request.

Cloud Providers and Packages

Every week we keep shipping provider updates. Two highlights on our native providers:

Azure Native shipped 20 new resources, including support for networkcloud and voiceservices (changelog)
AWS Native shipped 30 new resources in apigatewayv2, appflow, applicationautoscaling, cloudtrail, connect, directoryservice, ec2, fms, gamecast, gamelift, kendraranking, networkmanager, omics, organizations, sagemaker, secretsmanager, simspaceweaver, and systemsmanagersap modules (changelog)

Five new community packages were added to the Registry: Vultr, Zscaler Private Access (ZPA), Zscaler Internet Access (ZIA), Statuscake, and Nuage.

Pulumi Service & Pulumi.com

After launching Pulumi Deployments a few months ago and receiving a ton of community feedback (thank you!) we launched an integration with OpenID Connect (OIDC) to enable security first deployments and industry best practices.

OpenID Connect (OIDC) is an identity layer built on top of the OAuth 2.0 framework. It allows third-party applications to verify the identity of the end-user and to obtain basic user profile information. OIDC uses JSON web tokens (JWTs), which you can obtain using flows conforming to the OAuth 2.0 specifications.

Overview

The main benefits of using the OIDC integration for Pulumi Deployments are as follows:

Temporary credentials: You can move away from long term credentials within your organization to leverage temporary credentials that are generated every time a deployment runs. The Pulumi Service issues a new OIDC token that is limited to that deployment. This token is used to obtain cloud credentials that automatically expire, meaning you do not need to store sensitive credentials that are reused with each deployment.
Granular access control: The OIDC token generated by the Pulumi Service contains detailed information about the deployment with which it is associated. In AWS for example, you can configure our OIDC integration to only use temporary credentials if that token is associated with a particular Pulumi stack or Pulumi organization. This enables you to follow the principle of least privilege access and scope access down to only what is needed. For AWS, Azure, and GCP, Pulumi Deployments can automatically exchange its OIDC token for temporary credentials and make those credentials available to providers. For advanced scenarios or other cloud providers, the OIDC token can be manually exchanged for credentials by the user.

👉 Learn more on how to get started with Pulumi Deployments OIDC by reading the blog post.

If you haven’t yet tried out Pulumi Deployments

You should now see a “Deploy” tab under Stack Settings in the Pulumi Service console and have access to the Deployments REST API. View the documentation for help getting started and join our Slack (#pulumi-deployments channel) for questions and feedback.

Managing credentials with Open ID Connect for Pulumi Deployments

Pat Gavlin — Fri, 06 Jan 2023 11:10:50 -0800

After launching Pulumi Deployments a few months ago and receiving a ton of community feedback (thank you!) today we are launching an integration with OpenID Connect (OIDC) to enable temporary credentials and granular access controls. We would like to thank GitHub user eriklz for creating the original request for this functionality.

Overview

The main benefits of using the OIDC integration for Pulumi Deployments are as follows:

Temporary credentials: You can move away from long term credentials within your organization to leverage temporary credentials that are generated every time a deployment runs. The Pulumi Service issues a new OIDC token that is limited to that deployment. This token is used to obtain cloud credentials that automatically expire, meaning you do not need to store sensitive credentials that are reused with each deployment.
Granular access control: The OIDC token generated by the Pulumi Service contains detailed information about the deployment with which it is associated. In AWS for example, you can configure our OIDC integration to only use temporary credentials if that token is associated with a particular Pulumi stack or Pulumi organization. This enables you to follow the principle of least privilege access and scope access down to only what is needed. For AWS, Azure, and GCP, Pulumi Deployments can automatically exchange its OIDC token for temporary credentials and make those credentials available to providers. For advanced scenarios or other cloud providers, the OIDC token can be manually exchanged for credentials by the user.

Setup

Calling the Deployments REST API

You can call the Deployments REST API and use OIDC instead of cloud secrets. Here is an example of how that might look:

Enabling OIDC for your Stack for Git Push to Deploy

In order to use the Git Push to Deploy and Click to Deploy features in the Pulumi Service console you will need to enable OIDC for your stack in the console. In order to enable it you will need to be an admin of your Pulumi organization.

Navigate to your stack in the Pulumi Console.
Open the stack’s “Settings” tab.
Choose the “Deploy” panel.
Under the “OpenID Connect” header, toggle to your cloud provider
Follow the cloud provider specific docs below

Viewing Deployment logs

Once OIDC is set up you will see a new step in your Deployment logs “Fetch credentials via OIDC”

Getting access to Pulumi Deployments

Pulumi Deployments is available in preview today. Deployments are free during the preview, with usage-based pricing to come. Check out these resources to get started today:

Request access to the Pulumi Deployments preview
Pulumi Deployments examples repo
Pulumi Deployments documentation
Pulumi Deployments REST API documentation
Automation API examples

Please submit an issue in the Pulumi Service Requests repo if you have feature requests for the Pulumi Service or join our Slack channel(#pulumi-deployments) if you have questions.

🚀🚀🚀 Happy building! 🚀🚀🚀

Pulumi Release Notes: CED Launches, Skip Checkpoints flag, Automation API NodeJS parallel inline programs, and much more!

Meagan Cojocar — Tue, 08 Nov 2022 00:00:00 +0000

In addition to our Cloud Engineering Days launches, we have been busy shipping improvements in the last 2 months. Let’s walk through the release highlights across Pulumi engineering areas from September and October. If you want to learn more between release blogs, follow the CLI improvements in the pulumi/pulumi repo changelog and Pulumi Service features in the new features blogs.

Cloud Engineering Days Launches
Cloud Providers and Packages
- New Resources in our Providers
- Pulumi+Kubernetes Improvements
Pulumi CLI and core technologies
Pulumi Service & Pulumi.com
- Pulumi Deployments
- Architecture Templates Support

Cloud Engineering Days Launches

In the Pulumi Cloud Engineering Days 2022 we are announced a set of important new advancements in the Pulumi platform designed to help organizations scale with their infrastructure as code needs. These announcements were as follows:

Pulumi Deployments: The Pulumi Service now supports executing infrastructure deployments as a managed service, driven by click-to-deploy in the Console, Git push-to-deploy from GitHub, programmatic deployments via the Pulumi Service REST API as well as automation via the Pulumi Automation API. Pulumi Deployments enables organizations to scale up their infrastructure automation. Learn more in the Pulumi Service section below!
Pulumi YAML GA: In addition to supporting rich general purpose programming languages like TypeScript, Python, Go, .NET and Java, Pulumi added support last May for Pulumi YAML as a simple, declarative interface for accessing the full breadth of the Pulumi Universal Infrastructure as Code platform. Pulumi YAML is now Generally Available as part of its 1.0 release, enabling everyone in an organization to use Pulumi to manage cloud infrastructure.
Architecture Templates: New templates providing architecture-specific starting points for the most common infrastructure as code use cases - Containerized Services, Kubernetes Clusters, Serverless Applications, Static Websites and Virtual Machines - available across all major cloud platforms and supported languages.
Project Level Configuration: As cloud infrastructure projects grow in complexity, Pulumi provides many tools to manage the complexity of the software that defines these projects. But the configuration of that software also grows in complexity. New support for Project-Level Configuration across all Pulumi languages makes it easier to manage complex configuration as cloud projects scale up.

👉 Learn more in the Cloud Engineering Days Announcements blog.

Cloud Providers and Packages

New resources in our providers

We shipped new versions of the AWS Native provider, Google Native provider and the Azure Native provider that added support for 698 new resources in the last two months. 17 new resources were added to Google Native, 662 when including IAM binding and IAM member resources, 29 resources were added to the AWS Native provider and 7 were added to the Azure Native provider.

Pulumi+Kubernetes Improvements

In October we announced a set of major updates which deepen and extend Pulumi’s support for Kubernetes and the Kubernetes ecosystem. Here are a few of these exciting enhancements:

Pulumi Kubernetes Operator v1.10: New integration with Flux for richer GitOps support, and ability to deploy Pulumi stacks from directly within the Kubernetes resource model
New Pulumi Provider for Flux: Manage Flux with Infrastructure as Code
Pulumi Kubernetes Provider v3.22: Server Side Apply Option and Resource Patch

👉 Learn more in the Pulumi+Kubernetes: New Flux Integration and Inline Programs blog.

Pulumi CLI and core technologies

Pulumi CLI now displays time elapsed per resource

When you modify a resource the Pulumi CLI will display the time elapsed so you can determine how long the modification elapsed for. This change helps surface for each resource the time it takes to create or modify allowing you to plan for it next time. See it in action below!

👉 Learn more in the Elapsed Time per Resource blog post and in the resource creation times GitHub issue.

Automation API NodeJS parallel inline programs

Automation API now supports parallel execution of NodeJS inline programs in addition to the existing Python inline program support. Now users can run multiple NodeJS Pulumi programs in parallel when they are using Automation API. It uses NodeJS AsyncLocalContext to isolate state away from process environments and into context-specific stores owned by LocalWorkspaces. This functionality is also supported in Python and .NET.

👉 Learn more in the support parallel execution of NodeJS inline programs GitHub issue.

New –remove flag for `pulumi destroy`

A pulumi destroy operation destroys all existing resources in the stack, but not the stack itself. If you then wanted to delete the stack itself, once all the resources were destroyed, you would run a [pulumi stack rm]. A community member, mrod-io added a flag for pulumi destroy for when you want to remove the stack after its resources are destroyed: pulumi destroy --remove.

See it in action below:

meagancojocar@Meagans-MacBook-Pro demo % pulumi destroy --remove
Previewing destroy (demo/dev)

View Live: https://app.pulumi.com/meagan/demo/dev/previews/632658ed-41e0-411d-b8a8-f41cc6122ef4

 Type Name Plan
 - pulumi:pulumi:Stack dev delete
 - └─ aws:s3:Bucket my-bucket delete

Outputs:
 - bucketName: "my-bucket-ad305ba"

Resources:
 - 2 to delete

Do you want to perform this destroy? yes
Destroying (demo/dev)

👉 Learn more in the Add –remove as an option for pulumi destroy GitHub issue.

Add Output.format to the Python SDK

We added Output.format to the Python SDK to match helper methods in our other SDKs. Most languages have a standard way to format a string (String.Format in C#, fmt.Sprintf in Go, etc) but you can’t use those functions with our Output values. We have added a format method that’s similar to each language’s normal format method, but with support for outputs. We have added Output.format in Python which matches the standard Python str.format method. This is similar to how we have interpolate in TypeScript, and Output.Format in C#.

👉 Learn more in the Add Output.format to python SDK pull request.

Cloning Pulumi templates from Azure DevOps

We now support cloning Pulumi templates (pulumi new) from Azure DevOps repositories. When given a URL with a hostname of dev.azure.com, it will use system-installed Git and will clone the master/main branch.

Skip Checkpoints Experimental Flag

We have added a new experimental flag to improve performance for certain use cases. This flag makes Pulumi skip saving state checkpoints as it modifies resources and instead only save the final state of a deployment. This is an experimental feature that also requires an PULUMI_EXPERIMENTAL=true environment variable to be set. Using the feature introduces risk that in the case of network disconnect or crash state edits will be lost and may require manual recovery. When this risk is acceptable, using the feature can speed up Pulumi deployment times. As it is in experimental we may make changes to how it has been implemented down the track.

👉 Learn more by reviewing the Skip checkpoints pull request.

Token Authentication in Go Providers

Thank you to community member @aohoy for adding support for token authentication in the Go providers that use Git. You can now authenticated requests by tokens such as https://auth:token@gitlab.example.com/group/proj.git/tree/v0.0.0/path.

👉 Learn more by reviewing the Add token to Git Go module pull request.

Bump TypeScript version from v3.7.3 to v3.8.3

We updated the vendored version of TypeScript in the NodeJS SDK and runtime from v.3.7.3 to v3.8.3. Updating TypeScript versions enables you to use newer TypeScript features, such as the import type and export type syntax, which is required to use OpenTelemetry.

👉 Learn more by reviewing the Bump TSC version to 3.8.3 pull request.

Support lazy-loading Node modules

We now have opt-in support for generating lazy load code for Node modules. Modules that define functions and resources are loaded lazily when you opt-in. Generated code may use import type .. form when importing enums (“useTypeOnlyReferences” opt-in flag in schema). We plan to lazy-load Node modules in Azure Mative and other large Pulumi Packages to improve performance when using these libraries.

👉 Learn more by reviewing the Implement support for lazy-loaded Node modules pull request.

Pulumi Service & Pulumi.com

Pulumi Deployments

With Pulumi Deployments, you can run a Pulumi infrastructure as code action (a preview, update, destroy or refresh) inside Pulumi’s managed service. Pulumi provides scalability, observability, and security for deployments. It consists of five components:

The Deployments REST API - A fully managed REST API to execute Pulumi programs with the Pulumi Service. This includes APIs to observe your deployment and all associated logs.
Deployment Visualization - A dashboard for deployment observability including deployment status, logs, and queueing via the Pulumi Service console.
Click to Deploy - Deploy infrastructure with the click of a button from the Pulumi Service console.
Git push to Deploy - Configure automatic infrastructure deployments in response to git push events by installing and configuring the Pulumi GitHub App.
Remote Automation API - Our strongly typed infrastructure orchestration technology can now run deployments remotely in the cloud.

Click to Deploy

Pulumi Deployments removes the cloud as a bottleneck and unlocks it as an accelerant for your business. It is available in preview today. Deployments are free during the preview, with usage-based pricing to come. Check out these resources to get started today:

Request access to the Pulumi Deployments preview
The Pulumi Deployments demo video
Pulumi Deployments documentation
Pulumi Deployments REST API documentation
Automation API examples

👉 Learn more in the Pulumi Deployments launch blog post.

Architecture Templates Support

We added Architecture Templates support to the new project experience in the Pulumi Service console. Now in addition to selecting your cloud provider and language of choice you can select a template for your new project. We currently have Static Website, Serverless, Container Service, Virtual Machine, Kubernetes cluster and web application templates, with more to come!

Announcing: Pulumi Deployments, YAML GA, Arch Templates

Luke Hoban — Wed, 02 Nov 2022 06:15:00 -0700

Some of the code in this post is out of date. See the AWS guides for an updated overview and examples.

Our mission at Pulumi is to enable teams to scale up what they can build in the cloud. Scale up the sophistication and value of their cloud infrastructure investments through software engineering practices. Scale up the automation around delivering cloud infrastructure with software instead of just humans. And scale up the number of developers who can directly benefit from the rich cloud platform capabilities being built by central platform teams in every organization today.

As part of Pulumi Cloud Engineering Days 2022 today we are announcing a set of important new advancements in the Pulumi platform which are all designed to help organizations scale with their infrastructure as code needs.

Pulumi Deployments: The Pulumi Service now supports executing infrastructure deployments as a managed service, driven by click-to-deploy in the Console, Git push-to-deploy from GitHub, programmatic deployments via the Pulumi Service REST API as well as automation via the Pulumi Automation API. Pulumi Deployments enables organizations to scale up their infrastructure automation.
Pulumi YAML GA: In addition to supporting rich general purpose programming languages like TypeScript, Python, Go, .NET and Java, Pulumi added support last May for Pulumi YAML as a simple, declarative interface for accessing the full breadth of the Pulumi Universal Infrastructure as Code platform. Pulumi YAML is now Generally Available as part of its 1.0 release, enabling everyone in an organization to use Pulumi to manage cloud infrastructure.
Architecture Templates: New templates providing architecture-specific starting points for the most common infrastructure as code use cases - Containerized Services, Kubernetes Clusters, Serverless Applications, Static Websites and Virtual Machines - available across all major cloud platforms and supported languages.
Project Level Configuration: As cloud infrastructure projects grow in complexity, Pulumi provides many tools to manage the complexity of the software that defines these projects. But the configuration of that software also grows in complexity. New support for Project-Level Configuration across all Pulumi languages makes it easier to manage complex configuration as cloud projects scale up.

You can check out the Cloud Engineering Days 2022 Keynote to learn more about all of these features, or get started with Pulumi today!

Pulumi Deployments

We launched the Pulumi Automation API two years ago to enable organizations to build their own automated systems using the Pulumi Infrastructure as Code engine. Over the last two years, we’ve seen incredible adoption of Automation API, with millions of infrastructure updates driven by Automation API, and usage in everything from CI systems, to CLI tools, to self-service provisioning platforms, to complex deployment orchestration software.

To scale up how we deploy and manage cloud infrastructure and ultimately unlock the cloud’s value and agility, we believe it will be critical to build software systems around our Infrastructure as Code platforms - systems that scale with software, not just humans.

Today, we are taking the next step in this journey with the introduction of Pulumi Deployments to the Pulumi Service. With Pulumi Deployments, you can run a Pulumi infrastructure as code action (a preview, update, destroy or refresh) inside Pulumi’s managed service. Pulumi provides scalability, observability and security for deployments.

Pulumi Deployments are exposed as a new fundamental capability of the service via the Pulumi Service REST API. They are also now supported in Automation API as a new kind of Remote workspace that can be used in building your own custom infrastructure automation software.

We’ve also used Pulumi Deployments to build new features in the Pulumi Service console, including the ability to click to deploy infrastructure or to set up Git push to deploy workflows that trigger previews on GitHub PR commits and updates on merges to a tracking branch. This provides a simple point-and-click interface to ensure that any Pulumi user can benefit from continuous delivery of infrastructure changes.

Of course, because Pulumi Deployments exposes a convenient programmatic API, you can build your own automation on top of this. In our Pulumi Deploy Demos, we have examples of building Drift Detection automation, Time-to-live self-destruct stacks, per-PR environments and more.

Pulumi Deployments is available in preview now, and you can signup to get access today. Pulumi Deployments is free while in preview. We can’t wait to see what you’ll build on top of Pulumi Deployments! 🚀

👉 Read more in the Pulumi Deployments blog post.

Pulumi YAML

We introduced Pulumi YAML last May as a simple declarative interface to the full breadth of the Pulumi platform, ideal for smaller scale use cases and composition of higher level component building blocks. Developers who may not have been ready to learn or adopt a full blown language for their cloud infrastructure can use YAML to get started.

Our goal has always been to offer the broadest range of programming language options to empower every cloud builder so that they could benefit from the best of Pulumi’s Universal Infrastructure as Code platform.

Pulumi YAML makes it easy to compose any of the tens-of-thousands of resources available to the Pulumi Universal Infrastructure as Code platform, including mixtures of raw cloud resources (like an AWS Lambda Function below) and higher level components authored in other rich Pulumi languages (like the aws-apigateway packages high level RestAPI below).

name: serverlessapp
runtime: yaml
config:
 roleArn:
 type: string
resources:
 fn:
 type: aws:lambda:Function
 properties:
 runtime: python3.9
 handler: handler.handler
 role: ${roleArn}
 code:
 fn::fileArchive: ./function
 api:
 type: aws-apigateway:RestAPI
 properties:
 routes:
 - path: /
 localPath: www
 - path: /date
 method: GET
 eventHandler: ${fn}
outputs:
 url: ${api.url}

Pulumi YAML is the only declarative Infrastructure as Code language that supports convert - the ability to convert a YAML program into a program in any other Pulumi language (TypeScript, Python, Go, .NET and Java). If the complexity of a project ever grows beyond what YAML can comfortably express, you can seamlessly adopt another richer language to continue to scale.

pulumi convert --language python

With the Pulumi YAML GA, we’ve made dozens of improvements to Pulumi YAML. Richer type checking and error reporting makes Pulumi YAML easy to get started with. And IDE integration for VS Code and Emacs makes Pulumi YAML even more productive to work with day-to-day:

👉 Read more in the Pulumi YAML GA blog post.

Architecture Templates

The new Pulumi Architecture Templates offer the easiest way to get started with a complete working piece of infrastructure for one of the most common Infrastructure as Code use cases. Each architecture template is short (~100 lines of code), builds a deployable infrastructure project out of the box, is easy to customize, uses a mix of raw cloud resources and Pulumi components to operate at the right level of abstraction for the use case, and comes with useful documentation.

The initial set of architecture templates support common infrastructure as code use cases across AWS, Azure, Google Cloud and Kubernetes, and in all supported Pulumi languages.

For example, the AWS Containerized Service template provides this starting point, which builds and deploys a local container-based application into a set of managed infrastructure in AWS using ECS, ALB, ECR and IAM resources:

from pulumi import Config, Output, export
import pulumi_aws as aws
import pulumi_awsx as awsx

config = Config()
container_port = config.get_int("containerPort", 80)
cpu = config.get_int("cpu", 512)
memory = config.get_int("memory", 128)

# An ECS cluster to deploy into
cluster = aws.ecs.Cluster("cluster")

# An ALB to serve the container endpoint to the internet
loadbalancer = awsx.lb.ApplicationLoadBalancer("loadbalancer")

# An ECR repository to store our application's container image
repo = awsx.ecr.Repository("repo")

# Build and publish our application's container image from ./app to the ECR repository
image = awsx.ecr.Image(
 "image",
 repository_url=repo.url,
 path="./app")

# Deploy an ECS Service on Fargate to host the application container
service = awsx.ecs.FargateService(
 "service",
 cluster=cluster.arn,
 task_definition_args=awsx.ecs.FargateServiceTaskDefinitionArgs(
 container=awsx.ecs.TaskDefinitionContainerDefinitionArgs(
 image=image.image_uri,
 cpu=cpu,
 memory=memory,
 essential=True,
 port_mappings=[awsx.ecs.TaskDefinitionPortMappingArgs(
 container_port=container_port,
 target_group=loadbalancer.default_target_group,
 )],
 ),
 ))

# The URL at which the container's HTTP endpoint will be available
export("url", Output.concat("http://", loadbalancer.load_balancer.dns_name))

Pulumi Architecture Templates build on Pulumi’s support for creating your own templates. Pulumi programs can be templated by users to create their own starting points for cloud infrastructure projects using pulumi new. You can build your own templates and Deploy with Pulumi buttons to create similar architecture templates within your own team and organization.

👉 Read more in the Pulumi Architecture Templates blog post.

Project Level Configuration

On top of support for defining infrastructure as code using traditional programming languages, Pulumi has always offered a rich configuration system for specifying inputs to a program which might differ between instances of that program (“stacks”) - for example, across the dev, staging and prod environments, or across regional deployments, of what is logically the “same” infrastructure definition. Configuration in Pulumi has so far only been provided at the stack level - each stack provides the complete set of configuration needed to specify how that instance of the Pulumi program will be parameterized.

With the latest Pulumi release, we are excited to be adding Project Level Configuration to Pulumi as well. With project-level configuration, you can specify the configuration needed for each stack once, inside the project file. This specification can offer default values across all stacks, types to enforce validation of values used in each stack, descriptions, and enforcement that certain properties must be secret.

name: baseinfra
runtime: python
config:
 vm-count:
 type: integer
 description: the number of virtual machines to create
 default: 2
 subnets:
 type: array
 description: an array of subnets to create
 items:
 type: string
 vaultKey:
 type: string
 description: which vault key to use for this stack
 secret: true

This is the first step in our broader goals to bring Hierarchical Configuration to Pulumi, avoiding unnecessary copy/paste, simple abstraction of configuration data, and rich validation and contextual user interface support across all aspects of config in the Pulumi experience.

👉 Read more in the Project Level Configuration blog post.

Conclusion

Pulumi helps cloud engineering teams scale up what they can accomplish in the cloud. Today, we announced a collection of new features and enhancements that advance that platform and help you scale your cloud infrastructure teams. These features give platform team new powers to scale up what they can accomplish.

Check out the launch blog posts for each of the new features, sign up for access to Pulumi Deployments, or just dive right in and get started with Pulumi today!

Pulumi Blog: Releases

Now in Public Preview: Store Terraform State in Pulumi Cloud

Why this matters

What you get

How it works

Get started

Now GA: Up to 20x Faster Pulumi Operations for Everyone

What this means for you

Production results

What you need to do

New in Pulumi IaC: `onError` Resource Hook

Recovering from errors

Next steps

New in Pulumi IaC: `replacementTrigger` Resource Option

Forcing replacements

Defining replacement triggers

Next steps

Introducing the Stash Resource in Pulumi IaC

Why Stash?

Using Stash

When to replace

Learn more

New in Pulumi IaC: `replaceWith` Resource Option

Custom resource dependencies

Declaring our own dependencies

In code

Next steps

All Pulumi CLI flags are now supported as environment variables

Use cases

How it works

Using with direnv for project-level defaults

Wrapping up

Pulumi Google Cloud Provider Version 9.0.0

In case you missed it - new modules

What’s New in 9.0

Upstream Improvements

Improved Import Validation

Increased Field Validation

Documentation Improvements

Upgrading

NEW! Use Pulumi Neo to assist with your upgrade

Azure Native 3.8: Unified Credentials and Private Clouds

Simplified Authentication Across Environments

What’s New

Real-World Benefits

Getting Started

Private Azure Cloud Support

How To Use

Disabling Instance Discovery

How To Use

Authentication in AKS with Workload Identity

Walkthrough

Create an AKS Cluster

Install the Pulumi Kubernetes Operator

Configure a Pulumi Cloud Access Token

Update the Kubernetes Service Account

Define an Example Program

Run the Program using Workload Identity

Conclusion

Precise Resource Replacement with Pulumi State Taint

The New Commands

Why This Matters

Workflow

Get Started

Resource hooks

Hooks in action

Another example: application health checks

Where can we go from here?

The Pulumi Go Provider SDK is Now Generally Available

Unlock the Full Potential of Your Cloud Ecosystem

Key Enhancements for Your Team

Defining and Testing a Custom Resource

Creating a Custom Resource in Minutes, Not Days

Confidence Through Comprehensive Testing

Standardize Infrastructure with Powerful Component Resources

Seamless Multi-Language Support and Sharing

Understanding Provider Schema

Get Started Today!

New in Pulumi IaC: Support for skipping a resource

The challenge: partial operations on large stacks

The solution: introducing `--exclude` and `--exclude-dependents`