Advanced Automation API Patterns

Once you’ve built a first Automation API program by following Using Automation API, you’ll often need to go further: coordinate multiple stacks instead of one, tear resources down safely and in the right order, recover from failures gracefully, and manage the workspaces that back them. This guide covers those patterns with runnable TypeScript and Python examples. For terminology used throughout—Workspace, Stack, inline versus local programs—see Automation API concepts.

Prerequisites

The examples below assume the same setup as the getting-started guide: the Pulumi CLI on your PATH (or installed programmatically), the Node.js or Python runtime for your chosen language, and a Pulumi access token or another configured state backend.

Some examples below orchestrate multiple Pulumi projects from a single Automation API program. Each project—network and app in the examples—is an ordinary Pulumi project directory with its own Pulumi.yaml, deployed as a local program rather than an inline one. That’s a deliberate choice for these scenarios: an orchestrator managing multiple independently developed projects typically doesn’t own their program code, so it drives them by path instead of importing their logic as a function.

Cross-stack teardown ordering

When one stack’s program reads another stack’s outputs through a StackReference, the two stacks become coupled: the dependent stack’s resources may need values—a VPC ID, a subnet, a security group—that only exist while the referenced stack’s resources still exist. Automation API doesn’t infer this coupling for you, so destroying stacks in the wrong order can fail mid-destroy or, worse, leave orphaned resources behind.

The examples below use selectStack/select_stack and createOrSelectStack/create_or_select_stack directly; see Create, select, and create-or-select later in this guide for how those functions differ and when to reach for each.

Consider an app stack that reads networking details from a network stack:

// app/index.ts -- an ordinary Pulumi program, not an Automation API program.
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const config = new pulumi.Config();
const networkStackName = config.require("networkStackName");

const network = new pulumi.StackReference(networkStackName);
const vpcId = network.getOutput("vpcId");
const subnetIds = network.getOutput("subnetIds");

const server = new aws.ec2.Instance("app-server", {
    ami: "ami-0c55b159cbfafe1f0",
    instanceType: "t3.micro",
    subnetId: subnetIds.apply((ids: string[]) => ids[0]),
});

export const serverId = server.id;

# app/__main__.py -- an ordinary Pulumi program, not an Automation API program.
import pulumi
import pulumi_aws as aws

config = pulumi.Config()
network_stack_name = config.require("network_stack_name")

network = pulumi.StackReference(network_stack_name)
vpc_id = network.get_output("vpc_id")
subnet_ids = network.get_output("subnet_ids")

server = aws.ec2.Instance("app-server",
    ami="ami-0c55b159cbfafe1f0",
    instance_type="t3.micro",
    subnet_id=subnet_ids.apply(lambda ids: ids[0]))

pulumi.export("server_id", server.id)

Because app depends on network’s outputs, you must always destroy app before network—the reverse of the order you deployed them in. An orchestrator program can encode that rule directly instead of relying on whoever runs it to remember it:

import { LocalWorkspace } from "@pulumi/pulumi/automation";

async function selectStack(workDir: string, stackName: string) {
    return LocalWorkspace.selectStack({ stackName, workDir });
}

async function teardownEnvironment(envName: string) {
    const appStack = await selectStack("./app", `app-${envName}`);
    const networkStack = await selectStack("./network", `network-${envName}`);

    // Destroy the dependent stack first. If network's resources disappear
    // while app still references them, the app destroy can fail partway
    // through, leaving some of its resources behind.
    console.info(`Destroying app-${envName}...`);
    await appStack.destroy({ onOutput: console.info });

    console.info(`Destroying network-${envName}...`);
    await networkStack.destroy({ onOutput: console.info });
}

from pulumi import automation as auto


def select_stack(work_dir: str, stack_name: str) -> auto.Stack:
    return auto.select_stack(stack_name=stack_name, work_dir=work_dir)


def teardown_environment(env_name: str) -> None:
    app_stack = select_stack("./app", f"app-{env_name}")
    network_stack = select_stack("./network", f"network-{env_name}")

    # Destroy the dependent stack first. If network's resources disappear
    # while app still references them, the app destroy can fail partway
    # through, leaving some of its resources behind.
    print(f"Destroying app-{env_name}...")
    app_stack.destroy(on_output=print)

    print(f"Destroying network-{env_name}...")
    network_stack.destroy(on_output=print)

For a scenario with more than two stacks, build the same reverse-order list you used to deploy them and iterate over it: [app, cache, network] at teardown time for a [network, cache, app] deployment order, destroying each stack in turn before moving to the next.

Multi-stack orchestration

Automation API is well suited to deploying the same program across many environments—development, staging, production—or many regions, without hand-writing a stack for each one. Represent each target as data, then loop over it, creating or selecting the corresponding stack, and applying its own configuration:

import { LocalWorkspace, UpResult } from "@pulumi/pulumi/automation";

interface EnvironmentConfig {
    stackName: string;
    awsRegion: string;
}

const environments: EnvironmentConfig[] = [
    { stackName: "dev", awsRegion: "us-west-2" },
    { stackName: "staging", awsRegion: "us-east-1" },
    { stackName: "production", awsRegion: "us-east-1" },
];

async function deployAll(): Promise<void> {
    for (const env of environments) {
        const stack = await LocalWorkspace.createOrSelectStack({
            stackName: env.stackName,
            workDir: "./webapp",
        });

        await stack.setConfig("aws:region", { value: env.awsRegion });

        console.info(`Updating stack ${env.stackName}...`);
        const result: UpResult = await stack.up({ onOutput: console.info });

        const changes = result.summary.resourceChanges ?? {};
        console.info(
            `Stack ${env.stackName} updated: ${changes.same ?? 0} unchanged, ` +
            `${changes.create ?? 0} created, ${changes.update ?? 0} updated.`,
        );
    }
}

from pulumi import automation as auto

environments = [
    {"stack_name": "dev", "aws_region": "us-west-2"},
    {"stack_name": "staging", "aws_region": "us-east-1"},
    {"stack_name": "production", "aws_region": "us-east-1"},
]


def deploy_all() -> None:
    for env in environments:
        stack = auto.create_or_select_stack(
            stack_name=env["stack_name"],
            work_dir="./webapp",
        )

        stack.set_config("aws:region", auto.ConfigValue(value=env["aws_region"]))

        print(f"Updating stack {env['stack_name']}...")
        result = stack.up(on_output=print)

        changes = result.summary.resource_changes or {}
        print(
            f"Stack {env['stack_name']} updated: {changes.get('same', 0)} unchanged, "
            f"{changes.get('create', 0)} created, {changes.get('update', 0)} updated."
        )

CI/CD integration pattern

A common way to run this kind of program in a pipeline is to promote sequentially: deploy to dev, and only proceed to staging and production if it succeeds. Wrap each stack’s up in error handling (see the next section) and stop the loop—rather than continuing to the next environment—the first time a deployment fails, so a broken change never reaches production. See Continuous delivery for how to wire an Automation API program like this one into your CI/CD system.

Error handling and observability

Automation API surfaces failures as exceptions rather than nonzero exit codes, and it gives you structured results instead of text you’d otherwise have to parse from CLI output. Both make an orchestrator program considerably easier to operate than a script that shells out to pulumi.

Streaming output

The onOutput (on_output) callback you’ve seen throughout this guide receives the same incremental output a user watching pulumi up in a terminal would see. Pass any function that accepts a string—writing it to your own structured logger instead of stdout is often more useful in a service context than in a script:

const result = await stack.up({
    onOutput: (chunk: string) => logger.info("pulumi", { output: chunk.trimEnd() }),
});

result = stack.up(on_output=lambda chunk: logger.info("pulumi", extra={"output": chunk.rstrip()}))

Reading structured results

up, destroy, preview, and refresh all return a result object with a summary describing the update—its resulting permalink URL, per-resource-operation counts, and duration—plus, for up, the stack’s outputs. Use these instead of scraping console text to drive decisions in your program:

const result = await stack.up({ onOutput: console.info });

if (result.summary.result !== "succeeded") {
    throw new Error(`Update did not succeed: ${result.summary.result}`);
}

console.info(`Website URL: ${result.outputs.websiteUrl.value}`);

result = stack.up(on_output=print)

if result.summary.result != "succeeded":
    raise RuntimeError(f"Update did not succeed: {result.summary.result}")

print(f"Website URL: {result.outputs['website_url'].value}")

Catching engine errors

Failed updates raise a language-native exception rather than only returning a failed summary. Catch it around each stack operation so one failing stack doesn’t take down an orchestrator managing many others:

import { CommandError } from "@pulumi/pulumi/automation";

try {
    await stack.up({ onOutput: console.info });
} catch (err) {
    if (err instanceof CommandError) {
        console.error(`Update failed for stack ${env.stackName}: ${err.message}`);
        // Record the failure, alert, or stop promoting to later environments,
        // depending on how your orchestrator should react.
    } else {
        throw err;
    }
}

from pulumi.automation import CommandError

try:
    stack.up(on_output=print)
except CommandError as err:
    print(f"Update failed for stack {env['stack_name']}: {err}")
    # Record the failure, alert, or stop promoting to later environments,
    # depending on how your orchestrator should react.

For transient failures—a provider’s API rate-limiting you, a momentary network blip—wrap the call in your own retry loop with backoff rather than retrying blindly; a CommandError from a genuine configuration or program error will fail again immediately and shouldn’t be retried the same way.

Workspace lifecycle management

Every Stack operates against a Workspace, and how you construct that workspace determines where Pulumi looks for your program, your project settings, and your stack configuration.

Choosing how to associate a program

Automation API gives you three ways to associate a stack with a program, and the convenience functions (createStack, selectStack, createOrSelectStack) all accept either shape:

• Local program (workDir/work_dir): points at an existing project directory with its own Pulumi.yaml, exactly like a program you’d run with the CLI. Use this when the program already exists as a standalone project—the orchestration examples earlier in this guide use this style because they drive existing network and app projects.
• Inline program (program): a function defined in the same process as your Automation API code, with no Pulumi.yaml of its own. Use this when the Automation API program is the deployment tool—for example, a small CLI or service that both defines and deploys the infrastructure with no separate project to maintain. See Using Automation API for an inline example.
• Remote program (RemoteWorkspace): runs a program from a remote Git repository through Pulumi Deployments rather than on the machine executing your Automation API code.

Create, select, and create-or-select

The three convenience functions differ only in what they assume about the stack’s existence: createStack fails if the stack already exists, selectStack fails if it doesn’t, and createOrSelectStack does whichever is needed. Prefer createOrSelectStack for idempotent orchestrators that may run against a mix of new and existing environments—most of the examples in this guide use it for that reason. Reach for createStack or selectStack directly when you want the corresponding failure mode: createStack to guarantee you never silently reuse an existing stack’s state, selectStack (as in the teardown example above) to guarantee you never accidentally create one that shouldn’t exist yet.

Reusing workspaces across stacks

Creating a new Workspace for every stack works, but it’s wasteful when you’re operating on many stacks that share the same program: each LocalWorkspace you construct re-reads Pulumi.yaml and re-resolves the CLI. When you already have a Stack object, reuse its workspace property to select a sibling stack in the same project instead of building a new workspace from scratch:

import { LocalWorkspace, Stack } from "@pulumi/pulumi/automation";

const devStack = await LocalWorkspace.createOrSelectStack({
    stackName: "dev",
    workDir: "./webapp",
});

// Reuse the same workspace---and therefore the same Pulumi.yaml and
// CLI resolution---to operate on a sibling stack in the same project.
const stagingStack = await Stack.select("staging", devStack.workspace);

from pulumi.automation import Stack
from pulumi import automation as auto

dev_stack = auto.create_or_select_stack(stack_name="dev", work_dir="./webapp")

# Reuse the same workspace---and therefore the same Pulumi.yaml and
# CLI resolution---to operate on a sibling stack in the same project.
staging_stack = Stack.select("staging", dev_stack.workspace)

Cleaning up workspace and stack state

Automation API’s Workspace methods for removing state mirror the CLI commands they drive, and it’s worth being precise about which one you want:

• stack.destroy() tears down the stack’s deployed resources but leaves the stack itself—and its history—registered with your state backend. Use this for routine teardown, including the ordering pattern described earlier in this guide.
• workspace.removeStack(stackName) (remove_stack) deletes the stack’s registration and configuration entirely. Run this only after the stack’s resources are already destroyed—removing a stack that still has resources orphans them, since nothing is tracking them anymore.

await stack.destroy({ onOutput: console.info });
await stack.workspace.removeStack(stackName);

stack.destroy(on_output=print)
stack.workspace.remove_stack(stack_name)

For temporary environments—per-branch preview environments or integration test stacks, for example—pair these two calls so every stack you create programmatically is also fully cleaned up programmatically, rather than accumulating stale stacks in your state backend.

Next steps

• Review Automation API concepts for the underlying Workspace, Stack, and program model these patterns build on.
• See Using Automation API if you haven’t yet built a first Automation API program.
• Explore the automation-api-examples repository for complete, runnable programs in every supported language.