Pulumi Blog: Aws

Fully Automated AI Inference on AWS, Azure, and Google Cloud with Pulumi

Engin Diri — Tue, 30 Jun 2026 00:00:00 +0000

Putting Ollama on a cloud GPU is something I keep coming back to. A while ago I wrote up running open-source LLMs on an AWS EC2 box with Ollama and Pulumi, and the shape never really changes: a GPU instance, a model server, and a firewall rule in front. Infrastructure as code earned its place by making that kind of setup predictable and repeatable, and AI infrastructure is no exception. A GPU box serving a model is still a VM, a disk, and a firewall rule, and it should be declared like one.

Thorsten Hans made exactly that case in his Akamai post, Fully Automated AI Infrastructures with Terraform and Akamai Cloud, which stands up a single GPU instance on Linode, installs the drivers, runs Ollama, and pulls a model, with no manual steps after terraform apply.

I liked the shape of it, so this post ports the same idea to Pulumi and runs it across AWS, Azure, and Google Cloud instead of one. The result is one program shape per cloud: a single pulumi up brings up a GPU box that installs its own driver, runs Ollama, and pulls a model with no manual steps, and a single pulumi destroy takes it back down. Along the way it drops the two imperative bits the Terraform version leans on: a static access token sitting in an environment variable, and a null_resource running a shell loop to wait for the model. The first becomes an OIDC login from a Pulumi ESC environment, so no long-lived key lives anywhere. The second turns out not to be a resource at all.

What you are building

Strip away the per-cloud naming and every version of this is the same three things: a GPU virtual machine, a firewall in front of it, and a cloud-init script that turns a bare Ubuntu box into a running inference server. The model serving runs on Ollama, which exposes an HTTP API on port 11434 and keeps the model resident in GPU memory between requests.

flowchart LR
Dev([Your machine / curl]) -->|"HTTP :11434"| FW["Firewall / security group<br/>(allow 11434, optional 22)"]
FW --> VM["GPU VM (Ubuntu 24.04)<br/>NVIDIA driver + Ollama"]
VM -->|"resident in GPU memory"| Model["qwen2.5:14b"]
ESC["Pulumi ESC<br/>pulumi-idp/auth"] -.->|"OIDC login (short-lived creds)"| Pulumi["pulumi up"]
Pulumi -.->|"AWS / Azure / GCP"| VM

Component	Port	Description
GPU VM	-	Ubuntu 24.04 with an NVIDIA T4-class GPU. Installs the driver and Ollama from cloud-init, then pulls the model.
Ollama	11434	Serves the model over an HTTP API and keeps it in GPU memory between calls.
Firewall / security group	-	Allows inbound `11434` (and `22` when you ask for it). Open to the world for a demo; lock the CIDR down for anything real.
`pulumi-idp/auth` (ESC)	-	One environment that brokers an OIDC login into AWS, Azure, and GCP. No static keys in code or CI.

One detail here is worth pausing on, and it explains why the Pulumi version comes out shorter than the Terraform one. The Akamai project ends with a null_resource that runs a curl loop in local-exec to block terraform apply until the model finishes downloading. That is not infrastructure but a runtime check wearing a resource costume. Pulumi has a Command provider that would let you reproduce it line for line, and this post deliberately does not. The program provisions the box, the box pulls the model on its own, and the program prints the endpoint. Whether the model has finished downloading yet is a question you answer with a curl, not a question your IaC tool should be holding a deployment open to ask.

Prerequisites

Before getting started, ensure you have:

Pulumi CLI installed and configured
A Pulumi Cloud account (ESC and OIDC live here)
An account on at least one of AWS, Azure, or Google Cloud, with an OIDC trust set up for Pulumi (covered below)
Enough GPU quota in your target region for one T4-class instance; a fresh account often starts at zero, which is the most common reason a first deploy fails, so check it before you run pulumi up
Node.js 18+ for the TypeScript program
curl to talk to the inference endpoint once it comes up

This post serves the qwen2.5:14b model, the same one the Akamai article uses, because the 4-bit quant fits comfortably on a single 16 GB T4. The model is one config value (model), so swap in anything Ollama supports; match the GPU to the model’s memory footprint, because a model that overflows VRAM still runs but spills onto the CPU and crawls.

Credentials without the copy-paste

The Terraform version authenticates the only way a single-cloud demo can: you mint a personal access token, export it as LINODE_TOKEN, and the provider reads it from the environment. It works, but that token is long-lived, it sits in your shell history and your CI secrets, and you mint one per cloud. Across three clouds that is three static keys to rotate and worry about.

Pulumi ESC (Environments, Secrets, and Configuration) replaces it all with an OIDC login. The idea: instead of storing a cloud key, you store a trust relationship. At deploy time, ESC presents a short-lived OIDC token to AWS, Azure, or Google Cloud, and each one hands back temporary credentials scoped to a role you control. Nothing long-lived is ever written down.

I keep that wiring in one environment, pulumi-idp/auth, and every stack imports it. Here is the whole thing:

# pulumi-idp/auth: one ESC environment that brokers an OIDC login into all three
# clouds. Every stack imports it. No static cloud key lives here or anywhere else.
values:
 aws:
 login:
 fn::open::aws-login:
 oidc:
 roleArn: arn:aws:iam::123456789012:role/pulumi-esc
 sessionName: pulumi-esc
 azure:
 login:
 fn::open::azure-login:
 clientId: aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
 tenantId: aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
 subscriptionId: /subscriptions/00000000-0000-0000-0000-000000000000
 oidc: true
 gcp:
 login:
 fn::open::gcp-login:
 project: 123456789012 # numeric project number, not the project ID
 oidc:
 workloadPoolId: pulumi-esc
 providerId: pulumi-esc
 serviceAccount: pulumi-esc@my-project.iam.gserviceaccount.com
 environmentVariables:
 # AWS: read by the Pulumi AWS provider, the AWS SDKs, and the aws CLI
 AWS_ACCESS_KEY_ID: ${aws.login.accessKeyId}
 AWS_SECRET_ACCESS_KEY: ${aws.login.secretAccessKey}
 AWS_SESSION_TOKEN: ${aws.login.sessionToken}
 # Azure: read by the azure-native provider
 ARM_USE_OIDC: "true"
 ARM_CLIENT_ID: ${azure.login.clientId}
 ARM_TENANT_ID: ${azure.login.tenantId}
 ARM_SUBSCRIPTION_ID: ${azure.login.subscriptionId}
 ARM_OIDC_TOKEN: ${azure.login.oidc.token}
 # Google Cloud: read by the Pulumi Google Cloud provider
 GOOGLE_CLOUD_PROJECT: ${gcp.login.project}
 GOOGLE_OAUTH_ACCESS_TOKEN: ${gcp.login.accessToken}

You only need the blocks for the clouds you actually deploy to. Opening this environment makes ESC perform a live OIDC login for each cloud listed, so trim it to the one you use, or keep all three if, like me, you bounce between them.

Each cloud needs a one-time trust setup so this login is allowed at all: an IAM OIDC identity provider and role on AWS, a federated credential on an Azure app registration, and a Workload Identity Pool on Google Cloud. You do that once; after that, pulumi-idp/auth is the only thing any stack references for credentials.

A stack opts into it with one block in its stack config. The AWS stack’s Pulumi.aws.yaml:

# Pulumi.aws.yaml
environment:
 - pulumi-idp/auth

That is the entire credential story. No keys in the program, no keys in CI, nothing to rotate. The program below never mentions a secret; it only creates resources, and the ambient credentials from pulumi-idp/auth carry the request.

One cloud-init, three clouds

The part that turns a bare Ubuntu box into an inference server is identical on every cloud, so it lives in one file, cloud-init.yaml, that all three programs read. It follows the Akamai cloud-config closely, with a couple of robustness tweaks for a multi-cloud run, and it runs in a deliberate order because the GPU driver needs a reboot before Ollama can see the card:

Update packages and install the kernel headers and ubuntu-drivers.
Install the NVIDIA driver, then reboot so the kernel loads it.
On the next boot, a one-shot systemd service installs Ollama, binds it to 0.0.0.0:11434, and pulls the model.
The service disables itself, so it never runs again.

#cloud-config
# Zero-touch Ollama GPU box. Cloud-agnostic: no provider metadata, no static creds.
# The Pulumi program substitutes the model name on the `ollama pull` line below
# before this is passed as user-data (AWS/GCP) or custom-data (Azure).

write_files:
 # Make Ollama listen on every interface and never unload the model from VRAM.
 - path: /etc/systemd/system/ollama.service.d/override.conf
 content: |
 [Service]
 Environment="OLLAMA_HOST=0.0.0.0:11434"
 Environment="OLLAMA_KEEP_ALIVE=-1"

 # Runs once on the post-reboot boot, after the GPU driver is loaded.
 - path: /usr/local/bin/ollama-setup.sh
 permissions: '0755'
 content: |
 #!/usr/bin/env bash
 set -euxo pipefail

 # Install Ollama; the installer creates and starts the ollama systemd unit.
 curl -fsSL https://ollama.com/install.sh | sh

 # Pick up the OLLAMA_HOST / OLLAMA_KEEP_ALIVE override written above.
 systemctl daemon-reload
 systemctl enable ollama.service
 systemctl restart ollama.service

 # Wait for the daemon to bind its socket before pulling. This is the box
 # waiting on its OWN local daemon, not an external readiness gate.
 until curl -fsS http://127.0.0.1:11434/api/tags >/dev/null 2>&1; do
 sleep 2
 done

 # Pre-pull the model so the endpoint answers on the very first request.
 ollama pull __MODEL__

 # One-shot: never run again on future boots.
 systemctl disable ollama-setup.service

 - path: /etc/systemd/system/ollama-setup.service
 content: |
 [Unit]
 Description=One-time Ollama install and model pull
 After=network-online.target
 Wants=network-online.target

 [Service]
 Type=oneshot
 ExecStart=/usr/local/bin/ollama-setup.sh
 RemainAfterExit=true
 # The model pull runs for several minutes; without this, systemd's default
 # 90s start timeout kills the service mid-download and the model never lands.
 TimeoutStartSec=0

 [Install]
 WantedBy=multi-user.target

runcmd:
 - apt-get update
 - DEBIAN_FRONTEND=noninteractive apt-get -y upgrade
 # linux-headers-generic also covers the kernel the upgrade above may have pulled
 # in, so DKMS builds the NVIDIA module for the kernel that boots next.
 - DEBIAN_FRONTEND=noninteractive apt-get install -y "linux-headers-$(uname -r)" linux-headers-generic ubuntu-drivers-common
 # --gpgpu selects the headless server driver branch, the right one for a compute
 # GPU like the T4; bare `ubuntu-drivers install` would pull the desktop stack.
 - ubuntu-drivers install --gpgpu
 - systemctl enable ollama-setup.service

power_state:
 mode: reboot
 message: Rebooting to load the NVIDIA driver before Ollama setup
 condition: true

The model name is the literal token __MODEL__. The Pulumi program reads this file at deploy time and substitutes your model config value before handing it to the instance. Two environment settings are doing quiet but important work: OLLAMA_HOST=0.0.0.0:11434 makes Ollama listen on every interface instead of localhost alone, and OLLAMA_KEEP_ALIVE=-1 keeps the model pinned in GPU memory so only the first request pays the load cost.

The programs

Every program has the same five beats: read the config, template the cloud-init.yaml, open the inbound ports, launch the GPU instance with that cloud-init as its user data, and export the endpoint. What differs is only the dialect each cloud speaks for “GPU instance” and “firewall rule.”

The shared contract keeps the three listings legible side by side: the same model and allowSsh config keys, the same cloud-init.yaml, and the same four exports (publicIp, ollamaEndpoint, generateEndpoint, tagsEndpoint). Pick your cloud:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as fs from "fs";

const cfg = new pulumi.Config();
const model = cfg.get("model") ?? "qwen2.5:14b";
const allowSsh = cfg.getBoolean("allowSsh") ?? false;
const region = (cfg.get("region") ?? "us-east-1") as aws.Region;

// Inject the model name into cloud-init at deploy time (a file read, not a shell-out).
const userData = fs.readFileSync("cloud-init.yaml", "utf8").replace(/__MODEL__/g, model);

// Region comes from config; credentials arrive ambiently from pulumi-idp/auth.
const provider = new aws.Provider("aws", { region });

// Most recent Ubuntu 24.04 LTS (amd64, hvm, gp3) published by Canonical.
const ubuntu = aws.ec2.getAmi({
 mostRecent: true,
 owners: ["099720109477"],
 filters: [
 { name: "name", values: ["ubuntu/images/hvm-ssd-gp3/ubuntu-noble-24.04-amd64-server-*"] },
 { name: "virtualization-type", values: ["hvm"] },
 ],
}, { provider });

const ingress: aws.types.input.ec2.SecurityGroupIngress[] = [{
 description: "Ollama HTTP API. NOTE: restrict cidrBlocks to your own range in production.",
 fromPort: 11434,
 toPort: 11434,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
}];

if (allowSsh) {
 ingress.push({
 description: "SSH",
 fromPort: 22,
 toPort: 22,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 });
}

const sg = new aws.ec2.SecurityGroup("ollama", {
 description: "Ollama inferencing access",
 ingress,
 egress: [{
 description: "Allow all outbound",
 fromPort: 0,
 toPort: 0,
 protocol: "-1",
 cidrBlocks: ["0.0.0.0/0"],
 }],
}, {
 provider,
});

const server = new aws.ec2.Instance("ollama", {
 ami: ubuntu.then(a => a.id),
 instanceType: "g4dn.xlarge", // 1x NVIDIA T4
 vpcSecurityGroupIds: [sg.id],
 associatePublicIpAddress: true,
 userData, // plain text; the AWS provider base64-encodes it for you
 rootBlockDevice: {
 volumeSize: 40,
 volumeType: "gp3",
 },
 tags: { Name: "ollama" },
}, {
 provider,
});

export const publicIp = server.publicIp;
export const ollamaEndpoint = pulumi.interpolate`http://${publicIp}:11434`;
export const generateEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/generate`;
export const tagsEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/tags`;

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";
import * as fs from "fs";

const cfg = new pulumi.Config();
const model = cfg.get("model") ?? "qwen2.5:14b";
const allowSsh = cfg.getBoolean("allowSsh") ?? false;
const project = cfg.get("project");
const zone = cfg.get("zone") ?? "us-central1-a";

// Inject the model name into cloud-init at deploy time (a file read, not a shell-out).
const userData = fs.readFileSync("cloud-init.yaml", "utf8").replace(/__MODEL__/g, model);

// Latest Ubuntu 24.04 LTS amd64 image.
const ubuntu = gcp.compute.getImage({
 family: "ubuntu-2404-lts-amd64",
 project: "ubuntu-os-cloud",
});

// Network tag binds the firewall rule to this instance.
const networkTag = "ollama";

const firewall = new gcp.compute.Firewall("ollama-fw", {
 network: "default",
 project: project,
 // Inbound to Ollama from anywhere; restrict this CIDR for production.
 allows: allowSsh
 ? [{ protocol: "tcp", ports: ["11434"] }, { protocol: "tcp", ports: ["22"] }]
 : [{ protocol: "tcp", ports: ["11434"] }],
 sourceRanges: ["0.0.0.0/0"],
 targetTags: [networkTag],
});

const instance = new gcp.compute.Instance("ollama", {
 machineType: "n1-standard-4",
 zone: zone,
 project: project,
 tags: [networkTag],
 bootDisk: {
 initializeParams: {
 image: ubuntu.then(i => i.selfLink),
 size: 40,
 },
 },
 guestAccelerators: [{
 type: "nvidia-tesla-t4",
 count: 1,
 }],
 // GPUs cannot live-migrate, so host maintenance must terminate (and restart) the VM.
 scheduling: {
 onHostMaintenance: "TERMINATE",
 automaticRestart: true,
 },
 networkInterfaces: [{
 network: "default",
 accessConfigs: [{}], // an empty config requests an ephemeral public IP
 }],
 metadata: {
 "user-data": userData, // Ubuntu cloud-init reads this key, not startup-script
 },
}, {
 dependsOn: firewall,
});

export const publicIp = instance.networkInterfaces.apply(nics => nics[0].accessConfigs![0].natIp);
export const ollamaEndpoint = pulumi.interpolate`http://${publicIp}:11434`;
export const generateEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/generate`;
export const tagsEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/tags`;

import * as pulumi from "@pulumi/pulumi";
import * as resources from "@pulumi/azure-native/resources";
import * as network from "@pulumi/azure-native/network";
import * as compute from "@pulumi/azure-native/compute";
import * as random from "@pulumi/random";
import * as fs from "fs";

const cfg = new pulumi.Config();
const model = cfg.get("model") ?? "qwen2.5:14b";
const allowSsh = cfg.getBoolean("allowSsh") ?? false;
const location = cfg.get("location") ?? "eastus";

// Inject the model name into cloud-init at deploy time (a file read, not a shell-out).
const userData = fs.readFileSync("cloud-init.yaml", "utf8").replace(/__MODEL__/g, model);

const resourceGroup = new resources.ResourceGroup("ollama-rg", {
 location,
});

const vnet = new network.VirtualNetwork("ollama-vnet", {
 resourceGroupName: resourceGroup.name,
 addressSpace: { addressPrefixes: ["10.0.0.0/16"] },
});

const subnet = new network.Subnet("ollama-subnet", {
 resourceGroupName: resourceGroup.name,
 virtualNetworkName: vnet.name,
 addressPrefix: "10.0.1.0/24",
});

// Standard SKU public IPs use static allocation.
const publicIpAddress = new network.PublicIPAddress("ollama-pip", {
 resourceGroupName: resourceGroup.name,
 sku: { name: network.PublicIPAddressSkuName.Standard },
 publicIPAllocationMethod: network.IPAllocationMethod.Static,
});

// Azure permits all outbound by default, so only inbound rules are needed.
const nsg = new network.NetworkSecurityGroup("ollama-nsg", {
 resourceGroupName: resourceGroup.name,
 securityRules: [
 {
 name: "allow-ollama",
 priority: 1000,
 direction: network.SecurityRuleDirection.Inbound,
 access: network.SecurityRuleAccess.Allow,
 protocol: network.SecurityRuleProtocol.Tcp,
 sourcePortRange: "*",
 destinationPortRange: "11434",
 sourceAddressPrefix: "0.0.0.0/0", // prod: restrict to your client CIDR
 destinationAddressPrefix: "*",
 },
 ...(allowSsh ? [{
 name: "allow-ssh",
 priority: 1001,
 direction: network.SecurityRuleDirection.Inbound,
 access: network.SecurityRuleAccess.Allow,
 protocol: network.SecurityRuleProtocol.Tcp,
 sourcePortRange: "*",
 destinationPortRange: "22",
 sourceAddressPrefix: "0.0.0.0/0",
 destinationAddressPrefix: "*",
 }] : []),
 ],
});

const nic = new network.NetworkInterface("ollama-nic", {
 resourceGroupName: resourceGroup.name,
 networkSecurityGroup: { id: nsg.id },
 ipConfigurations: [{
 name: "ipconfig1",
 subnet: { id: subnet.id },
 publicIPAddress: { id: publicIpAddress.id },
 privateIPAllocationMethod: network.IPAllocationMethod.Dynamic,
 primary: true,
 }],
});

// Azure requires an admin credential even though we never log in. Generate one
// instead of hard-coding it; it stays a Pulumi secret and is never exported.
const adminPassword = new random.RandomPassword("ollama-admin-password", {
 length: 24,
 special: true,
 overrideSpecial: "!#$%*",
 minLower: 1,
 minUpper: 1,
 minNumeric: 1,
 minSpecial: 1, // Azure requires 3 of 4 character classes
});

const vm = new compute.VirtualMachine("ollama-vm", {
 resourceGroupName: resourceGroup.name,
 hardwareProfile: { vmSize: "Standard_NC4as_T4_v3" }, // 1x NVIDIA T4
 networkProfile: {
 networkInterfaces: [{ id: nic.id, primary: true }],
 },
 osProfile: {
 computerName: "ollama",
 adminUsername: "azureuser",
 adminPassword: adminPassword.result,
 customData: Buffer.from(userData).toString("base64"), // Azure wants base64
 linuxConfiguration: {
 disablePasswordAuthentication: false,
 },
 },
 storageProfile: {
 imageReference: {
 publisher: "Canonical",
 offer: "ubuntu-24_04-lts",
 sku: "server", // Gen2; NC4as_T4_v3 is a Gen2 size
 version: "latest",
 },
 osDisk: {
 name: "ollama-osdisk",
 createOption: "FromImage",
 diskSizeGB: 40,
 managedDisk: { storageAccountType: "StandardSSD_LRS" },
 },
 },
});

// A Standard/Static public IP is reserved at creation, so its address is known
// once the resource exists; no separate lookup needed.
export const publicIp = publicIpAddress.ipAddress.apply(ip => ip!);
export const ollamaEndpoint = pulumi.interpolate`http://${publicIp}:11434`;
export const generateEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/generate`;
export const tagsEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/tags`;

A few per-cloud details are worth calling out, since they are the places the “same program” abstraction leaks:

AWS is the shortest program, because the GPU comes with the instance shape: a g4dn.xlarge is a T4 box, so there is no separate accelerator to attach. The one thing to do before you deploy is raise the Running On-Demand G and VT instances vCPU quota in your region; a fresh account starts at zero, and pulumi up fails with VcpuLimitExceeded until you do.
Google Cloud attaches the GPU explicitly with guestAccelerators, and that brings the rule that trips people up: a GPU instance cannot live-migrate, so scheduling.onHostMaintenance must be "TERMINATE" or the apply is rejected. The cloud-init also has to ride on the user-data metadata key, not startup-script, and the empty accessConfigs: [{}] is what hands the box a public IP. T4 quota is also zero on a new project.
Azure is the longest listing, because the network is à la carte: the resource group, virtual network, subnet, public IP, security group, and NIC are each their own resource before you reach the VM. One detail surprises people: a Linux VM requires an admin credential even when you never log in, so the program generates a throwaway password with random.RandomPassword rather than committing one. The NC4as_T4_v3 is a compute GPU, so the standard server (CUDA) driver from the cloud-init is the right choice here; the GRID driver is only needed for GPU-accelerated visualization workloads.

The full programs, all three Pulumi.<cloud>.yaml files, and the shared cloud-init.yaml are in the companion repo:

github.com/dirien/fully-automated-ai-inference-pulumi

Deploying

Create a project, install the provider for your cloud, point the stack at pulumi-idp/auth, and deploy. For AWS:

mkdir ai-inference && cd ai-inference
pulumi new typescript
npm install @pulumi/aws

Drop the AWS listing into index.ts, copy the cloud-init.yaml shown earlier into the same directory (or grab it from the companion repo), point the stack at the auth environment, and set your region:

pulumi stack init aws
# add `environment: [pulumi-idp/auth]` to Pulumi.aws.yaml (shown above)
pulumi config set region us-east-1
pulumi config set model qwen2.5:14b
pulumi up

The other two clouds are the same flow with a different provider package and a couple of config keys: npm install @pulumi/gcp with pulumi config set project <your-project-id> and pulumi config set zone us-central1-a, or npm install @pulumi/azure-native @pulumi/random with pulumi config set location eastus.

The slow part is the GPU box: it boots, installs the driver, reboots, installs Ollama, and pulls the model, all without you. Depending on the instance, the region, and the model size, plan on roughly 10 to 15 minutes before the endpoint answers. When pulumi up finishes you get the endpoints straight back as stack outputs:

Outputs:
 generateEndpoint: "http://<public-ip>:11434/api/generate"
 ollamaEndpoint : "http://<public-ip>:11434"
 publicIp : "<public-ip>"
 tagsEndpoint : "http://<public-ip>:11434/api/tags"

Testing the inference server

pulumi up returns as soon as the infrastructure exists, which is before the model has finished downloading. This is exactly where Terraform reached for that null_resource loop, and where Pulumi hands you a URL instead. To check whether the model is ready, ask Ollama what it has loaded:

curl -s $(pulumi stack output tagsEndpoint) | grep -q "qwen2.5:14b" && echo "ready" || echo "still pulling"

Once it reports ready, send it a prompt. This is the same request the Akamai post makes, pointed at your generateEndpoint output:

curl -s $(pulumi stack output generateEndpoint) -d '{
 "model": "qwen2.5:14b",
 "system": "Answer every question with a three-line poem.",
 "prompt": "Why is the sky blue?",
 "stream": false
}'

The first call is slower, because Ollama loads the model into GPU memory before it answers. Every call after that is fast, since OLLAMA_KEEP_ALIVE=-1 keeps it resident:

{
 "model": "qwen2.5:14b",
 "response": "Sunlight scatters, short waves fly,\nBlue light paints the open sky,\nViolet fades as day drifts by.",
 "done": true
}

When you are done, take it all down with one command, so the GPU stops billing:

pulumi destroy

Cost

A GPU instance is the whole bill, and you pay by the hour whether the model is busy or idle. The numbers below are rough on-demand rates for a single T4-class box left running 24/7; in practice you spin it up, use it, and pulumi destroy it, so what you actually pay tracks the hours it stays up.

Cloud	Instance	GPU	~Hourly	~Monthly (24/7)
AWS	`g4dn.xlarge`	1× T4 (16 GB)	~$0.53	~$384
Azure	`Standard_NC4as_T4_v3`	1× T4 (16 GB)	~$0.53	~$384
Google Cloud	`n1-standard-4` + 1× T4	1× T4 (16 GB)	~$0.54	~$394

These are on-demand Linux rates in a cheap US region (us-east-1, eastus, us-central1) as of mid-2026, and the GPU dominates the bill. Spot or preemptible capacity cuts it sharply if your workload tolerates interruption (often 60 to 70 percent off), Google Cloud’s sustained-use discount trims an always-on instance on its own, and the 40 GB disk adds only a few dollars a month.

A GPU box left running overnight is the expensive mistake here. Because the whole stack is declarative, the safe habit is cheap: pulumi destroy when you stop using it, pulumi up when you need it back. The state and config are version-controlled, so standing it up again is one command, not a rebuild.

Security considerations

The architecture in this post matches the Akamai original on purpose, which means it carries the same caveat: Ollama is exposed over plain HTTP on a port open to the entire internet. That is fine for a demo on a box you tear down the same day. It is not fine for anything that outlives the afternoon, and self-hosted AI endpoints get found fast: scanners like Shodan enumerate a freshly exposed one within hours, and an open Ollama instance is free compute for whoever finds it first.

The credential side, though, is genuinely better than a static-token setup, and that is the part worth keeping. There is no long-lived cloud key in the program, in your shell, or in CI; every deploy gets short-lived credentials minted through pulumi-idp/auth and thrown away when it finishes.

Concern	Akamai/Terraform demo	This deployment
Cloud credentials	Static PAT in an env var	OIDC login, short-lived, via `pulumi-idp/auth`
Inbound exposure	Port 11434 open to `0.0.0.0/0`	Same by default; one config flag scopes the CIDR
Transport	Plain HTTP	Plain HTTP (front with TLS for real use)
Readiness wait	`null_resource` + `local-exec` shell loop	A runtime `curl`, no resource

My recommendations for taking this past a demo:

Scope the inbound rule to your own CIDR instead of 0.0.0.0/0; the allowSsh flag in the program is the pattern to copy for the Ollama port
Put a reverse proxy with TLS in front of Ollama, or keep the box off the public internet entirely and reach it over a private network or a Tailscale tailnet, the way I did for the Hermes agent
Keep credentials on OIDC through ESC; never fall back to a static key to “just get it working”
Treat the box as disposable, and pulumi destroy it when you are not using it, which shrinks both the bill and the attack window

What’s next?

The program is a foundation, and the interesting work is what you layer on once a model is a pulumi up away:

Lock it down. Scope the firewall, add TLS, or move the box behind a private network so the endpoint is not on the public internet at all.
Scale the model to the GPU. qwen2.5:14b on a T4 is the entry point. Bigger models want more memory, which means an A10G, an L4, or an A100, and the only change in the program is the instance type and the model value.
Swap the serving engine. Ollama is the simplest thing that works. For higher throughput, the same shape holds with vLLM in place of Ollama in the cloud-init.
Generate the next one. Pulumi Neo can take a target like “a GPU inference box on Azure behind a private endpoint” and produce a first draft of the program, which you then review in a PR like any other change.

Conclusion

Declaring an AI inference server as code buys you the same thing it buys for any other infrastructure: you can reproduce it on any of the three clouds, version-control it, and tear it down with a single pulumi destroy. Porting the Akamai project made the contrast sharp. The credentials moved from a static token to an OIDC login that leaves nothing behind, and the readiness wait disappeared entirely once you stop treating a runtime check as a resource.

That is the line worth carrying to the next thing you deploy. Not every step in a runbook is a resource. A GPU box and a firewall rule are; waiting for a download to finish is not. Draw that line first, and the program gets shorter on its own.

If you run into issues or have questions, drop by the Pulumi Community Slack or GitHub Discussions. New to Pulumi? Get started here.

Build an EKS Environment Factory with Pulumi and vCluster

Pablo Seibelt — Thu, 04 Jun 2026 00:00:00 +0000

AWS reports in an AWS Architecture Blog case study that Deloitte’s move to a virtual cluster model on Amazon EKS resulted in 89% faster testing environment provisioning. By consolidating dozens of disparate clusters into a single host cluster with over 50 vCluster instances, the case study says Deloitte saved about 500 QA hours per year. This “Environment Factory” pattern allows platform teams to provide isolated, ephemeral Kubernetes environments on demand without the cost or lag of full cluster provisioning.

This post adapts that general architecture with Pulumi to orchestrate Amazon EKS Auto Mode and vCluster.

The problem: environment sprawl and provisioning lag

Traditional development workflows often rely on one full EKS cluster per developer or feature branch. While this provides strong isolation, it introduces major pain points. Provisioning a full cluster can take 15 minutes or more, which slows down CI/CD pipelines. Managing dozens of clusters also leads to high costs and significant operational overhead.

Platform teams need a “soft multi-tenancy” model. This model should feel like a dedicated cluster to the developer but run on shared infrastructure to keep costs low and startup times fast.

Architecture overview: the host and the tenants

The environment factory architecture consists of two main layers.

Host cluster: A single, reliable EKS cluster managed with EKS Auto Mode. This cluster provides the underlying compute, networking, and storage.
Tenant environments: Virtual clusters (vCluster) running as pods within host namespaces.

According to the vCluster architecture, the virtual control plane handles API requests while a syncer maps virtual resources to the host cluster. This separation allows tenants to manage their own CRDs, namespaces, and RBAC while platform teams use quotas, NetworkPolicies, pod security, IAM boundaries, and node isolation controls to protect the host and other tenants.

Implementation: the EKS Auto Mode host

EKS Auto Mode simplifies the host cluster by automating infrastructure management. It handles node provisioning, scaling, and updates based on pod requirements.

The following snippet shows how to define an EKS cluster with Auto Mode enabled using Pulumi.

import * as awsx from "@pulumi/awsx";
import * as eks from "@pulumi/eks";
import * as k8s from "@pulumi/kubernetes";
import { SubnetType } from "@pulumi/awsx/ec2";

const clusterName = "environment-factory";

const vpc = new awsx.ec2.Vpc("environment-factory", {
 enableDnsHostnames: true,
 cidrBlock: "10.0.0.0/16",
 subnetSpecs: [
 {
 type: SubnetType.Public,
 tags: {
 [`kubernetes.io/cluster/${clusterName}`]: "shared",
 "kubernetes.io/role/elb": "1",
 },
 },
 {
 type: SubnetType.Private,
 tags: {
 [`kubernetes.io/cluster/${clusterName}`]: "shared",
 "kubernetes.io/role/internal-elb": "1",
 },
 },
 ],
 subnetStrategy: "Auto",
});

// Create an EKS cluster with Auto Mode enabled.
const hostCluster = new eks.Cluster("host-cluster", {
 name: clusterName,
 authenticationMode: eks.AuthenticationMode.Api, // Use API authentication mode for EKS access entries.
 vpcId: vpc.vpcId,
 publicSubnetIds: vpc.publicSubnetIds,
 privateSubnetIds: vpc.privateSubnetIds,
 autoMode: {
 enabled: true,
 },
});

const hostProvider = new k8s.Provider("host-provider", {
 kubeconfig: hostCluster.kubeconfig,
});

Implementation: the environment factory

Once the host cluster is ready, we can build the factory that stamps out tenant environments. Each tenant needs a dedicated namespace, resource quotas, and the vCluster itself.

Tenant guardrails

Before installing vCluster, we set up a namespace and resource quotas to ensure one tenant cannot consume all host resources.

import * as k8s from "@pulumi/kubernetes";

// Define a tenant namespace on the host cluster.
const tenantNamespace = new k8s.core.v1.Namespace("tenant-alpha", {
 metadata: { name: "tenant-alpha" },
}, { provider: hostProvider });

// Apply resource quotas to the tenant namespace.
const quota = new k8s.core.v1.ResourceQuota("tenant-quota", {
 metadata: { namespace: tenantNamespace.metadata.name },
 spec: {
 hard: {
 pods: "20",
 "requests.cpu": "4",
 "requests.memory": "8Gi",
 "limits.cpu": "8",
 "limits.memory": "16Gi",
 },
 },
}, { provider: hostProvider });

// Define a Role for the tenant within their namespace.
const tenantRole = new k8s.rbac.v1.Role("tenant-role", {
 metadata: { namespace: tenantNamespace.metadata.name },
 rules: [{
 apiGroups: [""],
 resources: ["pods", "services", "configmaps", "secrets"],
 verbs: ["get", "list", "watch", "create", "update", "patch", "delete"],
 }],
}, { provider: hostProvider });

// Bind the Role to a tenant user or group.
const tenantRoleBinding = new k8s.rbac.v1.RoleBinding("tenant-role-binding", {
 metadata: { namespace: tenantNamespace.metadata.name },
 subjects: [{
 kind: "User",
 // Replace "tenant-user" with the IAM-mapped user or group for this tenant.
 name: "tenant-user",
 apiGroup: "rbac.authorization.k8s.io",
 }],
 roleRef: {
 kind: "Role",
 name: tenantRole.metadata.name,
 apiGroup: "rbac.authorization.k8s.io",
 },
}, { provider: hostProvider });

For production use, map these Kubernetes identities to IAM principals using EKS Access Entries, with the legacy aws-auth ConfigMap still appearing in older clusters.

Deploying vCluster with Helm

We use the kubernetes.helm.v3.Release resource to install vCluster. This resource provides controlled Helm lifecycle management for the vCluster release. The values block should be adjusted for each tenant profile to control resource synchronization and control plane behavior. Review the vCluster release notes when changing chart versions because values schema and generated secret names can change across releases.

import * as k8s from "@pulumi/kubernetes";

// Install vCluster using the Helm Release resource.
const vcluster = new k8s.helm.v3.Release("vcluster-alpha", {
 name: "vcluster-alpha",
 chart: "vcluster",
 version: "0.20.0", // Tested with vCluster 0.20.x; review release notes before changing versions.
 repositoryOpts: {
 repo: "https://charts.loft.sh",
 },
 namespace: tenantNamespace.metadata.name,
 values: {
 // Explicit sync configuration; adjust per tenant profile.
 sync: {
 toHost: {
 pods: { enabled: true },
 },
 },
 },
}, { provider: hostProvider });

Accessing the virtual cluster

The vCluster generates a kubeconfig that allows developers to interact with the virtual API server. We must treat this kubeconfig as a secret, and the endpoint in that kubeconfig must be reachable from the Pulumi runner before using it to create resources inside the virtual cluster.

import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

// Retrieve the vCluster kubeconfig from the generated secret.
// The vCluster-generated secret can lag behind Helm release readiness on first creation,
// so teams may choose an explicit readiness check or rerun after the virtual control plane initializes.
const vclusterKubeconfig = k8s.core.v1.Secret.get("vcluster-secret",
 pulumi.interpolate`${tenantNamespace.metadata.name}/vc-vcluster-alpha`,
 {
 provider: hostProvider,
 dependsOn: [vcluster],
 }
).data.apply(data => Buffer.from(data["config"], "base64").toString());

// Export the kubeconfig as a secret.
export const tenantKubeconfig = pulumi.secret(vclusterKubeconfig);

// Create a provider for the virtual cluster using the secret kubeconfig.
const vclusterProvider = new k8s.Provider("vcluster-provider", {
 kubeconfig: tenantKubeconfig,
});

Operational caveats

RBAC and permissions: vCluster generates default RBAC rules that work for most scenarios. However, if your host cluster is heavily locked down, you may need to provide additional permissions to the vCluster service account.
Helm release previews: When using kubernetes.helm.v3.Release, Pulumi previews may not show every detail of the rendered Kubernetes resources. It primarily tracks the state of the Helm release itself.
EKS Auto Mode node lifetime: EKS Auto Mode uses immutable AMIs and has a 21-day node lifetime. Kubernetes reschedules vCluster pods and tenant workloads when nodes are replaced, so configure replicas, PodDisruptionBudgets, requests, and persistent storage for disruption tolerance.

Conclusion: ephemeral environments at scale

By combining Pulumi with EKS Auto Mode and vCluster, you can build a scalable environment factory. This approach provides the isolation developers need while maintaining the speed and cost-efficiency required by platform teams.

The snippets provided here are adapted for illustration. In a production environment, you would likely wrap these resources into a Pulumi ComponentResource to provide a clean, reusable API for your internal developers. When a feature branch is merged, deleting the Pulumi stack removes the resources managed by that stack, but validate namespace finalizers, persistent volume reclaim policies, and external cloud artifacts as part of cleanup.

For more on managing EKS with Pulumi, see the EKS guide.

Ten More Things You Can Do With Pulumi Neo

Adam Gordon Bell — Tue, 19 May 2026 05:00:00 -0700

Last fall, after launching Pulumi Neo, we wrote up 10 things you could do with it. In the months that followed, as platform teams handed Neo more real work, we watched and listened, shipping a steady stream of features like plan mode, read-only mode, AGENTS.md, an integration catalog, cross-cloud migration, and task sharing. With today’s release, Neo extends beyond the Pulumi Cloud console into the Pulumi CLI, GitHub, and Slack.

So here are 10 more things you can do with Neo.

1. Deploy your app to AWS without writing IaC

Hand Neo a repo. Neo picks the right services — ECS, AWS Fargate, ALB — writes the Pulumi, and opens a PR.

The cloud infrastructure part of getting a new service running, especially one in a new language, is always a few hours of boilerplate: a VPC and subnets, an IAM role, security groups, a load balancer, DNS, and a TLS cert.

With Neo, that work collapses into a prompt. Point Neo at a repo and ask:

Deploy this app to AWS as a publicly accessible service.

Plan mode comes back with the resources Neo will create, named and sized: ECS running on AWS Fargate, an ALB, and the VPC wiring. Approve, and Neo writes the Pulumi program, runs a preview, and opens a PR. You, the human in the loop, merge it after review.

Neo planning a PR and deploying an app to AWS.

Start a Neo task Ask Neo to deploy your app to AWS and make a PR

2. Diagnose a slow API from metrics, logs, and code

Slow endpoints live at the seam between runtime metrics and the stack that runs them. Neo reads both and proposes a fix with the metric evidence as the rationale.

Production incidents often involve multiple tools. When the checkout endpoint’s p95 latency climbs from 200ms to 1.2s, the metric is in Datadog, but the cause might be somewhere in your AWS account: maybe RDS is out of IOPS, maybe the connection pool is too small, maybe the autoscaler isn’t keeping up. Connecting “this metric looks bad” to a recent backend change and then to a one-line fix in your Pulumi program is an exercise in detective work.

Neo’s integration catalog bridges this gap. With built-in Datadog APM, PagerDuty, and Honeycomb integrations sitting alongside your Pulumi state, Neo can read traces and metrics from the tools your team already uses and take action.

Ask Neo:

Find the scaling bottleneck on /checkout from the last 7 days of metrics and propose a fix.

Neo pulls the metric history, matches the Datadog tag db.cluster=checkout-rds to the RDS instance in your prod-checkout Pulumi stack, and opens a PR with a Pulumi diff that bumps the storage IOPS and raises the connection-pool ceiling. You review and roll out the fix.

Toggle on the Honeycomb integration so Neo can read traces and metrics alongside your Pulumi stacks.

3. Automate PagerDuty incident response from Slack

A page comes in. You paste it into your on-call channel and tag Neo, and Neo replies with the cross-system view you’d otherwise spend the first 20 minutes assembling.

On-call triage is often about getting up to speed quickly. You get paged because something is in the red, and you don’t know why.

You mention Neo in the on-call Slack channel:

@neo, what’s going on with this alert?

Neo starts querying metrics and traces. With PagerDuty and Datadog in the integration catalog, it correlates the alert with every deploy and stack change tagged with the alert’s service in the last hour, and finds the change that lines up:

Two deploys in the last hour touched services tagged service:checkout: checkout-api@a3f9c2 (12 min ago, app-layer deploy) and Pulumi stack prod-checkout-rds (45 min ago, decreased max_connections from 200 → 100). p99 latency inflection at 14:03 lines up with the stack change. Likely cause: the connection-pool reduction is starving the API under current load.

You ask a couple of clarifying questions in-thread, then ask Neo to open a rollback PR against the Pulumi stack.

Authorize PagerDuty and Datadog in Neo's settings. Neo can then read alerts in your on-call Slack channel, find the change that correlates, and open a PR when you ask.

4. Implement a Jira or Linear ticket end-to-end

Hand Neo a ticket number from Linear, Jira, or GitHub Issues. Neo reads the description and acceptance criteria, plans against your stack, and opens a PR.

Tickets often pile up not because they’re unimportant, but because they’re not urgent. Ongoing maintenance quietly accumulates. Bumping a provider version, centralizing secret management, working through small policy violations: each one matters, but none of them ever moves to the top of the queue. Explaining each one to an agent is its own overhead.

The fix is letting Neo read the ticket itself. Connect Linear integration or Jira automation through the integration catalog (GitHub Issues works too), and Neo pulls the ticket the same way an engineer would: title, description, acceptance criteria.

Ask Neo:

Implement CAD-1234 in our payments stack.

Neo reads the ticket, plans against your existing stack, opens a PR, and drops a comment back on the ticket. The ticket and the PR end up linked, and your backlog shrinks.

Neo running locally in the Pulumi CLI: fielding a Linear issue, analyzing the codebase, and producing a PR that upgrades multiple projects to the latest Pulumi and AWS provider versions.

Start a Neo task Implement a Linear ticket end-to-end

5. Audit and tighten over-privileged IAM roles

Neo audits each role against what your stack code actually does, and proposes scoped policies that improve your security posture.

IAM cleanup is the kind of work nobody has the time to prioritize. Production has 40 roles. Half of them started with s3:* because nobody had time to scope them, and the cleanup slips quarter to quarter.

Ask Neo:

Audit IAM permissions across my accounts and propose narrower policies for over-privileged stack-managed roles.

Neo cross-references each role’s policy against what the stack code actually calls, and opens a PR per role. The PR body lists the API calls Neo found in the stack code, like s3:GetObject on audit-logs-* and s3:PutObject on audit-logs-staging, as the justification for the scoped policy. The evidence sits next to the diff.

If you’re unclear about which roles count as in-scope or what your team considers over-privileged, start in plan mode and agree on that with Neo first.

Neo auditing an over-privileged IAM role and proposing a narrower policy, with the actually-used permissions as evidence.

Start a Neo task Audit IAM and tighten over-privileged roles

6. Migrate from AWS CDK onto your platform’s golden paths

Neo reads your existing AWS CDK app and lands a PR that swaps AWS’s defaults for your team’s published components.

CDK’s L2 constructs encode AWS’s defaults. s3.Bucket with encryption: BucketEncryption.S3_MANAGED is a sane choice, but it’s AWS’s idea of sane, not yours. A platform team that’s published its own components to the Pulumi Private Registry has already decided what your bucket defaults look like: encryption with the right KMS key, tagging by cost center.

Ask Neo:

Migrate the payments-vpc CDK stack to Pulumi using our published components.¹

Neo reads the source CDK app and your registry side by side. It maps each CDK construct to its closest team-published equivalent, clarifying with you where the mapping is ambiguous.

// Before (AWS CDK, AWS's defaults)
const bucket = new s3.Bucket(this, "Assets", {
 bucketName: "payments-assets",
 encryption: s3.BucketEncryption.S3_MANAGED,
 versioned: true,
});

// After (Pulumi, your team's published component)
import * as platform from "@payments/platform";

const bucket = new platform.Bucket("assets", {
 bucketName: "payments-assets",
 classification: "internal",
});

Start a Neo task Migrate CDK onto your golden paths

Neo handles Terraform and Azure ARM migrations the same way.

7. Containerize a service and migrate it to Kubernetes from a runbook

Write the containerization pattern down once. Every service after that is a prompt away.

Containerizing an application and moving it to Kubernetes involves several small decisions: which Docker image, what labels go on deployments, how ingress is wired, and how secrets reach the pod. But after a team has moved two or three services, the pattern is set. The decisions get written down in a runbook, and every subsequent migration is mostly the same shape.

Ask Neo:

Containerize the billing-api service and write its Kubernetes manifests, following our K8s migration runbook in Confluence.

Neo reads the source repo and the runbook in Confluence via the integration catalog and starts working on your request.

You can save this as a Neo skill that splits the work into multiple PRs — Dockerfile first, ECR config next, Deployment/Service/Ingress manifests after — and link back to each runbook convention for ease of review. The output reflects your conventions: the labels you actually use, the ingress class you’ve standardized on, and the External Secrets Operator config your team prefers.

You’re still the one reviewing the PRs and deciding what the cutover looks like in production. Neo follows your internal standards, so the new service ends up shaped like the last one you migrated.

Neo migrating a VM-based service to Kubernetes step by step, following the team's Confluence runbook.

Once you’ve delegated something a few times, the next move is to automate it. The remaining three tasks are the kind Neo doesn’t need to be asked for. Drift, deps, compliance: they’re the operations you put on a schedule via Neo Automations.

8. Schedule daily configuration drift detection across your cloud infrastructure

Schedule a daily drift check across your cloud. Wake up to PRs that fix what changed overnight.

Configuration drift is an ongoing challenge. The security team rotated an IAM role at 04:47 UTC. Someone changed a security group in the AWS console three weeks ago. Left alone, drift turns into security gaps, into compliance issues, and into the kind of “wait, who changed that?” confusion nobody wants to chase down.

Pulumi Cloud is already good at configuration drift detection. Neo takes it a step further.

Ask Neo:

Every morning at 6 AM, check all production infrastructure for drift and create PRs to fix any issues you find.

From then on, the task runs on its own, and you wake up to a PR per drifted resource. The description spells out what happened (iam_role.audit-reader had inline policy AllowReadAuditLogs added at 04:47 UTC) and cites the section of infra/runbooks/drift.md Neo followed.

Some drift gets encoded into the Pulumi program, like the IAM rotation above. Some gets reverted, like the security group rule added from the console. Some gets ignored entirely, like autoscaler-managed Lambda concurrency reservations the runbook tells Neo to skip. You write the runbook once; Neo follows it every morning to decide what to do.

Neo's morning drift PR. The body names the resource, the change, when it happened, and the section of the runbook Neo followed to decide what to do.

Start a Neo task Schedule a daily drift check

9. Schedule weekly upgrades for outdated Lambda runtimes and providers

Lambda runtimes and container base images age out. Schedule the upgrade pass; review the PRs Neo opens.

AWS Lambda end-of-life notices come out months ahead. Node 20 stopped receiving updates as an AWS Lambda runtime at the end of April. Python 3.9 reached end-of-support last December. After the deadline, AWS blocks new deploys and eventually stops invoking the function. Each one needs to move to a supported runtime before the cutoff.²

Schedule it:

Every Sunday night at 10 PM, check our Lambda functions for runtimes nearing end-of-support and open PRs to upgrade them.

Neo reads the AWS Lambda runtime deprecation page, matches the end-of-support runtimes against every Lambda function in your stacks, and opens one PR per stack.

If Python 3.9 is reaching end-of-support, the upgrade is to Python 3.12, and datetime.utcnow() calls need to move to datetime.now(datetime.UTC). Neo can make all of those replacements in the same PR.

The same task can catch container base images with critical CVEs and bump them too.

Setting up a weekly task in the Scheduled Tasks UI. Once saved, Neo runs the prompt every Sunday night and opens PRs you review on Monday.

Start a Neo task Schedule a weekly runtime upgrade check

10. Fix AWS CIS Benchmark failures with daily PRs

Run the AWS CIS Benchmark on a schedule. Wake up to PRs that fix every failure.

The CIS AWS Foundations Benchmark, available through AWS Security Hub, is something every team should be keeping an eye on. The benchmark finds issues like S3 buckets that allow public read access (S3.1), root user access keys that shouldn’t exist (IAM.4), or CloudTrail not being enabled (CloudTrail.1). Scanning for these issues is a solved problem, but closing and addressing them is not. They pile up between audits because each one is a code change in a different stack, and nobody owns the cross-stack cleanup.³

Schedule the cleanup:

Every morning, read CIS Benchmark failures from Security Hub. For every failure on an IaC-managed resource, open a PR with the fix.

Neo opens one PR per failure. A bucket failing S3.1 arrives as a Pulumi diff that adds blockPublicAccess to the bucket in your prod-checkout stack. The PR body lists the CIS rule number, the resource ID, the diff, and a clean pulumi preview against the live infrastructure.

The runbook is where your security team writes down what each control means for your stacks. Block public S3 buckets, except the ones tagged public-content=true for CloudFront origins. Don’t auto-touch the break-glass IAM roles; page a human instead. Multi-region CloudTrail stays on, no exceptions. Neo reads that file, checks each Security Hub finding against it, and only opens a PR for the ones you’ve said are safe to fix. The rest get routed or ignored, the way your team already handles them.

A PR raised by Neo to fix a CIS Benchmark failure, with the failing rule, the resource, and the runbook decision laid out in the body.

Start a Neo task Schedule daily AWS CIS Benchmark scans

Neo: your newest platform engineer

Over the past year, many product teams have stopped treating AI as a request-by-request assistant and started delegating to it outright.⁴ Agents open pull requests, investigate issues, and iterate on review feedback.

But platform engineers have held back because a bad infrastructure change doesn’t just fail, it can take production down. Coding agents benefit from fast, forgiving feedback loops, but infrastructure recovery is rarely as simple as reverting a commit.

What was missing wasn’t the appetite. It was an agent with enough organizational context and grounding to plan reliably, enough guardrails to feel safe and contain mistakes, and enough discipline to keep working without being asked.

The theme across these tasks is clear. A thing platform engineers used to keep in their heads becomes a task you delegate, then becomes work that runs without you. Neo isn’t generating infrastructure from a template. It’s a teammate who knows your code, your providers, your conventions, your production metrics, and can raise PRs for you to review.

Neo now lives in your terminal, in your pull requests, in your Slack workspace, and in Pulumi Cloud. Pick one of these workflows and give it a try.

The observant reader will notice Terraform-to-Pulumi was covered in the original post. ↩︎
Also covered in the original post. Last year you could ask Neo to do it once. This year you can put it on a schedule. ↩︎
Also covered in the original post. Last year Neo could remediate violations on demand. This year Security Hub feeds findings to a scheduled task that knows your runbook’s interpretation of each control. ↩︎
For a concrete example, see Seven Rules for Building an AI-Native Software Factory: Ewan Dawson, CTO of Compostable AI, runs nineteen client deployments with five engineers, using Pulumi Neo to handle most of the infrastructure work. ↩︎

Scan AWS GovCloud and more partitions with Pulumi Insights

Alejandro Cotroneo — Tue, 14 Apr 2026 09:00:00 +0000

Pulumi Insights account scanning now supports every AWS partition. If your workloads run in GovCloud, China, the European Sovereign Cloud, or one of the ISO intelligence-community clouds, you can get the same resource discovery, cross-account search, and AI-assisted insights that commercial accounts already have.

Supported partitions

AWS Standard (Commercial)
AWS GovCloud (US)
AWS ISO (US)
AWS ISOB (US)
AWS ISOF (US)
AWS ISOE (Europe)
AWS European Sovereign Cloud
AWS China

You can also exclude specific regions from discovery — useful when regions are disabled by SCPs or fall outside an audit’s scope.

Set it up

In the Pulumi Cloud console:

Go to Accounts → Create account.
Select AWS as the provider.
Under Add your configuration, pick the target partition.
Supply credentials via a Pulumi ESC environment. The OIDC trust policy uses the partition-appropriate ARN prefix (arn:aws-us-gov:, arn:aws-cn:, etc.).

For IAM and ESC setup, see the Insights accounts docs. Log in to Pulumi Cloud to get started.

How We Load Data into Snowflake in Seconds with Pulumi

Pablo Seibelt — Mon, 23 Feb 2026 00:00:00 +0000

When you manage dozens of data-loading pipelines, copying and pasting IaC configurations between them is a recipe for mishap. IAM policies can drift, naming conventions diverge, and every new source is a new opportunity to make a mistake — not to mention compound the problem of duplication. In this post, we’ll show you how you can identify and encapsulate common patterns into composable components and walk through the production lessons we’ve learned running 25+ pipelines for over three years.

What we’ll cover

If you’re loading data into Snowflake and want reusable, composable infrastructure, this post is for you. Here’s what we’ll cover:

Handling and validating GitHub webhooks with AWS Lambda
Streaming webhook payloads directly into Snowflake with Amazon Data Firehose
Wiring it all up with a reusable Pulumi ComponentResource

The companion template also includes S3 auto-ingest and batch loading patterns, which we’ll cover in upcoming posts. We also use Pulumi ESC to handle authentication to both AWS and Snowflake using OpenID Connect.

Our own Josh Kodroff wrote an excellent introduction to Snowpipe with Pulumi. This post builds on his work using the newest Snowflake and AWS provider APIs and the direct Firehose-to-Snowflake destination, which wasn’t available when Josh wrote his post. Some resource names and grant patterns will also differ if you’re comparing the two.

Architecture overview

The following diagram shows the architecture in more detail:

GitHub sends webhook events to a Lambda Function URL.
Lambda validates the HMAC (Hash-based Message Authentication Code) signature and forwards the payload to Amazon Data Firehose.
Firehose streams records directly into Snowflake via the Snowpipe Streaming API. Data appears in Snowflake within seconds.
S3 is used only as a backup destination for failed records.

The direct Firehose-to-Snowflake destination is an AWS-native feature that works with any Snowflake account.

Project setup

Start a new Pulumi Python project and choose uv for dependency management when prompted:

mkdir snowpipe-data-loading && cd snowpipe-data-loading
pulumi new python

Notice Pulumi.yaml shows uv as your selected toolchain:

name: snowpipe-data-loading
runtime:
 name: python
 options:
 toolchain: uv

Add the provider dependencies for AWS, Snowflake, GitHub, and the Random and TLS providers:

uv add pulumi-aws pulumi-snowflake pulumi-github pulumi-random pulumi-tls

That’s it. uv creates the virtual environment and lockfile automatically, and Pulumi uses uv run under the hood to execute your program.

All examples in this post are in Python, but Pulumi supports multiple languages. You can implement the same components in TypeScript, Go, .NET, Java, or YAML.

Managing credentials with Pulumi ESC

This project needs credentials for AWS, Snowflake, and GitHub. Rather than managing long-lived secrets locally, we can use Pulumi ESC to obtain dynamic, short-lived OIDC credentials for AWS and Snowflake at runtime. When you run pulumi up, ESC exchanges a Pulumi-issued OIDC token for temporary credentials from each provider and injects them into your stack config automatically. If you prefer not to use ESC, you can set credentials directly with pulumi config set --secret.

Here is a single ESC environment that handles all three providers:

values:
 aws:
 login:
 fn::open::aws-login:
 oidc:
 duration: 1h
 roleArn: arn:aws:iam::123456789012:role/pulumi-esc-oidc
 sessionName: pulumi-snowpipe-demo
 snowflake:
 login:
 fn::open::snowflake-login:
 oidc:
 account: <your-snowflake-account>
 user: ESC_SERVICE_USER
 organizationName: <your-org-name>
 accountName: <your-account-name>
 environmentVariables:
 AWS_ACCESS_KEY_ID: ${aws.login.accessKeyId}
 AWS_SECRET_ACCESS_KEY: ${aws.login.secretAccessKey}
 AWS_SESSION_TOKEN: ${aws.login.sessionToken}
 SNOWFLAKE_USER: ${snowflake.login.user}
 SNOWFLAKE_TOKEN: ${snowflake.login.token}
 pulumiConfig:
 aws:region: us-west-2
 snowflake:organizationName: ${snowflake.organizationName}
 snowflake:accountName: ${snowflake.accountName}
 snowflake:authenticator: OAUTH
 snowflake:role: PULUMI_DEPLOYER
 github:token:
 fn::secret: <your-github-pat>
 github:owner: <your-github-org-or-user>

Then reference the environment from your stack config:

# Pulumi.<stack>.yaml
environment:
 - <project>/my-snowpipe-env
config:
 snowpipe-data-loading:database: LANDING_ZONE_WEBHOOKS
 snowpipe-data-loading:environment: dev
 snowpipe-data-loading:webhook-repo: <your-test-repo-name>
 snowflake:previewFeaturesEnabled:
 - snowflake_table_resource

Depending on your preferences, you can split credentials into separate per-provider environments and compose them with imports and reuse across stacks.

To set up OIDC trust for each provider, see the AWS OIDC guide and the Snowflake OIDC login guide. For GitHub authentication options (fine-grained PATs, classic PATs, or GitHub Apps), see the pulumi-github provider docs.

Shared infrastructure

The direct streaming pipeline needs an S3 bucket for backup/errors, a Snowflake database, and a schema.

Building the direct ingestion ComponentResource

Amazon Data Firehose supports Snowflake as a native destination via the Snowpipe Streaming API. Firehose streams records directly into Snowflake.

The Lambda handler

The Lambda function is the entry point for GitHub webhooks. It validates the HMAC-SHA256 signature, wraps the payload in an envelope with the event type, and forwards it to Firehose. Create this as lambda/webhook_handler.py:

import hashlib
import hmac
import json
import os

import boto3

firehose = boto3.client("firehose")

STREAM_NAME = os.environ["FIREHOSE_STREAM_NAME"]
WEBHOOK_SECRET = os.environ["WEBHOOK_SECRET"]


def handler(event, context):
 body = event.get("body", "")
 signature = (event.get("headers") or {}).get("x-hub-signature-256", "")

 # Validate HMAC-SHA256 signature
 expected = "sha256=" + hmac.new(
 WEBHOOK_SECRET.encode(), body.encode(), hashlib.sha256
 ).hexdigest()

 if not hmac.compare_digest(expected, signature):
 return {"statusCode": 401, "body": "Invalid signature"}

 github_event = (event.get("headers") or {}).get("x-github-event", "unknown")

 # Wrap in envelope - newline-delimited for S3 backup where Firehose concatenates records
 record = json.dumps({
 "github_event": github_event,
 "payload": json.loads(body),
 }) + "\n"

 firehose.put_record(
 DeliveryStreamName=STREAM_NAME,
 Record={"Data": record.encode()},
 )

 return {"statusCode": 200, "body": "OK"}

The envelope format {"github_event": "<type>", "payload": {...}}\n is important. The github_event field (e.g., push, pull_request, star) comes from the x-github-event header and lets downstream queries filter by event type. The trailing newline delimits records in the S3 backup destination, where Firehose concatenates them into files.

Another interesting part of this snippet is that instead of manually creating a secret which would have to be copy-pasted into both your webhook configuration and your Lambda environment, we are using random.RandomPassword to generate and store it securely it in Pulumi state.

The secret is automatically wired to both the Lambda env var and the GitHub webhook config, and it rotates cleanly if you ever need to replace it.

Why direct Firehose to Snowflake?

Data like this is normally written to and loaded from Amazon S3. But with an S3 intermediate path, you must wait for Firehose to buffer records (60 seconds), then for Snowpipe to detect the new file and load it. Total latency: about two minutes. With the direct Snowflake destination, Firehose uses the Snowpipe Streaming API to insert records as soon as they arrive, in seconds.

The direct path also removes the need for S3 event notifications, SQS queues, external stages, and pipe resources. S3 is still used, but only as a backup destination for failed records.

The DirectSnowflakeIngestion component

The component creates everything needed for the direct path: a TLS key pair for Snowflake authentication, a Snowflake service user with least-privilege grants, the landing table, a Firehose delivery stream with destination="snowflake", and a Lambda function with a public URL. Create an empty components/__init__.py so that Python treats the directory as a package.

The full component lives in components/direct_snowflake_ingestion.py:

import json
from dataclasses import dataclass

import pulumi
import pulumi_aws as aws
import pulumi_snowflake as snowflake
import pulumi_tls as tls

# In the example repository, you will find this class imported from a common library file instead
@dataclass
class ColumnDef:
 """Column definition for a Snowflake table."""

 name: str
 type: str
 nullable: bool = True

def strip_pem_headers(pem: str) -> str:
 """Remove PEM header/footer lines, returning only the base64 content."""
 lines = pem.strip().split("\n")
 return "".join(lines[1:-1])


@dataclass
class DirectSnowflakeIngestionArgs:
 bucket_arn: pulumi.Input[str]
 bucket_name: pulumi.Input[str]
 database: pulumi.Input[str]
 schema_name: pulumi.Input[str]
 table_name: str
 table_columns: list[ColumnDef]
 snowflake_account_url: pulumi.Input[str]
 snowflake_role_name: str
 lambda_code: pulumi.Archive
 lambda_handler: str
 lambda_environment: dict[str, pulumi.Input[str]]
 table_comment: str = ""
 s3_prefix: str = "direct-webhooks"
 s3_backup_mode: str = "FailedDataOnly"
 buffering_interval: int = 0
 buffering_size: int = 1
 retry_duration: int = 60
 data_loading_option: str = "VARIANT_CONTENT_AND_METADATA_MAPPING"
 content_column_name: str = "CONTENT"
 metadata_column_name: str = "METADATA"

class DirectSnowflakeIngestion(pulumi.ComponentResource):
 function_url: pulumi.Output[str]
 firehose_stream_name: pulumi.Output[str]
 snowflake_user_name: pulumi.Output[str]

 def __init__(
 self,
 name: str,
 args: DirectSnowflakeIngestionArgs,
 opts: pulumi.ResourceOptions | None = None,
 ):
 super().__init__(
 "snowpipe:direct:DirectSnowflakeIngestion", name, {}, opts
 )

 # --- TLS key pair for Snowflake auth ---
 key_pair = tls.PrivateKey(
 f"{name}-keypair",
 algorithm="RSA",
 rsa_bits=2048,
 opts=pulumi.ResourceOptions(parent=self),
 )

 # --- Snowflake role, user, and grants ---
 sf_role = snowflake.AccountRole(
 f"{name}-sf-role",
 name=args.snowflake_role_name,
 opts=pulumi.ResourceOptions(parent=self),
 )

 user_name = f"FIREHOSE_{name.upper().replace('-', '_')}_USER"
 sf_user = snowflake.ServiceUser(
 f"{name}-sf-user",
 name=user_name,
 login_name=user_name,
 default_role=sf_role.name,
 rsa_public_key=key_pair.public_key_pem.apply(strip_pem_headers),
 opts=pulumi.ResourceOptions(parent=self),
 )

 # Landing table
 table = snowflake.Table(
 f"{name}-table",
 name=args.table_name,
 database=args.database,
 schema=args.schema_name,
 comment=args.table_comment,
 columns=[
 snowflake.TableColumnArgs(
 name=col.name, type=col.type, nullable=col.nullable,
 )
 for col in args.table_columns
 ],
 opts=pulumi.ResourceOptions(parent=self),
 )

 # Grants: DB USAGE, schema USAGE, table INSERT+SELECT
 snowflake.GrantPrivilegesToAccountRole(
 f"{name}-grant-db-usage",
 account_role_name=sf_role.name,
 privileges=["USAGE"],
 on_account_object=snowflake.GrantPrivilegesToAccountRoleOnAccountObjectArgs(
 object_type="DATABASE",
 object_name=args.database,
 ),
 opts=pulumi.ResourceOptions(parent=self),
 )

 snowflake.GrantPrivilegesToAccountRole(
 f"{name}-grant-schema-usage",
 account_role_name=sf_role.name,
 privileges=["USAGE"],
 on_schema=snowflake.GrantPrivilegesToAccountRoleOnSchemaArgs(
 schema_name=pulumi.Output.all(
 args.database, args.schema_name
 ).apply(lambda parts: f'"{parts[0]}"."{parts[1]}"'),
 ),
 opts=pulumi.ResourceOptions(parent=self),
 )

 table_name = args.table_name
 snowflake.GrantPrivilegesToAccountRole(
 f"{name}-grant-table",
 account_role_name=sf_role.name,
 privileges=["INSERT", "SELECT"],
 on_schema_object=snowflake.GrantPrivilegesToAccountRoleOnSchemaObjectArgs(
 object_type="TABLE",
 object_name=pulumi.Output.all(
 args.database, args.schema_name
 ).apply(
 lambda parts: f'"{parts[0]}"."{parts[1]}"."{table_name}"'
 ),
 ),
 opts=pulumi.ResourceOptions(parent=self, depends_on=[table]),
 )

 # --- Firehose IAM role (S3 backup write) ---
 firehose_role = aws.iam.Role(
 f"{name}-firehose-role",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Action": "sts:AssumeRole",
 "Principal": {"Service": "firehose.amazonaws.com"},
 }],
 }),
 opts=pulumi.ResourceOptions(parent=self),
 )

 aws.iam.RolePolicy(
 f"{name}-firehose-s3-policy",
 role=firehose_role.id,
 policy=args.bucket_arn.apply(
 lambda arn: json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Action": [
 "s3:AbortMultipartUpload",
 "s3:GetBucketLocation",
 "s3:GetObject",
 "s3:ListBucket",
 "s3:ListBucketMultipartUploads",
 "s3:PutObject",
 ],
 "Resource": [arn, f"{arn}/*"],
 }],
 })
 ),
 opts=pulumi.ResourceOptions(parent=self),
 )

 # --- Firehose delivery stream (Snowflake destination) ---
 stream = aws.kinesis.FirehoseDeliveryStream(
 f"{name}-firehose",
 destination="snowflake",
 snowflake_configuration=aws.kinesis.FirehoseDeliveryStreamSnowflakeConfigurationArgs(
 account_url=args.snowflake_account_url,
 database=args.database,
 schema=args.schema_name,
 table=args.table_name,
 role_arn=firehose_role.arn,
 user=sf_user.name,
 private_key=key_pair.private_key_pem_pkcs8.apply(
 strip_pem_headers
 ),
 data_loading_option=args.data_loading_option,
 content_column_name=args.content_column_name,
 metadata_column_name=args.metadata_column_name,
 s3_backup_mode=args.s3_backup_mode,
 buffering_size=args.buffering_size,
 buffering_interval=args.buffering_interval,
 retry_duration=args.retry_duration,
 snowflake_role_configuration=aws.kinesis.FirehoseDeliveryStreamSnowflakeConfigurationSnowflakeRoleConfigurationArgs(
 enabled=True,
 snowflake_role=args.snowflake_role_name,
 ),
 s3_configuration=aws.kinesis.FirehoseDeliveryStreamSnowflakeConfigurationS3ConfigurationArgs(
 bucket_arn=args.bucket_arn,
 role_arn=firehose_role.arn,
 prefix=f"{args.s3_prefix}/backup/",
 error_output_prefix=f"{args.s3_prefix}/errors/",
 ),
 ),
 opts=pulumi.ResourceOptions(parent=self, depends_on=[table]),
 )

 # --- Lambda function + Function URL ---
 lambda_role = aws.iam.Role(
 f"{name}-lambda-role",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Action": "sts:AssumeRole",
 "Principal": {"Service": "lambda.amazonaws.com"},
 }],
 }),
 opts=pulumi.ResourceOptions(parent=self),
 )

 aws.iam.RolePolicyAttachment(
 f"{name}-lambda-basic-execution",
 role=lambda_role.name,
 policy_arn="arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole",
 opts=pulumi.ResourceOptions(parent=self),
 )

 aws.iam.RolePolicy(
 f"{name}-lambda-firehose-policy",
 role=lambda_role.id,
 policy=stream.arn.apply(
 lambda arn: json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Action": ["firehose:PutRecord"],
 "Resource": [arn],
 }],
 })
 ),
 opts=pulumi.ResourceOptions(parent=self),
 )

 env_vars = {
 **args.lambda_environment,
 "FIREHOSE_STREAM_NAME": stream.name,
 }

 fn = aws.lambda_.Function(
 f"{name}-handler",
 runtime="python3.11",
 handler=args.lambda_handler,
 role=lambda_role.arn,
 timeout=30,
 code=args.lambda_code,
 environment=aws.lambda_.FunctionEnvironmentArgs(
 variables=env_vars,
 ),
 opts=pulumi.ResourceOptions(parent=self),
 )

 fn_url = aws.lambda_.FunctionUrl(
 f"{name}-function-url",
 function_name=fn.name,
 authorization_type="NONE",
 opts=pulumi.ResourceOptions(parent=self),
 )

 aws.lambda_.Permission(
 f"{name}-function-url-permission",
 action="lambda:InvokeFunctionUrl",
 function=fn.name,
 principal="*",
 function_url_auth_type="NONE",
 opts=pulumi.ResourceOptions(parent=self),
 )

 # --- Outputs ---
 self.function_url = fn_url.function_url
 self.firehose_stream_name = stream.name
 self.snowflake_user_name = sf_user.name

 self.register_outputs({
 "function_url": self.function_url,
 "firehose_stream_name": self.firehose_stream_name,
 "snowflake_user_name": self.snowflake_user_name,
 })

A few things to note:

TLS key pair for authentication. The component generates an RSA key pair using the pulumi-tls provider. The public key is assigned to the Snowflake service user; the private key (PKCS#8 format, base64-encoded) is passed to Firehose. No passwords or OAuth tokens are stored.
ServiceUser instead of User. Snowflake service users can’t log in interactively. They authenticate only via key pair, which is exactly what Firehose needs.
destination="snowflake" on Firehose. This tells Firehose to use the Snowpipe Streaming API rather than writing to S3. The s3_configuration block is still required, but only for backup/error records.
Immediate flushing. buffering_interval=0 and buffering_size=1 ensure records are sent to Snowflake as soon as they arrive, minimizing latency. Tune according to your needs.

Amazon Data Firehose does not connect from fixed IP addresses, so you cannot use Snowflake network policies to restrict access by IP. If your Snowflake account uses network policies, you have three options: use AWS PrivateLink (requires Snowflake Business Critical edition), allow public internet access for the Firehose service user, or switch to S3 auto-ingest via Snowpipe which does not require direct network access to Snowflake from Firehose.

Wiring it together

With the component defined, __main__.py wires the direct ingestion pipeline:

import pulumi
import pulumi_aws as aws
import pulumi_github as github
import pulumi_random as random
import pulumi_snowflake as snowflake
from components.direct_snowflake_ingestion import (
 DirectSnowflakeIngestion, DirectSnowflakeIngestionArgs,
)
from components.snowpipe_pipeline import ColumnDef

config = pulumi.Config()
database_name = config.get("database") or "LANDING_ZONE_WEBHOOKS"
environment = config.get("environment") or "dev"

# --- Shared infrastructure ---

# S3 bucket for backup/errors
bucket = aws.s3.Bucket(
 "data-landing-bucket",
)

# Snowflake database and schema
database = snowflake.Database("demo-database", name=database_name)
schema = snowflake.Schema(
 "demo-schema",
 name="GITHUB",
 database=database.name,
)

# --- Direct ingestion pipeline ---

webhook_repo = config.require("webhook-repo")

# Snowflake account URL for Firehose configuration
snowflake_config = pulumi.Config("snowflake")
snowflake_account_url = (
 f"https://{snowflake_config.require('organizationName')}"
 f"-{snowflake_config.require('accountName')}"
 f".snowflakecomputing.com"
)

# Landing table columns: CONTENT (webhook JSON) + METADATA (Firehose metadata)
DIRECT_COLUMNS = [
 ColumnDef(name="CONTENT", type="VARIANT", nullable=True),
 ColumnDef(name="METADATA", type="VARIANT", nullable=True),
]

# Step 1: Generate webhook secret for HMAC validation
direct_webhook_secret = random.RandomPassword(
 "github-direct-webhook-secret", length=32, special=False
)

# Step 2: Direct ingestion pipeline - Lambda validates, Firehose streams to Snowflake
direct = DirectSnowflakeIngestion(
 "github-webhooks-direct",
 DirectSnowflakeIngestionArgs(
 bucket_arn=bucket.arn,
 bucket_name=bucket.bucket,
 database=database.name,
 schema_name=schema.name,
 table_name="REPOSITORY_EVENTS_DIRECT",
 table_columns=DIRECT_COLUMNS,
 table_comment="GitHub webhook events loaded via direct Firehose to Snowflake",
 snowflake_account_url=snowflake_account_url,
 snowflake_role_name="FIREHOSE_DIRECT_LOADER",
 lambda_code=pulumi.AssetArchive({
 "webhook_handler.py": pulumi.FileAsset("lambda/webhook_handler.py"),
 }),
 lambda_handler="webhook_handler.handler",
 lambda_environment={"WEBHOOK_SECRET": direct_webhook_secret.result},
 ),
)

# Step 3: GitHub webhook - sends events to the Lambda Function URL
github.RepositoryWebhook(
 "github-direct-webhook",
 repository=webhook_repo,
 configuration=github.RepositoryWebhookConfigurationArgs(
 url=direct.function_url,
 content_type="json",
 secret=direct_webhook_secret.result,
 ),
 events=["push", "pull_request", "issues", "star"],
)

# Exports
pulumi.export("webhook_url", direct.function_url)
pulumi.export("firehose_stream", direct.firehose_stream_name)

That’s the entire pipeline. One component, one GitHub webhook, one secret. The DirectSnowflakeIngestion component handles the TLS key pair, Snowflake service user, landing table, Firehose stream, and Lambda function internally, and now you can reuse this component for as many pipelines as you need.

The full code for this example is available on GitHub:

github.com/pulumi-demos/examples/tree/main/python/aws-snowflake-data-loading-real-time

Testing the pipeline

Deploy the stack:

pulumi up

The entire stack deploys in about two minutes. Immediately after deployment, you’ll start seeing GitHub events flowing into Snowflake.

Before querying, grant the DATA_READER role to your Snowflake user:

GRANT ROLE DATA_READER TO USER <your-user>;

In production, you can manage this grant through Pulumi, manually, or automatically via SCIM provisioning from your identity provider.

No need to craft test payloads. Just interact with the test repo. Star it, push a commit, or open an issue, then wait about 30 seconds and query Snowflake using the least-privilege reader role:

USE ROLE DATA_READER;

SELECT CONTENT:github_event::STRING AS event_type,
 CONTENT:payload:repository:full_name::STRING AS repo,
 METADATA:IngestionTime::TIMESTAMP AS ingested_at
FROM LANDING_ZONE_WEBHOOKS.GITHUB.REPOSITORY_EVENTS_DIRECT
ORDER BY ingested_at DESC;

You should see rows with event types like star, push, or issues, real GitHub events flowing through the entire pipeline. The METADATA column includes Firehose metadata like IngestionTime, which you can use to track end-to-end latency.

Other loading patterns

Direct streaming is the fastest path, but two other patterns are available in the companion template for different requirements:

S3 auto-ingest via Snowpipe. Firehose buffers to S3, and Snowpipe auto-ingests new files. Latency is about two minutes. Best when you need S3 as the system of record or can’t use direct Snowpipe Streaming.
Batch loading. Your orchestrator (Airflow, Prefect, cron, etc.) runs COPY INTO on a schedule. Best for full control over timing and deduplication.

We’ll walk through both patterns in detail in upcoming posts.

Publishing as reusable components

Once your components are battle-tested, you can share them across teams and projects instead of copying files around.

The most straightforward approach: push your components to a Git repository with a PulumiPlugin.yaml file at the root:

runtime: python
name: snowpipe-components
version: 1.0.0

Consumers add the package to their project with pulumi package add:

pulumi package add github.com/your-org/pulumi-snowpipe@v1.0.0

Pulumi downloads the package and generates typed SDKs automatically. The consumer’s __main__.py imports your components as if they were local, but they’re versioned and pinned.

Pulumi Cloud Private Registry

For organization-level discoverability, publish to the Pulumi Cloud Private Registry with pulumi package publish:

pulumi package publish ./schema.json

This gives you auto-generated API docs, usage tracking across teams, and cross-language SDK generation. Your Python components become usable from TypeScript, Go, and C# without any extra work. Teams browse available components in the Pulumi Cloud console, see who’s using what, and get notified when new versions are published. As an additional benefit, Neo will be able to use these components and build new pipelines in minutes from a natural language request.

Conclusion

ComponentResource is the key abstraction that makes this architecture scale. Instead of copying and pasting resources for each new data source, you instantiate a component with a handful of configuration parameters.

The DirectSnowflakeIngestion component in this post delivers data from GitHub webhooks into Snowflake in seconds: Lambda validates the HMAC signature, Firehose streams directly to Snowflake via the Snowpipe Streaming API, and the TLS key pair is managed entirely within Pulumi. No S3 intermediate, no SQS queues.

The component accepts pluggable Lambda handlers, so swapping GitHub for Stripe webhooks or any other source is just a matter of providing different lambda_code and lambda_environment arguments. We’ve been running this pattern in production for over three years across dozens of pipelines without significant changes to the infrastructure code.

You’ll find the complete example in the GitHub repository.

Passwordless PostgreSQL: IAM Authentication with Pulumi

Elisabeth Lichtie — Fri, 13 Feb 2026 00:00:00 +0000

Managing database credentials is one of the persistent challenges in cloud infrastructure. Passwords need to be rotated, secrets need to be stored securely, and access needs to be carefully controlled. AWS IAM authentication for RDS offers a better way: instead of managing long-lived passwords, your applications authenticate using short-lived tokens generated from IAM credentials. This approach is more secure, eliminates password rotation overhead, and integrates seamlessly with your existing IAM policies. With Pulumi, you can set up this entire system using reusable components that make IAM authentication a standard part of your infrastructure.

Why IAM authentication for PostgreSQL?

Traditional database authentication relies on usernames and passwords. These credentials need to be stored somewhere secure, rotated regularly, and distributed to applications that need them. Each of these steps introduces complexity and potential security risks.

IAM authentication changes this model fundamentally. Instead of passwords, your applications generate authentication tokens on demand using their IAM credentials. These tokens are valid for only 15 minutes, eliminating the need for password rotation. Access control happens through IAM policies, which you’re already using to manage other AWS resources. The result is a more secure system that’s easier to maintain and audit.

The benefits become even more pronounced when you componentize this setup with Pulumi. Rather than repeating the same configuration steps for each database, you can build reusable components that handle IAM authentication setup automatically. This turns a complex, multi-step process into a simple, repeatable pattern that your entire team can use. In this post, we’ll cover an example that sets up the complete flow: an RDS cluster with IAM authentication, the necessary IAM roles and policies, and a Kubernetes application that connects using IAM tokens.

Architecture overview

This example sets up a complete environment with IAM-authenticated database access from a Kubernetes application. The architecture includes:

VPC with public and private subnets across multiple availability zones
EKS cluster for running containerized applications
Aurora PostgreSQL cluster with IAM authentication enabled
IAM roles and policies that connect Kubernetes service accounts to database access
A demo application that authenticates to the database using IAM tokens

The key integration point is IAM Roles for Service Accounts (IRSA), which allows Kubernetes pods to assume IAM roles. This means your application doesn’t need any AWS credentials stored in environment variables or mounted secrets. The pod’s service account automatically has the permissions it needs to generate database authentication tokens.

Setting up the RDS cluster with IAM authentication

The foundation of this setup is an RDS cluster configured to accept IAM authentication. With Pulumi’s componentized approach, you can encapsulate all the RDS configuration in a reusable component:

const rdsCluster = new RdsCluster("iam-postgres", {
 vpcId: vpc.vpc.id,
 vpcCidrBlock: vpc.vpc.cidrBlock,
 subnetIds: vpc.publicSubnets.map((s) => s.id),
 databaseName: DATABASE_NAME,
 masterUsername: MASTER_USERNAME,
 masterPassword: dbMasterPassword,
 instanceClass: "db.t4g.medium",
 engineVersion: "17.4",
 iamDatabaseUser: IAM_DB_USERNAME,
});

Inside the component, the critical configuration is iamDatabaseAuthenticationEnabled:

this.cluster = new aws.rds.Cluster(
 `${name}-cluster`,
 {
 engine: "aurora-postgresql",
 engineVersion: engineVersion,
 databaseName: args.databaseName,
 masterUsername: args.masterUsername,
 masterPassword: args.masterPassword,
 dbSubnetGroupName: subnetGroup.name,
 vpcSecurityGroupIds: [this.securityGroup.id],
 iamDatabaseAuthenticationEnabled: true,
 skipFinalSnapshot: true,
 tags: { Name: `${name}-cluster` },
 },
 { parent: this }
);

This single flag tells RDS to accept authentication tokens in addition to traditional passwords. The cluster still needs a master password for initial setup and administrative tasks, but your applications can use IAM authentication instead.

Creating the database user with IAM permissions

Enabling IAM authentication on the cluster isn’t enough. You also need to create a database user and grant it the special rds_iam role that allows IAM authentication. This is where Pulumi’s PostgreSQL provider comes in:

const dbSetup = new DbSetup(
 "iam-postgres",
 {
 dbEndpoint: rdsCluster.cluster.endpoint,
 dbName: DATABASE_NAME,
 masterUsername: MASTER_USERNAME,
 masterPassword: dbMasterPassword,
 iamUsername: IAM_DB_USERNAME,
 },
 { dependsOn: [rdsCluster.instance] }
);

The DbSetup component handles all the database-level configuration. It creates the IAM user and grants the necessary permissions:

// Create IAM-enabled database user
this.iamRole = new postgresql.Role(
 `${name}-iam-user`,
 {
 name: args.iamUsername,
 login: true,
 },
 { parent: this, provider: this.provider }
);

// Grant rds_iam role to enable IAM authentication
new postgresql.GrantRole(
 `${name}-grant-rds-iam`,
 {
 role: this.iamRole.name,
 grantRole: "rds_iam",
 },
 { parent: this, provider: this.provider }
);

The component also grants the necessary database privileges (CONNECT, USAGE, CREATE, and table operations). By encapsulating this in a component, you ensure that every IAM-authenticated database user is set up consistently with the correct permissions.

Configuring IAM policies for database access

The next piece is the IAM role and policy that allows applications to generate authentication tokens. This involves two parts: the role that the application assumes, and the policy that grants database access.

For Kubernetes applications, this uses IRSA (IAM Roles for Service Accounts). The RdsCluster component includes a method that creates the appropriate IAM role:

const rdsIamRole = rdsCluster.createIamRole(
 eksCluster.oidcProvider,
 NAMESPACE,
 SERVICE_ACCOUNT_NAME
);

This creates an IAM role with a trust policy that allows the specified Kubernetes service account to assume it:

assumeRolePolicy: pulumi
 .all([oidcProvider.arn, oidcProvider.url])
 .apply(([arn, url]) =>
 JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Effect: "Allow",
 Principal: { Federated: arn },
 Action: "sts:AssumeRoleWithWebIdentity",
 Condition: {
 StringEquals: {
 [`${url}:sub`]: `system:serviceaccount:${namespace}:${serviceAccountName}`,
 },
 },
 },
 ],
 })
 ),

The role is then granted the rds-db:connect permission for the specific database user:

{
 Version: "2012-10-17",
 Statement: [
 {
 Effect: "Allow",
 Action: ["rds-db:connect"],
 Resource: `arn:aws:rds-db:${region}:${accountId}:dbuser:${resourceId}/${iamDatabaseUser}`,
 },
 ],
}

This policy is scoped to exactly one database user on one cluster. This level of granularity is one of the security advantages of IAM authentication: you can control database access with the same precision you use for other AWS resources.

Connecting from Kubernetes with IRSA

The application running in Kubernetes needs a service account annotated with the IAM role ARN:

this.serviceAccount = new k8s.core.v1.ServiceAccount(
 `${name}-sa`,
 {
 metadata: {
 name: args.serviceAccountName,
 namespace: args.namespace,
 annotations: {
 "eks.amazonaws.com/role-arn": args.iamRoleArn,
 },
 },
 },
 { parent: this, provider: args.provider }
);

When a pod uses this service account, EKS automatically configures the pod with temporary AWS credentials for the associated IAM role. The application can then use these credentials to generate database authentication tokens.

How IAM authentication works at runtime

The authentication flow happens at connection time:

%%{init: {'theme':'base', 'themeVariables': { 'primaryColor':'#FF9900','primaryTextColor':'#232F3E','primaryBorderColor':'#232F3E','lineColor':'#666','secondaryColor':'#527FFF','tertiaryColor':'#fff'}}}%%
sequenceDiagram
autonumber
participant App as Application Pod
participant SDK as AWS SDK
participant STS as AWS STS
participant IAM as AWS IAM
participant RDS as RDS Aurora
rect rgb(255, 249, 230)
Note over App,SDK: Phase 1: Token Generation
activate App
App->>+SDK: generate_db_auth_token()
SDK->>+STS: Get temporary credentials (IRSA)
STS-->>-SDK: Return AWS credentials
SDK->>SDK: Sign auth request
SDK-->>-App: Return IAM token (15 min TTL)
deactivate App
end
rect rgb(235, 245, 255)
Note over App,RDS: Phase 2: Database Connection
activate App
App->>+RDS: Connect (user + IAM token)
activate RDS
RDS->>+IAM: Validate token & permissions
IAM-->>-RDS: Token valid + rds-db:connect allowed
RDS->>RDS: Verify rds_iam role
RDS-->>-App: Connection established
deactivate App
end
Note over App,RDS: 💡 Token expires after 15 minutes - generate new token per connection

The application uses the AWS SDK to call generate_db_auth_token(), passing the database endpoint, port, and username
The SDK uses the pod’s IAM credentials (provided automatically by IRSA) to sign the request and generate a token
The application connects to PostgreSQL using the IAM username and the token as the password
RDS validates the token against IAM, verifying that the caller has rds-db:connect permission for that database user
If the token is valid, the connection is established

Here’s what this looks like in Python:

def get_iam_token():
 """Generate an IAM authentication token for RDS"""
 client = boto3.client('rds', region_name=AWS_REGION)
 token = client.generate_db_auth_token(
 DBHostname=DB_ENDPOINT,
 Port=DB_PORT,
 DBUsername=DB_USER,
 Region=AWS_REGION
 )
 return token

def get_db_connection():
 """Create a database connection with IAM auth"""
 token = get_iam_token()
 return psycopg2.connect(
 host=DB_ENDPOINT,
 port=DB_PORT,
 database=DB_NAME,
 user=DB_USER,
 password=token,
 sslmode='require',
 sslrootcert=RDS_CA_CERT
 )

The token is valid for 15 minutes. Applications should generate a new token for each connection or implement token caching with refresh logic.

Testing the setup with the demo application

The example includes an interactive demo application that lets you test the IAM authentication setup. Once deployed, the application provides a web interface for creating tables, adding messages, and querying the database:

Deploy the infrastructure and access the demo app:

pulumi up
export APP_URL=$(pulumi stack output appUrl)
echo "Demo app: http://$APP_URL"

The application shows the database endpoint and IAM username it’s using, and provides buttons to create tables and add data. Behind the scenes, every database operation authenticates using IAM tokens, demonstrating that the entire authentication flow is working correctly.

The full code for this example is available at https://github.com/pulumi-demos/examples/tree/main/typescript/aws-iam-for-postgres. The example includes all the component code referenced in this post.

Production considerations

This example demonstrates IAM authentication in a working environment, but you’ll want to make several adjustments for production use:

Network security: The example places RDS in public subnets to allow the PostgreSQL provider to connect during deployment. In production, place RDS in private subnets and ensure your deployment environment (like GitHub Actions or Pulumi Cloud) can reach the database for initial setup, either through a bastion host or VPN.

Connection pooling and performance: IAM tokens expire after 15 minutes. If you’re using connection pooling, you’ll need logic to refresh tokens before they expire. AWS recommends using IAM authentication only when your application creates fewer than 200 new connections per second. For higher connection rates, consider using Amazon RDS Proxy, which manages connection pooling and can reduce the overhead of IAM token generation.

Master password management: You still need a master password for database administration and for the initial user setup. Store this in AWS Secrets Manager or Pulumi ESC, and restrict access to it carefully.

Monitoring and auditing: Token generation via the AWS SDK is logged in CloudTrail. Note that CloudWatch and CloudTrail do not log database authentication attempts themselves, so you’ll need to rely on PostgreSQL’s native logging for connection monitoring.

Multi-region considerations: If your application runs in multiple regions, ensure that IAM policies and database access work correctly across regions. Token generation and validation must happen in the same region as the RDS cluster.

Cost: IAM authentication itself is free, but be aware that cross-AZ data transfer costs still apply for database connections.

Conclusion

IAM authentication for PostgreSQL transforms database credential management from a security burden into a seamless part of your infrastructure. By eliminating long-lived passwords, you reduce your attack surface and remove the operational overhead of password rotation. By integrating with IAM, you gain fine-grained access control and comprehensive audit logs.

Pulumi makes this setup practical and repeatable through componentization. The components in this example encapsulate the complexity of IAM authentication setup, making it easy to apply the same pattern across multiple databases and applications. This turns security best practices into your team’s default approach.

Whether you’re building a new application or improving the security posture of existing infrastructure, IAM authentication for RDS deserves consideration. The investment in setup pays dividends in security, maintainability, and operational simplicity.

Deploy OpenClaw on AWS or Hetzner Securely with Pulumi and Tailscale

Engin Diri — Mon, 26 Jan 2026 00:00:00 +0000

Update (January 2026): The lobster has molted into its final form! From Clawdbot to Moltbot to OpenClaw. With 100k+ GitHub stars and 2M visitors in a week, the project finally has a name that’ll stick. The CLI command is now openclaw and the new handle is @openclaw. Same mission: AI that actually does things. Your assistant. Your machine. Your rules. See the official getting started guide for updated installation instructions.

Update (April 2026): Refreshed for OpenClaw 2026.4.27. Upstream now recommends Node 24, but the cloud-init script in this post still installs Node 22 — both work. If you’d like Node 24, change the nvm install 22 lines to nvm install 24.

The short version: Deploy OpenClaw to AWS or Hetzner with a Pulumi TypeScript program that provisions the VM, installs Docker, Node, and OpenClaw, then joins the instance to your Tailscale network so the gateway and browser ports stay private. One pulumi up to deploy, one pulumi destroy to tear down. Total cost: about $33/month on AWS or $7/month on Hetzner.

OpenClaw is everywhere right now. The open-source AI assistant gained 9,000 GitHub stars in a single day, received public praise from former Tesla AI head Andrej Karpathy, and has sparked a global run on Mac Minis as developers scramble to give this “lobster assistant” a home. Users are calling it “Jarvis living in a hard drive” and “Claude with hands”—the personal AI assistant that Siri promised but never delivered.

The Mac Mini craze is real: people are buying dedicated hardware just to run OpenClaw, with some enthusiasts purchasing 40 Mac Minis at once. Even Logan Kilpatrick from Google DeepMind couldn’t resist ordering one. But here’s the thing: you don’t actually need a Mac Mini. OpenClaw runs anywhere: on a VPS, in the cloud, or on that old laptop gathering dust.

With all this hype, I had to try it myself. But instead of clicking through the AWS console or running manual commands on a VPS, I wanted to do it right from the start: infrastructure as code with Pulumi. Why? Because when I inevitably want to tear it down, spin up a new instance, or deploy to a different region, I don’t want to remember which buttons I clicked three weeks ago. I want a single pulumi up command.

Dan got the assignment right:

In this post, I’ll show you how to deploy OpenClaw to AWS or Hetzner Cloud (if you want European data residency or just want to spend less). We’ll use Pulumi to define the infrastructure and Tailscale to keep your AI assistant off the public internet.

What is OpenClaw?

OpenClaw is an open-source AI assistant created by Peter Steinberger that runs on your own infrastructure. It connects to WhatsApp, Slack, Discord, Google Chat, Signal, and iMessage. It can control browsers, generate videos and images, clone your voice for voice notes, and run scheduled tasks via cron. There’s a skills system for extending functionality, and you can run it on pretty much anything: Mac Mini, Raspberry Pi, VPS, laptop, or gaming PC.

The difference from cloud-hosted AI? OpenClaw runs on your server, not Anthropic’s. It’s available 24/7 across all your devices, can schedule automated tasks, and keeps your entire conversation history locally. Check the official OpenClaw documentation for the full feature list.

Prerequisites

Before getting started, ensure you have:

Pulumi CLI installed and configured
A Pulumi Cloud account
AWS account (for AWS deployment)
Hetzner Cloud account (for European deployment)
Anthropic API key
Node.js 22+ installed (Node 24 recommended; the cloud-init script in this post installs Node 22 inside the VM)
Tailscale account with HTTPS enabled (one-time setup in admin console)

This guide uses Anthropic’s API, but OpenClaw works with other providers too. Check the providers documentation if you’d rather use OpenAI, Google Gemini, or a local model via Ollama.

Understanding OpenClaw architecture

OpenClaw uses a gateway-centric architecture where a single daemon acts as the control plane for all messaging, tool execution, and client connections:

Component	Port	Description
Gateway	18789	WebSocket server handling channels, nodes, sessions, and hooks
Browser control	18791	Headless Chrome instance for web automation
Docker sandbox	-	Isolated container environment for running tools safely

The Gateway connects to messaging platforms (WhatsApp, Slack, Discord, etc.), the CLI, the web UI, and mobile apps. The Browser component lets OpenClaw open web pages, fill forms, scrape data, and download files. Docker sandboxing runs bash commands in isolated containers so your bot can execute code without risking your host system.

Setting up ESC for secrets management

Deploying OpenClaw means handling sensitive credentials: API keys, auth tokens, cloud provider secrets. You don’t want these hardcoded or scattered across environment variables. Pulumi ESC (Environments, Secrets, and Configuration) stores them securely and passes them directly to your Pulumi program — the same pattern Pulumi uses to eliminate long-lived CI secrets across 70+ repos.

Create a new ESC environment:

pulumi env init <your-org>/openclaw-secrets

Add your secrets to the environment:

values:
 anthropicApiKey:
 fn::secret: "sk-ant-xxxxx"
 tailscaleAuthKey:
 fn::secret: "tskey-auth-xxxxx"
 tailnetDnsName: "tailxxxxx.ts.net"
 hcloudToken:
 fn::secret: "your-hetzner-api-token"
 pulumiConfig:
 anthropicApiKey: ${anthropicApiKey}
 tailscaleAuthKey: ${tailscaleAuthKey}
 tailnetDnsName: ${tailnetDnsName}
 hcloud:token: ${hcloudToken}

To find your tailnet DNS name, go to the Tailscale admin console, look under the DNS section, and find your tailnet name (e.g., tailxxxxx.ts.net). This is the domain suffix used for all machines in your Tailscale network.

Then create a Pulumi.dev.yaml file in your project to reference the environment:

environment:
 - <your-org>/openclaw-secrets

This approach keeps your secrets out of your codebase and passes them directly to OpenClaw during automated onboarding. To prevent stack-level overrides of these values, see locking down ESC values with fn::final.

Securing with Tailscale

By default, deploying OpenClaw exposes SSH (port 22), the gateway (port 18789), and browser control (port 18791) to the public internet. This is convenient for testing but not ideal for production use.

Tailscale creates a secure mesh VPN that lets you access your OpenClaw instance without exposing unnecessary ports publicly. When you provide a Tailscale auth key, the Pulumi program:

Removes gateway and browser ports from public access
Keeps SSH as fallback for debugging if Tailscale setup fails
Installs Tailscale on the instance during provisioning (after other dependencies)
Enables Tailscale SSH so you can SSH via Tailscale without managing keys
Joins your tailnet automatically using the auth key

The Pulumi program installs Docker, Node.js, and OpenClaw first, then configures Tailscale last. This ensures that even if the Tailscale auth key is invalid or expired, you can still SSH in via the public IP to troubleshoot.

To generate a Tailscale auth key:

Go to Tailscale Admin Console
Click “Generate auth key”
Enable “Reusable” if you plan to redeploy
Copy the key and add it to your ESC environment

Deploying to AWS

Let’s walk through the complete AWS deployment. Create a new Pulumi project:

mkdir openclaw-aws && cd openclaw-aws
pulumi new typescript

Install the required dependencies:

npm install @pulumi/aws @pulumi/tls

Do not use t3.micro instances for OpenClaw. The 1 GB memory is insufficient for installation. Use t3.medium (4 GB) or t3.large (8 GB) instead.

The Pulumi program

Running OpenClaw on AWS means setting up a VPC, subnets, security groups, an EC2 instance, SSH keys, and a cloud-init script that installs everything. That’s a lot of clicking in the AWS console. The Pulumi program below defines all of it in code.

Replace the contents of index.ts with the following:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as tls from "@pulumi/tls";

const config = new pulumi.Config();

const instanceType = config.get("instanceType") ?? "t3.medium";
const anthropicApiKey = config.requireSecret("anthropicApiKey");
const model = config.get("model") ?? "anthropic/claude-sonnet-4";
const enableSandbox = config.getBoolean("enableSandbox") ?? true;
const gatewayPort = config.getNumber("gatewayPort") ?? 18789;
const browserPort = config.getNumber("browserPort") ?? 18791;

const tailscaleAuthKey = config.requireSecret("tailscaleAuthKey");
const tailnetDnsName = config.require("tailnetDnsName");

// Generate a random token for gateway authentication
const gatewayToken = new tls.PrivateKey("openclaw-gateway-token", {
 algorithm: "ED25519",
}).publicKeyOpenssh.apply(key => {
 // Create a deterministic token from the public key (take first 48 hex chars)
 const hash = require("crypto").createHash("sha256").update(key).digest("hex");
 return hash.substring(0, 48);
});

const sshKey = new tls.PrivateKey("openclaw-ssh-key", {
 algorithm: "ED25519",
});

const vpc = new aws.ec2.Vpc("openclaw-vpc", {
 cidrBlock: "10.0.0.0/16",
 enableDnsHostnames: true,
 enableDnsSupport: true,
 tags: { Name: "openclaw-vpc" },
});

const gateway = new aws.ec2.InternetGateway("openclaw-igw", {
 vpcId: vpc.id,
 tags: { Name: "openclaw-igw" },
});

const subnet = new aws.ec2.Subnet("openclaw-subnet", {
 vpcId: vpc.id,
 cidrBlock: "10.0.1.0/24",
 mapPublicIpOnLaunch: true,
 tags: { Name: "openclaw-subnet" },
});

const routeTable = new aws.ec2.RouteTable("openclaw-rt", {
 vpcId: vpc.id,
 routes: [
 {
 cidrBlock: "0.0.0.0/0",
 gatewayId: gateway.id,
 },
 ],
 tags: { Name: "openclaw-rt" },
});

new aws.ec2.RouteTableAssociation("openclaw-rta", {
 subnetId: subnet.id,
 routeTableId: routeTable.id,
});

const securityGroup = new aws.ec2.SecurityGroup("openclaw-sg", {
 vpcId: vpc.id,
 description: "Security group for OpenClaw instance",
 ingress: [
 {
 description: "SSH access (fallback)",
 fromPort: 22,
 toPort: 22,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 ],
 egress: [
 {
 fromPort: 0,
 toPort: 0,
 protocol: "-1",
 cidrBlocks: ["0.0.0.0/0"],
 },
 ],
 tags: { Name: "openclaw-sg" },
});

const keyPair = new aws.ec2.KeyPair("openclaw-keypair", {
 publicKey: sshKey.publicKeyOpenssh,
});

const ami = aws.ec2.getAmiOutput({
 owners: ["099720109477"],
 mostRecent: true,
 filters: [
 { name: "name", values: ["ubuntu/images/hvm-ssd-gp3/ubuntu-noble-24.04-amd64-server-*"] },
 { name: "virtualization-type", values: ["hvm"] },
 ],
});

const userData = pulumi
 .all([tailscaleAuthKey, anthropicApiKey, gatewayToken])
 .apply(([tsAuthKey, apiKey, gwToken]) => {
 return `#!/bin/bash
set -e

export DEBIAN_FRONTEND=noninteractive

# System updates
apt-get update
apt-get upgrade -y

# Install Docker
curl -fsSL https://get.docker.com | sh
systemctl enable docker
systemctl start docker
usermod -aG docker ubuntu

# Install NVM and Node.js for ubuntu user
sudo -u ubuntu bash << 'UBUNTU_SCRIPT'
set -e
cd ~

# Install NVM
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash

# Load NVM
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

# Install Node.js 22
nvm install 22
nvm use 22
nvm alias default 22

# Install OpenClaw
npm install -g openclaw@latest

# Add NVM to bashrc if not already there
if ! grep -q 'NVM_DIR' ~/.bashrc; then
 echo 'export NVM_DIR="$HOME/.nvm"' >> ~/.bashrc
 echo '[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"' >> ~/.bashrc
fi
UBUNTU_SCRIPT

# Set environment variables for ubuntu user
echo 'export ANTHROPIC_API_KEY="${apiKey}"' >> /home/ubuntu/.bashrc

# Install and configure Tailscale
echo "Installing Tailscale..."
curl -fsSL https://tailscale.com/install.sh | sh
tailscale up --authkey="${tsAuthKey}" --ssh || echo "WARNING: Tailscale setup failed. Run 'sudo tailscale up' manually."

# Enable systemd linger for ubuntu user (required for user services to run at boot)
loginctl enable-linger ubuntu

# Start user's systemd instance (required for user services during cloud-init)
systemctl start user@1000.service

# Run OpenClaw onboarding as ubuntu user (skip daemon install, do it separately)
echo "Running OpenClaw onboarding..."
sudo -H -u ubuntu ANTHROPIC_API_KEY="${apiKey}" GATEWAY_PORT="${gatewayPort}" bash -c '
export HOME=/home/ubuntu
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

openclaw onboard --non-interactive --accept-risk \
 --mode local \
 --auth-choice apiKey \
 --gateway-port $GATEWAY_PORT \
 --gateway-bind loopback \
 --skip-daemon \
 --skip-skills || echo "WARNING: OpenClaw onboarding failed. Run openclaw onboard manually."
'

# Install daemon service with XDG_RUNTIME_DIR set
echo "Installing OpenClaw daemon..."
sudo -H -u ubuntu XDG_RUNTIME_DIR=/run/user/1000 bash -c '
export HOME=/home/ubuntu
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

openclaw daemon install || echo "WARNING: Daemon install failed. Run openclaw daemon install manually."
'

# Configure gateway for Tailscale Serve (trustedProxies + skip device pairing + set token)
echo "Configuring gateway for Tailscale Serve..."
sudo -H -u ubuntu GATEWAY_TOKEN="${gwToken}" python3 << 'PYTHON_SCRIPT'
import json
import os
config_path = "/home/ubuntu/.openclaw/openclaw.json"
with open(config_path) as f:
 config = json.load(f)
config["gateway"]["trustedProxies"] = ["127.0.0.1"]
config["gateway"]["controlUi"] = {
 "enabled": True,
 "allowInsecureAuth": True
}
config["gateway"]["auth"] = {
 "mode": "token",
 "token": os.environ["GATEWAY_TOKEN"]
}
with open(config_path, "w") as f:
 json.dump(config, f, indent=2)
print("Configured gateway with trustedProxies, controlUi, and token")
PYTHON_SCRIPT

# Enable Tailscale HTTPS proxy (requires HTTPS to be enabled in Tailscale admin console)
echo "Enabling Tailscale HTTPS proxy..."
tailscale serve --bg ${gatewayPort} || echo "WARNING: tailscale serve failed. Enable HTTPS in your Tailscale admin console first."

echo "OpenClaw setup complete!"
`;
 });

const instance = new aws.ec2.Instance("openclaw-instance", {
 ami: ami.id,
 instanceType: instanceType,
 subnetId: subnet.id,
 vpcSecurityGroupIds: [securityGroup.id],
 keyName: keyPair.keyName,
 userData: userData,
 userDataReplaceOnChange: true,
 rootBlockDevice: {
 volumeSize: 30,
 volumeType: "gp3",
 },
 tags: { Name: "openclaw" },
});

export const publicIp = instance.publicIp;
export const publicDns = instance.publicDns;
export const privateKey = sshKey.privateKeyOpenssh;

// Construct the Tailscale MagicDNS hostname from the private IP
// AWS private IPs like 10.0.1.15 become hostnames like ip-10-0-1-15
const tailscaleHostname = instance.privateIp.apply(ip =>
 `ip-${ip.replace(/\./g, "-")}`
);

export const tailscaleUrlWithToken = pulumi.interpolate`https://${tailscaleHostname}.${tailnetDnsName}/?token=${gatewayToken}`;
export const gatewayTokenOutput = gatewayToken;

Deploying to Hetzner

Hetzner Cloud is a solid choice if you need European data residency or want to spend less money. Spoiler: it’s a lot less money.

Hetzner has similar concepts to AWS but different names. EC2 instances become Servers. Security groups become Firewalls. Same idea, different provider. The resource types come from @pulumi/hcloud.

Create a new project for Hetzner:

mkdir openclaw-hetzner && cd openclaw-hetzner
pulumi new typescript

Install the Hetzner provider:

npm install @pulumi/hcloud @pulumi/tls

The default server type cax21 is an ARM-based (Ampere) instance with 4 vCPUs and 8 GB RAM. ARM instances cost less for the same compute. If you need x86 architecture, use ccx13 or similar CCX series instead.

The Hetzner Pulumi program

Replace index.ts with the following:

import * as pulumi from "@pulumi/pulumi";
import * as hcloud from "@pulumi/hcloud";
import * as tls from "@pulumi/tls";

const config = new pulumi.Config();

const serverType = config.get("serverType") ?? "cax21";
const location = config.get("location") ?? "fsn1";
const anthropicApiKey = config.requireSecret("anthropicApiKey");
const model = config.get("model") ?? "anthropic/claude-sonnet-4";
const enableSandbox = config.getBoolean("enableSandbox") ?? true;
const gatewayPort = config.getNumber("gatewayPort") ?? 18789;
const browserPort = config.getNumber("browserPort") ?? 18791;

const tailscaleAuthKey = config.requireSecret("tailscaleAuthKey");
const tailnetDnsName = config.require("tailnetDnsName");

// Generate a random token for gateway authentication
const gatewayToken = new tls.PrivateKey("openclaw-gateway-token", {
 algorithm: "ED25519",
}).publicKeyOpenssh.apply(key => {
 const hash = require("crypto").createHash("sha256").update(key).digest("hex");
 return hash.substring(0, 48);
});

const sshKey = new tls.PrivateKey("openclaw-ssh-key", {
 algorithm: "ED25519",
});

const hcloudSshKey = new hcloud.SshKey("openclaw-sshkey", {
 publicKey: sshKey.publicKeyOpenssh,
});

const firewallRules: hcloud.types.input.FirewallRule[] = [
 {
 direction: "out",
 protocol: "tcp",
 port: "any",
 destinationIps: ["0.0.0.0/0", "::/0"],
 description: "Allow all outbound TCP",
 },
 {
 direction: "out",
 protocol: "udp",
 port: "any",
 destinationIps: ["0.0.0.0/0", "::/0"],
 description: "Allow all outbound UDP",
 },
 {
 direction: "out",
 protocol: "icmp",
 destinationIps: ["0.0.0.0/0", "::/0"],
 description: "Allow all outbound ICMP",
 },
 {
 direction: "in",
 protocol: "tcp",
 port: "22",
 sourceIps: ["0.0.0.0/0", "::/0"],
 description: "SSH access (fallback)",
 },
];

const firewall = new hcloud.Firewall("openclaw-firewall", {
 rules: firewallRules,
});

const userData = pulumi
 .all([tailscaleAuthKey, anthropicApiKey, gatewayToken])
 .apply(([tsAuthKey, apiKey, gwToken]) => {
 return `#!/bin/bash
set -e

export DEBIAN_FRONTEND=noninteractive

# System updates
apt-get update
apt-get upgrade -y

# Install Docker
curl -fsSL https://get.docker.com | sh
systemctl enable docker
systemctl start docker

# Create ubuntu user (Hetzner uses root by default)
useradd -m -s /bin/bash -G docker ubuntu || true

# Install NVM and Node.js for ubuntu user
sudo -u ubuntu bash << 'UBUNTU_SCRIPT'
set -e
cd ~

# Install NVM
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash

# Load NVM
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

# Install Node.js 22
nvm install 22
nvm use 22
nvm alias default 22

# Install OpenClaw
npm install -g openclaw@latest

# Add NVM to bashrc if not already there
if ! grep -q 'NVM_DIR' ~/.bashrc; then
 echo 'export NVM_DIR="$HOME/.nvm"' >> ~/.bashrc
 echo '[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"' >> ~/.bashrc
fi
UBUNTU_SCRIPT

# Set environment variables for ubuntu user
echo 'export ANTHROPIC_API_KEY="${apiKey}"' >> /home/ubuntu/.bashrc

# Install and configure Tailscale
echo "Installing Tailscale..."
curl -fsSL https://tailscale.com/install.sh | sh
tailscale up --authkey="${tsAuthKey}" --ssh || echo "WARNING: Tailscale setup failed. Run 'sudo tailscale up' manually."

# Enable systemd linger for ubuntu user (required for user services to run at boot)
loginctl enable-linger ubuntu

# Start user's systemd instance (required for user services during cloud-init)
systemctl start user@1000.service

# Run OpenClaw onboarding as ubuntu user (skip daemon install, do it separately)
echo "Running OpenClaw onboarding..."
sudo -H -u ubuntu ANTHROPIC_API_KEY="${apiKey}" GATEWAY_PORT="${gatewayPort}" bash -c '
export HOME=/home/ubuntu
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

openclaw onboard --non-interactive --accept-risk \
 --mode local \
 --auth-choice apiKey \
 --gateway-port $GATEWAY_PORT \
 --gateway-bind loopback \
 --skip-daemon \
 --skip-skills || echo "WARNING: OpenClaw onboarding failed. Run openclaw onboard manually."
'

# Install daemon service with XDG_RUNTIME_DIR set
echo "Installing OpenClaw daemon..."
sudo -H -u ubuntu XDG_RUNTIME_DIR=/run/user/1000 bash -c '
export HOME=/home/ubuntu
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

openclaw daemon install || echo "WARNING: Daemon install failed. Run openclaw daemon install manually."
'

# Configure gateway for Tailscale Serve (trustedProxies + skip device pairing + set token)
echo "Configuring gateway for Tailscale Serve..."
sudo -H -u ubuntu GATEWAY_TOKEN="${gwToken}" python3 << 'PYTHON_SCRIPT'
import json
import os
config_path = "/home/ubuntu/.openclaw/openclaw.json"
with open(config_path) as f:
 config = json.load(f)
config["gateway"]["trustedProxies"] = ["127.0.0.1"]
config["gateway"]["controlUi"] = {
 "enabled": True,
 "allowInsecureAuth": True
}
config["gateway"]["auth"] = {
 "mode": "token",
 "token": os.environ["GATEWAY_TOKEN"]
}
with open(config_path, "w") as f:
 json.dump(config, f, indent=2)
print("Configured gateway with trustedProxies, controlUi, and token")
PYTHON_SCRIPT

# Enable Tailscale HTTPS proxy (requires HTTPS to be enabled in Tailscale admin console)
echo "Enabling Tailscale HTTPS proxy..."
tailscale serve --bg ${gatewayPort} || echo "WARNING: tailscale serve failed. Enable HTTPS in your Tailscale admin console first."

echo "OpenClaw setup complete!"
`;
 });

const server = new hcloud.Server("openclaw-server", {
 serverType: serverType,
 location: location,
 image: "ubuntu-24.04",
 sshKeys: [hcloudSshKey.id],
 firewallIds: [firewall.id.apply(id => Number(id))],
 userData: userData,
 labels: {
 purpose: "openclaw",
 },
});

export const ipv4Address = server.ipv4Address;
export const privateKey = sshKey.privateKeyOpenssh;

// Construct the Tailscale MagicDNS hostname from the server name
// Hetzner servers use their name as the hostname
const tailscaleHostname = server.name;

export const tailscaleUrlWithToken = pulumi.interpolate`https://${tailscaleHostname}.${tailnetDnsName}/?token=${gatewayToken}`;
export const gatewayTokenOutput = gatewayToken;

You can find both programs in the Pulumi examples repo under openclaw/:

github.com/pulumi/examples

Cost comparison

Before deploying, let’s compare the costs between AWS and Hetzner for running OpenClaw 24/7:

	AWS (t3.medium)	Hetzner (cax21)
vCPUs	2	4
Memory	4 GB	8 GB
Storage	30 GB gp3 (+$2.40/mo)	80 GB NVMe (included)
Traffic	Pay per GB	20 TB included
Architecture	x86 (Intel/AMD)	ARM (Ampere)
Hourly price	$0.0416	€0.0104 (~$0.011)
Monthly price	~$33 (with storage)	€6.49 (~$7)
Annual cost	~$396	~$84

Hetzner gives you double the vCPUs, double the RAM, at less than a quarter of the price. The trade-off? ARM architecture instead of x86. But OpenClaw doesn’t care - it’s just Node.js and Docker.

Prices are for on-demand instances as of January 2026. AWS prices are for us-east-1; Hetzner prices exclude VAT. Both include standard networking and storage. Check AWS EC2 pricing and Hetzner Cloud pricing for current rates.

Running the deployment

With your ESC environment configured in Pulumi.dev.yaml, deploy with:

pulumi up

After deployment completes, you’ll see outputs similar to:

Outputs:
 gatewayTokenOutput : "786c099cc8f8bf20dbebf40b8b51b75cf5cdab25..."
 privateKey : [secret]
 publicDns : "ec2-x-x-x-x.compute-1.amazonaws.com"
 publicIp : "x.x.x.x"
 tailscaleUrlWithToken: "https://ip-10-0-1-x.tailxxxxx.ts.net/?token=786c099..."

The tailscaleUrlWithToken output provides the complete URL with authentication token. Copy and paste it into your browser to access the OpenClaw web UI.

Output names vary slightly between providers: AWS uses publicIp and publicDns, while Hetzner uses ipv4Address. The Tailscale hostname is derived from the instance’s private IP (AWS) or server name (Hetzner).

Automated onboarding

The Pulumi program runs OpenClaw’s non-interactive onboarding during instance provisioning. It uses your Anthropic API key from ESC, binds the gateway to loopback with Tailscale Serve as the HTTPS proxy, generates a secure gateway token (exported in Pulumi outputs), installs the daemon as a systemd user service, and configures trustedProxies and controlUi.allowInsecureAuth to skip device pairing when accessed via Tailscale.

The cloud-init script runs openclaw onboard --non-interactive with all necessary flags, then configures the gateway for secure Tailscale access. Your instance is ready as soon as provisioning finishes.

Access the web UI

Please wait 3-5 minutes after pulumi up completes. The cloud-init script runs after the instance launches and needs time to install Docker, Node.js, OpenClaw, and Tailscale, then run the onboarding process and start the daemon. Periodically refresh the page until it loads. If the page still doesn’t load after 5 minutes, see Verify the deployment to troubleshoot.

The easiest way to access the OpenClaw web UI is to use the tailscaleUrlWithToken output from Pulumi:

# Get the full URL with token
pulumi stack output tailscaleUrlWithToken

Copy and paste this URL into your browser. The URL includes both the Tailscale MagicDNS hostname and the authentication token, so you can access the web UI directly.

Finding your tailnet DNS name: Go to the Tailscale admin console and look under the DNS section for your tailnet name (e.g., tailxxxxx.ts.net). You can also find your machines and their MagicDNS hostnames in the Machines tab.

Token-based authentication provides an additional layer of security on top of Tailscale’s network-level authentication. Only devices on your tailnet can reach the URL, and the token prevents unauthorized access if someone gains access to your tailnet.

From the web UI, you can connect messaging channels (WhatsApp, Discord, Slack), configure skills and integrations, and manage settings.

Verify the deployment (optional)

If you encounter issues accessing the web UI, you can SSH into your instance to troubleshoot:

# Check your Tailscale admin console for the new machine
ssh ubuntu@<tailscale-ip>

# Check OpenClaw gateway status
systemctl --user status openclaw-gateway

Test your assistant

Open the gateway dashboard using the tailscaleUrlWithToken output and use the built-in chat to test your assistant:

Your personal AI assistant is now running 24/7 on your own infrastructure, accessible securely through Tailscale.

Security considerations

When self-hosting an AI assistant, security matters. OpenClaw’s rapid adoption meant thousands of instances spun up in days, and not everyone locked them down. The community noticed:

The tweet isn’t exaggerating. A quick Shodan search shows exposed gateways on port 18789 with shell access, browser automation, and API keys up for grabs:

Don’t let your instance be one of them.

Concern	Without Tailscale	With Tailscale
SSH access	Public (port 22 open)	Public fallback + Tailscale SSH
Gateway access	Public (port 18789 open)	Private (Tailscale only)
Browser control	Public (port 18791 open)	Private (Tailscale only)
API keys in transit	Exposed if gateway accessed over HTTP	Protected by Tailscale encryption
Attack surface	3 open ports	1 open port (SSH fallback)

SSH remains accessible as a fallback even with Tailscale enabled. This allows you to troubleshoot if Tailscale fails to connect. Once you’ve confirmed Tailscale is working, you can manually remove the SSH ingress rule from your security group for maximum security.

My recommendations:

Always use Tailscale for production
Rotate your auth keys periodically
Use Pulumi ESC for secrets instead of hardcoding
Enable Tailscale SSH to avoid managing keys manually
Monitor your Tailscale admin console for unauthorized devices
Remove the SSH fallback after confirming Tailscale works if you want zero public ports

What’s next?

Now that OpenClaw is running, you can install skills (voice generation, video creation, browser automation), set up scheduled tasks with cron, invite colleagues to your tailnet for shared access, or connect additional channels like WhatsApp and Discord.

For a deeper case study of running an AI assistant on Pulumi-managed infrastructure, see how we built Platybot, or read our KubeCon EU 2026 recap on the year AI moved into production on Kubernetes.

Conclusion

Deploying OpenClaw with infrastructure as code means you can reproduce your setup anytime, version control it, and tear it down with a single command. Adding Tailscale keeps it private - no exposed ports, no hoping you configured your firewall correctly at 2am.

If you run into issues or have questions, drop by the Pulumi Community Slack or GitHub Discussions.

New to Pulumi? Get started here.

Neo: Zero-downtime migration from CDK, Terraform & Azure ARM

Pulumi Neo Team — Wed, 21 Jan 2026 00:00:00 +0000

The barrier to migrating to Pulumi has always been the infrastructure you already have. Your existing resources can’t be disrupted, and manually importing them into a new tool is risky and time-consuming. Today, we’re excited to share how Neo removes this barrier entirely with automated, zero-downtime migration to Pulumi from AWS CDK, AWS CloudFormation, Terraform, CDKTF, and Azure ARM templates.

Why IaC migrations are hard

The promise of Infrastructure as Code is that your code perfectly describes your running infrastructure. But switching IaC tools breaks this promise in dangerous ways.

When you rewrite your infrastructure code in a new tool, you have two choices, both problematic. You can destroy and recreate all your resources to match the new code, accepting downtime and risk. Or you can try to import existing resources, which requires perfect knowledge of how every resource maps between the old and new systems. Many teams get stuck here, wanting Pulumi’s modern platform but unable to safely make the switch.

Neo’s insight: State knowledge enables perfect migrations

The key to safe migration isn’t just converting code - it’s understanding the complete relationship between your existing IaC tool’s state and your actual cloud resources. Each IaC tool maintains this relationship differently: CDK through CloudFormation stacks, Terraform/CDKTF through state files, and ARM through Azure deployments. But they all have complete knowledge of what they manage.

Neo leverages this existing state knowledge to orchestrate perfect resource transitions. Instead of asking you to manually find resource IDs and construct migration commands, Neo reads your current tool’s state, discovers every resource’s physical identity, and brings them under Pulumi management. This isn’t just automation - it’s using the source tool’s own knowledge against the migration problem.

In practice, this means Neo can bridge the gap between how tools name resources and where they actually live. A Lambda function that CDK knows as OrderHandler9I0J1K2L actually exists in AWS as my-app-OrderHandler-9I0J1K2L, while a Terraform resource at address aws_instance.web[2] maps to EC2 instance i-0abc123def456. Neo understands these mappings and handles the complex cases like composite IDs (FunctionName|StatementId), resource references, and dependency chains that must be migrated in order.

Because Neo uses the source tool’s own state knowledge, your infrastructure doesn’t change at all during migration. Not a single resource is modified, recreated, or even touched. We’re simply transferring ownership from one IaC tool to another.

This approach delivers three critical guarantees:

Zero downtime: Resources are never deleted or recreated
Zero risk: Since nothing changes, you can abandon the migration at any point without consequence
Zero surprises: Preview confirms no infrastructure changes before you commit

Migration in action: Four tools, one approach

Neo adapts its migration strategy to each tool’s unique characteristics while maintaining the same zero-downtime guarantee. Let’s explore how Neo handles migrations from each major IaC tool.

AWS CDK to Pulumi

For teams using AWS CDK, Neo leverages the CloudFormation layer that underpins CDK deployments. CDK’s architecture actually makes migration straightforward: since CDK synthesizes to CloudFormation templates, Neo can read the deployed stacks directly to understand every resource and its configuration. For detailed migration steps, see our CDK migration guide.

The challenge with CDK migrations isn’t the CloudFormation layer - it’s the cryptic resource naming. CDK generates logical IDs like OrdersTableA7B2C3D4 that map to physical resources with completely different names. These mappings are buried in CloudFormation metadata, and getting them wrong means either orphaning resources or accidentally creating duplicates. Neo navigates this complexity by reading CloudFormation’s own stack outputs and resource metadata, discovering the exact physical ID for every logical resource.

CDK also introduces complexity through its construct hierarchy. A single high-level construct might expand into dozens of CloudFormation resources, each with dependencies and references to others. Neo preserves these relationships during migration, ensuring that IAM roles still reference the right Lambda functions, API Gateway deployments still point to the correct stages, and security groups maintain their exact rules. The migration completes with your infrastructure unchanged and Pulumi’s preview confirming zero modifications.

Start a Neo task Migrate your CDK application

AWS CloudFormation to Pulumi

For teams using CloudFormation directly (rather than through CDK), Neo provides a streamlined migration path. CloudFormation stacks contain complete resource metadata - every resource’s logical ID maps to a physical resource in AWS, and CloudFormation tracks these relationships in its stack state. Neo reads this state directly to build a complete picture of your infrastructure. For detailed migration steps, see our CloudFormation migration guide.

The main challenge with CloudFormation migrations is the template language itself. CloudFormation templates use intrinsic functions like !Ref, !GetAtt, and !Sub that create implicit dependencies between resources. A security group might reference a VPC using !Ref MyVpc, while a Lambda function’s role uses !GetAtt LambdaRole.Arn. Neo evaluates these expressions against the actual deployed stack to resolve every reference to its concrete value.

CloudFormation also supports features like conditionals, mappings, and nested stacks that add layers of indirection. Neo handles these by examining what actually got deployed rather than trying to interpret every possible template path. The result is Pulumi code that manages your exact infrastructure configuration - not a theoretical interpretation of your template. The migration completes with pulumi preview confirming zero changes to your running resources.

Start a Neo task Migrate your CloudFormation stack

Terraform to Pulumi

Terraform and CDKTF migrations require two transformations: converting state to establish Pulumi’s connection to your resources, and transforming HCL configuration into Pulumi code. The state conversion is direct - Neo reads your Terraform state and converts it into Pulumi state, preserving the mappings between resource names and cloud IDs. This ensures Pulumi knows that aws_instance.web[2] corresponds to EC2 instance i-0abc123def456.

The code transformation analyzes your HCL to generate equivalent Pulumi code. Neo handles Terraform patterns like count and for_each loops, module structures, and resource dependencies, recreating them idiomatically in Pulumi. The generated code manages your existing infrastructure without modifications - pulumi preview confirms zero changes. For complete migration instructions, see our Terraform migration guide.

Start a Neo task Migrate your Terraform application

Azure ARM to Pulumi

ARM templates present unique migration challenges. Unlike CDK and Terraform, which maintain clear separation between code and state, ARM templates blur this line. The template is both the definition and, through deployment history, part of the state tracking. ARM’s template expression language, with its concat functions and resource ID constructors, makes it difficult to determine what resources actually exist until deployment time. For step-by-step migration guidance, see our ARM migration guide.

Neo orchestrates ARM migrations through intelligent AI-driven conversion. When an ARM template uses functions like concat(parameters('appName'), '-plan'), the conversion process evaluates these expressions using the actual parameter values to generate the correct resource names. Azure resource IDs follow predictable patterns - subscription IDs, resource groups, providers, and resource names - and Neo ensures these are correctly brought under Pulumi management using inline resource specifications directly in the generated code.

The biggest challenge with ARM migrations is handling the implicit dependencies and resource provider quirks. An App Service might implicitly create a service plan, a SQL database requires a server that might be defined in a linked template, and child resources like application settings need separate migration steps. Neo understands these Azure-specific patterns and generates the appropriate Pulumi code to manage every resource. The migration completes with a zero-diff preview, confirming your exact Azure configuration is preserved while giving you a more maintainable, type-safe way to manage it going forward.

Start a Neo task Migrate your ARM template

The architecture powering it all

While each tool requires specific handling, Neo’s core architecture remains consistent:

Universal orchestration layer

Neo acts as the intelligent migration coordinator, regardless of source tool:

Credential verification: Ensures proper cloud credentials are configured in Pulumi ESC
Resource inventory: Builds a complete catalog of existing resources
Conversion orchestration: Manages the code transformation
State migration: Brings existing resources under Pulumi management
Audit trail generation: Creates comprehensive migration reports

Unified state management engine

The state management engine works consistently across all tools:

Maps source resource IDs to Pulumi’s state management
Handles complex and composite resource identifiers
Provides fallback strategies for edge cases
Ensures idempotent operations

Human oversight

While Neo automates the heavy lifting, we maintain human checkpoints:

Review generated code before migrating resources
Verify preview shows zero changes
Approve the resulting pull request

What this means for your team

Migration friction no longer locks you into your current IaC tool. If you want Pulumi’s programming model, policy engine, and multi-cloud support, Neo gets you there without disrupting your infrastructure.

Ready to migrate? Check out our migration guides for CDK, CloudFormation, Terraform, or Azure ARM. Join us in the Pulumi Community Slack or reach out to your account team for a guided migration session.

How Ralph Wiggum Built a Serverless SaaS with Pulumi

Engin Diri — Wed, 14 Jan 2026 00:00:00 +0000

I was about to do something that felt either genius or completely reckless: hand over my AWS credentials to an AI and step away from my computer. The technique is called “Ralph Wiggum,” named after the Simpsons character who eats glue and says “I’m in danger” while everything burns around him. And honestly, that felt about right for what I was attempting.

The problem with AI coding assistants

If you have spent any time with AI coding assistants, you know the frustration. You are in the middle of a task. It is going great. Claude is writing beautiful infrastructure code, understanding your architecture, making progress… and then it says “I have completed your request.”

You stare at the screen. “No. No, you have not. You maybe did a third of what I asked.”

So you reprompt it. It does a little more. Then it stops again. You are babysitting. The whole point of using an AI assistant was to save time, but if you have to check on it every five minutes, you are not saving anything. You are just a very expensive supervisor.

This is the problem Geoffrey Huntley decided to solve.

What if we just do not let it stop?

Geoffrey Huntley, a developer based in rural Australia (who, according to internet lore, lives on a property with goats), had a deceptively simple idea: what if every time Claude Code finishes and tries to exit, we just feed it the prompt again?

He named the technique “Ralph Wiggum” because the character embodies a kind of childlike persistence. Ralph repeatedly fails, makes silly mistakes, yet stubbornly continues in an endless loop until he eventually succeeds. Sound familiar? That is exactly how AI coding agents work. They make mistakes. They try again. They iterate.

The philosophy behind Ralph is beautifully stated: “Better to fail predictably than succeed unpredictably.” Every failure is just data for the next iteration.

How the Ralph loop works

At its core, the Ralph Wiggum technique is almost embarrassingly simple. It is a bash loop:

while true; do
 cat PROMPT.md | claude --print --dangerously-skip-permissions
done

That is it. That is the whole thing. You write your instructions in a PROMPT.md file, and the loop keeps feeding it to Claude Code over and over.

But here is why it works: each time Claude starts up, it looks at the project. It sees the files that exist, the code that is already written, the git history. It is not starting from zero. It picks up where it left off, sees what still needs to be done, and keeps going.

The official Claude Code plugin formalizes this with some important additions:

Completion promises: You tell Claude to output a specific string like <promise>COMPLETE</promise> when it has genuinely finished
Max iterations: A safety limit so your API bill does not reach infinity
Stop hooks: The mechanism that intercepts exit attempts and re-injects the prompt

To install it:

/plugin install ralph-wiggum@ghuntley

For end-to-end testing of deployed infrastructure, you will also want the Playwright MCP server. This lets Claude interact with your deployed application in a real browser:

claude mcp add playwright npx @playwright/mcp@latest

Why Pulumi is perfect for Ralph

Infrastructure as code has something most application code does not: objective success criteria. Your Lambda either deploys or it does not. Your DynamoDB table either exists or it does not. Tests either pass or they fail.

This makes Pulumi an ideal candidate for autonomous AI development:

Clear completion signals: pulumi preview and pulumi up provide unambiguous feedback
Iterative feedback: Failed deployments tell Claude exactly what went wrong
Testable outputs: You can write integration tests that verify your infrastructure actually works
Registry access: With the Pulumi MCP server, Claude has real-time access to documentation and examples

The experiment

I decided to build a serverless URL shortener on AWS. Not because the world needs another URL shortener, but because it is the perfect test case: multiple AWS services, clear requirements, and objective success criteria. But I did not want just a basic demo. I wanted a full SaaS-like experience with a polished frontend.

Here is the PROMPT.md file I wrote:

# Build a Production-Ready URL Shortener SaaS on AWS

Using Pulumi TypeScript, create a complete URL shortener SaaS with a polished frontend experience.

## Infrastructure Requirements
- DynamoDB table for storing URL mappings (shortCode -> originalUrl, clickCount, createdAt)
- Lambda functions for: creating short URLs, redirecting to original URLs, getting stats
- API Gateway REST API exposing the Lambda functions
- S3 bucket for the React frontend with static website hosting
- CloudFront distribution with HTTPS for the frontend and API
- Use the Pulumi ESC environment pulumi-idp/auth for the AWS credentials

## Frontend Requirements (Use /frontend-design skill)
Create a polished, production-ready SaaS website with:

### Landing Page
- Hero section with catchy headline and URL input form
- Feature highlights (fast redirects, analytics, custom aliases)
- Pricing section (Free tier, Pro tier, Enterprise tier)
- Testimonials section with 3 fake but realistic customer reviews
- Footer with links to Docs, Privacy, Terms

### Dashboard Page
- URL shortening form with optional custom alias
- List of created short URLs with click counts
- Copy-to-clipboard functionality
- Delete URL option

### Documentation Page
- Getting started guide
- API reference (POST /shorten, GET /{code}, GET /stats/{code})
- Rate limits and usage policies
- FAQ section

### Design Requirements
- Modern, clean aesthetic (no generic AI look)
- Responsive design (mobile, tablet, desktop)
- Dark mode support
- Smooth animations and transitions
- Consistent color scheme and typography

## API Requirements
- POST /shorten: accepts { url: string, alias?: string }, returns { shortCode, shortUrl }
- GET /{shortCode}: redirects to original URL (301)
- GET /stats/{shortCode}: returns { originalUrl, clickCount, createdAt }

## Success Criteria
- All unit tests pass
- All integration tests pass
- pulumi preview shows no errors
- pulumi up deploys successfully

## End-to-End Verification (Required)
After deployment, use Playwright MCP to verify the live site:
1. Open the CloudFront URL and verify the landing page loads with all sections
1. Navigate to the docs page and verify content renders
1. Create a short URL using the dashboard
1. Verify the short URL redirects correctly (301 status)
1. Check the stats show the click was recorded
1. Test responsive design by resizing the viewport
1. Take screenshots of landing page, dashboard, and docs as proof

Only output <promise>COMPLETE</promise> after ALL of the above including E2E tests pass.

The key addition here is the end-to-end verification section. Without it, Claude reports success after pulumi up finishes, but you have no guarantee the deployed infrastructure actually works. By requiring Playwright to test the live URLs, you catch issues like the CloudFront 403 error I mentioned earlier.

Before running the loop, I started Claude Code with permission bypass mode to prevent it from stopping to ask for approval on every file write or command:

claude --permission-mode bypassPermissions

There are other ways to handle permissions, but this is the simplest for autonomous execution. Then I ran the Ralph loop:

/ralph-wiggum:ralph-loop PROMPT.md --max-iterations 25 --completion-promise "COMPLETE"

Then I stepped away and let it run unsupervised. For complex projects, you could even let it run overnight while you sleep.

Here is Claude Code in action during the Ralph loop, fixing CloudFront configuration issues and writing unit tests:

The results

The complete source code for this project is available on GitHub: url-shortener-saas

When Claude finished, it had:

Successfully built:

A complete Pulumi program with proper resource organization
DynamoDB table with a GSI for querying by creation date
Three Lambda functions with proper error handling
API Gateway with CORS configuration
S3 bucket with static website hosting
CloudFront distribution with proper cache behaviors

The frontend was genuinely impressive:

A polished landing page with hero section, feature cards, and pricing tiers
Working testimonials section with three fictional but believable customer reviews
A functional dashboard where you could create and manage short URLs
A documentation page with API reference and getting started guide
Dark mode toggle that actually worked
Responsive design that looked good on mobile

Home screen

Dashboard

Documentation

Here is a snippet of what it generated:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const stack = pulumi.getStack();
const projectName = "url-shortener";

// DynamoDB Table
const urlTable = new aws.dynamodb.Table("url-table", {
 name: `${projectName}-urls-${stack}`,
 billingMode: "PAY_PER_REQUEST",
 hashKey: "shortCode",
 attributes: [{ name: "shortCode", type: "S" }],
 tags: {
 Environment: stack,
 Project: projectName,
 ManagedBy: "Pulumi",
 },
});

// Lambda function for URL shortening
const shortenFunction = new aws.lambda.Function("shorten-function", {
 name: `${projectName}-shorten-${stack}`,
 runtime: aws.lambda.Runtime.NodeJS20dX,
 handler: "shorten.handler",
 role: lambdaRole.arn,
 timeout: 30,
 memorySize: 256,
 code: new pulumi.asset.AssetArchive({
 ".": new pulumi.asset.FileArchive("./lambda"),
 }),
 environment: {
 variables: {
 TABLE_NAME: urlTable.name,
 },
 },
});

// CloudFront Distribution with S3 and API Gateway origins
const distribution = new aws.cloudfront.Distribution("frontend-distribution", {
 enabled: true,
 defaultRootObject: "index.html",
 origins: [
 {
 domainName: frontendBucketWebsite.websiteEndpoint,
 originId: "S3Origin",
 customOriginConfig: {
 httpPort: 80,
 httpsPort: 443,
 originProtocolPolicy: "http-only",
 originSslProtocols: ["TLSv1.2"],
 },
 },
 {
 domainName: pulumi.interpolate`${api.id}.execute-api.${aws.config.region}.amazonaws.com`,
 originId: "APIGatewayOrigin",
 customOriginConfig: {
 httpPort: 80,
 httpsPort: 443,
 originProtocolPolicy: "https-only",
 originSslProtocols: ["TLSv1.2"],
 },
 },
 ],
 defaultCacheBehavior: {
 targetOriginId: "S3Origin",
 viewerProtocolPolicy: "redirect-to-https",
 allowedMethods: ["GET", "HEAD", "OPTIONS"],
 cachedMethods: ["GET", "HEAD"],
 forwardedValues: {
 queryString: false,
 cookies: { forward: "none" },
 },
 },
 orderedCacheBehaviors: [{
 pathPattern: "/api/*",
 targetOriginId: "APIGatewayOrigin",
 viewerProtocolPolicy: "https-only",
 allowedMethods: ["DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT"],
 cachedMethods: ["GET", "HEAD"],
 forwardedValues: {
 queryString: true,
 headers: ["Authorization", "Content-Type"],
 cookies: { forward: "none" },
 },
 minTtl: 0,
 defaultTtl: 0,
 maxTtl: 0,
 }],
 viewerCertificate: {
 cloudfrontDefaultCertificate: true,
 },
});

Where it struggled:

The first few iterations had the Lambda code inline instead of in separate files
It initially forgot to set up IAM permissions for the Lambda to access DynamoDB
The CloudFront configuration took three attempts to get the cache behaviors right

The funny parts:

At iteration 12, it decided to completely refactor the project structure. Then at iteration 13, it refactored it back
One commit message simply read: “fix the fix that fixed the previous fix”
It wrote a test, ran the test, fixed the code to pass the test, then realized the test was wrong

Best practices for Ralph with Pulumi

After running several experiments, here is what I have learned:

Write extremely specific prompts. Vague instructions lead to vague implementations. Be explicit about every requirement, every edge case, every success criterion.

Use pulumi preview as your feedback loop. Add this to your success criteria:

Success Criteria:
- pulumi preview completes with no errors
- pulumi preview shows expected resource count (approximately X resources)

Set realistic iteration limits. For infrastructure projects, I have found 20-30 iterations is usually sufficient. More than that and you are probably missing something in your prompt.

Include test requirements. Claude is surprisingly good at test-driven development when you tell it to write tests first:

## Development Approach
1. Write unit tests for each Lambda function first
1. Implement Lambda functions to pass the tests
1. Write integration tests that verify the deployed infrastructure
1. Only output COMPLETE when all tests pass

Mention cost awareness. Add a note like “Use PAY_PER_REQUEST billing for DynamoDB” or “Use the smallest Lambda memory allocation that works.” Claude will optimize for what you tell it to optimize for.

The honest assessment

Is this the future of infrastructure development? Probably not entirely. But it is a genuinely useful technique for specific scenarios:

Great for:

Greenfield projects where you have clear requirements
Large migrations or refactors
Building proof-of-concept infrastructure while you focus on other work
Tasks where iteration is more valuable than perfection on the first try

Not great for:

Production infrastructure changes (please do not let an AI modify your production AWS account unsupervised)
Complex architecture decisions that require human judgment
Anything involving sensitive data or security configurations
Small tweaks or quick fixes (overkill)

The API costs were reasonable for what I got. For a complete infrastructure project built autonomously, a few dollars in Claude API calls is a bargain compared to my hourly rate.

Try it yourself

If you want to experiment with Ralph Wiggum and Pulumi:

Install Claude Code and the Ralph Wiggum plugin
Set up your Pulumi account and AWS credentials
Write a detailed PROMPT.md with clear success criteria
Start with a small project and low iteration limits
Review the git history when it finishes

The combination of autonomous AI loops and infrastructure as code feels like a glimpse of where development is heading. Not replacing developers, but changing how we use our time. Instead of babysitting an AI through each step, you hand it a well-defined problem and come back to working infrastructure.

Just make sure you have billing alerts set up. Ralph Wiggum does not know when to stop spending your money.

Or try Pulumi Neo: Ralph built-in

If setting up the Ralph Wiggum plugin feels like too much ceremony, Pulumi has something similar built right into Pulumi Neo.

Neo Tasks work on the same principle: you describe what you want, and Neo plans and executes the infrastructure changes. The key is auto mode. When you set a task to auto mode, Neo runs without requesting approvals. It plans, executes, validates with pulumi preview, and iterates until the task is complete.

The experience is remarkably similar to Ralph:

Persistent execution: Tasks continue running even if you close your browser. Come back later and Neo shows you what it accomplished while you were away.
Iterative refinement: Failed deployments inform the next attempt, just like Ralph re-reading the project state.
Clear completion: Neo knows when the infrastructure matches your requirements.

The main difference is that Neo runs in Pulumi Cloud rather than on your local machine, and it is purpose-built for infrastructure tasks. You get the autonomous loop experience without needing to configure plugins or bash loops.

For teams already using Pulumi Cloud, Neo with auto mode might be the fastest path to autonomous infrastructure development.

Resources

From 'Works on My Machine' to Production-Ready: Building AI Agents with Amazon Bedrock AgentCore

Engin Diri — Mon, 15 Dec 2025 00:00:00 +0200

Every developer building AI agents knows the gap between a working prototype and production deployment. Your fraud detection agent works perfectly on your laptop, but how do you deploy it with proper authentication, memory persistence, observability, and guardrails? This post walks through a complete journey from local development to production-ready AI agents using Amazon Bedrock AgentCore, the Strands SDK, and Pulumi.

TL;DR: This post shows how to deploy AI agents from local prototype to production using:

Strands SDK for agent development
Amazon Bedrock AgentCore for managed runtime with Firecracker isolation
AgentCore Memory for short-term and long-term context
MCP Gateway for secure tool integration with OAuth2
Pulumi for infrastructure as code

Complete code: GitHub

The Challenge: Bridging the Gap

Building AI agents is getting easier. Tools like the Strands Agents SDK let you create functional agents in just a few lines of code. Production deployment introduces challenges:

Stateless to Stateful: How does your agent remember previous conversations?
Single-tenant to Multi-tenant: How do you isolate different users’ data?
Implicit Trust to Zero Trust: How do you authenticate requests and enforce boundaries?
Console Logs to Observability: How do you trace requests across distributed systems?
Ad-hoc Tools to Managed Services: How do you expose tools securely at scale?

The approach is progressive: start simple, add complexity only when needed, and use infrastructure as code to make it reproducible.

Agents Beyond Chat: Event-Driven AI

When most people think of AI agents, they imagine chatbots: interactive systems responding to user questions. But many production agents never interact with end users directly. Instead, they’re triggered by event streams: Kafka topics, SQS queues, webhook callbacks, or scheduled jobs.

Our fraud detection agent is a prime example. It doesn’t wait for customers to ask questions. It’s triggered automatically when the payment processing system detects a new transaction. The “user” is another system, and the “conversation” is a structured event payload.

This event-driven pattern is becoming the new stack for AI agents. As Kai Waehner describes in his analysis of agentic AI with Kafka and Flink:

“Protocols like A2A and MCP are essential for standardizing agent behavior and communication. But without an event-driven substrate like Kafka and a stream-native runtime like Flink, these agents remain stuck in isolated interactions.”

The emerging architecture looks like this:

In production, you might use Amazon MSK (Managed Kafka), Amazon SQS, or EventBridge to deliver transaction events to your agent. The agent processes each event autonomously, takes action (blocking cards, creating tickets), and publishes results back to another topic.

What We’re Building: A Fraud Detection Agent

Our demo is a fraud detection agent that:

Receives transaction alerts with user ID, amount, merchant, and location
Gathers context from tools (user profiles, transaction history)
Analyzes for fraud patterns (impossible travel, velocity attacks, amount anomalies)
Takes protective action (blocking cards) when fraud is detected

We’ll build this progressively:

Stage	What Changes	Key Additions
Local Development	Running on your laptop	Strands SDK, Claude Opus 4.5, mock tools
First Deployment	Running in AgentCore	Containerized, Firecracker isolation, guardrails
Production-Ready	Full enterprise features	Memory, MCP Gateway, X-Ray observability

Local Development with Strands SDK

The Strands Agents SDK is an open-source framework from AWS that takes a model-driven approach to agent development. Instead of hardcoding complex workflows, you let the LLM reason about tool usage autonomously.

Here’s our complete local agent in about 30 lines:

# agent.py
from strands import Agent
from strands.models import BedrockModel
from tools import get_user_profile, get_recent_transactions, block_credit_card

# Configure Bedrock model with cross-region inference for Claude Opus 4.5
# Note: Cross-region inference may have different pricing than single-region
bedrock_model = BedrockModel(
 model_id="us.anthropic.claude-opus-4-5-20251101-v1:0",
 region_name="us-east-1",
)

# Define the Fraud Detection Agent
fraud_agent = Agent(
 model=bedrock_model,
 tools=[get_user_profile, get_recent_transactions, block_credit_card],
 system_prompt="""
 You are a Senior Fraud Analyst Agent at a financial institution.
 Your mission is to protect customers from fraudulent transactions in real-time.

 When you receive a transaction alert:
 1. Gather context by checking the user's profile and recent transaction history.
 2. Analyze the transaction for fraud indicators:
 - Impossible travel: Can the user physically be at the new location?
 - Unusual amounts: Is this significantly larger than typical spending?
 - High-risk merchants: Electronics stores, gift cards, wire transfers.
 - Geographic anomalies: Transactions far from home location.
 3. Make a decision:
 - If fraud indicators are present, BLOCK the card immediately.
 - If the transaction appears legitimate, allow it to proceed.
 4. Document your reasoning clearly.

 Prioritize customer protection. When in doubt, block the card.
 """,
)

Tools are simple Python functions decorated with @tool:

from strands import tool

@tool
def get_user_profile(user_id: str) -> dict:
 """Get user profile including home location and account status."""
 return {
 "user_id": user_id,
 "name": "John Doe",
 "home_location": "London, UK",
 "account_status": "active",
 }

@tool
def block_credit_card(user_id: str, reason: str) -> dict:
 """Block a user's credit card and create a fraud ticket."""
 return {
 "status": "blocked",
 "ticket_id": f"FRAUD-{user_id}-001",
 "message": f"Card blocked for user {user_id}: {reason}",
 }

Running the Local Demo

Here’s how we simulate an incoming transaction event:

# demo_simulation.py
from agent import fraud_agent

# The "Trigger" Event (Simulated Payload from Kafka/SQS)
new_transaction_event = """
ALERT: New Transaction Attempt
User ID: user_123
Amount: $2000
Merchant: Electronics Store
Location: Tokyo, Japan
Time: 09:15
"""

def run_demo():
 print("--- INCOMING EVENT STREAM ---")
 print(new_transaction_event)
 print("--- AGENT ACTIVATED ---")

 # Run the agent - processes the event autonomously
 result = fraud_agent(new_transaction_event)

 print("--- ANALYSIS COMPLETE ---")

if __name__ == "__main__":
 run_demo()

When we run this, the agent autonomously gathers context, analyzes the transaction, and takes action:

$ python demo_simulation.py
--- INCOMING EVENT STREAM ---
ALERT: New Transaction Attempt
User ID: user_123
Amount: $2000
Merchant: Electronics Store
Location: Tokyo, Japan
Time: 09:15
-----------------------------------
AGENT ACTIVATED. Analyzing...
Tool #1: get_user_profile
Tool #2: get_recent_transactions
## 🚨 FRAUD ANALYSIS REPORT
**Customer:** John Doe (user_123)
### Analysis:
#### 1. ❌ IMPOSSIBLE TRAVEL DETECTED
- **Last Transaction:** 09:00 in **London, UK**
- **Current Transaction:** 09:15 in **Tokyo, Japan**
- **Time Elapsed:** 15 minutes
- **Distance:** ~9,500 km (5,900 miles)
- **Minimum Flight Time:** ~12 hours
**Verdict:** It is physically impossible for John Doe to travel from London
to Tokyo in 15 minutes. This is a critical fraud indicator.
#### 2. ⚠️ HIGH-RISK MERCHANT
- Electronics stores are frequently targeted for fraud due to high resale value
#### 3. ⚠️ HIGH-VALUE TRANSACTION
- $2,000 at an electronics store represents a substantial purchase
### Decision: 🛑 BLOCK CARD IMMEDIATELY
Tool #3: block_credit_card
## ✅ ACTION COMPLETED
| Action | Status |
|--------|--------|
| Card Blocked | ✅ Confirmed |
| Ticket ID | TICKET-999 |
--- ANALYSIS COMPLETE ---

Notice how the agent reasoned about the event using tools and took autonomous action. This works great for development, but it lacks everything needed for production: authentication, persistence, isolation, monitoring, and governance.

First Deployment with AgentCore Runtime

Our local prototype works, but where do you actually run it? Lambda times out after 15 minutes. ECS and Kubernetes add operational overhead. And AI agents have different requirements than typical microservices: they reason through multi-step workflows, sometimes taking minutes to analyze a complex fraud case. You need something built for this workload.

Amazon Bedrock AgentCore fills this gap. It’s a managed runtime specifically designed for AI agents, with support for tasks running up to 8 hours. No timeout workarounds, no cluster management.

The security model is particularly relevant for fraud detection. AgentCore runs each session in a dedicated microVM using AWS Firecracker (the same isolation technology behind Lambda). When your agent processes one customer’s transactions, it’s completely isolated from other sessions. No shared memory, no cross-tenant contamination.

The deployment model is straightforward: containerize your agent and point AgentCore at the image. It works with Strands, LangGraph, CrewAI, or custom code. The same agent we built locally runs unchanged. We just wrap it in FastAPI and add a Dockerfile.

Cold Start and Latency

For fraud detection, latency matters. AgentCore’s Firecracker microVMs introduce a cold start penalty on the first invocation, typically 2-5 seconds to spin up the container. Subsequent invocations within the same session are fast (sub-second overhead beyond model inference time).

For real-time fraud detection, this cold start is acceptable because:

The LLM reasoning itself takes 5-15 seconds for complex analysis
Most production deployments use session pooling to keep warm instances available
The alternative (always-on containers) trades latency for cost

If sub-second total latency is critical, consider Lambda for simple rule-based checks and reserve AgentCore for complex reasoning that justifies the cold start overhead.

Containerizing the Agent

First, we wrap our agent in a FastAPI endpoint:

# agent.py (production version)
from fastapi import FastAPI, Request
from strands import Agent
from strands.models import BedrockModel
import os

app = FastAPI()

bedrock_model = BedrockModel(
 model_id="us.anthropic.claude-opus-4-5-20251101-v1:0",
 region_name="us-east-1",
 guardrail_id=os.environ.get("BEDROCK_GUARDRAIL_ID"),
 guardrail_version=os.environ.get("BEDROCK_GUARDRAIL_VERSION"),
 guardrail_trace="enabled",
)

fraud_agent = Agent(
 model=bedrock_model,
 tools=[get_user_profile, get_recent_transactions, block_credit_card],
 system_prompt="...",
)

@app.post("/invocations")
async def invoke(request: Request):
 body = await request.json()
 prompt = body.get("input", {}).get("prompt", "")
 result = fraud_agent(prompt)
 return {"output": {"message": str(result)}}

Adding Guardrails

Amazon Bedrock Guardrails help ensure your agent stays on task. We configure content filters and topic restrictions using Pulumi:

fraud_detection_guardrail = bedrock.Guardrail(
 "fraud-detection-guardrail",
 name="fraud-detection-guardrail",
 description="Ensure agent only handles fraud detection tasks",
 blocked_input_messaging="This request is outside the scope of fraud detection.",
 blocked_outputs_messaging="This response was blocked.",
 content_policy_config=bedrock.GuardrailContentPolicyConfigArgs(
 filters_configs=[
 bedrock.GuardrailContentPolicyConfigFiltersConfigArgs(
 type="SEXUAL", input_strength="HIGH", output_strength="HIGH",
 ),
 bedrock.GuardrailContentPolicyConfigFiltersConfigArgs(
 type="VIOLENCE", input_strength="HIGH", output_strength="HIGH",
 ),
 bedrock.GuardrailContentPolicyConfigFiltersConfigArgs(
 type="PROMPT_ATTACK", input_strength="HIGH", output_strength="NONE",
 ),
 # ... additional filters
 ],
 ),
 topic_policy_config=bedrock.GuardrailTopicPolicyConfigArgs(
 topics_configs=[
 bedrock.GuardrailTopicPolicyConfigTopicsConfigArgs(
 name="entertainment-media",
 type="DENY",
 definition="Questions about movies, TV shows, celebrities...",
 examples=["What is LOTR?", "Tell me about Star Wars"],
 ),
 bedrock.GuardrailTopicPolicyConfigTopicsConfigArgs(
 name="general-assistance",
 type="DENY",
 definition="Requests for coding help, math problems...",
 examples=["Write me a poem", "Help me with Python code"],
 ),
 ],
 ),
)

Deploying with Pulumi

The infrastructure is straightforward: ECR repository, IAM roles, and AgentCore Runtime:

# Create ECR repository
repo = ecr.Repository(
 "strands-agent-repo",
 name="fraud-detection-agent",
 force_delete=True,
)

# Build and push Docker image
image = docker.Image(
 "strands-agent-image",
 image_name=repo.repository_url.apply(lambda url: f"{url}:latest"),
 build=docker.DockerBuildArgs(
 context="../fraud-detection-agent",
 platform="linux/arm64",
 ),
 registry=docker.RegistryArgs(
 server=repo.repository_url,
 username=auth_token.user_name,
 password=auth_token.password,
 ),
)

# Deploy AgentCore Runtime
agent_runtime = bedrock.AgentcoreAgentRuntime(
 "fraud-detection-agent-runtime",
 agent_runtime_name="fraud_detection_agent",
 description="Fraud Detection Agent with Guardrails",
 role_arn=agent_role.arn,
 agent_runtime_artifact=bedrock.AgentcoreAgentRuntimeAgentRuntimeArtifactArgs(
 container_configuration=bedrock.AgentcoreAgentRuntimeAgentRuntimeArtifactContainerConfigurationArgs(
 container_uri=image.image_name,
 ),
 ),
 network_configuration=bedrock.AgentcoreAgentRuntimeNetworkConfigurationArgs(
 network_mode="PUBLIC",
 ),
 environment_variables={
 "BEDROCK_GUARDRAIL_ID": fraud_detection_guardrail.guardrail_id,
 "BEDROCK_GUARDRAIL_VERSION": fraud_detection_guardrail.version,
 },
)

Testing the Basic Deployment

After pulumi up, we can invoke the deployed agent via the AWS SDK:

# invoke_agent.py
import boto3
import json

def invoke_agent(agent_runtime_arn: str, prompt: str) -> dict:
 client = boto3.client("bedrock-agentcore", region_name="us-east-1")

 response = client.invoke_agent_runtime(
 agentRuntimeArn=agent_runtime_arn,
 runtimeSessionId="session_abc123...",
 payload=json.dumps({"input": {"prompt": prompt}}),
 qualifier="DEFAULT",
 )

 return json.loads(response["response"].read())

Running the same fraud scenario against the deployed agent:

$ ./test.sh
Selecting Pulumi stack 'dev'...
Getting agent runtime ARN from Pulumi...
Agent Runtime ARN: arn:aws:bedrock-agentcore:us-east-1:052848974346:runtime/fraud_detection_agent-W7zFZT3Vcn
Invoking agent with fraud detection alert...
========================================
Sending prompt to agent...
Prompt: ALERT: New Transaction Attempt
User ID: user_123
Amount: $2000
Merchant: Electronics Store
Location: Tokyo, Japan
Time: 09:15
--------------------------------------------------
Agent Response:
{
"output": {
"message": {
"role": "assistant",
"content": [
{
"text": "**Analysis:** Impossible travel confirmed. User was in London at 09:00
and this transaction attempted in Tokyo at 09:15. Physically impossible.
Combined with high-risk merchant and $2000 amount, this is fraudulent.
Card blocked (TICKET-999)."
}
]
},
"timestamp": "2025-12-15T04:17:48.762738+00:00",
"model": "fraud-detection-agent"
}
}

The agent now runs in a managed, isolated environment with guardrails. Next, we add memory and external tool integration.

Production-Ready: Memory and MCP Gateway

Our basic deployment handles individual transactions, but fraud detection in the real world requires context. If a customer’s card was blocked five minutes ago, the agent shouldn’t try to block it again. If there’s a pattern of suspicious activity over the past week, the agent should factor that into its analysis.

This is where AgentCore Memory and the MCP Gateway come in.

AgentCore Memory: Short-term and Long-term

Memory in AgentCore works at two levels. Short-term memory stores raw conversation events within a session. If the agent blocks a card in one message, it remembers this in subsequent messages during the same session.

Long-term memory is more interesting. It uses semantic extraction to persist facts across sessions. The system runs your conversations through a smaller model (we use Claude Haiku) to extract key facts, then stores those facts in a namespace you define. When a new session starts, the agent can query for relevant facts about that user. Even if the customer calls back tomorrow, the agent knows their card was blocked yesterday.

Here’s how we configure the memory infrastructure:

# Create AgentCore Memory with 30-day event expiry
fraud_detection_memory = bedrock.AgentcoreMemory(
 "fraud-detection-memory",
 name="fraud_detection_memory",
 description="Memory for fraud detection agent",
 event_expiry_duration=30, # Events expire after 30 days
 memory_execution_role_arn=memory_execution_role.arn,
)

# Custom semantic extraction for fraud-related facts
FRAUD_EXTRACTION_PROMPT = """
Extract key fraud-related facts from this conversation about the user. Focus on:
1. Credit card status (blocked/active, ticket IDs if blocked)
2. Fraud alerts and patterns detected for this user
3. Transaction history patterns (locations, amounts, frequencies)
4. User profile details relevant to fraud detection
5. Previous fraud investigations and their outcomes

Format as concise facts that can help identify fraud patterns across sessions.
"""

# Create long-term memory strategy
fraud_semantic_strategy = bedrock.AgentcoreMemoryStrategy(
 "fraud-semantic-strategy",
 name="fraud_semantic_extraction",
 memory_id=fraud_detection_memory.id,
 type="CUSTOM",
 namespaces=["/fraud-detection/users/{actorId}"],
 configuration={
 "type": "SEMANTIC_OVERRIDE",
 "extraction": {
 "append_to_prompt": FRAUD_EXTRACTION_PROMPT,
 "model_id": "us.anthropic.claude-3-5-haiku-20241022-v1:0",
 },
 "consolidation": {
 "model_id": "us.anthropic.claude-3-5-haiku-20241022-v1:0",
 },
 },
)

The namespace pattern /fraud-detection/users/{actorId} ensures each user’s facts are isolated. Claude Haiku handles the extraction efficiently, keeping costs low while maintaining accuracy.

Long-term memory extraction is asynchronous. Plan for a 20-40 second delay between storing conversations and having facts available for retrieval. Use short-term memory for immediate needs while long-term memories are being processed.

In the agent code, we integrate memory using Strands hooks:

from bedrock_agentcore.memory import MemoryClient

memory_client = MemoryClient(region_name="us-east-1")

# Retrieve long-term facts before processing
memories = memory_client.retrieve_memories(
 memory_id=memory_id,
 namespace=f"/fraud-detection/users/{actor_id}",
 query=f"fraud detection facts for user {actor_id}"
)

# Store conversation after processing
memory_client.create_event(
 memory_id=memory_id,
 actor_id=actor_id,
 session_id=session_id,
 messages=[
 (user_message, "USER"),
 (assistant_message, "ASSISTANT")
 ]
)

MCP Gateway: Tool Integration at Scale

The tools we’ve used so far are baked into the agent container. That works for simple cases, but what happens when you have dozens of tools across multiple teams? Or when a tool needs to call an external service that requires its own authentication?

The Model Context Protocol (MCP) is an open standard for AI systems to communicate with tools. AgentCore Gateway provides a managed MCP endpoint that handles authentication (JWT inbound, OAuth2 outbound) and lets agents discover tools using natural language queries rather than knowing exact tool names.

We deploy an MCP server for risk scoring with three tools:

from mcp.server.fastmcp import FastMCP

mcp = FastMCP("Fraud Risk Scoring Server")

@mcp.tool()
def calculate_risk_score(user_id: str, amount: float, merchant: str, location: str) -> dict:
 """Calculate comprehensive fraud risk score (0-100)."""
 # Risk calculation logic
 return {
 "risk_score": 75,
 "factors": ["amount_anomaly", "high_risk_merchant"],
 "recommendation": "BLOCK"
 }

@mcp.tool()
def get_fraud_indicators(user_id: str) -> dict:
 """Get known fraud indicators for a user."""
 return {
 "indicators": ["velocity_alert", "device_inconsistency"],
 "severity": "HIGH"
 }

@mcp.tool()
def check_merchant_reputation(merchant_name: str) -> dict:
 """Check merchant risk rating and fraud history."""
 return {
 "risk_rating": "HIGH",
 "fraud_reports": 47,
 "chargeback_rate": 0.15
 }

The Gateway infrastructure requires careful attention to authentication. We use two separate Cognito user pools to handle the dual authentication flow:

Inbound authentication (Agent → Gateway): The agent authenticates to the Gateway using JWT tokens from the first Cognito pool. This ensures only authorized agents can discover and call tools.

# Gateway with MCP protocol and semantic search
agentcore_gateway = bedrock.AgentcoreGateway(
 "fraud-detection-gateway",
 name="fraud-detection-mcp-gateway",
 description="Gateway for fraud detection MCP tools",
 role_arn=gateway_role.arn,
 authorizer_type="CUSTOM_JWT",
 authorizer_configuration={
 "customJwtAuthorizer": {
 "discoveryUrl": gateway_discovery_url,
 "allowedClients": [gateway_app_client.id],
 },
 },
 protocol_type="MCP",
 protocol_configuration={
 "mcp": {
 "supportedVersions": ["2025-03-26"],
 "searchType": "SEMANTIC",
 "instructions": "Gateway for fraud detection risk scoring tools.",
 },
 },
)

The searchType: "SEMANTIC" configuration enables natural language tool discovery. Instead of the agent needing to know exact tool names, it can query for “tools that check merchant risk” and the Gateway returns relevant matches.

Outbound authentication (Gateway → MCP Server): When the Gateway calls the MCP server, it needs its own credentials. The OAuth2 Credential Provider manages this second authentication layer:

# OAuth2 Credential Provider for Gateway -> MCP Server auth
oauth2_credential_provider = bedrock.AgentcoreOauth2CredentialProvider(
 "mcp-runtime-oauth2-provider",
 name="fraud-mcp-runtime-oauth2",
 credential_provider_vendor="CustomOauth2",
 oauth2_provider_config={
 "customOauth2ProviderConfig": {
 "oauthDiscovery": {"discoveryUrl": mcp_runtime_discovery_url},
 "clientId": mcp_runtime_app_client.id,
 "clientSecret": mcp_runtime_app_client.client_secret,
 },
 },
)

Connecting the pieces: Finally, we create a Gateway Target. This links the Gateway to the MCP server endpoint. It also specifies which credential provider to use for outbound calls:

# Connect Gateway to MCP Server
gateway_target = bedrock.AgentcoreGatewayTarget(
 "mcp-risk-server-target",
 name="mcp-risk-server-target",
 gateway_identifier=agentcore_gateway.gateway_id,
 target_configuration={
 "mcp": {"mcpServer": {"endpoint": mcp_server_endpoint_url}},
 },
 credential_provider_configuration={
 "oauth": {
 "providerArn": oauth2_credential_provider.credential_provider_arn,
 "scopes": [mcp_runtime_scope_string],
 },
 },
)

This three-part configuration (Gateway, Credential Provider, Target) separates concerns cleanly: the Gateway handles inbound auth and protocol translation, the Credential Provider manages outbound secrets, and the Target defines routing.

Observability with X-Ray and OpenTelemetry

Production systems need observability. We configure CloudWatch Logs for application logs and X-Ray for distributed tracing:

# Agent application logs
agent_log_group = cloudwatch.LogGroup(
 "agent-runtime-log-group",
 name=pulumi.Output.concat(
 "/aws/vendedlogs/bedrock-agentcore/", agent_runtime.agent_runtime_id
 ),
 retention_in_days=30,
)

agent_logs_delivery = cloudwatch.LogDelivery(
 "agent-logs-delivery",
 delivery_source_name=agent_logs_delivery_source.name,
 delivery_destination_arn=agent_logs_delivery_destination.arn,
)

# X-Ray traces
agent_traces_delivery_destination = cloudwatch.LogDeliveryDestination(
 "agent-traces-delivery-destination",
 name="fraud-agent-traces-dest",
 delivery_destination_type="XRAY",
)

agent_traces_delivery = cloudwatch.LogDelivery(
 "agent-traces-delivery",
 delivery_source_name=agent_traces_delivery_source.name,
 delivery_destination_arn=agent_traces_delivery_destination.arn,
)

In the container, we use AWS Distro for OpenTelemetry for auto-instrumentation:

CMD ["uv", "run", "opentelemetry-instrument", "uvicorn", "agent:app", "--host", "0.0.0.0", "--port", "8080"]

This gives us end-to-end tracing across agent invocations, memory operations, and Gateway calls with zero code changes.

AWS has announced that X-Ray SDKs will be deprecated in favor of OpenTelemetry by February 2027. Using ADOT from the start ensures your observability stack is future-proof.

Running the Demo

Deploy the Infrastructure

cd advanced-bedrock-deployment/infra
uv sync
uv run pulumi up

This deploys approximately 40 AWS resources including:

ECR repositories for agent and MCP server
AgentCore Memory with semantic extraction strategy
Two Cognito user pools for authentication
AgentCore Gateway with MCP protocol
CloudWatch log groups and X-Ray trace delivery
IAM roles and policies

Test Short-term Memory

./test.sh demo

This runs 5 scenarios in the same session, demonstrating short-term memory:

======================================================================
FRAUD DETECTION DEMO - SHORT-TERM MEMORY (Same Session)
======================================================================
SCENARIO 1: John Doe - Impossible Travel Detection
Expected: FRAUD - impossible travel from London to Tokyo in 15 minutes
Actor ID: user_123
Session ID: session_cf171803eb324a2aa9ac456b3013fe45...
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": {
"text": "**FRAUD CONFIRMED - CARD BLOCKED**
| Finding | Details |
|---------|---------|
| **User** | John Doe |
| **Ticket** | TICKET-123-999 |
| **Fraud Type** | Impossible Travel + Spending Anomaly |
**Analysis:** Transaction in Tokyo 15 minutes after London purchase
is physically impossible. The $2,000 amount is 10x the user's typical
spending range ($50-200). Card blocked immediately."
},
"memory_enabled": true,
"long_term_facts_retrieved": 3
}
}
======================================================================
SCENARIO 3: John Doe - Follow-up Transaction (Short-term Memory Test)
Expected: FRAUD - card should already be blocked (SHORT-TERM MEMORY)
Actor ID: user_123
Session ID: session_cf171803eb324a2aa9ac456b3013fe45... (SAME SESSION!)
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": {
"text": "**Card for user_123 (John Doe) is ALREADY BLOCKED
(Ticket: TICKET-123-999). No further action needed.**
This appears to be another fraudulent attempt on an already-compromised
account. The card was previously blocked due to impossible travel fraud.
The current transaction follows the same pattern: Gift Card Kiosk
purchases are a common fraud tactic for converting stolen card access
to untraceable value."
},
"memory_enabled": true,
"long_term_facts_retrieved": 3
}
}
======================================================================
SHORT-TERM MEMORY DEMO SUMMARY
======================================================================
[OK] John Doe - Impossible Travel Detection
[OK] Jane Smith - Normal Transaction
[OK] John Doe - Follow-up Transaction (Short-term Memory Test)
[OK] Alice Chen - Impossible Travel Detection
[OK] Jane Smith - Second Normal Transaction

The agent remembers within the same session that John’s card is already blocked, so there’s no need to block it twice.

Test Long-term Memory

./test.sh longterm-fresh

This demonstrates cross-session memory with two phases:

======================================================================
PHASE 1: INITIAL FRAUD DETECTION
These interactions create long-term memory facts about users.
======================================================================
SCENARIO: John Doe - Initial Fraud (Card Blocked)
Session ID: session_xyz789... (NEW SESSION)
→ Card blocked, fact stored to long-term memory
======================================================================
WAITING FOR LONG-TERM MEMORY EXTRACTION...
The semantic strategy needs time to process conversations into facts.
Waiting 30 seconds...
======================================================================
======================================================================
PHASE 2: NEW SESSIONS - LONG-TERM MEMORY RETRIEVAL
These are COMPLETELY NEW sessions. The agent should remember
fraud-related facts about users from Phase 1.
======================================================================
SCENARIO: John Doe - NEW SESSION (Long-term Memory Test)
Session ID: session_new456... (DIFFERENT SESSION!)
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": "CARD ALREADY BLOCKED. According to my records,
this user's card was previously blocked due to fraud.
Ticket FRAUD-user_123-001 exists from a previous investigation.
The card remains blocked.",
"memory_enabled": true,
"long_term_facts_retrieved": 2
}
}

Even in a completely new session, the agent retrieves facts from long-term memory and knows John’s card was blocked yesterday.

Test MCP Gateway

./test.sh gateway-fresh

This runs 6 diverse fraud scenarios using both local tools and MCP Gateway tools:

======================================================================
LOADING MCP GATEWAY CONFIGURATION...
======================================================================
Gateway URL: https://fraud-detection-mcp-gateway-xxx.gateway.bedrock-agentcore.us-east-1.amazonaws.com/mcp
Token Endpoint: https://fraud-gateway-xxx.auth.us-east-1.amazoncognito.com/oauth2/token
Scope: fraud-detection-gateway/invoke
MCP Gateway configuration loaded successfully!
======================================================================
FRAUD DETECTION DEMO - MCP GATEWAY INTEGRATION
======================================================================
LOCAL TOOLS:
- get_user_profile: User details and home location
- get_recent_transactions: Last known transaction
- block_credit_card: Block card and create ticket
MCP GATEWAY TOOLS (via Risk Scoring Service):
- calculate_risk_score: Fraud risk score 0-100
- get_fraud_indicators: Known fraud indicators
- check_merchant_reputation: Merchant risk rating
======================================================================
SCENARIO 1: HIGH-RISK MERCHANT - Jane at CryptoExchange123
MCP Gateway: ENABLED
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": {
"text": "**✅ CARD BLOCKED** - Ticket: TICKET-456-999
**Summary:** Jane Smith's card blocked due to a $5,000 crypto exchange
transaction that is 10x her typical spend and outside her normal merchant
categories. Location matched home, but amount and merchant anomalies
indicate likely fraud or compromised credentials."
},
"mcp_gateway_enabled": true,
"mcp_tools_count": 4
}
}
======================================================================
SCENARIO 5: COMBINED SIGNALS - Alice's Multi-Factor Fraud
MCP Gateway: ENABLED
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": {
"text": "## 🚨 FRAUD CONFIRMED - CARD BLOCKED
**User:** Alice Chen (user_321)
**Ticket ID:** TICKET-321-999
**Fraud Indicators Detected:**
| Factor | Finding |
|--------|---------|
| ⛔ Impossible Travel | Singapore (10:00) → Los Angeles (11:30) = 90 min |
| ⛔ Amount Anomaly | $2,500 vs typical $50-300 (8x+ normal) |
| ⛔ Merchant Mismatch | Cash advance vs usual food/retail |
| ⛔ High-Risk Merchant | QuickCashAdvance (cash advance services) |
**Action Taken:** Card blocked immediately. This transaction exhibits
classic signs of stolen card credentials being used for rapid cash extraction."
},
"mcp_gateway_enabled": true,
"mcp_tools_count": 4
}
}
======================================================================
MCP GATEWAY DEMO SUMMARY
======================================================================
[OK] 1. HIGH-RISK MERCHANT - Jane at CryptoExchange123
Gateway: YES (4 tools)
[OK] 2. VELOCITY ATTACK - Bob's Rapid Transactions
Gateway: YES (4 tools)
[OK] 3. AMOUNT ANOMALY - Alice's Large Purchase
Gateway: YES (4 tools)
[OK] 4. KNOWN FRAUD PATTERNS - John's Indicator Check
Gateway: YES (4 tools)
[OK] 5. COMBINED SIGNALS - Alice's Multi-Factor Fraud
Gateway: YES (4 tools)
[OK] 6. CLEAN TRANSACTION - Jane's Normal Purchase
Gateway: YES (4 tools)

The MCP Gateway tools provide additional context (merchant reputation, risk scores, fraud indicators) that helps the agent make better decisions.

Architecture Summary

Here’s the complete architecture:

Key Takeaways

Event-driven first: Production agents are triggered by Kafka, SQS, or webhooks, not chat interfaces
Start simple: Build with Strands SDK locally, add AgentCore complexity only when needed
Guardrails early: Content filtering and topic restrictions prevent prompt injection
Memory has two levels: Short-term (session) and long-term (semantic extraction across sessions)
MCP for tools at scale: Gateway handles auth and discovery; you focus on tool logic
Observability required: X-Ray + CloudWatch are essential for debugging distributed AI
IaC for reproducibility: Pulumi deploys 40+ resources with proper dependencies

Cost Considerations

When evaluating AgentCore for production, consider these cost factors:

AgentCore Runtime charges per session-second. For fraud detection agents that typically run 10-30 seconds per transaction, this is cost-effective compared to maintaining always-on ECS containers. However, if your agents process thousands of transactions per second, the per-session costs add up. Consider batching or using Lambda for simple, fast agents.

Model costs dominate most deployments. Claude Opus 4.5 (used for reasoning) costs significantly more than Claude Haiku (used for memory extraction). We deliberately use Haiku for semantic extraction to keep memory costs low while maintaining accuracy. For high-volume production, consider whether Haiku or even smaller models could handle your primary reasoning.

Memory and Gateway have their own pricing tiers. AgentCore Memory charges for storage and retrieval operations. The Gateway adds overhead for OAuth2 token management and routing. For simple agents that don’t need cross-session memory or external tools, skip these components entirely.

Cognito user pools for authentication have a free tier (50,000 MAU) that covers most development and moderate production workloads. Beyond that, costs scale with active users.

Start with a basic deployment and add Memory/Gateway only when you have a clear need. The infrastructure cost savings of skipping unnecessary components often exceed the engineering cost of adding them later.

Conclusion

The fraud detection agent in this post grew from 30 lines of local Python to a production deployment with guardrails, dual-layer memory, MCP Gateway integration, and full observability, all defined as infrastructure as code.

The gap between prototype and production doesn’t have to be a chasm. AgentCore provides the managed runtime, Pulumi makes it reproducible, and the progressive approach lets you add complexity only when your use case demands it.

Get Started with Pulumi

Use Pulumi's open-source SDK to create, deploy, and manage infrastructure on any cloud.

AI Predictions for 2026: A DevOps Engineer's Guide

Engin Diri — Thu, 11 Dec 2025 00:00:00 +0000

The IDE is dying, and so is tool calling. OpenAI is not going to win. And next year, you’re going to be shipping code that you’ve never reviewed before, even as an experienced engineer.

These are bold claims, but the way we use AI in 2026 for coding and agents is going to look completely different. In this post, I want to cover my predictions and why they matter right now for DevOps engineers. Some of these are definitely hot takes, but that’s what makes this conversation worth having.

IDEs are dead (at least as we know them)

Traditional IDEs where the code is the focus of the interface are simply going to become irrelevant. We’re moving toward agent manager interfaces where we can kick off agents in parallel to work on different features in a codebase or even work on different projects at the exact same time.

We’re already seeing this transition. Google Antigravity combines a familiar AI-powered coding experience with a new agent-first interface. You can deploy agents that autonomously plan, execute, and verify complex tasks across your editor, terminal, and browser. Cursor 2.0 lets you run up to eight agents in parallel on a single prompt, using git worktrees or remote machines to prevent file conflicts. Each agent operates in its own isolated copy of your codebase.

AWS validated this direction at re:Invent 2025 by announcing “frontier agents” including Kiro for autonomous coding, along with dedicated security and DevOps agents. These agents maintain state, log actions, operate with policy guardrails, and integrate directly with CI/CD pipelines.

For infrastructure specifically, Pulumi Neo represents this same shift. Instead of writing code or running CLI commands for every operation, you describe what you need in natural language and Neo handles the implementation. It works across your entire infrastructure, understanding dependencies and creating execution plans that go through pull requests for review.

For DevOps engineers, this means your pipelines need to accommodate AI-generated code at scale. Multiple agents working simultaneously need isolated, reproducible environments. More generated code means more artifacts to track, version, and deploy.

The different paths to AI dominance

A lot of people think that in the future, a single large language model will have a monopoly and be the best at absolutely everything. But what’s really going to happen is different providers will specialize and focus on being the best at different things.

Google is going down the generalist route with Gemini, aiming to be the jack-of-all-trades. Anthropic is focusing on being the best for coding. You can see this in the benchmarks: when Opus 4.5 came out, the first benchmark they highlighted was for software engineering, because that’s what Anthropic is focusing on.

Amazon is carving out its own niche with the Nova model family, announced at re:Invent 2025. The Nova 2 lineup includes specialized models: Pro for complex reasoning, Sonic for real-time voice conversations, and Omni for simultaneous text, audio, and video processing. With Nova Forge, organizations can build custom frontier models by combining their proprietary data with AWS open weight models. The re:Invent message was clear: leveraging your first-party data is now fundamental to going beyond generic AI. We’re talking about 30-40% increases in accuracy when you bring your own data into the equation.

But here’s the hot take: I don’t think OpenAI is going to come out on top with any kind of specialization. They’ve disappointed time and time again with GPT-5 and GPT-4.5. With 4.5, they seemed to try to be the creative specialist, but it just didn’t work. The GPT-5 launch in August 2025 was described as “barely better than last month’s flavor of the month” and on some metrics it’s actually worse than earlier models.

For DevOps teams, this specialization means you’ll need infrastructure that’s model-agnostic and supports multiple AI backends. Plan for secrets management across multiple LLM providers and design your systems to swap models based on the task at hand.

The local AI breakthrough

2026 will be the year of local AI. We didn’t see that much this year besides DeepSeek at the start of 2025, which was a big deal. We had a couple of new models like Qwen 3, but nothing that fundamentally changed the game. Now we’re starting to see new hardware that makes it obvious we’re going to be able to run very large models on smaller devices.

There’s new AI chips that can run upwards of 120 billion parameter large language models on the edge, which would be a complete game-changer. Right now, hardware requirements are one of the biggest challenges for scaling local AI. If we can solve the hardware problem, we get 100% data privacy and zero-millisecond latency for our agents.

AWS is addressing this with Trainium3 UltraServers, their 3nm AI chips delivering 4.4x more compute than the previous generation. More significantly, AWS AI Factories allow organizations to deploy racks of Trainium chips and NVIDIA GPUs directly into their own data centers, addressing data sovereignty concerns while keeping AI inference close to the data.

For DevOps, this opens the door to zero-latency inference in CI/CD pipelines, complete data privacy for sensitive codebases, and reduced cloud costs for AI-heavy workloads.

Engineers as system architects

We’re finally going to get to the point where we’re not the coders. We delegate that entirely to our coding agents, and we become the system architects. This mirrors the evolution of other engineering disciplines. Civil engineers don’t fabricate the steel beams; they design the structure and verify the integrity.

I think of this as a three-step process:

Define: Set your objectives and the system that your agents will operate under
Orchestrate: Delegate the coding to your agents
Validate: Ensure the quality of the outputs and the overall system

We’re still in the loop. We are the final say in whatever is created, but we’re delegating the grunt work to our coding agents.

This is exactly the model that Pulumi Neo implements for infrastructure. When you give Neo a complex request, it creates a task plan outlining the steps it will take to accomplish your goal. This plan provides transparency into Neo’s approach and gives you the opportunity to adjust the strategy before execution begins. Neo operates in different modes: Review mode where everything requires approval, Balanced mode where only deployments need sign-off, or Auto mode for full autonomy. You define the boundaries, Neo orchestrates the work, and you validate through pull requests and previews.

For DevOps engineers, this shift means building robust validation infrastructure becomes critical. When AI writes the code, you need automated testing pipelines, security scanning, and verification systems that can operate at the speed of AI-generated changes.

Code execution is replacing tool calling

Here’s a key insight that kept coming up at re:Invent: models are no longer the bottleneck. Context is. Our agents are going to change a lot next year because code execution is starting to replace tool calling. The problem with tool calling right now is that all the capabilities you give an agent take up context upfront. When you try to give a lot of different tools to an agent, you completely overwhelm it.

Anthropic’s research on code execution with MCP addresses exactly this problem. Code execution is a massive token reduction, faster, and more flexible. You’re giving the agent the ability to generate its own capabilities at runtime by writing code to interact with APIs. A workflow that previously consumed about 150,000 tokens when tools were passed directly through the model was reimplemented with code execution and used only about 2,000 tokens. That’s a 98.7% reduction.

AWS embraced this pattern with Amazon Bedrock AgentCore, which now includes code interpretation capabilities. AgentCore supports any agent framework (CrewAI, LangGraph, OpenAI SDK) and provides memory, browser tools, and observability features that make code execution practical at enterprise scale.

For DevOps, this means you need sandboxed, secure execution environments for AI-generated code. Running agent-generated code requires appropriate isolation, resource limits, and monitoring.

Progressive disclosure and composable skills

The best part about code execution flexibility is it unlocks progressive disclosure. All I mean by that is: you have a lot of capabilities for an agent, but you don’t actually give all of them upfront. Instead, you allow the agent to discover capabilities and then leverage them in a more flexible way.

For each capability, you just have a bit of metadata or description that loads upfront. When the agent decides to leverage that capability, then you load the full instructions. Now you can practically scale to infinity because all capabilities don’t have to be loaded at runtime.

Claude Skills is a good example of this pattern. Skills are organized folders of instructions, scripts, and resources that agents can discover and load dynamically. At session start, the agent scans available skills and populates the system prompt with just a brief name and description (around 100 tokens). The full skill prompt loads only after Claude selects it, preventing context bloat while maintaining discoverability.

Kiro Powers addresses the same problem. Connecting five MCP servers can consume over 50,000 tokens, roughly 40% of an AI model’s context window, before you even type your first request. Powers bundle MCP servers, steering files, and hooks into units that load dynamically based on conversation context. Mention “payment” and the Stripe power activates. Datadog, Figma, and others have powers available.

For DevOps, this translates to modular infrastructure definitions, on-demand capability loading, and efficient resource utilization. Think about how you can apply this pattern to your own automation.

Agent-to-agent protocols are finally happening

Agent-to-agent protocols are where AI agents operate in a peer network, discover each other’s capabilities in real time, and interact autonomously. When Google released their A2A protocol earlier this year, there was a ton of buzz. A lot of people thought it was going to be the next big standard, like the next MCP. But then it kind of fell to the wayside.

The big reason is the chicken-and-egg problem. For A2A to be useful, you need many people to adopt it at the same time. Otherwise, if you build an A2A-compatible agent, it has no other agents to talk to. The whole value proposition is lost unless you already have a big network to attach to.

But that’s finally changing. The Linux Foundation launched the A2A project in June 2025, and adoption is accelerating. Adobe, Microsoft, SAP, ServiceNow, and S&P Global are all implementing A2A. In July 2025, Google released version 0.3 of the A2A protocol with a more stable interface critical to accelerating enterprise adoption.

Machines paying machines

My next big prediction is that machines paying machines is going to become a very big thing. Coinbase released the x402 protocol for exactly this: building AI agents that you expose over the internet but require payment whenever someone else interacts with them.

This goes really well with agent-to-agent protocols. You can create a peer network where you monetize your agents. They all leverage each other but make payments whenever they take advantage of another agent’s capabilities. Cryptocurrency is the perfect solution for this kind of machine-to-machine network because it enables micropayments to be made quickly and globally.

The x402 protocol has achieved 156,000 weekly transactions with 492% growth since launching in May 2025. It’s now integrated with Anthropic’s MCP Protocol, Google Gemini, OpenAI Codex, and other platforms. Stablecoins like USDC enable charging per request, per service, or per second of usage at near-zero transaction costs, enabling payments as low as $0.001 per request.

Artifact reviews instead of diff reviews

When we want to do a rigorous code review, traditionally, we look line by line at all the changes. But coding agents are getting to the point where they can prove their code works through artifacts. Instead of reviewing line by line, we can look at browser recordings, full working demos of a backend API, and other artifacts.

Google Antigravity is a perfect example. As part of its coding process, it can autonomously spin up your website, visit it, scroll through it, take screenshots, and record everything. Agents generate artifacts, including tangible deliverables like task lists, implementation plans, screenshots, and browser recordings. You can verify the agent’s logic at a glance.

Amazon Nova Act takes this further. It enables AI agents to automate browser-based tasks like form filling, QA testing, and workflow validation with over 90% reliability. The service includes built-in observability through live viewing, CloudTrail logging, and session replay, making it possible to review what an agent actually did rather than parsing through code changes.

Shipping code you’ve never read

For the last prediction, we’re tying everything together. We’ve talked about reviewing artifacts instead of diffs, creating systems instead of coding, and the new capabilities for agents with code execution.

We’re going to get to the point very quickly where we’re shipping code that we have never read before. And I’m not talking about people who vibe code. Even experienced engineers are going to trust their systems so much that they have the ability to review the code but they’re not going to. We’re just going to ship to production after reviewing the artifacts.

I presented on this exact topic at the Tel Aviv Pulumi User Group meetup at Qodo HQ back in October, where I demonstrated how Pulumi Neo’s autonomous decision-making capabilities can handle infrastructure tasks that we traditionally managed manually. Qodo is doing fascinating work in this space with their agentic development tools, building systems that let you trust the output without necessarily reviewing every line.

I’m not saying we’re taking the human completely out of the loop. I’m saying we’re going to have a lot of trust in our systems and a validation process that includes us, but that doesn’t necessarily have to be us actually looking at the code. Tools like Pulumi Neo create pull requests with clear documentation of changes, run previews to validate infrastructure modifications, and provide the transparency needed to ship with confidence.

The path forward

The predictions I’ve outlined point to a fundamental shift in how software gets built and deployed. For DevOps engineers, this isn’t a threat but an opportunity to become more strategic and less operational. We’re entering the battle of the agentic frameworks, where the winners will be those who can build faster, cheaper agentic applications through their platforms.

The immediate reality is that your CI/CD pipelines need to accommodate AI-generated code at scale, your secrets management needs to handle multiple LLM providers, and your execution environments need proper sandboxing for agent-generated code. These aren’t future concerns; they’re requirements for working effectively with the AI tools available today.

Looking further out, the engineers who thrive will be those who embrace the system architect role. Define clear objectives and constraints for your AI agents. Build validation frameworks that can verify outcomes without requiring line-by-line code review. Design infrastructure that’s modular enough to load capabilities on demand.

The technology to make this happen already exists. Agent orchestration platforms are shipping. Code execution is replacing tool calling. Progressive disclosure patterns are proven. The question isn’t whether these changes are coming; it’s whether you’ll be ready when they arrive.

Start building for the AI-driven future today

If you want to experience what this future looks like right now, Pulumi Neo is the place to start. Neo lets you make natural language requests for routine infrastructure tasks, analysis, and management. Instead of writing code for every operation, you describe what you need, and Neo handles the implementation, creating task plans, running previews, and opening pull requests for your review.

Whether you’re looking to update outdated resources across your infrastructure, analyze your cloud spend, or automate complex multi-step workflows, Neo provides the agent-first experience that’s defining the next generation of DevOps tooling.

Get started with Pulumi Neo and see how AI-powered infrastructure automation can transform your workflow.
Project: Build a production-ready AI agent by moving from local prototypes to deployed infrastructure with Amazon Bedrock AgentCore, the Strands SDK, and Pulumi.

AWS built an integrated AI Agent training pipeline and they want you to rent it

Adam Gordon Bell — Wed, 10 Dec 2025 00:00:00 +0000

AWS re:Invent 2025 delivered a myriad of announcements across AI, silicon, and cloud infrastructure. AWS unveiled the expanded Nova model family, introduced Nova Forge for custom model training, launched Trainium3 UltraServers, and added major production features to AgentCore. It was a lot, and taken at face value, it looks like another scattershot year of big releases.

But if you look past the firehose, a pattern emerges. These announcements fit together into a single bet about how enterprise AI will be built.

AWS built a vertically integrated agent-training pipeline; it’s expensive, ambitious, and not for everyone.

The clearest place to see that pattern is in how AWS talked about the Nova models. AWS dropped four new foundation models (Lite, Pro, Sonic, Omni) spanning text, multimodal, and speech. And AWS downplayed benchmarks entirely. A hint that the models aren’t the real story.

What AWS was really foregrounding wasn’t the models themselves, but the system used to shape them: Nova Forge.

Try it with Neo: Set up Nova 2 with Pulumi

Rent the lab: Nova Forge

Nova Forge is a managed way to run continued pretraining, fine-tuning, and reward-based alignment on Amazon’s Nova models using your data and your reinforcement loops. Instead of a finished, frozen model plus a thin fine-tuning API, you feed your data into earlier training stages while AWS handles the ugly parts: large-scale training runs, cluster management, and hosting. Access is $100,000 per year¹, plus compute costs!

If you can afford that, you bring big proprietary datasets (code, tickets, logs, documents) and they keep doing next-token pretraining on a mix of their data and yours, then instruction tuning (SFT), then “RL-style” preference optimization, but with your data and your reward signals mixed in.

Why does this exist? Because the kind of training this enables is usually out of reach. Training a GPT-4-class frontier model from scratch runs tens of millions to $100M+ in compute alone². You don’t own the weights and you’re locked into their stack, but you get frontier-level capabilities with your data baked in, without building datacenters or staffing ML teams.

Think of it as frontier-lab-as-a-service. No one else offers anything this close to a public, end-to-end training pipeline. And the only reason AWS can offer it is the next announcement.

The margin weapon: Trainium

The idealized Trainium flywheel: each generation should decrease training costs.

AWS built its own AI accelerator so it doesn’t have to live entirely on NVIDIA. Trainium is that chip. You don’t buy it; you rent it as a cloud box. This year: their third-gen chip (Trainium3) and new rack-scale Trn3 UltraServers are out, with 4× the performance and big energy/cost gains over the previous gen, positioned as a serious alternative to high-end GPUs for training and serving big models.

I’m sure one reason for Trainium’s is that AWS wants to stop handing NVIDIA half its AI training revenue. But the real story is bigger than cost-cutting. Trainium is the quiet machinery that makes AWS’s model-factory ambitions economically viable. You can only rent a frontier training pipeline if you can afford it, and Trainium makes it cheaper (if that word applies to six-figure entry costs).

Trainium is what turns Forge from a one-off experiment into an actual development pipeline. By compressing the marginal cost of each training cycle, AWS is trying to make iterative specialization economically viable. You can tune, test, and retrain until you converge on something useful.

AWS is clearly positioning Trainium3 to anchor a fully vertical stack

Spinning up a Trainium instance with Pulumi (where available):

import pulumi_aws as aws

# Assumes: ami, subnet, security_group already configured
trn1_instance = aws.ec2.Instance("trn1-instance",
 instance_type="trn1.2xlarge",
 ami=ami.id,
 subnet_id=subnet.id,
 vpc_security_group_ids=[security_group.id],
 associate_public_ip_address=True,
 tags={"Name": "trn1-training-instance"},
)

Try it with Neo: Provision Trainium instances

The data moat play

For most companies, this whole stack is overkill. If your AI roadmap is “add a chatbot and maybe summarize some tickets,” you don’t need Nova Forge, and you definitely don’t need Trainium. Hosted models plus RAG and an Agentic loop will get you 90% of the way there.

But this type of training is powerful, and it’s never been in reach to so many. If LLMs behave like the distributions they’re trained on, then getting your proprietary mess (logs, incident reports, claims histories, deal flows, call transcripts) into the core training loop means the model doesn’t just know your docs; it behaves like someone who’s lived inside your systems. That’s qualitatively different from “we stuffed a PDF into the context window.”

Latency and cost at scale matter too. For high-volume workflows like support triage, routing, code review, and fraud checks, “generic frontier model + giant prompt + RAG + tools” is slow and expensive. A smaller model that has your world baked into the weights can run with smaller contexts, simpler prompts, and fewer tool calls. And then there is reinforcement learning, which I’ll get to shortly.

But even if you get that far, a custom Nova model sitting in Bedrock is only half the story. You still need somewhere for it to act: a runtime, tools, policies, and an audit trail. That’s the gap AgentCore is meant to fill.

Where the models work: AgentCore

AgentCore: building blocks so you don't have to wire agents from scratch.

If Nova is the brain and Trainium is the muscle to build it, AgentCore is the nervous system.

AgentCore is a managed runtime for AI agents: instead of you wiring LLMs, tools, memory, auth, and logging together on Lambda or Fargate, AWS gives you a sticky per-session microVM, a standard way to call tools (Gateway), built-in long- and short-term memory, identity/permissions, and observability/evals. You package your agent, deploy it as an AgentCore runtime, and AWS handles the ugly parts: session isolation, scaling, policy guardrails, and tracing. You pay Fargate-ish per-vCPU/GB-hour pricing for the runtime plus normal Bedrock token and tool-call costs.

At re:Invent 2025, AgentCore picked up the missing “production” pieces: Policy, Evaluations, and episodic Memory. These handle guardrails, quality checks, and per-session state, so you don’t have to build them yourself.

Deploying an AgentCore runtime with Pulumi:

import pulumi_aws as aws

# Assumes: role, ecr_repo already configured
agent_runtime = aws.bedrock.AgentcoreAgentRuntime("my-agent",
 agent_runtime_name="my-agent-runtime",
 role_arn=role.arn,
 agent_runtime_artifact={
 "container_configuration": {
 "container_uri": f"{ecr_repo.repository_url}:latest",
 },
 },
 network_configuration={
 "network_mode": "PUBLIC",
 })

Try it with Neo: Deploy an AgentCore runtime — Requires a container image in ECR.

How does this come together? AWS shipped a use case.

The proof of concept: Nova Act

The AWS AI stack: vertically integrated from silicon to agent runtime. Nova Act uses the full stack.

Nova Act is the concrete example of this whole thing coming together. It handles browser-based UI automation: form filling, search-and-extract, QA testing. Amazon claims ~90% reliability. It deploys directly to AgentCore Runtime.

It’s not “an LLM plus Playwright.” Nova Act uses a specialized Nova 2 Lite variant trained on synthetic “web gym” environments: browser simulations that mirror enterprise UIs and provide an automatic reward signal when tasks are completed correctly. Instead of judging output quality, this model was trained on an RL loop that asks: Did the workflow succeed?

That specialized model is wrapped in AgentCore. The platform handles isolation, scaling, logging, and guardrails, so Nova Act behaves like a production automation system rather than a brittle demo.

Seen this way, Nova Act is Amazon’s reference implementation for a certain class of enterprise agents: start with a strong general model, specialize it through domain-specific RL in a controlled environment, and run it on AgentCore with tools and policies around it. It’s the pattern AWS expects customers to adopt.

One stack to rule them all

So Nova Forge, Trainium, AgentCore, and Nova Act connect. Trainium lowers the cost of big training runs. Nova Forge lets enterprises plug their own data and rewards into those runs. AgentCore is where the resulting models act, with tools, memory, and policy guardrails. Nova Act shows the pattern in action: a domain-specialized Nova model, trained in a controlled loop, running as a production agent.

Most enterprises still won’t choose this path. They don’t have the data, the reward loops, or the operational maturity to make early-stage training worthwhile.

But AWS’s bet is that enterprise AI is moving past stock foundation models and generic chatbots. AWS is expecting a world of agents shaped by proprietary data and domain feedback. Most companies won’t build the infrastructure to train and operate those agents, and so AWS is offering to rent them the whole pipeline.

Try it with Neo: Deploy a Bedrock-powered API with Pulumi

CNBC reporting on Nova Forge pricing. Source ↩︎
Sam Altman stated GPT-4 cost “more than $100 million” to train. Source ↩︎

Feature Flagging for Your Infrastructure

Elisabeth Lichtie — Sat, 06 Dec 2025 00:00:00 +0000

One of Pulumi’s foundational benefits is that it allows you to manage your infrastructure as software with rich programming languages, robust testing, and CI/CD patterns that you’d use with your application code. This post will cover applying another classic software development technique to your infrastructure: feature flagging. You can use feature flags to control change rollout, reduce the risk of new releases, and speed up the development of your infrastructure, the same way you do with your applications.

The examples in this post range from simply creating a flag and using it in a Lambda function to fully integrating with LaunchDarkly to build a comprehensive flagging system for your infrastructure.

Creating flags with Pulumi

Check out the code here: 01: Flag Application With LaunchDarkly.

When you define flags in your infrastructure as code, you can configure them alongside all the supporting infrastructure your code needs. You can also encourage flag creation as a best practice in Pulumi components or templates.

With Pulumi, you can use any Terraform provider to manage your feature flags, whether that’s LaunchDarkly, Statsig, Split, Flagsmith, or any other feature flagging tool with Terraform support. This lets you define flags, targeting rules, and the infrastructure they control in a single codebase. The example uses the LaunchDarkly provider to create flags and supporting infrastructure:

const flag = new launchdarkly.FeatureFlag("example-flag", {
 key: "exampleFlag",
 projectKey: "default",
 variationType: "boolean"
}, { provider });

The Lambda function evaluates the flag at runtime using the LaunchDarkly SDK key from environment variables, allowing you to toggle behavior without redeploying code.

Using flags to control infrastructure

Check out the code here: 02: Flaggable Infrastructure.

Before deploying this code, first deploy either 03 (for ESC Only) or 04 (for ESC and LaunchDarkly) so that the environment is created and available.

Once you have flags defined, you can use them to control infrastructure provisioning. Apply the same best practices you would for application feature flags: keep flags focused, remove them when they’re no longer needed, and document their purpose. In this example, a boolean flag controls whether internet-facing resources are created:

const config = new pulumi.Config();
const flags: Record<string, any> | undefined = config.getObject("flags");
const enableInternetAccess = flags ? flags["enableInternetAccess"] : false;

if (enableInternetAccess) {
 // Create Internet Gateway, public subnet, route table, and routes
}

To enable automatic infrastructure updates when flags change (as shown in examples 3 and 4), configure deployment settings in your stack. This requires:

The Pulumi GitHub App installed on your repository
Webhook configuration to trigger deployments when configuration values change

With these in place, changing a flag value automatically triggers a deployment.

Configuring values with ESC

Check out the code here: 03: ESC Auto-updating.

Ingesting config from ESC

Pulumi ESC provides a centralized way to manage configuration and secrets. For teams that don’t need the full feature set of a dedicated flagging service, you can store flag values directly in an ESC environment. Your infrastructure stack imports this environment and reads flag values at deployment time:

const pulumiProv = new pulumiservice.Environment("flaggingEnv", {
 name: "config",
 organization: pulumi.getOrganization(),
 yaml: new pulumi.asset.StringAsset(`values:
 pulumiConfig:
 flags:
 enableInternetAccess: 'true'`),
});

This approach keeps all configuration in one place, eliminates dependencies on external services, and works seamlessly with Pulumi’s existing stack configuration system. You can update flag values through the Pulumi CLI, API, or Cloud console.

Automatically triggering deployments with webhooks

There is some risk involved with automatically triggering updates. You can leave out the webhook and deploy manually if that’s safer for your team.

Set up a Pulumi Cloud webhook to automatically deploy infrastructure when ESC environment values change. The example creates an AWS Lambda function behind API Gateway that receives webhook events and triggers deployments via the Pulumi Deployments API:

const webhook = new pulumiservice.Webhook("escEnvironmentWebhook", {
 organizationName: pulumi.getOrganization(),
 displayName: "ESC Environment Change Webhook",
 payloadUrl: api.apiEndpoint.apply((endpoint) => `${endpoint}/webhook`),
 active: true,
 filters: ["environment_revision_created"],
});

When you update a flag value in ESC, the webhook fires, the Lambda identifies which stacks use that environment, and triggers their redeployment. This creates a continuous delivery pipeline for infrastructure configuration changes.

Ingesting flag values from LaunchDarkly

Check out the code here: 04: LaunchDarkly Auto-updating.

Ingesting flags from LaunchDarkly using ESC Connect

If your team already uses LaunchDarkly for application feature flags, you can integrate those same flags with your infrastructure using ESC Connect. This is also a good option if you need enterprise-grade features like advanced targeting rules or percentage rollouts.

This approach unifies application and infrastructure flag management. The ESC environment uses fn::open::external to query a Lambda function that fetches current flag values from LaunchDarkly:

values:
 pulumiConfig:
 flags: ${customSecrets.response}
 customSecrets:
 fn::open::external:
 url: ${getterUrl}
 request:
 secretName: ANY_SECRET
 secret: true

Your infrastructure code remains unchanged: it still imports the ESC environment and reads flag values the same way as in example 3. The difference is that flag values now come from LaunchDarkly, giving you access to advanced features like targeting rules, percentage rollouts, and flag scheduling.

Automatically triggering deployments with webhooks

There is some risk involved with automatically triggering updates. You can leave out the webhook and deploy manually if that’s safer for your team.

LaunchDarkly’s webhook system can trigger infrastructure updates whenever a flag changes. Here’s how the automation flow works:

graph LR
A[Update Flag in LaunchDarkly] --> B[LaunchDarkly Webhook]
B --> C[API Gateway]
C --> D[Lambda Function]
D --> E[Pulumi Deployments API]
E --> F[Infrastructure Updates]

When the infrastructure deploys, it retrieves the current flag values through ESC:

graph LR
A[Infrastructure Stack] --> B[ESC Environment]
B --> C[fn::open::external]
C --> D[Flag Getter Lambda]
D --> E[LaunchDarkly SDK]
E --> |Flag Value| D
D --> |Flag Value| C
C --> |Flag Value| B
B --> |Flag Value| A

The Lambda function is configured as a webhook endpoint in LaunchDarkly:

const launchDarklyWebhook = new LaunchDarklyWebhook("myWebhook", {
 pulumiProv: provCreation.pulumiEnv,
 flagKey: launchdarklyFlag.key,
 pulumiTokenSecret,
 lambdaRole,
}, { provider });

This creates a full automation loop: change a flag in LaunchDarkly’s UI, and your infrastructure automatically updates to reflect the new value. This pattern works with any feature flagging service that supports webhooks.

ESC Only vs ESC with LaunchDarkly

Both approaches enable automated infrastructure updates based on flag changes, but they suit different use cases:

Aspect	ESC Only (Example 3)	ESC with LaunchDarkly (Example 4)
Complexity	Simpler setup with fewer moving parts	More components but unified with application flags
Dependencies	No external services required	Requires LaunchDarkly subscription
Flag Management	Edit via Pulumi CLI or Cloud console	Edit via LaunchDarkly UI with full feature set
Advanced Features	Basic flag storage, versioning, and approvals	Targeting rules, percentage rollouts, experiments, scheduling
Best For	Teams wanting simple config management	Teams already using LaunchDarkly or needing sophisticated flag controls
Key Benefit	Centralized Pulumi-native configuration	Unified management of application and infrastructure flags

Conclusion

Infrastructure deserves the same development practices as application code. Feature flags let you control rollouts, reduce deployment risk, and coordinate changes across your entire system, not just in your applications, but in the infrastructure that supports them.

Pulumi makes infrastructure feature flagging practical. Whether you choose the simplicity of ESC-only flag management or integrate with an enterprise service like LaunchDarkly, you can build infrastructure that responds dynamically to configuration changes while maintaining the same development practices you use for application code.

Enforce AWS Organizations Tag Policies with Pulumi

Alejandro Cotroneo — Thu, 20 Nov 2025 10:00:00 -0800

Tags are the foundation of cloud governance, enabling cost allocation, ownership tracking, compliance reporting, and automation across your AWS infrastructure. Yet missing or inconsistent tags remain one of the most common governance challenges. Manual tag enforcement is error-prone, and discovering missing tags after deployment means your cost reports and compliance audits are already operating with incomplete data.

Today, we’re excited to announce a new pre-built policy pack created in partnership with AWS: AWS Organizations Tag Policies. This pack validates your infrastructure as code against tag policies configured in AWS Organizations, blocking deployments when required tags are missing and shifting tag governance left into your development workflow. Define your tag requirements once in AWS Organizations and enforce them consistently across all your Pulumi deployments.

How it works

The new policy pack integrates directly with your AWS Organizations Tag Policies as the single source of truth. No separate policy configuration or custom code required. When you run pulumi up, the pack retrieves your tag requirements from your AWS organization and validates that every resource has the required tags.

Start by enabling the pack in advisory mode to surface tagging violations in Pulumi Cloud’s Policy Findings hub without blocking deployments. This collaborative workspace allows your team to triage, prioritize, and systematically remediate missing tags. Once your infrastructure is compliant, switch to mandatory mode to prevent future non-compliant deployments.

Getting started

The pack works with both AWS Classic and AWS Native Pulumi providers, covering the full range of taggable AWS resources. To get started:

Configure tag policies in AWS Organizations following the AWS documentation.
Grant required permissions by adding the resourcegroupstaggingapi:ListRequiredTags permission to the IAM role or user that runs your Pulumi deployments.
Enable the pack in Pulumi Cloud:
1. From within your organization, navigate to the Policies tab
2. Under Policy Packs, select the Available tab
3. Select AWS Organizations Tag Policies and select Add to organization
4. From the Organizations tab, apply the policy to a Policy Group
Configure enforcement level: Set to advisory for warnings or mandatory to block non-compliant deployments.

Within minutes, every Pulumi deployment in your organization will validate tag compliance, ensuring that no resources are created without required tags.

Try it today

The AWS Organizations Tag Policies policy pack is now available to all Pulumi Team and Enterprise customers.

Get started with the integration
Learn about enforcing tag policies with AWS Organizations Tag Policies
Sign up for Pulumi Cloud if you’re new to Pulumi
Join the Community Slack to share feedback

We’re excited to partner with AWS on this capability and help organizations proactively enforce tag governance. Give it a try and let us know what you think!

10 Things You Can Do With Our Infrastructure Agent, Neo

Meagan Cojocar — Mon, 06 Oct 2025 00:00:00 +0000

Since launching Pulumi Neo two weeks ago, we’ve seen platform teams discover creative ways to put their newest AI teammate to work. We have also been using Neo internally for a handful of use cases. Neo shifts the conversation from “what could AI do for infrastructure?” to “what can I actually accomplish with Neo today?”

The answer is quite a bit. Here are 10 concrete workflows that platform teams can use Neo for right now, each one designed to save hours of manual work while keeping humans in the driver seat.

1. Explain infrastructure in plain English or diagrams

Neo answers questions about your infrastructure instantly, turning complex resource relationships into conversational insights (and diagrams!).

No more digging through AWS consoles or parsing state files to understand what you have deployed. Neo speaks your language about your infrastructure. Ask questions like “What databases do we have?”, “Which S3 buckets aren’t encrypted?”, “Show me all resources in us-east-1” or “draw an architecture diagram of our service” and get immediate, accurate answers.

2. Generate infrastructure code from natural language

Neo turns plain English descriptions into production-ready Pulumi code, eliminating the blank-page problem that slows down new projects.

Instead of starting from scratch or hunting through documentation, describe what you need and Neo generates the complete infrastructure. Ask Neo: “Create a serverless API with PostgreSQL database and Redis cache on AWS” and it produces fully configured resources with proper networking, security groups, and IAM policies.

3. Assist with Terraform to Pulumi migration

Neo analyzes existing Terraform code (and/or state files) and generates equivalent Pulumi programs, accelerating migration without starting from scratch.

Migrating from Terraform to Pulumi traditionally means rewriting everything. Neo understands both languages and can convert HCL to TypeScript, Python, or Go while preserving your infrastructure patterns. You can also use the Terraform state file. Ask Neo: “Analyze the provided terraform state. Based on it, generate a python pulumi program that imports the resources without requiring updates.” and it generates equivalent code with proper typing and best practices.

Using this prompt with a Terraform state file containing EC2 instances and supporting resources, Neo generated a complete Pulumi program that imported everything without requiring updates:

4. Automate CI/CD pipeline generation

Neo creates complete GitHub Actions workflows configured specifically for Pulumi deployments, with proper preview stages.

Setting up CI/CD for infrastructure projects usually involves copying workflows from other repos and adapting them. Neo generates purpose-built pipelines that understand your stack structure and deployment patterns. Ask Neo: “Create a GitHub Actions pipeline for this Pulumi project with preview and production stages” and it produces workflows with proper secrets management, approval gates, and deployment strategies.

Here is what Neo gives us using that prompt on a very basic Pulumi project:

name: Simple Pulumi Deploy

on:
 push:
 branches: [main]
 workflow_dispatch:

env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
 AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
 AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

jobs:
 deploy:
 runs-on: ubuntu-latest
 steps:
 - name: Checkout code
 uses: actions/checkout@v4

 - name: Install Pulumi
 uses: pulumi/actions@v4
 with:
 pulumi-version: latest

 - name: Configure AWS credentials
 uses: aws-actions/configure-aws-credentials@v4
 with:
 aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
 aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
 aws-region: us-west-2

 - name: Deploy infrastructure
 run: |
 pulumi stack select production
 pulumi up --yes
 working-directory: ./infrastructure

5. Identify inefficient infrastructure patterns

Neo analyzes your infrastructure and spots patterns that create unnecessary management overhead, helping you consolidate and optimize.

When you have 10 identical EC2 instances that should be an auto-scaling group, or multiple Lambda functions doing the same job, Neo spots these patterns. Ask Neo: “Analyze our infrastructure for inefficient patterns” and it can identify consolidation opportunities.

Example analysis:

Neo identified these optimization opportunities:

1. Duplicate EC2 Instances (High Priority)
 • Found 8 identical t3.medium instances running the same application
 • Recommendation: Consolidate into an Auto Scaling Group
 • Benefits: Simplified management, automatic scaling, reduced configuration drift

2. Scattered Lambda Functions (Medium Priority)
 • Found 12 Lambda functions with identical runtime and dependencies
 • Recommendation: Combine into a single function with routing logic
 • Benefits: Reduced cold starts, simplified monitoring, easier updates

3. Redundant Security Groups (Low Priority)
 • Found 15 security groups with identical rules
 • Recommendation: Create shared security groups for common patterns
 • Benefits: Easier rule management, reduced complexity

6. Upgrade EKS/Kubernetes clusters

Neo manages Kubernetes version upgrades across multiple clusters, planning upgrade paths and validating compatibility.

Kubernetes upgrades are complex, especially across multiple clusters with different workloads. Neo analyzes your cluster configurations, identifies upgrade blockers, and creates systematic upgrade plans. Ask Neo: “Plan an upgrade path for our EKS clusters from 1.27 to 1.28” and it maps dependencies, checks add-on compatibility, and sequences the upgrades.

7. Respond to security vulnerabilities

When CVEs are announced, Neo can help security teams move at machine speed addressing them.

Security vulnerabilities require immediate response across your entire infrastructure. Neo understands your resource inventory and can quickly identify what’s affected by new CVEs. When you ask Neo: “Identify all resources affected by CVE-2025-XXXXX and create remediation PRs”, it scans your infrastructure, maps vulnerable components, and generates targeted fixes.

8. Remediate policy violations

Neo detects non-compliant resources and generates pull requests to fix them, turning compliance headaches into automated workflows.

When your policy violations detect unencrypted EBS volumes, overly permissive security groups, or missing resource tags, Neo identifies the issues and proposes specific fixes. Ask Neo: “Fix all policy violations in our production account” and it analyzes your compliance dashboard, identifies the problematic resources, and creates PRs with the exact changes needed.

9. Update Lambda runtimes across multiple accounts

Neo identifies outdated Lambda runtimes, stages updates, and manages the rollout across your entire AWS organization.

When AWS announces runtime deprecations, Neo scans all your accounts, identifies functions running outdated versions, and creates a systematic upgrade plan. Ask Neo: “Upgrade all Lambda functions to Python 3.11” and it maps your functions, checks compatibility, and generates account-by-account PRs.

Before (outdated runtime):

// Create the process email Lambda function
const lambdaFunction = new aws.lambda.Function("emailProcessor", {
 runtime: aws.lambda.Runtime.Python3d8, // ❌ Deprecated
 code: new pulumi.asset.AssetArchive({
 ".": new pulumi.asset.FileArchive("./lambda"),
 }),
 // ... other configuration
});

After (Neo’s upgrade):

// Create the process email Lambda function
const lambdaFunction = new aws.lambda.Function("emailProcessor", {
 runtime: aws.lambda.Runtime.Python3d11, // ✅ Upgraded
 code: new pulumi.asset.AssetArchive({
 ".": new pulumi.asset.FileArchive("./lambda"),
 }),
 // ... other configuration
});

10. Enforce multi-account policy compliance

Neo applies organizational policies across your entire AWS Organizations structure, ensuring governance at scale without creating bottlenecks.

Large organizations struggle to maintain consistent policies across dozens of AWS accounts. Neo understands your organizational structure and can enforce tagging standards, encryption requirements, and security policies uniformly. Ask Neo: “Ensure all accounts follow our tagging and encryption policies” and it audits compliance across your organization.

Your newest platform engineer

These workflows represent just the beginning of what’s possible when an AI agent deeply understands infrastructure context.

The outcome is simple: spend less time on operational toil and more time on the architecture and policies that make your organizations successful.

Get started with Neo and discover what becomes possible when AI truly understands infrastructure.

The future of platform engineering isn’t about choosing between speed and safety. It’s about having tools that help you with both simultaneously.

Automatic Diagram Generation for Always-Accurate Diagrams

Elisabeth Lichtie — Mon, 22 Sep 2025 00:00:00 +0000

Architecture diagrams are one of the most tedious aspects of infrastructure management. Teams spend countless hours manually creating and updating visual representations of their systems, only to see them become outdated the moment infrastructure changes.

When you manage your infrastructure as code, you can automatically generate accurate, up-to-date diagrams directly from your IaC definitions and state. Pulumi’s preview capabilities let you anticipate exactly what changes will result from code modifications before deployment, providing the perfect foundation for automated diagramming workflows.

In this post, we’ll explore three practical approaches to automating diagram generation in your CI/CD pipeline:

Native Pulumi diagramming: Using Pulumi’s built-in diagramming capabilities to generate basic architecture diagrams that you can retrieve from GitHub Actions’ artifact archive
AI-powered diagramming: Leveraging the Claude app with Mermaid and Pulumi preview to generate detailed diagrams and automatically add them to pull requests for infrastructure changes
Pulumi Neo for enterprise diagramming: Using Pulumi’s AI-powered platform for organization-wide diagramming patterns, project-level and organization-level views, and intelligent customization for different stakeholders

These approaches eliminate manual diagramming work while ensuring your documentation stays current with your actual infrastructure, from simple automation to enterprise-scale intelligent diagramming.

Native Pulumi diagramming with GitHub Actions

Pulumi’s built-in pulumi stack graph command generates architecture diagrams directly from your deployed infrastructure state. This approach provides a reliable foundation for automated diagramming since it reflects your actual deployed resources rather than just the code.

Complete example: View the full working example on GitHub.

Here’s a complete GitHub Actions workflow that deploys infrastructure and automatically generates diagrams:

name: Pulumi Deploy & Diagram

on:
 push:
 branches: [main]

jobs:
 deploy:
 name: Deploy & Generate Diagram
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4

 - name: Setup Node.js
 uses: actions/setup-node@v4
 with:
 node-version: "20"
 cache: "npm"

 - name: Install dependencies
 run: npm ci

 - uses: pulumi/actions@v5
 with:
 command: up
 stack-name: dev
 env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}

 - name: Install Pulumi CLI
 run: |
 curl -fsSL https://get.pulumi.com | sh
 echo "$HOME/.pulumi/bin" >> $GITHUB_PATH

 - name: Generate Architecture Diagram
 run: pulumi stack graph architecture.dot --stack dev
 env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}

 - name: Install Graphviz
 run: sudo apt-get update && sudo apt-get install -y graphviz

 - name: Generate PNG from DOT file
 run: dot -Tpng architecture.dot -o architecture.png

 - name: Upload Architecture Diagrams
 uses: actions/upload-artifact@v4
 with:
 name: architecture-diagrams
 path: |
 architecture.dot
 architecture.png

How the workflow works

This workflow follows a straightforward pattern that ensures your diagrams always reflect your deployed infrastructure:

Deploy first: The workflow uses the official pulumi/actions@v5 action to deploy your infrastructure changes. This ensures the state reflects your latest code.
Generate diagram from state: The pulumi stack graph command reads your deployed stack’s state and generates a DOT format file containing the resource relationships and dependencies.
Convert to visual format: Graphviz converts the DOT file into a PNG image that’s easy to view and share.
Archive as artifacts: Both the raw DOT file and the rendered PNG are uploaded as GitHub Actions artifacts, making them downloadable from the workflow run.

Key advantages and limitations

Highest accuracy: Since diagrams are generated from deployed state rather than code, they reflect your actual infrastructure including any drift or manual changes. This is the biggest advantage of the native approach.

Zero maintenance: Once set up, diagrams update automatically with every deployment without any manual intervention.

Multiple formats: The DOT format can be converted to various output formats (PNG, SVG, PDF) depending on your needs.

Audit trail: Each workflow run preserves the diagram as an artifact, creating a visual history of your infrastructure changes.

However, this approach has some limitations:

Fixed styling: You’re locked into Pulumi’s diagram style and layout algorithms, with limited customization options.

Basic aesthetics: The generated diagrams are functional but not always aesthetically polished compared to hand-crafted or AI-generated alternatives.

Technical focus: Diagrams show all resources and relationships, which can be overwhelming for non-technical stakeholders who need higher-level architectural views.

Accessing your diagrams

After each workflow run, you can download the generated diagrams from the Actions tab in your repository. Click on any workflow run, scroll to the “Artifacts” section, and download the architecture-diagrams package.

For a more sophisticated setup, you could extend this workflow to automatically commit diagrams back to your repository, post them to Slack, or integrate them with your documentation system.

AI-powered diagramming with Claude and Mermaid

The second approach leverages Claude’s ability to analyze Pulumi preview output and generate aesthetically pleasing, contextually relevant diagrams using Mermaid syntax. This method addresses the styling limitations of native Pulumi diagrams while maintaining accuracy through pulumi preview integration.

Complete example: View the full working example on GitHub.

The automated workflow

This approach uses two GitHub Actions workflows working together:

Auto-comment workflow (auto-pr-comment.yml):

name: Auto PR Comment

on:
 pull_request:
 types: [opened]
 paths:
 - 'index.ts'
 - 'Pulumi.yaml'
 - 'Pulumi.*.yaml'

jobs:
 comment:
 if: github.actor != 'claude-code[bot]'
 runs-on: ubuntu-latest
 permissions:
 contents: read
 pull-requests: write
 steps:
 - name: Comment on PR
 uses: actions/github-script@v7
 with:
 github-token: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
 script: |
 github.rest.issues.createComment({
 issue_number: context.issue.number,
 owner: context.repo.owner,
 repo: context.repo.repo,
 body: '@claude update the diagrams to reflect any infrastructure changes.'
 })

Claude integration workflow (claude.yml):

name: Claude Code

on:
 issue_comment:
 types: [created]
 pull_request_review_comment:
 types: [created]

jobs:
 claude:
 if: contains(github.event.comment.body, '@claude')
 runs-on: ubuntu-latest
 permissions:
 contents: write
 pull-requests: write
 issues: write
 steps:
 - name: Checkout repository
 uses: actions/checkout@v4

 - name: Run Claude Code
 uses: anthropics/claude-code-action@beta
 env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
 with:
 anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
 allowed_tools: "Bash(npm install),Bash(pulumi stack select prod),Bash(pulumi preview --json),Bash(npx mmdc)"
 custom_instructions: |
 ## Pulumi Infrastructure Diagramming Assistant

 ### Step 1: Analyze Infrastructure State
 - Run `npm install` and `pulumi stack select prod`
 - Execute `pulumi preview --json` to get detailed resource plan
 - Parse JSON output to understand resource relationships and changes

 ### Step 2: Generate/Update Diagrams
 Update Mermaid diagram files in `/diagrams/`:
 - aws-infrastructure.mmd: Technical AWS resource diagram
 - c4-context.mmd: System context diagram
 - c4-container.mmd: Container-level view
 - c4-component.mmd: Component-level architecture

 ### Step 3: Generate SVG Files
 Convert .mmd files to .svg using mmdc CLI

How the AI approach works

Automatic triggering: When a pull request with infrastructure changes is opened, the auto-comment workflow adds a @claude mention, triggering Claude’s analysis.
Infrastructure analysis: Claude runs pulumi preview --json to get detailed information about planned infrastructure changes, including resource relationships and dependencies.
Intelligent diagram generation: Claude analyzes the preview output and updates multiple Mermaid diagrams:
- AWS infrastructure diagram: Technical view showing resources, networking, and data flow
- C4 diagrams: Context, container, and component views for different stakeholder needs
Visual rendering: The mmdc CLI converts Mermaid syntax to SVG files for immediate viewing.
PR integration: Updated diagrams are committed directly to the pull request branch, making them visible in code review.

Example generated diagram

Here’s a sample of the AWS infrastructure diagram that Claude generates:

graph TB
subgraph VPC["myVpc - CIDR: 10.0.0.0/16"]
subgraph PublicSubnet["myPublicSubnet - CIDR: 10.0.1.0/24"]
EC2["myInstance<br/>Type: t2.micro<br/>AMI: Latest Amazon Linux 2"]
end
IGW["myInternetGateway<br/>Internet connectivity"]
SG["mySecurityGroup<br/>HTTP (80) + SSH (22) access"]
end
WEB_USER["Web Users"] -->|HTTP requests| IGW
IGW --> PublicSubnet
SG --> EC2

Key advantages and limitations

Intelligent styling: Claude generates aesthetically pleasing diagrams with consistent formatting, colors, and layout that’s much more polished than native Pulumi output.

Multiple perspectives: Creates different diagram types (technical infrastructure, C4 architecture views) suitable for different audiences and stakeholder needs.

Context awareness: Claude understands the purpose and relationships of resources, creating meaningful groupings and flow representations.

Automated maintenance: Diagrams update automatically with infrastructure changes, staying current without manual intervention.

Integration with code review: Diagrams appear directly in pull requests, making infrastructure changes visible during review.

However, this approach also has limitations:

Potential for inaccuracy: Unlike native Pulumi diagrams that are generated directly from infrastructure state, AI-generated diagrams depend on Claude’s interpretation of preview output. While generally reliable, there’s inherent risk of misinterpretation or missed details that could lead to diagram inaccuracies.

AI dependency: Diagram accuracy is only as good as Claude’s understanding of complex infrastructure relationships, which may occasionally miss nuances or make incorrect assumptions.

Setup complexity: Requires more configuration including API keys, GitHub app installation, and workflow permissions.

Cost considerations: Each diagram update consumes Claude API credits, though costs are typically minimal for most projects.

Preview limitations: Diagrams reflect planned changes from pulumi preview rather than actual deployed state, which may differ if deployments fail.

This approach works best for teams that value diagram aesthetics and need multiple architectural views for different stakeholders, while accepting slightly more complexity and potential accuracy trade-offs in exchange for significantly better visual presentation.

Pulumi Neo: Enterprise-scale intelligent diagramming

Pulumi Neo brings the next evolution in infrastructure diagramming, with AI-powered intelligence that enables organization-wide diagram generation. Unlike the previous approaches that focus on individual stacks, Pulumi Neo provides enterprise-scale diagramming capabilities with intelligent customization for different stakeholders.

How the workflow works

Pulumi Neo streamlines enterprise diagramming through an intelligent, organization-aware approach that scales across multiple projects and stakeholders:

Repository setup: Create a dedicated repository to host your architecture diagrams and ensure your GitHub App has access. The repository structure helps Neo understand your diagramming patterns:
- overview/ folder for organization-wide infrastructure views
- stacks/ folder for individual stack diagrams
- Template files that Neo can model new diagrams from
- README documentation explaining the repository’s purpose and conventions

Neo prompt engineering: Use structured prompts to guide Neo’s diagram generation with enterprise-grade specificity. You can also select your architecture diagram repository to tell Neo where to store his work:

1. Scope: Create diagrams for all resources in my prod Pulumi stacks
1. Diagram Types:
- Individual stack diagrams showing resources within each stack
- Master overview diagram showing cross-stack dependencies and data flow
1. Visual Standards:
- Use official AWS/Azure/GCP service icons from their architecture icon sets
- Show resource relationships with labeled connections (security groups, subnets, etc.)
- Group related resources visually (VPC boundaries, availability zones)
1. Content Focus:
- Infrastructure resources only (compute, storage, networking, databases)
- Resource dependencies and data flow directions
- Cross-stack references and shared resources
1. Format: Generate diagrams as Mermaid code that can be rendered in documentation

Automated pull request generation: Neo analyzes your entire organization’s infrastructure state, identifies changes, and automatically creates pull requests with updated diagrams for review and approval.

Complete example: You can see my resulting architecture diagrams repository on GitHub.

Key advantages and limitations

Pulumi Neo offers the most sophisticated approach to infrastructure diagramming, particularly for enterprise environments:

Enterprise-scale intelligence: Neo understands your entire organization’s infrastructure landscape, generating diagrams that show cross-project dependencies and organization-wide resource relationships that individual stack-based approaches cannot capture.

Style customization: Customize your prompts to request different styles of diagrams.

Organizational consistency: Maintains consistent diagramming standards across teams and projects, ensuring visual coherence in large organizations with multiple infrastructure teams.

Cross-stack visibility: Unlike single-stack approaches, Neo provides organization-wide infrastructure views showing how different projects and teams’ resources interconnect.

Template-driven generation: Learns from your existing diagram patterns and templates to maintain consistency with your established visual standards and conventions.

AI-powered context understanding: Goes beyond simple resource mapping to understand business context, automatically highlighting critical paths, security boundaries, and architectural patterns.

However, Neo also has limitations:

AI interpretation risks: Like the Claude approach, AI-generated diagrams may miss nuances or make incorrect assumptions about complex enterprise infrastructure relationships.

Repository access requirements: Neo requires permissions to open pull requests against your repository.

Learning investment: Getting maximum value from Neo requires developing sophisticated prompt engineering skills and understanding how to structure repositories for optimal AI comprehension.

Conclusion

Automating diagram generation transforms one of the most tedious aspects of infrastructure management into a seamless part of your development workflow. All three approaches eliminate the manual overhead of keeping architecture diagrams current while solving the core problem of documentation drift.

Choose native Pulumi diagramming when accuracy is paramount and you need diagrams that perfectly reflect your deployed infrastructure. This approach offers the highest reliability with minimal setup complexity.

Choose AI-powered diagramming when you need polished, stakeholder-friendly diagrams and can accept slightly more complexity for significantly better aesthetics and multiple architectural perspectives.

Choose Pulumi Neo when you need enterprise-scale diagramming with organization-wide consistency, multi-level views across projects and stacks, and intelligent customization for different stakeholders with varying technical backgrounds.

Regardless of which approach you choose, you’ll eliminate hours of manual work while ensuring your documentation stays synchronized with your infrastructure changes. With Pulumi Neo’s AI capabilities now available, intelligent, context-aware diagramming has moved from possibility to reality, making infrastructure documentation both effortless and incredibly powerful.

The future of infrastructure documentation is here: automated, intelligent, and always accurate. With these examples as your foundation, you’re ready to transform how your team approaches architecture visualization at any scale.

Pulumi AWS Provider 7.0: Multi-Region Support, IAM Role Chaining, and S3 Resource Simplification

Cory Hall — Tue, 05 Aug 2025 09:00:00 -0400

Pulumi AWS provider 7.0 is here with powerful new capabilities that simplify and scale infrastructure as code on AWS. As the most widely used provider in the Pulumi ecosystem, it offers access to the full surface area of the upstream Terraform AWS Provider in Pulumi projects in all supported languages, like TypeScript, Python, Go, C#, Java, and YAML.

The 7.0 release brings fixes and improvements to the provider, including several breaking changes as part of the major version release.

What’s New in Pulumi AWS Provider 7.0

Multi-Region AWS Deployments with a Single Provider

Previously the Pulumi AWS provider each provider configuration could target a single AWS region. This meant that users would need to create a separate provider configuration for each region they wanted to deploy resources to. As the number of providers configured in an application grew, it could sometimes lead to increased memory usage.

With the 7.0 release, the AWS provider now supports multiple regions with a single provider. Each resource now has a region input property that allows the user to specify the region at the resource level, while still utilizing the same provider.

Some key highlights of this feature include:

Single provider configuration: This reduces the need to load multiple instances of the AWS provider, lowering memory usage.
Region input property: The region input property is added to all resources (except global resources)
Resource import enhancements: A new @<regionID> suffix allows importing resources from different regions

This example shows how to use the new region input property to configure cross-region VPC peering.

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const provider = new aws.Provider("provider", {region: "us-east-1"});
const mainVpc = new aws.ec2.Vpc("mainVpc", {cidrBlock: "10.0.0.0/16"}, {
 provider: provider,
});
const peerVpc = new aws.ec2.Vpc("peerVpc", {cidrBlock: "10.1.0.0/16"}, {
 provider: provider,
});
const peerConnection = new aws.ec2.VpcPeeringConnection("peerConnection", {
 vpcId: mainVpc.id,
 peerVpcId: peerVpc.id,
 peerRegion: "us-west-2",
 autoAccept: false,
}, {
 provider: provider,
});
const peerAcceptor = new aws.ec2.VpcPeeringConnectionAccepter("peerAcceptor", {
 region: "us-west-2",
 vpcPeeringConnectionId: peerConnection.id,
 autoAccept: true,
}, {
 provider: provider,
});

import pulumi
import pulumi_aws as aws

provider = aws.Provider("provider", region="us-east-1")
main_vpc = aws.ec2.Vpc("mainVpc", cidr_block="10.0.0.0/16",
opts = pulumi.ResourceOptions(provider=provider))
peer_vpc = aws.ec2.Vpc("peerVpc", cidr_block="10.1.0.0/16",
opts = pulumi.ResourceOptions(provider=provider))
peer_connection = aws.ec2.VpcPeeringConnection("peerConnection",
 vpc_id=main_vpc.id,
 peer_vpc_id=peer_vpc.id,
 peer_region="us-west-2",
 auto_accept=False,
 opts = pulumi.ResourceOptions(provider=provider))
peer_acceptor = aws.ec2.VpcPeeringConnectionAccepter("peerAcceptor",
 region="us-west-2",
 vpc_peering_connection_id=peer_connection.id,
 auto_accept=True,
 opts = pulumi.ResourceOptions(provider=provider))

package main

import (
 "github.com/pulumi/pulumi-aws/sdk/v7/go/aws"
 "github.com/pulumi/pulumi-aws/sdk/v7/go/aws/ec2"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 provider, err := aws.NewProvider(ctx, "provider", &aws.ProviderArgs{
 Region: pulumi.String("us-east-1"),
 })
 if err != nil {
 return err
 }
 mainVpc, err := ec2.NewVpc(ctx, "mainVpc", &ec2.VpcArgs{
 CidrBlock: pulumi.String("10.0.0.0/16"),
 }, pulumi.Provider(provider))
 if err != nil {
 return err
 }
 peerVpc, err := ec2.NewVpc(ctx, "peerVpc", &ec2.VpcArgs{
 CidrBlock: pulumi.String("10.1.0.0/16"),
 }, pulumi.Provider(provider))
 if err != nil {
 return err
 }
 peerConnection, err := ec2.NewVpcPeeringConnection(ctx, "peerConnection", &ec2.VpcPeeringConnectionArgs{
 VpcId: mainVpc.ID(),
 PeerVpcId: peerVpc.ID(),
 PeerRegion: pulumi.String("us-west-2"),
 AutoAccept: pulumi.Bool(false),
 }, pulumi.Provider(provider))
 if err != nil {
 return err
 }
 _, err = ec2.NewVpcPeeringConnectionAccepter(ctx, "peerAcceptor", &ec2.VpcPeeringConnectionAccepterArgs{
 Region: pulumi.String("us-west-2"),
 VpcPeeringConnectionId: peerConnection.ID(),
 AutoAccept: pulumi.Bool(true),
 }, pulumi.Provider(provider))
 if err != nil {
 return err
 }
 return nil
 })
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Aws = Pulumi.Aws;

return await Deployment.RunAsync(() =>
{
 var provider = new Aws.Provider("provider", new()
 {
 Region = "us-east-1",
 });

 var mainVpc = new Aws.Ec2.Vpc("mainVpc", new()
 {
 CidrBlock = "10.0.0.0/16",
 }, new CustomResourceOptions
 {
 Provider = provider,
 });

 var peerVpc = new Aws.Ec2.Vpc("peerVpc", new()
 {
 CidrBlock = "10.1.0.0/16",
 }, new CustomResourceOptions
 {
 Provider = provider,
 });

 var peerConnection = new Aws.Ec2.VpcPeeringConnection("peerConnection", new()
 {
 VpcId = mainVpc.Id,
 PeerVpcId = peerVpc.Id,
 PeerRegion = "us-west-2",
 AutoAccept = false,
 }, new CustomResourceOptions
 {
 Provider = provider,
 });

 var peerAcceptor = new Aws.Ec2.VpcPeeringConnectionAccepter("peerAcceptor", new()
 {
 Region = "us-west-2",
 VpcPeeringConnectionId = peerConnection.Id,
 AutoAccept = true,
 }, new CustomResourceOptions
 {
 Provider = provider,
 });

});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.aws.Provider;
import com.pulumi.aws.ProviderArgs;
import com.pulumi.aws.ec2.Vpc;
import com.pulumi.aws.ec2.VpcArgs;
import com.pulumi.aws.ec2.VpcPeeringConnection;
import com.pulumi.aws.ec2.VpcPeeringConnectionArgs;
import com.pulumi.aws.ec2.VpcPeeringConnectionAccepter;
import com.pulumi.aws.ec2.VpcPeeringConnectionAccepterArgs;
import com.pulumi.resources.CustomResourceOptions;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 var provider = new Provider("provider", ProviderArgs.builder()
 .region("us-east-1")
 .build());

 var mainVpc = new Vpc("mainVpc", VpcArgs.builder()
 .cidrBlock("10.0.0.0/16")
 .build(), CustomResourceOptions.builder()
 .provider(provider)
 .build());

 var peerVpc = new Vpc("peerVpc", VpcArgs.builder()
 .cidrBlock("10.1.0.0/16")
 .build(), CustomResourceOptions.builder()
 .provider(provider)
 .build());

 var peerConnection = new VpcPeeringConnection("peerConnection", VpcPeeringConnectionArgs.builder()
 .vpcId(mainVpc.id())
 .peerVpcId(peerVpc.id())
 .peerRegion("us-west-2")
 .autoAccept(false)
 .build(), CustomResourceOptions.builder()
 .provider(provider)
 .build());

 var peerAcceptor = new VpcPeeringConnectionAccepter("peerAcceptor", VpcPeeringConnectionAccepterArgs.builder()
 .region("us-west-2")
 .vpcPeeringConnectionId(peerConnection.id())
 .autoAccept(true)
 .build(), CustomResourceOptions.builder()
 .provider(provider)
 .build());

 }
}

name: cross-region-peering
runtime: yaml

resources:
 provider:
 type: pulumi:providers:aws
 properties:
 region: us-east-1
 mainVpc:
 type: aws:ec2:Vpc
 properties:
 cidrBlock: 10.0.0.0/16
 options:
 provider: ${provider}
 peerVpc:
 type: aws:ec2:Vpc
 properties:
 cidrBlock: 10.1.0.0/16
 options:
 provider: ${provider}
 peerConnection:
 type: aws:ec2:VpcPeeringConnection
 properties:
 vpcId: ${mainVpc.id}
 peerVpcId: ${peerVpc.id}
 peerRegion: us-west-2
 autoAccept: false
 options:
 provider: ${provider}
 peerAcceptor:
 type: aws:ec2:VpcPeeringConnectionAccepter
 properties:
 region: us-west-2
 vpcPeeringConnectionId: ${peerConnection.id}
 autoAccept: true
 options:
 provider: ${provider}

See the Enhanced Region Support guide for more details.

IAM Role Chaining for Secure AWS IaC Workflows

One of the most requested features, IAM role chaining, is now supported. This allows the Pulumi AWS Provider to assume multiple IAM roles in sequence to obtain credentials, streamlining secure AWS access across accounts and teams.

To assume a role with role chaining, you can now do the following:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const provider = new aws.Provider("provider", {assumeRoles: [
 {
 roleArn: "arn:aws:iam::123456789012:role/INITIAL_ROLE_NAME",
 },
 {
 roleArn: "arn:aws:iam::123456789012:role/FINAL_ROLE_NAME",
 },
]});

import pulumi
import pulumi_aws as aws

provider = aws.Provider("provider", assume_roles=[
 {
 "role_arn": "arn:aws:iam::123456789012:role/INITIAL_ROLE_NAME",
 },
 {
 "role_arn": "arn:aws:iam::123456789012:role/FINAL_ROLE_NAME",
 },
])

package main

import (
 "github.com/pulumi/pulumi-aws/sdk/v7/go/aws"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err := aws.NewProvider(ctx, "provider", &aws.ProviderArgs{
 AssumeRoles: aws.ProviderAssumeRoleArray{
 &aws.ProviderAssumeRoleArgs{
 RoleArn: pulumi.String("arn:aws:iam::123456789012:role/INITIAL_ROLE_NAME"),
 },
 &aws.ProviderAssumeRoleArgs{
 RoleArn: pulumi.String("arn:aws:iam::123456789012:role/FINAL_ROLE_NAME"),
 },
 },
 })
 if err != nil {
 return err
 }
 return nil
 })
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Aws = Pulumi.Aws;

return await Deployment.RunAsync(() =>
{
 var provider = new Aws.Provider("provider", new()
 {
 AssumeRoles = new[]
 {
 new Aws.Inputs.ProviderAssumeRoleArgs
 {
 RoleArn = "arn:aws:iam::123456789012:role/INITIAL_ROLE_NAME",
 },
 new Aws.Inputs.ProviderAssumeRoleArgs
 {
 RoleArn = "arn:aws:iam::123456789012:role/FINAL_ROLE_NAME",
 },
 },
 });

});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.aws.Provider;
import com.pulumi.aws.ProviderArgs;
import com.pulumi.aws.inputs.ProviderAssumeRoleArgs;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 var provider = new Provider("provider", ProviderArgs.builder()
 .assumeRoles(
 ProviderAssumeRoleArgs.builder()
 .roleArn("arn:aws:iam::123456789012:role/INITIAL_ROLE")
 .build(),
 ProviderAssumeRoleArgs.builder()
 .roleArn("arn:aws:iam::123456789012:role/FINAL_ROLE")
 .build())
 .build());

 }
}

name: example
runtime: yaml
resources:
 provider:
 type: pulumi:providers:aws
 properties:
 assumeRoles:
 - roleArn: arn:aws:iam::123456789012:role/INITIAL_ROLE_NAME
 - roleArn: arn:aws:iam::123456789012:role/FINAL_ROLE_NAME

With this configuration the provider will use the INITIAL_ROLE to assume the FINAL_ROLE which will then be used as the credentials for the provider.

Simplified S3 Bucket Resource Model

In v6 of the Pulumi AWS Provider the S3 Bucket and BucketV2 resources represented different resource implementations. BucketV2 represented the latest version of the upstream Terraform resource, while Bucket was a separate resource maintained by Pulumi to keep backwards compatibility with the v4 release of the upstream Terraform Provider.

In v7 we are taking the first step in unifying these two resources by moving the S3 Bucket resource to the latest upstream implementation. As a result, in v7 both Bucket and BucketV2 will represent the latest version of the upstream Terraform S3 Bucket resource. As part of this work BucketV2 has been deprecated and will be removed in v8 of the Pulumi AWS Provider.

We have also introduced new S3 Bucket configuration resources that are alternatives to their V2 counterparts. The V2 versions will be removed in v8 of the Pulumi AWS Provider.

BucketAccelerateConfigurationV2 => BucketAccelerateConfiguration
BucketRequestPaymentConfigurationV2 => BucketRequestPaymentConfiguration
BucketAclV2 => BucketAcl
BucketCorsConfigurationV2 => BucketCorsConfiguration
BucketLifecycleConfigurationV2 => BucketLifecycleConfiguration
BucketLoggingV2 => BucketLogging
BucketObjectLockConfigurationV2 => BucketObjectLockConfiguration
BucketServerSideEncryptionConfigurationV2 => BucketServerSideEncryptionConfiguration
BucketVersioningV2 => BucketVersioning
BucketWebsiteConfigurationV2 => BucketWebsiteConfiguration

See the Migration Guide for more details on this change.

Upstream Improvements and Breaking Changes

Pulumi AWS 7.0 includes all improvements and bug fixes from the upstream (terraform-provider-aws) versions from 6.0.0 to 6.3.0. It also contains a number of upstream breaking changes. Please refer to the changelog to navigate the entire list.

Migration Guide

You can see a full list of changes and learn more about migrating your existing programs in our Migration Guide.

Getting Started with Pulumi AWS 7.0

Pulumi makes it easy to define, deploy, and manage modern AWS infrastructure with code:

Use your favorite languages: TypeScript, Python, Go, C#, Java, YAML
Explore the Pulumi AWS Provider repro
Check out the API reference docs

Most Effective Infrastructure as Code (IaC) Tools

Asaf Ashirov — Thu, 26 Jun 2025 00:00:00 +0000

Infrastructure as Code (IaC) has evolved beyond simple automation into a fundamental shift toward applying software engineering practices to infrastructure management. In 2025, leading organizations aren’t just provisioning infrastructure—they’re treating it as software, complete with testing, version control, code reviews, and continuous integration.

As infrastructure complexity grows, teams increasingly seek approaches that provide the same developer productivity tools they use for application development. While template-based and domain-specific language approaches serve many use cases effectively, teams with complex requirements or programming backgrounds often find that general-purpose programming languages offer advantages in testing, abstraction, and collaboration.

This comprehensive guide examines the most effective infrastructure as code tools available today, providing detailed analysis of core IaC platforms, complementary tools, and related technologies through the lens of software engineering best practices. Whether you’re starting fresh with IaC or evaluating alternatives to overcome limitations in your current toolchain, we’ll help you navigate this complex landscape and choose solutions that truly bring software engineering to infrastructure.

What is Infrastructure as Code?

Infrastructure as Code (IaC) is an approach to automating the provisioning and management of infrastructure using software engineering principles, approaches, and tools. Rather than manually configuring servers, networks, and cloud resources through user interfaces or command-line tools, IaC enables you to define your entire infrastructure declaratively through code.

This approach brings the same benefits that have revolutionized software development—version control, automated testing, code reviews, and CI/CD pipelines—to infrastructure management.

What Are Infrastructure as Code Tools?

Infrastructure as Code tools are platforms and frameworks that enable you to define, provision, and manage infrastructure resources through code rather than manual processes. These tools translate your infrastructure definitions into API calls that create, modify, or destroy cloud resources across various providers.

The most effective IaC tools share several key characteristics:

Goal-state focus: Define your desired infrastructure outcome, whether through declarative syntax (like YAML/JSON templates) or imperative languages that express declarative intent
Multi-cloud support: Work across different cloud providers and services
State management: Track the current state of your infrastructure
Preview capabilities: Show what changes will be made before applying them
Idempotency: Safe to run multiple times with consistent results

Why Infrastructure as Code Tools Are Essential

The shift to IaC tools addresses fundamental challenges that manual infrastructure management cannot solve at scale:

Accelerates Deployment Velocity: Teams can provision complex multi-cloud architectures in minutes instead of weeks. This speed enables faster time-to-market and more frequent, reliable deployments.

Enables True Collaboration: Infrastructure becomes code that teams can review, test, and approve together. This collaborative approach reduces errors and ensures knowledge sharing across the organization.

Eliminates Configuration Drift: Manual changes lead to inconsistencies between environments. IaC ensures your production, staging, and development environments remain identical, eliminating the notorious “works on my machine” syndrome for infrastructure.

Provides Cost Control: Automated provisioning and deprovisioning prevents resource sprawl. Teams can easily track infrastructure costs, set budget alerts, and optimize resource usage across environments.

Ensures Compliance and Security: Codified security policies and compliance requirements are automatically enforced across all deployments. Audit trails become automatic, and policy violations are caught before deployment.

Guarantees Business Continuity: Complete infrastructure definitions stored in version control enable rapid disaster recovery. Organizations can reconstruct entire environments from code, minimizing downtime and data loss.

Infrastructure as Code Tools Overview

This guide covers the following infrastructure as code tools and platforms:

10 Most Used IaC Tools in 2025

Pulumi IaC - Modern IaC with general-purpose programming languages
Terraform - BSL-licensed IaC from HashiCorp that uses the HCL domain-specific language
AWS CDK - Cloud Development Kit for AWS
AWS CloudFormation - Native AWS integration
Azure Resource Manager (ARM) - Native Azure JSON templates
Azure Bicep - Azure-native domain-specific language that compiles to ARM
Google Cloud Infrastructure Manager - Terraform-based solution for Google Cloud
Kubernetes YAML - Native Kubernetes resource definitions
Crossplane - Kubernetes as universal control plane
OpenTofu - Open-source Terraform alternative

Configuration Management Tools

Chef - Configuration management and compliance automation
Puppet - Configuration management and compliance automation
Salt - Configuration management and remote execution

Application and Platform Management Tools

Kubernetes - Container orchestration platform

Security and Compliance Tools

Pulumi ESC - Configuration and secrets management platform
Pulumi Insights - Cloud resource search, analytics, and compliance platform
Snyk - Developer security platform with IaC scanning
Wiz - Comprehensive cloud security platform
HashiCorp Sentinel - Policy-as-code framework for Terraform
Checkov - Static analysis for IaC security
TFLint - Terraform linting and validation

Infrastructure Automation and Management Platforms

Pulumi Cloud - Managed service for Pulumi IaC with enterprise features
HashiCorp Cloud Platform - Enterprise SaaS platform for Terraform management and automation
Spacelift - Automation platform for IaC workflows (not an IaC tool itself)
Env0 - Governance and automation platform for existing IaC tools

Core Infrastructure as Code Tools

1. Pulumi

License: Apache 2.0
Best For: Teams who want flexible, language-agnostic IaC for infrastructure and operations

Pulumi IaC represents a modern approach to infrastructure as code, fundamentally changing how teams approach infrastructure by enabling the use of general-purpose programming languages like Python, TypeScript, Go, C#, and Java, plus YAML for simpler configurations. Unlike tools that force teams to learn proprietary domain-specific languages (DSLs), Pulumi leverages familiar languages and software engineering practices, providing unprecedented flexibility, powerful abstractions, and seamless integration with existing development workflows.

Pulumi’s approach combines the best of both imperative and declarative paradigms: you use imperative programming languages to define your desired infrastructure state, but the Pulumi engine processes this declaratively to determine what changes are needed to achieve your intended outcome.

Key Features:

Universal language support: Use Python, TypeScript, Go, C#, Java, or YAML configurations—no new DSL to learn
Any cloud, any architecture: Deploy to AWS, Azure, Google Cloud, Kubernetes, and 100+ other providers
Real programming constructs: Leverage loops, conditionals, functions, classes, packages, and third-party libraries
Superior developer experience: Full IDE support with IntelliSense, debugging, and refactoring
Built-in testing: Unit and integration testing for infrastructure code
Policy as Code: Enforce compliance and security policies with CrossGuard
Component ecosystem: Rich library of reusable infrastructure components

Universal Language Code Examples:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";

// Create a VPC with automatic subnets
const vpc = new awsx.ec2.Vpc("main-vpc", {
 cidrBlock: "10.0.0.0/16",
 numberOfAvailabilityZones: 2,
});

// Create an ECS cluster
const cluster = new aws.ecs.Cluster("app-cluster");

// Create an Application Load Balancer
const alb = new awsx.elasticloadbalancingv2.ApplicationLoadBalancer("app-alb", {
 vpcId: vpc.vpcId,
 subnetIds: vpc.publicSubnetIds,
});

// Deploy a containerized application
const service = new awsx.ecs.FargateService("app-service", {
 cluster: cluster.arn,
 taskDefinitionArgs: {
 container: {
 image: "nginx:latest",
 memory: 128,
 ports: [{
 containerPort: 80,
 targetGroup: alb.defaultTargetGroup,
 }],
 },
 },
});

export const vpcId = vpc.vpcId;
export const serviceUrl = alb.loadBalancer.dnsName;

import pulumi
import pulumi_aws as aws
import pulumi_awsx as awsx

# Create a VPC with automatic subnets
vpc = awsx.ec2.Vpc("main-vpc",
 cidr_block="10.0.0.0/16",
 number_of_availability_zones=2)

# Create an ECS cluster
cluster = aws.ecs.Cluster("app-cluster")

# Create an Application Load Balancer
alb = awsx.elasticloadbalancingv2.ApplicationLoadBalancer("app-alb",
 vpc_id=vpc.vpc_id,
 subnet_ids=vpc.public_subnet_ids)

# Deploy a containerized application
service = awsx.ecs.FargateService("app-service",
 cluster=cluster.arn,
 task_definition_args=awsx.ecs.FargateServiceTaskDefinitionArgs(
 container=awsx.ecs.TaskDefinitionContainerDefinitionArgs(
 image="nginx:latest",
 memory=128,
 ports=[awsx.ecs.TaskDefinitionPortMappingArgs(
 container_port=80,
 target_group=alb.default_target_group
 )]
 )
 ))

pulumi.export("vpc_id", vpc.vpc_id)
pulumi.export("service_url", alb.load_balancer.dns_name)

package main

import (
 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ecs"
 "github.com/pulumi/pulumi-awsx/sdk/v2/go/awsx/ec2"
 "github.com/pulumi/pulumi-awsx/sdk/v2/go/awsx/elasticloadbalancingv2"
 awsxecs "github.com/pulumi/pulumi-awsx/sdk/v2/go/awsx/ecs"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // Create a VPC with automatic subnets
 vpc, err := ec2.NewVpc(ctx, "main-vpc", &ec2.VpcArgs{
 CidrBlock: pulumi.String("10.0.0.0/16"),
 NumberOfAvailabilityZones: pulumi.Int(2),
 })
 if err != nil {
 return err
 }

 // Create an ECS cluster
 cluster, err := ecs.NewCluster(ctx, "app-cluster", nil)
 if err != nil {
 return err
 }

 // Create an Application Load Balancer
 alb, err := elasticloadbalancingv2.NewApplicationLoadBalancer(ctx, "app-alb", &elasticloadbalancingv2.ApplicationLoadBalancerArgs{
 VpcId: vpc.VpcId,
 SubnetIds: vpc.PublicSubnetIds,
 })
 if err != nil {
 return err
 }

 // Deploy a containerized application
 _, err = awsxecs.NewFargateService(ctx, "app-service", &awsxecs.FargateServiceArgs{
 Cluster: cluster.Arn,
 TaskDefinitionArgs: &awsxecs.FargateServiceTaskDefinitionArgs{
 Container: &awsxecs.TaskDefinitionContainerDefinitionArgs{
 Image: pulumi.String("nginx:latest"),
 Memory: pulumi.Int(128),
 Ports: awsxecs.TaskDefinitionPortMappingArray{
 &awsxecs.TaskDefinitionPortMappingArgs{
 ContainerPort: pulumi.Int(80),
 TargetGroup: alb.DefaultTargetGroup,
 },
 },
 },
 },
 })
 if err != nil {
 return err
 }

 ctx.Export("vpcId", vpc.VpcId)
 ctx.Export("serviceUrl", alb.LoadBalancer.DnsName())
 return nil
 })
}

using System.Collections.Generic;
using Pulumi;
using Aws = Pulumi.Aws;
using Awsx = Pulumi.Awsx;

return await Deployment.RunAsync(() =>
{
 // Create a VPC with automatic subnets
 var vpc = new Awsx.Ec2.Vpc("main-vpc", new()
 {
 CidrBlock = "10.0.0.0/16",
 NumberOfAvailabilityZones = 2,
 });

 // Create an ECS cluster
 var cluster = new Aws.Ecs.Cluster("app-cluster");

 // Create an Application Load Balancer
 var alb = new Awsx.ElasticLoadBalancingV2.ApplicationLoadBalancer("app-alb", new()
 {
 VpcId = vpc.VpcId,
 SubnetIds = vpc.PublicSubnetIds,
 });

 // Deploy a containerized application
 var service = new Awsx.Ecs.FargateService("app-service", new()
 {
 Cluster = cluster.Arn,
 TaskDefinitionArgs = new Awsx.Ecs.Inputs.FargateServiceTaskDefinitionArgs
 {
 Container = new Awsx.Ecs.Inputs.TaskDefinitionContainerDefinitionArgs
 {
 Image = "nginx:latest",
 Memory = 128,
 Ports = new[]
 {
 new Awsx.Ecs.Inputs.TaskDefinitionPortMappingArgs
 {
 ContainerPort = 80,
 TargetGroup = alb.DefaultTargetGroup,
 },
 },
 },
 },
 });

 return new Dictionary<string, object?>
 {
 ["vpcId"] = vpc.VpcId,
 ["serviceUrl"] = alb.LoadBalancer.Apply(lb => lb.DnsName),
 };
});

name: aws-ecs-example
runtime: yaml
description: An example that deploys an ECS Fargate service with load balancer

resources:
 # Create a VPC with automatic subnets
 main-vpc:
 type: awsx:ec2:Vpc
 properties:
 cidrBlock: "10.0.0.0/16"
 numberOfAvailabilityZones: 2

 # Create an ECS cluster
 app-cluster:
 type: aws:ecs:Cluster

 # Create an Application Load Balancer
 app-alb:
 type: awsx:elasticloadbalancingv2:ApplicationLoadBalancer
 properties:
 vpcId: ${main-vpc.vpcId}
 subnetIds: ${main-vpc.publicSubnetIds}

 # Deploy a containerized application
 app-service:
 type: awsx:ecs:FargateService
 properties:
 cluster: ${app-cluster.arn}
 taskDefinitionArgs:
 container:
 image: "nginx:latest"
 memory: 128
 ports:
 - containerPort: 80
 targetGroup: ${app-alb.defaultTargetGroup}

outputs:
 vpcId: ${main-vpc.vpcId}
 serviceUrl: ${app-alb.loadBalancer.dnsName}

Key Features:

General-purpose language support: Use Python, TypeScript, Go, C#, Java, or YAML without learning new DSLs
Software engineering practices: Full IDE support, comprehensive testing frameworks, debugging capabilities
Multi-cloud flexibility: Native cloud provider SDKs with same-day feature access across 150+ providers
Incremental adoption: Migration tools and state integration for gradual transitions
Open source licensing: Apache 2.0 ensures long-term freedom and flexibility

Considerations:

Learning curve: Teams new to programming may prefer template-based approaches initially
Ecosystem maturity: Smaller community compared to more established tools like Terraform
Tool complexity: Advanced features may require more setup than simpler template systems

Organizations like Unity, Snowflake, and Starburst have reported significant productivity improvements (80-90% deployment time reductions) when adopting programming language-based approaches. These improvements typically occur when transitioning from manual processes or basic template systems to automated approaches with comprehensive testing, IDE integration, and code reusability. Results vary based on starting point, team expertise, infrastructure complexity, and specific use cases.

Ready to get started? Experience Pulumi’s programming language approach and see how familiar languages can transform your infrastructure management with comprehensive testing, powerful abstractions, and seamless multi-cloud support.

2. Terraform

License: Business Source License (BSL) 1.1 (Not Open Source)
Best For: Teams with existing Terraform expertise and established workflows

Terraform uses HashiCorp Configuration Language (HCL) to define infrastructure across multiple cloud providers. However, its 2023 licensing change to BSL (no longer open source) and inherent limitations with domain-specific languages create challenges for teams requiring advanced software engineering practices.

Key Features:

Extensive provider ecosystem: Largest collection of community-maintained providers covering virtually every cloud service
Established workflows: Mature plan-and-apply process with extensive tooling and CI/CD integrations
Community and resources: Vast ecosystem of modules, extensive documentation, training materials, and community support
Enterprise adoption: Widely adopted in enterprises with established processes and expertise
Module ecosystem: Rich collection of reusable Terraform modules for common patterns

When Terraform Works Well:

Teams with existing HCL expertise and Terraform investments
Infrastructure patterns that fit well within HCL’s declarative model
Teams comfortable with template-based approaches

Considerations for Complex Scenarios:

Language constraints: HCL’s domain-specific nature can require workarounds for complex logic compared to general-purpose programming languages
Testing approach: Primarily supports integration testing; teams needing comprehensive unit testing may require additional tooling
IDE experience: While improving, HCL tooling provides less comprehensive support than mature programming language ecosystems
State management: Requires manual backend configuration and locking setup for team collaboration

Code Example:

data "aws_availability_zones" "available" {}

resource "aws_vpc" "main" {
 cidr_block = "10.0.0.0/16"

 tags = {
 Name = "main-vpc"
 }
}

resource "aws_subnet" "public" {
 count = 2
 vpc_id = aws_vpc.main.id
 cidr_block = "10.0.${count.index + 1}.0/24"
 availability_zone = data.aws_availability_zones.available.names[count.index]

 tags = {
 Name = "public-subnet-${count.index + 1}"
 }
}

3. AWS Cloud Development Kit (CDK)

License: Apache 2.0
Best For: AWS-focused teams who prefer programming languages over templates

AWS CDK allows you to define AWS infrastructure using familiar programming languages, synthesizing CloudFormation templates for deployment while providing higher-level abstractions. CDK addresses many limitations of traditional template-based approaches by enabling general-purpose programming languages.

Key Features:

General-purpose programming languages: TypeScript, Python, Java, C#, JavaScript support with full IDE integration (Go available in Developer Preview)
AWS-optimized constructs: High-level components encapsulating AWS best practices
Type safety: Compile-time checking and IntelliSense support
CloudFormation reliability: Built on AWS’s proven deployment engine

Notable Limitations:

AWS-only ecosystem: Locked into single cloud provider, limiting multi-cloud strategies
CloudFormation constraints: Inherits template size limits and deployment restrictions
Vendor lock-in: Deep AWS integration makes migration to other clouds challenging
Limited cross-cloud consistency: Teams need different tools and approaches for multi-cloud deployments

CDK represents a significant improvement over CloudFormation templates but constrains organizations to AWS-only infrastructure strategies.

Code Example:

import * as cdk from 'aws-cdk-lib';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as ecs from 'aws-cdk-lib/aws-ecs';
import { Construct } from 'constructs';

export class MyStack extends cdk.Stack {
 constructor(scope: Construct, id: string, props?: cdk.StackProps) {
 super(scope, id, props);

 const vpc = new ec2.Vpc(this, 'MyVpc', {
 maxAzs: 2
 });

 const cluster = new ecs.Cluster(this, 'MyCluster', {
 vpc: vpc
 });

 const taskDefinition = new ecs.FargateTaskDefinition(this, 'TaskDef', {
 memoryLimitMiB: 512,
 cpu: 256
 });

 taskDefinition.addContainer('web', {
 image: ecs.ContainerImage.fromRegistry('nginx:latest'),
 portMappings: [{ containerPort: 80 }]
 });

 new ecs.FargateService(this, 'MyService', {
 cluster: cluster,
 taskDefinition: taskDefinition
 });
 }
}

4. AWS CloudFormation

License: Proprietary (AWS Service)
Best For: AWS-only deployments requiring deep service integration

AWS CloudFormation provides the foundation for infrastructure as code on AWS, offering native integration with all AWS services and deep platform-specific features.

Pulumi Integration: Pulumi provides native AWS providers that offer the same comprehensive AWS service coverage as CloudFormation, with the added benefit of using general-purpose programming languages. You can also import existing CloudFormation stacks into Pulumi for gradual migration or hybrid management approaches.

Key Features:

AWS-native: First-party support for all AWS services
JSON/YAML templates: Declarative resource definitions
Stack management: Organized resource grouping and lifecycle management
Change sets: Preview infrastructure changes before deployment
Service integration: Deep integration with other AWS services

Code Example:

AWSTemplateFormatVersion: '2010-09-09'
Resources:
 MyVPC:
 Type: AWS::EC2::VPC
 Properties:
 CidrBlock: 10.0.0.0/16
 EnableDnsHostnames: true
 Tags:
 - Key: Name
 Value: MyVPC

 MySubnet:
 Type: AWS::EC2::Subnet
 Properties:
 VpcId: !Ref MyVPC
 CidrBlock: 10.0.1.0/24
 AvailabilityZone: !Select [0, !GetAZs '']

5. Azure Resource Manager (ARM)

License: Proprietary (Microsoft Service)
Best For: Azure-native deployments requiring comprehensive platform integration

Azure Resource Manager provides the foundational infrastructure as code solution for Microsoft Azure, offering complete support for Azure services through JSON-based ARM templates. As Azure’s native IaC solution, ARM templates provide the most comprehensive coverage of Azure services and features.

Pulumi Integration: Pulumi’s native Azure providers offer equivalent comprehensive Azure service coverage with general-purpose programming languages. ARM templates can be imported into Pulumi, and you can reference ARM deployments from Pulumi programs for hybrid scenarios.

Key Features:

Azure-native: Complete Azure service coverage with first-party support
JSON templates: Declarative resource definitions in JSON format
Resource groups: Logical organization of related resources
Deployment modes: Complete or incremental deployment options
Comprehensive integration: Deep integration with Azure services and features

Code Example:

{
 "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
 "contentVersion": "1.0.0.0",
 "resources": [
 {
 "type": "Microsoft.Storage/storageAccounts",
 "apiVersion": "2023-01-01",
 "name": "mystorageaccount",
 "location": "[resourceGroup().location]",
 "sku": {
 "name": "Standard_LRS"
 },
 "kind": "StorageV2"
 }
 ]
}

6. Azure Bicep

License: MIT
Best For: Azure deployments requiring improved readability and developer experience

Azure Bicep is a domain-specific language (DSL) that simplifies Azure Resource Manager template authoring. Bicep files compile transparently to ARM templates, providing all the capabilities of ARM with significantly improved syntax and developer experience.

Key Features:

Simplified syntax: Clean, readable syntax compared to JSON ARM templates
ARM compilation: Compiles to ARM templates for deployment, ensuring full compatibility
Type safety: Strong typing and IntelliSense support in development environments
Modular design: Support for modules and code reuse across projects
Azure-native: Complete Azure service coverage through ARM template compilation

Code Example:

@description('Storage Account type')
@allowed([
 'Standard_LRS'
 'Standard_GRS'
 'Standard_ZRS'
 'Premium_LRS'
])
param storageAccountType string = 'Standard_LRS'

@description('Location for the storage account.')
param location string = resourceGroup().location

resource storageAccount 'Microsoft.Storage/storageAccounts@2023-01-01' = {
 name: 'mystorageaccount'
 location: location
 sku: {
 name: storageAccountType
 }
 kind: 'StorageV2'
 properties: {}
}

7. Google Cloud Infrastructure Manager

License: Proprietary (Google Service)
Best For: Google Cloud Platform deployments using Terraform

Google Cloud Infrastructure Manager automates the deployment and management of Google Cloud infrastructure resources using Terraform configurations, representing Google’s modern approach to infrastructure as code. Infrastructure Manager replaces Google Cloud Deployment Manager, which will reach end of support on December 31, 2025.

Key Features:

Terraform-based: Uses standard Terraform configurations declaratively
Automated workflows: Handles Terraform init, validate, and apply operations
Version control integration: Supports Git repositories and Cloud Storage
Deployment tracking: Comprehensive metadata storage and logging
Multiple Terraform versions: Flexibility in Terraform version selection
Cloud Build integration: Leverages Google Cloud Build for execution environment
Migration path: Provides upgrade path from legacy Cloud Deployment Manager

Code Example:

# main.tf - Terraform configuration for Infrastructure Manager
resource "google_compute_instance" "vm_instance" {
 name = "my-vm"
 machine_type = "e2-medium"
 zone = "us-central1-a"

 boot_disk {
 initialize_params {
 image = "debian-cloud/debian-11"
 }
 }

 network_interface {
 network = "default"
 access_config {
 // Ephemeral public IP
 }
 }

 metadata = {
 startup-script = "echo Hello from Infrastructure Manager!"
 }
}

output "instance_ip" {
 value = google_compute_instance.vm_instance.network_interface[0].access_config[0].nat_ip
}

8. Kubernetes YAML

License: Apache 2.0
Best For: Teams managing container-native applications and cloud-native infrastructure

Kubernetes YAML manifests represent one of the most widely adopted forms of infrastructure as code, enabling teams to define, version, and manage containerized applications and their supporting infrastructure through declarative configuration files.

Key Features:

Declarative configuration: Define desired state through human-readable YAML files
Native Kubernetes integration: Direct integration with Kubernetes API without additional tools
GitOps compatibility: Version control YAML files for automated deployment workflows
Resource relationships: Define dependencies and relationships between Kubernetes resources
Extensive ecosystem: Rich ecosystem of operators and custom resources extending functionality

Code Example:

apiVersion: apps/v1
kind: Deployment
metadata:
 name: web-app
 labels:
 app: web-app
spec:
 replicas: 3
 selector:
 matchLabels:
 app: web-app
 template:
 metadata:
 labels:
 app: web-app
 spec:
 containers:
 - name: web-app
 image: nginx:1.21
 ports:
 - containerPort: 80
 resources:
 requests:
 memory: "64Mi"
 cpu: "250m"
 limits:
 memory: "128Mi"
 cpu: "500m"
---
apiVersion: v1
kind: Service
metadata:
 name: web-app-service
spec:
 selector:
 app: web-app
 ports:
 - port: 80
 targetPort: 80
 type: LoadBalancer

9. Crossplane

License: Apache 2.0
Best For: Kubernetes-first organizations managing multi-cloud infrastructure

Crossplane is a Cloud-Native Framework for Platform Engineering that extends Kubernetes to help organizations build custom infrastructure management platforms, allowing teams to provision and manage cloud resources using Kubernetes APIs and patterns.

Pulumi Integration: Pulumi offers the Pulumi Kubernetes Operator (PKO) that provides similar Kubernetes-native infrastructure management capabilities, plus support for YAML-based definitions. Teams can also use Pulumi programs to provision the underlying infrastructure that Crossplane manages, creating layered infrastructure management approaches.

Key Features:

Kubernetes-native: Uses CRDs and standard Kubernetes patterns
Composite resources: Create higher-level infrastructure abstractions
GitOps compatibility: Seamless integration with GitOps workflows
Multi-cloud support: Provision resources across cloud providers
Policy integration: Leverage Kubernetes RBAC and admission controllers

Code Example:

apiVersion: ec2.aws.crossplane.io/v1alpha1
kind: VPC
metadata:
 name: sample-vpc
spec:
 cidrBlock: 10.0.0.0/16
 region: us-east-1
 tags:
 Name: sample-vpc
 providerConfigRef:
 name: aws-provider-config

10. OpenTofu

License: Mozilla Public License 2.0
Best For: Teams seeking an open-source Terraform alternative with community governance

OpenTofu emerged as a fork of Terraform v1.5.x following HashiCorp’s license change, maintained by the Linux Foundation. It provides full compatibility with Terraform while ensuring long-term open-source availability under MPL 2.0 licensing.

Key Features:

True open source: MPL 2.0 license with community governance via Linux Foundation ensuring long-term accessibility
Terraform compatibility: Drop-in replacement maintaining existing workflows, modules, and provider ecosystem
Community-driven development: Transparent roadmap, open contribution process, and vendor-neutral governance
License certainty: Removes concerns about future licensing restrictions for commercial use

When OpenTofu Makes Sense:

Organizations requiring guaranteed open-source licensing for compliance or philosophical reasons
Teams with significant Terraform investments wanting to avoid vendor lock-in
Environments where community governance and transparency are priorities
Projects needing long-term license stability

Architectural Considerations: As a Terraform fork, OpenTofu inherits the same architectural approach, which means teams evaluating it should consider whether HCL-based infrastructure definition meets their long-term needs for testing, IDE integration, and developer productivity, or whether a programming language-based approach might better serve complex scenarios.

Code Example:

resource "aws_instance" "web" {
 ami = "ami-0c55b159cbfafe1d0"
 instance_type = "t3.micro"

 tags = {
 Name = "HelloWorld"
 }
}

output "instance_ip" {
 value = aws_instance.web.public_ip
}

Ansible

License: GPL v3
Best For: Configuration management with some infrastructure provisioning capabilities

Ansible is primarily a configuration management tool, not a pure Infrastructure as Code tool. While Ansible can provision some cloud resources, its core strength lies in configuring and managing software on existing systems rather than comprehensive infrastructure provisioning.

Ansible provides configuration management and limited infrastructure provisioning through its agentless architecture and simple YAML-based playbooks.

Pulumi Integration: Rather than competing with Ansible, Pulumi complements it perfectly. Use Pulumi for infrastructure provisioning and Ansible for configuration management. Pulumi’s Command provider can execute Ansible playbooks as part of your infrastructure deployment, and many Pulumi customers use both tools together for comprehensive infrastructure automation. See example: Deploy WordPress to AWS using Pulumi and Ansible.

Key Features:

Agentless architecture: No software installation required on target systems
YAML playbooks: Human-readable automation definitions
Idempotent operations: Safe to run multiple times
Large module library: Extensive built-in functionality for various systems
Push-based execution: Centralized control and execution

Code Example:

---
- name: Provision AWS infrastructure
 hosts: localhost
 tasks:
 - name: Create VPC
 amazon.aws.ec2_vpc_net:
 name: ansible-vpc
 cidr_block: 10.0.0.0/16
 region: us-east-1
 tags:
 Environment: production
 register: vpc

 - name: Create subnet
 amazon.aws.ec2_vpc_subnet:
 vpc_id: "{{ vpc.vpc.id }}"
 cidr: 10.0.1.0/24
 region: us-east-1
 tags:
 Name: ansible-subnet

Chef

License: Apache 2.0
Best For: Complex configuration management scenarios requiring programmable logic

Chef is a configuration management tool, not an Infrastructure as Code tool. Chef focuses on configuring and maintaining software, services, and system settings on existing infrastructure rather than provisioning cloud resources.

Chef provides configuration management and system automation using Ruby-based recipes and cookbooks, offering powerful programmability for complex configuration scenarios.

Key Features:

Ruby DSL: Full programming language for configuration logic
Agent-based architecture: Continuous compliance and drift detection
Cookbook ecosystem: Reusable configuration patterns and community recipes
Test Kitchen: Infrastructure testing and validation framework
Enterprise features: Advanced reporting and compliance capabilities

Code Example:

# cookbook/recipes/default.rb
package 'nginx' do
 action :install
end

service 'nginx' do
 action [:enable, :start]
end

template '/etc/nginx/sites-available/default' do
 source 'default.erb'
 owner 'root'
 group 'root'
 mode '0644'
 notifies :restart, 'service[nginx]'
end

Puppet

License: Apache 2.0
Best For: Enterprise environments requiring strong governance and compliance

Puppet is a configuration management tool, not an Infrastructure as Code tool. Puppet specializes in maintaining desired configuration state on existing systems and ensuring compliance, rather than provisioning cloud infrastructure.

Puppet offers enterprise-grade configuration management with a focus on compliance, governance, and declarative system state management.

Key Features:

Declarative language: Puppet DSL for describing desired system state
Compliance reporting: Built-in governance and audit capabilities
Forge marketplace: Community modules and enterprise content
Enterprise support: Professional services and enterprise features
Continuous enforcement: Ongoing configuration compliance monitoring

Code Example:

# manifests/webserver.pp
class webserver {
 package { 'nginx':
 ensure => installed,
 }

 service { 'nginx':
 ensure => running,
 enable => true,
 require => Package['nginx'],
 }

 file { '/var/www/html/index.html':
 ensure => file,
 content => '<h1>Hello from Puppet!</h1>',
 owner => 'www-data',
 group => 'www-data',
 mode => '0644',
 }
}

Salt

License: Apache 2.0
Best For: Python-oriented teams requiring high-performance configuration management

Salt is primarily a configuration management and remote execution tool, not a pure Infrastructure as Code tool. While Salt can manage some infrastructure components, its primary focus is on configuring systems and executing commands across large infrastructures.

Salt provides fast, scalable configuration management and remote execution using Python, designed for high-performance system automation at scale.

Key Features:

Python-based: Leverage the Python ecosystem and libraries
High performance: Fast execution across large infrastructures
Event-driven automation: Reactive automation and orchestration
Pillar data system: Secure, hierarchical data management
Flexible communication: Support for various communication patterns

Code Example:

# /srv/salt/webserver.sls
nginx:
 pkg.installed: []
 service.running:
 - enable: True
 - require:
 - pkg: nginx

/var/www/html/index.html:
 file.managed:
 - source: salt://files/index.html
 - user: www-data
 - group: www-data
 - mode: 644

Infrastructure as Code Tools

Kubernetes - Container Orchestration Platform

License: Apache 2.0
Best For: Container-native infrastructure and application management

While primarily a container orchestration platform, Kubernetes itself serves as an infrastructure as code tool through its declarative YAML manifests and API-driven resource management.

Key Features:

Declarative configuration: YAML-based resource definitions
API-driven: RESTful API for all resource operations
Self-healing: Automatic recovery and reconciliation
Extensible: Custom resources and controllers
GitOps compatible: Works seamlessly with Git-based workflows

Code Example:

apiVersion: apps/v1
kind: Deployment
metadata:
 name: nginx-deployment
spec:
 replicas: 3
 selector:
 matchLabels:
 app: nginx
 template:
 metadata:
 labels:
 app: nginx
 spec:
 containers:
 - name: nginx
 image: nginx:1.21
 ports:
 - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
 name: nginx-service
spec:
 selector:
 app: nginx
 ports:
 - port: 80
 targetPort: 80
 type: LoadBalancer

Infrastructure as Code Security and Compliance Tools

While the tools above focus on provisioning and managing infrastructure, a complete IaC ecosystem includes security scanning and compliance tools. These tools complement your primary IaC tool by providing security analysis, policy enforcement, and compliance checking:

Security Scanning Tools

Snyk - License: Proprietary
Leading developer security platform that includes comprehensive infrastructure as code scanning alongside container and application security. Provides real-time vulnerability detection, compliance checking, and automated remediation guidance. Integrates with popular development tools and CI/CD pipelines with extensive enterprise adoption.

Wiz - License: Proprietary
Comprehensive cloud security platform that includes infrastructure as code scanning capabilities. Provides vulnerability management, compliance monitoring, and security posture assessment across cloud environments. Offers integration with development workflows and supports multiple IaC formats with strong enterprise presence.

Checkov - License: Apache 2.0
Popular open-source static analysis tool for infrastructure as code that scans cloud infrastructure configurations for security and compliance issues. Supports Terraform, CloudFormation, Kubernetes, Helm, ARM templates, and more. Integrates with CI/CD pipelines and provides over 1000+ built-in policies covering CIS benchmarks, PCI DSS, and GDPR compliance.

Linting and Validation Tools

TFLint - License: MPL 2.0
Terraform linter focused on possible errors, best practices, and security issues in Terraform configurations. Provides pluggable rule sets for cloud providers (AWS, Azure, GCP) and helps enforce coding standards, detect deprecated syntax, and prevent common configuration errors.

These security tools integrate into CI/CD pipelines alongside your chosen IaC tool to provide comprehensive security coverage throughout the infrastructure lifecycle.

Pulumi ESC

License: Apache 2.0 (Open Source) / Proprietary (SaaS)
Best For: Teams needing centralized configuration and secrets management across environments and tools

Pulumi ESC (Environments, Secrets, and Configuration) is a comprehensive platform for managing configuration data, secrets, and environment variables across your entire infrastructure and application stack. ESC provides a single source of truth for configuration that works with any infrastructure tool, not just Pulumi.

Key Features:

Universal configuration management: Works with any infrastructure tool, CI/CD system, or application
Hierarchical environments: Compose configuration from multiple sources with inheritance and overrides
Dynamic secrets: Integration with cloud providers for short-lived credentials and just-in-time access
Policy-based access control: Fine-grained permissions and audit logging for configuration access
Multiple consumption methods: CLI, REST API, SDK integration, and direct cloud provider integration
GitOps workflows: Version control integration with pull request-based configuration management

Code Example:

# Production environment configuration
values:
 app:
 name: myapp
 version: ${fn.fromJSON(aws.ecr.getAuthorizationToken).password}
 database:
 connectionString: ${pulumi.database.connectionString}
imports:
 - shared/common
 - aws/production

Pulumi Insights

License: Proprietary (SaaS)
Best For: Organizations needing comprehensive cloud resource visibility, search, and compliance monitoring

Pulumi Insights provides cloud resource search, analytics, and compliance capabilities across your entire multi-cloud infrastructure, regardless of how resources were provisioned. It offers a unified view of your cloud resources with powerful search, cost analysis, and policy enforcement.

Key Features:

Universal cloud inventory: Discover and catalog resources across AWS, Azure, Google Cloud, and Kubernetes
Advanced search and analytics: Query resources by tags, properties, relationships, and compliance status
Cost optimization insights: Resource cost analysis, trends, and optimization recommendations
Compliance monitoring: Continuous policy evaluation and drift detection across all cloud resources
Resource relationships: Understand dependencies and relationships between cloud resources
Integration with any IaC tool: Works with resources created by Pulumi, Terraform, CloudFormation, or manually

Use Cases:

Cloud governance: Ensure compliance with organizational policies and industry standards
Cost management: Identify unused resources and optimization opportunities
Security auditing: Track resource configurations and detect security misconfigurations
Multi-cloud visibility: Single pane of glass for resources across different cloud providers

Infrastructure Automation and Management Platforms

While the above tools focus on defining and provisioning infrastructure, several platforms provide automation, orchestration, and management capabilities that work with Infrastructure as Code tools. Important: These are not IaC tools themselves, but rather automation platforms that rely on underlying IaC tools.

IaC Automation Platforms

Pulumi Cloud Best For: Teams using Pulumi IaC who need enterprise-grade collaboration, security, and governance

Pulumi Cloud is the smartest and easiest way to automate, secure, and manage everything you run in the cloud. It serves as the managed service companion to Pulumi’s open-source infrastructure as code tool, providing enterprise capabilities that extend beyond basic IaC functionality.

The relationship between Pulumi IaC and Pulumi Cloud follows a Git/GitHub model: Pulumi IaC is like Git (fully open source and functional on its own), while Pulumi Cloud is like GitHub (a managed service that makes the open-source tool much easier to use securely at scale with teams).

Key Features:

Managed state backend: Secure, scalable state management with transactional protocols
Team collaboration: Full audit logs, RBAC, and identity provider integration
Centralized secrets management: Pulumi ESC for configuration and secrets across environments
Cloud inventory and insights: Pulumi Insights for search, compliance, and drift detection
Remote deployments: CI/CD integrations and automated workflows
Policy enforcement: Built-in security and compliance policy as code
Multi-product platform: Integrated IaC, secrets management, and cloud insights

Pulumi IaC works with or without Pulumi Cloud - you can use DIY backends (S3, Azure Blob, etc.) or the managed service. However, Pulumi Cloud becomes the default experience when you install the Pulumi CLI, providing instant collaboration capabilities without the overhead of building and maintaining your own infrastructure management platform.

HashiCorp Cloud Platform (HCP) - License: Proprietary (SaaS)
Best For: Organizations standardizing on HashiCorp tools across infrastructure and security lifecycle management

HashiCorp Cloud Platform (HCP) is an enterprise-grade SaaS platform that provides unified lifecycle management for infrastructure and security operations. HCP Terraform (formerly Terraform Cloud) serves as the managed service for Terraform/OpenTofu workflows, while the broader platform integrates multiple HashiCorp tools.

Key Features:

HCP Terraform management: Workspace organization, remote execution, and state management for Terraform
Infrastructure lifecycle automation: Continuous validation, drift detection, and module lifecycle management
Private VCS access: Secure integration with private version control repositories
Policy enforcement: Sentinel policy-as-code framework for compliance and governance
Integrated security services: HCP Vault for secrets management, HCP Consul for service networking
Enterprise access controls: RBAC, SAML SSO integration, and fine-grained permissions
Module and image management: HCP Packer for automated machine/container image creation

HCP focuses specifically on HashiCorp tool integration and provides a comprehensive platform for organizations that have standardized on the HashiCorp ecosystem for infrastructure and security operations.

Spacelift - Spacelift is not an Infrastructure as Code tool—it’s an automation and workflow platform that relies on other IaC tools like Terraform, OpenTofu, Pulumi, CloudFormation, and Kubernetes. Spacelift provides CI/CD pipelines, policy enforcement, and collaboration features for teams using these underlying IaC tools.

Env0 - Env0 is not an Infrastructure as Code tool—it’s an automation platform that provides workflow management, governance, and collaboration features for existing IaC tools like Terraform, OpenTofu, and Terragrunt. It adds CI/CD pipelines, cost management, and policy enforcement on top of these tools.

Atlantis - An open-source tool that provides GitOps-style workflows for Terraform and OpenTofu by automatically running terraform plan and apply operations via pull request automation.

Development Environment Tools

Vagrant - Vagrant is not an Infrastructure as Code tool—it’s a development environment management tool that creates and configures lightweight, reproducible virtual development environments. While it can provision VMs, its focus is on local development environments rather than cloud infrastructure provisioning.

Docker Compose - While not an IaC tool, Docker Compose defines multi-container applications and can be used alongside IaC tools for application deployment after infrastructure provisioning.

These platforms and tools serve important roles in the infrastructure automation ecosystem but should not be confused with Infrastructure as Code tools themselves. They enhance and orchestrate the work of actual IaC tools rather than replacing them.

The Future of Infrastructure as Code

The infrastructure as code landscape is rapidly evolving toward software engineering maturity, with several transformative trends reshaping how organizations approach infrastructure:

Software Engineering Convergence: The most significant trend is the convergence of infrastructure and software engineering practices. Organizations are moving away from limited DSLs toward full programming languages that enable testing, debugging, refactoring, and other software engineering best practices. This shift enables infrastructure teams to leverage the same tools, skills, and methodologies that have proven successful in application development.

Real-Time Cloud Integration: Native cloud provider SDK integration is becoming the standard, replacing community-maintained providers that lag behind new cloud features. Organizations expect same-day access to new cloud services without waiting weeks or months for provider updates.

Comprehensive Testing Paradigms: Infrastructure testing is evolving beyond basic integration tests to include unit testing, property-based testing, and continuous validation. Teams are applying test-driven development principles to infrastructure, catching issues before deployment rather than discovering them in production.

Internal Developer Platform Evolution: Organizations are building sophisticated Internal Developer Platforms that provide self-service infrastructure capabilities while maintaining governance and compliance. These platforms leverage infrastructure as code tools to create standardized, reusable components that accelerate development velocity.

AI-Enhanced Development: Integration of AI tools to help generate, optimize, and troubleshoot infrastructure code, with particular strength in environments that use familiar programming languages where AI assistance is most mature. Emerging technologies like Model Context Protocol (MCP) and AI prompt templates are beginning to enable more sophisticated AI-infrastructure interactions.

These trends favor tools that embrace software engineering principles from the ground up, rather than attempting to retrofit programming capabilities onto template-based or DSL-limited approaches.

Migration and Adoption Strategies

Organizations don’t need to choose between maintaining existing infrastructure and adopting modern IaC approaches. Proven migration strategies enable gradual adoption while preserving operational stability.

Incremental Adoption Approaches

Team-by-Team Migration: Start with new projects or specific teams, allowing gradual skill development and process refinement. Teams can maintain existing infrastructure while building new capabilities with modern tools.

Project-by-Project Transition: Migrate individual applications or services incrementally, enabling teams to learn and optimize approaches before expanding scope.

Hybrid Operations: Use state integration and import tools to reference existing infrastructure while building new components with modern IaC tools.

Proven Migration Timelines

Real-world migrations demonstrate that adoption can be remarkably fast with proper tooling:

Atlassian Bitbucket: Complete Terraform to Pulumi migration in 2 days using automated conversion tools. This rapid migration was enabled by existing Python expertise on the team, well-structured Terraform code, and straightforward infrastructure patterns that converted cleanly.
Enterprise migrations: Typical team migrations complete in weeks, not months, depending on infrastructure complexity and team preparation
Learning curve: Teams with programming experience adapt to language-based IaC approaches within days, while those new to programming may require additional training time

Migration Tools and Resources

Automated Conversion: Tools like tf2pulumi and pulumi convert automatically translate existing infrastructure definitions, preserving logic and structure while upgrading to modern approaches.

State Integration: Import existing cloud resources and reference Terraform state during transition periods, enabling zero-downtime migrations.

Training and Support: Comprehensive documentation, tutorials, and community resources accelerate team onboarding and reduce migration risks.

Best Practices for Successful Adoption

Start with pilot projects to build confidence and establish patterns
Leverage automated migration tools to reduce manual conversion effort
Maintain state integration during transition periods for operational continuity
Invest in team training to maximize productivity gains from modern approaches
Establish testing practices early to realize reliability and quality benefits

The key insight: migration to modern IaC approaches is a technical upgrade, not a complete rebuild. Organizations can preserve existing investments while gaining access to superior tooling and practices.

Frequently Asked Questions

Which IaC tool should I choose for AWS?

For teams wanting programming languages: Pulumi IaC and AWS CDK both offer excellent developer experiences with general-purpose programming languages. Pulumi provides multi-cloud flexibility, while CDK offers deep AWS-native integration.

For AWS-only deployments: CloudFormation provides the deepest native AWS service support and direct integration with AWS services.

For Terraform ecosystem users: OpenTofu provides open-source licensing with Terraform compatibility, while Terraform offers extensive community resources and established workflows.

Is Terraform still worth learning in 2025?

Terraform remains a valuable skill and viable choice for many scenarios, though teams should consider their specific needs and long-term goals:

When Terraform Makes Sense:

Teams building on existing Terraform infrastructure and expertise
Organizations with successful Terraform workflows and established processes
Projects that benefit from Terraform’s extensive module ecosystem
Teams comfortable with HCL and template-based approaches

When to Consider Alternatives:

Teams wanting comprehensive testing capabilities (unit, property, integration testing)
Organizations requiring full IDE support with debugging and refactoring
Projects needing complex programmatic logic or extensive code reuse
Teams with strong programming backgrounds who prefer familiar languages

Key Considerations:

Licensing: Terraform’s Business Source License may create restrictions for some commercial scenarios
Ecosystem: Terraform has the largest provider ecosystem and community resources
Learning investment: Consider whether learning HCL aligns with your team’s skill development goals
Migration options: Tools exist for migrating between different IaC platforms when needs change

The choice often depends on team background, infrastructure complexity, and development workflow preferences rather than one approach being universally superior.

How do I get started with infrastructure as code?

Choose Based on Your Team Profile:

Programming-oriented teams: Consider Pulumi IaC for familiar languages with full IDE support, testing frameworks, and multi-cloud flexibility
Template-oriented teams: OpenTofu or Terraform offer extensive community resources, established workflows, and broad ecosystem support
Cloud-specific teams: Native tools (AWS CDK/CloudFormation, Azure ARM/Bicep, Google Cloud Infrastructure Manager) provide deep platform integration
Kubernetes-focused teams: Consider Crossplane or Pulumi Kubernetes Operator for container-native approaches

Getting Started Steps:

Start small: Begin with simple projects like deploying a single application or basic infrastructure
Use examples: Leverage existing templates, tutorials, and community examples to accelerate learning
Plan for growth: Consider how your tool choice will scale with team size and infrastructure complexity
Experiment: Most tools offer free tiers or trials - try multiple approaches with simple projects to find what fits your team best

What’s the difference between configuration management and infrastructure provisioning?

Infrastructure provisioning (Pulumi IaC, Terraform, CloudFormation) creates and manages cloud resources like VMs, networks, and databases
Configuration management (Ansible, Chef, Puppet) configures and maintains software on existing systems
Many modern tools do both, with Pulumi IaC and Ansible offering comprehensive capabilities across both domains

Can I use multiple IaC tools together?

Absolutely! Many organizations use complementary tools for different aspects of infrastructure management:

Pulumi + Configuration Management:

Pulumi + Ansible: Infrastructure provisioning + server configuration (example)
Pulumi + Chef/Puppet: Cloud resources + complex configuration management

Pulumi + Native Cloud Tools:

Reference existing CloudFormation/ARM deployments from Pulumi programs
Import CloudFormation/ARM resources into Pulumi for gradual migration
Use Pulumi alongside CDK for different parts of AWS infrastructure

Pulumi + Kubernetes:

Pulumi Kubernetes Operator (PKO) for GitOps workflows
Pulumi + Crossplane for layered infrastructure management
Pulumi + security scanners like Checkov or Terrascan for compliance

Terraform + Pulumi Coexistence:

Use Terraform for base infrastructure and Pulumi for Internal Developer Platforms (IDPs)
Reference existing Terraform state from Pulumi programs during gradual migration
Manage different infrastructure layers with different tools based on team expertise

Pulumi IaC is designed for heterogeneous environments where multiple tools may be in use. For example, you can manage existing Terraform or CloudFormation resources with Pulumi, either using both tools in tandem or for temporary management while migrating to native Pulumi IaC code. Other Pulumi tools, like Pulumi IDP, also enable you to manage IaC self-service workflows like other tools on this list. See what’s possible when migrating to Pulumi.

Which tool has the best learning resources?

Pulumi: Excellent documentation with examples in multiple languages, comprehensive tutorials
Terraform/OpenTofu: Extensive community content, many courses and books available
AWS CDK: Outstanding official documentation with workshops and examples
Kubernetes: Vast ecosystem but steeper learning curve

How important is open-source licensing for IaC tools?

Very important for long-term strategy:

Apache 2.0 (Pulumi, CDK) offers maximum flexibility
MPL 2.0 (OpenTofu) ensures open-source availability
BSL (Terraform) restricts commercial competitors
Proprietary (CloudFormation, ARM) ties you to specific vendors

Choose open-source tools like Pulumi or OpenTofu to avoid vendor lock-in.

What about security and compliance?

All major IaC tools support security best practices:

Built-in policy engines: Pulumi CrossGuard, AWS Config, Azure Policy
Third-party scanners: Checkov, KICS, Terrascan work with all tools
Compliance frameworks: Most tools support SOC 2, PCI DSS, CIS benchmarks
Secret management: Integration with Vault, AWS Secrets Manager, Azure Key Vault

Pulumi offers the most comprehensive built-in security features with CrossGuard policy as code.

Can I migrate from one IaC tool to another?

Yes, and many organizations are successfully migrating to overcome limitations in their current tools. Migration tools and proven approaches exist:

Proven Migration Success Stories:

Atlassian Bitbucket: Completed migration from Terraform to Pulumi’s Python-based IaC in just 2 days using migration tools
Starburst: Achieved 112x faster deployments after incrementally migrating from Terraform
Multiple organizations: Report 80-90% deployment time reductions after moving to programming language-based approaches

Available Migration Tools:

Pulumi: Offers pulumi convert for importing from Terraform, ARM, and CloudFormation with state integration
Terraformer: Can import existing cloud resources into Terraform/OpenTofu
CDK Migrate: Helps move from CloudFormation to CDK
Manual migration: Always possible by recreating resources in the new tool

Migration Strategies:

Incremental adoption: Migrate team-by-team or project-by-project without disruption
State integration: Reference existing infrastructure during transition periods
Zero downtime: Tools like Pulumi enable seamless migration without service interruption

The key is choosing tools that provide comprehensive migration support and incremental adoption paths rather than requiring “rip and replace” approaches.

Conclusion: The Evolution of Infrastructure as Code

The infrastructure as code landscape in 2025 reflects a maturing field where different approaches serve different organizational needs and team preferences. The evolution from manual processes to automated infrastructure has branched into multiple viable paths, each with distinct advantages.

The Spectrum of Approaches:

Template-based tools like Terraform and OpenTofu continue to serve teams effectively, particularly those comfortable with declarative configuration and established HCL workflows. Cloud-native solutions provide deep platform integration for single-cloud strategies. Programming language-based approaches offer familiar development experiences for teams seeking comprehensive testing, IDE integration, and code reusability.

Evidence of Success Across Approaches:

Organizations achieve significant improvements regardless of their chosen path—the key is selecting tools that align with team expertise and infrastructure requirements. Understanding what DevOps is and platform engineering concepts helps inform these decisions. Success stories span the entire ecosystem, from Terraform’s widespread enterprise adoption to programming language approaches enabling dramatic productivity gains at companies like Unity, Snowflake, and Starburst.

Choosing Your Path:

The most successful organizations focus on key decision criteria:

Team background: Match tools to existing skills and preferred development approaches
Infrastructure complexity: Consider testing, abstraction, and maintainability needs
Organizational constraints: Factor in licensing, governance, and compliance requirements
Growth trajectory: Plan for how needs may evolve with scale and team expansion
Integration requirements: Ensure compatibility with existing workflows and toolchains

The Future of Infrastructure:

The industry continues evolving toward treating infrastructure as software, but this transformation takes many forms. Organizations exploring serverless architectures and container strategies particularly benefit from programmable infrastructure approaches. Whether through enhanced DSLs, visual design tools, programming languages, or hybrid approaches, the goal remains consistent: enabling teams to manage infrastructure with the same reliability, collaboration, and velocity they expect from modern software development.

For teams ready to embrace programming language-based infrastructure as code, Get started with Pulumi to experience how familiar languages and software engineering practices can transform infrastructure management with comprehensive testing, powerful abstractions, and seamless multi-cloud support.

AWS RDS - Blue/Green Deployments for Low-Downtime Updates

Elisabeth Lichtie — Mon, 23 Jun 2025 00:00:00 +0000

AWS RDS supports blue/green deployments to support database maintenance. In a blue/green deployment, you have one production (blue) and one staging (green) database. You can safely make changes to the green instance without affecting production and promote it to be the main instance. When you enable blue/green updates, Pulumi will temporarily set up a blue/green deployment for the duration of the update to minimize downtime.

There are many benefits to using blue/green deployments for updates, especially for critical databases that need to be kept available and safe from untested changes.

How it works

By default, updates to a database will be applied in-place, meaning there may be service interruptions while a database is modified especially when making long-running updates like changing the engine version, parameter groups, or storage settings.

blueGreenUpdate can be enabled for Low-Downtime Updates. When this setting is on and an update is performed, the program starts the update by creating a blue/green deployment to maintain availability. Then, it performs the requested update on the green instance, performs a guardrail check, promotes the green instance, and finally deletes the old instance and removes the blue/green deployment. This leaves just the updated database running with minimal downtime during the update while minimizing the downtime.

Though this strategy typically increases the amount of time that updates need to run, it minimizes downtime for the application by keeping a current version of the database available during the update process.

How to use

Please see the full example code here: Blue/Green Deployment Updates on RDS. To use blue/green deployments for your database updates, start by creating a database and corresponding parameter group.

const dbVersion = "14.18";

const parameterGroup = new aws.rds.ParameterGroup(
 "parameter-group",
 {
 family: `postgres${dbVersion.split(".")[0]}`,
 parameters: [
 {
 name: "rds.logical_replication",
 value: "1",
 applyMethod: "pending-reboot",
 },
 ],
 },
 { deleteBeforeReplace: false } // This explicitly calls out that the new group should be created before the old is deleted so that database availability can be maintained
);

const database = new aws.rds.Instance(
 "rds-instance",
 {
 allocatedStorage: storageAllocation,
 engine: "postgres",
 engineVersion: dbVersion,
 instanceClass: aws.rds.InstanceType.T3_Micro,
 username: dbUsername,
 password: dbPW,
 storageEncrypted: true,
 backupRetentionPeriod: 7,
 blueGreenUpdate: { enabled: true },
 skipFinalSnapshot: true,
 parameterGroupName: parameterGroup.name,
 },
 { dependsOn: [parameterGroup] } //This line can be omitted since the dependency is also inferred by the reference to parameterGroup.name
);

After running pulumi up with the above code, you’ll have a database deployed and attached to your custom parameter group. It’s important to create and attach a custom parameter group so that you can keep it up to date with any engine version upgrades. If you use the default parameter group, updates to the database version may fail when the green deployment is created since its parameter group family won’t match the new version. You can avoid a failure by ensuring that the parameter group’s family and engineVersion are always changed in the same update.

You won’t see the effects of having blueGreenUpdate enabled until you run your first update. Change any part of the database configuration and run pulumi up again to start the update. In the example above, you can change dbVersion to 15.13 to trigger an update to both the parameter group and the instance.

Once the update kicks off, you can watch on AWS as the blue/green deployment is created and the original settings are copied to the green instance.

Then, the green instance will be updated with the new settings and the program will switch over to the green instance and clean up.

After that, your update has been completed with minimal disruption!

Additional Considerations

General limitations and considerations for Amazon RDS blue/green deployments are listed in the AWS documentation, but there are a few other considerations that should be taken into account when using them for updates with infrastructure as code (IaC).

No Custom Checks: Because IaC will automatically perform the switchover when the update is complete, you will not have the opportunity to perform custom checks on the green instance before promoting it. RDS will run some basic guardrails checks before performing the switchover, but you will not be able to perform any additional checks.
No Cluster Support: This feature only works with database instances, not with clusters or instances that are in a cluster.
Long Update Times: Because the IaC program will deploy temporary resources before performing the update and destroy them after, blue/green updates will take considerably longer than their in-place counterparts.
Manual Cleanup: Because the green instance and blue/green deployment are temporary resources designed to decrease downtime during the update, they are not tracked in the IaC state. If the update fails partway through, these temporary resources may have to be cleaned up manually. If the update is interrupted, you can delete the blue/green deployment and green database to get back into a manageable state.
Resource ID Change: On an update, the resource ID of the database might change since a new instance has been created. The endpoint and other details about the database will be maintained by AWS during switchover, but the resource ID should be dynamically retrieved from the stack outputs whenever it is used to ensure it stays up to date. Usually, no application or connection details will need to be updated since they don’t rely on the resource ID itself.
Types Of Updates: The blue/green update strategy will be used for changes that require a maintenance window including updates to the engine version, parameter group, storage and performance settings, and optimized writes.

Bring Your Own Keys With Pulumi ESC

Pablo Terradillos — Wed, 18 Jun 2025 13:25:26 -0300

Today we’re excited to launch support for Customer-Managed Keys (CMKs) in Pulumi ESC. This feature gives your organization full control over how your secrets and state are encrypted — empowering you to meet the most demanding compliance requirements like HIPAA, GDPR, and FedRAMP, all while maintaining the ease-of-use that Pulumi is known for.

Why Customer-Managed Keys?

Pulumi encrypts secrets and other sensitive information stored in ESC using data keys, which are encrypted with Pulumi-Managed Keys using strong security standards. However, customers might need to use their own keys to gain full control over their lifecycle and revocation, maintain a centralized audit trail, and enforce custom rotation policies. This is especially important to meet compliance requirements or specific regulations (e.g., HIPAA, GDPR, FedRAMP, etc.). Customer-Managed Keys give customers that control while keeping these aspects transparent for the consumers of those secrets.

How It Works

Customer-Managed Keys integrate seamlessly with your Pulumi Cloud Organization. Once enabled, all existing data keys used to encrypt your ESC secrets, previously encrypted with the Pulumi-managed key, will be automatically re-encrypted with the new CMK.

Setting it up takes just a few minutes:

Create a KMS key in your AWS account.
Grant access to the KMS key using OIDC in your AWS account.
Configure your Pulumi Cloud organization to use your new KMS key.

Available Today

Customer-Managed Keys with support for AWS KMS are available today for all organizations on the Pulumi Enterprise or Business Critical plans. You can set it up by following the instructions in the Pulumi Cloud documentation.

We expect to add support for more cloud providers in the future. Please vote on the following GitHub issues for your favorite to support prioritization:

We’re excited to see how Customer-Managed Keys help you and your organization meet your compliance goals with Pulumi Cloud. Your feedback is essential as we continue to evolve this feature:

Connect with us in the Pulumi Community Slack
Open an issue on GitHub
Read the full documentation on Customer-Managed Keys

Try this today, Get Started with Pulumi for free

How We Used Pulumi to Safely Migrate Oso's Global Infrastructure

Shaun Verch — Mon, 05 May 2025 00:00:00 +0000

Infrastructure as Code (IaC) tools such as Pulumi can provide enormous amounts of leverage, but they must be used correctly to also provide safety. One of our main jobs as infrastructure engineers is to not break things, so leverage without safety is useless. If something is safe, we can change things easily without even thinking about it. If it isn’t, we’ll be up until 2 a.m. fixing what we broke.

At Oso, we recently had to do a large infrastructure migration and learned three key principles that make a huge difference in using Pulumi for safe migrations. First, understand how it’s even possible to safely change your infrastructure independently of Pulumi. Second, write reusable and predictable Pulumi code. Finally, refactor your Pulumi code to your new version using zero-diff refactors.

Osoʼs Migration

Oso provides Authorization as a Service, so every application decision must first query Oso. This means that if Oso goes down, our customers will also go down. We can’t afford a moment of downtime.

However, we’re also growing quickly, and one of the side effects of growth is that architectural decisions that worked for us in the past quickly reach their expiration date. One example of this is when we hit a scaling limit and needed to quickly migrate from Amazon ECS Fargate to ECS on our own self-managed EC2 instances.

We can’t overstate the Pandora’s box of complexity this opened up for us. From the lifecycle of our data to our deployment orchestration, this touched nearly every part of our stack. On top of this, because performance and data locality are critical for us, we are deployed globally in 12 different regions. We need to be extremely careful to do this correctly.

How the Migration Actually Worked

The most important thing to realize when using an infrastructure tool like Pulumi is that ultimately you’re dealing with real resources. It doesn’t matter how cleverly you refactor your Pulumi code; if something gets deleted that isn’t supposed to be deleted, or if at any point a critical component is missing, there will be an outage.

At Oso, we were already using a Route 53 Traffic Policy to do latency-based per-region routing, so Route 53 was a convenient tool for us in this migration. The plan was to create an entirely new Oso infrastructure and include it in our traffic policy, but send 0% of our traffic to it. Then, we could run tests ourselves directly against the new infrastructure and gradually ramp up traffic until we were fully migrated.

That is the first big lesson we took away from this. If we didn’t have a point in our infrastructure where we could easily reroute customer traffic from one place to the other, this transition would have been impossible without downtime.

Writing Reusable Pulumi Code

The holy grail of infrastructure development is to achieve what we already have in application development: code reuse. Besides the benefit of reducing the burden of maintaining more code, it also reduces risk because we can concentrate our testing in one place and benefit from using code that has already been proven.

In Osoʼs case, because our migration involved creating a copy of our entire infrastructure with the new changes, code reuse was vital. Even the smallest reusable component meant fewer new bugs and less time fixing bugs we had already fixed in the old infrastructure.

In Pulumi, you can use the import and function system of the underlying programming language, so that part of managing libraries is not an issue. The real challenge lies in how to actually write the business logic of libraries in Pulumi because writing Pulumi code is not like writing normal code.

Take a look at this example of creating an ECS service:


let service = new aws.ecs.Serice("api",{ });

This line of code might seem trivial, but a lot is happening here, and it’s important to understand Pulumi’s programming model on a deeper level to know how to work with it properly.

The first thing to note is the first argument: api. This is the logical resource name in Pulumi. The subsequent arguments are more about how to actually create the resource, and we wonʼt talk about that here. This logical resource is how Pulumi identifies this resource in the internal state.

The internal state is the core of the Pulumi engine. It maps the “logical resource name" we provide to the actual resource that Pulumi either created or plans to create. The code example above, at some point deep inside the Pulumi engine, turns into something that roughly looks like this:

{
 "aws:ecs:Service": {
 "api": {...},
 ...
 },
 ...
}

This is the key to understanding why writing Pulumi code is not like writing normal code. There is one “state namespace” per Pulumi execution, which effectively means that there is one global object that we are adding resources to. So the new aws.ecs.Service function call wasn’t just creating a service object that we can pass around; it was registering the service with the global list of resources we plan to deploy.

After we understand that calls to the Pulumi library have side effects on this global state, we can start to write better Pulumi libraries. For example, say we have this function call:

deployWebStack("app",true,false,"enabled");

Do you know which resources this will create? Is it safe to call deployWebStack again with the same arguments? What would happen if we deleted this line of code? These are all the familiar perils of global state, and something we must proactively contend with to write a good Pulumi library.

Here are some rules that we recommend:

DO use Pulumi Component Resources in libraries that create resources.
- This encourages a more declarative object-based approach to resources, which fits better with Pulumi’s model and also causes these objects to show up in the Pulumi output.
Do NOT conditionally create resources inside library functions.
- If the function is called, all the resources should be created, and if it is not, the resources should not be created. Conditionals should be at the top level, because that’s where you are deciding what is deployed in a particular region or component.
Do NOT check the current infrastructure state inside library functions.
- This should be passed in as arguments; otherwise, the library function will have non-deterministic behavior.
Do NOT attempt to implement idempotent resource creation functions.
- See the previous rule. The only way to implement a function that creates a resource only if it hasn’t been created yet is to check if it has been created, leading to non-determinism.
DO write functional code as inputs to resources.
- You can still write normal code in some cases, as long as it doesn’t call into Pulumi. For example, the code that generates the list of environment variables for an app can be a normal function.
DO have a strategy for resource naming.
- Because any function might register new resources, and because you may even have naming conflicts on the cloud provider side, having a coherent strategy for naming resources is helpful. We don’t have anything prescriptive here, because it’s really a matter of preferences and specific situations.
DO favor assigning resources to named objects and variables over getter functions.
- Rather than implementing a function or some kind of “getter” to refer to another resource, save that resource object in a variable at some scope where it can be referenced by the other resources that need it. This can even be in the global scope to match the reality that all the resources end up in one resource namespace in the end.

There’s a lot of advice here, but it really all comes down to keeping in mind what Pulumi is actually doing with your code and remembering that calls into the Pulumi library are actually registering resources in the global state.

Infrastructure Snapshot Testing for Zero Diff Refactors

Now that we know what we want to change in our infrastructure, and we know how to write our Pulumi code in a reusable way, we’re faced with a familiar software engineering challenge: How can we safely refactor our existing code to get to where we want to be? To answer this question, we can step back from the Pulumi specifics for a moment and think about refactoring in general.

One method that is amazing for refactoring code is Snapshot Testing. With Snapshot Testing, you run the legacy code that you want to change, save the output, and create a unit test around that. Then, when you make a code change, you can detect if your code changes result in any unintended changes in behavior.

We can imitate this with Pulumi by taking advantage of the fact that we can see a preview of the changes Pulumi will make before we deploy them. While we don’t have direct access to Pulumi’s internal data model in the code itself to make this “snapshot,” the preview can give us this comparison. If we make changes to our infrastructure, as long as pulumi preview shows no changes, we are 100% sure that the refactor we made is safe and won’t cause any issues. Because of this, we at Oso internally call this method a “zero diff refactor.”

This is a relatively simple concept, but it is necessary to make this migration possible. In our original infrastructure, most of our resources were deployed in one monolithic Pulumi project, and resources were created at the top level rather than in libraries. The 12-region deployment also made running Pulumi on this project very slow, and we wanted a clean separation between the old and new infrastructure. Using snapshot testing to refactor allowed us to build and reuse our in-house infrastructure libraries with 100% confidence and let us focus on the parts that held the real risk.

Conclusion

This migration did not go smoothly. We constantly found problems and unexpected assumptions that our code was making about the underlying infrastructure.

But no one outside Oso noticed. We had a clear migration plan, factored out common Pulumi code, and used zero-diff refactors. Putting this all together meant we could benefit from the leverage that Pulumi has the potential to provide. This allowed us to make major changes with confidence that they wouldn’t break our system.

In the early days of Oso, people were often surprised by how much we were able to accomplish with such a small core team of engineers. Part of the secret to this is using the tools we have available to gain maximum leverage, but this isn’t the entire story. Knowing how to use them safely allows us to actually benefit from that leverage in practice.

If you’re interested in exploring Pulumi further, here are several ways to get involved:

Discover how Pulumi Crosswalk for AWS simplifies “day one” tasks in our AWS guide
Register for one of our upcoming workshops
But most importantly, try Pulumi today!

Enforcing Policy as Code on Discovered Resources with Pulumi

Tyler D — Tue, 04 Feb 2025 00:00:00 +0000

In this post, we’re introducing a powerful new capability in Pulumi Insights that extends policy as code (PaC) beyond infrastructure as code to automatically govern all cloud resources in your environment. By unifying policy enforcement across both IaC and discovered resources, you can now write policies once and apply them universally - dramatically simplifying how organizations maintain security and compliance standards at scale.

What is Policy as Code (PaC)?

Policy as Code (PaC) is a way to define and enforce infrastructure policies using code. Instead of relying on manual processes or static configuration files, PaC enables automated policy enforcement in a consistent, versioned way. Pulumi’s CrossGuard feature allows you to write policies in general purpose programming languages, applying them across various cloud platforms such as AWS, Azure, GCP, and Kubernetes.

You can read more about CrossGuard in Pulumi’s CrossGuard documentation.

Accounts and Insights

Before diving into how you can configure PaC for discovered resources, it’s important to understand how Pulumi discovers resources within your cloud environment regardless of how they were created. Pulumi Insights provides visibility into the state of your cloud resources by using read only access to your cloud environment in order to catalog the resources it contains. This is done through the Accounts page in Pulumi, which shows all the accounts for discovering resources you have added to your Pulumi organization. On this page, you can add new accounts which you can then add to policy groups to begin evaluating all of your cloud resources using your organization’s policy as code.

After you have added an account, you can connect it with one or multiple Policy Groups to enforce the policies within those groups across your stacks and Insights accounts. Once linked to a Policy Group, your resources will be automatically evaluated against the policies you define whenever those policies or resources are updated.

Setting Up Policy Groups for Discovered Resources

To apply policies to your discovered resources, you first need to set up a Policy Group. A Policy Group in Pulumi enforces policies across a group of stacks and accounts in your organization. Each Policy Group can contain multiple stacks and accounts, and you can assign multiple Policy Packs to these groups.

Here’s an example of the Policy Group configuration page, where you can create and manage policy groups:

Once your Policy Group is set up, you can add Insights Accounts to the group, which will ensure that the resources in those accounts are evaluated against the policies in the group.

Running Policies on Discovered Resources

Now that your Policy Group is set up, you can start running policies against your discovered resources. Currently, Pulumi Insights supports running resource policies against all resource types supported by Pulumi Insights with support for stack policies coming soon. Pulumi Insights will evaluate each resource against the policies defined in your Policy Pack. If the resources violate any of the defined policies, those violations will show up on the Policy Violations page, which gives you a detailed view of any non-compliant resources.

Each violation includes details about the resource and the reason for the violation, helping you quickly identify and address issues.

Key Takeaways

With Pulumi’s new PaC over discovered resources feature, you can now apply the same policies to both IaC and discovered resources across your cloud environment. This enables you to write a policy once and have it apply across all of your cloud resources. This results in consistent, compliant infrastructure no matter how resources are provisioned.

To begin securing your cloud infrastructure, here are the steps to take:

Set up Policy Groups to enforce policies across stacks and accounts.
Link Insights Accounts to your Policy Groups.
View violations on the Policy Violations page and take action to resolve them.

This new capability is available for AWS, Azure, OCI, and Kubernetes. Stay tuned for more updates on policy enforcement and discover how you can leverage Pulumi for cloud security and compliance.

Try it yourself: Start enforcing policies across your discovered resources in minutes with Pulumi Insights. Check out our getting started guide to begin securing your entire cloud environment with policy as code.

For more information on getting started with Policy as Code, check out the following resources:

Run Open-Source LLMs on AWS EC2 with Ollama and Pulumi

Engin Diri — Mon, 27 Jan 2025 00:00:00 +0000

TL;DR. Want to self-host an open-source LLM on AWS? Use a g4dn.xlarge ($0.526/hr on-demand, 16 GB GPU memory) for 7B/8B models, a g5.xlarge ($1.006/hr, 24 GB) for 13B–14B models, a g5.2xlarge ($1.212/hr, 24 GB) for 32B models, or a g6e.2xlarge ($2.242/hr, 48 GB) for 70B models. Deploy with the Pulumi program below and Ollama will run any model from its library: DeepSeek-R1, Llama 3, Qwen, or Mistral, with a one-line change.

This guide walks through that deployment end-to-end: a single Pulumi program that provisions a GPU-enabled EC2 instance, installs Ollama and Open WebUI via cloud-init, and exposes both a chat UI and an OpenAI-compatible API. The model is configurable, so you can swap DeepSeek-R1 for Llama 3.1, Qwen 2.5, or Mistral without touching the infrastructure code.

Why run open-source LLMs on AWS EC2?

Self-hosting an open-source LLM on AWS gives you three things hosted APIs can’t: data stays inside your VPC, per-token costs collapse to a flat hourly rate at high volume, and you can fine-tune or quantize models freely under permissive licenses. Ollama handles all three concerns from a single binary: it downloads, manages, and serves models behind an OpenAI-compatible API on port 11434.

The original version of this post focused on DeepSeek-R1 because it landed in late January 2025 and reset expectations for what an open-weight reasoning model could do. DeepSeek-R1 is still an excellent default (MIT-licensed, strong on math and coding, with distilled 1.5B–70B variants) but the same infrastructure runs Meta’s Llama 3, Alibaba’s Qwen, and Mistral equally well. Picking a model is now a config change, not an infrastructure decision.

For reference, DeepSeek-R1 scores 79.8% on AIME 2024 (vs. 79.2% for OpenAI o1-1217), 97.3% on MATH-500 (vs. 96.4%), 96.3% on Codeforces (vs. 96.6%), and 49.2% on SWE-bench Verified (vs. 48.9%)—near-parity with closed frontier models on most reasoning benchmarks, with the same caveat that benchmark scores age fast.

You might also like:

Which models can I run, and which EC2 instance do I need?

The bottleneck for inference is GPU memory (VRAM): the model weights have to fit in it, plus a few GB of headroom for the KV cache. The table below maps the most common Ollama models to the smallest AWS GPU instance that comfortably runs them at 4-bit (Q4) quantization, which is what ollama pull <model> gives you by default.

Model family	Sizes	Approx. VRAM (Q4)	Smallest EC2 instance	On-demand price (us-east-1)
DeepSeek-R1 (distill)	1.5B / 7B / 8B	1–6 GB	`g4dn.xlarge` (T4, 16 GB)	$0.526/hr
Llama 3.1 / Llama 3.2	8B	~5 GB	`g4dn.xlarge` (T4, 16 GB)	$0.526/hr
Qwen 2.5	7B	~5 GB	`g4dn.xlarge` (T4, 16 GB)	$0.526/hr
Mistral 7B / Mistral Nemo	7B / 12B	5–8 GB	`g4dn.xlarge` (T4, 16 GB)	$0.526/hr
DeepSeek-R1 (distill)	14B	~10 GB	`g5.xlarge` (A10G, 24 GB)	$1.006/hr
Llama 3.3 / DeepSeek-R1	32B / 32B distill	~20 GB	`g5.2xlarge` (A10G, 24 GB)	$1.212/hr
Llama 3.1 / DeepSeek-R1	70B	~42 GB	`g6e.2xlarge` (L40S, 48 GB)	$2.242/hr
DeepSeek-R1 (full)	671B (MoE)	400 GB+	`p5.48xlarge` or multi-node	$98.32/hr

For most workloads—internal tools, RAG backends, code assistants—a g4dn.xlarge running an 8B model is the right starting point. Move up only if quality is the bottleneck.

How much does this cost vs. hosted APIs?

A g4dn.xlarge running 24/7 costs ~$378/month on-demand, or ~$237/month with a 1-year reserved instance. Whether that’s cheaper than a hosted API depends entirely on your token volume.

Compare against hosted pricing as of April 2026 (input + output blended, rough numbers):

Provider	Model	Approx. blended price
OpenAI	GPT-4o-mini	~$0.30 per 1M tokens
OpenAI	GPT-4o	~$5.00 per 1M tokens
Anthropic	Claude Sonnet 4	~$6.00 per 1M tokens
DeepSeek (hosted)	DeepSeek-V3	~$0.50 per 1M tokens

A g4dn.xlarge running Llama 3.1 8B sustains roughly 40–60 tokens/sec under single-user load, or about 100–155M tokens/month at 100% utilization. At that ceiling the effective rate is ~$2.40/M tokens—cheaper than GPT-4o or Claude, more expensive than GPT-4o-mini or DeepSeek’s own hosted API.

The takeaway: self-hosting wins on data residency, latency, and predictable cost at high utilization. Hosted APIs win below ~10M tokens/month or when you need frontier-class quality. Run the math against your actual token volume before committing.

How do I deploy Ollama on AWS EC2 with Pulumi?

Prerequisites

Before we start, make sure you have the following:

An AWS account
Pulumi CLI installed
AWS CLI installed and configured
A working understanding of Ollama

What is Ollama?

Ollama is an open-source runtime that downloads, manages, and serves large language models from a single binary. It runs on macOS, Linux, and Windows, supports GPU acceleration through CUDA and Metal, and exposes both a native HTTP API and an OpenAI-compatible API on port 11434. Most clients written for the OpenAI SDK work against an Ollama endpoint with only a base-URL change.

It supports the major open-weight families—DeepSeek-R1, Llama 3, Qwen, Mistral, Gemma, Phi—plus quantized and distilled variants for each. You pull a model by name and tag (ollama pull llama3.1:8b) and run it (ollama run llama3.1:8b); Ollama handles the rest.

Architecture

Create a new Pulumi project

First, scaffold a new Pulumi project. Run the following from an empty directory:

# Replace <language> with typescript, python, go, csharp, or yaml
pulumi new aws-<language>

Pick the language you are most comfortable with. The template installs the AWS provider and creates a working sample. You can delete the sample code—we’ll replace it with the snippets below.

Step 1: Create an instance role with S3 access

The EC2 instance needs to download NVIDIA drivers from a public AWS-managed S3 bucket. Create an IAM role with S3 read access and attach it to an instance profile:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as fs from "fs";

const role = new aws.iam.Role("deepSeekRole", {
 name: "deepseek-role",
 assumeRolePolicy: JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Action: "sts:AssumeRole",
 Effect: "Allow",
 Principal: {
 Service: "ec2.amazonaws.com",
 },
 },
 ],
 }),
});

new aws.iam.RolePolicyAttachment("deepSeekS3Policy", {
 policyArn: "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess",
 role: role.name,
});

const instanceProfile = new aws.iam.InstanceProfile("deepSeekProfile", {
 name: "deepseek-profile",
 role: role.name,
});

import pulumi
import pulumi_aws as aws
import json
import os

# IAM Role for EC2 instances
role = aws.iam.Role(
 "deepSeekRole",
 name="deepseek-role",
 assume_role_policy=json.dumps(
 {
 "Version": "2012-10-17",
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": "ec2.amazonaws.com",
 },
 }
 ],
 }
 ),
)

# Attach S3 read-only policy to the IAM Role
iam_policy_attachment = aws.iam.RolePolicyAttachment(
 "deepSeekS3Policy",
 policy_arn="arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess",
 role=role.name,
)

# Instance Profile containing the IAM Role
instance_profile = aws.iam.InstanceProfile(
 "deepSeekProfile", name="deepseek-profile", role=role.name
)

package main

import (
 "encoding/json"
 "os"

 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/iam"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 rolePolicy, err := json.Marshal(map[string]interface{}{
 "Version": "2012-10-17",
 "Statement": []map[string]interface{}{
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": map[string]interface{}{"Service": "ec2.amazonaws.com"},
 },
 },
 })
 if err != nil {
 return err
 }

 role, err := iam.NewRole(ctx, "deepSeekRole", &iam.RoleArgs{
 Name: pulumi.String("deepseek-role"),
 AssumeRolePolicy: pulumi.String(rolePolicy),
 })
 if err != nil {
 return err
 }

 // Attach S3 read-only policy to the IAM Role
 _, err = iam.NewRolePolicyAttachment(ctx, "deepSeekS3Policy", &iam.RolePolicyAttachmentArgs{
 PolicyArn: pulumi.String("arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"),
 Role: role.Name,
 })
 if err != nil {
 return err
 }

 // Instance Profile containing the IAM Role
 instanceProfile, err := iam.NewInstanceProfile(ctx, "deepSeekProfile", &iam.InstanceProfileArgs{
 Name: pulumi.String("deepseek-profile"),
 Role: role.Name,
 })
 if err != nil {
 return err
 }
 return nil
 })
}

using Pulumi;
using Pulumi.Aws.Ec2;
using Pulumi.Aws.Iam;
using System.Collections.Generic;
using System.IO;
using Pulumi.Aws.Ec2.Inputs;
using System.Text.Json;

return await Deployment.RunAsync(() =>
{
 var rolePolicy = JsonSerializer.Serialize(new Dictionary<string, object>
 {
 {
 ["Version"] = "2012-10-17",
 ["Statement"] = new[]
 {
 new Dictionary<string, object>
 {
 ["Action"] = "sts:AssumeRole",
 ["Effect"] = "Allow",
 ["Principal"] = new Dictionary<string, string>
 {
 ["Service"] = "ec2.amazonaws.com"
 }
 }
 }
 });

 var role = new Role("deepSeekRole", new RoleArgs
 {
 Name = "deepseek-role",
 AssumeRolePolicy = rolePolicy
 });

 var rolePolicyAttachment = new RolePolicyAttachment("deepSeekS3Policy", new RolePolicyAttachmentArgs
 {
 PolicyArn = "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess",
 Role = role.Name
 });

 var instanceProfile = new InstanceProfile("deepSeekProfile", new InstanceProfileArgs
 {
 Name = "deepseek-profile",
 Role = role.Name
 });

 var vpc = new Vpc("deepSeekVpc", new VpcArgs
 {
 CidrBlock = "10.0.0.0/16",
 EnableDnsHostnames = true,
 EnableDnsSupport = true
 });

 var subnet = new Subnet("deepSeekSubnet", new SubnetArgs
 {
 VpcId = vpc.Id,
 CidrBlock = "10.0.48.0/20",
 AvailabilityZone = "eu-central-1a",
 MapPublicIpOnLaunch = true
 });

 var internetGateway = new InternetGateway("deepSeekInternetGateway", new InternetGatewayArgs
 {
 VpcId = vpc.Id
 });

 var routeTable = new RouteTable("deepSeekRouteTable", new RouteTableArgs

name: deepseek-ollama-yaml
description: DeepSeek Ollama AWS example
runtime: yaml

variables:
 publicKey:
 fn::readFile: ./deepseek.rsa
 userData:
 fn::readFile: ./cloud-init.yaml
 amiFilter: "amzn2-ami-hvm-*-x86_64-gp2"
 amiOwner: "137112412989"
 amiId:
 fn::invoke:
 function: aws:ec2:getAmi
 arguments:
 filters:
 - name: name
 values: ["${amiFilter}"]
 owners: ["${amiOwner}"]
 mostRecent: true
 return: id

resources:
 deepSeekRole:
 type: aws:iam:Role
 properties:
 name: deepseek-role
 assumeRolePolicy: |
 {
 "Version": "2012-10-17",
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": "ec2.amazonaws.com"
 }
 }
 ]
 }

 deepSeekS3Policy:
 type: aws:iam:RolePolicyAttachment
 properties:
 role: ${deepSeekRole.name}
 policyArn: arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

 deepSeekProfile:
 type: aws:iam:InstanceProfile
 properties:
 name: deepseek-profile
 role: ${deepSeekRole.name}

Step 2: Create the network

Next, create the VPC, subnet, internet gateway, and route table. The security group opens ports 22 (SSH), 3000 (Open WebUI), and 11434 (Ollama API):

const vpc = new aws.ec2.Vpc("deepSeekVpc", {
 cidrBlock: "10.0.0.0/16",
 enableDnsHostnames: true,
 enableDnsSupport: true,
});

const subnet = new aws.ec2.Subnet("deepSeekSubnet", {
 vpcId: vpc.id,
 cidrBlock: "10.0.48.0/20",
 availabilityZone: pulumi.interpolate`${aws.getAvailabilityZones().then(it => it.names[0])}`,
 mapPublicIpOnLaunch: true,
});

const internetGateway = new aws.ec2.InternetGateway("deepSeekInternetGateway", {
 vpcId: vpc.id,
});

const routeTable = new aws.ec2.RouteTable("deepSeekRouteTable", {
 vpcId: vpc.id,
 routes: [
 {
 cidrBlock: "0.0.0.0/0",
 gatewayId: internetGateway.id,
 },
 ],
});

const routeTableAssociation = new aws.ec2.RouteTableAssociation("deepSeekRouteTableAssociation", {
 subnetId: subnet.id,
 routeTableId: routeTable.id,
});

const securityGroup = new aws.ec2.SecurityGroup("deepSeekSecurityGroup", {
 vpcId: vpc.id,
 egress: [
 {
 fromPort: 0,
 toPort: 0,
 protocol: "-1",
 cidrBlocks: ["0.0.0.0/0"],
 },
 ],
 ingress: [
 {
 fromPort: 22,
 toPort: 22,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 {
 fromPort: 3000,
 toPort: 3000,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 {
 fromPort: 11434,
 toPort: 11434,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 ],
});


# Create a VPC
vpc = aws.ec2.Vpc(
 "deepSeekVpc",
 cidr_block="10.0.0.0/16",
 enable_dns_hostnames=True,
 enable_dns_support=True,
)

# Create a subnet
subnet = aws.ec2.Subnet(
 "deepSeekSubnet",
 vpc_id=vpc.id,
 cidr_block="10.0.48.0/20",
 availability_zone="eu-central-1a",
 map_public_ip_on_launch=True,
)

# Create an internet gateway
internet_gateway = aws.ec2.InternetGateway("deepSeekInternetGateway", vpc_id=vpc.id)

# Create a route table and route table association
route_table = aws.ec2.RouteTable(
 "deepSeekRouteTable",
 vpc_id=vpc.id,
 routes=[
 aws.ec2.RouteTableRouteArgs(
 cidr_block="0.0.0.0/0", gateway_id=internet_gateway.id
 )
 ],
)

route_table_association = aws.ec2.RouteTableAssociation(
 "deepSeekRouteTableAssociation", subnet_id=subnet.id, route_table_id=route_table.id
)

# Create a security group
security_group = aws.ec2.SecurityGroup(
 "deepSeekSecurityGroup",
 vpc_id=vpc.id,
 egress=[
 {
 "from_port": 0,
 "to_port": 0,
 "protocol": "-1",
 "cidr_blocks": ["0.0.0.0/0"],
 }
 ],
 ingress=[
 {
 "from_port": 22,
 "to_port": 22,
 "protocol": "tcp",
 "cidr_blocks": ["0.0.0.0/0"],
 },
 {
 "from_port": 3000,
 "to_port": 3000,
 "protocol": "tcp",
 "cidr_blocks": ["0.0.0.0/0"],
 },
 {
 "from_port": 11434,
 "to_port": 11434,
 "protocol": "tcp",
 "cidr_blocks": ["0.0.0.0/0"],
 },
 ],
)

package main

import (
 "encoding/json"
 "os"

 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/iam"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // omitted for brevity
 }

 // Create a VPC
 vpc, err := ec2.NewVpc(ctx, "deepSeekVpc", &ec2.VpcArgs{
 CidrBlock: pulumi.String("10.0.0.0/16"),
 EnableDnsHostnames: pulumi.Bool(true),
 EnableDnsSupport: pulumi.Bool(true),
 })
 if err != nil {
 return err
 }

 // Create a subnet
 subnet, err := ec2.NewSubnet(ctx, "deepSeekSubnet", &ec2.SubnetArgs{
 VpcId: vpc.ID(),
 CidrBlock: pulumi.String("10.0.48.0/20"),
 AvailabilityZone: pulumi.String("eu-central-1a"),
 MapPublicIpOnLaunch: pulumi.Bool(true),
 })
 if err != nil {
 return err
 }

 // Create an internet gateway
 internetGateway, err := ec2.NewInternetGateway(ctx, "deepSeekInternetGateway", &ec2.InternetGatewayArgs{
 VpcId: vpc.ID(),
 })
 if err != nil {
 return err
 }

 // Create a route table and route table association
 routeTable, err := ec2.NewRouteTable(ctx, "deepSeekRouteTable", &ec2.RouteTableArgs{
 VpcId: vpc.ID(),
 Routes: ec2.RouteTableRouteArray{
 &ec2.RouteTableRouteArgs{
 CidrBlock: pulumi.String("0.0.0.0/0"),
 GatewayId: internetGateway.ID(),
 },
 },
 })
 if err != nil {
 return err
 }

 _, err = ec2.NewRouteTableAssociation(ctx, "deepSeekRouteTableAssociation", &ec2.RouteTableAssociationArgs{
 SubnetId: subnet.ID(),
 RouteTableId: routeTable.ID(),
 })
 if err != nil {
 return err
 }

 // Create a security group
 securityGroup, err := ec2.NewSecurityGroup(ctx, "deepSeekSecurityGroup", &ec2.SecurityGroupArgs{
 VpcId: vpc.ID(),
 Egress: ec2.SecurityGroupEgressArray{
 &ec2.SecurityGroupEgressArgs{
 FromPort: pulumi.Int(0),
 ToPort: pulumi.Int(0),
 Protocol: pulumi.String("-1"),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 },
 Ingress: ec2.SecurityGroupIngressArray{
 &ec2.SecurityGroupIngressArgs{
 FromPort: pulumi.Int(22),
 ToPort: pulumi.Int(22),
 Protocol: pulumi.String("tcp"),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 &ec2.SecurityGroupIngressArgs{
 FromPort: pulumi.Int(3000),
 ToPort: pulumi.Int(3000),
 Protocol: pulumi.String("tcp"),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 &ec2.SecurityGroupIngressArgs{
 FromPort: pulumi.Int(11434),
 ToPort: pulumi.Int(11434),
 Protocol: pulumi.String("tcp"),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 },
 })
 if err != nil {
 return err
 }

 return nil
 })
}

using Pulumi;
using Pulumi.Aws.Ec2;
using Pulumi.Aws.Iam;
using System.Collections.Generic;
using System.IO;
using Pulumi.Aws.Ec2.Inputs;
using System.Text.Json;

return await Deployment.RunAsync(() =>
{
 var rolePolicy = JsonSerializer.Serialize(new Dictionary<string, object>
 {
 // omitted for brevity
 {
 VpcId = vpc.Id,
 Routes =
 {
 new RouteTableRouteArgs
 {
 CidrBlock = "0.0.0.0/0",
 GatewayId = internetGateway.Id
 }
 }
 });

 var routeTableAssociation = new RouteTableAssociation("deepSeekRouteTableAssociation", new RouteTableAssociationArgs
 {
 SubnetId = subnet.Id,
 RouteTableId = routeTable.Id
 });

 var securityGroup = new SecurityGroup("deepSeekSecurityGroup", new SecurityGroupArgs
 {
 VpcId = vpc.Id,
 Egress =
 {
 new SecurityGroupEgressArgs
 {
 FromPort = 0,
 ToPort = 0,
 Protocol = "-1",
 CidrBlocks = { "0.0.0.0/0" }
 }
 },
 Ingress =
 {
 new SecurityGroupIngressArgs
 {
 FromPort = 22,
 ToPort = 22,
 Protocol = "tcp",
 CidrBlocks = { "0.0.0.0/0" }
 },
 new SecurityGroupIngressArgs
 {
 FromPort = 3000,
 ToPort = 3000,
 Protocol = "tcp",
 CidrBlocks = { "0.0.0.0/0" }
 },
 new SecurityGroupIngressArgs
 {
 FromPort = 11434,
 ToPort = 11434,
 Protocol = "tcp",
 CidrBlocks = { "0.0.0.0/0" }
 }
 }
 });

 var publicKey = File.ReadAllText("deepseek.rsa");
 var keyPair = new KeyPair("deepSeekKey", new KeyPairArgs
 {
 PublicKey = publicKey
 });

 var amazonLinux = Pulumi.Aws.Ec2.GetAmi.Invoke(new()
 {
 MostRecent = true,
 Filters = new[]
 {
 new GetAmiFilterInputArgs
 {
 Name = "name",
 Values = new[] { "amzn2-ami-hvm-*-x86_64-gp2" }
 },
 new GetAmiFilterInputArgs
 {
 Name = "architecture",


 deepSeekVpc:
 type: aws:ec2:Vpc
 properties:
 cidrBlock: 10.0.0.0/16
 enableDnsHostnames: true
 enableDnsSupport: true

 deepSeekSubnet:
 type: aws:ec2:Subnet
 properties:
 vpcId: ${deepSeekVpc.id}
 cidrBlock: 10.0.48.0/20
 availabilityZone: eu-central-1a
 mapPublicIpOnLaunch: true

 deepSeekInternetGateway:
 type: aws:ec2:InternetGateway
 properties:
 vpcId: ${deepSeekVpc.id}

 deepSeekRouteTable:
 type: aws:ec2:RouteTable
 properties:
 vpcId: ${deepSeekVpc.id}
 routes:
 - cidrBlock: 0.0.0.0/0
 gatewayId: ${deepSeekInternetGateway.id}

 deepSeekRouteTableAssociation:
 type: aws:ec2:RouteTableAssociation
 properties:
 subnetId: ${deepSeekSubnet.id}
 routeTableId: ${deepSeekRouteTable.id}

 deepSeekSecurityGroup:
 type: aws:ec2:SecurityGroup
 properties:
 vpcId: ${deepSeekVpc.id}
 ingress:
 - fromPort: 22
 toPort: 22
 protocol: tcp
 cidrBlocks:
 - 0.0.0.0/0
 - fromPort: 3000
 toPort: 3000
 protocol: tcp
 cidrBlocks:
 - 0.0.0.0/0
 - fromPort: 11434
 toPort: 11434
 protocol: tcp
 cidrBlocks:
 - 0.0.0.0/0
 egress:
 - fromPort: 0
 toPort: 0
 protocol: -1
 cidrBlocks:
 - 0.0.0.0/0

Step 3: Launch the GPU EC2 instance

Now create the EC2 instance itself. The example uses Amazon Linux because the NVIDIA driver install path is well-trodden, plus an SSH key pair you generate locally.

The default instance type is g4dn.xlarge—the cheapest option that fits any 7B/8B model. Bump it up if you picked a larger model from the table above: g5.xlarge for 13B–14B, g5.2xlarge for 32B, g6e.2xlarge for 70B. AWS publishes full specs for the G4, G5, and G6 families.

Generate the key pair if you don’t already have one:

openssl genrsa -out deepseek.pem 2048
openssl rsa -in deepseek.pem -pubout > deepseek.pub
ssh-keygen -f mykey.pub -i -mPKCS8 > deepseek.pem

const keyPair = new aws.ec2.KeyPair("deepSeekKey", {
 publicKey: pulumi.output(fs.readFileSync("deepseek.rsa", "utf-8")),
});

const deepSeekAmi = aws.ec2
 .getAmi({
 filters: [
 {
 name: "name",
 values: ["amzn2-ami-hvm-2.0.*-x86_64-gp2"],
 },
 {
 name: "architecture",
 values: ["x86_64"],
 },
 ],
 owners: ["137112412989"], // Amazon
 mostRecent: true,
 })
 .then(ami => ami.id);

const deepSeekInstance = new aws.ec2.Instance("deepSeekInstance", {
 ami: deepSeekAmi,
 instanceType: "g4dn.xlarge",
 keyName: keyPair.keyName,
 rootBlockDevice: {
 volumeSize: 100,
 volumeType: "gp3",
 },
 subnetId: subnet.id,
 vpcSecurityGroupIds: [securityGroup.id],
 iamInstanceProfile: instanceProfile.name,
 userData: fs.readFileSync("cloud-init.yaml", "utf-8"),
 tags: {
 Name: "deepSeek-server",
 },
});

export const amiId = deepSeekAmi;
export const instanceId = deepSeekInstance.id;
export const instancePublicDns = deepSeekInstance.publicIp;


# Key pair for SSH access
public_key = open("deepseek.rsa", "r").read()
key_pair = aws.ec2.KeyPair("deepSeekKey", public_key=public_key)


# Get the latest Amazon Linux 2 AMI
ami = aws.ec2.get_ami(
 filters=[
 {"name": "name", "values": ["amzn2-ami-hvm-2.0.*-x86_64-gp2"]},
 {"name": "architecture", "values": ["x86_64"]},
 ],
 owners=["137112412989"], # Amazon
 most_recent=True,
).id

# Create an EC2 instance
user_data = open("cloud-init.yaml", "r").read()
instance = aws.ec2.Instance(
 "deepSeekInstance",
 ami=ami,
 instance_type="g4dn.xlarge",
 key_name=key_pair.key_name,
 root_block_device=aws.ec2.InstanceRootBlockDeviceArgs(
 volume_size=100, volume_type="gp3"
 ),
 subnet_id=subnet.id,
 vpc_security_group_ids=[security_group.id],
 iam_instance_profile=instance_profile.name,
 user_data=user_data,
 tags={"Name": "deepSeek-server"},
)

pulumi.export("amiId", ami)
pulumi.export("instanceId", instance.id)
pulumi.export("instancePublicDns", instance.public_ip)

package main

import (
 "encoding/json"
 "os"

 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/iam"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // omitted for brevity

 // Key pair for SSH access
 publicKey, err := os.ReadFile("deepseek.rsa")
 if err != nil {
 return err
 }

 keyPair, err := ec2.NewKeyPair(ctx, "deepSeekKey", &ec2.KeyPairArgs{
 PublicKey: pulumi.String(string(publicKey)),
 })
 if err != nil {
 return err
 }

 // Get the latest Amazon Linux 2 AMI
 mostRecent := true
 ami, err := ec2.LookupAmi(ctx, &ec2.LookupAmiArgs{
 Filters: []ec2.GetAmiFilter{
 {
 Name: "name",
 Values: []string{"amzn2-ami-hvm-2.0.*-x86_64-gp2"},
 },
 {
 Name: "architecture",
 Values: []string{"x86_64"},
 },
 },
 Owners: []string{"137112412989"},
 MostRecent: &mostRecent,
 })
 if err != nil {
 return err
 }

 // Create an EC2 instance
 userData, err := os.ReadFile("cloud-init.yaml")
 if err != nil {
 return err
 }

 instance, err := ec2.NewInstance(ctx, "deepSeekInstance", &ec2.InstanceArgs{
 Ami: pulumi.String(ami.Id),
 InstanceType: pulumi.String("g4dn.xlarge"),
 KeyName: keyPair.KeyName,
 RootBlockDevice: &ec2.InstanceRootBlockDeviceArgs{
 VolumeSize: pulumi.Int(100),
 VolumeType: pulumi.String("gp3"),
 },
 SubnetId: subnet.ID(),
 VpcSecurityGroupIds: pulumi.StringArray{securityGroup.ID()},
 IamInstanceProfile: instanceProfile.Name,
 UserData: pulumi.String(string(userData)),
 Tags: pulumi.StringMap{
 "Name": pulumi.String("deepSeek-server"),
 },
 })
 if err != nil {
 return err
 }

 ctx.Export("amiId", pulumi.String(ami.Id))
 ctx.Export("instanceId", instance.ID())
 ctx.Export("instancePublicDns", instance.PublicIp)

 return nil
 })
}

using Pulumi;
using Pulumi.Aws.Ec2;
using Pulumi.Aws.Iam;
using System.Collections.Generic;
using System.IO;
using Pulumi.Aws.Ec2.Inputs;
using System.Text.Json;

return await Deployment.RunAsync(() =>
{
 var rolePolicy = JsonSerializer.Serialize(new Dictionary<string, object>
 {
 // omitted for brevity
 Name = "architecture",
 Values = new[] { "x86_64" }
 }
 },
 Owners = new[] { "137112412989" }
 });

 var userData = File.ReadAllText("cloud-init.yaml");
 var instance = new Instance("deepSeekInstance", new InstanceArgs
 {
 Ami = amazonLinux.Apply(r => r.Id),
 InstanceType = "g4dn.xlarge",
 KeyName = keyPair.KeyName,
 RootBlockDevice = new InstanceRootBlockDeviceArgs
 {
 VolumeSize = 100,
 VolumeType = "gp3"
 },
 SubnetId = subnet.Id,
 VpcSecurityGroupIds = { securityGroup.Id },
 IamInstanceProfile = instanceProfile.Name,
 UserData = userData,
 Tags = { { "Name", "deepSeek-server" } }
 });

 return new Dictionary<string, object?>
 {
 ["amiId"] = amazonLinux.Apply(r => r.Id),
 ["instanceId"] = instance.Id,
 ["instancePublicDns"] = instance.PublicIp
 };
});


 deepSeekKey:
 type: aws:ec2:KeyPair
 properties:
 publicKey: ${publicKey}

 deepSeekInstance:
 type: aws:ec2:Instance
 properties:
 ami: ${amiId}
 instanceType: "g4dn.xlarge"
 keyName: ${deepSeekKey.keyName}
 rootBlockDevice:
 volumeSize: 100
 volumeType: gp3
 subnetId: ${deepSeekSubnet.id}
 vpcSecurityGroupIds:
 - ${deepSeekSecurityGroup.id}
 iamInstanceProfile: ${deepSeekProfile.name}
 userData: ${userData}
 tags:
 Name: deepSeek-server

outputs:
 AmiId: ${amiId}
 InstanceId: ${deepSeekInstance.id}
 InstancePublicDns: ${deepSeekInstance.publicIp}

Step 4: Install Ollama via cloud-init

The EC2 instance is a blank box until cloud-init runs. The user-data script below installs the NVIDIA GRID drivers, Docker, and the NVIDIA Container Toolkit, then starts the Ollama and Open WebUI containers. To switch models, edit the ollama run line—the rest is identical regardless of which model you want.

#cloud-config
users:
- default

package_update: true

packages:
- apt-transport-https
- ca-certificates
- curl
- openjdk-17-jre-headless
- gcc

runcmd:
- yum install -y gcc kernel-devel-$(uname -r)
- aws s3 cp --recursive s3://ec2-linux-nvidia-drivers/latest/ .
- chmod +x NVIDIA-Linux-x86_64*.run
- /bin/sh ./NVIDIA-Linux-x86_64*.run --tmpdir . --silent
- touch /etc/modprobe.d/nvidia.conf
- echo "options nvidia NVreg_EnableGpuFirmware=0" | sudo tee --append /etc/modprobe.d/nvidia.conf
- yum install -y docker
- usermod -a -G docker ec2-user
- systemctl enable docker.service
- systemctl start docker.service
- curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
- yum install -y nvidia-container-toolkit
- nvidia-ctk runtime configure --runtime=docker
- systemctl restart docker
- docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama --restart always ollama/ollama
- sleep 120
- docker exec ollama ollama run deepseek-r1:7b
- docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

You might also like:

Step 5: Deploy the infrastructure

Make sure your AWS credentials are configured:

aws configure

Pulumi supports several other authentication methods for the AWS provider. Once credentials are in place, deploy the infrastructure:

pulumi up

Pulumi previews the changes; type yes to confirm.

pulumi up
Please choose a stack, or create a new one: <create a new stack>
Please enter your desired stack name.
To create a stack in an organization, use the format <org-name>/<stack-name> (e.g. `acmecorp/dev`): dev
Please enter your desired stack name.
Created stack 'dev'
Previewing update (dev)
View in Browser (Ctrl+O): https://app.pulumi.com/dirien/deepseek-ollama-typescript/dev/previews/1dbb18ea-ba31-4d5b-9510-5dce19eb8ee8
Type Name Plan
+ pulumi:pulumi:Stack deepseek-ollama-typescript-dev create
+ ├─ aws:ec2:KeyPair deepSeekKey create
+ ├─ aws:ec2:Vpc deepSeekVpc create
+ ├─ aws:iam:Role deepSeekRole create
+ ├─ aws:iam:InstanceProfile deepSeekProfile create
+ ├─ aws:ec2:SecurityGroup deepSeekSecurityGroup create
+ ├─ aws:ec2:RouteTable deepSeekRouteTable create
+ ├─ aws:ec2:InternetGateway deepSeekInternetGateway create
+ ├─ aws:ec2:Subnet deepSeekSubnet create
+ ├─ aws:iam:RolePolicyAttachment deepSeekS3Policy create
+ ├─ aws:ec2:RouteTableAssociation deepSeekRouteTableAssociation create
+ └─ aws:ec2:Instance deepSeekInstance create
Outputs:
amiId : "ami-085131ff43045c877"
instanceId : output<string>
instancePublicDns: output<string>
Resources:
+ 12 to create
Do you want to perform this update? yes
Updating (dev)
View in Browser (Ctrl+O): https://app.pulumi.com/dirien/deepseek-ollama-typescript/dev/updates/1
Type Name Status
+ pulumi:pulumi:Stack deepseek-ollama-typescript-dev created (40s)
+ ├─ aws:ec2:KeyPair deepSeekKey created (0.47s)
+ ├─ aws:iam:Role deepSeekRole created (1s)
+ ├─ aws:ec2:Vpc deepSeekVpc created (12s)
+ ├─ aws:iam:InstanceProfile deepSeekProfile created (6s)
+ ├─ aws:iam:RolePolicyAttachment deepSeekS3Policy created (0.90s)
+ ├─ aws:ec2:InternetGateway deepSeekInternetGateway created (0.69s)
+ ├─ aws:ec2:Subnet deepSeekSubnet created (11s)
+ ├─ aws:ec2:SecurityGroup deepSeekSecurityGroup created (2s)
+ ├─ aws:ec2:RouteTable deepSeekRouteTable created (1s)
+ ├─ aws:ec2:RouteTableAssociation deepSeekRouteTableAssociation created (0.92s)
+ └─ aws:ec2:Instance deepSeekInstance created (12s)
Outputs:
amiId : "ami-085131ff43045c877"
instanceId : "i-0ae7495781ace3e81"
instancePublicDns: "18.159.211.136"
Resources:
+ 12 created
Duration: 42s

The infrastructure provisions in under a minute, but the cloud-init script needs another 5–10 minutes to install drivers, pull container images, and download the model weights. SSH in to watch the progress, or just wait and load the Web UI when it’s ready.

ssh -i deepseek.pem ec2-user@<instance-public-ip>

Check the container status:

sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c8714335e205 ghcr.io/open-webui/open-webui:main "bash start.sh" 6 minutes ago Up 6 minutes (healthy) 0.0.0.0:3000->8080/tcp, :::3000->8080/tcp open-webui
bf4bb3b7ede1 ollama/ollama "/bin/ollama serve" 8 minutes ago Up 7 minutes 0.0.0.0:11434->11434/tcp, :::11434->11434/tcp ollama
[ec2-user@ip-10-0-58-122 ~]$

Step 6: Access the Web UI or API

Once the containers are healthy, open http://<instance-public-ip>:3000 in your browser for Open WebUI:

Open WebUI is not secured by default. Restrict the security group to your IP, put it behind an authenticated proxy, or terminate TLS at an ALB before exposing it to the internet.

For programmatic access, Ollama exposes an OpenAI-compatible API on port 11434. Most clients written for the OpenAI SDK only need a base-URL change:

from openai import OpenAI

client = OpenAI(
 base_url="http://<instance-public-ip>:11434/v1",
 api_key="ollama", # required by the SDK, ignored by Ollama
)

response = client.chat.completions.create(
 model="llama3.1:8b",
 messages=[{"role": "user", "content": "Why is the sky blue?"}],
)
print(response.choices[0].message.content)

How do I switch models?

Ollama hosts every major open-weight family in its model library. Pulling a different model is two commands inside the EC2 instance—or a one-line edit to the cloud-init script if you want it provisioned automatically:

# DeepSeek-R1 distill (default in this guide)
docker exec ollama ollama run deepseek-r1:7b

# Llama 3.1 (Meta, 8B)
docker exec ollama ollama run llama3.1:8b

# Qwen 2.5 (Alibaba, 7B)
docker exec ollama ollama run qwen2.5:7b

# Mistral (7B)
docker exec ollama ollama run mistral:7b

# Larger reasoning model (needs g5.2xlarge or larger)
docker exec ollama ollama run deepseek-r1:32b

Tags follow a <size> or <size>-<quantization> pattern—8b, 8b-instruct-q4_K_M, 8b-instruct-q8_0. Q4 is the default and the right starting point; bump to Q8 only if you have spare VRAM and notice quality issues with Q4. Browse the full tag list for any model on its Ollama library page.

Cleaning up

When you’re done, tear everything down:

pulumi destroy

What are the next steps?

You now have a reproducible, IaC-managed deployment of any open-source LLM on AWS. The infrastructure is fixed; the model is a parameter. From here, the natural extensions are wiring this up to a real application, adding RAG over your own data, or moving the deployment behind an authenticated load balancer.

If you want to go further with AI on Pulumi, here are some related guides:

Deploy LangServe Apps with Pulumi on AWS (RAG & Chatbot) — Build a retrieval-augmented chatbot that could front-end this Ollama instance.
Deploy AI Models on Amazon SageMaker using Pulumi Python IaC — A SageMaker alternative when you’d rather not manage the EC2 host yourself.
Build an AI Slack Bot on AWS Using Embedchain & Pulumi — Wire an LLM into Slack as an internal assistant.
What is Infrastructure as Code? — Background on the IaC approach used throughout this guide.

Try Pulumi for Free

Changelog

2026-04-30 — Broadened scope from DeepSeek-only to any Ollama-supported model (Llama, Qwen, Mistral). Added TL;DR, instance-type recommendation table, cost-vs-hosted-API comparison, and HowTo structured data. Restructured headings as user questions. Verified Ollama and cloud-init commands against current versions.
2025-03-10 — Minor edits and corrections.
2025-01-27 — Original post: Run DeepSeek-R1 on AWS EC2 Using Ollama.

Getting Started with Amazon EKS Auto Mode in Pulumi

Florian Stadler — Mon, 16 Dec 2024 23:26:21 +0100

AWS recently announced Amazon EKS Auto Mode, a significant enhancement to Amazon EKS that streamlines cluster management by automating infrastructure decisions and operations. Today, we are excited to announce that Pulumi fully supports EKS Auto Mode across our AWS provider ecosystem, enabling you to leverage this powerful feature through infrastructure as code.

What is EKS Auto Mode?

EKS Auto Mode extends AWS management beyond the Kubernetes control plane to the underlying infrastructure that powers your workloads. While traditional EKS requires you to make numerous infrastructure decisions and maintain various components, Auto Mode simplifies this by automating these choices based on AWS’s operational expertise.

Instead of manually configuring and managing cluster add-ons, node groups, and infrastructure components, Auto Mode handles:

Compute Management: Automatically provisions and scales EC2 instances based on your workload demands while selecting optimal instance types and sizes. It also handles node lifecycle management, including OS updates and security patches.
Storage Automation: Eliminates the need to manually set up and configure the EBS CSI driver for block storage. Auto Mode manages volume provisioning, attachment, and cleanup for your persistent workloads.
Networking Simplification: Automatically configures and manages core networking components like Load Balancers.

This significantly reduces the operational overhead of running Kubernetes clusters. Instead of spending time on infrastructure decisions and maintenance, teams can now focus on their applications. For example, deploying a stateful application with load balancing no longer requires setting up and maintaining multiple add-ons and configurations - Auto Mode handles this automatically while following AWS best practices.

Common tasks that previously required significant expertise and time are now managed by AWS, making it simpler to run production-grade Kubernetes workloads. This includes:

Setting up and maintaining cluster autoscaling
Configuring and updating networking plugins
Managing storage drivers and volume provisioning
Handling node updates and security patches
Optimizing instance selection and placement

Pulumi Support for EKS Auto Mode

We have added EKS Auto Mode support across our AWS provider ecosystem:

Pulumi AWS Provider (v6.63.0+)
Pulumi AWS Cloud Control Provider (previously known as Pulumi AWS Native) (v1.13.0+)
Pulumi EKS Provider (v3.5.0+)

Creating an EKS Auto Mode Cluster

Let’s explore how to leverage EKS Auto Mode with Pulumi by deploying an example. You can enable EKS Auto Mode by setting autoMode.enabled to true.

First, we’ll create a new EKS cluster with Auto Mode enabled. This requires properly configured networking and authentication settings:

VPC subnets are tagged appropriately for EKS Auto Mode load balancer discovery
Authentication Mode supports Access Entries (Api or ApiAndConfigMap mode) as required by EKS Auto Mode
Default node groups and security groups are skipped as Auto Mode manages these

import * as pulumi from "@pulumi/pulumi";
import * as awsx from "@pulumi/awsx";
import * as eks from "@pulumi/eks";
import * as k8s from "@pulumi/kubernetes";
import { SubnetType } from "@pulumi/awsx/ec2";

const config = new pulumi.Config();
const clusterName = config.require("clusterName");

const eksVpc = new awsx.ec2.Vpc("eks-auto-mode", {
 enableDnsHostnames: true,
 cidrBlock: "10.0.0.0/16",
 subnetSpecs: [
 // Necessary tags for EKS Auto Mode to identify the subnets for the load balancers.
 // See: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/deploy/subnet_discovery/
 {type: SubnetType.Public, tags: {[`kubernetes.io/cluster/${clusterName}`]: "shared", "kubernetes.io/role/elb": "1"}},
 {type: SubnetType.Private, tags: {[`kubernetes.io/cluster/${clusterName}`]: "shared", "kubernetes.io/role/internal-elb": "1"}},
 ],
 subnetStrategy: "Auto"
});

const cluster = new eks.Cluster("eks-auto-mode", {
 name: clusterName,
 // EKS Auto Mode requires Access Entries, use either the `Api` or `ApiAndConfigMap` authentication mode.
 authenticationMode: eks.AuthenticationMode.Api,
 vpcId: eksVpc.vpcId,
 publicSubnetIds: eksVpc.publicSubnetIds,
 privateSubnetIds: eksVpc.privateSubnetIds,
 // Enables compute, storage and load balancing for the cluster.
 autoMode: {
 enabled: true,
 }
});

import pulumi
import pulumi_awsx as awsx
import pulumi_eks as eks
import pulumi_kubernetes as kubernetes

eks_vpc = awsx.ec2.Vpc("eksVpc",
 enable_dns_hostnames=True,
 cidr_block="10.0.0.0/16",
 # Necessary tags for EKS Auto Mode to identify the subnets for the load balancers.
 # See: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/deploy/subnet_discovery/
 subnet_specs=[
 {
 "type": awsx.ec2.SubnetType.PUBLIC,
 "tags": {
 "kubernetes_io_cluster_eks_auto_mode_demo": "shared",
 "kubernetes_io_role_elb": "1",
 },
 },
 {
 "type": awsx.ec2.SubnetType.PRIVATE,
 "tags": {
 "kubernetes_io_cluster_eks_auto_mode_demo": "shared",
 "kubernetes_io_role_internal_elb": "1",
 },
 },
 ],
 subnet_strategy=awsx.ec2.SubnetAllocationStrategy.AUTO)
cluster = eks.Cluster("cluster",
 name="eks-auto-mode-demo",
 # EKS Auto Mode requires Access Entries, use either the `Api` or `ApiAndConfigMap` authentication mode.
 authentication_mode=eks.AuthenticationMode.API,
 vpc_id=eks_vpc.vpc_id,
 public_subnet_ids=eks_vpc.public_subnet_ids,
 private_subnet_ids=eks_vpc.private_subnet_ids,
 # Enables compute, storage and load balancing for the cluster.
 auto_mode={
 "enabled": True,
 })

package main

import (
 "github.com/pulumi/pulumi-awsx/sdk/v2/go/awsx/ec2"
 "github.com/pulumi/pulumi-eks/sdk/v3/go/eks"
 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes"
 appsv1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/apps/v1"
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/core/v1"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/meta/v1"
 networkingv1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/networking/v1"
 storagev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/storage/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 eksVpc, err := ec2.NewVpc(ctx, "eksVpc", &ec2.VpcArgs{
 EnableDnsHostnames: pulumi.Bool(true),
 CidrBlock: "10.0.0.0/16",
 // Necessary tags for EKS Auto Mode to identify the subnets for the load balancers.
 // See: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/deploy/subnet_discovery/
 SubnetSpecs: []ec2.SubnetSpecArgs{
 {
 Type: ec2.SubnetTypePublic,
 Tags: {
 "kubernetes.io/cluster/eks-auto-mode-demo": pulumi.String("shared"),
 "kubernetes.io/role/elb": pulumi.String("1"),
 },
 },
 {
 Type: ec2.SubnetTypePrivate,
 Tags: {
 "kubernetes.io/cluster/eks-auto-mode-demo": pulumi.String("shared"),
 "kubernetes.io/role/internal-elb": pulumi.String("1"),
 },
 },
 },
 SubnetStrategy: ec2.SubnetAllocationStrategyAuto,
 })
 if err != nil {
 return err
 }
 cluster, err := eks.NewCluster(ctx, "cluster", &eks.ClusterArgs{
 Name: pulumi.String("eks-auto-mode-demo"),
 // EKS Auto Mode requires Access Entries, use either the `Api` or `ApiAndConfigMap` authentication mode.
 AuthenticationMode: eks.AuthenticationModeApi,
 VpcId: eksVpc.VpcId,
 PublicSubnetIds: eksVpc.PublicSubnetIds,
 PrivateSubnetIds: eksVpc.PrivateSubnetIds,
 // Enables compute, storage and load balancing for the cluster.
 AutoMode: &eks.AutoModeOptionsArgs{
 Enabled: true,
 },
 })
 if err != nil {
 return err
 }
 return nil
 })
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Awsx = Pulumi.Awsx;
using Eks = Pulumi.Eks;
using Kubernetes = Pulumi.Kubernetes;

return await Deployment.RunAsync(() =>
{
 var eksVpc = new Awsx.Ec2.Vpc("eksVpc", new()
 {
 EnableDnsHostnames = true,
 CidrBlock = "10.0.0.0/16",
 // Necessary tags for EKS Auto Mode to identify the subnets for the load balancers.
 // See: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/deploy/subnet_discovery/
 SubnetSpecs = new()
 {
 new Awsx.Ec2.Inputs.SubnetSpecArgs
 {
 Type = Awsx.Ec2.SubnetType.Public,
 Tags =
 {
 { "kubernetes.io/cluster/eks-auto-mode-demo", "shared" },
 { "kubernetes.io/role/elb", "1" },
 },
 },
 new Awsx.Ec2.Inputs.SubnetSpecArgs
 {
 Type = Awsx.Ec2.SubnetType.Private,
 Tags =
 {
 { "kubernetes.io/cluster/eks-auto-mode-demo", "shared" },
 { "kubernetes.io/role/internal-elb", "1" },
 },
 },
 },
 SubnetStrategy = Awsx.Ec2.SubnetAllocationStrategy.Auto,
 });

 var cluster = new Eks.Cluster("cluster", new()
 {
 Name = "eks-auto-mode-demo",
 // EKS Auto Mode requires Access Entries, use either the `Api` or `ApiAndConfigMap` authentication mode.
 AuthenticationMode = Eks.AuthenticationMode.Api,
 VpcId = eksVpc.VpcId,
 PublicSubnetIds = eksVpc.PublicSubnetIds,
 PrivateSubnetIds = eksVpc.PrivateSubnetIds,
 // Enables compute, storage and load balancing for the cluster.
 AutoMode = new Eks.Inputs.AutoModeOptionsArgs
 {
 Enabled = true,
 },
 });
});

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.awsx.ec2.Vpc;
import com.pulumi.awsx.ec2.VpcArgs;
import com.pulumi.awsx.ec2.inputs.SubnetSpecArgs;
import com.pulumi.eks.Cluster;
import com.pulumi.eks.ClusterArgs;
import com.pulumi.eks.inputs.AutoModeOptionsArgs;
import com.pulumi.resources.CustomResourceOptions;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 var eksVpc = new Vpc("eksVpc", VpcArgs.builder()
 .enableDnsHostnames(true)
 .cidrBlock("10.0.0.0/16")
 // Necessary tags for EKS Auto Mode to identify the subnets for the load balancers.
 // See: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/deploy/subnet_discovery/
 .subnetSpecs(
 SubnetSpecArgs.builder()
 .type("Public")
 .tags(Map.ofEntries(
 Map.entry("kubernetes.io/cluster/eks-auto-mode-demo", "shared"),
 Map.entry("kubernetes.io/role/elb", "1")
 ))
 .build(),
 SubnetSpecArgs.builder()
 .type("Private")
 .tags(Map.ofEntries(
 Map.entry("kubernetes.io/cluster/eks-auto-mode-demo", "shared"),
 Map.entry("kubernetes.io/role/internal-elb", "1")
 ))
 .build())
 .subnetStrategy("Auto")
 .build());

 var cluster = new Cluster("cluster", ClusterArgs.builder()
 .name("eks-auto-mode-demo")
 // EKS Auto Mode requires Access Entries, use either the `Api` or `ApiAndConfigMap` authentication mode.
 .authenticationMode("API")
 .vpcId(eksVpc.vpcId())
 .publicSubnetIds(eksVpc.publicSubnetIds())
 .privateSubnetIds(eksVpc.privateSubnetIds())
 // Enables compute, storage and load balancing for the cluster.
 .autoMode(AutoModeOptionsArgs.builder()
 .enabled(true)
 .build())
 .build());
 }
}

name: eks-auto-mode-demo
description: A demo of EKS Auto Mode
runtime: yaml
resources:
 eksVpc:
 type: awsx:ec2:Vpc
 properties:
 enableDnsHostnames: true
 cidrBlock: "10.0.0.0/16"
 # Necessary tags for EKS Auto Mode to identify the subnets for the load balancers.
 # See: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/deploy/subnet_discovery/
 subnetSpecs:
 - type: "Public"
 tags:
 kubernetes.io/cluster/eks-auto-mode-demo: shared
 kubernetes.io/role/elb: "1"
 - type: "Private"
 tags:
 kubernetes.io/cluster/eks-auto-mode-demo: shared
 kubernetes.io/role/internal-elb: "1"
 subnetStrategy: "Auto"
 cluster:
 type: eks:Cluster
 properties:
 name: eks-auto-mode-demo
 # EKS Auto Mode requires Access Entries, use either the `Api` or `ApiAndConfigMap` authentication mode.
 authenticationMode: API
 vpcId: ${eksVpc.vpcId}
 publicSubnetIds: ${eksVpc.publicSubnetIds}
 privateSubnetIds: ${eksVpc.privateSubnetIds}
 # Enables compute, storage and load balancing for the cluster.
 autoMode:
 enabled: true

Deploying a Load-Balanced Application

Now that we have provisioned an EKS cluster with Auto Mode enabled, let’s deploy a web application fronted by an Application Load Balancer. EKS Auto Mode will automatically provision appropriately sized nodes for your workload, set up the Application Load Balancer according to the Ingress configuration, and manage ongoing maintenance and updates.

//...
const appName = "nginx";
const ns = new k8s.core.v1.Namespace(appName, {
 metadata: { name: appName },
}, { provider: cluster.provider });

const configMap = new k8s.core.v1.ConfigMap(appName, {
 metadata: {
 namespace: ns.metadata.name,
 },
 data: {
 "index.html": "<html><body><h1>Hello, Pulumi!</h1></body></html>",
 },
}, { provider: cluster.provider });

const deployment = new k8s.apps.v1.Deployment(appName, {
 metadata: {
 namespace: ns.metadata.name
 },
 spec: {
 selector: { matchLabels: { app: appName } },
 replicas: 3,
 template: {
 metadata: { labels: { app: appName } },
 spec: {
 containers: [{
 name: appName,
 image: appName,
 ports: [{ containerPort: 80 }],
 volumeMounts: [{ name: "nginx-index", mountPath: "/usr/share/nginx/html" }],
 }],
 volumes: [{
 name: "nginx-index",
 configMap: { name: configMap.metadata.name },
 }],
 },
 },
 },
}, { provider: cluster.provider });

const service = new k8s.core.v1.Service(appName, {
 metadata: {
 name: appName,
 namespace: ns.metadata.name
 },
 spec: {
 selector: { app: appName },
 ports: [{ port: 80, targetPort: 80 }],
 },
}, { provider: cluster.provider, dependsOn: [deployment] });

const ingressClass = new k8s.networking.v1.IngressClass("alb", {
 metadata: {
 namespace: ns.metadata.name,
 labels: {
 "app.kubernetes.io/name": "LoadBalancerController",
 },
 name: "alb",
 },
 spec: {
 controller: "eks.amazonaws.com/alb",
 }
}, { provider: cluster.provider });

const ingress = new k8s.networking.v1.Ingress(appName, {
 metadata: {
 namespace: ns.metadata.name,
 // Annotations for EKS Auto Mode to identify the Ingress as internet-facing and target-type as IP.
 annotations: {
 "alb.ingress.kubernetes.io/scheme": "internet-facing",
 "alb.ingress.kubernetes.io/target-type": "ip",
 }
 },
 spec: {
 ingressClassName: ingressClass.metadata.name,
 rules: [{
 http: {
 paths: [{
 path: "/",
 pathType: "Prefix",
 backend: {
 service: {
 name: service.metadata.name,
 port: {
 number: 80,
 },
 },
 },
 }],
 },
 }],
 }
}, { provider: cluster.provider });

export const url = ingress.status.apply(status => status?.loadBalancer?.ingress?.[0]?.hostname);

# ...
k8s_provider = kubernetes.Provider("k8sProvider", kubeconfig=cluster.kubeconfig)
ns = kubernetes.core.v1.Namespace("ns", metadata={
 "name": "nginx",
},
opts = pulumi.ResourceOptions(provider=k8s_provider))
index_content = kubernetes.core.v1.ConfigMap("index-content",
 metadata={
 "namespace": ns.metadata.name,
 },
 data={
 "index.html": "<html><body><h1>Hello, Pulumi!</h1></body></html>",
 },
 opts = pulumi.ResourceOptions(provider=k8s_provider))
deployment = kubernetes.apps.v1.Deployment("deployment",
 metadata={
 "namespace": ns.metadata.name,
 },
 spec={
 "selector": {
 "match_labels": {
 "app": "nginx",
 },
 },
 "replicas": 3,
 "template": {
 "metadata": {
 "labels": {
 "app": "nginx",
 },
 },
 "spec": {
 "containers": [{
 "name": "nginx",
 "image": "nginx",
 "ports": [{
 "container_port": 80,
 }],
 "volume_mounts": [{
 "name": "nginx-index",
 "mount_path": "/usr/share/nginx/html",
 }],
 }],
 "volumes": [{
 "name": "nginx-index",
 "config_map": {
 "name": index_content.metadata.name,
 },
 }],
 },
 },
 },
 opts = pulumi.ResourceOptions(provider=k8s_provider))
service = kubernetes.core.v1.Service("service",
 metadata={
 "name": "nginx",
 "namespace": ns.metadata.name,
 },
 spec={
 "selector": {
 "app": "nginx",
 },
 "ports": [{
 "port": 80,
 "target_port": 80,
 }],
 },
 opts = pulumi.ResourceOptions(provider=k8s_provider,
 depends_on=[deployment]))
ingress_class = kubernetes.networking.v1.IngressClass("ingressClass",
 metadata={
 "namespace": ns.metadata.name,
 "labels": {
 "app_kubernetes_io_name": "LoadBalancerController",
 },
 "name": "alb",
 },
 spec={
 "controller": "eks.amazonaws.com/alb",
 },
 opts = pulumi.ResourceOptions(provider=k8s_provider))
ingress = kubernetes.networking.v1.Ingress("ingress",
 metadata={
 "namespace": ns.metadata.name,
 "annotations": {
 "alb_ingress_kubernetes_io_scheme": "internet-facing",
 "alb_ingress_kubernetes_io_target_type": "ip",
 },
 },
 spec={
 "ingress_class_name": ingress_class.metadata.name,
 "rules": [{
 "http": {
 "paths": [{
 "path": "/",
 "path_type": "Prefix",
 "backend": {
 "service": {
 "name": service.metadata.name,
 "port": {
 "number": 80,
 },
 },
 },
 }],
 },
 }],
 },
 opts = pulumi.ResourceOptions(provider=k8s_provider))

pulumi.export("url", ingress.status.load_balancer.ingress[0].hostname)

package main

import (
 "github.com/pulumi/pulumi-awsx/sdk/v2/go/awsx/ec2"
 "github.com/pulumi/pulumi-eks/sdk/v3/go/eks"
 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes"
 appsv1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/apps/v1"
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/core/v1"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/meta/v1"
 networkingv1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/networking/v1"
 storagev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/storage/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // ...
 k8sProvider, err := kubernetes.NewProvider(ctx, "k8sProvider", &kubernetes.ProviderArgs{
 Kubeconfig: cluster.Kubeconfig,
 })
 if err != nil {
 return err
 }
 ns, err := corev1.NewNamespace(ctx, "ns", &corev1.NamespaceArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("nginx"),
 },
 }, pulumi.Provider(k8sProvider))
 if err != nil {
 return err
 }
 indexContent, err := corev1.NewConfigMap(ctx, "index-content", &corev1.ConfigMapArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 Data: pulumi.StringMap{
 "index.html": pulumi.String("<html><body><h1>Hello, Pulumi!</h1></body></html>"),
 },
 }, pulumi.Provider(k8sProvider))
 if err != nil {
 return err
 }
 deployment, err := appsv1.NewDeployment(ctx, "deployment", &appsv1.DeploymentArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 Spec: &appsv1.DeploymentSpecArgs{
 Selector: &metav1.LabelSelectorArgs{
 MatchLabels: pulumi.StringMap{
 "app": pulumi.String("nginx"),
 },
 },
 Replicas: pulumi.Int(3),
 Template: &corev1.PodTemplateSpecArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Labels: pulumi.StringMap{
 "app": pulumi.String("nginx"),
 },
 },
 Spec: &corev1.PodSpecArgs{
 Containers: corev1.ContainerArray{
 &corev1.ContainerArgs{
 Name: pulumi.String("nginx"),
 Image: pulumi.String("nginx"),
 Ports: corev1.ContainerPortArray{
 &corev1.ContainerPortArgs{
 ContainerPort: pulumi.Int(80),
 },
 },
 VolumeMounts: corev1.VolumeMountArray{
 &corev1.VolumeMountArgs{
 Name: pulumi.String("nginx-index"),
 MountPath: pulumi.String("/usr/share/nginx/html"),
 },
 },
 },
 },
 Volumes: corev1.VolumeArray{
 &corev1.VolumeArgs{
 Name: pulumi.String("nginx-index"),
 ConfigMap: &corev1.ConfigMapVolumeSourceArgs{
 Name: indexContent.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 },
 },
 },
 },
 },
 }, pulumi.Provider(k8sProvider))
 if err != nil {
 return err
 }
 service, err := corev1.NewService(ctx, "service", &corev1.ServiceArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("nginx"),
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 Spec: &corev1.ServiceSpecArgs{
 Selector: pulumi.StringMap{
 "app": pulumi.String("nginx"),
 },
 Ports: corev1.ServicePortArray{
 &corev1.ServicePortArgs{
 Port: pulumi.Int(80),
 TargetPort: pulumi.Any(80),
 },
 },
 },
 }, pulumi.Provider(k8sProvider), pulumi.DependsOn([]pulumi.Resource{
 deployment,
 }))
 if err != nil {
 return err
 }
 ingressClass, err := networkingv1.NewIngressClass(ctx, "ingressClass", &networkingv1.IngressClassArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 Labels: pulumi.StringMap{
 "app.kubernetes.io/name": pulumi.String("LoadBalancerController"),
 },
 Name: pulumi.String("alb"),
 },
 Spec: &networkingv1.IngressClassSpecArgs{
 Controller: pulumi.String("eks.amazonaws.com/alb"),
 },
 }, pulumi.Provider(k8sProvider))
 if err != nil {
 return err
 }
 ingress, err := networkingv1.NewIngress(ctx, "ingress", &networkingv1.IngressArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 Annotations: pulumi.StringMap{
 "alb.ingress.kubernetes.io/scheme": pulumi.String("internet-facing"),
 "alb.ingress.kubernetes.io/target-type": pulumi.String("ip"),
 },
 },
 Spec: &networkingv1.IngressSpecArgs{
 IngressClassName: ingressClass.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 Rules: networkingv1.IngressRuleArray{
 &networkingv1.IngressRuleArgs{
 Http: &networkingv1.HTTPIngressRuleValueArgs{
 Paths: networkingv1.HTTPIngressPathArray{
 &networkingv1.HTTPIngressPathArgs{
 Path: pulumi.String("/"),
 PathType: pulumi.String("Prefix"),
 Backend: &networkingv1.IngressBackendArgs{
 Service: &networkingv1.IngressServiceBackendArgs{
 Name: service.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 Port: &networkingv1.ServiceBackendPortArgs{
 Number: pulumi.Int(80),
 },
 },
 },
 },
 },
 },
 },
 },
 },
 }, pulumi.Provider(k8sProvider))
 if err != nil {
 return err
 }

 ctx.Export("url", ingress.Status.ApplyT(func(status networkingv1.IngressStatus) (*string, error) {
 return &status.LoadBalancer.Ingress[0].Hostname, nil
 }).(pulumi.StringPtrOutput))
 return nil
 })
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Awsx = Pulumi.Awsx;
using Eks = Pulumi.Eks;
using Kubernetes = Pulumi.Kubernetes;

return await Deployment.RunAsync(() =>
{
 // ...
 var k8sProvider = new Kubernetes.Provider.Provider("k8sProvider", new()
 {
 KubeConfig = cluster.Kubeconfig,
 });

 var ns = new Kubernetes.Core.V1.Namespace("ns", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "nginx",
 },
 }, new CustomResourceOptions
 {
 Provider = k8sProvider,
 });

 var indexContent = new Kubernetes.Core.V1.ConfigMap("index-content", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 },
 Data =
 {
 { "index.html", "<html><body><h1>Hello, Pulumi!</h1></body></html>" },
 },
 }, new CustomResourceOptions
 {
 Provider = k8sProvider,
 });

 var deployment = new Kubernetes.Apps.V1.Deployment("deployment", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 },
 Spec = new Kubernetes.Types.Inputs.Apps.V1.DeploymentSpecArgs
 {
 Selector = new Kubernetes.Types.Inputs.Meta.V1.LabelSelectorArgs
 {
 MatchLabels =
 {
 { "app", "nginx" },
 },
 },
 Replicas = 3,
 Template = new Kubernetes.Types.Inputs.Core.V1.PodTemplateSpecArgs
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Labels =
 {
 { "app", "nginx" },
 },
 },
 Spec = new Kubernetes.Types.Inputs.Core.V1.PodSpecArgs
 {
 Containers = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.ContainerArgs
 {
 Name = "nginx",
 Image = "nginx",
 Ports = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.ContainerPortArgs
 {
 ContainerPortValue = 80,
 },
 },
 VolumeMounts = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.VolumeMountArgs
 {
 Name = "nginx-index",
 MountPath = "/usr/share/nginx/html",
 },
 },
 },
 },
 Volumes = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.VolumeArgs
 {
 Name = "nginx-index",
 ConfigMap = new Kubernetes.Types.Inputs.Core.V1.ConfigMapVolumeSourceArgs
 {
 Name = indexContent.Metadata.Apply(metadata => metadata.Name),
 },
 },
 },
 },
 },
 },
 }, new CustomResourceOptions
 {
 Provider = k8sProvider,
 });

 var service = new Kubernetes.Core.V1.Service("service", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "nginx",
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 },
 Spec = new Kubernetes.Types.Inputs.Core.V1.ServiceSpecArgs
 {
 Selector =
 {
 { "app", "nginx" },
 },
 Ports = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.ServicePortArgs
 {
 Port = 80,
 TargetPort = 80,
 },
 },
 },
 }, new CustomResourceOptions
 {
 Provider = k8sProvider,
 DependsOn =
 {
 deployment,
 },
 });

 var ingressClass = new Kubernetes.Networking.V1.IngressClass("ingressClass", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 Labels =
 {
 { "app.kubernetes.io/name", "LoadBalancerController" },
 },
 Name = "alb",
 },
 Spec = new Kubernetes.Types.Inputs.Networking.V1.IngressClassSpecArgs
 {
 Controller = "eks.amazonaws.com/alb",
 },
 }, new CustomResourceOptions
 {
 Provider = k8sProvider,
 });

 var ingress = new Kubernetes.Networking.V1.Ingress("ingress", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 Annotations =
 {
 { "alb.ingress.kubernetes.io/scheme", "internet-facing" },
 { "alb.ingress.kubernetes.io/target-type", "ip" },
 },
 },
 Spec = new Kubernetes.Types.Inputs.Networking.V1.IngressSpecArgs
 {
 IngressClassName = ingressClass.Metadata.Apply(metadata => metadata.Name),
 Rules = new[]
 {
 new Kubernetes.Types.Inputs.Networking.V1.IngressRuleArgs
 {
 Http = new Kubernetes.Types.Inputs.Networking.V1.HTTPIngressRuleValueArgs
 {
 Paths = new[]
 {
 new Kubernetes.Types.Inputs.Networking.V1.HTTPIngressPathArgs
 {
 Path = "/",
 PathType = "Prefix",
 Backend = new Kubernetes.Types.Inputs.Networking.V1.IngressBackendArgs
 {
 Service = new Kubernetes.Types.Inputs.Networking.V1.IngressServiceBackendArgs
 {
 Name = service.Metadata.Apply(metadata => metadata.Name),
 Port = new Kubernetes.Types.Inputs.Networking.V1.ServiceBackendPortArgs
 {
 Number = 80,
 },
 },
 },
 },
 },
 },
 },
 },
 },
 }, new CustomResourceOptions
 {
 Provider = k8sProvider,
 });

 return new Dictionary<string, object?>
 {
 ["url"] = ingress.Status.Apply(status => status?.LoadBalancer?.Ingress[0]?.Hostname),
 };
});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.awsx.ec2.Vpc;
import com.pulumi.awsx.ec2.VpcArgs;
import com.pulumi.awsx.ec2.inputs.SubnetSpecArgs;
import com.pulumi.eks.Cluster;
import com.pulumi.eks.ClusterArgs;
import com.pulumi.eks.inputs.AutoModeOptionsArgs;
import com.pulumi.kubernetes.Provider;
import com.pulumi.kubernetes.ProviderArgs;
import com.pulumi.kubernetes.core_v1.Namespace;
import com.pulumi.kubernetes.core_v1.NamespaceArgs;
import com.pulumi.kubernetes.meta_v1.inputs.ObjectMetaArgs;
import com.pulumi.kubernetes.core_v1.ConfigMap;
import com.pulumi.kubernetes.core_v1.ConfigMapArgs;
import com.pulumi.kubernetes.apps_v1.Deployment;
import com.pulumi.kubernetes.apps_v1.DeploymentArgs;
import com.pulumi.kubernetes.apps_v1.inputs.DeploymentSpecArgs;
import com.pulumi.kubernetes.meta_v1.inputs.LabelSelectorArgs;
import com.pulumi.kubernetes.core_v1.inputs.PodTemplateSpecArgs;
import com.pulumi.kubernetes.core_v1.inputs.PodSpecArgs;
import com.pulumi.kubernetes.core_v1.Service;
import com.pulumi.kubernetes.core_v1.ServiceArgs;
import com.pulumi.kubernetes.core_v1.inputs.ServiceSpecArgs;
import com.pulumi.kubernetes.networking.k8s.io_v1.IngressClass;
import com.pulumi.kubernetes.networking.k8s.io_v1.IngressClassArgs;
import com.pulumi.kubernetes.networking.k8s.io_v1.inputs.IngressClassSpecArgs;
import com.pulumi.kubernetes.networking.k8s.io_v1.Ingress;
import com.pulumi.kubernetes.networking.k8s.io_v1.IngressArgs;
import com.pulumi.kubernetes.networking.k8s.io_v1.inputs.IngressSpecArgs;
import com.pulumi.kubernetes.storage.k8s.io_v1.StorageClass;
import com.pulumi.kubernetes.storage.k8s.io_v1.StorageClassArgs;
import com.pulumi.kubernetes.core_v1.PersistentVolumeClaim;
import com.pulumi.kubernetes.core_v1.PersistentVolumeClaimArgs;
import com.pulumi.kubernetes.core_v1.inputs.PersistentVolumeClaimSpecArgs;
import com.pulumi.kubernetes.core_v1.inputs.VolumeResourceRequirementsArgs;
import com.pulumi.resources.CustomResourceOptions;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 // ...
 var k8sProvider = new Provider("k8sProvider", ProviderArgs.builder()
 .kubeconfig(cluster.kubeconfig())
 .build());

 var ns = new Namespace("ns", NamespaceArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("nginx")
 .build())
 .build(), CustomResourceOptions.builder()
 .provider(k8sProvider)
 .build());

 var indexContent = new ConfigMap("indexContent", ConfigMapArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .namespace(ns.metadata().applyValue(metadata -> metadata.name()))
 .build())
 .data(Map.of("index.html", "<html><body><h1>Hello, Pulumi!</h1></body></html>"))
 .build(), CustomResourceOptions.builder()
 .provider(k8sProvider)
 .build());

 var deployment = new Deployment("deployment", DeploymentArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .namespace(ns.metadata().applyValue(metadata -> metadata.name()))
 .build())
 .spec(DeploymentSpecArgs.builder()
 .selector(LabelSelectorArgs.builder()
 .matchLabels(Map.of("app", "nginx"))
 .build())
 .replicas(3)
 .template(PodTemplateSpecArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .labels(Map.of("app", "nginx"))
 .build())
 .spec(PodSpecArgs.builder()
 .containers(ContainerArgs.builder()
 .name("nginx")
 .image("nginx")
 .ports(ContainerPortArgs.builder()
 .containerPort(80)
 .build())
 .volumeMounts(VolumeMountArgs.builder()
 .name("nginx-index")
 .mountPath("/usr/share/nginx/html")
 .build())
 .build())
 .volumes(VolumeArgs.builder()
 .name("nginx-index")
 .configMap(ConfigMapVolumeSourceArgs.builder()
 .name(indexContent.metadata().applyValue(metadata -> metadata.name()))
 .build())
 .build())
 .build())
 .build())
 .build())
 .build(), CustomResourceOptions.builder()
 .provider(k8sProvider)
 .build());

 var service = new Service("service", ServiceArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("nginx")
 .namespace(ns.metadata().applyValue(metadata -> metadata.name()))
 .build())
 .spec(ServiceSpecArgs.builder()
 .selector(Map.of("app", "nginx"))
 .ports(ServicePortArgs.builder()
 .port(80)
 .targetPort(80)
 .build())
 .build())
 .build(), CustomResourceOptions.builder()
 .provider(k8sProvider)
 .dependsOn(deployment)
 .build());

 var ingressClass = new IngressClass("ingressClass", IngressClassArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .namespace(ns.metadata().applyValue(metadata -> metadata.name()))
 .labels(Map.of("app.kubernetes.io/name", "LoadBalancerController"))
 .name("alb")
 .build())
 .spec(IngressClassSpecArgs.builder()
 .controller("eks.amazonaws.com/alb")
 .build())
 .build(), CustomResourceOptions.builder()
 .provider(k8sProvider)
 .build());

 var ingress = new Ingress("ingress", IngressArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .namespace(ns.metadata().applyValue(metadata -> metadata.name()))
 .annotations(Map.ofEntries(
 Map.entry("alb.ingress.kubernetes.io/scheme", "internet-facing"),
 Map.entry("alb.ingress.kubernetes.io/target-type", "ip")
 ))
 .build())
 .spec(IngressSpecArgs.builder()
 .ingressClassName(ingressClass.metadata().applyValue(metadata -> metadata.name()))
 .rules(IngressRuleArgs.builder()
 .http(HTTPIngressRuleValueArgs.builder()
 .paths(HTTPIngressPathArgs.builder()
 .path("/")
 .pathType("Prefix")
 .backend(IngressBackendArgs.builder()
 .service(IngressServiceBackendArgs.builder()
 .name(service.metadata().applyValue(metadata -> metadata.name()))
 .port(ServiceBackendPortArgs.builder()
 .number(80)
 .build())
 .build())
 .build())
 .build())
 .build())
 .build())
 .build())
 .build(), CustomResourceOptions.builder()
 .provider(k8sProvider)
 .build());

 ctx.export("url", ingress.status().applyValue(status -> status.loadBalancer().ingress()[0].hostname()));
 }
}

name: eks-auto-mode-yaml
description: A Pulumi YAML program to deploy a Kubernetes cluster on AWS
runtime: yaml
resources:
 # ...
 k8sProvider:
 type: pulumi:providers:kubernetes
 properties:
 kubeconfig: ${cluster.kubeconfig}
 ns:
 type: kubernetes:core/v1:Namespace
 properties:
 metadata:
 name: nginx
 options:
 provider: ${k8sProvider}
 index-content:
 type: kubernetes:core/v1:ConfigMap
 properties:
 metadata:
 namespace: ${ns.metadata.name}
 data:
 "index.html": "<html><body><h1>Hello, Pulumi!</h1></body></html>"
 options:
 provider: ${k8sProvider}
 deployment:
 type: kubernetes:apps/v1:Deployment
 properties:
 metadata:
 namespace: ${ns.metadata.name}
 spec:
 selector:
 matchLabels:
 app: nginx
 replicas: 3
 template:
 metadata:
 labels:
 app: nginx
 spec:
 containers:
 - name: nginx
 image: nginx
 ports:
 - containerPort: 80
 volumeMounts:
 - name: nginx-index
 mountPath: /usr/share/nginx/html
 volumes:
 - name: nginx-index
 configMap:
 name: ${index-content.metadata.name}
 options:
 provider: ${k8sProvider}
 service:
 type: kubernetes:core/v1:Service
 properties:
 metadata:
 name: nginx
 namespace: ${ns.metadata.name}
 spec:
 selector:
 app: nginx
 ports:
 - port: 80
 targetPort: 80
 options:
 provider: ${k8sProvider}
 dependsOn:
 - ${deployment}
 ingressClass:
 type: kubernetes:networking.k8s.io/v1:IngressClass
 properties:
 metadata:
 namespace: ${ns.metadata.name}
 labels:
 app.kubernetes.io/name: LoadBalancerController
 name: alb
 spec:
 controller: eks.amazonaws.com/alb
 options:
 provider: ${k8sProvider}
 ingress:
 type: kubernetes:networking.k8s.io/v1:Ingress
 properties:
 metadata:
 namespace: ${ns.metadata.name}
 annotations:
 alb.ingress.kubernetes.io/scheme: internet-facing
 alb.ingress.kubernetes.io/target-type: ip
 spec:
 ingressClassName: ${ingressClass.metadata.name}
 rules:
 - http:
 paths:
 - path: /
 pathType: Prefix
 backend:
 service:
 name: ${service.metadata.name}
 port:
 number: 80
 options:
 provider: ${k8sProvider}
outputs:
 url: ${ingress.status.loadBalancer.ingress[0].hostname}

After deploying this with pulumi up, you can access your application with curl once the Load Balancer has been provisioned:

~ curl $(pulumi stack output url)
Hello, Pulumi!

Deploying a Stateful Workload

EKS Auto Mode simplifies persistent storage management by automatically handling EBS volume provisioning for your stateful workloads. To leverage this feature, you can create a StorageClass that uses the EBS CSI driver (ebs.csi.eks.amazonaws.com) as its provisioner. EKS Auto Mode will then automatically manage the lifecycle of your persistent volumes, including provisioning, attaching, and cleanup.

This is how you’d configure persistent storage for a sample application:

// ...
const storageClass = new k8s.storage.v1.StorageClass("storage-class", {
 metadata: {
 name: "auto-ebs-sc",
 annotations: {
 "storageclass.kubernetes.io/is-default-class": "true",
 },
 },
 provisioner: "ebs.csi.eks.amazonaws.com",
 volumeBindingMode: "WaitForFirstConsumer",
 parameters: {
 type: "gp3",
 encrypted: "true",
 },
}, {
 provider: cluster.provider,
});
const pvc = new k8s.core.v1.PersistentVolumeClaim("pvc", {
 metadata: {
 name: "auto-ebs-claim",
 namespace: ns.metadata.apply(metadata => metadata.name),
 },
 spec: {
 accessModes: ["ReadWriteOnce"],
 storageClassName: storageClass.metadata.apply(metadata => metadata.name),
 resources: {
 requests: {
 storage: "8Gi",
 },
 },
 },
}, {
 provider: cluster.provider,
});
const statefulWorkload = new k8s.apps.v1.Deployment("stateful-workload", {
 metadata: {
 name: "inflate-stateful",
 namespace: ns.metadata.apply(metadata => metadata.name),
 },
 spec: {
 replicas: 1,
 selector: {
 matchLabels: {
 app: "inflate-stateful",
 },
 },
 template: {
 metadata: {
 labels: {
 app: "inflate-stateful",
 },
 },
 spec: {
 terminationGracePeriodSeconds: 0,
 nodeSelector: {
 "eks.amazonaws.com/compute-type": "auto",
 },
 containers: [{
 name: "bash",
 image: "public.ecr.aws/docker/library/bash:4.4",
 command: ["/usr/local/bin/bash"],
 args: [
 "-c",
 "while true; do echo $(date -u) >> /data/out.txt; sleep 60; done",
 ],
 resources: {
 requests: {
 cpu: "1",
 },
 },
 volumeMounts: [{
 name: "persistent-storage",
 mountPath: "/data",
 }],
 }],
 volumes: [{
 name: "persistent-storage",
 persistentVolumeClaim: {
 claimName: pvc.metadata.name,
 },
 }],
 },
 },
 },
}, {
 provider: cluster.provider,
});

export const url = ingress.status.loadBalancer.ingress[0].hostname.apply(h => `http://${h}:80`);

# ...
storage_class = kubernetes.storage.v1.StorageClass("storage-class",
 metadata={
 "name": "auto-ebs-sc",
 "annotations": {
 "storageclass_kubernetes_io_is_default_class": "true",
 },
 },
 provisioner="ebs.csi.eks.amazonaws.com",
 volume_binding_mode="WaitForFirstConsumer",
 parameters={
 "type": "gp3",
 "encrypted": "true",
 },
 opts = pulumi.ResourceOptions(provider=k8_s_provider))
pvc = kubernetes.core.v1.PersistentVolumeClaim("pvc",
 metadata={
 "name": "auto-ebs-claim",
 "namespace": ns.metadata.name,
 },
 spec={
 "access_modes": ["ReadWriteOnce"],
 "storage_class_name": storage_class.metadata.name,
 "resources": {
 "requests": {
 "storage": "8Gi",
 },
 },
 },
 opts = pulumi.ResourceOptions(provider=k8_s_provider))
stateful_workload = kubernetes.apps.v1.Deployment("stateful-workload",
 metadata={
 "name": "inflate-stateful",
 "namespace": ns.metadata.name,
 },
 spec={
 "replicas": 1,
 "selector": {
 "match_labels": {
 "app": "inflate-stateful",
 },
 },
 "template": {
 "metadata": {
 "labels": {
 "app": "inflate-stateful",
 },
 },
 "spec": {
 "termination_grace_period_seconds": 0,
 "node_selector": {
 "eks_amazonaws_com_compute_type": "auto",
 },
 "containers": [{
 "name": "bash",
 "image": "public.ecr.aws/docker/library/bash:4.4",
 "command": ["/usr/local/bin/bash"],
 "args": [
 "-c",
 "while true; do echo $(date -u) >> /data/out.txt; sleep 60; done",
 ],
 "resources": {
 "requests": {
 "cpu": "1",
 },
 },
 "volume_mounts": [{
 "name": "persistent-storage",
 "mount_path": "/data",
 }],
 }],
 "volumes": [{
 "name": "persistent-storage",
 "persistent_volume_claim": {
 "claim_name": pvc.metadata.name,
 },
 }],
 },
 },
 },
 opts = pulumi.ResourceOptions(provider=k8_s_provider))

package main

import (
 "github.com/pulumi/pulumi-awsx/sdk/v2/go/awsx/ec2"
 "github.com/pulumi/pulumi-eks/sdk/v3/go/eks"
 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes"
 appsv1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/apps/v1"
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/core/v1"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/meta/v1"
 networkingv1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/networking/v1"
 storagev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/storage/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // ...
 storageClass, err := storagev1.NewStorageClass(ctx, "storage-class", &storagev1.StorageClassArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("auto-ebs-sc"),
 Annotations: pulumi.StringMap{
 "storageclass.kubernetes.io/is-default-class": pulumi.String("true"),
 },
 },
 Provisioner: pulumi.String("ebs.csi.eks.amazonaws.com"),
 VolumeBindingMode: pulumi.String("WaitForFirstConsumer"),
 Parameters: pulumi.StringMap{
 "type": pulumi.String("gp3"),
 "encrypted": pulumi.String("true"),
 },
 }, pulumi.Provider(k8SProvider))
 if err != nil {
 return err
 }
 pvc, err := corev1.NewPersistentVolumeClaim(ctx, "pvc", &corev1.PersistentVolumeClaimArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("auto-ebs-claim"),
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 Spec: &corev1.PersistentVolumeClaimSpecArgs{
 AccessModes: pulumi.StringArray{
 pulumi.String("ReadWriteOnce"),
 },
 StorageClassName: storageClass.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 Resources: &corev1.VolumeResourceRequirementsArgs{
 Requests: pulumi.StringMap{
 "storage": pulumi.String("8Gi"),
 },
 },
 },
 }, pulumi.Provider(k8SProvider))
 if err != nil {
 return err
 }
 _, err = appsv1.NewDeployment(ctx, "stateful-workload", &appsv1.DeploymentArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("inflate-stateful"),
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 Spec: &appsv1.DeploymentSpecArgs{
 Replicas: pulumi.Int(1),
 Selector: &metav1.LabelSelectorArgs{
 MatchLabels: pulumi.StringMap{
 "app": pulumi.String("inflate-stateful"),
 },
 },
 Template: &corev1.PodTemplateSpecArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Labels: pulumi.StringMap{
 "app": pulumi.String("inflate-stateful"),
 },
 },
 Spec: &corev1.PodSpecArgs{
 TerminationGracePeriodSeconds: pulumi.Int(0),
 NodeSelector: pulumi.StringMap{
 "eks.amazonaws.com/compute-type": pulumi.String("auto"),
 },
 Containers: corev1.ContainerArray{
 &corev1.ContainerArgs{
 Name: pulumi.String("bash"),
 Image: pulumi.String("public.ecr.aws/docker/library/bash:4.4"),
 Command: pulumi.StringArray{
 pulumi.String("/usr/local/bin/bash"),
 },
 Args: pulumi.StringArray{
 pulumi.String("-c"),
 pulumi.String("while true; do echo $(date -u) >> /data/out.txt; sleep 60; done"),
 },
 Resources: &corev1.ResourceRequirementsArgs{
 Requests: pulumi.StringMap{
 "cpu": pulumi.String("1"),
 },
 },
 VolumeMounts: corev1.VolumeMountArray{
 &corev1.VolumeMountArgs{
 Name: pulumi.String("persistent-storage"),
 MountPath: pulumi.String("/data"),
 },
 },
 },
 },
 Volumes: corev1.VolumeArray{
 &corev1.VolumeArgs{
 Name: pulumi.String("persistent-storage"),
 PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSourceArgs{
 ClaimName: pvc.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 },
 },
 },
 },
 },
 }, pulumi.Provider(k8SProvider))
 if err != nil {
 return err
 }
 return nil
 })
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Awsx = Pulumi.Awsx;
using Eks = Pulumi.Eks;
using Kubernetes = Pulumi.Kubernetes;

return await Deployment.RunAsync(() =>
{
 // ...
 var storageClass = new Kubernetes.Storage.V1.StorageClass("storage-class", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "auto-ebs-sc",
 Annotations =
 {
 { "storageclass.kubernetes.io/is-default-class", "true" },
 },
 },
 Provisioner = "ebs.csi.eks.amazonaws.com",
 VolumeBindingMode = "WaitForFirstConsumer",
 Parameters =
 {
 { "type", "gp3" },
 { "encrypted", "true" },
 },
 }, new CustomResourceOptions
 {
 Provider = k8SProvider,
 });

 var pvc = new Kubernetes.Core.V1.PersistentVolumeClaim("pvc", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "auto-ebs-claim",
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 },
 Spec = new Kubernetes.Types.Inputs.Core.V1.PersistentVolumeClaimSpecArgs
 {
 AccessModes = new[]
 {
 "ReadWriteOnce",
 },
 StorageClassName = storageClass.Metadata.Apply(metadata => metadata.Name),
 Resources = new Kubernetes.Types.Inputs.Core.V1.VolumeResourceRequirementsArgs
 {
 Requests =
 {
 { "storage", "8Gi" },
 },
 },
 },
 }, new CustomResourceOptions
 {
 Provider = k8SProvider,
 });

 var statefulWorkload = new Kubernetes.Apps.V1.Deployment("stateful-workload", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "inflate-stateful",
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 },
 Spec = new Kubernetes.Types.Inputs.Apps.V1.DeploymentSpecArgs
 {
 Replicas = 1,
 Selector = new Kubernetes.Types.Inputs.Meta.V1.LabelSelectorArgs
 {
 MatchLabels =
 {
 { "app", "inflate-stateful" },
 },
 },
 Template = new Kubernetes.Types.Inputs.Core.V1.PodTemplateSpecArgs
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Labels =
 {
 { "app", "inflate-stateful" },
 },
 },
 Spec = new Kubernetes.Types.Inputs.Core.V1.PodSpecArgs
 {
 TerminationGracePeriodSeconds = 0,
 NodeSelector =
 {
 { "eks.amazonaws.com/compute-type", "auto" },
 },
 Containers = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.ContainerArgs
 {
 Name = "bash",
 Image = "public.ecr.aws/docker/library/bash:4.4",
 Command = new[]
 {
 "/usr/local/bin/bash",
 },
 Args = new[]
 {
 "-c",
 "while true; do echo $(date -u) >> /data/out.txt; sleep 60; done",
 },
 Resources = new Kubernetes.Types.Inputs.Core.V1.ResourceRequirementsArgs
 {
 Requests =
 {
 { "cpu", "1" },
 },
 },
 VolumeMounts = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.VolumeMountArgs
 {
 Name = "persistent-storage",
 MountPath = "/data",
 },
 },
 },
 },
 Volumes = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.VolumeArgs
 {
 Name = "persistent-storage",
 PersistentVolumeClaim = new Kubernetes.Types.Inputs.Core.V1.PersistentVolumeClaimVolumeSourceArgs
 {
 ClaimName = pvc.Metadata.Apply(metadata => metadata.Name),
 },
 },
 },
 },
 },
 },
 }, new CustomResourceOptions
 {
 Provider = k8SProvider,
 });
});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.awsx.ec2.Vpc;
import com.pulumi.awsx.ec2.VpcArgs;
import com.pulumi.awsx.ec2.inputs.SubnetSpecArgs;
import com.pulumi.eks.Cluster;
import com.pulumi.eks.ClusterArgs;
import com.pulumi.eks.inputs.AutoModeOptionsArgs;
import com.pulumi.kubernetes.Provider;
import com.pulumi.kubernetes.ProviderArgs;
import com.pulumi.kubernetes.core_v1.Namespace;
import com.pulumi.kubernetes.core_v1.NamespaceArgs;
import com.pulumi.kubernetes.meta_v1.inputs.ObjectMetaArgs;
import com.pulumi.kubernetes.core_v1.ConfigMap;
import com.pulumi.kubernetes.core_v1.ConfigMapArgs;
import com.pulumi.kubernetes.apps_v1.Deployment;
import com.pulumi.kubernetes.apps_v1.DeploymentArgs;
import com.pulumi.kubernetes.apps_v1.inputs.DeploymentSpecArgs;
import com.pulumi.kubernetes.meta_v1.inputs.LabelSelectorArgs;
import com.pulumi.kubernetes.core_v1.inputs.PodTemplateSpecArgs;
import com.pulumi.kubernetes.core_v1.inputs.PodSpecArgs;
import com.pulumi.kubernetes.core_v1.Service;
import com.pulumi.kubernetes.core_v1.ServiceArgs;
import com.pulumi.kubernetes.core_v1.inputs.ServiceSpecArgs;
import com.pulumi.kubernetes.networking.k8s.io_v1.IngressClass;
import com.pulumi.kubernetes.networking.k8s.io_v1.IngressClassArgs;
import com.pulumi.kubernetes.networking.k8s.io_v1.inputs.IngressClassSpecArgs;
import com.pulumi.kubernetes.networking.k8s.io_v1.Ingress;
import com.pulumi.kubernetes.networking.k8s.io_v1.IngressArgs;
import com.pulumi.kubernetes.networking.k8s.io_v1.inputs.IngressSpecArgs;
import com.pulumi.kubernetes.storage.k8s.io_v1.StorageClass;
import com.pulumi.kubernetes.storage.k8s.io_v1.StorageClassArgs;
import com.pulumi.kubernetes.core_v1.PersistentVolumeClaim;
import com.pulumi.kubernetes.core_v1.PersistentVolumeClaimArgs;
import com.pulumi.kubernetes.core_v1.inputs.PersistentVolumeClaimSpecArgs;
import com.pulumi.kubernetes.core_v1.inputs.VolumeResourceRequirementsArgs;
import com.pulumi.resources.CustomResourceOptions;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 // ...
 var storageClass = new StorageClass("storageClass", StorageClassArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("auto-ebs-sc")
 .annotations(Map.of("storageclass.kubernetes.io/is-default-class", "true"))
 .build())
 .provisioner("ebs.csi.eks.amazonaws.com")
 .volumeBindingMode("WaitForFirstConsumer")
 .parameters(Map.ofEntries(
 Map.entry("type", "gp3"),
 Map.entry("encrypted", "true")
 ))
 .build(), CustomResourceOptions.builder()
 .provider(k8SProvider)
 .build());

 var pvc = new PersistentVolumeClaim("pvc", PersistentVolumeClaimArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("auto-ebs-claim")
 .namespace(ns.metadata().applyValue(metadata -> metadata.name()))
 .build())
 .spec(PersistentVolumeClaimSpecArgs.builder()
 .accessModes("ReadWriteOnce")
 .storageClassName(storageClass.metadata().applyValue(metadata -> metadata.name()))
 .resources(VolumeResourceRequirementsArgs.builder()
 .requests(Map.of("storage", "8Gi"))
 .build())
 .build())
 .build(), CustomResourceOptions.builder()
 .provider(k8SProvider)
 .build());

 var statefulWorkload = new Deployment("statefulWorkload", DeploymentArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("inflate-stateful")
 .namespace(ns.metadata().applyValue(metadata -> metadata.name()))
 .build())
 .spec(DeploymentSpecArgs.builder()
 .replicas(1)
 .selector(LabelSelectorArgs.builder()
 .matchLabels(Map.of("app", "inflate-stateful"))
 .build())
 .template(PodTemplateSpecArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .labels(Map.of("app", "inflate-stateful"))
 .build())
 .spec(PodSpecArgs.builder()
 .terminationGracePeriodSeconds(0)
 .nodeSelector(Map.of("eks.amazonaws.com/compute-type", "auto"))
 .containers(ContainerArgs.builder()
 .name("bash")
 .image("public.ecr.aws/docker/library/bash:4.4")
 .command("/usr/local/bin/bash")
 .args(
 "-c",
 "while true; do echo $(date -u) >> /data/out.txt; sleep 60; done")
 .resources(ResourceRequirementsArgs.builder()
 .requests(Map.of("cpu", "1"))
 .build())
 .volumeMounts(VolumeMountArgs.builder()
 .name("persistent-storage")
 .mountPath("/data")
 .build())
 .build())
 .volumes(VolumeArgs.builder()
 .name("persistent-storage")
 .persistentVolumeClaim(PersistentVolumeClaimVolumeSourceArgs.builder()
 .claimName(pvc.metadata().applyValue(metadata -> metadata.name()))
 .build())
 .build())
 .build())
 .build())
 .build())
 .build(), CustomResourceOptions.builder()
 .provider(k8SProvider)
 .build());
 }
}

name: eks-auto-mode-yaml
description: A Pulumi YAML program to deploy a Kubernetes cluster on AWS
runtime: yaml
resources:
 # ...
 storage-class:
 type: kubernetes:storage.k8s.io/v1:StorageClass
 properties:
 metadata:
 name: auto-ebs-sc
 annotations:
 storageclass.kubernetes.io/is-default-class: "true"
 provisioner: ebs.csi.eks.amazonaws.com
 volumeBindingMode: WaitForFirstConsumer
 parameters:
 type: gp3
 encrypted: "true"
 options:
 provider: ${k8sProvider}
 pvc:
 type: kubernetes:core/v1:PersistentVolumeClaim
 properties:
 metadata:
 name: auto-ebs-claim
 namespace: ${ns.metadata.name}
 spec:
 accessModes:
 - ReadWriteOnce
 storageClassName: ${storage-class.metadata.name}
 resources:
 requests:
 storage: 8Gi
 options:
 provider: ${k8sProvider}
 stateful-workload:
 type: kubernetes:apps/v1:Deployment
 properties:
 metadata:
 name: inflate-stateful
 namespace: ${ns.metadata.name}
 spec:
 replicas: 1
 selector:
 matchLabels:
 app: inflate-stateful
 template:
 metadata:
 labels:
 app: inflate-stateful
 spec:
 terminationGracePeriodSeconds: 0
 nodeSelector:
 eks.amazonaws.com/compute-type: auto
 containers:
 - name: bash
 image: public.ecr.aws/docker/library/bash:4.4
 command: ["/usr/local/bin/bash"]
 args: ["-c", "while true; do echo $(date -u) >> /data/out.txt; sleep 60; done"]
 resources:
 requests:
 cpu: "1"
 volumeMounts:
 - name: persistent-storage
 mountPath: /data
 volumes:
 - name: persistent-storage
 persistentVolumeClaim:
 claimName: ${pvc.metadata.name}
 options:
 provider: ${k8sProvider}
outputs:
 url: ${ingress.status.loadBalancer.ingress[0].hostname}

Summary

EKS Auto Mode represents a significant step forward in simplifying Kubernetes operations on AWS. Pulumi’s support for this feature allows you to define EKS Auto Mode clusters using your favorite programming language and integrate them with your existing infrastructure code.

To get started with EKS Auto Mode in Pulumi:

Update your AWS providers to the latest versions
Check out our EKS documentation
Join our Community Slack for support and discussions

We are excited to see what you build with EKS Auto Mode and Pulumi!

Announcing AWS Systems Manager - Parameter Store Support for Pulumi ESC

Sean Yeh — Mon, 16 Dec 2024 12:00:00 -0600

We are super excited to announce integration support for AWS Systems Manager - Parameter Store within Pulumi Environments, Secrets, and Configuration (ESC). Parameter Store is a popular managed service by AWS for storing and managing secrets and other configuration, and its integration with ESC has been highly requested among the community.

With the AWS Parameter Store ESC Provider, development teams can now import their AWS Parameter Store parameters as environment variables or as Pulumi Config in their workflows, all in a secure and convenient way.

As we continue to expand our roster of integrations, it’s easier now than ever to manage and use secrets and configuration all in one place by using ESC. If you or your team are users of AWS Parameter Store, say goodbye to managing multiple configuration sources and give this provider a try!

How does it work?

Add parameters to AWS Parameter Store.
Use aws-login to authenticate with AWS.
Define a path name, and enter the parameter name you want to retrieve. This will be imported into the path name that you have defined.

AWS Parameter Store provider syntax:

values:
 params:
 fn::open::aws-parameter-store:
 region: us-west-1
 login: ${aws.login}
 get:
 myKey:
 name: /myNamespace/myKey
 myKeyByVersion:
 name: /myNamespace/myKey:1
 myKeyByVersionLabel:
 name: /myNamespace/myKey:stable
 secureKey:
 name: /myNamespace/secureKey
 decrypt: true

Demo

Now that we know how to use the Pulumi ESC Parameter Store provider, let’s use it to pull some parameters!

Setup OpenID Connect for AWS. We will use OIDC to authenticate with AWS.
Store some parameters in Parameter Store. In the demo, we will store a String /test/api-endpoint and a SecureString /test/api-key
Create a new Pulumi ESC Environment with the following definition. Make sure to replace the OIDC roleArn with your own.

values:
 aws:
 login:
 fn::open::aws-login:
 oidc:
 duration: 1h
 roleArn: arn:aws:iam::123456789:role/esc-oidc
 sessionName: pulumi-environments-session
 params:
 fn::open::aws-parameter-store:
 region: us-west-1
 login: ${aws.login}
 get:
 api-endpoint:
 name: /test/api-endpoint
 api-key:
 name: /test/api-key
 decrypt: true
 environmentVariables:
 API_ENDPOINT: ${params.api-endpoint}
 API_KEY: ${params.api-key}

Open the environment to ensure we are able to successfully pull the credentials from AWS Parameter Store
Run pulumi env open <project/environment> --format shell to view the exported environment variables from this ESC environment.
Use pulumi env run to run any command with these environment variables. For example, we can now run pulumi env run <project/environment> -- <your command>

pulumi env run passes the configuration stored under the environmentVariables section for use with the provided command, without ever storing them locally on your machine.

Conclusion

We are excited to expand our roster of integrations with the new AWS Parameter Store ESC Provider, providing developers more flexibility and choices. We look forward to seeing what you will create next using Pulumi ESC!

Visit the AWS Parameter Store Provider Page for more detailed documentation on this provider.
Visit our Pulumi ESC docs to learn more about Pulumi ESC and its supported integrations

As always, we deeply value your insights. Your feedback is instrumental in helping us refine and enhance our solutions to better align with your needs.

Pulumi for AWS: Automate, Secure, and Manage Your Cloud

Aaron Kao — Mon, 02 Dec 2024 09:00:00 +0000

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

Pulumi is excited to be at AWS re:Invent this week, where we’re showcasing our broad and deep support for AWS across all our products. From automating infrastructure with Pulumi IaC to securing secrets with Pulumi ESC to managing cloud assets with Pulumi Insights, Pulumi makes AWS a competitive advantage. Whether you’re a developer, DevOps pro, or platform engineer, Pulumi delivers the tools you need to build and manage modern cloud applications with ease.

Stop by the Pulumi re:Invent booth #370 this week to chat with experts on the Pulumi team. If you can’t make it to re:Invent, join our workshop, Accelerating Platform Engineering with Pulumi on AWS, on December 11, 2024, to see how Pulumi can enhance your cloud operations on AWS.

Why Pulumi for AWS?

Pulumi empowers your organization to automate AWS cloud infrastructure through code, tame secrets sprawl through centralized secrets management, and manage cloud assets and compliance with the help of AI. Pulumi encourages infrastructure, platform, development, DevOps, and security teams to collaborate and accelerates time to market with greater control and minimized risk.

Pulumi will speed up your deployments and time to value, providing you with an approach to open source and community that gives stability and choice. It will also increase your productivity and velocity through better tooling. Lastly, Pulumi uses software engineering to tackle the scale of an infinite combination of cloud architectures. Read more about why engineers and developers switch to Pulumi.

Pulumi for AWS Over the Years

Pulumi was founded back in 2017 to tackle the mounting challenges of modern cloud applications, which have grown increasingly intricate, spanning multiple environments and relying on numerous cloud resources and SaaS services. This complexity creates significant challenges for managing infrastructure. Legacy tools are designed for “two virtual machines with a database” and not thousands or millions of resources across many clouds and accounts. Teams waste time wrestling with excessive YAML configurations, resorting to copy-paste solutions, or using limited domain-specific languages (DSLs) that fail to scale effectively.

The fragmentation between development, infrastructure, and security teams only compounds these challenges. Pulumi’s founding vision addresses these pain points by enabling teams to build and manage cloud infrastructure through infrastructure as code using familiar programming languages and tools. Pulumi Infrastructure as Code (IaC) was launched with strong support of the AWS cloud. We wanted to be the best way for you to manage and scale your AWS infrastructure through code. Over the years, we launched many AWS-specific features to simplify how to manage AWS infrastructure at scale.

Well-Architected AWS Best Practices as a Component

One of the first innovations made for AWS was Pulumi Crosswalk for AWS, a collection of libraries that automatically use well-architected best practices to make common infrastructure-as-code tasks in AWS easier and more secure. Released in 2019, Crosswalk for AWS supports “day one” tasks, such as creating your initial container-based workloads, using Amazon Elastic Container Service (ECS) – including Fargate or Kubernetes (EKS) – and creating serverless workloads using Amazon API Gateway and AWS Lambda. Secure and cost-conscious defaults are chosen so that simple programs automatically use best practices for the underlying infrastructure, enabling better productivity with confidence.

Crosswalk for AWS also supports “day two and beyond” tasks, such as scaling your workload, securing and integrating it with your existing infrastructure, and going to production in multiple complex environments. This includes Amazon Virtual Private Cloud (VPC) for network isolation, AWS Auto Scaling for dynamic scaling, and AWS Identity and Access Management (IAM) for securing your infrastructure.

Earlier this month at KubeCon 2024, we launched the Pulumi Elastic Kubernetes Service (EKS) Provider v3.0.0, which is one of the Crosswalk libraries that makes Kubernetes management on AWS even easier. This release simplifies multi-cluster deployments, improves resource handling, and offers enhanced compatibility with the latest AWS EKS features.

AWS Policies as Code

We released Pulumi CrossGuard to enforce your organization’s cloud governance — security, compliance, cost controls, and more. This enables Policy as Code within your organization so that you can define guardrails for your infrastructure, ensuring engineers are following best practices and putting security first. CrossGuard helps your organization prevent mistakes before they occur and respond rapidly to any incidents.

Building Controls Planes

Many AWS customers build internal developer platforms (IDP) for developers to self-service infrastructure. In 2020, Pulumi Automation API was created to help automate infrastructure for these IDP control planes. Automation API exposes the full power of infrastructure as code through a programmatic interface, instead of through CLI commands, and it lets you use the Pulumi engine as an SDK, enabling you to create software that can create, update, configure, and destroy infrastructure dynamically. This enables you to use Pulumi to build IDPs that are custom-tailored to your team, organization, or customers.

AWS CDK for Pulumi

In 2022, Pulumi brought the ability to use AWS Cloud Development Kit (CDK) constructs from within a Pulumi deployment. For users already using AWS CDK, this provides Pulumi as a new option for orchestrating deployments in place of CloudFormation, offering improved deployment speed and integration with the full set of features of the Pulumi Platform. Even better, you can also now combine AWS CDK and Pulumi resources in a single Pulumi infrastructure as code project - passing outputs from Pulumi resources into AWS CDK constructs, and outputs from AWS CDK constructs into Pulumi resources.

AWS Cloud Development Kit (CDK) on Pulumi is now generally available. This powerful integration includes full support for CDK features like assets, custom resources, aspects, and context values/methods. It provides access to the full ecosystem of over 1,200 CDK Construct libraries, and it also supports all 180+ Pulumi providers, allowing for CDK applications to include resources outside of AWS.

Cloud Control API

In 2021, AWS launched Cloud Control API as a standard set of APIs to Create, Read, Update, Delete, and List (CRUDL) resources across hundreds of AWS Services. Pulumi supported that launch with a native provider called the Pulumi Cloud Control Provider, formerly known as AWS Native. The provider is now generally available, enabling day 1 support for new AWS resources, seamless compatibility with Pulumi IaC projects that currently use the Pulumi AWS provider, and extended support for third-party resources.

Beyond Pulumi IaC

Over the last few years, our customers have shared a clear message: IaC alone is insufficient. The complexities of modern cloud environments demand more than just IaC to manage cloud resources, improve security, and ensure compliance. In response, Pulumi’s offerings have expanded into a comprehensive suite of tools that go beyond IaC.

Pulumi Insights: Intelligent Cloud Management

The complexities of modern cloud environments create significant challenges in making sense of the millions of resources and resource updates per month, across hundreds of clouds, regions, and accounts. Identifying cost saving opportunities is difficult amidst soaring cloud expenditures, and maintaining compliance and security is mission critical because errors can cause catastrophic damage.

Pulumi Insights empowers organizations to tackle cloud complexity by delivering visibility, actionable insights, and AI-assisted optimization for their infrastructure. With Pulumi Insights, companies can scan and sync their entire AWS resource inventory – including resources not managed with Pulumi IaC – ensuring a complete picture of their cloud assets. This Account Discovery capability is now in public preview.

Compliance checks can also be run against standards such as PCI DSS, ISO 27001, and CIS benchmarks, helping maintain regulatory compliance and mitigate risks. The AI-powered Pulumi Copilot provides an interactive assistant to gain visibility into a team’s activity, discover cost saving opportunities, get compliant, and debug cloud failures. Pulumi Insights enables companies to discover, understand, manage, and improve their cloud infrastructure on their journey towards intelligent infrastructure management.

Pulumi ESC: Centralized Secrets Management

Cloud complexity creates significant challenges in managing sensitive credentials, API keys, and configurations across development, testing, and production environments. As organizations scale, they often face issues of secrets and configuration sprawl, duplication of secrets, and too many long-lived static secrets. None of the existing secrets management solutions adequately address these challenges, exposing enterprises to errors that can lead to security breaches, unintended exposure of sensitive data, and unauthorized access to critical resources.

Pulumi ESC addresses these challenges by offering seamless two-way integration – both pull and sync – with AWS Secrets Manager and other popular secrets stores, as well as the capability to consume secrets securely across any application, tool, and CI/CD platform. Developers gain easy, secure access to secrets via CLI, API, Kubernetes operators, and SDKs – eliminating the need for .env files and reducing the security risks associated with them. Additionally, Pulumi ESC can issue dynamic, short-lived AWS authentication tokens on demand via OpenID Connect (OIDC), reducing the risks that come with long-lived access tokens, while RBAC, versioning, and a detailed audit log let you trust (and prove) your secrets are secure.

Try Pulumi Cloud Today

Check out the Pulumi roadmap for upcoming features, let us know your feature requests by creating an issue on GitHub, connect with the Pulumi community on Slack, and if you haven’t yet, sign-up for Pulumi Cloud to get access to our full suite of products – Pulumi IaC, Pulumi ESC, and Pulumi Insights – so you can automate, secure, and manage everything you run in the cloud.

Meta image credit: ESO/M. Zamani

Amazon EKS Hybrid Nodes: Bridging Cloud and On-Premises

Josh Kodroff — Sun, 01 Dec 2024 15:06:25 -0500

AWS has introduced a new feature for Amazon Elastic Kubernetes Service (EKS): Amazon EKS Hybrid Nodes. This addition to the Amazon EKS Hybrid/Edge portfolio allows organizations to maintain their Kubernetes control plane in AWS while running workloads on-premises or at the edge. This hybrid approach offers the best of both worlds - AWS’s reliable and scalable control plane management combined with the flexibility to run workloads wherever they make the most sense for your business.

Amazon EKS Hybrid Nodes are particularly valuable for organizations that need to keep certain workloads on-premises due to data sovereignty requirements, latency concerns, or existing infrastructure investments. By extending EKS to support on-premises workloads, AWS is addressing a critical need in the enterprise market, especially for industries like manufacturing, telecommunications, and healthcare that often require local data processing.

Pulumi’s strategic partnership with AWS has always focused on making cloud infrastructure easier to manage through modern infrastructure as code. As an EKS Service Ready Partner, Pulumi provides first-class support for AWS services, including EKS. This partnership ensures that as AWS releases new features like EKS Hybrid Nodes, Pulumi users can quickly adopt and integrate them into their infrastructure management workflows. Managing Hybrid Environments with Pulumi

Pulumi’s infrastructure as code platform is perfectly positioned to help organizations manage their hybrid EKS environments. Using Pulumi, teams can define and manage both their EKS clusters and Kubernetes resources using their favorite programming languages, bringing software engineering practices to infrastructure management. This includes managing Hybrid Node configurations, networking setup, and security policies across both cloud and on-premises environments.

Pulumi’s Environments Secrets and Config (ESC) adds another layer of capability to Hybrid Node deployments. Operators can securely manage OIDC IAM credentials, kubeconfig files and other sensitive credentials through ESC, ensuring secure access to EKS clusters regardless of where the workloads run. Furthermore, pods running on Hybrid Nodes can consume ESC secrets through Pulumi ESC’s integration with the Kubernetes External Secrets Operator, maintaining a consistent secrets management approach across your entire hybrid infrastructure.

Pulumi’s solutions for EKS Anywhere are automatically validated across a variety of EKS deployment models (including cloud, VMWare, Nutanix, and bare metal) via AWS’ Conformitron framework. The framework makes it simple for AWS partners to test their offerings across common customer environments without having to provision these environments themselves.

Automation and Operations

Organizations can leverage Pulumi’s CI/CD capabilities within their Hybrid Node environment. Pulumi Deployments or the Pulumi Kubernetes Operator can run directly on Hybrid Nodes, enabling automated infrastructure management from within your on-premises environment. This approach allows for consistent infrastructure automation while respecting network boundaries and security requirements.

Summary

The combination of EKS Hybrid Nodes and Pulumi’s infrastructure as code platform provides organizations with a powerful solution for managing hybrid Kubernetes environments. As this feature continues to evolve, Pulumi will provide comprehensive support for deploying and managing Hybrid Node configurations, making it easier for teams to adopt and maintain hybrid architectures while modernizing their application workloads.

Stay tuned for upcoming Pulumi examples for managing EKS Hybrid Nodes, which will enable you to manage your hybrid Kubernetes infrastructure using your preferred programming language, complete with type safety, testing capabilities, and modern development practices.

If you’d like to learn more about how you can harness the power of EKS with Pulumi, check out the latest version of our EKS package or join us for our AWS Immersion Day workshop following re:Invent 2024.

Fargate vs EC2

Adam Gordon Bell — Mon, 18 Nov 2024 08:42:34 -0500

Building an EKS cluster requires choosing how your containers will actually run - either on EC2 instances you manage or through AWS Fargate’s pod-by-pod approach. The differences can be pretty dramatic in practice. I’m setting up a demo cluster right now using Pulumi, so let me show you what I mean.

Here is my Fargate cluster:

import pulumi
import pulumi_awsx as awsx
import pulumi_eks as eks
import pulumi_kubernetes as k8s

# Create VPC with default CIDR
eks_vpc = awsx.ec2.Vpc("eks-vpc",
 enable_dns_hostnames=True)

# Create minimal EKS cluster
eks_cluster = eks.Cluster("eks-cluster",
 vpc_id=eks_vpc.vpc_id,
 private_subnet_ids=eks_vpc.private_subnet_ids,
 fargate=True,
)

pulumi.export("kubeconfig", eks_cluster.kubeconfig)

And when I look in k9s I see this:

You can see that each pod is on its own node. Even the core-dns pods in the system namespace are each on their own nodes. That is a lot of nodes, and it’s going to keep growing with my cluster.

If I change my cluster to be backed by EC2 nodes, I get something different.

eks_cluster = eks.Cluster("eks-cluster",
 vpc_id=eks_vpc.vpc_id,
 private_subnet_ids=eks_vpc.private_subnet_ids,
- fargate=True,
+ instance_type="t3.medium",
+ desired_capacity=2,
+ min_size=1,
+ max_size=2,
)

Now, there are only two nodes, with the pods distributed among them. Those are my two t3.mediums.

So when should you use Fargate, and when EC2 for EKS? Well, it really all relates to what we are seeing here with the number and size of nodes.

Let me explain.

Bin Packing

Normally, the Kubernetes scheduler has to solve a bin-packing problem—fitting M pods across N nodes based on available resources. This is a straight-up resource optimization problem of balancing CPU, memory, and other resource demands across the cluster.

This can be surprisingly complex in practice. Teams must balance selecting the right instance types for different workload needs while managing multiple node groups. Adding to this complexity are scheduling rules and resource quotas that need careful configuration. Perhaps most challenging is striking the right balance between high resource utilization and maintaining enough headroom for spikes in demand.

With Fargate, AWS sidesteps this challenge by providing a correctly sized bin for each pod. Each Fargate pod runs on its own dynamically provisioned, right-sized mini-environment, where the “bin” (the Fargate instance) is sized to match the pod’s requested resources. This means there’s no need for Kubernetes to optimize resource usage across a pool of shared nodes, as each pod effectively has its own “container” provided by Fargate that fits it’s needs precisely.

In other words, AWS effectively pushes this bin-packing responsibility to itself, so you don’t have to worry ( as much ) about it.

While Fargate abstracts away node management, it’s important to understand that Fargate pods still run on EC2 instances behind the scenes. And each Fargate pod gets its own ENI (Elastic Network Interface) that can sometimes limit your scaling because pods aren’t sharing a network namespace.

Pros and Cons

So yeah, Fargate abstracts away the complexity of managing nodes and scaling clusters by shifting bin-packing to AWS. With its extensive infrastructure, AWS is arguably better positioned to optimize placements at scale. They can monitor their entire cloud landscape, placing Fargate instances dynamically across available resources. This allows them to potentially leverage underutilized resources in ways that would be challenging or impractical at an individual account’s scale.

Responsibility	EC2	Fargate
Instance Types	You pick size and provision nodes	AWS provisions per pod request
Pod Isolation	You prevent noisy neighbors	AWS isolates each pod
Resource Allocation	You find space on nodes	AWS creates dedicated space

Customer vs AWS Responsibility EC2 vs Fargate

Ideally, this frees you to focus on the services rather than on the underlying infrastructure, but in practice, it’s a bit more complex.

Since each pod is treated as a standalone instance, you lose out on the cost efficiencies of co-locating multiple pods on a single node. Even though AWS is “solving” the bin-packing problem globally, to do so, they need to introduce hard barriers, and while noisy neighbors, therefore, aren’t a problem, sharing resources is now impossible. In a way, you are throwing away the run time benefit of containers, the shared kernel virtualization, for something more like VMs. And just like VMs, Fargate pods take longer to start than pods on existing EC2 nodes (typically 30-60 seconds vs near-immediate starts).

A Note on Scaling

Both EC2 and Fargate work with Kubernetes’ Horizontal Pod Autoscaling (HPA), which adds or removes pods based on metrics like CPU usage. The difference? With EC2, you need enough nodes available to handle new pods. With Fargate, AWS automatically provisions resources for each new pod - though you’ll face that 30-60 second startup delay and potential VPC limits since each pod needs its own network interface.

Feature	Fargate	EC2
Cost	Higher per pod	Lower cost per pod
Scalability	Scales on-demand, isolated resources	Requires manual node scaling
Resource Sharing	Not possible	Efficient resource sharing among pods
Startup Time	30-60 seconds	Near instant on existing nodes
Use Cases	Burst workloads, isolated pods	Cost-sensitive workloads, Resource sharing workloads

Maybe this will make more sense with an example.

Workload Example: Static Analysis

Have you ever had to run CPU-intensive batch jobs alongside lighter web services?

In a previous role, when my team and I were tasked with putting our static container image analyzer into production, we went the typical Kubernetes route, throwing it into an EC2-backed K8s cluster. The analyzer was built in Scala, running on the JVM, and was designed to take in work from a Kafka topic. Each analysis ultimately led to the results being sent to a database. We also had scheduled daily rescans of existing images whenever vulnerability lists were updated.

Then, a web app connected to this database and showed all your results. It was stateless and did far less work per request, mainly just answering an HTTP request by talking to a database.

The hard part though is that each analysis job involved uncompressing and analyzing Docker layers from S3 and could use a lot of CPU. Memory needs were around 2 GB per instance, which kept the JVM happy, but long sustained bursts in CPU usage were common, especially during intensive parts of the analysis.

In our K8s setup, the biggest pain point was handling this analysis workload’s burstiness and keeping a bunch of stateless web apps up and answering requests. We ran multiple instances of the static analyzer. Still, the analyzer was the worst type of noisy neighbor to other services in the cluster, so we’d end up over-provisioning to ensure we had enough headroom, which led to wasted resources and extra cost. The setup worked, but was not great.

Thinking back, I can see how Fargate could’ve streamlined things here. With Fargate, each static analyzer instance would run in its own isolated environment, so there’d be no resource contention or need for pre-provisioned node capacity. Each pod would be fully provisioned with the exact CPU and memory needed, and we could do some sort of on-demand scaling to deal with backlogs of analysis work in the Kafka topic.

Fargate on EKS could be a great solution for this type of chunky, resource-heavy macro-services. There are probably other ways to handle this situation, and we worked on decomposing the analysis into smaller chunks, but Fargate might have been a good solution.

But there are lots of situations where Fargate is less of a fit.

You might also like:

Example: Go Services for E-commerce

Ok, another workload that comes to mind is this one from a friend - He worked on an e-commerce backend with many lightweight Go services, each handling specific tasks—user profiles, catalog browsing, order processing, and notifications. These services were network-heavy, handling gRPC calls and database requests, but they didn’t need much CPU or memory per instance.

Fargate wouldn’t have made sense here—it would have isolated each pod, preventing resource sharing and driving up costs without real benefit. For lightweight services like these, EC2-backed Kubernetes is perfect, keeping costs low and maximizing resources by sharing nodes across small, efficient services.

So, for lightweight micro-services that can share resources, EC2 nodes with many pods each are probably the way to go.

Other Fargate Cases

Fargate can excel with long-running event-driven and bursty workloads, where you need compute resources briefly or unpredictably. Think file processing jobs that trigger on uploads, or batch processing that runs sporadically. When jobs are intermittent, Fargate’s ability to scale from zero and its isolated resources can outweigh its higher per-pod cost and startup delay. Just ensure your workload can tolerate the 30-60 second cold start.

Fargate has some hard limitations that might rule it out for your use case:

No DaemonSets
Maximum of 4 vCPU and 30GB memory per pod
Storage limitations: Only ephemeral storage up to 20GB per pod, no persistent volumes
No GPU support

Fargate vs EC2 Pricing

Ok, EC2 costs less than Fargate on a pure cost-of-compute basis. If my t3.medium ($0.0416/hr) can typically run 6-8 pods reliably, and this works out to under $0.01/hr per pod. Comparable Fargate pods (0.5 vCPU/1GB) cost about $0.025/hr each, so 2.5x.

These calculations can change based on how much headroom you need on EC2 and how many containers you can run on a given EC2 instance. Still, Fargate is just more expensive unless all your containers fit the exact computing sizes offered by Fargate. And don’t forget, AWS is likely incurring costs maintaining the warm compute capacity for launching Fargate pods, so it shouldn’t be surprising that Fargate is more expensive.

But really, pure compute cost calculations are an insufficient metric for making this decision - you need to factor in your team’s operational capacity, scaling patterns, and isolation requirements.

( Also, if Fargate makes sense because of your workload but the cost is a concern, then look into Fargate Spot instances, which can be up to 70% cheaper, but be aware that AWS can terminate them at any point to reclaim capacity. )

Misconceptions About Fargate

Fargate saves you from managing servers With containerized workloads and immutable infrastructure, your EC2 nodes are minimal, standardized, and automatically replaced. You’re not logging in, installing packages, or managing configuration - everything runs in containers. When a node needs updates, you replace it entirely.
Fargate handles all scaling challenges While it handles node scaling, you still face ENI limits, longer pod startup times (30-60s), and potential networking bottlenecks.
It’s simpler and more reliable It may be, but it may not be. The complexity shifts to networking, pod configuration, and working within Fargate’s constraints. You trade node management for new operational challenges like slower startups and possible networking complexity.

Managing Container Orchestration with Pulumi

Whether you choose EC2 or Fargate, Pulumi makes it easy to manage in your favorite programming languages. With its rich AWS support and infrastructure as code capabilities, you can define, deploy, and manage your entire Kubernetes infrastructure using familiar tools and practices. Pulumi’s AWS provider helps you implement the right solution for your needs.

With Pulumi, you get:

Complete control over EKS cluster configuration, including node types, scaling policies, and networking setup
Built-in support for both EC2 node groups and Fargate profiles within the same cluster
Infrastructure testing capabilities to validate your cluster configuration before deployment
Automation APIs for programmatic infrastructure management and custom tooling

Ready to streamline your EKS infrastructure management?

Try Pulumi for Free

Why Not Both

I hope my examples point to a pragmatic approach: use EC2 as your foundation for efficient microservices that can share resources, like our Go e-commerce setup. Add Fargate when you need isolation or flexible scaling - like for the static analysis workload. Better yet, combine both approaches in the same cluster, for cost-efficiency and isolation where needed.

If you’re interested in using Pulumi to manage EKS infra, here’s a practical EKS setup guide guide to get your started.

Now, back to getting this demo cluster up and working!

Pulumi EKS Provider Version 3.0.0

Florian Stadler — Thu, 17 Oct 2024 17:32:03 +0200

The latest major release of the Pulumi EKS Provider is available now!

This release delivers significant improvements in flexibility, security and introduces new features to enhance your Kubernetes experience on AWS. AWS recently announced the deprecation of two features used by default in Pulumi EKS: the aws-auth ConfigMap and the AL2 operating system. Pulumi EKS v3 addresses these deprecations, enhances the maintainability of the provider, and aligns it with EKS best practices.

Here are a few links to help you get started if you are new to Pulumi:

EKS Setup & Installation - Instructions on installing and configuring the Pulumi EKS provider
EKS How-to Guides - Learn how to use the EKS provider to provision and manage Kubernetes clusters on AWS
Pulumi Neo - Ask Pulumi Neo to help you create a new EKS project

Key Highlights of EKS V3

Support for Amazon Linux 2023 (AL2023) and Bottlerocket Operating Systems: Enhanced operating system options for node groups, allowing you to choose the OS that best fits your workloads and compliance needs. This addresses the upcoming deprecation of Amazon Linux 2 (AL2).
Access Entries for IAM Integration: Enables replacement of the deprecated aws-auth ConfigMap with Access Entries for managing Kubernetes authentication.
EKS Managed Addons: Simplified management of vpc-cni, coredns, and kube-proxy as EKS managed addons.
EKS Security Groups for Pods and Network Policies: Enhanced network security and control within EKS clusters.

New Features and Improvements

Support for Amazon Linux 2023 and Bottlerocket

We have expanded the operating system options for node groups in EKS v3 to address the upcoming deprecation of Amazon Linux 2 (AL2). You can now choose between Amazon Linux 2 (deprecated), Amazon Linux 2023 and Bottlerocket for your EKS nodes. This flexibility allows you to select the OS that best fits your workloads, security requirements, and compliance needs, while ensuring you are using a supported and actively maintained operating system. We’ve introduced a new operatingSystem property for node groups to facilitate this choice.

Access Entries for IAM Integration

AWS has introduced Access Entries as a new method for granting IAM principals access to Kubernetes resources. This approach relies solely on AWS resources for managing Kubernetes auth, replacing the deprecated aws-auth ConfigMap. You can now leverage Access Entries by setting the authenticationMode to API in your cluster configuration.

EKS Managed Addons

The EKS cluster components vpc-cni, coredns, and kube-proxy are now configured as EKS managed addons. This change simplifies management, especially for clusters with private API endpoints, and ensures that these critical components stay up to date automatically. Additionally it removes the dependency on kubectl, allowing pulumi-native management of clusters.

Cluster Autoscaler Integration

Pulumi EKS v3 introduces better support for the Kubernetes Cluster Autoscaler. A new ignoreScalingChanges parameter for node groups allows Pulumi to ignore external scaling changes, facilitating seamless integration with dynamic scaling solutions.

EKS Security Groups for Pods and Network Policies

We’ve added support for EKS security groups for pods (example) and EKS Network Policies (example), providing more granular control over pod-to-pod and pod-to-external network communication within your EKS clusters.

Migration Guide

To help you transition smoothly, we’ve prepared a migration guide with these key steps:

Update node groups to use AL2023 or explicitly configure AL2 if needed.
Replace the deprecated NodeGroup component with NodeGroupV2.
Update your code to handle new output types for certain properties.
Review and update your use of default security groups, which can now be disabled.

Please refer to our EKS v3 Migration Documentation for a detailed guide.

Conclusion

Pulumi EKS v3 represents a significant step forward in managing Kubernetes clusters on AWS, empowering you to build and manage more robust and efficient EKS clusters.

We encourage all users to upgrade to this latest version to take advantage of these improvements and ensure your EKS deployments remain secure and up-to-date.

For more information on getting started with Pulumi EKS v3, check out our documentation or join our community Slack for support and discussions.

Happy clustering with Pulumi EKS v3!

Pulumi AWS Cloud Control Provider is Generally Available

Tejitha Raju — Tue, 01 Oct 2024 09:00:00 -0700

We’re excited to announce the general availability of the new AWS Cloud Control Provider for Pulumi (previously “AWS Native”). AWS is one of the most-used cloud providers across the Pulumi ecosystem, and we are committed to providing the fastest possible access to new AWS capabilities to all Pulumi users. This provider delivers on that promise by providing native support for all resources in the AWS Cloud Control APIs. As part of this launch, we are also renaming the provider from “AWS Native” to “AWS Cloud Control Provider,” based on customer feedback during the preview phase, to provide users with a broader range of advanced cloud management tools exposed by the AWS Cloud Control API. This provider includes the following features and benefits,

Day 1 support for AWS Resources: Provides full coverage of the AWS Cloud Control API, typically on the day of the launch, ensuring users can adopt new features immediately.
Complementary Pulumi Experience: Works alongside Pulumi’s AWS Provider, enhancing Infrastructure as Code projects with the latest AWS capabilities without requiring significant changes to existing setups.
Extended support for third-party resources: Our platform offers enhanced flexibility and interoperability by supporting third-party resources available in the CloudFormation Registry, such as Atlassian, MongoDB, Datadog, and more.
Easy CloudFormation to Pulumi Migration: cf2pulumi provides the ability to migrate existing CloudFormation templates into Pulumi programs in your favorite language.

At Pulumi, we’re dedicated to empowering our customers with the tools they need to innovate and thrive in the cloud. The launch of the AWS Cloud Control Provider represents our unwavering commitment to providing best-in-class solutions that simplify cloud management, streamline operations, and drive business success. The Pulumi AWS Cloud Control Provider can be used side-by-side with the standard Pulumi AWS provider and nearly 200 additional Pulumi resource providers that cover a wide variety of cloud and SaaS platforms.

Using Pulumi AWS Cloud Control Provider with the Pulumi AWS Provider

Let’s walk through an example of using the AWS Cloud Control Provider alongside the Pulumi’s standard AWS Provider. This example shows how to create an S3 bucket with the standard AWS provider and deploy a Lambda function using the AWS Cloud Control provider. The Lambda function is set up to use the S3 bucket’s name and process events accordingly. This example showcases how both providers can work together, leveraging the standard AWS Provider for widely supported resources, while using AWS Cloud Control for the latest features and APIs provided by AWS.

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as awscc from "@pulumi/aws-native";

// ARN of the IAM role that the Lambda function will assume
const roleArn = "arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test";

// Create an S3 bucket using the AWS provider
const bucket = new aws.s3.Bucket("myBucket");

// Create the AWS Lambda function using the AWS Cloud Control provider
const lambdaFunction = new awscc.lambda.Function("myLambdaFunction", {
 functionName: "my-lambda-function",
 role: "arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test",
 runtime: "nodejs18.x",
 handler: "index.handler",
 code: {
 zipFile: `
 exports.handler = async function(event) {
 const bucketName = process.env.BUCKET_NAME;
 console.log("Event triggered: ", JSON.stringify(event, null, 2));
 return {
 statusCode: 200,
 body: JSON.stringify('Hello from Lambda!'),
 };
 };
 `,
 },
 environment: {
 variables: {
 BUCKET_NAME: bucket.bucket,
 },
 },
});

// Export the Lambda function ARN and the S3 bucket name
export const lambdaFunctionArn = lambdaFunction.arn;
export const s3BucketName = bucket.bucket;

import pulumi
import pulumi_aws as aws
import pulumi_aws_native as aws_cloudcontrol
# Create an S3 bucket using the standard AWS provider
my_bucket = aws.s3.Bucket("myBucket")

# Create the Lambda function using the AWS Cloud Control provider
my_lambda_function = aws_cloudcontrol.lambda_.Function("myLambdaFunction",
 function_name="my-lambda-function",
 role="arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test",
 runtime="nodejs18.x",
 handler="index.handler",
 code={
 "zip_file": """exports.handler = async function(event) {
 const bucketName = process.env.BUCKET_NAME;
 console.log("Event triggered: ", JSON.stringify(event, null, 2));
 return {
 statusCode: 200,
 body: JSON.stringify('Hello from Lambda!'),
 };
};
""",
 },
 environment={
 "variables": {
 "BUCKET_NAME": my_bucket.bucket,
 },
 })
pulumi.export("lambdaFunctionArn", my_lambda_function.arn) # Directly export the value, no need for a dictionary
pulumi.export("s3BucketName", my_bucket.bucket)

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Aws = Pulumi.Aws;
using AwsCloudControl = Pulumi.AwsNative;

return await Deployment.RunAsync(() =>
{
 // Create an S3 bucket
 var myBucket = new Aws.S3.Bucket("myBucket");

 // Create a Lambda function using AWS Cloud Control
 var myLambdaFunction = new AwsCloudControl.Lambda.Function("myLambdaFunction", new()
 {
 FunctionName = "my-lambda-function",
 Role = "arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test",
 Runtime = "nodejs18.x",
 Handler = "index.handler",
 Code = new AwsCloudControl.Lambda.Inputs.FunctionCodeArgs
 {
 ZipFile = @"exports.handler = async function(event) {
 const bucketName = process.env.BUCKET_NAME;
 console.log(""Event triggered: "", JSON.stringify(event, null, 2));
 return {
 statusCode: 200,
 body: JSON.stringify('Hello from Lambda!'),
 };
};
",
 },
 Environment = new AwsCloudControl.Lambda.Inputs.FunctionEnvironmentArgs
 {
 Variables =
 {
 { "BUCKET_NAME", myBucket.BucketName },
 },
 },
 });

 // Export the Lambda function ARN and the S3 bucket name directly
 return new Dictionary<string, object?>
 {
 ["lambdaFunctionArn"] = myLambdaFunction.Arn,
 ["s3BucketName"] = myBucket.BucketName,
 };
});


package main

import (
 "github.com/pulumi/pulumi-aws-native/sdk/go/aws/lambda"
 "github.com/pulumi/pulumi-aws/sdk/v7/go/aws/s3"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // Create an S3 bucket using the standard AWS provider
 myBucket, err := s3.NewBucket(ctx, "myBucket", nil)
 if err != nil {
 return err
 }
 // Create a Lambda function using AWS Cloud Control provider
 myLambdaFunction, err := lambda.NewFunction(ctx, "myLambdaFunction", &lambda.FunctionArgs{
 FunctionName: pulumi.String("my-lambda-function"),
 Role: pulumi.String("arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test"),
 Runtime: pulumi.String("nodejs18.x"),
 Handler: pulumi.String("index.handler"),
 Code: &lambda.FunctionCodeArgs{
ZipFile: pulumi.String(`exports.handler = async function(event) {
 const bucketName = process.env.BUCKET_NAME;
 console.log("Event triggered: ", JSON.stringify(event, null, 2));
 return {
 statusCode: 200,
 body: JSON.stringify('Hello from Lambda!'),
 };
};
`),
 },
 Environment: &lambda.FunctionEnvironmentArgs{
 Variables: pulumi.StringMap{
 "BUCKET_NAME": myBucket.Bucket,
 },
 },
 })
 if err != nil {
 return err
 }
 // Export the Lambda function's ARN so that it can be referenced later
 ctx.Export("lambdaFunctionArn",myLambdaFunction.Arn)
 // Export the S3 bucket name so it can be used in other parts of the infrastructure
 ctx.Export("s3BucketName", myBucket.Bucket)
 return nil
 })
}

name: your-project-name  # Replace with your project name
runtime: yaml

resources:
# Creating an S3 bucket using the standard AWS provider.
 myBucket:
 type: aws:s3/bucket:Bucket
 properties: {}

# Creating a Lambda function using the AWS Cloud Control provider.
 myLambdaFunction:
 type: aws-native:lambda:Function
 properties:
 functionName: my-lambda-function
 role: arn:aws:iam::YOUR_ACCOUNT_ID:role/cloudcontrol-test
 runtime: nodejs18.x
 handler: index.handler
 code:
 zipFile: |
 exports.handler = async function(event) {
 const bucketName = process.env.BUCKET_NAME;
 console.log("Event triggered: ", JSON.stringify(event, null, 2));
 return {
 statusCode: 200,
 body: JSON.stringify('Hello from Lambda!'),
 };
 };
# Lambda environment variables are set here.
 environment:
 variables:
 BUCKET_NAME: ${myBucket.bucket}

# Outputs section - Pulumi will output these values after deployment.
outputs:
 lambdaFunctionArn:
 value: ${myLambdaFunction.arn}
 s3BucketName:
 value: ${myBucket.bucket}

Leveraging AWS Cloud Control Provider for Advanced WAFv2 Configurations

The AWS Cloud Control Provider offers greater flexibility for handling complex configurations that may be challenging or impossible with the standard AWS Provider. For example, when configuring WAFv2, you can define arbitrary nested rules, which enables more advanced and customized security logic. With the Cloud Control Provider, you can compose these rules with reusable building blocks, creating configurations that are more modular and scalable. This level of granularity allows for more precise control over web traffic filtering, such as combining multiple rule sets, applying conditions across different layers of the request, and creating highly tailored security policies. This flexibility is particularly useful for organizations with complex security requirements that need to go beyond what the standard AWS Provider can offer.

For example, this configuration involves six levels of nesting rules but can be composed from reusable building blocks:

import * as pulumi from "@pulumi/pulumi";
import * as awscc from "@pulumi/aws-native";

type RuleGroupStatement = awscc.types.input.wafv2.RuleGroupStatementArgs;

// Create an AWS WAFv2 IP Set
const exampleIPSet = new awscc.wafv2.IpSet("exampleIPSet", {
 addresses: ["1.2.3.4/32", "5.6.7.8/32"],
 ipAddressVersion: "IPV4",
 scope: "REGIONAL",
});

const ipMatch: RuleGroupStatement = {
 ipSetReferenceStatement: {
 arn: exampleIPSet.arn,
 },
};

const testURIMatch: RuleGroupStatement = {
 byteMatchStatement: {
 searchString: "/test",
 fieldToMatch: {
 uriPath: {},
 },
 textTransformations: [
 {
 priority: 0,
 type: "NONE",
 },
 ],
 positionalConstraint: "EXACTLY",
 },
};

const hostMatch: RuleGroupStatement = {
 byteMatchStatement: {
 fieldToMatch: {
 singleHeader: {
 name: "host",
 },
 },
 positionalConstraint: "EXACTLY",
 searchString: "example.com",
 textTransformations: [
 {
 type: "NONE",
 priority: 0,
 },
 ],
 },
};

const internalLabelMatch: RuleGroupStatement = {
 labelMatchStatement: {
 scope: "LABEL",
 key: "internal",
 },
};
const xssMatch: RuleGroupStatement = {
 xssMatchStatement: {
 fieldToMatch: {
 body: {},
 },
 textTransformations: [
 {
 priority: 0,
 type: "NONE",
 },
 ],
 },
};

function matchNot(statement: RuleGroupStatement): RuleGroupStatement {
 return { notStatement: { statement } };
}

function matchAll(statements: RuleGroupStatement[]): RuleGroupStatement {
 return { andStatement: { statements: statements } };
}

function matchAny(statements: RuleGroupStatement[]): RuleGroupStatement {
 return { orStatement: { statements: statements } };
}

// Create an AWS WAFv2 Rule Group
const exampleRuleGroup = new awscc.wafv2.RuleGroup("exampleRuleGroup", {
 capacity: 2000,

 rules: [
 {
 action: { allow: {} },
 name: "rule-1",
 priority: 1,
 statement: matchAny([
 matchAll([ipMatch, xssMatch]),
 matchNot(
 matchAny([
 testURIMatch,
 matchAll([
 internalLabelMatch,
 matchNot(matchAll([ipMatch, hostMatch])),
 ]),
 ]),
 ),
 ]),

 visibilityConfig: {
 cloudWatchMetricsEnabled: true,
 metricName: "friendly-rule-metric-name",
 sampledRequestsEnabled: true,
 },
 },
 ],
 scope: "REGIONAL",
 visibilityConfig: {
 cloudWatchMetricsEnabled: true,
 metricName: "friendly-metric-name",
 sampledRequestsEnabled: true,
 },
});

// Create an AWS WAFv2 WebACL
const exampleWebACL = new awscc.wafv2.WebAcl("exampleWebACL", {
 defaultAction: { allow: {} },
 rules: [
 {
 name: "example-rule-1",
 overrideAction: { count: {} },
 priority: 1,
 statement: {
 ruleGroupReferenceStatement: {
 arn: exampleRuleGroup.arn,
 },
 },
 visibilityConfig: {
 cloudWatchMetricsEnabled: true,
 metricName: "friendly-rule-metric-name",
 sampledRequestsEnabled: true,
 },
 },
 ],
 scope: "REGIONAL",
 visibilityConfig: {
 cloudWatchMetricsEnabled: true,
 metricName: "friendly-metric-name",
 sampledRequestsEnabled: true,
 },
});

Conclusion

The AWS Cloud Control Provider for Pulumi is a major step forward in fully and faithfully supporting the AWS Cloud Control API and its ecosystem in Pulumi. By closely collaborating with AWS, we’ve created a provider that enables Pulumi users to leverage the latest AWS features as soon as they are released - without waiting for upstream provider changes to arrive in the standard Pulumi AWS Provider.

Our goal is to continually improve and simplify the Pulumi experience, empowering developers with the tools they need to manage cloud infrastructure seamlessly. This new provider represents our commitment to delivering cutting-edge capabilities as quickly as possible.

We look forward to your feedback and encourage you to explore the full potential of the Pulumi AWS Cloud Control Provider. Don’t hesitate to reach out via the Pulumi AWS Cloud Control Provider GitHub repository with any questions or suggestions as we continue to refine and enhance this experience.

Uploading documents to your AI Slackbot in real-time using S3, SQS and Pulumi on AWS

Tyler Mulligan — Mon, 03 Jun 2024 17:21:02 +0100

In the introductory blog post, we learned to Create an AI Slack Bot to Chat with Your Data Using Embedchain, Pulumi on AWS, and continued with Adding data to Pinecone using S3, Embedchain, and Pulumi on AWS for an AI Slack bot.

For reference, here’s what our architecture looked like at the end of the second blog post.

To follow along, clone the project, git clone https://github.com/catmeme/arti.git or view it on GitHub.

S3 Events and SQS

AWS provides a few ways to integrate with AWS S3 Events, for this project, we’re going to focus on the latest and most straight-forward, S3 Event Notifications. Depending on your use case, integrating with EventBridge may make more sense.

While we can integrate S3 events directly with Lambda, what happens when we add 100’s of files at the same time? Potentially, hundreds of Lambdas could be invoked simultaneously, which could become an issue depending on other factors. We’ll solve for this by Using Lambda with Amazon SQS, and reserved concurrency.

Our Pulumi program is updated to add a Queue and integrate it with our new Lambda SQS handler (policies left out for brevity, you can view them in the deploy/pulumi/__main__.py in the project).

# Add a queue
sqs_queue = aws.sqs.Queue(...)

# Add a docker image runtime for our Lambda SQS handler
docker_image_lambda_sqs_handler = docker.Image(...)

# Add Lambda SQS handler
app_sqs_lambda = aws.lambda_.Function(
 resource_name=f"{app_name}-sqs-lambda",
 environment=lambda_environment, # We've added the sqs_queue.url here as well
 image_uri=docker_image_lambda_sqs_handler.repo_digest,
 opts=pulumi.ResourceOptions(depends_on=[docker_image_lambda_sqs_handler]),
 **common_lambda_options,
)

# Add event source mapping
event_source_mapping = aws.lambda_.EventSourceMapping(
 f"{app_name}-lambda-sqs-event-source-mapping",
 event_source_arn=sqs_queue.arn,
 function_name=app_sqs_lambda.name,
 enabled=True,
 batch_size=10,
)

Add the application code responsible for processing S3 events coming from SQS.

def handler(event, context):
 for record in event.get("Records", []):
 s3_event = json.loads(record.get("body", "{}"))
 for s3_record in s3_event.get("Records", []):
 object_key = s3_record.get("s3", {}).get("object", {}).get("key")
 if object_key:
 logger.info("Processed %s in bucket %s.", object_key, app_bucket_name)
 load_data(asset_location=f"{app_bucket_name}/{object_key}")

 return {"statusCode": 200, "body": json.dumps("Successfully processed S3 event(s) from SQS messages.")}

We also have a scheduled event that we set up for ingestion in the last post, which we’re going to disable in favor of real-time events.

That’s pretty much it on the infrastructure side, here’s the updated diagram.

Slack to S3 file upload bridge

So what do we do with this new functionality? How do we expose it to the user? The most obvious approach is to reuse our existing Slack integration.

We’ll start by creating a Slack file upload bridge to S3, the bot watches for file_shared events on Slack and re-uploads files to our S3 bucket.

Using Slack Bolt, we add a new function handle_file_upload with the @app.event("file_shared") decorator, binding it to said event. When that event occurs, it’s processed, and the details of our file are retrieved, a new payload is constructed using requests, and the file gets uploaded to S3, this is our bridge.

@app.event("file_shared")
def handle_file_uploads(event, say, client, logger):
 file_id = event['file']['id']
 channel_id = event['channel_id']

 # Fetch file details using the Slack API
 file_info = client.files_info(token=slack_bot_token, file=file_id)['file']
 file_url = file_info['url_private_download']

 # Download file content from Slack
 headers = {'Authorization': f"Bearer {slack_bot_token}"}
 response = requests.get(file_url, headers=headers, stream=True)

 if response.status_code == 200:
 # Upload the file to S3
 s3_client.upload_fileobj(
 Fileobj=response.raw,
 Bucket=os.getenv("APP_BUCKET_NAME"),
 Key=file_info['name']
 )

 # Fetch recent messages to find the correct timestamp
 messages = client.conversations_history(channel=channel_id, limit=10)['messages']
 for msg in messages:
 if msg.get('files') and any(f['id'] == file_id for f in msg.get('files', [])):
 # Found the message with the correct file
 message_ts = msg['ts']
 client.reactions_add(channel=channel_id, name="white_check_mark", timestamp=message_ts)
 # Respond in a thread to the message containing the file
 say(text=f"File `{file_info['name']}` uploaded to S3 successfully! :tada:", thread_ts=message_ts)
 return

 logger.error("No valid message found for reaction.")

Keep in mind, we’re not doing any validation here yet. We’re blindly copying files from Slack to S3 and having our data loader process them. It’s probably not a good idea to deploy this bot to a public Slack channel without more moderation/validation in place.

Here’s what it looks like when we drag/drop a file into Slack now. On the right, a file was shared with the bot in private message. The bot acknowledged this upload with the checkmark reaction, the job is placed on the queue. Shortly after, on the left, the SQS Lambda handler has finished processing the uploaded file. We’re now able to use the bot to talk about the uploaded files.

Slack interactivity

In minutes, we were able to move from a scheduled task for processing files in a bucket, to real-time file processing integrated with Slack. We have our /arti command we can use to interrogate the documents we just uploaded, but we’re going to have a little more fun.

Thanks to Slack Block Kit Builder, we can build a form that exposes all of the configuration options for the query method in Embedchain’s app, providing the user with more options for interacting with the RAG model.

Our code to utilize the JSON we’ve crafted in the builder looks something like (we’ll call this /smarti):

@app.command("/smarti")
def open_smarti_modal(ack, body, client):
 ack()
 print("Opening SMARTI modal...")
 modal_file_path = Path(config.project_root) / 'config' / 'slack_modals' / 'chat.json'

 with open(modal_file_path, 'r') as json_file:
 modal_view = json.load(json_file)

 client.views_open(
 trigger_id=body["trigger_id"],
 view=modal_view
 )

And the code for handling that submission looks something like:

def handle_smarti_submission(client, logger, body, view):
 values = view["state"]["values"]
 question = values["question"]["input"]["value"]
 model = values["model"]["input"]["selected_option"]["value"]
 temperature = float(values["temperature"]["input"]["value"])
 max_tokens = int(values["max_tokens"]["input"]["value"])
 top_p = float(values["top_p"]["input"]["value"])
 prompt = values["prompt"]["input"]["value"]
 system_prompt = values["system_prompt"]["input"]["value"]
 citation_options = values["citations"]["input"].get("selected_options", [])
 citations = any(option['value'] == 'true' for option in citation_options)

 ai_response = ask_ai(
 question=question,
 model=model,
 temperature=temperature,
 max_tokens=max_tokens,
 top_p=top_p,
 prompt=prompt,
 system_prompt=system_prompt,
 citations=citations
 )
 ...

Submitting that form results in a response such as below. The bot honored our prompts, talking like Shakespeare, a thread is created, and citations were noted, showing where the data was retrieved from.

Lastly, if we want interact with the bot more publicly, then we can use the app_mention event.

@app.event("app_mention")
def handle_app_mention(event, say, client):
 text = event['text']
 bot_user_id = client.auth_test()["user_id"]

 # Strip the mention of the bot to isolate the command and arguments
 command_text = text.split(f'<@{bot_user_id}>')[1].strip() if f'<@{bot_user_id}>' in text else text

 # Parse the command and arguments
 parts = command_text.split(maxsplit=1)
 if not parts:
 say("I need a command to do something useful!")
 return

 # Get the command and the remainder of the message as arguments
 command = parts[0].lower()
 args = parts[1] if len(parts) > 1 else ""

 if command == "ask":
 if args:
 response = ask_ai(question=args)
 say(response)
 else:
 say("Please provide a question to ask.")
 elif command == "list":
 list_data_sources(say)
 else:
 say(f"Sorry, I don't recognize the command `{command}`.")

Resulting in an interaction such as

Of course, this same methodology can be used on any of the events exposed by Slack.

To learn more about extending this bot, see Slack Bolt’s documentation.

Conclusion

In this blog post, we took our existing Slack bot which has a data loading process that loads documents from an S3 bucket into a vector database on a schedule, and we wired it up to behave in real time.

To expose this functionality to the end user, we created a bridge between file uploads to Slack and S3, allowing drag/drop capabilities of files in conversational chat.

Furthering our interactions with our application, we leveraged Slack’s Block Kit to create a custom modal, and interact with events.

As always, we welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is easy and free. Get started today!

Happy AI building!

Adding data to Pinecone using S3, Embedchain and Pulumi on AWS for an AI Slack bot

Tyler Mulligan — Wed, 10 Apr 2024 17:00:00 +0100

In the introductory blog post, we learned to Create an AI Slack Bot to Chat with Your Data Using Embedchain, Pulumi on AWS. However, we made a few concessions in the application logic to illustrate the broader picture of what we were able to achieve combining these three technologies. Now that we have a solid foundation for deploying our Slack bot and querying our data, lets begin moving from proof-of-concept to production-ready, iteratively.

For reference, here’s what our architecture looked like at the end of the first iteration.

To follow along, clone the project, git clone https://github.com/catmeme/arti.git

Configuration

In our proof-of-concept, we defined our Embedchain config in our Python code. Best practices dictate strict separation of config from code, so the first thing we must address is getting the embedchain configuration out of our code.

A config/config.yaml made from our dict, is much easier to read, maintain and share.

app:
 config:
 id: arti
 collect_metrics: False
 log_level: INFO
llm:
 provider: openai
 config:
 model: 'gpt-3.5-turbo-1106'
 temperature: 0.5
 max_tokens: 1000
 top_p: 1
embedder:
 provider: openai
chunker:
 chunk_size: 2000
 chunk_overlap: 0
 length_function: len
vectordb:
 provider: chroma
 config:
 collection_name: arti-ai
 dir: /tmp/db
 allow_reset: True

Our goal is to replace our big inline dict, and read a YAML file instead, following this configuration specification.

def ask_ai(question):
 """Ask the AI a question."""
 print(f"Processing question: {question}")

 app_config = {
 # hardcoded configuration we're moving out to YAML
 }

 loader = DirectoryLoader(config={"recursive": True})
 app = App.from_config(config=app_config)
 app.add("assets", loader=loader)

 return app.query(question)

To do that, though, we don’t want to implement it directly as we already have a config module, src/arti_ai/config.py. Let’s define a new method to read that YAML file, and pass it to our application at startup.

class Config:
 def __init__(self, config_file="config/config.yaml", secrets_manager_client=None):
 self.secrets_manager_client = secrets_manager_client or boto3.client("secretsmanager")
 self.project_root = Path(__file__).resolve().parents[2]
 self.config_file = self.project_root / config_file
 load_dotenv()

 def get_embedchain_config(self):
 try:
 with open(self.config_file, 'r') as file:
 return yaml.safe_load(file)
 except FileNotFoundError as e:
 raise FileNotFoundError(f"Configuration file not found: {self.config_file}") from e
 except yaml.YAMLError as e:
 raise yaml.YAMLError(f"Error parsing YAML configuration: {self.config_file}") from e

Our app.py function simplifies, becoming more readable and flexible, we’ve even fixed a bug, making our application more portable with config.project_root, now arti can be run from any directory.

def ask_ai(question: str):
 """Ask the AI a question."""
 print(f"Processing question: {question}")

 from embedchain import App
 from embedchain.loaders.directory_loader import DirectoryLoader

 loader = DirectoryLoader(config={"recursive": True})
 app = App.from_config(config=config.get_embedchain_config()) # inline dict has become this
 app.add(f"{config.project_root}/assets", loader=loader)

 return app.query(question)

Now that we’ve separated out our configuration, we can have multiple scenarios preconfigured. In the repository, there are a few examples, including a private ai configuration, so that embedchain can be run 100% locally on your machine using gpt4all. The configuration difference boils down to updating the LLM and Embedder to open-source models.

llm:
 provider: gpt4all
 config:
 model: 'orca-mini-3b-gguf2-q4_0.gguf'
 max_tokens: 1000
 top_p: 1
embedder:
 provider: huggingface
 config:
 model: 'sentence-transformers/all-MiniLM-L6-v2'

Now that we have a dynamic approach to loading configuration from YAML, lets move on to improving the application architecture, which was not built to scale beyond relatively small data sets.

Application architecture

Knowing your limits

The current application architecture uses a single process with the expectation that a response is generated within 29 seconds, before the API Gateway timeout.

Slack also has an expectation, that we must acknowledge (or ack) a request within 3 seconds.

Currently, when the application receives a request, the API Gateway Lambda integration starts the application. The lifecycle of this single process approach is diagrammed below.

If we break this into two processes, Data Processing, and API Handling, we give our API handler more dedicated time within that 29 seconds. Simultaneously, we overcome another limitation, large amounts of data being loaded will take more than 29 seconds, by breaking out data processing into its own process, we can now have up to 15 minutes to load data using Lambda.

We’re going to use the same docker image for both Lambdas by adding an entrypoint script that can take an argument to select the handler, essentially deploying the same application twice as different Lambda Functions.

Our entrypoint.sh

#!/usr/bin/env sh
set -e

. /app/venv/bin/activate

if [ -n "$LAMBDA_HANDLER" ]; then
 /app/venv/bin/python -m awslambdaric $LAMBDA_HANDLER
else
 arti "$@"
fi

Here’s what the two process lifecycles look like separated

We have, however, introduced a new issue, and that is persistence. Our single process approach always loads the data, and warm requests will have the data cached, so it was okay that the Chroma database was ephemeral, it could always be recreated on the fly.

When our corpus is many gigabytes or terabytes, losing and recreating those embeddings for the vector database is a costly endeavor, both in time, and very likely monetarily.

How do we solve for that?

Multi-tier architecture and data persistence

By putting our vector database into another tier, we don’t couple its lifecycle with our application’s. It’s pretty cool that we can have an ephemeral vector database in Lambda for demonstration purposes, but it’s not an optimal configuration for production.

One option, is to deploy Chroma ourselves using a service such as EC2 or ECS, for example, but that incurs costs. For the ease of this demonstration, we’re going to use Pinecone, which Pulumi conveniently has a provider for. Though, there are many alternative options.

Navigate to Pinecone’s homepage, and click “Get started.” After confirming your account, login to Pinecone, and click “API Keys” in the sidebar. Create a new API key for Arti.

After configuring our Pinecone API key in AWS Secrets Manager, we are able to define our Pinecone index in our __main__.py Pulumi program.

import pulumi
import pinecone_pulumi as pinecone

pinecone_index = pinecone.PineconeIndex(
 f"{app_name}-pinecone-index",
 name="main-index",
 dimension=1536,
 metric=pinecone.IndexMetric.COSINE,
 spec=pinecone.PineconeSpecArgs(
 pod=pinecone.PineconePodSpecArgs(
 environment="gcp-starter",
 pod_type="starter",
 replicas=1,
 )
 ),
)

pulumi.export("output", {"value": pinecone_index.host})

Handling data load events

In the proof-of-concept, we are abusing docker to distribute our data sources, this isn’t optimal for a few reasons, the size of the docker image increases, which also increases build times. Do our documents really belong in the container, what about privacy, triggering builds, lead time to get new data added, etc.?

The obvious next place to put our documents is AWS S3 (Simple Storage Service). We don’t want to regress our existing implementation, which works locally as well as deployed to Lambda, so we abstract the concept of loading a path of assets, whether it’s a local path or S3. In the presence of an S3 bucket name, assume we intend to use that loader, otherwise use the directory loader.

def get_loader_and_asset_root():
 """Select and configures the appropriate loader based on AWS S3 bucket presence."""
 app_bucket_name = os.getenv("APP_BUCKET_NAME")
 root_path = os.getenv("ASSETS_ROOT_PATH", "assets")
 asset_location = root_path

 # pylint: disable=import-outside-toplevel
 if app_bucket_name:
 from arti_ai.s3_loader import S3BucketLoader # type: ignore

 loader = S3BucketLoader()
 asset_location = f"{app_bucket_name}/{root_path}"
 else:
 from embedchain.loaders.directory_loader import DirectoryLoader # type: ignore

 loader = DirectoryLoader(config={"recursive": True})

 # pylint: enable=import-outside-toplevel

 return loader, asset_location


def load_data():
 """Load data from data sources."""
 print("Loading data")

 # pylint: disable=import-outside-toplevel
 from embedchain import App

 # pylint: enable=import-outside-toplevel

 app = App.from_config(config=config.get_embedchain_config())

 loader, asset_location = get_loader_and_asset_root()
 response = app.add(asset_location, loader=loader)

 return response

With our data loading logic figured out, we’ll pass the APP_BUCKET_NAME in our Pulumi program to our Lambda Function, so it can retrieve the data.

app_bucket = aws.s3.Bucket(
 f"{app_name}-bucket",
 bucket=f"{app_name}-bucket",
 # ...
)

lambda_environment = pulumi.Output.all(
 app_bucket.id,
).apply(
 lambda args: {
 "variables": {
 # ...
 "APP_BUCKET_NAME": args[0],
 # ...
 }
 }
)

app_event_lambda = aws.lambda_.Function(
 f"{app_name}-lambda",
 architectures=["x86_64"],
 environment=lambda_environment,
 image_uri=docker_image_lambda_event_handler.repo_digest,
 # ...
)

If you’re new to Pulumi, you might be wondering what’s going on with the lambda_environment, why isn’t app_bucket.id referenced directly, and what is pulumi.Output.all? Simply put, this tells Pulumi to wait until the bucket is created before trying to use its name in a string we’re building. This is an important core feature of Pulumi that largely happens by nesting Outputs. To learn more, see Inputs & Outputs.

Triggering the data loader

When deployed, we’re going to need something that doesn’t require a local machine. One method for loading data is to use a scheduled task. AWS EventBridge makes that task simple, our Pulumi resource declarations are below. We define a rule/schedule (pulled from stack configuration), add our target (the event handler Lambda), and setup permissions to invoke the Lambda function.

event_rule_schedule = aws.cloudwatch.EventRule(
 f"{app_name}-event-rule-schedule",
 schedule_expression=data_load_event_rule_schedule,
 state=data_load_event_rule_state,
)

event_rule_target = aws.cloudwatch.EventTarget(
 f"{app_name}-event-target-schedule",
 rule=event_rule_schedule.name,
 arn=app_event_lambda.arn,
)

event_rule_target_permission = aws.lambda_.Permission(
 f"{app_name}-lambda-event-target-permissions-schedule",
 action="lambda:InvokeFunction",
 function=app_event_lambda.name,
 principal="events.amazonaws.com",
 source_arn=event_rule_schedule.arn,
)

Our event lambda handler looks something like

import json

from arti_ai.app import load_data


def handler(event, context):
 print("Received event: " + json.dumps(event, indent=2))
 print("Context AWS request ID: " + json.dumps(context, indent=2))

 load_data()

 return {"statusCode": 200, "body": json.dumps("Event processed successfully!")}

Locally, we can use our command-line interface (CLI), defined in __main__.py

if args.command == "ask":
 handle_ask(args.question)
elif args.command == "load":
 load_data()
elif args.command == "reset":
 reset_data()
else:
 parser.print_help()

Any of our YAML configuration files should work with the following commands:

arti ask "what can you tell me about fruits and vegetables?"
arti load
arti reset
arti -h

You could manage your live Pinecone data this way, whether you should is highly dependent on the use case.

Wrapping up

We reflected on the project that was presented in the original post, and reviewed short-comings with the proof-of-concept, working through improvements that touched upon architecture, application design, and infrastructure. Specifically, we have moved application configuration out of code into YAML, decoupled data-loading from querying, broken out our vector database to another tier, and integrated our application with AWS EventBridge to trigger scheduled tasks.

Our delivery on the concept was expedited by utilizing services that aligned with our improvements, and Pulumi helped us achieve our architecture changes using familiar patterns in just a few lines of code.

In the next post in this series, we’ll continue to make iterative improvements to both our applications and infrastructure, and start getting deeper into RAG.

As always, we welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is easy and free. Get started today!

Happy AI building!

Build an AI Slack Bot on AWS Using Embedchain & Pulumi

Tyler Mulligan — Mon, 18 Mar 2024 17:21:02 +0100

The integration of artificial intelligence (AI) to improve user experiences is gaining popularity in today’s world. One fascinating application of AI is the creation of chatbots, which can engage users in conversation and provide helpful information or services.

In this blog post, we’ll explore the process of building an AI-powered Slack bot using Embedchain, a Retrieval-Augmented Generation (RAG) framework powered by LangChain. Additionally, we’ll deploy our bot on AWS using Pulumi, a modern infrastructure as code (IaC) platform.

What are LLMs and RAG chatbots?

Large Language Models (LLMs) are sophisticated AI models trained on vast amounts of text data, enabling them to generate human-like text responses. Retrieval-Augmented Generated (RAG) is a cutting-edge approach to chatbot development that combines the capabilities of LLMs with information retrieval techniques. RAG chatbots can generate responses based on both the input query and relevant information retrieved from a knowledge base, resulting in more contextually relevant and informative interactions.

What’s an Embedchain?

Embedchain is a RAG framework powered by LangChain. It simplifies the process of creating RAG applications by providing high-level abstractions that reduce the complexity of AI technologies. With Embedchain, developers can focus on defining conversational flows and configuring the bot’s behavior using YAML configuration files, rather than dealing with low-level implementation details.

Embedchain supports configuration for various data sources, LLMs, vector databases, embedding models, and evaluation.

Creating a Slack bot

Start by cloning the project:

git clone https://github.com/catmeme/arti.git
cd arti

The files within the src/arti_ai directory can be summarized as:

The fourth file isn’t strictly necessary, but it illustrates a design pattern. The main application, app.py, is imported by slack_app but also __main__.py. This makes it easy to add additional user interfaces, such as APIs, or other chat services with minimal code.

The AI application

Let’s start with “the brain”, our Embedchain application. Below is a simple example (using explicit configuration for clarity). This code expects that OPENAI_API_KEY is populated in your environment. export OPENAI_API_KEY=<your key> to experiment with the code below.

from embedchain import App

app_config = {
 "app": {
 "config": {
 "collect_metrics": False,
 "log_level": "INFO",
 }
 },
 "llm": {
 "provider": "openai",
 "config": {
 "model": "gpt-3.5-turbo-1106",
 "temperature": 0.5,
 "max_tokens": 1000,
 "top_p": 1,
 },
 },
 "embedder": {"provider": "openai"},
 "chunker": {"chunk_size": 2000, "chunk_overlap": 0, "length_function": "len"},
 "vectordb": {
 "provider": "chroma", # we're writing to a vectordb in memory in lambda
 "config": {"collection_name": "arti-ai", "dir": "/tmp/db", "allow_reset": True},
 },
}

app = App.from_config(config=app_config)
app.add("https://www.pulumi.com/docs/", data_type="docs_site")
app.query("What is Pulumi?")

See how easy it was for us to configure our application, an LLM, an embedding model, chunker and vector database? Then we added a datasource, which is abstracting away more complex logic for us. The simple line app.add("https://www.pulumi.com/docs/", "docs_site") ends up crawling the website, chunking the documents, generating embeddings and inserting them into our vector database.

The app_config reference is the Embedchain configuration schema.

The Slack application

Now that our brain is built, we need a way for our users to interact with it. Slack Bolt is an official library for building Slack bots.

Follow their instructions to create a new Slack app.

OAuth & permissions

chat:write
channels:read
commands
im:read
im:write
users:read
users:write

Event subscriptions

message.channels
message.im

Code breakdown

The example below demonstrates two commands. knock knock is a basic ping/pong test that you expect to respond immediately. It’s not doing any complex logic, so that’s easy. What’s going on with this /arti command then? Slack requires your bot respond in 3 seconds or less. In our serverless architecture utilizing AWS API Gateway and Lambda, we’re operating on thin margins when we have cold starts. This enables us to respond quickly to Slack, but start our longer running process.

It is worth noting that in this architecture we’re still operating under 29 seconds for API Gateway’s response timeout. This can be overcome in a number of ways, but that’s out of scope for this demonstration.

from slack_bolt import App

from arti_ai.app import ask_ai, config

slack_bot_token, slack_bot_signing_secret = config.get_slack_credentials()
app = App(process_before_response=True, token=slack_bot_token, signing_secret=slack_bot_signing_secret)


def process_request(respond, body):
 """Process the request and respond to the user."""
 question = body["text"]
 response = ask_ai(question)
 respond(f"Q: _{question}_ A: {response}")


def respond_to_slack_within_3_seconds(ack):
 """Respond to the user within 3 seconds."""
 ack("Thinking...")


app.command("/arti")(ack=respond_to_slack_within_3_seconds, lazy=[process_request])


@app.message("knock knock")
def ask_who(_message, say):
 """Ask who's there."""
 say("_Who's there?_")

Configuration

The configuration module in this application normalizes retrieval of secrets from environment variables or AWS Secrets Manager. You may have noticed the Embedchain application configuration is hardcoded into the app.py. This isn’t considered best practice, and may be a fun first step in making this project yours. Otherwise, stay tuned for future posts in this series.

Deploying on AWS with Pulumi

Once our bot is ready, we’ll deploy it on AWS using Pulumi. Pulumi allows us to define our cloud infrastructure using familiar programming languages like TypeScript or Python, making it easy to manage and scale our deployment.

It’s important to note that in this selected architecture, we’re working within a few limitations. Lambda’s only writable directory is /tmp and it’s limited to 500MB, and we need to use a Docker container to overcome the 250MB filesize limitation of our application, which we’ll be bundling our data with.

The Pulumi program is broken up into the following sections:

Config: Get configuration from stack configuration, environment, and set up resource tagging
Networking: Set up VPC, VPC endpoints, security groups
S3: Bucket for later use
Application: Docker, Lambda, policies
API: API Gateway
Output: Stack outputs

Resources are contained within the VPC, policies and security groups exercise principle of least privilege, and logging is enabled. lambda_api_handler.py is responsible for handling API Gateway requests and imports the Slack app.

Prerequisites

Create the following keys in Secrets Manager. These names are configurable in deploy/pulumi/Pulumi.<stack>.yaml. Slack and Pinecone are only necessary if they are configured for use.

Secret Name	Schema
/catmeme/cloudplatform/sandbox/arti/access-token/openai	`{ "apiKey": "" }`
/catmeme/cloudplatform/sandbox/arti/access-token/slack	`{ "apiKey": "", "signingSecret": "" }`

Makefile

Makefiles offer a powerful, flexible, and efficient way to manage project tasks, contributing to smoother development processes, better team collaboration, and more reliable product builds. See the project README.md for more information.

cp -R /path/to/my/assets ./assets
cp sample.env .env
python3 -m venv venv
. venv/bin/activate
make develop

Ensure you have a matching AWS profile name to the one in the stack. The Makefile assumes <environment>-deployment. This script will attempt to copy the deploy/pulumi/Pulumi.catmeme.arti.sandbox.us-east-1.yaml as a base for configuration of the stack you deploy.

make deploy DEPLOY_ENVIRONMENT=dev

At this point, you should be able to use /arti to interact with your data via Slack.

Conclusion

In this blog post, we’ve explored the process of building an AI-powered Slack bot using Embedchain and deploying it on AWS with Pulumi. By harnessing the capabilities of LLMs and RAG chatbots, we can create intelligent conversational agents that enhance user interactions and provide valuable insights. With Embedchain’s high-level abstractions and Pulumi’s infrastructure as code platform, developers can streamline the development and deployment of AI-powered applications, unlocking new possibilities for innovation in the digital space.

As always, we welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is easy and free. Get started today!

Happy AI building!

Deploy Low-Code LLM Apps on AWS with Flowise and LocalAI

Engin Diri — Mon, 26 Feb 2024 06:00:00 -0800

Building low-code LLM applications just got easier! With Flowise, LocalAI, and Pulumi, you can quickly create, deploy, and manage AI-powered workflows on AWS EKS—without writing complex code.

In this guide, you’ll learn how to:

Set up a low-code LLM application using Flowise & LocalAI
Deploy it on AWS EKS with Pulumi & TypeScript
Create a chatbot workflow and expose it via API

Flowise simplifies LLM orchestration with a drag-and-drop interface, while LocalAI enables local inference without requiring a GPU. Pulumi streamlines cloud infrastructure deployment with just a few lines of code.

Whether you’re a developer looking to experiment with LLMs on AWS or an AI enthusiast exploring serverless AI applications, this step-by-step tutorial will guide you through the entire process. Let’s dive in!

Flowise: The Low-Code Tool for LLM Orchestration

Flowise is an open-source low-code/node tool for developers to build customized LLM orchestration workflows and AI agents. The low-code approach helps developers to cut down on the time and effort required to build LLM applications by delivering quick iterations to from testing to production. In Flowise nodes are referred to as integrations, and you can build a customized LLM orchestration flow, chatbot, or AI agent by dragging and dropping these integrations. There are plenty of ready-to-use integrations available in Flowise, like LangChain or LlamaIndex.

LocalAI: Open-Source LLM Inference for AWS & On-Prem

LocalAI is a free and Open Source alternative to OpenAI. It acts as a drop-in replacement for OpenAI’s REST API that is fully compatible with the OpenAI API specifications for local inference. It allows you to run LLMs, generate images, or audio locally or on-prem with consumer grade hardware supporting multiple model families and architectures. And the clue: It does not require a GPU, it runs on CPU only (optional GPU acceleration is still available).

Once LocalAI loaded the model the first time, it keeps it in memory for faster inference. It does not shell-out, but use bindings to the model for faster inference and better performance. It is a great way to test and benchmark different models directly on your local machine or on-prem. Additionally, you are not sending your data to a third-party service, which may contain sensitive information. This is a big advantage, especially in large enterprises or in the public sector where data privacy is a big concern.

Deploying Flowise & LocalAI on AWS with Pulumi

Covered a lot of the basics, but let’s get started with the real thing. To quickly get started, you can use the example project from the Pulumi examples repository.

The AWS architecture of the example project is as follows:

The Pulumi AWS architecture consists of the following resources:

An EKS cluster with a managed node group.
EBS volume support for the EKS cluster to store the downloaded LocalAI models.
Deployment of the Helm chart for Flowise and LocalAI.
A VPC, two subnets in different availability zones, and an internet gateway, and security groups for the EKS cluster.

As programming language, I will use TypeScript, but you can use any other language supported by Pulumi. Rewriting the example code should be straightforward. Feel free to reach out to us if you need help.

Create a Flowise Chatbot

Our first Flowise workflow will be a simple chatbot. This simple LLM chain will be stateless (no memory) with a prompt template and our LocalAI LLM model attached.

Deploy the Infrastructure with Pulumi

First clone the example repository, and head to the aws-ts-localai-flowise directory:

git clone https://github.com/pulumi/examples.git
cd examples/aws-ts-localai-flowise

To deploy the infrastructure with Pulumi, you can use the following command:

pulumi login # if you haven't logged in yet
pulumi up

Next to the definition of the AWS EKS cluster, the Pulumi program uses the pulumi-kubernetes provider to deploy the Helms charts for Flowise and LocalAI.

// imports omitted for brevity
import * as k8s from "@pulumi/kubernetes";

// code omitted for brevity

// Manually creation of the Kubernetes provider, with the kubeconfig from the EKS cluster and the enableServerSideApply set to true
const provider = new k8s.Provider("k8s", {
 kubeconfig: cluster.kubeconfigJson,
 enableServerSideApply: true,
});

// Deploy the Helm charts for LocalAI
const localai = new k8s.helm.v3.Release("local-ai", {
 chart: "local-ai",
 version: "3.2.0",
 repositoryOpts: {
 repo: "https://go-skynet.github.io/helm-charts",
 },
 forceUpdate: true,
 namespace: "local-ai",
 createNamespace: true,
 values: {
 deployment: {
 image: {
 repository: "quay.io/go-skynet/local-ai",
 tag: "latest",
 },
 env: {
 debug: "true",
 context_size: 512,
 modelsPath: "/models",
 },
 }, resources: {
 requests: {
 cpu: "8",
 memory: "32Gi",
 },
 },
 models: {
 list: [
 {
 url: "https://gpt4all.io/models/ggml-gpt4all-j.bin",
 name: "ggml-gpt4all-j",
 },
 ],
 },
 persistence: {
 models: {
 size: "50Gi",
 storageClass: "gp2",
 accessModes: "ReadWriteOnce",
 },
 output: {
 size: "10Gi",
 storageClass: "gp2",
 accessModes: "ReadWriteOnce",

 },
 },
 },
}, {
 provider: provider,
});

// Deploy the Helm charts for Flowise
const flowise = new k8s.helm.v3.Release("flowise", {
 chart: "flowise",
 version: "2.5.0",
 repositoryOpts: {
 repo: "https://cowboysysop.github.io/charts/",
 },
 namespace: "flowise",
 createNamespace: true,
}, {
 provider: provider,
});

Depending on your AWS region, the deployment can take a few minutes. Once the deployment is complete, you will have access to kubeconfig file to access the EKS cluster for the next steps.

To retrieve the kubeconfig file, you can use the following command:

pulumi stack output kubeconfig --show-secrets > kubeconfig.yaml

Now you can use the kubeconfig.yaml file either with kubectl or with the KUBECONFIG environment variable to access the EKS cluster.

As we are not using a load balancer, we will use the kubectl port-forward command to access the Flowise UI. The command is:

kubectl port-forward svc/$(kubectl get svc -n flowise | grep flowise | awk '{print $1}') 3000:3000 -n flowise

Start modeling the Chatbot with Flowise

Open the Flowise UI in your browser with the following URL: http://localhost:3000 and click the Chartflows menu item in the left sidebar. And then click the blue button + Add New

This should open a new window where you can drag and drop the nodes to the main window. To display the available nodes, click the + button on the left.

First we add our LocalAI node to the chartflow. Look under Chat Models for the ChatLocalAI node and drag it to the main canvas.

Fill out the form with the following values:

Base Path: This should be name ouf your LocalAI Kubernetes service. You can find it with the following command:
```
kubectl get svc -n local-ai
```
And should be in the form of http://<svc-name>.local-ai.svc.cluster.local:80
Model Name: The name of the model you want to use. You can find the available models with the following command:
```
kubectl exec -it $(kubectl get pods -n local-ai | grep local-ai | awk '{print $1}') -n local-ai -- curl http://localhost:8080/models | jq '.data[0].id'
```
This will return the name of the model you can use. For example ggml-gpt4all-j_f5d8f27287d3
Temperature: The temperature of the model. This is a float value between 0 and 1. The higher the value, the more creative the model will be. The lower the value, the more conservative the model will be. Keep the default value of 0.9 for now.

Now we can add the next node to our workflow. Search for Prompt Template and drag this node to the main canvas as well. You can fill out the Template field with the following value:

Question: {input}.

As a last step we need to add the LLM Chain node to the workflow. Head over to the Chains section of our nodes menu and drag the LLM Chain node to the main canvas. Give the chain a name chatbot.

This done, we can start to connect different nodes to each other. Start with the Prompt and Language Model nodes by dragging the arrow from the Language Model dot to the ChatLocalAI node. Same for the Prompt dot to the Prompt Template node.

Now we can save the chartflow and test it. Click the Save icon in the top right corner and then click the Dialog button to display the chatbot. You can now test the chatbot by typing a question in the input field and see the response. In my case, I asked the chatbot How does an airplane fly in the air? and got the following response:

There are several methods by which an airplane flies in the air, including:

Lift and Weight: An airplane flies by exploiting its weight in the vertical direction. Therefore, it is necessary for an aircraft to have a strong and heavy enough structure that can hold its weight in the air. The more an airplane weighs, the greater is its lift and climb capability.
Air Density: An airplane exploits the difference in air density below and above it. Airborne objects experience the same buoyant force that causes them to rise. By doing this, an aircraft can maintain a stable position and ascend or descend.
Airfoil Design: The shape of an aircraft’s airfoil is designed such that it allows for the efficient and effective generation of lift force. The shape, thickness and angle control the air flow over the wings of an aircraft.
Control Arm: The control bar controls the angle of attack, pitch and yaw control. By moving the bar, pilots can change and adjust these features of the aircraft.
Propulsion Systems: An airplane requires propulsion systems to provide the thrust needed for takeoff, climb and movement through the air. Engines such as piston, turbojet, diesel engine or rocket engines are commonly used. The combustion process in these systems produces heat, which provides thrust.
Radio Communication: Aircraft can communicate with air traffic control in real-time, allowing pilots and air traffic controllers to coordinate their actions. This communication is essential for safe and efficient air travel.

Go ahead and test your chatbot with different questions! It is a great way to test different LLM models and see how they behave.

Expose the Chatbot via API

If you want to expose the chatbot workflow via API or SDK, you can use Flowise for this as well. Flowise provides a REST API for this purpose. Head in your workflow view to the top right corner and click the </> button. This will open a new dialog with all the different methods you can use. This range from embedding the chatbot in your website to using to directly call the REST API.

There is even the option to enable authorization for the REST API. This is a great way to secure your chatbot and only allow authorized users to access it.

Housekeeping

If you want to destroy the stack, you can use the following command:

pulumi destroy

The Pulumi CLI will ask you to confirm the changes, and if you are happy with the changes, you can confirm them by typing yes and all the resources will be deleted.

In Conclusion

In this blog post, we have seen how easy it is to build a LLM application with Flowise and LocalAI without the need to use any code. This is a great way to quickly prototype and test different LLM models and workflows. We have also seen how easy it is to deploy the infrastructure with Pulumi and how easy it is to destroy the stack if we don’t need it anymore.

Want to get hands on with LangServe and walk through it live? We have two upcoming dates for a workshop focused on LangChain and LangServe!

Deploying LangChain Applications on AWS with Pulumi on March 14, 2024
Deploying LangChain Applications on AWS with Pulumi again on April 17, 2024

As always, we welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is easy and free. Get started today!

Happy AI building!

Deploy LangServe Apps with Pulumi on AWS (RAG & Chatbot)

Engin Diri — Tue, 13 Feb 2024 06:00:00 -0800

We all know how easy it is to create, deploy, and manage any cloud infrastructure with Pulumi using your favorite programming language. With the rise of artificial intelligence (AI) more and more developers are working on LLM-powered applications and services. And with this, the need to have the same ease of use for creating, deploying, and managing the infrastructure for these applications is growing.

In this blog post, we will show you how to this can be achieved with combining Pulumi and LangServe.

What is LangServe and How Does it Work with LongChain?

LangServe is part of the awesome 🦜️🔗 LangChain framework. LangChain simplifies the process of creating generative AI application interfaces. Generative AI applications are powered by large language models (LLMs) and deliver an answer to your question (called prompts) in natural language. This becomes even more interesting and useful once you integrate with your own private data and APIs. Some popular use cases could be:

Ask a question over your company’s internal knowledge base
Summarize a document
Text extraction or data labeling

Building these types of integrations often needs to build pipelines (called typically chains), starting from the prompt and enriching the prompt with your own data. Here is where LangChain comes into play.

LangChain gives you all the abstractions you need to start building your own LLM app. It has many components ready to use for you, like LLMs, chat and text embedding modes, and vector stores, and many more.

Once we have our prototype chain ready, we can now use LangServe to expose it as a REST API.

Getting Started with LangServe and Pulumi on AWS

After all the theory, let’s get our hands dirty and start building our first LangServe app with Pulumi on AWS. For this we can use one of the LangServe example projects from the Pulumi examples repository.

The AWS architecture you will get out of the box will look like this:

The Pulumi LangServe AWS architecture consists of the following components:

Docker Image resource, to build and push the LangServe app to the AWS Elastic Container Registry (ECR).
AWS Application Load Balancer (ALB) to expose the LangServe app to the public internet.
AWS Fargate to run the LangServe app in a serverless manner.
AWS Parameter Store to store the LangServe app’s configuration.
Some IAM roles and policies to allow the different components to interact with each other.
A VPC, Subnets, and Security Groups to isolate the LangServe app from the rest of your infrastructure.

I will use the Python example (aws-py-langserve), but you can use any of the other supported language. The examples are available here and ready to use as a base for our two demo projects.

Create a Gandalf Chatbot App 🧙‍♂️

Please make sure that you have OPENAI_API_KEY environment variable set with your OpenAI API key.

Our first example is a chatbot app with a twist. We will use LangChain to create chain that will answer questions about Middle-earth as Gandalf the wizard. The chain will be exposed as a REST API with LangServe and deployed with Pulumi.

First clone the example repository, and head to the aws-py-langserve directory:

git clone https://github.com/pulumi/examples.git
cd examples/aws-py-langserve

In the app folder create a new Python file called chain.py and add the following code:

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

_prompt = ChatPromptTemplate.from_template(
 "You are Gandalf the Grey, with his deep knowledge of Middle-earth, 'Let me tell you about {text}'"
)
_model = ChatOpenAI()
chain = _prompt | _model

Here we define a new chain with a prompt and a model. We provide the prompt to the model with a surrounding text to set up the context for the model. The model will then answer the prompt with a response.

This can be as simple or complex as you want. In our case, we will keep it simple, and tell the model that it is Gandalf and that he has deep knowledge of Middle-earth. Then we pass the text to the model and let it answer.

Open the server.py file and add the following code:

from app.chain import chain as gandalf_chain

# ... rest of the code

add_routes(
 app,
 gandalf_chain,
 path="/openai",
)

You can test the chain locally with the following command:

langchain serve

You can either use the playground or curl to test the chain. The playground is a web interface where you can test the chain. You can access it by opening your browser and navigating to http://localhost:8000/openai/playground.

Or you can use curl to test the chain:

curl http://127.0.0.1:8000/openai/stream -X POST -H "Content-Type: application/json" --data '{"input": {"text": "orcs"}}'

# ... omitted output
event: data
data: {"content":"ondrous","additional_kwargs":{},"type":"AIMessageChunk","example":false}

event: data
data: {"content":" land","additional_kwargs":{},"type":"AIMessageChunk","example":false}

event: data
data: {"content":".","additional_kwargs":{},"type":"AIMessageChunk","example":false}

event: data
data: {"content":"","additional_kwargs":{},"type":"AIMessageChunk","example":false}

event: end

To deploy the chain with Pulumi, you can use the following command:

pulumi login # if you haven't logged in yet
pulumi config set open-api-key --secret # your openai api key
pulumi up

This will first give you an outline of all the resources that will be created and then ask you to confirm the changes. If you are happy with the changes, you can confirm them by typing yes.

After the deployment is finished, you will get the URL of the deployed LangServe app. You can use this URL to test the app either with the playground or curl. Same as you did locally. The URL will look like this:

https://<your-alb-dns-name>/openai/playground

If you want to destroy the stack, you can use the following command:

pulumi destroy

The Pulumi CLI will ask you to confirm the changes, and if you are happy with the changes, you can confirm them by typing yes.

Create a Pinecone RAG App

Please make sure that you have the following API keys set:

PINECONE_API_KEY environment variable set with your Pinecone API key. You can get it from the Pinecone console.
OPENAI_API_KEY environment variable set with your OpenAI API key.

Our second example will show you how to create a RAG (Retrieval-Augmented Generation) app with Pinecone vector databases. The clue of this example here is to show you how easy you can extend the Pulumi LangServe example to deploy additional cloud resources. In this case, we will deploy a Pinecone index to use it in our LangServe app. Pulumi and Pinecone recently released a dedicated Pulumi provider for Pinecone. This provider allows you to create and manage Pinecone indexes with Pulumi.

As an external knowledge base, we will use Wikipedia to load information about Gandalf the Grey and ingest them into our Pinecone index.

First clone the example repository, and navigate to the aws-py-langserve directory:

git clone https://github.com/pulumi/examples.git
cd examples/aws-py-langserve

To use the Pulumi Pinecone provider, you need to install it first:

echo "pinecone_pulumi==0.4.0" >> requirements.txt

Now open the __main__.py file and add the following code at the end of the file:

import pinecone_pulumi as pinecone

# ... omitted code
pinecone_provider = pinecone.Provider("pinecone_provider",
 api_key=pinecone_api_key)

pinecone_rag_index = pinecone.PineconeIndex("pinecone-rag-index",
 name="test",
 metric=pinecone.IndexMetric.COSINE,
 spec=pinecone.PineconeSpecArgs(
 serverless=pinecone.PineconeServerlessSpecArgs(
 cloud=pinecone.ServerlessSpecCloud.AWS,
 region="us-west-2",
 ),
 ),
 opts=pulumi.ResourceOptions(provider=pinecone_provider))

Now we need to pass the Pinecone API key to the AWS Parameter Store. Open the __main__.py file and this following line after line 38:

# ... omitted code
pinecone_api_key = config.get("pinecone-api-key")
# ... omitted code

And add line 226 a new secret to the AWS Parameter Store:

# ... omitted code
langserve_ssm_parameter_pinecone = aws.ssm.Parameter("langserve-ssm-parameter-pinecone",
 type="SecureString",
 value=pinecone_api_key,
 key_id=langserve_key.key_id,
 name=f"/pulumi/{pulumi_project}/{pulumi_stack}/PINECONE_API_KEY",
 tags={
 "pulumi-application": pulumi_project,
 "pulumi-environment": pulumi_stack,
 })
# ... omitted code

And to the IAM policy in line 260:

# ... omitted code
"Resource": [langserve_ssm_parameter_arn, langserve_ssm_parameter_pinecone.arn, ],
# ... omitted code

Last but not least, change the Fargate task definition to use the Pinecone API key. Add a new element in line 338:

# ... omitted code
container_definitions = pulumi.Output.all(langserve_ecr_image.repo_digest, langserve_ssm_parameter.name,
 langserve_log_group.name, langserve_ssm_parameter_pinecone.name).apply(
 lambda repo_digest, langserve_ssm_parameter_name, langserve_log_group_name,
 langserve_ssm_parameter_pinecone_name: json.dumps([{
# ... omitted code

And add a new array element to line 349:

# ... omitted code
{
 "name": "PINECONE_API_KEY",
 "valueFrom": langserve_ssm_parameter_pinecone_name
}
# ... omitted code

This should create your Pinecone index and update the Fargate task definition to use the Pinecone API key; this is what we are going to need to use the Pinecone index in our LangServe app.

To use the Pinecone index in our LangServe app, we need first to add following packages to our LangChain app:

poetry add pinecone-client
poetry add langchain-community
poetry add langchain-openai
poetry add langchain
poetry add 'tiktoken>=0.5.2,<0.6.0'
poetry add wikipedia

Now we can add chain.py in the app directory and add the following code:

import os

from langchain_openai import ChatOpenAI
from langchain_community.embeddings import OpenAIEmbeddings
from langchain_community.vectorstores import Pinecone
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.pydantic_v1 import BaseModel
from langchain_core.runnables import RunnableParallel, RunnablePassthrough

if os.environ.get("PINECONE_API_KEY", None) is None:
 raise Exception("Missing `PINECONE_API_KEY` environment variable.")

PINECONE_INDEX_NAME = os.environ.get("PINECONE_INDEX", "test")

### Ingest code - you may need to run this the first time
# Load
from langchain_community.document_loaders import WikipediaLoader

loader = WikipediaLoader(query="Gandalf the Grey", load_max_docs=50)
data = loader.load()

# # Split
from langchain.text_splitter import RecursiveCharacterTextSplitter

text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)
all_splits = text_splitter.split_documents(data)

# # Add to vectorDB
vectorstore = Pinecone.from_documents(
 documents=all_splits, embedding=OpenAIEmbeddings(), index_name=PINECONE_INDEX_NAME
)
retriever = vectorstore.as_retriever()
### End Ingest code

vectorstore = Pinecone.from_existing_index(PINECONE_INDEX_NAME, OpenAIEmbeddings())
retriever = vectorstore.as_retriever()

# RAG prompt
template = """Answer the question based only on the following context:
{context}
Question: {question}
"""
prompt = ChatPromptTemplate.from_template(template)

# RAG
model = ChatOpenAI()
chain = (
 RunnableParallel({"context": retriever, "question": RunnablePassthrough()})
 | prompt
 | model
 | StrOutputParser()
)


# Add typing for input
class Question(BaseModel):
 __root__: str


chain = chain.with_types(input_type=Question)

Head over to the server.py file and add the following code:

from app.chain import chain as rag_chain

# ... rest of the code
add_routes(
 app,
 rag_chain,
 path="/rag-pinecone",
)

To deploy the chain with Pulumi, you can use the following command:

pulumi login # if you haven't logged in yet
pulumi config set open-api-key --secret # your openai api key
pulumi config set pinecone-api-key --secret # your pinecone api key
pulumi up

This will deploy create the AWS infrastructure, and the Pinecone index and then containerize and deploy the LangServe app to AWS Fargate.

After a couple of minutes, you will see in the Pinecone console that the index has been created and the first documents have been ingested from Wikipedia about Gandalf the Grey.

Here a screenshot of the Pinecone console:

And if you want to test the LangServe app, you can use the playground UI, where you should be able to ask questions about Gandalf the Grey and get answers based on the documents ingested into the Pinecone index.

When you’re ready to destroy the stack, you can use the following command:

pulumi destroy

The Pulumi CLI will ask you to confirm the changes, and if you are happy with the changes, you can confirm them by typing yes and all the resources will be deleted, including the Pinecone index.

Conclusion

In this blog post, we showed you how easy it is to create, deploy, and manage LangServe apps with Pulumi. The example projects from the Pulumi examples repository provide an excellent starting point to build your own LLM-powered apps. In the second example, we showed you how to extend the Pulumi code to deploy additional cloud resources, in this case a Pinecone serverless index to hold custom data for our Gandalf app.

You might also like:

As always, we welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is easy and free. Get started today!

Happy AI building!

10x Your Storage Performance with Amazon S3 Express One Zone and Pulumi

Joe Duffy — Mon, 04 Dec 2023 00:00:00 +0000

At AWS re:Invent 2023, AWS announced the new Amazon S3 Express One Zone storage class. This new service provides incredible performance by collocating your S3 buckets closer to the workloads that use the data. Compared to the S3 Standard storage class, the Express One Zone storage class is up to 10x faster, handles 100,000s of requests per second, offers single-digit millisecond latency, and can reduce request costs by 50%. This can be extremely beneficial for data-intensive workloads such as AI/ML, media, finance, realtime, and high-performance computing scenarios. This blog post shows how to get started with Amazon S3 Express One Zone using Pulumi infrastructure as code.

Introducing Amazon S3 Express One Zone

The new Amazon S3 Express One Zone storage class extends S3 with a new low latency option. This can improve runtime performance by spending less time moving data around. When creating a new bucket, you choose an Availability Zone (AZ) in which that bucket will be placed, which of course should be collocated with the compute workload that will consume that data.

A bucket must be designated as an Express One Zone bucket at creation time. There are a few things that are unique to these buckets that make them different from ordinary S3 buckets:

First, they are called “directory buckets” in AWS tools and in your infrastructure as code. This means how you interact with them sometimes differs from general-purpose S3 buckets.
They have a specific naming convention: [name]--[az]--x-s3. The [name] part is a bucket name with the usual restrictions. [az] is the ID of the AZ in which the bucket will reside. Note that this is an ID (e.g. usw2-az1), not the name (e.g. us-west-1c); read more here about the difference.
These buckets use region-specific endpoints, not the region-agnostic aws-global S3 endpoint scheme that general-purpose S3 buckets utilize. This means you’ll need a new AWS CLI or SDK that understands the new endpoint scheme in order to access these buckets (CLI version 2.14.5 and greater supports them). It also means that sometimes when running aws s3 and aws s3api commands, you will need to specify a --region explicitly. Other aspects of accessing these buckets differ, such as authorization, but the AWS SDK generally takes care of these for you.

This new storage class is not universally available in all regions and AZs yet, although Amazon has plans to expand regions over time. At the time of launch, us-east-1 (US North Virginia), us-west-2 (US Oregon), eu-north-1 (Europe Stockholm), and ap-northeast-1 (Asia Pacific Tokyo) are supported. Finally, because One Zone is explicitly single-AZ, there is always the risk of availability issues or data loss in the event that a single AZ becomes unavailable or is damaged; plan accordingly.

New BucketDirectory Resource Type

You can create a new Amazon S3 Express One Zone bucket using the aws s3api create-bucket CLI command or in the AWS console. For this blog post, however, you’ll use infrastructure as code instead, so that provisioning, updating, and managing our buckets is automated, reliable, and easy to scale.

To do this, you will use Pulumi’s new aws.s3.DirectoryBucket resource type in the AWS package. It takes a few parameters:

bucket, which is the name (conforming to the above-mentioned [name]--[az]--x-s3 convention)
dataRedundancy, which currently just accepts the sole value "SingleAvailabilityZone"
location, which specifies the AZ ID to use
forceDestroy, an optional boolean which, when set to true, allows deleting buckets that still have data in them (by default, only empty buckets can be deleted, to prevent accidental data loss)

Here is a simple program that declares a new Amazon S3 Express One Zone bucket:

import * as aws from "@pulumi/aws";
import * as pulumi from "@pulumi/pulumi";
import * as random from "@pulumi/random";

// Pick a single AZ for our S3 Express bucket.
const azId = aws.getAvailabilityZones().then(azs => azs.zoneIds.sort()[0]);

// Name our bucket according to S3 Express requirements ([name]--[az]--x-s3).
const bucketName = "my-bucket";
const suffix = new random.RandomString("bucket-name", {
 length: 6, upper: false, special: false });
const bucketFullName = pulumi.interpolate
 `${bucketName}-${suffix.result}--${azId}--x-s3`;

// Provision the bucket and export its ID.
const bucket = new aws.s3.DirectoryBucket(bucketName, {
 bucket: bucketFullName,
 dataRedundancy: "SingleAvailabilityZone",
 location: {
 type: "AvailabilityZone",
 name: azId,
 },
});

export const bucketID = bucket.id;

import pulumi
import pulumi_aws as aws
import pulumi_random as random

# Pick a single AZ for our S3 Express bucket.
az_ids = aws.get_availability_zones().zone_ids
az_ids.sort()
az_id = az_ids[0]

# Name our bucket according to S3 Express requirements ([name]--[az]--x-s3).
bucket_name = 'my-bucket'
suffix = random.RandomString(
 'bucket-name', length=6, upper=False, special=False)
bucket_full_name = pulumi.Output.format(
 '{0}-{1}--{2}--x-s3', bucket_name, suffix.result, az_id)

# Provision the bucket and export its ID.
bucket = aws.s3.DirectoryBucket(bucket_name,
 bucket = bucket_full_name,
 data_redundancy = 'SingleAvailabilityZone',
 location = {
 'type': 'AvailabilityZone',
 'name': az_id,
 },
)

pulumi.export('bucketID', bucket.id)

There are a few things to note here:

We’re picking the first AZ automatically in whatever region we’ve deployed to. Of course, we could hard-code this, or use a different technique for choosing an AZ. This approach makes it trivial to deploy the same project to multiple regions.
We use a random string suffix for our bucket name, just to avoid accidental collisions if we were to pick the same bucket name as another user (or even ourselves when deploying multiple stacks for the same project). Normally Pulumi uses auto-naming to avoid this problem by default, but because the bucket name must conform to a very precise format, it does not for this resource type.

Now let’s say we actually want to populate our bucket. We can use the standard aws.s3.BucketObjectv2 resource to do this. This program adds an object for every file in our project’s data directory, for example:

import * as fs from "fs";
import * as path from "path";

// ...

// Add a bucket object for every data file.
for (const file of fs.readdirSync("data")) {
 new aws.s3.BucketObjectv2(`my-file-${file}`, {
 bucket: bucket.id,
 source: new pulumi.asset.FileAsset(path.join("data", file)),
 }, { parent: bucket });
}

import os

# ...

# Add a bucket object for every data file.
for file in os.listdir('data'):
 path = os.path.join('data', file)
 if os.path.isfile(path):
 aws.s3.BucketObjectv2(f'my-file-{file}',
 bucket = bucket.id,
 source = pulumi.asset.FileAsset(path),
 )

Creating and Accessing an Express One Zone Bucket

Let’s go ahead and spin up a bucket. Assuming we have the above code in a valid Pulumi project (created with pulumi new), let’s ensure it is configured to one of the regions that supports Amazon S3 Express One Zone, such as us-west-2:

$ pulumi config set aws:region us-west-2

Now let’s create some random files in our data directory:

$ mkdir data
$ for i in {0..9}; do \
 dd if=/dev/urandom of=data/file${i}.txt bs=100K count=1; \
 done

Next, we’ll run pulumi up which will show us a preview of the deployment and ask us to confirm, which we’ll do:

$ pulumi up
 Type Name Status
 + pulumi:pulumi:Stack s3-express-one-zone-py-dev created (4s)
 + ├─ random:index:RandomString bucket-name created (0.09s)
 + ├─ aws:s3:DirectoryBucket my-bucket created (2s)
 + ├─ aws:s3:BucketObjectv2 my-file-file2.txt created (1s)
 + ├─ aws:s3:BucketObjectv2 my-file-file3.txt created (1s)
 + ├─ aws:s3:BucketObjectv2 my-file-file1.txt created (1s)
 + ├─ aws:s3:BucketObjectv2 my-file-file4.txt created (1s)
 + ├─ aws:s3:BucketObjectv2 my-file-file5.txt created (1s)
 + ├─ aws:s3:BucketObjectv2 my-file-file7.txt created (2s)
 + ├─ aws:s3:BucketObjectv2 my-file-file8.txt created (2s)
 + ├─ aws:s3:BucketObjectv2 my-file-file9.txt created (2s)
 + ├─ aws:s3:BucketObjectv2 my-file-file0.txt created (3s)
 + └─ aws:s3:BucketObjectv2 my-file-file6.txt created (3s)

Outputs:
 bucketID: "my-bucket-z5tn74--usw2-az1--x-s3"

Resources:
 + 13 created

Duration: 11s

And there we go – our new Amazon S3 Express One Zone bucket is available and its ID has been printed. We can check out its contents:

$ aws s3api list-objects-v2 \
 --region=us-west-2 \
 --bucket $(pulumi stack output bucketID)

Note that the storage class of these objects is listed as EXPRESS_ONEZONE:

{
"Contents": [
{
"Key": "my-file-file6.txt",
"LastModified": "2023-12-03T17:18:06+00:00",
"ETag": "\"1254d05e685245ef9fdcca10f3c547bc\"",
"ChecksumAlgorithm": [
"CRC32"
],
"Size": 102400,
"StorageClass": "EXPRESS_ONEZONE"
},
...
{
"Key": "my-file-file2.txt",
"LastModified": "2023-12-03T17:18:04+00:00",
"ETag": "\"d5310d3efd394c96bf5bf4379fc81bff\"",
"ChecksumAlgorithm": [
"CRC32"
],
"Size": 102400,
"StorageClass": "EXPRESS_ONEZONE"
}
]
}

We can also see our bucket and its contents in the S3 Management Console:

General-purpose buckets and directory buckets have different tabs. There are also utilities available to import data from other buckets and perform batch operations.

Benchmarking Express One Zone Performance

To demonstrate how much faster Express One Zone buckets are to access, let’s run a few benchmarks. To do this, we set up two buckets – an Express One Zone and another using Standard storage class – along with an EC2 instance in the same AZ as our One Zone bucket.

For the first test, we’ll allocate 100 files, each 10MB in size, for a total of 1GB of data. We’ll then run an aws s3 sync from our EC2 instance to copy the entirety of the bucket locally. We’ll loop this 10 times and simply measure how long it took for the copy to complete. Obviously this only tests read performance, but it will immediately give us a sense of how much faster Express One Zone can be:

Standard class: 8m5s
Express One Zone class: 2m41s
Speedup: 3.0x

This was a nice bump in performance, however, the new storage class really shines for smaller objects. Storage latency typically impacts small files much more than larger files, as storage overheads and latencies become more of a bottleneck than streaming the data over the network. For the second test, we’ll try 100,000 files, each 100K in size instead – still 1GB total, but comprised of many smaller files:

Standard class: 30m45s
Express class: 5m36s
Speedup: 5.5x

From 3x to more than 5x the performance! With more fine tuning of our sync parallelism and other factors, we could have eked out even more gains, but these simple tests quickly showed us how much faster Express One Zone can be for some very basic examples.

In summary, here are the results:

Conclusion

In this post, we learned all about Amazon S3 Express One Zone, how to manage new buckets and their contents with infrastructure as code, and saw the awesome performance gains in action. To be up and running with Amazon S3 Express One Zone and infrastructure as code in minutes, go ahead and get started with Pulumi for free today. Happy cloud spelunking!

Pulumi and Redis Cloud: Real-Time Data for Modern Apps

Josh Kodroff — Tue, 28 Nov 2023 00:00:00 +0000

Redis Enterprise Cloud makes apps faster by providing an in-memory database that enables teams to create a real-time data platform. In this post, you’ll learn how to enable connectivity between Redis Enterprise Cloud and your workloads using Pulumi and the Redis Cloud provider.

The full code for this example is available at https://github.com/pulumi/examples/tree/master/redis-cloud-aws-ts. The full code differs slightly from the example below in that it has some parameterization that makes the code more portable, whereas in this post we show hardcoded values for simplicity.

About Redis Enterprise Cloud

Redis Enterprise Cloud is a fully managed database as-a-service (DBaaS) provider that offers managed Redis clusters. Redis Enterprise Cloud clusters can be accessed via network peering in AWS or Google Cloud, or over the internet. The Redis Cloud provider allows you to manage Redis Enterprise Cloud resources in Pulumi programs.

Some of Redis Enterprise Cloud’s key features include:

Automated failover, backups, and scaling.
Linear scaling based on a serverless architecture, with an optimized engine that provides higher throughput and lower latencies compared to solutions based on Redis’ open-source version.
A subscription-based pricing pricing model based on consumption (as opposed to being billed hourly on provisioned hardware).

Architecture

One of the most common scenarios for using Redis Enterprise Cloud is to run a workload in a hyperscaler (like AWS or Google Cloud), and then to establish a low-latency connection between your workload and your managed Redis cluster via network peering. In this post, you’ll create an architecture for a sample AWS workload. The architecture you’ll build will include:

A Redis Enterprise Cloud database cluster.
An AWS VPC across 3 Availability Zones, with public and private subnets.
VPC Peering to enable connectivity between the Redis cluster and the VPC.

You’ll also prove the architecture works by spinning up an EC2 instance and running simple commands that target the Redis cluster.

The following diagram shows the major components of the architecture:

Project Initialization and Redis Enterprise Cloud Resources

Before you begin this tutorial, you will need to add a payment method in the Redis Cloud console.

To begin, initialize your Pulumi program in an empty directory:

pulumi new typescript

Then, add a reference to the Redis Cloud provider:

npm i @rediscloud/pulumi-rediscloud

Then, in your index.ts file, add use the getPaymentMethodOutput function to obtain your payment information, and use the resulting card ID to create a Redis Subscription:

Please note the following:

The Redis Cloud provider does not support Fixed subscriptions. You’ll need to create a Flexible subscription in order to use the provider to manage resources with Pulumi.
The snippet below uses hard-coded values for things like payment info or CIDR blocks for simplicity. In a multi-stack Pulumi program, these should be refactored to configurable values.

import * as pulumi from "@pulumi/pulumi";
import * as rediscloud from "@rediscloud/pulumi-rediscloud";

const card = rediscloud.getPaymentMethodOutput({
 cardType: "<YOUR-CARD-TYPE>" // e.g. "Visa" (capitalization matters here),
 lastFourNumbers: "1234",
});

const subscription = new rediscloud.Subscription("redis-subscription", {
 name: "pulumi-redis-aws-example",
 paymentMethod: "credit-card",
 paymentMethodId: card.id,
 cloudProvider: {
 regions: [
 {
 region: "us-east-1",
 // This CIDR block must not conflict with the CIDR block of the AWS
 // VPC to which you wish to connect:
 networkingDeploymentCidr: "10.0.0.0/24",
 // We use a single AZ here to minimize the cost incurred from running
 // this example. For a production workload, consider multiple availability
 // zones:
 multipleAvailabilityZones: false,
 preferredAvailabilityZones: ["use1-az1"],
 },
 ],
 },

 creationPlan: {
 memoryLimitInGb: 10,
 quantity: 1,
 replication: true,
 supportOssClusterApi: false,
 throughputMeasurementBy: "operations-per-second",
 throughputMeasurementValue: 20000,
 modules: ["RedisJSON"],
 },
});

Then, create a database in your subscription:

const database = new rediscloud.SubscriptionDatabase("redis-db", {
 name: "my-db",
 subscriptionId: subscription.id,
 protocol: "redis",
 memoryLimitInGb: 10,
 dataPersistence: "aof-every-1-second",
 throughputMeasurementBy: "operations-per-second",
 throughputMeasurementValue: 20000,
 replication: true,
});

AWS Networking Resources

Now you are ready to add your AWS resources. First, you’ll need to add some reference to your Pulumi program. Add the Pulumi AWS Provider, which provides basic AWS resources, and the AWSX package, which provides a convenient VPC component:

npm i @pulumi/awsx @pulumi/aws

Now you can add some AWS resources to your program. Start by adding import statements for the packages you added to our project, then create a VPC with with the AWSX package:

import * as awsx from "@pulumi/awsx";
import * as aws from "@pulumi/aws";

const vpc = new awsx.ec2.Vpc("vpc", {
 cidrBlock: "10.1.0.0/16", // Cannot conflict with the Redis CIDR block,
 natGateways: {
 strategy: "Single"
 }
});

Next, peer the Redis Cloud subscription with your AWS VPC:

const callerIdentity = aws.getCallerIdentity();

const peering = new rediscloud.SubscriptionPeering("redis-peering", {
 subscriptionId: subscription.id,
 region: region,
 awsAccountId: callerIdentity.then(x => x.accountId),
 vpcId: vpc.vpcId,
 vpcCidr: vpc.vpc.cidrBlock,
});

new aws.ec2.VpcPeeringConnectionAccepter("aws-peering-accepter", {
 vpcPeeringConnectionId: peering.awsPeeringId,
 autoAccept: true
});

To complete network connectivity between your Redis Cloud database and your AWS VPC, add routes to the private subnets in which your workloads will run that allow traffic to to through the peering connection to the Redis Cloud database’s network:

vpc.privateSubnetIds.apply(ids => {
 ids.forEach((id, index) => {
 const routeTable = aws.ec2.getRouteTableOutput({
 subnetId: id,
 });

 new aws.ec2.Route(`peering-route-${index}`, {
 routeTableId: routeTable.id,
 destinationCidrBlock: redisVpcCidr,
 vpcPeeringConnectionId: peering.awsPeeringId,
 });
 });
});

If you are unfamiliar with the call to apply(), you can learn more about why this is necessary in Inputs and Outputs. Creating resources within apply is usually not necessary, but this is one of the few cases where it is.

Now, deploy the infrastructure using the pulumi up command:

pulumi up

Creating an EC2 Instance to Test

To test the architecture, create an EC2 instance running Amazon Linux 2 and enable AWS SSM Systems Manager to allow you easy access to a shell prompt without having to worry about SSH keys. From the shell prompt, you will run the Redis CLI to prove that network connectivity is working as expected.

First, create a Security Group to allow all egress traffic. This will allow connectivity to Redis, SSM Systems Manager, and HTTP/S to update the underlying operating system and install dependencies:

const sg = new aws.ec2.SecurityGroup("instance-sg", {
 description: "Allow all egress traffic.",
 vpcId: vpc.vpcId,
 egress: [{
 cidrBlocks: ["0.0.0.0/0"],
 description: "Allow all",
 protocol: "-1",
 fromPort: 0,
 toPort: 0,
 }]
});

Next, create an EC2 instance role that grants the necessary permissions to use SSM Systems Manager:

const instanceRole = new aws.iam.Role("instance-role", {
 assumeRolePolicy: JSON.stringify({
 "Version": "2012-10-17",
 "Statement": {
 "Effect": "Allow",
 "Principal": {
 "Service": "ec2.amazonaws.com",
 },
 "Action": "sts:AssumeRole",
 },
 })
});

new aws.iam.RolePolicyAttachment("instance-role-attachment", {
 role: instanceRole.name,
 policyArn: "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore",
});

const instanceProfile = new aws.iam.InstanceProfile("instance-profile", {
 role: instanceRole.name,
});

Then, create the EC2 instance based on the latest Amazon Linux 2 AMI. Note that the userData input of the EC2 instance resource installs the Redis CLI, which is not included in the base operating system by default:

const amazonLinux2 = aws.ec2.getAmiOutput({
 mostRecent: true,
 owners: ["amazon"],
 filters: [
 { name: "name", values: ["amzn2-ami-hvm-*-x86_64-gp2"] },
 { name: "owner-alias", values: ["amazon"] },
 ]
});

const instance = new aws.ec2.Instance("instance", {
 ami: amazonLinux2.id,
 instanceType: "t3.micro",
 vpcSecurityGroupIds: [sg.id],
 subnetId: vpc.privateSubnetIds[0],
 tags: {
 Name: "pulumi-redis-cloud-tester"
 },
 iamInstanceProfile: instanceProfile.name,
 userData: `#!/bin/bash
 sudo amazon-linux-extras install redis6
 `
});

And again, deploy your stack:

pulumi up

If you’re new to Pulumi, you may notice that when you run the pulumi up command this time, Pulumi does not attempt to re-create the resources you’ve already declared. This is because Pulumi is a declarative tool: You only need to specify the desired end state of your resources, and Pulumi will figure out what steps (if any) are necessary to get from your infrastructure’s current state to your declared desired state.

Testing the Architecture

Now that you have your EC2 instance up and running, it’s time to verify that you can connect to your Redis cluster and run commands:

In the AWS Console, find the EC2 instance named pulumi-redis-cloud-tester, and connect to it via SSM Systems Manager:
In the Redis Cloud console, locate your database, and select “Connect”:
In the connection dialog, copy the connection info for the Redis CLI:
Paste the command into the Systems Manager shell prompt, which will connect you to the Redis instance. From there you can run Redis commands like INCR that will write to the Redis cluster:

Now your workloads can use your Redis Cloud cluster! You can tear down all the resources you created with the following command:

pulumi destroy

Next Steps and Conclusion

Now that you have proven that the architecture works, here are some next steps you might want to take with this codebase:

Replace the EC2 test instance with a real workload that uses Redis:
- If your workload is running on EC2, then the code in this post should get you up and running. However, if you find you need some help Pulumi Neo can be a very helpful resource. Simply enter a natural language query and select a support Pulumi language, and Pulumi AI will generate a Pulumi program that can be copy/pasted into your program and iterated on in the same Pulumi AI conversation.
- If you’re using another AWS compute platform, Pulumi also has helpful guides for creating workloads on ECS, EKS, or Lambda.
Try creating Redis Cloud clusters peered with Google Cloud.
If you’re comfortable using Pulumi and are thinking of adopting it at a team or organizational level, consider creating an organization in Pulumi Cloud and enjoying a 2-week free trial of our full set of features, no credit card required!

With Redis Cloud you can enjoy lightning-fast access to whatever data your application needs. When Redis Cloud is coupled with Pulumi, you can manage your Redis databases and all your other cloud infrastructure using a single tool, in any language. That’s powerful stuff!

Dynamic Credentials for Every AWS CLI Command: A Step-by-Step Guide to 'pulumi env run'

Torian Crane — Fri, 27 Oct 2023 00:00:00 +0000

In a world where cloud computing is the backbone of modern applications, managing environments and secrets is of the utmost importance. Earlier this month we released a new service called Pulumi ESC (Environments, Secrets, and Configuration), the focus of which is to help alleviate the burden of managing cloud configuration and secrets by providing a centralized way to handle these critical aspects of cloud development. It’s like having a Swiss Army knife in your toolkit, ready to tackle the challenges of cloud infrastructure. This post will highlight the specific challenge of credentials management, and we’ll specifically dive into how using the pulumi env run functionality of Pulumi ESC will make that easier.

Credentials Management in the Cloud

Dealing with cloud credentials can be a major headache, especially when you’re trying to manage them at scale. If you’ve ever agonized over things like how to:

securely share credentials with teammates in a consistent way,
minimize the risks associated with locally configured, long-lived and highly privileged credentials,
ensure teams can easily and safely run commands like aws s3 ls without needing to be a security wizard,

then this post has got you covered. The pulumi env run command was made to solve exactly these kinds of problems.

If we take a look at the Pulumi documentation for the pulumi env run command, it states the following:

This command opens the environment with the given name and runs the given command. If the opened environment contains a top-level ’environmentVariables’ object, each key-value pair in the object is made available to the command as an environment variable.

But what does this actually mean? Using AWS as an example, it means that we can run the aws s3 ls command without having to manually configure AWS credentials in local environments because Pulumi ESC will dynamically generate and project those credentials on our behalf. In this post, we’ll show you how to use Pulumi ESC with the AWS CLI so that every command you run incorporates security best practices like short-term, scoped credentials issued via OpenID Connect (OIDC).

Getting Started with pulumi env run

Now that we’ve covered the basics of what pulumi env run does, let’s dive into how you can start using it! We’ll walk you through everything you need to be able to run the aws s3 ls command without manually configuring local AWS credentials.

Step 1: Install and log in to the Pulumi CLI

To begin, you’ll need to install the Pulumi CLI. Once the installation is complete, run the pulumi login command and follow the steps to log in to the CLI.

$ pulumi login
Manage your Pulumi stacks by logging in.
Run `pulumi login --help` for alternative login options.
Enter your access token from https://app.pulumi.com/account/tokens
or hit <ENTER> to log in using your browser :
Logged in to pulumi.com as …

Step 2: Create the OIDC configuration

Pulumi ESC offers you the ability to manually set your credentials as secrets in your Pulumi ESC environment files. When it comes to something like OIDC configuration, a more secure and efficient alternative is to leverage yet another great feature of Pulumi ESC: dynamic credentials.

This service can dynamically generate credentials on your behalf each time you need to interact with your AWS environments. To do so, you’ll need to configure OIDC between Pulumi and AWS. On the AWS side, there are two resources you’ll need to create:

An IAM OIDC provider
An IAM Role

Let’s start by creating the OIDC provider resource. There are a number of ways you can do this in AWS, but we will demonstrate how to create it via the AWS console.

Please note that while we’re providing steps and screenshots that are accurate as of the date of this post, AWS documentation is subject to change. For the most current and precise information, always refer to the official AWS documentation.

a. Add identity provider

In the navigation pane of the IAM console, choose Identity providers, and then choose Add provider.

In the Provider type section, click the radio button next to OpenID Connect. For the Provider URL, provide the following URL:

https://api.pulumi.com/oidc

For the Audience field, provide the name of your Pulumi organization. Then scroll down to the bottom of the page and click Add provider.

b. Assign IAM role

You’ll see a notification at the top of your screen prompting you to assign an IAM role to your newly created provider. Click the Assign role button.

This will prompt you to either create a new role or use an existing role. For the purposes of this blog, we will select the Create a new role option and demonstrate how to configure it for use with Pulumi ESC.

This will then direct you to the IAM Create role page. The Web identity radio button should already be selected for you but if not, make sure that it is selected. Then scroll down and make sure that the following is selected in the Web identity section:

the api.pulumi.com/oidc provider is selected under Identity provider
the name of your Pulumi organization is selected under Audience

Click Next which will direct you to the Add permissions page. This is where you will select what permissions you will allow your Pulumi ESC environment to have in your AWS account. Since we want to interact with S3, let’s go ahead and select the AmazonS3FullAccess policy from the list. Then scroll down and click Next.

You will be directed to the last page of the wizard, where you can provide the name and an optional description for the role and review all of the role’s configuration details before creating it.

c. Review trust policy

One area that we want to draw your attention to is the Select trusted entities section which is where the trust policy of the role is defined.

This definition currently allows any Pulumi service to assume this role, but only if the request comes from your organization. You can edit this policy to further limit access to this role to just the Pulumi ESC service, and you can make it even more granular by limiting access to a specific environment. This more detailed configuration is outside the scope of this blog post, but you can learn more about it by checking out the relevant Pulumi documentation.

After creating your role, make sure to make a note of your role’s ARN value as you will need it in the next step.

Step 3: Create a new environment

Now that we’ve created all the necessary AWS resources, let’s move over to the Pulumi side of things. You’ll first want to create a new environment in the Pulumi Cloud. Make sure that you have the correct organization selected in the left-hand navigation menu. Then click the Environments link, and click the Create environment button. In the following pop-up, provide a name for your environment before clicking the Create environment button.

Step 4: Add the AWS provider integration

Once you’ve created your new environment, you will be presented with a split-pane document view. You’ll want to clear out the default placeholder content in the editor on the left-hand side and replace it with the following code, making sure to replace <your-oidc-iam-role-arn> with the value of your IAM role ARN from the previous step:

values:
 aws:
 login:
 fn::open::aws-login:
 oidc:
 duration: 1h
 roleArn: <your-oidc-iam-role-arn>
 sessionName: pulumi-environments-session
 environmentVariables:
 AWS_ACCESS_KEY_ID: ${aws.login.accessKeyId}
 AWS_SECRET_ACCESS_KEY: ${aws.login.secretAccessKey}
 AWS_SESSION_TOKEN: ${aws.login.sessionToken}

The variables defined under the environmentVariables parameter above are the same environment variables that the AWS CLI uses if you were authenticating with something like the aws configure command. What the above configuration is doing is dynamically generating the credentials and projecting those credential values into your local environment. From there, the AWS CLI picks up those environment variables and runs the designated command. You can find out more about this provider definition and how it works in the Pulumi ESC documentation for the AWS provider as well as the documentation for projecting environment variables.

Click Save.

Step 5: Run your command

Now to the fun part! With your environment set up, try it out by listing all of the S3 buckets in your AWS account. Start by running the aws s3 ls command as normal, making sure to run it in an environment that does not have any AWS credentials configured. You should see the following response:

$ aws s3 ls
Unable to locate credentials. You can configure credentials by running "aws configure".

Now run the command using pulumi env run as shown below, making sure to replace <your-pulumi-org-name> and <your-environment-name> with the names of your own Pulumi organization and environment respectively:

pulumi env run <your-pulumi-org-name>/<your-environment-name> -i aws s3 ls

Voila! You’ve just run an AWS command without any local AWS credential configuration.

The Power of pulumi env run

This is just the tip of the iceberg of how the pulumi env run command can be a game-changer in how you and your team manage and interact with cloud resources. It’s a significant stride towards making your cloud interactions more efficient and less error-prone, and here’s a deeper dive into why:

Seamless Command Execution - The pulumi env run command lets you execute AWS commands effortlessly, freeing you from the intricacies of managing AWS credentials on your local machine. As a developer, you could get temporary credentials for a particular environment to check if your application is working as expected. Simply put, it significantly reduces the overhead of credential setup and maintenance.
Enhanced Security - One of the standout features of pulumi env run is its commitment to security. By removing the local storage of credentials, it drastically reduces the risk of accidental exposure. Your credentials and secrets are securely managed within the Pulumi environment.
Streamlined Collaboration - Because credentials will be centralized, pulumi env run facilitates smoother team collaboration by providing a consistent environment for all team members to run commands with. Everyone can access the same secure environment which reduces the complexities of coordinating credentials and configurations across teams.

Real-World Use Cases

To further illustrate the power of Pulumi ESC, let’s consider a real-world use case: automated data backup. Imagine you have a web application that generates daily logs stored in an S3 bucket. You want to schedule a daily backup job to copy these logs to a separate bucket without exposing your AWS credentials in your scripts or having to manually configure them each time.

With Pulumi ESC, you can create an environment, dynamically generate your AWS credentials, and schedule a daily backup job using a simple script that runs the aws s3 cp command. This ensures your backups are automated, secure, and easily maintainable, all thanks to Pulumi ESC.

This is one example from several dozen use cases possible via Pulumi ESC. Other examples include but are not limited to:

Disaster Recovery:
- Aside from data backups, Pulumi ESC can also be used in disaster recovery scenarios. The pulumi env run command can be utilized to automate the recovery processes, ensuring quicker restoration of services with the correct configurations.
Environment Segregation and Management:
- In organizations with multiple development, staging, and production environments, Pulumi ESC facilitates easy segregation and management of these environments. The pulumi env run command allows developers to execute commands in designated environments without the hassle of manual configuration changes.
Continuous Integration/Deployment (CI/CD) Pipelines:
- Pulumi ESC can be integrated into CI/CD pipelines to automate deployment processes. The pulumi env run command can be used to execute deployment scripts in a controlled and secure manner, making the deployment process more efficient and error-free.
Application Testing and Debugging:
- The pulumi env run command facilitates smooth testing and debugging processes by allowing developers to run test scripts and debugging commands in designated environments with the necessary configurations set automatically.

Conclusion

Pulumi ESC, and the pulumi env run command in particular, is designed to make your cloud journey smoother, more secure, and less daunting. So, the next time you find yourself drowning in a sea of cloud configurations, remember, with Pulumi ESC and a dash of code, you can say goodbye to manual configuration hassles and hello to efficient, secure, and scalable cloud development with Pulumi ESC.

Want to learn more about Pulumi? Our Get Started with Pulumi hub is a great place to start.

If you’re looking for an automated (and faster) way to configure OIDC with Pulumi, we’ve got you covered! Check out our Examples repo where we offer a wide variety of automations using Pulumi Infrastructure as Code (IaC). For OIDC configuration in particular, we have example automation for:

Feel free to join our community on Slack and let us know what you think!

Using Go Generics with Pulumi

Zaid Ajaj — Wed, 20 Sep 2023 00:00:00 +0000

Pulumi loves Go, it’s what powers Pulumi. We’ve kept a close eye on the design and development of support for generics in the Go programming language over the years, a feature that allows developers to write type-safe, concise, and reusable code. We’ve been exploring what it’d look like to improve Pulumi’s Go SDKs with generics and recently published a public RFC detailing our plans. We’ve been making progress on the implementation and are excited to announce preview support for Go generics in our core and AWS Go SDKs. If you’re using Go with Pulumi, we’d love for you to give it a try and share your feedback!

// Given
var a pulumi.IntOutput
var b pulumi.StringOutput

// Before (could panic at runtime if you got something wrong)
o := pulumi.All(a, b).ApplyT(func(vs []interface{}) string { // could panic
 a := vs[0].(int) // could panic
 b := vs[1].(string) // could panic
 return strconv.Itoa(a) + b
}).(pulumi.StringOutput) // could panic

// After (compile-time type-safety)
o := pulumix.Apply2(a, b, func(a int, b string) string {
 return strconv.Itoa(a) + b
})

Why Generics?

Before looking at what’s available in the preview, let’s first review the motivations for adding support for generics, from the RFC.

Unsafe Inputs, Outputs, and Apply

A common complaint about the Pulumi Go SDK is the lack of type safety. The Input and Output types, and the apply operation rely heavily on interface{}. This makes them effectively untyped at compile-time, instead relying on type matching at runtime.

For instance, the apply operation is implemented in Go as the following method:

func (*OutputState) ApplyT(applier interface{}) Output

The contract for ApplyT states that applier must be a function with one of the following signatures:

func(U) T
func(U) (T, error)

Where U must be assignable from the type contained in the underlying Output. If it isn’t, the operation will panic.

The caller will usually want to cast the result into a specific output type. If they choose the wrong output type, this operation will panic.

So given a typical usage like the following, there are two spots that can panic: the ApplyT invocation and the IntOutput cast.

intOutput := stringOutput.ApplyT(func(v string) int {
 return len(v)
}).(pulumi.IntOutput)

Excessive code generation

The Go SDK relies heavily on code generation in an attempt to alleviate the pain of untyped operations. It generates type-specific input and output wrapper types for nearly every type it encounters.

In the core Pulumi SDK, this includes all primitive types, their pointers, containers, and containers of containers. For instance, the core SDK has 22 String* types besides type String itself, covering nearly all variations of the following pattern:

String(Ptr|(Array|Map)?(Array|Map))?(Input|Output)?

This pattern of code generation carries over to code generated for provider SDKs, though not to this degree of nesting. Every type gets an input and output type, a pointer variant if the type is referenced as a pointer, and container variants if the type is referenced from a container.

In general, this generates extremely large packages that are ungainly to navigate—both in source form and in their API reference.

Introducing Go Generics in the Pulumi Go SDK

We can address the above by leveraging Go generics to incorporate type-safety into the core concepts (inputs, outputs, and apply) and reduce the amount of generated code.

Starting with v3.80 of the Pulumi Go SDK, we have added a new pulumix subpackage (containing “extensions”), which includes new generic Input[T] and Output[T] types as well as type-safe generics-based APIs to work with them such as Apply.

With the new support for generics, the previous untyped example can now be written in a type-safe manner as follows:

intOutput := pulumix.Apply(stringOutput, func(v string) int {
 return len(v)
})

This is checked at compile-time: if you mistakenly specified an incorrect type, it will be caught at compile-time rather than panicking at runtime.

We recommend using Go 1.21 or later, which has improved type inference for generics, allowing you to write the call to Apply above without having to explicitly specify generic type parameters.

Adopting Generics in Your Pulumi Programs

Here’s a larger example that uses the non-generic pulumi.All function and ApplyT method to produce a JSON string representing a policy for a BucketPolicy:

package main

import (
 "encoding/json"

 "github.com/pulumi/pulumi-aws/sdk/v7/go/aws/cloudfront"
 "github.com/pulumi/pulumi-aws/sdk/v7/go/aws/s3"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 bucket, err := s3.NewBucket(ctx, "content-bucket", &s3.BucketArgs{
 Acl: pulumi.String("private"),
 Website: &s3.BucketWebsiteArgs{
 IndexDocument: pulumi.String("index.html"),
 ErrorDocument: pulumi.String("404.html"),
 },
 })
 if err != nil {
 return err
 }

 originAccessIdentity, err := cloudfront.NewOriginAccessIdentity(ctx, "cloudfront", &cloudfront.OriginAccessIdentityArgs{
 Comment: pulumi.Sprintf("OAI-%s", bucket.ID()),
 })
 if err != nil {
 return err
 }

 _, err = s3.NewBucketPolicy(ctx, "cloudfront-bucket-policy", &s3.BucketPolicyArgs{
 Bucket: bucket.ID(),
 Policy: pulumi.All(bucket.Arn, originAccessIdentity.IamArn).ApplyT(
 func(args []any) (string, error) {
 bucketArn := args[0].(string)
 iamArn := args[1].(string)

 policy, err := json.Marshal(map[string]any{
 "Version": "2012-10-17",
 "Statement": []map[string]any{
 {
 "Sid": "CloudfrontAllow",
 "Effect": "Allow",
 "Principal": map[string]any{
 "AWS": iamArn,
 },
 "Action": "s3:GetObject",
 "Resource": bucketArn + "/*",
 },
 },
 })
 if err != nil {
 return "", err
 }
 return string(policy), nil
 }).(pulumi.StringOutput),
 }, pulumi.Parent(bucket))
 if err != nil {
 return err
 }

 return nil
 })
}

pulumi.All is used to combine bucket.Arn and originAccessIdentity.IamArn into a single Output and then uses ApplyT construct a policy in JSON using the values.

If we wanted to take advantage of the type-safety provided by the new pulumix subpackage, we could rewrite the above example to use pulumix.Apply. First, import "github.com/pulumi/pulumi/sdk/v3/go/pulumix". Then, replace the use of pulumi.All and ApplyT with the generic pulumix.Apply:

 Policy: pulumix.Apply2Err(bucket.Arn, originAccessIdentity.IamArn,
 func(bucketArn, iamArn string) (string, error) {
 policy, err := json.Marshal(map[string]any{
 "Version": "2012-10-17",
 "Statement": []map[string]any{
 {
 "Sid": "CloudfrontAllow",
 "Effect": "Allow",
 "Principal": map[string]any{
 "AWS": iamArn,
 },
 "Action": "s3:GetObject",
 "Resource": bucketArn + "/*",
 },
 },
 })
 if err != nil {
 return "", err
 }
 return string(policy), nil
 }),

Here, we’re specifically using pulumix.Apply2Err, a variant of pulumix.Apply that accepts two generic arguments and a callback function that supports returning an error. There are other variants of pulumix.Apply that accept up to 8 generic arguments, and variants that return an error and accept a context.Context.

The result of our call to pulumix.Apply2Err in this example is pulumix.Output[string]. We can pass this directly to the Policy argument because it happens to be weakly typed as pulumi.Input, which pulumix.Output[T] implements.

If the Policy argument had been typed as pulumi.StringInput, for example, we would have had to cast the pulumix.Output[string] to pulumi.StringOutput using the pulumix.Cast function:

policy := pulumix.Apply2Err(bucket.Arn, originAccessIdentity.IamArn,
 func(bucketArn, iamArn string) (string, error) {
 // ...
 })

// ...

Policy: pulumix.Cast[pulumi.StringOutput](policy),

Here’s the updated example in full:

package main

import (
 "encoding/json"

 "github.com/pulumi/pulumi-aws/sdk/v7/go/aws/cloudfront"
 "github.com/pulumi/pulumi-aws/sdk/v7/go/aws/s3"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumix"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 bucket, err := s3.NewBucket(ctx, "content-bucket", &s3.BucketArgs{
 Acl: pulumi.String("private"),
 Website: &s3.BucketWebsiteArgs{
 IndexDocument: pulumi.String("index.html"),
 ErrorDocument: pulumi.String("404.html"),
 },
 })
 if err != nil {
 return err
 }

 originAccessIdentity, err := cloudfront.NewOriginAccessIdentity(ctx, "cloudfront", &cloudfront.OriginAccessIdentityArgs{
 Comment: pulumi.Sprintf("OAI-%s", bucket.ID()),
 })
 if err != nil {
 return err
 }

 _, err = s3.NewBucketPolicy(ctx, "bucket-policy", &s3.BucketPolicyArgs{
 Bucket: bucket.ID(),
 Policy: pulumix.Apply2Err(bucket.Arn, originAccessIdentity.IamArn,
 func(bucketArn, iamArn string) (string, error) {
 policy, err := json.Marshal(map[string]any{
 "Version": "2012-10-17",
 "Statement": []map[string]any{
 {
 "Sid": "CloudfrontAllow",
 "Effect": "Allow",
 "Principal": map[string]any{
 "AWS": iamArn,
 },
 "Action": "s3:GetObject",
 "Resource": bucketArn + "/*",
 },
 },
 })
 if err != nil {
 return "", err
 }
 return string(policy), nil
 }),
 }, pulumi.Parent(bucket))
 if err != nil {
 return err
 }

 return nil
 })
}

Notice how we are importing pulumix alongside pulumi. Being a subpackage means you can start using the generics-based types and functions alongside the current APIs as necessary without having to switch your code fully from using the old types to the new generic ones. Instead, you can migrate your code gradually to these new APIs where it makes sense. When needed, there are APIs to convert between generics-based APIs and the old ones using functions such as pulumix.Cast and pulumix.ConvertTyped.

Generics-Based Cloud Provider SDKs

We’ve seen how the pulumix subpackage on its own can be used with existing Pulumi provider SDKs. We can go further and adopt generics in the provider SDKs themselves. To make the transition non-breaking and gradual for your Pulumi programs, we intend to publish generic-based variants of the provider Go SDKs under a subpackage called x (for “extensions”). This allows you to migrate your code resource by resource.

Migrating an AWS Program to leverage Go generics

The following example is an AWS Pulumi program which we’ll migrate into the generics-based API. Here is what pulumi new aws-go gives you today:

package main

import (
 "github.com/pulumi/pulumi-aws/sdk/v7/go/aws/s3"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // Create an AWS resource (S3 Bucket)
 bucket, err := s3.NewBucket(ctx, "my-bucket", &s3.BucketArgs{
 Website: &s3.BucketWebsiteArgs{
 IndexDocument: pulumi.String("index.html"),
 },
 })
 if err != nil {
 return err
 }

 // Export the name of the bucket
 ctx.Export("bucketName", bucket.ID())
 return nil
 })
}

To start using the generics-based version of the S3 module from AWS, you need to change the import from "https://github.com/pulumi/pulumi-aws/sdk/v7/go/aws/s3" to "https://github.com/pulumi/pulumi-aws/sdk/v7/go/aws/x/s3" (notice the x after aws/), then change input values to use pulumix.Ptr instead of pulumi.String.

Rather than passing values as pulumi.String("foo") and pulumi.Int(42), with generics, you can simply use pulumix.Val("foo") and pulumix.Val(42). Similarly, you can use the pulumix.Ptr function for pointer values.

package main

import (
 "github.com/pulumi/pulumi-aws/sdk/v7/go/aws/x/s3"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumix"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // Create an AWS resource (S3 Bucket)
 bucket, err := s3.NewBucket(ctx, "my-bucket", &s3.BucketArgs{
 Website: &s3.BucketWebsiteArgs{
 IndexDocument: pulumix.Ptr("index.html"),
 },
 })
 if err != nil {
 return err
 }

 // Export the name of the bucket
 ctx.Export("bucketName", bucket.ID())
 return nil
 })
}

You’ll also need to update your go.mod to use the preview branch of the Pulumi AWS Go SDK that includes the x subpackage with generic APIs. Run the following:

go get github.com/pulumi/pulumi-aws/sdk/v7@generics
go mod tidy

The changes look minimal, except the fact that the inputs of s3.BucketArgs and the outputs of Bucket are now fully generics-based. Here is the Bucket type you get from NewBucket:

type Bucket struct {
 pulumi.CustomResourceState
 AccelerationStatus pulumix.Output[string] `pulumi:"accelerationStatus"`
 Acl pulumix.Output[*string] `pulumi:"acl"`
 Arn pulumix.Output[string] `pulumi:"arn"`
 Bucket pulumix.Output[string] `pulumi:"bucket"`
 // ....
}

And here are its input arguments:

type BucketArgs struct {
 AccelerationStatus pulumix.Input[*string]
 Acl pulumix.Input[*string]

 Arn pulumix.Input[*string]
 Bucket pulumix.Input[*string]
 // ...
}

Authoring Generics-based Go SDKs

For Pulumi package authors, it is possible to start generating Go SDKs for your providers that include the generics-based variant under the x subpackage using v3.84.0 of Pulumi. By default, this is not enabled so you will have to explicitly opt-in to it using a specific option in the go language section of your Pulumi schema. The new option is called "generics" and it has three possible values:

"none" is the default value when the option is not set. This doesn’t generate the generics-based variant
"side-by-side" is the option we are using for our select providers which generates the variant under the x subdirectory.
"generics-only" if you want your provider to only emit generics-based APIs. Since this is a preview feature, things might still be rough around the edges, so use this option with caution as fixes in the SDK-generation code might result in breaking changes in your APIs.

Here is how this looks in JSON:

{
 "language": {
 "go": {
 "generics": "side-by-side"
 }
 }
}

We eventually intend to make "side-by-side" be the default, such that all providers get a generics-based variant under the x subpackage, so that all users can use the full generics API.

Try it out!

We are excited for you to try out these new APIs and would love to hear your feedback! You can learn more details on the design and add comments to the RFC, or chat with us in the #golang channel on the Pulumi Community Slack.

If you’d like to learn more about building cloud applications with Go and Pulumi, we have a workshop coming up in October. Sign up here.

We’ll also be at GopherCon, if you’d like to meet with us.

Deploy AI Models on Amazon SageMaker using Pulumi Python IaC

Kat Morgan — Mon, 11 Sep 2023 00:00:00 +0000

Running models from Hugging Face on Amazon SageMaker is a popular deployment option for AI/ML services. While the SageMaker console allows for provisioning these cloud resources, this deployment pattern is labor intensive to document and vulnerable to human errors when reproducing as a regular operations practice. Infrastructure as Code (IaC) offers a reliable and easy to duplicate deployment practice. By developing this IaC with Pulumi, practitioners can choose to write their infrastructure code in Python and seamlessly develop both AI application code and IaC code in the same language.

In this short tutorial we will deploy a publicly available Meta AI LlaMa 2 based model from Hugging Face, on Amazon SageMaker. Then we will test it with a natural language prompt using a short Python script.

Pulumi makes AI/ML Easy?

We will use the sagemaker-aws-python Pulumi Template to bootstrap our Python SageMaker IaC. Templates allow you to quickly bootstrap new Pulumi projects from a working scaffold which you can then customize further to your needs. Out of the box, this template provisions Amazon IAM Roles to assign SageMaker privileges, CloudWatch alarms to alert in case of latency or error spikes for the endpoint, and of course a Meta AI LlaMa 2 based LLM (NousResearch/Llama-2-7b-chat-hf) hosted on Hugging Face.

Requirements

Instructions

Let’s begin by logging into Pulumi Cloud:

# There are many ways to store Pulumi state, here we use Pulumi Cloud
# Other state backends include S3, local file, and more
pulumi login

If you’re unsure of which Pulumi state backend you’re using, you can check by running pulumi whoami (or pulumi whoami -v for more info).

2. Prepare a new Pulumi project

Here we create our new Pulumi project directory and populate it from the sagemaker-aws-python Pulumi template

# Create a new directory & change directories into it
mkdir newSageMaker && cd newSageMaker

# Start your project from the sagemaker-aws-python template
# Follow along with the prompts to create your new project and initialize a stack
pulumi new sagemaker-aws-python

While creating a new project from the sagemaker-aws-python Pulumi template, you will be prompted for a project name, description, stack name, and Amazon Web Service Region. You can proceed with defaults, or supply your own.

After responding to all prompts, the pulumi new command will proceed to setup a Python Virtual Environment (venv) and download all dependencies into the venv.

3. Deploy your model as a new SageMaker endpoint

This step may take between 10 and 20 minutes while Amazon builds your infrastructure and deploys the configured model. You can follow along in the console as resources are provisioned, or open the link displayed in terminal to view the deployment status and other stack information in Pulumi Cloud.

pulumi up

4. Try your new SageMaker endpoint

Once your stack has finished deploying, use this rudimentary Python snippet to test the deployed SageMaker endpoint.

NOTE: Notice that we are using us-east-1 in this script. Be sure to change the region in Python to match the region you deployed the SageMaker endpoint into.

First, save the following python snippet as test.py:

import json, boto3, argparse

def main(endpoint_name):
 client = boto3.client('SageMaker-runtime', region_name='us-east-1')
 payload = json.dumps({"inputs": "In 3 words, name the biggest mountain on earth?"})
 response = client.invoke_endpoint(EndpointName=endpoint_name, ContentType="application/json", Body=payload)
 print("Response:", json.loads(response['Body'].read().decode()))

if __name__ == "__main__":
 parser = argparse.ArgumentParser()
 parser.add_argument("endpoint_name")
 main(parser.parse_args().endpoint_name)

Once you have the test.py script created, source the Python virtual environment named venv (created automatically by Pulumi) and run the script using the name of your new endpoint, taken directly from the Pulumi stack output:

# Source the venv
# On Linux & MacOS
source venv/bin/activate

# Execute test.py
python3 test.py $(pulumi stack output EndpointName)

5. Cleanup all resources

Finally, when you’re finished with testing your Hugging Face model on SageMaker, you can easily clean up un-used resources with one easy command.

pulumi destroy

Conclusion

To recap, in a few commands, we created a new Pulumi Python project from a ready-to-roll template, deployed an LLM endpoint on Amazon SageMaker, and tested it with a short Python script to generate a response from our model!

AI and ML is rapidly becoming a necessity with every new day. It may appear daunting or out of reach at first glance, but with the power of IaC written as Pulumi Python programs, getting started has never been easier.

If you followed along then tell us how it worked out for you! We would love to know what you are looking forward to, or if you have ideas for future installments of the Pulumi Python #PulumiMLOps series!

Join us on Twitter, and on the Pulumi Community Slack to decide what #PulumiMLOps we take on next!

Pulumi Blog: Aws

Fully Automated AI Inference on AWS, Azure, and Google Cloud with Pulumi

What you are building

Prerequisites

Credentials without the copy-paste

One cloud-init, three clouds

The programs

Deploying

Testing the inference server

Cost

Security considerations

What’s next?

Conclusion

Build an EKS Environment Factory with Pulumi and vCluster

The problem: environment sprawl and provisioning lag

Architecture overview: the host and the tenants

Implementation: the EKS Auto Mode host

Implementation: the environment factory

Tenant guardrails

Deploying vCluster with Helm

Accessing the virtual cluster

Operational caveats

Conclusion: ephemeral environments at scale

Ten More Things You Can Do With Pulumi Neo

1. Deploy your app to AWS without writing IaC

2. Diagnose a slow API from metrics, logs, and code

3. Automate PagerDuty incident response from Slack

4. Implement a Jira or Linear ticket end-to-end

5. Audit and tighten over-privileged IAM roles

6. Migrate from AWS CDK onto your platform’s golden paths

7. Containerize a service and migrate it to Kubernetes from a runbook

8. Schedule daily configuration drift detection across your cloud infrastructure

9. Schedule weekly upgrades for outdated Lambda runtimes and providers

10. Fix AWS CIS Benchmark failures with daily PRs

Neo: your newest platform engineer

Scan AWS GovCloud and more partitions with Pulumi Insights

Supported partitions

Set it up

How We Load Data into Snowflake in Seconds with Pulumi

What we’ll cover

Architecture overview

Project setup

Managing credentials with Pulumi ESC

Shared infrastructure

Building the direct ingestion ComponentResource

The Lambda handler

Why direct Firehose to Snowflake?

The DirectSnowflakeIngestion component

Wiring it together

Testing the pipeline

Other loading patterns

Publishing as reusable components

Git-based sharing

Pulumi Cloud Private Registry

Conclusion

Passwordless PostgreSQL: IAM Authentication with Pulumi

Why IAM authentication for PostgreSQL?

Architecture overview

Setting up the RDS cluster with IAM authentication

Creating the database user with IAM permissions

Configuring IAM policies for database access

Connecting from Kubernetes with IRSA

How IAM authentication works at runtime

Testing the setup with the demo application

Production considerations

Conclusion

Deploy OpenClaw on AWS or Hetzner Securely with Pulumi and Tailscale

What is OpenClaw?

Prerequisites

Understanding OpenClaw architecture

Setting up ESC for secrets management

Securing with Tailscale

Deploying to AWS

The Pulumi program

Deploying to Hetzner

The Hetzner Pulumi program

Cost comparison

Running the deployment

Automated onboarding

Access the web UI