Pulumi Blog: Ai

Fully Automated AI Inference on AWS, Azure, and Google Cloud with Pulumi

Engin Diri — Tue, 30 Jun 2026 00:00:00 +0000

Putting Ollama on a cloud GPU is something I keep coming back to. A while ago I wrote up running open-source LLMs on an AWS EC2 box with Ollama and Pulumi, and the shape never really changes: a GPU instance, a model server, and a firewall rule in front. Infrastructure as code earned its place by making that kind of setup predictable and repeatable, and AI infrastructure is no exception. A GPU box serving a model is still a VM, a disk, and a firewall rule, and it should be declared like one.

Thorsten Hans made exactly that case in his Akamai post, Fully Automated AI Infrastructures with Terraform and Akamai Cloud, which stands up a single GPU instance on Linode, installs the drivers, runs Ollama, and pulls a model, with no manual steps after terraform apply.

I liked the shape of it, so this post ports the same idea to Pulumi and runs it across AWS, Azure, and Google Cloud instead of one. The result is one program shape per cloud: a single pulumi up brings up a GPU box that installs its own driver, runs Ollama, and pulls a model with no manual steps, and a single pulumi destroy takes it back down. Along the way it drops the two imperative bits the Terraform version leans on: a static access token sitting in an environment variable, and a null_resource running a shell loop to wait for the model. The first becomes an OIDC login from a Pulumi ESC environment, so no long-lived key lives anywhere. The second turns out not to be a resource at all.

What you are building

Strip away the per-cloud naming and every version of this is the same three things: a GPU virtual machine, a firewall in front of it, and a cloud-init script that turns a bare Ubuntu box into a running inference server. The model serving runs on Ollama, which exposes an HTTP API on port 11434 and keeps the model resident in GPU memory between requests.

flowchart LR
Dev([Your machine / curl]) -->|"HTTP :11434"| FW["Firewall / security group<br/>(allow 11434, optional 22)"]
FW --> VM["GPU VM (Ubuntu 24.04)<br/>NVIDIA driver + Ollama"]
VM -->|"resident in GPU memory"| Model["qwen2.5:14b"]
ESC["Pulumi ESC<br/>pulumi-idp/auth"] -.->|"OIDC login (short-lived creds)"| Pulumi["pulumi up"]
Pulumi -.->|"AWS / Azure / GCP"| VM

Component	Port	Description
GPU VM	-	Ubuntu 24.04 with an NVIDIA T4-class GPU. Installs the driver and Ollama from cloud-init, then pulls the model.
Ollama	11434	Serves the model over an HTTP API and keeps it in GPU memory between calls.
Firewall / security group	-	Allows inbound `11434` (and `22` when you ask for it). Open to the world for a demo; lock the CIDR down for anything real.
`pulumi-idp/auth` (ESC)	-	One environment that brokers an OIDC login into AWS, Azure, and GCP. No static keys in code or CI.

One detail here is worth pausing on, and it explains why the Pulumi version comes out shorter than the Terraform one. The Akamai project ends with a null_resource that runs a curl loop in local-exec to block terraform apply until the model finishes downloading. That is not infrastructure but a runtime check wearing a resource costume. Pulumi has a Command provider that would let you reproduce it line for line, and this post deliberately does not. The program provisions the box, the box pulls the model on its own, and the program prints the endpoint. Whether the model has finished downloading yet is a question you answer with a curl, not a question your IaC tool should be holding a deployment open to ask.

Prerequisites

Before getting started, ensure you have:

Pulumi CLI installed and configured
A Pulumi Cloud account (ESC and OIDC live here)
An account on at least one of AWS, Azure, or Google Cloud, with an OIDC trust set up for Pulumi (covered below)
Enough GPU quota in your target region for one T4-class instance; a fresh account often starts at zero, which is the most common reason a first deploy fails, so check it before you run pulumi up
Node.js 18+ for the TypeScript program
curl to talk to the inference endpoint once it comes up

This post serves the qwen2.5:14b model, the same one the Akamai article uses, because the 4-bit quant fits comfortably on a single 16 GB T4. The model is one config value (model), so swap in anything Ollama supports; match the GPU to the model’s memory footprint, because a model that overflows VRAM still runs but spills onto the CPU and crawls.

Credentials without the copy-paste

The Terraform version authenticates the only way a single-cloud demo can: you mint a personal access token, export it as LINODE_TOKEN, and the provider reads it from the environment. It works, but that token is long-lived, it sits in your shell history and your CI secrets, and you mint one per cloud. Across three clouds that is three static keys to rotate and worry about.

Pulumi ESC (Environments, Secrets, and Configuration) replaces it all with an OIDC login. The idea: instead of storing a cloud key, you store a trust relationship. At deploy time, ESC presents a short-lived OIDC token to AWS, Azure, or Google Cloud, and each one hands back temporary credentials scoped to a role you control. Nothing long-lived is ever written down.

I keep that wiring in one environment, pulumi-idp/auth, and every stack imports it. Here is the whole thing:

# pulumi-idp/auth: one ESC environment that brokers an OIDC login into all three
# clouds. Every stack imports it. No static cloud key lives here or anywhere else.
values:
 aws:
 login:
 fn::open::aws-login:
 oidc:
 roleArn: arn:aws:iam::123456789012:role/pulumi-esc
 sessionName: pulumi-esc
 azure:
 login:
 fn::open::azure-login:
 clientId: aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
 tenantId: aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
 subscriptionId: /subscriptions/00000000-0000-0000-0000-000000000000
 oidc: true
 gcp:
 login:
 fn::open::gcp-login:
 project: 123456789012 # numeric project number, not the project ID
 oidc:
 workloadPoolId: pulumi-esc
 providerId: pulumi-esc
 serviceAccount: pulumi-esc@my-project.iam.gserviceaccount.com
 environmentVariables:
 # AWS: read by the Pulumi AWS provider, the AWS SDKs, and the aws CLI
 AWS_ACCESS_KEY_ID: ${aws.login.accessKeyId}
 AWS_SECRET_ACCESS_KEY: ${aws.login.secretAccessKey}
 AWS_SESSION_TOKEN: ${aws.login.sessionToken}
 # Azure: read by the azure-native provider
 ARM_USE_OIDC: "true"
 ARM_CLIENT_ID: ${azure.login.clientId}
 ARM_TENANT_ID: ${azure.login.tenantId}
 ARM_SUBSCRIPTION_ID: ${azure.login.subscriptionId}
 ARM_OIDC_TOKEN: ${azure.login.oidc.token}
 # Google Cloud: read by the Pulumi Google Cloud provider
 GOOGLE_CLOUD_PROJECT: ${gcp.login.project}
 GOOGLE_OAUTH_ACCESS_TOKEN: ${gcp.login.accessToken}

You only need the blocks for the clouds you actually deploy to. Opening this environment makes ESC perform a live OIDC login for each cloud listed, so trim it to the one you use, or keep all three if, like me, you bounce between them.

Each cloud needs a one-time trust setup so this login is allowed at all: an IAM OIDC identity provider and role on AWS, a federated credential on an Azure app registration, and a Workload Identity Pool on Google Cloud. You do that once; after that, pulumi-idp/auth is the only thing any stack references for credentials.

A stack opts into it with one block in its stack config. The AWS stack’s Pulumi.aws.yaml:

# Pulumi.aws.yaml
environment:
 - pulumi-idp/auth

That is the entire credential story. No keys in the program, no keys in CI, nothing to rotate. The program below never mentions a secret; it only creates resources, and the ambient credentials from pulumi-idp/auth carry the request.

One cloud-init, three clouds

The part that turns a bare Ubuntu box into an inference server is identical on every cloud, so it lives in one file, cloud-init.yaml, that all three programs read. It follows the Akamai cloud-config closely, with a couple of robustness tweaks for a multi-cloud run, and it runs in a deliberate order because the GPU driver needs a reboot before Ollama can see the card:

Update packages and install the kernel headers and ubuntu-drivers.
Install the NVIDIA driver, then reboot so the kernel loads it.
On the next boot, a one-shot systemd service installs Ollama, binds it to 0.0.0.0:11434, and pulls the model.
The service disables itself, so it never runs again.

#cloud-config
# Zero-touch Ollama GPU box. Cloud-agnostic: no provider metadata, no static creds.
# The Pulumi program substitutes the model name on the `ollama pull` line below
# before this is passed as user-data (AWS/GCP) or custom-data (Azure).

write_files:
 # Make Ollama listen on every interface and never unload the model from VRAM.
 - path: /etc/systemd/system/ollama.service.d/override.conf
 content: |
 [Service]
 Environment="OLLAMA_HOST=0.0.0.0:11434"
 Environment="OLLAMA_KEEP_ALIVE=-1"

 # Runs once on the post-reboot boot, after the GPU driver is loaded.
 - path: /usr/local/bin/ollama-setup.sh
 permissions: '0755'
 content: |
 #!/usr/bin/env bash
 set -euxo pipefail

 # Install Ollama; the installer creates and starts the ollama systemd unit.
 curl -fsSL https://ollama.com/install.sh | sh

 # Pick up the OLLAMA_HOST / OLLAMA_KEEP_ALIVE override written above.
 systemctl daemon-reload
 systemctl enable ollama.service
 systemctl restart ollama.service

 # Wait for the daemon to bind its socket before pulling. This is the box
 # waiting on its OWN local daemon, not an external readiness gate.
 until curl -fsS http://127.0.0.1:11434/api/tags >/dev/null 2>&1; do
 sleep 2
 done

 # Pre-pull the model so the endpoint answers on the very first request.
 ollama pull __MODEL__

 # One-shot: never run again on future boots.
 systemctl disable ollama-setup.service

 - path: /etc/systemd/system/ollama-setup.service
 content: |
 [Unit]
 Description=One-time Ollama install and model pull
 After=network-online.target
 Wants=network-online.target

 [Service]
 Type=oneshot
 ExecStart=/usr/local/bin/ollama-setup.sh
 RemainAfterExit=true
 # The model pull runs for several minutes; without this, systemd's default
 # 90s start timeout kills the service mid-download and the model never lands.
 TimeoutStartSec=0

 [Install]
 WantedBy=multi-user.target

runcmd:
 - apt-get update
 - DEBIAN_FRONTEND=noninteractive apt-get -y upgrade
 # linux-headers-generic also covers the kernel the upgrade above may have pulled
 # in, so DKMS builds the NVIDIA module for the kernel that boots next.
 - DEBIAN_FRONTEND=noninteractive apt-get install -y "linux-headers-$(uname -r)" linux-headers-generic ubuntu-drivers-common
 # --gpgpu selects the headless server driver branch, the right one for a compute
 # GPU like the T4; bare `ubuntu-drivers install` would pull the desktop stack.
 - ubuntu-drivers install --gpgpu
 - systemctl enable ollama-setup.service

power_state:
 mode: reboot
 message: Rebooting to load the NVIDIA driver before Ollama setup
 condition: true

The model name is the literal token __MODEL__. The Pulumi program reads this file at deploy time and substitutes your model config value before handing it to the instance. Two environment settings are doing quiet but important work: OLLAMA_HOST=0.0.0.0:11434 makes Ollama listen on every interface instead of localhost alone, and OLLAMA_KEEP_ALIVE=-1 keeps the model pinned in GPU memory so only the first request pays the load cost.

The programs

Every program has the same five beats: read the config, template the cloud-init.yaml, open the inbound ports, launch the GPU instance with that cloud-init as its user data, and export the endpoint. What differs is only the dialect each cloud speaks for “GPU instance” and “firewall rule.”

The shared contract keeps the three listings legible side by side: the same model and allowSsh config keys, the same cloud-init.yaml, and the same four exports (publicIp, ollamaEndpoint, generateEndpoint, tagsEndpoint). Pick your cloud:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as fs from "fs";

const cfg = new pulumi.Config();
const model = cfg.get("model") ?? "qwen2.5:14b";
const allowSsh = cfg.getBoolean("allowSsh") ?? false;
const region = (cfg.get("region") ?? "us-east-1") as aws.Region;

// Inject the model name into cloud-init at deploy time (a file read, not a shell-out).
const userData = fs.readFileSync("cloud-init.yaml", "utf8").replace(/__MODEL__/g, model);

// Region comes from config; credentials arrive ambiently from pulumi-idp/auth.
const provider = new aws.Provider("aws", { region });

// Most recent Ubuntu 24.04 LTS (amd64, hvm, gp3) published by Canonical.
const ubuntu = aws.ec2.getAmi({
 mostRecent: true,
 owners: ["099720109477"],
 filters: [
 { name: "name", values: ["ubuntu/images/hvm-ssd-gp3/ubuntu-noble-24.04-amd64-server-*"] },
 { name: "virtualization-type", values: ["hvm"] },
 ],
}, { provider });

const ingress: aws.types.input.ec2.SecurityGroupIngress[] = [{
 description: "Ollama HTTP API. NOTE: restrict cidrBlocks to your own range in production.",
 fromPort: 11434,
 toPort: 11434,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
}];

if (allowSsh) {
 ingress.push({
 description: "SSH",
 fromPort: 22,
 toPort: 22,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 });
}

const sg = new aws.ec2.SecurityGroup("ollama", {
 description: "Ollama inferencing access",
 ingress,
 egress: [{
 description: "Allow all outbound",
 fromPort: 0,
 toPort: 0,
 protocol: "-1",
 cidrBlocks: ["0.0.0.0/0"],
 }],
}, {
 provider,
});

const server = new aws.ec2.Instance("ollama", {
 ami: ubuntu.then(a => a.id),
 instanceType: "g4dn.xlarge", // 1x NVIDIA T4
 vpcSecurityGroupIds: [sg.id],
 associatePublicIpAddress: true,
 userData, // plain text; the AWS provider base64-encodes it for you
 rootBlockDevice: {
 volumeSize: 40,
 volumeType: "gp3",
 },
 tags: { Name: "ollama" },
}, {
 provider,
});

export const publicIp = server.publicIp;
export const ollamaEndpoint = pulumi.interpolate`http://${publicIp}:11434`;
export const generateEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/generate`;
export const tagsEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/tags`;

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";
import * as fs from "fs";

const cfg = new pulumi.Config();
const model = cfg.get("model") ?? "qwen2.5:14b";
const allowSsh = cfg.getBoolean("allowSsh") ?? false;
const project = cfg.get("project");
const zone = cfg.get("zone") ?? "us-central1-a";

// Inject the model name into cloud-init at deploy time (a file read, not a shell-out).
const userData = fs.readFileSync("cloud-init.yaml", "utf8").replace(/__MODEL__/g, model);

// Latest Ubuntu 24.04 LTS amd64 image.
const ubuntu = gcp.compute.getImage({
 family: "ubuntu-2404-lts-amd64",
 project: "ubuntu-os-cloud",
});

// Network tag binds the firewall rule to this instance.
const networkTag = "ollama";

const firewall = new gcp.compute.Firewall("ollama-fw", {
 network: "default",
 project: project,
 // Inbound to Ollama from anywhere; restrict this CIDR for production.
 allows: allowSsh
 ? [{ protocol: "tcp", ports: ["11434"] }, { protocol: "tcp", ports: ["22"] }]
 : [{ protocol: "tcp", ports: ["11434"] }],
 sourceRanges: ["0.0.0.0/0"],
 targetTags: [networkTag],
});

const instance = new gcp.compute.Instance("ollama", {
 machineType: "n1-standard-4",
 zone: zone,
 project: project,
 tags: [networkTag],
 bootDisk: {
 initializeParams: {
 image: ubuntu.then(i => i.selfLink),
 size: 40,
 },
 },
 guestAccelerators: [{
 type: "nvidia-tesla-t4",
 count: 1,
 }],
 // GPUs cannot live-migrate, so host maintenance must terminate (and restart) the VM.
 scheduling: {
 onHostMaintenance: "TERMINATE",
 automaticRestart: true,
 },
 networkInterfaces: [{
 network: "default",
 accessConfigs: [{}], // an empty config requests an ephemeral public IP
 }],
 metadata: {
 "user-data": userData, // Ubuntu cloud-init reads this key, not startup-script
 },
}, {
 dependsOn: firewall,
});

export const publicIp = instance.networkInterfaces.apply(nics => nics[0].accessConfigs![0].natIp);
export const ollamaEndpoint = pulumi.interpolate`http://${publicIp}:11434`;
export const generateEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/generate`;
export const tagsEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/tags`;

import * as pulumi from "@pulumi/pulumi";
import * as resources from "@pulumi/azure-native/resources";
import * as network from "@pulumi/azure-native/network";
import * as compute from "@pulumi/azure-native/compute";
import * as random from "@pulumi/random";
import * as fs from "fs";

const cfg = new pulumi.Config();
const model = cfg.get("model") ?? "qwen2.5:14b";
const allowSsh = cfg.getBoolean("allowSsh") ?? false;
const location = cfg.get("location") ?? "eastus";

// Inject the model name into cloud-init at deploy time (a file read, not a shell-out).
const userData = fs.readFileSync("cloud-init.yaml", "utf8").replace(/__MODEL__/g, model);

const resourceGroup = new resources.ResourceGroup("ollama-rg", {
 location,
});

const vnet = new network.VirtualNetwork("ollama-vnet", {
 resourceGroupName: resourceGroup.name,
 addressSpace: { addressPrefixes: ["10.0.0.0/16"] },
});

const subnet = new network.Subnet("ollama-subnet", {
 resourceGroupName: resourceGroup.name,
 virtualNetworkName: vnet.name,
 addressPrefix: "10.0.1.0/24",
});

// Standard SKU public IPs use static allocation.
const publicIpAddress = new network.PublicIPAddress("ollama-pip", {
 resourceGroupName: resourceGroup.name,
 sku: { name: network.PublicIPAddressSkuName.Standard },
 publicIPAllocationMethod: network.IPAllocationMethod.Static,
});

// Azure permits all outbound by default, so only inbound rules are needed.
const nsg = new network.NetworkSecurityGroup("ollama-nsg", {
 resourceGroupName: resourceGroup.name,
 securityRules: [
 {
 name: "allow-ollama",
 priority: 1000,
 direction: network.SecurityRuleDirection.Inbound,
 access: network.SecurityRuleAccess.Allow,
 protocol: network.SecurityRuleProtocol.Tcp,
 sourcePortRange: "*",
 destinationPortRange: "11434",
 sourceAddressPrefix: "0.0.0.0/0", // prod: restrict to your client CIDR
 destinationAddressPrefix: "*",
 },
 ...(allowSsh ? [{
 name: "allow-ssh",
 priority: 1001,
 direction: network.SecurityRuleDirection.Inbound,
 access: network.SecurityRuleAccess.Allow,
 protocol: network.SecurityRuleProtocol.Tcp,
 sourcePortRange: "*",
 destinationPortRange: "22",
 sourceAddressPrefix: "0.0.0.0/0",
 destinationAddressPrefix: "*",
 }] : []),
 ],
});

const nic = new network.NetworkInterface("ollama-nic", {
 resourceGroupName: resourceGroup.name,
 networkSecurityGroup: { id: nsg.id },
 ipConfigurations: [{
 name: "ipconfig1",
 subnet: { id: subnet.id },
 publicIPAddress: { id: publicIpAddress.id },
 privateIPAllocationMethod: network.IPAllocationMethod.Dynamic,
 primary: true,
 }],
});

// Azure requires an admin credential even though we never log in. Generate one
// instead of hard-coding it; it stays a Pulumi secret and is never exported.
const adminPassword = new random.RandomPassword("ollama-admin-password", {
 length: 24,
 special: true,
 overrideSpecial: "!#$%*",
 minLower: 1,
 minUpper: 1,
 minNumeric: 1,
 minSpecial: 1, // Azure requires 3 of 4 character classes
});

const vm = new compute.VirtualMachine("ollama-vm", {
 resourceGroupName: resourceGroup.name,
 hardwareProfile: { vmSize: "Standard_NC4as_T4_v3" }, // 1x NVIDIA T4
 networkProfile: {
 networkInterfaces: [{ id: nic.id, primary: true }],
 },
 osProfile: {
 computerName: "ollama",
 adminUsername: "azureuser",
 adminPassword: adminPassword.result,
 customData: Buffer.from(userData).toString("base64"), // Azure wants base64
 linuxConfiguration: {
 disablePasswordAuthentication: false,
 },
 },
 storageProfile: {
 imageReference: {
 publisher: "Canonical",
 offer: "ubuntu-24_04-lts",
 sku: "server", // Gen2; NC4as_T4_v3 is a Gen2 size
 version: "latest",
 },
 osDisk: {
 name: "ollama-osdisk",
 createOption: "FromImage",
 diskSizeGB: 40,
 managedDisk: { storageAccountType: "StandardSSD_LRS" },
 },
 },
});

// A Standard/Static public IP is reserved at creation, so its address is known
// once the resource exists; no separate lookup needed.
export const publicIp = publicIpAddress.ipAddress.apply(ip => ip!);
export const ollamaEndpoint = pulumi.interpolate`http://${publicIp}:11434`;
export const generateEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/generate`;
export const tagsEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/tags`;

A few per-cloud details are worth calling out, since they are the places the “same program” abstraction leaks:

AWS is the shortest program, because the GPU comes with the instance shape: a g4dn.xlarge is a T4 box, so there is no separate accelerator to attach. The one thing to do before you deploy is raise the Running On-Demand G and VT instances vCPU quota in your region; a fresh account starts at zero, and pulumi up fails with VcpuLimitExceeded until you do.
Google Cloud attaches the GPU explicitly with guestAccelerators, and that brings the rule that trips people up: a GPU instance cannot live-migrate, so scheduling.onHostMaintenance must be "TERMINATE" or the apply is rejected. The cloud-init also has to ride on the user-data metadata key, not startup-script, and the empty accessConfigs: [{}] is what hands the box a public IP. T4 quota is also zero on a new project.
Azure is the longest listing, because the network is à la carte: the resource group, virtual network, subnet, public IP, security group, and NIC are each their own resource before you reach the VM. One detail surprises people: a Linux VM requires an admin credential even when you never log in, so the program generates a throwaway password with random.RandomPassword rather than committing one. The NC4as_T4_v3 is a compute GPU, so the standard server (CUDA) driver from the cloud-init is the right choice here; the GRID driver is only needed for GPU-accelerated visualization workloads.

The full programs, all three Pulumi.<cloud>.yaml files, and the shared cloud-init.yaml are in the companion repo:

github.com/dirien/fully-automated-ai-inference-pulumi

Deploying

Create a project, install the provider for your cloud, point the stack at pulumi-idp/auth, and deploy. For AWS:

mkdir ai-inference && cd ai-inference
pulumi new typescript
npm install @pulumi/aws

Drop the AWS listing into index.ts, copy the cloud-init.yaml shown earlier into the same directory (or grab it from the companion repo), point the stack at the auth environment, and set your region:

pulumi stack init aws
# add `environment: [pulumi-idp/auth]` to Pulumi.aws.yaml (shown above)
pulumi config set region us-east-1
pulumi config set model qwen2.5:14b
pulumi up

The other two clouds are the same flow with a different provider package and a couple of config keys: npm install @pulumi/gcp with pulumi config set project <your-project-id> and pulumi config set zone us-central1-a, or npm install @pulumi/azure-native @pulumi/random with pulumi config set location eastus.

The slow part is the GPU box: it boots, installs the driver, reboots, installs Ollama, and pulls the model, all without you. Depending on the instance, the region, and the model size, plan on roughly 10 to 15 minutes before the endpoint answers. When pulumi up finishes you get the endpoints straight back as stack outputs:

Outputs:
 generateEndpoint: "http://<public-ip>:11434/api/generate"
 ollamaEndpoint : "http://<public-ip>:11434"
 publicIp : "<public-ip>"
 tagsEndpoint : "http://<public-ip>:11434/api/tags"

Testing the inference server

pulumi up returns as soon as the infrastructure exists, which is before the model has finished downloading. This is exactly where Terraform reached for that null_resource loop, and where Pulumi hands you a URL instead. To check whether the model is ready, ask Ollama what it has loaded:

curl -s $(pulumi stack output tagsEndpoint) | grep -q "qwen2.5:14b" && echo "ready" || echo "still pulling"

Once it reports ready, send it a prompt. This is the same request the Akamai post makes, pointed at your generateEndpoint output:

curl -s $(pulumi stack output generateEndpoint) -d '{
 "model": "qwen2.5:14b",
 "system": "Answer every question with a three-line poem.",
 "prompt": "Why is the sky blue?",
 "stream": false
}'

The first call is slower, because Ollama loads the model into GPU memory before it answers. Every call after that is fast, since OLLAMA_KEEP_ALIVE=-1 keeps it resident:

{
 "model": "qwen2.5:14b",
 "response": "Sunlight scatters, short waves fly,\nBlue light paints the open sky,\nViolet fades as day drifts by.",
 "done": true
}

When you are done, take it all down with one command, so the GPU stops billing:

pulumi destroy

Cost

A GPU instance is the whole bill, and you pay by the hour whether the model is busy or idle. The numbers below are rough on-demand rates for a single T4-class box left running 24/7; in practice you spin it up, use it, and pulumi destroy it, so what you actually pay tracks the hours it stays up.

Cloud	Instance	GPU	~Hourly	~Monthly (24/7)
AWS	`g4dn.xlarge`	1× T4 (16 GB)	~$0.53	~$384
Azure	`Standard_NC4as_T4_v3`	1× T4 (16 GB)	~$0.53	~$384
Google Cloud	`n1-standard-4` + 1× T4	1× T4 (16 GB)	~$0.54	~$394

These are on-demand Linux rates in a cheap US region (us-east-1, eastus, us-central1) as of mid-2026, and the GPU dominates the bill. Spot or preemptible capacity cuts it sharply if your workload tolerates interruption (often 60 to 70 percent off), Google Cloud’s sustained-use discount trims an always-on instance on its own, and the 40 GB disk adds only a few dollars a month.

A GPU box left running overnight is the expensive mistake here. Because the whole stack is declarative, the safe habit is cheap: pulumi destroy when you stop using it, pulumi up when you need it back. The state and config are version-controlled, so standing it up again is one command, not a rebuild.

Security considerations

The architecture in this post matches the Akamai original on purpose, which means it carries the same caveat: Ollama is exposed over plain HTTP on a port open to the entire internet. That is fine for a demo on a box you tear down the same day. It is not fine for anything that outlives the afternoon, and self-hosted AI endpoints get found fast: scanners like Shodan enumerate a freshly exposed one within hours, and an open Ollama instance is free compute for whoever finds it first.

The credential side, though, is genuinely better than a static-token setup, and that is the part worth keeping. There is no long-lived cloud key in the program, in your shell, or in CI; every deploy gets short-lived credentials minted through pulumi-idp/auth and thrown away when it finishes.

Concern	Akamai/Terraform demo	This deployment
Cloud credentials	Static PAT in an env var	OIDC login, short-lived, via `pulumi-idp/auth`
Inbound exposure	Port 11434 open to `0.0.0.0/0`	Same by default; one config flag scopes the CIDR
Transport	Plain HTTP	Plain HTTP (front with TLS for real use)
Readiness wait	`null_resource` + `local-exec` shell loop	A runtime `curl`, no resource

My recommendations for taking this past a demo:

Scope the inbound rule to your own CIDR instead of 0.0.0.0/0; the allowSsh flag in the program is the pattern to copy for the Ollama port
Put a reverse proxy with TLS in front of Ollama, or keep the box off the public internet entirely and reach it over a private network or a Tailscale tailnet, the way I did for the Hermes agent
Keep credentials on OIDC through ESC; never fall back to a static key to “just get it working”
Treat the box as disposable, and pulumi destroy it when you are not using it, which shrinks both the bill and the attack window

What’s next?

The program is a foundation, and the interesting work is what you layer on once a model is a pulumi up away:

Lock it down. Scope the firewall, add TLS, or move the box behind a private network so the endpoint is not on the public internet at all.
Scale the model to the GPU. qwen2.5:14b on a T4 is the entry point. Bigger models want more memory, which means an A10G, an L4, or an A100, and the only change in the program is the instance type and the model value.
Swap the serving engine. Ollama is the simplest thing that works. For higher throughput, the same shape holds with vLLM in place of Ollama in the cloud-init.
Generate the next one. Pulumi Neo can take a target like “a GPU inference box on Azure behind a private endpoint” and produce a first draft of the program, which you then review in a PR like any other change.

Conclusion

Declaring an AI inference server as code buys you the same thing it buys for any other infrastructure: you can reproduce it on any of the three clouds, version-control it, and tear it down with a single pulumi destroy. Porting the Akamai project made the contrast sharp. The credentials moved from a static token to an OIDC login that leaves nothing behind, and the readiness wait disappeared entirely once you stop treating a runtime check as a resource.

That is the line worth carrying to the next thing you deploy. Not every step in a runbook is a resource. A GPU box and a firewall rule are; waiting for a download to finish is not. Draw that line first, and the program gets shorter on its own.

If you run into issues or have questions, drop by the Pulumi Community Slack or GitHub Discussions. New to Pulumi? Get started here.

Deploy a Private Hermes Agent on Render Securely with Pulumi, Modal, and Tailscale

Engin Diri — Wed, 24 Jun 2026 00:00:00 +0000

Personal AI agents had their breakout this year. OpenClaw crossed 100,000 GitHub stars within months of launching, and self-hosting your own assistant went from a hobbyist trick to something a lot of developers actually do. I wrote up how to deploy that lobster to AWS or Hetzner back when it was everywhere.

The one people are switching to now is Hermes, the open-source runtime from Nous Research, and it caught on just as quickly. The reason shows up in every “I ditched OpenClaw for Hermes” thread: it actually learns, building up memory and writing its own skills as it goes instead of running off a static, human-written list.

Here is the part the launch videos skip. Hermes writes and runs its own code, with no human approving the commands. A model that can write code will eventually write a bad one, and the only thing between that command and your credentials is the sandbox it runs in. That is the box you do not want on the public internet. Researchers found 175,000 exposed Ollama servers sitting open in early 2026, and attackers hijack the ones they find for compute. The fix is not a better lock on the front door. It is to have no front door at all.

So this post deploys a private Hermes agent as one Pulumi program across Render, Modal, and Tailscale. The agent and its chat UI run as Render private services with no public URL, Tailscale puts the UI on your tailnet, and Modal runs the agent’s code in throwaway sandboxes. One pulumi up to stand it up, one pulumi destroy to tear it down, around $50 a month. No Makefile, no CLI escape hatches, and an honest look at the few places “nothing on the public internet” still leaks.

What is Hermes?

Hermes is the open-source, MIT-licensed agent runtime from Nous Research. It runs continuously on a server rather than living in a browser tab, and you talk to it through a chat UI or a messaging platform like Telegram, Discord, Slack, or Signal. A few things make it more than a chatbot:

It is model-agnostic. You point it at any provider. This deployment uses OpenRouter, one key that fronts 400-plus models across providers, with automatic failover when one is down.
It remembers across restarts. Memory persists on disk, so it builds up context over time instead of resetting every session.
It writes its own skills. When it works through a task, it can save that approach as a reusable skill and reach for it next time.
It schedules itself. A built-in cron runs automations on a schedule, driven in plain language.
It runs code in a sandbox. When the agent needs to execute code, it hands that off to an isolated, ephemeral container instead of running it next to its own process.

The difference from a cloud-hosted assistant is the usual self-hosting trade: it runs on your infrastructure with your keys, and your conversation history and memory stay on your disk.

“Hermes” here is the agent runtime from Nous Research, not the Hermes family of language models that share the name. The agent is model-agnostic, and the model it talks to is a separate choice you make through OpenRouter.

Prerequisites

Before getting started, ensure you have:

Pulumi CLI installed and configured
A Pulumi Cloud account
A Render account, an API key, and your workspace owner ID
A Modal account and a token pair (token ID and secret)
An OpenRouter API key with credit on the account
A container registry you can push to; this post uses GitHub Container Registry (a GHCR username and token)
Docker running locally (the image build rides on a local buildx engine), or a Docker-enabled CI runner
A Tailscale account with HTTPS enabled and a reusable auth key
Node.js 18+ (for TypeScript) or Python 3.9+ (for Python), depending on the language you pick

This guide routes models through OpenRouter, but Hermes works with other providers too. The model is just a routing string you pass (gpt-5.5 here), so any model OpenRouter supports works by changing that one value.

Understanding the Hermes architecture

Strip away the dashboard clicks and the architecture is four pieces. One rule governs all of them: what the public internet is allowed to touch.

flowchart LR
User([Your devices]) -->|"tailnet (WireGuard)"| TS["Tailscale<br/>private mesh"]
TS -->|"open-webui.&lt;tailnet&gt;.ts.net"| WebUI["Open WebUI<br/>(Render private service,<br/>joined to your tailnet)"]
WebUI -->|"private network<br/>&lt;hermes-slug&gt;:8642"| Hermes["Hermes gateway<br/>(Render private service)"]
Hermes -->|"Modal SDK at runtime"| Modal["Modal sandbox<br/>isolated code execution"]
Hermes -->|model calls| OR["OpenRouter"]

Component	Port	Description
Hermes gateway	8642	Holds secrets and memory, talks to the model, creates Modal sandboxes. The most dangerous box, so it has no public URL.
Open WebUI	8080	The chat front end. Joins your tailnet and reaches Hermes over Render’s private network.
Modal sandbox	-	Isolated, ephemeral container for running the agent’s code.
Tailscale	-	A private WireGuard mesh that makes the UI reachable to your devices only.

The agent that can run code and read secrets is never routable from the internet, and neither is the UI in front of it. There is no inbound ingress on any box (the outbound calls to the model, to Modal, and to the image registry still cross the internet, as they have to). Expressing “which box can reach which” in code is what makes the boundary auditable: the rule lives in the program rather than in a dashboard.

The four pieces split cleanly in two: two you declare as resources, and two you hand a credential and let run.

Layer	How you provision it	Status
Render services, disks, secrets	official `render-oss` Terraform provider, bridged	Declarative, via one `pulumi package add`.
Container images	`@pulumi/docker-build`	Built and pushed during `pulumi up`. No `docker` CLI.
Modal sandboxes	nothing	No Terraform provider exists. A runtime credential.
Tailnet access (Tailscale)	nothing	No provider needed. A runtime credential, like Modal.

Two of these four layers are credentials, not resources. That distinction often gets missed, and it is worth getting right: a provider can only manage what it can provision, and spinning up a Modal sandbox is a runtime act, not a provisioned resource. We come back to it at each layer.

Setting up ESC for secrets management

Deploying Hermes means handling a pile of credentials: a Render API key, registry credentials, the OpenRouter key, the Modal token pair, and the Tailscale auth key. You don’t want these hardcoded or scattered across environment variables. Pulumi ESC (Environments, Secrets, and Configuration) stores them securely and passes them directly to your Pulumi program.

Create a new ESC environment:

pulumi env init <your-org>/hermes-secrets

Add your secrets to the environment:

values:
 renderApiKey:
 fn::secret: "rnd_xxxxx"
 renderOwnerId: "tea-xxxxx"
 ghcrUser: "your-github-username"
 ghcrToken:
 fn::secret: "ghp_xxxxx"
 openrouterApiKey:
 fn::secret: "sk-or-xxxxx"
 modalTokenId:
 fn::secret: "ak-xxxxx"
 modalTokenSecret:
 fn::secret: "as-xxxxx"
 tailscaleAuthKey:
 fn::secret: "tskey-auth-xxxxx"
 tailnetDnsName: "tailxxxxx.ts.net"
 pulumiConfig:
 renderApiKey: ${renderApiKey}
 ownerId: ${renderOwnerId}
 ghcrUser: ${ghcrUser}
 ghcrToken: ${ghcrToken}
 openrouterApiKey: ${openrouterApiKey}
 modalTokenId: ${modalTokenId}
 modalTokenSecret: ${modalTokenSecret}
 tailscaleAuthKey: ${tailscaleAuthKey}
 tailnetDnsName: ${tailnetDnsName}

To find your tailnet DNS name, go to the Tailscale admin console, look under the DNS section, and find your tailnet name (e.g., tailxxxxx.ts.net). This is the domain suffix used for all machines in your Tailscale network.

Then create a Pulumi.dev.yaml file in your project to reference the environment:

environment:
 - <your-org>/hermes-secrets

This keeps your secrets out of your codebase and hands them to the program at deploy time. The Modal token pair and the Tailscale auth key never become resources; they ride straight through as encrypted env vars on the services that need them.

Generating the Render SDK

Render has no first-party Pulumi provider. What it has is an official, maintained Terraform provider, and Pulumi can turn any Terraform provider into a typed local SDK with Any Terraform Provider:

pulumi package add terraform-provider render-oss/render 1.8.0

That one command generates a render SDK you import like any other package, in whatever language your stack is written in. There is no published @pulumi/render package and no vendor to wait on.

This is also where the provider map shows its edges. The bridge wraps a Terraform provider that already exists, so it works for Render. But Modal has no Terraform provider anywhere, so pulumi package add terraform-provider modal-labs/modal has nothing to fetch. That is not a gap to work around. Modal is not a provisioned resource here; Hermes creates sandboxes at runtime through the Modal SDK, and Pulumi’s only job is to hold the token pair and hand it over. Tailscale lands in the same bucket: a Tailscale provider exists, but it manages your tailnet’s settings, not the act of putting a container on the network, which is a runtime job done inside the image.

Securing with Tailscale

The usual way to make a self-hosted UI reachable is to give it a public hostname and put an identity gate in front: an SSO provider, an access policy, a public DNS record, and a certificate to keep renewing. Tailscale collapses all of it into nothing you declare. There is no gate to bolt in front of a public URL, because there is no public URL. Open WebUI joins your tailnet and answers only to your own devices.

This is a stronger position than a public UI behind a login. Because there is no public URL, there is no login to harden and no certificate to renew, and Open WebUI answers only to your own devices.

Tailscale lives in the Open WebUI image rather than the Pulumi program. The image adds Tailscale and an entrypoint that brings up tailscaled in userspace mode (a container has no /dev/net/tun), joins your tailnet, and runs tailscale serve to expose Open WebUI on it. serve publishes over HTTPS to authenticated members of your tailnet only, where its sibling tailscale funnel would put the same port on the public internet, the one thing this design avoids.

FROM ghcr.io/open-webui/open-webui:main-slim
RUN apt-get update && apt-get install -y --no-install-recommends curl ca-certificates \
 && curl -fsSL https://tailscale.com/install.sh | sh
COPY entrypoint.sh /usr/local/bin/ts-entrypoint
ENTRYPOINT ["/usr/local/bin/ts-entrypoint"]

#!/usr/bin/env bash
set -e
PORT="${PORT:-8080}"
mkdir -p /app/backend/data/tailscale
# Userspace networking: a container has no /dev/net/tun. Keep node state on the disk
# so it survives restarts, and accept-dns=false so Render's DNS still resolves the
# private Hermes hostname.
tailscaled --tun=userspace-networking --statedir=/app/backend/data/tailscale &
n=0
until tailscale up --authkey="${TS_AUTHKEY}" --hostname="${TS_HOSTNAME}" --accept-dns=false; do
 n=$((n + 1)); [ "${n}" -ge 30 ] && { echo "tailnet join failed; check TS_AUTHKEY"; exit 1; }
 sleep 2
done
tailscale serve --bg "${PORT}" # serve Open WebUI on the tailnet over HTTPS
cd /app/backend && exec bash start.sh

The TS_AUTHKEY env var on the service, pulled from ESC, is the only thing Pulumi contributes; everything else is the image doing its own networking.

There is one prerequisite, the same one any Tailscale node needs for serve: enable HTTPS for your tailnet once in the admin console so it can get a certificate. Use a reusable auth key, since the service re-authenticates on every redeploy, and the join loop has a ceiling so a dead key fails the deploy in a minute instead of hanging it.

To generate the reusable auth key:

Go to the Tailscale admin console.
Click Generate auth key.
Enable Reusable, since the service re-authenticates on every redeploy.
Copy the key into your ESC environment as tailscaleAuthKey.

Deploying the agent

This is the part a Makefile usually handles. Two private services, two disks, every secret an encrypted env var, and both images built and pushed during the same pulumi up that creates the services. No CLI runs anywhere, though the build still rides on a local Docker engine through buildx: the docker call is gone, the daemon it talks to is not, so pulumi up needs Docker running.

Create a new Pulumi project (pick the language you prefer; the companion repo has all three):

mkdir hermes-agent-pulumi && cd hermes-agent-pulumi
pulumi new typescript

Install the dependencies and generate the Render SDK:

npm install @pulumi/docker-build @pulumi/random
pulumi package add terraform-provider render-oss/render 1.8.0

Both services need the 2 GB Standard plan. The gateway and Open WebUI each exhaust memory on the 512 MB Starter and OOM on boot.

Two facts make this clean. Render env var values are encrypted at rest, so a secret is only an env var, with no separate resource. And @pulumi/docker-build builds and pushes an image through an embedded buildx during pulumi up, so the image pipeline is declarative too. Render then pulls the pushed image by digest.

The complete program

Hermes comes first: a private Render service with no public URL, a disk for its memory, and every secret as an encrypted env var. The shared bearer key is generated in state with random.RandomBytes, the clean replacement for openssl rand -base64 32; the Modal token pair and the OpenRouter key come from config or Pulumi ESC. Render has no port field, so Hermes binds 0.0.0.0:8642 and Open WebUI reaches it at the service’s read-only slug. Open WebUI is almost the mirror image, private too, except its image joins your tailnet (the tailscaled entrypoint from the previous section) and it carries the operational env vars that keep it lean: no local embedding models, and ENABLE_PERSISTENT_CONFIG=false so your config always wins over a stale copy in its database. One render.Provider carries your renderApiKey and ownerId into every Render resource.

Here is the whole program, copy-paste ready as your index.ts (or __main__.py, or Pulumi.yaml):

import * as pulumi from "@pulumi/pulumi";
import * as dockerbuild from "@pulumi/docker-build";
import * as random from "@pulumi/random";
// Local SDK generated by: pulumi package add terraform-provider render-oss/render 1.8.0
// (no @pulumi/render is published; the SDK is bridged from the official Render provider).
import * as render from "@pulumi/render";

const cfg = new pulumi.Config();

// ---------------------------------------------------------------------------
// Configuration. Non-secret values via `pulumi config set`, secrets via
// `pulumi config set --secret` or a Pulumi ESC environment (see README).
// The Render provider reads renderApiKey/renderOwnerId (or RENDER_API_KEY/RENDER_OWNER_ID).
// ---------------------------------------------------------------------------
const region = cfg.get("renderRegion") ?? "oregon";
const modelDefault = cfg.get("modelDefault") ?? "gpt-5.5";

// Container registry the images are pushed to and Render pulls from (GHCR here).
const ghcrUser = cfg.require("ghcrUser");
const ghcrToken = cfg.requireSecret("ghcrToken"); // GHCR PAT with write:packages

// Long-lived secrets supplied by you (dashboard-minted). Hold in config or ESC.
const modalTokenId = cfg.requireSecret("modalTokenId");
const modalTokenSecret = cfg.requireSecret("modalTokenSecret");
const openrouterApiKey = cfg.requireSecret("openrouterApiKey");

// Tailscale: the only "gateway". Open WebUI joins your tailnet and is reachable
// at https://<hostname>.<tailnet>.ts.net by your devices only — never public.
// Both values are already in your ESC. Enable HTTPS once in the Tailscale admin console.
const tailscaleAuthKey = cfg.requireSecret("tailscaleAuthKey");
const tailnetDnsName = cfg.require("tailnetDnsName"); // e.g. tailc6fb4e.ts.net
const webuiHostname = "open-webui";
const webuiTailnetUrl = `https://${webuiHostname}.${tailnetDnsName}`;

// Render provider auth (from ESC: pulumiConfig.renderApiKey / pulumiConfig.renderOwnerId).
const renderProvider = new render.Provider("render", {
 apiKey: cfg.requireSecret("renderApiKey"),
 ownerId: cfg.require("renderOwnerId"),
});

// ---------------------------------------------------------------------------
// Secrets generated in state (the in-IaC replacement for `openssl rand -base64 32`).
// The shared bearer authenticates Open WebUI -> Hermes; it lands on both services.
// ---------------------------------------------------------------------------
const sharedKey = new random.RandomBytes("hermes-shared-key", { length: 32 });
const webuiSecretKey = new random.RandomBytes("webui-secret-key", { length: 32 });

// ---------------------------------------------------------------------------
// Build and push both images during `pulumi up` (no docker CLI; the provider
// embeds buildx). Render then pulls them by their digest-pinned ref.
// ---------------------------------------------------------------------------
const registryAuth = [{ address: "ghcr.io", username: ghcrUser, password: ghcrToken }];

const hermesImage = new dockerbuild.Image("hermes-image", {
 context: { location: "./hermes" },
 dockerfile: { location: "./hermes/Dockerfile" },
 platforms: ["linux/amd64"],
 tags: [pulumi.interpolate`ghcr.io/${ghcrUser}/hermes-agent:latest`],
 push: true,
 registries: registryAuth,
});

const webuiImage = new dockerbuild.Image("webui-image", {
 context: { location: "./openwebui" },
 dockerfile: { location: "./openwebui/Dockerfile" },
 platforms: ["linux/amd64"],
 tags: [pulumi.interpolate`ghcr.io/${ghcrUser}/open-webui:latest`],
 push: true,
 registries: registryAuth,
});

// Credential Render uses to pull the private GHCR images.
const ghcrCredential = new render.RegistryCredential("ghcr", {
 name: "ghcr",
 registry: "GITHUB",
 username: ghcrUser,
 authToken: ghcrToken,
}, { provider: renderProvider });

// ---------------------------------------------------------------------------
// Hermes: a PRIVATE Render service. No public URL; reachable only over Render's
// private network. A disk holds its memory and workspace. Env values are
// encrypted at rest, which is how secrets ride on Render (no separate flag).
// ---------------------------------------------------------------------------
const hermes = new render.PrivateService("hermes-agent", {
 name: "hermes-agent",
 plan: "standard", // 2 GB; the gateway OOMs on starter's 512 MB
 region: region,
 runtimeSource: {
 // Render wants the bare repo in imageUrl; the tag/digest go in their own field.
 image: {
 imageUrl: pulumi.interpolate`ghcr.io/${ghcrUser}/hermes-agent`,
 digest: hermesImage.digest,
 registryCredentialId: ghcrCredential.id,
 },
 },
 disk: { name: "hermes-data", mountPath: "/opt/data", sizeGb: 5 },
 envVars: {
 PORT: { value: "8642" },
 API_SERVER_ENABLED: { value: "true" },
 API_SERVER_HOST: { value: "0.0.0.0" },
 API_SERVER_PORT: { value: "8642" },
 API_SERVER_MODEL_NAME: { value: modelDefault },
 // Auto-approve the agent's shell/terminal hooks (no TTY in a container),
 // so it can run code in the Modal sandbox without an interactive prompt.
 HERMES_ACCEPT_HOOKS: { value: "1" },
 // Secrets (encrypted at rest on Render):
 API_SERVER_KEY: { value: sharedKey.base64 },
 OPENROUTER_API_KEY: { value: openrouterApiKey },
 // Modal is a runtime credential, not a provisioned resource: Hermes creates
 // sandboxes at runtime with this token pair. No Modal provider, no Modal CLI.
 MODAL_TOKEN_ID: { value: modalTokenId },
 MODAL_TOKEN_SECRET: { value: modalTokenSecret },
 },
}, { provider: renderProvider });

// Open WebUI reaches Hermes at its Render-assigned internal hostname (the read-only
// slug), not the bare service name. Same region + same workspace share the network.
const hermesApiBaseUrl = pulumi.interpolate`http://${hermes.slug}:8642/v1`;

// ---------------------------------------------------------------------------
// Open WebUI: also a PRIVATE Render service, but the container joins your tailnet
// (tailscaled in userspace + `tailscale serve`), so you reach it at
// https://open-webui.<tailnet>.ts.net from your devices. Never public; the gate
// is tailnet membership. It still reaches Hermes over Render's private network.
// ---------------------------------------------------------------------------
const webui = new render.PrivateService("open-webui", {
 name: "open-webui",
 plan: "standard", // 2 GB; Open WebUI OOMs on starter's 512 MB
 region: region, // must match Hermes for private networking
 runtimeSource: {
 image: {
 imageUrl: pulumi.interpolate`ghcr.io/${ghcrUser}/open-webui`,
 digest: webuiImage.digest,
 registryCredentialId: ghcrCredential.id,
 },
 },
 disk: { name: "openwebui-data", mountPath: "/app/backend/data", sizeGb: 5 },
 envVars: {
 PORT: { value: "8080" }, // tailscale serve points at this; Open WebUI binds it
 OPENAI_API_BASE_URL: { value: hermesApiBaseUrl },
 ENABLE_OPENAI_API: { value: "true" },
 ENABLE_OLLAMA_API: { value: "false" },
 DEFAULT_MODELS: { value: modelDefault },
 WEBUI_AUTH: { value: "true" },
 WEBUI_URL: { value: webuiTailnetUrl },
 // Reached only over the tailnet, so the first sign-up is the admin.
 ENABLE_LOGIN_FORM: { value: "true" },
 ENABLE_SIGNUP: { value: "true" },
 DEFAULT_USER_ROLE: { value: "admin" },
 // Always apply env config; otherwise Open WebUI persists the first run's config
 // to its DB and ignores later changes.
 ENABLE_PERSISTENT_CONFIG: { value: "false" },
 // Don't load local embedding/RAG models (a big memory saver; the agent is the brain).
 RAG_EMBEDDING_ENGINE: { value: "openai" },
 BYPASS_EMBEDDING_AND_RETRIEVAL: { value: "true" },
 OFFLINE_MODE: { value: "true" },
 HF_HUB_OFFLINE: { value: "1" },
 // Secrets (encrypted at rest):
 OPENAI_API_KEY: { value: sharedKey.base64 }, // shared bearer to call Hermes
 WEBUI_SECRET_KEY: { value: webuiSecretKey.base64 },
 // Tailscale joins this container to your tailnet and serves Open WebUI on it.
 TS_AUTHKEY: { value: tailscaleAuthKey },
 TS_HOSTNAME: { value: webuiHostname },
 },
}, { dependsOn: hermes, provider: renderProvider });

export const hermesInternalUrl = hermesApiBaseUrl;
// Open this from any device on your tailnet:
export const openWebUiTailnetUrl = webuiTailnetUrl;

import pulumi
import pulumi_docker_build as docker_build
import pulumi_random as random

# Local SDK generated by: pulumi package add terraform-provider render-oss/render 1.8.0
# (no pulumi_render is published; the SDK is bridged from the official Render provider).
import pulumi_render as render

cfg = pulumi.Config()

# ---------------------------------------------------------------------------
# Configuration. Non-secret values via `pulumi config set`, secrets via
# `pulumi config set --secret` or a Pulumi ESC environment (see README).
# The Render provider reads renderApiKey/renderOwnerId (or RENDER_API_KEY/RENDER_OWNER_ID).
# ---------------------------------------------------------------------------
region = cfg.get("renderRegion") or "oregon"
model_default = cfg.get("modelDefault") or "gpt-5.5"

# Container registry the images are pushed to and Render pulls from (GHCR here).
ghcr_user = cfg.require("ghcrUser")
ghcr_token = cfg.require_secret("ghcrToken") # GHCR PAT with write:packages

# Long-lived secrets supplied by you (dashboard-minted). Hold in config or ESC.
modal_token_id = cfg.require_secret("modalTokenId")
modal_token_secret = cfg.require_secret("modalTokenSecret")
openrouter_api_key = cfg.require_secret("openrouterApiKey")

# Tailscale: the only "gateway". Open WebUI joins your tailnet and is reachable
# at https://<hostname>.<tailnet>.ts.net by your devices only -- never public.
# Enable HTTPS once in the Tailscale admin console so `tailscale serve` can get a cert.
tailscale_auth_key = cfg.require_secret("tailscaleAuthKey")
tailnet_dns_name = cfg.require("tailnetDnsName") # e.g. tailc6fb4e.ts.net
webui_hostname = "open-webui"
webui_tailnet_url = f"https://{webui_hostname}.{tailnet_dns_name}"

# Render provider auth (from ESC: pulumiConfig.renderApiKey / pulumiConfig.renderOwnerId).
render_provider = render.Provider(
 "render",
 api_key=cfg.require_secret("renderApiKey"),
 owner_id=cfg.require("renderOwnerId"),
)

# ---------------------------------------------------------------------------
# Secrets generated in state (the in-IaC replacement for `openssl rand -base64 32`).
# The shared bearer authenticates Open WebUI -> Hermes; it lands on both services.
# ---------------------------------------------------------------------------
shared_key = random.RandomBytes("hermes-shared-key", length=32)
webui_secret_key = random.RandomBytes("webui-secret-key", length=32)

# ---------------------------------------------------------------------------
# Build and push both images during `pulumi up` (no docker CLI; the provider
# embeds buildx). Render then pulls them by their digest-pinned ref.
# ---------------------------------------------------------------------------
registry_auth = [{"address": "ghcr.io", "username": ghcr_user, "password": ghcr_token}]

hermes_image = docker_build.Image(
 "hermes-image",
 context={"location": "./hermes"},
 dockerfile={"location": "./hermes/Dockerfile"},
 platforms=["linux/amd64"],
 tags=[pulumi.Output.concat("ghcr.io/", ghcr_user, "/hermes-agent:latest")],
 push=True,
 registries=registry_auth,
)

webui_image = docker_build.Image(
 "webui-image",
 context={"location": "./openwebui"},
 dockerfile={"location": "./openwebui/Dockerfile"},
 platforms=["linux/amd64"],
 tags=[pulumi.Output.concat("ghcr.io/", ghcr_user, "/open-webui:latest")],
 push=True,
 registries=registry_auth,
)

# Credential Render uses to pull the private GHCR images.
ghcr_credential = render.RegistryCredential(
 "ghcr",
 name="ghcr",
 registry="GITHUB",
 username=ghcr_user,
 auth_token=ghcr_token,
 opts=pulumi.ResourceOptions(provider=render_provider),
)

# ---------------------------------------------------------------------------
# Hermes: a PRIVATE Render service. No public URL; reachable only over Render's
# private network. A disk holds its memory and workspace. Env values are
# encrypted at rest, which is how secrets ride on Render (no separate flag).
# ---------------------------------------------------------------------------
hermes = render.PrivateService(
 "hermes-agent",
 name="hermes-agent",
 plan="standard", # 2 GB; the gateway OOMs on starter's 512 MB
 region=region,
 runtime_source={
 "image": {
 "image_url": pulumi.Output.concat("ghcr.io/", ghcr_user, "/hermes-agent"),
 "digest": hermes_image.digest,
 "registry_credential_id": ghcr_credential.id,
 },
 },
 disk={"name": "hermes-data", "mount_path": "/opt/data", "size_gb": 5},
 env_vars={
 "PORT": {"value": "8642"},
 "API_SERVER_ENABLED": {"value": "true"},
 "API_SERVER_HOST": {"value": "0.0.0.0"},
 "API_SERVER_PORT": {"value": "8642"},
 "API_SERVER_MODEL_NAME": {"value": model_default},
 # Auto-approve the agent's shell/terminal hooks (no TTY in a container).
 "HERMES_ACCEPT_HOOKS": {"value": "1"},
 # Secrets (encrypted at rest on Render):
 "API_SERVER_KEY": {"value": shared_key.base64},
 "OPENROUTER_API_KEY": {"value": openrouter_api_key},
 # Modal is a runtime credential: Hermes creates sandboxes at runtime with this pair.
 "MODAL_TOKEN_ID": {"value": modal_token_id},
 "MODAL_TOKEN_SECRET": {"value": modal_token_secret},
 },
 opts=pulumi.ResourceOptions(provider=render_provider),
)

# Open WebUI reaches Hermes at its Render-assigned internal hostname (the read-only slug).
hermes_api_base_url = pulumi.Output.concat("http://", hermes.slug, ":8642/v1")

# ---------------------------------------------------------------------------
# Open WebUI: also a PRIVATE Render service, but the container joins your tailnet
# (tailscaled in userspace + `tailscale serve`), so you reach it at
# https://open-webui.<tailnet>.ts.net from your devices. Never public.
# ---------------------------------------------------------------------------
webui = render.PrivateService(
 "open-webui",
 name="open-webui",
 plan="standard", # 2 GB; Open WebUI OOMs on starter's 512 MB
 region=region, # must match Hermes for private networking
 runtime_source={
 "image": {
 "image_url": pulumi.Output.concat("ghcr.io/", ghcr_user, "/open-webui"),
 "digest": webui_image.digest,
 "registry_credential_id": ghcr_credential.id,
 },
 },
 disk={"name": "openwebui-data", "mount_path": "/app/backend/data", "size_gb": 5},
 env_vars={
 "PORT": {"value": "8080"}, # tailscale serve points at this; Open WebUI binds it
 "OPENAI_API_BASE_URL": {"value": hermes_api_base_url},
 "ENABLE_OPENAI_API": {"value": "true"},
 "ENABLE_OLLAMA_API": {"value": "false"},
 "DEFAULT_MODELS": {"value": model_default},
 "WEBUI_AUTH": {"value": "true"},
 "WEBUI_URL": {"value": webui_tailnet_url},
 # Reached only over the tailnet, so the first sign-up is the admin.
 "ENABLE_LOGIN_FORM": {"value": "true"},
 "ENABLE_SIGNUP": {"value": "true"},
 "DEFAULT_USER_ROLE": {"value": "admin"},
 # Always apply env config; otherwise Open WebUI persists the first run's config.
 "ENABLE_PERSISTENT_CONFIG": {"value": "false"},
 # Don't load local embedding/RAG models (a big memory saver; the agent is the brain).
 "RAG_EMBEDDING_ENGINE": {"value": "openai"},
 "BYPASS_EMBEDDING_AND_RETRIEVAL": {"value": "true"},
 "OFFLINE_MODE": {"value": "true"},
 "HF_HUB_OFFLINE": {"value": "1"},
 # Secrets (encrypted at rest):
 "OPENAI_API_KEY": {"value": shared_key.base64},
 "WEBUI_SECRET_KEY": {"value": webui_secret_key.base64},
 # Tailscale joins this container to your tailnet and serves Open WebUI on it.
 "TS_AUTHKEY": {"value": tailscale_auth_key},
 "TS_HOSTNAME": {"value": webui_hostname},
 },
 opts=pulumi.ResourceOptions(depends_on=[hermes], provider=render_provider),
)

pulumi.export("hermesInternalUrl", hermes_api_base_url)
# Open this from any device on your tailnet:
pulumi.export("openWebUiTailnetUrl", webui_tailnet_url)

name: hermes-agent-pulumi
runtime: yaml
description: Deploy a Hermes agent (Render + Modal + Tailscale) as one declarative Pulumi program.

# The Render SDK is bridged from the official Terraform provider. Run once:
# pulumi package add terraform-provider render-oss/render 1.8.0
# Then `pulumi install` regenerates it on a fresh checkout.
packages:
 render:
 source: terraform-provider
 version: 1.1.4
 parameters:
 - render-oss/render
 - 1.8.0

config:
 renderRegion:
 type: string
 default: oregon
 modelDefault:
 type: string
 default: gpt-5.5
 ghcrUser:
 type: string
 ghcrToken:
 type: string
 secret: true
 modalTokenId:
 type: string
 secret: true
 modalTokenSecret:
 type: string
 secret: true
 openrouterApiKey:
 type: string
 secret: true
 tailscaleAuthKey:
 type: string
 secret: true
 tailnetDnsName:
 type: string
 renderApiKey:
 type: string
 secret: true
 renderOwnerId:
 type: string

variables:
 registryAuth:
 - address: ghcr.io
 username: ${ghcrUser}
 password: ${ghcrToken}
 # Open WebUI reaches Hermes at its private slug; you reach Open WebUI on the tailnet.
 hermesApiBaseUrl: http://${hermes-agent.slug}:8642/v1
 webuiTailnetUrl: https://open-webui.${tailnetDnsName}

resources:
 # Render provider auth (from config or ESC).
 render:
 type: pulumi:providers:render
 properties:
 apiKey: ${renderApiKey}
 ownerId: ${renderOwnerId}

 # Secrets generated in state: the replacement for `openssl rand -base64 32`.
 hermes-shared-key:
 type: random:RandomBytes
 properties:
 length: 32
 webui-secret-key:
 type: random:RandomBytes
 properties:
 length: 32

 # Build + push both images during `pulumi up` (no docker CLI).
 hermes-image:
 type: docker-build:Image
 properties:
 context:
 location: ./hermes
 dockerfile:
 location: ./hermes/Dockerfile
 platforms: [linux/amd64]
 tags:
 - ghcr.io/${ghcrUser}/hermes-agent:latest
 push: true
 registries: ${registryAuth}
 webui-image:
 type: docker-build:Image
 properties:
 context:
 location: ./openwebui
 dockerfile:
 location: ./openwebui/Dockerfile
 platforms: [linux/amd64]
 tags:
 - ghcr.io/${ghcrUser}/open-webui:latest
 push: true
 registries: ${registryAuth}

 # Credential Render uses to pull the private GHCR images.
 ghcr:
 type: render:RegistryCredential
 properties:
 name: ghcr
 registry: GITHUB
 username: ${ghcrUser}
 authToken: ${ghcrToken}
 options:
 provider: ${render}

 # Hermes: a PRIVATE Render service. No public URL; encrypted env vars are the secret mechanism.
 hermes-agent:
 type: render:PrivateService
 properties:
 name: hermes-agent
 plan: standard  # 2 GB; the gateway OOMs on starter's 512 MB
 region: ${renderRegion}
 runtimeSource:
 image:
 imageUrl: ghcr.io/${ghcrUser}/hermes-agent
 digest: ${hermes-image.digest}
 registryCredentialId: ${ghcr.id}
 disk:
 name: hermes-data
 mountPath: /opt/data
 sizeGb: 5
 envVars:
 PORT: { value: "8642" }
 API_SERVER_ENABLED: { value: "true" }
 API_SERVER_HOST: { value: "0.0.0.0" }
 API_SERVER_PORT: { value: "8642" }
 API_SERVER_MODEL_NAME: { value: "${modelDefault}" }
 HERMES_ACCEPT_HOOKS: { value: "1" }
 API_SERVER_KEY: { value: "${hermes-shared-key.base64}" }
 OPENROUTER_API_KEY: { value: "${openrouterApiKey}" }
 MODAL_TOKEN_ID: { value: "${modalTokenId}" }
 MODAL_TOKEN_SECRET: { value: "${modalTokenSecret}" }
 options:
 provider: ${render}

 # Open WebUI: also a PRIVATE Render service, joined to your tailnet by its image.
 open-webui:
 type: render:PrivateService
 properties:
 name: open-webui
 plan: standard  # 2 GB; Open WebUI OOMs on starter's 512 MB
 region: ${renderRegion}  # must match Hermes for private networking
 runtimeSource:
 image:
 imageUrl: ghcr.io/${ghcrUser}/open-webui
 digest: ${webui-image.digest}
 registryCredentialId: ${ghcr.id}
 disk:
 name: openwebui-data
 mountPath: /app/backend/data
 sizeGb: 5
 envVars:
 PORT: { value: "8080" }
 OPENAI_API_BASE_URL: { value: "${hermesApiBaseUrl}" }
 ENABLE_OPENAI_API: { value: "true" }
 ENABLE_OLLAMA_API: { value: "false" }
 DEFAULT_MODELS: { value: "${modelDefault}" }
 WEBUI_AUTH: { value: "true" }
 WEBUI_URL: { value: "${webuiTailnetUrl}" }
 ENABLE_LOGIN_FORM: { value: "true" }
 ENABLE_SIGNUP: { value: "true" }
 DEFAULT_USER_ROLE: { value: "admin" }
 ENABLE_PERSISTENT_CONFIG: { value: "false" }
 RAG_EMBEDDING_ENGINE: { value: "openai" }
 BYPASS_EMBEDDING_AND_RETRIEVAL: { value: "true" }
 OFFLINE_MODE: { value: "true" }
 HF_HUB_OFFLINE: { value: "1" }
 OPENAI_API_KEY: { value: "${hermes-shared-key.base64}" }
 WEBUI_SECRET_KEY: { value: "${webui-secret-key.base64}" }
 TS_AUTHKEY: { value: "${tailscaleAuthKey}" }
 TS_HOSTNAME: { value: "open-webui" }
 options:
 provider: ${render}
 dependsOn:
 - ${hermes-agent}

outputs:
 hermesInternalUrl: ${hermesApiBaseUrl}
 # Open this from any device on your tailnet:
 openWebUiTailnetUrl: ${webuiTailnetUrl}

Neither box is published to the internet. The same code, plus the Dockerfiles and the Tailscale entrypoint, is in the companion repo:

github.com/dirien/hermes-agent-pulumi

Cost

Two details shape the bill. The host starts on Render’s free Hobby workspace, but a private service with a persistent disk runs on paid compute, so you need a card on file the moment you add them. There is no free-tier version of this architecture. Both services also need the 2 GB Standard plan, since the gateway and Open WebUI each OOM on the 512 MB Starter.

	Hermes gateway	Open WebUI
Plan	Standard (2 GB)	Standard (2 GB)
Compute	~$25/mo	~$25/mo
Disk	5 GB	5 GB
Public URL	none	none (tailnet only)

That lands the host around $50/month, in line with what a comparable managed setup runs. Modal and OpenRouter bill by usage on top, so the total tracks how hard the agent works rather than a flat number. Tailscale is free for a personal tailnet, so the private gateway adds nothing to that.

The persistent disk pins each service to a single instance, so there is no autoscaling, and a redeploy drops a few seconds of connectivity while the old instance lets go of the disk. For a single-user agent that is a fine trade, and it is why “stand it up in a second region” is a copy-the-config move and not a live failover.

The OpenRouter Activity dashboard: 56 requests and 738K tokens of gpt-5.5 usage, all routed through one key.

Running the deployment

With your ESC environment referenced in Pulumi.dev.yaml, deploy with:

pulumi up

This generates the Render SDK’s resources, builds and pushes both images through buildx, and creates the two private services with their disks and encrypted env vars, all in one run, with no docker CLI and no secret import hiding in a script. When it finishes, both services show up in the Render dashboard with no public URL.

The hermes-agent and open-webui private services in the Render dashboard, both deployed with no public URL.

Access the agent

Give it a few minutes after pulumi up finishes. The images still have to be pulled and the containers booted, and Open WebUI has to join your tailnet before the URL resolves. Refresh until it loads.

Open WebUI is reachable only from a device on your tailnet, at https://open-webui.<your-tailnet>.ts.net, with no domain to buy, no DNS to point, and no certificate to manage. Open it, sign up so the first account becomes the admin, and ask the agent to run a Python command. It should fire up a Modal sandbox and return the result, with nothing exposed to the public internet at any point.

Open WebUI on the tailnet URL: the gpt-5.5 agent runs sum(i*i for i in range(1, 1001)) in a Modal sandbox and returns 333833500.

When the agent runs that code, Modal builds a throwaway sandbox specifically for it. There is no Modal resource to declare, no Modal CLI to call, and no Modal provider to bridge. The Hermes service carries MODAL_TOKEN_ID and MODAL_TOKEN_SECRET as two encrypted env vars, and that is the whole footprint.

The Modal dashboard: the per-task sandboxes the Hermes agent spun up at runtime to execute code.

Verify the deployment (optional)

If the chat UI does not load, give the services a few minutes to finish their first boot, then check two things:

Render dashboard logs for the hermes-agent and open-webui services, to confirm both started and pulled their images.
Tailscale admin console (Machines), to confirm the open-webui node joined your tailnet. If it did not, the TS_AUTHKEY is likely expired or not reusable.

Security considerations

Self-hosted AI servers get found fast: scanners like Shodan and Censys enumerate a freshly exposed one within hours to days, and the code-capable ones are already under active attack. Running an always-on agent on a machine you also use for everything else invites prompt injection on top of that. The answer here is to keep the agent unroutable from the internet and to isolate the code it runs. That defeats the scanner: nobody finds an open port to attack. It does less against the agent being turned against you from the inside, which is the harder problem.

One detail makes that isolation essential. In a headless container there is no one at the terminal to approve anything, and the same is true of its cron and CI runs, so the deployment runs Hermes with HERMES_ACCEPT_HOOKS=1 to auto-accept the shell hooks it would otherwise pause on. The practical result is that the agent’s code runs with no human in the loop, which is exactly why the Modal sandbox has to carry the weight: when the model writes a bad command, the throwaway container it runs in is what stands between that command and the gateway’s credentials, not a confirmation prompt.

Concern	Typical self-host	This deployment
Gateway reachability	Public port, or behind a login	Private service, no public URL
Chat UI reachability	Public hostname + auth	Tailnet only (`tailscale serve`)
Code execution	Runs next to the gateway	Isolated Modal sandbox, torn down after
Command approval	Human approves dangerous commands	Headless: isolation replaces the prompt
Secrets in transit	Exposed if accessed over HTTP	Encrypted env vars; tailnet-encrypted access

This is private, which is not the same as fully trusted. A few places the boundary still leaks:

The shared bearer never rotates. Hermes is unroutable from the internet, but it is wide open to Open WebUI, and the two share one static bearer key. Whoever owns the front end owns the gateway behind it. Rotate it periodically.
The tailnet policy is click-ops. The ACLs, the node’s tags, and the one-time HTTPS toggle live in the Tailscale admin console, not in pulumi up, and the reusable key has no tag scoping its blast radius. The Tailscale provider can manage that policy declaratively if you want it in code.
Signup defaults to admin. Fine on a single-user tailnet, the first thing to lock down (ENABLE_SIGNUP, DEFAULT_USER_ROLE) once more than one person can reach the URL.
Every messaging client is another entry point. Each platform you wire up adds another way in, which is why the tailnet is the baseline for securing this, not the full extent of it.
A compromised agent can still reach out. The private core stops external discovery, but it does not contain the agent itself. The gateway holds the Modal and OpenRouter tokens as env vars and can call out freely, so a prompt-injected agent could read its own environment and send them somewhere; the Modal sandbox isolates the code the agent runs, not the credentials it holds. Cap and rotate the OpenRouter key, keep the Modal token off the gateway, and allowlist outbound traffic, which on Render means a forward-proxy sidecar (Render gives you dedicated outbound IPs and inbound rules, but no destination egress filter).

My recommendations:

Keep both services private; never give Hermes a public URL
Run code in Modal sandboxes, since HERMES_ACCEPT_HOOKS=1 means no human approves commands
Rotate the shared bearer key and the Tailscale auth key periodically
Cap and rotate the OpenRouter key; a leaked credential is the likelier failure mode than an open port
Use Pulumi ESC for secrets instead of hardcoding
Lock down Open WebUI signup once more than one person can reach it
Manage Tailscale ACLs and tags deliberately, in the admin console or with the Tailscale provider

What’s next?

The deployment is the foundation the agent builds on. A private agent becomes useful when it remembers earlier context, runs code without a human at the terminal, and initiates contact on its own. The disk is what makes this persist: its memory and its schedule live on that disk, so they survive every restart.

It schedules itself. Hermes ships a first-class cron you drive in plain language, either by asking the agent in the chat window or through its CLI. The schedule takes intervals, five-field cron, or one-shots:

hermes cron create "every 1h" \
 "Scrape <listing-url> for the price, compare it with seen.json in the sandbox, message me on Telegram only if it dropped, and otherwise reply exactly [SILENT]." \
 --deliver telegram

The [SILENT] reply suppresses delivery when nothing changed, so the watcher only speaks up when there is something to say. Scheduled runs skip memory by default, so polling does not pollute what the agent has learned.

It connects where you work, with the same secret pattern. Messaging clients attach to the same gateway, and their tokens ride as encrypted env vars on the Hermes service, the exact shape you used for the Modal pair:

// alongside MODAL_TOKEN_ID on the hermes service:
TELEGRAM_BOT_TOKEN: { value: cfg.requireSecret("telegramBotToken") },

plus a platform block you add to config.yaml (the image ships only the model and terminal blocks; Hermes supports the messaging platforms on top, per its docs):

gateway:
 platforms:
 telegram:
 require_mention: true

With those two pieces, a few things are genuinely doable today:

A price or uptime watcher that only speaks up when something changed, using the [SILENT] and persistent-sandbox dedup pattern above.
A daily briefing that ranks and dedups your feeds and delivers the top items each morning, the documented daily-briefing bot. If you want it to adapt to your taste over time, that is the optional Honcho memory provider’s job; it reads your preferences out of the conversation itself. The built-in file memory has no version of that.
A PR-review teammate that reviews open pull requests against your repo’s AGENTS.md conventions and saves recurring patterns as a skill, so the fifth review needs less hand-holding than the first.

A few limits are worth stating plainly, since demos tend to overstate the capabilities. Hermes does not ship Gmail, Calendar, or market integrations out of the box; those are wired up as MCP servers or skills, which is more work than a launch video implies. It curates its own memory and improves its own skills, but it does not retrain the underlying model or autonomously change its high-level strategy, so “self-learning” here means whatever it writes into memory and skills each run.

Conclusion

Deploying a Hermes agent with infrastructure as code means you can reproduce the setup anytime, version control it, and tear it down with a single pulumi destroy. Keeping both services private, with no public URL and no exposed ports, means the code-executing core never sits on the internet for someone to find.

The part worth carrying to the next service you deploy: not every layer has a provider, and that is fine. The task is to tell a real resource apart from a runtime dependency wearing a costume. Render and the images are resources you declare; Modal and Tailscale are credentials you hand over. Draw that map first, and the program follows from it.

To generate that map instead of writing it by hand, Pulumi Neo can take a target architecture like this one and produce a first draft of the program. The agent side of this shift is covered in How Building AI Agents Has Changed in 2026 and the platform side in Agent Sprawl Is Here. Your IaC Platform Is the Answer..

If you run into issues or have questions, drop by the Pulumi Community Slack or GitHub Discussions. New to Pulumi? Get started here.

Neo code reviews: AI code review built for infrastructure

Pulumi Neo Team — Mon, 22 Jun 2026 08:00:00 -0700

Today we’re introducing Pulumi Neo code reviews, now in public preview. Neo code reviews analyze pull request changes in conjunction with what Pulumi Cloud knows about your running infrastructure, providing both high-level and code-level feedback.

Normal code review agents can’t reliably anticipate the impact an infrastructure-as-code change will have. This is because they don’t have access to critical aspects of the IaC workflow: the potential impact the update will have, in this case the pulumi preview output; and the current state of the cloud infrastructure. Neo not only has access to both of those, but also to the entirety of your other cloud context, such as stack relationships and dependencies.

Running reviews

Neo can review every pull request automatically, or only when someone mentions @pulumi-neo. Either way, it skips draft pull requests and those opened by bots by default.

A review is a comment, so it informs the person approving the merge and sits alongside the required checks and branch protection you already enforce. Neo code reviews run inside the same governance as every other Neo task, with the RBAC, guardrails, and audit logging your organization has set.

Enable code reviews

Neo code reviews are available on GitHub during public preview. They require Pulumi Neo to be enabled for your organization, the Pulumi GitHub App installed on the repositories you want reviewed, and a one-time grant from each organization user to access their GitHub account under Management > Version control. If Neo currently posts preview summaries on your pull requests, code reviews are already enabled, and they take the place of those summaries.

Neo code reviews are free while in public preview. On July 1, 2026, they’ll be generally available, and reviews will begin counting toward your organization’s Neo token usage, at the same per-token rate as any other Neo task. The pricing page shows that rate and the monthly token allotment included with each plan.

Give it a try

Open a pull request against a stack Pulumi manages and see Neo’s review. We want to hear what it catches and what it misses, so hop into the Pulumi Community Slack and tell us.

Stop Prompting. Design the Loop.

Engin Diri — Tue, 09 Jun 2026 00:00:00 +0000

For about two years, the unit of work with a coding agent was the prompt. You wrote a good one, you gave it enough context, you read what came back, and you wrote the next one. The agent was a tool, and you were holding it the entire time, one turn after another.

That part is ending. Addy Osmani, a director of AI at Google Cloud, has a name for what replaces it, and I have not stopped thinking about it since: loop engineering. You stop being the person who prompts the agent. You design the loop that prompts it for you.

In my phrasing: you stop being the thing that runs, and start designing the thing that runs. The leverage moves up a layer. What I want to do here is take an honest look at the pieces, and at the part nobody automates.

The leverage moved up a layer

The people building these tools have already made the jump. Peter Steinberger has been posting it as a monthly reminder.

Peter Steinberger (@steipete) on X.

Boris Cherny, who heads Claude Code at Anthropic, says the same thing about his own job. He does not prompt Claude anymore. He has loops running that prompt Claude and decide what to do next, scanning the issue tracker, the team chat, and the timeline for what to build. “My job is to write loops.”

A loop is a goal that prompts itself. You set the purpose, and the system keeps iterating until it’s met. In practice it finds the work, hands it out, checks the result, writes down what got finished, and decides the next thing, then it pokes the agent instead of you. You build that small system once and let it run.

Look closer, and a loop is really two loops nested. The inner one does the work against a spec. The outer one decides what the work should be: it watches an issue tracker, an error feed, a changelog, then writes the next spec and hands it down. Most people are still running that outer loop by hand, in their head, and calling it a backlog.

The part that surprised me is that this is barely a tooling problem anymore. A year ago a loop meant a pile of bash you wrote and maintained forever. Now the pieces ship inside the products, and the same shapes show up in Claude Code and in Codex. Osmani puts loop engineering one floor above the harness, the context and tooling you wire around a single agent. I wrote about that harness a couple of weeks ago. The loop is the thing that runs on top of it: it runs on a timer, it spawns helpers, and it feeds itself.

The five pieces, and the one that holds them together

Strip loop engineering down and you get roughly five building blocks, plus one place to remember things. Both Claude Code and Codex have all five now. The names differ here and there; the capability is the same.

Automations are the heartbeat. They are what make a loop an actual loop and not one run you did once. A prompt or command on a cadence, a scheduled task, a hook that fires at a point in the agent’s lifecycle, or a job on CI that keeps running after you close the laptop. Discovery and triage run themselves, and the findings that matter come to you.
Worktrees keep parallel from turning into chaos. The second you run more than one agent, the files start colliding. Two agents writing the same file is the same headache as two engineers committing to the same lines with nobody talking first. A git worktree is a separate working directory on its own branch, so one agent’s edits cannot touch another’s checkout.
Skills are intent, written down. An agent starts every session cold and fills any hole in your intent with a confident guess. A skill is that intent written on the outside: the conventions, the build steps, the “we don’t do it like this because of that one incident,” recorded once where the agent reads it every run. Without skills, the loop re-derives your whole project from zero every cycle.
Connectors let the loop touch your real tools. Built on MCP, they let the agent read the issue tracker, query a database, hit a staging API, or drop a message in chat. This is the difference between an agent that says “here is the fix” and a loop that opens the pull request, links the ticket, and pings the channel once CI goes green.
Sub-agents keep the maker away from the checker. The model that wrote the code is far too generous grading its own homework. A second agent with different instructions, and sometimes a different model, catches the things the first one talked itself into. Worktrees and a cold-context reviewer are two pieces I have written about before, back when the question was running agents in parallel without them trampling each other.

Then the sixth thing: memory. A markdown file, a Linear board, a state file, anything that lives outside the single conversation and holds what is done and what is next. It sounds too dumb to matter, and it’s the whole game. The model forgets everything between runs, so the memory has to live on disk, not in the context window. The agent forgets. The repo does not.

What makes a loop hold together

A loop running unattended is also a loop making mistakes unattended. The one thing that keeps it honest is verification, and verification needs an oracle, something outside the model that returns a hard yes or no. Passing tests, a clean build, a green pipeline, a real production signal. Without an oracle, the loop compounds confidently wrong work, faster than you can read it.

The cleanest version of this already ships in the tools. Claude Code’s /goal keeps working across turns until a condition you actually wrote holds, something like “every test in auth/ passes and lint is clean,” and after every turn a separate, faster model reads the transcript and decides whether you are there yet. The agent that wrote the code is not the one that grades it. That is the maker-and-checker split applied to the stop condition itself. Codex’s /goal reaches the same finish line a different way: the agent audits its own work against the evidence before it can call the goal done.

What the loop still won’t do for you

The loop changes the shape of the work. It does not take it off your desk. And a few things get sharper as the loop gets better, not softer.

Verification is still on you. The split reviewer is what makes “it’s done” mean something, but “done” is a claim, not a proof. Your job is still to ship code you confirmed works, which is harder to remember when the diff arrived while you were at lunch.

The bill comes in two currencies. Tokens and attention. A single unattended run can burn through millions of tokens, and that is only worth it when the tokens buy something worth more than they cost. The quieter trap is the second currency: memory is what lets a loop compound over time, and slop compounds right alongside it. A loop pointed at a vague goal does not get tired and stop. It gets faster.

Your understanding rots if you let it. The faster the loop ships code you did not write, the wider the gap between what exists in the repo and what you actually understand. A smooth loop grows that gap faster, not slower, unless you read what it made. The comfortable posture, where you stop having an opinion and take whatever the loop gives back, is the risky one. Two engineers can build the exact same loop and get opposite results, one moving faster on work they understand deeply, the other avoiding the work entirely. The loop cannot tell which one you are.

When the loop reaches production

Most of this thinking grew up around application code, where a bad run costs you a revert. When the loop reaches into infrastructure, the blast radius is a production outage rather than a revert, and the verification bar has to rise to meet it. The upside is that infrastructure hands the loop a better oracle than application code does. A plan diff is deterministic and machine-readable, a policy check returns a hard verdict, and drift and cost are numbers you can put a threshold on. A reviewer, whether human or agent, can read the change cold, with no memory of the prompt that produced it. That cold-context check is exactly what an unattended loop needs, and it’s the reason an infrastructure loop can be built to hold together while you sleep. Pulumi Neo reasons over the state graph directly, so the checker grounds every claim in what the change actually does, not in what the writer says it does.

Where to start

Pick the loop you can actually trust first. In order:

Start where “done” is unambiguous. CI triage, dependency bumps, a flaky-test hunt, a failing job you keep re-running by hand. Loops need an oracle, so begin where the oracle already exists.
Write the memory file before the loop. One markdown file, or a board. What is done, what is next, what was tried and failed. This is the spine, and everything else hangs off it.
Split the checker from the maker. Use /goal with a verifiable condition, or a second agent with its own instructions. Never let the agent that did the work be the one that decides the work is finished.
Cap it, then read everything. A max-iteration count, a token budget, a teardown step. Run it once, end to end, then read every line it shipped. The first run is the measurement, not the payoff.

Then look at what you built. You designed it once, and it ran without you steering each step. That is the real shift. But the leverage only holds if you wire the loop like an engineer, not like someone looking for permission to stop thinking. Read what it ships. Keep an opinion. Judgment is the one part that does not move up a layer.

The loop will do the typing. The thinking is the work.

See how Pulumi Neo closes the loop on your infrastructure

Use Your Mac for AI Agents: Self-Host Gemma 4 12 B with Pulumi and Tailscale

Pablo Seibelt — Thu, 04 Jun 2026 00:00:00 +0000

If you run AI tools and agents, you’ve probably accepted three tradeoffs: your data leaves your network, you can’t work offline, and your bill scales with usage.

Open-weight models now run well on consumer hardware. Once the model is on your machine, your data stays local, inference works offline, and tokens cost nothing. If you own a modern Mac, you can run a high-quality model yourself.

Gemma 4 is an open-weights model family from Google. This post focuses on Gemma 4 12 B, released in June 2026, using Unsloth’s Q8_0 GGUF. The 12 B model fits comfortably on a modern Mac while leaving enough headroom for local llama.cpp and a chat UI.

We’ll use llama.cpp for host-native inference, k3d for a local Kubernetes cluster, Pulumi for infrastructure as code, and Tailscale for secure access.

Prerequisites

This setup was validated on the following hardware:

macOS 26 Tahoe, version 26.5
MacBook Pro with Apple M3 Max
36 GB RAM

On this machine, llama.cpp reported about 20 output tokens per second for a 160-token validation response with unsloth/gemma-4-12b-it-GGUF, gemma-4-12b-it-Q8_0.gguf, and a 131,072-token context. Sustained throughput varies by prompt length, thermal state, and llama.cpp settings.

You’ll need brew, docker, pulumi, and tailscale installed. We’ll also install k3d during the process.

Run Gemma 4 with host-native llama.cpp

We use llama.cpp directly on macOS to leverage Apple Metal acceleration. Running the LLM on the host is more efficient than trying to pass GPU access into a local Kubernetes VM.

Install the build tools:

brew install cmake git

Then build llama.cpp from source and download the multimodal projector. In validation, Homebrew llama.cpp 9430 could run text inference, but it could not load the new Gemma 4 12 B projector and failed with unknown projector type: gemma4uv. Building current llama.cpp from source fixed that.

llm_home="$HOME/pulumi-gemma4-llm"
mkdir -p "$llm_home/models" "$llm_home/logs"

if [ ! -d "$llm_home/llama.cpp/.git" ]; then
 git clone --depth 1 https://github.com/ggml-org/llama.cpp.git "$llm_home/llama.cpp"
fi

cmake -S "$llm_home/llama.cpp" \
 -B "$llm_home/llama.cpp/build" \
 -DGGML_METAL=ON \
 -DGGML_BLAS=ON \
 -DCMAKE_BUILD_TYPE=Release

cmake --build "$llm_home/llama.cpp/build" --target llama-server -j 10

curl -L --fail \
 --output "$llm_home/models/mmproj-F16.gguf" \
 https://huggingface.co/unsloth/gemma-4-12b-it-GGUF/resolve/main/mmproj-F16.gguf

Then download and run the model with this command:

"$HOME/pulumi-gemma4-llm/llama.cpp/build/bin/llama-server" \
 --hf-repo unsloth/gemma-4-12b-it-GGUF \
 --hf-file gemma-4-12b-it-Q8_0.gguf \
 --mmproj "$HOME/pulumi-gemma4-llm/models/mmproj-F16.gguf" \
 --host 127.0.0.1 \
 --port 18080 \
 --ctx-size 131072 \
 --parallel 1 \
 --jinja \
 --reasoning off

We use port 18080 because 8080 is commonly used and is likely to conflict with another service you may already have running locally. If your port 8080 is free, you can use it and adjust the Pulumi config later.

The model file is about 12.65 GB, and the projector is about 116 MB. Gemma 4 12 B advertises a 131,072-token context, and this Mac loaded that full context with --parallel 1. llama.cpp projected about 15.1 GiB of Apple Metal device memory for the text model and about 258 MiB worst-case memory for the projector, leaving enough headroom for Open WebUI and the rest of the local stack. The --reasoning off flag keeps OpenAI-compatible chat responses visible in clients that do not read separate reasoning fields.

With --mmproj, /v1/models advertised capabilities: ["completion","multimodal"]. In local validation, Open WebUI accepted an uploaded Pulumi logo image and Gemma 4 described it correctly. A small WAV file also worked through the OpenAI-compatible input_audio request shape, though llama.cpp logs still mark audio input as experimental.

Verify the LLM API

Open a new terminal and check if llama.cpp is responding:

curl http://127.0.0.1:18080/v1/models

The /v1/models endpoint should return the model ID unsloth/gemma-4-12b-it-GGUF. Now try a chat completion:

curl http://127.0.0.1:18080/v1/chat/completions \
 -H "Content-Type: application/json" \
 -d '{
 "model": "unsloth/gemma-4-12b-it-GGUF",
 "messages": [{"role": "user", "content": "Reply with exactly: OK"}],
 "temperature": 0,
 "max_tokens": 32
 }'

The chat prompt Reply with exactly: OK should return content OK. In validation, llama.cpp reported an output token velocity of about 20 tokens per second for a longer 160-token response.

Keep llama.cpp running after reboot

For a permanent setup, put the llama.cpp startup script and logs under a folder in your home directory and let launchd restart it when you sign in:

llm_home="$HOME/pulumi-gemma4-llm"
mkdir -p "$llm_home/logs" "$HOME/Library/LaunchAgents"

cat > "$llm_home/start-llama-server.sh" <<'EOF'
#!/bin/zsh
set -euo pipefail

export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"

exec "$HOME/pulumi-gemma4-llm/llama.cpp/build/bin/llama-server" \
 --hf-repo unsloth/gemma-4-12b-it-GGUF \
 --hf-file gemma-4-12b-it-Q8_0.gguf \
 --mmproj "$HOME/pulumi-gemma4-llm/models/mmproj-F16.gguf" \
 --host 127.0.0.1 \
 --port 18080 \
 --ctx-size 131072 \
 --parallel 1 \
 --jinja \
 --reasoning off
EOF

chmod +x "$llm_home/start-llama-server.sh"

cat > "$HOME/Library/LaunchAgents/com.pulumi.gemma4.llama-server.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
 <key>Label</key>
 <string>com.pulumi.gemma4.llama-server</string>
 <key>ProgramArguments</key>
 <array>
 <string>$llm_home/start-llama-server.sh</string>
 </array>
 <key>WorkingDirectory</key>
 <string>$llm_home</string>
 <key>RunAtLoad</key>
 <true/>
 <key>KeepAlive</key>
 <true/>
 <key>StandardOutPath</key>
 <string>$llm_home/logs/llama-server.out.log</string>
 <key>StandardErrorPath</key>
 <string>$llm_home/logs/llama-server.err.log</string>
</dict>
</plist>
EOF

launchctl bootout gui/$(id -u)/com.pulumi.gemma4.llama-server 2>/dev/null || true
launchctl bootstrap gui/$(id -u) "$HOME/Library/LaunchAgents/com.pulumi.gemma4.llama-server.plist"
launchctl kickstart -k gui/$(id -u)/com.pulumi.gemma4.llama-server

Check the launchd service and llama.cpp logs:

launchctl print gui/$(id -u)/com.pulumi.gemma4.llama-server
tail -f "$HOME/pulumi-gemma4-llm/logs/llama-server.err.log"

If you want to stop llama.cpp later, unload the launchd service:

launchctl bootout gui/$(id -u)/com.pulumi.gemma4.llama-server

Deploy Open WebUI with Pulumi and k3d

Now we’ll deploy Open WebUI into a local Kubernetes cluster. This provides a polished chat interface that connects to our host-native LLM.

First, install k3d if you haven’t already:

brew install k3d

Create a new cluster for this project:

k3d cluster create pulumi-gemma4-blog-qa

We’ll use the Pulumi program in pulumi/examples. This program defaults to runtimeMode=host, which creates a Kubernetes ExternalName service pointing to your host machine.

Why not run the LLM inside Kubernetes on this Mac? Pulumi can do that, and the example supports it with runtimeMode=cluster, but that path is meant for Linux hosts with NVIDIA or AMD GPU device plugins.

On macOS, llama.cpp enables Metal by default, and Metal acceleration is available to native macOS processes. k3d runs Linux containers through Docker Desktop, so those pods do not get direct access to the Mac’s Metal device. Docker’s own vLLM Metal announcement calls out the same boundary: Metal-backed inference runs natively on the host because there is no Metal GPU passthrough for containers. That is why this setup keeps inference host-native and lets Pulumi manage the Kubernetes UI, service wiring, and optional Tailscale access around it.

Clone the examples repo, navigate to the program directory, and initialize a new stack:

git clone https://github.com/pulumi/examples.git
cd examples/kubernetes-py-self-host-gemma4-llm
pulumi stack init gemma4-local

Configure the program to match your local setup:

pulumi config set hostLlmPort 18080
pulumi config set llmBaseUrl http://llm-server:18080/v1

The Kubernetes service named llm-server maps to host.k3d.internal. In our validation, we confirmed that a disposable k3d pod could reach the Mac’s llama.cpp API at http://llm-server:18080/v1/models after a CoreDNS restart.

kubectl rollout restart deployment coredns -n kube-system

Run pulumi up to deploy Open WebUI and connect it to host-native llama.cpp:

pulumi up

In our validation environment, this command successfully reached the resource synthesis phase without requiring Tailscale credentials because Tailscale exposure is opt-in.

Access Open WebUI through Tailscale

Tailscale allows you to access your private Open WebUI instance from any device on your tailnet. Note that we only expose the web interface, not the raw LLM API, to keep the system secure.

The base Open WebUI deployment works without Tailscale credentials. To expose the web UI on your tailnet, enable Tailscale resources and provide an explicit api_key or OAuth/identity token:

pulumi config set enableTailscale true
pulumi config set tailscale:apiKey YOUR_API_KEY --secret

Once configured, Pulumi will create a Tailscale device or proxy that routes traffic to your Open WebUI service.

Use the model with Pi

Open WebUI gives you a browser-based chat interface, but local models are also useful from coding agents. Pi is the local coding agent used for this validation; if you do not use Pi, treat this section as an example of how any OpenAI-compatible client can point at the same local endpoint. Pi can point at the same OpenAI-compatible llama.cpp endpoint and use the model running on your Mac.

For a fresh Pi config, create ~/.pi/agent/models.json with a local provider that points at the llama.cpp API:

{
 "providers": {
 "local-llama": {
 "baseUrl": "http://127.0.0.1:18080/v1",
 "api": "openai-completions",
 "apiKey": "local",
 "compat": {
 "supportsDeveloperRole": false,
 "supportsReasoningEffort": false
 },
 "models": [
 {
 "id": "unsloth/gemma-4-12b-it-GGUF",
 "name": "Gemma 4 12B Q8 (local llama.cpp)",
 "reasoning": false,
 "input": ["text"],
 "contextWindow": 131072,
 "maxTokens": 1024,
 "cost": {
 "input": 0,
 "output": 0,
 "cacheRead": 0,
 "cacheWrite": 0
 }
 }
 ]
 }
 }
}

Then set Pi to use that provider and model by default in ~/.pi/agent/settings.json:

{
 "defaultProvider": "local-llama",
 "defaultModel": "unsloth/gemma-4-12b-it-GGUF",
 "defaultThinkingLevel": "off",
 "hideThinkingBlock": true
}

If you already have Pi configuration files, merge the local-llama provider and defaults into your existing JSON instead of replacing the files.

Advanced: Linux GPU in-cluster serving

If you’re running on a Linux host with an NVIDIA or AMD GPU, you can run the LLM directly inside the Kubernetes cluster. This requires the NVIDIA or ROCm device plugins.

The Pulumi program supports this through runtimeMode=cluster. In this mode, it deploys a LlmServer pod that manages the llama.cpp process within the cluster, using GPU resource requests to ensure hardware acceleration.

Cleanup

When you’re done, you can tear down the resources:

pulumi destroy
k3d cluster delete pulumi-gemma4-blog-qa
# Stop llama.cpp using the PID from your terminal
kill <PID>

Five Stacks Before Lunch: The Parallel Coding Playbook for Pulumi

Engin Diri — Tue, 02 Jun 2026 00:00:00 +0000

AI coding has two shapes right now. One agent in a loop, sequential work, you babysitting the chat window. Call that 2x. Most teams live here. Five agents in worktrees, parallel work, fresh-context review on every change. Call that 10x. The trick: 2x is mostly prompting, 10x is mostly plumbing.

The parallel coding playbook is a five-pattern setup for running multiple AI coding agents at the same time without them stepping on each other: an issue used as the spec, a plan/build/validate loop, parallel git worktrees, fresh-session review, and a self-healing layer. The whole thing targets application code. The interesting question, and the one I keep ending up at, is what changes when the five agents are touching infrastructure.

2x is prompting, 10x is plumbing

2x is one human, one agent, one repo, one branch. The agent writes, you review, you tell it to try again, it tries again. The bottleneck is your attention. Whatever the agent’s raw throughput, your reading speed sets the ceiling.

10x moves you out of the per-change loop and into the issue loop. You write five issues with sharp acceptance criteria, send each one to its own agent in its own worktree, and let them plan, build, and validate end-to-end. You read five PRs at lunch instead of pair-programming on one all morning.

Concurrent isolation does the work. And isolation is mostly an infrastructure problem.

The five pillars

The five pillars, in one sentence each.

Issue is the spec. The GitHub issue carries the acceptance criteria. The pull request is the artifact that gets validated. Input and output of every implementation are versioned, scoped, and reviewable on their own.
Plan, build, validate. Three stages, three artifacts. A markdown plan you can read in thirty seconds. A build that produces a diff. A validate step that checks the diff against the spec.
Parallel worktrees. Each agent runs in its own git worktree so concurrent changes never trample each other. One repo, five working trees, five branches.
Fresh-session review. A different agent, a different conversation, no shared context, reads the output and judges it. The reviewer never sees the writer’s chat. An agent reviewing its own output in the same context is theater.
Self-healing layer. When the same issue keeps coming back, fix the system that allowed it. Update the rules, the skills, the AGENTS.md. The agent gets better; the bug class disappears.

The application-code version of this playbook leans on ports, node_modules, and databases to get isolation right. The infrastructure version has a different toolbox.

What changes when the agents are touching infrastructure

Walk the pillars again, this time with a Pulumi shop in mind.

Issue is the spec. For application code, the spec describes behavior. For infrastructure, the spec is a Pulumi component contract plus a Pulumi Policies excerpt. “The resulting bucket is private, lives in eu-west-1, has SSE-KMS, and is tagged owner=team-x.” That sentence compiles to a typed component signature and three policy assertions. The agent does not get to interpret “looks right.” The acceptance criteria are deterministic, which is the whole reason this works.

Plan, build, validate. Pulumi already ships the validate step. pulumi preview produces a deterministic, machine-readable diff a second reviewer can judge without the conversation that produced it. The plan is a markdown doc the agent writes before touching code. The build is pulumi up --target against a review stack scoped to the resources the issue covers. The validate step is the preview output plus the Pulumi Policies verdict.

Parallel worktrees. Worktrees alone are not enough. Two worktrees pointing at the same Pulumi stack will fight over state on the first concurrent up. The unit of isolation for infrastructure is the stack, not the worktree. Each worktree gets its own ephemeral review stack and its own ESC environment for credentials. State branches with the work, credentials branch with the work, and the cloud account does not see five agents elbowing each other.

Fresh-session review. The hardest part of the application-code version is keeping the reviewer cold. For infrastructure, the substrate hands you the cold context. The pulumi preview JSON has no memory of the prompt that produced it. A separate agent reading it has the same starting point a human reviewer has: a diff, a stack name, a policy report. Pulumi Neo reasons over the state graph directly, so the reviewer grounds every claim in what the change actually does, not what the writer says it does. Reviewer quality still depends on how well your policies cover the stack, but the cold-context part comes built in.

Self-healing layer. Most Pulumi Policies rule messages today read like assertions. “S3 bucket has no encryption.” A self-healing layer needs them to read like instructions. “S3 bucket has no encryption. Set serverSideEncryptionConfiguration with SSE-KMS to fix.” That single rewrite is the difference between an agent flailing and an agent fixing the violation on the first try. When the same rule keeps tripping, the fix is upstream of the next pull request: in the rules, in the skills, in the policy itself.

The five catches, infra edition

Every parallelism story has a catch list. The application-code version lists port conflicts, node_modules sprawl, database conflicts, token blowouts, and PR pile-up. The infrastructure equivalents map almost one to one.

Port conflicts become stack-name collisions. Two agents naming their stack dev and racing each other into Pulumi Cloud. The fix is the same hash-the-path trick the app-code playbook uses: derive the stack name from pulumi.getProject() plus a hash of the worktree path. Resource names follow the same pattern. Collisions go away.

node_modules sprawl becomes provider plugin sprawl, mostly already solved. Three worktrees each pulling their own copy of pulumi-aws would add up fast, except Pulumi already shares plugins through a single cache at ~/.pulumi/plugins. Identical provider versions are reused across worktrees automatically. Per-worktree language SDKs (node_modules, venv) still need the usual care, but the provider layer is free.

Database conflicts become state conflicts. Two agents racing each other into pulumi up on the same stack is the same hazard as two agents writing to the same migrated database. The app-code playbook reaches for Neon branches or per-worktree SQLite files to isolate state. The infra answer is simpler: each worktree gets its own review stack. State branches with the work, by construction.

Token blowouts become cloud spend per ephemeral stack. The cost vector flips. For app code, the worry is LLM bills. For infrastructure, the worry is what your five agents just spun up in five review stacks. The mitigations are boring and they work. Use TTL stacks to tear review stacks down on a schedule. Avoid retainOnDelete on review-stack resources so the teardown actually frees them. Cap retries per spec. Watch the bill.

PR pile-up is the same problem. Five reviewed diffs are still five things waiting on the merge queue. The infra-flavored mitigations: stack-scoped reviewers (the human who owns the stack approves the change to it), the Pulumi Cloud audit log for grouping by stack and time, and auto-merge for the narrow class of changes where the preview diff is clean and every policy passes. That last one is where most of the throughput hides.

Where to start, this afternoon

Three steps, in order, on a stack with a small blast radius.

Write an AGENTS.md for the repo. Five paragraphs is enough. The component library, the stack naming convention, the policy rules, the review-stack TTL, and the one thing in this repo that bites every newcomer. Neo reads AGENTS.md natively, as do most coding agents. This file is the spec for how the agent should behave even before you write a spec for what it should build.
Cut a 24-hour review-stack TTL. Spin up a review stack on PR open, tear it down on PR close or after 24 hours, whichever comes first. This is the gate that turns “ephemeral” from a slogan into a line item that does not appear on next month’s bill.
Run three issues in parallel. Pick three open issues that touch unrelated resources. Spin up three worktrees, three review stacks, three ESC environments. Let each agent run end-to-end against its own stack. Then have a fourth agent read each preview JSON cold and produce a one-paragraph review. Read three PRs plus the reviewer’s summary at lunch.

That last step is the measurement. The first time you run it, half of the changes will fail validation. The second time, fewer. By the third time you will know whether your spec quality, your policies, and your stack hygiene are good enough to scale this to five, then ten, then to every issue tagged infra:fix.

If three issues finish cleanly, you have the substrate. If they do not, the gap is almost always in the spec or the policy rules, not the agent. Fix the spec, tighten the rule, run it again.

Five stacks before lunch

10x is five concurrent agents, working from five issues, against five stacks, behind five fresh-session reviews. The substrate is already there. Stacks isolate state. ESC isolates credentials. pulumi preview is the deterministic artifact a fresh reviewer can read cold. Pulumi Policies is the self-healing layer when you write the rule messages as instructions.

The remaining work is small and mostly wiring. Write the AGENTS.md. Cut the TTL. Pick three issues that touch unrelated resources. Read three PRs at lunch. Five stacks before the room empties out is a realistic Monday.

See how Pulumi Neo runs your stacks

Stop Tuning Prompts. Build a Harness.

Engin Diri — Tue, 26 May 2026 00:00:00 +0000

Anthropic shipped a piece earlier this month called How Claude Code Works in Large Codebases. I have not read anything more useful about coding agents this year. The core claim, in their words: “the ecosystem built around the model—the harness—determines how Claude Code performs more than the model alone.” In my phrasing: in a real codebase, the model is the smaller variable. The layer of context and tooling you wire around the agent matters more than which version of Sonnet or Opus is behind it.

The post stays high-level, which is the right move for a launch piece. What I want to do here is land it. Same seven pieces, but with the wiring you would actually put in a repo, in the order I would put it.

How Claude Code navigates without an index

Anthropic’s writeup says Claude Code works from the live codebase and does not require a codebase index to be built, maintained, or uploaded. The agent navigates the way an engineer would, with grep, find, ls, file reads, and reference-following. Anthropic calls this agentic search, and the upside is obvious: no separate index exists for you to keep fresh.

The downside is also obvious. An engineer who has never seen your repo and only has shell tools will flounder if you drop them in the root with no map. That is your agent on day one. Everything that follows is about giving it the map.

The AI layer in seven pieces

Every codebase used to have two artifacts engineers cared about: the code and the tests. A third exists now. Call it the AI layer, or the harness, or whatever you want. This layer is the set of context and tools you give your coding agent to operate in this specific repo. Anthropic breaks it into seven pieces, and each one solves a different scaling problem.

Anthropic gives each piece a role: CLAUDE.md is the foundation, hooks do self-improvement, skills are progressive disclosure, plugins handle distribution, LSP gives navigation, MCP is extension, subagents split exploration from editing. They are not equal in usage either. CLAUDE.md is read at the start of each session and stays in context for the duration. The others fire when relevant.

Lean and layered CLAUDE.md

The single biggest mistake I see is a root CLAUDE.md that has grown into a small book. Two thousand lines of conventions for parts of the repo the current task will never touch. Every session pays the tax. Anthropic’s own guidance is to keep these files focused on what applies broadly so they do not become a drag on performance, and you can feel that drag in practice: the agent gets cautious, slow, and oddly literal.

Keep the root file lean. What is this repo, broadly. The tech stack. The commands the agent will need (make test, make lint, how to run the dev server). General conventions that apply everywhere. That is most of what belongs there.

Local conventions go in subdirectory CLAUDE.md files. When the agent starts in a subdirectory, Claude Code walks upward from the working directory and loads every CLAUDE.md it finds on the way to the repo root, so root context is never lost and intermediate layers stack in the order you would expect. Claude Code can also discover files below the current working directory when it reads files in those subdirectories. That means services/api/CLAUDE.md only joins the session when the work reaches that service. Same for services/billing/, the frontend, the data layer.

If you already know the task is scoped to one service, start the agent in that subdirectory. The working directory becomes the focus, and the agent stays out of unrelated code unless you tell it otherwise. Most of the time, you know.

Two more cheap wins live in the same neighborhood. Scope the make test and make lint commands so the subdirectory version runs only the slice the agent is working in, instead of the whole repo on every change. And version-control your exclusion rules in .claude/settings.json so the agent never reads dist/, generated SDKs, or vendored code. Every file the agent skips is tokens you keep for the work that matters. If your directory layout is unconventional or has historical baggage, Anthropic also suggests adding a short codebase map to the root CLAUDE.md so the agent has somewhere to anchor.

Hooks that make the harness self-improving

Most teams use hooks as guardrails. Block edits in vendor/, refuse to delete migrations, kill the run if a secret turns up in a diff. That is fine and you should do it. But hooks have a second life that almost no one uses, and that second life is the more interesting one.

Both kinds register the same way, in .claude/settings.json, against named events Claude Code fires during a session:

{
 "hooks": {
 "SessionStart": [
 {
 "hooks": [
 {
 "type": "command",
 "command": "uv run --directory \"$CLAUDE_PROJECT_DIR\" python .claude/hooks/session_start_context.py"
 }
 ]
 }
 ],
 "Stop": [
 {
 "hooks": [
 {
 "type": "command",
 "command": "uv run --directory \"$CLAUDE_PROJECT_DIR\" python .claude/hooks/propose_claude_md.py"
 }
 ]
 }
 ]
 }
}

A SessionStart hook fires before the agent has done anything. Whatever the script prints to stdout is injected straight into the session as context, so you can preload the things the agent would otherwise have to spend a turn discovering: the current branch, the uncommitted diff, the last few commits. For a larger team you might fetch the Confluence or Notion page that owns the directory the engineer is working in. Every developer starts each session pre-oriented, with no manual setup.

"""SessionStart hook — prints orientation Claude reads as session context."""
import os, subprocess
from pathlib import Path

root = Path(os.environ.get("CLAUDE_PROJECT_DIR", "."))

def git(*args):
 out = subprocess.run(["git", *args], cwd=root, capture_output=True, text=True)
 return out.stdout.strip()

print("# Orientation\n")
print(f"## Branch\n{git('rev-parse', '--abbrev-ref', 'HEAD')}\n")
print(f"## Uncommitted changes\n{git('status', '--porcelain') or '(clean)'}\n")
print(f"## Recent commits\n{git('log', '-5', '--oneline')}\n")

The Stop hook is the more interesting one. It fires when the agent finishes its turn. At that moment the session context is still fresh, the diff is still small, and you have a free shot at a question nobody asks: did anything I changed invalidate the rules I wrote down? Spawn a separate headless Claude session, hand it the diff and the relevant CLAUDE.md files, ask it to propose updates, and write the result to a markdown review file. You read it when you are ready. The CLAUDE.md files stop going stale on their own.

The trick is to make the hook itself cheap and dispatch the LLM call in the background, so the end of every turn does not block on a reflection:

"""Stop hook — dispatch a headless Claude reflection in the background."""
import os, subprocess, sys
from pathlib import Path

# The reflector spawns its own headless Claude, whose Stop hook lands back
# here. The lock prevents infinite recursion.
if os.environ.get("REFLECT_LOCK"):
 sys.exit(0)

root = Path(os.environ.get("CLAUDE_PROJECT_DIR", "."))
diff = subprocess.run(
 ["git", "diff", "HEAD"], cwd=root, capture_output=True, text=True
).stdout
if not diff.strip():
 sys.exit(0)

env = {**os.environ, "REFLECT_LOCK": "1"}
subprocess.Popen(
 ["uv", "run", "python", ".claude/hooks/reflect_claude_md.py"],
 cwd=root, env=env,
 stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
)

reflect_claude_md.py is the part that calls a headless claude against the diff and writes .claude/claude-md-review.md. You can grow it from twenty lines to two hundred without ever blocking the agent.

The pattern that ties the two together: hooks let the harness improve itself in the background while you do the actual work.

Path-scoped skills

Skills are where the agent learns how to do a thing. CLAUDE.md is conventions (“every route is registered here”). Skills are workflows (“here is how you add a new route in this repo, end to end”). The two overlap, but the framing keeps me honest: rules in CLAUDE.md, recipes in skills.

The piece of the skills system most teams miss is the path scope. A skill can declare which directories it activates in. A create-api-endpoint skill that only loads when the agent is editing under services/api/ is invisible the rest of the time. With dozens of skills in a real repo, scoping is the difference between a useful library and a wall of irrelevant prompts.

The mental model: progressive disclosure for expertise. Most knowledge in a large codebase is local. Load it locally.

Symbol-level search through LSP and MCP

grep is fine until it isn’t. Past six-digit line counts, plain string search gets slow, returns too much, and burns tokens reading files the agent did not need to open. You also lose what every IDE has done for decades: jump-to-definition, find-references, hover-for-types.

You can give the agent the same navigation. Run a language server locally, wrap it in a small MCP server, expose two or three tools: where_is, find_references, goto_definition. The agent now searches by symbol, not by string. A request like “find every place monthly_total_cents is referenced” returns one definition and the actual references, instead of fifty grep hits that mention the substring in unrelated comments.

This is also where bigger orgs invest. Custom MCP servers that expose internal search systems, the code-ownership graph, the design-doc index. The patterns are the same; the targets are domain-specific. The point is that the agent does not have to brute-force its way through your repo when you already have better tools for finding things.

Image: Anthropic, How Claude Code Works in Large Codebases.

Subagents for exploration

The rule I follow: split exploration from editing. A subagent runs in its own context window. You ask which files implement the billing webhook flow, or what the user model looks like across services. It does the digging, and only the summary comes back to your primary session.

The win is context budget, not parallelism. Exploration is wasteful by nature. The agent reads forty files to find the three that matter, and most of those forty get thrown away. If that happens in your primary session, your editing turns start with a context window already half full of noise. If it happens in a subagent, the noise stays there. You get the answer.

Use the built-in Explore subagent liberally. Custom subagents earn their place when you have a workflow specific enough that a generic explorer is the wrong tool. The file shape is small: a single markdown file under .claude/agents/, a short frontmatter block, and a prompt body. name, description, tools, and model are enough to start:

---
name: explorer
description: Read-only repo explorer. Map a service or package without burning the main session's context, then return findings.
tools: Read, Grep, Glob
model: sonnet
---

You are a read-only explorer. The parent agent will hand you one service or
package to map. Read its `CLAUDE.md` if there is one, then trace entry points,
the public surface, and dependencies. Return findings as your final response.
No edits.

Restricting tools to read-only is the load-bearing line. The model only sees the tools you expose, so an explorer subagent without Write or Edit has nothing to call when it gets tempted, even if the prompt body forgot to say so. Treat that as a strong default. If you need a hard guarantee, layer a PreToolUse hook on top.

Don’t let it rot

The harness is not a one-time setup. Models improve, and rules written for last year’s model often constrain this year’s. A note like “always split refactors into single-file changes” might have saved you in 2024 and might block a beneficial cross-file edit in 2026. Anthropic suggests reviewing your CLAUDE.md files every three to six months, or whenever performance feels like it has plateaued after a major model release. The stop-hook reflection gives you a head start. The rest is on you.

Assign an owner

The last piece is not technical. The teams that get value out of Claude Code at scale have someone who owns the harness. A small platform-engineering team, or one DRI, or a hybrid PM/engineer doing it half-time. Their job is the same shape as owning a CI pipeline: write the conventions, build the skills, run the LSP wrapper, version the hooks, evangelize what works, retire what does not.

Plugins are the distribution vehicle. A good harness that lives in one engineer’s dotfiles stays tribal. The same harness packaged as a plugin (or a private marketplace) is how a team of five hundred ends up running the same skills, the same MCP servers, and the same hooks without anyone having to remember to copy a config.

The pattern that fails: ship Claude Code to the org on a Friday, hope adoption goes viral, watch every team grow its own slightly different version of CLAUDE.md for six months. The pattern that works: a quiet build-out period, a small set of approved skills, a working plugin or two, a documented governance story, then broad access.

Treat the harness like infrastructure.

Where to start

The order that has worked for me, in any repo:

Trim the root CLAUDE.md until it fits on one screen. Move the rest into subdirectories.
Add a Stop hook that proposes updates to those CLAUDE.md files in headless mode.
Convert your three most common repeated tasks into path-scoped skills.
Run a language server behind an MCP server. Stop searching by string.
Get comfortable dispatching exploration to subagents.

Most teams will plateau on step one for a week and find the agent is already noticeably sharper. The rest compounds. I have written more on the agent-tooling shift this is part of in How Building AI Agents Has Changed in 2026, and on the workflow side in The Claude Skills I Actually Use for DevOps and Superpowers, GSD, and GSTACK.

The model will keep getting better. The harness is the work.

Best AI Infrastructure Tools in 2026

Alex Leventer — Mon, 25 May 2026 00:00:00 +0000

“AI infrastructure tools” covers two distinct markets: infrastructure for AI (GPU clouds like CoreWeave, MLOps platforms like Weights & Biases) and AI for infrastructure (agentic platforms like Pulumi Neo that generate, deploy, and govern cloud resources for you). Most teams need tools from both categories, and picking the wrong one wastes budget and adoption goodwill.

The pressure to get this right is real. McKinsey research puts the productivity lift from generative AI in software development at 20–45%, which is great for application teams and a problem for platform teams trying to keep up with the resulting feature flow. Infrastructure investment is climbing on both fronts: more spend on the compute that trains and serves models, more spend on AI tools that manage everything else.

This guide covers both categories: the compute and MLOps stack in Part 1, and AI-powered infrastructure management in Part 2, where the more interesting product shift is happening.

AI infrastructure tools overview

Tools for building AI infrastructure

CoreWeave: GPU cloud built for AI workloads
Lambda Labs: straightforward GPU cloud for research and startups
Modal: serverless GPU compute
Weights & Biases: ML experiment tracking and model management
MLflow: open-source ML lifecycle platform
Hyperscaler AI platforms: AWS SageMaker, Google Vertex AI, Azure ML

AI-powered infrastructure management tools

Pulumi Neo: agentic AI with policy automation
Firefly AIaC: asset codification and IaC generation
env0 Cloud Compass: multi-IaC insights and analysis
Spacelift AI: run explanation and troubleshooting
Crossplane with Upbound: Kubernetes-native infrastructure
General-purpose code assistants: Copilot, Claude Code, Cursor, Gemini
AWS Application Composer: visual serverless builder

Quick picks

If you only have two minutes:

Enterprise compliance: Pulumi Neo. Executes changes (not only suggestions), ships with policy packs for CIS, HITRUST, NIST, and PCI DSS, and works with Terraform, CloudFormation, and resources created by hand.
Serious GPU compute: CoreWeave. Purpose-built for AI workloads, deep NVIDIA partnership, and prices that generally undercut the hyperscalers.
Best developer experience for ML: Modal. Decorate a Python function, get a GPU, pay by the second.
Open-source MLOps: MLflow. No vendor lock-in, runs anywhere, plays well with everything.

What is AI infrastructure?

The term covers two distinct categories that share almost no vendors.

Infrastructure for AI is the compute, storage, and orchestration that AI workloads run on. Training a large model is not a normal cloud workload: it wants thousands of GPUs talking to each other over fat, low-latency networks for weeks at a time. Inference is different again: lower latency, smarter batching, different hardware. General-purpose cloud was not designed for either case, which is why specialized GPU clouds and MLOps platforms exist.

AI-powered infrastructure management is the inverse: AI tools that manage cloud infrastructure. They generate infrastructure as code, run deployments, detect drift, and remediate policy violations. The pitch is that modern infrastructure (multi-cloud, containers, microservices, regulated workloads) has gotten too complex for humans to manage by hand and too varied for scripted automation to keep up with.

Most organizations end up needing both: somewhere to run their ML workloads, and something to keep the rest of the cloud sane.

Part 1: Tools for building AI infrastructure

These are the platforms you run AI and ML workloads on: GPU clouds for raw compute, MLOps platforms for the lifecycle around them.

CoreWeave

CoreWeave is the GPU cloud that broke out of the AI hype cycle into a real public company. They went public in 2025, signed a multi-billion-dollar capacity deal with OpenAI, and acquired Weights & Biases. Their thesis from day one was that AI workloads deserve infrastructure designed for AI workloads, not a GPU SKU bolted onto a general-purpose cloud.

License: Proprietary
Best for: Large-scale training and high-throughput inference; teams that need dedicated GPU capacity with first access to new NVIDIA hardware
Strengths: GPU infrastructure designed for AI; Kubernetes-native; direct NVIDIA partnership; handles distributed training at scale
Watch out for: Smaller global footprint than AWS/GCP/Azure; not a general-purpose cloud, so if you need RDS, S3, and a managed Kafka in the same provider, this isn’t it

Lambda Labs

Lambda has been the approachable GPU cloud for a long time. Environments come pre-configured with PyTorch and TensorFlow, and you can be running on an H100 in about as long as it takes to copy your SSH key.

License: Proprietary
Best for: Research teams, startups, and individual practitioners who want GPUs without a configuration tax
Strengths: Straightforward to start on; pre-configured deep learning environments; competitive on-demand pricing; strong learning resources
Watch out for: Smaller scale than CoreWeave or the hyperscalers; availability gets tight during demand spikes

Modal’s pitch is that you write a Python function, decorate it, and Modal handles the GPU. No capacity planning, no idle instances burning money overnight, no Dockerfile to maintain.

License: Proprietary
Best for: Variable ML workloads where reserved capacity would sit idle; data scientists who’d rather not learn Kubernetes
Strengths: Strong developer experience; serverless GPUs with automatic scaling; pay-per-second pricing; cold starts are fast for what they are
Watch out for: You give up infrastructure control. Not ideal for long training jobs that need reserved hardware or strict configuration requirements.

Weights & Biases

Weights & Biases is the de facto standard for ML experiment tracking and model management, integrated with essentially every framework and cloud you’d plausibly use. CoreWeave acquired the company in 2025, which has accelerated the joint roadmap but raised some neutrality questions for teams that prefer their tooling cloud-agnostic.

License: Proprietary with a free tier
Best for: ML teams that need shared experiment tracking, model versioning, and reporting
Strengths: Industry-leading experiment tracking and visualization; comprehensive model registry; strong team collaboration; broad integration surface
Watch out for: Costs scale quickly past the free tier; some teams self-host alternatives for data residency reasons

MLflow

MLflow is the leading open-source MLOps platform: experiment tracking, packaging, registry, and serving, with no lock-in. Originally built at Databricks, it’s now a broad open-source ecosystem with managed offerings from multiple vendors (including Databricks and the major clouds).

License: Apache 2.0
Best for: Teams that want MLOps without a vendor; or want the option to start managed and self-host later
Strengths: Open source; covers the full ML lifecycle; runs locally, on-prem, or managed; broad framework support
Watch out for: Self-hosting carries the usual operational tax; commercial alternatives have stronger collaboration UX out of the box

Hyperscaler AI platforms

The major clouds all sell end-to-end ML platforms. Each leads on the dimensions that line up with its parent cloud (Vertex for Google’s models and TPUs, SageMaker for AWS-native data pipelines, Azure ML for Microsoft-stack integration), but the wider integration with the rest of the cloud is the deciding factor.

AWS SageMaker: end-to-end ML on AWS, deeply integrated with S3 and Glue, with first-class connections to Lambda for serverless inference and to the rest of the AWS data stack. The default pick if your data already lives in AWS.
Google Vertex AI: Google’s ML stack, including TPUs for workloads that need them, plus access to Google’s foundation models. Strongest when paired with BigQuery.
Azure Machine Learning: the natural choice when the rest of your stack is Microsoft; first-party MLOps integrations across GitHub Actions, Azure DevOps, and Microsoft Fabric for downstream reporting. The right choice if you’re already an Azure shop with Microsoft compliance requirements.

The shared tradeoff: hyperscaler GPU compute typically runs 2–3x the per-hour price of specialized providers, and the platforms work best when you commit to them top to bottom. For organizations already inside one cloud, the unified billing and single support contract usually justifies the premium. For a new ML team starting from scratch, it rarely does.

Part 2: AI-powered infrastructure management tools

This is where the more interesting product shift is happening. Instead of running AI on infrastructure, these tools point AI at your infrastructure and let it do work. They’re a newer, AI-native layer on top of the broader infrastructure as code tooling landscape.

From code generation to agentic execution

Before the tool list, one distinction matters more than any feature comparison: whether the tool generates code or executes changes.

Code generation tools like GitHub Copilot suggest infrastructure code based on context. You review it, maybe edit it, run it yourself. The AI helps, but you’re still the one doing the work.

Agentic platforms generate the code and run it, with the guardrails you define. They understand your environment, handle multi-step workflows, and enforce policies on the way through. You describe the outcome; the platform makes it happen.

Capability	Code generation	Agentic execution
Generates infrastructure code	Yes	Yes
Understands infrastructure context	Limited	Deep
Executes changes	No	Yes
Handles multi-step workflows	No	Yes
Enforces policies automatically	No	Yes
Remediates drift and violations	No	Yes

Where you want to land on this spectrum is mostly a governance question, not a productivity one.

Pulumi Neo

Pulumi Neo is Pulumi’s agentic AI for infrastructure. The distinguishing claim is execution: Neo doesn’t only suggest a Terraform snippet, it figures out the right resources, generates the code, and runs the deployment inside whatever guardrails you’ve set.

License: Proprietary (Pulumi Cloud)
Best for: Platform engineering teams that want AI automation with real policy controls, especially in regulated industries

A few things that set it apart in practice:

Policy automation and compliance. Neo is integrated with Pulumi Insights and Governance, which ships pre-built policy packs for CIS benchmarks, HITRUST CSF, NIST SP 800-53, and PCI DSS. Detection and remediation run in the same loop: Neo finds a violation, generates a fix, and (subject to approvals) applies it. You can batch-remediate across stacks and accounts with prompts like “find and fix all unencrypted S3 buckets across our AWS accounts.”

Works with infrastructure you didn’t create with Pulumi. Neo’s governance applies to Pulumi-managed resources, Terraform state, CloudFormation stacks, and resources someone clicked together in the AWS console. That matters because the realistic adoption path is to point Neo at what you have, audit it, and gradually bring it under management, not to migrate everything first.

Progressive autonomy. Trust levels are configurable. Start with human approval for everything; loosen it for well-defined, low-risk operations as confidence builds; keep production and sensitive resources behind strict approvals. This is the part that tends to determine whether enterprises actually deploy agentic AI in anger, versus letting it sit as a sandbox toy.

IDE and CI/CD integration. The Pulumi MCP Server brings Neo into Cursor, Claude Code, Claude Desktop, Windsurf, and any other MCP-compatible client. The Pulumi Cloud UI is the home base for approvals, history, and remediation status. Neo also slots into CI/CD pipelines for pre-merge policy remediation.

Case studies:

Werner Enterprises reduced infrastructure provisioning time from 3 days to 4 hours using Pulumi.
Spear AI cut their Authority to Operate (ATO) timeline from an expected 1.5 years to roughly 3 months by using policy-as-code to evidence compliance controls for auditors.

Tradeoff to be honest about: Neo gets more valuable the deeper you are in the Pulumi ecosystem. If you’re running IaC, ESC, and policy packs already, Neo has a lot of context to draw on. If you’re kicking the tires, it’s still useful, but the differentiating capability (context-aware, policy-respecting agentic execution) is harder to feel.

Firefly AIaC

Firefly is an asset management platform with AI features bolted on top of a strong core. The core capability is asset codification: it discovers cloud resources you already have and generates the IaC for them.

License: Proprietary
Best for: Teams that need to codify existing cloud footprints or generate IaC from natural language

Strengths: solid asset discovery, multi-cloud coverage, natural-language IaC generation, drift detection with remediation hooks. Caveat: AI features here are supplementary to the asset management product, not the main event, and Firefly is less focused on agentic execution than on inventory and policy.

env0 Cloud Compass

env0’s Cloud Compass adds AI to env0’s IaC automation platform, focusing on analysis rather than autonomous execution.

License: Proprietary
Best for: Multi-IaC shops that want AI-generated PR summaries, drift explanations, and cost insights

Strengths: multi-tool support across Terraform, OpenTofu, Pulumi, and Terragrunt; AI-generated PR summaries; drift cause analysis; cost estimation. Caveat: this is analysis and explanation, not action: Cloud Compass complements an agentic tool rather than replacing one.

Spacelift AI

Spacelift’s AI work is focused on the post-run experience: explaining what happened in a deployment and helping troubleshoot failures.

License: Proprietary
Best for: GitOps shops that want AI assistance reading complex runs and diagnosing failed deployments

Strengths: AI-powered run explanation; troubleshooting guidance for failures; broad IaC tool support; mature CI/CD integration. Caveat: like Spacelift as a whole, this is observation and explanation, not generation or execution. Pair with something that writes the code.

Crossplane with Upbound

Crossplane brings Kubernetes-style declarative management to cloud resources. Upbound is the company that commercializes it, and is layering AI-native control-plane capabilities into the 2.0 generation.

License: Apache 2.0 (Crossplane); proprietary (Upbound)
Best for: Teams already deep in Kubernetes that want to manage cloud resources the same way

Strengths: Kubernetes-native model; native GitOps fit; very active OSS community; AI control-plane work emerging from Upbound. Caveat: the learning curve is real if you’re not already living in Kubernetes; the commercial AI features are still maturing.

General-purpose code assistants

General-purpose AI coding assistants are the tools your developers already have open: GitHub Copilot, Claude Code, Cursor, and Google’s Gemini and Antigravity. They write Terraform HCL, Pulumi programs, and CloudFormation templates competently, about as well as they write anything else.

License: Proprietary (subscription), varies by tool
Best for: Developers who want broad code assistance, including infrastructure code, inside their existing editor

Strengths: excellent line-by-line code completion; broad language support; first-class editor integration; trained on huge corpora. Caveat: no infrastructure context. They don’t know what’s in your account, what your policies are, or which subnet you should pick. Treat their IaC suggestions as first-pass scaffolding, not production output.

AWS Application Composer

Application Composer is AWS’s visual builder for serverless applications. Drag services onto a canvas, get a CloudFormation template out, with AI suggestions for service configuration along the way.

License: Proprietary (AWS, included)
Best for: Teams building AWS serverless apps who prefer a visual workflow

Strengths: visual development for serverless; direct AWS integration; AI suggestions for service configuration; emits CloudFormation. Caveat: AWS-only, CloudFormation-only, and best suited to serverless rather than general infrastructure.

Comparison tables

Infrastructure for AI

Tool	Category	Key strength	Limitation	Pricing	Best for
CoreWeave	GPU cloud	Purpose-built GPU infra, NVIDIA partnership	Not a general-purpose cloud	Per-GPU-hour	Large-scale AI training
Lambda Labs	GPU cloud	Approachable, pre-configured environments	Smaller scale	Per-GPU-hour	Research teams, startups
Modal	Serverless GPU	Developer experience, pay-per-second	Less infrastructure control	Pay-per-use	Variable ML workloads
Weights & Biases	MLOps	Industry-standard experiment tracking	Costs scale quickly	Free tier + paid	ML team collaboration
MLflow	MLOps	Open source, no lock-in	Self-hosting overhead	Free (self-hosted)	Flexible ML lifecycle
AWS SageMaker	Hyperscaler	AWS ecosystem integration	Higher cost, lock-in	Per-use	AWS-native orgs
Google Vertex AI	Hyperscaler	Google models, TPU access	Lock-in	Per-use	Google Cloud users
Azure ML	Hyperscaler	Microsoft integration, enterprise features	Lock-in	Per-use	Microsoft ecosystem

AI-powered infrastructure management

Tool	Approach	Key strength	Limitation	Pricing	Best for
Pulumi Neo	Agentic AI	Execution + policy automation	Best within Pulumi ecosystem	Pulumi Cloud tiers	Enterprise platform teams
Firefly AIaC	Asset management	Asset codification, IaC generation	AI is supplementary	Proprietary	Codifying existing infra
env0 Cloud Compass	Multi-IaC platform	Multi-tool support, PR analysis	Analysis, not execution	Proprietary	Multi-IaC environments
Spacelift AI	CI/CD platform	Run explanation, troubleshooting	Observation, not action	Proprietary	GitOps workflows
Crossplane / Upbound	Kubernetes-native	K8s patterns for infra	Requires K8s expertise	Open source + commercial	Kubernetes-native teams
Code assistants	Code assistant	Broad language support, IDE	No infrastructure context	Subscription	General code assistance
AWS Composer	Visual builder	Visual serverless development	AWS- and CFN-only	Included with AWS	AWS serverless apps

How to choose

There’s no universal best tool. Five questions sort the field quickly:

Cloud strategy. Multi-cloud means tools like Pulumi Neo, Firefly, env0, or Crossplane. Single-cloud commitment means hyperscaler-native tools may integrate more deeply (AWS Composer, SageMaker, and so on).
Team expertise. Programmers gravitate to tools that use real languages (Pulumi Neo, Pulumi IaC). Kubernetes teams find Crossplane natural; everyone else finds it steep. Teams that prefer visual workflows should look at AWS Composer or env0’s UI.
Compliance. Regulated industries (healthcare, finance, government) get the most value from tools with pre-built compliance packs and audit trails. Pulumi Neo’s CIS/HITRUST/NIST/PCI packs are the most direct fit. If preventative policy enforcement matters, prefer tools that block non-compliant deployments rather than flag them after the fact.
Existing footprint. Greenfield projects can use anything. Brownfield is where it gets interesting: Pulumi Neo works against Terraform, CloudFormation, and manually-created resources, which lets you adopt incrementally instead of migrating first. Mixed-IaC shops should also look at env0.
Budget. Open source first: MLflow for MLOps, Crossplane for Kubernetes-native infra. Open source is not free, though: self-hosting carries a real total cost of ownership in hosting, maintenance, and the expertise to operate it. Commercial tools (Pulumi Cloud, env0, Spacelift) fold that operational cost into the price, on top of support, SLAs, and the enterprise-tier features open source can lack.

Before adopting anything, get visibility into what you have today, pilot on staging where mistakes are cheap, and define success metrics up front: time to provision, policy violation rates, mean time to remediate. The best AI infrastructure tool is the one your team will actually use, which means meeting developers where they already work.

Key trends and outlook

From copilots to agents. “AI suggests code” and “AI runs the deploy” are different products with different governance implications. The teams getting value from agentic tools have figured out which tasks to delegate fully, which to keep human-in-the-loop, and which to leave alone.

Progressive autonomy. Enterprise adoption follows a predictable shape: visibility → recommendations → human-approved execution → autonomous execution for well-understood scenarios. Tools that support that graduation will see stronger enterprise traction than tools that force an all-or-nothing choice.

Policy as the control plane. As AI takes on more infrastructure tasks, policy frameworks become the primary control plane. Done well, policy becomes an enabler (guardrails that let you safely expand automation) rather than a brake on velocity.

MCP standardization. The Model Context Protocol is becoming the integration standard between AI assistants and infrastructure tools. The practical upshot is that the IDE is increasingly a viable surface for managing infrastructure, with AI mediating between natural language and the underlying APIs.

Consolidation. CoreWeave acquiring Weights & Biases and NVIDIA acquiring Run:ai both point toward integrated platforms across the AI infrastructure stack. For tool selection today, that’s an argument for picking vendors with clear strategic direction over point solutions likely to be acquired or out-competed.

Frequently asked questions

What is the best AI agent for cloud infrastructure management?

For enterprise governance plus true agentic capability, Pulumi Neo is currently the most complete offering: it executes changes (not just suggests them), integrates with pre-built compliance frameworks, and works with infrastructure regardless of how it was provisioned. For Kubernetes-native shops, Crossplane with Upbound’s emerging AI features is worth tracking.

How can I use generative AI to manage cloud infrastructure?

Start by identifying the repetitive, time-consuming infrastructure work in your team. The highest-value early use cases tend to be:

Code generation: write IaC from natural-language descriptions, then review.
Documentation: explain unfamiliar configurations and reduce onboarding time.
Troubleshooting: analyze logs, errors, and configs to suggest likely causes.
Security and compliance: scan for violations and generate fixes.
Full automation: for shops that want it, agentic platforms like Pulumi Neo execute provisioning workflows end-to-end with governance controls intact.

What is agentic AI for infrastructure?

Agentic AI for infrastructure means AI systems that autonomously execute infrastructure tasks, not just generate code suggestions. The difference from a code assistant is action: an agent understands your environment, respects your policies, and performs multi-step work (provisioning, configuration, security controls) within the boundaries you’ve defined.

How do AI agents improve DevOps workflows?

By automating the repetitive parts (provisioning, drift remediation, policy enforcement), reducing context-switching, and catching issues earlier. Teams that have rolled out agentic tools well report faster provisioning, fewer policy violations slipping into production, and quicker compliance remediation. The compounding effect (engineers freed for higher-value work as the agent absorbs the routine) is the actual point.

What’s the difference between AI code generation and agentic execution?

Code generation suggests IaC for a human to review and run. Agentic execution generates the code and runs it, with policy and governance enforced along the way. It’s the difference between a knowledgeable colleague who suggests an approach and a knowledgeable colleague who also ships the change with appropriate oversight.

Can AI generate Terraform or Pulumi programs?

Yes. Most general-purpose AI assistants (Copilot, Claude, Gemini, ChatGPT, Cursor) can produce Terraform HCL, Pulumi programs in TypeScript / Python / Go, and CloudFormation. Quality varies. Generic assistants lack environment context and will happily emit syntactically correct but operationally wrong code. Infrastructure-specific tools like Pulumi Neo generate code that’s aware of your existing resources, policies, and provider constraints.

Can AI help with infrastructure compliance and policy automation?

Yes, and this is one of the highest-leverage uses of AI in infrastructure. Tools like Pulumi Neo detect policy violations across your footprint (including resources created outside IaC), generate compliant remediation, and apply it with the approvals you require. Pre-built frameworks for CIS, HITRUST, NIST, and PCI DSS shorten what would otherwise be a long manual compliance project.

Are AI infrastructure tools secure for enterprise use?

Enterprise-grade ones are. Look for RBAC, full audit logging of AI actions, preventative policy enforcement (not just detection), and human-in-the-loop approvals for sensitive operations. SOC 2, data residency options, and configurable autonomy levels are table stakes. The risk to avoid is wiring a consumer AI assistant directly into a production cloud account without those controls.

How do I choose between different AI infrastructure tools?

Match the tool to your context: existing clouds and IaC, team skills, compliance requirements, budget. Enterprise platform teams with governance needs should evaluate Pulumi Neo first. MLOps-focused teams should look at Weights & Biases or MLflow. For general code assistance inside the editor, a general-purpose assistant like Copilot, Cursor, or Gemini is the default. Most organizations end up with more than one: a code assistant for daily development and an agentic platform for production infrastructure.

What are the best tools for machine learning infrastructure?

For GPU compute, CoreWeave leads at scale, Modal wins for variable workloads and developer experience, and the hyperscalers are the default pick if you’re already inside one of them. For experiment tracking and model management, Weights & Biases is the leading commercial platform; MLflow is the leading open-source one. Most teams pick on the deploy model and pricing fit rather than capability gap. For the cloud infrastructure underneath the ML workloads, the same infrastructure management story applies: Pulumi Neo can provision and govern ML infrastructure the same way it handles everything else.

Conclusion

Two categories, two problems. GPU clouds and MLOps platforms (CoreWeave, Lambda, Modal, hyperscaler trio, W&B, MLflow) solve the compute and lifecycle problem for running AI workloads. AI-powered infrastructure tools (Neo, Firefly, env0, Spacelift, Crossplane, code assistants, Composer) solve the management problem for everything else.

For GPU workloads, the choice mostly comes down to scale and where you already are. For infrastructure management, the real question is how much you actually want AI to do. Code assistants help you write IaC faster, but you’re still running it. Agentic platforms like Pulumi Neo execute changes and enforce policy on the way through, with the guardrails you control.

The pattern from teams getting real value: treat AI as a force multiplier on routine work (provisioning, drift, compliance) and keep human judgment in the loop for the architecture and the edge cases.

If you want to see agentic infrastructure management running against real resources, start with Pulumi Neo.

Bringing Neo to GitHub and Slack

Pulumi Neo Team — Thu, 21 May 2026 08:00:00 -0700

This week, Pulumi Neo started working in two more places: GitHub and Slack. The agent that already runs Pulumi tasks from the Cloud console and the terminal now participates in the threads where your team discusses changes.

Mention @pulumi-neo in a pull request or issue and Neo replies in the thread. Mention @Neo in a Slack channel and Neo starts a task, continuing the conversation as you reply.

Neo in GitHub

Mention @pulumi-neo in a pull request description, a top-level or inline review comment, or an issue. Neo sees the diff, the stacks linked to the repository, and their current state. Reviewers can ask Neo to walk through what a proposed change does, including resources that change in stacks the PR doesn’t touch directly. Responses land in the same thread, so the analysis becomes part of the review record and any follow-up stays with it.

Neo in Slack

Mention @Neo in any channel where Neo has been added, and Neo starts a task in the thread. The reply lands in the same thread, and follow-up messages continue the conversation there. The rest of the channel can see what was asked and what Neo found. Neo has the same capabilities here as in the Pulumi Cloud console or the terminal: check stack state, investigate failures, walk through what a change will do, or carry out actions the team has approved.

Integrations in action

A teammate posts in #platform-engineering: “API latency p95 has been climbing for two days, nobody can figure out why.” You reply:

You: @Neo check the production API stack. Anything change in the last 72 hours?

Neo starts a task in the thread, walks the stack history, and finds a configuration change to the load balancer’s idle-timeout setting that landed Friday afternoon. It posts the change, who deployed it, and when. The rest of the channel sees the finding without you having to retell it.

You: @Neo open a PR to revert idle-timeout to the previous value.

Neo edits the stack’s Pulumi program, runs pulumi preview to confirm the change touches only the load balancer, and opens a pull request with the diff and the preview output. A reviewer pulls it up:

Reviewer: @pulumi-neo what else does this change affect downstream?

Neo replies in the same review thread with the resources that change: the listener config and the target group health check. The reviewer reads, approves, and the change ships.

The investigation moved from Slack to GitHub, and both threads keep the record.

Permissions and governance

Whether the conversation starts in GitHub or Slack, Neo runs with the RBAC permissions of your Pulumi Cloud user. Stack-level controls, organization-level guardrails, and audit logging apply the same way they do for a task started from the console. Starting a conversation in a new place doesn’t grant Neo new permissions; it just changes where the conversation happens.

Try it out

Both integrations are available now for Neo-enabled organizations. The Code Reviews docs and Slack integration docs cover the one-time setup. From there, every engineer with a linked Pulumi Cloud identity can mention Neo from the threads they already work in.

Today’s launch is part of a bigger story. Read our launch-day piece on the agentic infrastructure era for the broader vision, the Neo CLI launch post for Neo’s new home in the terminal, and the Neo Integrations post for the MCP servers and cloud CLIs that ship with this release.

As always, we’d love to hear what you think — and if you have any suggestions for places we should put Neo next, file an issue in pulumi-cloud-requests.

Neo Automations: Scheduled Tasks Shipped as Pull Requests

Pulumi Neo Team — Thu, 21 May 2026 07:00:00 -0700

Recurring platform work slips: provider versions fall behind, drift accumulates between checks, and the quarterly audit keeps getting pushed back another month. Pulumi Neo can now run any task on a cadence you set, opening a pull request for each run.

Automations in action

Your platform team runs stacks across staging and production, and the AWS, GCP, and Kubernetes providers keep shipping new versions. Nobody has time to bump them stack by stack.

You write one automation:

Every Monday at 8 AM, check the infra/ project for stacks where the AWS, GCP, or Kubernetes provider is more than two minor versions behind. For each one, bump the out-of-date provider, run pulumi preview, and open a PR if the preview is clean.

Monday morning, Neo runs the prompt. It finds three stacks behind on the AWS provider, edits each program, runs preview, and opens a PR for each clean run. You review the PRs like you would any other dependency bump, merge them, and Neo runs again next Monday.

What automations are for

The launch includes four built-in templates: a provider freshness check, an encryption audit, a backup audit, and an activity digest. You can also skip the templates and write your own prompt.

Pick from hourly, daily, weekdays, or weekly cadences. Each automation gets its own page in the Automations tab, where you can edit the prompt, change the schedule, run it once on demand, or pause it.

Safe by default

Automations default to two settings that fit recurring work. Approval mode is auto, so a run doesn’t wait for human confirmation between steps. Permission mode is read-only, so a run can read state and propose changes through pull requests but can’t apply changes directly. You can override either default per automation.

How automations fit with the rest of Neo

A scheduled task uses the same context as an interactive Neo task. Custom Instructions at the organization and project level apply, so a scheduled run respects the same naming conventions, tagging policies, and architecture rules your team has written down.

MCP integrations and CLI integrations work in scheduled tasks the same way they work in interactive ones, so a weekly drift check can query AWS through the aws CLI, file Linear issues, and link related PagerDuty incidents. Scheduled tasks also run with the RBAC permissions of the user who scheduled them, checked at run time; if permissions change between scheduling and execution, the new permissions apply.

Try it out

Open Neo in Pulumi Cloud, switch to the Automations tab, and pick a template or write your own prompt. The automations docs cover the form, scheduling options, and per-automation overrides.

Today’s launch is part of a bigger story. Read our launch-day piece on the agentic infrastructure era for the broader vision, and the Neo Integrations post for the third-party tools and CLIs your automations can use.

As always, we’d love to hear what you think — and if you have any suggestions for automations that’d make Neo even better, file an issue in pulumi-cloud-requests.

Seven Rules for Building an AI-Native Software Factory

Ewan Dawson — Thu, 21 May 2026 06:00:00 -0700

Ewan Dawson is CTO of Compostable AI, where five engineers run an AI-native software factory: nineteen clients, custom AWS deployments, most of them shipped within a day of contract signing. This article is adapted from his recent Pulumi webinar, and covers rules in more depth than we had time for on stage.

For the past twenty years, I’ve viewed software development as a craft. The best engineers drew on decades of experience to get every function right.

But two years into the agentic AI revolution, I realised software is going to look more like a factory than a craft. The economics have changed. We can’t treat code as bespoke anymore. To scale, we have to think industrial — use the tools to ship more value with fewer engineers.

I joined Compostable AI soon after it was founded 2.5 years ago, and I built the engineering org AI-native from day one. The technology has come a long way since then, and so has my understanding of what AI-native actually means. Here are seven rules I keep coming back to.

An AI software factory is a software operation where autonomous agents write and ship most of the code. The engineers stop writing it by hand and spend their time deciding what gets built and talking to customers. The rules below are our rules for building and running an AI-native software factory.

1. Transform, don’t enhance

Going AI-native isn’t an upgrade to your existing process. If you treat AI as a way to hand your developers smarter tools, you leave most of the value on the table. You get the leverage by rebuilding how you write software — and the culture and processes around it.

I know that’s a tall order for a large, mature engineering org. My advice: start small. Pick one team or one business area and run it as a fully AI-native function. Take what you learn and roll it out from there. And do the political work early, especially with your Governance, Risk, and Compliance function. Get GRC on your side early. Otherwise AI becomes a compliance fight instead of a structural advantage.

Don’t bolt AI onto your existing workflow. Redesign the workflow around what agents can do.

Most of the leverage in this technology comes from rebuilding around it. The tool change is the small part.

2. Remove the problem, don’t solve it

Going AI-native flips which problems are hard and which are easy. The right move often isn’t to engineer a solution. It’s to reframe the problem so it goes away.

Here’s an example. Serving multiple clients with agents writing the code, blast radius wasn’t a hypothetical. One bad agent run could trash a customer’s database, or leak one client’s data into another’s. Our instinct was to build a secure multi-tenant sandbox with guardrails, approvals, rollback. But every version we tried still had agents loose in a shared environment, one bug away from making one customer’s data visible to another’s. So we removed the problem: every client gets two dedicated AWS accounts, one for production and one “digital twin” staging account. Agents iterate on staging until the work checks out. Only then does it ship to production. We have nineteen accounts now, one per client.

Managing nineteen AWS accounts with five engineers used to be an administrative nightmare. When code is cheap, infrastructure-as-code tools like AWS Control Tower and Pulumi make it the easier path.

Remove the problem before you try to solve it.

It’s cheaper to reframe the problem than to engineer your way through it.

3. Pick tools your agents can drive

Removing problems is the process side. The other side is tooling. If you want an automated factory, your tech stack has to be something agents can drive. This overlaps a lot with tools that have great developer experience. If a tool has a robust API plus a clean CLI, agents can drive it. If it’s heavy click-ops around a web UI, agents stop there.

We didn’t get there first try. Our first IaC tool worked fine when we had a couple of clients. As we added more, accounts drifted, deployments slowed, retries got complicated. We needed something built for where we were heading.

I went looking, and Pulumi fit. We express infrastructure as type-safe code — TypeScript, in our case, rather than HCL — and agents are good at writing it. Pair that with Pulumi Neo — pre-loaded with domain-specific Pulumi skills — and we ship infrastructure that follows best practices. One of my colleagues put it: “The scary thing about Neo is it just seems to know everything about what we do.” Pulumi IaC plus Pulumi ESC for configuration beats stitching tools together. And TypeScript lets us build higher-level abstractions that keep the AWS account fleet tractable.

“I don’t actually care if it’s HCL or TypeScript, as long as my software development agents can write it. And they do a better job with TypeScript than HCL.”

Tools have to share your AI-native mindset. If they don’t integrate deeply, the human becomes the glue.

If part of your stack still requires a human to click through a web UI to provision an account, your agents stop there.

4. Don’t let one agent do everything

When I first started with agents, I reached for a god prompt: one massive system prompt meant to guide a single agent through the whole software lifecycle. It didn’t work. Agents struggle when you give them multiple goals. The writer is lenient on its own work — it won’t catch what it just shipped. You don’t want it reviewing the code, checking for security flaws, or hunting bugs.

We get better results from a constellation of specialized agents, each handling one part of the line. Pulumi Neo handles infrastructure. Alongside it sit agents specialized in:

Code implementation
Code review and testing
Security auditing
Internal standards compliance
Documentation updates

Tasks pass down the line. Clean code comes out the other end, with almost no human involved.

Don’t let any agent mark its own homework. Specialize by job.

Treat agents the way you’d treat a team. The one who writes the code shouldn’t be the one signing it off.

5. Measure human hours per unit of value

Once we had agents writing and agents reviewing, throughput went up — but the bottleneck moved past the PR. Engineering hours were still the most expensive thing in the building, so my core metric is human hours per unit of value produced. Minimize that.

That means hunting for every step that still goes through a person — especially the mid-pipeline steps between ideation and production. Automate the human touchpoints along that line, and the factory runs 24/7.

Pushing automation this hard also forces good engineering. A chaotic, undocumented process is impossible to automate. Good engineering is still good engineering, AI or not. Agents won’t fix a weak process.

Measure human hours per unit of value. Treat every one as a bottleneck to remove.

You can’t automate what you can’t describe. Every human in the pipeline marks a piece that hasn’t been described yet.

6. Design for convergence, not one-shot correctness

Even with the human touchpoints removed, the agents don’t ship right the first try. Once you embrace the factory pipeline, you stop needing them to. We design for convergence instead — a system that lands on the right answer through automated iteration.

The loop we run looks like this:

Refinement: agents iterate on the Product Requirements Document until the problem is clear.
Planning: agents draft multiple technical approaches, and evaluation agents pick the best one.
Implementation: coding agents write the software.
Review: specialized checking agents look for bugs, API misuse, and security flaws.

If the checkers find a problem, they hand it back to the implementation agent. The loop repeats until the tests pass and the agents agree on a clean PR. Once it converges, we merge and deploy to staging.

Two things have to be true. You need a way to evaluate the output. Without that, you don’t know when to stop. And the loop has to converge — each pass has to get closer. A checker that fails every PR for a different reason isn’t helping — it just keeps the work going in circles. The feedback has to narrow the search, not widen it.

Once it converges, the question moves on. How cheap can we make it? Lower the time to PR, reduce token count, drop the overall cost. The optimization never really ends.

Don’t aim for one-shot correctness. Design for convergence.

It doesn’t matter how many tries it takes, as long as the loop closes without a human in it. Get convergence first. The optimization comes after.

7. Run the factory in the cloud, not on a laptop

Even a converged factory has to live somewhere. Try running a fully automated factory on individual developers’ laptops, and it falls apart. Laptops are highly trusted machines. Put autonomous agents on them and your security posture drops, fast. And the factory has to run 24/7. Events come from elsewhere — PR comments, Slack threads, errors in test environments.

Cloud also kills configuration drift across a dozen developer machines. The same prompts run against different model versions, and env vars sit half-set on half the laptops. The thing you’re trying to optimize lives in different states across the team. Cloud isn’t just where the factory runs; it’s the only place a team can iterate on it together. Keep everything in one place — AWS, Pulumi Cloud, GitHub. The specific stack matters less than the principle of one place.

And the part that matters most: the factory keeps running, testing, and deploying long after we’ve closed our laptops and gone to sleep — the dark factory pattern, where the line keeps producing with the lights off.

Build the factory somewhere you can work on it — not just somewhere it can run.

A factory scattered across laptops can’t be improved as a system. Cloud keeps it in one shape, 24/7, and lets the team iterate together.

How to build an AI-native software factory

There’s no single blueprint for how to build a software factory, but these seven rules are the software factory model we run at Compostable AI: transform the workflow rather than bolt AI onto it, remove problems instead of engineering around them, pick tools your agents can drive, split work across specialized agents, measure human hours per unit of value, design for convergence over one-shot correctness, and run the factory in the cloud so it operates 24/7.

Closing thought

I’ve shipped more code in the last two years than I did in the fifteen before that. Most of it in languages I couldn’t write by hand. And that’s after a stretch in leadership where I wrote almost none.

If you’re where I was two years ago: don’t ask how AI fits into what you already do. The factory is built one rule at a time, and it’s not a template — it’s the practice of finding where you’re taking advantage of the new economics and where you’re not, where your practices still need an update. The leverage is in finding these places and improving them.

Frequently asked questions

What is a software factory?

A software factory runs software delivery as a repeatable industrial process instead of hand-crafting each feature, the way a physical factory turns out products on a line. The idea predates AI. An AI-native software factory is the version where agents do most of the building and the engineers steer.

How is a software factory different from traditional software development?

Traditional development treats code as a craft, each function written and reviewed by hand. A software factory treats software delivery as an industrial process: agents handle the writing and checking through an automated, converging loop, and the expensive human hours move to defining the work and deciding what to build. The goal is to minimize human hours per unit of value shipped.

How do you keep AI agents from breaking things across customers?

Rather than build a shared multi-tenant sandbox, Compostable AI removes the problem: every client gets two dedicated AWS accounts, one for production and a “digital twin” staging account. Agents iterate on staging until the work checks out, and only then does it ship to production. Infrastructure-as-code tools like AWS Control Tower and Pulumi make running that account fleet tractable for a small team.

What tools does an AI-native software factory need?

Tools your agents can actually drive, anything with a solid API and a clean CLI, rather than click-ops around a web UI. Compostable AI expresses infrastructure as type-safe TypeScript with Pulumi, pairs it with Pulumi Neo for domain-specific infrastructure skills, and uses Pulumi ESC for configuration. If part of your stack still requires a human to click through a UI, your agents stop there.

Watch the original Pulumi webinar. Read the Compostable AI case study, and learn more about Compostable AI and Pulumi Neo.

Neo, Now in the Terminal

Pulumi Neo Team — Wed, 20 May 2026 00:00:00 +0000

Since launching Pulumi Neo, over 4,500 organizations have used it to delegate real infrastructure work: scaffolding, migrating, investigating, operationalizing, and more. Though that usage has come entirely through Pulumi Cloud, we know a large portion of Pulumi users live in the terminal, and increasingly that’s where AI tools run too. Now we’re bringing Neo there.

pulumi neo brings the same Neo experience you’ve had in Pulumi Cloud to your terminal. Running locally means there’s no separate branch to push, no credentials to provision, and no context to paste: Neo picks up the setup you already have.

What local execution unlocks

Neo inherits your setup when it runs locally. The CLIs you’ve authenticated, the environment variables and kubeconfigs you’ve configured, and the project you’re editing right now are all available without any setup on your part. That means Neo can run the same commands you would, against the same systems you have access to.

That makes pulumi neo a fit for paired, interactive sessions where you and Neo work through a problem together. For asynchronous, autonomous tasks you set up and come back to, Pulumi Cloud Neo is still the surface to reach for. Both reach the same Neo.

You can also hand tasks to Neo from other agent sessions. Simply ask your agent, such as Claude Code or Codex, to hand the task off to Neo, and the Neo handoff skill packages the current thread (goal, repo pointers, conversation summary) and starts a Neo task using pulumi neo under the hood. This works anywhere skills are supported, without leaving your current session.

What carries over

Local tools and context are what’s new. The full set of controls you have in Pulumi Cloud Neo applies in the terminal: approval modes (manual, balanced, auto) for tool calls, permission modes (default, read-only) for what Neo can change, and Plan Mode for research and planning before execution.

Integrations carry over too. The integration catalog (connectors to Atlassian, Datadog, Linear, PagerDuty, and others) works the same way from the terminal. Identity, RBAC, and audit all run through your pulumi login, the same way they do in the console. See the Pulumi Neo docs for details.

Get started

pulumi neo ships with the latest Pulumi CLI. To start a session:

Authenticate to Pulumi Cloud with pulumi login.
Run pulumi neo, or pass an initial prompt: pulumi neo "what's in this stack?".

pulumi neo is part of a broader launch on agentic infrastructure. See the pulumi neo command reference and the Pulumi Neo docs for details. 10 things you can do with Neo is a good starting point for tasks to try. The Pulumi Community Slack is the place for questions and feedback.

Neo Integrations: MCP Servers and Cloud CLIs

Pulumi Neo Team — Wed, 20 May 2026 00:00:00 +0000

Pulumi Neo already understands your infrastructure: your code, your stacks, your state. Today we’re launching new capabilities that extend Neo’s reach in two directions: into the third-party systems your team uses to plan and observe, and out to the cloud CLIs that actually drive your infrastructure.

The first half is MCP integrations: connections to Atlassian, Datadog, Honeycomb, Linear, PagerDuty, and Supabase that show up as tools Neo can call during a task. The second half is CLI integrations: scopable access to aws, gcloud, az, and kubectl. Both are configured once at the org level and available to every Neo task in the organization.

Integrations in action

A PagerDuty alert just fired: RDS storage on payments-prod is at 90% and climbing. You want to know how fast, and whether you can buy yourself any runway before it fills.

You: Neo, RDS storage on payments-prod just paged at 90%. How fast is it growing, and what do we have configured?

Neo pulls the active incident from PagerDuty, decides on its own to check Datadog for the storage-utilization curve over the last 30 days, and runs aws rds describe-db-instances --db-instance-identifier payments-prod through your production-aws CLI integration (the name your org gave its production AWS credentials). The database has been growing about 5 GB a day. The instance has AllocatedStorage at 200 GB and MaxAllocatedStorage also at 200, so storage autoscaling is effectively disabled. At current growth, the disk fills in three days.

You: Bump max allocated storage to 500. Open a PR.

Neo edits the payments stack’s Pulumi program to raise maxAllocatedStorage from 200 to 500 on the RDS instance, runs pulumi preview to confirm the change is scoped to that one resource, and opens a pull request with the diff, the preview output, and links to the PagerDuty incident and the Datadog graph. You review the PR and merge it. Pulumi applies the change, and Neo posts the resolution back to PagerDuty.

With three integrations and one conversation, the change is reviewed, shipped, and the alert resolved a few minutes later.

MCP integrations: context from your existing tools

The launch catalog covers six services that show up most often in infrastructure investigations: Atlassian for Jira issues and Confluence runbooks, Datadog for metrics and logs, Honeycomb for traces, Linear for issue tracking, PagerDuty for incidents and on-call schedules, and Supabase for managed database changes. Each connects Neo to a remote MCP server hosted by the provider, so the agent has access to the full set of tools the vendor chooses to expose.

Integrations can be enabled by organization administrators on the Neo Settings page. Once configured, they’re available to every Neo task in your organization.

CLI integrations: live cloud insights

CLI integrations cover what MCP doesn’t reach: live cloud insights. With AWS, GCP, Azure, or Kubernetes connected, Neo can check live database utilization, look up the current state of a running service, verify a service quota before scaling, or reach into resources that aren’t managed by any Pulumi stack.

An admin enables a CLI integration the same way as an MCP one, from your org’s Neo settings. Each integration gets a name your team chooses, like production-aws or staging-gcloud, and tasks reference that name to tell Neo which environment to reach into. You can connect multiple instances of the same CLI (for example, production-aws and staging-aws) so Neo can investigate staging without touching production. Credentials are backed by Pulumi ESC environments your org owns; the CLI integrations docs walk through setup.

Per-task control and failure handling

Both surfaces default to org-wide availability, with per-task overrides. Before starting a task, you can toggle individual MCP integrations off. The toggles only affect that task; the org-level configuration is unchanged.

Failures behave the same way for both. If an integration can’t be reached, Neo logs a warning, skips it, and continues with the rest. A single broken integration doesn’t stop a task. CLI integration connect and disconnect events go to your organization’s audit log, and Neo’s individual CLI calls appear in the task transcript alongside its other tool calls.

Try it out

Both MCP and CLI integrations are available now for Neo-enabled organizations. Open your org’s Neo settings, connect the MCP server or CLI of your choice, and let Neo do the next investigation against the tools you already use. The MCP integrations docs and CLI integrations docs walk through credential setup for each one, and the Neo integrations hub ties it all together.

Today’s launch is part of a bigger story. Read our launch-day piece on the agentic infrastructure era for the broader vision, and the Neo CLI launch post for Neo’s new home in the terminal.

As always, we’d love to hear what you think — and if you have any suggestions for integrations that’d make Neo even better, file an issue in pulumi-cloud-requests.

Ten More Things You Can Do With Pulumi Neo

Adam Gordon Bell — Tue, 19 May 2026 05:00:00 -0700

Last fall, after launching Pulumi Neo, we wrote up 10 things you could do with it. In the months that followed, as platform teams handed Neo more real work, we watched and listened, shipping a steady stream of features like plan mode, read-only mode, AGENTS.md, an integration catalog, cross-cloud migration, and task sharing. With today’s release, Neo extends beyond the Pulumi Cloud console into the Pulumi CLI, GitHub, and Slack.

So here are 10 more things you can do with Neo.

1. Deploy your app to AWS without writing IaC

Hand Neo a repo. Neo picks the right services — ECS, AWS Fargate, ALB — writes the Pulumi, and opens a PR.

The cloud infrastructure part of getting a new service running, especially one in a new language, is always a few hours of boilerplate: a VPC and subnets, an IAM role, security groups, a load balancer, DNS, and a TLS cert.

With Neo, that work collapses into a prompt. Point Neo at a repo and ask:

Deploy this app to AWS as a publicly accessible service.

Plan mode comes back with the resources Neo will create, named and sized: ECS running on AWS Fargate, an ALB, and the VPC wiring. Approve, and Neo writes the Pulumi program, runs a preview, and opens a PR. You, the human in the loop, merge it after review.

Neo planning a PR and deploying an app to AWS.

Start a Neo task Ask Neo to deploy your app to AWS and make a PR

2. Diagnose a slow API from metrics, logs, and code

Slow endpoints live at the seam between runtime metrics and the stack that runs them. Neo reads both and proposes a fix with the metric evidence as the rationale.

Production incidents often involve multiple tools. When the checkout endpoint’s p95 latency climbs from 200ms to 1.2s, the metric is in Datadog, but the cause might be somewhere in your AWS account: maybe RDS is out of IOPS, maybe the connection pool is too small, maybe the autoscaler isn’t keeping up. Connecting “this metric looks bad” to a recent backend change and then to a one-line fix in your Pulumi program is an exercise in detective work.

Neo’s integration catalog bridges this gap. With built-in Datadog APM, PagerDuty, and Honeycomb integrations sitting alongside your Pulumi state, Neo can read traces and metrics from the tools your team already uses and take action.

Ask Neo:

Find the scaling bottleneck on /checkout from the last 7 days of metrics and propose a fix.

Neo pulls the metric history, matches the Datadog tag db.cluster=checkout-rds to the RDS instance in your prod-checkout Pulumi stack, and opens a PR with a Pulumi diff that bumps the storage IOPS and raises the connection-pool ceiling. You review and roll out the fix.

Toggle on the Honeycomb integration so Neo can read traces and metrics alongside your Pulumi stacks.

3. Automate PagerDuty incident response from Slack

A page comes in. You paste it into your on-call channel and tag Neo, and Neo replies with the cross-system view you’d otherwise spend the first 20 minutes assembling.

On-call triage is often about getting up to speed quickly. You get paged because something is in the red, and you don’t know why.

You mention Neo in the on-call Slack channel:

@neo, what’s going on with this alert?

Neo starts querying metrics and traces. With PagerDuty and Datadog in the integration catalog, it correlates the alert with every deploy and stack change tagged with the alert’s service in the last hour, and finds the change that lines up:

Two deploys in the last hour touched services tagged service:checkout: checkout-api@a3f9c2 (12 min ago, app-layer deploy) and Pulumi stack prod-checkout-rds (45 min ago, decreased max_connections from 200 → 100). p99 latency inflection at 14:03 lines up with the stack change. Likely cause: the connection-pool reduction is starving the API under current load.

You ask a couple of clarifying questions in-thread, then ask Neo to open a rollback PR against the Pulumi stack.

Authorize PagerDuty and Datadog in Neo's settings. Neo can then read alerts in your on-call Slack channel, find the change that correlates, and open a PR when you ask.

4. Implement a Jira or Linear ticket end-to-end

Hand Neo a ticket number from Linear, Jira, or GitHub Issues. Neo reads the description and acceptance criteria, plans against your stack, and opens a PR.

Tickets often pile up not because they’re unimportant, but because they’re not urgent. Ongoing maintenance quietly accumulates. Bumping a provider version, centralizing secret management, working through small policy violations: each one matters, but none of them ever moves to the top of the queue. Explaining each one to an agent is its own overhead.

The fix is letting Neo read the ticket itself. Connect Linear integration or Jira automation through the integration catalog (GitHub Issues works too), and Neo pulls the ticket the same way an engineer would: title, description, acceptance criteria.

Ask Neo:

Implement CAD-1234 in our payments stack.

Neo reads the ticket, plans against your existing stack, opens a PR, and drops a comment back on the ticket. The ticket and the PR end up linked, and your backlog shrinks.

Neo running locally in the Pulumi CLI: fielding a Linear issue, analyzing the codebase, and producing a PR that upgrades multiple projects to the latest Pulumi and AWS provider versions.

Start a Neo task Implement a Linear ticket end-to-end

5. Audit and tighten over-privileged IAM roles

Neo audits each role against what your stack code actually does, and proposes scoped policies that improve your security posture.

IAM cleanup is the kind of work nobody has the time to prioritize. Production has 40 roles. Half of them started with s3:* because nobody had time to scope them, and the cleanup slips quarter to quarter.

Ask Neo:

Audit IAM permissions across my accounts and propose narrower policies for over-privileged stack-managed roles.

Neo cross-references each role’s policy against what the stack code actually calls, and opens a PR per role. The PR body lists the API calls Neo found in the stack code, like s3:GetObject on audit-logs-* and s3:PutObject on audit-logs-staging, as the justification for the scoped policy. The evidence sits next to the diff.

If you’re unclear about which roles count as in-scope or what your team considers over-privileged, start in plan mode and agree on that with Neo first.

Neo auditing an over-privileged IAM role and proposing a narrower policy, with the actually-used permissions as evidence.

Start a Neo task Audit IAM and tighten over-privileged roles

6. Migrate from AWS CDK onto your platform’s golden paths

Neo reads your existing AWS CDK app and lands a PR that swaps AWS’s defaults for your team’s published components.

CDK’s L2 constructs encode AWS’s defaults. s3.Bucket with encryption: BucketEncryption.S3_MANAGED is a sane choice, but it’s AWS’s idea of sane, not yours. A platform team that’s published its own components to the Pulumi Private Registry has already decided what your bucket defaults look like: encryption with the right KMS key, tagging by cost center.

Ask Neo:

Migrate the payments-vpc CDK stack to Pulumi using our published components.¹

Neo reads the source CDK app and your registry side by side. It maps each CDK construct to its closest team-published equivalent, clarifying with you where the mapping is ambiguous.

// Before (AWS CDK, AWS's defaults)
const bucket = new s3.Bucket(this, "Assets", {
 bucketName: "payments-assets",
 encryption: s3.BucketEncryption.S3_MANAGED,
 versioned: true,
});

// After (Pulumi, your team's published component)
import * as platform from "@payments/platform";

const bucket = new platform.Bucket("assets", {
 bucketName: "payments-assets",
 classification: "internal",
});

Start a Neo task Migrate CDK onto your golden paths

Neo handles Terraform and Azure ARM migrations the same way.

7. Containerize a service and migrate it to Kubernetes from a runbook

Write the containerization pattern down once. Every service after that is a prompt away.

Containerizing an application and moving it to Kubernetes involves several small decisions: which Docker image, what labels go on deployments, how ingress is wired, and how secrets reach the pod. But after a team has moved two or three services, the pattern is set. The decisions get written down in a runbook, and every subsequent migration is mostly the same shape.

Ask Neo:

Containerize the billing-api service and write its Kubernetes manifests, following our K8s migration runbook in Confluence.

Neo reads the source repo and the runbook in Confluence via the integration catalog and starts working on your request.

You can save this as a Neo skill that splits the work into multiple PRs — Dockerfile first, ECR config next, Deployment/Service/Ingress manifests after — and link back to each runbook convention for ease of review. The output reflects your conventions: the labels you actually use, the ingress class you’ve standardized on, and the External Secrets Operator config your team prefers.

You’re still the one reviewing the PRs and deciding what the cutover looks like in production. Neo follows your internal standards, so the new service ends up shaped like the last one you migrated.

Neo migrating a VM-based service to Kubernetes step by step, following the team's Confluence runbook.

Once you’ve delegated something a few times, the next move is to automate it. The remaining three tasks are the kind Neo doesn’t need to be asked for. Drift, deps, compliance: they’re the operations you put on a schedule via Neo Automations.

8. Schedule daily configuration drift detection across your cloud infrastructure

Schedule a daily drift check across your cloud. Wake up to PRs that fix what changed overnight.

Configuration drift is an ongoing challenge. The security team rotated an IAM role at 04:47 UTC. Someone changed a security group in the AWS console three weeks ago. Left alone, drift turns into security gaps, into compliance issues, and into the kind of “wait, who changed that?” confusion nobody wants to chase down.

Pulumi Cloud is already good at configuration drift detection. Neo takes it a step further.

Ask Neo:

Every morning at 6 AM, check all production infrastructure for drift and create PRs to fix any issues you find.

From then on, the task runs on its own, and you wake up to a PR per drifted resource. The description spells out what happened (iam_role.audit-reader had inline policy AllowReadAuditLogs added at 04:47 UTC) and cites the section of infra/runbooks/drift.md Neo followed.

Some drift gets encoded into the Pulumi program, like the IAM rotation above. Some gets reverted, like the security group rule added from the console. Some gets ignored entirely, like autoscaler-managed Lambda concurrency reservations the runbook tells Neo to skip. You write the runbook once; Neo follows it every morning to decide what to do.

Neo's morning drift PR. The body names the resource, the change, when it happened, and the section of the runbook Neo followed to decide what to do.

Start a Neo task Schedule a daily drift check

9. Schedule weekly upgrades for outdated Lambda runtimes and providers

Lambda runtimes and container base images age out. Schedule the upgrade pass; review the PRs Neo opens.

AWS Lambda end-of-life notices come out months ahead. Node 20 stopped receiving updates as an AWS Lambda runtime at the end of April. Python 3.9 reached end-of-support last December. After the deadline, AWS blocks new deploys and eventually stops invoking the function. Each one needs to move to a supported runtime before the cutoff.²

Schedule it:

Every Sunday night at 10 PM, check our Lambda functions for runtimes nearing end-of-support and open PRs to upgrade them.

Neo reads the AWS Lambda runtime deprecation page, matches the end-of-support runtimes against every Lambda function in your stacks, and opens one PR per stack.

If Python 3.9 is reaching end-of-support, the upgrade is to Python 3.12, and datetime.utcnow() calls need to move to datetime.now(datetime.UTC). Neo can make all of those replacements in the same PR.

The same task can catch container base images with critical CVEs and bump them too.

Setting up a weekly task in the Scheduled Tasks UI. Once saved, Neo runs the prompt every Sunday night and opens PRs you review on Monday.

Start a Neo task Schedule a weekly runtime upgrade check

10. Fix AWS CIS Benchmark failures with daily PRs

Run the AWS CIS Benchmark on a schedule. Wake up to PRs that fix every failure.

The CIS AWS Foundations Benchmark, available through AWS Security Hub, is something every team should be keeping an eye on. The benchmark finds issues like S3 buckets that allow public read access (S3.1), root user access keys that shouldn’t exist (IAM.4), or CloudTrail not being enabled (CloudTrail.1). Scanning for these issues is a solved problem, but closing and addressing them is not. They pile up between audits because each one is a code change in a different stack, and nobody owns the cross-stack cleanup.³

Schedule the cleanup:

Every morning, read CIS Benchmark failures from Security Hub. For every failure on an IaC-managed resource, open a PR with the fix.

Neo opens one PR per failure. A bucket failing S3.1 arrives as a Pulumi diff that adds blockPublicAccess to the bucket in your prod-checkout stack. The PR body lists the CIS rule number, the resource ID, the diff, and a clean pulumi preview against the live infrastructure.

The runbook is where your security team writes down what each control means for your stacks. Block public S3 buckets, except the ones tagged public-content=true for CloudFront origins. Don’t auto-touch the break-glass IAM roles; page a human instead. Multi-region CloudTrail stays on, no exceptions. Neo reads that file, checks each Security Hub finding against it, and only opens a PR for the ones you’ve said are safe to fix. The rest get routed or ignored, the way your team already handles them.

A PR raised by Neo to fix a CIS Benchmark failure, with the failing rule, the resource, and the runbook decision laid out in the body.

Start a Neo task Schedule daily AWS CIS Benchmark scans

Neo: your newest platform engineer

Over the past year, many product teams have stopped treating AI as a request-by-request assistant and started delegating to it outright.⁴ Agents open pull requests, investigate issues, and iterate on review feedback.

But platform engineers have held back because a bad infrastructure change doesn’t just fail, it can take production down. Coding agents benefit from fast, forgiving feedback loops, but infrastructure recovery is rarely as simple as reverting a commit.

What was missing wasn’t the appetite. It was an agent with enough organizational context and grounding to plan reliably, enough guardrails to feel safe and contain mistakes, and enough discipline to keep working without being asked.

The theme across these tasks is clear. A thing platform engineers used to keep in their heads becomes a task you delegate, then becomes work that runs without you. Neo isn’t generating infrastructure from a template. It’s a teammate who knows your code, your providers, your conventions, your production metrics, and can raise PRs for you to review.

Neo now lives in your terminal, in your pull requests, in your Slack workspace, and in Pulumi Cloud. Pick one of these workflows and give it a try.

The observant reader will notice Terraform-to-Pulumi was covered in the original post. ↩︎
Also covered in the original post. Last year you could ask Neo to do it once. This year you can put it on a schedule. ↩︎
Also covered in the original post. Last year Neo could remediate violations on demand. This year Security Hub feeds findings to a scheduled task that knows your runbook’s interpretation of each control. ↩︎
For a concrete example, see Seven Rules for Building an AI-Native Software Factory: Ewan Dawson, CTO of Compostable AI, runs nineteen client deployments with five engineers, using Pulumi Neo to handle most of the infrastructure work. ↩︎

The Agentic Infrastructure Era

Joe Duffy — Tue, 19 May 2026 03:00:00 -0700

The first frontier agents excelled at was coding. The reason is evident: we have billions of lines of self-documenting code available on the internet for the LLMs to learn from. We can measure their performance on coding thanks to linters, type checkers, compilers, and test suites. The most advanced agentic systems to hit product/market fit have been coding-oriented, and it has resulted in an intense velocity increase in how much and how fast code we can write.

But as the AI tsunami whips up reams of code, what happens to it becomes just as critical. As an industry, we’ve moved beyond just coding to engineering, which includes documentation, tests, automation, and, yes, managing the very infrastructure our applications need to run. The deeper into production you go, however, the less good agents naturally are at helping. At Pulumi, we live and breathe infrastructure, and have seen this firsthand. But we’ve also been hard at work building the platform this new era runs on. In this post, I’ll share our point of view, what we’ve built, what we’re launching today, and why all infrastructure is about to be agentic.

LLMs are natural coders

It is remarkable to look back and note that frontier models, less than two years ago, in August 2024, scored just 33% on SWE-bench Verified. Present-day models score 86%, which represents a 4x reduction in the errors models will make when coding. This enables models to solve increasingly difficult coding problems, and humans can lean more heavily on them to offload tasks. Anthropic’s new Mythos model scores 94% and, although it isn’t generally available at the time of this article, there’s no question we’ll close in on 95% by the end of 2026. That is another 2.3x reduction in error rates. This very naturally puts us onto the last mile of fully agentic coding.

This has been the result of code being highly in-distribution combined with the relentless pursuit of solving coding problems from the frontier labs, especially with Anthropic’s Claude Code, but now with OpenAI’s Codex, driving a tight feedback loop that turns into an improvement flywheel.

Given that LLMs are natural coders, most of us simply assumed that the breakout success we’ve seen with agents for coding would automatically translate into new problem domains. And for sure, we have seen some success. But perhaps not as much as we’d like. Not all problem domains are documented equally well so that the models can naturally learn about them.

Andrej Karpathy noted nearly a year ago that, “Building a modern app is a bit like assembling IKEA furniture,” observing that, though writing the code was easy, fun, and fast, the next mile of actually getting the application running in production entailed many things the LLM wasn’t naturally good at, including “services, API keys, configurations, dev/prod deployments.”

At the same time, we’re seeing something magical happen here at Pulumi: LLMs are now doing over 20% of the infrastructure deployments, up from virtually zero a year ago. We expect this to grow to over 50% before the end of this year and well beyond afterwards.

The agentic infrastructure era is here. Today we’re announcing several new platform capabilities to accelerate it further.

Before getting to what’s new, however, why are we seeing this happening in reality?

Turning infrastructure problems into coding problems

We began our journey with our open-source infrastructure-as-code project nearly ten years ago. Having spent much of my career working on programming languages and compilers, I had a strong conviction that the right substrate for infrastructure was the languages developers already knew and loved, not yet another DSL and certainly not piles of YAML. So we focused first on great ergonomics for humans, and for us, that meant letting you use programming languages, tools, and ecosystems that humans already know and love. Languages like Python, TypeScript, Go, C#, Java, and more. Infrastructure as code also, importantly, comes with guardrails to make infrastructure deployments dependable, reviewable, and auditable. Over time we’ve built out an entire platform with security, compliance, and governance capabilities.

One way of thinking about this is we modeled the realm of cloud infrastructure inside the realm of programming languages: cloud resources become objects, configurations are just variables, dependencies between resources are just references, standard blueprints become classes. Doing so turns the cloud into something that is suddenly programmable, and allows us to apply real software engineering patterns and practices to infrastructure.

Here’s the twist we’ve realized this past year. It is a happy accident that the combination of this plus the verifiability of every change is the exact combination that empowers agents to do infrastructure. LLMs are natural coders. By mapping infrastructure space in code space, agents can do what they’re great at – code – and the infrastructure as code engine is the link that maps those code edits back down into infrastructure space. And as the SWE-bench curve plays out, every domain expressed in code rides it. Infrastructure, thanks to the bet we made nine years ago, is now one of them.

This isn’t just about volume of in-distribution code. Most production infrastructure code lives in private repositories. In contrast, public Python and TypeScript include vast amounts of genuinely production-grade open source, demonstrating real patterns at scale. Public infrastructure DSL corpora skew heavily toward tutorial-grade content that doesn’t scale. By modeling infrastructure in real languages, Pulumi inherits at-scale in-distribution directly. Patterns in Python or TypeScript generalize from what the model has practiced, while DSL patterns are idiosyncratic by definition and don’t compound the same way. Code can express everything an engineer can do, which is exactly why models trained on it have become prolific engineers.

What does this look like in practice? For example, an agent can define reusable infrastructure:

import pulumi
import pulumi_coreweave as coreweave
import pulumi_nvidia_aicr as aicr

class TrainingCluster(pulumi.ComponentResource):
 """A reusable training cluster resource."""
 kubeconfig: pulumi.Output[str]
 def __init__(self, name: str, *, nodes: int, accelerator: str):
 super().__init__("training:Cluster", name)
 self.cluster = coreweave.KubernetesCluster(f"{name}-k8s",
 accelerator=accelerator, # gb200/b200/h100
 count=nodes)
 self.stack = aicr.ClusterStack(f"{name}-aicr",
 kubeconfig=self.cluster.kubeconfig,
 accelerator=accelerator,
 intent="training",
 platform="kubeflow")

And then flexibly consume it – all using languages, like Python, it is already fluent in:

import pulumi
from infra import TrainingCluster
from orchestration import RLFleet

# Spin up an RL stack with 1 learner and 8 rollout workers.
learner = TrainingCluster("learner",
 nodes=4, accelerator="gb200")
rollouts = [
 TrainingCluster(f"rollout-{i}",
 nodes=2, accelerator="b200")
 for i in range(8)
]

fleet = RLFleet("ppo",
 learner=learner, workers=rollouts)

pulumi.export("login", fleet.head_node)

Beyond just being in-distribution, infrastructure as code also gives us a critical piece: the ability to diff changes at the infrastructure layer. Just as we wouldn’t vibe code without git showing us the source changes, we shouldn’t vibe infrastructure without a tool that shows what it will do before it does it, and what it has already done in the past. It’s like git diff for your infrastructure.

The combination of languages, type systems, linting, testing, and infrastructure diffs gives agents enough of what they need to create a fully closed, verifiable, self-correcting loop. Verifiability is the thing that ultimately matters. Every action an agent takes can be previewed, policy-checked, reviewed, and audited. By humans, by other agents, by CI/CD automation. Guardrails guide the agent and verifiability proves the end result.

When I say “infrastructure” by the way, I mean something quite broad. It includes public cloud hyperscalers, of course, like AWS, Azure, Google Cloud, and CoreWeave. It includes cloud native technologies like Kubernetes and Helm. It also includes private clouds powered by VMware and other data center technologies. It includes broad cloud platforms like Cloudflare. But, subtly, also any hosted managed infrastructure service, such as Snowflake, MongoDB, Databricks, and more. And increasingly, the observability, security, and operational tooling that wraps everything else, like Datadog, Sentry, PagerDuty, and other systems that give agents the context to reason about what’s actually happening in production. We support thousands of providers with a consistent programming model and agents are increasingly writing those too.

What we’re shipping today

The foundation of expressing infrastructure as code, combined with verifiable changes, makes a 100% agentic infrastructure future even conceivable. But we still have work to realize this vision.

Meeting agents where they work (the CLI)

We know real code is the right substrate for agentic infrastructure – but we also know agents want to start with the smallest possible commitment and progressively level up. So we’ve designed the Pulumi experience as a progression: an agent should be able to sign up, run its first command, and grow into a full infrastructure as code workflow without ever hitting a wall or manual intervention. Here’s what we’re launching today to make that journey work end-to-end:

Agent accounts. Now agents can use free, ephemeral Pulumi Cloud accounts straight from Claude Code, Codex, OpenCode, Copilot, Cursor, and others. This lets an agent do anything a human could with the Pulumi product without a manual signup step, removing friction while letting agents benefit from robust infrastructure management. A human can claim that account at any time to make it permanent and to share with their team.

One-command execution. Today an agent needs to know how to find, download, and install Pulumi, and manage subsequent versioning and upgrades. Anything that adds friction is discouraging to an agent. We now have an npm package that enables npx pulumi <anything>-style commands so agents can run any Pulumi command anywhere without needing to worry about installation or version pinning.

Imperative infrastructure operations. The new pulumi do command enables direct create, read, update, delete, list, and API operations with a single command. For example, pulumi do create eks:Cluster spins up an AWS EKS cluster with built-in best practices, and pulumi do update cloudflare:r2:Bucket --tags '{ "Foo": "Bar" }' updates a bucket’s tags. All thousands of providers and hundreds of thousands of resource types in the Pulumi ecosystem are supported across clouds. As the complexity of your scenario outgrows singular commands, it is easy to eject out into a full-blown infrastructure-as-code project.

Pulumi Cloud in the CLI. Over the years, we’ve added tons of great capabilities to Pulumi Cloud – things like change history, time-to-live stacks, drift detection, resource discovery and search, private registries, IDP, audit logs, secrets management, team-wide policy enforcement – but didn’t add the respective ergonomic commands to the CLI. These are the sort of features that matter at scale. Now it’s all there in your terminal where agents can use them. Over 30 new commands, you can think of this as the equivalent of the gh CLI which agents really like.

Read the blog: Better CLI Interactions for Agents and Humans →

Neo, everywhere you work

Once an agent (or a team) has graduated through these capabilities, what comes next is asynchronous infrastructure work and, increasingly, autonomy. That’s where Neo comes in: Pulumi’s own infrastructure agent, now upgraded with the surfaces, integrations, and cadences teams actually need to run agentic infrastructure in production. We are shipping:

Neo in the CLI. The new pulumi neo command lets you run the same agent that is already in Pulumi Cloud. It even shares the same agentic loop and uses a sophisticated architecture where the agent workstation is your local computer, allowing it to more seamlessly access source context and tools. This allows you to do agentic infrastructure wherever you prefer: Neo in the cloud for asynchronous, autonomous tasks, or your workstation for more intensive deep-dive paired sessions.

Neo GitHub and Slack Apps. Now you can @-mention Neo from GitHub pull requests, and/or straight from Slack, to kick off agentic infrastructure workflows wherever it is most convenient, complete with whatever guardrails you’ve configured already.

Neo Integration Catalog. A new integration catalog lets you configure connectors to other systems that bring valuable infrastructure management context, including Atlassian, Datadog, Honeycomb, Linear, PagerDuty, and Supabase. This lets you tap into additional planning, specification, observability, and live site incident data, expanding what agents can do on Days 0, 1, and especially 2.

Scheduled Tasks and Read-Only Sessions. Now you can automate recurring infrastructure tasks, including confining Neo to read-only operations for extra safety. This opens up scenarios like reporting on infrastructure patterns weekly, automatically cleaning up waste in your dev accounts daily, or scheduled maintenance and upgrades.

Read the blog: 10 More Things You Can Do with Pulumi Neo →

Partnering with the frontier of AI infrastructure

We’re investing in deep partnerships with the companies building the frontier of AI infrastructure, and are shipping two new partner providers in close collaboration with the teams behind them:

CoreWeave provider. This new provider brings the full power of the CoreWeave platform, the leader in GPU infrastructure, to help teams provision AI workloads with ease. This includes support for all of the CoreWeave services including CoreWeave Kubernetes Service (CKS) and includes several examples and templates out of the box for training and inference architectures written in AI-native languages.

NVIDIA AI Cluster Runtime (AICR) provider. This new provider delivers fully functional NVIDIA software stacks atop the underlying cloud provider’s GPU infrastructure. This packages out-of-the-box components like NVIDIA GPU Operator, Kubeflow, NIM Operator, and dozens more otherwise tricky-to-configure cluster components. We’re excited to showcase the new AICR provider in tandem with our new CoreWeave provider: you can now spin up a CKS cluster and configure it with AICR in one program.

Educating and measuring agentic infrastructure intelligence

We’ve also invested in making Pulumi maximally legible to agents, because the best tools only matter if agents can find them, read them, and know which one to reach for and when:

Agent-friendly docs. We now serve our docs website in markdown to agents and did a pass over our documentation and CLI text to ensure it’s maximally useful to agents.

Agent-friendly CLI ergonomics. We have added --json and structured errors across the CLI to help agents parse and react to outputs appropriately.

New skills. We’ve done a pass over our skills and added a new uber-skill that describes how an agent should leverage the full suite of capabilities we now offer, from Level 0 (agent accounts), to Level 1 (pulumi do), to Level 2 (full IaC and pulumi up), to Level 3 (Pulumi Cloud governance and full autonomy with Neo), and beyond. It’s handy for humans too.

InfraBench. We have created a new benchmark to measure how well an agent performs on a wide array of representative infrastructure tasks. This is the infrastructure equivalent to SWE-bench and lets us measure stock agents, agents plus the new tools and skills, and Neo, and how they’re improving over time. Although we are still hard at work on building out the comprehensive suite, and will keep it internally for now, we will begin publishing our progress against InfraBench as we continue tackling new problems and improving agent performance.

What comes next

We’re primarily focused on two pillars as we go forward:

First, we are making agentic infrastructure a reality with increasingly autonomous workflows. The pace and scale of infrastructure in an AI era are beyond anything we’ve seen before, and we need to ensure infrastructure doesn’t become the bottleneck to velocity. We will be shipping steady improvements that help agents score better on InfraBench as a measure of their abilities. This will begin with new building blocks and in-context improvements, but we’re really excited that every improvement the frontier labs ship in general-purpose coding ability is automatically an improvement in agentic infrastructure on Pulumi, at zero cost to us or to our users. The result is that Pulumi-equipped agents will consistently and measurably outperform raw agents on real infrastructure work, and that gap will compound as the substrate and the models both improve.

Next, a world in which agents are provisioning, updating, monitoring, and managing all of our infrastructure changes the human/agent/infrastructure interface greatly. We need better change management, policy enforcement, and observability tools. Going from a single human doing a single infrastructure task at a time, to a single agent doing that task, to teams of agents doing lots of autonomous infrastructure tasks at once, will reveal new bottlenecks needed to fully govern your infrastructure. We have a lot of those facilities in Pulumi Cloud already but we will continue pushing here to improve them and ensure they scale up as fast as they need to. This is the layer that matters even as agents themselves get great at infrastructure. The smartest agent in the world still needs guardrails, audit trails, and policy enforcement to be trusted with production systems at scale, and that layer gets more valuable as agents get more capable, not less.

The combination of these two things will lead to a world where agents are able to manage more infrastructure, and to do so with increasing autonomy, while human operators retain full visibility, understanding, and control over what their teams of agents are doing at all times. This is the full engineering loop applied to infrastructure. Not just code generation, but design, review, deployment, and operation, with agents and humans collaborating across every step.

It’s a new era, and we’re just getting started

It was remarkable to see in practice and at scale that what’s good for humans is good for agents: ergonomic infrastructure as code. We’ve seen every point along the spectrum, ranging from Wiz managing 1 million resources and doing 100k daily deployments, to Compostable.ai already living the 100% agentic infrastructure dream, to a frontier lab who grew their infrastructure footprint by 982% in a year, all using infrastructure modeled in true code. All very different scales and teams, but with one thing in common: it was code all the way down.

Languages and verifiability already provide strong foundations, however, there are still clear areas we need to improve, beginning with making the full suite of infrastructure automation, security, and governance tools more accessible to agents. That’s what today is all about. To deep dive on details of the launches, check out the companion blog posts released today.

Agentic infrastructure is a super exciting one-way door for our industry. We’re pushing hard to take this beyond 20% and toward 100% in the years ahead. This is uniquely enabled by infrastructure as code in languages the models already speak combined with built-in verifiability that makes agentic work automatically safe. There’s more to be done, however, today’s launch will bend the curve even further upwards. We’re still only just getting started.

Happy clouding,

-Joe

How Building AI Agents Has Changed in 2026

Engin Diri — Thu, 14 May 2026 00:00:00 +0000

Twelve months ago, building an AI agent meant picking a framework, defining your tools, standing up a RAG pipeline, and writing a stack of glue code to wire it all together. That was the default playbook. The post-mortem on six months of work usually went the same way: half the time went into infrastructure that had nothing to do with the agent’s actual job.

That isn’t where the work is anymore. Most of the middle layer is gone. The SDKs ship with the tools, the skills system replaced the upfront tool registry, and longer context windows pushed vector search out of the default slot it held all of last year.

The shape is the same as a lot of infrastructure shifts before it. The hard thing got cheap, the cheap thing got expected, and the question moved up a level.

The old playbook

A 2024 to 2025 agent project looked like this. You picked a framework, usually LangChain, LlamaIndex, or an early version of Pydantic AI. You wrote tool definitions, usually a wrapper around an API the agent would call. You stood up a RAG pipeline: chunk your documents, embed them, pick a vector database, write retrievers, layer reranking on top. Then you wrote the agent loop yourself, including prompt assembly, tool dispatch, retry logic, and observability.

This was the default for good reasons. Foundation models had short context windows. They didn’t ship with file access. They couldn’t run code. If you wanted an agent to do anything useful with your data, you had to bring the data to the model in pre-digested chunks.

The cost wasn’t only setup time. It was infra bills, retries against embedding APIs, and a context strategy that fought the model as the model got better. By mid-2025 the retrieval layer was often the bottleneck on quality. The agent would ask a question, get five plausible-looking chunks, and answer from those instead of the document you actually wanted it to read. Chunking decisions made on a Tuesday in March were still hurting answer quality six months later.

Most teams I talked to in 2025 were tuning their RAG pipeline. Almost nobody enjoyed it.

The shift: three things changed at once

Three changes landed close enough together that they collapsed the middle layer.

Built-in tools. The Claude Agent SDK ships with Read, Write, Edit, Bash, Grep, Glob, WebSearch, and WebFetch out of the box. OpenAI’s Codex SDK is similar in shape, with shell and file tools available to the agent by default. These are the tools every agent project was rebuilding in 2024, often as a side quest to the work the agent was actually meant to do. A Read that handles binary files. A Bash that streams output and respects working directory. A Grep that doesn’t choke on large files. The 80% of agent tooling everyone was paying their team to reimplement is now table stakes.

The consequence is that you can give an agent the ability to do real work with about ten lines of configuration. The flip side is that the differentiator moved up a layer. The value isn’t in having Read. It’s in what the agent does with it.

Anything outside the built-in toolbox plugs in through MCP servers. The registry has grown nearly 8x since early 2025, and every major model vendor now ships first-party support. The picture in 2026 is more layered than that, though. A lot of what used to call for an MCP server is now better served by the agent invoking a CLI through Bash and wrapping the recipe in a skill. Benchmarks put CLI-based tool calls at a fraction of the context cost of equivalent MCP calls, with fewer round-trips and fewer failure modes. MCP still earns its place for protocol-heavy work like browser control, OAuth flows, and streaming services, but it stopped being the automatic answer to “how do I give my agent a new capability.”

Skills replaced tool stuffing. The old way was to register every tool the agent might need at startup, eating context every turn whether the agent used the tool or not. A hundred tools meant a heavy system prompt before the agent had thought about anything. The skills pattern flips that. A skill is a small markdown package with a name and a one-line description. The agent sees the description (around 100 tokens) and only loads the body when it decides the skill is relevant. A hundred skills no longer means a hundred tools’ worth of context tax. Anthropic frames this as progressive disclosure: because the body only loads on demand, the amount of content you can bundle into a single skill is effectively unbounded.

Progressive disclosure isn’t a new idea. What’s new is that the agent harness now treats it as the default loading strategy instead of something you have to engineer.

RAG got demoted. This is the change with the biggest blast radius and the smallest amount of commentary. A year ago, “we need to add RAG” was the reflex answer when somebody asked how an agent would handle a corpus. Today that question splits three ways. If the corpus fits in the context window, put it in. If the agent can grep the filesystem, let it grep. If the corpus is genuinely too large for either, vector search is still right, but you’ll find that’s a smaller set of cases than it used to be. You can see this in the coding agents that already ship today. Cursor, Claude Code, and Devin lean on grep, find, and direct file reads more than vector search. LlamaIndex’s own writing on agentic retrieval is one of the clearer reads on where this is going.

Vector search didn’t get worse. The context around it improved enough that it stopped being the right first move.

Taken together, what got pulled into the SDK is the middle of an agent project: the tools layer, the retrieval layer, and the loop. What’s left for the team is the system prompt, the skills, and the policies around what the agent is allowed to do.

When you still need a framework

The first reaction to a lot of this is to declare that frameworks are over. They aren’t, but the cases where you reach for one have narrowed.

Pydantic AI is still the right choice when you want strong typing, deterministic output schemas, and an evaluation loop that matches how the rest of your Python codebase already thinks. LangGraph is still the right choice when your problem is genuinely a graph of agent states with branching and human approval steps. OpenAI’s Agents SDK is built around explicit handoffs between agents and earns its place when that pattern fits how you want to decompose the work. CrewAI is the fastest path I’ve seen for prototyping a multi-agent system, as long as you can live with its opinions. Any team running production traffic across multiple model providers is going to want a routing layer that the official SDK from any single vendor isn’t going to give them. Anthropic’s own writing on building effective agents lands in the same place: start with the simplest thing, add complexity only when the problem demands it.

The mental model that works for me: start with the SDK, reach for a framework when you outgrow it. “Outgrow” usually means one of four things:

Multi-provider routing. You’re running production traffic across more than one model vendor and need a routing layer the official SDKs don’t ship.
Multi-agent orchestration. Your problem genuinely decomposes into separate agents with handoffs, branching, or human approval steps.
Deterministic typing. You need strong schemas and validation around inputs and outputs, and the rest of your codebase already thinks that way.
Production observability. You need eval loops, replay, or tracing beyond what the SDK provides out of the box.

If none of those four are biting, the SDK is probably enough, and adding a framework on top is a layer you’ll regret in six months.

Where this lands for infrastructure work

Two things from the new agent shape map cleanly onto infrastructure work. The first is that “built-in tools plus governed actions” is the model an IaC platform was already running. The SDK assumes the agent has tools that do real work. The platform assumes those tools have policies, audit logs, and short-lived credentials around them. Those assumptions stack.

The second is that a state graph is already structured context. You don’t need to chunk it. You don’t need to embed it. An agent reasoning over a Pulumi stack can grep its way through the program graph the same way it greps a codebase, and the answers are grounded in the same source of truth the rest of your platform uses. I wrote the deeper version in Grounded AI: Why Neo Knows Your Infrastructure. The dark-factory and sprawl posts (The Dark Factory Pattern for Infrastructure and Agent Sprawl Is Here. Your IaC Platform Is the Answer.) are the places to go if you want to push on this further.

Start with the SDK

A year ago, an agent project was 80% glue code and 20% the thing the agent actually did. On most projects today that ratio is flipped. If you’ve been sitting on an agent idea, build it the SDK way first and reach for a framework only when you hit something the SDK genuinely can’t do. Most teams will be surprised how often they don’t.

There’s one agent you don’t have to build at all. Pulumi Neo is the same SDK-first shape applied to the IaC slice: tools that reason directly over your state graph, governed by the controls the rest of your platform already runs on. Save your own SDK time for the agents only you can build.

See how Pulumi Neo works

The Dark Factory Pattern for Infrastructure: Running Pulumi Lights-Out

Engin Diri — Tue, 05 May 2026 00:00:00 +0000

The original dark factory was Fanuc’s robotics plant in Oshino, Japan, where the lights are off because nobody is on the floor. Robots build robots. Parts move through the line for weeks at a time without a person walking past them.

The same pattern is now showing up in software. Three engineers at StrongDM shipped roughly 32,000 lines of production code without writing or reviewing any of it. Stripe’s “Minions” agent system merges over a thousand pull requests every week. In January, Dan Shapiro of Glowforge published a five-level autonomy ladder that landed cleanly enough to become the shorthand most people now use, and BCG put out a piece calling it the dark software factory.

Almost every public writeup so far is about application code. The harder question is what this looks like for infrastructure.

What a dark factory actually is

Shapiro’s ladder is the cleanest framing I’ve seen. He borrows it from the SAE’s self-driving levels, and it fits surprisingly well:

Level	What it is	Driving analogy
0	Spicy autocomplete	Stick shift; you do everything.
1	Coding intern (boilerplate)	Cruise control.
2	Junior developer (interactive pair)	One hand on the wheel.
3	AI writes the majority; you review every PR	Eyes still on the road.
4	Spec-driven; agent runs unattended for hours; you review later	Sleeping at the wheel, you can still wake up.
5	Dark factory; no human review of code before production	No steering wheel at all.

Most teams are at level 2 or 3. A few of the more aggressive ones are at 4. Level 5 is the experiment. Most teams won’t get there safely, and probably shouldn’t try to. The interesting design question is what has to be true for level 5 to be safe at all, and that question gets sharper when the thing being shipped is infrastructure.

A dark factory is not a coding harness. A harness is the framework an agent runs inside; the dark factory is the surrounding system that makes a harness’s output mergeable without a human reading the diff. Copilot and Cursor sit at the other end: interactive, the human stays in the loop on every keystroke. The dark factory takes the human out of the per-change loop entirely and puts them at the top, writing the spec and the acceptance criteria.

The wall between generator and validator

Strip the dark factory down to its layers and there are four of them.

flowchart LR
A[Inputs<br/>Humans] --> B[Code Generation<br/>Autonomous]
B --> C[Validation<br/>Autonomous, isolated]
C -->|pass| D[Merge & Deploy<br/>Autonomous + existing CI/CD]
C -->|fail| B
A -.->|holdout scenarios<br/>generator never sees these| C

The single most important rule is that Code Generation and Validation must be completely isolated. The generator never sees the acceptance scenarios. A separate evaluator does, and it judges the generator’s output against scenarios the generator could not have memorized.

The reason is sycophancy. LLMs are too eager to agree with their own prior turns and too willing to declare victory on something they just produced. Without isolation, the same model that wrote the change is the one telling you it’s fine. The practical concern is direct: a test stored in the same codebase as the implementation will get lazily rewritten to match the code, not the other way around. It isn’t malice; it’s the agent doing exactly what it was asked, badly. The wall is what stops that.

StrongDM’s pattern for this is holdout scenarios: plain-English BDD acceptance tests stored where the generator cannot reach them. Each scenario runs three times against an ephemeral deployment, two of three must pass, and the overall pass rate has to clear 90% before the change moves forward. If the generator fails, it gets a one-line failure message (“SQL Injection Detection failed: endpoint returned 500”), not the scenario text. It cannot game the test.

Without that wall, you don’t have a quality gate. You have theater.

Why infrastructure is the harder version

Application code factories can lean on tests, linters, and type checkers. Infrastructure adds blast radius, drift, secrets, irreversible actions, and multi-region state. A code dark factory shipping a broken UI causes a bad user experience. An infrastructure dark factory shipping a broken IAM policy ends in a postmortem.

A few things make this manageable on Pulumi specifically.

The orchestrator does not need to be invented. The Pulumi Automation API is the engine as an SDK in Python, TypeScript, Go, .NET, Java, or YAML, which is the same surface a dark factory orchestrator runs on. Credentials don’t have to be long-lived: ESC and OIDC issue short-lived ones per run, so the agent never sees a static secret.

Policy doesn’t have to be probabilistic: CrossGuard enforces deterministic rules at preview time. Execution doesn’t have to happen on a laptop: Pulumi Cloud Deployments runs pulumi up inside a governed runner with audit logs and approval rules already wired. And the reasoning layer doesn’t have to start from scratch: Pulumi Neo is grounded in your state graph and ships with three modes (Auto, Balanced, Review) that line up cleanly with Shapiro’s levels 5, 4, and 3.

That doesn’t make Pulumi a dark factory by itself. It means the parts that an application-code factory has to build from scratch are pieces a Pulumi shop already has: a credential broker, a policy engine, a governed runner, a state-aware reasoning layer, an audit trail.

And one more piece nobody talks about: pulumi preview produces a clean, deterministic validation artifact, and CrossGuard evaluates that artifact without ever seeing the conversation that produced the program. That’s the same context-free judgment the holdout pattern depends on, applied at the policy layer instead of the acceptance-test layer. For infrastructure, half the wall is already built.

The interesting work is the part that nobody ships in a box.

The interesting work

What no platform ships for you is the wall: the holdout scenarios for infrastructure, the isolated evaluator that runs them, and the agreement on which stacks are even allowed to run lights-out.

The happy-path orchestrator is small. It pulls a spec, runs preview, hands the preview to an isolated evaluator (with its own credentials and its own access to the cloud, no access to the generator’s prompt or output), and branches on the verdict. Auto mode runs up immediately. Balanced mode submits a deployment that requires approval. Review mode opens a PR for a human. Every branch records a stack version traceable in the audit log. Retries, observability, secret rotation, and the rest of the production-grade plumbing add up to real code, but the shape is small.

The wall is the part that takes a week to get right. You write five plain-English scenarios for one stack (“after pulumi up, the bucket is private, has SSE-KMS, lives in eu-west-1, and is tagged owner=team-x”) and a janky evaluator that runs preview and up against an ephemeral copy, queries the cloud, and asks a separate model whether the resulting state satisfies the scenario. Triple-run, 90% pass gate. Then you watch it for a few weeks before you let anything auto-apply.

A four-phase rollout

This is the same path the application-code factories walked, with the gates tightened.

Phase 1: better context, this afternoon

Write an AGENTS.md for your most active stack repo. Pulumi Neo reads it natively, as do most coding agents. While you’re there, look at your CrossGuard rules and rewrite the error messages as instructions. Not “S3 bucket has no encryption” but “S3 bucket has no encryption. Set serverSideEncryptionConfiguration with SSE-KMS to fix.” That single change is the difference between an agent flailing and an agent fixing the policy violation on the first try. Wire pulumi preview as a build-before-push gate so PRs don’t show up just to fail CI.

Phase 2: spec-driven with holdouts, this week

Pick one stack with a small blast radius. A review-stack lifecycle is ideal. Write five plain-English holdout scenarios for it and the janky evaluator above. Humans still approve every PR. Don’t auto-merge yet. You’re earning the data, not declaring trust.

Phase 3: take the human out of the merge

Only after the three measurable gates hold over twenty PRs (scenario pass rate above 90%, false positive rate below 5%, human override rate below 10%) flip auto-apply on for that one stack. Add a weekly drift sweep that goes through the same scenario gate as everything else.

Phase 4: lights out

Expand the auto-apply flag to every stack with strong scenario numbers. Wire your issue tracker so tickets tagged infra:fix flow through the pipeline. Mock the cloud APIs that are slow or flaky enough to make scenario evaluation expensive. At this point the orchestrator is configuration, not architecture.

What could go wrong

None of these have clean fixes. The mitigations below reduce risk; they don’t eliminate it. Any team running level 5 should expect to eat one or two of these in the first year.

The validator approves a bad change. This is the obvious one. The standard mitigation is layered: triple-run each scenario with a 2-of-3 threshold, a 90% gate over the run set, a human audit of the first fifty auto-applied changes, and your existing policies still run after the validator says yes.

The agent gets a destroy permission it shouldn’t have. There’s a class of operations that should not sit in the autonomous loop yet: dropping a database, deleting a hosted zone, rotating a root key, anything that crosses a regulated data boundary. Scope what each agent identity can do at the credential layer, require human approval for anything destructive, and start every stack at Review mode. Tag changes, security-group adjustments, and instance resizes can run autonomously today. Release-branch cuts and config promotions can probably run by next quarter. The destructive class earns its way in over months.

You need all three of those layers. Approvals without policy means anything a human approves in a hurry ships. Policy without approvals means a sufficiently clever spec eventually finds the gap. Both without a human kill switch means an incident at 3 a.m. has nobody to escalate to.

Costs blow up. Cap retries at three per spec, alert on token spend per run, and remember that StrongDM reported roughly $1,000 per day per engineer-equivalent. That’s still cheaper than a salary, but only if you put the cap in place before you find out.

Where to start

Most of what a dark factory needs already exists in any reasonably mature platform. Whatever you have for state, policy, credentials, audit, and a deployment runner is the substrate. The interesting work is not building the factory. It’s the wall: the holdout scenarios that make the gap between “the model says it’s fine” and “the system is actually fine” mean something.

For most teams, Phase 1 alone is the win. Full Level 5 may stay out of reach indefinitely, and that’s fine. The path itself forces useful work: clearer specs, named bottlenecks, the deterministic gates humans had been running in their heads.

Write an AGENTS.md and five holdout scenarios for one stack this week. That’s enough to get a real signal on whether the pattern fits your team. The rest of the path is the same problem the application-code factories have already worked through, with the gates set tighter.

Agent Sprawl Is Here. Your IaC Platform Is the Answer.

Engin Diri — Wed, 22 Apr 2026 00:00:00 +0000

Somewhere in your company right now, a developer is building an AI agent. Maybe it’s a release agent that cuts tags when tests pass. Maybe it’s a cost agent that shuts down idle EC2 overnight. It’s running, it’s in production, and there’s a decent chance the platform team doesn’t know it exists.

This isn’t a thought experiment. OutSystems just surveyed 1,900 IT leaders and the numbers are rough: 96% of enterprises run AI agents in production today, 94% say the sprawl is becoming a real security problem, and only 12% have any central way to manage it. Twelve percent. You can read the full report here.

The real question is where those agents run. Inside the platform you’ve already built, or somewhere off to the side where nobody on the platform team can see them.

The new platform tension

Platform teams have always had two jobs that pull in opposite directions. Let developers ship without waiting on a ticket. Keep the infrastructure coherent while they do. Golden paths, review stacks, a catalog of components that don’t fight each other.

Agents break the second half of that deal.

A developer with a sharp prompt can spin up an SRE agent that watches a queue, a release agent that cuts tags when the test suite goes green, or a cost agent that kills idle infra at 2 a.m. That’s useful. It’s also running on your production cloud account, using credentials you never provisioned, writing to systems you never approved, and the only audit trail is whatever the developer remembered to log. The Salesforce 2026 Connectivity Benchmark pegs the average enterprise at twelve agents today, projected to grow 67% over the next two years. Most teams aren’t ready for one, let alone twenty.

This is the same shape as every sprawl problem before it. I wrote about the last one in How Secrets Sprawl Is Slowing You Down, and the pattern keeps repeating. When something useful gets cheap, it proliferates. When it proliferates without structure, it becomes a liability.

The clock is also ticking on the compliance side. The EU AI Act’s high-risk obligations kick in on 2 August 2026. Colorado’s AI Act goes live on 30 June 2026 after last year’s delay. A folder of unreviewed agent scripts isn’t going to hold up against either of those.

Three ways to respond (only one of them works)

There are roughly three paths from here.

Do nothing. Accept the sprawl and hope nothing catches fire. This is the default, and it’s also how you end up explaining to an auditor why some finance agent moved data between three systems last Thursday and nobody remembers which prompt triggered it.

Mandate centralization. Tell developers every agent has to be registered and approved before it runs. This sounds responsible on a slide, and it falls apart inside a sprint. Developers route around friction. If the official path takes a week and the unofficial path takes an afternoon, the unofficial path wins, and you’ve just pushed the sprawl underground where you can’t see it anymore.

Make the platform the obvious path. Build the thing developers actually want to use. A place where an agent inherits the guardrails, credentials, policies, and audit trail by default, because that’s what’s on offer. Adoption becomes a side effect of shipping something good.

Option three is the only one that scales. It’s also the one where most platform teams look at their existing stack and assume they need to build a pile of new scaffolding. I don’t think they do, and the rest of this post is why.

The seven things an AI agent needs from your platform

An agent needs seven concrete things from the platform it runs on. Each one maps to a Pulumi primitive you already own.

1. A trustworthy context lake

Agents are only as good as the context they can reason over. Drop a generic LLM into your cloud account and you’ll get plausible-sounding nonsense, because the model has never seen your environment. What you actually need is a grounded source of truth: what resources exist, how they relate, which stack owns what, which version is running where.

Pulumi state is already that. Your program graph, your stack outputs, your resource metadata, all of it adds up to a structured record of what you’ve actually deployed. Pulumi Neo reasons directly over that graph, which is why it can tell you why a deployment drifted instead of guessing. I wrote the long version of that argument there. Short version: you already have the context lake. Point agents at it.

2. Pre-cleared integrations

An agent that needs to touch five systems shouldn’t need five separate credential dances. That’s where credential sprawl starts. Every agent gets a long-lived key, every key ends up in somebody’s .env, and every rotation turns into an incident.

The Pulumi surface here is the 200+ providers plus Pulumi ESC handling dynamic credentials through OIDC. An agent doesn’t ask for an AWS access key. It asks ESC for a short-lived, scoped token bound to the environment it’s allowed to operate in, and the token expires when the task ends. No static keys, no rotation pain, no awkward postmortem about how something got committed to GitHub. The ESC patterns I walked through in the Claude skills post work just as well for an autonomous agent as they do for a human developer, which is really the whole point.

3. Governed actions

There’s a real difference between “an agent can see your infrastructure” and “an agent can change your infrastructure.” The second one is where you actually need structure. Pulumi Deployments gives you that structure: defined workflows, controlled triggers, running inside your Pulumi Cloud boundary instead of whatever environment the developer happened to spin up. The Automation API lets you build higher-order orchestration on the same primitives your developers already use.

The framing I keep coming back to goes like this. An agent shouldn’t call pulumi up directly. It should submit an action to a governed pipeline that runs pulumi up on its behalf, inside an environment you control, with a log trail and the guardrails already in place. Same effect, very different threat model.

4. Deterministic policy

Real governance lives outside the prompt. “Please don’t delete production” is a wish written into a system prompt, not an enforced control. And when an agent overrides your intent to do what it thought you meant, it’s behaving exactly the way the technology was designed to behave.

Pulumi Policies is the answer the IaC community landed on years ago: policy as code, written in a real programming language, evaluated deterministically at preview and update time. Disallow production RDS deletions. Require encryption at rest. Block S3 buckets with public ACLs. An agent running through Pulumi hits those gates whether it “wants” to or not, because the gates live in the pipeline and not in the prompt. This is the pillar most teams underweight, and it’s the first one most auditors ask about.

5. An audit trail

When something goes wrong at 3 a.m. (and with enough agents running, something will), you need answers fast. What changed, who changed it, and why. Not just “which agent,” but which version of which agent, triggered by what event, authorized by which policy, touching which resources.

Pulumi Cloud’s activity log, the stack update history, and ESC audit logs already capture all of that. Every update is versioned. Every secret access is logged. Every policy evaluation is recorded. When an agent submits a change through your Pulumi pipeline, it inherits that audit surface for free. The alternative is reconstructing an incident from a mix of Slack messages, container logs, and developer memory, which is roughly the state most teams without a platform are in today.

6. A review process

Not every agent action should wait for a human. But agents do need a promotion path, the same way new platform components do. Experimental, then reviewed, then trusted, then autonomous. That’s exactly what pulumi preview, review stacks, and Deployments PR workflows already model for human contributors. An agent that wants to make a change should have to submit it the same way a junior engineer would. As a diff, with a plan, against a preview environment, until it earns the trust to skip steps.

This connects back to the pattern I laid out in Golden Paths: Infrastructure Components and Templates. Golden paths were never only for humans. They’re just paths, and agents can walk them too.

7. Human-in-the-loop approval

The last pillar is the one that keeps the other six honest. Some decisions shouldn’t be automated, full stop. Production rollbacks outside business hours. Destructive changes above a certain blast-radius threshold. Anything that touches a regulated data boundary. For those cases, you want a forced human checkpoint that the agent can’t route around.

Pulumi Deployments approvals already play that role for human changes. Pulumi Neo’s review steps add the AI-aware version: a structured plan, a diff, a named approver, and a record of what they decided and why. I walked through what this looks like in practice in Self-Verifying AI Agents. Short version: an agent that proposes is much safer than an agent that commits.

Why IaC is the natural substrate for this

Step back from the seven pillars and look at what they have in common. Context, integrations, governed actions, deterministic policy, audit, review, approval. None of those are new problems that AI agents invented. They’re the problems infrastructure-as-code has been quietly solving for a decade, for human developers.

Every meaningful agent action ends up being a change, whether that’s to infrastructure, configuration, secrets, or state. IaC is the one layer in your stack that already treats change as the unit of work. Plan, preview, apply, record. If you want governance for agents and you don’t want to build it twice, the most efficient move is to route agent changes through the same substrate your humans already use.

I made the same point from a different angle in Token Efficiency vs Cognitive Efficiency: Choosing IaC for AI Agents. An IaC platform that models your world as a graph of typed resources is a much better reasoning surface for an agent than a stack of YAML or a bash script somebody wrote on a Friday. The structure is what makes it work.

What this means for the platform engineer

There’s a narrative floating around that AI is going to make platform engineers less relevant. I haven’t seen it hold up against an actual production environment. Every stat I’ve looked at points the other way. Gartner expects 70% of enterprises to deploy agentic AI as part of IT infrastructure and operations by 2029, up from less than 5% in 2025. LangChain’s State of Agent Engineering report already has 57% of teams running agents in production today. And Gartner projects that 80% of large software engineering orgs will have a platform team by end of 2026, up from 45% in 2022. More agents means more changes, more changes means more blast radius, and more blast radius means more need for the thing platform teams are uniquely equipped to provide.

Your classic responsibilities haven’t gone anywhere either. Golden paths, service catalogs, CI/CD, on-call rotations, all of that is still yours. Agents are an additional layer that needs the same discipline. The upside is that if your platform already runs on a mature IaC surface, you’re extending a muscle you’ve been building for years instead of growing a new one.

The developer-facing side matters too. A developer building an agent needs to know what’s available to them, needs templates that work on the first try, and needs to see what teammates have already built so they don’t start from a blank page. That’s the territory the Claude skills post and IDP Strategy: Self-Service Infrastructure That Balances Autonomy With Control cover. That’s the experience layer that makes developers actually choose your platform instead of routing around it. You need both sides working at once. The governance your security team cares about, and the experience your developers will actually reach for.

Close the window

The agents your developers are shipping this week are going to outlive the experiment that started them. Some of them will become critical. At least one will cause an incident. At least one will eventually show up in an audit. All of them are going to be easier to govern if they were built on your platform from day one than if you try to wrap policy around them later.

If you want the longer view on where this is going, AI Predictions for 2026: A DevOps Engineer’s Guide is the companion piece. If you want the developer-facing version of the grounding argument, Grounded AI is what to read next.

Either way, here’s where I land. The substrate for agent governance is already running in your stack. You’ve been pointing it at human changes for years. Now point it at the agents too.

See how Pulumi Neo governs agent actions

Superpowers, GSD, and GSTACK: Picking the Right Framework for Your Coding Agent

Engin Diri — Mon, 13 Apr 2026 00:00:00 +0000

Three community frameworks have emerged that fix the specific ways AI coding agents break down on real projects. Superpowers enforces test-driven development. GSD prevents context rot. GSTACK adds role-based governance. All three started with Claude Code but now work across Cursor, Codex, Windsurf, Gemini CLI, and more.

Pulumi uses general-purpose programming languages to define infrastructure. TypeScript, Python, Go, C#, Java. Every framework that makes AI agents write better TypeScript also makes your pulumi up better. After spending a few weeks with each one, I have opinions about when to use which.

The problem all three frameworks solve

AI coding agents are impressive for the first 30 minutes. Then things go sideways. The patterns are predictable enough that three separate teams independently built frameworks to fix them.

Context rot. Every LLM has a context window. As that window fills up, earlier instructions fade. You start a session asking for an S3 bucket with AES-256 encryption, proper ACLs, and access logging. Two hours and 200K tokens later, the agent creates a new bucket with none of those requirements. The context window got crowded and your original instructions lost weight.

No test discipline. Agents write code that looks plausible. Plausible code compiles. Plausible code even runs, for a while. But plausible code without tests is a liability. The agent adds a feature and quietly breaks two others because nothing verified the existing behavior was preserved.

Scope drift. You ask for a VPC with three subnets. The agent decides you also need a NAT gateway, a transit gateway, a VPN endpoint, and a custom DNS resolver. Helpful in theory. In practice, you now have infrastructure you never requested and barely understand. You will also pay for it monthly.

These problems are not specific to Claude Code or any particular agent. They happen with Cursor, Codex, Windsurf, and every other LLM-powered coding tool. The context window does not care which brand name is on the wrapper.

Superpowers: the test-driven discipline enforcer

Superpowers was created by Jesse Vincent and has accumulated over 149K GitHub stars. The core idea is simple: no production code gets written without a failing test first.

The framework enforces a 7-phase workflow. Brainstorm the approach. Write a spec. Create a plan. Write failing tests (TDD). Spin up subagents to implement. Review. Finalize. Every phase has gates. You cannot skip ahead. The iron law is that production code only exists to make a failing test pass.

This sounds rigid. It is. That is the point.

Superpowers includes a Visual Companion for design decisions, which helps when you are making architectural choices that need visual reasoning. The main orchestrator manages the entire workflow from a single context window, delegating implementation work to subagents that run in isolation.

The tradeoff is that the mega-orchestrator pattern means the orchestrator itself can hit context limits on very long sessions. One big brain coordinating everything works well until the big brain fills up. For most projects, this is not an issue. For marathon sessions with dozens of files, keep it in mind.

The workflow breaks down into skills that trigger automatically:

Skill	Phase	What it does
`brainstorming`	Design	Refines rough ideas through Socratic questions, saves design doc
`writing-plans`	Planning	Breaks work into 2-5 minute tasks with exact file paths and code
`test-driven-development`	Implementation	RED-GREEN-REFACTOR: failing test first, minimal code, commit
`subagent-driven-development`	Implementation	Dispatches fresh subagent per task with two-stage review
`requesting-code-review`	Review	Reviews against plan, blocks progress on critical issues
`finishing-a-development-branch`	Finalize	Verifies tests pass, presents merge/PR/keep/discard options

The results speak for themselves. The chardet maintainer used Superpowers to rewrite chardet v7.0.0 from scratch, achieving a 41x performance improvement. Not a 41% improvement. 41 times faster. That is what happens when every code change has to pass a test: the agent optimizes aggressively because it has a safety net.

Superpowers works with Claude Code, Cursor, Codex, OpenCode, GitHub Copilot CLI, and Gemini CLI.

GSD: preventing context rot before it ruins your project

GSD (Get Shit Done) was created by Lex Christopherson and has over 51K stars. Where Superpowers focuses on test discipline, GSD attacks the context window problem directly.

The key architectural decision: GSD does not use a single mega-orchestrator. Instead, it assigns a separate orchestrator to each phase of work. Each orchestrator stays under 50% of its context capacity. When a phase completes, the orchestrator writes its state to disk as Markdown files, then a fresh orchestrator picks up where the last one left off.

Think about why this matters. With a single orchestrator, your 200K token context window is a shared resource. Instructions from hour one compete with code from hour three. GSD sidesteps this entirely. Every phase starts with a full context budget because the previous phase’s orchestrator handed off cleanly and shut down.

The state files use XML-formatted instructions because (it turns out) LLMs parse structured XML more reliably than freeform Markdown. GSD also includes quality gates that detect schema drift and scope reduction. If the agent starts cutting corners or wandering from the plan, the gates catch it.

GSD evolved from v1 (pure Markdown configuration) to v2 (TypeScript SDK), which tells you something about the level of engineering behind it. The v2 SDK gives you programmatic control over orchestration, not just static instruction files.

The tradeoff: GSD has more ceremony than the other two frameworks. For a quick script or a single-file change, the phase-based workflow is overkill. GSD earns its keep on projects that span multiple files, multiple sessions, or multiple days.

The core commands map to a phase-based workflow:

Command	What it does
`/gsd-new-project`	Full initialization: questions, research, requirements, roadmap
`/gsd-discuss-phase`	Capture implementation decisions before planning starts
`/gsd-plan-phase`	Research, plan, and verify for a single phase
`/gsd-execute-phase`	Execute all plans in parallel waves, verify when complete
`/gsd-verify-work`	Manual user acceptance testing
`/gsd-ship`	Create PR from verified phase work with auto-generated body
`/gsd-fast`	Inline trivial tasks, skips planning entirely

GSD supports the widest range of agents: 14 and counting. Claude Code, Cursor, Windsurf, Codex, Copilot, Gemini CLI, Cline, Augment, Trae, Qwen Code, and more.

GSTACK: when you need a whole team, not just an engineer

GSTACK was created by Garry Tan (CEO of Y Combinator) and has over 71K stars. It takes a fundamentally different approach from the other two frameworks.

Instead of disciplining a single agent, GSTACK models a 23-person team. CEO, product manager, QA lead, engineer, designer, security reviewer. Each role has its own responsibilities, its own constraints, and its own slice of the problem.

The framework enforces five layers of constraint. Role focus keeps each specialist in their lane. Data flow controls what information passes between roles. Quality control gates ensure standards at handoff points. The “boil the lake” principle means each role finishes what it can do perfectly and skips what it cannot, rather than producing mediocre work across everything. And the simplicity layer pushes back against unnecessary complexity.

The role isolation is what makes GSTACK distinctive. The engineer role does not see the product roadmap. The QA role does not see the implementation details. Each role only receives the context it needs to do its job. This is not just about efficiency. It prevents the kind of scope creep where an agent that knows everything tries to do everything.

“Boil the lake” is my favorite principle across all three frameworks. It is the opposite of how most agents work. Agents default to attempting everything and producing something mediocre. GSTACK says: do fewer things, but do them right.

The tradeoff: 23 specialist roles feels heavy for pure infrastructure work. If you are writing Pulumi programs and deploying cloud resources with component resources, you probably do not need a product manager role or a designer role. GSTACK shines when you are building a product, not just provisioning infrastructure.

Each slash command activates a different specialist:

Command	Role	What it does
`/office-hours`	YC partner	Six forcing questions that reframe your product before you write code
`/plan-ceo-review`	CEO	Four modes: expand scope, selective expand, hold, reduce
`/plan-eng-review`	Engineering manager	Lock architecture, map data flow, list edge cases
`/review`	Staff engineer	Find bugs that pass CI but break in production, auto-fix the obvious ones
`/qa`	QA lead	Real Playwright browser testing, not simulated
`/ship`	Release engineer	One-command deploy with coverage audit
`/cso`	Security officer	OWASP and STRIDE security audits

GSTACK works with Claude Code, Codex CLI, OpenCode, Cursor, Factory Droid, Slate, and Kiro.

Where each framework fits

	Superpowers	GSD	GSTACK
What it locks down	The dev process itself	The execution environment	Who decides what
Orchestration	Single orchestrator	Per-phase orchestrators	23 specialist roles
Context management	One window	State-to-disk, fresh per phase	Role-scoped handoffs
Where it shines	TDD, subagent delegation, disciplined plan execution	Marathon sessions, parallel workstreams, crash recovery	Product strategy, multi-perspective review, real browser QA
Where it struggles	Anything beyond the build phase	Overkill for small tasks, no role separation	The actual writing-code part
Best for	Solo devs who need test discipline	Complex projects that span days or weeks	Founder-engineers shipping a product
GitHub stars	149K	51K	71K
Agent support	6 agents	14+ agents	7 agents

For infrastructure work, GSD’s context management matters most. Long Pulumi sessions that provision dozens of resources across multiple stacks are exactly the scenario where context rot bites hardest. GSD’s phase-based approach keeps each orchestrator fresh.

Superpowers’ TDD workflow maps well to application code where unit tests are straightforward. Infrastructure testing is different. You cannot unit test whether an IAM policy actually grants the right permissions. You can test the shape of the policy with Pulumi’s testing frameworks, but the real validation happens at pulumi preview and pulumi up. Superpowers still helps here (discipline is discipline), but the TDD cycle is less natural for infra than for app code.

GSTACK shines when the project has product dimensions. If you are building a SaaS platform where the infrastructure serves a product vision, GSTACK’s multi-role governance keeps the product thinking connected to the engineering work. For pure infra provisioning, the extra roles add overhead without much benefit.

My honest take: none of these is universally best. Knowing your failure mode is the real decision.

What keeps going wrong	Try this	The reason
Code works today, breaks tomorrow	Superpowers	Forces every change through a failing test first
Quality drops after the first hour	GSD	Fresh context per phase, nothing carries over
You ship features nobody asked for	GSTACK	Product review before engineering starts
All of the above	GSTACK for direction, bolt on Superpowers TDD	No single framework covers everything yet

Combining frameworks with Pulumi workflows

These frameworks solve the “how” of agent orchestration. Skills (like the ones from Pulumi Agent Skills) solve the “what,” teaching agents the right patterns for specific technologies. Frameworks and skills complement each other. A skill tells the agent to use OIDC instead of hardcoded credentials. A framework makes sure the agent still remembers that instruction 200K tokens later.

GSD’s state-to-disk approach pairs naturally with Pulumi stack outputs. Each phase can read the previous phase’s stack outputs from the state files, so a networking phase can provision a VPC and the compute phase can reference the subnet IDs without any context window gymnastics.

Superpowers’ TDD cycle maps to infrastructure validation. Write a failing test (the expected shape of your infrastructure). Run pulumi preview (red, the resources do not exist yet). Run pulumi up (green, the infrastructure matches the test). This is not a perfect analogy since infrastructure tests are broader than unit tests, but the discipline of “verify before moving on” translates directly.

You do not have to pick one framework and commit forever. Try GSD for a long multi-stack project. Try Superpowers for a focused library. See which failure mode bites you most and let that guide your choice.

Getting started

github.com/obra/superpowers

github.com/gsd-build/get-shit-done

github.com/garrytan/gstack

All three frameworks support multiple agents. For Claude Code, the install commands are straightforward:

# Superpowers
/plugin install superpowers@claude-plugins-official

# GSD (the installer asks which agents and whether to install globally or locally)
npx get-shit-done-cc@latest

# GSTACK
git clone --single-branch --depth 1 https://github.com/garrytan/gstack.git ~/.claude/skills/gstack && cd ~/.claude/skills/gstack && ./setup

Check each repository’s README for Cursor, Codex, Windsurf, and other agents.

If you want a managed experience that handles orchestration for you, Pulumi Neo is grounded in your actual infrastructure, not internet patterns. It understands your stacks, your dependencies, and your deployment history. The 10 things you can do with Neo post shows what that looks like in practice.

Pick one and give it a project. You will know within an hour whether it fixes your particular failure mode.

Try Pulumi for Free

Introducing the pulumi policy analyze Command for Existing Stacks

Fraser Waters — Fri, 03 Apr 2026 00:00:00 +0000

You can now run policy packs against your existing stack state without running your Pulumi program or making provider calls. The new pulumi policy analyze command evaluates your current infrastructure against local policy packs directly, turning policy validation into a fast, repeatable check.

Why this command matters

Policy authoring and policy updates usually involve an iteration loop:

Make a policy change.
Run a policy check.
Inspect violations or remediations.
Repeat until the policy behavior matches intent.

Before this command, that loop often depended on pulumi preview or pulumi up, which can be heavier than you need when your goal is validating policy logic against known state.

With pulumi policy analyze, you can evaluate your current stack state directly and quickly.

Basic usage

At minimum, provide a policy pack path and optionally a stack:

pulumi policy analyze \
 --policy-pack ./policy-pack \
 --stack dev

You can also pass a config file for each policy pack:

pulumi policy analyze \
 --policy-pack ./policy-pack \
 --policy-pack-config ./policy-config.dev.json \
 --stack dev

If any mandatory policy violations are found, the command exits non-zero.

If remediation policies fire, those changes are reported in output, but stack state is not modified.

Testing new policy packs as a developer

For policy pack development, this command is useful as a tight local feedback loop:

Pick a representative stack (dev, staging, or a fixture stack).
Run pulumi policy analyze against that stack after each policy change.
Use the output to verify mandatory, advisory, and remediation behavior.
Repeat before publishing the policy pack or attaching it to broader policy groups.

Two output modes are especially useful:

--diff for a concise, human-readable view while iterating locally.
--json for structured output that can be consumed in scripts and CI.

Using it in AI and agent workflows

This command is also a good primitive for AI-assisted policy workflows.

Because pulumi policy analyze can emit JSON and a clear process exit code, agents can use it for deterministic policy evaluation steps:

Propose or edit policy rules.
Run pulumi policy analyze --json against target stacks.
Parse violations and remediation signals.
Suggest policy fixes, config adjustments, or targeted infrastructure changes.
Re-run analysis until mandatory violations are resolved.

For example, an agent tasked with fixing a policy violation can run pulumi policy analyze --json to get a structured list of violations, identify which resources are non-compliant, generate targeted infrastructure changes, then re-run analysis to confirm the violations are resolved, all without triggering a full preview on each iteration. The same loop works for policy authoring: an agent can propose a new policy rule, test it against several representative stacks, and surface unintended violations before the rule is published.

This works well for automation because the command doesn’t execute your Pulumi program or make provider calls, so there are no side effects or runtime variance between runs. The JSON output and non-zero exit code on failure give agents a clear pass/fail contract to build on.

Try it out

pulumi policy analyze is available in Pulumi v3.229.0.

If you are authoring or tuning policy packs, start by running this command against a known stack in your environment. It is a quick way to validate policy behavior before rollout.

For implementation details, see the merged PR: pulumi/pulumi#22250.

Get started with policy as code

KubeCon EU 2026 Recap: The Year AI Moved Into Production on Kubernetes

Engin Diri — Wed, 01 Apr 2026 00:00:00 -0700

Amsterdam in late March still has that sharp North Sea wind, but inside the RAI Convention Centre, 13,350 people generated enough energy to heat the building twice over. KubeCon + CloudNativeCon EU 2026 was the biggest European edition yet, and the shift from previous years was impossible to miss. AI dominated the conference.

I spent most of the conference at the Pulumi booth, and that turned out to be the best vantage point. Hundreds of visitors stopped by over four days, and I kept asking the same question: what are you actually running in production with AI on Kubernetes? The answers shaped this post more than any keynote did. Almost everyone had a proof of concept. Almost nobody had a production story they were happy with.

Here is the stat that framed the entire conference for me: 66% of organizations use Kubernetes to host generative AI workloads, but only 7% deploy to production daily. That gap between experimentation and actual production use matched what I was hearing at the booth. The CNCF’s own survey now counts 19.9 million cloud native developers worldwide, 7.3 million of them building AI workloads. The tooling and the infrastructure need to catch up.

My takeaway after four days on the ground: lots of working demos, very few production setups people trust. Teams are trying to scale inference, put guardrails around agents, and make GPU infrastructure behave like anything else they run.

Here is what I saw.

From training to inference: the big pivot

About 67% of AI compute now goes to inference, not training. The inference market is projected to hit $255 billion by 2030. It’s also where most of the operational complexity lives.

NVIDIA leaned into this hard. Their open-source stack around NeMo and Dynamo got significant stage time, but the bigger move was donating three projects to the CNCF: the DRA driver for fractional GPU allocation, the KAI Scheduler for GPU-aware scheduling, and Grove. Moving these to community governance signals that GPU infra is becoming part of the standard Kubernetes toolkit.

The CNCF donations that will reshape AI on Kubernetes

Every KubeCon has its crop of new CNCF projects, but this year’s batch felt different. We are starting to see the building blocks of an AI runtime for Kubernetes.

llm-d was the headline donation. Created by IBM Research, Red Hat, and Google Cloud, it splits inference workloads by separating prefill and decode phases across different pods. The collaborator list reads like an industry consortium: NVIDIA, CoreWeave, AMD, Cisco, Hugging Face, Intel, Lambda, Mistral AI, UC Berkeley, and UChicago. When that many organizations agree on a single approach to distributed inference, pay attention.

NVIDIA’s DRA driver enables fractional GPU allocation and multi-node NVLink support. GPU multi-tenancy is one of the hardest unsolved problems in Kubernetes right now. Scheduling, isolation, cost attribution — all of it breaks down when multiple workloads share a GPU. The DRA driver does not solve everything, but it gives the community a real starting point.

KAI Scheduler entered the CNCF Sandbox for GPU-aware scheduling. If llm-d handles the inference runtime and the DRA driver handles allocation, KAI Scheduler handles placement. Together, these three projects form the skeleton of a GPU-native Kubernetes stack.

Velero, donated by Broadcom, moved into CNCF Sandbox for backup and restore. AI workloads are stateful now (model weights, checkpoints, fine-tuning data), and backup is no longer optional. Good timing.

Microsoft AI Runway is an open-source Kubernetes API for inference that plugs in Hugging Face model discovery, GPU memory fit calculations, and cost estimates. Think of it as a model-aware control plane. HolmesGPT and Dalec, also from Microsoft, entered CNCF Sandbox for AI-powered troubleshooting and dependency analysis.

The Kubernetes AI Conformance Program is growing fast, with certifications nearly doubled and three new requirements proposed for Kubernetes 1.36. Conformance programs are boring until they are not. This one will determine which distributions can credibly claim AI readiness.

Agentic AI gets an identity layer

If inference was this year’s production story, agentic AI was the architecture story. Agents are proliferating, and nobody has quite figured out how to manage and secure them inside Kubernetes yet.

kagent, donated to CNCF Sandbox by Solo.io, defines agents as Kubernetes CRDs. It ships with pre-built MCP (Model Context Protocol) servers for Kubernetes, Istio, Helm, Argo, Prometheus, Grafana, and Cilium. An agent becomes a first-class Kubernetes resource, schedulable and observable and subject to RBAC, instead of a rogue process running in someone’s notebook.

kagenti from IBM goes after the identity problem directly. Using SPIFFE/SPIRE, it gives agents cryptographic identities. When an agent calls an API, you can verify exactly which agent made the call, what trust domain it belongs to, and whether it is authorized. This kind of security work needs to happen before agents proliferate across production clusters. Retrofitting identity later is ugly.

Dapr Agents took a different angle with the actor model and durable execution. Each agent gets reliable state management and exactly-once messaging semantics. If your workflows cannot tolerate lost messages or duplicate actions, this matters.

agentregistry showed up as a centralized discovery service for MCP servers and agents. As agents and tool servers multiply, you need a registry to find and manage them, the same way container registries became necessary for images.

David Soria Parra from Anthropic gave a talk on MCP evolving beyond simple tool-calling into richer interaction patterns (sched). Google announced the Kubernetes Agent Sandbox for running agentic AI workloads in secure, isolated environments.

AI gateways and inference routing

Gateway infrastructure had its own mini-conference within KubeCon. The Gateway API Inference Extension from the Kubernetes SIG introduces model-aware routing and load balancing at the gateway level. Instead of routing by URL path, your gateway routes by model name, version, and capacity. That changes how inference traffic flows through a cluster in a fundamental way.

Envoy AI Gateway builds on Envoy’s existing proxy capabilities with token-aware rate limiting and provider failover. If your primary inference provider is saturated, traffic shifts to a secondary automatically. Rate limiting by token count rather than request count makes much more sense for LLM workloads, where a single request can consume vastly different amounts of compute.

I want to call out Agentgateway specifically. Written in Rust, it proxies LLM traffic, MCP connections, and agent-to-agent communication, with Cedar and CEL policy engines for fine-grained access control. Rust’s performance characteristics matter here because inference gateway latency adds directly to user-perceived response time.

Kuadrant, now in CNCF Sandbox, layers policy on top of gateway infrastructure and includes MCP server aggregation. Gateways are evolving from dumb traffic proxies into intelligent control planes for AI workloads, and these four projects are driving that shift.

Platform engineering absorbs LLMOps

The observability and platform engineering vendors showed up in force. The message was consistent: LLMOps is just platform engineering with new requirements.

Chronosphere demonstrated parallel AI investigation, with multiple agents analyzing different aspects of an incident simultaneously and combining their findings. SUSE Liz takes a domain-specialized approach, deploying different AI agents for different operational domains rather than one general-purpose assistant. groundcover combines eBPF with OpenTelemetry to give coding agents rich runtime context about the systems they are modifying. That last one is subtle but important: if an AI agent is writing code that touches a service, it should understand that service’s actual runtime behavior, not just its source code.

Dynatrace and DevCycle partnered to make feature flags observable primitives via OpenFeature. Rolling out AI features behind feature flags is table stakes, but having those flags show up in your observability pipeline as first-class signals closes a real gap.

Shadow AI governance emerged as its own theme. CAST AI’s Kimchi can route requests across 50+ models while providing centralized visibility into what models are being used, by whom, and at what cost. Every large organization I talked to had some version of the same problem: teams spinning up model endpoints without central oversight, burning through GPU budgets, creating compliance blind spots they did not even know about.

GPU multi-tenancy remains genuinely unsolved. Scheduling, workload isolation, cost attribution across shared GPUs — all of it breaks down at scale. Multiple talks addressed pieces of this, but nobody had a complete answer.

Sovereignty shapes infrastructure architecture

Regulation came up in almost every conversation. The EU Cyber Resilience Act is driving compliance requirements deep into software supply chains, and every European organization I spoke with is feeling the pressure. Teams are already changing how they build and deploy software.

Sovereign Kubernetes is a platform architecture requirement now, not something you can defer to next quarter. Organizations need Kubernetes distributions and cloud regions that guarantee data residency, and they need the tooling to enforce those guarantees programmatically. Self-hosted models are proliferating partly because of capability and cost, but data sovereignty is the accelerant. If your data cannot leave a jurisdiction, neither can your model.

Runtime isolation is expanding beyond containers. Several talks covered KVM-based isolation for AI workloads, which is heavier than containers but necessary when the threat model includes side-channel attacks on shared GPU memory. The sandboxing conversation has gotten more sophisticated since last year.

These constraints are not uniquely European. Any organization operating across jurisdictions faces similar pressures, and the regulatory direction globally is toward more data sovereignty requirements, not fewer.

What this means for your team

Four days in Amsterdam distilled into five things I would act on now:

Treat inference workloads like production services. If you are still deploying models with scripts and hope, stop. Inference infrastructure needs the same IaC discipline as any other production system: version-controlled, tested, policy-enforced.
Evaluate the Gateway API Inference Extension and llm-d. These are not speculative projects. They have broad industry backing and solve real problems around inference routing and distributed serving. Get them into your test environments.
Plan agent identity before agents proliferate. SPIFFE/SPIRE for agent identity is not optional if you are running agents in production. Retrofitting identity onto an existing agent fleet is painful. Start with kagenti now.
Platform teams should own AI infrastructure. Shadow AI is already happening in your organization. The platform engineering team needs to provide self-service AI infrastructure with guardrails before ungoverned model endpoints become a security and cost problem.
Sovereignty and GPU multi-tenancy are universal. Even if you are not subject to the EU Cyber Resilience Act today, data residency requirements are spreading globally. GPU multi-tenancy will affect every organization running inference at scale.

Kubernetes spent the past decade proving it could orchestrate containers. The next decade will test whether it can orchestrate intelligence. Based on what I saw in Amsterdam, the community is building the right pieces, but the gap between what exists and what production demands is still wide. That gap is where the interesting work happens.

Neo Plan Mode: Iterate Before You Execute

Pulumi Neo Team — Wed, 01 Apr 2026 00:00:00 +0000

Infrastructure work ranges from simple updates to complex multi-stack operations. For straightforward tasks, jumping straight to execution is often fine. But complex tasks benefit from deliberate upfront thinking: understanding what exists, identifying dependencies, and agreeing on an approach before anything changes. Today we’re launching Plan Mode, a dedicated experience for collaborating with Neo on a detailed plan before execution begins.

Plan Mode

Without dedicated planning, Neo balances planning with progress toward execution. That works well for many tasks, but complex operations benefit from more thorough upfront discovery. Plan Mode now makes upfront deliberation a first-class workflow, where instead of focusing on getting to execution, Neo focuses entirely on discovery and synthesis until you explicitly approve the plan.

How it works

Enter Plan Mode by selecting the plan button when starting a task. Neo shifts its behavior:

Discovery: Neo investigates your environment — examining existing infrastructure, reading relevant code, checking dependencies, and researching patterns.
Synthesis: From that research, Neo produces a plan explaining what it will do and why. The plan references specific things Neo discovered, like a particular stack configuration or dependency.
Refinement: You refine the plan through normal conversation, challenging assumptions, asking for an alternative approach, or requesting more detail on a specific area.
Approval: Once you’re satisfied, you approve the plan and execution begins. Neo carries forward everything it learned during discovery, so the transition from planning to execution is seamless.

When to use it

Plan Mode is opt-in. You choose it when you want to work through an approach before committing:

Complex multi-stack operations where understanding dependencies matters
Unfamiliar infrastructure where discovery reduces churn
Autonomous execution where plan approval is your key control point before Neo runs without step-by-step oversight

Get started

Plan Mode is available now for all Pulumi Cloud organizations. It works with any task mode, so you can pair thorough upfront planning with whatever level of execution autonomy fits the situation.

To try it, open Neo in Pulumi Cloud. For more details, see the Plan Mode documentation.

Introducing Read-Only Mode for Pulumi Neo

Florian Stadler — Wed, 01 Apr 2026 00:00:00 +0000

A platform engineer with broad access might want Neo to analyze infrastructure and suggest changes, but include guarantees it won’t actually apply them. Read-only mode makes that possible: Neo does the heavy lifting and hands off a pull request for your existing deployment process to pick up.

Control what Neo can change

Neo runs with the permissions of the user who creates a task, but you often want a tighter boundary. Read-only mode solves this by letting you cap Neo’s permissions at task creation time. Neo can still read your infrastructure, run previews, and open pull requests, but it cannot deploy, update, or destroy resources.

How it works

When you create a Neo task, you now choose between two permission levels:

Option	What Neo can do	Availability
Use my permissions	Full access (current default behavior)	All tiers
Read-only	Read, preview, and create PRs. No infrastructure mutations.	All tiers

Read-only mode takes your existing permissions and removes the ability to make changes. Neo remains fully active, meaning it can still read your infrastructure state, run previews, write and refactor code, create branches, and open pull requests. If Neo encounters an operation it can’t perform in read-only mode, the operation fails and Neo reports what it would have done. The only difference is that Neo cannot trigger deployments or other write operations in Pulumi Cloud directly.

Read-only mode and auto-approve

Neo’s operating modes let you choose how much oversight you want: review mode for full approval at each step, balanced mode for approving only mutating operations, and auto mode for hands-off execution.

Read-only mode pairs well with auto-approve. Because Neo cannot perform write operations like deployments or destroys, you can let it run autonomously and trust that the output is a pull request, not a production change. Kick off a task, let Neo work in the background, and come back to a ready-to-review PR.

Getting started

Read-only mode is available today for all Pulumi Cloud users.

Sign in to Pulumi Cloud and select Read-only when creating your next Neo task
Read the Neo documentation for detailed guides on permission levels
Join the Community Slack to share your feedback

Treating Prompts Like Code: A Content Engineer's AI Workflow

Cam Soper — Mon, 09 Mar 2026 00:00:00 +0000

Pulumi has a lot of engineers. It has marketers, solution architects, developer advocates. Everyone has something to contribute to docs and blog posts — domain expertise, hard-won lessons, real-world examples. What they don’t all have is familiarity with our Hugo setup, our style guide, our metadata conventions, or where a new document is supposed to live in the navigation tree. I joined Pulumi in July 2025 as a Senior Technical Content Engineer. A few weeks in, my sole teammate departed. The docs practice was now, functionally, me.

The problem was clear enough: how do you take one docs engineer’s accumulated knowledge and make it available to everyone who needs it, without that engineer becoming a bottleneck?

I started packaging it. Here’s what that looked like in practice.

The real problem AI solves

Everyone talks about AI making you faster. That’s not wrong, but it’s not the most interesting part — at least not for me.

The most interesting part is what it does to the starting problem. I have an ADHD brain (not formally diagnosed, but with enough self-recognition to know what’s going on). I know what that means for my relationship with most tasks: I can see the problem, I understand it, I want to fix it, and then the sheer weight of starting crushes me flat.

When I’m stuck on a task, the issue is almost never that I don’t know what to do. It’s that my brain is trying to hold the entire finished product in working memory while simultaneously producing the first step. That’s an enormous cognitive tax, and for an ADHD brain it’s often insurmountable.

Talking through a problem conversationally is a completely different cognitive load. I can tell Claude “here’s the issue, here’s what I’m trying to accomplish, here’s what’s weird about it,” and suddenly I’m not staring at a blank page anymore. I’m in a conversation. The scaffold exists. I can build on it.

That dynamic isn’t new for me. In a previous role writing training modules at Microsoft, I did some of my best work, not because the work was easy, but because I had a collaborator. A friend to think out loud with. Someone to say “okay, so what are we actually trying to say here?” That conversational scaffolding was the difference between spinning and shipping.

In my current role as a team of one, AI turned out to be that collaborator.

This isn’t really a productivity story. It’s closer to a cognitive accommodation story. And I’d bet a lot of people — diagnosed or not — will recognize what I’m describing.

Treating prompts like code

If conversational scaffolding could lower my own activation energy, the next question was obvious: could I build that for anyone who needed it? I knew I wanted to use AI to solve this problem, but I didn’t want to just write a bunch of one-off prompts. That would be a maintenance nightmare, and it wouldn’t scale beyond me. I needed a system. Claude Code calls these reusable prompts skills — other platforms have the same idea under names like plugins or extensions. My first real experiment was /docs-review — a reusable prompt that would run my writing through a consistent set of criteria before I committed it. Nothing fancy. I just wanted a reliable bar that didn’t depend on my mood or how much coffee I’d had.

Then it occurred to me: every PR to our docs repo should get this automatically. So I wired it into our CI/CD pipeline. Meagan, my manager, loved it — and after a few weeks, she noticed that PR quality had improved dramatically. On almost every PR, contributors were now spontaneously pushing an “Addressing feedback” commit right after the automated review posts — catching and fixing issues before I ever saw the PR.

That’s when something clicked: I wasn’t writing prompts anymore. I was writing modules — reusable, composable pieces of my own expertise.

The insight was straightforward, but it changed how I thought about the whole system: if multiple skills need the same context — our style guide, our review criteria, our content standards — that context should live in one place and get consumed by everything that needs it. Just like a shared library. Just like any decent software project.

I created a REVIEW-CRITERIA.md file as the single source of truth for what a “good” docs PR review looks like at Pulumi. Every skill that does any kind of review pulls from it. Change it once, and everything gets smarter at once. Likewise with our style guide, our Hugo conventions, our navigation structure. All of that lives in central reference files that any skill can pull from. If something changes, I change it in one place and all the skills get the update.

This also matters for token efficiency — which sounds like a nerdy footnote but isn’t, especially when automated reviews are running on every PR. Duplicating context across skills bloats token usage fast. Modularizing keeps it lean. Your CI/CD pipeline doesn’t care about elegance, but it definitely cares about cost.

The mental model I kept coming back to: Don’t Repeat Yourself. It’s the same principle that makes good software maintainable. It turns out it makes good AI workflows maintainable too.

The skill catalog

From there, the system grew organically. Whenever I found myself doing something more than once, I asked: “Can I turn this into a skill?” Here’s a sampling of what that produced:

/fix-issue — takes a GitHub issue and recommends a concrete plan of attack, so I go from “here’s a ticket” to “here’s what I’m doing” without the spinning-up tax.

/shipit — runs pre-commit checks, writes a focused commit message, and drafts a PR description.

/pr-review — full doc review on a PR branch: style guide, code examples, screenshots, optional test deployment, then an Approve/Merge/Request Changes dialog with a drafted comment.

/slack-to-issue — converts #docs Slack conversations into properly formed GitHub issues. Slack is where decisions happen; issues are where work gets tracked.

/glow-up — runs an older doc through the modern style guide and flags outdated screenshots, for digging out of accumulated technical debt.

/new-doc and /new-blog-post — guide anyone through adding a new document or blog post with the right location, metadata, and navigation wiring. Engineers, marketers, whoever. The barrier to contributing just dropped significantly.

/docs-tools — helps other repo users discover that any of this exists. Discoverability is a real problem with internal tooling.

Slack’s built-in Claude integration isn’t the same Claude running your Claude Code workflows — they don’t share context or custom instructions. If you want consistent criteria across both surfaces, you need to bring your own backend. That’s exactly what /slack-to-issue handles.

Other people started contributing skills to the repo — not because I asked, but because the pattern was legible enough to extend. Someone built a skill for SEO analysis. Marketing added their own review criteria. Engineers contributed workflows I never would have thought to build.

The thing I’d built as a personal survival tool had become a shared platform. That happened because I treated the prompts like code: modular, reusable, documented, open for contribution.

Honest limitations

It’s not a replacement for human judgment. These are probabilistic tools — they’re right most of the time, not all of the time. /pr-review doesn’t approve PRs autonomously. It highlights things and then asks me, the human, to read them and make the call. The AI does the first pass; I do the last one. That’s not a workaround for a limitation — that’s the design.

The system isn’t finished, either. It’s probably never finished. I’m still tweaking review criteria, still finding edge cases where a skill produces something weird, still adding new tools as new pain points emerge. Treating prompts like code means treating them like software: you ship, you iterate, you maintain. There’s no version 1.0 and done.

And the ADHD angle is real but it’s not magic. There are still days where the paralysis wins. AI lowers the activation energy for starting; it doesn’t eliminate it. I’m still the one who has to show up. I suppose I could automate that too, but then we’d be in a whole different kind of dystopia.

Know your models and their costs. At Pulumi we primarily use Claude, and I work in Claude Code; for most tasks I reach for Sonnet rather than Opus. Opus is excellent, but it’s significantly more expensive, and well-crafted instructions to Sonnet handle the vast majority of my work just as effectively.

Treat it like a coworker. Don’t just issue commands and wait for output. Ask what it thinks. Push back when it’s wrong. Explain your reasoning. The more you engage conversationally, the better the results tend to be. That extends to alignment, too — before diving into a complex task, talk through the approach first. A few minutes of alignment up front beats iterating on a misunderstood spec. I’ve gone as far as adding personal instructions to my config — things like playing along when I’m pretending to be Captain Picard, or using colorful language when the context calls for it. (Yes, those are literal config settings.) That sounds frivolous, but it isn’t: a tool you actually enjoy using is a tool you’ll reach for instead of avoid.

Modularize your workflow. Don’t write one giant monolithic prompt that tries to do everything. Break it into focused skills that do one thing well and share common context through a central reference file. Easier to maintain, easier to debug, cheaper to run.

Version control your prompts. Your skills are code. Treat them like code. Commit them, review them, iterate on them. If a skill starts producing weird output after a tweak, you’ll want to know what changed.

Think about token burn rate. This matters most when running automation in CI/CD. Keep your skills focused — a skill that checks style doesn’t need to load your Hugo navigation conventions. The model only reads what you give it, so give it only what it needs.

Not everything needs to be a prompt. This one is underappreciated: skills can include scripts, and that’s often the right call. When my team moves a doc in the repo, it needs to happen via git mv to preserve history, and we need to add a redirect alias to the front matter to prevent 404s and protect SEO. That’s not something I want an AI to reason through from scratch every time — it’s a solved problem. So it’s a script. The skill just knows the script exists and what it does. Claude orchestrates; the script executes. That’s a cleaner, more reliable division of labor than asking an LLM to reinvent the wheel on every run.

Not everything needs to be generative. Corollary to the last point: if you need deterministic output, don’t use probabilistic tools. We have a skill that generates the meta image for blog posts — procedurally, not generatively. No AI-generated imagery. We have a brand to protect, and “let the AI vibe it out” isn’t a content strategy. The skill follows our visual standards programmatically and produces something consistent every time. Know what you’re automating and why.

What’s next

The next frontier is bringing some of this tooling to the less technical members of the team — marketing, in particular. The skills I’ve built assume a certain comfort level with terminals and repos. That’s fine for engineers. It’s a barrier for everyone else. A friendly interface would lower that bar significantly — that’s the direction I’m currently exploring.

If you’re a technical writer, a developer advocate, or a solo practitioner figuring out how AI fits into your workflow, the approach described here is a solid starting point. The tools matter, but the mental model matters more: treat your prompts like code. Make them reusable. Document them. Share them.

Our docs repo is public, so the skills are there for anyone who wants them. If you’re building something similar, steal freely — or contribute back.

The blank page is still there. It’s just a lot less intimidating when you’ve got a good collaborator and a solid set of tools.

See our docs skills for inspiration

Token Efficiency vs Cognitive Efficiency: Choosing IaC for AI Agents

Engin Diri — Tue, 03 Mar 2026 00:00:00 +0000

When an AI agent writes infrastructure code, two things matter: how compact the output is (token efficiency) and how well the model actually reasons about what it’s writing (cognitive efficiency). HCL produces fewer tokens for the same resource. But does that make it the better choice when agents need to refactor, debug, and iterate? We ran a benchmark across Claude Opus 4.6 and GPT-5.2-Codex to find out.

These two goals pull in opposite directions

You might assume that the language producing fewer tokens is also the one models reason about best. Research into LLM-driven infrastructure generation suggests otherwise.

Where HCL wins on tokens

HCL is declarative and minimal. It requires no imports, no runtime constructs, and no language scaffolding. For simple infrastructure generation, HCL leads to fewer tokens and lower generation cost.

For a straightforward resource definition, HCL gets straight to the point:

resource "aws_s3_bucket" "example" {
 bucket = "my-bucket"
}

Compare that with the Pulumi TypeScript equivalent:

import * as aws from "@pulumi/aws";

const bucket = new aws.s3.Bucket("example", {
 bucket: "my-bucket",
});

The HCL version requires fewer tokens. No import statement, no variable declaration, no constructor syntax. For single-shot generation of simple resources, that compactness matters. But the picture changes once you account for deployability and refactoring.

What the data shows

HCL’s token advantage is real for simple generation. But agents don’t just generate once. They validate, repair failures, and refactor. We built a benchmark that measures the full cycle: an open-source tool that sends identical prompts to Claude Opus 4.6 (claude-opus-4-6) and OpenAI GPT-5.2-Codex (gpt-5.2-codex), requesting both Terraform HCL and Pulumi TypeScript for the same AWS infrastructure (VPC with public and private subnets across 2 AZs, an EC2 instance with security groups for SSH and HTTP, and an RDS PostgreSQL instance with a security group allowing port 5432 only from the EC2 security group, plus all networking: internet gateway, NAT gateway, route tables, and associations). We measured token consumption, cost, and deployability across two scenarios: initial generation and refactoring into reusable components.

Methodology: Temperature 0, 5 runs per combination, randomized execution order. Each generated output goes through a three-stage validation pipeline: formatting (terraform fmt for HCL, prettier for TypeScript), static analysis (terraform validate for HCL, tsc --noEmit for TypeScript), and provider-level validation (terraform plan for HCL, pulumi preview for TypeScript). Both plan and preview check against real AWS provider schemas without creating resources, making their pass rates comparable across formats. If plan/preview fails, the benchmark feeds the error back to the model for one self-repair attempt. At temperature 0, Claude Opus 4.6 produced near-identical outputs across runs (sd=0-4 tokens). GPT-5.2-Codex showed more natural variation (sd=130-165 tokens). With 5 runs per combination the results are directional, not statistically conclusive. Costs are estimates based on published pricing as of 2026-02-22. Full methodology and reproducible code:

github.com/dirien/iac-token-benchmark

Scenario 1: Generation

HCL uses fewer tokens for generation:

Provider	Format	Output tokens (mean)	LOC (mean)	Cost (mean)	Plan/Preview pass	Repairs needed
Claude Opus 4.6	Terraform	2,007	212	$0.051	5/5	0/5
Claude Opus 4.6	Pulumi TS	2,555	200	$0.065	5/5	0/5
GPT-5.2-Codex	Terraform	1,565	110	$0.022	2/5	2/5
GPT-5.2-Codex	Pulumi TS	2,322	147	$0.033	0/5	5/5

HCL produces 21-33% fewer output tokens across both models. For simple resource generation, this translates directly to lower cost. Pulumi TypeScript uses more tokens for fewer lines of code because imports, type annotations, and constructor syntax add tokens without adding functional lines.

The Plan/Preview column tells a more complete story. Claude Opus 4.6 produced deployable code on the first pass for both formats: 5/5 for Terraform and 5/5 for Pulumi. Neither needed repairs. GPT-5.2-Codex struggled with both formats, but Terraform fared slightly better (2/5 vs 0/5).

Scenario 2: Refactoring into reusable components

We took each model’s generation output and asked it to refactor the code into a reusable module or component with parameterized environment name, instance sizes, and availability zone count.

Provider	Format	Output tokens (mean)	LOC (mean)	Cost (mean)	Plan/Preview pass
Claude Opus 4.6	Pulumi TS	2,720	218	$0.082	5/5
Claude Opus 4.6	Terraform	3,379	345	$0.095	5/5
GPT-5.2-Codex	Pulumi TS	2,477	248	$0.038	4/5
GPT-5.2-Codex	Terraform	1,356	119	$0.021	0/5

This is where the results get interesting. Opus + Pulumi refactoring used 20% fewer tokens, cost 14% less, and passed pulumi preview on every run (5/5) with zero repairs. Opus + Terraform also ended up at 5/5 for terraform plan, but it needed repair cycles to get there. The benchmark run log tells the story:

# Pulumi refactoring: runs 29-33, sequential, no gaps = zero repairs
✓ [29/40] anthropic/pulumi-ts/refactor run 1 — 2721 tokens, $0.0817
✓ [30/40] anthropic/pulumi-ts/refactor run 2 — 2721 tokens, $0.0817
✓ [31/40] anthropic/pulumi-ts/refactor run 3 — 2721 tokens, $0.0817
✓ [32/40] anthropic/pulumi-ts/refactor run 4 — 2721 tokens, $0.0817
✓ [33/40] anthropic/pulumi-ts/refactor run 5 — 2714 tokens, $0.0816

# Terraform refactoring: 34→36→38→40→42, every run skips a number = every run triggered self-repair
✓ [34/40] anthropic/terraform/refactor run 1 — 3388 tokens, $0.0956
✓ [36/40] anthropic/terraform/refactor run 2 — 3339 tokens, $0.0944
✓ [38/40] anthropic/terraform/refactor run 3 — 3390 tokens, $0.0957
✓ [40/40] anthropic/terraform/refactor run 4 — 3388 tokens, $0.0956
✓ [42/40] anthropic/terraform/refactor run 5 — 3388 tokens, $0.0956

The skipped numbers (35, 37, 39, 41) are self-repair turns where the model received terraform plan errors and regenerated the code. Each repair consumed additional tokens that do not show up in the first-generation cost but do show up in the total pipeline cost.

GPT-5.2-Codex tells a different story. Both formats needed repair on every run, but what happened after repair is what matters:

# Codex + Terraform refactoring: repaired, but still failed plan (0/5 deployable)
terraform run 1 turn 2: plan_valid=False tokens=1559
terraform run 2 turn 2: plan_valid=False tokens=1165
terraform run 3 turn 2: plan_valid=False tokens=1068
terraform run 4 turn 2: plan_valid=False tokens=1134
terraform run 5 turn 2: plan_valid=False tokens=1101

# Codex + Pulumi refactoring: repaired, and preview passed (4/5 deployable)
pulumi-ts run 1 turn 2: plan_valid=True tokens=2246
pulumi-ts run 2 turn 2: plan_valid=True tokens=2567
pulumi-ts run 3 turn 2: plan_valid=True tokens=2187
pulumi-ts run 4 turn 2: plan_valid=True tokens=2531
pulumi-ts run 5 turn 2: plan_valid=False tokens=2647

Codex + Terraform used fewer tokens but produced zero deployable refactored code after repair. Codex + Pulumi used more tokens but recovered to deployable code 4 out of 5 times. TypeScript’s type errors gave the model enough information to fix the problems. HCL’s plan errors did not.

Total pipeline cost

This is the number that matters for production agent workflows. It includes generation, any self-repair cycles, and refactoring:

Provider	Format	Total tokens (mean)	Total cost (mean)
Claude Opus 4.6	Pulumi TS	8,183	$0.146
Claude Opus 4.6	Terraform	14,669	$0.249
GPT-5.2-Codex	Terraform	8,723	$0.084
GPT-5.2-Codex	Pulumi TS	15,211	$0.138

Opus + Pulumi had the lowest total pipeline cost at $0.146, 41% cheaper than Opus + Terraform at $0.249. The difference comes entirely from repair cycles: Pulumi needed zero repairs across both scenarios, while Terraform refactoring triggered self-repair on every run.

With Codex, Terraform had the lower pipeline cost ($0.084 vs $0.138), driven by its smaller token output. But Codex + Terraform produced zero deployable refactored code (0/5 plan pass), while Codex + Pulumi produced deployable code 4 out of 5 times.

What this means

HCL uses fewer tokens per generation. For single-shot resource creation, HCL’s compactness saves 21-33% on output tokens. That advantage is consistent across both models.
Pulumi produces deployable refactored code more reliably. With Opus, Pulumi refactoring passed preview 5/5 with zero repairs. Codex + Pulumi passed 4/5. Codex + Terraform passed 0/5.
Total pipeline cost favors Pulumi with Opus. Opus + Pulumi cost 41% less than Opus + Terraform across the full pipeline ($0.146 vs $0.249), because Terraform refactoring needed repair cycles that Pulumi did not.
The tradeoff depends on your model and workflow. Codex + Terraform is cheapest on raw tokens but produces no deployable refactored code. Codex + Pulumi costs more per token but actually deploys. Opus + Pulumi is the best of both: fewer refactoring tokens and zero repairs.

Why Pulumi refactoring deploys cleaner

The TerraFormer project identified what they call the correctness-congruence gap: LLMs generate configurations that look valid but fail to match the user’s architectural intent. An error taxonomy study cataloged the same pattern across models. A survey of LLMs for IaC found the gap between syntax validity and architectural correctness widens with infrastructure complexity.

Refactoring is where this gap bites hardest. Turning a flat resource list into a parameterized, reusable module requires the model to restructure dependencies, introduce variables, and compose abstractions. With Pulumi, the model can use TypeScript’s standard refactoring patterns: extract a class, add typed constructor parameters, compose functions. These are patterns it has practiced across millions of repositories during training. With HCL, the same refactoring requires count, for_each, dynamic blocks, and module variable plumbing, domain-specific constructs that have far less representation in training data.

Our benchmark confirms this directly. Opus produced 2,720 tokens for Pulumi refactoring versus 3,379 for Terraform, a 20% reduction, and every Pulumi run passed pulumi preview without repair. The Terraform refactoring runs all triggered self-repair because the restructured HCL modules had issues that terraform plan caught.

Training data distribution makes this structural. LLMs have far more TypeScript than HCL in their corpora. A model refactoring TypeScript draws on patterns from the entire open-source ecosystem. A model refactoring HCL modules has a much smaller pool. Since general-purpose languages dominate new code production, this gap will widen over time.

Tooling can close the gap further. The Pulumi MCP server gives AI agents direct access to resource schemas at generation time. A tool like get-resource returns every property, type, and required field for a given cloud resource. The agent does not have to guess from what it memorized during training. It can look up the correct schema before writing a single line of code.

This changes the workflow from “generate, fail, read error, retry” to “look up schema, generate correctly.” Agent skills push this further by encoding working Pulumi idioms as structured prompts, so the model starts from a known-good baseline. Terraform has no equivalent to this MCP-based schema lookup. That difference matters more with every iteration.

Where the industry is heading

One way to think about IaC language choice is through the lens of AI engineering maturity levels. At Level 3 (agentic coding), agents generate infrastructure from prompts. HCL’s 21-33% token savings on generation matters here. At Levels 4-5, agents iterate on specifications, refactor code, and maintain systems over time. Our benchmark shows this is where Pulumi pulls ahead: 41% lower total pipeline cost with Opus, and more deployable refactored output with both models.

The industry is moving toward Levels 4-5. Agents are taking on refactoring, feature flags, environment parameterization. 43% of DevOps teams need four or more deployment iterations before infrastructure is production-ready. The first-generation token advantage HCL holds applies to the task that is shrinking as a share of agent workloads. The refactoring and deployability advantages that Pulumi offers apply to the tasks that are growing.

Choosing based on your workflow

If your agents primarily generate well-scoped resource definitions, HCL saves 21-33% on output tokens. That advantage is real and consistent.

If your agents need deployable output on the first pass (which avoids repair costs entirely), our data shows Opus + Pulumi is the strongest combination: 5/5 plan/preview pass for both generation and refactoring, zero repairs, lowest total pipeline cost.

If your agents evolve infrastructure over time through refactoring and modularizing, Pulumi produced deployable refactored code more reliably across both models we tested (5/5 and 4/5 vs 5/5 and 0/5 for Terraform).

Pulumi covers the full iteration loop, from generation through repair to refactoring, using the same language patterns and tooling that models already know.

Get started

Ready to explore how AI agents work with Pulumi? Check out Pulumi AI to see LLM-powered infrastructure generation in action, or get started with Pulumi’s documentation.

Join the conversation in the Pulumi Community Slack or Pulumi Community Discussions.

How We Built Platybot: An AI-Powered Analytics Assistant

Pablo Seibelt — Wed, 11 Feb 2026 00:00:00 +0000

Before Platybot, our #analytics Slack channel was a support queue. Every day, people from every team would ask questions: “Which customers use feature X?”, “What’s our ARR by plan type?”, “Do we have a report for template usage?” Our two-person data team was a bottleneck.

Our #analytics channel, before Platybot (dramatized).

We didn’t want to just throw an LLM at our Snowflake warehouse either. Without guardrails, large language models generate SQL that may work but silently gets the answer wrong. Different join logic, wrong filters, missing snapshot handling, incorrect summarization. We needed something that could answer reliably for most queries, otherwise we’d switch to fixing LLM SQL queries.

So we built Platybot (platypus + bot, named after our mascot), an AI-powered analytics assistant that any Pulumi employee can use to query our Data Warehouse in natural language. It’s available as a Web App, a Slack bot, and a Model Context Protocol (MCP) server. The infrastructure is deployed with Pulumi IaC (Infrastructure as Code). But the most important thing we learned building it is that the AI was the easy part. The semantic layer is what makes it work.

The problem with throwing AI at your Data Warehouse

The naive solution is obvious: connect an LLM to your database and let it write SQL. But this fails in practice, and the failure mode is insidious. Consider a few examples from our warehouse:

ARR is a snapshot metric. If you query ARR (Annual Recurring Revenue) without filtering by end-of-period dates (last day of month or quarter), you get duplicate rows and wildly inflated numbers. An LLM doesn’t automatically know this.
Account queries need exclusions. Most queries should exclude Pulumi’s own internal accounts and deleted ones. Without these filters, you’re counting test data alongside real customers.
User queries need employee filters. Querying active users without excluding Pulumi employees inflates adoption metrics.

The danger shows up when results feel decision-ready before anyone has validated how the numbers were derived. A confidently wrong ARR figure presented to leadership is worse than no answer at all.

Many organizations run into the same constraint once data usage spreads. Dashboards answer yesterday’s questions, not today’s. Ad-hoc LLM queries answer today’s questions, but incorrectly. The gap between “I have a question” and “I have a trustworthy answer” is where data teams can get stuck.

Why we built a semantic layer first

Before writing a single line of AI code, we built a semantic layer using Cube (open source). This was the hardest, least glamorous, and most important part of the entire project.

A semantic layer is a shared, versioned definition of what your business metrics mean.

“Monthly active users” starts with COUNT(DISTINCT user_id), but the aggregation is only the outer layer. It has to be applied to the right table (fct_pulumi_operations), with the right filters (exclude Pulumi employees, exclude deleted organizations), scoped to a calendar month, and counting only users who performed real operations — not just previews. The semantic layer encodes all of this once, and the AI can use it to build queries without needing to guess which tables are related to each other, whether it’s one-to-one or many-to-many, etc.

We organized our data into seven domains: Revenue, Cloud, Core, Clickstream, Community, Support, and People. Each domain contains cubes (think of them as well-defined, composable views) with explicit measures, dimensions, and joins. Here’s a real example from our Cloud domain (trimmed for readability):

cubes:
 - name: fct_pulumi_operations
 sql_table: CLOUD.FCT_PULUMI_OPERATIONS
 description: >
 Pulumi CLI operations (update, preview, destroy, refresh).
 Each row is one operation with resource changes, duration,
 and CLI environment.

 joins:
 - name: dim_organization
 sql: '{CUBE}.ORGANIZATION_HK = {dim_organization}.ORGANIZATION_HK'
 relationship: many_to_one
 - name: dim_stacks
 sql: '{CUBE}.STACK_PROGRAM_HK = {dim_stacks}.STACK_PROGRAM_HK'
 relationship: many_to_one
 - name: dim_user
 sql: '{CUBE}.USER_HK = {dim_user}.USER_HK'
 relationship: many_to_one

 measures:
 - name: count
 type: count
 - name: resource_count
 sql: '{CUBE}.RESOURCE_COUNT'
 type: sum
 description: Number of resources active when the operation finished
 - name: operations_succeeded
 sql: "CASE WHEN {CUBE}.OPERATION_STATE = 'succeeded' THEN 1 ELSE 0 END"
 type: sum
 description: Count of operations that succeeded
 # ... plus other measures

 dimensions:
 - name: operation_type
 sql: '{CUBE}.OPERATION_TYPE'
 type: string
 description: Type of operation (Refresh, Update, Destroy, etc)
 - name: operation_state
 sql: '{CUBE}.OPERATION_STATE'
 type: string
 description: Last known state of this operation
 # ... 20+ more dimensions

The key insight: the semantic layer makes the AI’s job tractable. Instead of generating arbitrary SQL from scratch, where the search space is “any possible SQL query against hundreds of tables,” the AI picks from a defined set of measures, dimensions, and joins. The search space shrinks from almost infinite to a well-bounded set of valid combinations.

A semantic layer is to an AI data assistant what a type system is to a programming language. It doesn’t eliminate errors, but it makes entire categories of mistakes structurally impossible. The AI can’t calculate ARR wrong because it doesn’t calculate ARR at all. It references a pre-defined measure that already encodes the correct logic.

Building the semantic layer was mainly data engineering work. We already had agreed-upon metric definitions across the company. The challenge was encoding those definitions into Cube: specifying the correct joins between tables, wiring up the right filters, and making sure every measure matched the logic our dashboards already used. Tedious, but essential.

Adding the AI layer

With the semantic layer in place, the AI becomes a translation problem: convert natural language into a Cube query.

Platybot supports multiple models: Claude Opus 4.6, Claude Sonnet 4.5, and Gemini 3 Pro. Users can choose which model to use. We found that different models have different strengths. Claude excels at structured data queries, while Gemini performs very well on text-heavy tasks like analyzing call transcriptions.

The system prompt gives the model awareness of available cubes, their measures, dimensions, and joins. When a user asks “What’s the ARR breakdown by plan type?”, the model doesn’t write SQL. Instead, it constructs a Cube query, selecting the total_arr measure from the ARR table and grouping by the sku from the subscriptions dimension. Cube handles the SQL generation, the joins, and the filters. For edge cases the semantic layer doesn’t cover, the model can fall back to direct (read-only) SQL against Snowflake, but it may already have a basic query that is already close to what it needs.

Query generation follows a workflow that maps to the same tools a human analyst would use:

flowchart TB
subgraph Entry["Entry points"]
Web["Web UI"]
Slack["Slack (@platybot)"]
MCP["MCP Server"]
end
subgraph Core["Platybot core"]
BE["Backend (Express + LLM)"]
end
subgraph Data["Data layer"]
Cube["Cube semantic layer"]
SF["Snowflake"]
end
Web --> BE
Slack --> BE
MCP --> BE
BE -->|"Structured queries"| Cube
BE -->|"Fallback SQL (read-only)"| SF
Cube --> SF

The workflow is: discover domains, explore cubes, understand the schema, construct and execute the query. This mirrors how a data analyst would work. You don’t jump straight to SQL; you first understand what data is available and what the metrics mean.

The system’s role is narrower than people expect. It’s a translator between human intent and a well-defined data model, not a general-purpose data scientist. This is a feature, not a limitation. By constraining Platybot to operate within the semantic layer, we get reliability. The creativity goes into understanding the question, not into inventing SQL.

Meeting users where they are

The backend solved query generation. It did not solve usage and if people don’t use it, it doesn’t matter. We launched Platybot across three interfaces, each designed for a different workflow.

The web UI

The web app is the primary interface: a React 19 + TypeScript + Vite + Tailwind conversational UI where employees can explore data through multi-turn conversations. It supports table visualization, data export, and conversation history so you can pick up where you left off.

Every analysis produces a shareable report with a permanent link, protected behind company authentication. The report shows the agent’s reasoning so you can verify its approach, and lists every query and table used along with a sample of the data. Each query includes a direct link to run it in Metabase, our reporting tool, so any useful query can be saved, scheduled, or extended without starting from scratch.

The web UI sees the highest adoption of all three access points. It’s where people go for deeper data exploration: multi-step analyses, follow-up questions, and comparing metrics across different dimensions.

One fun detail we’ve added during our last hackathon: while queries iterate through analysis steps (discovering domains, exploring cubes, executing queries), users are entertained by a platypus-themed runner game. Think Chrome’s dinosaur game, but with our mascot on a skateboard. Sometimes Platybot can take a long time iterating, so you can try to beat your high score in the meantime!

Slack

Platybot is also available as a Slack bot. Users @mention it in any channel, and it replies in a thread with the answer, keeping the channel clean while making results visible to the whole team. It’s best suited for quick lookups — “what’s the ARR for account X?” — where the answer fits in a message rather than a deep analysis. Most usage stayed in the web UI, but the Slack bot fills a different niche: answers that benefit from being shared in context.

MCP: Making Platybot usable by Agents

The Model Context Protocol (MCP) server is the newest addition, launched February 4, 2026. It exposes six tools that any MCP-compatible client can use:

list_domains - discover available data domains
list_cubes - explore cubes within a domain
get_cube_details - get measures, dimensions, and joins for a cube
execute_cube_query - run a structured query
execute_sql_query - raw SQL fallback (read-only)
get_generated_sql - preview SQL without executing

Setup is a single command:

claude mcp add --transport http platybot <platybot-mcp-url>

This is a paradigm shift. Platybot goes from a destination (open the app, ask a question) to a capability that any AI tool can use. An engineer writing a postmortem can pull live metrics without leaving their terminal. An analyst building a report in Claude Code can query data inline. The Data Warehouse becomes ambient, always available, never in the way.

Security uses OAuth 2.0 Device Authorization Flow with PKCE for CLI-friendly authentication, with @pulumi.com domain restriction, read-only enforcement, rate limiting, and full audit logging.

A meta moment: we used the Platybot MCP server to query our Slack messages table to find the real usage data cited above.

Deploying with Pulumi

Platybot’s infrastructure runs on AWS and is managed entirely with Pulumi IaC. The stack includes ECS Fargate for the application, an Application Load Balancer for traffic routing, EFS for persistent storage, ECR for container images, and Route 53 for DNS. The setup is intentionally simple and we use a small instance size since it’s just for internal usage.

Here’s a representative snippet of the Fargate service definition:

const service = new awsx.ecs.FargateService("platybot", {
 cluster: cluster.arn,
 assignPublicIp: false,
 taskDefinitionArgs: {
 container: {
 name: "platybot",
 image: image.imageUri,
 essential: true,
 portMappings: [{
 containerPort: 3000,
 targetGroup: targetGroup,
 }],
 environment: [
 { name: "CUBE_API_URL", value: cubeApiUrl },
 { name: "NODE_ENV", value: "production" },
 ],
 secrets: [
 { name: "ANTHROPIC_API_KEY", valueFrom: anthropicKeySecret.arn },
 { name: "SNOWFLAKE_PASSWORD", valueFrom: snowflakePasswordSecret.arn },
 ],
 logConfiguration: {
 logDriver: "awslogs",
 options: {
 "awslogs-group": logGroup.name,
 "awslogs-region": aws.config.region!,
 "awslogs-stream-prefix": "platybot",
 },
 },
 },
 },
});

Dogfooding Pulumi for our internal tools has real benefits beyond the obvious. Staging environments are trivial: spin up a full copy of the stack with different parameters. Secrets management is built in, so API keys and database credentials never touch a config file. And when something needs to change, the diff-and-preview workflow (pulumi preview) catches mistakes before they hit production.

Results

Since launch in September 2025: over 1,700 questions from 83 employees across every team. Usage grew steadily — from around 8 questions per day in the first month to 18 per day by January 2026, with 51 unique users that month alone. This was real production work, not experimentation. Customer analysis for sales calls, resource breakdowns for account managers, blog performance metrics for marketing, policy adoption research for product, ARR deep-dives for leadership.

The impact on the data team was immediate. Questions that used to land in the #analytics channel and wait for a human now get answered in seconds. The data team shifted from answering routine queries to building better models and improving data quality. We went from being a help desk to being a platform team.

Rumors of our replacement have been greatly exaggerated. Platybot just freed us up for the work that doesn't fit in a Slack reply.

Accuracy is harder to quantify, but the semantic layer gives us confidence. Because Platybot uses pre-defined measures instead of generating arbitrary SQL, entire categories of errors (wrong joins, missing filters, incorrect aggregations) are structurally less likely. When the model does get something wrong, it’s usually in interpreting the question, not in the data — and users can verify that through the report’s reasoning trace.

What we learned

The semantic layer matters more than the model. We spent more time defining metrics in Cube than we did on any AI work. That investment pays for itself: swap the model, and the answers stay correct. Swap the semantic layer, and nothing works.

Meet users where they already are. Different UIs handle different cognitive loads. Slack catches the quick questions, the web UI handles deep exploration, and MCP makes data ambient for AI-native workflows. Each interface serves a different mode of thinking.

Transparency builds trust. Showing the AI’s reasoning, the queries it ran, and linking to Metabase for verification turned skeptics into regular users. People don’t trust a black box, but they’ll trust a tool that shows its work.

The Claude Skills I Actually Use for DevOps

Engin Diri — Mon, 09 Feb 2026 00:00:00 +0000

When Claude Code first released skills, I ignored them. They looked like fancy prompts, another feature to add to the pile of things I would get around to learning eventually. Then I watched a few engineers demonstrate what skills actually do, and something clicked. By default, language models do not write good code. They write plausible code based on what they have read. Plausible code turns into bugs, horrible UX, and infrastructure that breaks at 3am.

Skills fill that gap. They package engineering expertise into something Claude can use. The workflows and judgment matter more than the raw information. Without skills, every conversation starts from zero. You explain the same conventions and correct the same mistakes. Every morning, back to zero.

Think about what separates a junior engineer from a senior one. Both can write code that compiles. Both can deploy infrastructure that runs. The difference is that the senior engineer knows the patterns that prevent problems before they happen. They know when to use component resources instead of plain resources. They know that creating infrastructure inside an apply() callback (Pulumi’s way of transforming outputs that are not known until deployment) breaks preview. They know that hardcoded credentials will eventually end up in a git log somewhere embarrassing.

This knowledge takes years to accumulate through painful experience. Skills let you transfer that knowledge to Claude in minutes. And here is the thing that makes them practical: if you find yourself doing the same type of task with different content each time, that is a skill waiting to be built. You encode the process once, then feed it new inputs forever.

The mechanic, the tools, and the manual

I heard an analogy recently that made skills click for me. Imagine Claude as a mechanic. A capable mechanic who knows engines, can diagnose problems, and fix most cars that come through the shop.

MCP servers are like giving that mechanic a set of tools. Wrenches, diagnostic equipment, lift systems. Without tools, the mechanic cannot do much. With tools, the mechanic can work on whatever comes through the door.

But what happens when someone brings in a Formula 1 race car? Or a 1967 Ford Mustang that has been modified beyond recognition? The mechanic knows engines in general, but these specific vehicles require specific knowledge. The F1 car has procedures that must be followed in exact order. The vintage Mustang has quirks that only someone who has worked on that model would know.

Skills are the user manuals and standard operating procedures for these specific vehicles. They tell the mechanic what needs to happen and when. They encode the expertise of someone who has done this work a thousand times.

Or think about it from a carpenter’s perspective. Skills are the process to make the table: the measurements, the design, the exact steps. MCPs are the tools: the saw, the hammer, the drill. You need both. The process alone is theoretical, and tools without a process just sit in the garage.

For DevOps engineers working with Pulumi, this matters because infrastructure as code has its own quirks and patterns. Generic AI assistance produces code that looks reasonable but breaks conventions the community learned the hard way. Skills teach Claude those conventions.

Why skills instead of MCPs

Before skills clicked for me, I tried solving the expertise problem with MCPs. I kept adding servers until I noticed Claude getting slower and making worse decisions. Turns out the GitHub MCP alone eats 46,000 tokens across 91 tools before you type anything. Cursor eventually capped MCPs at 40 tools because too many options made everything worse.

Slash commands were another option, but you had to remember to invoke them. Anthropic apparently agreed, because in January 2026 they merged slash commands into skills. One unified system instead of two.

Skills avoid this through progressive disclosure. Claude reads just the description at startup, maybe a hundred tokens. The full procedures only load when Claude decides they are relevant. Unlike those massive system prompts that used to eat through your context window, skills stay out of the way until they are needed. For DevOps engineers running long infrastructure sessions with dozens of resources, this matters. You keep your context budget for the actual work instead of burning it on instructions. Skills can also fork context, spinning up isolated subagents that do work without polluting your main conversation. Think of it like handing a colleague a written brief. They go work on it, hand back a summary, and never sit in on your conversation.

I still use MCPs for connecting Claude to external systems. The Pulumi MCP server lets Claude query the registry and validate code. But MCPs give Claude access to things. Skills teach Claude how to think about things. Different jobs. They get more useful when you combine them. One engineer built a financial reporting skill that connects to his Mercury bank account via MCP, pulls every transaction for a given month, classifies the expenses into categories, and generates a styled HTML report with totals and breakdowns. A skill that knows your deployment process connecting to MCPs that talk to your actual infrastructure is the same idea, just pointed at ops instead of accounting.

Skills are portable. They follow an open standard, so a skill you write for Claude Code works in Cursor, GitHub Copilot, or anywhere else that supports agent skills. You can even copy the skill content into ChatGPT as a starting prompt. No vendor lock-in.

Teaching Claude to write Pulumi like an expert

The first time you ask Claude to help with a Pulumi project, the process is painful. You have to explain the patterns you want. You correct mistakes. You explain why creating resources inside apply() breaks things. By the third or fourth project, you start copying your corrections from previous conversations.

I built the dirien/claude-skills pulumi-typescript skill after going through this painful process too many times. It knows the patterns that prevent common mistakes: Pulumi ESC (Environments, Secrets, and Configuration) integration, OIDC (OpenID Connect) instead of hardcoded access keys, ComponentResource abstractions (reusable groups of related resources), and proper output structuring so dependent stacks can consume them cleanly.

Skill	What it teaches Claude
`pulumi-typescript`	Pulumi with TypeScript, ESC secrets management, component patterns, and multi-cloud deployment

npx skills add https://github.com/dirien/claude-skills --skill pulumi-typescript

Skills install as markdown files in your project’s .claude/skills/ directory, so they travel with your repo and are easy to review.

The next time you ask Claude to create infrastructure, it applies these patterns automatically. You do not have to remember to invoke the skill or correct the same mistakes repeatedly.

The official Pulumi skills

Pulumi maintains its own skills repository at pulumi/agent-skills, which they announced recently. The repo includes skills for ComponentResource patterns, Automation API, and migration from Terraform, CDK, CloudFormation, and ARM. The two I use daily are pulumi-esc and pulumi-best-practices.

The pulumi-esc skill teaches Claude how to work with Pulumi ESC (Environments, Secrets, and Configuration). It knows the difference between pulumi env get, pulumi env open, and pulumi env run. It sets up OIDC for dynamic credentials, integrates with external secret stores like AWS Secrets Manager and Vault, and structures layered environment composition so your dev, staging, and production configs inherit from a shared base.

The pulumi-best-practices skill catches the mistakes that burn you in production. It stops Claude from creating resources inside apply() callbacks, enforces proper parent relationships in ComponentResources, encrypts secrets from day one, and makes sure pulumi preview runs before any deployment. These are the patterns that took me years to internalize, and now Claude follows them by default.

Skill	What it teaches Claude
`pulumi-esc`	Environment, secrets, and configuration management with OIDC, dynamic credentials, and secret store integration
`pulumi-best-practices`	Resource dependencies, ComponentResource patterns, secret encryption, and safe refactoring

npx skills add https://github.com/pulumi/agent-skills --skill pulumi-esc
npx skills add https://github.com/pulumi/agent-skills --skill pulumi-best-practices

Making Claude a monitoring expert by default

You deploy something, it works, and six months later something breaks and you realize you never added monitoring. We have all been there. The monitoring skills from the community teach Claude to add observability from the start.

The jeffallan/claude-skills repository contains a monitoring-expert skill that knows Prometheus, Grafana, and DataDog.

Skill	What it teaches Claude
`monitoring-expert`	Structured logging, metrics, distributed tracing, alerting, and performance testing for production systems

npx skills add https://github.com/jeffallan/claude-skills --skill monitoring-expert

In my testing, deploying a static website with these skills installed looks different from vanilla Claude. Instead of just creating the S3 bucket and CloudFront distribution, Claude asked about error rate thresholds before writing any code. It suggested CloudWatch alarms and created an SNS topic for alerts. The results are not always this clean. Sometimes the monitoring suggestions are generic or miss your specific SLO requirements. But the baseline shifted from “no monitoring at all” to “monitoring that needs tuning,” and that is a better starting point.

Kubernetes configuration that actually passes security review

Kubernetes has hundreds of configuration options. Most deployments use a handful of them. The problem is that the important options like security contexts, resource limits, and pod disruption budgets are easy to forget when you are focused on getting something to run.

The jeffallan/claude-skills kubernetes-specialist skill focuses on configurations that production deployments actually need. Without it, ask Claude for a deployment and you get something that runs: the right image, the right ports, maybe a service. With the skill, the same request comes back with runAsNonRoot: true in the security context, resource requests and limits that reflect actual usage patterns, liveness and readiness probes with sensible intervals, and a pod disruption budget. These are the things that make the difference between “it works in staging” and “it survives a node failure in production.” The skill also understands when RollingUpdate makes sense versus Recreate, which is the kind of judgment call that usually requires context a generic model does not have.

The wshobson/agents repository fills in the gaps around Kubernetes with CI/CD, cost management, and deployment workflows:

Skill	What it teaches Claude
`kubernetes-specialist`	Production cluster management, security hardening, and cloud-native architectures
`cost-optimization`	Cloud cost reduction across AWS, Azure, and GCP with right-sizing and reserved instances
`github-actions-templates`	CI/CD workflows, Docker builds, Kubernetes deployments, security scanning, and matrix builds
`gitops-workflow`	ArgoCD and Flux CD for automated Kubernetes deployments

npx skills add https://github.com/jeffallan/claude-skills --skill kubernetes-specialist
npx skills add https://github.com/wshobson/agents --skill gitops-workflow
npx skills add https://github.com/wshobson/agents --skill github-actions-templates
npx skills add https://github.com/wshobson/agents --skill cost-optimization

Debugging like a senior engineer

The obra/superpowers repository contains a skill that changed how I debug with Claude. The systematic-debugging skill implements a four-phase framework: root cause investigation, pattern analysis, hypothesis testing, and implementation.

Without this skill, Claude tends to suggest solutions immediately. Something is broken, here are five things that might fix it. This feels helpful but often wastes time because none of the suggestions address the actual problem.

With the systematic debugging skill, Claude approaches problems differently. It asks clarifying questions. It wants to see logs. It builds a model of what is happening before suggesting changes. When it proposes a fix, it explains why that fix addresses the root cause. Sometimes skills find problems you did not know about. One engineer pointed a skills-equipped Claude at a set of SEO pages and discovered they had been decaying for months with nobody watching. The infrastructure parallel is obvious: configuration drift, unused resources, permissions that expanded over time. A debugging skill that investigates before prescribing will find these things.

Skill	What it teaches Claude
`systematic-debugging`	Root cause investigation, pattern analysis, hypothesis testing, and verified implementation

npx skills add https://github.com/obra/superpowers --skill systematic-debugging

Catching security issues before they ship

Two skills cover different sides of security review. The wshobson/agents k8s-security-policies skill handles Kubernetes-specific hardening: NetworkPolicies, Pod Security Standards, RBAC, OPA Gatekeeper constraints, and service mesh mTLS configuration. The sickn33/antigravity-awesome-skills security-review skill covers application-level concerns like secrets management, SQL injection, XSS prevention, and input validation.

I asked Claude to check a Pulumi program that created an S3 bucket. Without the security skills, Claude confirmed the code was correct and moved on. With the skills loaded, it flagged that the bucket had no server-side encryption configured, the bucket policy allowed s3:* from an overly broad principal, and there was no access logging enabled. On the Kubernetes side, the k8s-security-policies skill catches things like missing default-deny NetworkPolicies and containers running as root. These skills are not a replacement for deterministic tools like tfsec, checkov, or trivy. Those catch known issues every time. Skills are probabilistic and work best as an extra layer during development, not as your only security gate.

Skill	What it teaches Claude
`k8s-security-policies`	Network policies, pod security standards, RBAC, and admission control for defense-in-depth
`security-review`	Secrets management, input validation, SQL injection, XSS/CSRF prevention, and dependency auditing

npx skills add https://github.com/wshobson/agents --skill k8s-security-policies
npx skills add https://github.com/sickn33/antigravity-awesome-skills --skill security-review

Incident response before you need it

At 3am when something breaks, you want runbooks. The incident-runbook-templates skill from wshobson/agents helps Claude create these before you need them. It includes a four-level severity model (SEV1 through SEV4) with response time expectations, escalation decision trees, and communication templates for status updates.

When you ask Claude to document your deployment process, it produces runbooks with diagnostic steps, rollback protocols, and verification checks. It knows kubectl commands for Kubernetes recovery and SQL procedures for PostgreSQL troubleshooting. The output needs editing. Generated runbooks tend to be thorough on the happy path but thin on the failure modes that matter most at 3am. I treat them as a first draft that gets me to 60% in minutes instead of hours, then fill in the gaps from experience.

Skill	What it teaches Claude
`incident-runbook-templates`	Detection, triage, mitigation, resolution, and communication procedures for production incidents

npx skills add https://github.com/wshobson/agents --skill incident-runbook-templates

General-purpose DevOps and SRE skills

The skills above target specific problems. The jeffallan/claude-skills repository also includes two broader skills that cover the day-to-day work that does not fit neatly into one category.

The devops-engineer skill gives Claude a senior DevOps engineer persona covering CI/CD pipelines, container management, deployment strategies like blue-green and canary, and infrastructure as code across AWS, GCP, and Azure. It enforces constraints I care about: no deploying to production without approval, no secrets in code, no unversioned container images.

The sre-engineer skill focuses on reliability: SLO/SLI definitions, error budget calculations, golden signal dashboards, and toil reduction through automation. It produces Prometheus/Grafana configs, remediation runbooks, and reliability assessments. If you run production systems and want Claude to think about error budgets instead of just uptime, this is the skill.

Skill	What it teaches Claude
`devops-engineer`	CI/CD pipelines, container management, deployment strategies, and infrastructure as code across clouds
`sre-engineer`	SLI/SLO management, error budgets, monitoring, automation, and incident response

npx skills add https://github.com/jeffallan/claude-skills --skill devops-engineer
npx skills add https://github.com/jeffallan/claude-skills --skill sre-engineer

Vetting the skills you install

Before you install everything in sight, a warning. Skills run with the same permissions as your AI agent. A malicious skill can exfiltrate credentials, download backdoors, or disable safety mechanisms, and it will look like your agent doing it.

Snyk researchers published ToxicSkills in February 2026 after scanning 3,984 skills from public registries. 13.4% had critical-level vulnerabilities, and they found 76 confirmed malicious payloads. The attack techniques included base64-encoded commands that steal AWS credentials, skills that direct you to download password-protected executables from attacker infrastructure, and jailbreak attempts that try to disable safety mechanisms. 91% of malicious skills combine code-level malware with prompt injection, so they attack on two fronts simultaneously.

Treat skills like you treat any third-party dependency:

Read the source before installing. Skills are markdown and YAML files. If you cannot read the full skill in a few minutes, that is a red flag.
Check the repository. Look at stars, contributors, and commit history. A single-commit repository from an unknown account deserves scrutiny.
Run uvx mcp-scan@latest --skills to scan installed skills for known malicious patterns, prompt injection, and credential exposure.
Be cautious with skills that fetch external content at runtime. The Snyk research found 17.7% of skills on ClawHub pull from third-party URLs, which means the skill’s behavior can change after you install it.
Stick to known repositories. Every skill recommended in this post comes from a repository with visible maintainers and community activity.

Eight malicious skills were still publicly available on ClawHub when Snyk published their findings. The skills ecosystem is young, and the vetting infrastructure is still catching up.

Putting it together

Stacking skills is where this pays off. Install the Pulumi skills and Claude writes better infrastructure code. Add monitoring and security on top and you start catching problems that used to slip through to production.

A note on stacking: I have not hit conflicts running all of these simultaneously, but more skills means more descriptions for Claude to evaluate at startup. If you notice Claude getting slower or making odd choices, pare back to the skills you actually use for that project. Start with the Pulumi and monitoring skills, add others as you need them.

Here is how to set up a new project with all of them:

mkdir -p pulumi-skills-demo && cd pulumi-skills-demo
pulumi new aws-typescript --name skills-demo --yes

npx skills add https://github.com/dirien/claude-skills --skill pulumi-typescript
npx skills add https://github.com/pulumi/agent-skills --skill pulumi-esc
npx skills add https://github.com/pulumi/agent-skills --skill pulumi-best-practices
npx skills add https://github.com/obra/superpowers --skill systematic-debugging
npx skills add https://github.com/jeffallan/claude-skills --skill monitoring-expert
npx skills add https://github.com/jeffallan/claude-skills --skill kubernetes-specialist
npx skills add https://github.com/wshobson/agents --skill gitops-workflow
npx skills add https://github.com/wshobson/agents --skill github-actions-templates
npx skills add https://github.com/wshobson/agents --skill cost-optimization
npx skills add https://github.com/wshobson/agents --skill incident-runbook-templates
npx skills add https://github.com/jeffallan/claude-skills --skill devops-engineer
npx skills add https://github.com/jeffallan/claude-skills --skill sre-engineer
npx skills add https://github.com/wshobson/agents --skill k8s-security-policies
npx skills add https://github.com/sickn33/antigravity-awesome-skills --skill security-review

Then try these two prompts to see how many skills activate at once:

# Static website — triggers Pulumi TypeScript, monitoring, and security skills
Create a Pulumi TypeScript program for a static website on AWS with S3, CloudFront, OIDC credentials via Pulumi ESC, CloudWatch monitoring, and /security-review the infrastructure before deploying

# EKS cluster — stacks Kubernetes, GitOps, incident response, cost, and SRE skills
Create a Pulumi TypeScript program for an EKS cluster with /kubernetes-specialist security hardening, /gitops-workflow for ArgoCD deployment, /incident-runbook-templates for the cluster, /cost-optimization recommendations, and /sre-engineer SLO definitions for the services

The first prompt triggers the Pulumi TypeScript, monitoring, and security review skills in a single conversation. The second stacks Kubernetes, GitOps, incident response, cost, and SRE skills on one cluster build. You get infrastructure code, operational runbooks, and security policies from a single request.

What changes

Fair warning: not every skill works perfectly on the first try. Some need iteration. Some produce output that you have to review and tweak before it matches your standards. Skills do not replace your judgment.

That said, after a few weeks with these skills installed, I stopped correcting the same mistakes. The code Claude writes now looks like code I would write, not code I would have to fix. That is the whole point. Skills just stop you from repeating the same corrections across every conversation.

Get started

Every skill in this post includes its install command. Pick the section that matches your biggest pain point, run the npx skills add command, and try it on your next task. Skills work in Claude Code, Cursor, GitHub Copilot, and anything else that supports the Agent Skills standard.

The Pulumi Agent Skills announcement has more details, and the GitHub repository has the source. If you want something that goes further, with organizational context and deployment governance, look at Pulumi Neo. Neo is grounded in your actual infrastructure, not internet patterns. The 10 things you can do with Neo post shows what that looks like in practice.

Give it one project. That is all it took for me.

Try Pulumi for Free

Pulumi Neo Now Supports AGENTS.md

Pulumi Neo Team — Fri, 06 Feb 2026 00:00:00 +0000

Neo now reads AGENTS.md files, the open standard for giving AI coding tools context about your project. If you’re already using AGENTS.md, Neo will pick up those same instructions automatically.

The problem AGENTS.md solves

Every codebase has conventions that aren’t captured in linters or formatters. Maybe your team uses a specific naming pattern for infrastructure resources. Maybe there’s a particular way you structure tests, or commands that need to run in a certain order. These are the things you’d explain to a new team member, and now you can explain them to AI tools too.

Without something like AGENTS.md, you end up repeating yourself. Every conversation starts with “remember to use TypeScript” or “make sure you add the environment tag.” It’s tedious, and things slip through.

AGENTS.md gives these instructions a home. You write them once, commit the file to your repo, and any tool that supports the format picks them up automatically.

What to put in yours

Think about what you’d tell someone on their first day working in the codebase. How do you run tests? Are there naming conventions? Any gotchas they should know about?

Here’s an example for a Pulumi project:

# Infrastructure conventions

Run tests with `make test`. This spins up LocalStack, so Docker must be running.

Stacks are named `{service}-{region}-{env}` (e.g., `payments-us-west-2-prod`).
Only the platform team deploys to prod stacks.

All resources need these tags: `cost-center`, `team`, `environment`.

Reusable components live in `components/`. Check there before writing
something new.

There’s no required structure, just markdown. Some teams write a few lines, others write detailed guides. Start small and add things as you notice yourself repeating instructions.

How Neo handles AGENTS.md

When you point Neo at a repository, it reads any AGENTS.md file it finds and applies those instructions to its work. You don’t need to mention the file or remind Neo about your conventions.

If you have a monorepo, you can put AGENTS.md files in subdirectories too. Neo uses the nearest one to wherever it’s working, so you can have general instructions at the root and more specific ones in subpackages.

Your instructions in conversation always take precedence, so you can override the file when you need to. If you’ve also set up Custom Instructions at the organization level, Neo applies those first, then AGENTS.md on top.

Works with the tools you’re already using

AGENTS.md is an open format supported by most AI coding tools: Cursor, Windsurf, GitHub Copilot, Zed, and now Neo. If your team uses different tools for different tasks, they’ll all follow the same project conventions without any extra configuration.

The format is managed by the Agentic AI Foundation under the Linux Foundation, and it’s already in use in over 60,000 open source projects. See agents.md for the full specification.

Get started

Add an AGENTS.md file to your repository and Neo will start using it on your next task. For more on configuring Neo, including organization-wide Custom Instructions and Slash Commands, see the Settings documentation.

Neo: Share Tasks for Collaborative AI Infrastructure Operations

Pulumi Neo Team — Wed, 04 Feb 2026 00:00:00 +0000

Neo shows its work, but until now that context was only viewable by the user that initiated the conversation. When you wanted a teammate’s input on a decision Neo made, you had to describe it in Slack or screenshot fragments of the conversation. Today we’re introducing task sharing: share a read-only view of any Neo task with anyone in your organization, full context preserved.

To share a Neo task, click the share button to generate a read-only link, then send it to a teammate. They see the complete picture: the original prompt, Neo’s reasoning process, the actions it took, and the outcome. Instead of writing up what happened and losing detail in the retelling, you share the task itself.

We built this with security as a core constraint. The original task system enforced strict RBAC, ensuring users could only see and act on resources they had permission to access. Task sharing preserves these guarantees. Viewers can see the conversation with Neo, but they cannot trigger any actions, and links within the shared task to stacks or resources still enforce the viewer’s existing permissions.

The feature is available now. The next time you want a second opinion or need to show a colleague how you solved something, share the task. You’re no longer working alone.

Pulumi Agent Skills: Best practices and more for AI coding assistants

Pulumi Neo Team — Thu, 29 Jan 2026 00:00:00 +0000

AI coding assistants have transformed how developers write software, including infrastructure code. Tools like Claude Code, Cursor, and GitHub Copilot can generate code, explain complex systems, and automate tedious tasks. But when it comes to infrastructure, these tools often produce code that works but misses the mark on patterns that matter: proper secret handling, correct resource dependencies, idiomatic component structure, and the dozens of other details that separate working infrastructure from production-ready infrastructure.

We built Neo for teams that want deep Pulumi expertise combined with organizational context and deployment governance. But developers have preferred tools, and we want people to succeed with Pulumi wherever they work. Some teams live in Claude Code. Others use Cursor, Copilot, Codex, Gemini CLI, or other platforms. That is why we are releasing Pulumi Agent Skills, a collection of packaged expertise that teaches any AI coding assistant how to work with Pulumi the way an experienced practitioner would.

What are agent skills?

Skills are structured knowledge packages that follow the open Agent Skills specification. They work across multiple AI coding platforms including Claude Code, GitHub Copilot, Cursor, VS Code, Codex, and Gemini CLI. When you install Pulumi skills, your AI assistant gains access to detailed workflows, code patterns, and decision trees for common infrastructure tasks.

Available Pulumi skills

We are launching a set of skills organized into two plugin groups: authoring and migration. You can install all skills at once or choose specific plugin groups based on your needs.

Authoring skills

This plugin includes four skills focused on code quality, reusability, and configuration.

Pulumi best practices encodes the patterns that prevent common mistakes. It covers output handling, component structure, secrets management, safe refactoring with aliases, and deployment workflows. The skill flags anti-patterns that can cause issues with preview, dependencies, and production deployments.
Pulumi Component provides a complete guide for authoring ComponentResource classes. The skill covers designing component interfaces, multi-language support, and distribution. It teaches assistants how to build reusable infrastructure abstractions that work across TypeScript, Python, Go, C#, Java, and YAML.
Pulumi Automation API covers programmatic orchestration of Pulumi operations. The skill explains when to use Automation API versus the CLI, the tradeoffs between local source and inline programs, and patterns for multi-stack deployments.
Pulumi ESC covers centralized secrets and configuration management. The skill guides assistants through setting up dynamic OIDC credentials, composing environments, and integrating secrets into Pulumi programs and other applications.

Migration skills

Convert and import infrastructure from other tools to Pulumi. This plugin includes four skills covering complete migration workflows, not just syntax translation.

Terraform to Pulumi walks through the full migration workflow. It handles state translation, provider version alignment, and the iterative process of achieving a clean pulumi preview with no unexpected changes.
CloudFormation to Pulumi covers the complete AWS CloudFormation migration workflow, from template conversion and stack import to handling CloudFormation-specific constructs.
CDK to Pulumi covers the complete AWS CDK migration workflow end to end, from conversion and import to handling CDK-specific constructs like Lambda-backed custom resources and cross-stack references.
Azure to Pulumi covers the complete Azure Resource Manager and Bicep migration workflow, handling template conversion and resource import with guidance on achieving zero-diff validation.

How to install

Claude Code plugin marketplace

For Claude Code users, the plugin system provides the simplest installation experience:

claude plugin marketplace add pulumi/agent-skills
claude plugin install pulumi-authoring # Install authoring skills
claude plugin install pulumi-migration # Install migration skills

You can install both plugin groups or choose only the ones you need.

Universal installation

For Cursor, GitHub Copilot, VS Code, Codex, Gemini and other platforms, use the universal Agent Skills CLI:

npx skills add pulumi/agent-skills --skill '*'

This works across all platforms that support the Agent Skills specification.

Using skills

Once installed, skills activate automatically based on context. When you ask your assistant to help migrate a Terraform project, it draws on the Terraform skill’s workflow. When you are debugging why resources are being recreated unexpectedly, the best practices skill helps the assistant check for missing aliases.

In Codex and Claude Code, you can invoke skills directly via slash commands.

/pulumi-terraform-to-pulumi

Or describe what you need in natural language:

“Help me migrate this CDK application to Pulumi”

“Review this Pulumi code for best practices issues”

“Create a reusable component for a web service with load balancer”

The assistant will follow the skill’s procedures, ask clarifying questions when needed, and produce output that reflects Pulumi best practices rather than generic code generation.

Get started

We expect this collection to grow. If you have Pulumi expertise worth packaging, whether provider-specific patterns, debugging workflows, or operational practices, we welcome contributions. See the contributing guide for details.

The skills are available now in the agent-skills repository. Install them in your preferred AI coding environment and let us know what you build.

Deploy OpenClaw on AWS or Hetzner Securely with Pulumi and Tailscale

Engin Diri — Mon, 26 Jan 2026 00:00:00 +0000

Update (January 2026): The lobster has molted into its final form! From Clawdbot to Moltbot to OpenClaw. With 100k+ GitHub stars and 2M visitors in a week, the project finally has a name that’ll stick. The CLI command is now openclaw and the new handle is @openclaw. Same mission: AI that actually does things. Your assistant. Your machine. Your rules. See the official getting started guide for updated installation instructions.

Update (April 2026): Refreshed for OpenClaw 2026.4.27. Upstream now recommends Node 24, but the cloud-init script in this post still installs Node 22 — both work. If you’d like Node 24, change the nvm install 22 lines to nvm install 24.

The short version: Deploy OpenClaw to AWS or Hetzner with a Pulumi TypeScript program that provisions the VM, installs Docker, Node, and OpenClaw, then joins the instance to your Tailscale network so the gateway and browser ports stay private. One pulumi up to deploy, one pulumi destroy to tear down. Total cost: about $33/month on AWS or $7/month on Hetzner.

OpenClaw is everywhere right now. The open-source AI assistant gained 9,000 GitHub stars in a single day, received public praise from former Tesla AI head Andrej Karpathy, and has sparked a global run on Mac Minis as developers scramble to give this “lobster assistant” a home. Users are calling it “Jarvis living in a hard drive” and “Claude with hands”—the personal AI assistant that Siri promised but never delivered.

The Mac Mini craze is real: people are buying dedicated hardware just to run OpenClaw, with some enthusiasts purchasing 40 Mac Minis at once. Even Logan Kilpatrick from Google DeepMind couldn’t resist ordering one. But here’s the thing: you don’t actually need a Mac Mini. OpenClaw runs anywhere: on a VPS, in the cloud, or on that old laptop gathering dust.

With all this hype, I had to try it myself. But instead of clicking through the AWS console or running manual commands on a VPS, I wanted to do it right from the start: infrastructure as code with Pulumi. Why? Because when I inevitably want to tear it down, spin up a new instance, or deploy to a different region, I don’t want to remember which buttons I clicked three weeks ago. I want a single pulumi up command.

Dan got the assignment right:

In this post, I’ll show you how to deploy OpenClaw to AWS or Hetzner Cloud (if you want European data residency or just want to spend less). We’ll use Pulumi to define the infrastructure and Tailscale to keep your AI assistant off the public internet.

What is OpenClaw?

OpenClaw is an open-source AI assistant created by Peter Steinberger that runs on your own infrastructure. It connects to WhatsApp, Slack, Discord, Google Chat, Signal, and iMessage. It can control browsers, generate videos and images, clone your voice for voice notes, and run scheduled tasks via cron. There’s a skills system for extending functionality, and you can run it on pretty much anything: Mac Mini, Raspberry Pi, VPS, laptop, or gaming PC.

The difference from cloud-hosted AI? OpenClaw runs on your server, not Anthropic’s. It’s available 24/7 across all your devices, can schedule automated tasks, and keeps your entire conversation history locally. Check the official OpenClaw documentation for the full feature list.

Prerequisites

Before getting started, ensure you have:

Pulumi CLI installed and configured
A Pulumi Cloud account
AWS account (for AWS deployment)
Hetzner Cloud account (for European deployment)
Anthropic API key
Node.js 22+ installed (Node 24 recommended; the cloud-init script in this post installs Node 22 inside the VM)
Tailscale account with HTTPS enabled (one-time setup in admin console)

This guide uses Anthropic’s API, but OpenClaw works with other providers too. Check the providers documentation if you’d rather use OpenAI, Google Gemini, or a local model via Ollama.

Understanding OpenClaw architecture

OpenClaw uses a gateway-centric architecture where a single daemon acts as the control plane for all messaging, tool execution, and client connections:

Component	Port	Description
Gateway	18789	WebSocket server handling channels, nodes, sessions, and hooks
Browser control	18791	Headless Chrome instance for web automation
Docker sandbox	-	Isolated container environment for running tools safely

The Gateway connects to messaging platforms (WhatsApp, Slack, Discord, etc.), the CLI, the web UI, and mobile apps. The Browser component lets OpenClaw open web pages, fill forms, scrape data, and download files. Docker sandboxing runs bash commands in isolated containers so your bot can execute code without risking your host system.

Setting up ESC for secrets management

Deploying OpenClaw means handling sensitive credentials: API keys, auth tokens, cloud provider secrets. You don’t want these hardcoded or scattered across environment variables. Pulumi ESC (Environments, Secrets, and Configuration) stores them securely and passes them directly to your Pulumi program — the same pattern Pulumi uses to eliminate long-lived CI secrets across 70+ repos.

Create a new ESC environment:

pulumi env init <your-org>/openclaw-secrets

Add your secrets to the environment:

values:
 anthropicApiKey:
 fn::secret: "sk-ant-xxxxx"
 tailscaleAuthKey:
 fn::secret: "tskey-auth-xxxxx"
 tailnetDnsName: "tailxxxxx.ts.net"
 hcloudToken:
 fn::secret: "your-hetzner-api-token"
 pulumiConfig:
 anthropicApiKey: ${anthropicApiKey}
 tailscaleAuthKey: ${tailscaleAuthKey}
 tailnetDnsName: ${tailnetDnsName}
 hcloud:token: ${hcloudToken}

Then create a Pulumi.dev.yaml file in your project to reference the environment:

environment:
 - <your-org>/openclaw-secrets

This approach keeps your secrets out of your codebase and passes them directly to OpenClaw during automated onboarding. To prevent stack-level overrides of these values, see locking down ESC values with fn::final.

Securing with Tailscale

By default, deploying OpenClaw exposes SSH (port 22), the gateway (port 18789), and browser control (port 18791) to the public internet. This is convenient for testing but not ideal for production use.

Tailscale creates a secure mesh VPN that lets you access your OpenClaw instance without exposing unnecessary ports publicly. When you provide a Tailscale auth key, the Pulumi program:

Removes gateway and browser ports from public access
Keeps SSH as fallback for debugging if Tailscale setup fails
Installs Tailscale on the instance during provisioning (after other dependencies)
Enables Tailscale SSH so you can SSH via Tailscale without managing keys
Joins your tailnet automatically using the auth key

The Pulumi program installs Docker, Node.js, and OpenClaw first, then configures Tailscale last. This ensures that even if the Tailscale auth key is invalid or expired, you can still SSH in via the public IP to troubleshoot.

To generate a Tailscale auth key:

Go to Tailscale Admin Console
Click “Generate auth key”
Enable “Reusable” if you plan to redeploy
Copy the key and add it to your ESC environment

Deploying to AWS

Let’s walk through the complete AWS deployment. Create a new Pulumi project:

mkdir openclaw-aws && cd openclaw-aws
pulumi new typescript

Install the required dependencies:

npm install @pulumi/aws @pulumi/tls

Do not use t3.micro instances for OpenClaw. The 1 GB memory is insufficient for installation. Use t3.medium (4 GB) or t3.large (8 GB) instead.

The Pulumi program

Running OpenClaw on AWS means setting up a VPC, subnets, security groups, an EC2 instance, SSH keys, and a cloud-init script that installs everything. That’s a lot of clicking in the AWS console. The Pulumi program below defines all of it in code.

Replace the contents of index.ts with the following:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as tls from "@pulumi/tls";

const config = new pulumi.Config();

const instanceType = config.get("instanceType") ?? "t3.medium";
const anthropicApiKey = config.requireSecret("anthropicApiKey");
const model = config.get("model") ?? "anthropic/claude-sonnet-4";
const enableSandbox = config.getBoolean("enableSandbox") ?? true;
const gatewayPort = config.getNumber("gatewayPort") ?? 18789;
const browserPort = config.getNumber("browserPort") ?? 18791;

const tailscaleAuthKey = config.requireSecret("tailscaleAuthKey");
const tailnetDnsName = config.require("tailnetDnsName");

// Generate a random token for gateway authentication
const gatewayToken = new tls.PrivateKey("openclaw-gateway-token", {
 algorithm: "ED25519",
}).publicKeyOpenssh.apply(key => {
 // Create a deterministic token from the public key (take first 48 hex chars)
 const hash = require("crypto").createHash("sha256").update(key).digest("hex");
 return hash.substring(0, 48);
});

const sshKey = new tls.PrivateKey("openclaw-ssh-key", {
 algorithm: "ED25519",
});

const vpc = new aws.ec2.Vpc("openclaw-vpc", {
 cidrBlock: "10.0.0.0/16",
 enableDnsHostnames: true,
 enableDnsSupport: true,
 tags: { Name: "openclaw-vpc" },
});

const gateway = new aws.ec2.InternetGateway("openclaw-igw", {
 vpcId: vpc.id,
 tags: { Name: "openclaw-igw" },
});

const subnet = new aws.ec2.Subnet("openclaw-subnet", {
 vpcId: vpc.id,
 cidrBlock: "10.0.1.0/24",
 mapPublicIpOnLaunch: true,
 tags: { Name: "openclaw-subnet" },
});

const routeTable = new aws.ec2.RouteTable("openclaw-rt", {
 vpcId: vpc.id,
 routes: [
 {
 cidrBlock: "0.0.0.0/0",
 gatewayId: gateway.id,
 },
 ],
 tags: { Name: "openclaw-rt" },
});

new aws.ec2.RouteTableAssociation("openclaw-rta", {
 subnetId: subnet.id,
 routeTableId: routeTable.id,
});

const securityGroup = new aws.ec2.SecurityGroup("openclaw-sg", {
 vpcId: vpc.id,
 description: "Security group for OpenClaw instance",
 ingress: [
 {
 description: "SSH access (fallback)",
 fromPort: 22,
 toPort: 22,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 ],
 egress: [
 {
 fromPort: 0,
 toPort: 0,
 protocol: "-1",
 cidrBlocks: ["0.0.0.0/0"],
 },
 ],
 tags: { Name: "openclaw-sg" },
});

const keyPair = new aws.ec2.KeyPair("openclaw-keypair", {
 publicKey: sshKey.publicKeyOpenssh,
});

const ami = aws.ec2.getAmiOutput({
 owners: ["099720109477"],
 mostRecent: true,
 filters: [
 { name: "name", values: ["ubuntu/images/hvm-ssd-gp3/ubuntu-noble-24.04-amd64-server-*"] },
 { name: "virtualization-type", values: ["hvm"] },
 ],
});

const userData = pulumi
 .all([tailscaleAuthKey, anthropicApiKey, gatewayToken])
 .apply(([tsAuthKey, apiKey, gwToken]) => {
 return `#!/bin/bash
set -e

export DEBIAN_FRONTEND=noninteractive

# System updates
apt-get update
apt-get upgrade -y

# Install Docker
curl -fsSL https://get.docker.com | sh
systemctl enable docker
systemctl start docker
usermod -aG docker ubuntu

# Install NVM and Node.js for ubuntu user
sudo -u ubuntu bash << 'UBUNTU_SCRIPT'
set -e
cd ~

# Install NVM
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash

# Load NVM
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

# Install Node.js 22
nvm install 22
nvm use 22
nvm alias default 22

# Install OpenClaw
npm install -g openclaw@latest

# Add NVM to bashrc if not already there
if ! grep -q 'NVM_DIR' ~/.bashrc; then
 echo 'export NVM_DIR="$HOME/.nvm"' >> ~/.bashrc
 echo '[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"' >> ~/.bashrc
fi
UBUNTU_SCRIPT

# Set environment variables for ubuntu user
echo 'export ANTHROPIC_API_KEY="${apiKey}"' >> /home/ubuntu/.bashrc

# Install and configure Tailscale
echo "Installing Tailscale..."
curl -fsSL https://tailscale.com/install.sh | sh
tailscale up --authkey="${tsAuthKey}" --ssh || echo "WARNING: Tailscale setup failed. Run 'sudo tailscale up' manually."

# Enable systemd linger for ubuntu user (required for user services to run at boot)
loginctl enable-linger ubuntu

# Start user's systemd instance (required for user services during cloud-init)
systemctl start user@1000.service

# Run OpenClaw onboarding as ubuntu user (skip daemon install, do it separately)
echo "Running OpenClaw onboarding..."
sudo -H -u ubuntu ANTHROPIC_API_KEY="${apiKey}" GATEWAY_PORT="${gatewayPort}" bash -c '
export HOME=/home/ubuntu
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

openclaw onboard --non-interactive --accept-risk \
 --mode local \
 --auth-choice apiKey \
 --gateway-port $GATEWAY_PORT \
 --gateway-bind loopback \
 --skip-daemon \
 --skip-skills || echo "WARNING: OpenClaw onboarding failed. Run openclaw onboard manually."
'

# Install daemon service with XDG_RUNTIME_DIR set
echo "Installing OpenClaw daemon..."
sudo -H -u ubuntu XDG_RUNTIME_DIR=/run/user/1000 bash -c '
export HOME=/home/ubuntu
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

openclaw daemon install || echo "WARNING: Daemon install failed. Run openclaw daemon install manually."
'

# Configure gateway for Tailscale Serve (trustedProxies + skip device pairing + set token)
echo "Configuring gateway for Tailscale Serve..."
sudo -H -u ubuntu GATEWAY_TOKEN="${gwToken}" python3 << 'PYTHON_SCRIPT'
import json
import os
config_path = "/home/ubuntu/.openclaw/openclaw.json"
with open(config_path) as f:
 config = json.load(f)
config["gateway"]["trustedProxies"] = ["127.0.0.1"]
config["gateway"]["controlUi"] = {
 "enabled": True,
 "allowInsecureAuth": True
}
config["gateway"]["auth"] = {
 "mode": "token",
 "token": os.environ["GATEWAY_TOKEN"]
}
with open(config_path, "w") as f:
 json.dump(config, f, indent=2)
print("Configured gateway with trustedProxies, controlUi, and token")
PYTHON_SCRIPT

# Enable Tailscale HTTPS proxy (requires HTTPS to be enabled in Tailscale admin console)
echo "Enabling Tailscale HTTPS proxy..."
tailscale serve --bg ${gatewayPort} || echo "WARNING: tailscale serve failed. Enable HTTPS in your Tailscale admin console first."

echo "OpenClaw setup complete!"
`;
 });

const instance = new aws.ec2.Instance("openclaw-instance", {
 ami: ami.id,
 instanceType: instanceType,
 subnetId: subnet.id,
 vpcSecurityGroupIds: [securityGroup.id],
 keyName: keyPair.keyName,
 userData: userData,
 userDataReplaceOnChange: true,
 rootBlockDevice: {
 volumeSize: 30,
 volumeType: "gp3",
 },
 tags: { Name: "openclaw" },
});

export const publicIp = instance.publicIp;
export const publicDns = instance.publicDns;
export const privateKey = sshKey.privateKeyOpenssh;

// Construct the Tailscale MagicDNS hostname from the private IP
// AWS private IPs like 10.0.1.15 become hostnames like ip-10-0-1-15
const tailscaleHostname = instance.privateIp.apply(ip =>
 `ip-${ip.replace(/\./g, "-")}`
);

export const tailscaleUrlWithToken = pulumi.interpolate`https://${tailscaleHostname}.${tailnetDnsName}/?token=${gatewayToken}`;
export const gatewayTokenOutput = gatewayToken;

Deploying to Hetzner

Hetzner Cloud is a solid choice if you need European data residency or want to spend less money. Spoiler: it’s a lot less money.

Hetzner has similar concepts to AWS but different names. EC2 instances become Servers. Security groups become Firewalls. Same idea, different provider. The resource types come from @pulumi/hcloud.

Create a new project for Hetzner:

mkdir openclaw-hetzner && cd openclaw-hetzner
pulumi new typescript

Install the Hetzner provider:

npm install @pulumi/hcloud @pulumi/tls

The default server type cax21 is an ARM-based (Ampere) instance with 4 vCPUs and 8 GB RAM. ARM instances cost less for the same compute. If you need x86 architecture, use ccx13 or similar CCX series instead.

The Hetzner Pulumi program

Replace index.ts with the following:

import * as pulumi from "@pulumi/pulumi";
import * as hcloud from "@pulumi/hcloud";
import * as tls from "@pulumi/tls";

const config = new pulumi.Config();

const serverType = config.get("serverType") ?? "cax21";
const location = config.get("location") ?? "fsn1";
const anthropicApiKey = config.requireSecret("anthropicApiKey");
const model = config.get("model") ?? "anthropic/claude-sonnet-4";
const enableSandbox = config.getBoolean("enableSandbox") ?? true;
const gatewayPort = config.getNumber("gatewayPort") ?? 18789;
const browserPort = config.getNumber("browserPort") ?? 18791;

const tailscaleAuthKey = config.requireSecret("tailscaleAuthKey");
const tailnetDnsName = config.require("tailnetDnsName");

// Generate a random token for gateway authentication
const gatewayToken = new tls.PrivateKey("openclaw-gateway-token", {
 algorithm: "ED25519",
}).publicKeyOpenssh.apply(key => {
 const hash = require("crypto").createHash("sha256").update(key).digest("hex");
 return hash.substring(0, 48);
});

const sshKey = new tls.PrivateKey("openclaw-ssh-key", {
 algorithm: "ED25519",
});

const hcloudSshKey = new hcloud.SshKey("openclaw-sshkey", {
 publicKey: sshKey.publicKeyOpenssh,
});

const firewallRules: hcloud.types.input.FirewallRule[] = [
 {
 direction: "out",
 protocol: "tcp",
 port: "any",
 destinationIps: ["0.0.0.0/0", "::/0"],
 description: "Allow all outbound TCP",
 },
 {
 direction: "out",
 protocol: "udp",
 port: "any",
 destinationIps: ["0.0.0.0/0", "::/0"],
 description: "Allow all outbound UDP",
 },
 {
 direction: "out",
 protocol: "icmp",
 destinationIps: ["0.0.0.0/0", "::/0"],
 description: "Allow all outbound ICMP",
 },
 {
 direction: "in",
 protocol: "tcp",
 port: "22",
 sourceIps: ["0.0.0.0/0", "::/0"],
 description: "SSH access (fallback)",
 },
];

const firewall = new hcloud.Firewall("openclaw-firewall", {
 rules: firewallRules,
});

const userData = pulumi
 .all([tailscaleAuthKey, anthropicApiKey, gatewayToken])
 .apply(([tsAuthKey, apiKey, gwToken]) => {
 return `#!/bin/bash
set -e

export DEBIAN_FRONTEND=noninteractive

# System updates
apt-get update
apt-get upgrade -y

# Install Docker
curl -fsSL https://get.docker.com | sh
systemctl enable docker
systemctl start docker

# Create ubuntu user (Hetzner uses root by default)
useradd -m -s /bin/bash -G docker ubuntu || true

# Install NVM and Node.js for ubuntu user
sudo -u ubuntu bash << 'UBUNTU_SCRIPT'
set -e
cd ~

# Install NVM
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash

# Load NVM
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

# Install Node.js 22
nvm install 22
nvm use 22
nvm alias default 22

# Install OpenClaw
npm install -g openclaw@latest

# Add NVM to bashrc if not already there
if ! grep -q 'NVM_DIR' ~/.bashrc; then
 echo 'export NVM_DIR="$HOME/.nvm"' >> ~/.bashrc
 echo '[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"' >> ~/.bashrc
fi
UBUNTU_SCRIPT

# Set environment variables for ubuntu user
echo 'export ANTHROPIC_API_KEY="${apiKey}"' >> /home/ubuntu/.bashrc

# Install and configure Tailscale
echo "Installing Tailscale..."
curl -fsSL https://tailscale.com/install.sh | sh
tailscale up --authkey="${tsAuthKey}" --ssh || echo "WARNING: Tailscale setup failed. Run 'sudo tailscale up' manually."

# Enable systemd linger for ubuntu user (required for user services to run at boot)
loginctl enable-linger ubuntu

# Start user's systemd instance (required for user services during cloud-init)
systemctl start user@1000.service

# Run OpenClaw onboarding as ubuntu user (skip daemon install, do it separately)
echo "Running OpenClaw onboarding..."
sudo -H -u ubuntu ANTHROPIC_API_KEY="${apiKey}" GATEWAY_PORT="${gatewayPort}" bash -c '
export HOME=/home/ubuntu
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

openclaw onboard --non-interactive --accept-risk \
 --mode local \
 --auth-choice apiKey \
 --gateway-port $GATEWAY_PORT \
 --gateway-bind loopback \
 --skip-daemon \
 --skip-skills || echo "WARNING: OpenClaw onboarding failed. Run openclaw onboard manually."
'

# Install daemon service with XDG_RUNTIME_DIR set
echo "Installing OpenClaw daemon..."
sudo -H -u ubuntu XDG_RUNTIME_DIR=/run/user/1000 bash -c '
export HOME=/home/ubuntu
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

openclaw daemon install || echo "WARNING: Daemon install failed. Run openclaw daemon install manually."
'

# Configure gateway for Tailscale Serve (trustedProxies + skip device pairing + set token)
echo "Configuring gateway for Tailscale Serve..."
sudo -H -u ubuntu GATEWAY_TOKEN="${gwToken}" python3 << 'PYTHON_SCRIPT'
import json
import os
config_path = "/home/ubuntu/.openclaw/openclaw.json"
with open(config_path) as f:
 config = json.load(f)
config["gateway"]["trustedProxies"] = ["127.0.0.1"]
config["gateway"]["controlUi"] = {
 "enabled": True,
 "allowInsecureAuth": True
}
config["gateway"]["auth"] = {
 "mode": "token",
 "token": os.environ["GATEWAY_TOKEN"]
}
with open(config_path, "w") as f:
 json.dump(config, f, indent=2)
print("Configured gateway with trustedProxies, controlUi, and token")
PYTHON_SCRIPT

# Enable Tailscale HTTPS proxy (requires HTTPS to be enabled in Tailscale admin console)
echo "Enabling Tailscale HTTPS proxy..."
tailscale serve --bg ${gatewayPort} || echo "WARNING: tailscale serve failed. Enable HTTPS in your Tailscale admin console first."

echo "OpenClaw setup complete!"
`;
 });

const server = new hcloud.Server("openclaw-server", {
 serverType: serverType,
 location: location,
 image: "ubuntu-24.04",
 sshKeys: [hcloudSshKey.id],
 firewallIds: [firewall.id.apply(id => Number(id))],
 userData: userData,
 labels: {
 purpose: "openclaw",
 },
});

export const ipv4Address = server.ipv4Address;
export const privateKey = sshKey.privateKeyOpenssh;

// Construct the Tailscale MagicDNS hostname from the server name
// Hetzner servers use their name as the hostname
const tailscaleHostname = server.name;

export const tailscaleUrlWithToken = pulumi.interpolate`https://${tailscaleHostname}.${tailnetDnsName}/?token=${gatewayToken}`;
export const gatewayTokenOutput = gatewayToken;

You can find both programs in the Pulumi examples repo under openclaw/:

github.com/pulumi/examples

Cost comparison

Before deploying, let’s compare the costs between AWS and Hetzner for running OpenClaw 24/7:

	AWS (t3.medium)	Hetzner (cax21)
vCPUs	2	4
Memory	4 GB	8 GB
Storage	30 GB gp3 (+$2.40/mo)	80 GB NVMe (included)
Traffic	Pay per GB	20 TB included
Architecture	x86 (Intel/AMD)	ARM (Ampere)
Hourly price	$0.0416	€0.0104 (~$0.011)
Monthly price	~$33 (with storage)	€6.49 (~$7)
Annual cost	~$396	~$84

Hetzner gives you double the vCPUs, double the RAM, at less than a quarter of the price. The trade-off? ARM architecture instead of x86. But OpenClaw doesn’t care - it’s just Node.js and Docker.

Prices are for on-demand instances as of January 2026. AWS prices are for us-east-1; Hetzner prices exclude VAT. Both include standard networking and storage. Check AWS EC2 pricing and Hetzner Cloud pricing for current rates.

Running the deployment

With your ESC environment configured in Pulumi.dev.yaml, deploy with:

pulumi up

After deployment completes, you’ll see outputs similar to:

Outputs:
 gatewayTokenOutput : "786c099cc8f8bf20dbebf40b8b51b75cf5cdab25..."
 privateKey : [secret]
 publicDns : "ec2-x-x-x-x.compute-1.amazonaws.com"
 publicIp : "x.x.x.x"
 tailscaleUrlWithToken: "https://ip-10-0-1-x.tailxxxxx.ts.net/?token=786c099..."

The tailscaleUrlWithToken output provides the complete URL with authentication token. Copy and paste it into your browser to access the OpenClaw web UI.

Output names vary slightly between providers: AWS uses publicIp and publicDns, while Hetzner uses ipv4Address. The Tailscale hostname is derived from the instance’s private IP (AWS) or server name (Hetzner).

Automated onboarding

The Pulumi program runs OpenClaw’s non-interactive onboarding during instance provisioning. It uses your Anthropic API key from ESC, binds the gateway to loopback with Tailscale Serve as the HTTPS proxy, generates a secure gateway token (exported in Pulumi outputs), installs the daemon as a systemd user service, and configures trustedProxies and controlUi.allowInsecureAuth to skip device pairing when accessed via Tailscale.

The cloud-init script runs openclaw onboard --non-interactive with all necessary flags, then configures the gateway for secure Tailscale access. Your instance is ready as soon as provisioning finishes.

Access the web UI

Please wait 3-5 minutes after pulumi up completes. The cloud-init script runs after the instance launches and needs time to install Docker, Node.js, OpenClaw, and Tailscale, then run the onboarding process and start the daemon. Periodically refresh the page until it loads. If the page still doesn’t load after 5 minutes, see Verify the deployment to troubleshoot.

The easiest way to access the OpenClaw web UI is to use the tailscaleUrlWithToken output from Pulumi:

# Get the full URL with token
pulumi stack output tailscaleUrlWithToken

Copy and paste this URL into your browser. The URL includes both the Tailscale MagicDNS hostname and the authentication token, so you can access the web UI directly.

Finding your tailnet DNS name: Go to the Tailscale admin console and look under the DNS section for your tailnet name (e.g., tailxxxxx.ts.net). You can also find your machines and their MagicDNS hostnames in the Machines tab.

Token-based authentication provides an additional layer of security on top of Tailscale’s network-level authentication. Only devices on your tailnet can reach the URL, and the token prevents unauthorized access if someone gains access to your tailnet.

From the web UI, you can connect messaging channels (WhatsApp, Discord, Slack), configure skills and integrations, and manage settings.

Verify the deployment (optional)

If you encounter issues accessing the web UI, you can SSH into your instance to troubleshoot:

# Check your Tailscale admin console for the new machine
ssh ubuntu@<tailscale-ip>

# Check OpenClaw gateway status
systemctl --user status openclaw-gateway

Test your assistant

Open the gateway dashboard using the tailscaleUrlWithToken output and use the built-in chat to test your assistant:

Your personal AI assistant is now running 24/7 on your own infrastructure, accessible securely through Tailscale.

Security considerations

When self-hosting an AI assistant, security matters. OpenClaw’s rapid adoption meant thousands of instances spun up in days, and not everyone locked them down. The community noticed:

The tweet isn’t exaggerating. A quick Shodan search shows exposed gateways on port 18789 with shell access, browser automation, and API keys up for grabs:

Don’t let your instance be one of them.

Concern	Without Tailscale	With Tailscale
SSH access	Public (port 22 open)	Public fallback + Tailscale SSH
Gateway access	Public (port 18789 open)	Private (Tailscale only)
Browser control	Public (port 18791 open)	Private (Tailscale only)
API keys in transit	Exposed if gateway accessed over HTTP	Protected by Tailscale encryption
Attack surface	3 open ports	1 open port (SSH fallback)

SSH remains accessible as a fallback even with Tailscale enabled. This allows you to troubleshoot if Tailscale fails to connect. Once you’ve confirmed Tailscale is working, you can manually remove the SSH ingress rule from your security group for maximum security.

My recommendations:

Always use Tailscale for production
Rotate your auth keys periodically
Use Pulumi ESC for secrets instead of hardcoding
Enable Tailscale SSH to avoid managing keys manually
Monitor your Tailscale admin console for unauthorized devices
Remove the SSH fallback after confirming Tailscale works if you want zero public ports

What’s next?

Now that OpenClaw is running, you can install skills (voice generation, video creation, browser automation), set up scheduled tasks with cron, invite colleagues to your tailnet for shared access, or connect additional channels like WhatsApp and Discord.

For a deeper case study of running an AI assistant on Pulumi-managed infrastructure, see how we built Platybot, or read our KubeCon EU 2026 recap on the year AI moved into production on Kubernetes.

Conclusion

Deploying OpenClaw with infrastructure as code means you can reproduce your setup anytime, version control it, and tear it down with a single command. Adding Tailscale keeps it private - no exposed ports, no hoping you configured your firewall correctly at 2am.

If you run into issues or have questions, drop by the Pulumi Community Slack or GitHub Discussions.

New to Pulumi? Get started here.

Neo: Zero-downtime migration from CDK, Terraform & Azure ARM

Pulumi Neo Team — Wed, 21 Jan 2026 00:00:00 +0000

The barrier to migrating to Pulumi has always been the infrastructure you already have. Your existing resources can’t be disrupted, and manually importing them into a new tool is risky and time-consuming. Today, we’re excited to share how Neo removes this barrier entirely with automated, zero-downtime migration to Pulumi from AWS CDK, AWS CloudFormation, Terraform, CDKTF, and Azure ARM templates.

Why IaC migrations are hard

The promise of Infrastructure as Code is that your code perfectly describes your running infrastructure. But switching IaC tools breaks this promise in dangerous ways.

When you rewrite your infrastructure code in a new tool, you have two choices, both problematic. You can destroy and recreate all your resources to match the new code, accepting downtime and risk. Or you can try to import existing resources, which requires perfect knowledge of how every resource maps between the old and new systems. Many teams get stuck here, wanting Pulumi’s modern platform but unable to safely make the switch.

Neo’s insight: State knowledge enables perfect migrations

The key to safe migration isn’t just converting code - it’s understanding the complete relationship between your existing IaC tool’s state and your actual cloud resources. Each IaC tool maintains this relationship differently: CDK through CloudFormation stacks, Terraform/CDKTF through state files, and ARM through Azure deployments. But they all have complete knowledge of what they manage.

Neo leverages this existing state knowledge to orchestrate perfect resource transitions. Instead of asking you to manually find resource IDs and construct migration commands, Neo reads your current tool’s state, discovers every resource’s physical identity, and brings them under Pulumi management. This isn’t just automation - it’s using the source tool’s own knowledge against the migration problem.

In practice, this means Neo can bridge the gap between how tools name resources and where they actually live. A Lambda function that CDK knows as OrderHandler9I0J1K2L actually exists in AWS as my-app-OrderHandler-9I0J1K2L, while a Terraform resource at address aws_instance.web[2] maps to EC2 instance i-0abc123def456. Neo understands these mappings and handles the complex cases like composite IDs (FunctionName|StatementId), resource references, and dependency chains that must be migrated in order.

Because Neo uses the source tool’s own state knowledge, your infrastructure doesn’t change at all during migration. Not a single resource is modified, recreated, or even touched. We’re simply transferring ownership from one IaC tool to another.

This approach delivers three critical guarantees:

Zero downtime: Resources are never deleted or recreated
Zero risk: Since nothing changes, you can abandon the migration at any point without consequence
Zero surprises: Preview confirms no infrastructure changes before you commit

Migration in action: Four tools, one approach

Neo adapts its migration strategy to each tool’s unique characteristics while maintaining the same zero-downtime guarantee. Let’s explore how Neo handles migrations from each major IaC tool.

AWS CDK to Pulumi

For teams using AWS CDK, Neo leverages the CloudFormation layer that underpins CDK deployments. CDK’s architecture actually makes migration straightforward: since CDK synthesizes to CloudFormation templates, Neo can read the deployed stacks directly to understand every resource and its configuration. For detailed migration steps, see our CDK migration guide.

The challenge with CDK migrations isn’t the CloudFormation layer - it’s the cryptic resource naming. CDK generates logical IDs like OrdersTableA7B2C3D4 that map to physical resources with completely different names. These mappings are buried in CloudFormation metadata, and getting them wrong means either orphaning resources or accidentally creating duplicates. Neo navigates this complexity by reading CloudFormation’s own stack outputs and resource metadata, discovering the exact physical ID for every logical resource.

CDK also introduces complexity through its construct hierarchy. A single high-level construct might expand into dozens of CloudFormation resources, each with dependencies and references to others. Neo preserves these relationships during migration, ensuring that IAM roles still reference the right Lambda functions, API Gateway deployments still point to the correct stages, and security groups maintain their exact rules. The migration completes with your infrastructure unchanged and Pulumi’s preview confirming zero modifications.

Start a Neo task Migrate your CDK application

AWS CloudFormation to Pulumi

For teams using CloudFormation directly (rather than through CDK), Neo provides a streamlined migration path. CloudFormation stacks contain complete resource metadata - every resource’s logical ID maps to a physical resource in AWS, and CloudFormation tracks these relationships in its stack state. Neo reads this state directly to build a complete picture of your infrastructure. For detailed migration steps, see our CloudFormation migration guide.

The main challenge with CloudFormation migrations is the template language itself. CloudFormation templates use intrinsic functions like !Ref, !GetAtt, and !Sub that create implicit dependencies between resources. A security group might reference a VPC using !Ref MyVpc, while a Lambda function’s role uses !GetAtt LambdaRole.Arn. Neo evaluates these expressions against the actual deployed stack to resolve every reference to its concrete value.

CloudFormation also supports features like conditionals, mappings, and nested stacks that add layers of indirection. Neo handles these by examining what actually got deployed rather than trying to interpret every possible template path. The result is Pulumi code that manages your exact infrastructure configuration - not a theoretical interpretation of your template. The migration completes with pulumi preview confirming zero changes to your running resources.

Start a Neo task Migrate your CloudFormation stack

Terraform to Pulumi

Terraform and CDKTF migrations require two transformations: converting state to establish Pulumi’s connection to your resources, and transforming HCL configuration into Pulumi code. The state conversion is direct - Neo reads your Terraform state and converts it into Pulumi state, preserving the mappings between resource names and cloud IDs. This ensures Pulumi knows that aws_instance.web[2] corresponds to EC2 instance i-0abc123def456.

The code transformation analyzes your HCL to generate equivalent Pulumi code. Neo handles Terraform patterns like count and for_each loops, module structures, and resource dependencies, recreating them idiomatically in Pulumi. The generated code manages your existing infrastructure without modifications - pulumi preview confirms zero changes. For complete migration instructions, see our Terraform migration guide.

Start a Neo task Migrate your Terraform application

Azure ARM to Pulumi

ARM templates present unique migration challenges. Unlike CDK and Terraform, which maintain clear separation between code and state, ARM templates blur this line. The template is both the definition and, through deployment history, part of the state tracking. ARM’s template expression language, with its concat functions and resource ID constructors, makes it difficult to determine what resources actually exist until deployment time. For step-by-step migration guidance, see our ARM migration guide.

Neo orchestrates ARM migrations through intelligent AI-driven conversion. When an ARM template uses functions like concat(parameters('appName'), '-plan'), the conversion process evaluates these expressions using the actual parameter values to generate the correct resource names. Azure resource IDs follow predictable patterns - subscription IDs, resource groups, providers, and resource names - and Neo ensures these are correctly brought under Pulumi management using inline resource specifications directly in the generated code.

The biggest challenge with ARM migrations is handling the implicit dependencies and resource provider quirks. An App Service might implicitly create a service plan, a SQL database requires a server that might be defined in a linked template, and child resources like application settings need separate migration steps. Neo understands these Azure-specific patterns and generates the appropriate Pulumi code to manage every resource. The migration completes with a zero-diff preview, confirming your exact Azure configuration is preserved while giving you a more maintainable, type-safe way to manage it going forward.

Start a Neo task Migrate your ARM template

The architecture powering it all

While each tool requires specific handling, Neo’s core architecture remains consistent:

Universal orchestration layer

Neo acts as the intelligent migration coordinator, regardless of source tool:

Credential verification: Ensures proper cloud credentials are configured in Pulumi ESC
Resource inventory: Builds a complete catalog of existing resources
Conversion orchestration: Manages the code transformation
State migration: Brings existing resources under Pulumi management
Audit trail generation: Creates comprehensive migration reports

Unified state management engine

The state management engine works consistently across all tools:

Maps source resource IDs to Pulumi’s state management
Handles complex and composite resource identifiers
Provides fallback strategies for edge cases
Ensures idempotent operations

Human oversight

While Neo automates the heavy lifting, we maintain human checkpoints:

Review generated code before migrating resources
Verify preview shows zero changes
Approve the resulting pull request

What this means for your team

Migration friction no longer locks you into your current IaC tool. If you want Pulumi’s programming model, policy engine, and multi-cloud support, Neo gets you there without disrupting your infrastructure.

Ready to migrate? Check out our migration guides for CDK, CloudFormation, Terraform, or Azure ARM. Join us in the Pulumi Community Slack or reach out to your account team for a guided migration session.

Self-Verifying AI Agents: Vercel's Agent-Browser in the Ralph Wiggum Loop

Engin Diri — Tue, 20 Jan 2026 00:00:00 +0000

In my previous post about the Ralph Wiggum technique, Claude Code built a complete serverless URL shortener on AWS. The setup included Playwright MCP for end-to-end testing. It worked. But I kept wondering if there was something better for AI-driven browser automation. Then Vercel released agent-browser, and I had to try it.

Why browser automation matters for AI coding

When an AI coding agent builds a frontend, someone has to verify it works. Without browser automation, that someone is you. The AI finishes and says “done,” but you can’t trust that claim until you open a browser and click around yourself.

Browser automation changes this. The AI verifies its own work. It builds a component, launches a browser, tests the interaction, confirms behavior matches expectations. If something breaks, it fixes the code and tests again. The validation loop runs without you.

This matters more as AI agents take on larger tasks. A quick component fix might not need automated testing. But when an agent builds an entire dashboard from scratch, you need proof that the deployed application actually works.

The context problem with browser automation

Playwright MCP gives you powerful browser control, but it comes with overhead. Every screenshot, every DOM snapshot, every accessibility tree adds tokens to your context window. GitHub issue #889 documents a 6x token increase between versions 0.0.30 and 0.0.32. Users report single screenshots consuming over 15,000 tokens. Some exhausted their entire five-hour token allocation in just a few automation steps.

The problem is verbose output. Full accessibility trees contain every element on the page with all their properties. Console message logging adds more. Each piece of information might be useful for debugging, but together they overwhelm the context window.

This creates a frustrating tradeoff. You want rich browser interaction for thorough testing, but that richness costs tokens. Fewer tokens means fewer iterations before hitting limits.

Vercel’s “less is more” philosophy

Vercel ran into something similar with their D0 text-to-SQL agent. Their research documents what happened when they reduced tool complexity.

The original architecture used 17 specialized tools. Each tool handled one specific operation: create tables, insert rows, run queries, validate schemas. The approach seemed logical. Specialized tools should produce better results than general ones.

The numbers told a different story. With 17 tools, they got 80% success (4 out of 5 queries), averaging 274.8 seconds and ~102,000 tokens. The worst case took 724 seconds, burned 145,463 tokens, and still failed.

Then they rebuilt with just two tools: ExecuteCommand and ExecuteSQL. Success jumped to 100%. Average execution dropped to 77.4 seconds (3.5x faster). Token usage fell to ~61,000 (37% reduction). The worst case took 141 seconds, used 67,483 tokens, and succeeded.

Vercel’s takeaway: “We were constraining reasoning because we didn’t trust the model to reason.” Fewer tools meant the model could think more freely about how to accomplish tasks. The simplicity reduced confusion and context waste.

Applying “less is more” to browser automation

agent-browser takes the same approach. Instead of separate tools for clicking, typing, scrolling, and navigating, it has a unified CLI with one clever idea: the snapshot + refs system.

When you request a page snapshot, agent-browser returns something like this:

- button "Sign In" [ref=e1]
- textbox "Email" [ref=e2]
- textbox "Password" [ref=e3]
- link "Documentation" [ref=e4]

Those @e1, @e2 references are stable element identifiers. To click the sign-in button, you run click @e1. No CSS selectors. No XPath expressions. No waiting for the DOM to stabilize. The reference points to that exact element.

This cuts out the verbosity of full accessibility trees. You get just enough information to understand the page structure and interact with elements. The AI can reason about what it sees without drowning in metadata.

Installing agent-browser

Setup is simple:

npm install -g agent-browser && agent-browser install

This installs the CLI and downloads a bundled browser. You don’t need to install any additional browsers.

For Claude Code integration, install the official skill from the agent-browser package. You can either copy it from node_modules:

cp -r node_modules/agent-browser/skills/agent-browser .claude/skills/

Or download it directly:

mkdir -p .claude/skills/agent-browser
curl -o .claude/skills/agent-browser/SKILL.md \
 https://raw.githubusercontent.com/vercel-labs/agent-browser/main/skills/agent-browser/SKILL.md

The skill gives Claude Code better context than just running agent-browser --help.

Unlike Playwright MCP, there’s no server configuration. Agent-browser runs as a standalone CLI that Claude Code invokes directly through bash.

The experiment: adding analytics

I wanted a realistic test scenario, so I added an analytics dashboard to the url-shortener-saas project from my previous post. The feature tracks click events with metadata (browser, device, country, referrer) and displays them through interactive charts.

This follows the same “Ralph Wiggum” workflow from my previous post: write a feature-prompt.md that describes what you want, then let Claude Code implement, deploy with Pulumi, and verify with browser automation. Here’s a snippet from the feature prompt:

## Backend Requirements

### New DynamoDB Table: Analytics Events

Create a new DynamoDB table `url-shortener-analytics-{stack}` with:
- Partition Key: shortCode (String)
- Sort Key: timestamp (String, ISO 8601 format)
- Attributes: browser, deviceType, country, referrer, etc.

### New Lambda: analytics.ts

GET /api/analytics/{shortCode}
- Returns timeline, browsers, devices, countries, referrers
- Query params: from, to, granularity

## Success Criteria

1. `pulumi up` deploys successfully without errors
2. All existing E2E tests still pass
3. New analytics E2E tests pass
4. Charts render with real data after generating test clicks

Claude Code reads this, writes the Lambda handlers, updates the Pulumi infrastructure code, and deploys. The infrastructure changes are straightforward:

// Analytics events table
const analyticsTable = new aws.dynamodb.Table("analytics-table", {
 name: `${projectName}-analytics-${stack}`,
 billingMode: "PAY_PER_REQUEST",
 hashKey: "shortCode",
 rangeKey: "timestamp",
 attributes: [
 { name: "shortCode", type: "S" },
 { name: "timestamp", type: "S" },
 ],
 globalSecondaryIndexes: [{
 name: "by-date",
 hashKey: "shortCode",
 rangeKey: "timestamp",
 projectionType: "ALL",
 }],
});

After pulumi up succeeds, the agent needs to verify the deployed feature works. That’s where browser automation comes in. The feature prompt specifies verification requirements:

## End-to-End Verification

After deployment, use the `/agent-browser` skill to verify:
1. Analytics link appears in navigation
2. Analytics page loads without errors
3. Charts render (timeline, browser, device, geographic)
4. URL-specific analytics work from dashboard
5. Click data appears after generating test clicks

The test suite covered six scenarios: homepage load, URL shortening, dashboard view, analytics navigation, analytics overview, and date filter functionality. Nothing fancy, just the basics you’d want to verify after deploying.

I ran the exact same tests with both Playwright MCP and agent-browser to see how much context each consumed.

Agent-browser workflow in practice

Here’s what the CLI interaction looked like during my test run.

Navigate to the deployed application:

$ agent-browser open https://d1232drths1aav.cloudfront.net
✓ Snip.ly - Modern URL Shortener
 https://d1232drths1aav.cloudfront.net/

Snapshot to see available elements (the -i flag filters to interactive elements only):

$ agent-browser snapshot -i
- link "S Snip.ly" [ref=e1]
- link "Home" [ref=e2]
- link "Dashboard" [ref=e3]
- link "Analytics" [ref=e4]
- link "Docs" [ref=e5]
- button "Switch to dark mode" [ref=e6]
- link "Get Started" [ref=e7]
- textbox "Paste your long URL here..." [ref=e8]
- button "Shorten URL" [ref=e9]

That entire homepage snapshot is 280 characters. Playwright MCP returned 8,247 characters for the same page.

Fill the URL input and click the shorten button:

$ agent-browser fill @e8 "https://example.com/agent-browser-e2e-test"
✓ Done

$ agent-browser click @e9
✓ Done

Each action confirmation is 6 characters. Playwright MCP returns the full page state after every click - 12,891 characters when I clicked the shorten button.

Navigate to analytics:

$ agent-browser click @e4
✓ Done

$ agent-browser wait --load networkidle && agent-browser snapshot -i
✓ Done
- link "S Snip.ly" [ref=e1]
- link "Home" [ref=e2]
- link "Dashboard" [ref=e3]
- link "Analytics" [ref=e4]
- button "Last 7 days" [ref=e8]
- button "Last 30 days" [ref=e9]
- button "Last 90 days" [ref=e10]
- link "1 analytics-e2e-test https://example.com/analytics-test-verification 5" [ref=e11]
- link "2 test-url https://www.example.com/this-is-a-very-long-url... 4" [ref=e12]

The analytics snapshot shows date filter buttons and top URLs with click counts. 385 characters versus Playwright MCP’s 4,127.

Test the date filter:

$ agent-browser click @e9
✓ Done

$ agent-browser screenshot analytics-30-day-filter.png
✓ Screenshot saved to analytics-30-day-filter.png

The ref-based workflow is deterministic. Each command operates on a specific element from the snapshot. No guessing about selectors.

The numbers: 82.5% context reduction

Same six tests, both tools:

Metric	Playwright MCP	Agent-Browser	Reduction
Total response characters	31,117	5,455	82.5%
Largest single response	12,891	2,847	77.9%
Average response size	3,112	328	89.5%
Homepage snapshot	8,247	280	96.6%
Dashboard snapshot	12,891	2,847	77.9%

The difference is what each tool returns after every action.

Playwright MCP returns the full accessibility tree after every click:

### Page state
- Page URL: https://d1232drths1aav.cloudfront.net/dashboard
- Page Title: Snip.ly - Modern URL Shortener
- Page Snapshot:
- generic [ref=e2]:
 - banner [ref=e3]:
 - generic [ref=e4]:
 - link "S Snip.ly" [ref=e5] [cursor=pointer]:
 - /url: /
 - generic [ref=e6]: S
 - generic [ref=e7]: Snip.ly
 - navigation [ref=e8]:
 - link "Home" [ref=e9] [cursor=pointer]:
 - /url: /
 - text: Home
 ...
[12,891 characters continues...]

Agent-browser returns:

✓ Done

6 characters versus 12,891 for the same button click.

Six tests consumed ~31K characters with Playwright MCP versus ~5.5K with agent-browser. At roughly 4 characters per token, that’s ~7,800 tokens versus ~1,400. An AI agent could run 5.7x more tests in the same context budget.

What I noticed

Past the numbers, working with agent-browser felt different in ways that are hard to quantify.

The compact snapshots changed the rhythm of iteration. A typical page fit in a few hundred tokens instead of thousands, so Claude Code could run more test cycles before hitting limits. I spent less time worrying about context and more time actually testing.

Element refs removed a frustration I’d had with Playwright. CSS selectors break when someone changes a class name or restructures the DOM. With agent-browser, a button is just @e9. It doesn’t care about styling. That predictability was a relief.

What worked well:

Snapshots stay small enough that you don’t think about them
Refs mean no selector debugging
The CLI just works with Claude Code’s bash tool
No MCP server to configure

Agent-browser is early. Documentation is thin, and I read the source more than once to figure out edge cases.

Where I hit friction:

Modals that appear after API calls needed manual wait logic
Playwright’s waiting mechanisms are more mature
Hidden or dynamically loaded elements sometimes didn’t show up without explicit waits

For my URL shortener tests, agent-browser used fewer tokens per cycle, and the ref-based approach was more predictable than selector-driven automation.

Playwright MCP still wins on depth. Network interception, multi-tab handling, PDF generation, better waiting logic—if you need those, you need Playwright. For complex browser automation, it’s the more capable tool.

When to use each

Agent-browser fits long autonomous sessions where context budget matters, basic navigation and verification tasks, and setups where you want to skip MCP configuration. CLI-based skills also avoid the schema overhead—tool definitions for multi-tool MCPs eat tokens even when idle.

Playwright MCP fits when you need the advanced stuff: network interception, PDF generation, multi-tab workflows, or sophisticated synchronization. It’s also the right choice if you have existing Playwright test suites. If the full MCP schema feels heavy, Playwright skills offer a lighter wrapper.

Start with agent-browser for AI validation loops. Move to Playwright when you outgrow it.

How Ralph Wiggum Built a Serverless SaaS with Pulumi

Engin Diri — Wed, 14 Jan 2026 00:00:00 +0000

I was about to do something that felt either genius or completely reckless: hand over my AWS credentials to an AI and step away from my computer. The technique is called “Ralph Wiggum,” named after the Simpsons character who eats glue and says “I’m in danger” while everything burns around him. And honestly, that felt about right for what I was attempting.

The problem with AI coding assistants

If you have spent any time with AI coding assistants, you know the frustration. You are in the middle of a task. It is going great. Claude is writing beautiful infrastructure code, understanding your architecture, making progress… and then it says “I have completed your request.”

You stare at the screen. “No. No, you have not. You maybe did a third of what I asked.”

So you reprompt it. It does a little more. Then it stops again. You are babysitting. The whole point of using an AI assistant was to save time, but if you have to check on it every five minutes, you are not saving anything. You are just a very expensive supervisor.

This is the problem Geoffrey Huntley decided to solve.

What if we just do not let it stop?

Geoffrey Huntley, a developer based in rural Australia (who, according to internet lore, lives on a property with goats), had a deceptively simple idea: what if every time Claude Code finishes and tries to exit, we just feed it the prompt again?

He named the technique “Ralph Wiggum” because the character embodies a kind of childlike persistence. Ralph repeatedly fails, makes silly mistakes, yet stubbornly continues in an endless loop until he eventually succeeds. Sound familiar? That is exactly how AI coding agents work. They make mistakes. They try again. They iterate.

The philosophy behind Ralph is beautifully stated: “Better to fail predictably than succeed unpredictably.” Every failure is just data for the next iteration.

How the Ralph loop works

At its core, the Ralph Wiggum technique is almost embarrassingly simple. It is a bash loop:

while true; do
 cat PROMPT.md | claude --print --dangerously-skip-permissions
done

That is it. That is the whole thing. You write your instructions in a PROMPT.md file, and the loop keeps feeding it to Claude Code over and over.

But here is why it works: each time Claude starts up, it looks at the project. It sees the files that exist, the code that is already written, the git history. It is not starting from zero. It picks up where it left off, sees what still needs to be done, and keeps going.

The official Claude Code plugin formalizes this with some important additions:

Completion promises: You tell Claude to output a specific string like <promise>COMPLETE</promise> when it has genuinely finished
Max iterations: A safety limit so your API bill does not reach infinity
Stop hooks: The mechanism that intercepts exit attempts and re-injects the prompt

To install it:

/plugin install ralph-wiggum@ghuntley

For end-to-end testing of deployed infrastructure, you will also want the Playwright MCP server. This lets Claude interact with your deployed application in a real browser:

claude mcp add playwright npx @playwright/mcp@latest

Why Pulumi is perfect for Ralph

Infrastructure as code has something most application code does not: objective success criteria. Your Lambda either deploys or it does not. Your DynamoDB table either exists or it does not. Tests either pass or they fail.

This makes Pulumi an ideal candidate for autonomous AI development:

Clear completion signals: pulumi preview and pulumi up provide unambiguous feedback
Iterative feedback: Failed deployments tell Claude exactly what went wrong
Testable outputs: You can write integration tests that verify your infrastructure actually works
Registry access: With the Pulumi MCP server, Claude has real-time access to documentation and examples

The experiment

I decided to build a serverless URL shortener on AWS. Not because the world needs another URL shortener, but because it is the perfect test case: multiple AWS services, clear requirements, and objective success criteria. But I did not want just a basic demo. I wanted a full SaaS-like experience with a polished frontend.

Here is the PROMPT.md file I wrote:

# Build a Production-Ready URL Shortener SaaS on AWS

Using Pulumi TypeScript, create a complete URL shortener SaaS with a polished frontend experience.

## Infrastructure Requirements
- DynamoDB table for storing URL mappings (shortCode -> originalUrl, clickCount, createdAt)
- Lambda functions for: creating short URLs, redirecting to original URLs, getting stats
- API Gateway REST API exposing the Lambda functions
- S3 bucket for the React frontend with static website hosting
- CloudFront distribution with HTTPS for the frontend and API
- Use the Pulumi ESC environment pulumi-idp/auth for the AWS credentials

## Frontend Requirements (Use /frontend-design skill)
Create a polished, production-ready SaaS website with:

### Landing Page
- Hero section with catchy headline and URL input form
- Feature highlights (fast redirects, analytics, custom aliases)
- Pricing section (Free tier, Pro tier, Enterprise tier)
- Testimonials section with 3 fake but realistic customer reviews
- Footer with links to Docs, Privacy, Terms

### Dashboard Page
- URL shortening form with optional custom alias
- List of created short URLs with click counts
- Copy-to-clipboard functionality
- Delete URL option

### Documentation Page
- Getting started guide
- API reference (POST /shorten, GET /{code}, GET /stats/{code})
- Rate limits and usage policies
- FAQ section

### Design Requirements
- Modern, clean aesthetic (no generic AI look)
- Responsive design (mobile, tablet, desktop)
- Dark mode support
- Smooth animations and transitions
- Consistent color scheme and typography

## API Requirements
- POST /shorten: accepts { url: string, alias?: string }, returns { shortCode, shortUrl }
- GET /{shortCode}: redirects to original URL (301)
- GET /stats/{shortCode}: returns { originalUrl, clickCount, createdAt }

## Success Criteria
- All unit tests pass
- All integration tests pass
- pulumi preview shows no errors
- pulumi up deploys successfully

## End-to-End Verification (Required)
After deployment, use Playwright MCP to verify the live site:
1. Open the CloudFront URL and verify the landing page loads with all sections
1. Navigate to the docs page and verify content renders
1. Create a short URL using the dashboard
1. Verify the short URL redirects correctly (301 status)
1. Check the stats show the click was recorded
1. Test responsive design by resizing the viewport
1. Take screenshots of landing page, dashboard, and docs as proof

Only output <promise>COMPLETE</promise> after ALL of the above including E2E tests pass.

The key addition here is the end-to-end verification section. Without it, Claude reports success after pulumi up finishes, but you have no guarantee the deployed infrastructure actually works. By requiring Playwright to test the live URLs, you catch issues like the CloudFront 403 error I mentioned earlier.

Before running the loop, I started Claude Code with permission bypass mode to prevent it from stopping to ask for approval on every file write or command:

claude --permission-mode bypassPermissions

There are other ways to handle permissions, but this is the simplest for autonomous execution. Then I ran the Ralph loop:

/ralph-wiggum:ralph-loop PROMPT.md --max-iterations 25 --completion-promise "COMPLETE"

Then I stepped away and let it run unsupervised. For complex projects, you could even let it run overnight while you sleep.

Here is Claude Code in action during the Ralph loop, fixing CloudFront configuration issues and writing unit tests:

The results

The complete source code for this project is available on GitHub: url-shortener-saas

When Claude finished, it had:

Successfully built:

A complete Pulumi program with proper resource organization
DynamoDB table with a GSI for querying by creation date
Three Lambda functions with proper error handling
API Gateway with CORS configuration
S3 bucket with static website hosting
CloudFront distribution with proper cache behaviors

The frontend was genuinely impressive:

A polished landing page with hero section, feature cards, and pricing tiers
Working testimonials section with three fictional but believable customer reviews
A functional dashboard where you could create and manage short URLs
A documentation page with API reference and getting started guide
Dark mode toggle that actually worked
Responsive design that looked good on mobile

Home screen

Dashboard

Documentation

Here is a snippet of what it generated:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const stack = pulumi.getStack();
const projectName = "url-shortener";

// DynamoDB Table
const urlTable = new aws.dynamodb.Table("url-table", {
 name: `${projectName}-urls-${stack}`,
 billingMode: "PAY_PER_REQUEST",
 hashKey: "shortCode",
 attributes: [{ name: "shortCode", type: "S" }],
 tags: {
 Environment: stack,
 Project: projectName,
 ManagedBy: "Pulumi",
 },
});

// Lambda function for URL shortening
const shortenFunction = new aws.lambda.Function("shorten-function", {
 name: `${projectName}-shorten-${stack}`,
 runtime: aws.lambda.Runtime.NodeJS20dX,
 handler: "shorten.handler",
 role: lambdaRole.arn,
 timeout: 30,
 memorySize: 256,
 code: new pulumi.asset.AssetArchive({
 ".": new pulumi.asset.FileArchive("./lambda"),
 }),
 environment: {
 variables: {
 TABLE_NAME: urlTable.name,
 },
 },
});

// CloudFront Distribution with S3 and API Gateway origins
const distribution = new aws.cloudfront.Distribution("frontend-distribution", {
 enabled: true,
 defaultRootObject: "index.html",
 origins: [
 {
 domainName: frontendBucketWebsite.websiteEndpoint,
 originId: "S3Origin",
 customOriginConfig: {
 httpPort: 80,
 httpsPort: 443,
 originProtocolPolicy: "http-only",
 originSslProtocols: ["TLSv1.2"],
 },
 },
 {
 domainName: pulumi.interpolate`${api.id}.execute-api.${aws.config.region}.amazonaws.com`,
 originId: "APIGatewayOrigin",
 customOriginConfig: {
 httpPort: 80,
 httpsPort: 443,
 originProtocolPolicy: "https-only",
 originSslProtocols: ["TLSv1.2"],
 },
 },
 ],
 defaultCacheBehavior: {
 targetOriginId: "S3Origin",
 viewerProtocolPolicy: "redirect-to-https",
 allowedMethods: ["GET", "HEAD", "OPTIONS"],
 cachedMethods: ["GET", "HEAD"],
 forwardedValues: {
 queryString: false,
 cookies: { forward: "none" },
 },
 },
 orderedCacheBehaviors: [{
 pathPattern: "/api/*",
 targetOriginId: "APIGatewayOrigin",
 viewerProtocolPolicy: "https-only",
 allowedMethods: ["DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT"],
 cachedMethods: ["GET", "HEAD"],
 forwardedValues: {
 queryString: true,
 headers: ["Authorization", "Content-Type"],
 cookies: { forward: "none" },
 },
 minTtl: 0,
 defaultTtl: 0,
 maxTtl: 0,
 }],
 viewerCertificate: {
 cloudfrontDefaultCertificate: true,
 },
});

Where it struggled:

The first few iterations had the Lambda code inline instead of in separate files
It initially forgot to set up IAM permissions for the Lambda to access DynamoDB
The CloudFront configuration took three attempts to get the cache behaviors right

The funny parts:

At iteration 12, it decided to completely refactor the project structure. Then at iteration 13, it refactored it back
One commit message simply read: “fix the fix that fixed the previous fix”
It wrote a test, ran the test, fixed the code to pass the test, then realized the test was wrong

Best practices for Ralph with Pulumi

After running several experiments, here is what I have learned:

Write extremely specific prompts. Vague instructions lead to vague implementations. Be explicit about every requirement, every edge case, every success criterion.

Use pulumi preview as your feedback loop. Add this to your success criteria:

Success Criteria:
- pulumi preview completes with no errors
- pulumi preview shows expected resource count (approximately X resources)

Set realistic iteration limits. For infrastructure projects, I have found 20-30 iterations is usually sufficient. More than that and you are probably missing something in your prompt.

Include test requirements. Claude is surprisingly good at test-driven development when you tell it to write tests first:

## Development Approach
1. Write unit tests for each Lambda function first
1. Implement Lambda functions to pass the tests
1. Write integration tests that verify the deployed infrastructure
1. Only output COMPLETE when all tests pass

Mention cost awareness. Add a note like “Use PAY_PER_REQUEST billing for DynamoDB” or “Use the smallest Lambda memory allocation that works.” Claude will optimize for what you tell it to optimize for.

The honest assessment

Is this the future of infrastructure development? Probably not entirely. But it is a genuinely useful technique for specific scenarios:

Great for:

Greenfield projects where you have clear requirements
Large migrations or refactors
Building proof-of-concept infrastructure while you focus on other work
Tasks where iteration is more valuable than perfection on the first try

Not great for:

Production infrastructure changes (please do not let an AI modify your production AWS account unsupervised)
Complex architecture decisions that require human judgment
Anything involving sensitive data or security configurations
Small tweaks or quick fixes (overkill)

The API costs were reasonable for what I got. For a complete infrastructure project built autonomously, a few dollars in Claude API calls is a bargain compared to my hourly rate.

Try it yourself

If you want to experiment with Ralph Wiggum and Pulumi:

Install Claude Code and the Ralph Wiggum plugin
Set up your Pulumi account and AWS credentials
Write a detailed PROMPT.md with clear success criteria
Start with a small project and low iteration limits
Review the git history when it finishes

The combination of autonomous AI loops and infrastructure as code feels like a glimpse of where development is heading. Not replacing developers, but changing how we use our time. Instead of babysitting an AI through each step, you hand it a well-defined problem and come back to working infrastructure.

Just make sure you have billing alerts set up. Ralph Wiggum does not know when to stop spending your money.

Or try Pulumi Neo: Ralph built-in

If setting up the Ralph Wiggum plugin feels like too much ceremony, Pulumi has something similar built right into Pulumi Neo.

Neo Tasks work on the same principle: you describe what you want, and Neo plans and executes the infrastructure changes. The key is auto mode. When you set a task to auto mode, Neo runs without requesting approvals. It plans, executes, validates with pulumi preview, and iterates until the task is complete.

The experience is remarkably similar to Ralph:

Persistent execution: Tasks continue running even if you close your browser. Come back later and Neo shows you what it accomplished while you were away.
Iterative refinement: Failed deployments inform the next attempt, just like Ralph re-reading the project state.
Clear completion: Neo knows when the infrastructure matches your requirements.

The main difference is that Neo runs in Pulumi Cloud rather than on your local machine, and it is purpose-built for infrastructure tasks. You get the autonomous loop experience without needing to configure plugins or bash loops.

For teams already using Pulumi Cloud, Neo with auto mode might be the fastest path to autonomous infrastructure development.

Resources

From 'Works on My Machine' to Production-Ready: Building AI Agents with Amazon Bedrock AgentCore

Engin Diri — Mon, 15 Dec 2025 00:00:00 +0200

Every developer building AI agents knows the gap between a working prototype and production deployment. Your fraud detection agent works perfectly on your laptop, but how do you deploy it with proper authentication, memory persistence, observability, and guardrails? This post walks through a complete journey from local development to production-ready AI agents using Amazon Bedrock AgentCore, the Strands SDK, and Pulumi.

TL;DR: This post shows how to deploy AI agents from local prototype to production using:

Strands SDK for agent development
Amazon Bedrock AgentCore for managed runtime with Firecracker isolation
AgentCore Memory for short-term and long-term context
MCP Gateway for secure tool integration with OAuth2
Pulumi for infrastructure as code

Complete code: GitHub

The Challenge: Bridging the Gap

Building AI agents is getting easier. Tools like the Strands Agents SDK let you create functional agents in just a few lines of code. Production deployment introduces challenges:

Stateless to Stateful: How does your agent remember previous conversations?
Single-tenant to Multi-tenant: How do you isolate different users’ data?
Implicit Trust to Zero Trust: How do you authenticate requests and enforce boundaries?
Console Logs to Observability: How do you trace requests across distributed systems?
Ad-hoc Tools to Managed Services: How do you expose tools securely at scale?

The approach is progressive: start simple, add complexity only when needed, and use infrastructure as code to make it reproducible.

Agents Beyond Chat: Event-Driven AI

When most people think of AI agents, they imagine chatbots: interactive systems responding to user questions. But many production agents never interact with end users directly. Instead, they’re triggered by event streams: Kafka topics, SQS queues, webhook callbacks, or scheduled jobs.

Our fraud detection agent is a prime example. It doesn’t wait for customers to ask questions. It’s triggered automatically when the payment processing system detects a new transaction. The “user” is another system, and the “conversation” is a structured event payload.

This event-driven pattern is becoming the new stack for AI agents. As Kai Waehner describes in his analysis of agentic AI with Kafka and Flink:

“Protocols like A2A and MCP are essential for standardizing agent behavior and communication. But without an event-driven substrate like Kafka and a stream-native runtime like Flink, these agents remain stuck in isolated interactions.”

The emerging architecture looks like this:

In production, you might use Amazon MSK (Managed Kafka), Amazon SQS, or EventBridge to deliver transaction events to your agent. The agent processes each event autonomously, takes action (blocking cards, creating tickets), and publishes results back to another topic.

What We’re Building: A Fraud Detection Agent

Our demo is a fraud detection agent that:

Receives transaction alerts with user ID, amount, merchant, and location
Gathers context from tools (user profiles, transaction history)
Analyzes for fraud patterns (impossible travel, velocity attacks, amount anomalies)
Takes protective action (blocking cards) when fraud is detected

We’ll build this progressively:

Stage	What Changes	Key Additions
Local Development	Running on your laptop	Strands SDK, Claude Opus 4.5, mock tools
First Deployment	Running in AgentCore	Containerized, Firecracker isolation, guardrails
Production-Ready	Full enterprise features	Memory, MCP Gateway, X-Ray observability

Local Development with Strands SDK

The Strands Agents SDK is an open-source framework from AWS that takes a model-driven approach to agent development. Instead of hardcoding complex workflows, you let the LLM reason about tool usage autonomously.

Here’s our complete local agent in about 30 lines:

# agent.py
from strands import Agent
from strands.models import BedrockModel
from tools import get_user_profile, get_recent_transactions, block_credit_card

# Configure Bedrock model with cross-region inference for Claude Opus 4.5
# Note: Cross-region inference may have different pricing than single-region
bedrock_model = BedrockModel(
 model_id="us.anthropic.claude-opus-4-5-20251101-v1:0",
 region_name="us-east-1",
)

# Define the Fraud Detection Agent
fraud_agent = Agent(
 model=bedrock_model,
 tools=[get_user_profile, get_recent_transactions, block_credit_card],
 system_prompt="""
 You are a Senior Fraud Analyst Agent at a financial institution.
 Your mission is to protect customers from fraudulent transactions in real-time.

 When you receive a transaction alert:
 1. Gather context by checking the user's profile and recent transaction history.
 2. Analyze the transaction for fraud indicators:
 - Impossible travel: Can the user physically be at the new location?
 - Unusual amounts: Is this significantly larger than typical spending?
 - High-risk merchants: Electronics stores, gift cards, wire transfers.
 - Geographic anomalies: Transactions far from home location.
 3. Make a decision:
 - If fraud indicators are present, BLOCK the card immediately.
 - If the transaction appears legitimate, allow it to proceed.
 4. Document your reasoning clearly.

 Prioritize customer protection. When in doubt, block the card.
 """,
)

Tools are simple Python functions decorated with @tool:

from strands import tool

@tool
def get_user_profile(user_id: str) -> dict:
 """Get user profile including home location and account status."""
 return {
 "user_id": user_id,
 "name": "John Doe",
 "home_location": "London, UK",
 "account_status": "active",
 }

@tool
def block_credit_card(user_id: str, reason: str) -> dict:
 """Block a user's credit card and create a fraud ticket."""
 return {
 "status": "blocked",
 "ticket_id": f"FRAUD-{user_id}-001",
 "message": f"Card blocked for user {user_id}: {reason}",
 }

Running the Local Demo

Here’s how we simulate an incoming transaction event:

# demo_simulation.py
from agent import fraud_agent

# The "Trigger" Event (Simulated Payload from Kafka/SQS)
new_transaction_event = """
ALERT: New Transaction Attempt
User ID: user_123
Amount: $2000
Merchant: Electronics Store
Location: Tokyo, Japan
Time: 09:15
"""

def run_demo():
 print("--- INCOMING EVENT STREAM ---")
 print(new_transaction_event)
 print("--- AGENT ACTIVATED ---")

 # Run the agent - processes the event autonomously
 result = fraud_agent(new_transaction_event)

 print("--- ANALYSIS COMPLETE ---")

if __name__ == "__main__":
 run_demo()

When we run this, the agent autonomously gathers context, analyzes the transaction, and takes action:

$ python demo_simulation.py
--- INCOMING EVENT STREAM ---
ALERT: New Transaction Attempt
User ID: user_123
Amount: $2000
Merchant: Electronics Store
Location: Tokyo, Japan
Time: 09:15
-----------------------------------
AGENT ACTIVATED. Analyzing...
Tool #1: get_user_profile
Tool #2: get_recent_transactions
## 🚨 FRAUD ANALYSIS REPORT
**Customer:** John Doe (user_123)
### Analysis:
#### 1. ❌ IMPOSSIBLE TRAVEL DETECTED
- **Last Transaction:** 09:00 in **London, UK**
- **Current Transaction:** 09:15 in **Tokyo, Japan**
- **Time Elapsed:** 15 minutes
- **Distance:** ~9,500 km (5,900 miles)
- **Minimum Flight Time:** ~12 hours
**Verdict:** It is physically impossible for John Doe to travel from London
to Tokyo in 15 minutes. This is a critical fraud indicator.
#### 2. ⚠️ HIGH-RISK MERCHANT
- Electronics stores are frequently targeted for fraud due to high resale value
#### 3. ⚠️ HIGH-VALUE TRANSACTION
- $2,000 at an electronics store represents a substantial purchase
### Decision: 🛑 BLOCK CARD IMMEDIATELY
Tool #3: block_credit_card
## ✅ ACTION COMPLETED
| Action | Status |
|--------|--------|
| Card Blocked | ✅ Confirmed |
| Ticket ID | TICKET-999 |
--- ANALYSIS COMPLETE ---

Notice how the agent reasoned about the event using tools and took autonomous action. This works great for development, but it lacks everything needed for production: authentication, persistence, isolation, monitoring, and governance.

First Deployment with AgentCore Runtime

Our local prototype works, but where do you actually run it? Lambda times out after 15 minutes. ECS and Kubernetes add operational overhead. And AI agents have different requirements than typical microservices: they reason through multi-step workflows, sometimes taking minutes to analyze a complex fraud case. You need something built for this workload.

Amazon Bedrock AgentCore fills this gap. It’s a managed runtime specifically designed for AI agents, with support for tasks running up to 8 hours. No timeout workarounds, no cluster management.

The security model is particularly relevant for fraud detection. AgentCore runs each session in a dedicated microVM using AWS Firecracker (the same isolation technology behind Lambda). When your agent processes one customer’s transactions, it’s completely isolated from other sessions. No shared memory, no cross-tenant contamination.

The deployment model is straightforward: containerize your agent and point AgentCore at the image. It works with Strands, LangGraph, CrewAI, or custom code. The same agent we built locally runs unchanged. We just wrap it in FastAPI and add a Dockerfile.

Cold Start and Latency

For fraud detection, latency matters. AgentCore’s Firecracker microVMs introduce a cold start penalty on the first invocation, typically 2-5 seconds to spin up the container. Subsequent invocations within the same session are fast (sub-second overhead beyond model inference time).

For real-time fraud detection, this cold start is acceptable because:

The LLM reasoning itself takes 5-15 seconds for complex analysis
Most production deployments use session pooling to keep warm instances available
The alternative (always-on containers) trades latency for cost

If sub-second total latency is critical, consider Lambda for simple rule-based checks and reserve AgentCore for complex reasoning that justifies the cold start overhead.

Containerizing the Agent

First, we wrap our agent in a FastAPI endpoint:

# agent.py (production version)
from fastapi import FastAPI, Request
from strands import Agent
from strands.models import BedrockModel
import os

app = FastAPI()

bedrock_model = BedrockModel(
 model_id="us.anthropic.claude-opus-4-5-20251101-v1:0",
 region_name="us-east-1",
 guardrail_id=os.environ.get("BEDROCK_GUARDRAIL_ID"),
 guardrail_version=os.environ.get("BEDROCK_GUARDRAIL_VERSION"),
 guardrail_trace="enabled",
)

fraud_agent = Agent(
 model=bedrock_model,
 tools=[get_user_profile, get_recent_transactions, block_credit_card],
 system_prompt="...",
)

@app.post("/invocations")
async def invoke(request: Request):
 body = await request.json()
 prompt = body.get("input", {}).get("prompt", "")
 result = fraud_agent(prompt)
 return {"output": {"message": str(result)}}

Adding Guardrails

Amazon Bedrock Guardrails help ensure your agent stays on task. We configure content filters and topic restrictions using Pulumi:

fraud_detection_guardrail = bedrock.Guardrail(
 "fraud-detection-guardrail",
 name="fraud-detection-guardrail",
 description="Ensure agent only handles fraud detection tasks",
 blocked_input_messaging="This request is outside the scope of fraud detection.",
 blocked_outputs_messaging="This response was blocked.",
 content_policy_config=bedrock.GuardrailContentPolicyConfigArgs(
 filters_configs=[
 bedrock.GuardrailContentPolicyConfigFiltersConfigArgs(
 type="SEXUAL", input_strength="HIGH", output_strength="HIGH",
 ),
 bedrock.GuardrailContentPolicyConfigFiltersConfigArgs(
 type="VIOLENCE", input_strength="HIGH", output_strength="HIGH",
 ),
 bedrock.GuardrailContentPolicyConfigFiltersConfigArgs(
 type="PROMPT_ATTACK", input_strength="HIGH", output_strength="NONE",
 ),
 # ... additional filters
 ],
 ),
 topic_policy_config=bedrock.GuardrailTopicPolicyConfigArgs(
 topics_configs=[
 bedrock.GuardrailTopicPolicyConfigTopicsConfigArgs(
 name="entertainment-media",
 type="DENY",
 definition="Questions about movies, TV shows, celebrities...",
 examples=["What is LOTR?", "Tell me about Star Wars"],
 ),
 bedrock.GuardrailTopicPolicyConfigTopicsConfigArgs(
 name="general-assistance",
 type="DENY",
 definition="Requests for coding help, math problems...",
 examples=["Write me a poem", "Help me with Python code"],
 ),
 ],
 ),
)

Deploying with Pulumi

The infrastructure is straightforward: ECR repository, IAM roles, and AgentCore Runtime:

# Create ECR repository
repo = ecr.Repository(
 "strands-agent-repo",
 name="fraud-detection-agent",
 force_delete=True,
)

# Build and push Docker image
image = docker.Image(
 "strands-agent-image",
 image_name=repo.repository_url.apply(lambda url: f"{url}:latest"),
 build=docker.DockerBuildArgs(
 context="../fraud-detection-agent",
 platform="linux/arm64",
 ),
 registry=docker.RegistryArgs(
 server=repo.repository_url,
 username=auth_token.user_name,
 password=auth_token.password,
 ),
)

# Deploy AgentCore Runtime
agent_runtime = bedrock.AgentcoreAgentRuntime(
 "fraud-detection-agent-runtime",
 agent_runtime_name="fraud_detection_agent",
 description="Fraud Detection Agent with Guardrails",
 role_arn=agent_role.arn,
 agent_runtime_artifact=bedrock.AgentcoreAgentRuntimeAgentRuntimeArtifactArgs(
 container_configuration=bedrock.AgentcoreAgentRuntimeAgentRuntimeArtifactContainerConfigurationArgs(
 container_uri=image.image_name,
 ),
 ),
 network_configuration=bedrock.AgentcoreAgentRuntimeNetworkConfigurationArgs(
 network_mode="PUBLIC",
 ),
 environment_variables={
 "BEDROCK_GUARDRAIL_ID": fraud_detection_guardrail.guardrail_id,
 "BEDROCK_GUARDRAIL_VERSION": fraud_detection_guardrail.version,
 },
)

Testing the Basic Deployment

After pulumi up, we can invoke the deployed agent via the AWS SDK:

# invoke_agent.py
import boto3
import json

def invoke_agent(agent_runtime_arn: str, prompt: str) -> dict:
 client = boto3.client("bedrock-agentcore", region_name="us-east-1")

 response = client.invoke_agent_runtime(
 agentRuntimeArn=agent_runtime_arn,
 runtimeSessionId="session_abc123...",
 payload=json.dumps({"input": {"prompt": prompt}}),
 qualifier="DEFAULT",
 )

 return json.loads(response["response"].read())

Running the same fraud scenario against the deployed agent:

$ ./test.sh
Selecting Pulumi stack 'dev'...
Getting agent runtime ARN from Pulumi...
Agent Runtime ARN: arn:aws:bedrock-agentcore:us-east-1:052848974346:runtime/fraud_detection_agent-W7zFZT3Vcn
Invoking agent with fraud detection alert...
========================================
Sending prompt to agent...
Prompt: ALERT: New Transaction Attempt
User ID: user_123
Amount: $2000
Merchant: Electronics Store
Location: Tokyo, Japan
Time: 09:15
--------------------------------------------------
Agent Response:
{
"output": {
"message": {
"role": "assistant",
"content": [
{
"text": "**Analysis:** Impossible travel confirmed. User was in London at 09:00
and this transaction attempted in Tokyo at 09:15. Physically impossible.
Combined with high-risk merchant and $2000 amount, this is fraudulent.
Card blocked (TICKET-999)."
}
]
},
"timestamp": "2025-12-15T04:17:48.762738+00:00",
"model": "fraud-detection-agent"
}
}

The agent now runs in a managed, isolated environment with guardrails. Next, we add memory and external tool integration.

Production-Ready: Memory and MCP Gateway

Our basic deployment handles individual transactions, but fraud detection in the real world requires context. If a customer’s card was blocked five minutes ago, the agent shouldn’t try to block it again. If there’s a pattern of suspicious activity over the past week, the agent should factor that into its analysis.

This is where AgentCore Memory and the MCP Gateway come in.

AgentCore Memory: Short-term and Long-term

Memory in AgentCore works at two levels. Short-term memory stores raw conversation events within a session. If the agent blocks a card in one message, it remembers this in subsequent messages during the same session.

Long-term memory is more interesting. It uses semantic extraction to persist facts across sessions. The system runs your conversations through a smaller model (we use Claude Haiku) to extract key facts, then stores those facts in a namespace you define. When a new session starts, the agent can query for relevant facts about that user. Even if the customer calls back tomorrow, the agent knows their card was blocked yesterday.

Here’s how we configure the memory infrastructure:

# Create AgentCore Memory with 30-day event expiry
fraud_detection_memory = bedrock.AgentcoreMemory(
 "fraud-detection-memory",
 name="fraud_detection_memory",
 description="Memory for fraud detection agent",
 event_expiry_duration=30, # Events expire after 30 days
 memory_execution_role_arn=memory_execution_role.arn,
)

# Custom semantic extraction for fraud-related facts
FRAUD_EXTRACTION_PROMPT = """
Extract key fraud-related facts from this conversation about the user. Focus on:
1. Credit card status (blocked/active, ticket IDs if blocked)
2. Fraud alerts and patterns detected for this user
3. Transaction history patterns (locations, amounts, frequencies)
4. User profile details relevant to fraud detection
5. Previous fraud investigations and their outcomes

Format as concise facts that can help identify fraud patterns across sessions.
"""

# Create long-term memory strategy
fraud_semantic_strategy = bedrock.AgentcoreMemoryStrategy(
 "fraud-semantic-strategy",
 name="fraud_semantic_extraction",
 memory_id=fraud_detection_memory.id,
 type="CUSTOM",
 namespaces=["/fraud-detection/users/{actorId}"],
 configuration={
 "type": "SEMANTIC_OVERRIDE",
 "extraction": {
 "append_to_prompt": FRAUD_EXTRACTION_PROMPT,
 "model_id": "us.anthropic.claude-3-5-haiku-20241022-v1:0",
 },
 "consolidation": {
 "model_id": "us.anthropic.claude-3-5-haiku-20241022-v1:0",
 },
 },
)

The namespace pattern /fraud-detection/users/{actorId} ensures each user’s facts are isolated. Claude Haiku handles the extraction efficiently, keeping costs low while maintaining accuracy.

Long-term memory extraction is asynchronous. Plan for a 20-40 second delay between storing conversations and having facts available for retrieval. Use short-term memory for immediate needs while long-term memories are being processed.

In the agent code, we integrate memory using Strands hooks:

from bedrock_agentcore.memory import MemoryClient

memory_client = MemoryClient(region_name="us-east-1")

# Retrieve long-term facts before processing
memories = memory_client.retrieve_memories(
 memory_id=memory_id,
 namespace=f"/fraud-detection/users/{actor_id}",
 query=f"fraud detection facts for user {actor_id}"
)

# Store conversation after processing
memory_client.create_event(
 memory_id=memory_id,
 actor_id=actor_id,
 session_id=session_id,
 messages=[
 (user_message, "USER"),
 (assistant_message, "ASSISTANT")
 ]
)

MCP Gateway: Tool Integration at Scale

The tools we’ve used so far are baked into the agent container. That works for simple cases, but what happens when you have dozens of tools across multiple teams? Or when a tool needs to call an external service that requires its own authentication?

The Model Context Protocol (MCP) is an open standard for AI systems to communicate with tools. AgentCore Gateway provides a managed MCP endpoint that handles authentication (JWT inbound, OAuth2 outbound) and lets agents discover tools using natural language queries rather than knowing exact tool names.

We deploy an MCP server for risk scoring with three tools:

from mcp.server.fastmcp import FastMCP

mcp = FastMCP("Fraud Risk Scoring Server")

@mcp.tool()
def calculate_risk_score(user_id: str, amount: float, merchant: str, location: str) -> dict:
 """Calculate comprehensive fraud risk score (0-100)."""
 # Risk calculation logic
 return {
 "risk_score": 75,
 "factors": ["amount_anomaly", "high_risk_merchant"],
 "recommendation": "BLOCK"
 }

@mcp.tool()
def get_fraud_indicators(user_id: str) -> dict:
 """Get known fraud indicators for a user."""
 return {
 "indicators": ["velocity_alert", "device_inconsistency"],
 "severity": "HIGH"
 }

@mcp.tool()
def check_merchant_reputation(merchant_name: str) -> dict:
 """Check merchant risk rating and fraud history."""
 return {
 "risk_rating": "HIGH",
 "fraud_reports": 47,
 "chargeback_rate": 0.15
 }

The Gateway infrastructure requires careful attention to authentication. We use two separate Cognito user pools to handle the dual authentication flow:

Inbound authentication (Agent → Gateway): The agent authenticates to the Gateway using JWT tokens from the first Cognito pool. This ensures only authorized agents can discover and call tools.

# Gateway with MCP protocol and semantic search
agentcore_gateway = bedrock.AgentcoreGateway(
 "fraud-detection-gateway",
 name="fraud-detection-mcp-gateway",
 description="Gateway for fraud detection MCP tools",
 role_arn=gateway_role.arn,
 authorizer_type="CUSTOM_JWT",
 authorizer_configuration={
 "customJwtAuthorizer": {
 "discoveryUrl": gateway_discovery_url,
 "allowedClients": [gateway_app_client.id],
 },
 },
 protocol_type="MCP",
 protocol_configuration={
 "mcp": {
 "supportedVersions": ["2025-03-26"],
 "searchType": "SEMANTIC",
 "instructions": "Gateway for fraud detection risk scoring tools.",
 },
 },
)

The searchType: "SEMANTIC" configuration enables natural language tool discovery. Instead of the agent needing to know exact tool names, it can query for “tools that check merchant risk” and the Gateway returns relevant matches.

Outbound authentication (Gateway → MCP Server): When the Gateway calls the MCP server, it needs its own credentials. The OAuth2 Credential Provider manages this second authentication layer:

# OAuth2 Credential Provider for Gateway -> MCP Server auth
oauth2_credential_provider = bedrock.AgentcoreOauth2CredentialProvider(
 "mcp-runtime-oauth2-provider",
 name="fraud-mcp-runtime-oauth2",
 credential_provider_vendor="CustomOauth2",
 oauth2_provider_config={
 "customOauth2ProviderConfig": {
 "oauthDiscovery": {"discoveryUrl": mcp_runtime_discovery_url},
 "clientId": mcp_runtime_app_client.id,
 "clientSecret": mcp_runtime_app_client.client_secret,
 },
 },
)

Connecting the pieces: Finally, we create a Gateway Target. This links the Gateway to the MCP server endpoint. It also specifies which credential provider to use for outbound calls:

# Connect Gateway to MCP Server
gateway_target = bedrock.AgentcoreGatewayTarget(
 "mcp-risk-server-target",
 name="mcp-risk-server-target",
 gateway_identifier=agentcore_gateway.gateway_id,
 target_configuration={
 "mcp": {"mcpServer": {"endpoint": mcp_server_endpoint_url}},
 },
 credential_provider_configuration={
 "oauth": {
 "providerArn": oauth2_credential_provider.credential_provider_arn,
 "scopes": [mcp_runtime_scope_string],
 },
 },
)

This three-part configuration (Gateway, Credential Provider, Target) separates concerns cleanly: the Gateway handles inbound auth and protocol translation, the Credential Provider manages outbound secrets, and the Target defines routing.

Observability with X-Ray and OpenTelemetry

Production systems need observability. We configure CloudWatch Logs for application logs and X-Ray for distributed tracing:

# Agent application logs
agent_log_group = cloudwatch.LogGroup(
 "agent-runtime-log-group",
 name=pulumi.Output.concat(
 "/aws/vendedlogs/bedrock-agentcore/", agent_runtime.agent_runtime_id
 ),
 retention_in_days=30,
)

agent_logs_delivery = cloudwatch.LogDelivery(
 "agent-logs-delivery",
 delivery_source_name=agent_logs_delivery_source.name,
 delivery_destination_arn=agent_logs_delivery_destination.arn,
)

# X-Ray traces
agent_traces_delivery_destination = cloudwatch.LogDeliveryDestination(
 "agent-traces-delivery-destination",
 name="fraud-agent-traces-dest",
 delivery_destination_type="XRAY",
)

agent_traces_delivery = cloudwatch.LogDelivery(
 "agent-traces-delivery",
 delivery_source_name=agent_traces_delivery_source.name,
 delivery_destination_arn=agent_traces_delivery_destination.arn,
)

In the container, we use AWS Distro for OpenTelemetry for auto-instrumentation:

CMD ["uv", "run", "opentelemetry-instrument", "uvicorn", "agent:app", "--host", "0.0.0.0", "--port", "8080"]

This gives us end-to-end tracing across agent invocations, memory operations, and Gateway calls with zero code changes.

AWS has announced that X-Ray SDKs will be deprecated in favor of OpenTelemetry by February 2027. Using ADOT from the start ensures your observability stack is future-proof.

Running the Demo

Deploy the Infrastructure

cd advanced-bedrock-deployment/infra
uv sync
uv run pulumi up

This deploys approximately 40 AWS resources including:

ECR repositories for agent and MCP server
AgentCore Memory with semantic extraction strategy
Two Cognito user pools for authentication
AgentCore Gateway with MCP protocol
CloudWatch log groups and X-Ray trace delivery
IAM roles and policies

Test Short-term Memory

./test.sh demo

This runs 5 scenarios in the same session, demonstrating short-term memory:

======================================================================
FRAUD DETECTION DEMO - SHORT-TERM MEMORY (Same Session)
======================================================================
SCENARIO 1: John Doe - Impossible Travel Detection
Expected: FRAUD - impossible travel from London to Tokyo in 15 minutes
Actor ID: user_123
Session ID: session_cf171803eb324a2aa9ac456b3013fe45...
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": {
"text": "**FRAUD CONFIRMED - CARD BLOCKED**
| Finding | Details |
|---------|---------|
| **User** | John Doe |
| **Ticket** | TICKET-123-999 |
| **Fraud Type** | Impossible Travel + Spending Anomaly |
**Analysis:** Transaction in Tokyo 15 minutes after London purchase
is physically impossible. The $2,000 amount is 10x the user's typical
spending range ($50-200). Card blocked immediately."
},
"memory_enabled": true,
"long_term_facts_retrieved": 3
}
}
======================================================================
SCENARIO 3: John Doe - Follow-up Transaction (Short-term Memory Test)
Expected: FRAUD - card should already be blocked (SHORT-TERM MEMORY)
Actor ID: user_123
Session ID: session_cf171803eb324a2aa9ac456b3013fe45... (SAME SESSION!)
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": {
"text": "**Card for user_123 (John Doe) is ALREADY BLOCKED
(Ticket: TICKET-123-999). No further action needed.**
This appears to be another fraudulent attempt on an already-compromised
account. The card was previously blocked due to impossible travel fraud.
The current transaction follows the same pattern: Gift Card Kiosk
purchases are a common fraud tactic for converting stolen card access
to untraceable value."
},
"memory_enabled": true,
"long_term_facts_retrieved": 3
}
}
======================================================================
SHORT-TERM MEMORY DEMO SUMMARY
======================================================================
[OK] John Doe - Impossible Travel Detection
[OK] Jane Smith - Normal Transaction
[OK] John Doe - Follow-up Transaction (Short-term Memory Test)
[OK] Alice Chen - Impossible Travel Detection
[OK] Jane Smith - Second Normal Transaction

The agent remembers within the same session that John’s card is already blocked, so there’s no need to block it twice.

Test Long-term Memory

./test.sh longterm-fresh

This demonstrates cross-session memory with two phases:

======================================================================
PHASE 1: INITIAL FRAUD DETECTION
These interactions create long-term memory facts about users.
======================================================================
SCENARIO: John Doe - Initial Fraud (Card Blocked)
Session ID: session_xyz789... (NEW SESSION)
→ Card blocked, fact stored to long-term memory
======================================================================
WAITING FOR LONG-TERM MEMORY EXTRACTION...
The semantic strategy needs time to process conversations into facts.
Waiting 30 seconds...
======================================================================
======================================================================
PHASE 2: NEW SESSIONS - LONG-TERM MEMORY RETRIEVAL
These are COMPLETELY NEW sessions. The agent should remember
fraud-related facts about users from Phase 1.
======================================================================
SCENARIO: John Doe - NEW SESSION (Long-term Memory Test)
Session ID: session_new456... (DIFFERENT SESSION!)
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": "CARD ALREADY BLOCKED. According to my records,
this user's card was previously blocked due to fraud.
Ticket FRAUD-user_123-001 exists from a previous investigation.
The card remains blocked.",
"memory_enabled": true,
"long_term_facts_retrieved": 2
}
}

Even in a completely new session, the agent retrieves facts from long-term memory and knows John’s card was blocked yesterday.

Test MCP Gateway

./test.sh gateway-fresh

This runs 6 diverse fraud scenarios using both local tools and MCP Gateway tools:

======================================================================
LOADING MCP GATEWAY CONFIGURATION...
======================================================================
Gateway URL: https://fraud-detection-mcp-gateway-xxx.gateway.bedrock-agentcore.us-east-1.amazonaws.com/mcp
Token Endpoint: https://fraud-gateway-xxx.auth.us-east-1.amazoncognito.com/oauth2/token
Scope: fraud-detection-gateway/invoke
MCP Gateway configuration loaded successfully!
======================================================================
FRAUD DETECTION DEMO - MCP GATEWAY INTEGRATION
======================================================================
LOCAL TOOLS:
- get_user_profile: User details and home location
- get_recent_transactions: Last known transaction
- block_credit_card: Block card and create ticket
MCP GATEWAY TOOLS (via Risk Scoring Service):
- calculate_risk_score: Fraud risk score 0-100
- get_fraud_indicators: Known fraud indicators
- check_merchant_reputation: Merchant risk rating
======================================================================
SCENARIO 1: HIGH-RISK MERCHANT - Jane at CryptoExchange123
MCP Gateway: ENABLED
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": {
"text": "**✅ CARD BLOCKED** - Ticket: TICKET-456-999
**Summary:** Jane Smith's card blocked due to a $5,000 crypto exchange
transaction that is 10x her typical spend and outside her normal merchant
categories. Location matched home, but amount and merchant anomalies
indicate likely fraud or compromised credentials."
},
"mcp_gateway_enabled": true,
"mcp_tools_count": 4
}
}
======================================================================
SCENARIO 5: COMBINED SIGNALS - Alice's Multi-Factor Fraud
MCP Gateway: ENABLED
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": {
"text": "## 🚨 FRAUD CONFIRMED - CARD BLOCKED
**User:** Alice Chen (user_321)
**Ticket ID:** TICKET-321-999
**Fraud Indicators Detected:**
| Factor | Finding |
|--------|---------|
| ⛔ Impossible Travel | Singapore (10:00) → Los Angeles (11:30) = 90 min |
| ⛔ Amount Anomaly | $2,500 vs typical $50-300 (8x+ normal) |
| ⛔ Merchant Mismatch | Cash advance vs usual food/retail |
| ⛔ High-Risk Merchant | QuickCashAdvance (cash advance services) |
**Action Taken:** Card blocked immediately. This transaction exhibits
classic signs of stolen card credentials being used for rapid cash extraction."
},
"mcp_gateway_enabled": true,
"mcp_tools_count": 4
}
}
======================================================================
MCP GATEWAY DEMO SUMMARY
======================================================================
[OK] 1. HIGH-RISK MERCHANT - Jane at CryptoExchange123
Gateway: YES (4 tools)
[OK] 2. VELOCITY ATTACK - Bob's Rapid Transactions
Gateway: YES (4 tools)
[OK] 3. AMOUNT ANOMALY - Alice's Large Purchase
Gateway: YES (4 tools)
[OK] 4. KNOWN FRAUD PATTERNS - John's Indicator Check
Gateway: YES (4 tools)
[OK] 5. COMBINED SIGNALS - Alice's Multi-Factor Fraud
Gateway: YES (4 tools)
[OK] 6. CLEAN TRANSACTION - Jane's Normal Purchase
Gateway: YES (4 tools)

The MCP Gateway tools provide additional context (merchant reputation, risk scores, fraud indicators) that helps the agent make better decisions.

Architecture Summary

Here’s the complete architecture:

Key Takeaways

Event-driven first: Production agents are triggered by Kafka, SQS, or webhooks, not chat interfaces
Start simple: Build with Strands SDK locally, add AgentCore complexity only when needed
Guardrails early: Content filtering and topic restrictions prevent prompt injection
Memory has two levels: Short-term (session) and long-term (semantic extraction across sessions)
MCP for tools at scale: Gateway handles auth and discovery; you focus on tool logic
Observability required: X-Ray + CloudWatch are essential for debugging distributed AI
IaC for reproducibility: Pulumi deploys 40+ resources with proper dependencies

Cost Considerations

When evaluating AgentCore for production, consider these cost factors:

AgentCore Runtime charges per session-second. For fraud detection agents that typically run 10-30 seconds per transaction, this is cost-effective compared to maintaining always-on ECS containers. However, if your agents process thousands of transactions per second, the per-session costs add up. Consider batching or using Lambda for simple, fast agents.

Model costs dominate most deployments. Claude Opus 4.5 (used for reasoning) costs significantly more than Claude Haiku (used for memory extraction). We deliberately use Haiku for semantic extraction to keep memory costs low while maintaining accuracy. For high-volume production, consider whether Haiku or even smaller models could handle your primary reasoning.

Memory and Gateway have their own pricing tiers. AgentCore Memory charges for storage and retrieval operations. The Gateway adds overhead for OAuth2 token management and routing. For simple agents that don’t need cross-session memory or external tools, skip these components entirely.

Cognito user pools for authentication have a free tier (50,000 MAU) that covers most development and moderate production workloads. Beyond that, costs scale with active users.

Start with a basic deployment and add Memory/Gateway only when you have a clear need. The infrastructure cost savings of skipping unnecessary components often exceed the engineering cost of adding them later.

Conclusion

The fraud detection agent in this post grew from 30 lines of local Python to a production deployment with guardrails, dual-layer memory, MCP Gateway integration, and full observability, all defined as infrastructure as code.

The gap between prototype and production doesn’t have to be a chasm. AgentCore provides the managed runtime, Pulumi makes it reproducible, and the progressive approach lets you add complexity only when your use case demands it.

Get Started with Pulumi

Use Pulumi's open-source SDK to create, deploy, and manage infrastructure on any cloud.

AI Predictions for 2026: A DevOps Engineer's Guide

Engin Diri — Thu, 11 Dec 2025 00:00:00 +0000

The IDE is dying, and so is tool calling. OpenAI is not going to win. And next year, you’re going to be shipping code that you’ve never reviewed before, even as an experienced engineer.

These are bold claims, but the way we use AI in 2026 for coding and agents is going to look completely different. In this post, I want to cover my predictions and why they matter right now for DevOps engineers. Some of these are definitely hot takes, but that’s what makes this conversation worth having.

IDEs are dead (at least as we know them)

Traditional IDEs where the code is the focus of the interface are simply going to become irrelevant. We’re moving toward agent manager interfaces where we can kick off agents in parallel to work on different features in a codebase or even work on different projects at the exact same time.

We’re already seeing this transition. Google Antigravity combines a familiar AI-powered coding experience with a new agent-first interface. You can deploy agents that autonomously plan, execute, and verify complex tasks across your editor, terminal, and browser. Cursor 2.0 lets you run up to eight agents in parallel on a single prompt, using git worktrees or remote machines to prevent file conflicts. Each agent operates in its own isolated copy of your codebase.

AWS validated this direction at re:Invent 2025 by announcing “frontier agents” including Kiro for autonomous coding, along with dedicated security and DevOps agents. These agents maintain state, log actions, operate with policy guardrails, and integrate directly with CI/CD pipelines.

For infrastructure specifically, Pulumi Neo represents this same shift. Instead of writing code or running CLI commands for every operation, you describe what you need in natural language and Neo handles the implementation. It works across your entire infrastructure, understanding dependencies and creating execution plans that go through pull requests for review.

For DevOps engineers, this means your pipelines need to accommodate AI-generated code at scale. Multiple agents working simultaneously need isolated, reproducible environments. More generated code means more artifacts to track, version, and deploy.

The different paths to AI dominance

A lot of people think that in the future, a single large language model will have a monopoly and be the best at absolutely everything. But what’s really going to happen is different providers will specialize and focus on being the best at different things.

Google is going down the generalist route with Gemini, aiming to be the jack-of-all-trades. Anthropic is focusing on being the best for coding. You can see this in the benchmarks: when Opus 4.5 came out, the first benchmark they highlighted was for software engineering, because that’s what Anthropic is focusing on.

Amazon is carving out its own niche with the Nova model family, announced at re:Invent 2025. The Nova 2 lineup includes specialized models: Pro for complex reasoning, Sonic for real-time voice conversations, and Omni for simultaneous text, audio, and video processing. With Nova Forge, organizations can build custom frontier models by combining their proprietary data with AWS open weight models. The re:Invent message was clear: leveraging your first-party data is now fundamental to going beyond generic AI. We’re talking about 30-40% increases in accuracy when you bring your own data into the equation.

But here’s the hot take: I don’t think OpenAI is going to come out on top with any kind of specialization. They’ve disappointed time and time again with GPT-5 and GPT-4.5. With 4.5, they seemed to try to be the creative specialist, but it just didn’t work. The GPT-5 launch in August 2025 was described as “barely better than last month’s flavor of the month” and on some metrics it’s actually worse than earlier models.

For DevOps teams, this specialization means you’ll need infrastructure that’s model-agnostic and supports multiple AI backends. Plan for secrets management across multiple LLM providers and design your systems to swap models based on the task at hand.

The local AI breakthrough

2026 will be the year of local AI. We didn’t see that much this year besides DeepSeek at the start of 2025, which was a big deal. We had a couple of new models like Qwen 3, but nothing that fundamentally changed the game. Now we’re starting to see new hardware that makes it obvious we’re going to be able to run very large models on smaller devices.

There’s new AI chips that can run upwards of 120 billion parameter large language models on the edge, which would be a complete game-changer. Right now, hardware requirements are one of the biggest challenges for scaling local AI. If we can solve the hardware problem, we get 100% data privacy and zero-millisecond latency for our agents.

AWS is addressing this with Trainium3 UltraServers, their 3nm AI chips delivering 4.4x more compute than the previous generation. More significantly, AWS AI Factories allow organizations to deploy racks of Trainium chips and NVIDIA GPUs directly into their own data centers, addressing data sovereignty concerns while keeping AI inference close to the data.

For DevOps, this opens the door to zero-latency inference in CI/CD pipelines, complete data privacy for sensitive codebases, and reduced cloud costs for AI-heavy workloads.

Engineers as system architects

We’re finally going to get to the point where we’re not the coders. We delegate that entirely to our coding agents, and we become the system architects. This mirrors the evolution of other engineering disciplines. Civil engineers don’t fabricate the steel beams; they design the structure and verify the integrity.

I think of this as a three-step process:

Define: Set your objectives and the system that your agents will operate under
Orchestrate: Delegate the coding to your agents
Validate: Ensure the quality of the outputs and the overall system

We’re still in the loop. We are the final say in whatever is created, but we’re delegating the grunt work to our coding agents.

This is exactly the model that Pulumi Neo implements for infrastructure. When you give Neo a complex request, it creates a task plan outlining the steps it will take to accomplish your goal. This plan provides transparency into Neo’s approach and gives you the opportunity to adjust the strategy before execution begins. Neo operates in different modes: Review mode where everything requires approval, Balanced mode where only deployments need sign-off, or Auto mode for full autonomy. You define the boundaries, Neo orchestrates the work, and you validate through pull requests and previews.

For DevOps engineers, this shift means building robust validation infrastructure becomes critical. When AI writes the code, you need automated testing pipelines, security scanning, and verification systems that can operate at the speed of AI-generated changes.

Code execution is replacing tool calling

Here’s a key insight that kept coming up at re:Invent: models are no longer the bottleneck. Context is. Our agents are going to change a lot next year because code execution is starting to replace tool calling. The problem with tool calling right now is that all the capabilities you give an agent take up context upfront. When you try to give a lot of different tools to an agent, you completely overwhelm it.

Anthropic’s research on code execution with MCP addresses exactly this problem. Code execution is a massive token reduction, faster, and more flexible. You’re giving the agent the ability to generate its own capabilities at runtime by writing code to interact with APIs. A workflow that previously consumed about 150,000 tokens when tools were passed directly through the model was reimplemented with code execution and used only about 2,000 tokens. That’s a 98.7% reduction.

AWS embraced this pattern with Amazon Bedrock AgentCore, which now includes code interpretation capabilities. AgentCore supports any agent framework (CrewAI, LangGraph, OpenAI SDK) and provides memory, browser tools, and observability features that make code execution practical at enterprise scale.

For DevOps, this means you need sandboxed, secure execution environments for AI-generated code. Running agent-generated code requires appropriate isolation, resource limits, and monitoring.

Progressive disclosure and composable skills

The best part about code execution flexibility is it unlocks progressive disclosure. All I mean by that is: you have a lot of capabilities for an agent, but you don’t actually give all of them upfront. Instead, you allow the agent to discover capabilities and then leverage them in a more flexible way.

For each capability, you just have a bit of metadata or description that loads upfront. When the agent decides to leverage that capability, then you load the full instructions. Now you can practically scale to infinity because all capabilities don’t have to be loaded at runtime.

Claude Skills is a good example of this pattern. Skills are organized folders of instructions, scripts, and resources that agents can discover and load dynamically. At session start, the agent scans available skills and populates the system prompt with just a brief name and description (around 100 tokens). The full skill prompt loads only after Claude selects it, preventing context bloat while maintaining discoverability.

Kiro Powers addresses the same problem. Connecting five MCP servers can consume over 50,000 tokens, roughly 40% of an AI model’s context window, before you even type your first request. Powers bundle MCP servers, steering files, and hooks into units that load dynamically based on conversation context. Mention “payment” and the Stripe power activates. Datadog, Figma, and others have powers available.

For DevOps, this translates to modular infrastructure definitions, on-demand capability loading, and efficient resource utilization. Think about how you can apply this pattern to your own automation.

Agent-to-agent protocols are finally happening

Agent-to-agent protocols are where AI agents operate in a peer network, discover each other’s capabilities in real time, and interact autonomously. When Google released their A2A protocol earlier this year, there was a ton of buzz. A lot of people thought it was going to be the next big standard, like the next MCP. But then it kind of fell to the wayside.

The big reason is the chicken-and-egg problem. For A2A to be useful, you need many people to adopt it at the same time. Otherwise, if you build an A2A-compatible agent, it has no other agents to talk to. The whole value proposition is lost unless you already have a big network to attach to.

But that’s finally changing. The Linux Foundation launched the A2A project in June 2025, and adoption is accelerating. Adobe, Microsoft, SAP, ServiceNow, and S&P Global are all implementing A2A. In July 2025, Google released version 0.3 of the A2A protocol with a more stable interface critical to accelerating enterprise adoption.

Machines paying machines

My next big prediction is that machines paying machines is going to become a very big thing. Coinbase released the x402 protocol for exactly this: building AI agents that you expose over the internet but require payment whenever someone else interacts with them.

This goes really well with agent-to-agent protocols. You can create a peer network where you monetize your agents. They all leverage each other but make payments whenever they take advantage of another agent’s capabilities. Cryptocurrency is the perfect solution for this kind of machine-to-machine network because it enables micropayments to be made quickly and globally.

The x402 protocol has achieved 156,000 weekly transactions with 492% growth since launching in May 2025. It’s now integrated with Anthropic’s MCP Protocol, Google Gemini, OpenAI Codex, and other platforms. Stablecoins like USDC enable charging per request, per service, or per second of usage at near-zero transaction costs, enabling payments as low as $0.001 per request.

Artifact reviews instead of diff reviews

When we want to do a rigorous code review, traditionally, we look line by line at all the changes. But coding agents are getting to the point where they can prove their code works through artifacts. Instead of reviewing line by line, we can look at browser recordings, full working demos of a backend API, and other artifacts.

Google Antigravity is a perfect example. As part of its coding process, it can autonomously spin up your website, visit it, scroll through it, take screenshots, and record everything. Agents generate artifacts, including tangible deliverables like task lists, implementation plans, screenshots, and browser recordings. You can verify the agent’s logic at a glance.

Amazon Nova Act takes this further. It enables AI agents to automate browser-based tasks like form filling, QA testing, and workflow validation with over 90% reliability. The service includes built-in observability through live viewing, CloudTrail logging, and session replay, making it possible to review what an agent actually did rather than parsing through code changes.

Shipping code you’ve never read

For the last prediction, we’re tying everything together. We’ve talked about reviewing artifacts instead of diffs, creating systems instead of coding, and the new capabilities for agents with code execution.

We’re going to get to the point very quickly where we’re shipping code that we have never read before. And I’m not talking about people who vibe code. Even experienced engineers are going to trust their systems so much that they have the ability to review the code but they’re not going to. We’re just going to ship to production after reviewing the artifacts.

I presented on this exact topic at the Tel Aviv Pulumi User Group meetup at Qodo HQ back in October, where I demonstrated how Pulumi Neo’s autonomous decision-making capabilities can handle infrastructure tasks that we traditionally managed manually. Qodo is doing fascinating work in this space with their agentic development tools, building systems that let you trust the output without necessarily reviewing every line.

I’m not saying we’re taking the human completely out of the loop. I’m saying we’re going to have a lot of trust in our systems and a validation process that includes us, but that doesn’t necessarily have to be us actually looking at the code. Tools like Pulumi Neo create pull requests with clear documentation of changes, run previews to validate infrastructure modifications, and provide the transparency needed to ship with confidence.

The path forward

The predictions I’ve outlined point to a fundamental shift in how software gets built and deployed. For DevOps engineers, this isn’t a threat but an opportunity to become more strategic and less operational. We’re entering the battle of the agentic frameworks, where the winners will be those who can build faster, cheaper agentic applications through their platforms.

The immediate reality is that your CI/CD pipelines need to accommodate AI-generated code at scale, your secrets management needs to handle multiple LLM providers, and your execution environments need proper sandboxing for agent-generated code. These aren’t future concerns; they’re requirements for working effectively with the AI tools available today.

Looking further out, the engineers who thrive will be those who embrace the system architect role. Define clear objectives and constraints for your AI agents. Build validation frameworks that can verify outcomes without requiring line-by-line code review. Design infrastructure that’s modular enough to load capabilities on demand.

The technology to make this happen already exists. Agent orchestration platforms are shipping. Code execution is replacing tool calling. Progressive disclosure patterns are proven. The question isn’t whether these changes are coming; it’s whether you’ll be ready when they arrive.

Start building for the AI-driven future today

If you want to experience what this future looks like right now, Pulumi Neo is the place to start. Neo lets you make natural language requests for routine infrastructure tasks, analysis, and management. Instead of writing code for every operation, you describe what you need, and Neo handles the implementation, creating task plans, running previews, and opening pull requests for your review.

Whether you’re looking to update outdated resources across your infrastructure, analyze your cloud spend, or automate complex multi-step workflows, Neo provides the agent-first experience that’s defining the next generation of DevOps tooling.

Get started with Pulumi Neo and see how AI-powered infrastructure automation can transform your workflow.
Project: Build a production-ready AI agent by moving from local prototypes to deployed infrastructure with Amazon Bedrock AgentCore, the Strands SDK, and Pulumi.

Encode What You Know With Neo: Custom Instructions and Slash Commands

Pulumi Neo Team — Tue, 09 Dec 2025 00:00:00 +0000

Every organization builds up knowledge over time: naming standards, compliance requirements, patterns your team has settled on, and proven approaches to common tasks. Until now, bringing this knowledge into Neo meant repeating it manually each time - specifying preferences, describing how your team works, and recreating prompts that someone already perfected.

Two new features change this. Custom Instructions teach Neo your standards so it applies them automatically. Slash Commands capture proven prompts so anyone on your team can use them with a keystroke.

Custom Instructions: Standards Applied Automatically

Custom Instructions let you define what Neo should know about your organization and how it should behave. This includes naming conventions, required tags and compliance requirements, technology preferences, and cost guidelines - but also actions Neo should take automatically, like including a rough cost estimate whenever it proposes new infrastructure. You configure them once in your organization settings, and Neo applies them to every task from that point forward.

Consider the difference. Before Custom Instructions, a simple request required loading context:

“Neo, update our Lambda functions to Node 20. Remember, we use TypeScript exclusively, our naming convention is service-region-env, we always deploy to us-east-1 first for testing, and all resources need our standard compliance tags including CostCenter and DataClassification.”

With those details captured in Custom Instructions, the same request becomes:

“Neo, update our Lambda functions to Node 20.”

Neo already knows how your team works, so you can focus on what you’re trying to accomplish.

Slash Commands: Capture What Works

Over time, your team figures out the right way to ask Neo for certain tasks. Maybe someone wrote the perfect prompt for checking policy violations, or discovered an approach to drift detection that catches issues others miss. That knowledge tends to live in someone’s head or buried in a Slack thread.

Slash Commands turn these prompts into shortcuts anyone can use. When you type / in Neo, you’ll see available commands, select one, and Neo receives the full prompt behind it.

Neo ships with built-in commands for common tasks:

Command	What it does
`/get-started`	Learn what Neo can do and how to structure effective requests
`/policy-issues-report`	Lists your most severe policy violations
`/component-version-report`	Lists components that are outdated in your private registry
`/provider-version-report`	Lists providers that are outdated

You can also create your own. In Pulumi Cloud, you define the prompt - no coding required. Once saved, your team can start using it immediately. If a command needs more information than what’s provided, Neo will ask follow-up questions to fill in the gaps.

Get Started

Custom Instructions and Slash Commands are available now. You can configure Custom Instructions in Neo Settings. Slash Commands come with several built-in options, and you can create custom ones tailored to your workflow.

Future of the Cloud: 10 Trends Shaping 2026 and Beyond

Sara Huddleston — Thu, 04 Dec 2025 07:56:40 +0000

In 2026, several trends will dominate cloud computing, driving innovation, efficiency, and scalability. From Infrastructure as Code (IaC) to AI/ML, platform engineering to multi-cloud and hybrid strategies, and security practices, let’s explore the 10 biggest emerging trends.

On This Article

1. Cloud Will Become a Business Necessity by 2028
2. Hyperscalers Accelerate AI-Driven Cloud Expansion
3. Hybrid and Multi-Cloud to Drive Innovation
4. Enterprises Rebuild Their Cloud Foundations to Operationalize AI
5. IaC Drives Scalable Cloud, Multi-Cloud, and AI Operations
6. DevSecOps Evolves Into AI-Integrated Security
7. Platform Engineering and Internal Developer Platforms
8. AIOps Matures Into a Cloud Operations Standard
9. Kubernetes Dominance and Increased Complexity
10. AI Code Assistants in the Enterprise
The Future of Cloud: Reinvented for an AI-First Decade

1. Cloud Will Become a Business Necessity by 2028

According to Gartner, by 2028 the cloud will be the key driver for business innovation, and estimates that over 95% of new digital workloads will be deployed on cloud-native platforms.

The future of cloud computing. Credit: Gartner

According to McKinsey & Company’s “In search of cloud value” report:

Cloud value is driven by innovation, worth 5x more than cost savings.
EBITDA uplift of 20–30% by 2030 for high-performing organizations.
Asia leads in projected cloud value, followed by the US and Europe.
High-ROI organizations excel by aligning cloud strategy with business priorities, building strong cloud foundations, and using modern operating models.

Teams succeeding in this transition increasingly use Infrastructure as Code, automation, and unified governance frameworks like Pulumi Insights + Policies to operationalize this value.

2. Hyperscalers Accelerate AI-Driven Cloud Expansion

Hyperscalers are making the largest infrastructure investments in cloud history — nearly all centered on AI workloads, inference, and high-performance compute.

AWS has integrated Anthropic’s Claude 3 and Claude 4 models into Amazon Bedrock for enterprise LLM workflows. “Claude Opus 4 and Claude Sonnet 4 are available today in Amazon Bedrock, enabling customers to build agents with stronger reasoning, memory, and tool use.” — AWS, May 2025
Microsoft Azure revenue rose 33% year-over-year in Q3 (ended March 31), outperforming estimates of ~29.7%. AI contributed 16 percentage points to this growth, up from 13 points in the prior quarter. “Microsoft is on track to invest approximately $80 billion to build out AI-enabled datacenters to train AI models and deploy AI and cloud-based applications around the world,” said Brad Smith, the Microsoft Vice Chair and President.
Google Cloud is committing $25 billion over two years for data center and AI infrastructure expansion across the PJM grid, with total capital expenditure for 2025 ranging from $75–85 billion. “As our CEO has said, in these early days of a very transformative technology, the risks of under-investing are dramatically higher than the risks of over-investing,” said Eunice Huang, Head of AI and Emerging Tech Policy.
Oracle anticipates 15–20% cloud revenue growth in FY 2026–2027 attributable to AI infrastructure demand, tied to its partnership in the Stargate initiative.

As hyperscalers integrate AI deeper into their service layers, engineering teams must adapt with IaC-driven automation, reusable patterns, and policy controls to deploy cloud and AI infrastructure consistently. See how organizations deploy AWS infrastructure at the speed of AI with Pulumi and Pulumi Policies.

3. Hybrid and Multi-Cloud to Drive Innovation

Hybrid and multi-cloud strategies are now mainstream:

Hybrid cloud will grow from $130B to $310–330B by 2030 (ResearchAndMarkets).
87% of enterprises run workloads across multiple clouds (Mordor Intelligence).
Gartner predicts that 40% of enterprises will adopt hybrid compute architectures in mission-critical workflows by 2028 (up from 8%).

Credit: Cloud Worldwide Service, Forbes

As AI and regulatory requirements grow, organizations must deploy workloads across AWS, Azure, Google Cloud, on-prem, and edge — while maintaining consistent security, compliance, and configuration.

Modern cloud teams use:

Infrastructure as Code for consistent multi-cloud provisioning and environment standardization, forming the backbone of AI infrastructure orchestration
Reusable components and internal platforms to define scalable architecture patterns and accelerate delivery across Kubernetes, AI/ML pipelines, and hybrid environments
Policy-driven guardrails to maintain cost, security, and compliance across environments, supporting cloud governance automation and modern cloud cost governance

Pulumi enables all three through its multi-cloud IaC model, Pulumi Policies, and internal developer platform capabilities.

4. Enterprises Rebuild Their Cloud Foundations to Operationalize AI

While hyperscalers are transforming the global cloud platform, enterprises face a different challenge: adapting their own cloud foundations to support AI at scale. Organizations are moving beyond prototypes and integrating AI into core products, internal workflows, and customer-facing systems, requiring new levels of automation, governance, and AI infrastructure orchestration.

According to Gartner, global AI infrastructure spending is expected to surpass $2 trillion in 2026. IDC predicts that by 2027, more than 50% of enterprises will use AI agents to drive core workflows, which requires scalable, secure, and automated cloud architectures to support model execution and orchestration.

To enable this transition, enterprises are investing in:

GPU provisioning and orchestration, data pipelines, vector databases, feature stores, and LLM infrastructure needed for real-time AI workloads.
Data pipelines, vector databases, and feature stores needed for real-time AI workloads
Model-serving infrastructure, including gateways, inference routers, and autoscaling layers
Strong identity, secrets, and access controls as AI systems increase security exposure
Automation through Infrastructure as Code to ensure reproducibility and reduce drift
Policy-driven governance to secure cost, compliance, and architectural consistency

As AI becomes deeply embedded across engineering organizations, teams are increasingly using software engineering approaches such as Infrastructure as Code, reusable components, platform engineering, and policy automation to standardize how AI infrastructure is deployed, scaled, and secured across clouds.

To support this shift, Pulumi’s perspective on Superintelligence Infrastructure explains why AI workloads, from pre-training to inference at massive scale, require dynamic infrastructure orchestration rather than static configuration.

Pulumi users increasingly rely on:

Pulumi IaC for standardized AI infrastructure
Pulumi ESC to manage all secrets and configuration at scale
Pulumi Insights for visibility and misconfiguration analysis
Pulumi Policies for AI-specific guardrails in code, cost detection, and to provide automated compliance protections

5. IaC Drives Scalable Cloud, Multi-Cloud, and AI Operations

As cloud environments expand and AI workloads demand highly dynamic infrastructure, Infrastructure as Code (IaC) is becoming the foundation for scaling reliably across all environments. Organizations are increasingly adopting IaC in general-purpose languages to unify development and infrastructure workflows, reduce configuration drift, and deliver cloud resources at speed.

Modern Infrastructure as Code is advancing far beyond simple provisioning:

Standardizing multi-cloud and hybrid patterns so teams can deploy consistently across AWS, Azure, Google Cloud, on-prem, and edge environments.
Integrating seamlessly with cloud providers and third-party services, including data platforms and messaging systems like CockroachDB, Confluent Cloud, and Kafka.
Providing deeper validation and type-safety, ensuring parameters, dependencies, and security controls are correct before deployment.
Improving cloud resource efficiency and visibility with tools like Pulumi Insights Discovery.
Embedding security and compliance through Policy as Code, enforcing guardrails, cost controls, and regulatory requirements automatically, enabling truly policy-driven cloud management.
Enabling intelligent automation, from unit and integration tests to auto-remediation policies and policy-driven approvals.
Incorporating AI-driven optimization and insights, helping teams detect misconfigurations, analyze usage patterns, and generate infrastructure updates with tools like Pulumi Neo and Pulumi Policies.

As organizations scale both traditional cloud workloads and AI-driven systems, IaC has become critical for achieving secure, repeatable, and high-velocity operations across every environment.

6. DevSecOps Evolves Into AI-Integrated Security

As AI becomes embedded across cloud-native systems, DevSecOps is entering a new era. Gartner predicts that by 2028, over 50% of enterprises will use AI security platforms to protect their AI investments. Below are the 3 key predictions for the future of DevSecOps:

AI-driven security automation: Teams will increasingly rely on AI to detect threats, enforce policies, and generate secure infrastructure patches. See Pulumi’s capabilities in AI-powered remediation.
More focus on secrets management: With AI systems accessing more sensitive data, secure secret storage will be essential. Pulumi ESC helps teams centralize and govern credentials, keys, and tokens safely.
Greater cross-team collaboration: Dev, Sec, and Ops workflows will converge under shared frameworks: IaC, policy automation, runtime scanning, and GitOps.

As organizations increase their use of AI across cloud-native systems, the need for tightly aligned security, governance, and cloud governance automation becomes even more urgent. At the Gartner Data & Analytics Summit in Sydney, Carlie Idoine, VP Analyst at Gartner, emphasized this growing dependency:

"[AI]… it doesn’t deliver value on its own – AI needs to be tightly aligned with data, analytics, and governance to enable intelligent, adaptive decisions and actions across the organization."

This perspective mirrors what we’re seeing across modern DevSecOps practices: AI can amplify security, but only when paired with strong foundations in secrets management, governance, and cross-team collaboration.

7. Platform Engineering & Internal Developer Platforms (IDPs)

According to Gartner, by 2026, 80% of large software engineering organizations will establish platform engineering teams as internal providers of reusable services, components, and tools for application delivery. Platform engineering will ultimately solve the central problem of cooperation between software developers and operators.

Mid-size to large companies will begin or continue to invest in implementing platform engineering practices, with large tech companies as first adopters. They will provide Internal Developer Platforms (IDP) to elevate the Developer Experience (DX, sometimes referred to as DE or DevEx), helping them work faster, like abstracting the complexities of configuring, testing, and validation, deploying infrastructure, and scanning their code for security.

Internal developer platform-in-a-box. Credit: Pulumi

IDPs are reshaping how developers interact with cloud infrastructure, bringing together platform engineering, automation, and emerging AI platform engineering practices.

8. AIOps Matures into a Cloud Operations Standard

AIOps is becoming mainstream, helping teams predict failures, auto-scale infrastructure, and resolve incidents with minimal manual effort. As AI and automation continue to evolve, the fusion of these technologies will enable organizations to achieve unprecedented levels of efficiency and scalability.

Proactive Operations: AI-powered tools will assist teams in foreseeing issues with greater accuracy, minimizing downtime, and reducing the firefighting nature of incident management. These tools will automatically detect anomalies, optimize performance, and trigger remediation actions.
Intelligent Automation: Routine operational tasks like patching, monitoring, and resource scaling will be fully automated. AI-driven decision-making will allow for smarter resource allocation and optimization, dynamically adjusting infrastructure and workloads in response to real-time demands and predictions.
Data-Driven Insights: AIOps will analyze vast amounts of operational data and provide actionable insights, enabling teams to focus on high-impact tasks such as improving system architecture and user experience. The AI-powered insights will also inform better strategic decisions, helping teams to continuously evolve their DevOps practices.
Collaboration Across Teams: AIOps will bridge the gap between DevOps, SecOps, and IT operations by bridging monitoring and automation. Cross-team collaboration will improve as AI systems consolidate and interpret data from various departments, allowing for a more cohesive approach to system management.

AIOps features include observability, automation, and real-time analytics to bridge DevOps, SRE, and IT operations.

9. Kubernetes Dominance and Increased Complexity

Kubernetes will continue its ascent in 2026. According to Research & Markets, the global Kubernetes market was valued at USD 2.3 billion in 2024 and is projected to reach USD 8.2 billion by 2030, with a CAGR of 23.8% over the forecast period.

The CNCF Annual Survey shows AI/ML workloads rapidly moving onto Kubernetes — including batch pipelines, model experimentation, real-time inference, and data preprocessing — even as only 41% of ML/AI developers are cloud-native today. This shift is accelerating as teams need flexible GPU scheduling, distributed pipelines, and portable execution environments.

Kubernetes is also evolving in response to AI demands. Inference workloads, powered by LLMs and GPUs, now require low-latency execution closer to the user. This shift is pushing organizations to build intelligent orchestration layers that schedule AI pipelines across edge and core clusters, often leveraging Kubernetes as the common control plane for AI cluster orchestration.

As we move into 2026, three patterns are becoming clear:

Kubernetes is evolving to support AI through GPU-aware scheduling, Kubernetes GPU scheduling optimizations, and more advanced workload orchestration.
Governance and consistency matter more than ever, as teams struggle to secure and manage multi-cluster, multi-cloud environments.
Platform engineering is essential, providing curated patterns and automation rather than raw YAML to reduce cognitive load.

Kubernetes will remain a strategic foundation — but operating it effectively now depends on robust automation, strong security controls, and standardized delivery models that scale across clouds, clusters, and AI pipelines.

10. AI Code Assistants in the Enterprise

AI-powered coding assistants like GitHub Copilot, Claude Code, Cursor, and others are rapidly becoming part of modern software development.

By 2027, the use of AI assistants will dramatically increase developer velocity to meet functional business requirements for 70% of new digital solutions in production (source: IDC).

The value of AI code assistants. Credit: Gartner

According to Gartner, by 2028, 75% of enterprise software engineers will use dedicated AI code assistants, and 63% of organizations are currently piloting, deploying or beginning to use AI code assistants.

As enterprise adoption of AI assistants increases, expectations are rising: they must not only generate code but also understand the state of infrastructure, configurations, and security posture. That means being able to answer questions about environments, surface misconfigurations, or act directly on infrastructure.

One of the newest developments is the release of Pulumi Agent Skills, a collection of infrastructure expertise packaged for use in AI coding assistants. These skills teach tools such as Claude Code, Cursor, or Gemini CLI to reason about Pulumi projects, reducing hallucination and improving outputs based on real infrastructure conventions and practices.

Combined with infrastructure access via tools like Pulumi’s Remote MCP Server, teams can build secure, AI-driven workflows where assistants provide insights and Pulumi Neo safely executes actions with previews, policies, and orchestration.

AI code assistants are no longer experimental; they’re fast becoming a competitive advantage in cloud software development.

The Future of Cloud: Reinvented for an AI-First Decade

Cloud infrastructure is entering its most transformative era since the rise of Kubernetes. The trends shaping 2026 reveal a clear pattern: AI is no longer a workload — it’s becoming the organizing principle of cloud strategy.

AI-native cloud architectures that require elastic compute, GPU orchestration, fast data access, and governance built into every layer
Infrastructure as Code as the operational backbone, standardizing deployments across AI, cloud, and hybrid environments
Platform engineering and IDPs to enable self-service, gold-standard patterns, and automated guardrails
Security integrated into every pipeline, with AI-assisted threat detection, strong secrets management, and policy-driven compliance
AIOps and intelligent automation are becoming standard for scaling modern cloud systems
Kubernetes evolving for AI, driving new orchestration patterns across edge, core, and inference clusters
Multi-cloud and hybrid ecosystems accelerating to support interoperability, resilience, and global workload placement

Taken together, these shifts point to a new model of cloud operations that is intelligent, automated, policy-aware, and built on software engineering principles rather than manual configuration.

Organizations that invest now in modern IaC, unified governance, reusable components, and policy frameworks — all core capabilities of the Pulumi Cloud platform — will be positioned to lead in an AI-first world. The gap between teams that modernize and those that do not will widen rapidly in 2026 and beyond.

Try Pulumi for Free

Announcing the Next Generation of Pulumi Policies: AI-Accelerated Governance for the Cloud

Craig Symonds — Wed, 05 Nov 2025 00:05:00 +0000

The era of AI-accelerated development has created a paradox: the faster developers move, the bigger the governance challenge becomes. For years, security and platform teams have worked to “shift left,” but the tools available have been incomplete. Most focus on detection, which is necessary but not sufficient. They identify thousands of policy violations across an organization’s infrastructure but leave teams with an overwhelming backlog and no scalable way to remediate it. This creates a persistent gap between finding a problem and fixing it. The result is an impossible choice between development velocity and organizational control, forcing leadership to slow down innovation to manage risk.

Today, we end that compromise.

We are thrilled to announce the next generation of Pulumi Policies, a comprehensive governance solution that moves beyond detection to deliver AI-powered remediation at scale. We’re introducing a new lifecycle to secure your cloud: first, Get Clean by using AI to fix your existing policy violations. Second, Stay Clean by using policy as a universal guardrail that makes AI-driven development not just fast, but fundamentally safe. These are not strictly sequential steps; you can begin enforcing “Stay Clean” policies for all new infrastructure while you simultaneously work on the “Get Clean” process for your existing footprint.

Watch our Launch Video:

Part 1: Get Clean - From Thousands of Issues to a Compliant State

The first step to a secure cloud is tackling the mountain of existing misconfigurations spread across your environments.

First, Gain Complete Visibility with the New Policy Findings Hub

To fix your issues, you first need to see them clearly. We have introduced a powerful Policy Findings Hub designed to give every stakeholder the exact view they need.

The Overview Tab: A high-level dashboard with compliance scores for leadership to track trends and measure your organization’s posture.
The Compliance Tab: A control-centric view for auditors and infosec teams, grouping findings by policy (e.g., CIS, NIST) to simplify evidence gathering and prove compliance.
The Issues Tab: A collaborative workspace for platform and development teams to triage, assign, prioritize, and track the remediation of every policy issue.

This hub is powered by our flexible audit capabilities. With Audit Scans for IaC Stacks, you can get an instant compliance baseline on all your existing Pulumi-managed infrastructure without blocking developers or re-deploying thousands of stacks. This is combined with discovery scans of your cloud accounts to give you a single, unified view of every resource, whether managed by Pulumi or not.

The Solution to Remediation at Scale: AI-Powered Fixes with Pulumi Neo

Visibility creates a new problem: an overwhelming backlog. Manually fixing thousands of issues is an impossible task.

This is where Pulumi Neo provides a powerful solution.

Pulumi Neo, our AI platform engineer, is now integrated directly into the Policy Findings hub to automate the most difficult part of the process: the fix itself. From the Issues tab, your teams can now select a group of policy issues, assign them to Neo, and trigger a remediation flow.

Neo is smart. It will:

Analyze the non-compliant resources and the policies they are violating.
Understand the required configuration to make them compliant.
Generate a pull request with the exact code changes needed to fix the issues.

Most powerfully, for an unmanaged resource discovered via a cloud scan, Neo will generate the code to import the resource into a Pulumi stack and apply the fix. This “Import and Fix” workflow transforms unmanaged infrastructure into governed, compliant code, turning a task that could take a developer hours into a simple review-and-merge process. Your teams can finally burn down their issue backlog and achieve a state of continuous compliance.

Part 2: Stay Clean - The Universal Guardrail for Humans and AI

Once you begin cleaning your environment, the next challenge is to stay clean. As AI accelerates infrastructure creation, you need robust guardrails to ensure it doesn’t also accelerate the creation of security risks.

The answer is Policy as Real Code. Pulumi Policies uses general-purpose languages like TypeScript and Python to create sophisticated guardrails that govern every change. To help you establish these controls immediately, we are launching a new suite of pre-built compliance packs authored and maintained by Pulumi experts.

Framework	AWS	Azure	Google Cloud
CIS Controls v8.1	✅	✅	✅
NIST SP 800-53 Rev. 5	✅
PCI DSS v4.0	✅
HITRUST CSF v11.5	✅	✅	✅
Pulumi Best Practices	✅	✅	✅

These policies act as a universal guardrail for your entire organization by blocking non-compliant changes during pulumi up, before they are ever created. Learn more about our compliance packs.

And now, you can even use Neo to author new policies. You can ask Neo in plain English to “create a policy that prevents overly permissive IAM roles,” and it will generate the code for you. This creates a powerful, dynamic governance system where AI can help you build the very rules that then govern its own actions. If you then ask Neo to create an admin role, the deployment will be blocked by the policy it just helped write. This is how we make AI safe to go fast.

A New Era of Collaboration

This “Get Clean, Stay Clean” lifecycle transforms how teams work together:

Platform Teams lead prevention by building real-code guardrails, proving their value with measurable compliance scores.
Security Teams drive the “Get Clean” process with continuous, non-blocking audit scans and use the Findings hub to manage compliance without slowing development.

Governance That Accelerates You

The age of AI-driven development demands AI-powered governance. With this new generation of Pulumi Policies, we are providing the essential infrastructure for platform engineering teams to not just survive, but thrive. You can finally build preventative guardrails that developers love, secure your cloud at scale, and transform governance from a blocker into a business accelerator. Learn more about Pulumi Insights & Governance capabilities.

This powerful new experience is available today. Navigate to the Policies and Policy Findings tab in Pulumi Cloud to explore your new governance capabilities and meet the future of platform engineering.

Try Pulumi Policies

Ready to try these features?

Sign up for Pulumi Cloud and start a Neo task
Read the Get Started guide to set up and apply a policy group to stacks and clouds.
Join the Community Slack to share feedback on the new features

For complete documentation, visit our Policies documentation.

Grounded AI: Why Neo Knows Your Infrastructure

Engin Diri — Fri, 24 Oct 2025 00:00:00 +0000

Ask a generic LLM to “fix my broken deployment,” and you’ll get generic advice. Ask Pulumi Neo the same question, and you’ll get a fix plan grounded in your actual infrastructure state.

The difference isn’t about better prompts or newer models. It’s about what the AI actually knows. Generic LLMs have been trained on the internet. Neo has been trained on your infrastructure.

This distinction matters more than you’d think.

The grounding problem

Most AI tools treat infrastructure like a text generation problem. You describe what you want, the model produces code, and you hope it works.

This approach fails more often than it succeeds because the AI has no connection to your actual infrastructure. It doesn’t know what resources you’ve already deployed, what dependencies exist, or what policies you need to follow. It’s generating code from patterns it learned on the internet, not from understanding your environment.

Pulumi Neo takes a different approach. It’s trained on your infrastructure context: your programs, your state, your resource relationships. When you ask Neo about drift, it doesn’t guess. It queries your actual deployment history. When it suggests a fix, it reasons from your program graph, not generic examples.

Neo doesn’t just autocomplete code. It understands state, resources, dependencies, and cloud behavior. When something fails, Neo explains why, not just what.

This is what makes it grounded AI: it’s anchored to the reality of your infrastructure, not floating in a sea of internet-trained probabilities.

What makes Neo different

The foundation matters more than you’d think.

Traditional AI tools fail in DevOps because they operate in a vacuum. Ask a generic LLM to “fix my broken deployment,” and you’ll get generic advice that ignores your specific infrastructure, your state, your constraints.

The difference with systems like Pulumi Neo comes down to what powers them: not just data, but context.

Think of it this way: a data lake is a massive repository of information sitting inert, waiting to be queried. A context lake is something else entirely. It’s a structured repository that aggregates knowledge about your domain and feeds it to AI systems.

A context lake contains things that matter:

Your infrastructure programs, resource definitions, API schemas, service dependencies. In Pulumi’s case, this includes your program graph, component models, stack configurations.

Real-time metrics, deployment history, drift detection, policy violations. The live pulse of your infrastructure as it actually runs in production.

Ownership information, compliance policies, access controls, quality signals. The organizational context that determines what changes are safe, who can make them, and why they matter.

Pulumi’s approach builds on this principle. Your infrastructure programs, state files, resource metadata, policy definitions. All of this becomes queryable context. Neo doesn’t hallucinate solutions because it’s grounded in your actual infrastructure. It knows what you’ve deployed, how resources relate to each other, what dependencies exist, what’s drifted.

This is the architectural shift that makes AI-powered DevOps actually work. AI agents are only as effective as the context they can access and the guardrails you have in place to keep them in check. You’re not just automating actions anymore. You’re automating understanding.

Grounded reasoning in practice

Early DevOps automated actions. Modern AI agents automate decisions. But grounded AI does something different: it automates understanding.

When you combine observability data, infrastructure-as-code programs, and deployment history into a context lake, the AI doesn’t just predict what might happen. It reasons about what is happening based on your actual infrastructure state.

Neo can watch infrastructure drift, identify whether it stems from a manual change or a misconfigured resource, and generate a fix plan that maps back to your Pulumi program. That feedback isn’t guesswork. It’s grounded in the same infrastructure metadata developers already use. The context lake ensures Neo isn’t making probabilistic predictions. It’s reasoning from your specific infrastructure truth.

This is what separates grounded AI from generic LLMs. Neo doesn’t just automate deployment. It understands the intent behind it.

The human layer stays critical

Grounded AI doesn’t replace engineering judgment. Every engineer who’s tried to prompt a generic LLM to “write a Pulumi program for me” knows how quickly hallucinations creep in. You still need human context: the judgment to choose the right platform, the discipline to model dependencies, the awareness of compliance and cost.

That’s where grounded AI like Neo fits best. It’s not a chatbot for infrastructure. It’s an extension of your own reasoning.

Neo learns from the same program graph and state data you already manage, drawing from a continuously updated context lake of your infrastructure reality. Its recommendations stay grounded in your environment, not floating in a generic prompt window trained on the public internet.

The opportunity isn’t about replacing engineers. It’s about creating smarter feedback loops between human expertise and AI that actually understands your infrastructure.

What grounded AI enables

The first generation of DevOps automated deployment. Grounded AI automates understanding.

Infrastructure becomes queryable. AI-powered observability correlates incidents before they cascade. CI/CD becomes continuous reasoning rather than continuous execution. Platform teams spend less time fighting YAML and more time guiding systems that reason alongside them.

The context lake architecture makes this possible. Instead of static documentation and scattered tribal knowledge, your infrastructure context becomes something AI agents can actually query and reason over.

Grounded AI won’t replace engineers. But it will change what engineering means: less time translating intent into YAML, more time reasoning about systems at a higher level.

Try it yourself

Want to see what infrastructure cognition looks like in practice? Get started with Neo and ask it about your infrastructure. Watch it reason through drift detection, generate fix plans, or explain complex resource relationships in plain English. For a wider survey of the category, see our guide to the best AI infrastructure tools.

Neo is available today for teams using Pulumi Cloud. The cognitive layer isn’t coming. It’s already here.

Neo Gets Smarter: New Modes, CLI Access & Sonnet 4.5

Pulumi Neo Team — Mon, 20 Oct 2025 00:00:00 +0000

Neo just got significantly more capable. We’ve shipped three major updates: Operating Modes for flexible control, full ecosystem tool access, and Claude Sonnet 4.5 for better performance on complex infrastructure tasks.

You decide how much control you keep

The same action has different risks in different contexts. Building a new dev environment is low risk. Opening a PR that changes shared infrastructure requires more scrutiny.

Operating Modes let you adjust autonomy based on context:

Review Mode: You approve the task plan, preview, and PR. You see everything before it happens.
Balanced Mode: Neo handles planning and previews. You approve mutating operations, such as updates or destroys. Less friction, control where it matters.
Auto Mode: Neo runs without stopping. For when you need speed and trust the outcome.

You can use Review Mode when Neo is updating production infrastructure. Use Balanced Mode when deploying application updates where you want to verify destructive changes but trust routine operations. Switch to Auto Mode when spinning up temporary dev environments for testing. You pick the mode that matches your situation. The same agent, flexible levels of autonomy.

Neo can do more

Neo now has access to more tools and information. This means it can handle more complex scenarios and work with the broader ecosystem your infrastructure depends on.

Full Pulumi CLI access: Neo can now run all Pulumi CLI operations, including stack imports, state management, and plugin operations that previously required manual intervention.
Ecosystem tooling built in: Neo now includes kubectl, Helm, AWS CLI, GCP, and Oracle Cloud CLI. This means Neo can verify Kubernetes deployments, install Helm charts, and run cloud-specific operations without leaving your workflow.
Private registry search: Your team builds reusable components to accelerate development and codify standards. Neo can now search your organization’s private registry, including READMEs, to find and use them. You get the benefit of your team’s existing work.

Sonnet 4.5

Neo now runs on Anthropic’s Sonnet 4.5 by default. Sonnet 4.5 delivers better performance on complex, multi-step infrastructure tasks that require sustained reasoning across multiple files and services.

Try Neo

Ready to try these features?

Sign in to Pulumi Cloud and start a Neo task
Read the Neo documentation for detailed guides
Join the Community Slack to share feedback on the new features

Announcing Pulumi Remote MCP Server

Artur Laksberg — Tue, 07 Oct 2025 00:00:00 +0000

We’re excited to announce the Pulumi Remote MCP Server—a hosted service that brings AI-powered infrastructure management to any AI assistant that supports the Model Context Protocol. Connect your favorite AI assistant to https://mcp.ai.pulumi.com/mcp and instantly access your Pulumi Cloud infrastructure, search resources across stacks, and delegate complex automation tasks to Pulumi Neo.

The Evolution of Pulumi MCP

Earlier this year, we launched the Pulumi MCP server as a local npm package that brought AI-assisted infrastructure management to developers’ machines. The adoption and feedback from users and partners has been positive, validating the power of combining AI assistants with infrastructure-as-code.

As the MCP ecosystem has matured and more organizations have adopted the protocol, a clear pattern has emerged: remote MCP servers are becoming the industry standard. Remote servers provide a key advantage—accessibility. One endpoint works everywhere, with no per-machine setup.

Following industry trends and feedback from users and partners, we’re introducing the Remote MCP Server to ease installation and version management. The remote server preserves everything developers love about the local version while adding powerful new capabilities like seamless Pulumi Neo integration.

Note: The local MCP server continues to be available and fully supported for developers who prefer local tooling or need offline capabilities.

Why Remote MCP?

The Pulumi Remote MCP Server runs as a hosted service. Instead of managing local installations, you configure it once and get automatic updates and consistent functionality across all your development environments.

Zero local setup, universal access

Instead of installing npm packages, you simply configure your AI assistant with a single URL: https://mcp.ai.pulumi.com/mcp. That’s it.

No per-machine installations - Works the same on your laptop, desktop, or cloud workstation
No manual updates - New features and improvements roll out automatically to all users
Works with any MCP-compatible AI assistant - Cursor, Claude Code, Windsurf, Claude Desktop, and more

For instructions on how to configure different AI assistants, see Pulumi MCP Server.

Centralized authentication & secrets

Remote MCP also solves a critical security challenge: credential management. Instead of scattering Pulumi Access Tokens across laptops, containers, and scripts, the Remote MCP Server uses OAuth-based authentication with your Pulumi Cloud organization.

When you first connect, a browser window opens where you:

Enter your Pulumi Access Token (which is validated server-side)
Select which organization to access
Return to your AI assistant - now authenticated

Your credentials are stored securely in Pulumi Cloud, not on your individual machine.

What Can You Do With It?

The Remote MCP Server is your AI assistant’s gateway to your entire Pulumi infrastructure. It combines real-time access to your cloud resources with the power of autonomous infrastructure automation through Pulumi Neo.

Discover and query infrastructure

Your AI assistant can instantly explore what you’ve deployed across your entire organization:

List all stacks in your organization
Search for specific resources across all stacks
Find resources by type, name, tags, or any property
Check for policy violations
View organization members and their roles
Identify security gaps, untagged resources, or misconfigured infrastructure

Ask questions like:

“Show me all RDS databases without encryption enabled”
“Which stacks have resources in us-east-1?”
“Find all Lambda functions using deprecated runtimes”

Generate infrastructure code

The MCP server connects directly to the Pulumi Registry, giving your AI assistant access to thousands of cloud resources with complete type information:

Browse available resources
Get detailed resource schemas
Access property documentation, input/output types, and examples

Your AI assistant can:

Look up the exact properties for any cloud resource
Generate type-safe infrastructure code in TypeScript, Python, Go, or any Pulumi language
Include proper configurations, security settings, and best practices
Reference real documentation and examples

This means code generation is more accurate and up-to-date with the latest provider versions.

Autonomous infrastructure with Pulumi Neo

This is where the Remote MCP Server truly shines. For complex infrastructure tasks that require multiple steps, code changes, testing, and pull requests, your AI assistant can delegate directly to Pulumi Neo—Pulumi’s autonomous infrastructure AI agent.

What makes Neo special:

Neo isn’t just an AI that writes code—it’s an AI that ships infrastructure changes autonomously:

Multi-step planning - Neo breaks down complex requests into actionable plans
Code generation at scale - Works across multiple stacks and repositories
Automated testing - Validates changes before creating pull requests
Pull request workflows - Creates PRs with detailed explanations and comments
Continuous execution - Runs in Pulumi Cloud, not consuming your local resources
Human-in-the-loop - Pauses for approval on critical changes

Real-world Neo examples:

Security remediation: “Ask Neo to find all security groups allowing SSH from 0.0.0.0/0 and create a PR restricting them to my office IP”

Neo will:

Search your infrastructure for overly permissive security groups
Create a detailed plan for restricting access
Generate infrastructure code changes
Create a pull request with explanations
Wait for your approval to merge

Runtime migrations: “Ask Neo to migrate all Lambda functions from Python 3.8 to Python 3.12 and create PRs for each affected stack”

Neo handles:

Finding all Lambda functions with Python 3.8
Checking for compatibility issues
Updating runtime configurations
Running tests to ensure functionality
Creating separate PRs per stack for easy review

Policy compliance: “Ask Neo to scan for policy violations and fix them automatically”

Neo will:

Identify violations across all stacks
Generate fixes following your policy rules
Test changes to ensure compliance
Create PRs with clear explanations of what was fixed and why

Cost optimization: “Ask Neo to find idle resources and create a plan to shut them down”

Neo analyzes usage patterns, identifies waste, and proposes infrastructure changes to reduce costs—all autonomously.

The key difference: your AI assistant identifies what needs to be done, and Neo does it—writing code, running tests, creating PRs, and managing the entire workflow in Pulumi Cloud.

Real-World Workflow

Here’s what a typical session looks like:

You: “What stacks do I have with ‘production’ in the name?”

AI Assistant: Uses get-stacks to list: api-production, web-production, data-production

You: “Are there any policy violations in those stacks?”

AI Assistant: Uses get-policy-violations and reports: 3 S3 buckets without encryption, 2 security groups too permissive

You: “Ask Neo to fix those violations and create a PR”

AI Assistant: Uses neo-bridge to launch Neo task, provides link

Neo: Autonomously creates plan, generates fixes, tests changes, creates PR with detailed explanation

You: Review PR, approve, merge

Getting Started

Ready to try it? Check out our documentation for configuration instructions for your AI assistant of choice.

Key points:

Configure once - Add https://mcp.ai.pulumi.com/mcp to your AI assistant’s MCP settings
Authenticate - Browser popup for token entry and org selection (one time)
Start asking - Query your infrastructure, generate code, delegate to Neo

The Remote MCP Server is available now for all Pulumi users. No installation required—just configure and connect.

Learn More

We’re excited to see what you build with AI-assisted infrastructure management. Let us know what you think in our Community Slack!

10 Things You Can Do With Our Infrastructure Agent, Neo

Meagan Cojocar — Mon, 06 Oct 2025 00:00:00 +0000

Since launching Pulumi Neo two weeks ago, we’ve seen platform teams discover creative ways to put their newest AI teammate to work. We have also been using Neo internally for a handful of use cases. Neo shifts the conversation from “what could AI do for infrastructure?” to “what can I actually accomplish with Neo today?”

The answer is quite a bit. Here are 10 concrete workflows that platform teams can use Neo for right now, each one designed to save hours of manual work while keeping humans in the driver seat.

1. Explain infrastructure in plain English or diagrams

Neo answers questions about your infrastructure instantly, turning complex resource relationships into conversational insights (and diagrams!).

No more digging through AWS consoles or parsing state files to understand what you have deployed. Neo speaks your language about your infrastructure. Ask questions like “What databases do we have?”, “Which S3 buckets aren’t encrypted?”, “Show me all resources in us-east-1” or “draw an architecture diagram of our service” and get immediate, accurate answers.

2. Generate infrastructure code from natural language

Neo turns plain English descriptions into production-ready Pulumi code, eliminating the blank-page problem that slows down new projects.

Instead of starting from scratch or hunting through documentation, describe what you need and Neo generates the complete infrastructure. Ask Neo: “Create a serverless API with PostgreSQL database and Redis cache on AWS” and it produces fully configured resources with proper networking, security groups, and IAM policies.

3. Assist with Terraform to Pulumi migration

Neo analyzes existing Terraform code (and/or state files) and generates equivalent Pulumi programs, accelerating migration without starting from scratch.

Migrating from Terraform to Pulumi traditionally means rewriting everything. Neo understands both languages and can convert HCL to TypeScript, Python, or Go while preserving your infrastructure patterns. You can also use the Terraform state file. Ask Neo: “Analyze the provided terraform state. Based on it, generate a python pulumi program that imports the resources without requiring updates.” and it generates equivalent code with proper typing and best practices.

Using this prompt with a Terraform state file containing EC2 instances and supporting resources, Neo generated a complete Pulumi program that imported everything without requiring updates:

4. Automate CI/CD pipeline generation

Neo creates complete GitHub Actions workflows configured specifically for Pulumi deployments, with proper preview stages.

Setting up CI/CD for infrastructure projects usually involves copying workflows from other repos and adapting them. Neo generates purpose-built pipelines that understand your stack structure and deployment patterns. Ask Neo: “Create a GitHub Actions pipeline for this Pulumi project with preview and production stages” and it produces workflows with proper secrets management, approval gates, and deployment strategies.

Here is what Neo gives us using that prompt on a very basic Pulumi project:

name: Simple Pulumi Deploy

on:
 push:
 branches: [main]
 workflow_dispatch:

env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
 AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
 AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

jobs:
 deploy:
 runs-on: ubuntu-latest
 steps:
 - name: Checkout code
 uses: actions/checkout@v4

 - name: Install Pulumi
 uses: pulumi/actions@v4
 with:
 pulumi-version: latest

 - name: Configure AWS credentials
 uses: aws-actions/configure-aws-credentials@v4
 with:
 aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
 aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
 aws-region: us-west-2

 - name: Deploy infrastructure
 run: |
 pulumi stack select production
 pulumi up --yes
 working-directory: ./infrastructure

5. Identify inefficient infrastructure patterns

Neo analyzes your infrastructure and spots patterns that create unnecessary management overhead, helping you consolidate and optimize.

When you have 10 identical EC2 instances that should be an auto-scaling group, or multiple Lambda functions doing the same job, Neo spots these patterns. Ask Neo: “Analyze our infrastructure for inefficient patterns” and it can identify consolidation opportunities.

Example analysis:

Neo identified these optimization opportunities:

1. Duplicate EC2 Instances (High Priority)
 • Found 8 identical t3.medium instances running the same application
 • Recommendation: Consolidate into an Auto Scaling Group
 • Benefits: Simplified management, automatic scaling, reduced configuration drift

2. Scattered Lambda Functions (Medium Priority)
 • Found 12 Lambda functions with identical runtime and dependencies
 • Recommendation: Combine into a single function with routing logic
 • Benefits: Reduced cold starts, simplified monitoring, easier updates

3. Redundant Security Groups (Low Priority)
 • Found 15 security groups with identical rules
 • Recommendation: Create shared security groups for common patterns
 • Benefits: Easier rule management, reduced complexity

6. Upgrade EKS/Kubernetes clusters

Neo manages Kubernetes version upgrades across multiple clusters, planning upgrade paths and validating compatibility.

Kubernetes upgrades are complex, especially across multiple clusters with different workloads. Neo analyzes your cluster configurations, identifies upgrade blockers, and creates systematic upgrade plans. Ask Neo: “Plan an upgrade path for our EKS clusters from 1.27 to 1.28” and it maps dependencies, checks add-on compatibility, and sequences the upgrades.

7. Respond to security vulnerabilities

When CVEs are announced, Neo can help security teams move at machine speed addressing them.

Security vulnerabilities require immediate response across your entire infrastructure. Neo understands your resource inventory and can quickly identify what’s affected by new CVEs. When you ask Neo: “Identify all resources affected by CVE-2025-XXXXX and create remediation PRs”, it scans your infrastructure, maps vulnerable components, and generates targeted fixes.

8. Remediate policy violations

Neo detects non-compliant resources and generates pull requests to fix them, turning compliance headaches into automated workflows.

When your policy violations detect unencrypted EBS volumes, overly permissive security groups, or missing resource tags, Neo identifies the issues and proposes specific fixes. Ask Neo: “Fix all policy violations in our production account” and it analyzes your compliance dashboard, identifies the problematic resources, and creates PRs with the exact changes needed.

9. Update Lambda runtimes across multiple accounts

Neo identifies outdated Lambda runtimes, stages updates, and manages the rollout across your entire AWS organization.

When AWS announces runtime deprecations, Neo scans all your accounts, identifies functions running outdated versions, and creates a systematic upgrade plan. Ask Neo: “Upgrade all Lambda functions to Python 3.11” and it maps your functions, checks compatibility, and generates account-by-account PRs.

Before (outdated runtime):

// Create the process email Lambda function
const lambdaFunction = new aws.lambda.Function("emailProcessor", {
 runtime: aws.lambda.Runtime.Python3d8, // ❌ Deprecated
 code: new pulumi.asset.AssetArchive({
 ".": new pulumi.asset.FileArchive("./lambda"),
 }),
 // ... other configuration
});

After (Neo’s upgrade):

// Create the process email Lambda function
const lambdaFunction = new aws.lambda.Function("emailProcessor", {
 runtime: aws.lambda.Runtime.Python3d11, // ✅ Upgraded
 code: new pulumi.asset.AssetArchive({
 ".": new pulumi.asset.FileArchive("./lambda"),
 }),
 // ... other configuration
});

10. Enforce multi-account policy compliance

Neo applies organizational policies across your entire AWS Organizations structure, ensuring governance at scale without creating bottlenecks.

Large organizations struggle to maintain consistent policies across dozens of AWS accounts. Neo understands your organizational structure and can enforce tagging standards, encryption requirements, and security policies uniformly. Ask Neo: “Ensure all accounts follow our tagging and encryption policies” and it audits compliance across your organization.

Your newest platform engineer

These workflows represent just the beginning of what’s possible when an AI agent deeply understands infrastructure context.

The outcome is simple: spend less time on operational toil and more time on the architecture and policies that make your organizations successful.

Get started with Neo and discover what becomes possible when AI truly understands infrastructure.

The future of platform engineering isn’t about choosing between speed and safety. It’s about having tools that help you with both simultaneously.

Policy Comes to Team and Enterprise, with New Out-of-the-box Policies and Management Experience

Luke Ward — Wed, 17 Sep 2025 00:00:00 +0000

Pulumi’s Infrastructure as Code has included a powerful policy engine from day one. Over the past year, we’ve been enhancing it significantly to provide stronger governance for modern cloud platforms. Until now, these capabilities were limited to our Business Critical tier. Today, we’re excited to announce that policy guardrails are now available to all Team and Enterprise customers. Alongside this, we’re launching a redesigned policy management experience and introducing out-of-the-box policy packs that make it easier than ever to secure, govern, and optimize your cloud environments—even when powered by AI agents like Pulumi Neo.

Policy as Guardrails for AI Infrastructure

As developer velocity accelerates, AI agents like Neo are helping teams create and manage infrastructure at unprecedented speed. That speed creates a governance challenge: how do you ensure security, compliance, and cost-effectiveness without slowing developers down?

Policy guardrails provide the answer. Pulumi’s Policy as Code (PaC) framework applies DevOps best practices to governance, letting you codify rules, test them through pull requests, and version them alongside your infrastructure. With policies in place, you can ensure that every change stays within your organization’s standards, whether it is a human developer or Neo making a change.

What Makes Pulumi Policy Different

Unlike many other policy solutions, Pulumi Policy uses general-purpose programming languages such as TypeScript and Python rather than restrictive DSLs or YAML. This unlocks richer and more flexible capabilities, including:

Conditional logic to codify your organization’s nuanced rules.
Reusable functions and abstractions to create a shared library of policies.
Unit tests to validate policy behavior before rollout.

With today’s updates, this foundation is easier to adopt and scale across your teams than ever before.

New Simplified Experience

While Pulumi Policy has always allowed you to enforce compliance using TypeScript and Python, the existing workflow could be cumbersome—discovering policies required switching between the console and CLI, and applying them involved multiple steps.

We’ve removed that friction to provide the seamless guardrails needed for this high-velocity, AI-driven world. The new experience is built around a streamlined, in-console workflow for discovering, applying, and managing your policies.

In-Console Discoverability

We are making it easier to find and understand policy packs. You can now browse a rich set of pre-built policy packs directly within the Pulumi Cloud console—no CLI or context switching required.

You’ll find:

A rich browsing experience: See all available policy packs in one place.
Detailed information: Each pack comes with a clear description of the policies included.
Organization-approved packs: Org admins can curate a list of approved policy packs, ensuring your teams—and AI agents like Neo—are using a vetted set of policies.

New Pre-Built Policy Packs

To help you establish guardrails immediately, we have authored several pre-built policy packs. We are excited to highlight two that are available today:

Pulumi Best Practices: A foundational set of recommended governance and security controls that serves as a strong starting point for any organization.
HITRUST CSF v11.5: Provides predefined controls that help align cloud resources with HITRUST CSF requirements.

Streamlined Policy Management and Application

The new policy management experience dramatically simplifies policy application:

Intuitive Interface: Browse, select, and apply policies through a streamlined workflow using Policy Groups to bundle related policies and apply them to your stacks or cloud accounts.
Expanded Scale: We’ve eliminated the 4,000 stack UI limit on policy groups, allowing you to scale policies across your entire infrastructure.
Granular Enforcement: Configure the enforcement level (advisory or mandatory) for policies to either warn developers or block non-compliant deployments entirely.

Preventative and Audit Approaches

Pulumi Policy supports two complementary approaches to governance.

Preventative policies: Block non-compliant deployments during pulumi up, providing real-time guardrails for developers and AI agents.
Audit policies: Continuously scan existing resources for ongoing compliance monitoring, giving you a complete picture of your cloud security posture.

This dual approach ensures new deployments meet standards while maintaining visibility across your entire infrastructure. Learn more in our Preventative vs. Audit Policies guide.

How to Get Started

Getting started with the new policy management experience is straightforward:

Browse Policy Packs: Navigate to the Policies tab in your Pulumi Cloud console to discover pre-built policy packs like Pulumi Best Practices and HITRUST CSF. Learn more about Policy as Code configuration and setup.
Choose Your Approach: Decide whether you need preventative policies (to block non-compliant deployments during pulumi up) or audit policies (for continuous compliance monitoring across all cloud resources).
Create Policy Groups: Set up policy groups to bundle related policies and apply them to your stacks or cloud accounts.
Configure Enforcement: Set enforcement levels (advisory, mandatory, or remediate) for each policy based on your requirements.

Now Available To Team and Enterprise Customers

To ensure every organization can build with confidence, these policy management enhancements and the pulumi-best-practices policy packs are now available to Team and Enterprise customers. Checkout our pricing page for more information.

Conclusion

Governance doesn’t need to be a roadblock—it can be a foundation that accelerates innovation. With enhanced policy management and ready-to-use guardrails, Pulumi helps your teams and AI agents like Neo move fast while staying secure, compliant, and cost effective.

Try the new experience in Pulumi Cloud today, and let us know what you think in the Pulumi Community Slack.

Meet Neo, Your Newest Platform Engineer

Pulumi Neo Team — Tue, 16 Sep 2025 07:00:00 -0600

AI coding assistants have transformed the speed at which developers can write and deploy code. Pull request velocity has increased significantly. Feature delivery has accelerated beyond what we thought possible just two years ago. This should be a victory for everyone in the software organization.

Instead, it’s created significant challenges for infrastructure and platform teams.

Every line of code that ships faster creates new platform needs: monitoring, secrets management, deployment pipelines, and compliance checks. The rapid pace also increases the risk of insecure or non-compliant code reaching production. Developer velocity has increased dramatically, but platform teams haven’t scaled to match.

Platform teams are struggling because they lack the proper tools for their unique challenges. They’re caught in a velocity trap where generic AI accelerates developers while leaving platform teams behind. We’re optimizing software development speed while creating global infrastructure debt.

The very success of developer AI has become a challenge for the platform team.

Introducing Pulumi Neo

Neo is a purpose-built infrastructure automation agent that amplifies your platform engineering capabilities. Built on Pulumi’s platform, Neo has a deep understanding of cloud context, IaC, secrets and configuration, internal developer platforms, and more. It automatically respects your security and policy guardrails and works in tandem with human-in-the-loop approvals and controls.

For years, we’ve worked closely with platform and infrastructure teams. We’ve observed how the demands on these teams have intensified as development velocity has increased. We understand the challenges they face in keeping pace with rapidly evolving infrastructure needs.

The pattern became clear. Generic AI tools excel at generating code but lack the context to understand the unique challenges of infrastructure, including state management, tracking dependencies across services, scaling across tens to hundreds of repositories, and anticipating the effects of changes across an organization.

We’ve heard countless stories from platform engineers about spending hours tracking down dependencies, credential locations, and configuration relationships across their systems. When they try to use generic AI tools for these tasks, the tools often suggest changes without understanding the broader organizational context and dependencies.

That’s when we realized: Platform engineers don’t need faster AI tools. They need their own dedicated AI tool, one that understands infrastructure context, speaks their language, and works within their constraints.

These insights led us to develop Neo, an AI agent specifically designed for platform and infrastructure engineering teams. Unlike generic coding assistants, Neo understands the full context of infrastructure: it knows that changes span multiple repositories, that every modification has downstream effects, and that compliance and security policies matter as much as the code itself.

Built for Enterprise Trust

Neo represents a fundamentally different approach to AI in infrastructure. We view the Pulumi platform’s existing features, like IaC, ESC, and policies, as the foundation for trustworthy AI automation. These aren’t additional guardrails we’ve added; they’re the very platform capabilities you’re already using. Understanding the impact scope through your infrastructure graph gives you confidence to move faster. Knowing compliance implications through your existing policies before making changes is what actually accelerates platform teams.

While other solutions retrofit generic AI models with infrastructure plugins, Neo is built from the ground up on proven enterprise foundations. It operates within your existing Pulumi governance frameworks, respects your policies, and maintains the audit trails and compliance controls your organization requires. Your investment in Pulumi’s platform features—the same ones that govern your infrastructure today—becomes Neo’s operational guardrails.

This isn’t experimental AI, it’s enterprise-ready automation that amplifies your expertise while working within the same platform controls that keep your infrastructure secure and compliant. The Pulumi features you rely on for governance become the accelerants that enable confident, rapid automation.

The Reinforcement Cycle

Neo becomes more capable the more you invest in infrastructure as code. Every Pulumi component you write, every policy you define, every environment configuration you establish becomes Neo’s operational context that guides its decisions.

This is the key shift. We’re talking about AI amplifying platform expertise. The engineers who understand the why behind infrastructure decisions become even more valuable when they have tools that can execute the how at scale.

The teams that lean into this cycle will find themselves accelerating while others are still struggling. They’ll be writing policies while others are responding to issues. They’ll be designing systems while others are updating versions.

The Platform Engineering Evolution

The bigger picture extends far beyond task automation. Platform engineering evolves from reactive to proactive. Teams shift from ticket takers to strategic enablers. Infrastructure becomes self-healing and self-documenting. The profession transforms from responding to urgent issues to crafting foundations.

Imagine infrastructure that evolves as fast as the code it supports. Imagine platform teams that can actually think about next quarter instead of just surviving next sprint. Imagine organizations that can sustain AI-driven development velocity because their infrastructure keeps pace automatically.

For the industry, this means the end of the “platform team bottleneck” narrative. No longer will infrastructure be the reason features don’t ship. No longer will platform engineers be seen as blockers in the velocity story. Instead, they’ll be the enablers who made sustainable speed possible.

This is the rise of strategic platform engineering. These are professionals who shape how organizations build and operate software, who establish the foundations that make everything else possible, and who finally have tools that match the complexity and importance of their work.

The Choice Before Us

What we’re launching is more than an AI agent. We’re introducing your newest platform engineer, who gets smarter as your infrastructure evolves. We’re launching a new era for platform engineering, one where the teams that enable everyone else finally have tools built specifically for their reality.

Neo represents our belief that platform engineers shouldn’t have to choose between speed and safety, between automation and control, between serving developers and driving strategy. These are false choices created by tools that lack an understanding of infrastructure.

The age of generic AI taught us what’s possible. The era of specialized AI shows us what’s practical. Platform teams have been overloaded by everyone else’s velocity long enough.

It’s time for platform engineering to keep pace instead of falling behind.

Neo is available today in public preview. Ready to meet your newest platform engineer? Join thousands of teams already transforming their infrastructure workflows with the Pulumi platform.

The future of platform engineering starts now, and it starts with Neo.

Pulumi Updates, Explained: AI-Powered Features in Pulumi CLI

Artur Laksberg — Thu, 22 May 2025 00:00:00 +0000

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

We’re excited to announce the new AI capabilities for Pulumi CLI powered by Pulumi Copilot that translate complex infrastructure changes into clear, human-readable explanations and help resolve deployment errors with actionable guidance. Enable these preview features with the --copilot flag.

At Pulumi, we’re committed to helping you deploy infrastructure efficiently and with minimal friction. Anyone who has worked with cloud infrastructure knows the frustration of sifting through large previews with numerous changes and the difficulty of interpreting cryptic error messages from cloud provider APIs when deployments fail.

There are two key challenges we’ve identified:

Understanding Preview Changes: Reviewing raw update previews can be overwhelming, especially for complex deployments with many resources. It’s difficult to quickly grasp the full scope and impact of pending infrastructure changes.
Interpreting Error Messages: When deployments fail, cloud providers often return error messages that, while comprehensive, bury the core issue within extensive diagnostics, requiring specialized expertise to interpret and address.

Our latest CLI update introduces AI capabilities that tackle both issues. Let’s explore how these new features improve your infrastructure management workflow.

Update Explanation

To address the first challenge above, we’ve introduced a new “explain” menu item in the CLI. When running pulumi preview or pulumi up with the --copilot flag, you’ll see this option in the interactive menu:

In this screenshot, you can see the standard Pulumi update preview showing an AWS VPC with multiple subnets, route tables, and gateways that need to be updated. The interactive menu at the bottom shows the new “explain” option with a sparkle emoji, indicating the AI-powered feature. When selected, this tool analyzes and provides a clear summary of all changes in your pending infrastructure update:

In this example, the AI explanation immediately identifies that this is a “VPC Renaming Operation - Tag Updates Only” and explains that the update is simply changing tags across all components. It clearly states this is a “safe, non-disruptive change that only affects resource labels.”

This explanation serves as a valuable sanity check before deployment. It presents a plain-language summary of exactly what changes will occur, helping you verify that the planned modifications align with your intentions. This visibility is particularly valuable when managing complex infrastructure with many resources, as it helps identify unexpected changes or potential drift between your code and deployed resources. The system also proactively flags potential issues such as security vulnerabilities or destructive updates that might impact your production environment.

Diagnostics Summarization

For the second challenge, when infrastructure deployments fail, cloud providers often return error messages that are technically accurate but difficult to decipher without specialized knowledge. Consider the following scenario where a Go-based Docker image build fails with vendoring issues:

In this example, the standard Go build process failed with a cryptic error about inconsistent vendoring. While the raw output does contain the necessary information, it’s buried within extensive build logs. Our AI-powered Copilot diagnostics immediately identifies the problem and provides a clear, actionable solution: “Docker image update failed due to inconsistent vendoring. Sync vendor with ‘go mod vendor’.”

This clear explanation identifies root causes and provides actionable guidance, transforming dense technical output into concise explanations. It reduces time to resolution by pinpointing both the problem and the exact command needed to fix it, eliminating the need to search through documentation or guess.

Getting Started

The AI-powered features are currently in preview. You can enable them by adding the --copilot flag to your Pulumi CLI commands:

pulumi preview --copilot
pulumi up --copilot

These features require Pulumi CLI version v3.171.0 or greater and the Pulumi Cloud backend. Organization administrators must enable Pulumi Copilot by navigating to Settings > Access Management > Pulumi Copilot in the Pulumi Cloud console.

During this preview period, we’re focused on:

Refining explanations based on user feedback
Adding new capabilities like suggesting fixes for common errors
Improving performance to ensure minimal impact on command execution time

We will continue to monitor usage patterns and incorporate your feedback before enabling these features by default.

We’re excited to see how these AI-powered insights enhance your infrastructure deployment workflow! Please share your experiences with us:

Connect with us in the Pulumi Community Slack
Open an issue on GitHub
Learn more about Pulumi Copilot

Try this today, Get Started with Pulumi for free

Your input will directly influence how we evolve these features from preview to general availability.

AI-Assisted Infrastructure as Code with Pulumi's Model Context Protocol Server

Mikhail Shilkov — Tue, 08 Apr 2025 00:00:00 +0000

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

Infrastructure as Code (IaC) has revolutionized how we manage cloud resources, but navigating complex cloud provider APIs, writing boilerplate code, and iterating through deployment cycles can still be time-consuming. Pulumi offers a fantastic developer experience using familiar programming languages. But what if we could make it even faster and more intuitive by integrating powerful AI assistants directly into the development loop?

This is where the Pulumi Model Context Protocol (MCP) Server integration shines. MCP is a specification that allows language models (like the AI in your coding assistant) to interact with external tools and data sources in a structured way. By connecting AI-powered code assistants with Pulumi’s CLI and registry via MCP, we can bring real-time resource information and infrastructure management directly into the development environment, dramatically reducing friction and accelerating workflows.

Several AI coding assistants like GitHub Copilot, Anthropic’s Claude Code, Windsurf and others are rapidly evolving; this post will use Cursor (an AI-first code editor) to demonstrate a real-world example of this synergy in action.

Setting up the Pulumi MCP Integration in Cursor

Before diving in, you typically need to configure your AI assistant to communicate with the Pulumi MCP server. For Cursor, you can do so by clicking the following button:

The assistant can now leverage Pulumi tools seamlessly. These tools are specific actions enabled by the MCP server—like searching the Pulumi Registry or running a pulumi command—allowing the assistant to gather information or interact with your Pulumi project.

For detailed setup instructions for Cursor, Claude Code, Windsurf, Claude Desktop, and other AI assistants, see the Pulumi MCP Server documentation.

The Goal: Provisioning an AKS Cluster

Our objective for this walkthrough is to provision a temporary Azure Kubernetes Service (AKS) cluster for a short experiment using Pulumi and TypeScript. We need the cluster created with minimal fuss and its kubeconfig exported for access.

The Traditional Approach

The conventional method for this task involves significant context switching: searching Azure and Pulumi documentation in a browser, writing code in an editor, running commands (pulumi preview, pulumi up) in a terminal, and manually correlating information between these different environments. This process can be slow, requires deep knowledge recall (or constant lookups), and is prone to errors during the manual translation from documentation to code.

The AI Assistant + Pulumi MCP Approach

Let’s walk through how the same task unfolds much more efficiently using an AI assistant integrated with Pulumi MCP.

Understanding the AI Assistant + Tool Interaction

Before detailing the steps, it’s helpful to understand the user interface flow when the AI assistant uses integrated tools like those provided by the Pulumi MCP integration. Typically:

The developer provides a prompt or instruction in the chat interface within their editor.
The AI assistant analyzes the request and determines that it needs specific information (like resource properties) or needs to perform an action (like running pulumi preview).
The assistant indicates it will use a specific tool (e.g., pulumi_registry_listResources, pulumi_cli_preview). This often appears as a distinct UI element in the chat, showing the tool name and parameters being used.
The tool executes via the MCP server, interacting with the Pulumi CLI or registry as needed.
The output or result from the tool (e.g., registry listings, preview results, deployment errors) is displayed directly in the chat interface.
The AI assistant processes this output and uses it to continue the task – either by generating code, providing an answer, or deciding on the next step (like suggesting a fix for an error).

This tight loop keeps the developer focused within their editor environment, minimizing disruptions.

Step-by-Step Walkthrough

Here’s how the AKS provisioning task played out:

The Request

The developer starts with a natural language request to the AI assistant within the editor:

“I have an empty Pulumi project with TypeScript. Please edit the program to provision an AKS cluster for me. It’s a temporary AKS cluster that I need for a short experiment, so I don’t need any particular configuration of it. Just export its kubeconfig when you are done. Please use the tools to lookup resource information and to run pulumi preview to make sure your code works. When done, run pulumi up for me.”

AI-Powered Resource Discovery

Instead of the developer manually searching docs, the AI assistant leverages the MCP integration.

The AI assistant first queries the Pulumi Registry via MCP to list resources within the Azure Native provider’s containerservice module (using the pulumi_registry_listResources tool). This immediately identified ManagedCluster as the relevant resource.
The AI assistant then requests detailed information specifically for ManagedCluster (using pulumi_registry_getResource), directly retrieving its required properties, descriptions, and structure without leaving the editor.

AI Code Generation & Editing

Using the information retrieved from the Pulumi Registry, the assistant generated the necessary TypeScript code. It defined the ManagedCluster resource with a basic configuration and used its code editing capabilities to insert the code directly into the developer’s index.ts file.

Integrated Validation

Before attempting a potentially time-consuming deployment, the assistant ran pulumi preview using the integrated CLI tool (pulumi_cli_preview). The preview output appeared directly in the chat.

The preview succeeded, showing the resources that would be created. The assistant then proceeded to run pulumi up using the integrated CLI tool (pulumi_cli_up). However, the deployment flagged an error: an incorrect attempt to interpolate a Pulumi Output<string> directly into a resource property.

AI-Assisted Debugging & Iteration

The assistant analyzed the error message from the tool:

It identified the incorrect string interpolation and proposed a fix (using pulumi.interpolate), applying it via the editing tool.
Self-correction: The first fix inadvertently introduced a circular dependency (caught by linters integrated into the editor and surfaced to the assistant). The assistant recognized this and further refined the code by removing the problematic property, correctly relying on Azure’s default behavior.
Another attempt to deploy using pulumi up failed, this time surfacing an Azure API error: the provided SSH key format was invalid.

Collaborative Problem Solving & Tooling

The developer, seeing the SSH key error, suggested using the Pulumi TLS package to generate a valid key dynamically.

The assistant used the integrated terminal tool to execute npm install @pulumi/tls.
It then edited the index.ts file again, incorporating the tls.PrivateKey resource and correctly using its publicKeyOpenssh output for the ManagedCluster’s Linux profile.

Successful Deployment

The subsequent pulumi preview showed the correct plan (including the new TLS key resource). The assistant then executed pulumi up, which completed successfully. The success message and resource summary appeared in the chat.

Accessing Outputs

Finally, retrieving the kubeconfig was trivial. The assistant used the stack output tool (pulumi_cli_stack_output kubeconfig) to fetch and display the configuration directly.

Why This Changes the Game

Integrating AI assistants with the Pulumi Model Context Protocol offers tangible benefits:

Reduced Context Switching: The entire workflow – discovery, coding, validation, deployment, debugging, output retrieval – happens primarily inside the editor. No more juggling browser tabs and terminal windows.
Accelerated Discovery: MCP integration provides immediate, context-aware access to Pulumi resource schemas and documentation snippets.
Faster Coding: AI generates boilerplate and resource definitions quickly based on registry data and natural language requests.
Tighter Feedback Loop: pulumi preview and pulumi up results are instantly available within the coding environment, enabling rapid iteration.
Intelligent Assistance & Collaboration: The AI assists with debugging complex errors, incorporates developer suggestions, and leverages integrated tooling effectively.

Conclusion

The synergy between AI coding assistants and the Pulumi Model Context Protocol (MCP) integration creates a remarkably efficient environment for Infrastructure as Code development. By bringing cloud resource knowledge, code generation, and the Pulumi CLI workflow directly into the editor, developers can build, deploy, and iterate on their infrastructure faster and with significantly less friction.

Whether you use Cursor, Copilot, Claude Code, Windsurf, or another emerging AI tool, integrating it with the Pulumi MCP server offers a glimpse into the future of streamlined, intelligent IaC development, ultimately boosting productivity and improving the overall developer experience.

AI/ML on Kubernetes: Deploying Models with Pulumi on Google Cloud

Sara Huddleston — Mon, 24 Mar 2025 07:32:19 +0000

Kubernetes has transformed cloud infrastructure by enabling scalable, containerized applications. While it initially gained traction for managing web applications and microservices, its capabilities now extend to AI/ML workloads, making it the go-to platform for data scientists and machine learning engineers.

Running AI/ML workloads on Kubernetes presents unique challenges, including:

Specialized hardware requirements (e.g., GPUs, TPUs)
Scalability for model training and inference
Complex data pipelines that integrate various cloud services
Infrastructure automation for seamless deployment

Google Cloud Kubernetes (GKE) provides a robust foundation for AI/ML workloads, but managing infrastructure manually can be cumbersome. This is where Pulumi comes in—enabling Infrastructure as Code (IaC) to automate and simplify AI/ML infrastructure on Kubernetes.

Pulumi: Automating AI/ML Infrastructure on Google Cloud

Pulumi is a modern Infrastructure as Code (IaC) tool that allows teams to define and manage cloud infrastructure using general-purpose programming languages like Python, TypeScript, and Go. This approach is particularly beneficial for AI/ML teams, as Python is already the dominant language in data science and machine learning.

With Pulumi, you can:

Provision and scale Kubernetes clusters on Google Cloud automatically.
Define AI/ML environments as code, making deployments repeatable and version-controlled.
Integrate infrastructure with machine learning pipelines, reducing operational overhead.

Deploying AI/ML Workloads on Kubernetes

Below, we explore two use cases for running AI/ML workloads on Google Kubernetes Engine (GKE) using Pulumi.

Use Case 1: Deploying a Large Language Model (LLM) with Retrieval Augmented Generation (RAG)

Large Language Models (LLMs) like GPT-3, Whisper, and DALL-E require significant infrastructure for training and inference. Retrieval Augmented Generation (RAG) enhances LLMs by integrating external knowledge sources, improving accuracy and relevance.

Using Pulumi, you can automate the deployment of an open-source LLM with RAG on Kubernetes.

Step 1: Set Up a Kubernetes Cluster on Google Cloud

import pulumi
import pulumi_gcp as gcp

# Create a GKE cluster for AI/ML workloads
cluster = gcp.container.Cluster("ml-cluster",
 location="us-central1",
 initial_node_count=3,
 node_version="1.23",
 min_master_version="1.23")

# Create a node pool optimized for ML workloads
node_pool = gcp.container.NodePool("ml-node-pool",
 cluster=cluster.name,
 node_config=gcp.container.NodePoolNodeConfigArgs(
 machine_type="n1-standard-4",
 oauth_scopes=[
 "https://www.googleapis.com/auth/devstorage.read_only",
 "https://www.googleapis.com/auth/logging.write",
 ],
 labels={"team": "ml"},
 shielded_instance_config=gcp.container.NodePoolNodeConfigShieldedInstanceConfigArgs(
 enable_secure_boot=True,
 enable_integrity_monitoring=True,
 ),
 ),
 initial_node_count=2,
 location="us-central1",
 version="1.23")
...

This Pulumi script:

Provisions a Kubernetes cluster on Google Cloud.
Creates a dedicated node pool optimized for AI/ML workloads.
Ensures security best practices for machine learning environments.

Step 2: Deploy the LLM with RAG Model

Once the cluster is set up, we can deploy the LLM with RAG model using Pulumi’s Kubernetes provider:

Define Kubernetes Deployments for the LLM and RAG model.
Package models as Docker containers and deploy them to the cluster.
Configure Kubernetes Services to expose APIs for model inference.
Use ConfigMaps and Secrets to manage parameters and credentials.

By defining these resources in Pulumi, deployments become fully automated, repeatable, and scalable.

Use Case 2: Training and Serving Custom Machine Learning Models

Beyond pre-trained LLMs, Kubernetes is ideal for training and serving custom AI/ML models. Pulumi can help automate every stage of the ML lifecycle.

Step 1: Set Up the Model Training Environment

Using Pulumi, we define a training environment with:

A Kubernetes Deployment for training jobs (GPU-enabled).
Persistent Volume Claims (PVCs) for storing training data and model artifacts.
Monitoring tools for tracking performance.

Step 2: Deploy and Serve the Trained Model

Once the model is trained, Pulumi can be used to:

Deploy the trained model as a Kubernetes Deployment.
Expose the model via a Kubernetes Service (REST API or gRPC).
Add autoscaling rules for dynamic inference scaling.

Pulumi allows teams to manage the entire AI/ML pipeline in a structured and automated way.

Try Jay’s demo code on Creating an AI Training Platform on GKE with Pulumi.

Why Use Pulumi for AI/ML on Kubernetes?

Pulumi provides several advantages for AI/ML teams running workloads on Kubernetes:

1. Use General-Purpose Languages for Infrastructure as Code

Most AI/ML engineers already work with Python or Go, and Pulumi lets them manage infrastructure using the same language.
No need to learn YAML or Kubernetes manifests—define everything programmatically.

2. Automate AI/ML Workflows

Define infrastructure, training jobs, and model serving in one unified IaC framework.
Ensure consistency across development, staging, and production environments.

3. Improve Scalability and Cost Efficiency

Pulumi integrates with Google Cloud AI services for optimized compute resources.
Automate autoscaling and resource allocation for AI/ML workloads.

4. Increase Security and Compliance

Manage credentials and secrets securely with Pulumi ESC (Secrets Management).
Apply policy-as-code to enforce security best practices.

Get Started with AI/ML on Kubernetes with Pulumi

Pulumi makes it easy to deploy, scale, and manage AI/ML workloads on Kubernetes, leveraging Google Cloud’s AI infrastructure. Whether you’re serving LLMs, training custom models, or automating ML pipelines, Pulumi provides a developer-friendly, scalable, and secure solution.

By combining Kubernetes, Google Cloud, and Pulumi, you can accelerate AI/ML innovation while reducing infrastructure complexity.

Run Open-Source LLMs on AWS EC2 with Ollama and Pulumi

Engin Diri — Mon, 27 Jan 2025 00:00:00 +0000

TL;DR. Want to self-host an open-source LLM on AWS? Use a g4dn.xlarge ($0.526/hr on-demand, 16 GB GPU memory) for 7B/8B models, a g5.xlarge ($1.006/hr, 24 GB) for 13B–14B models, a g5.2xlarge ($1.212/hr, 24 GB) for 32B models, or a g6e.2xlarge ($2.242/hr, 48 GB) for 70B models. Deploy with the Pulumi program below and Ollama will run any model from its library: DeepSeek-R1, Llama 3, Qwen, or Mistral, with a one-line change.

This guide walks through that deployment end-to-end: a single Pulumi program that provisions a GPU-enabled EC2 instance, installs Ollama and Open WebUI via cloud-init, and exposes both a chat UI and an OpenAI-compatible API. The model is configurable, so you can swap DeepSeek-R1 for Llama 3.1, Qwen 2.5, or Mistral without touching the infrastructure code.

Why run open-source LLMs on AWS EC2?

Self-hosting an open-source LLM on AWS gives you three things hosted APIs can’t: data stays inside your VPC, per-token costs collapse to a flat hourly rate at high volume, and you can fine-tune or quantize models freely under permissive licenses. Ollama handles all three concerns from a single binary: it downloads, manages, and serves models behind an OpenAI-compatible API on port 11434.

The original version of this post focused on DeepSeek-R1 because it landed in late January 2025 and reset expectations for what an open-weight reasoning model could do. DeepSeek-R1 is still an excellent default (MIT-licensed, strong on math and coding, with distilled 1.5B–70B variants) but the same infrastructure runs Meta’s Llama 3, Alibaba’s Qwen, and Mistral equally well. Picking a model is now a config change, not an infrastructure decision.

For reference, DeepSeek-R1 scores 79.8% on AIME 2024 (vs. 79.2% for OpenAI o1-1217), 97.3% on MATH-500 (vs. 96.4%), 96.3% on Codeforces (vs. 96.6%), and 49.2% on SWE-bench Verified (vs. 48.9%)—near-parity with closed frontier models on most reasoning benchmarks, with the same caveat that benchmark scores age fast.

You might also like:

Which models can I run, and which EC2 instance do I need?

The bottleneck for inference is GPU memory (VRAM): the model weights have to fit in it, plus a few GB of headroom for the KV cache. The table below maps the most common Ollama models to the smallest AWS GPU instance that comfortably runs them at 4-bit (Q4) quantization, which is what ollama pull <model> gives you by default.

Model family	Sizes	Approx. VRAM (Q4)	Smallest EC2 instance	On-demand price (us-east-1)
DeepSeek-R1 (distill)	1.5B / 7B / 8B	1–6 GB	`g4dn.xlarge` (T4, 16 GB)	$0.526/hr
Llama 3.1 / Llama 3.2	8B	~5 GB	`g4dn.xlarge` (T4, 16 GB)	$0.526/hr
Qwen 2.5	7B	~5 GB	`g4dn.xlarge` (T4, 16 GB)	$0.526/hr
Mistral 7B / Mistral Nemo	7B / 12B	5–8 GB	`g4dn.xlarge` (T4, 16 GB)	$0.526/hr
DeepSeek-R1 (distill)	14B	~10 GB	`g5.xlarge` (A10G, 24 GB)	$1.006/hr
Llama 3.3 / DeepSeek-R1	32B / 32B distill	~20 GB	`g5.2xlarge` (A10G, 24 GB)	$1.212/hr
Llama 3.1 / DeepSeek-R1	70B	~42 GB	`g6e.2xlarge` (L40S, 48 GB)	$2.242/hr
DeepSeek-R1 (full)	671B (MoE)	400 GB+	`p5.48xlarge` or multi-node	$98.32/hr

For most workloads—internal tools, RAG backends, code assistants—a g4dn.xlarge running an 8B model is the right starting point. Move up only if quality is the bottleneck.

How much does this cost vs. hosted APIs?

A g4dn.xlarge running 24/7 costs ~$378/month on-demand, or ~$237/month with a 1-year reserved instance. Whether that’s cheaper than a hosted API depends entirely on your token volume.

Compare against hosted pricing as of April 2026 (input + output blended, rough numbers):

Provider	Model	Approx. blended price
OpenAI	GPT-4o-mini	~$0.30 per 1M tokens
OpenAI	GPT-4o	~$5.00 per 1M tokens
Anthropic	Claude Sonnet 4	~$6.00 per 1M tokens
DeepSeek (hosted)	DeepSeek-V3	~$0.50 per 1M tokens

A g4dn.xlarge running Llama 3.1 8B sustains roughly 40–60 tokens/sec under single-user load, or about 100–155M tokens/month at 100% utilization. At that ceiling the effective rate is ~$2.40/M tokens—cheaper than GPT-4o or Claude, more expensive than GPT-4o-mini or DeepSeek’s own hosted API.

The takeaway: self-hosting wins on data residency, latency, and predictable cost at high utilization. Hosted APIs win below ~10M tokens/month or when you need frontier-class quality. Run the math against your actual token volume before committing.

How do I deploy Ollama on AWS EC2 with Pulumi?

Prerequisites

Before we start, make sure you have the following:

An AWS account
Pulumi CLI installed
AWS CLI installed and configured
A working understanding of Ollama

What is Ollama?

Ollama is an open-source runtime that downloads, manages, and serves large language models from a single binary. It runs on macOS, Linux, and Windows, supports GPU acceleration through CUDA and Metal, and exposes both a native HTTP API and an OpenAI-compatible API on port 11434. Most clients written for the OpenAI SDK work against an Ollama endpoint with only a base-URL change.

It supports the major open-weight families—DeepSeek-R1, Llama 3, Qwen, Mistral, Gemma, Phi—plus quantized and distilled variants for each. You pull a model by name and tag (ollama pull llama3.1:8b) and run it (ollama run llama3.1:8b); Ollama handles the rest.

Architecture

Create a new Pulumi project

First, scaffold a new Pulumi project. Run the following from an empty directory:

# Replace <language> with typescript, python, go, csharp, or yaml
pulumi new aws-<language>

Pick the language you are most comfortable with. The template installs the AWS provider and creates a working sample. You can delete the sample code—we’ll replace it with the snippets below.

Step 1: Create an instance role with S3 access

The EC2 instance needs to download NVIDIA drivers from a public AWS-managed S3 bucket. Create an IAM role with S3 read access and attach it to an instance profile:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as fs from "fs";

const role = new aws.iam.Role("deepSeekRole", {
 name: "deepseek-role",
 assumeRolePolicy: JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Action: "sts:AssumeRole",
 Effect: "Allow",
 Principal: {
 Service: "ec2.amazonaws.com",
 },
 },
 ],
 }),
});

new aws.iam.RolePolicyAttachment("deepSeekS3Policy", {
 policyArn: "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess",
 role: role.name,
});

const instanceProfile = new aws.iam.InstanceProfile("deepSeekProfile", {
 name: "deepseek-profile",
 role: role.name,
});

import pulumi
import pulumi_aws as aws
import json
import os

# IAM Role for EC2 instances
role = aws.iam.Role(
 "deepSeekRole",
 name="deepseek-role",
 assume_role_policy=json.dumps(
 {
 "Version": "2012-10-17",
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": "ec2.amazonaws.com",
 },
 }
 ],
 }
 ),
)

# Attach S3 read-only policy to the IAM Role
iam_policy_attachment = aws.iam.RolePolicyAttachment(
 "deepSeekS3Policy",
 policy_arn="arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess",
 role=role.name,
)

# Instance Profile containing the IAM Role
instance_profile = aws.iam.InstanceProfile(
 "deepSeekProfile", name="deepseek-profile", role=role.name
)

package main

import (
 "encoding/json"
 "os"

 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/iam"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 rolePolicy, err := json.Marshal(map[string]interface{}{
 "Version": "2012-10-17",
 "Statement": []map[string]interface{}{
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": map[string]interface{}{"Service": "ec2.amazonaws.com"},
 },
 },
 })
 if err != nil {
 return err
 }

 role, err := iam.NewRole(ctx, "deepSeekRole", &iam.RoleArgs{
 Name: pulumi.String("deepseek-role"),
 AssumeRolePolicy: pulumi.String(rolePolicy),
 })
 if err != nil {
 return err
 }

 // Attach S3 read-only policy to the IAM Role
 _, err = iam.NewRolePolicyAttachment(ctx, "deepSeekS3Policy", &iam.RolePolicyAttachmentArgs{
 PolicyArn: pulumi.String("arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"),
 Role: role.Name,
 })
 if err != nil {
 return err
 }

 // Instance Profile containing the IAM Role
 instanceProfile, err := iam.NewInstanceProfile(ctx, "deepSeekProfile", &iam.InstanceProfileArgs{
 Name: pulumi.String("deepseek-profile"),
 Role: role.Name,
 })
 if err != nil {
 return err
 }
 return nil
 })
}

using Pulumi;
using Pulumi.Aws.Ec2;
using Pulumi.Aws.Iam;
using System.Collections.Generic;
using System.IO;
using Pulumi.Aws.Ec2.Inputs;
using System.Text.Json;

return await Deployment.RunAsync(() =>
{
 var rolePolicy = JsonSerializer.Serialize(new Dictionary<string, object>
 {
 {
 ["Version"] = "2012-10-17",
 ["Statement"] = new[]
 {
 new Dictionary<string, object>
 {
 ["Action"] = "sts:AssumeRole",
 ["Effect"] = "Allow",
 ["Principal"] = new Dictionary<string, string>
 {
 ["Service"] = "ec2.amazonaws.com"
 }
 }
 }
 });

 var role = new Role("deepSeekRole", new RoleArgs
 {
 Name = "deepseek-role",
 AssumeRolePolicy = rolePolicy
 });

 var rolePolicyAttachment = new RolePolicyAttachment("deepSeekS3Policy", new RolePolicyAttachmentArgs
 {
 PolicyArn = "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess",
 Role = role.Name
 });

 var instanceProfile = new InstanceProfile("deepSeekProfile", new InstanceProfileArgs
 {
 Name = "deepseek-profile",
 Role = role.Name
 });

 var vpc = new Vpc("deepSeekVpc", new VpcArgs
 {
 CidrBlock = "10.0.0.0/16",
 EnableDnsHostnames = true,
 EnableDnsSupport = true
 });

 var subnet = new Subnet("deepSeekSubnet", new SubnetArgs
 {
 VpcId = vpc.Id,
 CidrBlock = "10.0.48.0/20",
 AvailabilityZone = "eu-central-1a",
 MapPublicIpOnLaunch = true
 });

 var internetGateway = new InternetGateway("deepSeekInternetGateway", new InternetGatewayArgs
 {
 VpcId = vpc.Id
 });

 var routeTable = new RouteTable("deepSeekRouteTable", new RouteTableArgs

name: deepseek-ollama-yaml
description: DeepSeek Ollama AWS example
runtime: yaml

variables:
 publicKey:
 fn::readFile: ./deepseek.rsa
 userData:
 fn::readFile: ./cloud-init.yaml
 amiFilter: "amzn2-ami-hvm-*-x86_64-gp2"
 amiOwner: "137112412989"
 amiId:
 fn::invoke:
 function: aws:ec2:getAmi
 arguments:
 filters:
 - name: name
 values: ["${amiFilter}"]
 owners: ["${amiOwner}"]
 mostRecent: true
 return: id

resources:
 deepSeekRole:
 type: aws:iam:Role
 properties:
 name: deepseek-role
 assumeRolePolicy: |
 {
 "Version": "2012-10-17",
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": "ec2.amazonaws.com"
 }
 }
 ]
 }

 deepSeekS3Policy:
 type: aws:iam:RolePolicyAttachment
 properties:
 role: ${deepSeekRole.name}
 policyArn: arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

 deepSeekProfile:
 type: aws:iam:InstanceProfile
 properties:
 name: deepseek-profile
 role: ${deepSeekRole.name}

Step 2: Create the network

Next, create the VPC, subnet, internet gateway, and route table. The security group opens ports 22 (SSH), 3000 (Open WebUI), and 11434 (Ollama API):

const vpc = new aws.ec2.Vpc("deepSeekVpc", {
 cidrBlock: "10.0.0.0/16",
 enableDnsHostnames: true,
 enableDnsSupport: true,
});

const subnet = new aws.ec2.Subnet("deepSeekSubnet", {
 vpcId: vpc.id,
 cidrBlock: "10.0.48.0/20",
 availabilityZone: pulumi.interpolate`${aws.getAvailabilityZones().then(it => it.names[0])}`,
 mapPublicIpOnLaunch: true,
});

const internetGateway = new aws.ec2.InternetGateway("deepSeekInternetGateway", {
 vpcId: vpc.id,
});

const routeTable = new aws.ec2.RouteTable("deepSeekRouteTable", {
 vpcId: vpc.id,
 routes: [
 {
 cidrBlock: "0.0.0.0/0",
 gatewayId: internetGateway.id,
 },
 ],
});

const routeTableAssociation = new aws.ec2.RouteTableAssociation("deepSeekRouteTableAssociation", {
 subnetId: subnet.id,
 routeTableId: routeTable.id,
});

const securityGroup = new aws.ec2.SecurityGroup("deepSeekSecurityGroup", {
 vpcId: vpc.id,
 egress: [
 {
 fromPort: 0,
 toPort: 0,
 protocol: "-1",
 cidrBlocks: ["0.0.0.0/0"],
 },
 ],
 ingress: [
 {
 fromPort: 22,
 toPort: 22,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 {
 fromPort: 3000,
 toPort: 3000,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 {
 fromPort: 11434,
 toPort: 11434,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 ],
});


# Create a VPC
vpc = aws.ec2.Vpc(
 "deepSeekVpc",
 cidr_block="10.0.0.0/16",
 enable_dns_hostnames=True,
 enable_dns_support=True,
)

# Create a subnet
subnet = aws.ec2.Subnet(
 "deepSeekSubnet",
 vpc_id=vpc.id,
 cidr_block="10.0.48.0/20",
 availability_zone="eu-central-1a",
 map_public_ip_on_launch=True,
)

# Create an internet gateway
internet_gateway = aws.ec2.InternetGateway("deepSeekInternetGateway", vpc_id=vpc.id)

# Create a route table and route table association
route_table = aws.ec2.RouteTable(
 "deepSeekRouteTable",
 vpc_id=vpc.id,
 routes=[
 aws.ec2.RouteTableRouteArgs(
 cidr_block="0.0.0.0/0", gateway_id=internet_gateway.id
 )
 ],
)

route_table_association = aws.ec2.RouteTableAssociation(
 "deepSeekRouteTableAssociation", subnet_id=subnet.id, route_table_id=route_table.id
)

# Create a security group
security_group = aws.ec2.SecurityGroup(
 "deepSeekSecurityGroup",
 vpc_id=vpc.id,
 egress=[
 {
 "from_port": 0,
 "to_port": 0,
 "protocol": "-1",
 "cidr_blocks": ["0.0.0.0/0"],
 }
 ],
 ingress=[
 {
 "from_port": 22,
 "to_port": 22,
 "protocol": "tcp",
 "cidr_blocks": ["0.0.0.0/0"],
 },
 {
 "from_port": 3000,
 "to_port": 3000,
 "protocol": "tcp",
 "cidr_blocks": ["0.0.0.0/0"],
 },
 {
 "from_port": 11434,
 "to_port": 11434,
 "protocol": "tcp",
 "cidr_blocks": ["0.0.0.0/0"],
 },
 ],
)

package main

import (
 "encoding/json"
 "os"

 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/iam"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // omitted for brevity
 }

 // Create a VPC
 vpc, err := ec2.NewVpc(ctx, "deepSeekVpc", &ec2.VpcArgs{
 CidrBlock: pulumi.String("10.0.0.0/16"),
 EnableDnsHostnames: pulumi.Bool(true),
 EnableDnsSupport: pulumi.Bool(true),
 })
 if err != nil {
 return err
 }

 // Create a subnet
 subnet, err := ec2.NewSubnet(ctx, "deepSeekSubnet", &ec2.SubnetArgs{
 VpcId: vpc.ID(),
 CidrBlock: pulumi.String("10.0.48.0/20"),
 AvailabilityZone: pulumi.String("eu-central-1a"),
 MapPublicIpOnLaunch: pulumi.Bool(true),
 })
 if err != nil {
 return err
 }

 // Create an internet gateway
 internetGateway, err := ec2.NewInternetGateway(ctx, "deepSeekInternetGateway", &ec2.InternetGatewayArgs{
 VpcId: vpc.ID(),
 })
 if err != nil {
 return err
 }

 // Create a route table and route table association
 routeTable, err := ec2.NewRouteTable(ctx, "deepSeekRouteTable", &ec2.RouteTableArgs{
 VpcId: vpc.ID(),
 Routes: ec2.RouteTableRouteArray{
 &ec2.RouteTableRouteArgs{
 CidrBlock: pulumi.String("0.0.0.0/0"),
 GatewayId: internetGateway.ID(),
 },
 },
 })
 if err != nil {
 return err
 }

 _, err = ec2.NewRouteTableAssociation(ctx, "deepSeekRouteTableAssociation", &ec2.RouteTableAssociationArgs{
 SubnetId: subnet.ID(),
 RouteTableId: routeTable.ID(),
 })
 if err != nil {
 return err
 }

 // Create a security group
 securityGroup, err := ec2.NewSecurityGroup(ctx, "deepSeekSecurityGroup", &ec2.SecurityGroupArgs{
 VpcId: vpc.ID(),
 Egress: ec2.SecurityGroupEgressArray{
 &ec2.SecurityGroupEgressArgs{
 FromPort: pulumi.Int(0),
 ToPort: pulumi.Int(0),
 Protocol: pulumi.String("-1"),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 },
 Ingress: ec2.SecurityGroupIngressArray{
 &ec2.SecurityGroupIngressArgs{
 FromPort: pulumi.Int(22),
 ToPort: pulumi.Int(22),
 Protocol: pulumi.String("tcp"),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 &ec2.SecurityGroupIngressArgs{
 FromPort: pulumi.Int(3000),
 ToPort: pulumi.Int(3000),
 Protocol: pulumi.String("tcp"),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 &ec2.SecurityGroupIngressArgs{
 FromPort: pulumi.Int(11434),
 ToPort: pulumi.Int(11434),
 Protocol: pulumi.String("tcp"),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 },
 })
 if err != nil {
 return err
 }

 return nil
 })
}

using Pulumi;
using Pulumi.Aws.Ec2;
using Pulumi.Aws.Iam;
using System.Collections.Generic;
using System.IO;
using Pulumi.Aws.Ec2.Inputs;
using System.Text.Json;

return await Deployment.RunAsync(() =>
{
 var rolePolicy = JsonSerializer.Serialize(new Dictionary<string, object>
 {
 // omitted for brevity
 {
 VpcId = vpc.Id,
 Routes =
 {
 new RouteTableRouteArgs
 {
 CidrBlock = "0.0.0.0/0",
 GatewayId = internetGateway.Id
 }
 }
 });

 var routeTableAssociation = new RouteTableAssociation("deepSeekRouteTableAssociation", new RouteTableAssociationArgs
 {
 SubnetId = subnet.Id,
 RouteTableId = routeTable.Id
 });

 var securityGroup = new SecurityGroup("deepSeekSecurityGroup", new SecurityGroupArgs
 {
 VpcId = vpc.Id,
 Egress =
 {
 new SecurityGroupEgressArgs
 {
 FromPort = 0,
 ToPort = 0,
 Protocol = "-1",
 CidrBlocks = { "0.0.0.0/0" }
 }
 },
 Ingress =
 {
 new SecurityGroupIngressArgs
 {
 FromPort = 22,
 ToPort = 22,
 Protocol = "tcp",
 CidrBlocks = { "0.0.0.0/0" }
 },
 new SecurityGroupIngressArgs
 {
 FromPort = 3000,
 ToPort = 3000,
 Protocol = "tcp",
 CidrBlocks = { "0.0.0.0/0" }
 },
 new SecurityGroupIngressArgs
 {
 FromPort = 11434,
 ToPort = 11434,
 Protocol = "tcp",
 CidrBlocks = { "0.0.0.0/0" }
 }
 }
 });

 var publicKey = File.ReadAllText("deepseek.rsa");
 var keyPair = new KeyPair("deepSeekKey", new KeyPairArgs
 {
 PublicKey = publicKey
 });

 var amazonLinux = Pulumi.Aws.Ec2.GetAmi.Invoke(new()
 {
 MostRecent = true,
 Filters = new[]
 {
 new GetAmiFilterInputArgs
 {
 Name = "name",
 Values = new[] { "amzn2-ami-hvm-*-x86_64-gp2" }
 },
 new GetAmiFilterInputArgs
 {
 Name = "architecture",


 deepSeekVpc:
 type: aws:ec2:Vpc
 properties:
 cidrBlock: 10.0.0.0/16
 enableDnsHostnames: true
 enableDnsSupport: true

 deepSeekSubnet:
 type: aws:ec2:Subnet
 properties:
 vpcId: ${deepSeekVpc.id}
 cidrBlock: 10.0.48.0/20
 availabilityZone: eu-central-1a
 mapPublicIpOnLaunch: true

 deepSeekInternetGateway:
 type: aws:ec2:InternetGateway
 properties:
 vpcId: ${deepSeekVpc.id}

 deepSeekRouteTable:
 type: aws:ec2:RouteTable
 properties:
 vpcId: ${deepSeekVpc.id}
 routes:
 - cidrBlock: 0.0.0.0/0
 gatewayId: ${deepSeekInternetGateway.id}

 deepSeekRouteTableAssociation:
 type: aws:ec2:RouteTableAssociation
 properties:
 subnetId: ${deepSeekSubnet.id}
 routeTableId: ${deepSeekRouteTable.id}

 deepSeekSecurityGroup:
 type: aws:ec2:SecurityGroup
 properties:
 vpcId: ${deepSeekVpc.id}
 ingress:
 - fromPort: 22
 toPort: 22
 protocol: tcp
 cidrBlocks:
 - 0.0.0.0/0
 - fromPort: 3000
 toPort: 3000
 protocol: tcp
 cidrBlocks:
 - 0.0.0.0/0
 - fromPort: 11434
 toPort: 11434
 protocol: tcp
 cidrBlocks:
 - 0.0.0.0/0
 egress:
 - fromPort: 0
 toPort: 0
 protocol: -1
 cidrBlocks:
 - 0.0.0.0/0

Step 3: Launch the GPU EC2 instance

Now create the EC2 instance itself. The example uses Amazon Linux because the NVIDIA driver install path is well-trodden, plus an SSH key pair you generate locally.

The default instance type is g4dn.xlarge—the cheapest option that fits any 7B/8B model. Bump it up if you picked a larger model from the table above: g5.xlarge for 13B–14B, g5.2xlarge for 32B, g6e.2xlarge for 70B. AWS publishes full specs for the G4, G5, and G6 families.

Generate the key pair if you don’t already have one:

openssl genrsa -out deepseek.pem 2048
openssl rsa -in deepseek.pem -pubout > deepseek.pub
ssh-keygen -f mykey.pub -i -mPKCS8 > deepseek.pem

const keyPair = new aws.ec2.KeyPair("deepSeekKey", {
 publicKey: pulumi.output(fs.readFileSync("deepseek.rsa", "utf-8")),
});

const deepSeekAmi = aws.ec2
 .getAmi({
 filters: [
 {
 name: "name",
 values: ["amzn2-ami-hvm-2.0.*-x86_64-gp2"],
 },
 {
 name: "architecture",
 values: ["x86_64"],
 },
 ],
 owners: ["137112412989"], // Amazon
 mostRecent: true,
 })
 .then(ami => ami.id);

const deepSeekInstance = new aws.ec2.Instance("deepSeekInstance", {
 ami: deepSeekAmi,
 instanceType: "g4dn.xlarge",
 keyName: keyPair.keyName,
 rootBlockDevice: {
 volumeSize: 100,
 volumeType: "gp3",
 },
 subnetId: subnet.id,
 vpcSecurityGroupIds: [securityGroup.id],
 iamInstanceProfile: instanceProfile.name,
 userData: fs.readFileSync("cloud-init.yaml", "utf-8"),
 tags: {
 Name: "deepSeek-server",
 },
});

export const amiId = deepSeekAmi;
export const instanceId = deepSeekInstance.id;
export const instancePublicDns = deepSeekInstance.publicIp;


# Key pair for SSH access
public_key = open("deepseek.rsa", "r").read()
key_pair = aws.ec2.KeyPair("deepSeekKey", public_key=public_key)


# Get the latest Amazon Linux 2 AMI
ami = aws.ec2.get_ami(
 filters=[
 {"name": "name", "values": ["amzn2-ami-hvm-2.0.*-x86_64-gp2"]},
 {"name": "architecture", "values": ["x86_64"]},
 ],
 owners=["137112412989"], # Amazon
 most_recent=True,
).id

# Create an EC2 instance
user_data = open("cloud-init.yaml", "r").read()
instance = aws.ec2.Instance(
 "deepSeekInstance",
 ami=ami,
 instance_type="g4dn.xlarge",
 key_name=key_pair.key_name,
 root_block_device=aws.ec2.InstanceRootBlockDeviceArgs(
 volume_size=100, volume_type="gp3"
 ),
 subnet_id=subnet.id,
 vpc_security_group_ids=[security_group.id],
 iam_instance_profile=instance_profile.name,
 user_data=user_data,
 tags={"Name": "deepSeek-server"},
)

pulumi.export("amiId", ami)
pulumi.export("instanceId", instance.id)
pulumi.export("instancePublicDns", instance.public_ip)

package main

import (
 "encoding/json"
 "os"

 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/iam"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // omitted for brevity

 // Key pair for SSH access
 publicKey, err := os.ReadFile("deepseek.rsa")
 if err != nil {
 return err
 }

 keyPair, err := ec2.NewKeyPair(ctx, "deepSeekKey", &ec2.KeyPairArgs{
 PublicKey: pulumi.String(string(publicKey)),
 })
 if err != nil {
 return err
 }

 // Get the latest Amazon Linux 2 AMI
 mostRecent := true
 ami, err := ec2.LookupAmi(ctx, &ec2.LookupAmiArgs{
 Filters: []ec2.GetAmiFilter{
 {
 Name: "name",
 Values: []string{"amzn2-ami-hvm-2.0.*-x86_64-gp2"},
 },
 {
 Name: "architecture",
 Values: []string{"x86_64"},
 },
 },
 Owners: []string{"137112412989"},
 MostRecent: &mostRecent,
 })
 if err != nil {
 return err
 }

 // Create an EC2 instance
 userData, err := os.ReadFile("cloud-init.yaml")
 if err != nil {
 return err
 }

 instance, err := ec2.NewInstance(ctx, "deepSeekInstance", &ec2.InstanceArgs{
 Ami: pulumi.String(ami.Id),
 InstanceType: pulumi.String("g4dn.xlarge"),
 KeyName: keyPair.KeyName,
 RootBlockDevice: &ec2.InstanceRootBlockDeviceArgs{
 VolumeSize: pulumi.Int(100),
 VolumeType: pulumi.String("gp3"),
 },
 SubnetId: subnet.ID(),
 VpcSecurityGroupIds: pulumi.StringArray{securityGroup.ID()},
 IamInstanceProfile: instanceProfile.Name,
 UserData: pulumi.String(string(userData)),
 Tags: pulumi.StringMap{
 "Name": pulumi.String("deepSeek-server"),
 },
 })
 if err != nil {
 return err
 }

 ctx.Export("amiId", pulumi.String(ami.Id))
 ctx.Export("instanceId", instance.ID())
 ctx.Export("instancePublicDns", instance.PublicIp)

 return nil
 })
}

using Pulumi;
using Pulumi.Aws.Ec2;
using Pulumi.Aws.Iam;
using System.Collections.Generic;
using System.IO;
using Pulumi.Aws.Ec2.Inputs;
using System.Text.Json;

return await Deployment.RunAsync(() =>
{
 var rolePolicy = JsonSerializer.Serialize(new Dictionary<string, object>
 {
 // omitted for brevity
 Name = "architecture",
 Values = new[] { "x86_64" }
 }
 },
 Owners = new[] { "137112412989" }
 });

 var userData = File.ReadAllText("cloud-init.yaml");
 var instance = new Instance("deepSeekInstance", new InstanceArgs
 {
 Ami = amazonLinux.Apply(r => r.Id),
 InstanceType = "g4dn.xlarge",
 KeyName = keyPair.KeyName,
 RootBlockDevice = new InstanceRootBlockDeviceArgs
 {
 VolumeSize = 100,
 VolumeType = "gp3"
 },
 SubnetId = subnet.Id,
 VpcSecurityGroupIds = { securityGroup.Id },
 IamInstanceProfile = instanceProfile.Name,
 UserData = userData,
 Tags = { { "Name", "deepSeek-server" } }
 });

 return new Dictionary<string, object?>
 {
 ["amiId"] = amazonLinux.Apply(r => r.Id),
 ["instanceId"] = instance.Id,
 ["instancePublicDns"] = instance.PublicIp
 };
});


 deepSeekKey:
 type: aws:ec2:KeyPair
 properties:
 publicKey: ${publicKey}

 deepSeekInstance:
 type: aws:ec2:Instance
 properties:
 ami: ${amiId}
 instanceType: "g4dn.xlarge"
 keyName: ${deepSeekKey.keyName}
 rootBlockDevice:
 volumeSize: 100
 volumeType: gp3
 subnetId: ${deepSeekSubnet.id}
 vpcSecurityGroupIds:
 - ${deepSeekSecurityGroup.id}
 iamInstanceProfile: ${deepSeekProfile.name}
 userData: ${userData}
 tags:
 Name: deepSeek-server

outputs:
 AmiId: ${amiId}
 InstanceId: ${deepSeekInstance.id}
 InstancePublicDns: ${deepSeekInstance.publicIp}

Step 4: Install Ollama via cloud-init

The EC2 instance is a blank box until cloud-init runs. The user-data script below installs the NVIDIA GRID drivers, Docker, and the NVIDIA Container Toolkit, then starts the Ollama and Open WebUI containers. To switch models, edit the ollama run line—the rest is identical regardless of which model you want.

#cloud-config
users:
- default

package_update: true

packages:
- apt-transport-https
- ca-certificates
- curl
- openjdk-17-jre-headless
- gcc

runcmd:
- yum install -y gcc kernel-devel-$(uname -r)
- aws s3 cp --recursive s3://ec2-linux-nvidia-drivers/latest/ .
- chmod +x NVIDIA-Linux-x86_64*.run
- /bin/sh ./NVIDIA-Linux-x86_64*.run --tmpdir . --silent
- touch /etc/modprobe.d/nvidia.conf
- echo "options nvidia NVreg_EnableGpuFirmware=0" | sudo tee --append /etc/modprobe.d/nvidia.conf
- yum install -y docker
- usermod -a -G docker ec2-user
- systemctl enable docker.service
- systemctl start docker.service
- curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
- yum install -y nvidia-container-toolkit
- nvidia-ctk runtime configure --runtime=docker
- systemctl restart docker
- docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama --restart always ollama/ollama
- sleep 120
- docker exec ollama ollama run deepseek-r1:7b
- docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

You might also like:

Step 5: Deploy the infrastructure

Make sure your AWS credentials are configured:

aws configure

Pulumi supports several other authentication methods for the AWS provider. Once credentials are in place, deploy the infrastructure:

pulumi up

Pulumi previews the changes; type yes to confirm.

pulumi up
Please choose a stack, or create a new one: <create a new stack>
Please enter your desired stack name.
To create a stack in an organization, use the format <org-name>/<stack-name> (e.g. `acmecorp/dev`): dev
Please enter your desired stack name.
Created stack 'dev'
Previewing update (dev)
View in Browser (Ctrl+O): https://app.pulumi.com/dirien/deepseek-ollama-typescript/dev/previews/1dbb18ea-ba31-4d5b-9510-5dce19eb8ee8
Type Name Plan
+ pulumi:pulumi:Stack deepseek-ollama-typescript-dev create
+ ├─ aws:ec2:KeyPair deepSeekKey create
+ ├─ aws:ec2:Vpc deepSeekVpc create
+ ├─ aws:iam:Role deepSeekRole create
+ ├─ aws:iam:InstanceProfile deepSeekProfile create
+ ├─ aws:ec2:SecurityGroup deepSeekSecurityGroup create
+ ├─ aws:ec2:RouteTable deepSeekRouteTable create
+ ├─ aws:ec2:InternetGateway deepSeekInternetGateway create
+ ├─ aws:ec2:Subnet deepSeekSubnet create
+ ├─ aws:iam:RolePolicyAttachment deepSeekS3Policy create
+ ├─ aws:ec2:RouteTableAssociation deepSeekRouteTableAssociation create
+ └─ aws:ec2:Instance deepSeekInstance create
Outputs:
amiId : "ami-085131ff43045c877"
instanceId : output<string>
instancePublicDns: output<string>
Resources:
+ 12 to create
Do you want to perform this update? yes
Updating (dev)
View in Browser (Ctrl+O): https://app.pulumi.com/dirien/deepseek-ollama-typescript/dev/updates/1
Type Name Status
+ pulumi:pulumi:Stack deepseek-ollama-typescript-dev created (40s)
+ ├─ aws:ec2:KeyPair deepSeekKey created (0.47s)
+ ├─ aws:iam:Role deepSeekRole created (1s)
+ ├─ aws:ec2:Vpc deepSeekVpc created (12s)
+ ├─ aws:iam:InstanceProfile deepSeekProfile created (6s)
+ ├─ aws:iam:RolePolicyAttachment deepSeekS3Policy created (0.90s)
+ ├─ aws:ec2:InternetGateway deepSeekInternetGateway created (0.69s)
+ ├─ aws:ec2:Subnet deepSeekSubnet created (11s)
+ ├─ aws:ec2:SecurityGroup deepSeekSecurityGroup created (2s)
+ ├─ aws:ec2:RouteTable deepSeekRouteTable created (1s)
+ ├─ aws:ec2:RouteTableAssociation deepSeekRouteTableAssociation created (0.92s)
+ └─ aws:ec2:Instance deepSeekInstance created (12s)
Outputs:
amiId : "ami-085131ff43045c877"
instanceId : "i-0ae7495781ace3e81"
instancePublicDns: "18.159.211.136"
Resources:
+ 12 created
Duration: 42s

The infrastructure provisions in under a minute, but the cloud-init script needs another 5–10 minutes to install drivers, pull container images, and download the model weights. SSH in to watch the progress, or just wait and load the Web UI when it’s ready.

ssh -i deepseek.pem ec2-user@<instance-public-ip>

Check the container status:

sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c8714335e205 ghcr.io/open-webui/open-webui:main "bash start.sh" 6 minutes ago Up 6 minutes (healthy) 0.0.0.0:3000->8080/tcp, :::3000->8080/tcp open-webui
bf4bb3b7ede1 ollama/ollama "/bin/ollama serve" 8 minutes ago Up 7 minutes 0.0.0.0:11434->11434/tcp, :::11434->11434/tcp ollama
[ec2-user@ip-10-0-58-122 ~]$

Step 6: Access the Web UI or API

Once the containers are healthy, open http://<instance-public-ip>:3000 in your browser for Open WebUI:

Open WebUI is not secured by default. Restrict the security group to your IP, put it behind an authenticated proxy, or terminate TLS at an ALB before exposing it to the internet.

For programmatic access, Ollama exposes an OpenAI-compatible API on port 11434. Most clients written for the OpenAI SDK only need a base-URL change:

from openai import OpenAI

client = OpenAI(
 base_url="http://<instance-public-ip>:11434/v1",
 api_key="ollama", # required by the SDK, ignored by Ollama
)

response = client.chat.completions.create(
 model="llama3.1:8b",
 messages=[{"role": "user", "content": "Why is the sky blue?"}],
)
print(response.choices[0].message.content)

How do I switch models?

Ollama hosts every major open-weight family in its model library. Pulling a different model is two commands inside the EC2 instance—or a one-line edit to the cloud-init script if you want it provisioned automatically:

# DeepSeek-R1 distill (default in this guide)
docker exec ollama ollama run deepseek-r1:7b

# Llama 3.1 (Meta, 8B)
docker exec ollama ollama run llama3.1:8b

# Qwen 2.5 (Alibaba, 7B)
docker exec ollama ollama run qwen2.5:7b

# Mistral (7B)
docker exec ollama ollama run mistral:7b

# Larger reasoning model (needs g5.2xlarge or larger)
docker exec ollama ollama run deepseek-r1:32b

Tags follow a <size> or <size>-<quantization> pattern—8b, 8b-instruct-q4_K_M, 8b-instruct-q8_0. Q4 is the default and the right starting point; bump to Q8 only if you have spare VRAM and notice quality issues with Q4. Browse the full tag list for any model on its Ollama library page.

Cleaning up

When you’re done, tear everything down:

pulumi destroy

What are the next steps?

You now have a reproducible, IaC-managed deployment of any open-source LLM on AWS. The infrastructure is fixed; the model is a parameter. From here, the natural extensions are wiring this up to a real application, adding RAG over your own data, or moving the deployment behind an authenticated load balancer.

If you want to go further with AI on Pulumi, here are some related guides:

Deploy LangServe Apps with Pulumi on AWS (RAG & Chatbot) — Build a retrieval-augmented chatbot that could front-end this Ollama instance.
Deploy AI Models on Amazon SageMaker using Pulumi Python IaC — A SageMaker alternative when you’d rather not manage the EC2 host yourself.
Build an AI Slack Bot on AWS Using Embedchain & Pulumi — Wire an LLM into Slack as an internal assistant.
What is Infrastructure as Code? — Background on the IaC approach used throughout this guide.

Try Pulumi for Free

Changelog

2026-04-30 — Broadened scope from DeepSeek-only to any Ollama-supported model (Llama, Qwen, Mistral). Added TL;DR, instance-type recommendation table, cost-vs-hosted-API comparison, and HowTo structured data. Restructured headings as user questions. Verified Ollama and cloud-init commands against current versions.
2025-03-10 — Minor edits and corrections.
2025-01-27 — Original post: Run DeepSeek-R1 on AWS EC2 Using Ollama.

A Recipe for a Better AI-based Code Generator

Artur Laksberg — Tue, 07 Jan 2025 20:00:00 -0500

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

When asked about his research process, Anthony Bourdain would describe how he’d blend his formal culinary training with deep dives into local food culture - from market stalls to family recipes. Modern AI code generation follows a similar path: it can’t just rely on what it knows - it must tap into continuously evolving, domain-specific knowledge bases. Just as Bourdain would combine his classical French training with techniques learned from local kitchens, AI code generators blend their built-in knowledge with retrieved code snippets and type definitions to generate code that accurately represents the user’s intent.

This fusion of base knowledge with contextual understanding is especially valuable for Infrastructure as Code (IaC), where rapidly evolving cloud providers and libraries make manual development challenging, traditional debugging cycles impractical, and errors catastrophically expensive.

The role of IaC won’t diminish in the age of AI - if anything, it will become even more central as systems grow increasingly complex and automated. Trustworthy code generators will be a key ingredient in the recipe for modern infrastructure management.

In this post, we share how we developed code generation for Pulumi and what we learned, based on both our production IaC generator powering Pulumi AI and Pulumi Copilot, as well as features and approaches we’re still exploring.

Using RAG for code generation

Pulumi supports over 120 providers, including major cloud providers such as AWS, Azure, Google Cloud, and Kubernetes, as well as many other services and platforms. New providers are continuously added, and existing providers change as their capabilities grow.

Our goal is to generate the most accurate code for every provider - code that is not only correct but also reflects their latest capabilities.

LLMs are great at generating code however they are limited to what they have learned before their knowledge cutoff date. This means that the latest changes in the providers will not be reflected in the generated code. This also could lead to hallucinations when the model struggles to answer a question that requires up to date knowledge.

To help us solve this, we rely on the technique known as the Retrieval Augmented Generation (RAG). RAG helps code generation by integrating information retrieved from external data sources. In Pulumi, we call this data source the Registry - it’s the database we maintain that contains type schema and usage information for every provider.

At a high level, using RAG involves the following steps:

Analyze user’s question.
Look up the pertinent information from the Registry.
Format that information in a system prompt that LLM can understand.
Make the LLM call asking it to generate the code using the additional information in the prompt.

There is lot of fascinating details here, so let’s dig in!

The anatomy of Pulumi Copilot RAG

Before going into the details, let’s consider a simple yet essential insight: an effective dataset for RAG must meet two key requirements:

It must contain the necessary information.
The information must be easily searchable.

Let’s take a sample user query:

“Generate code for S3 Bucket”

To fulfil the request, the system needs to intuit the following:

The cloud provider. This can be determined based on the fact that S3 is a common storage solution in AWS, and possibly the fact that the user’s organization has stated AWS as their preferred cloud provider.
The programming language. This information can again come directly from the organizational preferences, or from the user’s prior conversations.
The information about the type (or types) that must be created - its name and schema, the package it is in, and the capabilities it supports. Some of this information comes from the built-in LLM knowledge, supported and augmented by the Registry.

Putting it all together, we can now expand the original user query into the following prompt that is going to guide the code generation:

“Generate TypeScript code for S3 Bucket, the AWS resource defined in package @pulumi/aws, type aws.s3.Bucket with its schema defined as follows: …

While we had to rely on some guesswork to come up with this prompt, fortunately this process can be iterative - if we don’t get all of it right the first time, we can try again with additional information that will help us refine the results. This is an important point we return to later in the post.

Assessing search quality using recall and precision

To assess how effective the RAG is, we need to first understand the two fundamental concepts used in the information retrieval systems: the recall and the precision. Imagine that you’re looking for apple pie recipes in one of Jamie Oliver’s cookbooks. The book has a recipe for a classic American apple pie, a Dutch apple pie and a modern take on a French apple tart. Due to the book’s narrative approach with the recipes woven into the stories and context, you’ve managed to retrieve only the first two recipes but missed the French apple tart. Having retrieved 2 ouf 3 relevant documents, you have achieved a 67% recall.

Because you were looking for the word “pie”, you also retrieved a recipe for a Shepherd’s pie, which, while delicious, does not qualify as an apple pie. Another document that came up was a fish pie - a classic British dish that does not contain apples or even a pastry crust. Since only 2 of your 4 retrieved documents can be legitimately classified as apple pies, you have achieved a 50% precision.

Now let’s formalize this a bit. Recall measures the ratio of the relevant documents retrieved to the total number of relevant documents in RAG dataset:

$$Recall = \frac{N(Retrieved \cap Relevant)}{N(Relevant)}$$

Where

$N(Retrieved \cap Relevant)$ is the number of documents that are both retrieved and relevant.
$N(Relevant)$ is the total number of relevant documents in the database.

Good recall means that many documents relevant to the query were retrieved.

Precision is the ratio of the relevant documents retrieved to the total number of retrieved documents:

$$Precision = \frac{N(Retrieved \cap Relevant)}{N(Retrieved)}$$

Where $N(Retrieved)$ is the total number of documents that were retrieved.

High precision means that many of the retrieved documents were relevant.

Naturally, we want to maximize both the recall and the precision. It’s been said that high recall is essential to ensure relevant content is available to the code generator while precision is the parameter you want to optimize for to avoid hallucinations.

Practical concerns

Precision and recall are essential in understanding the information retrieval quality, but they are quite hard to measure in practice. Unlike a cookbook, Pulumi registry contains thousands of ever changing documents, and evaluating how many of them are relevant for every user-submitted query is impractical. This makes recall evaluation for live traffic next to impossible. Things are a little easier with precision, where we’re dealing with a small number of documents, but even that metric requires a non-trivial evaluation of relevance, which needs an LLM call or a human judge.

Fortunately, other metrics that often can effectively estimate retrieval quality have been developed. We have found a metric that can predict, with some degree of accuracy, whether the generated code will successfully compile. For this metric, we compare the tokens present in the LLM-produced prompt with the number of tokens present in the generated code. (By token here we understand a compiler token - an identifier such as the name of a class, method or a field and not a traditional LLM token concept), Intuitively, if a token present in the prompt also appears in the generated program, it can be assumed that the token contributed to the generated program. Tokens in the generated program that were not part of the prompt are not necessarily wrong but they are less trusted (they can come from the LLM built-in knowledge or were guessed)

$$prompt \ coverage = \frac{N(\text{Tokens in prompt} \cap \text{Tokens in code})}{N(\text{Tokens in code})}$$

Prompt coverage is a metric we can observe in production, and it’s one of several metrics we use when updating providers to ensure we haven’t regressed the quality of the RAG.

Semantic search with vector embeddings

Semantic search is based on the conceptual similarity of the term you’re looking for with the elements in the data store. For example, searching for “dumplings” can return terms like pierogi and gyoza - words with different spelling but both representing different types of filled dough preparations.

A common way to determine the similarity between the two strings is to first turn these strings into vector embeddings - arrays of floating point values representing the semantic meaning of each string - and then calculate the cosine similarity between the two vectors, which is the cosine of the angle between the vectors. Various methods of producing vector embeddings are fascinating but we will not cover them here in depth.

For Pulumi code generation we are using the OpenAI’s Ada-002 embedding model which at this moment represents a good balance between performance and cost.

Producing vector embeddings from the user query is the standard approach in this situation. However, for Pulumi code generator we added a little twist - to increase the odds of getting more relevant information from the Registry (i.e. to increase the recall) we first make an LLM call to generate a small set of relevant search terms that will produce an array of vector embeddings.

Getting vector embeddings from user query

We then use the array of vector embeddings to retrieve the set of relevant documents from the Registry.

Full text vs semantic search

While semantic search is essential for any modern information retrieval system, we should not forget simple and effective methods for text search already exist. The “S3 Bucket” part of the user search happens to be easily searchable using traditional text search operations (such as SQL LIKE operator).

To be effective, our RAG must be able to handle queries that require semantic understanding (such as “simple storage service in AWS”) as well as the traditional text search to support situations where the user knows exactly what they are looking for. To that end, the industry has adopted an approach known as the “hybrid search”, in which the results of full-text search and semantic search are combined to provide the final result.

For each of these search terms we generate a query that combines the full text search with the semantic search. The resulting documents are then evaluated based on the following criteria:

Dense score: Vector similarity using cosine distance between query embedding and stored embeddings
Sparse score: Text search relevance according to the full-using algorithm (PostgreSQL full-text search in our case)

Both the dense and the sparse scores contribute to the final score that is used to sort the documents. Even though the final score is the average of the two in our current implementation, it would not be correct to say that they have the same weight, or the influence on the end result. This is so because the normalization of the scores must take into account their distribution in the real-world queries and calculating that is quite complex.

Boosting the influence of one score relative to the other is an area we’re actively exploring, and achieving the optimal result depends highly on what parameters we decide to optimize for - high quality code, time to result and so on.

Finally, we apply the rank-based scoring technique in which the results are penalized based on their position in the ranking. This creates separation between results that might have similar initial scores and gives preference to results that appear earlier in the ranking.

Pruning the results

Our Pulumi code generator employs a two-phase document selection strategy. The first phase casts a wide net, gathering all potentially relevant documentation to maximize recall. The second phase applies precision-focused filtering to distill this collection down to the most pertinent documents.

This filtering step serves two purposes. First, it prevents LLM hallucinations that arise from similarly-named types across different providers. Second, it optimizes performance by keeping prompts concise - a critical consideration given that larger prompts increase both latency and computational costs, even when within context window constraints.

Through empirical testing with the Pulumi Registry search, we’ve established these baseline parameters: a maximum of 10 documents per query term selected by relevance score, and a 20K token ceiling for prompts. While these parameters have yielded good results in practice, they are likely not optimal for all scenarios. We continue to iterate on these values through ongoing experimentation.

Prompt generation

We’re now ready to create the system prompt for the code generation LLM call! We’ve already discussed some of the elements needed to build the effective prompt - the original user query, the search terms and the vector embeddings that produce the set of documents that will guide the code generation process.

There is another element that goes into the prompt - a concise set of instructions produced by an LLM call based on the original user’s query. The approach when the output of one prompt is used as input for another is known as “prompt chaining”, and we used it to provide step-by-step instructions to the code generator. For our query, this set of instructions can look as follows:

Create an S3 bucket using Pulumi in TypeScript using the following steps:

Import the necessary Pulumi AWS package.

Define a new S3 bucket resource with basic configuration.

Export the bucket name as an output.

Composition of the system prompt for code generation

Finally, we use the resulting prompt to call the LLM and ask it to generate the code. We’re done!

Self-debugging

Unfortunately, this isn’t always the final step of the process. Despite our best efforts, the code produced by the LLM will not always be correct.

At this point, it’s worth pondering what “correct code” means. The generated program might have the following problems:

It might be syntactically or semantically incorrect, i.e. it might not compile or fail to typecheck.
It might fail at runtime - for example by referring to a non-existing resource, a region and so on.
It might run “successfully” but not do what the user intended - either because the user did not express themselves clearly or because the request was misunderstood.
Finally, the code might run and do exactly what the user wants, but lead to an undesired outcome, for example loss of an asset, or a security vulnerability.

Solutions to many of these problems go well beyond the domain of code generation.

However, the first of these problems can be addressed by the approach known as “self-debugging”.

We observed that many generated TypeScript programs that fail to typecheck contain only a few errors, and asking the LLM to fix these errors often produces a valid program. While this approach is still experimental, we see promising results where getting to a correct program requires only 1-2 iterations of self-debugging. The biggest challenge with this approach is doing it in real-time. The user is staring at the screen waiting for an answer, so latency is a major concern.

Monitoring these typechecking errors in production can also provide valuable insight into the quality of the data used by the RAG and even suggest specific solutions. For example, failure to typecheck a member-access expression is a likely indicator of a missing type schema (a recall problem) or a “wrong” schema brought in by an irrelevant document (a precision problem).

Self-debugging can also be extended to include the pulumi preview command, which is a “dry run” operation before the actual deployment and can detect many real or potential problems such as destructive actions, incorrect configurations that cannot be detected at compile time, dependency conflicts, and policy violations.

From kitchen to table: testing what works

The landscape of LLM-based code generation is moving fast, and we need to keep learning and adapting as we go. But with all this rapid technological change, it’s crucial to ground our decisions in real numbers. We need to make sure each new advancement actually makes things better, both in our test environments and out in the real world.

The probabilistic nature of LLM-based code generation means we can’t rely solely on pre-production testing. Instead, we adopt multiple layers of quality control working together. Here’s what we’ve learned works best:

Building confidence through testing

Our testing strategy combines several approaches:

Systematic experimentation: We run “what-if” scenarios against various test datasets to understand how the system behaves under different conditions.
Local evaluation pipeline: While local testing has its limitations, it helps catch obvious issues early in development. We evaluate the performance of the code generator and run the code through the typechecker.
Deterministic generation: We set the LLM temperature to 0 to ensure consistent outputs. For code generation, consistency and repeatability matters more than creative variations.

Monitoring quality in production

We track several key metrics to ensure the system performs well:

RAG effectiveness: We measure prompt coverage to evaluate how well our retrieval system performs.
Code quality indicators: We track the percentage of generated code that successfully typechecks for all supported languages along with how this metric responds to system changes.
User feedback: Every “thumbs down” report gets analyzed to identify patterns and potential improvements. This direct user feedback has been invaluable in refining our system.

Wrapping up

Building an effective AI-powered code generator requires carefully balancing multiple concerns: the raw capabilities of LLMs with retrieved contextual knowledge, semantic search with traditional text matching, and thorough testing with real-world scenarios.

Our experience has taught us that success lies in treating code generation like a delicate soufflé - it requires precise measurements, careful monitoring, and constant refinement of technique. The key ingredients are:

A robust RAG system with well-tuned recall and precision.
End-to-end testing and monitoring across development and production.
Self-debugging capabilities to handle common issues like type checking errors.

As we continue to evolve Pulumi’s code generation capabilities, we’re excited about expanding our self-debugging features and further refining our RAG implementation.

We invite you to try these capabilities in your own projects and share your experiences. Your feedback helps us continue improving and advancing the state of AI-assisted infrastructure development.