Pulumi Blog: Tutorials

Fully Automated AI Inference on AWS, Azure, and Google Cloud with Pulumi

Engin Diri — Tue, 30 Jun 2026 00:00:00 +0000

Putting Ollama on a cloud GPU is something I keep coming back to. A while ago I wrote up running open-source LLMs on an AWS EC2 box with Ollama and Pulumi, and the shape never really changes: a GPU instance, a model server, and a firewall rule in front. Infrastructure as code earned its place by making that kind of setup predictable and repeatable, and AI infrastructure is no exception. A GPU box serving a model is still a VM, a disk, and a firewall rule, and it should be declared like one.

Thorsten Hans made exactly that case in his Akamai post, Fully Automated AI Infrastructures with Terraform and Akamai Cloud, which stands up a single GPU instance on Linode, installs the drivers, runs Ollama, and pulls a model, with no manual steps after terraform apply.

I liked the shape of it, so this post ports the same idea to Pulumi and runs it across AWS, Azure, and Google Cloud instead of one. The result is one program shape per cloud: a single pulumi up brings up a GPU box that installs its own driver, runs Ollama, and pulls a model with no manual steps, and a single pulumi destroy takes it back down. Along the way it drops the two imperative bits the Terraform version leans on: a static access token sitting in an environment variable, and a null_resource running a shell loop to wait for the model. The first becomes an OIDC login from a Pulumi ESC environment, so no long-lived key lives anywhere. The second turns out not to be a resource at all.

What you are building

Strip away the per-cloud naming and every version of this is the same three things: a GPU virtual machine, a firewall in front of it, and a cloud-init script that turns a bare Ubuntu box into a running inference server. The model serving runs on Ollama, which exposes an HTTP API on port 11434 and keeps the model resident in GPU memory between requests.

flowchart LR
Dev([Your machine / curl]) -->|"HTTP :11434"| FW["Firewall / security group<br/>(allow 11434, optional 22)"]
FW --> VM["GPU VM (Ubuntu 24.04)<br/>NVIDIA driver + Ollama"]
VM -->|"resident in GPU memory"| Model["qwen2.5:14b"]
ESC["Pulumi ESC<br/>pulumi-idp/auth"] -.->|"OIDC login (short-lived creds)"| Pulumi["pulumi up"]
Pulumi -.->|"AWS / Azure / GCP"| VM

Component	Port	Description
GPU VM	-	Ubuntu 24.04 with an NVIDIA T4-class GPU. Installs the driver and Ollama from cloud-init, then pulls the model.
Ollama	11434	Serves the model over an HTTP API and keeps it in GPU memory between calls.
Firewall / security group	-	Allows inbound `11434` (and `22` when you ask for it). Open to the world for a demo; lock the CIDR down for anything real.
`pulumi-idp/auth` (ESC)	-	One environment that brokers an OIDC login into AWS, Azure, and GCP. No static keys in code or CI.

One detail here is worth pausing on, and it explains why the Pulumi version comes out shorter than the Terraform one. The Akamai project ends with a null_resource that runs a curl loop in local-exec to block terraform apply until the model finishes downloading. That is not infrastructure but a runtime check wearing a resource costume. Pulumi has a Command provider that would let you reproduce it line for line, and this post deliberately does not. The program provisions the box, the box pulls the model on its own, and the program prints the endpoint. Whether the model has finished downloading yet is a question you answer with a curl, not a question your IaC tool should be holding a deployment open to ask.

Prerequisites

Before getting started, ensure you have:

Pulumi CLI installed and configured
A Pulumi Cloud account (ESC and OIDC live here)
An account on at least one of AWS, Azure, or Google Cloud, with an OIDC trust set up for Pulumi (covered below)
Enough GPU quota in your target region for one T4-class instance; a fresh account often starts at zero, which is the most common reason a first deploy fails, so check it before you run pulumi up
Node.js 18+ for the TypeScript program
curl to talk to the inference endpoint once it comes up

This post serves the qwen2.5:14b model, the same one the Akamai article uses, because the 4-bit quant fits comfortably on a single 16 GB T4. The model is one config value (model), so swap in anything Ollama supports; match the GPU to the model’s memory footprint, because a model that overflows VRAM still runs but spills onto the CPU and crawls.

Credentials without the copy-paste

The Terraform version authenticates the only way a single-cloud demo can: you mint a personal access token, export it as LINODE_TOKEN, and the provider reads it from the environment. It works, but that token is long-lived, it sits in your shell history and your CI secrets, and you mint one per cloud. Across three clouds that is three static keys to rotate and worry about.

Pulumi ESC (Environments, Secrets, and Configuration) replaces it all with an OIDC login. The idea: instead of storing a cloud key, you store a trust relationship. At deploy time, ESC presents a short-lived OIDC token to AWS, Azure, or Google Cloud, and each one hands back temporary credentials scoped to a role you control. Nothing long-lived is ever written down.

I keep that wiring in one environment, pulumi-idp/auth, and every stack imports it. Here is the whole thing:

# pulumi-idp/auth: one ESC environment that brokers an OIDC login into all three
# clouds. Every stack imports it. No static cloud key lives here or anywhere else.
values:
 aws:
 login:
 fn::open::aws-login:
 oidc:
 roleArn: arn:aws:iam::123456789012:role/pulumi-esc
 sessionName: pulumi-esc
 azure:
 login:
 fn::open::azure-login:
 clientId: aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
 tenantId: aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
 subscriptionId: /subscriptions/00000000-0000-0000-0000-000000000000
 oidc: true
 gcp:
 login:
 fn::open::gcp-login:
 project: 123456789012 # numeric project number, not the project ID
 oidc:
 workloadPoolId: pulumi-esc
 providerId: pulumi-esc
 serviceAccount: pulumi-esc@my-project.iam.gserviceaccount.com
 environmentVariables:
 # AWS: read by the Pulumi AWS provider, the AWS SDKs, and the aws CLI
 AWS_ACCESS_KEY_ID: ${aws.login.accessKeyId}
 AWS_SECRET_ACCESS_KEY: ${aws.login.secretAccessKey}
 AWS_SESSION_TOKEN: ${aws.login.sessionToken}
 # Azure: read by the azure-native provider
 ARM_USE_OIDC: "true"
 ARM_CLIENT_ID: ${azure.login.clientId}
 ARM_TENANT_ID: ${azure.login.tenantId}
 ARM_SUBSCRIPTION_ID: ${azure.login.subscriptionId}
 ARM_OIDC_TOKEN: ${azure.login.oidc.token}
 # Google Cloud: read by the Pulumi Google Cloud provider
 GOOGLE_CLOUD_PROJECT: ${gcp.login.project}
 GOOGLE_OAUTH_ACCESS_TOKEN: ${gcp.login.accessToken}

You only need the blocks for the clouds you actually deploy to. Opening this environment makes ESC perform a live OIDC login for each cloud listed, so trim it to the one you use, or keep all three if, like me, you bounce between them.

Each cloud needs a one-time trust setup so this login is allowed at all: an IAM OIDC identity provider and role on AWS, a federated credential on an Azure app registration, and a Workload Identity Pool on Google Cloud. You do that once; after that, pulumi-idp/auth is the only thing any stack references for credentials.

A stack opts into it with one block in its stack config. The AWS stack’s Pulumi.aws.yaml:

# Pulumi.aws.yaml
environment:
 - pulumi-idp/auth

That is the entire credential story. No keys in the program, no keys in CI, nothing to rotate. The program below never mentions a secret; it only creates resources, and the ambient credentials from pulumi-idp/auth carry the request.

One cloud-init, three clouds

The part that turns a bare Ubuntu box into an inference server is identical on every cloud, so it lives in one file, cloud-init.yaml, that all three programs read. It follows the Akamai cloud-config closely, with a couple of robustness tweaks for a multi-cloud run, and it runs in a deliberate order because the GPU driver needs a reboot before Ollama can see the card:

Update packages and install the kernel headers and ubuntu-drivers.
Install the NVIDIA driver, then reboot so the kernel loads it.
On the next boot, a one-shot systemd service installs Ollama, binds it to 0.0.0.0:11434, and pulls the model.
The service disables itself, so it never runs again.

#cloud-config
# Zero-touch Ollama GPU box. Cloud-agnostic: no provider metadata, no static creds.
# The Pulumi program substitutes the model name on the `ollama pull` line below
# before this is passed as user-data (AWS/GCP) or custom-data (Azure).

write_files:
 # Make Ollama listen on every interface and never unload the model from VRAM.
 - path: /etc/systemd/system/ollama.service.d/override.conf
 content: |
 [Service]
 Environment="OLLAMA_HOST=0.0.0.0:11434"
 Environment="OLLAMA_KEEP_ALIVE=-1"

 # Runs once on the post-reboot boot, after the GPU driver is loaded.
 - path: /usr/local/bin/ollama-setup.sh
 permissions: '0755'
 content: |
 #!/usr/bin/env bash
 set -euxo pipefail

 # Install Ollama; the installer creates and starts the ollama systemd unit.
 curl -fsSL https://ollama.com/install.sh | sh

 # Pick up the OLLAMA_HOST / OLLAMA_KEEP_ALIVE override written above.
 systemctl daemon-reload
 systemctl enable ollama.service
 systemctl restart ollama.service

 # Wait for the daemon to bind its socket before pulling. This is the box
 # waiting on its OWN local daemon, not an external readiness gate.
 until curl -fsS http://127.0.0.1:11434/api/tags >/dev/null 2>&1; do
 sleep 2
 done

 # Pre-pull the model so the endpoint answers on the very first request.
 ollama pull __MODEL__

 # One-shot: never run again on future boots.
 systemctl disable ollama-setup.service

 - path: /etc/systemd/system/ollama-setup.service
 content: |
 [Unit]
 Description=One-time Ollama install and model pull
 After=network-online.target
 Wants=network-online.target

 [Service]
 Type=oneshot
 ExecStart=/usr/local/bin/ollama-setup.sh
 RemainAfterExit=true
 # The model pull runs for several minutes; without this, systemd's default
 # 90s start timeout kills the service mid-download and the model never lands.
 TimeoutStartSec=0

 [Install]
 WantedBy=multi-user.target

runcmd:
 - apt-get update
 - DEBIAN_FRONTEND=noninteractive apt-get -y upgrade
 # linux-headers-generic also covers the kernel the upgrade above may have pulled
 # in, so DKMS builds the NVIDIA module for the kernel that boots next.
 - DEBIAN_FRONTEND=noninteractive apt-get install -y "linux-headers-$(uname -r)" linux-headers-generic ubuntu-drivers-common
 # --gpgpu selects the headless server driver branch, the right one for a compute
 # GPU like the T4; bare `ubuntu-drivers install` would pull the desktop stack.
 - ubuntu-drivers install --gpgpu
 - systemctl enable ollama-setup.service

power_state:
 mode: reboot
 message: Rebooting to load the NVIDIA driver before Ollama setup
 condition: true

The model name is the literal token __MODEL__. The Pulumi program reads this file at deploy time and substitutes your model config value before handing it to the instance. Two environment settings are doing quiet but important work: OLLAMA_HOST=0.0.0.0:11434 makes Ollama listen on every interface instead of localhost alone, and OLLAMA_KEEP_ALIVE=-1 keeps the model pinned in GPU memory so only the first request pays the load cost.

The programs

Every program has the same five beats: read the config, template the cloud-init.yaml, open the inbound ports, launch the GPU instance with that cloud-init as its user data, and export the endpoint. What differs is only the dialect each cloud speaks for “GPU instance” and “firewall rule.”

The shared contract keeps the three listings legible side by side: the same model and allowSsh config keys, the same cloud-init.yaml, and the same four exports (publicIp, ollamaEndpoint, generateEndpoint, tagsEndpoint). Pick your cloud:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as fs from "fs";

const cfg = new pulumi.Config();
const model = cfg.get("model") ?? "qwen2.5:14b";
const allowSsh = cfg.getBoolean("allowSsh") ?? false;
const region = (cfg.get("region") ?? "us-east-1") as aws.Region;

// Inject the model name into cloud-init at deploy time (a file read, not a shell-out).
const userData = fs.readFileSync("cloud-init.yaml", "utf8").replace(/__MODEL__/g, model);

// Region comes from config; credentials arrive ambiently from pulumi-idp/auth.
const provider = new aws.Provider("aws", { region });

// Most recent Ubuntu 24.04 LTS (amd64, hvm, gp3) published by Canonical.
const ubuntu = aws.ec2.getAmi({
 mostRecent: true,
 owners: ["099720109477"],
 filters: [
 { name: "name", values: ["ubuntu/images/hvm-ssd-gp3/ubuntu-noble-24.04-amd64-server-*"] },
 { name: "virtualization-type", values: ["hvm"] },
 ],
}, { provider });

const ingress: aws.types.input.ec2.SecurityGroupIngress[] = [{
 description: "Ollama HTTP API. NOTE: restrict cidrBlocks to your own range in production.",
 fromPort: 11434,
 toPort: 11434,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
}];

if (allowSsh) {
 ingress.push({
 description: "SSH",
 fromPort: 22,
 toPort: 22,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 });
}

const sg = new aws.ec2.SecurityGroup("ollama", {
 description: "Ollama inferencing access",
 ingress,
 egress: [{
 description: "Allow all outbound",
 fromPort: 0,
 toPort: 0,
 protocol: "-1",
 cidrBlocks: ["0.0.0.0/0"],
 }],
}, {
 provider,
});

const server = new aws.ec2.Instance("ollama", {
 ami: ubuntu.then(a => a.id),
 instanceType: "g4dn.xlarge", // 1x NVIDIA T4
 vpcSecurityGroupIds: [sg.id],
 associatePublicIpAddress: true,
 userData, // plain text; the AWS provider base64-encodes it for you
 rootBlockDevice: {
 volumeSize: 40,
 volumeType: "gp3",
 },
 tags: { Name: "ollama" },
}, {
 provider,
});

export const publicIp = server.publicIp;
export const ollamaEndpoint = pulumi.interpolate`http://${publicIp}:11434`;
export const generateEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/generate`;
export const tagsEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/tags`;

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";
import * as fs from "fs";

const cfg = new pulumi.Config();
const model = cfg.get("model") ?? "qwen2.5:14b";
const allowSsh = cfg.getBoolean("allowSsh") ?? false;
const project = cfg.get("project");
const zone = cfg.get("zone") ?? "us-central1-a";

// Inject the model name into cloud-init at deploy time (a file read, not a shell-out).
const userData = fs.readFileSync("cloud-init.yaml", "utf8").replace(/__MODEL__/g, model);

// Latest Ubuntu 24.04 LTS amd64 image.
const ubuntu = gcp.compute.getImage({
 family: "ubuntu-2404-lts-amd64",
 project: "ubuntu-os-cloud",
});

// Network tag binds the firewall rule to this instance.
const networkTag = "ollama";

const firewall = new gcp.compute.Firewall("ollama-fw", {
 network: "default",
 project: project,
 // Inbound to Ollama from anywhere; restrict this CIDR for production.
 allows: allowSsh
 ? [{ protocol: "tcp", ports: ["11434"] }, { protocol: "tcp", ports: ["22"] }]
 : [{ protocol: "tcp", ports: ["11434"] }],
 sourceRanges: ["0.0.0.0/0"],
 targetTags: [networkTag],
});

const instance = new gcp.compute.Instance("ollama", {
 machineType: "n1-standard-4",
 zone: zone,
 project: project,
 tags: [networkTag],
 bootDisk: {
 initializeParams: {
 image: ubuntu.then(i => i.selfLink),
 size: 40,
 },
 },
 guestAccelerators: [{
 type: "nvidia-tesla-t4",
 count: 1,
 }],
 // GPUs cannot live-migrate, so host maintenance must terminate (and restart) the VM.
 scheduling: {
 onHostMaintenance: "TERMINATE",
 automaticRestart: true,
 },
 networkInterfaces: [{
 network: "default",
 accessConfigs: [{}], // an empty config requests an ephemeral public IP
 }],
 metadata: {
 "user-data": userData, // Ubuntu cloud-init reads this key, not startup-script
 },
}, {
 dependsOn: firewall,
});

export const publicIp = instance.networkInterfaces.apply(nics => nics[0].accessConfigs![0].natIp);
export const ollamaEndpoint = pulumi.interpolate`http://${publicIp}:11434`;
export const generateEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/generate`;
export const tagsEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/tags`;

import * as pulumi from "@pulumi/pulumi";
import * as resources from "@pulumi/azure-native/resources";
import * as network from "@pulumi/azure-native/network";
import * as compute from "@pulumi/azure-native/compute";
import * as random from "@pulumi/random";
import * as fs from "fs";

const cfg = new pulumi.Config();
const model = cfg.get("model") ?? "qwen2.5:14b";
const allowSsh = cfg.getBoolean("allowSsh") ?? false;
const location = cfg.get("location") ?? "eastus";

// Inject the model name into cloud-init at deploy time (a file read, not a shell-out).
const userData = fs.readFileSync("cloud-init.yaml", "utf8").replace(/__MODEL__/g, model);

const resourceGroup = new resources.ResourceGroup("ollama-rg", {
 location,
});

const vnet = new network.VirtualNetwork("ollama-vnet", {
 resourceGroupName: resourceGroup.name,
 addressSpace: { addressPrefixes: ["10.0.0.0/16"] },
});

const subnet = new network.Subnet("ollama-subnet", {
 resourceGroupName: resourceGroup.name,
 virtualNetworkName: vnet.name,
 addressPrefix: "10.0.1.0/24",
});

// Standard SKU public IPs use static allocation.
const publicIpAddress = new network.PublicIPAddress("ollama-pip", {
 resourceGroupName: resourceGroup.name,
 sku: { name: network.PublicIPAddressSkuName.Standard },
 publicIPAllocationMethod: network.IPAllocationMethod.Static,
});

// Azure permits all outbound by default, so only inbound rules are needed.
const nsg = new network.NetworkSecurityGroup("ollama-nsg", {
 resourceGroupName: resourceGroup.name,
 securityRules: [
 {
 name: "allow-ollama",
 priority: 1000,
 direction: network.SecurityRuleDirection.Inbound,
 access: network.SecurityRuleAccess.Allow,
 protocol: network.SecurityRuleProtocol.Tcp,
 sourcePortRange: "*",
 destinationPortRange: "11434",
 sourceAddressPrefix: "0.0.0.0/0", // prod: restrict to your client CIDR
 destinationAddressPrefix: "*",
 },
 ...(allowSsh ? [{
 name: "allow-ssh",
 priority: 1001,
 direction: network.SecurityRuleDirection.Inbound,
 access: network.SecurityRuleAccess.Allow,
 protocol: network.SecurityRuleProtocol.Tcp,
 sourcePortRange: "*",
 destinationPortRange: "22",
 sourceAddressPrefix: "0.0.0.0/0",
 destinationAddressPrefix: "*",
 }] : []),
 ],
});

const nic = new network.NetworkInterface("ollama-nic", {
 resourceGroupName: resourceGroup.name,
 networkSecurityGroup: { id: nsg.id },
 ipConfigurations: [{
 name: "ipconfig1",
 subnet: { id: subnet.id },
 publicIPAddress: { id: publicIpAddress.id },
 privateIPAllocationMethod: network.IPAllocationMethod.Dynamic,
 primary: true,
 }],
});

// Azure requires an admin credential even though we never log in. Generate one
// instead of hard-coding it; it stays a Pulumi secret and is never exported.
const adminPassword = new random.RandomPassword("ollama-admin-password", {
 length: 24,
 special: true,
 overrideSpecial: "!#$%*",
 minLower: 1,
 minUpper: 1,
 minNumeric: 1,
 minSpecial: 1, // Azure requires 3 of 4 character classes
});

const vm = new compute.VirtualMachine("ollama-vm", {
 resourceGroupName: resourceGroup.name,
 hardwareProfile: { vmSize: "Standard_NC4as_T4_v3" }, // 1x NVIDIA T4
 networkProfile: {
 networkInterfaces: [{ id: nic.id, primary: true }],
 },
 osProfile: {
 computerName: "ollama",
 adminUsername: "azureuser",
 adminPassword: adminPassword.result,
 customData: Buffer.from(userData).toString("base64"), // Azure wants base64
 linuxConfiguration: {
 disablePasswordAuthentication: false,
 },
 },
 storageProfile: {
 imageReference: {
 publisher: "Canonical",
 offer: "ubuntu-24_04-lts",
 sku: "server", // Gen2; NC4as_T4_v3 is a Gen2 size
 version: "latest",
 },
 osDisk: {
 name: "ollama-osdisk",
 createOption: "FromImage",
 diskSizeGB: 40,
 managedDisk: { storageAccountType: "StandardSSD_LRS" },
 },
 },
});

// A Standard/Static public IP is reserved at creation, so its address is known
// once the resource exists; no separate lookup needed.
export const publicIp = publicIpAddress.ipAddress.apply(ip => ip!);
export const ollamaEndpoint = pulumi.interpolate`http://${publicIp}:11434`;
export const generateEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/generate`;
export const tagsEndpoint = pulumi.interpolate`http://${publicIp}:11434/api/tags`;

A few per-cloud details are worth calling out, since they are the places the “same program” abstraction leaks:

AWS is the shortest program, because the GPU comes with the instance shape: a g4dn.xlarge is a T4 box, so there is no separate accelerator to attach. The one thing to do before you deploy is raise the Running On-Demand G and VT instances vCPU quota in your region; a fresh account starts at zero, and pulumi up fails with VcpuLimitExceeded until you do.
Google Cloud attaches the GPU explicitly with guestAccelerators, and that brings the rule that trips people up: a GPU instance cannot live-migrate, so scheduling.onHostMaintenance must be "TERMINATE" or the apply is rejected. The cloud-init also has to ride on the user-data metadata key, not startup-script, and the empty accessConfigs: [{}] is what hands the box a public IP. T4 quota is also zero on a new project.
Azure is the longest listing, because the network is à la carte: the resource group, virtual network, subnet, public IP, security group, and NIC are each their own resource before you reach the VM. One detail surprises people: a Linux VM requires an admin credential even when you never log in, so the program generates a throwaway password with random.RandomPassword rather than committing one. The NC4as_T4_v3 is a compute GPU, so the standard server (CUDA) driver from the cloud-init is the right choice here; the GRID driver is only needed for GPU-accelerated visualization workloads.

The full programs, all three Pulumi.<cloud>.yaml files, and the shared cloud-init.yaml are in the companion repo:

github.com/dirien/fully-automated-ai-inference-pulumi

Deploying

Create a project, install the provider for your cloud, point the stack at pulumi-idp/auth, and deploy. For AWS:

mkdir ai-inference && cd ai-inference
pulumi new typescript
npm install @pulumi/aws

Drop the AWS listing into index.ts, copy the cloud-init.yaml shown earlier into the same directory (or grab it from the companion repo), point the stack at the auth environment, and set your region:

pulumi stack init aws
# add `environment: [pulumi-idp/auth]` to Pulumi.aws.yaml (shown above)
pulumi config set region us-east-1
pulumi config set model qwen2.5:14b
pulumi up

The other two clouds are the same flow with a different provider package and a couple of config keys: npm install @pulumi/gcp with pulumi config set project <your-project-id> and pulumi config set zone us-central1-a, or npm install @pulumi/azure-native @pulumi/random with pulumi config set location eastus.

The slow part is the GPU box: it boots, installs the driver, reboots, installs Ollama, and pulls the model, all without you. Depending on the instance, the region, and the model size, plan on roughly 10 to 15 minutes before the endpoint answers. When pulumi up finishes you get the endpoints straight back as stack outputs:

Outputs:
 generateEndpoint: "http://<public-ip>:11434/api/generate"
 ollamaEndpoint : "http://<public-ip>:11434"
 publicIp : "<public-ip>"
 tagsEndpoint : "http://<public-ip>:11434/api/tags"

Testing the inference server

pulumi up returns as soon as the infrastructure exists, which is before the model has finished downloading. This is exactly where Terraform reached for that null_resource loop, and where Pulumi hands you a URL instead. To check whether the model is ready, ask Ollama what it has loaded:

curl -s $(pulumi stack output tagsEndpoint) | grep -q "qwen2.5:14b" && echo "ready" || echo "still pulling"

Once it reports ready, send it a prompt. This is the same request the Akamai post makes, pointed at your generateEndpoint output:

curl -s $(pulumi stack output generateEndpoint) -d '{
 "model": "qwen2.5:14b",
 "system": "Answer every question with a three-line poem.",
 "prompt": "Why is the sky blue?",
 "stream": false
}'

The first call is slower, because Ollama loads the model into GPU memory before it answers. Every call after that is fast, since OLLAMA_KEEP_ALIVE=-1 keeps it resident:

{
 "model": "qwen2.5:14b",
 "response": "Sunlight scatters, short waves fly,\nBlue light paints the open sky,\nViolet fades as day drifts by.",
 "done": true
}

When you are done, take it all down with one command, so the GPU stops billing:

pulumi destroy

Cost

A GPU instance is the whole bill, and you pay by the hour whether the model is busy or idle. The numbers below are rough on-demand rates for a single T4-class box left running 24/7; in practice you spin it up, use it, and pulumi destroy it, so what you actually pay tracks the hours it stays up.

Cloud	Instance	GPU	~Hourly	~Monthly (24/7)
AWS	`g4dn.xlarge`	1× T4 (16 GB)	~$0.53	~$384
Azure	`Standard_NC4as_T4_v3`	1× T4 (16 GB)	~$0.53	~$384
Google Cloud	`n1-standard-4` + 1× T4	1× T4 (16 GB)	~$0.54	~$394

These are on-demand Linux rates in a cheap US region (us-east-1, eastus, us-central1) as of mid-2026, and the GPU dominates the bill. Spot or preemptible capacity cuts it sharply if your workload tolerates interruption (often 60 to 70 percent off), Google Cloud’s sustained-use discount trims an always-on instance on its own, and the 40 GB disk adds only a few dollars a month.

A GPU box left running overnight is the expensive mistake here. Because the whole stack is declarative, the safe habit is cheap: pulumi destroy when you stop using it, pulumi up when you need it back. The state and config are version-controlled, so standing it up again is one command, not a rebuild.

Security considerations

The architecture in this post matches the Akamai original on purpose, which means it carries the same caveat: Ollama is exposed over plain HTTP on a port open to the entire internet. That is fine for a demo on a box you tear down the same day. It is not fine for anything that outlives the afternoon, and self-hosted AI endpoints get found fast: scanners like Shodan enumerate a freshly exposed one within hours, and an open Ollama instance is free compute for whoever finds it first.

The credential side, though, is genuinely better than a static-token setup, and that is the part worth keeping. There is no long-lived cloud key in the program, in your shell, or in CI; every deploy gets short-lived credentials minted through pulumi-idp/auth and thrown away when it finishes.

Concern	Akamai/Terraform demo	This deployment
Cloud credentials	Static PAT in an env var	OIDC login, short-lived, via `pulumi-idp/auth`
Inbound exposure	Port 11434 open to `0.0.0.0/0`	Same by default; one config flag scopes the CIDR
Transport	Plain HTTP	Plain HTTP (front with TLS for real use)
Readiness wait	`null_resource` + `local-exec` shell loop	A runtime `curl`, no resource

My recommendations for taking this past a demo:

Scope the inbound rule to your own CIDR instead of 0.0.0.0/0; the allowSsh flag in the program is the pattern to copy for the Ollama port
Put a reverse proxy with TLS in front of Ollama, or keep the box off the public internet entirely and reach it over a private network or a Tailscale tailnet, the way I did for the Hermes agent
Keep credentials on OIDC through ESC; never fall back to a static key to “just get it working”
Treat the box as disposable, and pulumi destroy it when you are not using it, which shrinks both the bill and the attack window

What’s next?

The program is a foundation, and the interesting work is what you layer on once a model is a pulumi up away:

Lock it down. Scope the firewall, add TLS, or move the box behind a private network so the endpoint is not on the public internet at all.
Scale the model to the GPU. qwen2.5:14b on a T4 is the entry point. Bigger models want more memory, which means an A10G, an L4, or an A100, and the only change in the program is the instance type and the model value.
Swap the serving engine. Ollama is the simplest thing that works. For higher throughput, the same shape holds with vLLM in place of Ollama in the cloud-init.
Generate the next one. Pulumi Neo can take a target like “a GPU inference box on Azure behind a private endpoint” and produce a first draft of the program, which you then review in a PR like any other change.

Conclusion

Declaring an AI inference server as code buys you the same thing it buys for any other infrastructure: you can reproduce it on any of the three clouds, version-control it, and tear it down with a single pulumi destroy. Porting the Akamai project made the contrast sharp. The credentials moved from a static token to an OIDC login that leaves nothing behind, and the readiness wait disappeared entirely once you stop treating a runtime check as a resource.

That is the line worth carrying to the next thing you deploy. Not every step in a runbook is a resource. A GPU box and a firewall rule are; waiting for a download to finish is not. Draw that line first, and the program gets shorter on its own.

If you run into issues or have questions, drop by the Pulumi Community Slack or GitHub Discussions. New to Pulumi? Get started here.

Build a Governed Databricks Workspace with Pulumi

Pablo Seibelt — Thu, 25 Jun 2026 00:00:00 +0000

Platform teams responsible for Databricks often find themselves manually configuring clusters and notebooks for every new data science team. This manual overhead leads to inconsistent cluster policies, runaway costs from oversized instances, and security gaps in workspace operations. Without a standardized way to provision workspace-local resources, data platforms become a fragmented collection of bespoke environments that are impossible to govern at scale.

As Databricks usage grows across the enterprise, the lack of a governed workspace baseline becomes a major operational risk. Inconsistent policies lead to unpredictable billing and audit failures. Standardizing your Databricks environment with Pulumi ensures that every workspace starts with the correct cost controls, cluster policies, notebooks, and automated jobs, allowing your data teams to move faster without compromising governance.

What you’ll build

In this post, you will learn how to provision a governed Databricks workspace baseline using Pulumi. You will build:

Cluster policies to enforce instance types and cost controls.
Workspace notebooks to standardize workload locations.
Secret scopes and permissions to make access reviewable.
Automated jobs that run on policy-constrained compute.

By the end, you will have a reproducible workspace configuration that you can deploy to any new Databricks environment.

The Databricks management boundary

When managing Databricks with Pulumi, understand the boundary between account-level and workspace-level resources.

Account-level: Creating cloud prerequisites such as VPCs, subnets, IAM roles, and Azure resource groups uses the cloud provider for your target platform. On AWS and GCP, Databricks workspace creation and account-level identity use the Databricks provider’s account APIs, such as databricks.MwsWorkspaces; on Azure, the workspace itself is commonly managed with azure-native.
Workspace-level: Managing clusters, jobs, notebooks, permissions, and secret scopes within a specific workspace. This example uses the @pulumi/databricks provider against an Azure Databricks workspace URL, and the same workspace-resource pattern applies once your provider is configured for the target workspace.

For this tutorial, we’ll assume you have an existing Databricks workspace and want to manage the resources within it.

Configuring credentials with Pulumi ESC

Before writing code, we need to configure our credentials. Pulumi ESC (Environments, Secrets, and Configuration) manages these securely.

You can create an environment that maps Databricks credentials into Pulumi configuration for the Databricks provider:

values:
 databricks:
 host: "https://adb-123456789.0.azuredatabricks.net"
 token:
 fn::secret: databricks-token
 pulumiConfig:
 databricks:host: ${databricks.host}
 databricks:token: ${databricks.token}

Once configured, import this environment into your Pulumi stack. The Databricks provider reads configuration values such as databricks:host and databricks:token without requiring static secrets in stack config.

Defining your data infrastructure

The following Pulumi program in TypeScript sets up separate cluster policies for interactive and job compute, a governed shared cluster, a secret scope, explicit permissions, a notebook, and a scheduled job that runs on policy-constrained compute.

import * as databricks from "@pulumi/databricks";

const interactivePolicy = new databricks.ClusterPolicy("interactive-fair-use", {
 name: "Interactive Fair Use Policy",
 definition: JSON.stringify({
 "dbus_per_hour": {
 "type": "range",
 "maxValue": 10,
 },
 "autotermination_minutes": {
 "type": "fixed",
 "value": 20,
 "hidden": true,
 },
 }),
});

const jobPolicy = new databricks.ClusterPolicy("job-fair-use", {
 name: "Job Fair Use Policy",
 definition: JSON.stringify({
 "dbus_per_hour": {
 "type": "range",
 "maxValue": 10,
 },
 }),
});

const sharedCluster = new databricks.Cluster("shared-cluster", {
 clusterName: "Shared Engineering Cluster",
 sparkVersion: "17.3.x-scala2.13",
 nodeTypeId: "Standard_DS3_v2",
 autoterminationMinutes: 20,
 numWorkers: 2,
 policyId: interactivePolicy.id,
});

const clusterPermissions = new databricks.Permissions("shared-cluster-permissions", {
 clusterId: sharedCluster.id,
 accessControls: [{
 groupName: "data-engineers",
 permissionLevel: "CAN_RESTART",
 }, {
 groupName: "analysts",
 permissionLevel: "CAN_ATTACH_TO",
 }],
});

const secretScope = new databricks.SecretScope("etl-secrets", {
 name: "etl-secrets",
});

const secretScopeAcl = new databricks.SecretAcl("etl-secrets-readers", {
 scope: secretScope.name,
 principal: "data-engineers",
 permission: "READ",
});

const etlNotebook = new databricks.Notebook("etl-notebook", {
 path: "/Shared/ETL/NightlyProcess",
 language: "PYTHON",
 contentBase64: Buffer.from("print('Running ETL process...')").toString("base64"),
});

const job = new databricks.Job("nightly-etl", {
 name: "Nightly ETL Job",
 tasks: [{
 taskKey: "etl-task",
 newCluster: {
 numWorkers: 2,
 sparkVersion: "17.3.x-scala2.13",
 nodeTypeId: "Standard_DS3_v2",
 policyId: jobPolicy.id,
 },
 notebookTask: {
 notebookPath: etlNotebook.path,
 },
 }],
 schedule: {
 quartzCronExpression: "0 0 2 * * ?",
 timezoneId: "UTC",
 },
});

This baseline covers the resources a platform team needs to standardize first:

Interactive cluster policy: caps DBU consumption and enforces auto-termination for shared interactive compute.
Job cluster policy: caps DBU consumption for ephemeral job compute without applying interactive auto-termination settings.
Shared cluster: gives teams a governed interactive compute target.
Secret scope and ACL: makes ETL secret access reviewable.
Cluster permissions: assigns group-level access to the shared cluster.
Notebook: stores the workload artifact in the workspace.
Job: schedules the notebook on policy-constrained compute.

Validation

After running pulumi up, you can validate your Databricks baseline:

Databricks UI: Navigate to the “Compute” section and verify that the interactive and job fair-use policies exist, with auto-termination fixed only on the interactive policy.
Cluster check: Confirm that the shared cluster references the interactive policy and uses the expected node type.
Permission check: Confirm that cluster and secret-scope permissions match the intended Databricks groups.
Notebook check: Confirm that /Shared/ETL/NightlyProcess exists in the workspace.
Job check: Go to the “Workflows” section and confirm that the “Nightly ETL Job” is scheduled and references the notebook.

Managing your Databricks workspace as code gives every team a governed, cost-aware baseline. Use Pulumi stacks to deploy the same cluster policies, permissions, secret scope, notebook, and job pattern to multiple workspaces with environment-specific configuration.

# Deploy to development
pulumi stack select dev
pulumi up

# Promote to production
pulumi stack select prod
pulumi up

This keeps production ETL jobs, cluster policies, secret scopes, and permission assignments aligned with development while still letting each stack bind to its own Databricks workspace and ESC environment.

Conclusion

Next, connect this workspace baseline to your CI/CD pipeline and promote it through Pulumi stacks so Databricks jobs, cluster policies, and permissions move through the same review path as application code. Start with the Databricks provider docs and Pulumi ESC for credential handling.

Deploy a Private Hermes Agent on Render Securely with Pulumi, Modal, and Tailscale

Engin Diri — Wed, 24 Jun 2026 00:00:00 +0000

Personal AI agents had their breakout this year. OpenClaw crossed 100,000 GitHub stars within months of launching, and self-hosting your own assistant went from a hobbyist trick to something a lot of developers actually do. I wrote up how to deploy that lobster to AWS or Hetzner back when it was everywhere.

The one people are switching to now is Hermes, the open-source runtime from Nous Research, and it caught on just as quickly. The reason shows up in every “I ditched OpenClaw for Hermes” thread: it actually learns, building up memory and writing its own skills as it goes instead of running off a static, human-written list.

Here is the part the launch videos skip. Hermes writes and runs its own code, with no human approving the commands. A model that can write code will eventually write a bad one, and the only thing between that command and your credentials is the sandbox it runs in. That is the box you do not want on the public internet. Researchers found 175,000 exposed Ollama servers sitting open in early 2026, and attackers hijack the ones they find for compute. The fix is not a better lock on the front door. It is to have no front door at all.

So this post deploys a private Hermes agent as one Pulumi program across Render, Modal, and Tailscale. The agent and its chat UI run as Render private services with no public URL, Tailscale puts the UI on your tailnet, and Modal runs the agent’s code in throwaway sandboxes. One pulumi up to stand it up, one pulumi destroy to tear it down, around $50 a month. No Makefile, no CLI escape hatches, and an honest look at the few places “nothing on the public internet” still leaks.

What is Hermes?

Hermes is the open-source, MIT-licensed agent runtime from Nous Research. It runs continuously on a server rather than living in a browser tab, and you talk to it through a chat UI or a messaging platform like Telegram, Discord, Slack, or Signal. A few things make it more than a chatbot:

It is model-agnostic. You point it at any provider. This deployment uses OpenRouter, one key that fronts 400-plus models across providers, with automatic failover when one is down.
It remembers across restarts. Memory persists on disk, so it builds up context over time instead of resetting every session.
It writes its own skills. When it works through a task, it can save that approach as a reusable skill and reach for it next time.
It schedules itself. A built-in cron runs automations on a schedule, driven in plain language.
It runs code in a sandbox. When the agent needs to execute code, it hands that off to an isolated, ephemeral container instead of running it next to its own process.

The difference from a cloud-hosted assistant is the usual self-hosting trade: it runs on your infrastructure with your keys, and your conversation history and memory stay on your disk.

“Hermes” here is the agent runtime from Nous Research, not the Hermes family of language models that share the name. The agent is model-agnostic, and the model it talks to is a separate choice you make through OpenRouter.

Prerequisites

Before getting started, ensure you have:

Pulumi CLI installed and configured
A Pulumi Cloud account
A Render account, an API key, and your workspace owner ID
A Modal account and a token pair (token ID and secret)
An OpenRouter API key with credit on the account
A container registry you can push to; this post uses GitHub Container Registry (a GHCR username and token)
Docker running locally (the image build rides on a local buildx engine), or a Docker-enabled CI runner
A Tailscale account with HTTPS enabled and a reusable auth key
Node.js 18+ (for TypeScript) or Python 3.9+ (for Python), depending on the language you pick

This guide routes models through OpenRouter, but Hermes works with other providers too. The model is just a routing string you pass (gpt-5.5 here), so any model OpenRouter supports works by changing that one value.

Understanding the Hermes architecture

Strip away the dashboard clicks and the architecture is four pieces. One rule governs all of them: what the public internet is allowed to touch.

flowchart LR
User([Your devices]) -->|"tailnet (WireGuard)"| TS["Tailscale<br/>private mesh"]
TS -->|"open-webui.&lt;tailnet&gt;.ts.net"| WebUI["Open WebUI<br/>(Render private service,<br/>joined to your tailnet)"]
WebUI -->|"private network<br/>&lt;hermes-slug&gt;:8642"| Hermes["Hermes gateway<br/>(Render private service)"]
Hermes -->|"Modal SDK at runtime"| Modal["Modal sandbox<br/>isolated code execution"]
Hermes -->|model calls| OR["OpenRouter"]

Component	Port	Description
Hermes gateway	8642	Holds secrets and memory, talks to the model, creates Modal sandboxes. The most dangerous box, so it has no public URL.
Open WebUI	8080	The chat front end. Joins your tailnet and reaches Hermes over Render’s private network.
Modal sandbox	-	Isolated, ephemeral container for running the agent’s code.
Tailscale	-	A private WireGuard mesh that makes the UI reachable to your devices only.

The agent that can run code and read secrets is never routable from the internet, and neither is the UI in front of it. There is no inbound ingress on any box (the outbound calls to the model, to Modal, and to the image registry still cross the internet, as they have to). Expressing “which box can reach which” in code is what makes the boundary auditable: the rule lives in the program rather than in a dashboard.

The four pieces split cleanly in two: two you declare as resources, and two you hand a credential and let run.

Layer	How you provision it	Status
Render services, disks, secrets	official `render-oss` Terraform provider, bridged	Declarative, via one `pulumi package add`.
Container images	`@pulumi/docker-build`	Built and pushed during `pulumi up`. No `docker` CLI.
Modal sandboxes	nothing	No Terraform provider exists. A runtime credential.
Tailnet access (Tailscale)	nothing	No provider needed. A runtime credential, like Modal.

Two of these four layers are credentials, not resources. That distinction often gets missed, and it is worth getting right: a provider can only manage what it can provision, and spinning up a Modal sandbox is a runtime act, not a provisioned resource. We come back to it at each layer.

Setting up ESC for secrets management

Deploying Hermes means handling a pile of credentials: a Render API key, registry credentials, the OpenRouter key, the Modal token pair, and the Tailscale auth key. You don’t want these hardcoded or scattered across environment variables. Pulumi ESC (Environments, Secrets, and Configuration) stores them securely and passes them directly to your Pulumi program.

Create a new ESC environment:

pulumi env init <your-org>/hermes-secrets

Add your secrets to the environment:

values:
 renderApiKey:
 fn::secret: "rnd_xxxxx"
 renderOwnerId: "tea-xxxxx"
 ghcrUser: "your-github-username"
 ghcrToken:
 fn::secret: "ghp_xxxxx"
 openrouterApiKey:
 fn::secret: "sk-or-xxxxx"
 modalTokenId:
 fn::secret: "ak-xxxxx"
 modalTokenSecret:
 fn::secret: "as-xxxxx"
 tailscaleAuthKey:
 fn::secret: "tskey-auth-xxxxx"
 tailnetDnsName: "tailxxxxx.ts.net"
 pulumiConfig:
 renderApiKey: ${renderApiKey}
 ownerId: ${renderOwnerId}
 ghcrUser: ${ghcrUser}
 ghcrToken: ${ghcrToken}
 openrouterApiKey: ${openrouterApiKey}
 modalTokenId: ${modalTokenId}
 modalTokenSecret: ${modalTokenSecret}
 tailscaleAuthKey: ${tailscaleAuthKey}
 tailnetDnsName: ${tailnetDnsName}

To find your tailnet DNS name, go to the Tailscale admin console, look under the DNS section, and find your tailnet name (e.g., tailxxxxx.ts.net). This is the domain suffix used for all machines in your Tailscale network.

Then create a Pulumi.dev.yaml file in your project to reference the environment:

environment:
 - <your-org>/hermes-secrets

This keeps your secrets out of your codebase and hands them to the program at deploy time. The Modal token pair and the Tailscale auth key never become resources; they ride straight through as encrypted env vars on the services that need them.

Generating the Render SDK

Render has no first-party Pulumi provider. What it has is an official, maintained Terraform provider, and Pulumi can turn any Terraform provider into a typed local SDK with Any Terraform Provider:

pulumi package add terraform-provider render-oss/render 1.8.0

That one command generates a render SDK you import like any other package, in whatever language your stack is written in. There is no published @pulumi/render package and no vendor to wait on.

This is also where the provider map shows its edges. The bridge wraps a Terraform provider that already exists, so it works for Render. But Modal has no Terraform provider anywhere, so pulumi package add terraform-provider modal-labs/modal has nothing to fetch. That is not a gap to work around. Modal is not a provisioned resource here; Hermes creates sandboxes at runtime through the Modal SDK, and Pulumi’s only job is to hold the token pair and hand it over. Tailscale lands in the same bucket: a Tailscale provider exists, but it manages your tailnet’s settings, not the act of putting a container on the network, which is a runtime job done inside the image.

Securing with Tailscale

The usual way to make a self-hosted UI reachable is to give it a public hostname and put an identity gate in front: an SSO provider, an access policy, a public DNS record, and a certificate to keep renewing. Tailscale collapses all of it into nothing you declare. There is no gate to bolt in front of a public URL, because there is no public URL. Open WebUI joins your tailnet and answers only to your own devices.

This is a stronger position than a public UI behind a login. Because there is no public URL, there is no login to harden and no certificate to renew, and Open WebUI answers only to your own devices.

Tailscale lives in the Open WebUI image rather than the Pulumi program. The image adds Tailscale and an entrypoint that brings up tailscaled in userspace mode (a container has no /dev/net/tun), joins your tailnet, and runs tailscale serve to expose Open WebUI on it. serve publishes over HTTPS to authenticated members of your tailnet only, where its sibling tailscale funnel would put the same port on the public internet, the one thing this design avoids.

FROM ghcr.io/open-webui/open-webui:main-slim
RUN apt-get update && apt-get install -y --no-install-recommends curl ca-certificates \
 && curl -fsSL https://tailscale.com/install.sh | sh
COPY entrypoint.sh /usr/local/bin/ts-entrypoint
ENTRYPOINT ["/usr/local/bin/ts-entrypoint"]

#!/usr/bin/env bash
set -e
PORT="${PORT:-8080}"
mkdir -p /app/backend/data/tailscale
# Userspace networking: a container has no /dev/net/tun. Keep node state on the disk
# so it survives restarts, and accept-dns=false so Render's DNS still resolves the
# private Hermes hostname.
tailscaled --tun=userspace-networking --statedir=/app/backend/data/tailscale &
n=0
until tailscale up --authkey="${TS_AUTHKEY}" --hostname="${TS_HOSTNAME}" --accept-dns=false; do
 n=$((n + 1)); [ "${n}" -ge 30 ] && { echo "tailnet join failed; check TS_AUTHKEY"; exit 1; }
 sleep 2
done
tailscale serve --bg "${PORT}" # serve Open WebUI on the tailnet over HTTPS
cd /app/backend && exec bash start.sh

The TS_AUTHKEY env var on the service, pulled from ESC, is the only thing Pulumi contributes; everything else is the image doing its own networking.

There is one prerequisite, the same one any Tailscale node needs for serve: enable HTTPS for your tailnet once in the admin console so it can get a certificate. Use a reusable auth key, since the service re-authenticates on every redeploy, and the join loop has a ceiling so a dead key fails the deploy in a minute instead of hanging it.

To generate the reusable auth key:

Go to the Tailscale admin console.
Click Generate auth key.
Enable Reusable, since the service re-authenticates on every redeploy.
Copy the key into your ESC environment as tailscaleAuthKey.

Deploying the agent

This is the part a Makefile usually handles. Two private services, two disks, every secret an encrypted env var, and both images built and pushed during the same pulumi up that creates the services. No CLI runs anywhere, though the build still rides on a local Docker engine through buildx: the docker call is gone, the daemon it talks to is not, so pulumi up needs Docker running.

Create a new Pulumi project (pick the language you prefer; the companion repo has all three):

mkdir hermes-agent-pulumi && cd hermes-agent-pulumi
pulumi new typescript

Install the dependencies and generate the Render SDK:

npm install @pulumi/docker-build @pulumi/random
pulumi package add terraform-provider render-oss/render 1.8.0

Both services need the 2 GB Standard plan. The gateway and Open WebUI each exhaust memory on the 512 MB Starter and OOM on boot.

Two facts make this clean. Render env var values are encrypted at rest, so a secret is only an env var, with no separate resource. And @pulumi/docker-build builds and pushes an image through an embedded buildx during pulumi up, so the image pipeline is declarative too. Render then pulls the pushed image by digest.

The complete program

Hermes comes first: a private Render service with no public URL, a disk for its memory, and every secret as an encrypted env var. The shared bearer key is generated in state with random.RandomBytes, the clean replacement for openssl rand -base64 32; the Modal token pair and the OpenRouter key come from config or Pulumi ESC. Render has no port field, so Hermes binds 0.0.0.0:8642 and Open WebUI reaches it at the service’s read-only slug. Open WebUI is almost the mirror image, private too, except its image joins your tailnet (the tailscaled entrypoint from the previous section) and it carries the operational env vars that keep it lean: no local embedding models, and ENABLE_PERSISTENT_CONFIG=false so your config always wins over a stale copy in its database. One render.Provider carries your renderApiKey and ownerId into every Render resource.

Here is the whole program, copy-paste ready as your index.ts (or __main__.py, or Pulumi.yaml):

import * as pulumi from "@pulumi/pulumi";
import * as dockerbuild from "@pulumi/docker-build";
import * as random from "@pulumi/random";
// Local SDK generated by: pulumi package add terraform-provider render-oss/render 1.8.0
// (no @pulumi/render is published; the SDK is bridged from the official Render provider).
import * as render from "@pulumi/render";

const cfg = new pulumi.Config();

// ---------------------------------------------------------------------------
// Configuration. Non-secret values via `pulumi config set`, secrets via
// `pulumi config set --secret` or a Pulumi ESC environment (see README).
// The Render provider reads renderApiKey/renderOwnerId (or RENDER_API_KEY/RENDER_OWNER_ID).
// ---------------------------------------------------------------------------
const region = cfg.get("renderRegion") ?? "oregon";
const modelDefault = cfg.get("modelDefault") ?? "gpt-5.5";

// Container registry the images are pushed to and Render pulls from (GHCR here).
const ghcrUser = cfg.require("ghcrUser");
const ghcrToken = cfg.requireSecret("ghcrToken"); // GHCR PAT with write:packages

// Long-lived secrets supplied by you (dashboard-minted). Hold in config or ESC.
const modalTokenId = cfg.requireSecret("modalTokenId");
const modalTokenSecret = cfg.requireSecret("modalTokenSecret");
const openrouterApiKey = cfg.requireSecret("openrouterApiKey");

// Tailscale: the only "gateway". Open WebUI joins your tailnet and is reachable
// at https://<hostname>.<tailnet>.ts.net by your devices only — never public.
// Both values are already in your ESC. Enable HTTPS once in the Tailscale admin console.
const tailscaleAuthKey = cfg.requireSecret("tailscaleAuthKey");
const tailnetDnsName = cfg.require("tailnetDnsName"); // e.g. tailc6fb4e.ts.net
const webuiHostname = "open-webui";
const webuiTailnetUrl = `https://${webuiHostname}.${tailnetDnsName}`;

// Render provider auth (from ESC: pulumiConfig.renderApiKey / pulumiConfig.renderOwnerId).
const renderProvider = new render.Provider("render", {
 apiKey: cfg.requireSecret("renderApiKey"),
 ownerId: cfg.require("renderOwnerId"),
});

// ---------------------------------------------------------------------------
// Secrets generated in state (the in-IaC replacement for `openssl rand -base64 32`).
// The shared bearer authenticates Open WebUI -> Hermes; it lands on both services.
// ---------------------------------------------------------------------------
const sharedKey = new random.RandomBytes("hermes-shared-key", { length: 32 });
const webuiSecretKey = new random.RandomBytes("webui-secret-key", { length: 32 });

// ---------------------------------------------------------------------------
// Build and push both images during `pulumi up` (no docker CLI; the provider
// embeds buildx). Render then pulls them by their digest-pinned ref.
// ---------------------------------------------------------------------------
const registryAuth = [{ address: "ghcr.io", username: ghcrUser, password: ghcrToken }];

const hermesImage = new dockerbuild.Image("hermes-image", {
 context: { location: "./hermes" },
 dockerfile: { location: "./hermes/Dockerfile" },
 platforms: ["linux/amd64"],
 tags: [pulumi.interpolate`ghcr.io/${ghcrUser}/hermes-agent:latest`],
 push: true,
 registries: registryAuth,
});

const webuiImage = new dockerbuild.Image("webui-image", {
 context: { location: "./openwebui" },
 dockerfile: { location: "./openwebui/Dockerfile" },
 platforms: ["linux/amd64"],
 tags: [pulumi.interpolate`ghcr.io/${ghcrUser}/open-webui:latest`],
 push: true,
 registries: registryAuth,
});

// Credential Render uses to pull the private GHCR images.
const ghcrCredential = new render.RegistryCredential("ghcr", {
 name: "ghcr",
 registry: "GITHUB",
 username: ghcrUser,
 authToken: ghcrToken,
}, { provider: renderProvider });

// ---------------------------------------------------------------------------
// Hermes: a PRIVATE Render service. No public URL; reachable only over Render's
// private network. A disk holds its memory and workspace. Env values are
// encrypted at rest, which is how secrets ride on Render (no separate flag).
// ---------------------------------------------------------------------------
const hermes = new render.PrivateService("hermes-agent", {
 name: "hermes-agent",
 plan: "standard", // 2 GB; the gateway OOMs on starter's 512 MB
 region: region,
 runtimeSource: {
 // Render wants the bare repo in imageUrl; the tag/digest go in their own field.
 image: {
 imageUrl: pulumi.interpolate`ghcr.io/${ghcrUser}/hermes-agent`,
 digest: hermesImage.digest,
 registryCredentialId: ghcrCredential.id,
 },
 },
 disk: { name: "hermes-data", mountPath: "/opt/data", sizeGb: 5 },
 envVars: {
 PORT: { value: "8642" },
 API_SERVER_ENABLED: { value: "true" },
 API_SERVER_HOST: { value: "0.0.0.0" },
 API_SERVER_PORT: { value: "8642" },
 API_SERVER_MODEL_NAME: { value: modelDefault },
 // Auto-approve the agent's shell/terminal hooks (no TTY in a container),
 // so it can run code in the Modal sandbox without an interactive prompt.
 HERMES_ACCEPT_HOOKS: { value: "1" },
 // Secrets (encrypted at rest on Render):
 API_SERVER_KEY: { value: sharedKey.base64 },
 OPENROUTER_API_KEY: { value: openrouterApiKey },
 // Modal is a runtime credential, not a provisioned resource: Hermes creates
 // sandboxes at runtime with this token pair. No Modal provider, no Modal CLI.
 MODAL_TOKEN_ID: { value: modalTokenId },
 MODAL_TOKEN_SECRET: { value: modalTokenSecret },
 },
}, { provider: renderProvider });

// Open WebUI reaches Hermes at its Render-assigned internal hostname (the read-only
// slug), not the bare service name. Same region + same workspace share the network.
const hermesApiBaseUrl = pulumi.interpolate`http://${hermes.slug}:8642/v1`;

// ---------------------------------------------------------------------------
// Open WebUI: also a PRIVATE Render service, but the container joins your tailnet
// (tailscaled in userspace + `tailscale serve`), so you reach it at
// https://open-webui.<tailnet>.ts.net from your devices. Never public; the gate
// is tailnet membership. It still reaches Hermes over Render's private network.
// ---------------------------------------------------------------------------
const webui = new render.PrivateService("open-webui", {
 name: "open-webui",
 plan: "standard", // 2 GB; Open WebUI OOMs on starter's 512 MB
 region: region, // must match Hermes for private networking
 runtimeSource: {
 image: {
 imageUrl: pulumi.interpolate`ghcr.io/${ghcrUser}/open-webui`,
 digest: webuiImage.digest,
 registryCredentialId: ghcrCredential.id,
 },
 },
 disk: { name: "openwebui-data", mountPath: "/app/backend/data", sizeGb: 5 },
 envVars: {
 PORT: { value: "8080" }, // tailscale serve points at this; Open WebUI binds it
 OPENAI_API_BASE_URL: { value: hermesApiBaseUrl },
 ENABLE_OPENAI_API: { value: "true" },
 ENABLE_OLLAMA_API: { value: "false" },
 DEFAULT_MODELS: { value: modelDefault },
 WEBUI_AUTH: { value: "true" },
 WEBUI_URL: { value: webuiTailnetUrl },
 // Reached only over the tailnet, so the first sign-up is the admin.
 ENABLE_LOGIN_FORM: { value: "true" },
 ENABLE_SIGNUP: { value: "true" },
 DEFAULT_USER_ROLE: { value: "admin" },
 // Always apply env config; otherwise Open WebUI persists the first run's config
 // to its DB and ignores later changes.
 ENABLE_PERSISTENT_CONFIG: { value: "false" },
 // Don't load local embedding/RAG models (a big memory saver; the agent is the brain).
 RAG_EMBEDDING_ENGINE: { value: "openai" },
 BYPASS_EMBEDDING_AND_RETRIEVAL: { value: "true" },
 OFFLINE_MODE: { value: "true" },
 HF_HUB_OFFLINE: { value: "1" },
 // Secrets (encrypted at rest):
 OPENAI_API_KEY: { value: sharedKey.base64 }, // shared bearer to call Hermes
 WEBUI_SECRET_KEY: { value: webuiSecretKey.base64 },
 // Tailscale joins this container to your tailnet and serves Open WebUI on it.
 TS_AUTHKEY: { value: tailscaleAuthKey },
 TS_HOSTNAME: { value: webuiHostname },
 },
}, { dependsOn: hermes, provider: renderProvider });

export const hermesInternalUrl = hermesApiBaseUrl;
// Open this from any device on your tailnet:
export const openWebUiTailnetUrl = webuiTailnetUrl;

import pulumi
import pulumi_docker_build as docker_build
import pulumi_random as random

# Local SDK generated by: pulumi package add terraform-provider render-oss/render 1.8.0
# (no pulumi_render is published; the SDK is bridged from the official Render provider).
import pulumi_render as render

cfg = pulumi.Config()

# ---------------------------------------------------------------------------
# Configuration. Non-secret values via `pulumi config set`, secrets via
# `pulumi config set --secret` or a Pulumi ESC environment (see README).
# The Render provider reads renderApiKey/renderOwnerId (or RENDER_API_KEY/RENDER_OWNER_ID).
# ---------------------------------------------------------------------------
region = cfg.get("renderRegion") or "oregon"
model_default = cfg.get("modelDefault") or "gpt-5.5"

# Container registry the images are pushed to and Render pulls from (GHCR here).
ghcr_user = cfg.require("ghcrUser")
ghcr_token = cfg.require_secret("ghcrToken") # GHCR PAT with write:packages

# Long-lived secrets supplied by you (dashboard-minted). Hold in config or ESC.
modal_token_id = cfg.require_secret("modalTokenId")
modal_token_secret = cfg.require_secret("modalTokenSecret")
openrouter_api_key = cfg.require_secret("openrouterApiKey")

# Tailscale: the only "gateway". Open WebUI joins your tailnet and is reachable
# at https://<hostname>.<tailnet>.ts.net by your devices only -- never public.
# Enable HTTPS once in the Tailscale admin console so `tailscale serve` can get a cert.
tailscale_auth_key = cfg.require_secret("tailscaleAuthKey")
tailnet_dns_name = cfg.require("tailnetDnsName") # e.g. tailc6fb4e.ts.net
webui_hostname = "open-webui"
webui_tailnet_url = f"https://{webui_hostname}.{tailnet_dns_name}"

# Render provider auth (from ESC: pulumiConfig.renderApiKey / pulumiConfig.renderOwnerId).
render_provider = render.Provider(
 "render",
 api_key=cfg.require_secret("renderApiKey"),
 owner_id=cfg.require("renderOwnerId"),
)

# ---------------------------------------------------------------------------
# Secrets generated in state (the in-IaC replacement for `openssl rand -base64 32`).
# The shared bearer authenticates Open WebUI -> Hermes; it lands on both services.
# ---------------------------------------------------------------------------
shared_key = random.RandomBytes("hermes-shared-key", length=32)
webui_secret_key = random.RandomBytes("webui-secret-key", length=32)

# ---------------------------------------------------------------------------
# Build and push both images during `pulumi up` (no docker CLI; the provider
# embeds buildx). Render then pulls them by their digest-pinned ref.
# ---------------------------------------------------------------------------
registry_auth = [{"address": "ghcr.io", "username": ghcr_user, "password": ghcr_token}]

hermes_image = docker_build.Image(
 "hermes-image",
 context={"location": "./hermes"},
 dockerfile={"location": "./hermes/Dockerfile"},
 platforms=["linux/amd64"],
 tags=[pulumi.Output.concat("ghcr.io/", ghcr_user, "/hermes-agent:latest")],
 push=True,
 registries=registry_auth,
)

webui_image = docker_build.Image(
 "webui-image",
 context={"location": "./openwebui"},
 dockerfile={"location": "./openwebui/Dockerfile"},
 platforms=["linux/amd64"],
 tags=[pulumi.Output.concat("ghcr.io/", ghcr_user, "/open-webui:latest")],
 push=True,
 registries=registry_auth,
)

# Credential Render uses to pull the private GHCR images.
ghcr_credential = render.RegistryCredential(
 "ghcr",
 name="ghcr",
 registry="GITHUB",
 username=ghcr_user,
 auth_token=ghcr_token,
 opts=pulumi.ResourceOptions(provider=render_provider),
)

# ---------------------------------------------------------------------------
# Hermes: a PRIVATE Render service. No public URL; reachable only over Render's
# private network. A disk holds its memory and workspace. Env values are
# encrypted at rest, which is how secrets ride on Render (no separate flag).
# ---------------------------------------------------------------------------
hermes = render.PrivateService(
 "hermes-agent",
 name="hermes-agent",
 plan="standard", # 2 GB; the gateway OOMs on starter's 512 MB
 region=region,
 runtime_source={
 "image": {
 "image_url": pulumi.Output.concat("ghcr.io/", ghcr_user, "/hermes-agent"),
 "digest": hermes_image.digest,
 "registry_credential_id": ghcr_credential.id,
 },
 },
 disk={"name": "hermes-data", "mount_path": "/opt/data", "size_gb": 5},
 env_vars={
 "PORT": {"value": "8642"},
 "API_SERVER_ENABLED": {"value": "true"},
 "API_SERVER_HOST": {"value": "0.0.0.0"},
 "API_SERVER_PORT": {"value": "8642"},
 "API_SERVER_MODEL_NAME": {"value": model_default},
 # Auto-approve the agent's shell/terminal hooks (no TTY in a container).
 "HERMES_ACCEPT_HOOKS": {"value": "1"},
 # Secrets (encrypted at rest on Render):
 "API_SERVER_KEY": {"value": shared_key.base64},
 "OPENROUTER_API_KEY": {"value": openrouter_api_key},
 # Modal is a runtime credential: Hermes creates sandboxes at runtime with this pair.
 "MODAL_TOKEN_ID": {"value": modal_token_id},
 "MODAL_TOKEN_SECRET": {"value": modal_token_secret},
 },
 opts=pulumi.ResourceOptions(provider=render_provider),
)

# Open WebUI reaches Hermes at its Render-assigned internal hostname (the read-only slug).
hermes_api_base_url = pulumi.Output.concat("http://", hermes.slug, ":8642/v1")

# ---------------------------------------------------------------------------
# Open WebUI: also a PRIVATE Render service, but the container joins your tailnet
# (tailscaled in userspace + `tailscale serve`), so you reach it at
# https://open-webui.<tailnet>.ts.net from your devices. Never public.
# ---------------------------------------------------------------------------
webui = render.PrivateService(
 "open-webui",
 name="open-webui",
 plan="standard", # 2 GB; Open WebUI OOMs on starter's 512 MB
 region=region, # must match Hermes for private networking
 runtime_source={
 "image": {
 "image_url": pulumi.Output.concat("ghcr.io/", ghcr_user, "/open-webui"),
 "digest": webui_image.digest,
 "registry_credential_id": ghcr_credential.id,
 },
 },
 disk={"name": "openwebui-data", "mount_path": "/app/backend/data", "size_gb": 5},
 env_vars={
 "PORT": {"value": "8080"}, # tailscale serve points at this; Open WebUI binds it
 "OPENAI_API_BASE_URL": {"value": hermes_api_base_url},
 "ENABLE_OPENAI_API": {"value": "true"},
 "ENABLE_OLLAMA_API": {"value": "false"},
 "DEFAULT_MODELS": {"value": model_default},
 "WEBUI_AUTH": {"value": "true"},
 "WEBUI_URL": {"value": webui_tailnet_url},
 # Reached only over the tailnet, so the first sign-up is the admin.
 "ENABLE_LOGIN_FORM": {"value": "true"},
 "ENABLE_SIGNUP": {"value": "true"},
 "DEFAULT_USER_ROLE": {"value": "admin"},
 # Always apply env config; otherwise Open WebUI persists the first run's config.
 "ENABLE_PERSISTENT_CONFIG": {"value": "false"},
 # Don't load local embedding/RAG models (a big memory saver; the agent is the brain).
 "RAG_EMBEDDING_ENGINE": {"value": "openai"},
 "BYPASS_EMBEDDING_AND_RETRIEVAL": {"value": "true"},
 "OFFLINE_MODE": {"value": "true"},
 "HF_HUB_OFFLINE": {"value": "1"},
 # Secrets (encrypted at rest):
 "OPENAI_API_KEY": {"value": shared_key.base64},
 "WEBUI_SECRET_KEY": {"value": webui_secret_key.base64},
 # Tailscale joins this container to your tailnet and serves Open WebUI on it.
 "TS_AUTHKEY": {"value": tailscale_auth_key},
 "TS_HOSTNAME": {"value": webui_hostname},
 },
 opts=pulumi.ResourceOptions(depends_on=[hermes], provider=render_provider),
)

pulumi.export("hermesInternalUrl", hermes_api_base_url)
# Open this from any device on your tailnet:
pulumi.export("openWebUiTailnetUrl", webui_tailnet_url)

name: hermes-agent-pulumi
runtime: yaml
description: Deploy a Hermes agent (Render + Modal + Tailscale) as one declarative Pulumi program.

# The Render SDK is bridged from the official Terraform provider. Run once:
# pulumi package add terraform-provider render-oss/render 1.8.0
# Then `pulumi install` regenerates it on a fresh checkout.
packages:
 render:
 source: terraform-provider
 version: 1.1.4
 parameters:
 - render-oss/render
 - 1.8.0

config:
 renderRegion:
 type: string
 default: oregon
 modelDefault:
 type: string
 default: gpt-5.5
 ghcrUser:
 type: string
 ghcrToken:
 type: string
 secret: true
 modalTokenId:
 type: string
 secret: true
 modalTokenSecret:
 type: string
 secret: true
 openrouterApiKey:
 type: string
 secret: true
 tailscaleAuthKey:
 type: string
 secret: true
 tailnetDnsName:
 type: string
 renderApiKey:
 type: string
 secret: true
 renderOwnerId:
 type: string

variables:
 registryAuth:
 - address: ghcr.io
 username: ${ghcrUser}
 password: ${ghcrToken}
 # Open WebUI reaches Hermes at its private slug; you reach Open WebUI on the tailnet.
 hermesApiBaseUrl: http://${hermes-agent.slug}:8642/v1
 webuiTailnetUrl: https://open-webui.${tailnetDnsName}

resources:
 # Render provider auth (from config or ESC).
 render:
 type: pulumi:providers:render
 properties:
 apiKey: ${renderApiKey}
 ownerId: ${renderOwnerId}

 # Secrets generated in state: the replacement for `openssl rand -base64 32`.
 hermes-shared-key:
 type: random:RandomBytes
 properties:
 length: 32
 webui-secret-key:
 type: random:RandomBytes
 properties:
 length: 32

 # Build + push both images during `pulumi up` (no docker CLI).
 hermes-image:
 type: docker-build:Image
 properties:
 context:
 location: ./hermes
 dockerfile:
 location: ./hermes/Dockerfile
 platforms: [linux/amd64]
 tags:
 - ghcr.io/${ghcrUser}/hermes-agent:latest
 push: true
 registries: ${registryAuth}
 webui-image:
 type: docker-build:Image
 properties:
 context:
 location: ./openwebui
 dockerfile:
 location: ./openwebui/Dockerfile
 platforms: [linux/amd64]
 tags:
 - ghcr.io/${ghcrUser}/open-webui:latest
 push: true
 registries: ${registryAuth}

 # Credential Render uses to pull the private GHCR images.
 ghcr:
 type: render:RegistryCredential
 properties:
 name: ghcr
 registry: GITHUB
 username: ${ghcrUser}
 authToken: ${ghcrToken}
 options:
 provider: ${render}

 # Hermes: a PRIVATE Render service. No public URL; encrypted env vars are the secret mechanism.
 hermes-agent:
 type: render:PrivateService
 properties:
 name: hermes-agent
 plan: standard  # 2 GB; the gateway OOMs on starter's 512 MB
 region: ${renderRegion}
 runtimeSource:
 image:
 imageUrl: ghcr.io/${ghcrUser}/hermes-agent
 digest: ${hermes-image.digest}
 registryCredentialId: ${ghcr.id}
 disk:
 name: hermes-data
 mountPath: /opt/data
 sizeGb: 5
 envVars:
 PORT: { value: "8642" }
 API_SERVER_ENABLED: { value: "true" }
 API_SERVER_HOST: { value: "0.0.0.0" }
 API_SERVER_PORT: { value: "8642" }
 API_SERVER_MODEL_NAME: { value: "${modelDefault}" }
 HERMES_ACCEPT_HOOKS: { value: "1" }
 API_SERVER_KEY: { value: "${hermes-shared-key.base64}" }
 OPENROUTER_API_KEY: { value: "${openrouterApiKey}" }
 MODAL_TOKEN_ID: { value: "${modalTokenId}" }
 MODAL_TOKEN_SECRET: { value: "${modalTokenSecret}" }
 options:
 provider: ${render}

 # Open WebUI: also a PRIVATE Render service, joined to your tailnet by its image.
 open-webui:
 type: render:PrivateService
 properties:
 name: open-webui
 plan: standard  # 2 GB; Open WebUI OOMs on starter's 512 MB
 region: ${renderRegion}  # must match Hermes for private networking
 runtimeSource:
 image:
 imageUrl: ghcr.io/${ghcrUser}/open-webui
 digest: ${webui-image.digest}
 registryCredentialId: ${ghcr.id}
 disk:
 name: openwebui-data
 mountPath: /app/backend/data
 sizeGb: 5
 envVars:
 PORT: { value: "8080" }
 OPENAI_API_BASE_URL: { value: "${hermesApiBaseUrl}" }
 ENABLE_OPENAI_API: { value: "true" }
 ENABLE_OLLAMA_API: { value: "false" }
 DEFAULT_MODELS: { value: "${modelDefault}" }
 WEBUI_AUTH: { value: "true" }
 WEBUI_URL: { value: "${webuiTailnetUrl}" }
 ENABLE_LOGIN_FORM: { value: "true" }
 ENABLE_SIGNUP: { value: "true" }
 DEFAULT_USER_ROLE: { value: "admin" }
 ENABLE_PERSISTENT_CONFIG: { value: "false" }
 RAG_EMBEDDING_ENGINE: { value: "openai" }
 BYPASS_EMBEDDING_AND_RETRIEVAL: { value: "true" }
 OFFLINE_MODE: { value: "true" }
 HF_HUB_OFFLINE: { value: "1" }
 OPENAI_API_KEY: { value: "${hermes-shared-key.base64}" }
 WEBUI_SECRET_KEY: { value: "${webui-secret-key.base64}" }
 TS_AUTHKEY: { value: "${tailscaleAuthKey}" }
 TS_HOSTNAME: { value: "open-webui" }
 options:
 provider: ${render}
 dependsOn:
 - ${hermes-agent}

outputs:
 hermesInternalUrl: ${hermesApiBaseUrl}
 # Open this from any device on your tailnet:
 openWebUiTailnetUrl: ${webuiTailnetUrl}

Neither box is published to the internet. The same code, plus the Dockerfiles and the Tailscale entrypoint, is in the companion repo:

github.com/dirien/hermes-agent-pulumi

Cost

Two details shape the bill. The host starts on Render’s free Hobby workspace, but a private service with a persistent disk runs on paid compute, so you need a card on file the moment you add them. There is no free-tier version of this architecture. Both services also need the 2 GB Standard plan, since the gateway and Open WebUI each OOM on the 512 MB Starter.

	Hermes gateway	Open WebUI
Plan	Standard (2 GB)	Standard (2 GB)
Compute	~$25/mo	~$25/mo
Disk	5 GB	5 GB
Public URL	none	none (tailnet only)

That lands the host around $50/month, in line with what a comparable managed setup runs. Modal and OpenRouter bill by usage on top, so the total tracks how hard the agent works rather than a flat number. Tailscale is free for a personal tailnet, so the private gateway adds nothing to that.

The persistent disk pins each service to a single instance, so there is no autoscaling, and a redeploy drops a few seconds of connectivity while the old instance lets go of the disk. For a single-user agent that is a fine trade, and it is why “stand it up in a second region” is a copy-the-config move and not a live failover.

The OpenRouter Activity dashboard: 56 requests and 738K tokens of gpt-5.5 usage, all routed through one key.

Running the deployment

With your ESC environment referenced in Pulumi.dev.yaml, deploy with:

pulumi up

This generates the Render SDK’s resources, builds and pushes both images through buildx, and creates the two private services with their disks and encrypted env vars, all in one run, with no docker CLI and no secret import hiding in a script. When it finishes, both services show up in the Render dashboard with no public URL.

The hermes-agent and open-webui private services in the Render dashboard, both deployed with no public URL.

Access the agent

Give it a few minutes after pulumi up finishes. The images still have to be pulled and the containers booted, and Open WebUI has to join your tailnet before the URL resolves. Refresh until it loads.

Open WebUI is reachable only from a device on your tailnet, at https://open-webui.<your-tailnet>.ts.net, with no domain to buy, no DNS to point, and no certificate to manage. Open it, sign up so the first account becomes the admin, and ask the agent to run a Python command. It should fire up a Modal sandbox and return the result, with nothing exposed to the public internet at any point.

Open WebUI on the tailnet URL: the gpt-5.5 agent runs sum(i*i for i in range(1, 1001)) in a Modal sandbox and returns 333833500.

When the agent runs that code, Modal builds a throwaway sandbox specifically for it. There is no Modal resource to declare, no Modal CLI to call, and no Modal provider to bridge. The Hermes service carries MODAL_TOKEN_ID and MODAL_TOKEN_SECRET as two encrypted env vars, and that is the whole footprint.

The Modal dashboard: the per-task sandboxes the Hermes agent spun up at runtime to execute code.

Verify the deployment (optional)

If the chat UI does not load, give the services a few minutes to finish their first boot, then check two things:

Render dashboard logs for the hermes-agent and open-webui services, to confirm both started and pulled their images.
Tailscale admin console (Machines), to confirm the open-webui node joined your tailnet. If it did not, the TS_AUTHKEY is likely expired or not reusable.

Security considerations

Self-hosted AI servers get found fast: scanners like Shodan and Censys enumerate a freshly exposed one within hours to days, and the code-capable ones are already under active attack. Running an always-on agent on a machine you also use for everything else invites prompt injection on top of that. The answer here is to keep the agent unroutable from the internet and to isolate the code it runs. That defeats the scanner: nobody finds an open port to attack. It does less against the agent being turned against you from the inside, which is the harder problem.

One detail makes that isolation essential. In a headless container there is no one at the terminal to approve anything, and the same is true of its cron and CI runs, so the deployment runs Hermes with HERMES_ACCEPT_HOOKS=1 to auto-accept the shell hooks it would otherwise pause on. The practical result is that the agent’s code runs with no human in the loop, which is exactly why the Modal sandbox has to carry the weight: when the model writes a bad command, the throwaway container it runs in is what stands between that command and the gateway’s credentials, not a confirmation prompt.

Concern	Typical self-host	This deployment
Gateway reachability	Public port, or behind a login	Private service, no public URL
Chat UI reachability	Public hostname + auth	Tailnet only (`tailscale serve`)
Code execution	Runs next to the gateway	Isolated Modal sandbox, torn down after
Command approval	Human approves dangerous commands	Headless: isolation replaces the prompt
Secrets in transit	Exposed if accessed over HTTP	Encrypted env vars; tailnet-encrypted access

This is private, which is not the same as fully trusted. A few places the boundary still leaks:

The shared bearer never rotates. Hermes is unroutable from the internet, but it is wide open to Open WebUI, and the two share one static bearer key. Whoever owns the front end owns the gateway behind it. Rotate it periodically.
The tailnet policy is click-ops. The ACLs, the node’s tags, and the one-time HTTPS toggle live in the Tailscale admin console, not in pulumi up, and the reusable key has no tag scoping its blast radius. The Tailscale provider can manage that policy declaratively if you want it in code.
Signup defaults to admin. Fine on a single-user tailnet, the first thing to lock down (ENABLE_SIGNUP, DEFAULT_USER_ROLE) once more than one person can reach the URL.
Every messaging client is another entry point. Each platform you wire up adds another way in, which is why the tailnet is the baseline for securing this, not the full extent of it.
A compromised agent can still reach out. The private core stops external discovery, but it does not contain the agent itself. The gateway holds the Modal and OpenRouter tokens as env vars and can call out freely, so a prompt-injected agent could read its own environment and send them somewhere; the Modal sandbox isolates the code the agent runs, not the credentials it holds. Cap and rotate the OpenRouter key, keep the Modal token off the gateway, and allowlist outbound traffic, which on Render means a forward-proxy sidecar (Render gives you dedicated outbound IPs and inbound rules, but no destination egress filter).

My recommendations:

Keep both services private; never give Hermes a public URL
Run code in Modal sandboxes, since HERMES_ACCEPT_HOOKS=1 means no human approves commands
Rotate the shared bearer key and the Tailscale auth key periodically
Cap and rotate the OpenRouter key; a leaked credential is the likelier failure mode than an open port
Use Pulumi ESC for secrets instead of hardcoding
Lock down Open WebUI signup once more than one person can reach it
Manage Tailscale ACLs and tags deliberately, in the admin console or with the Tailscale provider

What’s next?

The deployment is the foundation the agent builds on. A private agent becomes useful when it remembers earlier context, runs code without a human at the terminal, and initiates contact on its own. The disk is what makes this persist: its memory and its schedule live on that disk, so they survive every restart.

It schedules itself. Hermes ships a first-class cron you drive in plain language, either by asking the agent in the chat window or through its CLI. The schedule takes intervals, five-field cron, or one-shots:

hermes cron create "every 1h" \
 "Scrape <listing-url> for the price, compare it with seen.json in the sandbox, message me on Telegram only if it dropped, and otherwise reply exactly [SILENT]." \
 --deliver telegram

The [SILENT] reply suppresses delivery when nothing changed, so the watcher only speaks up when there is something to say. Scheduled runs skip memory by default, so polling does not pollute what the agent has learned.

It connects where you work, with the same secret pattern. Messaging clients attach to the same gateway, and their tokens ride as encrypted env vars on the Hermes service, the exact shape you used for the Modal pair:

// alongside MODAL_TOKEN_ID on the hermes service:
TELEGRAM_BOT_TOKEN: { value: cfg.requireSecret("telegramBotToken") },

plus a platform block you add to config.yaml (the image ships only the model and terminal blocks; Hermes supports the messaging platforms on top, per its docs):

gateway:
 platforms:
 telegram:
 require_mention: true

With those two pieces, a few things are genuinely doable today:

A price or uptime watcher that only speaks up when something changed, using the [SILENT] and persistent-sandbox dedup pattern above.
A daily briefing that ranks and dedups your feeds and delivers the top items each morning, the documented daily-briefing bot. If you want it to adapt to your taste over time, that is the optional Honcho memory provider’s job; it reads your preferences out of the conversation itself. The built-in file memory has no version of that.
A PR-review teammate that reviews open pull requests against your repo’s AGENTS.md conventions and saves recurring patterns as a skill, so the fifth review needs less hand-holding than the first.

A few limits are worth stating plainly, since demos tend to overstate the capabilities. Hermes does not ship Gmail, Calendar, or market integrations out of the box; those are wired up as MCP servers or skills, which is more work than a launch video implies. It curates its own memory and improves its own skills, but it does not retrain the underlying model or autonomously change its high-level strategy, so “self-learning” here means whatever it writes into memory and skills each run.

Conclusion

Deploying a Hermes agent with infrastructure as code means you can reproduce the setup anytime, version control it, and tear it down with a single pulumi destroy. Keeping both services private, with no public URL and no exposed ports, means the code-executing core never sits on the internet for someone to find.

The part worth carrying to the next service you deploy: not every layer has a provider, and that is fine. The task is to tell a real resource apart from a runtime dependency wearing a costume. Render and the images are resources you declare; Modal and Tailscale are credentials you hand over. Draw that map first, and the program follows from it.

To generate that map instead of writing it by hand, Pulumi Neo can take a target architecture like this one and produce a first draft of the program. The agent side of this shift is covered in How Building AI Agents Has Changed in 2026 and the platform side in Agent Sprawl Is Here. Your IaC Platform Is the Answer..

If you run into issues or have questions, drop by the Pulumi Community Slack or GitHub Discussions. New to Pulumi? Get started here.

Cloudflare-First Networking as Code with Pulumi

Pablo Seibelt — Tue, 23 Jun 2026 00:00:00 +0000

Platform teams managing multi-cloud applications face a dangerous visibility gap. While origin infrastructure is tightly controlled, the edge configuration often drifts through manual console tweaks. DNS records point to stale origins, WAF rules are inconsistent across environments, and Zero Trust policies fail to keep pace with team changes. This edge drift leads to application exposure or routing failures that origin teams only notice after users report them.

As applications span multiple clouds, the edge often becomes the most consistent layer for enforcing security and traffic policies. The cost of waiting for a unified edge strategy is high. Every manual change is a potential security hole or a performance bottleneck that bypasses your standard CI/CD rigor.

What you’ll build

In this post, you will build a standardized Cloudflare edge baseline. You’ll use Pulumi to define:

DNS records to point to your multi-cloud origins.
WAF custom rules to block malicious traffic before it hits your network.
A Worker canary to handle edge logic and header validation.
Zero Trust Access policies to secure internal tools and origin access.

By the end, you will have a version-controlled edge that you can validate and deploy across any environment.

The Pulumi program

We’ll build a Pulumi program that sets up a complete edge environment. This approach ensures that your edge security and routing are always in sync with your application code.

Setting up the provider

First, ensure you have the @pulumi/cloudflare package installed and your credentials configured. The examples below are checked against @pulumi/cloudflare@6.15.0, the current v6 Pulumi Registry API at the time of writing.

npm install @pulumi/cloudflare@6.15.0
pulumi config set --secret cloudflare:apiToken "$CLOUDFLARE_API_TOKEN"

Defining the infrastructure

Here is the complete Pulumi program in TypeScript. If your zone already has a custom WAF ruleset for the same phase, import that ruleset or add the rule to your existing ruleset instead of creating another phase-level ruleset that may fail or conflict.

import * as cloudflare from "@pulumi/cloudflare";

const accountId = "<your-cloudflare-account-id>";
const trustedAdminIp = "198.51.100.10";

const zone = new cloudflare.Zone("my-zone", {
 account: {
 id: accountId,
 },
 name: "example.com",
});

const www = new cloudflare.DnsRecord("www", {
 zoneId: zone.id,
 name: "www",
 content: "1.2.3.4",
 ttl: 1,
 type: "A",
 proxied: true,
});

const admin = new cloudflare.DnsRecord("admin", {
 zoneId: zone.id,
 name: "admin",
 content: "1.2.3.4",
 ttl: 1,
 type: "A",
 proxied: true,
});

const wafRule = new cloudflare.Ruleset("waf-rule", {
 zoneId: zone.id,
 name: "Block suspicious traffic",
 description: "Custom WAF rule to block specific patterns",
 kind: "zone",
 phase: "http_request_firewall_custom",
 rules: [{
 action: "block",
 expression: `(http.request.uri.path contains "/admin" and not ip.src in {${trustedAdminIp}})`,
 description: "Block unauthorized admin access",
 }],
});

const worker = new cloudflare.WorkersScript("canary-worker", {
 accountId: accountId,
 scriptName: "edge-canary",
 compatibilityDate: "2026-01-01",
 mainModule: "worker.js",
 content: `
 export default {
 async fetch(request) {
 return new Response("Edge Baseline Validated", {
 headers: { "x-edge-baseline": "active" },
 });
 },
 };
 `,
});

const workerRoute = new cloudflare.WorkersRoute("canary-route", {
 zoneId: zone.id,
 pattern: "www.example.com/*",
 script: worker.scriptName,
});

const adminGroup = new cloudflare.ZeroTrustAccessGroup("admin-group", {
 accountId: accountId,
 name: "Administrators",
 includes: [{
 email: {
 email: "admin@example.com",
 },
 }],
});

const accessApp = new cloudflare.ZeroTrustAccessApplication("internal-tool", {
 accountId: accountId,
 name: "Internal Admin Tool",
 domain: "admin.example.com",
 type: "self_hosted",
 policies: [{
 name: "Allow administrators",
 precedence: 1,
 decision: "allow",
 includes: [{
 group: {
 id: adminGroup.id,
 },
 }],
 }],
});

Validation

Once you run pulumi up, you can validate your edge baseline with these steps:

DNS check: Run dig www.example.com to confirm the record points to Cloudflare’s proxied IPs.
WAF test: Attempt to access www.example.com/admin from an unauthorized IP and verify you receive a 403 Forbidden response.
Worker canary: Use curl -I https://www.example.com and check for the x-edge-baseline: active header.
Access policy: Navigate to admin.example.com and verify you are redirected to the Cloudflare Access login page.

By managing these resources as code, you eliminate the risk of manual drift and ensure that every environment starts with a secure, validated edge baseline.

Six Live Kubernetes Recommendations: AKS, Cilium, Rate Limiting, and More

Adam Gordon Bell — Mon, 15 Jun 2026 12:00:00 +0000

On June 10th, Engin and I ran a live workshop building an AKS cluster, an Azure Container Registry, and a random-cat web app from scratch in C#. This is the writeup, including the parts we didn’t get to live.

Live demos keep you honest. On June 10th my AKS workshop went a little sideways. Partway through, Docker Hub rate-limited my image pull and we had to adapt the content on the fly. The original plan was to stand up an AKS cluster with Cilium, an Azure Container Registry with the cluster’s pull permission wired in code, and a random-cat web app, then split the infrastructure from the workload into separate Pulumi stacks. Live, we didn’t make it through all of that, but we had some fun tangents and it turned out to be a great session. So here are my six recommendations for working with Kubernetes on Azure from this recent workshop.

1. Pick the language your team already uses

I opened the session by asking the room what language they prefer to work in, with one caveat from me: “Hopefully, it’s not PowerShell because this is not in PowerShell.” We got some C# answers and zero requests for PowerShell, which works out, because the whole workshop is in ~~PowerShell~~ C#.

I chose C# because, first, I genuinely love the language. I spent the early part of my career as a C# developer, and on Azure it just makes sense. Second, an Azure workshop tends to draw a big turnout of C# devs, and if they aren’t C# devs themselves, they often work somewhere the backend team uses C#. Pulumi lets you bring that language to your infrastructure rather than learn a new DSL.

AI coding assistants also came up. LLMs have substantially more C# and Go in their training data than HCL or Bicep. As I put it live: “If you’re working in HCL, well, there’s not that much out there in the training set, but there’s lots of C#. There’s lots of Go.” I co-authored parts of the workshop code with Claude.

Pick the language your team already writes.

Pulumi runs on the language your backend engineers know, and that language has more AI assistant training data behind it too.

Aside: everyone’s path ran through Docker Swarm

We also had some fun historical conversations about orchestrators. I asked Engin: “Did you use anything before Kubernetes became popular? Like Mesosphere or any of those other interesting solutions of the heyday?”

Engin had walked the same road most of us did.

Engin: “We used the classical Ansible powered provisioning of VMs… And then we switched to Consul to have a little bit more control… And then we had a short intermezzo with Docker Swarm… So Docker Swarm with Portainer was really goated. And Kubernetes then took over.”

My own history rhymed with his.

Adam: “I worked at a place where we used a bunch of shell scripts to have like JVM services, like a red green deploy… And then it was Docker Swarm for a little bit. And then it was Kubernetes.”

No single tool won. Teams picked what fit their existing skills and shifted gradually as the ecosystem matured.

2. Use Cilium on AKS if you can

By default, Kubernetes on AKS doesn’t turn on Cilium. My strong recommendation is to turn it on. The option is called Azure CNI Powered by Cilium, and it takes one config block:

NetworkProfile = new ACI.ContainerServiceNetworkProfileArgs
{
 NetworkDataplane = "cilium",
 NetworkPlugin = "azure",
 NetworkPluginMode = "overlay",
 NetworkPolicy = "cilium",
 PodCidr = "192.168.0.0/16",
},

This buys you eBPF networking instead of kube-proxy and iptables. eBPF lets you run sandboxed programs inside the Linux kernel itself, so packet handling and NetworkPolicy enforcement happen down in the kernel instead of up in userspace. You also get Hubble flow visibility built in, the same technology as GKE’s Dataplane V2. As I said live, “this is very easy to set up on Azure. You just network data plane, set it to Cilium. Network policy, set it to Cilium.”

Self-hosted, Cilium is helm install cilium with a dozen flags and a lifecycle you own forever. On AKS, it’s these five lines and Azure runs it. On a getting-started cluster, I’ll take the managed version every time. Owning that lifecycle yourself buys you nothing here.

Full code: 01-cluster/Program.cs

On managed Kubernetes, complexity you’d self-host becomes a field you set.

AKS gives you the managed version of eBPF networking. Pulumi makes the configuration code instead of portal clicks.

3. Debugging Kubernetes is easier when your tools have the full context

The rate limit wasn’t even the first failure of the hour. That honor went to Azure, which refused to provision the cluster on the first pulumi up. I was running in Canada Central, and Pulumi Neo read the error and diagnosed it:

Neo: “Capacity constraints in the selected region. We might need to change our location.”

Adam: “Maybe this Canadian Azure instance is not all it’s cracked up to be.”

Engin: “East US is always a good bet.”

We switched the region to East US, and the cluster came up. That is the kind of diagnosis a stack-aware assistant is good at, reading the actual provisioning error instead of leaving me to guess.

Then the image step hit. The workshop’s original flow pulled the cat app image into ACR with az acr import --source docker.io/agbell/my-random-cat, and Docker Hub rate-limited me, even though I was the only person in the room running the code. So how do you blow through an anonymous pull limit all by yourself? The trick is that az acr import runs server-side. Azure does the pull from Docker Hub, not my laptop, and it goes out over an egress IP shared by who-knows-how-many other Azure tenants. Docker Hub counts anonymous pulls per IP address, so all of that shared traffic lands in one bucket. It flagged me at the worst possible moment: “I just got flagged by Docker Hub. That is awesome. That will be a challenge for this demo.”

Nothing focuses a workshop like watching your own demo get rate-limited in front of a live audience.

Live, I asked Neo to route around it: “we’re using Neo. And Neo is going to switch us out to a nice Hello World project.” Neo could see the whole stack, down to the failing image reference, and it swapped the workload over to a standard hello-world image hosted on Azure, so the deploy no longer depended on Docker Hub. A few minutes later: “we should have a Hello World container running here. It’s way less fun than my image… just a bunch of random cats. But here we can see that this is working.” I won’t oversell it. The cat was gone, and we didn’t get through every planned stage. But the cluster was up with a workload deployed, and the session kept moving.

That’s the case for a stack-aware assistant over a chatbot. A general chatbot can tell you that Docker Hub has rate limits and that you should host images closer to home. Neo was working with the actual program state, so the workaround it proposed was already wired to my resources. Engin put it this way: “for me, Pulumi Neo, the subcommand is the new Pulumi app.” When something breaks mid-demo, the tool that can read your stack is the one that helps.

The long-term fix was switching to az acr build, building the image in ACR from source, so the app image never has to come from Docker Hub at all. Every stage in the repo now builds the cat image this way, from source shipped in each stage’s app/ folder:

az acr build --registry <your-acr> --image my-random-cat:latest app

No local Docker, and the app image never touches Docker Hub. The only Docker Hub dependency left is the base image (python:3.9-slim), pulled server-side by ACR’s build instead of by an import you re-run every time.

Full code: 02-app/Program.cs

An assistant that can read your stack beats one that can’t.

A general chatbot knows Docker Hub has rate limits. Neo could see the deployment and the failing image reference, and that context is what turned advice into a working fix mid-session.

4. You don’t have to abandon Terraform to start

One attendee, Martin, had been using Terraform for eight-plus years. His question, put directly: “I’m using TF for eight plus years. Why should I consider Pulumi? Where is the strength?… It seems unnecessary to TF for me. At least from what I’m seeing now.”

Fair question, and the honest answer is you don’t have to abandon your modules. Pulumi has first-class Terraform module support, so you can reference existing modules from a Pulumi program. Python ML teams can import infrastructure modules, and platform teams can wrap them in typed abstractions. The portfolio doesn’t change. The programs that use it just get more capable.

The sharper reason is about edges. HCL and Bicep are purpose-built DSLs, and when you hit what they can’t express, there’s no escape hatch inside the language. With a general-purpose language you can always drop to the SDK or shell out. The edges still exist. They’re just further out, and that’s the whole difference once a project gets past the basics.

You don’t have to rip and replace.

Pulumi can reference existing Terraform modules. Move what makes sense now. The DSL/language tradeoff shows up at the edges, not the basics.

5. Split slow infrastructure from fast workloads

Once we had the cluster up and the app running, the single program had grown: cluster, ACR, AcrPull, image build, deployment, service. Every new workload meant editing the file that also declares the cluster. We had planned this turn in the session in advance. I posed the problem and Engin had the recommendation ready:

Adam: “What do you think the problem is with my code here, Engin? … The problem is like every time I want to add a new service to my Kubernetes cluster, I have to go into this file that declares the cluster and all its infrastructure. I think I would prefer to split it.”

Engin: “Create other stacks which are then referencing each other to get the values out. Actually, I highly recommend.”

Adam: “Yes, I totally agree with you, Engin. We should split these out into two separate stacks.”

So that’s what we did. The infrastructure stack exports the kubeconfig and registry URL. The workload stack references those outputs via StackReference. The slow-moving infra and the fast-moving app become independent deployment units. The infra team and the app team can ship on their own cadences, without fighting over one file.

// workload/Program.cs — pulls cluster outputs, never touches cluster resources
var clusterStack = new StackReference(cfg.Require("clusterStack"));
var kubeconfig = clusterStack.GetOutput("kubeconfig");
var acrLoginServer = clusterStack.GetOutput("acrLoginServer");

The tricky part is one we didn’t cover live, which is how you split a running workload without taking down the running services. The real answer is migrating the stack state, moving resources between stacks instead of destroying and recreating them. As I noted in the session: “now we are just removing the service from our project. If this was actual production stuff, I don’t think I’d be just shutting down the service… You can actually just make changes to the state rather than tearing them down.”

For the workshop, a simpler move kept the Kubernetes cluster up and the demo moving: keep the project name. Pulumi identifies stacks as org/project/stack, so if you move the cluster code into aks-cluster/ while keeping the project name kube-kitties, Pulumi still sees org/kube-kitties/dev. Same stack, nothing recreated. Rename the project and you get a second cluster.

In stage four of the demo, I showed how to extend this pattern to work with existing YAML manifests: ConfigGroup drives raw Kubernetes YAML through the Pulumi provider, giving you previews and dependency ordering without rewriting files. One transformation swaps the Docker Hub image reference for the ACR copy, so manifests stay portable and pods pull from your private registry.

Code: 03-split/ and 04-split-yaml/

Keep the project name when you split, and the cluster doesn’t move.

Pulumi identifies stacks as org/project/stack. Same project name means the same stack. Rename it and you spin up a new cluster.

6. GitOps is the way

The final stage of the workshop is left as homework, and it’s a great one to pick up. GitOps is the standard way to run Kubernetes at this point, the One True Way™ depending on who you ask. My position from the call: “I think I’m on team GitOps.”

The take-home is the 05-gitops folder. We didn’t demo it live, but the code is real and verified: Pulumi stands up the cluster, installs Argo CD, and registers the cat as an Argo Application. From there, changes are git push.

To see Kubernetes and GitOps worked through end to end, watch the video above. Engin also recently ran a GCP workshop with great examples of GitOps with Pulumi, using Flux.

Let Pulumi stand up the cluster, then let GitOps run the apps.

Pulumi installs Argo CD or Flux as part of the infrastructure. From there, workload changes are a git push. Stage 5 of the workshop repo is a complete Argo CD example.

Try it yourself

That’s the whole workshop, six recommendations and two live failures that ended with a cat app running on AKS. To run it yourself, clone the repo. There’s a DEMO.md with every step of this session, and the stage folders mirror the arc above. 01-cluster is the infrastructure checkpoint, 02-app adds the cat app built into your own registry, 03-split and 04-split-yaml separate infrastructure from workload, and 05-gitops is the homework.

Pick a stage and bring it up:

git clone https://github.com/pulumi/workshops
cd workshops/az-getting-started-aks/02-app
pulumi up
curl http://$(pulumi stack output catServiceIp)/

Expect 5 to 8 minutes waiting on cluster create. That’s just how long AKS takes. And since each folder is a complete, runnable checkpoint with its own stack, run pulumi destroy before moving to the next stage, or you’ll have several clusters billing at once.

You’ll need an Azure subscription, the Pulumi CLI, and the .NET SDK. Basic Kubernetes familiarity helps. For more on the building blocks, see the Azure Native provider and the Kubernetes provider in the Pulumi Registry.

The full workshop code lives at github.com/pulumi/workshops under az-getting-started-aks. Check out our upcoming sessions on the workshop page, or subscribe to the newsletter to hear what’s coming up.

If there’s a seventh recommendation hiding in here, it’s the one I learned the hard way: never pull from Docker Hub live again. I’ll be keeping my images on a paid cloud registry from now on.

Engin and I will be doing more of these, and we’d love to see you at the next one.

Parts of the workshop code were authored by Claude and Pulumi Neo, and parts of this writeup got help from both as well.

Use Your Mac for AI Agents: Self-Host Gemma 4 12 B with Pulumi and Tailscale

Pablo Seibelt — Thu, 04 Jun 2026 00:00:00 +0000

If you run AI tools and agents, you’ve probably accepted three tradeoffs: your data leaves your network, you can’t work offline, and your bill scales with usage.

Open-weight models now run well on consumer hardware. Once the model is on your machine, your data stays local, inference works offline, and tokens cost nothing. If you own a modern Mac, you can run a high-quality model yourself.

Gemma 4 is an open-weights model family from Google. This post focuses on Gemma 4 12 B, released in June 2026, using Unsloth’s Q8_0 GGUF. The 12 B model fits comfortably on a modern Mac while leaving enough headroom for local llama.cpp and a chat UI.

We’ll use llama.cpp for host-native inference, k3d for a local Kubernetes cluster, Pulumi for infrastructure as code, and Tailscale for secure access.

Prerequisites

This setup was validated on the following hardware:

macOS 26 Tahoe, version 26.5
MacBook Pro with Apple M3 Max
36 GB RAM

On this machine, llama.cpp reported about 20 output tokens per second for a 160-token validation response with unsloth/gemma-4-12b-it-GGUF, gemma-4-12b-it-Q8_0.gguf, and a 131,072-token context. Sustained throughput varies by prompt length, thermal state, and llama.cpp settings.

You’ll need brew, docker, pulumi, and tailscale installed. We’ll also install k3d during the process.

Run Gemma 4 with host-native llama.cpp

We use llama.cpp directly on macOS to leverage Apple Metal acceleration. Running the LLM on the host is more efficient than trying to pass GPU access into a local Kubernetes VM.

Install the build tools:

brew install cmake git

Then build llama.cpp from source and download the multimodal projector. In validation, Homebrew llama.cpp 9430 could run text inference, but it could not load the new Gemma 4 12 B projector and failed with unknown projector type: gemma4uv. Building current llama.cpp from source fixed that.

llm_home="$HOME/pulumi-gemma4-llm"
mkdir -p "$llm_home/models" "$llm_home/logs"

if [ ! -d "$llm_home/llama.cpp/.git" ]; then
 git clone --depth 1 https://github.com/ggml-org/llama.cpp.git "$llm_home/llama.cpp"
fi

cmake -S "$llm_home/llama.cpp" \
 -B "$llm_home/llama.cpp/build" \
 -DGGML_METAL=ON \
 -DGGML_BLAS=ON \
 -DCMAKE_BUILD_TYPE=Release

cmake --build "$llm_home/llama.cpp/build" --target llama-server -j 10

curl -L --fail \
 --output "$llm_home/models/mmproj-F16.gguf" \
 https://huggingface.co/unsloth/gemma-4-12b-it-GGUF/resolve/main/mmproj-F16.gguf

Then download and run the model with this command:

"$HOME/pulumi-gemma4-llm/llama.cpp/build/bin/llama-server" \
 --hf-repo unsloth/gemma-4-12b-it-GGUF \
 --hf-file gemma-4-12b-it-Q8_0.gguf \
 --mmproj "$HOME/pulumi-gemma4-llm/models/mmproj-F16.gguf" \
 --host 127.0.0.1 \
 --port 18080 \
 --ctx-size 131072 \
 --parallel 1 \
 --jinja \
 --reasoning off

We use port 18080 because 8080 is commonly used and is likely to conflict with another service you may already have running locally. If your port 8080 is free, you can use it and adjust the Pulumi config later.

The model file is about 12.65 GB, and the projector is about 116 MB. Gemma 4 12 B advertises a 131,072-token context, and this Mac loaded that full context with --parallel 1. llama.cpp projected about 15.1 GiB of Apple Metal device memory for the text model and about 258 MiB worst-case memory for the projector, leaving enough headroom for Open WebUI and the rest of the local stack. The --reasoning off flag keeps OpenAI-compatible chat responses visible in clients that do not read separate reasoning fields.

With --mmproj, /v1/models advertised capabilities: ["completion","multimodal"]. In local validation, Open WebUI accepted an uploaded Pulumi logo image and Gemma 4 described it correctly. A small WAV file also worked through the OpenAI-compatible input_audio request shape, though llama.cpp logs still mark audio input as experimental.

Verify the LLM API

Open a new terminal and check if llama.cpp is responding:

curl http://127.0.0.1:18080/v1/models

The /v1/models endpoint should return the model ID unsloth/gemma-4-12b-it-GGUF. Now try a chat completion:

curl http://127.0.0.1:18080/v1/chat/completions \
 -H "Content-Type: application/json" \
 -d '{
 "model": "unsloth/gemma-4-12b-it-GGUF",
 "messages": [{"role": "user", "content": "Reply with exactly: OK"}],
 "temperature": 0,
 "max_tokens": 32
 }'

The chat prompt Reply with exactly: OK should return content OK. In validation, llama.cpp reported an output token velocity of about 20 tokens per second for a longer 160-token response.

Keep llama.cpp running after reboot

For a permanent setup, put the llama.cpp startup script and logs under a folder in your home directory and let launchd restart it when you sign in:

llm_home="$HOME/pulumi-gemma4-llm"
mkdir -p "$llm_home/logs" "$HOME/Library/LaunchAgents"

cat > "$llm_home/start-llama-server.sh" <<'EOF'
#!/bin/zsh
set -euo pipefail

export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"

exec "$HOME/pulumi-gemma4-llm/llama.cpp/build/bin/llama-server" \
 --hf-repo unsloth/gemma-4-12b-it-GGUF \
 --hf-file gemma-4-12b-it-Q8_0.gguf \
 --mmproj "$HOME/pulumi-gemma4-llm/models/mmproj-F16.gguf" \
 --host 127.0.0.1 \
 --port 18080 \
 --ctx-size 131072 \
 --parallel 1 \
 --jinja \
 --reasoning off
EOF

chmod +x "$llm_home/start-llama-server.sh"

cat > "$HOME/Library/LaunchAgents/com.pulumi.gemma4.llama-server.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
 <key>Label</key>
 <string>com.pulumi.gemma4.llama-server</string>
 <key>ProgramArguments</key>
 <array>
 <string>$llm_home/start-llama-server.sh</string>
 </array>
 <key>WorkingDirectory</key>
 <string>$llm_home</string>
 <key>RunAtLoad</key>
 <true/>
 <key>KeepAlive</key>
 <true/>
 <key>StandardOutPath</key>
 <string>$llm_home/logs/llama-server.out.log</string>
 <key>StandardErrorPath</key>
 <string>$llm_home/logs/llama-server.err.log</string>
</dict>
</plist>
EOF

launchctl bootout gui/$(id -u)/com.pulumi.gemma4.llama-server 2>/dev/null || true
launchctl bootstrap gui/$(id -u) "$HOME/Library/LaunchAgents/com.pulumi.gemma4.llama-server.plist"
launchctl kickstart -k gui/$(id -u)/com.pulumi.gemma4.llama-server

Check the launchd service and llama.cpp logs:

launchctl print gui/$(id -u)/com.pulumi.gemma4.llama-server
tail -f "$HOME/pulumi-gemma4-llm/logs/llama-server.err.log"

If you want to stop llama.cpp later, unload the launchd service:

launchctl bootout gui/$(id -u)/com.pulumi.gemma4.llama-server

Deploy Open WebUI with Pulumi and k3d

Now we’ll deploy Open WebUI into a local Kubernetes cluster. This provides a polished chat interface that connects to our host-native LLM.

First, install k3d if you haven’t already:

brew install k3d

Create a new cluster for this project:

k3d cluster create pulumi-gemma4-blog-qa

We’ll use the Pulumi program in pulumi/examples. This program defaults to runtimeMode=host, which creates a Kubernetes ExternalName service pointing to your host machine.

Why not run the LLM inside Kubernetes on this Mac? Pulumi can do that, and the example supports it with runtimeMode=cluster, but that path is meant for Linux hosts with NVIDIA or AMD GPU device plugins.

On macOS, llama.cpp enables Metal by default, and Metal acceleration is available to native macOS processes. k3d runs Linux containers through Docker Desktop, so those pods do not get direct access to the Mac’s Metal device. Docker’s own vLLM Metal announcement calls out the same boundary: Metal-backed inference runs natively on the host because there is no Metal GPU passthrough for containers. That is why this setup keeps inference host-native and lets Pulumi manage the Kubernetes UI, service wiring, and optional Tailscale access around it.

Clone the examples repo, navigate to the program directory, and initialize a new stack:

git clone https://github.com/pulumi/examples.git
cd examples/kubernetes-py-self-host-gemma4-llm
pulumi stack init gemma4-local

Configure the program to match your local setup:

pulumi config set hostLlmPort 18080
pulumi config set llmBaseUrl http://llm-server:18080/v1

The Kubernetes service named llm-server maps to host.k3d.internal. In our validation, we confirmed that a disposable k3d pod could reach the Mac’s llama.cpp API at http://llm-server:18080/v1/models after a CoreDNS restart.

kubectl rollout restart deployment coredns -n kube-system

Run pulumi up to deploy Open WebUI and connect it to host-native llama.cpp:

pulumi up

In our validation environment, this command successfully reached the resource synthesis phase without requiring Tailscale credentials because Tailscale exposure is opt-in.

Access Open WebUI through Tailscale

Tailscale allows you to access your private Open WebUI instance from any device on your tailnet. Note that we only expose the web interface, not the raw LLM API, to keep the system secure.

The base Open WebUI deployment works without Tailscale credentials. To expose the web UI on your tailnet, enable Tailscale resources and provide an explicit api_key or OAuth/identity token:

pulumi config set enableTailscale true
pulumi config set tailscale:apiKey YOUR_API_KEY --secret

Once configured, Pulumi will create a Tailscale device or proxy that routes traffic to your Open WebUI service.

Use the model with Pi

Open WebUI gives you a browser-based chat interface, but local models are also useful from coding agents. Pi is the local coding agent used for this validation; if you do not use Pi, treat this section as an example of how any OpenAI-compatible client can point at the same local endpoint. Pi can point at the same OpenAI-compatible llama.cpp endpoint and use the model running on your Mac.

For a fresh Pi config, create ~/.pi/agent/models.json with a local provider that points at the llama.cpp API:

{
 "providers": {
 "local-llama": {
 "baseUrl": "http://127.0.0.1:18080/v1",
 "api": "openai-completions",
 "apiKey": "local",
 "compat": {
 "supportsDeveloperRole": false,
 "supportsReasoningEffort": false
 },
 "models": [
 {
 "id": "unsloth/gemma-4-12b-it-GGUF",
 "name": "Gemma 4 12B Q8 (local llama.cpp)",
 "reasoning": false,
 "input": ["text"],
 "contextWindow": 131072,
 "maxTokens": 1024,
 "cost": {
 "input": 0,
 "output": 0,
 "cacheRead": 0,
 "cacheWrite": 0
 }
 }
 ]
 }
 }
}

Then set Pi to use that provider and model by default in ~/.pi/agent/settings.json:

{
 "defaultProvider": "local-llama",
 "defaultModel": "unsloth/gemma-4-12b-it-GGUF",
 "defaultThinkingLevel": "off",
 "hideThinkingBlock": true
}

If you already have Pi configuration files, merge the local-llama provider and defaults into your existing JSON instead of replacing the files.

Advanced: Linux GPU in-cluster serving

If you’re running on a Linux host with an NVIDIA or AMD GPU, you can run the LLM directly inside the Kubernetes cluster. This requires the NVIDIA or ROCm device plugins.

The Pulumi program supports this through runtimeMode=cluster. In this mode, it deploys a LlmServer pod that manages the llama.cpp process within the cluster, using GPU resource requests to ensure hardware acceleration.

Cleanup

When you’re done, you can tear down the resources:

pulumi destroy
k3d cluster delete pulumi-gemma4-blog-qa
# Stop llama.cpp using the PID from your terminal
kill <PID>

Build an EKS Environment Factory with Pulumi and vCluster

Pablo Seibelt — Thu, 04 Jun 2026 00:00:00 +0000

AWS reports in an AWS Architecture Blog case study that Deloitte’s move to a virtual cluster model on Amazon EKS resulted in 89% faster testing environment provisioning. By consolidating dozens of disparate clusters into a single host cluster with over 50 vCluster instances, the case study says Deloitte saved about 500 QA hours per year. This “Environment Factory” pattern allows platform teams to provide isolated, ephemeral Kubernetes environments on demand without the cost or lag of full cluster provisioning.

This post adapts that general architecture with Pulumi to orchestrate Amazon EKS Auto Mode and vCluster.

The problem: environment sprawl and provisioning lag

Traditional development workflows often rely on one full EKS cluster per developer or feature branch. While this provides strong isolation, it introduces major pain points. Provisioning a full cluster can take 15 minutes or more, which slows down CI/CD pipelines. Managing dozens of clusters also leads to high costs and significant operational overhead.

Platform teams need a “soft multi-tenancy” model. This model should feel like a dedicated cluster to the developer but run on shared infrastructure to keep costs low and startup times fast.

Architecture overview: the host and the tenants

The environment factory architecture consists of two main layers.

Host cluster: A single, reliable EKS cluster managed with EKS Auto Mode. This cluster provides the underlying compute, networking, and storage.
Tenant environments: Virtual clusters (vCluster) running as pods within host namespaces.

According to the vCluster architecture, the virtual control plane handles API requests while a syncer maps virtual resources to the host cluster. This separation allows tenants to manage their own CRDs, namespaces, and RBAC while platform teams use quotas, NetworkPolicies, pod security, IAM boundaries, and node isolation controls to protect the host and other tenants.

Implementation: the EKS Auto Mode host

EKS Auto Mode simplifies the host cluster by automating infrastructure management. It handles node provisioning, scaling, and updates based on pod requirements.

The following snippet shows how to define an EKS cluster with Auto Mode enabled using Pulumi.

import * as awsx from "@pulumi/awsx";
import * as eks from "@pulumi/eks";
import * as k8s from "@pulumi/kubernetes";
import { SubnetType } from "@pulumi/awsx/ec2";

const clusterName = "environment-factory";

const vpc = new awsx.ec2.Vpc("environment-factory", {
 enableDnsHostnames: true,
 cidrBlock: "10.0.0.0/16",
 subnetSpecs: [
 {
 type: SubnetType.Public,
 tags: {
 [`kubernetes.io/cluster/${clusterName}`]: "shared",
 "kubernetes.io/role/elb": "1",
 },
 },
 {
 type: SubnetType.Private,
 tags: {
 [`kubernetes.io/cluster/${clusterName}`]: "shared",
 "kubernetes.io/role/internal-elb": "1",
 },
 },
 ],
 subnetStrategy: "Auto",
});

// Create an EKS cluster with Auto Mode enabled.
const hostCluster = new eks.Cluster("host-cluster", {
 name: clusterName,
 authenticationMode: eks.AuthenticationMode.Api, // Use API authentication mode for EKS access entries.
 vpcId: vpc.vpcId,
 publicSubnetIds: vpc.publicSubnetIds,
 privateSubnetIds: vpc.privateSubnetIds,
 autoMode: {
 enabled: true,
 },
});

const hostProvider = new k8s.Provider("host-provider", {
 kubeconfig: hostCluster.kubeconfig,
});

Implementation: the environment factory

Once the host cluster is ready, we can build the factory that stamps out tenant environments. Each tenant needs a dedicated namespace, resource quotas, and the vCluster itself.

Tenant guardrails

Before installing vCluster, we set up a namespace and resource quotas to ensure one tenant cannot consume all host resources.

import * as k8s from "@pulumi/kubernetes";

// Define a tenant namespace on the host cluster.
const tenantNamespace = new k8s.core.v1.Namespace("tenant-alpha", {
 metadata: { name: "tenant-alpha" },
}, { provider: hostProvider });

// Apply resource quotas to the tenant namespace.
const quota = new k8s.core.v1.ResourceQuota("tenant-quota", {
 metadata: { namespace: tenantNamespace.metadata.name },
 spec: {
 hard: {
 pods: "20",
 "requests.cpu": "4",
 "requests.memory": "8Gi",
 "limits.cpu": "8",
 "limits.memory": "16Gi",
 },
 },
}, { provider: hostProvider });

// Define a Role for the tenant within their namespace.
const tenantRole = new k8s.rbac.v1.Role("tenant-role", {
 metadata: { namespace: tenantNamespace.metadata.name },
 rules: [{
 apiGroups: [""],
 resources: ["pods", "services", "configmaps", "secrets"],
 verbs: ["get", "list", "watch", "create", "update", "patch", "delete"],
 }],
}, { provider: hostProvider });

// Bind the Role to a tenant user or group.
const tenantRoleBinding = new k8s.rbac.v1.RoleBinding("tenant-role-binding", {
 metadata: { namespace: tenantNamespace.metadata.name },
 subjects: [{
 kind: "User",
 // Replace "tenant-user" with the IAM-mapped user or group for this tenant.
 name: "tenant-user",
 apiGroup: "rbac.authorization.k8s.io",
 }],
 roleRef: {
 kind: "Role",
 name: tenantRole.metadata.name,
 apiGroup: "rbac.authorization.k8s.io",
 },
}, { provider: hostProvider });

For production use, map these Kubernetes identities to IAM principals using EKS Access Entries, with the legacy aws-auth ConfigMap still appearing in older clusters.

Deploying vCluster with Helm

We use the kubernetes.helm.v3.Release resource to install vCluster. This resource provides controlled Helm lifecycle management for the vCluster release. The values block should be adjusted for each tenant profile to control resource synchronization and control plane behavior. Review the vCluster release notes when changing chart versions because values schema and generated secret names can change across releases.

import * as k8s from "@pulumi/kubernetes";

// Install vCluster using the Helm Release resource.
const vcluster = new k8s.helm.v3.Release("vcluster-alpha", {
 name: "vcluster-alpha",
 chart: "vcluster",
 version: "0.20.0", // Tested with vCluster 0.20.x; review release notes before changing versions.
 repositoryOpts: {
 repo: "https://charts.loft.sh",
 },
 namespace: tenantNamespace.metadata.name,
 values: {
 // Explicit sync configuration; adjust per tenant profile.
 sync: {
 toHost: {
 pods: { enabled: true },
 },
 },
 },
}, { provider: hostProvider });

Accessing the virtual cluster

The vCluster generates a kubeconfig that allows developers to interact with the virtual API server. We must treat this kubeconfig as a secret, and the endpoint in that kubeconfig must be reachable from the Pulumi runner before using it to create resources inside the virtual cluster.

import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

// Retrieve the vCluster kubeconfig from the generated secret.
// The vCluster-generated secret can lag behind Helm release readiness on first creation,
// so teams may choose an explicit readiness check or rerun after the virtual control plane initializes.
const vclusterKubeconfig = k8s.core.v1.Secret.get("vcluster-secret",
 pulumi.interpolate`${tenantNamespace.metadata.name}/vc-vcluster-alpha`,
 {
 provider: hostProvider,
 dependsOn: [vcluster],
 }
).data.apply(data => Buffer.from(data["config"], "base64").toString());

// Export the kubeconfig as a secret.
export const tenantKubeconfig = pulumi.secret(vclusterKubeconfig);

// Create a provider for the virtual cluster using the secret kubeconfig.
const vclusterProvider = new k8s.Provider("vcluster-provider", {
 kubeconfig: tenantKubeconfig,
});

Operational caveats

RBAC and permissions: vCluster generates default RBAC rules that work for most scenarios. However, if your host cluster is heavily locked down, you may need to provide additional permissions to the vCluster service account.
Helm release previews: When using kubernetes.helm.v3.Release, Pulumi previews may not show every detail of the rendered Kubernetes resources. It primarily tracks the state of the Helm release itself.
EKS Auto Mode node lifetime: EKS Auto Mode uses immutable AMIs and has a 21-day node lifetime. Kubernetes reschedules vCluster pods and tenant workloads when nodes are replaced, so configure replicas, PodDisruptionBudgets, requests, and persistent storage for disruption tolerance.

Conclusion: ephemeral environments at scale

By combining Pulumi with EKS Auto Mode and vCluster, you can build a scalable environment factory. This approach provides the isolation developers need while maintaining the speed and cost-efficiency required by platform teams.

The snippets provided here are adapted for illustration. In a production environment, you would likely wrap these resources into a Pulumi ComponentResource to provide a clean, reusable API for your internal developers. When a feature branch is merged, deleting the Pulumi stack removes the resources managed by that stack, but validate namespace finalizers, persistent volume reclaim policies, and external cloud artifacts as part of cleanup.

For more on managing EKS with Pulumi, see the EKS guide.

How We Load Data into Snowflake in Seconds with Pulumi

Pablo Seibelt — Mon, 23 Feb 2026 00:00:00 +0000

When you manage dozens of data-loading pipelines, copying and pasting IaC configurations between them is a recipe for mishap. IAM policies can drift, naming conventions diverge, and every new source is a new opportunity to make a mistake — not to mention compound the problem of duplication. In this post, we’ll show you how you can identify and encapsulate common patterns into composable components and walk through the production lessons we’ve learned running 25+ pipelines for over three years.

What we’ll cover

If you’re loading data into Snowflake and want reusable, composable infrastructure, this post is for you. Here’s what we’ll cover:

Handling and validating GitHub webhooks with AWS Lambda
Streaming webhook payloads directly into Snowflake with Amazon Data Firehose
Wiring it all up with a reusable Pulumi ComponentResource

The companion template also includes S3 auto-ingest and batch loading patterns, which we’ll cover in upcoming posts. We also use Pulumi ESC to handle authentication to both AWS and Snowflake using OpenID Connect.

Our own Josh Kodroff wrote an excellent introduction to Snowpipe with Pulumi. This post builds on his work using the newest Snowflake and AWS provider APIs and the direct Firehose-to-Snowflake destination, which wasn’t available when Josh wrote his post. Some resource names and grant patterns will also differ if you’re comparing the two.

Architecture overview

The following diagram shows the architecture in more detail:

GitHub sends webhook events to a Lambda Function URL.
Lambda validates the HMAC (Hash-based Message Authentication Code) signature and forwards the payload to Amazon Data Firehose.
Firehose streams records directly into Snowflake via the Snowpipe Streaming API. Data appears in Snowflake within seconds.
S3 is used only as a backup destination for failed records.

The direct Firehose-to-Snowflake destination is an AWS-native feature that works with any Snowflake account.

Project setup

Start a new Pulumi Python project and choose uv for dependency management when prompted:

mkdir snowpipe-data-loading && cd snowpipe-data-loading
pulumi new python

Notice Pulumi.yaml shows uv as your selected toolchain:

name: snowpipe-data-loading
runtime:
 name: python
 options:
 toolchain: uv

Add the provider dependencies for AWS, Snowflake, GitHub, and the Random and TLS providers:

uv add pulumi-aws pulumi-snowflake pulumi-github pulumi-random pulumi-tls

That’s it. uv creates the virtual environment and lockfile automatically, and Pulumi uses uv run under the hood to execute your program.

All examples in this post are in Python, but Pulumi supports multiple languages. You can implement the same components in TypeScript, Go, .NET, Java, or YAML.

Managing credentials with Pulumi ESC

This project needs credentials for AWS, Snowflake, and GitHub. Rather than managing long-lived secrets locally, we can use Pulumi ESC to obtain dynamic, short-lived OIDC credentials for AWS and Snowflake at runtime. When you run pulumi up, ESC exchanges a Pulumi-issued OIDC token for temporary credentials from each provider and injects them into your stack config automatically. If you prefer not to use ESC, you can set credentials directly with pulumi config set --secret.

Here is a single ESC environment that handles all three providers:

values:
 aws:
 login:
 fn::open::aws-login:
 oidc:
 duration: 1h
 roleArn: arn:aws:iam::123456789012:role/pulumi-esc-oidc
 sessionName: pulumi-snowpipe-demo
 snowflake:
 login:
 fn::open::snowflake-login:
 oidc:
 account: <your-snowflake-account>
 user: ESC_SERVICE_USER
 organizationName: <your-org-name>
 accountName: <your-account-name>
 environmentVariables:
 AWS_ACCESS_KEY_ID: ${aws.login.accessKeyId}
 AWS_SECRET_ACCESS_KEY: ${aws.login.secretAccessKey}
 AWS_SESSION_TOKEN: ${aws.login.sessionToken}
 SNOWFLAKE_USER: ${snowflake.login.user}
 SNOWFLAKE_TOKEN: ${snowflake.login.token}
 pulumiConfig:
 aws:region: us-west-2
 snowflake:organizationName: ${snowflake.organizationName}
 snowflake:accountName: ${snowflake.accountName}
 snowflake:authenticator: OAUTH
 snowflake:role: PULUMI_DEPLOYER
 github:token:
 fn::secret: <your-github-pat>
 github:owner: <your-github-org-or-user>

Then reference the environment from your stack config:

# Pulumi.<stack>.yaml
environment:
 - <project>/my-snowpipe-env
config:
 snowpipe-data-loading:database: LANDING_ZONE_WEBHOOKS
 snowpipe-data-loading:environment: dev
 snowpipe-data-loading:webhook-repo: <your-test-repo-name>
 snowflake:previewFeaturesEnabled:
 - snowflake_table_resource

Depending on your preferences, you can split credentials into separate per-provider environments and compose them with imports and reuse across stacks.

To set up OIDC trust for each provider, see the AWS OIDC guide and the Snowflake OIDC login guide. For GitHub authentication options (fine-grained PATs, classic PATs, or GitHub Apps), see the pulumi-github provider docs.

Shared infrastructure

The direct streaming pipeline needs an S3 bucket for backup/errors, a Snowflake database, and a schema.

Building the direct ingestion ComponentResource

Amazon Data Firehose supports Snowflake as a native destination via the Snowpipe Streaming API. Firehose streams records directly into Snowflake.

The Lambda handler

The Lambda function is the entry point for GitHub webhooks. It validates the HMAC-SHA256 signature, wraps the payload in an envelope with the event type, and forwards it to Firehose. Create this as lambda/webhook_handler.py:

import hashlib
import hmac
import json
import os

import boto3

firehose = boto3.client("firehose")

STREAM_NAME = os.environ["FIREHOSE_STREAM_NAME"]
WEBHOOK_SECRET = os.environ["WEBHOOK_SECRET"]


def handler(event, context):
 body = event.get("body", "")
 signature = (event.get("headers") or {}).get("x-hub-signature-256", "")

 # Validate HMAC-SHA256 signature
 expected = "sha256=" + hmac.new(
 WEBHOOK_SECRET.encode(), body.encode(), hashlib.sha256
 ).hexdigest()

 if not hmac.compare_digest(expected, signature):
 return {"statusCode": 401, "body": "Invalid signature"}

 github_event = (event.get("headers") or {}).get("x-github-event", "unknown")

 # Wrap in envelope - newline-delimited for S3 backup where Firehose concatenates records
 record = json.dumps({
 "github_event": github_event,
 "payload": json.loads(body),
 }) + "\n"

 firehose.put_record(
 DeliveryStreamName=STREAM_NAME,
 Record={"Data": record.encode()},
 )

 return {"statusCode": 200, "body": "OK"}

The envelope format {"github_event": "<type>", "payload": {...}}\n is important. The github_event field (e.g., push, pull_request, star) comes from the x-github-event header and lets downstream queries filter by event type. The trailing newline delimits records in the S3 backup destination, where Firehose concatenates them into files.

Another interesting part of this snippet is that instead of manually creating a secret which would have to be copy-pasted into both your webhook configuration and your Lambda environment, we are using random.RandomPassword to generate and store it securely it in Pulumi state.

The secret is automatically wired to both the Lambda env var and the GitHub webhook config, and it rotates cleanly if you ever need to replace it.

Why direct Firehose to Snowflake?

Data like this is normally written to and loaded from Amazon S3. But with an S3 intermediate path, you must wait for Firehose to buffer records (60 seconds), then for Snowpipe to detect the new file and load it. Total latency: about two minutes. With the direct Snowflake destination, Firehose uses the Snowpipe Streaming API to insert records as soon as they arrive, in seconds.

The direct path also removes the need for S3 event notifications, SQS queues, external stages, and pipe resources. S3 is still used, but only as a backup destination for failed records.

The DirectSnowflakeIngestion component

The component creates everything needed for the direct path: a TLS key pair for Snowflake authentication, a Snowflake service user with least-privilege grants, the landing table, a Firehose delivery stream with destination="snowflake", and a Lambda function with a public URL. Create an empty components/__init__.py so that Python treats the directory as a package.

The full component lives in components/direct_snowflake_ingestion.py:

import json
from dataclasses import dataclass

import pulumi
import pulumi_aws as aws
import pulumi_snowflake as snowflake
import pulumi_tls as tls

# In the example repository, you will find this class imported from a common library file instead
@dataclass
class ColumnDef:
 """Column definition for a Snowflake table."""

 name: str
 type: str
 nullable: bool = True

def strip_pem_headers(pem: str) -> str:
 """Remove PEM header/footer lines, returning only the base64 content."""
 lines = pem.strip().split("\n")
 return "".join(lines[1:-1])


@dataclass
class DirectSnowflakeIngestionArgs:
 bucket_arn: pulumi.Input[str]
 bucket_name: pulumi.Input[str]
 database: pulumi.Input[str]
 schema_name: pulumi.Input[str]
 table_name: str
 table_columns: list[ColumnDef]
 snowflake_account_url: pulumi.Input[str]
 snowflake_role_name: str
 lambda_code: pulumi.Archive
 lambda_handler: str
 lambda_environment: dict[str, pulumi.Input[str]]
 table_comment: str = ""
 s3_prefix: str = "direct-webhooks"
 s3_backup_mode: str = "FailedDataOnly"
 buffering_interval: int = 0
 buffering_size: int = 1
 retry_duration: int = 60
 data_loading_option: str = "VARIANT_CONTENT_AND_METADATA_MAPPING"
 content_column_name: str = "CONTENT"
 metadata_column_name: str = "METADATA"

class DirectSnowflakeIngestion(pulumi.ComponentResource):
 function_url: pulumi.Output[str]
 firehose_stream_name: pulumi.Output[str]
 snowflake_user_name: pulumi.Output[str]

 def __init__(
 self,
 name: str,
 args: DirectSnowflakeIngestionArgs,
 opts: pulumi.ResourceOptions | None = None,
 ):
 super().__init__(
 "snowpipe:direct:DirectSnowflakeIngestion", name, {}, opts
 )

 # --- TLS key pair for Snowflake auth ---
 key_pair = tls.PrivateKey(
 f"{name}-keypair",
 algorithm="RSA",
 rsa_bits=2048,
 opts=pulumi.ResourceOptions(parent=self),
 )

 # --- Snowflake role, user, and grants ---
 sf_role = snowflake.AccountRole(
 f"{name}-sf-role",
 name=args.snowflake_role_name,
 opts=pulumi.ResourceOptions(parent=self),
 )

 user_name = f"FIREHOSE_{name.upper().replace('-', '_')}_USER"
 sf_user = snowflake.ServiceUser(
 f"{name}-sf-user",
 name=user_name,
 login_name=user_name,
 default_role=sf_role.name,
 rsa_public_key=key_pair.public_key_pem.apply(strip_pem_headers),
 opts=pulumi.ResourceOptions(parent=self),
 )

 # Landing table
 table = snowflake.Table(
 f"{name}-table",
 name=args.table_name,
 database=args.database,
 schema=args.schema_name,
 comment=args.table_comment,
 columns=[
 snowflake.TableColumnArgs(
 name=col.name, type=col.type, nullable=col.nullable,
 )
 for col in args.table_columns
 ],
 opts=pulumi.ResourceOptions(parent=self),
 )

 # Grants: DB USAGE, schema USAGE, table INSERT+SELECT
 snowflake.GrantPrivilegesToAccountRole(
 f"{name}-grant-db-usage",
 account_role_name=sf_role.name,
 privileges=["USAGE"],
 on_account_object=snowflake.GrantPrivilegesToAccountRoleOnAccountObjectArgs(
 object_type="DATABASE",
 object_name=args.database,
 ),
 opts=pulumi.ResourceOptions(parent=self),
 )

 snowflake.GrantPrivilegesToAccountRole(
 f"{name}-grant-schema-usage",
 account_role_name=sf_role.name,
 privileges=["USAGE"],
 on_schema=snowflake.GrantPrivilegesToAccountRoleOnSchemaArgs(
 schema_name=pulumi.Output.all(
 args.database, args.schema_name
 ).apply(lambda parts: f'"{parts[0]}"."{parts[1]}"'),
 ),
 opts=pulumi.ResourceOptions(parent=self),
 )

 table_name = args.table_name
 snowflake.GrantPrivilegesToAccountRole(
 f"{name}-grant-table",
 account_role_name=sf_role.name,
 privileges=["INSERT", "SELECT"],
 on_schema_object=snowflake.GrantPrivilegesToAccountRoleOnSchemaObjectArgs(
 object_type="TABLE",
 object_name=pulumi.Output.all(
 args.database, args.schema_name
 ).apply(
 lambda parts: f'"{parts[0]}"."{parts[1]}"."{table_name}"'
 ),
 ),
 opts=pulumi.ResourceOptions(parent=self, depends_on=[table]),
 )

 # --- Firehose IAM role (S3 backup write) ---
 firehose_role = aws.iam.Role(
 f"{name}-firehose-role",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Action": "sts:AssumeRole",
 "Principal": {"Service": "firehose.amazonaws.com"},
 }],
 }),
 opts=pulumi.ResourceOptions(parent=self),
 )

 aws.iam.RolePolicy(
 f"{name}-firehose-s3-policy",
 role=firehose_role.id,
 policy=args.bucket_arn.apply(
 lambda arn: json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Action": [
 "s3:AbortMultipartUpload",
 "s3:GetBucketLocation",
 "s3:GetObject",
 "s3:ListBucket",
 "s3:ListBucketMultipartUploads",
 "s3:PutObject",
 ],
 "Resource": [arn, f"{arn}/*"],
 }],
 })
 ),
 opts=pulumi.ResourceOptions(parent=self),
 )

 # --- Firehose delivery stream (Snowflake destination) ---
 stream = aws.kinesis.FirehoseDeliveryStream(
 f"{name}-firehose",
 destination="snowflake",
 snowflake_configuration=aws.kinesis.FirehoseDeliveryStreamSnowflakeConfigurationArgs(
 account_url=args.snowflake_account_url,
 database=args.database,
 schema=args.schema_name,
 table=args.table_name,
 role_arn=firehose_role.arn,
 user=sf_user.name,
 private_key=key_pair.private_key_pem_pkcs8.apply(
 strip_pem_headers
 ),
 data_loading_option=args.data_loading_option,
 content_column_name=args.content_column_name,
 metadata_column_name=args.metadata_column_name,
 s3_backup_mode=args.s3_backup_mode,
 buffering_size=args.buffering_size,
 buffering_interval=args.buffering_interval,
 retry_duration=args.retry_duration,
 snowflake_role_configuration=aws.kinesis.FirehoseDeliveryStreamSnowflakeConfigurationSnowflakeRoleConfigurationArgs(
 enabled=True,
 snowflake_role=args.snowflake_role_name,
 ),
 s3_configuration=aws.kinesis.FirehoseDeliveryStreamSnowflakeConfigurationS3ConfigurationArgs(
 bucket_arn=args.bucket_arn,
 role_arn=firehose_role.arn,
 prefix=f"{args.s3_prefix}/backup/",
 error_output_prefix=f"{args.s3_prefix}/errors/",
 ),
 ),
 opts=pulumi.ResourceOptions(parent=self, depends_on=[table]),
 )

 # --- Lambda function + Function URL ---
 lambda_role = aws.iam.Role(
 f"{name}-lambda-role",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Action": "sts:AssumeRole",
 "Principal": {"Service": "lambda.amazonaws.com"},
 }],
 }),
 opts=pulumi.ResourceOptions(parent=self),
 )

 aws.iam.RolePolicyAttachment(
 f"{name}-lambda-basic-execution",
 role=lambda_role.name,
 policy_arn="arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole",
 opts=pulumi.ResourceOptions(parent=self),
 )

 aws.iam.RolePolicy(
 f"{name}-lambda-firehose-policy",
 role=lambda_role.id,
 policy=stream.arn.apply(
 lambda arn: json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Action": ["firehose:PutRecord"],
 "Resource": [arn],
 }],
 })
 ),
 opts=pulumi.ResourceOptions(parent=self),
 )

 env_vars = {
 **args.lambda_environment,
 "FIREHOSE_STREAM_NAME": stream.name,
 }

 fn = aws.lambda_.Function(
 f"{name}-handler",
 runtime="python3.11",
 handler=args.lambda_handler,
 role=lambda_role.arn,
 timeout=30,
 code=args.lambda_code,
 environment=aws.lambda_.FunctionEnvironmentArgs(
 variables=env_vars,
 ),
 opts=pulumi.ResourceOptions(parent=self),
 )

 fn_url = aws.lambda_.FunctionUrl(
 f"{name}-function-url",
 function_name=fn.name,
 authorization_type="NONE",
 opts=pulumi.ResourceOptions(parent=self),
 )

 aws.lambda_.Permission(
 f"{name}-function-url-permission",
 action="lambda:InvokeFunctionUrl",
 function=fn.name,
 principal="*",
 function_url_auth_type="NONE",
 opts=pulumi.ResourceOptions(parent=self),
 )

 # --- Outputs ---
 self.function_url = fn_url.function_url
 self.firehose_stream_name = stream.name
 self.snowflake_user_name = sf_user.name

 self.register_outputs({
 "function_url": self.function_url,
 "firehose_stream_name": self.firehose_stream_name,
 "snowflake_user_name": self.snowflake_user_name,
 })

A few things to note:

TLS key pair for authentication. The component generates an RSA key pair using the pulumi-tls provider. The public key is assigned to the Snowflake service user; the private key (PKCS#8 format, base64-encoded) is passed to Firehose. No passwords or OAuth tokens are stored.
ServiceUser instead of User. Snowflake service users can’t log in interactively. They authenticate only via key pair, which is exactly what Firehose needs.
destination="snowflake" on Firehose. This tells Firehose to use the Snowpipe Streaming API rather than writing to S3. The s3_configuration block is still required, but only for backup/error records.
Immediate flushing. buffering_interval=0 and buffering_size=1 ensure records are sent to Snowflake as soon as they arrive, minimizing latency. Tune according to your needs.

Amazon Data Firehose does not connect from fixed IP addresses, so you cannot use Snowflake network policies to restrict access by IP. If your Snowflake account uses network policies, you have three options: use AWS PrivateLink (requires Snowflake Business Critical edition), allow public internet access for the Firehose service user, or switch to S3 auto-ingest via Snowpipe which does not require direct network access to Snowflake from Firehose.

Wiring it together

With the component defined, __main__.py wires the direct ingestion pipeline:

import pulumi
import pulumi_aws as aws
import pulumi_github as github
import pulumi_random as random
import pulumi_snowflake as snowflake
from components.direct_snowflake_ingestion import (
 DirectSnowflakeIngestion, DirectSnowflakeIngestionArgs,
)
from components.snowpipe_pipeline import ColumnDef

config = pulumi.Config()
database_name = config.get("database") or "LANDING_ZONE_WEBHOOKS"
environment = config.get("environment") or "dev"

# --- Shared infrastructure ---

# S3 bucket for backup/errors
bucket = aws.s3.Bucket(
 "data-landing-bucket",
)

# Snowflake database and schema
database = snowflake.Database("demo-database", name=database_name)
schema = snowflake.Schema(
 "demo-schema",
 name="GITHUB",
 database=database.name,
)

# --- Direct ingestion pipeline ---

webhook_repo = config.require("webhook-repo")

# Snowflake account URL for Firehose configuration
snowflake_config = pulumi.Config("snowflake")
snowflake_account_url = (
 f"https://{snowflake_config.require('organizationName')}"
 f"-{snowflake_config.require('accountName')}"
 f".snowflakecomputing.com"
)

# Landing table columns: CONTENT (webhook JSON) + METADATA (Firehose metadata)
DIRECT_COLUMNS = [
 ColumnDef(name="CONTENT", type="VARIANT", nullable=True),
 ColumnDef(name="METADATA", type="VARIANT", nullable=True),
]

# Step 1: Generate webhook secret for HMAC validation
direct_webhook_secret = random.RandomPassword(
 "github-direct-webhook-secret", length=32, special=False
)

# Step 2: Direct ingestion pipeline - Lambda validates, Firehose streams to Snowflake
direct = DirectSnowflakeIngestion(
 "github-webhooks-direct",
 DirectSnowflakeIngestionArgs(
 bucket_arn=bucket.arn,
 bucket_name=bucket.bucket,
 database=database.name,
 schema_name=schema.name,
 table_name="REPOSITORY_EVENTS_DIRECT",
 table_columns=DIRECT_COLUMNS,
 table_comment="GitHub webhook events loaded via direct Firehose to Snowflake",
 snowflake_account_url=snowflake_account_url,
 snowflake_role_name="FIREHOSE_DIRECT_LOADER",
 lambda_code=pulumi.AssetArchive({
 "webhook_handler.py": pulumi.FileAsset("lambda/webhook_handler.py"),
 }),
 lambda_handler="webhook_handler.handler",
 lambda_environment={"WEBHOOK_SECRET": direct_webhook_secret.result},
 ),
)

# Step 3: GitHub webhook - sends events to the Lambda Function URL
github.RepositoryWebhook(
 "github-direct-webhook",
 repository=webhook_repo,
 configuration=github.RepositoryWebhookConfigurationArgs(
 url=direct.function_url,
 content_type="json",
 secret=direct_webhook_secret.result,
 ),
 events=["push", "pull_request", "issues", "star"],
)

# Exports
pulumi.export("webhook_url", direct.function_url)
pulumi.export("firehose_stream", direct.firehose_stream_name)

That’s the entire pipeline. One component, one GitHub webhook, one secret. The DirectSnowflakeIngestion component handles the TLS key pair, Snowflake service user, landing table, Firehose stream, and Lambda function internally, and now you can reuse this component for as many pipelines as you need.

The full code for this example is available on GitHub:

github.com/pulumi-demos/examples/tree/main/python/aws-snowflake-data-loading-real-time

Testing the pipeline

Deploy the stack:

pulumi up

The entire stack deploys in about two minutes. Immediately after deployment, you’ll start seeing GitHub events flowing into Snowflake.

Before querying, grant the DATA_READER role to your Snowflake user:

GRANT ROLE DATA_READER TO USER <your-user>;

In production, you can manage this grant through Pulumi, manually, or automatically via SCIM provisioning from your identity provider.

No need to craft test payloads. Just interact with the test repo. Star it, push a commit, or open an issue, then wait about 30 seconds and query Snowflake using the least-privilege reader role:

USE ROLE DATA_READER;

SELECT CONTENT:github_event::STRING AS event_type,
 CONTENT:payload:repository:full_name::STRING AS repo,
 METADATA:IngestionTime::TIMESTAMP AS ingested_at
FROM LANDING_ZONE_WEBHOOKS.GITHUB.REPOSITORY_EVENTS_DIRECT
ORDER BY ingested_at DESC;

You should see rows with event types like star, push, or issues, real GitHub events flowing through the entire pipeline. The METADATA column includes Firehose metadata like IngestionTime, which you can use to track end-to-end latency.

Other loading patterns

Direct streaming is the fastest path, but two other patterns are available in the companion template for different requirements:

S3 auto-ingest via Snowpipe. Firehose buffers to S3, and Snowpipe auto-ingests new files. Latency is about two minutes. Best when you need S3 as the system of record or can’t use direct Snowpipe Streaming.
Batch loading. Your orchestrator (Airflow, Prefect, cron, etc.) runs COPY INTO on a schedule. Best for full control over timing and deduplication.

We’ll walk through both patterns in detail in upcoming posts.

Publishing as reusable components

Once your components are battle-tested, you can share them across teams and projects instead of copying files around.

The most straightforward approach: push your components to a Git repository with a PulumiPlugin.yaml file at the root:

runtime: python
name: snowpipe-components
version: 1.0.0

Consumers add the package to their project with pulumi package add:

pulumi package add github.com/your-org/pulumi-snowpipe@v1.0.0

Pulumi downloads the package and generates typed SDKs automatically. The consumer’s __main__.py imports your components as if they were local, but they’re versioned and pinned.

Pulumi Cloud Private Registry

For organization-level discoverability, publish to the Pulumi Cloud Private Registry with pulumi package publish:

pulumi package publish ./schema.json

This gives you auto-generated API docs, usage tracking across teams, and cross-language SDK generation. Your Python components become usable from TypeScript, Go, and C# without any extra work. Teams browse available components in the Pulumi Cloud console, see who’s using what, and get notified when new versions are published. As an additional benefit, Neo will be able to use these components and build new pipelines in minutes from a natural language request.

Conclusion

ComponentResource is the key abstraction that makes this architecture scale. Instead of copying and pasting resources for each new data source, you instantiate a component with a handful of configuration parameters.

The DirectSnowflakeIngestion component in this post delivers data from GitHub webhooks into Snowflake in seconds: Lambda validates the HMAC signature, Firehose streams directly to Snowflake via the Snowpipe Streaming API, and the TLS key pair is managed entirely within Pulumi. No S3 intermediate, no SQS queues.

The component accepts pluggable Lambda handlers, so swapping GitHub for Stripe webhooks or any other source is just a matter of providing different lambda_code and lambda_environment arguments. We’ve been running this pattern in production for over three years across dozens of pipelines without significant changes to the infrastructure code.

You’ll find the complete example in the GitHub repository.

Passwordless PostgreSQL: IAM Authentication with Pulumi

Elisabeth Lichtie — Fri, 13 Feb 2026 00:00:00 +0000

Managing database credentials is one of the persistent challenges in cloud infrastructure. Passwords need to be rotated, secrets need to be stored securely, and access needs to be carefully controlled. AWS IAM authentication for RDS offers a better way: instead of managing long-lived passwords, your applications authenticate using short-lived tokens generated from IAM credentials. This approach is more secure, eliminates password rotation overhead, and integrates seamlessly with your existing IAM policies. With Pulumi, you can set up this entire system using reusable components that make IAM authentication a standard part of your infrastructure.

Why IAM authentication for PostgreSQL?

Traditional database authentication relies on usernames and passwords. These credentials need to be stored somewhere secure, rotated regularly, and distributed to applications that need them. Each of these steps introduces complexity and potential security risks.

IAM authentication changes this model fundamentally. Instead of passwords, your applications generate authentication tokens on demand using their IAM credentials. These tokens are valid for only 15 minutes, eliminating the need for password rotation. Access control happens through IAM policies, which you’re already using to manage other AWS resources. The result is a more secure system that’s easier to maintain and audit.

The benefits become even more pronounced when you componentize this setup with Pulumi. Rather than repeating the same configuration steps for each database, you can build reusable components that handle IAM authentication setup automatically. This turns a complex, multi-step process into a simple, repeatable pattern that your entire team can use. In this post, we’ll cover an example that sets up the complete flow: an RDS cluster with IAM authentication, the necessary IAM roles and policies, and a Kubernetes application that connects using IAM tokens.

Architecture overview

This example sets up a complete environment with IAM-authenticated database access from a Kubernetes application. The architecture includes:

VPC with public and private subnets across multiple availability zones
EKS cluster for running containerized applications
Aurora PostgreSQL cluster with IAM authentication enabled
IAM roles and policies that connect Kubernetes service accounts to database access
A demo application that authenticates to the database using IAM tokens

The key integration point is IAM Roles for Service Accounts (IRSA), which allows Kubernetes pods to assume IAM roles. This means your application doesn’t need any AWS credentials stored in environment variables or mounted secrets. The pod’s service account automatically has the permissions it needs to generate database authentication tokens.

Setting up the RDS cluster with IAM authentication

The foundation of this setup is an RDS cluster configured to accept IAM authentication. With Pulumi’s componentized approach, you can encapsulate all the RDS configuration in a reusable component:

const rdsCluster = new RdsCluster("iam-postgres", {
 vpcId: vpc.vpc.id,
 vpcCidrBlock: vpc.vpc.cidrBlock,
 subnetIds: vpc.publicSubnets.map((s) => s.id),
 databaseName: DATABASE_NAME,
 masterUsername: MASTER_USERNAME,
 masterPassword: dbMasterPassword,
 instanceClass: "db.t4g.medium",
 engineVersion: "17.4",
 iamDatabaseUser: IAM_DB_USERNAME,
});

Inside the component, the critical configuration is iamDatabaseAuthenticationEnabled:

this.cluster = new aws.rds.Cluster(
 `${name}-cluster`,
 {
 engine: "aurora-postgresql",
 engineVersion: engineVersion,
 databaseName: args.databaseName,
 masterUsername: args.masterUsername,
 masterPassword: args.masterPassword,
 dbSubnetGroupName: subnetGroup.name,
 vpcSecurityGroupIds: [this.securityGroup.id],
 iamDatabaseAuthenticationEnabled: true,
 skipFinalSnapshot: true,
 tags: { Name: `${name}-cluster` },
 },
 { parent: this }
);

This single flag tells RDS to accept authentication tokens in addition to traditional passwords. The cluster still needs a master password for initial setup and administrative tasks, but your applications can use IAM authentication instead.

Creating the database user with IAM permissions

Enabling IAM authentication on the cluster isn’t enough. You also need to create a database user and grant it the special rds_iam role that allows IAM authentication. This is where Pulumi’s PostgreSQL provider comes in:

const dbSetup = new DbSetup(
 "iam-postgres",
 {
 dbEndpoint: rdsCluster.cluster.endpoint,
 dbName: DATABASE_NAME,
 masterUsername: MASTER_USERNAME,
 masterPassword: dbMasterPassword,
 iamUsername: IAM_DB_USERNAME,
 },
 { dependsOn: [rdsCluster.instance] }
);

The DbSetup component handles all the database-level configuration. It creates the IAM user and grants the necessary permissions:

// Create IAM-enabled database user
this.iamRole = new postgresql.Role(
 `${name}-iam-user`,
 {
 name: args.iamUsername,
 login: true,
 },
 { parent: this, provider: this.provider }
);

// Grant rds_iam role to enable IAM authentication
new postgresql.GrantRole(
 `${name}-grant-rds-iam`,
 {
 role: this.iamRole.name,
 grantRole: "rds_iam",
 },
 { parent: this, provider: this.provider }
);

The component also grants the necessary database privileges (CONNECT, USAGE, CREATE, and table operations). By encapsulating this in a component, you ensure that every IAM-authenticated database user is set up consistently with the correct permissions.

Configuring IAM policies for database access

The next piece is the IAM role and policy that allows applications to generate authentication tokens. This involves two parts: the role that the application assumes, and the policy that grants database access.

For Kubernetes applications, this uses IRSA (IAM Roles for Service Accounts). The RdsCluster component includes a method that creates the appropriate IAM role:

const rdsIamRole = rdsCluster.createIamRole(
 eksCluster.oidcProvider,
 NAMESPACE,
 SERVICE_ACCOUNT_NAME
);

This creates an IAM role with a trust policy that allows the specified Kubernetes service account to assume it:

assumeRolePolicy: pulumi
 .all([oidcProvider.arn, oidcProvider.url])
 .apply(([arn, url]) =>
 JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Effect: "Allow",
 Principal: { Federated: arn },
 Action: "sts:AssumeRoleWithWebIdentity",
 Condition: {
 StringEquals: {
 [`${url}:sub`]: `system:serviceaccount:${namespace}:${serviceAccountName}`,
 },
 },
 },
 ],
 })
 ),

The role is then granted the rds-db:connect permission for the specific database user:

{
 Version: "2012-10-17",
 Statement: [
 {
 Effect: "Allow",
 Action: ["rds-db:connect"],
 Resource: `arn:aws:rds-db:${region}:${accountId}:dbuser:${resourceId}/${iamDatabaseUser}`,
 },
 ],
}

This policy is scoped to exactly one database user on one cluster. This level of granularity is one of the security advantages of IAM authentication: you can control database access with the same precision you use for other AWS resources.

Connecting from Kubernetes with IRSA

The application running in Kubernetes needs a service account annotated with the IAM role ARN:

this.serviceAccount = new k8s.core.v1.ServiceAccount(
 `${name}-sa`,
 {
 metadata: {
 name: args.serviceAccountName,
 namespace: args.namespace,
 annotations: {
 "eks.amazonaws.com/role-arn": args.iamRoleArn,
 },
 },
 },
 { parent: this, provider: args.provider }
);

When a pod uses this service account, EKS automatically configures the pod with temporary AWS credentials for the associated IAM role. The application can then use these credentials to generate database authentication tokens.

How IAM authentication works at runtime

The authentication flow happens at connection time:

%%{init: {'theme':'base', 'themeVariables': { 'primaryColor':'#FF9900','primaryTextColor':'#232F3E','primaryBorderColor':'#232F3E','lineColor':'#666','secondaryColor':'#527FFF','tertiaryColor':'#fff'}}}%%
sequenceDiagram
autonumber
participant App as Application Pod
participant SDK as AWS SDK
participant STS as AWS STS
participant IAM as AWS IAM
participant RDS as RDS Aurora
rect rgb(255, 249, 230)
Note over App,SDK: Phase 1: Token Generation
activate App
App->>+SDK: generate_db_auth_token()
SDK->>+STS: Get temporary credentials (IRSA)
STS-->>-SDK: Return AWS credentials
SDK->>SDK: Sign auth request
SDK-->>-App: Return IAM token (15 min TTL)
deactivate App
end
rect rgb(235, 245, 255)
Note over App,RDS: Phase 2: Database Connection
activate App
App->>+RDS: Connect (user + IAM token)
activate RDS
RDS->>+IAM: Validate token & permissions
IAM-->>-RDS: Token valid + rds-db:connect allowed
RDS->>RDS: Verify rds_iam role
RDS-->>-App: Connection established
deactivate App
end
Note over App,RDS: 💡 Token expires after 15 minutes - generate new token per connection

The application uses the AWS SDK to call generate_db_auth_token(), passing the database endpoint, port, and username
The SDK uses the pod’s IAM credentials (provided automatically by IRSA) to sign the request and generate a token
The application connects to PostgreSQL using the IAM username and the token as the password
RDS validates the token against IAM, verifying that the caller has rds-db:connect permission for that database user
If the token is valid, the connection is established

Here’s what this looks like in Python:

def get_iam_token():
 """Generate an IAM authentication token for RDS"""
 client = boto3.client('rds', region_name=AWS_REGION)
 token = client.generate_db_auth_token(
 DBHostname=DB_ENDPOINT,
 Port=DB_PORT,
 DBUsername=DB_USER,
 Region=AWS_REGION
 )
 return token

def get_db_connection():
 """Create a database connection with IAM auth"""
 token = get_iam_token()
 return psycopg2.connect(
 host=DB_ENDPOINT,
 port=DB_PORT,
 database=DB_NAME,
 user=DB_USER,
 password=token,
 sslmode='require',
 sslrootcert=RDS_CA_CERT
 )

The token is valid for 15 minutes. Applications should generate a new token for each connection or implement token caching with refresh logic.

Testing the setup with the demo application

The example includes an interactive demo application that lets you test the IAM authentication setup. Once deployed, the application provides a web interface for creating tables, adding messages, and querying the database:

Deploy the infrastructure and access the demo app:

pulumi up
export APP_URL=$(pulumi stack output appUrl)
echo "Demo app: http://$APP_URL"

The application shows the database endpoint and IAM username it’s using, and provides buttons to create tables and add data. Behind the scenes, every database operation authenticates using IAM tokens, demonstrating that the entire authentication flow is working correctly.

The full code for this example is available at https://github.com/pulumi-demos/examples/tree/main/typescript/aws-iam-for-postgres. The example includes all the component code referenced in this post.

Production considerations

This example demonstrates IAM authentication in a working environment, but you’ll want to make several adjustments for production use:

Network security: The example places RDS in public subnets to allow the PostgreSQL provider to connect during deployment. In production, place RDS in private subnets and ensure your deployment environment (like GitHub Actions or Pulumi Cloud) can reach the database for initial setup, either through a bastion host or VPN.

Connection pooling and performance: IAM tokens expire after 15 minutes. If you’re using connection pooling, you’ll need logic to refresh tokens before they expire. AWS recommends using IAM authentication only when your application creates fewer than 200 new connections per second. For higher connection rates, consider using Amazon RDS Proxy, which manages connection pooling and can reduce the overhead of IAM token generation.

Master password management: You still need a master password for database administration and for the initial user setup. Store this in AWS Secrets Manager or Pulumi ESC, and restrict access to it carefully.

Monitoring and auditing: Token generation via the AWS SDK is logged in CloudTrail. Note that CloudWatch and CloudTrail do not log database authentication attempts themselves, so you’ll need to rely on PostgreSQL’s native logging for connection monitoring.

Multi-region considerations: If your application runs in multiple regions, ensure that IAM policies and database access work correctly across regions. Token generation and validation must happen in the same region as the RDS cluster.

Cost: IAM authentication itself is free, but be aware that cross-AZ data transfer costs still apply for database connections.

Conclusion

IAM authentication for PostgreSQL transforms database credential management from a security burden into a seamless part of your infrastructure. By eliminating long-lived passwords, you reduce your attack surface and remove the operational overhead of password rotation. By integrating with IAM, you gain fine-grained access control and comprehensive audit logs.

Pulumi makes this setup practical and repeatable through componentization. The components in this example encapsulate the complexity of IAM authentication setup, making it easy to apply the same pattern across multiple databases and applications. This turns security best practices into your team’s default approach.

Whether you’re building a new application or improving the security posture of existing infrastructure, IAM authentication for RDS deserves consideration. The investment in setup pays dividends in security, maintainability, and operational simplicity.

Deploy OpenClaw on AWS or Hetzner Securely with Pulumi and Tailscale

Engin Diri — Mon, 26 Jan 2026 00:00:00 +0000

Update (January 2026): The lobster has molted into its final form! From Clawdbot to Moltbot to OpenClaw. With 100k+ GitHub stars and 2M visitors in a week, the project finally has a name that’ll stick. The CLI command is now openclaw and the new handle is @openclaw. Same mission: AI that actually does things. Your assistant. Your machine. Your rules. See the official getting started guide for updated installation instructions.

Update (April 2026): Refreshed for OpenClaw 2026.4.27. Upstream now recommends Node 24, but the cloud-init script in this post still installs Node 22 — both work. If you’d like Node 24, change the nvm install 22 lines to nvm install 24.

The short version: Deploy OpenClaw to AWS or Hetzner with a Pulumi TypeScript program that provisions the VM, installs Docker, Node, and OpenClaw, then joins the instance to your Tailscale network so the gateway and browser ports stay private. One pulumi up to deploy, one pulumi destroy to tear down. Total cost: about $33/month on AWS or $7/month on Hetzner.

OpenClaw is everywhere right now. The open-source AI assistant gained 9,000 GitHub stars in a single day, received public praise from former Tesla AI head Andrej Karpathy, and has sparked a global run on Mac Minis as developers scramble to give this “lobster assistant” a home. Users are calling it “Jarvis living in a hard drive” and “Claude with hands”—the personal AI assistant that Siri promised but never delivered.

The Mac Mini craze is real: people are buying dedicated hardware just to run OpenClaw, with some enthusiasts purchasing 40 Mac Minis at once. Even Logan Kilpatrick from Google DeepMind couldn’t resist ordering one. But here’s the thing: you don’t actually need a Mac Mini. OpenClaw runs anywhere: on a VPS, in the cloud, or on that old laptop gathering dust.

With all this hype, I had to try it myself. But instead of clicking through the AWS console or running manual commands on a VPS, I wanted to do it right from the start: infrastructure as code with Pulumi. Why? Because when I inevitably want to tear it down, spin up a new instance, or deploy to a different region, I don’t want to remember which buttons I clicked three weeks ago. I want a single pulumi up command.

Dan got the assignment right:

In this post, I’ll show you how to deploy OpenClaw to AWS or Hetzner Cloud (if you want European data residency or just want to spend less). We’ll use Pulumi to define the infrastructure and Tailscale to keep your AI assistant off the public internet.

What is OpenClaw?

OpenClaw is an open-source AI assistant created by Peter Steinberger that runs on your own infrastructure. It connects to WhatsApp, Slack, Discord, Google Chat, Signal, and iMessage. It can control browsers, generate videos and images, clone your voice for voice notes, and run scheduled tasks via cron. There’s a skills system for extending functionality, and you can run it on pretty much anything: Mac Mini, Raspberry Pi, VPS, laptop, or gaming PC.

The difference from cloud-hosted AI? OpenClaw runs on your server, not Anthropic’s. It’s available 24/7 across all your devices, can schedule automated tasks, and keeps your entire conversation history locally. Check the official OpenClaw documentation for the full feature list.

Prerequisites

Before getting started, ensure you have:

Pulumi CLI installed and configured
A Pulumi Cloud account
AWS account (for AWS deployment)
Hetzner Cloud account (for European deployment)
Anthropic API key
Node.js 22+ installed (Node 24 recommended; the cloud-init script in this post installs Node 22 inside the VM)
Tailscale account with HTTPS enabled (one-time setup in admin console)

This guide uses Anthropic’s API, but OpenClaw works with other providers too. Check the providers documentation if you’d rather use OpenAI, Google Gemini, or a local model via Ollama.

Understanding OpenClaw architecture

OpenClaw uses a gateway-centric architecture where a single daemon acts as the control plane for all messaging, tool execution, and client connections:

Component	Port	Description
Gateway	18789	WebSocket server handling channels, nodes, sessions, and hooks
Browser control	18791	Headless Chrome instance for web automation
Docker sandbox	-	Isolated container environment for running tools safely

The Gateway connects to messaging platforms (WhatsApp, Slack, Discord, etc.), the CLI, the web UI, and mobile apps. The Browser component lets OpenClaw open web pages, fill forms, scrape data, and download files. Docker sandboxing runs bash commands in isolated containers so your bot can execute code without risking your host system.

Setting up ESC for secrets management

Deploying OpenClaw means handling sensitive credentials: API keys, auth tokens, cloud provider secrets. You don’t want these hardcoded or scattered across environment variables. Pulumi ESC (Environments, Secrets, and Configuration) stores them securely and passes them directly to your Pulumi program — the same pattern Pulumi uses to eliminate long-lived CI secrets across 70+ repos.

Create a new ESC environment:

pulumi env init <your-org>/openclaw-secrets

Add your secrets to the environment:

values:
 anthropicApiKey:
 fn::secret: "sk-ant-xxxxx"
 tailscaleAuthKey:
 fn::secret: "tskey-auth-xxxxx"
 tailnetDnsName: "tailxxxxx.ts.net"
 hcloudToken:
 fn::secret: "your-hetzner-api-token"
 pulumiConfig:
 anthropicApiKey: ${anthropicApiKey}
 tailscaleAuthKey: ${tailscaleAuthKey}
 tailnetDnsName: ${tailnetDnsName}
 hcloud:token: ${hcloudToken}

Then create a Pulumi.dev.yaml file in your project to reference the environment:

environment:
 - <your-org>/openclaw-secrets

This approach keeps your secrets out of your codebase and passes them directly to OpenClaw during automated onboarding. To prevent stack-level overrides of these values, see locking down ESC values with fn::final.

Securing with Tailscale

By default, deploying OpenClaw exposes SSH (port 22), the gateway (port 18789), and browser control (port 18791) to the public internet. This is convenient for testing but not ideal for production use.

Tailscale creates a secure mesh VPN that lets you access your OpenClaw instance without exposing unnecessary ports publicly. When you provide a Tailscale auth key, the Pulumi program:

Removes gateway and browser ports from public access
Keeps SSH as fallback for debugging if Tailscale setup fails
Installs Tailscale on the instance during provisioning (after other dependencies)
Enables Tailscale SSH so you can SSH via Tailscale without managing keys
Joins your tailnet automatically using the auth key

The Pulumi program installs Docker, Node.js, and OpenClaw first, then configures Tailscale last. This ensures that even if the Tailscale auth key is invalid or expired, you can still SSH in via the public IP to troubleshoot.

To generate a Tailscale auth key:

Go to Tailscale Admin Console
Click “Generate auth key”
Enable “Reusable” if you plan to redeploy
Copy the key and add it to your ESC environment

Deploying to AWS

Let’s walk through the complete AWS deployment. Create a new Pulumi project:

mkdir openclaw-aws && cd openclaw-aws
pulumi new typescript

Install the required dependencies:

npm install @pulumi/aws @pulumi/tls

Do not use t3.micro instances for OpenClaw. The 1 GB memory is insufficient for installation. Use t3.medium (4 GB) or t3.large (8 GB) instead.

The Pulumi program

Running OpenClaw on AWS means setting up a VPC, subnets, security groups, an EC2 instance, SSH keys, and a cloud-init script that installs everything. That’s a lot of clicking in the AWS console. The Pulumi program below defines all of it in code.

Replace the contents of index.ts with the following:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as tls from "@pulumi/tls";

const config = new pulumi.Config();

const instanceType = config.get("instanceType") ?? "t3.medium";
const anthropicApiKey = config.requireSecret("anthropicApiKey");
const model = config.get("model") ?? "anthropic/claude-sonnet-4";
const enableSandbox = config.getBoolean("enableSandbox") ?? true;
const gatewayPort = config.getNumber("gatewayPort") ?? 18789;
const browserPort = config.getNumber("browserPort") ?? 18791;

const tailscaleAuthKey = config.requireSecret("tailscaleAuthKey");
const tailnetDnsName = config.require("tailnetDnsName");

// Generate a random token for gateway authentication
const gatewayToken = new tls.PrivateKey("openclaw-gateway-token", {
 algorithm: "ED25519",
}).publicKeyOpenssh.apply(key => {
 // Create a deterministic token from the public key (take first 48 hex chars)
 const hash = require("crypto").createHash("sha256").update(key).digest("hex");
 return hash.substring(0, 48);
});

const sshKey = new tls.PrivateKey("openclaw-ssh-key", {
 algorithm: "ED25519",
});

const vpc = new aws.ec2.Vpc("openclaw-vpc", {
 cidrBlock: "10.0.0.0/16",
 enableDnsHostnames: true,
 enableDnsSupport: true,
 tags: { Name: "openclaw-vpc" },
});

const gateway = new aws.ec2.InternetGateway("openclaw-igw", {
 vpcId: vpc.id,
 tags: { Name: "openclaw-igw" },
});

const subnet = new aws.ec2.Subnet("openclaw-subnet", {
 vpcId: vpc.id,
 cidrBlock: "10.0.1.0/24",
 mapPublicIpOnLaunch: true,
 tags: { Name: "openclaw-subnet" },
});

const routeTable = new aws.ec2.RouteTable("openclaw-rt", {
 vpcId: vpc.id,
 routes: [
 {
 cidrBlock: "0.0.0.0/0",
 gatewayId: gateway.id,
 },
 ],
 tags: { Name: "openclaw-rt" },
});

new aws.ec2.RouteTableAssociation("openclaw-rta", {
 subnetId: subnet.id,
 routeTableId: routeTable.id,
});

const securityGroup = new aws.ec2.SecurityGroup("openclaw-sg", {
 vpcId: vpc.id,
 description: "Security group for OpenClaw instance",
 ingress: [
 {
 description: "SSH access (fallback)",
 fromPort: 22,
 toPort: 22,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 ],
 egress: [
 {
 fromPort: 0,
 toPort: 0,
 protocol: "-1",
 cidrBlocks: ["0.0.0.0/0"],
 },
 ],
 tags: { Name: "openclaw-sg" },
});

const keyPair = new aws.ec2.KeyPair("openclaw-keypair", {
 publicKey: sshKey.publicKeyOpenssh,
});

const ami = aws.ec2.getAmiOutput({
 owners: ["099720109477"],
 mostRecent: true,
 filters: [
 { name: "name", values: ["ubuntu/images/hvm-ssd-gp3/ubuntu-noble-24.04-amd64-server-*"] },
 { name: "virtualization-type", values: ["hvm"] },
 ],
});

const userData = pulumi
 .all([tailscaleAuthKey, anthropicApiKey, gatewayToken])
 .apply(([tsAuthKey, apiKey, gwToken]) => {
 return `#!/bin/bash
set -e

export DEBIAN_FRONTEND=noninteractive

# System updates
apt-get update
apt-get upgrade -y

# Install Docker
curl -fsSL https://get.docker.com | sh
systemctl enable docker
systemctl start docker
usermod -aG docker ubuntu

# Install NVM and Node.js for ubuntu user
sudo -u ubuntu bash << 'UBUNTU_SCRIPT'
set -e
cd ~

# Install NVM
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash

# Load NVM
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

# Install Node.js 22
nvm install 22
nvm use 22
nvm alias default 22

# Install OpenClaw
npm install -g openclaw@latest

# Add NVM to bashrc if not already there
if ! grep -q 'NVM_DIR' ~/.bashrc; then
 echo 'export NVM_DIR="$HOME/.nvm"' >> ~/.bashrc
 echo '[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"' >> ~/.bashrc
fi
UBUNTU_SCRIPT

# Set environment variables for ubuntu user
echo 'export ANTHROPIC_API_KEY="${apiKey}"' >> /home/ubuntu/.bashrc

# Install and configure Tailscale
echo "Installing Tailscale..."
curl -fsSL https://tailscale.com/install.sh | sh
tailscale up --authkey="${tsAuthKey}" --ssh || echo "WARNING: Tailscale setup failed. Run 'sudo tailscale up' manually."

# Enable systemd linger for ubuntu user (required for user services to run at boot)
loginctl enable-linger ubuntu

# Start user's systemd instance (required for user services during cloud-init)
systemctl start user@1000.service

# Run OpenClaw onboarding as ubuntu user (skip daemon install, do it separately)
echo "Running OpenClaw onboarding..."
sudo -H -u ubuntu ANTHROPIC_API_KEY="${apiKey}" GATEWAY_PORT="${gatewayPort}" bash -c '
export HOME=/home/ubuntu
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

openclaw onboard --non-interactive --accept-risk \
 --mode local \
 --auth-choice apiKey \
 --gateway-port $GATEWAY_PORT \
 --gateway-bind loopback \
 --skip-daemon \
 --skip-skills || echo "WARNING: OpenClaw onboarding failed. Run openclaw onboard manually."
'

# Install daemon service with XDG_RUNTIME_DIR set
echo "Installing OpenClaw daemon..."
sudo -H -u ubuntu XDG_RUNTIME_DIR=/run/user/1000 bash -c '
export HOME=/home/ubuntu
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

openclaw daemon install || echo "WARNING: Daemon install failed. Run openclaw daemon install manually."
'

# Configure gateway for Tailscale Serve (trustedProxies + skip device pairing + set token)
echo "Configuring gateway for Tailscale Serve..."
sudo -H -u ubuntu GATEWAY_TOKEN="${gwToken}" python3 << 'PYTHON_SCRIPT'
import json
import os
config_path = "/home/ubuntu/.openclaw/openclaw.json"
with open(config_path) as f:
 config = json.load(f)
config["gateway"]["trustedProxies"] = ["127.0.0.1"]
config["gateway"]["controlUi"] = {
 "enabled": True,
 "allowInsecureAuth": True
}
config["gateway"]["auth"] = {
 "mode": "token",
 "token": os.environ["GATEWAY_TOKEN"]
}
with open(config_path, "w") as f:
 json.dump(config, f, indent=2)
print("Configured gateway with trustedProxies, controlUi, and token")
PYTHON_SCRIPT

# Enable Tailscale HTTPS proxy (requires HTTPS to be enabled in Tailscale admin console)
echo "Enabling Tailscale HTTPS proxy..."
tailscale serve --bg ${gatewayPort} || echo "WARNING: tailscale serve failed. Enable HTTPS in your Tailscale admin console first."

echo "OpenClaw setup complete!"
`;
 });

const instance = new aws.ec2.Instance("openclaw-instance", {
 ami: ami.id,
 instanceType: instanceType,
 subnetId: subnet.id,
 vpcSecurityGroupIds: [securityGroup.id],
 keyName: keyPair.keyName,
 userData: userData,
 userDataReplaceOnChange: true,
 rootBlockDevice: {
 volumeSize: 30,
 volumeType: "gp3",
 },
 tags: { Name: "openclaw" },
});

export const publicIp = instance.publicIp;
export const publicDns = instance.publicDns;
export const privateKey = sshKey.privateKeyOpenssh;

// Construct the Tailscale MagicDNS hostname from the private IP
// AWS private IPs like 10.0.1.15 become hostnames like ip-10-0-1-15
const tailscaleHostname = instance.privateIp.apply(ip =>
 `ip-${ip.replace(/\./g, "-")}`
);

export const tailscaleUrlWithToken = pulumi.interpolate`https://${tailscaleHostname}.${tailnetDnsName}/?token=${gatewayToken}`;
export const gatewayTokenOutput = gatewayToken;

Deploying to Hetzner

Hetzner Cloud is a solid choice if you need European data residency or want to spend less money. Spoiler: it’s a lot less money.

Hetzner has similar concepts to AWS but different names. EC2 instances become Servers. Security groups become Firewalls. Same idea, different provider. The resource types come from @pulumi/hcloud.

Create a new project for Hetzner:

mkdir openclaw-hetzner && cd openclaw-hetzner
pulumi new typescript

Install the Hetzner provider:

npm install @pulumi/hcloud @pulumi/tls

The default server type cax21 is an ARM-based (Ampere) instance with 4 vCPUs and 8 GB RAM. ARM instances cost less for the same compute. If you need x86 architecture, use ccx13 or similar CCX series instead.

The Hetzner Pulumi program

Replace index.ts with the following:

import * as pulumi from "@pulumi/pulumi";
import * as hcloud from "@pulumi/hcloud";
import * as tls from "@pulumi/tls";

const config = new pulumi.Config();

const serverType = config.get("serverType") ?? "cax21";
const location = config.get("location") ?? "fsn1";
const anthropicApiKey = config.requireSecret("anthropicApiKey");
const model = config.get("model") ?? "anthropic/claude-sonnet-4";
const enableSandbox = config.getBoolean("enableSandbox") ?? true;
const gatewayPort = config.getNumber("gatewayPort") ?? 18789;
const browserPort = config.getNumber("browserPort") ?? 18791;

const tailscaleAuthKey = config.requireSecret("tailscaleAuthKey");
const tailnetDnsName = config.require("tailnetDnsName");

// Generate a random token for gateway authentication
const gatewayToken = new tls.PrivateKey("openclaw-gateway-token", {
 algorithm: "ED25519",
}).publicKeyOpenssh.apply(key => {
 const hash = require("crypto").createHash("sha256").update(key).digest("hex");
 return hash.substring(0, 48);
});

const sshKey = new tls.PrivateKey("openclaw-ssh-key", {
 algorithm: "ED25519",
});

const hcloudSshKey = new hcloud.SshKey("openclaw-sshkey", {
 publicKey: sshKey.publicKeyOpenssh,
});

const firewallRules: hcloud.types.input.FirewallRule[] = [
 {
 direction: "out",
 protocol: "tcp",
 port: "any",
 destinationIps: ["0.0.0.0/0", "::/0"],
 description: "Allow all outbound TCP",
 },
 {
 direction: "out",
 protocol: "udp",
 port: "any",
 destinationIps: ["0.0.0.0/0", "::/0"],
 description: "Allow all outbound UDP",
 },
 {
 direction: "out",
 protocol: "icmp",
 destinationIps: ["0.0.0.0/0", "::/0"],
 description: "Allow all outbound ICMP",
 },
 {
 direction: "in",
 protocol: "tcp",
 port: "22",
 sourceIps: ["0.0.0.0/0", "::/0"],
 description: "SSH access (fallback)",
 },
];

const firewall = new hcloud.Firewall("openclaw-firewall", {
 rules: firewallRules,
});

const userData = pulumi
 .all([tailscaleAuthKey, anthropicApiKey, gatewayToken])
 .apply(([tsAuthKey, apiKey, gwToken]) => {
 return `#!/bin/bash
set -e

export DEBIAN_FRONTEND=noninteractive

# System updates
apt-get update
apt-get upgrade -y

# Install Docker
curl -fsSL https://get.docker.com | sh
systemctl enable docker
systemctl start docker

# Create ubuntu user (Hetzner uses root by default)
useradd -m -s /bin/bash -G docker ubuntu || true

# Install NVM and Node.js for ubuntu user
sudo -u ubuntu bash << 'UBUNTU_SCRIPT'
set -e
cd ~

# Install NVM
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash

# Load NVM
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

# Install Node.js 22
nvm install 22
nvm use 22
nvm alias default 22

# Install OpenClaw
npm install -g openclaw@latest

# Add NVM to bashrc if not already there
if ! grep -q 'NVM_DIR' ~/.bashrc; then
 echo 'export NVM_DIR="$HOME/.nvm"' >> ~/.bashrc
 echo '[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"' >> ~/.bashrc
fi
UBUNTU_SCRIPT

# Set environment variables for ubuntu user
echo 'export ANTHROPIC_API_KEY="${apiKey}"' >> /home/ubuntu/.bashrc

# Install and configure Tailscale
echo "Installing Tailscale..."
curl -fsSL https://tailscale.com/install.sh | sh
tailscale up --authkey="${tsAuthKey}" --ssh || echo "WARNING: Tailscale setup failed. Run 'sudo tailscale up' manually."

# Enable systemd linger for ubuntu user (required for user services to run at boot)
loginctl enable-linger ubuntu

# Start user's systemd instance (required for user services during cloud-init)
systemctl start user@1000.service

# Run OpenClaw onboarding as ubuntu user (skip daemon install, do it separately)
echo "Running OpenClaw onboarding..."
sudo -H -u ubuntu ANTHROPIC_API_KEY="${apiKey}" GATEWAY_PORT="${gatewayPort}" bash -c '
export HOME=/home/ubuntu
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

openclaw onboard --non-interactive --accept-risk \
 --mode local \
 --auth-choice apiKey \
 --gateway-port $GATEWAY_PORT \
 --gateway-bind loopback \
 --skip-daemon \
 --skip-skills || echo "WARNING: OpenClaw onboarding failed. Run openclaw onboard manually."
'

# Install daemon service with XDG_RUNTIME_DIR set
echo "Installing OpenClaw daemon..."
sudo -H -u ubuntu XDG_RUNTIME_DIR=/run/user/1000 bash -c '
export HOME=/home/ubuntu
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"

openclaw daemon install || echo "WARNING: Daemon install failed. Run openclaw daemon install manually."
'

# Configure gateway for Tailscale Serve (trustedProxies + skip device pairing + set token)
echo "Configuring gateway for Tailscale Serve..."
sudo -H -u ubuntu GATEWAY_TOKEN="${gwToken}" python3 << 'PYTHON_SCRIPT'
import json
import os
config_path = "/home/ubuntu/.openclaw/openclaw.json"
with open(config_path) as f:
 config = json.load(f)
config["gateway"]["trustedProxies"] = ["127.0.0.1"]
config["gateway"]["controlUi"] = {
 "enabled": True,
 "allowInsecureAuth": True
}
config["gateway"]["auth"] = {
 "mode": "token",
 "token": os.environ["GATEWAY_TOKEN"]
}
with open(config_path, "w") as f:
 json.dump(config, f, indent=2)
print("Configured gateway with trustedProxies, controlUi, and token")
PYTHON_SCRIPT

# Enable Tailscale HTTPS proxy (requires HTTPS to be enabled in Tailscale admin console)
echo "Enabling Tailscale HTTPS proxy..."
tailscale serve --bg ${gatewayPort} || echo "WARNING: tailscale serve failed. Enable HTTPS in your Tailscale admin console first."

echo "OpenClaw setup complete!"
`;
 });

const server = new hcloud.Server("openclaw-server", {
 serverType: serverType,
 location: location,
 image: "ubuntu-24.04",
 sshKeys: [hcloudSshKey.id],
 firewallIds: [firewall.id.apply(id => Number(id))],
 userData: userData,
 labels: {
 purpose: "openclaw",
 },
});

export const ipv4Address = server.ipv4Address;
export const privateKey = sshKey.privateKeyOpenssh;

// Construct the Tailscale MagicDNS hostname from the server name
// Hetzner servers use their name as the hostname
const tailscaleHostname = server.name;

export const tailscaleUrlWithToken = pulumi.interpolate`https://${tailscaleHostname}.${tailnetDnsName}/?token=${gatewayToken}`;
export const gatewayTokenOutput = gatewayToken;

You can find both programs in the Pulumi examples repo under openclaw/:

github.com/pulumi/examples

Cost comparison

Before deploying, let’s compare the costs between AWS and Hetzner for running OpenClaw 24/7:

	AWS (t3.medium)	Hetzner (cax21)
vCPUs	2	4
Memory	4 GB	8 GB
Storage	30 GB gp3 (+$2.40/mo)	80 GB NVMe (included)
Traffic	Pay per GB	20 TB included
Architecture	x86 (Intel/AMD)	ARM (Ampere)
Hourly price	$0.0416	€0.0104 (~$0.011)
Monthly price	~$33 (with storage)	€6.49 (~$7)
Annual cost	~$396	~$84

Hetzner gives you double the vCPUs, double the RAM, at less than a quarter of the price. The trade-off? ARM architecture instead of x86. But OpenClaw doesn’t care - it’s just Node.js and Docker.

Prices are for on-demand instances as of January 2026. AWS prices are for us-east-1; Hetzner prices exclude VAT. Both include standard networking and storage. Check AWS EC2 pricing and Hetzner Cloud pricing for current rates.

Running the deployment

With your ESC environment configured in Pulumi.dev.yaml, deploy with:

pulumi up

After deployment completes, you’ll see outputs similar to:

Outputs:
 gatewayTokenOutput : "786c099cc8f8bf20dbebf40b8b51b75cf5cdab25..."
 privateKey : [secret]
 publicDns : "ec2-x-x-x-x.compute-1.amazonaws.com"
 publicIp : "x.x.x.x"
 tailscaleUrlWithToken: "https://ip-10-0-1-x.tailxxxxx.ts.net/?token=786c099..."

The tailscaleUrlWithToken output provides the complete URL with authentication token. Copy and paste it into your browser to access the OpenClaw web UI.

Output names vary slightly between providers: AWS uses publicIp and publicDns, while Hetzner uses ipv4Address. The Tailscale hostname is derived from the instance’s private IP (AWS) or server name (Hetzner).

Automated onboarding

The Pulumi program runs OpenClaw’s non-interactive onboarding during instance provisioning. It uses your Anthropic API key from ESC, binds the gateway to loopback with Tailscale Serve as the HTTPS proxy, generates a secure gateway token (exported in Pulumi outputs), installs the daemon as a systemd user service, and configures trustedProxies and controlUi.allowInsecureAuth to skip device pairing when accessed via Tailscale.

The cloud-init script runs openclaw onboard --non-interactive with all necessary flags, then configures the gateway for secure Tailscale access. Your instance is ready as soon as provisioning finishes.

Access the web UI

Please wait 3-5 minutes after pulumi up completes. The cloud-init script runs after the instance launches and needs time to install Docker, Node.js, OpenClaw, and Tailscale, then run the onboarding process and start the daemon. Periodically refresh the page until it loads. If the page still doesn’t load after 5 minutes, see Verify the deployment to troubleshoot.

The easiest way to access the OpenClaw web UI is to use the tailscaleUrlWithToken output from Pulumi:

# Get the full URL with token
pulumi stack output tailscaleUrlWithToken

Copy and paste this URL into your browser. The URL includes both the Tailscale MagicDNS hostname and the authentication token, so you can access the web UI directly.

Finding your tailnet DNS name: Go to the Tailscale admin console and look under the DNS section for your tailnet name (e.g., tailxxxxx.ts.net). You can also find your machines and their MagicDNS hostnames in the Machines tab.

Token-based authentication provides an additional layer of security on top of Tailscale’s network-level authentication. Only devices on your tailnet can reach the URL, and the token prevents unauthorized access if someone gains access to your tailnet.

From the web UI, you can connect messaging channels (WhatsApp, Discord, Slack), configure skills and integrations, and manage settings.

Verify the deployment (optional)

If you encounter issues accessing the web UI, you can SSH into your instance to troubleshoot:

# Check your Tailscale admin console for the new machine
ssh ubuntu@<tailscale-ip>

# Check OpenClaw gateway status
systemctl --user status openclaw-gateway

Test your assistant

Open the gateway dashboard using the tailscaleUrlWithToken output and use the built-in chat to test your assistant:

Your personal AI assistant is now running 24/7 on your own infrastructure, accessible securely through Tailscale.

Security considerations

When self-hosting an AI assistant, security matters. OpenClaw’s rapid adoption meant thousands of instances spun up in days, and not everyone locked them down. The community noticed:

The tweet isn’t exaggerating. A quick Shodan search shows exposed gateways on port 18789 with shell access, browser automation, and API keys up for grabs:

Don’t let your instance be one of them.

Concern	Without Tailscale	With Tailscale
SSH access	Public (port 22 open)	Public fallback + Tailscale SSH
Gateway access	Public (port 18789 open)	Private (Tailscale only)
Browser control	Public (port 18791 open)	Private (Tailscale only)
API keys in transit	Exposed if gateway accessed over HTTP	Protected by Tailscale encryption
Attack surface	3 open ports	1 open port (SSH fallback)

SSH remains accessible as a fallback even with Tailscale enabled. This allows you to troubleshoot if Tailscale fails to connect. Once you’ve confirmed Tailscale is working, you can manually remove the SSH ingress rule from your security group for maximum security.

My recommendations:

Always use Tailscale for production
Rotate your auth keys periodically
Use Pulumi ESC for secrets instead of hardcoding
Enable Tailscale SSH to avoid managing keys manually
Monitor your Tailscale admin console for unauthorized devices
Remove the SSH fallback after confirming Tailscale works if you want zero public ports

What’s next?

Now that OpenClaw is running, you can install skills (voice generation, video creation, browser automation), set up scheduled tasks with cron, invite colleagues to your tailnet for shared access, or connect additional channels like WhatsApp and Discord.

For a deeper case study of running an AI assistant on Pulumi-managed infrastructure, see how we built Platybot, or read our KubeCon EU 2026 recap on the year AI moved into production on Kubernetes.

Conclusion

Deploying OpenClaw with infrastructure as code means you can reproduce your setup anytime, version control it, and tear it down with a single command. Adding Tailscale keeps it private - no exposed ports, no hoping you configured your firewall correctly at 2am.

If you run into issues or have questions, drop by the Pulumi Community Slack or GitHub Discussions.

New to Pulumi? Get started here.

How Ralph Wiggum Built a Serverless SaaS with Pulumi

Engin Diri — Wed, 14 Jan 2026 00:00:00 +0000

I was about to do something that felt either genius or completely reckless: hand over my AWS credentials to an AI and step away from my computer. The technique is called “Ralph Wiggum,” named after the Simpsons character who eats glue and says “I’m in danger” while everything burns around him. And honestly, that felt about right for what I was attempting.

The problem with AI coding assistants

If you have spent any time with AI coding assistants, you know the frustration. You are in the middle of a task. It is going great. Claude is writing beautiful infrastructure code, understanding your architecture, making progress… and then it says “I have completed your request.”

You stare at the screen. “No. No, you have not. You maybe did a third of what I asked.”

So you reprompt it. It does a little more. Then it stops again. You are babysitting. The whole point of using an AI assistant was to save time, but if you have to check on it every five minutes, you are not saving anything. You are just a very expensive supervisor.

This is the problem Geoffrey Huntley decided to solve.

What if we just do not let it stop?

Geoffrey Huntley, a developer based in rural Australia (who, according to internet lore, lives on a property with goats), had a deceptively simple idea: what if every time Claude Code finishes and tries to exit, we just feed it the prompt again?

He named the technique “Ralph Wiggum” because the character embodies a kind of childlike persistence. Ralph repeatedly fails, makes silly mistakes, yet stubbornly continues in an endless loop until he eventually succeeds. Sound familiar? That is exactly how AI coding agents work. They make mistakes. They try again. They iterate.

The philosophy behind Ralph is beautifully stated: “Better to fail predictably than succeed unpredictably.” Every failure is just data for the next iteration.

How the Ralph loop works

At its core, the Ralph Wiggum technique is almost embarrassingly simple. It is a bash loop:

while true; do
 cat PROMPT.md | claude --print --dangerously-skip-permissions
done

That is it. That is the whole thing. You write your instructions in a PROMPT.md file, and the loop keeps feeding it to Claude Code over and over.

But here is why it works: each time Claude starts up, it looks at the project. It sees the files that exist, the code that is already written, the git history. It is not starting from zero. It picks up where it left off, sees what still needs to be done, and keeps going.

The official Claude Code plugin formalizes this with some important additions:

Completion promises: You tell Claude to output a specific string like <promise>COMPLETE</promise> when it has genuinely finished
Max iterations: A safety limit so your API bill does not reach infinity
Stop hooks: The mechanism that intercepts exit attempts and re-injects the prompt

To install it:

/plugin install ralph-wiggum@ghuntley

For end-to-end testing of deployed infrastructure, you will also want the Playwright MCP server. This lets Claude interact with your deployed application in a real browser:

claude mcp add playwright npx @playwright/mcp@latest

Why Pulumi is perfect for Ralph

Infrastructure as code has something most application code does not: objective success criteria. Your Lambda either deploys or it does not. Your DynamoDB table either exists or it does not. Tests either pass or they fail.

This makes Pulumi an ideal candidate for autonomous AI development:

Clear completion signals: pulumi preview and pulumi up provide unambiguous feedback
Iterative feedback: Failed deployments tell Claude exactly what went wrong
Testable outputs: You can write integration tests that verify your infrastructure actually works
Registry access: With the Pulumi MCP server, Claude has real-time access to documentation and examples

The experiment

I decided to build a serverless URL shortener on AWS. Not because the world needs another URL shortener, but because it is the perfect test case: multiple AWS services, clear requirements, and objective success criteria. But I did not want just a basic demo. I wanted a full SaaS-like experience with a polished frontend.

Here is the PROMPT.md file I wrote:

# Build a Production-Ready URL Shortener SaaS on AWS

Using Pulumi TypeScript, create a complete URL shortener SaaS with a polished frontend experience.

## Infrastructure Requirements
- DynamoDB table for storing URL mappings (shortCode -> originalUrl, clickCount, createdAt)
- Lambda functions for: creating short URLs, redirecting to original URLs, getting stats
- API Gateway REST API exposing the Lambda functions
- S3 bucket for the React frontend with static website hosting
- CloudFront distribution with HTTPS for the frontend and API
- Use the Pulumi ESC environment pulumi-idp/auth for the AWS credentials

## Frontend Requirements (Use /frontend-design skill)
Create a polished, production-ready SaaS website with:

### Landing Page
- Hero section with catchy headline and URL input form
- Feature highlights (fast redirects, analytics, custom aliases)
- Pricing section (Free tier, Pro tier, Enterprise tier)
- Testimonials section with 3 fake but realistic customer reviews
- Footer with links to Docs, Privacy, Terms

### Dashboard Page
- URL shortening form with optional custom alias
- List of created short URLs with click counts
- Copy-to-clipboard functionality
- Delete URL option

### Documentation Page
- Getting started guide
- API reference (POST /shorten, GET /{code}, GET /stats/{code})
- Rate limits and usage policies
- FAQ section

### Design Requirements
- Modern, clean aesthetic (no generic AI look)
- Responsive design (mobile, tablet, desktop)
- Dark mode support
- Smooth animations and transitions
- Consistent color scheme and typography

## API Requirements
- POST /shorten: accepts { url: string, alias?: string }, returns { shortCode, shortUrl }
- GET /{shortCode}: redirects to original URL (301)
- GET /stats/{shortCode}: returns { originalUrl, clickCount, createdAt }

## Success Criteria
- All unit tests pass
- All integration tests pass
- pulumi preview shows no errors
- pulumi up deploys successfully

## End-to-End Verification (Required)
After deployment, use Playwright MCP to verify the live site:
1. Open the CloudFront URL and verify the landing page loads with all sections
1. Navigate to the docs page and verify content renders
1. Create a short URL using the dashboard
1. Verify the short URL redirects correctly (301 status)
1. Check the stats show the click was recorded
1. Test responsive design by resizing the viewport
1. Take screenshots of landing page, dashboard, and docs as proof

Only output <promise>COMPLETE</promise> after ALL of the above including E2E tests pass.

The key addition here is the end-to-end verification section. Without it, Claude reports success after pulumi up finishes, but you have no guarantee the deployed infrastructure actually works. By requiring Playwright to test the live URLs, you catch issues like the CloudFront 403 error I mentioned earlier.

Before running the loop, I started Claude Code with permission bypass mode to prevent it from stopping to ask for approval on every file write or command:

claude --permission-mode bypassPermissions

There are other ways to handle permissions, but this is the simplest for autonomous execution. Then I ran the Ralph loop:

/ralph-wiggum:ralph-loop PROMPT.md --max-iterations 25 --completion-promise "COMPLETE"

Then I stepped away and let it run unsupervised. For complex projects, you could even let it run overnight while you sleep.

Here is Claude Code in action during the Ralph loop, fixing CloudFront configuration issues and writing unit tests:

The results

The complete source code for this project is available on GitHub: url-shortener-saas

When Claude finished, it had:

Successfully built:

A complete Pulumi program with proper resource organization
DynamoDB table with a GSI for querying by creation date
Three Lambda functions with proper error handling
API Gateway with CORS configuration
S3 bucket with static website hosting
CloudFront distribution with proper cache behaviors

The frontend was genuinely impressive:

A polished landing page with hero section, feature cards, and pricing tiers
Working testimonials section with three fictional but believable customer reviews
A functional dashboard where you could create and manage short URLs
A documentation page with API reference and getting started guide
Dark mode toggle that actually worked
Responsive design that looked good on mobile

Home screen

Dashboard

Documentation

Here is a snippet of what it generated:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const stack = pulumi.getStack();
const projectName = "url-shortener";

// DynamoDB Table
const urlTable = new aws.dynamodb.Table("url-table", {
 name: `${projectName}-urls-${stack}`,
 billingMode: "PAY_PER_REQUEST",
 hashKey: "shortCode",
 attributes: [{ name: "shortCode", type: "S" }],
 tags: {
 Environment: stack,
 Project: projectName,
 ManagedBy: "Pulumi",
 },
});

// Lambda function for URL shortening
const shortenFunction = new aws.lambda.Function("shorten-function", {
 name: `${projectName}-shorten-${stack}`,
 runtime: aws.lambda.Runtime.NodeJS20dX,
 handler: "shorten.handler",
 role: lambdaRole.arn,
 timeout: 30,
 memorySize: 256,
 code: new pulumi.asset.AssetArchive({
 ".": new pulumi.asset.FileArchive("./lambda"),
 }),
 environment: {
 variables: {
 TABLE_NAME: urlTable.name,
 },
 },
});

// CloudFront Distribution with S3 and API Gateway origins
const distribution = new aws.cloudfront.Distribution("frontend-distribution", {
 enabled: true,
 defaultRootObject: "index.html",
 origins: [
 {
 domainName: frontendBucketWebsite.websiteEndpoint,
 originId: "S3Origin",
 customOriginConfig: {
 httpPort: 80,
 httpsPort: 443,
 originProtocolPolicy: "http-only",
 originSslProtocols: ["TLSv1.2"],
 },
 },
 {
 domainName: pulumi.interpolate`${api.id}.execute-api.${aws.config.region}.amazonaws.com`,
 originId: "APIGatewayOrigin",
 customOriginConfig: {
 httpPort: 80,
 httpsPort: 443,
 originProtocolPolicy: "https-only",
 originSslProtocols: ["TLSv1.2"],
 },
 },
 ],
 defaultCacheBehavior: {
 targetOriginId: "S3Origin",
 viewerProtocolPolicy: "redirect-to-https",
 allowedMethods: ["GET", "HEAD", "OPTIONS"],
 cachedMethods: ["GET", "HEAD"],
 forwardedValues: {
 queryString: false,
 cookies: { forward: "none" },
 },
 },
 orderedCacheBehaviors: [{
 pathPattern: "/api/*",
 targetOriginId: "APIGatewayOrigin",
 viewerProtocolPolicy: "https-only",
 allowedMethods: ["DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT"],
 cachedMethods: ["GET", "HEAD"],
 forwardedValues: {
 queryString: true,
 headers: ["Authorization", "Content-Type"],
 cookies: { forward: "none" },
 },
 minTtl: 0,
 defaultTtl: 0,
 maxTtl: 0,
 }],
 viewerCertificate: {
 cloudfrontDefaultCertificate: true,
 },
});

Where it struggled:

The first few iterations had the Lambda code inline instead of in separate files
It initially forgot to set up IAM permissions for the Lambda to access DynamoDB
The CloudFront configuration took three attempts to get the cache behaviors right

The funny parts:

At iteration 12, it decided to completely refactor the project structure. Then at iteration 13, it refactored it back
One commit message simply read: “fix the fix that fixed the previous fix”
It wrote a test, ran the test, fixed the code to pass the test, then realized the test was wrong

Best practices for Ralph with Pulumi

After running several experiments, here is what I have learned:

Write extremely specific prompts. Vague instructions lead to vague implementations. Be explicit about every requirement, every edge case, every success criterion.

Use pulumi preview as your feedback loop. Add this to your success criteria:

Success Criteria:
- pulumi preview completes with no errors
- pulumi preview shows expected resource count (approximately X resources)

Set realistic iteration limits. For infrastructure projects, I have found 20-30 iterations is usually sufficient. More than that and you are probably missing something in your prompt.

Include test requirements. Claude is surprisingly good at test-driven development when you tell it to write tests first:

## Development Approach
1. Write unit tests for each Lambda function first
1. Implement Lambda functions to pass the tests
1. Write integration tests that verify the deployed infrastructure
1. Only output COMPLETE when all tests pass

Mention cost awareness. Add a note like “Use PAY_PER_REQUEST billing for DynamoDB” or “Use the smallest Lambda memory allocation that works.” Claude will optimize for what you tell it to optimize for.

The honest assessment

Is this the future of infrastructure development? Probably not entirely. But it is a genuinely useful technique for specific scenarios:

Great for:

Greenfield projects where you have clear requirements
Large migrations or refactors
Building proof-of-concept infrastructure while you focus on other work
Tasks where iteration is more valuable than perfection on the first try

Not great for:

Production infrastructure changes (please do not let an AI modify your production AWS account unsupervised)
Complex architecture decisions that require human judgment
Anything involving sensitive data or security configurations
Small tweaks or quick fixes (overkill)

The API costs were reasonable for what I got. For a complete infrastructure project built autonomously, a few dollars in Claude API calls is a bargain compared to my hourly rate.

Try it yourself

If you want to experiment with Ralph Wiggum and Pulumi:

Install Claude Code and the Ralph Wiggum plugin
Set up your Pulumi account and AWS credentials
Write a detailed PROMPT.md with clear success criteria
Start with a small project and low iteration limits
Review the git history when it finishes

The combination of autonomous AI loops and infrastructure as code feels like a glimpse of where development is heading. Not replacing developers, but changing how we use our time. Instead of babysitting an AI through each step, you hand it a well-defined problem and come back to working infrastructure.

Just make sure you have billing alerts set up. Ralph Wiggum does not know when to stop spending your money.

Or try Pulumi Neo: Ralph built-in

If setting up the Ralph Wiggum plugin feels like too much ceremony, Pulumi has something similar built right into Pulumi Neo.

Neo Tasks work on the same principle: you describe what you want, and Neo plans and executes the infrastructure changes. The key is auto mode. When you set a task to auto mode, Neo runs without requesting approvals. It plans, executes, validates with pulumi preview, and iterates until the task is complete.

The experience is remarkably similar to Ralph:

Persistent execution: Tasks continue running even if you close your browser. Come back later and Neo shows you what it accomplished while you were away.
Iterative refinement: Failed deployments inform the next attempt, just like Ralph re-reading the project state.
Clear completion: Neo knows when the infrastructure matches your requirements.

The main difference is that Neo runs in Pulumi Cloud rather than on your local machine, and it is purpose-built for infrastructure tasks. You get the autonomous loop experience without needing to configure plugins or bash loops.

For teams already using Pulumi Cloud, Neo with auto mode might be the fastest path to autonomous infrastructure development.

Resources

Kubernetes ConfigMap Revisions with Pulumi

Matan Baruch — Tue, 13 Jan 2026 00:00:00 +0000

ConfigMaps in Kubernetes don’t have built-in revision support, which can create challenges when deploying applications with canary strategies.

When using Argo Rollouts with AWS Spot instances, ConfigMap deletions during canary deployments can cause older pods to fail when they try to reload configuration. We solved this by implementing a custom ConfigMap revision system using Pulumi’s ConfigMapPatch and Kubernetes owner references.

The Problem

When deploying applications to Kubernetes using canary strategies with Argo Rollouts, we encountered a specific challenge:

Pulumi ConfigMap replacement behavior: By default, when a ConfigMap’s data changes, Pulumi may replace it rather than update it in place, which for auto-named ConfigMaps results in a new generated name (suffix).
Canary deployment issues: During canary deployments, the old ConfigMap gets deleted, but older pods (especially on AWS Spot instances that can be replaced during canary) may fail to reload
No native revision support: Neither Kubernetes nor Pulumi natively supports ConfigMap revisions like they do for deployments

Going to KubeCon Europe 2026? Tame Kubernetes complexity with code.

Pulumi demos, platform engineering talks, AI-powered Neo in action, and yes, free plushies. Booth 784.

The solution: ConfigMap revisions with owner references

Our solution leverages Kubernetes’ garbage collection mechanism by using owner references to tie ConfigMaps to ReplicaSets created during canary deployments.

Pulumi makes this approach practical by letting us express patching logic, owner references, and rollout coordination directly in code, instead of encoding complex behavior in static YAML.

Key components

Pulumi’s ConfigMapPatch: Patches existing ConfigMaps with owner references
ReplicaSet Owner References: Links ConfigMaps to ReplicaSets for automatic cleanup
Kubernetes Garbage Collection: Automatically cleans up ConfigMaps when ReplicaSets are deleted
Retain on Delete: Protects ConfigMaps from immediate deletion during Pulumi updates

Implementation

Here’s how we implemented this solution in our rollout component:

import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";
import * as k8sClient from "@kubernetes/client-node";

interface RolloutComponentArgs {
 namespace: string;
 configMapPatch?: boolean;
 kubeconfig: pulumi.Output<any>;
 configMapName: pulumi.Output<string>;
 rolloutSpec: k8s.types.input.apiextensions.CustomResourceArgs["spec"];
}

export class ConfigMapRevisionRollout extends pulumi.ComponentResource {
 public readonly rollout: k8s.apiextensions.CustomResource;

 constructor(
 name: string,
 args: RolloutComponentArgs,
 opts?: pulumi.ComponentResourceOptions
 ) {
 super("pulumi:component:ConfigMapRevisionRollout", name, {}, opts);

 // Create the Argo Rollout using CustomResource
 this.rollout = new k8s.apiextensions.CustomResource(
 `${name}-rollout`,
 {
 apiVersion: "argoproj.io/v1alpha1",
 kind: "Rollout",
 metadata: {
 name: name,
 namespace: args.namespace,
 },
 spec: args.rolloutSpec,
 },
 { parent: this, ...opts }
 );

 // Apply ConfigMap revision patching if enabled
 if (args.configMapPatch) {
 this.setupConfigMapRevisions(name, args);
 }

 this.registerOutputs({
 rollout: this.rollout,
 });
 }

 private setupConfigMapRevisions(name: string, args: RolloutComponentArgs): void {
 pulumi
 .all([args.kubeconfig, args.configMapName])
 .apply(async ([kubeconfig, configMapName]) => {
 try {
 // Create Server-Side Apply enabled provider
 const ssaProvider = new k8s.Provider(`${name}-ssa-provider`, {
 kubeconfig: JSON.stringify(kubeconfig),
 enableServerSideApply: true,
 });

 // Wait for rollout to stabilize and create ReplicaSets
 await this.waitForRolloutStabilization();

 // Get ReplicaSets associated with this rollout
 const replicaSets = await this.getAssociatedReplicaSets(
 args.namespace,
 configMapName,
 kubeconfig
 );

 if (replicaSets.length === 0) {
 pulumi.log.warn("No ReplicaSets found for ConfigMap patching");
 return;
 }

 // Create owner references for the ConfigMap
 const ownerReferences = replicaSets.map(rs => ({
 apiVersion: "apps/v1",
 kind: "ReplicaSet",
 name: rs.metadata?.name!,
 uid: rs.metadata?.uid!,
 controller: false,
 blockOwnerDeletion: false,
 }));

 // Patch the ConfigMap with owner references
 new k8s.core.v1.ConfigMapPatch(
 `${configMapName}-revision-patch`,
 {
 metadata: {
 name: configMapName,
 namespace: args.namespace,
 ownerReferences: ownerReferences,
 annotations: {
 "pulumi.com/patchForce": "true",
 "configmap.kubernetes.io/revision-managed": "true",
 },
 },
 },
 {
 provider: ssaProvider,
 retainOnDelete: true,
 parent: this,
 }
 );

 pulumi.log.info(
 `Successfully patched ConfigMap ${configMapName} with ${ownerReferences.length} owner references`
 );
 } catch (error) {
 pulumi.log.error(`Failed to setup ConfigMap revisions: ${error}`);
 throw error;
 }
 });
 }

 private async waitForRolloutStabilization(): Promise<void> {
 // Wait for rollout to create and stabilize ReplicaSets
 // In production, consider using a more sophisticated polling mechanism
 await new Promise(resolve => setTimeout(resolve, 10000));
 }

 private async getAssociatedReplicaSets(
 namespace: string,
 configMapName: string,
 kubeconfig: any
 ): Promise<k8sClient.V1ReplicaSet[]> {
 const kc = new k8sClient.KubeConfig();
 kc.loadFromString(JSON.stringify(kubeconfig));

 const appsV1Api = kc.makeApiClient(k8sClient.AppsV1Api);

 try {
 const response = await appsV1Api.listNamespacedReplicaSet(
 namespace,
 undefined, // pretty
 false, // allowWatchBookmarks
 undefined, // continue
 undefined, // fieldSelector
 `configMap=${configMapName}` // labelSelector
 );

 return response.body.items;
 } catch (error) {
 pulumi.log.error(`Failed to list ReplicaSets: ${error}`);
 return [];
 }
 }
}

How do ConfigMap revisions work with Argo Rollouts?

Rollout Creation: When a new rollout is created, Argo Rollouts generates new ReplicaSets for the canary deployment
ConfigMap Patching: Our code waits for the ReplicaSet creation, then patches the ConfigMap with owner references pointing to these ReplicaSets
Garbage Collection: Kubernetes automatically tracks the relationship between ConfigMaps and ReplicaSets
Automatic Cleanup: When ReplicaSets are cleaned up (based on the default 10 revision history), their associated ConfigMaps are also garbage collected

Benefits

Revision Control: ConfigMaps now have revision-like behavior tied to ReplicaSet history
Automatic Cleanup: No manual intervention needed for ConfigMap cleanup
Canary Safety: Old ConfigMaps remain available during canary deployments until ReplicaSets are cleaned up
Spot Instance Resilience: Pods that get replaced during canary deployments can still access their original ConfigMaps

Configuration options

interface RolloutComponentArgs {
 namespace: string;
 configMapPatch?: boolean;
 kubeconfig: pulumi.Output<any>;
 configMapName: pulumi.Output<string>;
 rolloutSpec: k8s.types.input.apiextensions.CustomResourceArgs["spec"];
}

To enable this feature in your rollout:

import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

// Create your EKS cluster
const cluster = new k8s.Provider("k8s-provider", {
 kubeconfig: clusterKubeconfig,
});

// Create ConfigMap
const appConfig = new k8s.core.v1.ConfigMap("app-config", {
 metadata: {
 name: "my-app-config",
 namespace: "default",
 labels: {
 app: "my-app",
 configMap: "my-app-config", // Important for ReplicaSet selection
 },
 },
 data: {
 "app.properties": "key=value\nother=setting",
 },
}, { provider: cluster });

// Create rollout with ConfigMap revision management
const rollout = new ConfigMapRevisionRollout("my-app", {
 namespace: "default",
 configMapPatch: true,
 kubeconfig: clusterKubeconfig,
 configMapName: appConfig.metadata.name,
 rolloutSpec: {
 replicas: 3,
 selector: {
 matchLabels: { app: "my-app" },
 },
 template: {
 metadata: {
 labels: { app: "my-app" },
 },
 spec: {
 containers: [{
 name: "app",
 image: "nginx:latest",
 volumeMounts: [{
 name: "config",
 mountPath: "/etc/config",
 }],
 }],
 volumes: [{
 name: "config",
 configMap: {
 name: appConfig.metadata.name,
 },
 }],
 },
 },
 strategy: {
 canary: {
 maxSurge: 1,
 maxUnavailable: 0,
 steps: [
 { setWeight: 20 },
 { pause: { duration: "1m" } },
 { setWeight: 50 },
 { pause: { duration: "2m" } },
 ],
 },
 },
 },
});

Key dependencies

The solution uses several key packages:

@pulumi/kubernetes: For Kubernetes resources and ConfigMapPatch
@kubernetes/client-node: For direct Kubernetes API access
Argo Rollouts CRDs installed in your cluster

When should you use this approach?

This pattern is a good fit if you:

Use Argo Rollouts with canary deployments
Rely on ConfigMaps that must remain available across rollout revisions
Run workloads on Spot or preemptible instances
Need automatic cleanup without custom controllers

Conclusion

This approach gives us ConfigMap revision functionality that doesn’t exist natively in Kubernetes or Pulumi. By leveraging Kubernetes’ garbage collection mechanism and Pulumi’s patching capabilities, we created a robust solution for managing ConfigMap lifecycles during canary deployments.

The solution is particularly valuable when:

Running canary deployments with Argo Rollouts
Using AWS Spot instances that can be replaced during deployments
Needing automatic cleanup of old ConfigMaps without manual intervention
Wanting to maintain configuration availability for older pods during deployment transitions

This pattern can be extended to other scenarios where you need revision control for Kubernetes resources that don’t natively support it.

Want to learn how to put these practices into action? Meet us at KubeCon Europe 2026 (Booth 784) or register for our upcoming Zero to Production in Kubernetes workshop.

Try Pulumi for Free

How to Move to the Gateway API: post ingress-nginx Retirement

Engin Diri — Fri, 02 Jan 2026 11:35:51 +0100

The upcoming retirement of ingress-nginx in early 2026 gives infrastructure teams both a deadline and an opportunity to rethink traffic management. Configuring the Ingress API often meant relying on controller-specific annotations that varied between implementations. The Gateway API offers a cleaner, standardized alternative. This post investigates the practical reality of this migration and explores why kgateway emerges as a robust solution for the future.

With ingress-nginx entering its sunset phase in early 2026, the Kubernetes community faces a decision point. While the controller served as the default standard for a decade, its architecture now struggles to meet modern requirements. The transition to the Gateway API offers a chance to adopt a standard designed for contemporary traffic patterns.

The Gateway API addresses the portability issues of its predecessor by establishing a standardized, expressive approach that behaves consistently across implementations. A technical evaluation of the available options points to kgateway as a particularly strong candidate for production workloads.

Why ingress-nginx is retiring

The Kubernetes SIG Network and Security Response Committee was direct in its announcement regarding the project’s future. After March 2026, ingress-nginx will cease to receive releases, bug fixes, or security patches. The repositories will become read-only, leaving existing deployments functional but unmaintained.

This decision stems from both resource constraints and technical debt. The project relied on a very small group of maintainers working primarily in their spare time. Furthermore, features that once provided flexibility, such as “snippets” for arbitrary NGINX configuration injection, now pose significant security liabilities. With no path to modernize the codebase, retirement was inevitable.

It is worth noting that the Ingress API itself remains supported. The retirement affects only the ingress-nginx controller. NGINX as a web server continues unchanged. However, clusters relying on this specific controller must prepare for a transition.

To identify affected resources, checking the cluster for specific pods can reveal the scope of the dependency:

kubectl get pods --all-namespaces --selector app.kubernetes.io/name=ingress-nginx

Understanding the Gateway API

The Gateway API represents a fundamental shift in traffic management concepts. Some elements will look familiar to Ingress users, but the underlying philosophy addresses the “baggage” that the previous standard accumulated.

The core improvement lies in expressiveness. Advanced routing requirements (header matching, weighted traffic splitting, and traffic policies) are now native parts of the specification. This eliminates the need for the non-portable annotations that plagued the Ingress ecosystem.

The API also introduces a role-oriented structure that mirrors actual organizational workflows.

GatewayClass resources are managed by infrastructure teams to define the underlying controller.
Gateway resources are created by platform teams to specify entry points and listeners.
HTTPRoute resources are owned by application teams to define service traffic rules.

This separation allows for genuine self-service. Application teams can manage their routing logic without requiring broad cluster privileges.

The relationship between these resources defines the traffic flow:

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
 name: kgateway
spec:
 controllerName: kgateway.dev/kgateway

A Gateway references this class to establish the entry point:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
 name: my-gateway
 namespace: default
spec:
 gatewayClassName: kgateway
 listeners:
 - name: http
 port: 8080
 protocol: HTTP
 allowedRoutes:
 namespaces:
 from: All

An HTTPRoute then attaches to the gateway:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
 name: my-route
 namespace: default
spec:
 parentRefs:
 - name: my-gateway
 hostnames:
 - "example.com"
 rules:
 - matches:
 - path:
 type: PathPrefix
 value: /api
 backendRefs:
 - name: api-service
 port: 8080

Beyond standard HTTP, the API supports GRPCRoute, TCPRoute, UDPRoute, and TLSRoute, offering a protocol diversity that the original Ingress spec lacked.

Cross-namespace routing and ReferenceGrant

The Gateway API enables a security model where routes can cross namespace boundaries, but only with explicit permission. The ReferenceGrant resource controls these connections. Without a grant, an HTTPRoute in one namespace cannot reference a Service in another.

apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
 name: allow-routes-from-apps
 namespace: gateway-system
spec:
 from:
 - group: gateway.networking.k8s.io
 kind: HTTPRoute
 namespace: my-app
 to:
 - group: ""
 kind: Service

This mechanism allows platform teams to strictly control which namespaces can attach to shared gateways.

Evaluating Gateway API implementations

Several implementations have emerged to support the new standard, each with a distinct philosophy.

NGINX Gateway Fabric leverages NGINX as the data plane, offering continuity for teams in the F5 ecosystem. It provides impressive throughput and integrates with enterprise security tools, making it a logical choice for existing NGINX shops.

Traefik focuses on simplicity. As a single binary with no external dependencies, it aligns well with environments that prioritize developer experience and rapid iteration. Its declarative configuration is particularly friendly to GitOps workflows.

Envoy Gateway is built on the Envoy Proxy and operates under a community-driven governance model. It appeals to those seeking strong conformance to the Gateway API spec without vendor lock-in. Its companion project, Envoy AI Gateway, adds support for LLM routing.

kgateway, formerly Gloo Gateway, brings seven years of production history to the table. Donated to the CNCF in early 2025, it combines an Envoy data plane with a control plane optimized for scale. Its internal architecture uses the “krt” framework to handle massive route tables efficiently, avoiding the performance bottlenecks often seen in snapshot-based systems.

Aspect	NGINX Gateway Fabric	Traefik	Envoy Gateway	kgateway
Base Technology	NGINX (C)	Custom Go	Envoy Proxy (C++)	Envoy Proxy (C++)
Maturity	Production-ready	Mature	GA (v1.0+)	Battle-tested (7+ years as Gloo)
Setup Complexity	Medium	Low	Low	Low
Commercial Support	F5 Enterprise	Traefik Labs	Multiple vendors	Solo.io
AI/LLM Support	No	No	Via Envoy AI Gateway	Native (Agentgateway)
Learning Curve	Familiar for NGINX users	Gentle	Moderate	Moderate
Community Size	Growing	Large	Large	Established
Best For	Teams with NGINX expertise, high-throughput requirements	Teams prioritizing simplicity and rapid deployment	Standards-focused teams, multi-vendor environments	Enterprise scale, AI workloads, Istio integration

Decision framework

NGINX Gateway Fabric suits teams needing F5 support or deep NGINX tuning.
Traefik fits best where simplicity and operational ease are paramount.
Envoy Gateway works for those valuing community governance and strict standards adherence.
kgateway is the choice for enterprise scale, native AI gateway needs, and Istio integration.

This exploration uses kgateway for its proven track record and native handling of both traditional microservices and AI traffic.

Migrating from Ingress to Gateway API

The official migration guide outlines the transition process. It’s less translation, more restructuring.

The process typically involves:

Defining a Gateway resource: Unlike Ingress, listeners must be explicitly defined. This provides granular control over ports and protocols.
Creating HTTPRoute resources: These replace Ingress routing rules. You can split complex Ingress resources into multiple HTTPRoutes for better management.
Configuring filters: Annotations for headers or redirects are replaced by standardized filters within the HTTPRoute spec.

The ingress2gateway tool can automate much of the initial conversion, providing a baseline of resources to review and refine.

Migration planning

Scope the migration effort first:

1. Assessment Identify the volume and complexity of existing resources.

# Count your Ingress resources across all namespaces
kubectl get ingress -A | wc -l

# List all Ingress resources with their hosts
kubectl get ingress -A -o jsonpath='{range .items[*]}{.metadata.namespace}/{.metadata.name}: {.spec.rules[*].host}{"\n"}{end}'

2. Parallel Operations Running kgateway alongside ingress-nginx allows for incremental migration. Services can be moved one by one, validating the new configuration without risking the entire cluster’s traffic.

3. Rollback Strategy Keeping the original Ingress manifests ensures that traffic can be quickly reverted to the old controller if issues arise during the transition.

Why kgateway stands out

kgateway distinguishes itself through its maturity and architectural decisions. Having started as Gloo Gateway in 2018, it has hardened through years of production use.

Its control plane uses the krt framework, originally developed for Istio. This allows it to track dependencies precisely and recalculate only the parts of the configuration that have changed. In large clusters where pods churn constantly, this incremental approach prevents the control plane from becoming a bottleneck, enabling it to handle over 10,000 routes efficiently.

For teams managing AI workloads, kgateway includes the Agentgateway component. This Rust-based data plane is built for LLM traffic, supporting the Model Context Protocol (MCP) and providing token-based rate limiting. It unifies the management of AI and traditional traffic under a single control plane.

Additional features include:

Route Delegation: Allows large routing tables to be split across teams with clear inheritance.
Security Integration: Seamless mTLS with Istio ambient mesh and external authorization support.
Traffic Management: Advanced capabilities like traffic mirroring and session affinity are available without complex Envoy configuration.

Putting it all together with Pulumi

Seeing these concepts in code clarifies the architecture. The following Pulumi program demonstrates a complete setup: a DigitalOcean Kubernetes cluster, the kgateway installation via Helm, Gateway API resources, and an httpbin application to validate the configuration.

The program provisions infrastructure in a specific sequence. The cluster comes first, followed by the Gateway API CRDs, then kgateway itself. Once the control plane is running, the program creates a GatewayClass, a Gateway listener, and an HTTPRoute that directs traffic to the sample application. A ReferenceGrant permits the cross-namespace reference between the HTTPRoute and the backend Service.

import * as pulumi from "@pulumi/pulumi";
import * as digitalocean from "@pulumi/digitalocean";
import * as k8s from "@pulumi/kubernetes";

// Configuration
const config = new pulumi.Config();
const clusterName = config.get("clusterName") || "kgateway-demo-cluster";
const region = config.get("region") || "fra1";
const nodeSize = config.get("nodeSize") || "s-2vcpu-4gb";
const nodeCount = config.getNumber("nodeCount") || 2;
const k8sVersion = config.get("k8sVersion") || "1.32.10-do.2";

// Create a DigitalOcean Kubernetes cluster
const cluster = new digitalocean.KubernetesCluster(clusterName, {
 name: clusterName,
 region: region,
 version: k8sVersion,
 nodePool: {
 name: "default-pool",
 size: nodeSize,
 nodeCount: nodeCount,
 },
});

// Create a Kubernetes provider using the cluster's kubeconfig
const k8sProvider = new k8s.Provider("k8s-provider", {
 kubeconfig: cluster.kubeConfigs[0].rawConfig,
});

// Install Gateway API CRDs (standard channel)
const gatewayApiCrds = new k8s.yaml.ConfigFile("gateway-api-crds", {
 file: "https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/standard-install.yaml",
}, { provider: k8sProvider });

// Create kgateway-system namespace
const kgatewayNamespace = new k8s.core.v1.Namespace("kgateway-system", {
 metadata: {
 name: "kgateway-system",
 },
}, { provider: k8sProvider });

// Install kgateway CRDs via Helm
const kgatewayCrds = new k8s.helm.v3.Release("kgateway-crds", {
 chart: "oci://cr.kgateway.dev/kgateway-dev/charts/kgateway-crds",
 version: "v2.1.2",
 namespace: kgatewayNamespace.metadata.name,
 createNamespace: false,
}, { provider: k8sProvider, dependsOn: [gatewayApiCrds, kgatewayNamespace] });

// Install kgateway control plane via Helm
const kgateway = new k8s.helm.v3.Release("kgateway", {
 chart: "oci://cr.kgateway.dev/kgateway-dev/charts/kgateway",
 version: "v2.1.2",
 namespace: kgatewayNamespace.metadata.name,
 createNamespace: false,
}, { provider: k8sProvider, dependsOn: [kgatewayCrds] });

// Create hello-world namespace
const helloWorldNamespace = new k8s.core.v1.Namespace("hello-world", {
 metadata: {
 name: "hello-world",
 },
}, { provider: k8sProvider });

// Deploy hello world application (using httpbin as shown in kgateway docs)
const helloWorldLabels = { app: "httpbin" };

const helloWorldDeployment = new k8s.apps.v1.Deployment("httpbin", {
 metadata: {
 name: "httpbin",
 namespace: helloWorldNamespace.metadata.name,
 },
 spec: {
 replicas: 1,
 selector: {
 matchLabels: helloWorldLabels,
 },
 template: {
 metadata: {
 labels: helloWorldLabels,
 },
 spec: {
 containers: [{
 name: "httpbin",
 image: "mccutchen/go-httpbin:v2.15.0",
 ports: [{
 containerPort: 8080,
 name: "http",
 }],
 }],
 },
 },
 },
}, { provider: k8sProvider, dependsOn: [helloWorldNamespace] });

const helloWorldService = new k8s.core.v1.Service("httpbin", {
 metadata: {
 name: "httpbin",
 namespace: helloWorldNamespace.metadata.name,
 },
 spec: {
 selector: helloWorldLabels,
 ports: [{
 port: 8000,
 targetPort: 8080,
 name: "http",
 }],
 },
}, { provider: k8sProvider, dependsOn: [helloWorldDeployment] });

// Create GatewayClass for kgateway
const gatewayClass = new k8s.apiextensions.CustomResource("kgateway-gateway-class", {
 apiVersion: "gateway.networking.k8s.io/v1",
 kind: "GatewayClass",
 metadata: {
 name: "kgateway",
 },
 spec: {
 controllerName: "kgateway.dev/kgateway",
 },
}, { provider: k8sProvider, dependsOn: [kgateway] });

// Create Gateway with LoadBalancer service (matching kgateway docs pattern)
const gateway = new k8s.apiextensions.CustomResource("http-gateway", {
 apiVersion: "gateway.networking.k8s.io/v1",
 kind: "Gateway",
 metadata: {
 name: "http",
 namespace: kgatewayNamespace.metadata.name,
 },
 spec: {
 gatewayClassName: "kgateway",
 listeners: [{
 name: "http",
 port: 8080,
 protocol: "HTTP",
 allowedRoutes: {
 namespaces: {
 from: "All",
 },
 },
 }],
 },
}, { provider: k8sProvider, dependsOn: [gatewayClass] });

// Create HTTPRoute for httpbin application
const httpRoute = new k8s.apiextensions.CustomResource("httpbin-route", {
 apiVersion: "gateway.networking.k8s.io/v1",
 kind: "HTTPRoute",
 metadata: {
 name: "httpbin",
 namespace: helloWorldNamespace.metadata.name,
 },
 spec: {
 parentRefs: [{
 name: "http",
 namespace: kgatewayNamespace.metadata.name,
 }],
 hostnames: [
 "*.nip.io",
 ],
 rules: [{
 backendRefs: [{
 name: "httpbin",
 port: 8000,
 }],
 }],
 },
}, { provider: k8sProvider, dependsOn: [gateway, helloWorldService] });

// Create a reference grant to allow the HTTPRoute to reference the service cross-namespace
const referenceGrant = new k8s.apiextensions.CustomResource("httpbin-reference-grant", {
 apiVersion: "gateway.networking.k8s.io/v1beta1",
 kind: "ReferenceGrant",
 metadata: {
 name: "allow-gateway-to-httpbin",
 namespace: helloWorldNamespace.metadata.name,
 },
 spec: {
 from: [{
 group: "gateway.networking.k8s.io",
 kind: "HTTPRoute",
 namespace: helloWorldNamespace.metadata.name,
 }],
 to: [{
 group: "",
 kind: "Service",
 }],
 },
}, { provider: k8sProvider, dependsOn: [gatewayApiCrds] });

// Create a LoadBalancer Service for the Gateway proxy
const gatewayProxyService = new k8s.core.v1.Service("gateway-proxy", {
 metadata: {
 name: "gateway-proxy",
 namespace: kgatewayNamespace.metadata.name,
 },
 spec: {
 type: "LoadBalancer",
 selector: {
 "gateway.networking.k8s.io/gateway-name": "http",
 },
 ports: [{
 name: "http",
 port: 8080,
 targetPort: 8080,
 protocol: "TCP",
 }],
 },
}, { provider: k8sProvider, dependsOn: [gateway] });

// Extract the LoadBalancer IP from our managed Service
const gatewayIP = gatewayProxyService.status.apply(status => {
 const ingress = status?.loadBalancer?.ingress;
 if (ingress && ingress.length > 0) {
 return ingress[0].ip || ingress[0].hostname || "pending";
 }
 return "pending";
});

// Exports
export const clusterNameOutput = cluster.name;
export const clusterEndpoint = cluster.endpoint;
export const kubeconfig = pulumi.secret(cluster.kubeConfigs[0].rawConfig);

// Export the LoadBalancer IP and nip.io URL
export const gatewayLoadBalancerIP = gatewayIP;
export const httpbinUrl = gatewayIP.apply(ip =>
 ip !== "pending" ? `http://httpbin.${ip}.nip.io:8080/get` : "Waiting for LoadBalancer IP..."
);

The demo uses nip.io, a wildcard DNS service that resolves *.IP.nip.io hostnames to the specified IP address. This eliminates the need to configure separate DNS records during testing. Once deployed, the httpbinUrl output provides a ready-to-use endpoint for validating the gateway configuration.

The short-term option: Chainguard’s ingress-nginx fork

For organizations that cannot immediately migrate to the Gateway API, there is another path forward. Chainguard has forked ingress-nginx as part of their EmeritOSS program, providing continued maintenance after the upstream project’s retirement.

This fork is not about adding new features. Chainguard is explicit that they are maintaining stability, not continuing development. Their commitment includes keeping dependencies updated, addressing CVEs on a best-efforts basis, and providing commercial container images with low vulnerability counts and SLAs. FIPS-compliant versions are also available for regulated environments.

The chainguard-forks/ingress-nginx repository on GitHub provides the maintained codebase. For teams running ingress-nginx in production, this fork offers a viable bridge while evaluating the Gateway API or other alternatives. However, this buys time rather than solving the problem permanently. The architectural limitations of the Ingress API remain, and the eventual move to a more expressive standard like the Gateway API is still the recommended path forward.

Looking ahead

The retirement of ingress-nginx marks the end of an era and the beginning of a more structured approach to Kubernetes networking. The Gateway API offers a robust framework that reflects how teams actually build and secure applications today.

While March 2026 provides a comfortable runway, early planning prevents rushed decisions. Setting up a test environment with kgateway to validate the new API constructs is a prudent next step.

Moving to the Gateway API means building a networking foundation ready for the next decade of infrastructure challenges.

Get Started with Pulumi

Use Pulumi's open-source SDK to create, deploy, and manage infrastructure on any cloud.

Resources

Gateway API Documentation – Comprehensive guides and examples.
Migration Guide – Official steps for transitioning from Ingress.
ingress2gateway – Tooling to automate resource conversion.
kgateway Documentation – Installation and configuration details.
Designing kgateway for Scalability – Deep dive into the krt framework.
Kubernetes Blog: ingress-nginx Retirement – Full context on the deprecation.

From 'Works on My Machine' to Production-Ready: Building AI Agents with Amazon Bedrock AgentCore

Engin Diri — Mon, 15 Dec 2025 00:00:00 +0200

Every developer building AI agents knows the gap between a working prototype and production deployment. Your fraud detection agent works perfectly on your laptop, but how do you deploy it with proper authentication, memory persistence, observability, and guardrails? This post walks through a complete journey from local development to production-ready AI agents using Amazon Bedrock AgentCore, the Strands SDK, and Pulumi.

TL;DR: This post shows how to deploy AI agents from local prototype to production using:

Strands SDK for agent development
Amazon Bedrock AgentCore for managed runtime with Firecracker isolation
AgentCore Memory for short-term and long-term context
MCP Gateway for secure tool integration with OAuth2
Pulumi for infrastructure as code

Complete code: GitHub

The Challenge: Bridging the Gap

Building AI agents is getting easier. Tools like the Strands Agents SDK let you create functional agents in just a few lines of code. Production deployment introduces challenges:

Stateless to Stateful: How does your agent remember previous conversations?
Single-tenant to Multi-tenant: How do you isolate different users’ data?
Implicit Trust to Zero Trust: How do you authenticate requests and enforce boundaries?
Console Logs to Observability: How do you trace requests across distributed systems?
Ad-hoc Tools to Managed Services: How do you expose tools securely at scale?

The approach is progressive: start simple, add complexity only when needed, and use infrastructure as code to make it reproducible.

Agents Beyond Chat: Event-Driven AI

When most people think of AI agents, they imagine chatbots: interactive systems responding to user questions. But many production agents never interact with end users directly. Instead, they’re triggered by event streams: Kafka topics, SQS queues, webhook callbacks, or scheduled jobs.

Our fraud detection agent is a prime example. It doesn’t wait for customers to ask questions. It’s triggered automatically when the payment processing system detects a new transaction. The “user” is another system, and the “conversation” is a structured event payload.

This event-driven pattern is becoming the new stack for AI agents. As Kai Waehner describes in his analysis of agentic AI with Kafka and Flink:

“Protocols like A2A and MCP are essential for standardizing agent behavior and communication. But without an event-driven substrate like Kafka and a stream-native runtime like Flink, these agents remain stuck in isolated interactions.”

The emerging architecture looks like this:

In production, you might use Amazon MSK (Managed Kafka), Amazon SQS, or EventBridge to deliver transaction events to your agent. The agent processes each event autonomously, takes action (blocking cards, creating tickets), and publishes results back to another topic.

What We’re Building: A Fraud Detection Agent

Our demo is a fraud detection agent that:

Receives transaction alerts with user ID, amount, merchant, and location
Gathers context from tools (user profiles, transaction history)
Analyzes for fraud patterns (impossible travel, velocity attacks, amount anomalies)
Takes protective action (blocking cards) when fraud is detected

We’ll build this progressively:

Stage	What Changes	Key Additions
Local Development	Running on your laptop	Strands SDK, Claude Opus 4.5, mock tools
First Deployment	Running in AgentCore	Containerized, Firecracker isolation, guardrails
Production-Ready	Full enterprise features	Memory, MCP Gateway, X-Ray observability

Local Development with Strands SDK

The Strands Agents SDK is an open-source framework from AWS that takes a model-driven approach to agent development. Instead of hardcoding complex workflows, you let the LLM reason about tool usage autonomously.

Here’s our complete local agent in about 30 lines:

# agent.py
from strands import Agent
from strands.models import BedrockModel
from tools import get_user_profile, get_recent_transactions, block_credit_card

# Configure Bedrock model with cross-region inference for Claude Opus 4.5
# Note: Cross-region inference may have different pricing than single-region
bedrock_model = BedrockModel(
 model_id="us.anthropic.claude-opus-4-5-20251101-v1:0",
 region_name="us-east-1",
)

# Define the Fraud Detection Agent
fraud_agent = Agent(
 model=bedrock_model,
 tools=[get_user_profile, get_recent_transactions, block_credit_card],
 system_prompt="""
 You are a Senior Fraud Analyst Agent at a financial institution.
 Your mission is to protect customers from fraudulent transactions in real-time.

 When you receive a transaction alert:
 1. Gather context by checking the user's profile and recent transaction history.
 2. Analyze the transaction for fraud indicators:
 - Impossible travel: Can the user physically be at the new location?
 - Unusual amounts: Is this significantly larger than typical spending?
 - High-risk merchants: Electronics stores, gift cards, wire transfers.
 - Geographic anomalies: Transactions far from home location.
 3. Make a decision:
 - If fraud indicators are present, BLOCK the card immediately.
 - If the transaction appears legitimate, allow it to proceed.
 4. Document your reasoning clearly.

 Prioritize customer protection. When in doubt, block the card.
 """,
)

Tools are simple Python functions decorated with @tool:

from strands import tool

@tool
def get_user_profile(user_id: str) -> dict:
 """Get user profile including home location and account status."""
 return {
 "user_id": user_id,
 "name": "John Doe",
 "home_location": "London, UK",
 "account_status": "active",
 }

@tool
def block_credit_card(user_id: str, reason: str) -> dict:
 """Block a user's credit card and create a fraud ticket."""
 return {
 "status": "blocked",
 "ticket_id": f"FRAUD-{user_id}-001",
 "message": f"Card blocked for user {user_id}: {reason}",
 }

Running the Local Demo

Here’s how we simulate an incoming transaction event:

# demo_simulation.py
from agent import fraud_agent

# The "Trigger" Event (Simulated Payload from Kafka/SQS)
new_transaction_event = """
ALERT: New Transaction Attempt
User ID: user_123
Amount: $2000
Merchant: Electronics Store
Location: Tokyo, Japan
Time: 09:15
"""

def run_demo():
 print("--- INCOMING EVENT STREAM ---")
 print(new_transaction_event)
 print("--- AGENT ACTIVATED ---")

 # Run the agent - processes the event autonomously
 result = fraud_agent(new_transaction_event)

 print("--- ANALYSIS COMPLETE ---")

if __name__ == "__main__":
 run_demo()

When we run this, the agent autonomously gathers context, analyzes the transaction, and takes action:

$ python demo_simulation.py
--- INCOMING EVENT STREAM ---
ALERT: New Transaction Attempt
User ID: user_123
Amount: $2000
Merchant: Electronics Store
Location: Tokyo, Japan
Time: 09:15
-----------------------------------
AGENT ACTIVATED. Analyzing...
Tool #1: get_user_profile
Tool #2: get_recent_transactions
## 🚨 FRAUD ANALYSIS REPORT
**Customer:** John Doe (user_123)
### Analysis:
#### 1. ❌ IMPOSSIBLE TRAVEL DETECTED
- **Last Transaction:** 09:00 in **London, UK**
- **Current Transaction:** 09:15 in **Tokyo, Japan**
- **Time Elapsed:** 15 minutes
- **Distance:** ~9,500 km (5,900 miles)
- **Minimum Flight Time:** ~12 hours
**Verdict:** It is physically impossible for John Doe to travel from London
to Tokyo in 15 minutes. This is a critical fraud indicator.
#### 2. ⚠️ HIGH-RISK MERCHANT
- Electronics stores are frequently targeted for fraud due to high resale value
#### 3. ⚠️ HIGH-VALUE TRANSACTION
- $2,000 at an electronics store represents a substantial purchase
### Decision: 🛑 BLOCK CARD IMMEDIATELY
Tool #3: block_credit_card
## ✅ ACTION COMPLETED
| Action | Status |
|--------|--------|
| Card Blocked | ✅ Confirmed |
| Ticket ID | TICKET-999 |
--- ANALYSIS COMPLETE ---

Notice how the agent reasoned about the event using tools and took autonomous action. This works great for development, but it lacks everything needed for production: authentication, persistence, isolation, monitoring, and governance.

First Deployment with AgentCore Runtime

Our local prototype works, but where do you actually run it? Lambda times out after 15 minutes. ECS and Kubernetes add operational overhead. And AI agents have different requirements than typical microservices: they reason through multi-step workflows, sometimes taking minutes to analyze a complex fraud case. You need something built for this workload.

Amazon Bedrock AgentCore fills this gap. It’s a managed runtime specifically designed for AI agents, with support for tasks running up to 8 hours. No timeout workarounds, no cluster management.

The security model is particularly relevant for fraud detection. AgentCore runs each session in a dedicated microVM using AWS Firecracker (the same isolation technology behind Lambda). When your agent processes one customer’s transactions, it’s completely isolated from other sessions. No shared memory, no cross-tenant contamination.

The deployment model is straightforward: containerize your agent and point AgentCore at the image. It works with Strands, LangGraph, CrewAI, or custom code. The same agent we built locally runs unchanged. We just wrap it in FastAPI and add a Dockerfile.

Cold Start and Latency

For fraud detection, latency matters. AgentCore’s Firecracker microVMs introduce a cold start penalty on the first invocation, typically 2-5 seconds to spin up the container. Subsequent invocations within the same session are fast (sub-second overhead beyond model inference time).

For real-time fraud detection, this cold start is acceptable because:

The LLM reasoning itself takes 5-15 seconds for complex analysis
Most production deployments use session pooling to keep warm instances available
The alternative (always-on containers) trades latency for cost

If sub-second total latency is critical, consider Lambda for simple rule-based checks and reserve AgentCore for complex reasoning that justifies the cold start overhead.

Containerizing the Agent

First, we wrap our agent in a FastAPI endpoint:

# agent.py (production version)
from fastapi import FastAPI, Request
from strands import Agent
from strands.models import BedrockModel
import os

app = FastAPI()

bedrock_model = BedrockModel(
 model_id="us.anthropic.claude-opus-4-5-20251101-v1:0",
 region_name="us-east-1",
 guardrail_id=os.environ.get("BEDROCK_GUARDRAIL_ID"),
 guardrail_version=os.environ.get("BEDROCK_GUARDRAIL_VERSION"),
 guardrail_trace="enabled",
)

fraud_agent = Agent(
 model=bedrock_model,
 tools=[get_user_profile, get_recent_transactions, block_credit_card],
 system_prompt="...",
)

@app.post("/invocations")
async def invoke(request: Request):
 body = await request.json()
 prompt = body.get("input", {}).get("prompt", "")
 result = fraud_agent(prompt)
 return {"output": {"message": str(result)}}

Adding Guardrails

Amazon Bedrock Guardrails help ensure your agent stays on task. We configure content filters and topic restrictions using Pulumi:

fraud_detection_guardrail = bedrock.Guardrail(
 "fraud-detection-guardrail",
 name="fraud-detection-guardrail",
 description="Ensure agent only handles fraud detection tasks",
 blocked_input_messaging="This request is outside the scope of fraud detection.",
 blocked_outputs_messaging="This response was blocked.",
 content_policy_config=bedrock.GuardrailContentPolicyConfigArgs(
 filters_configs=[
 bedrock.GuardrailContentPolicyConfigFiltersConfigArgs(
 type="SEXUAL", input_strength="HIGH", output_strength="HIGH",
 ),
 bedrock.GuardrailContentPolicyConfigFiltersConfigArgs(
 type="VIOLENCE", input_strength="HIGH", output_strength="HIGH",
 ),
 bedrock.GuardrailContentPolicyConfigFiltersConfigArgs(
 type="PROMPT_ATTACK", input_strength="HIGH", output_strength="NONE",
 ),
 # ... additional filters
 ],
 ),
 topic_policy_config=bedrock.GuardrailTopicPolicyConfigArgs(
 topics_configs=[
 bedrock.GuardrailTopicPolicyConfigTopicsConfigArgs(
 name="entertainment-media",
 type="DENY",
 definition="Questions about movies, TV shows, celebrities...",
 examples=["What is LOTR?", "Tell me about Star Wars"],
 ),
 bedrock.GuardrailTopicPolicyConfigTopicsConfigArgs(
 name="general-assistance",
 type="DENY",
 definition="Requests for coding help, math problems...",
 examples=["Write me a poem", "Help me with Python code"],
 ),
 ],
 ),
)

Deploying with Pulumi

The infrastructure is straightforward: ECR repository, IAM roles, and AgentCore Runtime:

# Create ECR repository
repo = ecr.Repository(
 "strands-agent-repo",
 name="fraud-detection-agent",
 force_delete=True,
)

# Build and push Docker image
image = docker.Image(
 "strands-agent-image",
 image_name=repo.repository_url.apply(lambda url: f"{url}:latest"),
 build=docker.DockerBuildArgs(
 context="../fraud-detection-agent",
 platform="linux/arm64",
 ),
 registry=docker.RegistryArgs(
 server=repo.repository_url,
 username=auth_token.user_name,
 password=auth_token.password,
 ),
)

# Deploy AgentCore Runtime
agent_runtime = bedrock.AgentcoreAgentRuntime(
 "fraud-detection-agent-runtime",
 agent_runtime_name="fraud_detection_agent",
 description="Fraud Detection Agent with Guardrails",
 role_arn=agent_role.arn,
 agent_runtime_artifact=bedrock.AgentcoreAgentRuntimeAgentRuntimeArtifactArgs(
 container_configuration=bedrock.AgentcoreAgentRuntimeAgentRuntimeArtifactContainerConfigurationArgs(
 container_uri=image.image_name,
 ),
 ),
 network_configuration=bedrock.AgentcoreAgentRuntimeNetworkConfigurationArgs(
 network_mode="PUBLIC",
 ),
 environment_variables={
 "BEDROCK_GUARDRAIL_ID": fraud_detection_guardrail.guardrail_id,
 "BEDROCK_GUARDRAIL_VERSION": fraud_detection_guardrail.version,
 },
)

Testing the Basic Deployment

After pulumi up, we can invoke the deployed agent via the AWS SDK:

# invoke_agent.py
import boto3
import json

def invoke_agent(agent_runtime_arn: str, prompt: str) -> dict:
 client = boto3.client("bedrock-agentcore", region_name="us-east-1")

 response = client.invoke_agent_runtime(
 agentRuntimeArn=agent_runtime_arn,
 runtimeSessionId="session_abc123...",
 payload=json.dumps({"input": {"prompt": prompt}}),
 qualifier="DEFAULT",
 )

 return json.loads(response["response"].read())

Running the same fraud scenario against the deployed agent:

$ ./test.sh
Selecting Pulumi stack 'dev'...
Getting agent runtime ARN from Pulumi...
Agent Runtime ARN: arn:aws:bedrock-agentcore:us-east-1:052848974346:runtime/fraud_detection_agent-W7zFZT3Vcn
Invoking agent with fraud detection alert...
========================================
Sending prompt to agent...
Prompt: ALERT: New Transaction Attempt
User ID: user_123
Amount: $2000
Merchant: Electronics Store
Location: Tokyo, Japan
Time: 09:15
--------------------------------------------------
Agent Response:
{
"output": {
"message": {
"role": "assistant",
"content": [
{
"text": "**Analysis:** Impossible travel confirmed. User was in London at 09:00
and this transaction attempted in Tokyo at 09:15. Physically impossible.
Combined with high-risk merchant and $2000 amount, this is fraudulent.
Card blocked (TICKET-999)."
}
]
},
"timestamp": "2025-12-15T04:17:48.762738+00:00",
"model": "fraud-detection-agent"
}
}

The agent now runs in a managed, isolated environment with guardrails. Next, we add memory and external tool integration.

Production-Ready: Memory and MCP Gateway

Our basic deployment handles individual transactions, but fraud detection in the real world requires context. If a customer’s card was blocked five minutes ago, the agent shouldn’t try to block it again. If there’s a pattern of suspicious activity over the past week, the agent should factor that into its analysis.

This is where AgentCore Memory and the MCP Gateway come in.

AgentCore Memory: Short-term and Long-term

Memory in AgentCore works at two levels. Short-term memory stores raw conversation events within a session. If the agent blocks a card in one message, it remembers this in subsequent messages during the same session.

Long-term memory is more interesting. It uses semantic extraction to persist facts across sessions. The system runs your conversations through a smaller model (we use Claude Haiku) to extract key facts, then stores those facts in a namespace you define. When a new session starts, the agent can query for relevant facts about that user. Even if the customer calls back tomorrow, the agent knows their card was blocked yesterday.

Here’s how we configure the memory infrastructure:

# Create AgentCore Memory with 30-day event expiry
fraud_detection_memory = bedrock.AgentcoreMemory(
 "fraud-detection-memory",
 name="fraud_detection_memory",
 description="Memory for fraud detection agent",
 event_expiry_duration=30, # Events expire after 30 days
 memory_execution_role_arn=memory_execution_role.arn,
)

# Custom semantic extraction for fraud-related facts
FRAUD_EXTRACTION_PROMPT = """
Extract key fraud-related facts from this conversation about the user. Focus on:
1. Credit card status (blocked/active, ticket IDs if blocked)
2. Fraud alerts and patterns detected for this user
3. Transaction history patterns (locations, amounts, frequencies)
4. User profile details relevant to fraud detection
5. Previous fraud investigations and their outcomes

Format as concise facts that can help identify fraud patterns across sessions.
"""

# Create long-term memory strategy
fraud_semantic_strategy = bedrock.AgentcoreMemoryStrategy(
 "fraud-semantic-strategy",
 name="fraud_semantic_extraction",
 memory_id=fraud_detection_memory.id,
 type="CUSTOM",
 namespaces=["/fraud-detection/users/{actorId}"],
 configuration={
 "type": "SEMANTIC_OVERRIDE",
 "extraction": {
 "append_to_prompt": FRAUD_EXTRACTION_PROMPT,
 "model_id": "us.anthropic.claude-3-5-haiku-20241022-v1:0",
 },
 "consolidation": {
 "model_id": "us.anthropic.claude-3-5-haiku-20241022-v1:0",
 },
 },
)

The namespace pattern /fraud-detection/users/{actorId} ensures each user’s facts are isolated. Claude Haiku handles the extraction efficiently, keeping costs low while maintaining accuracy.

Long-term memory extraction is asynchronous. Plan for a 20-40 second delay between storing conversations and having facts available for retrieval. Use short-term memory for immediate needs while long-term memories are being processed.

In the agent code, we integrate memory using Strands hooks:

from bedrock_agentcore.memory import MemoryClient

memory_client = MemoryClient(region_name="us-east-1")

# Retrieve long-term facts before processing
memories = memory_client.retrieve_memories(
 memory_id=memory_id,
 namespace=f"/fraud-detection/users/{actor_id}",
 query=f"fraud detection facts for user {actor_id}"
)

# Store conversation after processing
memory_client.create_event(
 memory_id=memory_id,
 actor_id=actor_id,
 session_id=session_id,
 messages=[
 (user_message, "USER"),
 (assistant_message, "ASSISTANT")
 ]
)

MCP Gateway: Tool Integration at Scale

The tools we’ve used so far are baked into the agent container. That works for simple cases, but what happens when you have dozens of tools across multiple teams? Or when a tool needs to call an external service that requires its own authentication?

The Model Context Protocol (MCP) is an open standard for AI systems to communicate with tools. AgentCore Gateway provides a managed MCP endpoint that handles authentication (JWT inbound, OAuth2 outbound) and lets agents discover tools using natural language queries rather than knowing exact tool names.

We deploy an MCP server for risk scoring with three tools:

from mcp.server.fastmcp import FastMCP

mcp = FastMCP("Fraud Risk Scoring Server")

@mcp.tool()
def calculate_risk_score(user_id: str, amount: float, merchant: str, location: str) -> dict:
 """Calculate comprehensive fraud risk score (0-100)."""
 # Risk calculation logic
 return {
 "risk_score": 75,
 "factors": ["amount_anomaly", "high_risk_merchant"],
 "recommendation": "BLOCK"
 }

@mcp.tool()
def get_fraud_indicators(user_id: str) -> dict:
 """Get known fraud indicators for a user."""
 return {
 "indicators": ["velocity_alert", "device_inconsistency"],
 "severity": "HIGH"
 }

@mcp.tool()
def check_merchant_reputation(merchant_name: str) -> dict:
 """Check merchant risk rating and fraud history."""
 return {
 "risk_rating": "HIGH",
 "fraud_reports": 47,
 "chargeback_rate": 0.15
 }

The Gateway infrastructure requires careful attention to authentication. We use two separate Cognito user pools to handle the dual authentication flow:

Inbound authentication (Agent → Gateway): The agent authenticates to the Gateway using JWT tokens from the first Cognito pool. This ensures only authorized agents can discover and call tools.

# Gateway with MCP protocol and semantic search
agentcore_gateway = bedrock.AgentcoreGateway(
 "fraud-detection-gateway",
 name="fraud-detection-mcp-gateway",
 description="Gateway for fraud detection MCP tools",
 role_arn=gateway_role.arn,
 authorizer_type="CUSTOM_JWT",
 authorizer_configuration={
 "customJwtAuthorizer": {
 "discoveryUrl": gateway_discovery_url,
 "allowedClients": [gateway_app_client.id],
 },
 },
 protocol_type="MCP",
 protocol_configuration={
 "mcp": {
 "supportedVersions": ["2025-03-26"],
 "searchType": "SEMANTIC",
 "instructions": "Gateway for fraud detection risk scoring tools.",
 },
 },
)

The searchType: "SEMANTIC" configuration enables natural language tool discovery. Instead of the agent needing to know exact tool names, it can query for “tools that check merchant risk” and the Gateway returns relevant matches.

Outbound authentication (Gateway → MCP Server): When the Gateway calls the MCP server, it needs its own credentials. The OAuth2 Credential Provider manages this second authentication layer:

# OAuth2 Credential Provider for Gateway -> MCP Server auth
oauth2_credential_provider = bedrock.AgentcoreOauth2CredentialProvider(
 "mcp-runtime-oauth2-provider",
 name="fraud-mcp-runtime-oauth2",
 credential_provider_vendor="CustomOauth2",
 oauth2_provider_config={
 "customOauth2ProviderConfig": {
 "oauthDiscovery": {"discoveryUrl": mcp_runtime_discovery_url},
 "clientId": mcp_runtime_app_client.id,
 "clientSecret": mcp_runtime_app_client.client_secret,
 },
 },
)

Connecting the pieces: Finally, we create a Gateway Target. This links the Gateway to the MCP server endpoint. It also specifies which credential provider to use for outbound calls:

# Connect Gateway to MCP Server
gateway_target = bedrock.AgentcoreGatewayTarget(
 "mcp-risk-server-target",
 name="mcp-risk-server-target",
 gateway_identifier=agentcore_gateway.gateway_id,
 target_configuration={
 "mcp": {"mcpServer": {"endpoint": mcp_server_endpoint_url}},
 },
 credential_provider_configuration={
 "oauth": {
 "providerArn": oauth2_credential_provider.credential_provider_arn,
 "scopes": [mcp_runtime_scope_string],
 },
 },
)

This three-part configuration (Gateway, Credential Provider, Target) separates concerns cleanly: the Gateway handles inbound auth and protocol translation, the Credential Provider manages outbound secrets, and the Target defines routing.

Observability with X-Ray and OpenTelemetry

Production systems need observability. We configure CloudWatch Logs for application logs and X-Ray for distributed tracing:

# Agent application logs
agent_log_group = cloudwatch.LogGroup(
 "agent-runtime-log-group",
 name=pulumi.Output.concat(
 "/aws/vendedlogs/bedrock-agentcore/", agent_runtime.agent_runtime_id
 ),
 retention_in_days=30,
)

agent_logs_delivery = cloudwatch.LogDelivery(
 "agent-logs-delivery",
 delivery_source_name=agent_logs_delivery_source.name,
 delivery_destination_arn=agent_logs_delivery_destination.arn,
)

# X-Ray traces
agent_traces_delivery_destination = cloudwatch.LogDeliveryDestination(
 "agent-traces-delivery-destination",
 name="fraud-agent-traces-dest",
 delivery_destination_type="XRAY",
)

agent_traces_delivery = cloudwatch.LogDelivery(
 "agent-traces-delivery",
 delivery_source_name=agent_traces_delivery_source.name,
 delivery_destination_arn=agent_traces_delivery_destination.arn,
)

In the container, we use AWS Distro for OpenTelemetry for auto-instrumentation:

CMD ["uv", "run", "opentelemetry-instrument", "uvicorn", "agent:app", "--host", "0.0.0.0", "--port", "8080"]

This gives us end-to-end tracing across agent invocations, memory operations, and Gateway calls with zero code changes.

AWS has announced that X-Ray SDKs will be deprecated in favor of OpenTelemetry by February 2027. Using ADOT from the start ensures your observability stack is future-proof.

Running the Demo

Deploy the Infrastructure

cd advanced-bedrock-deployment/infra
uv sync
uv run pulumi up

This deploys approximately 40 AWS resources including:

ECR repositories for agent and MCP server
AgentCore Memory with semantic extraction strategy
Two Cognito user pools for authentication
AgentCore Gateway with MCP protocol
CloudWatch log groups and X-Ray trace delivery
IAM roles and policies

Test Short-term Memory

./test.sh demo

This runs 5 scenarios in the same session, demonstrating short-term memory:

======================================================================
FRAUD DETECTION DEMO - SHORT-TERM MEMORY (Same Session)
======================================================================
SCENARIO 1: John Doe - Impossible Travel Detection
Expected: FRAUD - impossible travel from London to Tokyo in 15 minutes
Actor ID: user_123
Session ID: session_cf171803eb324a2aa9ac456b3013fe45...
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": {
"text": "**FRAUD CONFIRMED - CARD BLOCKED**
| Finding | Details |
|---------|---------|
| **User** | John Doe |
| **Ticket** | TICKET-123-999 |
| **Fraud Type** | Impossible Travel + Spending Anomaly |
**Analysis:** Transaction in Tokyo 15 minutes after London purchase
is physically impossible. The $2,000 amount is 10x the user's typical
spending range ($50-200). Card blocked immediately."
},
"memory_enabled": true,
"long_term_facts_retrieved": 3
}
}
======================================================================
SCENARIO 3: John Doe - Follow-up Transaction (Short-term Memory Test)
Expected: FRAUD - card should already be blocked (SHORT-TERM MEMORY)
Actor ID: user_123
Session ID: session_cf171803eb324a2aa9ac456b3013fe45... (SAME SESSION!)
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": {
"text": "**Card for user_123 (John Doe) is ALREADY BLOCKED
(Ticket: TICKET-123-999). No further action needed.**
This appears to be another fraudulent attempt on an already-compromised
account. The card was previously blocked due to impossible travel fraud.
The current transaction follows the same pattern: Gift Card Kiosk
purchases are a common fraud tactic for converting stolen card access
to untraceable value."
},
"memory_enabled": true,
"long_term_facts_retrieved": 3
}
}
======================================================================
SHORT-TERM MEMORY DEMO SUMMARY
======================================================================
[OK] John Doe - Impossible Travel Detection
[OK] Jane Smith - Normal Transaction
[OK] John Doe - Follow-up Transaction (Short-term Memory Test)
[OK] Alice Chen - Impossible Travel Detection
[OK] Jane Smith - Second Normal Transaction

The agent remembers within the same session that John’s card is already blocked, so there’s no need to block it twice.

Test Long-term Memory

./test.sh longterm-fresh

This demonstrates cross-session memory with two phases:

======================================================================
PHASE 1: INITIAL FRAUD DETECTION
These interactions create long-term memory facts about users.
======================================================================
SCENARIO: John Doe - Initial Fraud (Card Blocked)
Session ID: session_xyz789... (NEW SESSION)
→ Card blocked, fact stored to long-term memory
======================================================================
WAITING FOR LONG-TERM MEMORY EXTRACTION...
The semantic strategy needs time to process conversations into facts.
Waiting 30 seconds...
======================================================================
======================================================================
PHASE 2: NEW SESSIONS - LONG-TERM MEMORY RETRIEVAL
These are COMPLETELY NEW sessions. The agent should remember
fraud-related facts about users from Phase 1.
======================================================================
SCENARIO: John Doe - NEW SESSION (Long-term Memory Test)
Session ID: session_new456... (DIFFERENT SESSION!)
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": "CARD ALREADY BLOCKED. According to my records,
this user's card was previously blocked due to fraud.
Ticket FRAUD-user_123-001 exists from a previous investigation.
The card remains blocked.",
"memory_enabled": true,
"long_term_facts_retrieved": 2
}
}

Even in a completely new session, the agent retrieves facts from long-term memory and knows John’s card was blocked yesterday.

Test MCP Gateway

./test.sh gateway-fresh

This runs 6 diverse fraud scenarios using both local tools and MCP Gateway tools:

======================================================================
LOADING MCP GATEWAY CONFIGURATION...
======================================================================
Gateway URL: https://fraud-detection-mcp-gateway-xxx.gateway.bedrock-agentcore.us-east-1.amazonaws.com/mcp
Token Endpoint: https://fraud-gateway-xxx.auth.us-east-1.amazoncognito.com/oauth2/token
Scope: fraud-detection-gateway/invoke
MCP Gateway configuration loaded successfully!
======================================================================
FRAUD DETECTION DEMO - MCP GATEWAY INTEGRATION
======================================================================
LOCAL TOOLS:
- get_user_profile: User details and home location
- get_recent_transactions: Last known transaction
- block_credit_card: Block card and create ticket
MCP GATEWAY TOOLS (via Risk Scoring Service):
- calculate_risk_score: Fraud risk score 0-100
- get_fraud_indicators: Known fraud indicators
- check_merchant_reputation: Merchant risk rating
======================================================================
SCENARIO 1: HIGH-RISK MERCHANT - Jane at CryptoExchange123
MCP Gateway: ENABLED
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": {
"text": "**✅ CARD BLOCKED** - Ticket: TICKET-456-999
**Summary:** Jane Smith's card blocked due to a $5,000 crypto exchange
transaction that is 10x her typical spend and outside her normal merchant
categories. Location matched home, but amount and merchant anomalies
indicate likely fraud or compromised credentials."
},
"mcp_gateway_enabled": true,
"mcp_tools_count": 4
}
}
======================================================================
SCENARIO 5: COMBINED SIGNALS - Alice's Multi-Factor Fraud
MCP Gateway: ENABLED
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": {
"text": "## 🚨 FRAUD CONFIRMED - CARD BLOCKED
**User:** Alice Chen (user_321)
**Ticket ID:** TICKET-321-999
**Fraud Indicators Detected:**
| Factor | Finding |
|--------|---------|
| ⛔ Impossible Travel | Singapore (10:00) → Los Angeles (11:30) = 90 min |
| ⛔ Amount Anomaly | $2,500 vs typical $50-300 (8x+ normal) |
| ⛔ Merchant Mismatch | Cash advance vs usual food/retail |
| ⛔ High-Risk Merchant | QuickCashAdvance (cash advance services) |
**Action Taken:** Card blocked immediately. This transaction exhibits
classic signs of stolen card credentials being used for rapid cash extraction."
},
"mcp_gateway_enabled": true,
"mcp_tools_count": 4
}
}
======================================================================
MCP GATEWAY DEMO SUMMARY
======================================================================
[OK] 1. HIGH-RISK MERCHANT - Jane at CryptoExchange123
Gateway: YES (4 tools)
[OK] 2. VELOCITY ATTACK - Bob's Rapid Transactions
Gateway: YES (4 tools)
[OK] 3. AMOUNT ANOMALY - Alice's Large Purchase
Gateway: YES (4 tools)
[OK] 4. KNOWN FRAUD PATTERNS - John's Indicator Check
Gateway: YES (4 tools)
[OK] 5. COMBINED SIGNALS - Alice's Multi-Factor Fraud
Gateway: YES (4 tools)
[OK] 6. CLEAN TRANSACTION - Jane's Normal Purchase
Gateway: YES (4 tools)

The MCP Gateway tools provide additional context (merchant reputation, risk scores, fraud indicators) that helps the agent make better decisions.

Architecture Summary

Here’s the complete architecture:

Key Takeaways

Event-driven first: Production agents are triggered by Kafka, SQS, or webhooks, not chat interfaces
Start simple: Build with Strands SDK locally, add AgentCore complexity only when needed
Guardrails early: Content filtering and topic restrictions prevent prompt injection
Memory has two levels: Short-term (session) and long-term (semantic extraction across sessions)
MCP for tools at scale: Gateway handles auth and discovery; you focus on tool logic
Observability required: X-Ray + CloudWatch are essential for debugging distributed AI
IaC for reproducibility: Pulumi deploys 40+ resources with proper dependencies

Cost Considerations

When evaluating AgentCore for production, consider these cost factors:

AgentCore Runtime charges per session-second. For fraud detection agents that typically run 10-30 seconds per transaction, this is cost-effective compared to maintaining always-on ECS containers. However, if your agents process thousands of transactions per second, the per-session costs add up. Consider batching or using Lambda for simple, fast agents.

Model costs dominate most deployments. Claude Opus 4.5 (used for reasoning) costs significantly more than Claude Haiku (used for memory extraction). We deliberately use Haiku for semantic extraction to keep memory costs low while maintaining accuracy. For high-volume production, consider whether Haiku or even smaller models could handle your primary reasoning.

Memory and Gateway have their own pricing tiers. AgentCore Memory charges for storage and retrieval operations. The Gateway adds overhead for OAuth2 token management and routing. For simple agents that don’t need cross-session memory or external tools, skip these components entirely.

Cognito user pools for authentication have a free tier (50,000 MAU) that covers most development and moderate production workloads. Beyond that, costs scale with active users.

Start with a basic deployment and add Memory/Gateway only when you have a clear need. The infrastructure cost savings of skipping unnecessary components often exceed the engineering cost of adding them later.

Conclusion

The fraud detection agent in this post grew from 30 lines of local Python to a production deployment with guardrails, dual-layer memory, MCP Gateway integration, and full observability, all defined as infrastructure as code.

The gap between prototype and production doesn’t have to be a chasm. AgentCore provides the managed runtime, Pulumi makes it reproducible, and the progressive approach lets you add complexity only when your use case demands it.

Get Started with Pulumi

Use Pulumi's open-source SDK to create, deploy, and manage infrastructure on any cloud.

Automatic Diagram Generation for Always-Accurate Diagrams

Elisabeth Lichtie — Mon, 22 Sep 2025 00:00:00 +0000

Architecture diagrams are one of the most tedious aspects of infrastructure management. Teams spend countless hours manually creating and updating visual representations of their systems, only to see them become outdated the moment infrastructure changes.

When you manage your infrastructure as code, you can automatically generate accurate, up-to-date diagrams directly from your IaC definitions and state. Pulumi’s preview capabilities let you anticipate exactly what changes will result from code modifications before deployment, providing the perfect foundation for automated diagramming workflows.

In this post, we’ll explore three practical approaches to automating diagram generation in your CI/CD pipeline:

Native Pulumi diagramming: Using Pulumi’s built-in diagramming capabilities to generate basic architecture diagrams that you can retrieve from GitHub Actions’ artifact archive
AI-powered diagramming: Leveraging the Claude app with Mermaid and Pulumi preview to generate detailed diagrams and automatically add them to pull requests for infrastructure changes
Pulumi Neo for enterprise diagramming: Using Pulumi’s AI-powered platform for organization-wide diagramming patterns, project-level and organization-level views, and intelligent customization for different stakeholders

These approaches eliminate manual diagramming work while ensuring your documentation stays current with your actual infrastructure, from simple automation to enterprise-scale intelligent diagramming.

Native Pulumi diagramming with GitHub Actions

Pulumi’s built-in pulumi stack graph command generates architecture diagrams directly from your deployed infrastructure state. This approach provides a reliable foundation for automated diagramming since it reflects your actual deployed resources rather than just the code.

Complete example: View the full working example on GitHub.

Here’s a complete GitHub Actions workflow that deploys infrastructure and automatically generates diagrams:

name: Pulumi Deploy & Diagram

on:
 push:
 branches: [main]

jobs:
 deploy:
 name: Deploy & Generate Diagram
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4

 - name: Setup Node.js
 uses: actions/setup-node@v4
 with:
 node-version: "20"
 cache: "npm"

 - name: Install dependencies
 run: npm ci

 - uses: pulumi/actions@v5
 with:
 command: up
 stack-name: dev
 env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}

 - name: Install Pulumi CLI
 run: |
 curl -fsSL https://get.pulumi.com | sh
 echo "$HOME/.pulumi/bin" >> $GITHUB_PATH

 - name: Generate Architecture Diagram
 run: pulumi stack graph architecture.dot --stack dev
 env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}

 - name: Install Graphviz
 run: sudo apt-get update && sudo apt-get install -y graphviz

 - name: Generate PNG from DOT file
 run: dot -Tpng architecture.dot -o architecture.png

 - name: Upload Architecture Diagrams
 uses: actions/upload-artifact@v4
 with:
 name: architecture-diagrams
 path: |
 architecture.dot
 architecture.png

How the workflow works

This workflow follows a straightforward pattern that ensures your diagrams always reflect your deployed infrastructure:

Deploy first: The workflow uses the official pulumi/actions@v5 action to deploy your infrastructure changes. This ensures the state reflects your latest code.
Generate diagram from state: The pulumi stack graph command reads your deployed stack’s state and generates a DOT format file containing the resource relationships and dependencies.
Convert to visual format: Graphviz converts the DOT file into a PNG image that’s easy to view and share.
Archive as artifacts: Both the raw DOT file and the rendered PNG are uploaded as GitHub Actions artifacts, making them downloadable from the workflow run.

Key advantages and limitations

Highest accuracy: Since diagrams are generated from deployed state rather than code, they reflect your actual infrastructure including any drift or manual changes. This is the biggest advantage of the native approach.

Zero maintenance: Once set up, diagrams update automatically with every deployment without any manual intervention.

Multiple formats: The DOT format can be converted to various output formats (PNG, SVG, PDF) depending on your needs.

Audit trail: Each workflow run preserves the diagram as an artifact, creating a visual history of your infrastructure changes.

However, this approach has some limitations:

Fixed styling: You’re locked into Pulumi’s diagram style and layout algorithms, with limited customization options.

Basic aesthetics: The generated diagrams are functional but not always aesthetically polished compared to hand-crafted or AI-generated alternatives.

Technical focus: Diagrams show all resources and relationships, which can be overwhelming for non-technical stakeholders who need higher-level architectural views.

Accessing your diagrams

After each workflow run, you can download the generated diagrams from the Actions tab in your repository. Click on any workflow run, scroll to the “Artifacts” section, and download the architecture-diagrams package.

For a more sophisticated setup, you could extend this workflow to automatically commit diagrams back to your repository, post them to Slack, or integrate them with your documentation system.

AI-powered diagramming with Claude and Mermaid

The second approach leverages Claude’s ability to analyze Pulumi preview output and generate aesthetically pleasing, contextually relevant diagrams using Mermaid syntax. This method addresses the styling limitations of native Pulumi diagrams while maintaining accuracy through pulumi preview integration.

Complete example: View the full working example on GitHub.

The automated workflow

This approach uses two GitHub Actions workflows working together:

Auto-comment workflow (auto-pr-comment.yml):

name: Auto PR Comment

on:
 pull_request:
 types: [opened]
 paths:
 - 'index.ts'
 - 'Pulumi.yaml'
 - 'Pulumi.*.yaml'

jobs:
 comment:
 if: github.actor != 'claude-code[bot]'
 runs-on: ubuntu-latest
 permissions:
 contents: read
 pull-requests: write
 steps:
 - name: Comment on PR
 uses: actions/github-script@v7
 with:
 github-token: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
 script: |
 github.rest.issues.createComment({
 issue_number: context.issue.number,
 owner: context.repo.owner,
 repo: context.repo.repo,
 body: '@claude update the diagrams to reflect any infrastructure changes.'
 })

Claude integration workflow (claude.yml):

name: Claude Code

on:
 issue_comment:
 types: [created]
 pull_request_review_comment:
 types: [created]

jobs:
 claude:
 if: contains(github.event.comment.body, '@claude')
 runs-on: ubuntu-latest
 permissions:
 contents: write
 pull-requests: write
 issues: write
 steps:
 - name: Checkout repository
 uses: actions/checkout@v4

 - name: Run Claude Code
 uses: anthropics/claude-code-action@beta
 env:
 PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
 with:
 anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
 allowed_tools: "Bash(npm install),Bash(pulumi stack select prod),Bash(pulumi preview --json),Bash(npx mmdc)"
 custom_instructions: |
 ## Pulumi Infrastructure Diagramming Assistant

 ### Step 1: Analyze Infrastructure State
 - Run `npm install` and `pulumi stack select prod`
 - Execute `pulumi preview --json` to get detailed resource plan
 - Parse JSON output to understand resource relationships and changes

 ### Step 2: Generate/Update Diagrams
 Update Mermaid diagram files in `/diagrams/`:
 - aws-infrastructure.mmd: Technical AWS resource diagram
 - c4-context.mmd: System context diagram
 - c4-container.mmd: Container-level view
 - c4-component.mmd: Component-level architecture

 ### Step 3: Generate SVG Files
 Convert .mmd files to .svg using mmdc CLI

How the AI approach works

Automatic triggering: When a pull request with infrastructure changes is opened, the auto-comment workflow adds a @claude mention, triggering Claude’s analysis.
Infrastructure analysis: Claude runs pulumi preview --json to get detailed information about planned infrastructure changes, including resource relationships and dependencies.
Intelligent diagram generation: Claude analyzes the preview output and updates multiple Mermaid diagrams:
- AWS infrastructure diagram: Technical view showing resources, networking, and data flow
- C4 diagrams: Context, container, and component views for different stakeholder needs
Visual rendering: The mmdc CLI converts Mermaid syntax to SVG files for immediate viewing.
PR integration: Updated diagrams are committed directly to the pull request branch, making them visible in code review.

Example generated diagram

Here’s a sample of the AWS infrastructure diagram that Claude generates:

graph TB
subgraph VPC["myVpc - CIDR: 10.0.0.0/16"]
subgraph PublicSubnet["myPublicSubnet - CIDR: 10.0.1.0/24"]
EC2["myInstance<br/>Type: t2.micro<br/>AMI: Latest Amazon Linux 2"]
end
IGW["myInternetGateway<br/>Internet connectivity"]
SG["mySecurityGroup<br/>HTTP (80) + SSH (22) access"]
end
WEB_USER["Web Users"] -->|HTTP requests| IGW
IGW --> PublicSubnet
SG --> EC2

Key advantages and limitations

Intelligent styling: Claude generates aesthetically pleasing diagrams with consistent formatting, colors, and layout that’s much more polished than native Pulumi output.

Multiple perspectives: Creates different diagram types (technical infrastructure, C4 architecture views) suitable for different audiences and stakeholder needs.

Context awareness: Claude understands the purpose and relationships of resources, creating meaningful groupings and flow representations.

Automated maintenance: Diagrams update automatically with infrastructure changes, staying current without manual intervention.

Integration with code review: Diagrams appear directly in pull requests, making infrastructure changes visible during review.

However, this approach also has limitations:

Potential for inaccuracy: Unlike native Pulumi diagrams that are generated directly from infrastructure state, AI-generated diagrams depend on Claude’s interpretation of preview output. While generally reliable, there’s inherent risk of misinterpretation or missed details that could lead to diagram inaccuracies.

AI dependency: Diagram accuracy is only as good as Claude’s understanding of complex infrastructure relationships, which may occasionally miss nuances or make incorrect assumptions.

Setup complexity: Requires more configuration including API keys, GitHub app installation, and workflow permissions.

Cost considerations: Each diagram update consumes Claude API credits, though costs are typically minimal for most projects.

Preview limitations: Diagrams reflect planned changes from pulumi preview rather than actual deployed state, which may differ if deployments fail.

This approach works best for teams that value diagram aesthetics and need multiple architectural views for different stakeholders, while accepting slightly more complexity and potential accuracy trade-offs in exchange for significantly better visual presentation.

Pulumi Neo: Enterprise-scale intelligent diagramming

Pulumi Neo brings the next evolution in infrastructure diagramming, with AI-powered intelligence that enables organization-wide diagram generation. Unlike the previous approaches that focus on individual stacks, Pulumi Neo provides enterprise-scale diagramming capabilities with intelligent customization for different stakeholders.

How the workflow works

Pulumi Neo streamlines enterprise diagramming through an intelligent, organization-aware approach that scales across multiple projects and stakeholders:

Repository setup: Create a dedicated repository to host your architecture diagrams and ensure your GitHub App has access. The repository structure helps Neo understand your diagramming patterns:
- overview/ folder for organization-wide infrastructure views
- stacks/ folder for individual stack diagrams
- Template files that Neo can model new diagrams from
- README documentation explaining the repository’s purpose and conventions

Neo prompt engineering: Use structured prompts to guide Neo’s diagram generation with enterprise-grade specificity. You can also select your architecture diagram repository to tell Neo where to store his work:

1. Scope: Create diagrams for all resources in my prod Pulumi stacks
1. Diagram Types:
- Individual stack diagrams showing resources within each stack
- Master overview diagram showing cross-stack dependencies and data flow
1. Visual Standards:
- Use official AWS/Azure/GCP service icons from their architecture icon sets
- Show resource relationships with labeled connections (security groups, subnets, etc.)
- Group related resources visually (VPC boundaries, availability zones)
1. Content Focus:
- Infrastructure resources only (compute, storage, networking, databases)
- Resource dependencies and data flow directions
- Cross-stack references and shared resources
1. Format: Generate diagrams as Mermaid code that can be rendered in documentation

Automated pull request generation: Neo analyzes your entire organization’s infrastructure state, identifies changes, and automatically creates pull requests with updated diagrams for review and approval.

Complete example: You can see my resulting architecture diagrams repository on GitHub.

Key advantages and limitations

Pulumi Neo offers the most sophisticated approach to infrastructure diagramming, particularly for enterprise environments:

Enterprise-scale intelligence: Neo understands your entire organization’s infrastructure landscape, generating diagrams that show cross-project dependencies and organization-wide resource relationships that individual stack-based approaches cannot capture.

Style customization: Customize your prompts to request different styles of diagrams.

Organizational consistency: Maintains consistent diagramming standards across teams and projects, ensuring visual coherence in large organizations with multiple infrastructure teams.

Cross-stack visibility: Unlike single-stack approaches, Neo provides organization-wide infrastructure views showing how different projects and teams’ resources interconnect.

Template-driven generation: Learns from your existing diagram patterns and templates to maintain consistency with your established visual standards and conventions.

AI-powered context understanding: Goes beyond simple resource mapping to understand business context, automatically highlighting critical paths, security boundaries, and architectural patterns.

However, Neo also has limitations:

AI interpretation risks: Like the Claude approach, AI-generated diagrams may miss nuances or make incorrect assumptions about complex enterprise infrastructure relationships.

Repository access requirements: Neo requires permissions to open pull requests against your repository.

Learning investment: Getting maximum value from Neo requires developing sophisticated prompt engineering skills and understanding how to structure repositories for optimal AI comprehension.

Conclusion

Automating diagram generation transforms one of the most tedious aspects of infrastructure management into a seamless part of your development workflow. All three approaches eliminate the manual overhead of keeping architecture diagrams current while solving the core problem of documentation drift.

Choose native Pulumi diagramming when accuracy is paramount and you need diagrams that perfectly reflect your deployed infrastructure. This approach offers the highest reliability with minimal setup complexity.

Choose AI-powered diagramming when you need polished, stakeholder-friendly diagrams and can accept slightly more complexity for significantly better aesthetics and multiple architectural perspectives.

Choose Pulumi Neo when you need enterprise-scale diagramming with organization-wide consistency, multi-level views across projects and stacks, and intelligent customization for different stakeholders with varying technical backgrounds.

Regardless of which approach you choose, you’ll eliminate hours of manual work while ensuring your documentation stays synchronized with your infrastructure changes. With Pulumi Neo’s AI capabilities now available, intelligent, context-aware diagramming has moved from possibility to reality, making infrastructure documentation both effortless and incredibly powerful.

The future of infrastructure documentation is here: automated, intelligent, and always accurate. With these examples as your foundation, you’re ready to transform how your team approaches architecture visualization at any scale.

I Tried Jenkins in 2025 with Pulumi: Here's How It Went

Engin Diri — Tue, 29 Jul 2025 10:35:00 +0200

It’s funny how technology has a way of sneaking back into your life just when you think you’ve moved on for good. Jenkins and I have quite the history. Think of it as that reliable but slightly temperamental friend from your college days who you haven’t seen in years.

A Blast from the Jenkins Past

The last time Jenkins and I were on speaking terms was during my tenure at my former workplace, back when the CI/CD landscape looked very different than it does today. We weren’t just casual acquaintances either. We were deep in the trenches together, orchestrating builds, managing deployments, and occasionally cursing at each other when things went sideways (which, let’s be honest, happened more often than I’d like to admit).

Those were the days when Jenkins was transitioning from its scrappy, plugin-heavy adolescence into something resembling maturity. I was part of a significant modernization effort that would make any DevOps engineer’s heart race: migrating our entire Jenkins infrastructure from Jenkins 1.x to the shiny new Jenkins 2.0. If you’ve never experienced a Jenkins major version upgrade in a production environment, imagine performing heart surgery while the patient is running a marathon. Technically possible, but requiring nerves of steel and copious amounts of coffee.

The migration wasn’t just about version numbers. Jenkins 2.0 brought with it the Pipeline plugin as a first-class citizen, transforming how we thought about continuous integration. Gone were the days of clicking through endless GUI configurations and praying that someone had documented the build process correctly. Instead, we embraced the revolutionary concept of “Pipeline as Code”, storing our build definitions in Jenkinsfile scripts that lived alongside our application code.

But the real game-changer came when we decided to containerize our Jenkins worker nodes using Docker. This was back when Docker was still the new kid on the block, and the idea of running build agents in containers felt both cutting-edge and slightly reckless. We implemented the Docker plugin for Jenkins, which allowed us to spin up fresh, isolated environments for each build. No more “but it works on my machine” syndrome. Every build got its own pristine container, complete with exactly the dependencies it needed.

The transformation was remarkable. What used to take us hours of manual environment setup and dependency management became as simple as defining a Dockerfile. Our build agents went from being precious, hand-crafted snowflakes that required constant care and feeding to becoming ephemeral, reproducible environments that could be destroyed and recreated at will.

Here’s what a typical Jenkinsfile looked like during that era. Notice how we could seamlessly combine different technologies and services, each running in their own containers:

pipeline {
 agent none
 stages {
 stage('Back-end') {
 agent {
 docker { image 'maven:3.9.10-eclipse-temurin-21-alpine' }
 }
 steps {
 sh 'mvn clean compile test'
 junit 'target/surefire-reports/*.xml'
 }
 }
 stage('Front-end') {
 agent {
 docker { image 'node:22.17.0-alpine3.22' }
 }
 steps {
 sh 'npm install && npm test'
 }
 }
 stage('Integration Tests') {
 agent any
 steps {
 script {
 docker.image('postgres:13').withRun('-e POSTGRES_PASSWORD=test') { db ->
 docker.image('redis:7-alpine').withRun() { redis ->
 docker.build('myapp:test').inside("--link ${db.id}:postgres --link ${redis.id}:redis") {
 sh 'npm run integration-tests'
 }
 }
 }
 }
 }
 }
 }
}

This pipeline definition encapsulated everything we’d learned about containerized CI/CD: isolated environments, service orchestration, and the beautiful simplicity of treating infrastructure as disposable. Each stage could use exactly the tools it needed, without worrying about conflicts or environmental drift.

Fast forward to 2025, and here I am, staring at a Jenkins instance again. But this time, there’s a twist: instead of the traditional XML configurations and GUI-heavy setup processes I remember, I’m armed with Pulumi. This tool promises to bring infrastructure-as-code practices to Jenkins deployment itself. The irony isn’t lost on me: using modern infrastructure tooling to deploy a CI/CD platform that helped define what modern infrastructure tooling could become.

So buckle up for this nostalgic yet forward-looking journey. We’re about to find out whether Jenkins has kept up with the times, and more importantly, whether the combination of Jenkins and Pulumi can create something greater than the sum of its parts.

How I’d Set Up Jenkins in 2025: The Kubernetes Way

Fast forward to 2025, and the infrastructure landscape has evolved dramatically. While my old Jenkins setup was running on virtual machines with Docker containers as build agents, the modern approach demands something more cloud-native. Today, if you’re serious about CI/CD at scale, Kubernetes isn’t just an option. It’s practically a requirement.

The beauty of running Jenkins on Kubernetes lies in its perfect marriage of concepts. Jenkins already understood the value of ephemeral build agents, and Kubernetes takes that philosophy to its logical conclusion: everything is ephemeral, everything is disposable, and everything can be recreated on demand. Your Jenkins controller becomes just another workload in your cluster, and your build agents become pods that spin up, do their work, and disappear without a trace.

But here’s where it gets interesting. Instead of clicking through installation wizards or crafting complex shell scripts, I’m going to use Pulumi to define my entire Jenkins infrastructure as code. This isn’t just about following the latest trends. It’s about bringing the same “everything as code” philosophy that made Jenkins 2.0 so compelling to the infrastructure layer itself.

The Pulumi Advantage

Why Pulumi for this task? After years of wrestling with YAML manifests and Helm charts, I’ve come to appreciate Pulumi’s approach to infrastructure as code. With Pulumi, I can leverage real programming languages (in this case, Python) to define my infrastructure, complete with loops, conditionals, and all the abstractions that make complex deployments manageable.

Let me show you what a modern Jenkins deployment looks like when we combine Pulumi’s infrastructure-as-code capabilities with Kubernetes’ orchestration power. This isn’t just theoretical. This is production-ready code that you could deploy in any Kubernetes cluster.

Setting Up Our Local Kubernetes Cluster with Kind

Before we dive into the Pulumi code, we need a Kubernetes cluster to deploy to. For this demonstration, I’m using Kind (Kubernetes in Docker), which creates a local Kubernetes cluster using Docker containers as nodes. It’s perfect for development, testing, and learning scenarios like this.

Kind has become the de facto standard for local Kubernetes development because it’s lightweight, fast, and provides a genuine multi-node Kubernetes experience without the overhead of virtual machines. Plus, it’s what the Kubernetes project itself uses for testing, so you know it’s reliable.

Creating the Cluster:

Once Kind is installed, creating a cluster is straightforward:

kind create cluster --name jenkins-cluster

This command spins up a single-node Kubernetes cluster running in a Docker container. The beauty of Kind is that it configures your kubectl context automatically, so you’re immediately ready to start deploying workloads.

You can verify your cluster is running:

kubectl cluster-info --context kind-jenkins-cluster
kubectl get nodes

For our Jenkins deployment, the default single-node cluster is sufficient, but Kind also supports multi-node clusters if you need to test more complex scenarios. The cluster will persist until you explicitly delete it with

kind delete cluster --name jenkins-cluster

Now that we have our Kubernetes cluster ready, let’s deploy Jenkins using Pulumi.

import pulumi
import pulumi_kubernetes as k8s
from pulumi_kubernetes.helm.v3 import Chart, ChartOpts, FetchOpts

# Configuration - making it parameterizable
config = pulumi.Config()
namespace_name = config.get("namespace", "jenkins")
storage_class = config.get("storageClass", "standard") # Kind uses 'standard' by default
jenkins_admin_password = config.require_secret("adminPassword")

# Create a dedicated namespace for Jenkins
jenkins_namespace = k8s.core.v1.Namespace(
 "jenkins-namespace",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name=namespace_name,
 labels={"app": "jenkins", "managed-by": "pulumi"}
 )
)

# Create persistent storage for Jenkins
jenkins_pvc = k8s.core.v1.PersistentVolumeClaim(
 "jenkins-pvc",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name="jenkins-pv-claim",
 namespace=jenkins_namespace.metadata.name,
 ),
 spec=k8s.core.v1.PersistentVolumeClaimSpecArgs(
 access_modes=["ReadWriteOnce"],
 storage_class_name=storage_class,
 resources=k8s.core.v1.ResourceRequirementsArgs(
 requests={"storage": "20Gi"}
 )
 )
)

# Create a service account for Jenkins with proper RBAC
jenkins_service_account = k8s.core.v1.ServiceAccount(
 "jenkins-service-account",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name="jenkins",
 namespace=jenkins_namespace.metadata.name,
 )
)

# Create cluster role for Jenkins to manage pods (for Kubernetes plugin)
jenkins_cluster_role = k8s.rbac.v1.ClusterRole(
 "jenkins-cluster-role",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name="jenkins-cluster-role",
 ),
 rules=[
 k8s.rbac.v1.PolicyRuleArgs(
 api_groups=[""],
 resources=["pods", "pods/exec", "pods/log", "persistentvolumeclaims"],
 verbs=["get", "list", "watch", "create", "update", "patch", "delete"]
 ),
 k8s.rbac.v1.PolicyRuleArgs(
 api_groups=["apps"],
 resources=["deployments", "replicasets"],
 verbs=["get", "list", "watch", "create", "update", "patch", "delete"]
 )
 ]
)

# Bind the cluster role to the service account
jenkins_cluster_role_binding = k8s.rbac.v1.ClusterRoleBinding(
 "jenkins-cluster-role-binding",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name="jenkins-cluster-role-binding",
 ),
 role_ref=k8s.rbac.v1.RoleRefArgs(
 api_group="rbac.authorization.k8s.io",
 kind="ClusterRole",
 name=jenkins_cluster_role.metadata.name
 ),
 subjects=[
 k8s.rbac.v1.SubjectArgs(
 kind="ServiceAccount",
 name=jenkins_service_account.metadata.name,
 namespace=jenkins_namespace.metadata.name
 )
 ]
)

# Create a secret for Jenkins admin credentials
jenkins_admin_secret = k8s.core.v1.Secret(
 "jenkins-admin-secret",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name="jenkins-admin-secret",
 namespace=jenkins_namespace.metadata.name,
 ),
 string_data={
 "jenkins-admin-user": "admin",
 "jenkins-admin-password": jenkins_admin_password,
 }
)

# Deploy Jenkins using the official Helm chart from https://charts.jenkins.io
jenkins_chart = Chart(
 "jenkins",
 ChartOpts(
 chart="jenkins",
 version="5.8.72", # Using latest stable version
 namespace=jenkins_namespace.metadata.name,
 fetch_opts=FetchOpts(
 repo="https://charts.jenkins.io" # Official Jenkins Helm repository
 ),
 values={
 "controller": {
 "admin": {
 "existingSecret": jenkins_admin_secret.metadata.name,
 "userKey": "jenkins-admin-user",
 "passwordKey": "jenkins-admin-password",
 },
 "resources": {
 "requests": {
 "cpu": "500m",
 "memory": "1Gi"
 },
 "limits": {
 "cpu": "2000m",
 "memory": "4Gi"
 }
 },
 "installPlugins": [
 "kubernetes",
 "workflow-aggregator",
 "git",
 "configuration-as-code",
 "blueocean" # Modern Jenkins UI
 ],
 "JCasC": { # Jenkins Configuration as Code
 "enabled": True,
 "defaultConfig": True,
 "configScripts": {
 "kubernetes-config": '''
 jenkins:
 clouds:
 - kubernetes:
 name: "kubernetes"
 serverUrl: "https://kubernetes.default:443"
 namespace: "''' + namespace_name + '''"
 credentialsId: ""
 jenkinsTunnel: "jenkins-agent:50000"
 jenkinsUrl: "http://jenkins:8080"
 maxRequestsPerHostStr: 64
 retentionTimeout: 5
 connectTimeout: 10
 templates:
 - name: "pulumi-agent"
 namespace: "''' + namespace_name + '''"
 label: "pulumi"
 nodeUsageMode: NORMAL
 containers:
 - name: "pulumi"
 image: "pulumi/pulumi:latest"
 alwaysPullImage: true
 workingDir: "/home/jenkins/agent"
 command: ""
 args: ""
 resourceRequestCpu: "500m"
 resourceRequestMemory: "1Gi"
 resourceLimitCpu: "2000m"
 resourceLimitMemory: "4Gi"
 serviceAccount: "jenkins"
 - name: "default"
 namespace: "''' + namespace_name + '''"
 label: "jenkins-agent"
 nodeUsageMode: NORMAL
 containers:
 - name: "jnlp"
 image: "jenkins/inbound-agent:latest"
 alwaysPullImage: true
 workingDir: "/home/jenkins/agent"
 command: ""
 args: ""
 resourceRequestCpu: "500m"
 resourceRequestMemory: "512Mi"
 resourceLimitCpu: "1000m"
 resourceLimitMemory: "1Gi"
 '''
 }
 }
 },
 "serviceAccount": {
 "create": False,
 "name": jenkins_service_account.metadata.name
 },
 "persistence": {
 "existingClaim": jenkins_pvc.metadata.name
 },
 "agent": {
 "enabled": True,
 "namespace": jenkins_namespace.metadata.name
 }
 }
 )
)

# Export useful information
pulumi.export("namespace", jenkins_namespace.metadata.name)
pulumi.export("jenkins_url", "http://jenkins.{}.svc.cluster.local:8080".format(namespace_name))
pulumi.export("admin_user", "admin")

In the browser, you can access Jenkins at http://localhost:8080 (or the appropriate URL for your cluster).

What Makes This Approach Special

This Pulumi program does several things that would be much more cumbersome with traditional approaches:

Infrastructure as Code: Every aspect of the Jenkins deployment is version-controlled and reproducible. No more “it works on my machine” scenarios with infrastructure.

Kubernetes-Native: The deployment leverages Kubernetes’ native capabilities for service discovery, persistent storage, and RBAC, making Jenkins a first-class citizen in your cluster.

Configuration as Code: Using Jenkins Configuration as Code (JCasC), the Jenkins instance comes pre-configured with the Kubernetes plugin and agent templates, eliminating manual setup steps.

Security by Default: Proper RBAC configurations ensure Jenkins has exactly the permissions it needs, no more and no less.

Scalable Agent Model: Build agents are ephemeral pods that scale based on demand, just like the containerized agents we used years ago, but with Kubernetes orchestration.

The beauty of this approach is that it combines the lessons learned from our Docker-based Jenkins setup with modern cloud-native practices. Agents are still containerized and ephemeral, but now they’re managed by Kubernetes rather than the Docker plugin. Storage is persistent but managed by Kubernetes PVCs. And everything is defined as code using a real programming language rather than YAML.

This setup gives you a production-ready Jenkins instance that can handle serious workloads while being completely reproducible and maintainable. But more importantly, it sets the foundation for the kind of advanced CI/CD practices that modern teams need. These practices go far beyond what we could achieve with our old VM-based approach.

Configuring Our First Pipeline to Deploy a Pulumi Program

Now comes the fun part: putting our Kubernetes-based Jenkins setup to work with a real-world scenario. What better way to test our new infrastructure than by creating a pipeline that deploys infrastructure itself? We’re going to configure Jenkins to automatically deploy Pulumi programs using Kubernetes agents, creating a complete infrastructure-as-code CI/CD workflow.

This scenario hits close to home for any modern DevOps team. Gone are the days when infrastructure changes required manual server configurations or clicking through cloud provider consoles. Today’s teams need automated, reliable ways to manage infrastructure changes through the same rigorous processes they use for application code, including peer reviews, automated testing, and controlled deployments.

The Power of Kubernetes Agents

One of the most compelling features of our Kubernetes-based Jenkins setup is its ability to spawn dedicated agent pods for specific workloads. Unlike traditional Jenkins agents that run on persistent VMs or containers, Kubernetes agents are truly ephemeral. They exist only for the duration of a single build, then disappear without a trace.

This approach solves several problems that plagued our old Docker-based setup:

Resource Efficiency: Agents consume cluster resources only when actively building, not sitting idle waiting for work.

Perfect Isolation: Each pipeline run gets its own dedicated environment with no risk of interference from previous builds.

Unlimited Scalability: Need to run 50 concurrent builds? Kubernetes will happily schedule 50 agent pods across your cluster.

Tool-Specific Environments: Different pipelines can use completely different toolchains without conflicts.

For Pulumi deployments, this means we can create specialized agent pods that include exactly the tools we need: the Pulumi CLI, cloud provider SDKs, and any additional dependencies our infrastructure code requires.

A Complete Pulumi Deployment Pipeline

Now let’s create a Jenkinsfile that demonstrates the full power of our setup. This pipeline will:

Check out a Pulumi program from source control
Install dependencies and validate the Pulumi code
Run a preview to show planned changes
Deploy the infrastructure changes (with manual approval for production)
Clean up temporary resources

pipeline {
 agent {
 kubernetes {
 label 'pulumi-deployment'
 yaml """
apiVersion: v1
kind: Pod
spec:
 serviceAccountName: jenkins
 containers:
 - name: pulumi
 image: pulumi/pulumi:latest
 command:
 - sleep
 args:
 - 99d
 env:
 - name: PULUMI_SKIP_UPDATE_CHECK
 value: "true"
 resources:
 requests:
 memory: "1Gi"
 cpu: "500m"
 limits:
 memory: "4Gi"
 cpu: "2000m"
"""
 }
 }

 parameters {
 string(name: 'GIT_URL', defaultValue: 'https://github.com/pulumi/examples', description: 'Git repository URL') // Official Pulumi examples
 string(name: 'PULUMI_FOLDER', defaultValue: 'aws-ts-s3-folder', description: 'Folder containing Pulumi program')
 }

 environment {
 PULUMI_ACCESS_TOKEN = credentials('pulumi-access-token')
 }

 stages {
 stage('Checkout') {
 steps {
 git url: "${params.GIT_URL}",
 branch: "master"
 }
 }

 stage('Install Dependencies') {
 steps {
 container('pulumi') {
 sh '''
 # Navigate to the Pulumi program directory
 cd ${PULUMI_FOLDER}/

 # Install Python dependencies if using Python
 if [ -f "requirements.txt" ]; then
 pip install -r requirements.txt
 fi

 # Install Node.js dependencies if using TypeScript/JavaScript
 if [ -f "package.json" ]; then
 npm install
 fi

 # Install Go dependencies if using Go
 if [ -f "go.mod" ]; then
 go mod download
 fi

 # Login to Pulumi service
 pulumi login

 echo "environment:\n- pulumi-ultimate-gitops/dev" > Pulumi.dev.yaml
 cat Pulumi.dev.yaml
 '''
 }
 }
 }

 stage('Preview Changes') {
 steps {
 container('pulumi') {
 sh '''
 cd ${PULUMI_FOLDER}/
 pulumi stack select dev

 # Generate a preview of changes
 pulumi preview --diff --cwd . > preview.txt

 # Display the preview
 cat preview.txt
 '''
 }

 // Archive the preview for later reference
 archiveArtifacts artifacts: "${params.PULUMI_FOLDER}/preview.txt", allowEmptyArchive: true
 }
 }

 stage('Approval') {
 steps {
 script {
 def preview = readFile "${params.PULUMI_FOLDER}/preview.txt"
 def userInput = input(
 id: 'deploy-approval',
 message: 'Deploy these infrastructure changes?',
 parameters: [
 text(name: 'PREVIEW', defaultValue: preview, description: 'Preview of changes')
 ],
 submitterParameter: 'APPROVER'
 )
 env.APPROVER = userInput
 }
 }
 }

 stage('Deploy Infrastructure') {
 steps {
 container('pulumi') {
 sh '''
 cd ${PULUMI_FOLDER}/
 pulumi stack select dev
 pulumi up --yes --cwd .
 '''
 }
 }
 }
 }

 post {
 always {
 sh "rm -f ${params.PULUMI_FOLDER}/preview.txt"
 }

 success {
 script {
 if (env.APPROVER) {
 echo "Infrastructure deployment completed successfully for stack ${env.PULUMI_STACK}. Approved by: ${env.APPROVER}"
 }
 }
 }

 failure {
 echo "Infrastructure deployment failed for stack ${env.PULUMI_STACK}. Check the build logs for details."
 }
 }
}

Open the Jenkins UI, create a new pipeline job, and paste this Jenkinsfile into the pipeline configuration.

Now we can run this pipeline to deploy a Pulumi program.

Show the preview of changes to be applied:

And wait for manual approval before proceeding with the deployment:

Key Features of This Pipeline

This Jenkinsfile demonstrates several advanced concepts that make modern CI/CD pipelines robust and production-ready:

Kubernetes-Native Agent Pods: The pipeline uses a dedicated Kubernetes pod with a Pulumi-specific container image, providing an isolated environment with all necessary tools pre-installed. The pod exists only for the duration of the build, ensuring perfect isolation between pipeline runs.

Parameterized Deployments: The pipeline accepts parameters for Git repository URL and Pulumi folder path, making it reusable across different Pulumi projects and repositories. This flexibility allows teams to standardize their deployment process while supporting diverse infrastructure codebases.

Multi-Language Support: The dependency installation stage automatically detects and installs dependencies for Python (requirements.txt), Node.js/TypeScript (package.json), and Go (go.mod) projects, making the pipeline adaptable to any Pulumi program regardless of the chosen language.

Manual Approval Gates: All deployments require explicit manual approval, with the complete preview of infrastructure changes displayed to the approver. This ensures that infrastructure modifications are always reviewed by a human before being applied to live environments.

Stack Configuration Management: The pipeline dynamically creates and configures Pulumi stack files, demonstrating how to manage environment-specific configurations programmatically within the CI/CD process.

Comprehensive Artifact Management: Preview outputs are automatically archived for audit trails and debugging, providing a historical record of what changes were planned and approved for each deployment.

You might also like:

Wrap Up

After this deep dive into Jenkins with Pulumi in 2025, I’m reminded of why Jenkins became the workhorse of CI/CD in the first place. It’s reliable, it’s battle-tested, and most importantly, it adapts. Just as we successfully containerized our build agents years ago, Jenkins has seamlessly evolved to thrive in the Kubernetes era.

The transformation from those early days of XML configurations and GUI-heavy setup processes to today’s infrastructure-as-code approach is remarkable. What we’ve built here (a fully automated Jenkins deployment with Kubernetes agents and Pulumi integration) represents the kind of setup that would have been a months-long project back in the day. Now, it’s defined in a single Python file and deployable in minutes.

But here’s the honest truth: Would I choose Jenkins for a greenfield project today? Probably not.

The modern DevOps landscape offers compelling alternatives that were purpose-built for cloud-native environments. GitHub Actions provides seamless integration with source control and requires zero infrastructure management. GitLab CI/CD offers similar convenience with its integrated approach. Cloud-native solutions like Tekton or Argo Workflows were designed from the ground up for Kubernetes, without the historical baggage that Jenkins carries.

These newer platforms offer several advantages over Jenkins:

Zero infrastructure overhead: No need to manage controllers, agents, or storage
Native cloud integration: Built-in support for modern authentication, secrets management, and scaling
Simpler configuration: Pipelines without the complexity of Jenkins’ plugin ecosystem
Better resource efficiency: No persistent controllers consuming resources when idle

However, Jenkins still has its place. If you’re working with legacy systems, have extensive existing Jenkins infrastructure, or need the ultimate flexibility that comes from its massive plugin ecosystem, Jenkins remains a solid choice. The setup we’ve demonstrated proves that Jenkins can absolutely hold its own in modern environments.

The beauty of what we’ve accomplished here isn’t just about Jenkins. It’s about the power of infrastructure-as-code. Whether you choose Jenkins, GitHub Actions, or any other CI/CD platform, the principles remain the same: define your infrastructure declaratively, version control everything, and automate the deployment process.

Jenkins may be the old workhorse, but as we’ve seen, this workhorse still has plenty of life left in it.

Next Steps

This post demonstrated how to modernize Jenkins deployment using Pulumi and Kubernetes. By leveraging infrastructure-as-code principles, we created a completely reproducible CI/CD platform that bridges traditional practices with cloud-native approaches.

If you want to explore more modern CI/CD approaches or dive deeper into infrastructure automation with Pulumi, check out our documentation and try building your own infrastructure-as-code solutions. For more insights on DevOps best practices, explore our guide on IaC best practices.

Try Pulumi for Free

AWS RDS - Blue/Green Deployments for Low-Downtime Updates

Elisabeth Lichtie — Mon, 23 Jun 2025 00:00:00 +0000

AWS RDS supports blue/green deployments to support database maintenance. In a blue/green deployment, you have one production (blue) and one staging (green) database. You can safely make changes to the green instance without affecting production and promote it to be the main instance. When you enable blue/green updates, Pulumi will temporarily set up a blue/green deployment for the duration of the update to minimize downtime.

There are many benefits to using blue/green deployments for updates, especially for critical databases that need to be kept available and safe from untested changes.

How it works

By default, updates to a database will be applied in-place, meaning there may be service interruptions while a database is modified especially when making long-running updates like changing the engine version, parameter groups, or storage settings.

blueGreenUpdate can be enabled for Low-Downtime Updates. When this setting is on and an update is performed, the program starts the update by creating a blue/green deployment to maintain availability. Then, it performs the requested update on the green instance, performs a guardrail check, promotes the green instance, and finally deletes the old instance and removes the blue/green deployment. This leaves just the updated database running with minimal downtime during the update while minimizing the downtime.

Though this strategy typically increases the amount of time that updates need to run, it minimizes downtime for the application by keeping a current version of the database available during the update process.

How to use

Please see the full example code here: Blue/Green Deployment Updates on RDS. To use blue/green deployments for your database updates, start by creating a database and corresponding parameter group.

const dbVersion = "14.18";

const parameterGroup = new aws.rds.ParameterGroup(
 "parameter-group",
 {
 family: `postgres${dbVersion.split(".")[0]}`,
 parameters: [
 {
 name: "rds.logical_replication",
 value: "1",
 applyMethod: "pending-reboot",
 },
 ],
 },
 { deleteBeforeReplace: false } // This explicitly calls out that the new group should be created before the old is deleted so that database availability can be maintained
);

const database = new aws.rds.Instance(
 "rds-instance",
 {
 allocatedStorage: storageAllocation,
 engine: "postgres",
 engineVersion: dbVersion,
 instanceClass: aws.rds.InstanceType.T3_Micro,
 username: dbUsername,
 password: dbPW,
 storageEncrypted: true,
 backupRetentionPeriod: 7,
 blueGreenUpdate: { enabled: true },
 skipFinalSnapshot: true,
 parameterGroupName: parameterGroup.name,
 },
 { dependsOn: [parameterGroup] } //This line can be omitted since the dependency is also inferred by the reference to parameterGroup.name
);

After running pulumi up with the above code, you’ll have a database deployed and attached to your custom parameter group. It’s important to create and attach a custom parameter group so that you can keep it up to date with any engine version upgrades. If you use the default parameter group, updates to the database version may fail when the green deployment is created since its parameter group family won’t match the new version. You can avoid a failure by ensuring that the parameter group’s family and engineVersion are always changed in the same update.

You won’t see the effects of having blueGreenUpdate enabled until you run your first update. Change any part of the database configuration and run pulumi up again to start the update. In the example above, you can change dbVersion to 15.13 to trigger an update to both the parameter group and the instance.

Once the update kicks off, you can watch on AWS as the blue/green deployment is created and the original settings are copied to the green instance.

Then, the green instance will be updated with the new settings and the program will switch over to the green instance and clean up.

After that, your update has been completed with minimal disruption!

Additional Considerations

General limitations and considerations for Amazon RDS blue/green deployments are listed in the AWS documentation, but there are a few other considerations that should be taken into account when using them for updates with infrastructure as code (IaC).

No Custom Checks: Because IaC will automatically perform the switchover when the update is complete, you will not have the opportunity to perform custom checks on the green instance before promoting it. RDS will run some basic guardrails checks before performing the switchover, but you will not be able to perform any additional checks.
No Cluster Support: This feature only works with database instances, not with clusters or instances that are in a cluster.
Long Update Times: Because the IaC program will deploy temporary resources before performing the update and destroy them after, blue/green updates will take considerably longer than their in-place counterparts.
Manual Cleanup: Because the green instance and blue/green deployment are temporary resources designed to decrease downtime during the update, they are not tracked in the IaC state. If the update fails partway through, these temporary resources may have to be cleaned up manually. If the update is interrupted, you can delete the blue/green deployment and green database to get back into a manageable state.
Resource ID Change: On an update, the resource ID of the database might change since a new instance has been created. The endpoint and other details about the database will be maintained by AWS during switchover, but the resource ID should be dynamically retrieved from the stack outputs whenever it is used to ensure it stays up to date. Usually, no application or connection details will need to be updated since they don’t rely on the resource ID itself.
Types Of Updates: The blue/green update strategy will be used for changes that require a maintenance window including updates to the engine version, parameter group, storage and performance settings, and optimized writes.

AI-Assisted Infrastructure as Code with Pulumi's Model Context Protocol Server

Mikhail Shilkov — Tue, 08 Apr 2025 00:00:00 +0000

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

Infrastructure as Code (IaC) has revolutionized how we manage cloud resources, but navigating complex cloud provider APIs, writing boilerplate code, and iterating through deployment cycles can still be time-consuming. Pulumi offers a fantastic developer experience using familiar programming languages. But what if we could make it even faster and more intuitive by integrating powerful AI assistants directly into the development loop?

This is where the Pulumi Model Context Protocol (MCP) Server integration shines. MCP is a specification that allows language models (like the AI in your coding assistant) to interact with external tools and data sources in a structured way. By connecting AI-powered code assistants with Pulumi’s CLI and registry via MCP, we can bring real-time resource information and infrastructure management directly into the development environment, dramatically reducing friction and accelerating workflows.

Several AI coding assistants like GitHub Copilot, Anthropic’s Claude Code, Windsurf and others are rapidly evolving; this post will use Cursor (an AI-first code editor) to demonstrate a real-world example of this synergy in action.

Setting up the Pulumi MCP Integration in Cursor

Before diving in, you typically need to configure your AI assistant to communicate with the Pulumi MCP server. For Cursor, you can do so by clicking the following button:

The assistant can now leverage Pulumi tools seamlessly. These tools are specific actions enabled by the MCP server—like searching the Pulumi Registry or running a pulumi command—allowing the assistant to gather information or interact with your Pulumi project.

For detailed setup instructions for Cursor, Claude Code, Windsurf, Claude Desktop, and other AI assistants, see the Pulumi MCP Server documentation.

The Goal: Provisioning an AKS Cluster

Our objective for this walkthrough is to provision a temporary Azure Kubernetes Service (AKS) cluster for a short experiment using Pulumi and TypeScript. We need the cluster created with minimal fuss and its kubeconfig exported for access.

The Traditional Approach

The conventional method for this task involves significant context switching: searching Azure and Pulumi documentation in a browser, writing code in an editor, running commands (pulumi preview, pulumi up) in a terminal, and manually correlating information between these different environments. This process can be slow, requires deep knowledge recall (or constant lookups), and is prone to errors during the manual translation from documentation to code.

The AI Assistant + Pulumi MCP Approach

Let’s walk through how the same task unfolds much more efficiently using an AI assistant integrated with Pulumi MCP.

Understanding the AI Assistant + Tool Interaction

Before detailing the steps, it’s helpful to understand the user interface flow when the AI assistant uses integrated tools like those provided by the Pulumi MCP integration. Typically:

The developer provides a prompt or instruction in the chat interface within their editor.
The AI assistant analyzes the request and determines that it needs specific information (like resource properties) or needs to perform an action (like running pulumi preview).
The assistant indicates it will use a specific tool (e.g., pulumi_registry_listResources, pulumi_cli_preview). This often appears as a distinct UI element in the chat, showing the tool name and parameters being used.
The tool executes via the MCP server, interacting with the Pulumi CLI or registry as needed.
The output or result from the tool (e.g., registry listings, preview results, deployment errors) is displayed directly in the chat interface.
The AI assistant processes this output and uses it to continue the task – either by generating code, providing an answer, or deciding on the next step (like suggesting a fix for an error).

This tight loop keeps the developer focused within their editor environment, minimizing disruptions.

Step-by-Step Walkthrough

Here’s how the AKS provisioning task played out:

The Request

The developer starts with a natural language request to the AI assistant within the editor:

“I have an empty Pulumi project with TypeScript. Please edit the program to provision an AKS cluster for me. It’s a temporary AKS cluster that I need for a short experiment, so I don’t need any particular configuration of it. Just export its kubeconfig when you are done. Please use the tools to lookup resource information and to run pulumi preview to make sure your code works. When done, run pulumi up for me.”

AI-Powered Resource Discovery

Instead of the developer manually searching docs, the AI assistant leverages the MCP integration.

The AI assistant first queries the Pulumi Registry via MCP to list resources within the Azure Native provider’s containerservice module (using the pulumi_registry_listResources tool). This immediately identified ManagedCluster as the relevant resource.
The AI assistant then requests detailed information specifically for ManagedCluster (using pulumi_registry_getResource), directly retrieving its required properties, descriptions, and structure without leaving the editor.

AI Code Generation & Editing

Using the information retrieved from the Pulumi Registry, the assistant generated the necessary TypeScript code. It defined the ManagedCluster resource with a basic configuration and used its code editing capabilities to insert the code directly into the developer’s index.ts file.

Integrated Validation

Before attempting a potentially time-consuming deployment, the assistant ran pulumi preview using the integrated CLI tool (pulumi_cli_preview). The preview output appeared directly in the chat.

The preview succeeded, showing the resources that would be created. The assistant then proceeded to run pulumi up using the integrated CLI tool (pulumi_cli_up). However, the deployment flagged an error: an incorrect attempt to interpolate a Pulumi Output<string> directly into a resource property.

AI-Assisted Debugging & Iteration

The assistant analyzed the error message from the tool:

It identified the incorrect string interpolation and proposed a fix (using pulumi.interpolate), applying it via the editing tool.
Self-correction: The first fix inadvertently introduced a circular dependency (caught by linters integrated into the editor and surfaced to the assistant). The assistant recognized this and further refined the code by removing the problematic property, correctly relying on Azure’s default behavior.
Another attempt to deploy using pulumi up failed, this time surfacing an Azure API error: the provided SSH key format was invalid.

Collaborative Problem Solving & Tooling

The developer, seeing the SSH key error, suggested using the Pulumi TLS package to generate a valid key dynamically.

The assistant used the integrated terminal tool to execute npm install @pulumi/tls.
It then edited the index.ts file again, incorporating the tls.PrivateKey resource and correctly using its publicKeyOpenssh output for the ManagedCluster’s Linux profile.

Successful Deployment

The subsequent pulumi preview showed the correct plan (including the new TLS key resource). The assistant then executed pulumi up, which completed successfully. The success message and resource summary appeared in the chat.

Accessing Outputs

Finally, retrieving the kubeconfig was trivial. The assistant used the stack output tool (pulumi_cli_stack_output kubeconfig) to fetch and display the configuration directly.

Why This Changes the Game

Integrating AI assistants with the Pulumi Model Context Protocol offers tangible benefits:

Reduced Context Switching: The entire workflow – discovery, coding, validation, deployment, debugging, output retrieval – happens primarily inside the editor. No more juggling browser tabs and terminal windows.
Accelerated Discovery: MCP integration provides immediate, context-aware access to Pulumi resource schemas and documentation snippets.
Faster Coding: AI generates boilerplate and resource definitions quickly based on registry data and natural language requests.
Tighter Feedback Loop: pulumi preview and pulumi up results are instantly available within the coding environment, enabling rapid iteration.
Intelligent Assistance & Collaboration: The AI assists with debugging complex errors, incorporates developer suggestions, and leverages integrated tooling effectively.

Conclusion

The synergy between AI coding assistants and the Pulumi Model Context Protocol (MCP) integration creates a remarkably efficient environment for Infrastructure as Code development. By bringing cloud resource knowledge, code generation, and the Pulumi CLI workflow directly into the editor, developers can build, deploy, and iterate on their infrastructure faster and with significantly less friction.

Whether you use Cursor, Copilot, Claude Code, Windsurf, or another emerging AI tool, integrating it with the Pulumi MCP server offers a glimpse into the future of streamlined, intelligent IaC development, ultimately boosting productivity and improving the overall developer experience.

Host your Python app for $1.12 a month

Adam Gordon Bell — Wed, 29 Jan 2025 00:00:00 +0000

TL;DR (2026 pricing): Packaging a Flask app as a container and running it on AWS Lambda behind an HTTP API Gateway costs ~$1.12/month worst-case, or $0 when idle. The breakdown: $0.04 for API Gateway requests, $0.07 for Lambda compute (fully absorbed by the always-free tier), and $1.08 for 12 GB of egress at $0.09/GB (covered by AWS’s 100 GB/month free egress allowance unless other services in the account have already used it). Pulumi handles the entire infrastructure. Verified against AWS pricing as of April 2026.

How cheap can you host a Python app in 2026? For a low-traffic Flask API (say, 40,000 requests per month at 512 MB of memory), the answer is roughly $1.12/month on AWS worst-case, dropping to $0 when idle. The trick is to stop thinking of AWS Lambda as “one function per endpoint” and instead package your entire web framework as a container, deploy it to Lambda, and put it behind an HTTP API Gateway. Your code stays standard Flask. Your bill stays in the loose-change zone.

This post walks through the whole setup with Pulumi, then compares the resulting cost against Google Cloud Run, Fly.io, Railway, and Vercel using current 2026 prices.

Most developers maintain at least one low-traffic service that still needs to be reliably available: an internal reporting API, a webhook receiver, a side project with occasional use. Traditional hosting means paying for 24/7 server time, even when the service sits idle. AWS Lambda’s container support flips that: pay only when your API is being called, and any web framework that handles HTTP requests (Flask, FastAPI, Express) works without modification.

How does the pricing break down? (2026)

Plugging 40,000 requests per month at 512 MB memory and ~200 ms average execution into 2026 AWS pricing:

Lambda compute: 40,000 × 0.5 GB × 0.2 s = 4,000 GB-seconds × $0.0000166667 = $0.067, fully covered by the always-free tier (400,000 GB-seconds free per month).
Lambda requests: 40,000 × $0.20 / 1,000,000 = $0.008, also covered by the free tier (1M requests free per month).
HTTP API Gateway: 40,000 × $1.00 / 1,000,000 = $0.04.
Data transfer out: 12 GB × $0.09/GB = $1.08 if billed, but AWS’s always-free tier (expanded to 100 GB/month outbound in late 2021) covers this row outright unless other services in the same account have already exhausted that allowance.
Total: ~$0.04/month (egress in the free tier, only API Gateway bills) to ~$1.12/month (12 GB egress billed in full: $1.08 + $0.04 API Gateway), with Lambda compute and requests always fully covered by the free tier.

The underlying math hasn’t changed since this post was first published in early 2025; Lambda and API Gateway pricing has been remarkably stable, and AWS has only expanded free-tier allowances since.

Cheap Python hosting compared (2026)

Provider	Approx. cost / month	2026 notes
AWS Lambda + HTTP API	~$1.12	~$1.08 for 12 GB egress at $0.09/GB + $0.04 for 40k API Gateway requests at $1/M; Lambda compute ($0.067) is fully covered by the always-free tier (1M req/400k GB-s).
Google Cloud Run	~$1.44	2M requests, 360,000 GiB-seconds of memory, and 180,000 vCPU-seconds are free every month — compute is $0. Egress dominates: Premium tier is ~~$0.12/GB, so 12 GB ≈ $1.44. Standard tier (~~$0.085/GB) lands near $1.02.
Fly.io	~$2.18	Cheapest persistent `shared-cpu-1x@256MB` machine is ~$1.94/mo; 12 GB outbound at $0.02/GB (NA/EU) adds ~$0.24 (free tier was retired in late 2024).
Railway	~$6.00	Hobby plan starts at $5/mo and includes $5 of usage; a 512 MB always-on container plus 12 GB egress lands around $5.50–6.50 total.
Vercel (Pro)	$20+	Pro plan has a $20/mo floor; 40k function invocations are well under the included quotas, so you mostly pay for the seat.

Approximate 2026 hosting costs for a 512 MB Python API at 40,000 requests/month with 12 GB egress. Egress dominates at this scale; even 1 million requests/month stays under $30 on Lambda.

What we’ll cover:

Why use AWS Lambda for low-traffic Python apps?

“But Lambda means one function per REST endpoint, right?” While that’s how serverless was initially positioned, the Lambda monolith approach here is a more straightforward pattern. Take any REST API service, package it as a container, put it in Lambda, and give it an entire HTTP route behind an IP, domain name, or API Gateway prefix. Done.

This approach gives you several benefits:

Minimal Serverless Knowledge Required: You don’t need to be an expert in Lambda architecture or serverless patterns. If you can build a REST API and containerize it, you’re ready to go.
Standard Development Experience: Write your API exactly as you would for any other hosting platform: no special frameworks, no Lambda-specific patterns to learn.
Simple Local Development: Run your container locally, and it will behave exactly like it will in production. No complex serverless emulation is needed.
Zero Cost at Zero Usage: Get all the cost benefits of serverless without restructuring your application.

The trade-off? Cold starts when Lambda spins up a new container instance. But this is often a worthwhile exchange for operational simplicity and zero-cost idle periods for low-traffic services.

Let me show you how to set this up. With some Pulumi Python code for setting up the Infra and some containerization code, you get an easy hosting solution.

Why containers?

Containers let us package everything—code, system dependencies, runtime—into one unit. Any HTTP service that fits in a container will work: Flask, Express, Rails, plus whatever system tools you need. Local testing? Just start the container. No Lambda-specific configuration is required.

How do I build a Flask app for Lambda?

Let’s start with the fun part - writing some Python code, but I will keep it really simple. We’ll build a simple Flask application that demonstrates why running a full web framework in Lambda makes sense.

Here’s our starting point in scripts/simple_server.py

from typing import Dict
from flask import Flask, jsonify, request
from mangum import Mangum

# Create Flask app
app = Flask(__name__)

@app.route("/hello")
def hello() -> Dict[str, str]:
 return jsonify({"message": "Hello from Flask!"})

@app.route("/echo", methods=['POST'])
def echo() -> Dict[str, str]:
 data = request.get_json()
 return jsonify({"reversed": data['message'][::-1]})

# Create Lambda handler
lambda_handler = Mangum(app)

if __name__ == "__main__":
 app.run(host="0.0.0.0", port=3000)

Notice a few key things about this code:

It’s Just Flask: No Lambda-specific code, no special frameworks. If you know Flask, you know how to modify this.
Multiple Endpoints: We have several routes demonstrating different HTTP methods and URL patterns.

How do I containerize a Flask app for Lambda?

To containerize our Flask app for Lambda, we’ll start with a Dockerfile that meets AWS’s requirements. The key is using the right base image and adapting our application to Lambda’s runtime interface.

First, let’s create our Dockerfile:

FROM public.ecr.aws/lambda/python:3.12

# Set up working directory
WORKDIR /app
COPY scripts/simple_server.py scripts/
COPY scripts/simple_entrypoint.sh scripts/

# Install dependencies
RUN pip install --no-cache-dir \
 flask>=3.0.0 \
 mangum>=0.17.0

# Make entrypoint script executable
RUN chmod +x scripts/simple_entrypoint.sh

# Set environment variables
ENV FLASK_APP=scripts/simple_server.py
ENV PYTHONPATH=/app
ENV RUN_MODE=lambda
EXPOSE 3000

# Use our entrypoint script
ENTRYPOINT ["/app/scripts/simple_entrypoint.sh"]

Notice that:

We’re using AWS’s official Lambda Python base image. You can use your own base image and your Python package manager of choice, but starting with this base and pip install is the easiest.
The ${LAMBDA_TASK_ROOT} is a special directory where Lambda expects to find your code
We’re introducing a new file: simple_entrypoint.sh

Do I need to learn Lambda-specific patterns?

The beauty of everything covered so far is that it’s just a container. You can work with it locally with Docker run, put it in Kubernetes, or run it in one of the 1000 places you can run containers. Lambda has some odd requirements about function calling that get in the way of this, but simple_entrypoint.sh saves the day:

#!/bin/bash

if [ "$RUN_MODE" = "local" ]; then
 echo "Starting in local mode..."
 exec python scripts/simple_server.py
else
 echo "Starting in Lambda mode..."
 exec python -m awslambdaric scripts.simple_server.lambda_handler
fi

All this means is that locally, you don’t need to know about lambdas at all. If you supply RUN_MODE=local it runs the standard way you are used to:

#!/bin/bash

# Build the container
docker build -t simple-flask -f infra/Dockerfile .
docker run -d -p 3000:3000 -e RUN_MODE=local simple-flask

# Test the endpoints:
# 1. Test GET /hello
curl http://localhost:3000/hello

# 2. Test POST /echo with a message
curl -X POST "http://localhost:3000/echo" \
 -H "Content-Type: application/json" \
 -d '{"message": "hello world"}'

This local testing setup is crucial because it helps catch issues before deployment.

The beauty of this setup is that once it works locally, it’ll work the same way in AWS.

And since this is just a standard Python application in a container, you can use all your favorite Python tools and patterns – your preferred package manager (pip, poetry, uv), pytest for testing, standard logging, type hints, linting, and pre-commit hooks.

( I’m skipping those details here to focus on deployment, but the container approach means there are no serverless-specific limitations on your Python workflow. )

How do I deploy a Lambda container with Pulumi?

Now let’s get our containerized Flask app into AWS. While you could click around in the AWS console, we’re going to do this properly with infrastructure as code. You are welcome to use Cloudformation or manual setup, but it’s nice to be able to keep everything in Python, and this is the Pulumi blog. Using Python to define our AWS resources means we get type-checking and proper IDE support.

Here’s our infrastructure code:

Don’t let the infrastructure code below intimidate you. It’s just typed Python code, hooking up the Lambda, that you can version control and modify easily.

from typing import Any
import pulumi
import pulumi_aws as aws
import pulumi_awsx as awsx
import json

# First, create an API Gateway
api_gateway: aws.apigatewayv2.Api = aws.apigatewayv2.Api("flask-api",
 name="Flask Lambda API",
 protocol_type="HTTP",
 route_selection_expression="$request.method $request.path"
)

# Create IAM role for Lambda
lambda_role: aws.iam.Role = aws.iam.Role("flask-lambda-role",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": "sts:AssumeRole",
 "Principal": {
 "Service": "lambda.amazonaws.com"
 },
 "Effect": "Allow",
 "Sid": ""
 }]
 })
)

# Attach basic execution policy - Lambda needs this to write logs
lambda_role_policy: aws.iam.RolePolicyAttachment = aws.iam.RolePolicyAttachment("lambda-role-policy",
 role=lambda_role.name,
 policy_arn="arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
)

# Create ECR repository for our container
repository: aws.ecr.Repository = aws.ecr.Repository("flask-app-repo",
 name="flask-app-repo",
 force_delete=True, # Makes cleanup easier for testing
 image_scanning_configuration=aws.ecr.RepositoryImageScanningConfigurationArgs(
 scan_on_push=True,
 )
)

# Build and push the Docker image to ECR
image: awsx.ecr.Image = awsx.ecr.Image("flask-app-image",
 repository_url=repository.repository_url,
 path=".", # Path to your Dockerfile
 platform="linux/amd64" # Important for M1/M2 Mac users
)

# Create the Lambda function
lambda_function: aws.lambda_.Function = aws.lambda_.Function("flask-app",
 name="flask-app",
 package_type="Image",
 image_uri=image.image_uri,
 role=lambda_role.arn,
 timeout=30,
 memory_size=512,
 environment={
 "variables": {
 "FLASK_ENV": "production",
 "PYTHONUNBUFFERED": "1"
 }
 }
)

# Create API Gateway stage
stage: aws.apigatewayv2.Stage = aws.apigatewayv2.Stage("api-stage",
 api_id=api_gateway.id,
 name="$default",
 auto_deploy=True
)

# Connect API Gateway to Lambda
integration: aws.apigatewayv2.Integration = aws.apigatewayv2.Integration("lambda-integration",
 api_id=api_gateway.id,
 integration_type="AWS_PROXY",
 integration_uri=lambda_function.arn,
 integration_method="POST",
 payload_format_version="2.0"
)

# Create catch-all route
route: aws.apigatewayv2.Route = aws.apigatewayv2.Route("catch-all-route",
 api_id=api_gateway.id,
 route_key="ANY /{proxy+}",
 target=integration.id.apply(lambda id: f"integrations/{id}")
)

# Allow API Gateway to invoke Lambda
lambda_permission: aws.lambda_.Permission = aws.lambda_.Permission("api-lambda-permission",
 action="lambda:InvokeFunction",
 function=lambda_function.name,
 principal="apigateway.amazonaws.com",
 source_arn=api_gateway.execution_arn.apply(lambda arn: f"{arn}/*/*")
)

# Export the API Gateway URL
pulumi.export("url", api_gateway.api_endpoint)

Let’s break down what’s happening here:

API Gateway: We create an HTTP API (v2) that will receive all web requests and forward them to our Lambda.
IAM Role: Lambda needs permissions to execute and write logs. We create a role with the minimum necessary permissions.
Container Registry: We create an ECR repository and use Pulumi’s awsx package to build and push our Docker image automatically.
Lambda Function: We create the Lambda function using our container image. Notice how we set memory and timeout - these are important for a web application.
Integration: We connect API Gateway to Lambda using the AWS_PROXY integration type. ANY /{proxy+} means we route everything through with the full path.

Need a custom domain, that’s easy to use as well. Setup some Route53 records, CNAME a domain name appropriately and you’ll have a container backed by a custom domain name.

Why is this approach worth it?

The cool thing about using Pulumi and lambdas is that they make complex infrastructure changes safe and repeatable.

Want to increase the Lambda memory? Change one number and run pulumi up.
You get complete AWS Lambda and API Gateway management for your low-traffic REST APIs
And Infrastructure testing capabilities mean you can validate your serverless configuration before deployment

You can find the full code in the service status monitor repo from my Python devops article. A larger version with rolling updates is coming soon.

Alternative approaches

There are other ways to run containers that scale to zero. Google Cloud Run offers similar functionality on GCP, and AWS App Runner is another AWS service that can do this. Both have similar pricing models—very cheap for low-volume services. And SST is a similar solution for getting TypeScript / JavaScript solutions into an AWS Lambda.

The beauty of the container approach used here is that switching between these services is straightforward. Since we’re just running a standard container with a REST endpoint, there’s nothing AWS-specific in our application code. If you need to move to a different platform later, you can take your container with you.

Conclusion

Running Flask applications in Lambda containers represents a sweet spot between development simplicity and operational efficiency. You get:

True zero-cost periods when your application isn’t being used
Familiar development patterns - it’s just Flask
Infrastructure as code with Pulumi
Automatic scaling without managing servers

The trade-offs - primarily cold starts and some AWS-specific configuration - are worth it for many applications, especially:

APIs with sporadic traffic
Cost-sensitive projects

The key is understanding that this isn’t just about saving money - it’s about reducing operational complexity. No more worrying about scaling policies, server maintenance, or capacity planning. Your application scales automatically, and you only pay for what you use.

Is this approach right for every application? No, but this combination of familiar local development with serverless operations is under utilized.

I’d love to hear about your experiences with monolithic serverless applications. What hosting solutions have you tried? What challenges have you encountered?

Get started with Pulumi on AWS: the official quickstart for the tooling used in this post.
AWS Lambda function reference: every property on the resource we used above.
Code, deploy, and manage a serverless REST API on AWS with Pulumi: the function-per-route alternative to the monolith pattern.
Controlling AWS costs with Lambda and Pulumi: patterns for keeping your bill predictable as traffic grows.
The cursed container iceberg: every other place you can run the same container if you outgrow Lambda.
Python for DevOps: the companion post that ships the full source code for this example.

Changelog

2026-04-30: Refreshed for 2026. Re-verified Lambda/API Gateway/egress math, replaced the comparison table with current Cloud Run, Fly.io, Railway, and Vercel pricing, restructured H2s as user questions, added an answer-first intro and TL;DR, added a worked cost example and a HowTo schema for the deployment steps.
2025-03-13: Minor copy edits.
2025-01-29: Original publication.

Run Open-Source LLMs on AWS EC2 with Ollama and Pulumi

Engin Diri — Mon, 27 Jan 2025 00:00:00 +0000

TL;DR. Want to self-host an open-source LLM on AWS? Use a g4dn.xlarge ($0.526/hr on-demand, 16 GB GPU memory) for 7B/8B models, a g5.xlarge ($1.006/hr, 24 GB) for 13B–14B models, a g5.2xlarge ($1.212/hr, 24 GB) for 32B models, or a g6e.2xlarge ($2.242/hr, 48 GB) for 70B models. Deploy with the Pulumi program below and Ollama will run any model from its library: DeepSeek-R1, Llama 3, Qwen, or Mistral, with a one-line change.

This guide walks through that deployment end-to-end: a single Pulumi program that provisions a GPU-enabled EC2 instance, installs Ollama and Open WebUI via cloud-init, and exposes both a chat UI and an OpenAI-compatible API. The model is configurable, so you can swap DeepSeek-R1 for Llama 3.1, Qwen 2.5, or Mistral without touching the infrastructure code.

Why run open-source LLMs on AWS EC2?

Self-hosting an open-source LLM on AWS gives you three things hosted APIs can’t: data stays inside your VPC, per-token costs collapse to a flat hourly rate at high volume, and you can fine-tune or quantize models freely under permissive licenses. Ollama handles all three concerns from a single binary: it downloads, manages, and serves models behind an OpenAI-compatible API on port 11434.

The original version of this post focused on DeepSeek-R1 because it landed in late January 2025 and reset expectations for what an open-weight reasoning model could do. DeepSeek-R1 is still an excellent default (MIT-licensed, strong on math and coding, with distilled 1.5B–70B variants) but the same infrastructure runs Meta’s Llama 3, Alibaba’s Qwen, and Mistral equally well. Picking a model is now a config change, not an infrastructure decision.

For reference, DeepSeek-R1 scores 79.8% on AIME 2024 (vs. 79.2% for OpenAI o1-1217), 97.3% on MATH-500 (vs. 96.4%), 96.3% on Codeforces (vs. 96.6%), and 49.2% on SWE-bench Verified (vs. 48.9%)—near-parity with closed frontier models on most reasoning benchmarks, with the same caveat that benchmark scores age fast.

You might also like:

Which models can I run, and which EC2 instance do I need?

The bottleneck for inference is GPU memory (VRAM): the model weights have to fit in it, plus a few GB of headroom for the KV cache. The table below maps the most common Ollama models to the smallest AWS GPU instance that comfortably runs them at 4-bit (Q4) quantization, which is what ollama pull <model> gives you by default.

Model family	Sizes	Approx. VRAM (Q4)	Smallest EC2 instance	On-demand price (us-east-1)
DeepSeek-R1 (distill)	1.5B / 7B / 8B	1–6 GB	`g4dn.xlarge` (T4, 16 GB)	$0.526/hr
Llama 3.1 / Llama 3.2	8B	~5 GB	`g4dn.xlarge` (T4, 16 GB)	$0.526/hr
Qwen 2.5	7B	~5 GB	`g4dn.xlarge` (T4, 16 GB)	$0.526/hr
Mistral 7B / Mistral Nemo	7B / 12B	5–8 GB	`g4dn.xlarge` (T4, 16 GB)	$0.526/hr
DeepSeek-R1 (distill)	14B	~10 GB	`g5.xlarge` (A10G, 24 GB)	$1.006/hr
Llama 3.3 / DeepSeek-R1	32B / 32B distill	~20 GB	`g5.2xlarge` (A10G, 24 GB)	$1.212/hr
Llama 3.1 / DeepSeek-R1	70B	~42 GB	`g6e.2xlarge` (L40S, 48 GB)	$2.242/hr
DeepSeek-R1 (full)	671B (MoE)	400 GB+	`p5.48xlarge` or multi-node	$98.32/hr

For most workloads—internal tools, RAG backends, code assistants—a g4dn.xlarge running an 8B model is the right starting point. Move up only if quality is the bottleneck.

How much does this cost vs. hosted APIs?

A g4dn.xlarge running 24/7 costs ~$378/month on-demand, or ~$237/month with a 1-year reserved instance. Whether that’s cheaper than a hosted API depends entirely on your token volume.

Compare against hosted pricing as of April 2026 (input + output blended, rough numbers):

Provider	Model	Approx. blended price
OpenAI	GPT-4o-mini	~$0.30 per 1M tokens
OpenAI	GPT-4o	~$5.00 per 1M tokens
Anthropic	Claude Sonnet 4	~$6.00 per 1M tokens
DeepSeek (hosted)	DeepSeek-V3	~$0.50 per 1M tokens

A g4dn.xlarge running Llama 3.1 8B sustains roughly 40–60 tokens/sec under single-user load, or about 100–155M tokens/month at 100% utilization. At that ceiling the effective rate is ~$2.40/M tokens—cheaper than GPT-4o or Claude, more expensive than GPT-4o-mini or DeepSeek’s own hosted API.

The takeaway: self-hosting wins on data residency, latency, and predictable cost at high utilization. Hosted APIs win below ~10M tokens/month or when you need frontier-class quality. Run the math against your actual token volume before committing.

How do I deploy Ollama on AWS EC2 with Pulumi?

Prerequisites

Before we start, make sure you have the following:

An AWS account
Pulumi CLI installed
AWS CLI installed and configured
A working understanding of Ollama

What is Ollama?

Ollama is an open-source runtime that downloads, manages, and serves large language models from a single binary. It runs on macOS, Linux, and Windows, supports GPU acceleration through CUDA and Metal, and exposes both a native HTTP API and an OpenAI-compatible API on port 11434. Most clients written for the OpenAI SDK work against an Ollama endpoint with only a base-URL change.

It supports the major open-weight families—DeepSeek-R1, Llama 3, Qwen, Mistral, Gemma, Phi—plus quantized and distilled variants for each. You pull a model by name and tag (ollama pull llama3.1:8b) and run it (ollama run llama3.1:8b); Ollama handles the rest.

Architecture

Create a new Pulumi project

First, scaffold a new Pulumi project. Run the following from an empty directory:

# Replace <language> with typescript, python, go, csharp, or yaml
pulumi new aws-<language>

Pick the language you are most comfortable with. The template installs the AWS provider and creates a working sample. You can delete the sample code—we’ll replace it with the snippets below.

Step 1: Create an instance role with S3 access

The EC2 instance needs to download NVIDIA drivers from a public AWS-managed S3 bucket. Create an IAM role with S3 read access and attach it to an instance profile:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as fs from "fs";

const role = new aws.iam.Role("deepSeekRole", {
 name: "deepseek-role",
 assumeRolePolicy: JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Action: "sts:AssumeRole",
 Effect: "Allow",
 Principal: {
 Service: "ec2.amazonaws.com",
 },
 },
 ],
 }),
});

new aws.iam.RolePolicyAttachment("deepSeekS3Policy", {
 policyArn: "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess",
 role: role.name,
});

const instanceProfile = new aws.iam.InstanceProfile("deepSeekProfile", {
 name: "deepseek-profile",
 role: role.name,
});

import pulumi
import pulumi_aws as aws
import json
import os

# IAM Role for EC2 instances
role = aws.iam.Role(
 "deepSeekRole",
 name="deepseek-role",
 assume_role_policy=json.dumps(
 {
 "Version": "2012-10-17",
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": "ec2.amazonaws.com",
 },
 }
 ],
 }
 ),
)

# Attach S3 read-only policy to the IAM Role
iam_policy_attachment = aws.iam.RolePolicyAttachment(
 "deepSeekS3Policy",
 policy_arn="arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess",
 role=role.name,
)

# Instance Profile containing the IAM Role
instance_profile = aws.iam.InstanceProfile(
 "deepSeekProfile", name="deepseek-profile", role=role.name
)

package main

import (
 "encoding/json"
 "os"

 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/iam"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 rolePolicy, err := json.Marshal(map[string]interface{}{
 "Version": "2012-10-17",
 "Statement": []map[string]interface{}{
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": map[string]interface{}{"Service": "ec2.amazonaws.com"},
 },
 },
 })
 if err != nil {
 return err
 }

 role, err := iam.NewRole(ctx, "deepSeekRole", &iam.RoleArgs{
 Name: pulumi.String("deepseek-role"),
 AssumeRolePolicy: pulumi.String(rolePolicy),
 })
 if err != nil {
 return err
 }

 // Attach S3 read-only policy to the IAM Role
 _, err = iam.NewRolePolicyAttachment(ctx, "deepSeekS3Policy", &iam.RolePolicyAttachmentArgs{
 PolicyArn: pulumi.String("arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"),
 Role: role.Name,
 })
 if err != nil {
 return err
 }

 // Instance Profile containing the IAM Role
 instanceProfile, err := iam.NewInstanceProfile(ctx, "deepSeekProfile", &iam.InstanceProfileArgs{
 Name: pulumi.String("deepseek-profile"),
 Role: role.Name,
 })
 if err != nil {
 return err
 }
 return nil
 })
}

using Pulumi;
using Pulumi.Aws.Ec2;
using Pulumi.Aws.Iam;
using System.Collections.Generic;
using System.IO;
using Pulumi.Aws.Ec2.Inputs;
using System.Text.Json;

return await Deployment.RunAsync(() =>
{
 var rolePolicy = JsonSerializer.Serialize(new Dictionary<string, object>
 {
 {
 ["Version"] = "2012-10-17",
 ["Statement"] = new[]
 {
 new Dictionary<string, object>
 {
 ["Action"] = "sts:AssumeRole",
 ["Effect"] = "Allow",
 ["Principal"] = new Dictionary<string, string>
 {
 ["Service"] = "ec2.amazonaws.com"
 }
 }
 }
 });

 var role = new Role("deepSeekRole", new RoleArgs
 {
 Name = "deepseek-role",
 AssumeRolePolicy = rolePolicy
 });

 var rolePolicyAttachment = new RolePolicyAttachment("deepSeekS3Policy", new RolePolicyAttachmentArgs
 {
 PolicyArn = "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess",
 Role = role.Name
 });

 var instanceProfile = new InstanceProfile("deepSeekProfile", new InstanceProfileArgs
 {
 Name = "deepseek-profile",
 Role = role.Name
 });

 var vpc = new Vpc("deepSeekVpc", new VpcArgs
 {
 CidrBlock = "10.0.0.0/16",
 EnableDnsHostnames = true,
 EnableDnsSupport = true
 });

 var subnet = new Subnet("deepSeekSubnet", new SubnetArgs
 {
 VpcId = vpc.Id,
 CidrBlock = "10.0.48.0/20",
 AvailabilityZone = "eu-central-1a",
 MapPublicIpOnLaunch = true
 });

 var internetGateway = new InternetGateway("deepSeekInternetGateway", new InternetGatewayArgs
 {
 VpcId = vpc.Id
 });

 var routeTable = new RouteTable("deepSeekRouteTable", new RouteTableArgs

name: deepseek-ollama-yaml
description: DeepSeek Ollama AWS example
runtime: yaml

variables:
 publicKey:
 fn::readFile: ./deepseek.rsa
 userData:
 fn::readFile: ./cloud-init.yaml
 amiFilter: "amzn2-ami-hvm-*-x86_64-gp2"
 amiOwner: "137112412989"
 amiId:
 fn::invoke:
 function: aws:ec2:getAmi
 arguments:
 filters:
 - name: name
 values: ["${amiFilter}"]
 owners: ["${amiOwner}"]
 mostRecent: true
 return: id

resources:
 deepSeekRole:
 type: aws:iam:Role
 properties:
 name: deepseek-role
 assumeRolePolicy: |
 {
 "Version": "2012-10-17",
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": "ec2.amazonaws.com"
 }
 }
 ]
 }

 deepSeekS3Policy:
 type: aws:iam:RolePolicyAttachment
 properties:
 role: ${deepSeekRole.name}
 policyArn: arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

 deepSeekProfile:
 type: aws:iam:InstanceProfile
 properties:
 name: deepseek-profile
 role: ${deepSeekRole.name}

Step 2: Create the network

Next, create the VPC, subnet, internet gateway, and route table. The security group opens ports 22 (SSH), 3000 (Open WebUI), and 11434 (Ollama API):

const vpc = new aws.ec2.Vpc("deepSeekVpc", {
 cidrBlock: "10.0.0.0/16",
 enableDnsHostnames: true,
 enableDnsSupport: true,
});

const subnet = new aws.ec2.Subnet("deepSeekSubnet", {
 vpcId: vpc.id,
 cidrBlock: "10.0.48.0/20",
 availabilityZone: pulumi.interpolate`${aws.getAvailabilityZones().then(it => it.names[0])}`,
 mapPublicIpOnLaunch: true,
});

const internetGateway = new aws.ec2.InternetGateway("deepSeekInternetGateway", {
 vpcId: vpc.id,
});

const routeTable = new aws.ec2.RouteTable("deepSeekRouteTable", {
 vpcId: vpc.id,
 routes: [
 {
 cidrBlock: "0.0.0.0/0",
 gatewayId: internetGateway.id,
 },
 ],
});

const routeTableAssociation = new aws.ec2.RouteTableAssociation("deepSeekRouteTableAssociation", {
 subnetId: subnet.id,
 routeTableId: routeTable.id,
});

const securityGroup = new aws.ec2.SecurityGroup("deepSeekSecurityGroup", {
 vpcId: vpc.id,
 egress: [
 {
 fromPort: 0,
 toPort: 0,
 protocol: "-1",
 cidrBlocks: ["0.0.0.0/0"],
 },
 ],
 ingress: [
 {
 fromPort: 22,
 toPort: 22,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 {
 fromPort: 3000,
 toPort: 3000,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 {
 fromPort: 11434,
 toPort: 11434,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 ],
});


# Create a VPC
vpc = aws.ec2.Vpc(
 "deepSeekVpc",
 cidr_block="10.0.0.0/16",
 enable_dns_hostnames=True,
 enable_dns_support=True,
)

# Create a subnet
subnet = aws.ec2.Subnet(
 "deepSeekSubnet",
 vpc_id=vpc.id,
 cidr_block="10.0.48.0/20",
 availability_zone="eu-central-1a",
 map_public_ip_on_launch=True,
)

# Create an internet gateway
internet_gateway = aws.ec2.InternetGateway("deepSeekInternetGateway", vpc_id=vpc.id)

# Create a route table and route table association
route_table = aws.ec2.RouteTable(
 "deepSeekRouteTable",
 vpc_id=vpc.id,
 routes=[
 aws.ec2.RouteTableRouteArgs(
 cidr_block="0.0.0.0/0", gateway_id=internet_gateway.id
 )
 ],
)

route_table_association = aws.ec2.RouteTableAssociation(
 "deepSeekRouteTableAssociation", subnet_id=subnet.id, route_table_id=route_table.id
)

# Create a security group
security_group = aws.ec2.SecurityGroup(
 "deepSeekSecurityGroup",
 vpc_id=vpc.id,
 egress=[
 {
 "from_port": 0,
 "to_port": 0,
 "protocol": "-1",
 "cidr_blocks": ["0.0.0.0/0"],
 }
 ],
 ingress=[
 {
 "from_port": 22,
 "to_port": 22,
 "protocol": "tcp",
 "cidr_blocks": ["0.0.0.0/0"],
 },
 {
 "from_port": 3000,
 "to_port": 3000,
 "protocol": "tcp",
 "cidr_blocks": ["0.0.0.0/0"],
 },
 {
 "from_port": 11434,
 "to_port": 11434,
 "protocol": "tcp",
 "cidr_blocks": ["0.0.0.0/0"],
 },
 ],
)

package main

import (
 "encoding/json"
 "os"

 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/iam"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // omitted for brevity
 }

 // Create a VPC
 vpc, err := ec2.NewVpc(ctx, "deepSeekVpc", &ec2.VpcArgs{
 CidrBlock: pulumi.String("10.0.0.0/16"),
 EnableDnsHostnames: pulumi.Bool(true),
 EnableDnsSupport: pulumi.Bool(true),
 })
 if err != nil {
 return err
 }

 // Create a subnet
 subnet, err := ec2.NewSubnet(ctx, "deepSeekSubnet", &ec2.SubnetArgs{
 VpcId: vpc.ID(),
 CidrBlock: pulumi.String("10.0.48.0/20"),
 AvailabilityZone: pulumi.String("eu-central-1a"),
 MapPublicIpOnLaunch: pulumi.Bool(true),
 })
 if err != nil {
 return err
 }

 // Create an internet gateway
 internetGateway, err := ec2.NewInternetGateway(ctx, "deepSeekInternetGateway", &ec2.InternetGatewayArgs{
 VpcId: vpc.ID(),
 })
 if err != nil {
 return err
 }

 // Create a route table and route table association
 routeTable, err := ec2.NewRouteTable(ctx, "deepSeekRouteTable", &ec2.RouteTableArgs{
 VpcId: vpc.ID(),
 Routes: ec2.RouteTableRouteArray{
 &ec2.RouteTableRouteArgs{
 CidrBlock: pulumi.String("0.0.0.0/0"),
 GatewayId: internetGateway.ID(),
 },
 },
 })
 if err != nil {
 return err
 }

 _, err = ec2.NewRouteTableAssociation(ctx, "deepSeekRouteTableAssociation", &ec2.RouteTableAssociationArgs{
 SubnetId: subnet.ID(),
 RouteTableId: routeTable.ID(),
 })
 if err != nil {
 return err
 }

 // Create a security group
 securityGroup, err := ec2.NewSecurityGroup(ctx, "deepSeekSecurityGroup", &ec2.SecurityGroupArgs{
 VpcId: vpc.ID(),
 Egress: ec2.SecurityGroupEgressArray{
 &ec2.SecurityGroupEgressArgs{
 FromPort: pulumi.Int(0),
 ToPort: pulumi.Int(0),
 Protocol: pulumi.String("-1"),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 },
 Ingress: ec2.SecurityGroupIngressArray{
 &ec2.SecurityGroupIngressArgs{
 FromPort: pulumi.Int(22),
 ToPort: pulumi.Int(22),
 Protocol: pulumi.String("tcp"),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 &ec2.SecurityGroupIngressArgs{
 FromPort: pulumi.Int(3000),
 ToPort: pulumi.Int(3000),
 Protocol: pulumi.String("tcp"),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 &ec2.SecurityGroupIngressArgs{
 FromPort: pulumi.Int(11434),
 ToPort: pulumi.Int(11434),
 Protocol: pulumi.String("tcp"),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 },
 })
 if err != nil {
 return err
 }

 return nil
 })
}

using Pulumi;
using Pulumi.Aws.Ec2;
using Pulumi.Aws.Iam;
using System.Collections.Generic;
using System.IO;
using Pulumi.Aws.Ec2.Inputs;
using System.Text.Json;

return await Deployment.RunAsync(() =>
{
 var rolePolicy = JsonSerializer.Serialize(new Dictionary<string, object>
 {
 // omitted for brevity
 {
 VpcId = vpc.Id,
 Routes =
 {
 new RouteTableRouteArgs
 {
 CidrBlock = "0.0.0.0/0",
 GatewayId = internetGateway.Id
 }
 }
 });

 var routeTableAssociation = new RouteTableAssociation("deepSeekRouteTableAssociation", new RouteTableAssociationArgs
 {
 SubnetId = subnet.Id,
 RouteTableId = routeTable.Id
 });

 var securityGroup = new SecurityGroup("deepSeekSecurityGroup", new SecurityGroupArgs
 {
 VpcId = vpc.Id,
 Egress =
 {
 new SecurityGroupEgressArgs
 {
 FromPort = 0,
 ToPort = 0,
 Protocol = "-1",
 CidrBlocks = { "0.0.0.0/0" }
 }
 },
 Ingress =
 {
 new SecurityGroupIngressArgs
 {
 FromPort = 22,
 ToPort = 22,
 Protocol = "tcp",
 CidrBlocks = { "0.0.0.0/0" }
 },
 new SecurityGroupIngressArgs
 {
 FromPort = 3000,
 ToPort = 3000,
 Protocol = "tcp",
 CidrBlocks = { "0.0.0.0/0" }
 },
 new SecurityGroupIngressArgs
 {
 FromPort = 11434,
 ToPort = 11434,
 Protocol = "tcp",
 CidrBlocks = { "0.0.0.0/0" }
 }
 }
 });

 var publicKey = File.ReadAllText("deepseek.rsa");
 var keyPair = new KeyPair("deepSeekKey", new KeyPairArgs
 {
 PublicKey = publicKey
 });

 var amazonLinux = Pulumi.Aws.Ec2.GetAmi.Invoke(new()
 {
 MostRecent = true,
 Filters = new[]
 {
 new GetAmiFilterInputArgs
 {
 Name = "name",
 Values = new[] { "amzn2-ami-hvm-*-x86_64-gp2" }
 },
 new GetAmiFilterInputArgs
 {
 Name = "architecture",


 deepSeekVpc:
 type: aws:ec2:Vpc
 properties:
 cidrBlock: 10.0.0.0/16
 enableDnsHostnames: true
 enableDnsSupport: true

 deepSeekSubnet:
 type: aws:ec2:Subnet
 properties:
 vpcId: ${deepSeekVpc.id}
 cidrBlock: 10.0.48.0/20
 availabilityZone: eu-central-1a
 mapPublicIpOnLaunch: true

 deepSeekInternetGateway:
 type: aws:ec2:InternetGateway
 properties:
 vpcId: ${deepSeekVpc.id}

 deepSeekRouteTable:
 type: aws:ec2:RouteTable
 properties:
 vpcId: ${deepSeekVpc.id}
 routes:
 - cidrBlock: 0.0.0.0/0
 gatewayId: ${deepSeekInternetGateway.id}

 deepSeekRouteTableAssociation:
 type: aws:ec2:RouteTableAssociation
 properties:
 subnetId: ${deepSeekSubnet.id}
 routeTableId: ${deepSeekRouteTable.id}

 deepSeekSecurityGroup:
 type: aws:ec2:SecurityGroup
 properties:
 vpcId: ${deepSeekVpc.id}
 ingress:
 - fromPort: 22
 toPort: 22
 protocol: tcp
 cidrBlocks:
 - 0.0.0.0/0
 - fromPort: 3000
 toPort: 3000
 protocol: tcp
 cidrBlocks:
 - 0.0.0.0/0
 - fromPort: 11434
 toPort: 11434
 protocol: tcp
 cidrBlocks:
 - 0.0.0.0/0
 egress:
 - fromPort: 0
 toPort: 0
 protocol: -1
 cidrBlocks:
 - 0.0.0.0/0

Step 3: Launch the GPU EC2 instance

Now create the EC2 instance itself. The example uses Amazon Linux because the NVIDIA driver install path is well-trodden, plus an SSH key pair you generate locally.

The default instance type is g4dn.xlarge—the cheapest option that fits any 7B/8B model. Bump it up if you picked a larger model from the table above: g5.xlarge for 13B–14B, g5.2xlarge for 32B, g6e.2xlarge for 70B. AWS publishes full specs for the G4, G5, and G6 families.

Generate the key pair if you don’t already have one:

openssl genrsa -out deepseek.pem 2048
openssl rsa -in deepseek.pem -pubout > deepseek.pub
ssh-keygen -f mykey.pub -i -mPKCS8 > deepseek.pem

const keyPair = new aws.ec2.KeyPair("deepSeekKey", {
 publicKey: pulumi.output(fs.readFileSync("deepseek.rsa", "utf-8")),
});

const deepSeekAmi = aws.ec2
 .getAmi({
 filters: [
 {
 name: "name",
 values: ["amzn2-ami-hvm-2.0.*-x86_64-gp2"],
 },
 {
 name: "architecture",
 values: ["x86_64"],
 },
 ],
 owners: ["137112412989"], // Amazon
 mostRecent: true,
 })
 .then(ami => ami.id);

const deepSeekInstance = new aws.ec2.Instance("deepSeekInstance", {
 ami: deepSeekAmi,
 instanceType: "g4dn.xlarge",
 keyName: keyPair.keyName,
 rootBlockDevice: {
 volumeSize: 100,
 volumeType: "gp3",
 },
 subnetId: subnet.id,
 vpcSecurityGroupIds: [securityGroup.id],
 iamInstanceProfile: instanceProfile.name,
 userData: fs.readFileSync("cloud-init.yaml", "utf-8"),
 tags: {
 Name: "deepSeek-server",
 },
});

export const amiId = deepSeekAmi;
export const instanceId = deepSeekInstance.id;
export const instancePublicDns = deepSeekInstance.publicIp;


# Key pair for SSH access
public_key = open("deepseek.rsa", "r").read()
key_pair = aws.ec2.KeyPair("deepSeekKey", public_key=public_key)


# Get the latest Amazon Linux 2 AMI
ami = aws.ec2.get_ami(
 filters=[
 {"name": "name", "values": ["amzn2-ami-hvm-2.0.*-x86_64-gp2"]},
 {"name": "architecture", "values": ["x86_64"]},
 ],
 owners=["137112412989"], # Amazon
 most_recent=True,
).id

# Create an EC2 instance
user_data = open("cloud-init.yaml", "r").read()
instance = aws.ec2.Instance(
 "deepSeekInstance",
 ami=ami,
 instance_type="g4dn.xlarge",
 key_name=key_pair.key_name,
 root_block_device=aws.ec2.InstanceRootBlockDeviceArgs(
 volume_size=100, volume_type="gp3"
 ),
 subnet_id=subnet.id,
 vpc_security_group_ids=[security_group.id],
 iam_instance_profile=instance_profile.name,
 user_data=user_data,
 tags={"Name": "deepSeek-server"},
)

pulumi.export("amiId", ami)
pulumi.export("instanceId", instance.id)
pulumi.export("instancePublicDns", instance.public_ip)

package main

import (
 "encoding/json"
 "os"

 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/iam"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // omitted for brevity

 // Key pair for SSH access
 publicKey, err := os.ReadFile("deepseek.rsa")
 if err != nil {
 return err
 }

 keyPair, err := ec2.NewKeyPair(ctx, "deepSeekKey", &ec2.KeyPairArgs{
 PublicKey: pulumi.String(string(publicKey)),
 })
 if err != nil {
 return err
 }

 // Get the latest Amazon Linux 2 AMI
 mostRecent := true
 ami, err := ec2.LookupAmi(ctx, &ec2.LookupAmiArgs{
 Filters: []ec2.GetAmiFilter{
 {
 Name: "name",
 Values: []string{"amzn2-ami-hvm-2.0.*-x86_64-gp2"},
 },
 {
 Name: "architecture",
 Values: []string{"x86_64"},
 },
 },
 Owners: []string{"137112412989"},
 MostRecent: &mostRecent,
 })
 if err != nil {
 return err
 }

 // Create an EC2 instance
 userData, err := os.ReadFile("cloud-init.yaml")
 if err != nil {
 return err
 }

 instance, err := ec2.NewInstance(ctx, "deepSeekInstance", &ec2.InstanceArgs{
 Ami: pulumi.String(ami.Id),
 InstanceType: pulumi.String("g4dn.xlarge"),
 KeyName: keyPair.KeyName,
 RootBlockDevice: &ec2.InstanceRootBlockDeviceArgs{
 VolumeSize: pulumi.Int(100),
 VolumeType: pulumi.String("gp3"),
 },
 SubnetId: subnet.ID(),
 VpcSecurityGroupIds: pulumi.StringArray{securityGroup.ID()},
 IamInstanceProfile: instanceProfile.Name,
 UserData: pulumi.String(string(userData)),
 Tags: pulumi.StringMap{
 "Name": pulumi.String("deepSeek-server"),
 },
 })
 if err != nil {
 return err
 }

 ctx.Export("amiId", pulumi.String(ami.Id))
 ctx.Export("instanceId", instance.ID())
 ctx.Export("instancePublicDns", instance.PublicIp)

 return nil
 })
}

using Pulumi;
using Pulumi.Aws.Ec2;
using Pulumi.Aws.Iam;
using System.Collections.Generic;
using System.IO;
using Pulumi.Aws.Ec2.Inputs;
using System.Text.Json;

return await Deployment.RunAsync(() =>
{
 var rolePolicy = JsonSerializer.Serialize(new Dictionary<string, object>
 {
 // omitted for brevity
 Name = "architecture",
 Values = new[] { "x86_64" }
 }
 },
 Owners = new[] { "137112412989" }
 });

 var userData = File.ReadAllText("cloud-init.yaml");
 var instance = new Instance("deepSeekInstance", new InstanceArgs
 {
 Ami = amazonLinux.Apply(r => r.Id),
 InstanceType = "g4dn.xlarge",
 KeyName = keyPair.KeyName,
 RootBlockDevice = new InstanceRootBlockDeviceArgs
 {
 VolumeSize = 100,
 VolumeType = "gp3"
 },
 SubnetId = subnet.Id,
 VpcSecurityGroupIds = { securityGroup.Id },
 IamInstanceProfile = instanceProfile.Name,
 UserData = userData,
 Tags = { { "Name", "deepSeek-server" } }
 });

 return new Dictionary<string, object?>
 {
 ["amiId"] = amazonLinux.Apply(r => r.Id),
 ["instanceId"] = instance.Id,
 ["instancePublicDns"] = instance.PublicIp
 };
});


 deepSeekKey:
 type: aws:ec2:KeyPair
 properties:
 publicKey: ${publicKey}

 deepSeekInstance:
 type: aws:ec2:Instance
 properties:
 ami: ${amiId}
 instanceType: "g4dn.xlarge"
 keyName: ${deepSeekKey.keyName}
 rootBlockDevice:
 volumeSize: 100
 volumeType: gp3
 subnetId: ${deepSeekSubnet.id}
 vpcSecurityGroupIds:
 - ${deepSeekSecurityGroup.id}
 iamInstanceProfile: ${deepSeekProfile.name}
 userData: ${userData}
 tags:
 Name: deepSeek-server

outputs:
 AmiId: ${amiId}
 InstanceId: ${deepSeekInstance.id}
 InstancePublicDns: ${deepSeekInstance.publicIp}

Step 4: Install Ollama via cloud-init

The EC2 instance is a blank box until cloud-init runs. The user-data script below installs the NVIDIA GRID drivers, Docker, and the NVIDIA Container Toolkit, then starts the Ollama and Open WebUI containers. To switch models, edit the ollama run line—the rest is identical regardless of which model you want.

#cloud-config
users:
- default

package_update: true

packages:
- apt-transport-https
- ca-certificates
- curl
- openjdk-17-jre-headless
- gcc

runcmd:
- yum install -y gcc kernel-devel-$(uname -r)
- aws s3 cp --recursive s3://ec2-linux-nvidia-drivers/latest/ .
- chmod +x NVIDIA-Linux-x86_64*.run
- /bin/sh ./NVIDIA-Linux-x86_64*.run --tmpdir . --silent
- touch /etc/modprobe.d/nvidia.conf
- echo "options nvidia NVreg_EnableGpuFirmware=0" | sudo tee --append /etc/modprobe.d/nvidia.conf
- yum install -y docker
- usermod -a -G docker ec2-user
- systemctl enable docker.service
- systemctl start docker.service
- curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
- yum install -y nvidia-container-toolkit
- nvidia-ctk runtime configure --runtime=docker
- systemctl restart docker
- docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama --restart always ollama/ollama
- sleep 120
- docker exec ollama ollama run deepseek-r1:7b
- docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

You might also like:

Step 5: Deploy the infrastructure

Make sure your AWS credentials are configured:

aws configure

Pulumi supports several other authentication methods for the AWS provider. Once credentials are in place, deploy the infrastructure:

pulumi up

Pulumi previews the changes; type yes to confirm.

pulumi up
Please choose a stack, or create a new one: <create a new stack>
Please enter your desired stack name.
To create a stack in an organization, use the format <org-name>/<stack-name> (e.g. `acmecorp/dev`): dev
Please enter your desired stack name.
Created stack 'dev'
Previewing update (dev)
View in Browser (Ctrl+O): https://app.pulumi.com/dirien/deepseek-ollama-typescript/dev/previews/1dbb18ea-ba31-4d5b-9510-5dce19eb8ee8
Type Name Plan
+ pulumi:pulumi:Stack deepseek-ollama-typescript-dev create
+ ├─ aws:ec2:KeyPair deepSeekKey create
+ ├─ aws:ec2:Vpc deepSeekVpc create
+ ├─ aws:iam:Role deepSeekRole create
+ ├─ aws:iam:InstanceProfile deepSeekProfile create
+ ├─ aws:ec2:SecurityGroup deepSeekSecurityGroup create
+ ├─ aws:ec2:RouteTable deepSeekRouteTable create
+ ├─ aws:ec2:InternetGateway deepSeekInternetGateway create
+ ├─ aws:ec2:Subnet deepSeekSubnet create
+ ├─ aws:iam:RolePolicyAttachment deepSeekS3Policy create
+ ├─ aws:ec2:RouteTableAssociation deepSeekRouteTableAssociation create
+ └─ aws:ec2:Instance deepSeekInstance create
Outputs:
amiId : "ami-085131ff43045c877"
instanceId : output<string>
instancePublicDns: output<string>
Resources:
+ 12 to create
Do you want to perform this update? yes
Updating (dev)
View in Browser (Ctrl+O): https://app.pulumi.com/dirien/deepseek-ollama-typescript/dev/updates/1
Type Name Status
+ pulumi:pulumi:Stack deepseek-ollama-typescript-dev created (40s)
+ ├─ aws:ec2:KeyPair deepSeekKey created (0.47s)
+ ├─ aws:iam:Role deepSeekRole created (1s)
+ ├─ aws:ec2:Vpc deepSeekVpc created (12s)
+ ├─ aws:iam:InstanceProfile deepSeekProfile created (6s)
+ ├─ aws:iam:RolePolicyAttachment deepSeekS3Policy created (0.90s)
+ ├─ aws:ec2:InternetGateway deepSeekInternetGateway created (0.69s)
+ ├─ aws:ec2:Subnet deepSeekSubnet created (11s)
+ ├─ aws:ec2:SecurityGroup deepSeekSecurityGroup created (2s)
+ ├─ aws:ec2:RouteTable deepSeekRouteTable created (1s)
+ ├─ aws:ec2:RouteTableAssociation deepSeekRouteTableAssociation created (0.92s)
+ └─ aws:ec2:Instance deepSeekInstance created (12s)
Outputs:
amiId : "ami-085131ff43045c877"
instanceId : "i-0ae7495781ace3e81"
instancePublicDns: "18.159.211.136"
Resources:
+ 12 created
Duration: 42s

The infrastructure provisions in under a minute, but the cloud-init script needs another 5–10 minutes to install drivers, pull container images, and download the model weights. SSH in to watch the progress, or just wait and load the Web UI when it’s ready.

ssh -i deepseek.pem ec2-user@<instance-public-ip>

Check the container status:

sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c8714335e205 ghcr.io/open-webui/open-webui:main "bash start.sh" 6 minutes ago Up 6 minutes (healthy) 0.0.0.0:3000->8080/tcp, :::3000->8080/tcp open-webui
bf4bb3b7ede1 ollama/ollama "/bin/ollama serve" 8 minutes ago Up 7 minutes 0.0.0.0:11434->11434/tcp, :::11434->11434/tcp ollama
[ec2-user@ip-10-0-58-122 ~]$

Step 6: Access the Web UI or API

Once the containers are healthy, open http://<instance-public-ip>:3000 in your browser for Open WebUI:

Open WebUI is not secured by default. Restrict the security group to your IP, put it behind an authenticated proxy, or terminate TLS at an ALB before exposing it to the internet.

For programmatic access, Ollama exposes an OpenAI-compatible API on port 11434. Most clients written for the OpenAI SDK only need a base-URL change:

from openai import OpenAI

client = OpenAI(
 base_url="http://<instance-public-ip>:11434/v1",
 api_key="ollama", # required by the SDK, ignored by Ollama
)

response = client.chat.completions.create(
 model="llama3.1:8b",
 messages=[{"role": "user", "content": "Why is the sky blue?"}],
)
print(response.choices[0].message.content)

How do I switch models?

Ollama hosts every major open-weight family in its model library. Pulling a different model is two commands inside the EC2 instance—or a one-line edit to the cloud-init script if you want it provisioned automatically:

# DeepSeek-R1 distill (default in this guide)
docker exec ollama ollama run deepseek-r1:7b

# Llama 3.1 (Meta, 8B)
docker exec ollama ollama run llama3.1:8b

# Qwen 2.5 (Alibaba, 7B)
docker exec ollama ollama run qwen2.5:7b

# Mistral (7B)
docker exec ollama ollama run mistral:7b

# Larger reasoning model (needs g5.2xlarge or larger)
docker exec ollama ollama run deepseek-r1:32b

Tags follow a <size> or <size>-<quantization> pattern—8b, 8b-instruct-q4_K_M, 8b-instruct-q8_0. Q4 is the default and the right starting point; bump to Q8 only if you have spare VRAM and notice quality issues with Q4. Browse the full tag list for any model on its Ollama library page.

Cleaning up

When you’re done, tear everything down:

pulumi destroy

What are the next steps?

You now have a reproducible, IaC-managed deployment of any open-source LLM on AWS. The infrastructure is fixed; the model is a parameter. From here, the natural extensions are wiring this up to a real application, adding RAG over your own data, or moving the deployment behind an authenticated load balancer.

If you want to go further with AI on Pulumi, here are some related guides:

Deploy LangServe Apps with Pulumi on AWS (RAG & Chatbot) — Build a retrieval-augmented chatbot that could front-end this Ollama instance.
Deploy AI Models on Amazon SageMaker using Pulumi Python IaC — A SageMaker alternative when you’d rather not manage the EC2 host yourself.
Build an AI Slack Bot on AWS Using Embedchain & Pulumi — Wire an LLM into Slack as an internal assistant.
What is Infrastructure as Code? — Background on the IaC approach used throughout this guide.

Try Pulumi for Free

Changelog

2026-04-30 — Broadened scope from DeepSeek-only to any Ollama-supported model (Llama, Qwen, Mistral). Added TL;DR, instance-type recommendation table, cost-vs-hosted-API comparison, and HowTo structured data. Restructured headings as user questions. Verified Ollama and cloud-init commands against current versions.
2025-03-10 — Minor edits and corrections.
2025-01-27 — Original post: Run DeepSeek-R1 on AWS EC2 Using Ollama.

Master Kubernetes Secrets with Pulumi ESC + Secrets Store CSI Driver

Engin Diri — Wed, 18 Dec 2024 09:58:01 +0100

Welcome to the second blog post of the Pulumi ESC and Kubernetes secrets series. If you haven’t had the chance to read the first blog post, go ahead and read it here.

In the previous blog post, we have learned how to manage secrets with Pulumi ESC and the External Secrets Operator. While the External Secrets Operator is a great tool to manage secrets in a cloud-native way, it still creates Kubernetes secrets in the cluster. Depending on your security requirements, you might want to avoid the use of Kubernetes secrets in your cluster at all. This is the point where you hit the limits of the External Secrets Operator.

But don’t worry, we also have a solution for this problem. In this blog post, we will introduce the Secrets Store CSI Driver.

TL;DR?

Jump straight to the comparison of the External Secrets Operator and the Secrets Store CSI Driver here.

Refresher: Why Should You Avoid Using Kubernetes Secrets?

Before we dive into the Secrets Store CSI Driver, let’s quickly recap why you might want to avoid using native Kubernetes secrets. I wrote in depth about this topic in the previous blog post, but here is a quick summary:

By default, etcd stores them in a base64-encoded format, which is not secure.
Developers create them either manually by using kubectl commands or by using a manifest file, which makes them hard to manage at scale.
Hard to manage and synchronise secrets across different environments and clusters.
There is no default way to rotate secrets automatically.

Point 1 is the most critical one, as there is even a warning in the Kubernetes documentation:

Kubernetes Secrets. Credit: kubernetes.io/docs/concepts/configuration/secret/

What is the Secrets Store CSI Driver?

The first thing you may notice is that in the name of the Secret Store CSI Driver, there is a reference to the Container Storage Interface (CSI) standard.

What is CSI and why it is used in the Secret Store CSI Driver*?*

How to write a Container Storage Interface (CSI) plugin. Credit: arslan.io/2018/06/21/how-to-write-a-container-storage-interface-csi-plugin/

Container Storage Interface (CSI)

The Container Storage Interface (CSI) is a standard to unify the interface between container orchestrators (like Kubernetes) and different storage vendors (like NetApp, Ceph, etc.). This helps to guarantee that implementing a CSI for a storage vendor is going to work with all orchestrators that support CSI.

Before CSI, we needed to create volume plugins for every orchestrator and storage vendor. This meant that developers coupled volume plugin development with the Kubernetes version and made it dependent on it. Bugs in volume plugins could break the Kubernetes components, instead of the volume plugin. Besides, volume plugins had full privileges on the Kubernetes components like the kubelet.

Combining CSI, Kubernetes and Secrets = Secret Store CSI Driver

The Secret Store CSI Driver is a CSI driver that allows you to mount many secrets, certificates, and keys from external secret stores into Kubernetes pods as volumes. After attaching the volume, the system mounts the secrets into the container file system.

The benefits of using the Secret Store CSI Driver are that you manage the lifecycle of the secrets outside of Kubernetes while still providing a Kubernetes-native experience of using the secrets in your pods.

The Architecture of the Secret Store CSI Driver

The Secret Store CSI Driver is a daemonset that lets all kublets communicate with each other and uses gRPC to talk to a provider. A SecretProviderClass custom resource specifies the definition of the external Secret Store. Then the system mounts a volume in the pod as tmpfs, and it injects the secret into the volume. When you delete the pod, the system cleans up the volume and removes the secret from the tmpfs volume.

How to Use the Secrets Store CSI Driver with Pulumi ESC

Before we start, make sure you have the following prerequisites:

A Pulumi Cloud account. If you don’t have one, you can create one for free.
A Kubernetes cluster (I will be using a local KinD cluster, but you can use any Kubernetes cluster)
Pulumi CLI installed. You can use the pulumi env commands to manage secrets and configurations.
kubectl CLI installed for some debugging

Step 1: Deploy the Secrets Store CSI Driver and Pulumi ESC CSI Provider

We will use Pulumi with the pulumi-kubernetes provider to deploy the Secrets Store CSI Driver and the Pulumi ESC CSI Provider to our Kubernetes cluster.

And here we also use the integration of Pulumi ESC with Pulumi IaC to supply the Pulumi access token in a secure way.

First, we need to create a new Pulumi project:

# Choose your favorite Pulumi supported language
pulumi new kubernetes-<your-programming-language> --name

Before we dig into the code, let’s head to the Pulumi Cloud Console and create a new Pulumi ESC project with the name esc-secrets-store-csi-driver-demo.

And create the environment dev:

In the editor add the following yaml into the Environment definition:

values:
 pulumiConfig:
 pulumi-pat:
 fn::secret: <your-pulumi-pat>

Don’t know how to create a PAT? Check out the official Pulumi documentation.

If you prefer to use the Pulumi CLI, you can create the environment by running:

pulumi env init <your-org>/esc-secrets-store-csi-driver-demo/dev

And set the configuration by running the env edit command and copy the above YAML into the editor:

pulumi env edit <your-org>/esc-secrets-store-csi-driver-demo/dev

Now, we need to link the Pulumi ESC project to the Pulumi IaC project. To do this, we need to add the following to your Pulumi.dev.yaml:

environment:
- esc-secrets-store-csi-driver-demo/dev

import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

const secretsStoreCSIDriver = new k8s.helm.v4.Chart("secrets-store-csi-driver", {
 chart: "secrets-store-csi-driver",
 namespace: "kube-system",
 repositoryOpts: {
 repo: "https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts",
 },
 values: {
 nodeSelector: {
 "kubernetes.io/os": "linux",
 },
 },
});

const secretsStoreCSIPulumiESCProvider = new k8s.helm.v4.Chart(
 "secrets-store-csi-pulumi-esc-provider",
 {
 chart: "oci://ghcr.io/pulumi/helm-charts/pulumi-esc-csi-provider",
 namespace: "kube-system",
 values: {
 nodeSelector: {
 "kubernetes.io/os": "linux",
 },
 },
 },
 { dependsOn: secretsStoreCSIDriver },
);

const config = new pulumi.Config();

const mySecret = new k8s.core.v1.Secret(
 "my-secret",
 {
 metadata: {
 namespace: "default",
 name: "pulumi-access-token",
 },
 stringData: {
 "pulumi-access-token": config.require("pulumi-pat"),
 },
 type: "Opaque",
 },
 { dependsOn: secretsStoreCSIPulumiESCProvider },
);

import pulumi
import pulumi_kubernetes as k8s

secrets_store_csi_driver = k8s.helm.v4.Chart(
 "secrets-store-csi-driver",
 k8s.helm.v4.ChartArgs(
 chart="secrets-store-csi-driver",
 namespace="kube-system",
 repository_opts=k8s.helm.v4.RepositoryOptsArgs(
 repo="https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts",
 ),
 values={
 "nodeSelector": {
 "kubernetes.io/os": "linux",
 },
 },
 ),
)

secrets_store_csi_pulumi_esc_provider = k8s.helm.v4.Chart(
 "secrets-store-csi-pulumi-esc-provider",
 k8s.helm.v4.ChartArgs(
 chart="oci://ghcr.io/pulumi/helm-charts/pulumi-esc-csi-provider",
 namespace="kube-system",
 values={
 "nodeSelector": {
 "kubernetes.io/os": "linux",
 },
 },
 ),
 opts=pulumi.ResourceOptions(depends_on=[secrets_store_csi_driver]),
)

config = pulumi.Config()

my_secret = k8s.core.v1.Secret(
 "my-secret",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 namespace="default", name="pulumi-access-token"
 ),
 string_data={
 "pulumi-access-token": config.require("pulumi-pat"),
 },
 type="Opaque",
 opts=pulumi.ResourceOptions(depends_on=[secrets_store_csi_pulumi_esc_provider]),
)

package main

import (
 k8s "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes"
 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/apiextensions"
 appv1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/apps/v1"
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/core/v1"
 chartv4 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/helm/v4"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/meta/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi/config"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 secretsStoreCsiDriver, err := chartv4.NewChart(ctx, "secrets-store-csi-driver", &chartv4.ChartArgs{
 Chart: pulumi.String("secrets-store-csi-driver"),
 Namespace: pulumi.String("kube-system"),
 RepositoryOpts: chartv4.RepositoryOptsArgs{
 Repo: pulumi.String("https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts"),
 },
 Values: pulumi.Map{
 "nodeSelector": pulumi.Map{
 "kubernetes.io/os": pulumi.String("linux"),
 },
 },
 })
 if err != nil {
 return err
 }

 secretsStoreCsiPulumiEscProvider, err := chartv4.NewChart(ctx, "secrets-store-csi-pulumi-esc-provider", &chartv4.ChartArgs{
 Chart: pulumi.String("oci://ghcr.io/pulumi/helm-charts/pulumi-esc-csi-provider"),
 Namespace: pulumi.String("kube-system"),
 Values: pulumi.Map{
 "nodeSelector": pulumi.Map{
 "kubernetes.io/os": pulumi.String("linux"),
 },
 },
 }, pulumi.DependsOn([]pulumi.Resource{secretsStoreCsiDriver}))
 if err != nil {
 return err
 }

 pulumiPAT := config.Get(ctx, "pulumi-pat")

 mySecret, err := corev1.NewSecret(ctx, "my-secret", &corev1.SecretArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Namespace: pulumi.String("default"),
 Name: pulumi.String("pulumi-access-token"),
 },
 StringData: pulumi.StringMap{
 "pulumi-access-token": pulumi.String(pulumiPAT),
 },
 Type: pulumi.String("Opaque"),
 }, pulumi.DependsOn([]pulumi.Resource{secretsStoreCsiPulumiEscProvider}))
 if err != nil {
 return err
 }


 return nil
 })
}

using Pulumi;
using Pulumi.Kubernetes.Core.V1;
using Pulumi.Kubernetes.Types.Inputs.Core.V1;
using Pulumi.Kubernetes.Helm.V3;
using Pulumi.Kubernetes.Helm;
using Pulumi.Kubernetes.Types.Inputs.Meta.V1;
using Pulumi.Kubernetes.ApiExtensions;
using System.Collections.Generic;



return await Deployment.RunAsync(() =>
{
 var secretsStoreCsiDriver = new Release("secrets-store-csi-driver", new()
 {
 Chart = "secrets-store-csi-driver",
 Namespace = "kube-system",
 RepositoryOpts = new Pulumi.Kubernetes.Types.Inputs.Helm.V3.RepositoryOptsArgs
 {
 Repo = "https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts",
 },
 Values = new Dictionary<string, object>
 {
 { "nodeSelector", new Dictionary<string, object>
 {
 { "kubernetes.io/os", "linux" },
 } },
 },
 });

 var secretsStoreCsiPulumiEscProvider = new Release("secrets-store-csi-pulumi-esc-provider", new()
 {
 Chart = "oci://ghcr.io/pulumi/helm-charts/pulumi-esc-csi-provider",
 Namespace = "kube-system",
 Values = new Dictionary<string, object>
 {
 { "nodeSelector", new Dictionary<string, object>
 {
 { "kubernetes.io/os", "linux" },
 } },
 },
 }, new CustomResourceOptions { DependsOn = { secretsStoreCsiDriver } });

 var config = new Config();
 var pulumiPAT = config.Require("pulumi-pat");

 var mySecret = new Secret("my-secret", new SecretArgs
 {
 Metadata = new ObjectMetaArgs
 {
 Namespace = "default",
 Name = "pulumi-access-token",
 },
 StringData =
 {
 { "pulumi-access-token", pulumiPAT }
 },
 Type = "Opaque",
 }, new CustomResourceOptions { DependsOn = { secretsStoreCsiPulumiEscProvider } });

});


class SecretProviderClassArgs : CustomResourceArgs
{
 public SecretProviderClassArgs(): base("secrets-store.csi.x-k8s.io/v1", "SecretProviderClass")
 {
 }

 [Input("spec")]
 public Input<SecretProviderClassSpecArgs>? Spec { get; set; }
}


class SecretProviderClassSpecArgs : ResourceArgs
{
 [Input("provider")]
 public Input<string>? Provider { get; set; }

 [Input("parameters")]
 public Input<InputMap<object>>? Parameters { get; set; }
}

class SecretProviderParametersArgs : ResourceArgs
{
 [Input("apiUrl")]
 public Input<string>? ApiUrl { get; set; }

 [Input("organization")]
 public Input<string>? Organization { get; set; }

 [Input("project")]
 public Input<string>? Project { get; set; }

 [Input("environment")]
 public Input<string>? Environment { get; set; }

 [Input("authSecretName")]
 public Input<string>? AuthSecretName { get; set; }

 [Input("authSecretNamespace")]
 public Input<string>? AuthSecretNamespace { get; set; }

 [Input("secrets")]
 public Input<string>? Secrets { get; set; }
}

Deploy the stack by running:

pulumi up

And you should see that the secret was created in the Kubernetes cluster and the ESO instance was deployed successfully.

kubectl get secret pulumi-access-token -o jsonpath='{.data.PULUMI_ACCESS_TOKEN}' | base64 -d

Step 2: Create a SecretProviderClass

Now, we will create a secret in the Pulumi ESC project and synchronize it into the Kubernetes cluster by creating an SecretProviderClass:

Create a new ESC environment called csi-secrets-store-app in the esc-secrets-store-csi-driver-demo project:

values:
 app:
 hello: world
 hello-secret:
 fn::secret: world

If you prefer to use the Pulumi CLI, you can create the environment by running:

pulumi env init <your-org>/esc-secrets-store-csi-driver-demo/csi-secrets-store-app

And set the configuration by running the env edit command and copy the above YAML into the editor:

pulumi env edit <your-org>/esc-secrets-store-csi-driver-demo/csi-secrets-store-app

Either way, you should see following environment configuration in the Pulumi Cloud Console:

We can now create the SecretProviderClass in the Kubernetes cluster:

const secretProviderClass = new k8s.apiextensions.CustomResource(
 "example-provider-pulumi-esc",
 {
 apiVersion: "secrets-store.csi.x-k8s.io/v1",
 kind: "SecretProviderClass",
 metadata: {
 name: "example-provider-pulumi-esc",
 namespace: "default",
 },
 spec: {
 provider: "pulumi",
 parameters: {
 apiUrl: "https://api.pulumi.com/api/esc",
 organization: "dirien",
 project: "esc-secrets-store-csi-driver-demo",
 environment: "csi-secrets-store-app",
 authSecretName: mySecret.metadata.name,
 authSecretNamespace: mySecret.metadata.namespace,
 secrets: `- secretPath: "/"
 fileName: "hello"
 secretKey: "app.hello"
`,
 },
 },
 },
 { dependsOn: secretsStoreCSIPulumiESCProvider },
);

secret_provider_class = k8s.apiextensions.CustomResource(
 "example-provider-pulumi-esc",
 api_version="secrets-store.csi.x-k8s.io/v1",
 kind="SecretProviderClass",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name="example-provider-pulumi-esc", namespace="default"
 ),
 spec={
 "provider": "pulumi",
 "parameters": {
 "apiUrl": "https://api.pulumi.com/api/esc",
 "organization": "dirien",
 "project": "esc-secrets-store-csi-driver-demo",
 "environment": "csi-secrets-store-app",
 "authSecretName": my_secret.metadata["name"],
 "authSecretNamespace": my_secret.metadata["namespace"],
 "secrets": '- secretPath: "/"\n fileName: "hello"\n secretKey: "app.hello"\n',
 },
 },
 opts=pulumi.ResourceOptions(depends_on=[secrets_store_csi_pulumi_esc_provider]),
)

 secretProviderClass, err := apiextensions.NewCustomResource(ctx, "example-provider-pulumi-esc", &apiextensions.CustomResourceArgs{
 ApiVersion: pulumi.String("secrets-store.csi.x-k8s.io/v1"),
 Kind: pulumi.String("SecretProviderClass"),
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("example-provider-pulumi-esc"),
 Namespace: pulumi.String("default"),
 },
 OtherFields: k8s.UntypedArgs{
 "provider": pulumi.String("pulumi"),
 "parameters": pulumi.Map{
 "apiUrl": pulumi.String("https://api.pulumi.com/api/esc"),
 "organization": pulumi.String("dirien"),
 "project": pulumi.String("esc-secrets-store-csi-driver-demo"),
 "environment": pulumi.String("csi-secrets-store-app"),
 "authSecretName": mySecret.Metadata.Name().Elem(),
 "authSecretNamespace": mySecret.Metadata.Namespace().Elem(),
 "secrets": pulumi.String(`- secretPath: "/"
 fileName: "hello"
 secretKey: "app.hello"
`),
 },
 },
 }, pulumi.DependsOn([]pulumi.Resource{secretsStoreCsiPulumiEscProvider}))
 if err != nil {
 return err
 }

 var secretProviderClass = new Pulumi.Kubernetes.ApiExtensions.CustomResource("example-provider-pulumi-esc", new SecretProviderClassArgs
 {
 Metadata = new ObjectMetaArgs
 {
 Name = "example-provider-pulumi-esc",
 Namespace = "default",
 },
 Spec = new SecretProviderClassSpecArgs
 {
 Provider = "pulumi",
 Parameters = new InputMap<object>
 {
 { "apiUrl", "https://api.pulumi.com/api/esc" },
 { "organization", "dirien" },
 { "project", "esc-secrets-store-csi-driver-demo" },
 { "environment", "csi-secrets-store-app" },
 { "authSecretName", mySecret.Metadata.Apply(metadata => metadata.Name) },
 { "authSecretNamespace", mySecret.Metadata.Apply(metadata => metadata.Namespace) },
 { "secrets", "- secretPath: \"/\"\n fileName: \"hello\"\n secretKey: \"app.hello\"\n" }
 },
 },
 }, new CustomResourceOptions { DependsOn = { secretsStoreCsiPulumiEscProvider } });

We can check that the secret was successfully synchronized by running:

kubectl get secretproviderclasses example-provider-pulumi-esc

You might also like:

Step 3: Deploy an Application and Mount the Secret

Now, we can deploy an application that references the secret from the Kubernetes cluster. I am going to use busybox that reads mounted file in.

const deployment = new k8s.apps.v1.Deployment("example-provider-pulumi-esc", {
 metadata: {
 name: "example-provider-pulumi-esc",
 namespace: "default",
 labels: {
 app: "example-provider-pulumi-esc",
 },
 },
 spec: {
 replicas: 1,
 selector: {
 matchLabels: {
 app: "example-provider-pulumi-esc",
 },
 },
 template: {
 metadata: {
 labels: {
 app: "example-provider-pulumi-esc",
 },
 },
 spec: {
 containers: [
 {
 name: "client",
 image: "busybox:latest",
 command: ["sh", "-c"],
 args: [
 `set -eux
 ls /run/secrets
 find /run/secrets/ -mindepth 1 -maxdepth 1 -not -name '.*' | xargs -t -I {} sh -c 'echo "$(cat "{}")"'
 tail -f /dev/null`,
 ],
 volumeMounts: [
 {
 name: "data",
 mountPath: "/run/secrets",
 },
 ],
 },
 ],
 volumes: [
 {
 name: "data",
 csi: {
 driver: "secrets-store.csi.k8s.io",
 readOnly: true,
 volumeAttributes: {
 secretProviderClass: secretProviderClass.metadata.name,
 },
 },
 },
 ],
 },
 },
 },
});

export const deploymentName = deployment.metadata.name;

deployment = k8s.apps.v1.Deployment(
 "example-provider-pulumi-esc",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 namespace="default",
 name="example-provider-pulumi-esc",
 labels={
 "app": "example-provider-pulumi-esc",
 },
 ),
 spec=k8s.apps.v1.DeploymentSpecArgs(
 replicas=1,
 selector=k8s.meta.v1.LabelSelectorArgs(
 match_labels={
 "app": "example-provider-pulumi-esc",
 },
 ),
 template=k8s.core.v1.PodTemplateSpecArgs(
 metadata=k8s.meta.v1.ObjectMetaArgs(
 labels={
 "app": "example-provider-pulumi-esc",
 },
 ),
 spec=k8s.core.v1.PodSpecArgs(
 containers=[
 k8s.core.v1.ContainerArgs(
 name="client",
 image="busybox:latest",
 command=["sh", "-c"],
 args=[
 "set -eux\nls /run/secrets\nfind /run/secrets/ -mindepth 1 -maxdepth 1 -not -name '.*' | xargs -t -I {} sh -c 'echo \"$(cat \"{}\")\"'\ntail -f /dev/null",
 ],
 volume_mounts=[
 k8s.core.v1.VolumeMountArgs(
 name="data",
 mount_path="/run/secrets",
 ),
 ],
 ),
 ],
 volumes=[
 k8s.core.v1.VolumeArgs(
 name="data",
 csi=k8s.core.v1.CSIVolumeSourceArgs(
 driver="secrets-store.csi.k8s.io",
 read_only=True,
 volume_attributes={
 "secretProviderClass": secret_provider_class.metadata[
 "name"
 ],
 },
 ),
 ),
 ],
 ),
 ),
 ),
)

pulumi.export("deploymentName", deployment.metadata["name"])

 deployment, err := appv1.NewDeployment(ctx, "example-provider-pulumi-esc", &appv1.DeploymentArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("example-provider-pulumi-esc"),
 Namespace: pulumi.String("default"),
 Labels: pulumi.StringMap{
 "app": pulumi.String("example-provider-pulumi-esc"),
 },
 },
 Spec: &appv1.DeploymentSpecArgs{
 Replicas: pulumi.Int(1),
 Selector: &metav1.LabelSelectorArgs{
 MatchLabels: pulumi.StringMap{
 "app": pulumi.String("example-provider-pulumi-esc"),
 },
 },
 Template: &corev1.PodTemplateSpecArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Labels: pulumi.StringMap{
 "app": pulumi.String("example-provider-pulumi-esc"),
 },
 },
 Spec: &corev1.PodSpecArgs{
 Containers: corev1.ContainerArray{
 &corev1.ContainerArgs{
 Name: pulumi.String("client"),
 Image: pulumi.String("busybox:latest"),
 Command: pulumi.StringArray{
 pulumi.String("sh"),
 pulumi.String("-c"),
 },
 Args: pulumi.StringArray{
 pulumi.String(`set -eux
 ls /run/secrets
 find /run/secrets/ -mindepth 1 -maxdepth 1 -not -name '.*' | xargs -t -I {} sh -c 'echo "$(cat "{}")"'
 tail -f /dev/null`),
 },
 VolumeMounts: &corev1.VolumeMountArray{
 &corev1.VolumeMountArgs{
 Name: pulumi.String("data"),
 MountPath: pulumi.String("/run/secrets"),
 },
 },
 },
 },
 Volumes: &corev1.VolumeArray{
 &corev1.VolumeArgs{
 Name: pulumi.String("data"),
 Csi: corev1.CSIVolumeSourceArgs{
 Driver: pulumi.String("secrets-store.csi.k8s.io"),
 ReadOnly: pulumi.Bool(true),
 VolumeAttributes: pulumi.StringMap{
 "secretProviderClass": secretProviderClass.Metadata.Name().Elem(),
 },
 },
 },
 },
 },
 },
 },
 }, pulumi.DependsOn([]pulumi.Resource{secretProviderClass}))
 if err != nil {
 return err
 }

 var deployment = new Pulumi.Kubernetes.Apps.V1.Deployment("example-provider-pulumi-esc", new Pulumi.Kubernetes.Types.Inputs.Apps.V1.DeploymentArgs
 {
 Metadata = new ObjectMetaArgs
 {
 Name = "example-provider-pulumi-esc",
 Namespace = "default",
 Labels =
 {
 { "app", "example-provider-pulumi-esc" },
 },
 },
 Spec = new Pulumi.Kubernetes.Types.Inputs.Apps.V1.DeploymentSpecArgs
 {
 Replicas = 1,
 Selector = new Pulumi.Kubernetes.Types.Inputs.Meta.V1.LabelSelectorArgs
 {
 MatchLabels =
 {
 { "app", "example-provider-pulumi-esc" },
 },
 },
 Template = new Pulumi.Kubernetes.Types.Inputs.Core.V1.PodTemplateSpecArgs
 {
 Metadata = new ObjectMetaArgs
 {
 Labels =
 {
 { "app", "example-provider-pulumi-esc" },
 },
 },
 Spec = new Pulumi.Kubernetes.Types.Inputs.Core.V1.PodSpecArgs
 {
 Containers =
 {
 new Pulumi.Kubernetes.Types.Inputs.Core.V1.ContainerArgs
 {
 Name = "client",
 Image = "busybox:latest",
 Command =
 {
 "sh",
 "-c",
 },
 Args =
 {
 "set -eux\nls /run/secrets\nfind /run/secrets/ -mindepth 1 -maxdepth 1 -not -name '.*' | xargs -t -I {} sh -c 'echo \"$(cat \"{}\")\"'\ntail -f /dev/null",
 },
 VolumeMounts =
 {
 new Pulumi.Kubernetes.Types.Inputs.Core.V1.VolumeMountArgs
 {
 Name = "data",
 MountPath = "/run/secrets",
 },
 },
 },
 },
 Volumes =
 {
 new Pulumi.Kubernetes.Types.Inputs.Core.V1.VolumeArgs
 {
 Name = "data",
 Csi = new Pulumi.Kubernetes.Types.Inputs.Core.V1.CSIVolumeSourceArgs
 {
 Driver = "secrets-store.csi.k8s.io",
 ReadOnly = true,
 VolumeAttributes =
 {
 { "secretProviderClass", secretProviderClass.Metadata.Apply(metadata => metadata.Name) },
 },
 },
 },
 },
 },
 },
 },
 }, new CustomResourceOptions { DependsOn = { secretProviderClass } });

After deploying the stack, you can get the logs of the busybox pod to see that the secret was successfully mounted:

NAME=$(kubectl get pods -o name | grep example-provider-pulumi-esc | cut -d'/' -f2)
kubectl logs $NAME

You should see the following output:

+ ls /run/secrets
hello
+ find /run/secrets/ -mindepth 1 -maxdepth 1 -not -name '.*'
+ xargs -t -I '{}' sh -c 'echo "$(cat "{}")"'
sh -c echo "$(cat "/run/secrets/hello")"
world
+ tail -f /dev/null

Or you can exec into the pod and check the content of the mounted secret:

kubectl exec -it $NAME -- cat /run/secrets/hello

You should see the following output:

world

Step 4: Clean Up

After you are done with the demo, you can clean up the resources by running:

pulumi destroy

Conclusion

The Secret Store CSI Driver is a great option for managing secrets in a cloud-native way when the use of Kubernetes secrets is not an option due to enhanced security requirements.

Below is a quick comparison of the External Secrets Operator and the Secrets Store CSI Driver to help you decide which one is the best fit for your use case. As always, the best way to find out is to try both solutions and see which one fits your requirements best.

The good part is, whatever you choose, you can use Pulumi ESC to manage your secrets and avoid any secret sprawl in your organization as both solutions are supported by Pulumi ESC.

Comparison of the External Secrets Operator and the Secrets Store CSI Driver

Feature/Aspect	External Secrets Operator	CSI Secrets Driver
Primary Use Case	Synchronizing external secrets into Kubernetes as native secrets.	Mounting secrets directly to pods as files or environment variables.
Integration with External Secret Stores	Supports multiple external secret stores like AWS Secrets Manager, Google Secret Manager, Vault, Azure Key Vault, etc.	Primarily supports integrations defined by the CSI specification (e.g., Vault, AWS, Azure Key Vault, Pulumi ESC).
Mechanism for Secrets Delivery	Secrets are pulled and stored as Kubernetes `Secret` resources.	Secrets are mounted directly into pods as files or injected as environment variables.
Kubernetes Resource Requirements	Requires a custom Kubernetes resource (`ExternalSecret`) and the External Secrets Operator.	Relies on the Kubernetes CSI (Container Storage Interface) standard and requires a specific CSI driver.
Secret Rotation	Supports automatic rotation by polling the external secret store at defined intervals.	Supports rotation, but the application needs to handle reloading mounted secrets.
Security Considerations	Secrets are stored temporarily in Kubernetes as native secrets, which may pose a risk if not properly managed.	Secrets are mounted directly and not stored as native Kubernetes secrets, reducing exposure but relying on file permissions.
Customization and Features	Highly customizable with fine-grained mappings and transformation options.	Limited customization, focused on mounting secrets as-is.
Application Compatibility	Applications access secrets as Kubernetes secrets (e.g., via env variables or volumes).	Applications access secrets directly via mounted files or injected env variables.
Performance Considerations	Potential slight delay due to syncing secrets to Kubernetes secrets.	Faster access as secrets are directly mounted.
Community and Ecosystem	Supported by a growing community; integrates well with diverse secret management solutions.	Part of the Kubernetes ecosystem; supports CSI-compliant secret storage solutions.
Best Use Cases	Ideal for scenarios where secrets need to be accessed as Kubernetes native secrets.	Best for applications requiring direct access to secrets without intermediate storage in Kubernetes secrets.

You can find detailed information about all current available Pulumi ESC Kubernetes integrations in the Pulumi ESC documentation.

Let us know what you think about the Secret Store CSI Driver and how you are managing secrets in your Kubernetes clusters. We would love to hear your feedback and experiences.

Try Pulumi for Free

Getting Started with Amazon EKS Auto Mode in Pulumi

Florian Stadler — Mon, 16 Dec 2024 23:26:21 +0100

AWS recently announced Amazon EKS Auto Mode, a significant enhancement to Amazon EKS that streamlines cluster management by automating infrastructure decisions and operations. Today, we are excited to announce that Pulumi fully supports EKS Auto Mode across our AWS provider ecosystem, enabling you to leverage this powerful feature through infrastructure as code.

What is EKS Auto Mode?

EKS Auto Mode extends AWS management beyond the Kubernetes control plane to the underlying infrastructure that powers your workloads. While traditional EKS requires you to make numerous infrastructure decisions and maintain various components, Auto Mode simplifies this by automating these choices based on AWS’s operational expertise.

Instead of manually configuring and managing cluster add-ons, node groups, and infrastructure components, Auto Mode handles:

Compute Management: Automatically provisions and scales EC2 instances based on your workload demands while selecting optimal instance types and sizes. It also handles node lifecycle management, including OS updates and security patches.
Storage Automation: Eliminates the need to manually set up and configure the EBS CSI driver for block storage. Auto Mode manages volume provisioning, attachment, and cleanup for your persistent workloads.
Networking Simplification: Automatically configures and manages core networking components like Load Balancers.

This significantly reduces the operational overhead of running Kubernetes clusters. Instead of spending time on infrastructure decisions and maintenance, teams can now focus on their applications. For example, deploying a stateful application with load balancing no longer requires setting up and maintaining multiple add-ons and configurations - Auto Mode handles this automatically while following AWS best practices.

Common tasks that previously required significant expertise and time are now managed by AWS, making it simpler to run production-grade Kubernetes workloads. This includes:

Setting up and maintaining cluster autoscaling
Configuring and updating networking plugins
Managing storage drivers and volume provisioning
Handling node updates and security patches
Optimizing instance selection and placement

Pulumi Support for EKS Auto Mode

We have added EKS Auto Mode support across our AWS provider ecosystem:

Pulumi AWS Provider (v6.63.0+)
Pulumi AWS Cloud Control Provider (previously known as Pulumi AWS Native) (v1.13.0+)
Pulumi EKS Provider (v3.5.0+)

Creating an EKS Auto Mode Cluster

Let’s explore how to leverage EKS Auto Mode with Pulumi by deploying an example. You can enable EKS Auto Mode by setting autoMode.enabled to true.

First, we’ll create a new EKS cluster with Auto Mode enabled. This requires properly configured networking and authentication settings:

VPC subnets are tagged appropriately for EKS Auto Mode load balancer discovery
Authentication Mode supports Access Entries (Api or ApiAndConfigMap mode) as required by EKS Auto Mode
Default node groups and security groups are skipped as Auto Mode manages these

import * as pulumi from "@pulumi/pulumi";
import * as awsx from "@pulumi/awsx";
import * as eks from "@pulumi/eks";
import * as k8s from "@pulumi/kubernetes";
import { SubnetType } from "@pulumi/awsx/ec2";

const config = new pulumi.Config();
const clusterName = config.require("clusterName");

const eksVpc = new awsx.ec2.Vpc("eks-auto-mode", {
 enableDnsHostnames: true,
 cidrBlock: "10.0.0.0/16",
 subnetSpecs: [
 // Necessary tags for EKS Auto Mode to identify the subnets for the load balancers.
 // See: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/deploy/subnet_discovery/
 {type: SubnetType.Public, tags: {[`kubernetes.io/cluster/${clusterName}`]: "shared", "kubernetes.io/role/elb": "1"}},
 {type: SubnetType.Private, tags: {[`kubernetes.io/cluster/${clusterName}`]: "shared", "kubernetes.io/role/internal-elb": "1"}},
 ],
 subnetStrategy: "Auto"
});

const cluster = new eks.Cluster("eks-auto-mode", {
 name: clusterName,
 // EKS Auto Mode requires Access Entries, use either the `Api` or `ApiAndConfigMap` authentication mode.
 authenticationMode: eks.AuthenticationMode.Api,
 vpcId: eksVpc.vpcId,
 publicSubnetIds: eksVpc.publicSubnetIds,
 privateSubnetIds: eksVpc.privateSubnetIds,
 // Enables compute, storage and load balancing for the cluster.
 autoMode: {
 enabled: true,
 }
});

import pulumi
import pulumi_awsx as awsx
import pulumi_eks as eks
import pulumi_kubernetes as kubernetes

eks_vpc = awsx.ec2.Vpc("eksVpc",
 enable_dns_hostnames=True,
 cidr_block="10.0.0.0/16",
 # Necessary tags for EKS Auto Mode to identify the subnets for the load balancers.
 # See: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/deploy/subnet_discovery/
 subnet_specs=[
 {
 "type": awsx.ec2.SubnetType.PUBLIC,
 "tags": {
 "kubernetes_io_cluster_eks_auto_mode_demo": "shared",
 "kubernetes_io_role_elb": "1",
 },
 },
 {
 "type": awsx.ec2.SubnetType.PRIVATE,
 "tags": {
 "kubernetes_io_cluster_eks_auto_mode_demo": "shared",
 "kubernetes_io_role_internal_elb": "1",
 },
 },
 ],
 subnet_strategy=awsx.ec2.SubnetAllocationStrategy.AUTO)
cluster = eks.Cluster("cluster",
 name="eks-auto-mode-demo",
 # EKS Auto Mode requires Access Entries, use either the `Api` or `ApiAndConfigMap` authentication mode.
 authentication_mode=eks.AuthenticationMode.API,
 vpc_id=eks_vpc.vpc_id,
 public_subnet_ids=eks_vpc.public_subnet_ids,
 private_subnet_ids=eks_vpc.private_subnet_ids,
 # Enables compute, storage and load balancing for the cluster.
 auto_mode={
 "enabled": True,
 })

package main

import (
 "github.com/pulumi/pulumi-awsx/sdk/v2/go/awsx/ec2"
 "github.com/pulumi/pulumi-eks/sdk/v3/go/eks"
 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes"
 appsv1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/apps/v1"
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/core/v1"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/meta/v1"
 networkingv1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/networking/v1"
 storagev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/storage/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 eksVpc, err := ec2.NewVpc(ctx, "eksVpc", &ec2.VpcArgs{
 EnableDnsHostnames: pulumi.Bool(true),
 CidrBlock: "10.0.0.0/16",
 // Necessary tags for EKS Auto Mode to identify the subnets for the load balancers.
 // See: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/deploy/subnet_discovery/
 SubnetSpecs: []ec2.SubnetSpecArgs{
 {
 Type: ec2.SubnetTypePublic,
 Tags: {
 "kubernetes.io/cluster/eks-auto-mode-demo": pulumi.String("shared"),
 "kubernetes.io/role/elb": pulumi.String("1"),
 },
 },
 {
 Type: ec2.SubnetTypePrivate,
 Tags: {
 "kubernetes.io/cluster/eks-auto-mode-demo": pulumi.String("shared"),
 "kubernetes.io/role/internal-elb": pulumi.String("1"),
 },
 },
 },
 SubnetStrategy: ec2.SubnetAllocationStrategyAuto,
 })
 if err != nil {
 return err
 }
 cluster, err := eks.NewCluster(ctx, "cluster", &eks.ClusterArgs{
 Name: pulumi.String("eks-auto-mode-demo"),
 // EKS Auto Mode requires Access Entries, use either the `Api` or `ApiAndConfigMap` authentication mode.
 AuthenticationMode: eks.AuthenticationModeApi,
 VpcId: eksVpc.VpcId,
 PublicSubnetIds: eksVpc.PublicSubnetIds,
 PrivateSubnetIds: eksVpc.PrivateSubnetIds,
 // Enables compute, storage and load balancing for the cluster.
 AutoMode: &eks.AutoModeOptionsArgs{
 Enabled: true,
 },
 })
 if err != nil {
 return err
 }
 return nil
 })
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Awsx = Pulumi.Awsx;
using Eks = Pulumi.Eks;
using Kubernetes = Pulumi.Kubernetes;

return await Deployment.RunAsync(() =>
{
 var eksVpc = new Awsx.Ec2.Vpc("eksVpc", new()
 {
 EnableDnsHostnames = true,
 CidrBlock = "10.0.0.0/16",
 // Necessary tags for EKS Auto Mode to identify the subnets for the load balancers.
 // See: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/deploy/subnet_discovery/
 SubnetSpecs = new()
 {
 new Awsx.Ec2.Inputs.SubnetSpecArgs
 {
 Type = Awsx.Ec2.SubnetType.Public,
 Tags =
 {
 { "kubernetes.io/cluster/eks-auto-mode-demo", "shared" },
 { "kubernetes.io/role/elb", "1" },
 },
 },
 new Awsx.Ec2.Inputs.SubnetSpecArgs
 {
 Type = Awsx.Ec2.SubnetType.Private,
 Tags =
 {
 { "kubernetes.io/cluster/eks-auto-mode-demo", "shared" },
 { "kubernetes.io/role/internal-elb", "1" },
 },
 },
 },
 SubnetStrategy = Awsx.Ec2.SubnetAllocationStrategy.Auto,
 });

 var cluster = new Eks.Cluster("cluster", new()
 {
 Name = "eks-auto-mode-demo",
 // EKS Auto Mode requires Access Entries, use either the `Api` or `ApiAndConfigMap` authentication mode.
 AuthenticationMode = Eks.AuthenticationMode.Api,
 VpcId = eksVpc.VpcId,
 PublicSubnetIds = eksVpc.PublicSubnetIds,
 PrivateSubnetIds = eksVpc.PrivateSubnetIds,
 // Enables compute, storage and load balancing for the cluster.
 AutoMode = new Eks.Inputs.AutoModeOptionsArgs
 {
 Enabled = true,
 },
 });
});

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.awsx.ec2.Vpc;
import com.pulumi.awsx.ec2.VpcArgs;
import com.pulumi.awsx.ec2.inputs.SubnetSpecArgs;
import com.pulumi.eks.Cluster;
import com.pulumi.eks.ClusterArgs;
import com.pulumi.eks.inputs.AutoModeOptionsArgs;
import com.pulumi.resources.CustomResourceOptions;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 var eksVpc = new Vpc("eksVpc", VpcArgs.builder()
 .enableDnsHostnames(true)
 .cidrBlock("10.0.0.0/16")
 // Necessary tags for EKS Auto Mode to identify the subnets for the load balancers.
 // See: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/deploy/subnet_discovery/
 .subnetSpecs(
 SubnetSpecArgs.builder()
 .type("Public")
 .tags(Map.ofEntries(
 Map.entry("kubernetes.io/cluster/eks-auto-mode-demo", "shared"),
 Map.entry("kubernetes.io/role/elb", "1")
 ))
 .build(),
 SubnetSpecArgs.builder()
 .type("Private")
 .tags(Map.ofEntries(
 Map.entry("kubernetes.io/cluster/eks-auto-mode-demo", "shared"),
 Map.entry("kubernetes.io/role/internal-elb", "1")
 ))
 .build())
 .subnetStrategy("Auto")
 .build());

 var cluster = new Cluster("cluster", ClusterArgs.builder()
 .name("eks-auto-mode-demo")
 // EKS Auto Mode requires Access Entries, use either the `Api` or `ApiAndConfigMap` authentication mode.
 .authenticationMode("API")
 .vpcId(eksVpc.vpcId())
 .publicSubnetIds(eksVpc.publicSubnetIds())
 .privateSubnetIds(eksVpc.privateSubnetIds())
 // Enables compute, storage and load balancing for the cluster.
 .autoMode(AutoModeOptionsArgs.builder()
 .enabled(true)
 .build())
 .build());
 }
}

name: eks-auto-mode-demo
description: A demo of EKS Auto Mode
runtime: yaml
resources:
 eksVpc:
 type: awsx:ec2:Vpc
 properties:
 enableDnsHostnames: true
 cidrBlock: "10.0.0.0/16"
 # Necessary tags for EKS Auto Mode to identify the subnets for the load balancers.
 # See: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/deploy/subnet_discovery/
 subnetSpecs:
 - type: "Public"
 tags:
 kubernetes.io/cluster/eks-auto-mode-demo: shared
 kubernetes.io/role/elb: "1"
 - type: "Private"
 tags:
 kubernetes.io/cluster/eks-auto-mode-demo: shared
 kubernetes.io/role/internal-elb: "1"
 subnetStrategy: "Auto"
 cluster:
 type: eks:Cluster
 properties:
 name: eks-auto-mode-demo
 # EKS Auto Mode requires Access Entries, use either the `Api` or `ApiAndConfigMap` authentication mode.
 authenticationMode: API
 vpcId: ${eksVpc.vpcId}
 publicSubnetIds: ${eksVpc.publicSubnetIds}
 privateSubnetIds: ${eksVpc.privateSubnetIds}
 # Enables compute, storage and load balancing for the cluster.
 autoMode:
 enabled: true

Deploying a Load-Balanced Application

Now that we have provisioned an EKS cluster with Auto Mode enabled, let’s deploy a web application fronted by an Application Load Balancer. EKS Auto Mode will automatically provision appropriately sized nodes for your workload, set up the Application Load Balancer according to the Ingress configuration, and manage ongoing maintenance and updates.

//...
const appName = "nginx";
const ns = new k8s.core.v1.Namespace(appName, {
 metadata: { name: appName },
}, { provider: cluster.provider });

const configMap = new k8s.core.v1.ConfigMap(appName, {
 metadata: {
 namespace: ns.metadata.name,
 },
 data: {
 "index.html": "<html><body><h1>Hello, Pulumi!</h1></body></html>",
 },
}, { provider: cluster.provider });

const deployment = new k8s.apps.v1.Deployment(appName, {
 metadata: {
 namespace: ns.metadata.name
 },
 spec: {
 selector: { matchLabels: { app: appName } },
 replicas: 3,
 template: {
 metadata: { labels: { app: appName } },
 spec: {
 containers: [{
 name: appName,
 image: appName,
 ports: [{ containerPort: 80 }],
 volumeMounts: [{ name: "nginx-index", mountPath: "/usr/share/nginx/html" }],
 }],
 volumes: [{
 name: "nginx-index",
 configMap: { name: configMap.metadata.name },
 }],
 },
 },
 },
}, { provider: cluster.provider });

const service = new k8s.core.v1.Service(appName, {
 metadata: {
 name: appName,
 namespace: ns.metadata.name
 },
 spec: {
 selector: { app: appName },
 ports: [{ port: 80, targetPort: 80 }],
 },
}, { provider: cluster.provider, dependsOn: [deployment] });

const ingressClass = new k8s.networking.v1.IngressClass("alb", {
 metadata: {
 namespace: ns.metadata.name,
 labels: {
 "app.kubernetes.io/name": "LoadBalancerController",
 },
 name: "alb",
 },
 spec: {
 controller: "eks.amazonaws.com/alb",
 }
}, { provider: cluster.provider });

const ingress = new k8s.networking.v1.Ingress(appName, {
 metadata: {
 namespace: ns.metadata.name,
 // Annotations for EKS Auto Mode to identify the Ingress as internet-facing and target-type as IP.
 annotations: {
 "alb.ingress.kubernetes.io/scheme": "internet-facing",
 "alb.ingress.kubernetes.io/target-type": "ip",
 }
 },
 spec: {
 ingressClassName: ingressClass.metadata.name,
 rules: [{
 http: {
 paths: [{
 path: "/",
 pathType: "Prefix",
 backend: {
 service: {
 name: service.metadata.name,
 port: {
 number: 80,
 },
 },
 },
 }],
 },
 }],
 }
}, { provider: cluster.provider });

export const url = ingress.status.apply(status => status?.loadBalancer?.ingress?.[0]?.hostname);

# ...
k8s_provider = kubernetes.Provider("k8sProvider", kubeconfig=cluster.kubeconfig)
ns = kubernetes.core.v1.Namespace("ns", metadata={
 "name": "nginx",
},
opts = pulumi.ResourceOptions(provider=k8s_provider))
index_content = kubernetes.core.v1.ConfigMap("index-content",
 metadata={
 "namespace": ns.metadata.name,
 },
 data={
 "index.html": "<html><body><h1>Hello, Pulumi!</h1></body></html>",
 },
 opts = pulumi.ResourceOptions(provider=k8s_provider))
deployment = kubernetes.apps.v1.Deployment("deployment",
 metadata={
 "namespace": ns.metadata.name,
 },
 spec={
 "selector": {
 "match_labels": {
 "app": "nginx",
 },
 },
 "replicas": 3,
 "template": {
 "metadata": {
 "labels": {
 "app": "nginx",
 },
 },
 "spec": {
 "containers": [{
 "name": "nginx",
 "image": "nginx",
 "ports": [{
 "container_port": 80,
 }],
 "volume_mounts": [{
 "name": "nginx-index",
 "mount_path": "/usr/share/nginx/html",
 }],
 }],
 "volumes": [{
 "name": "nginx-index",
 "config_map": {
 "name": index_content.metadata.name,
 },
 }],
 },
 },
 },
 opts = pulumi.ResourceOptions(provider=k8s_provider))
service = kubernetes.core.v1.Service("service",
 metadata={
 "name": "nginx",
 "namespace": ns.metadata.name,
 },
 spec={
 "selector": {
 "app": "nginx",
 },
 "ports": [{
 "port": 80,
 "target_port": 80,
 }],
 },
 opts = pulumi.ResourceOptions(provider=k8s_provider,
 depends_on=[deployment]))
ingress_class = kubernetes.networking.v1.IngressClass("ingressClass",
 metadata={
 "namespace": ns.metadata.name,
 "labels": {
 "app_kubernetes_io_name": "LoadBalancerController",
 },
 "name": "alb",
 },
 spec={
 "controller": "eks.amazonaws.com/alb",
 },
 opts = pulumi.ResourceOptions(provider=k8s_provider))
ingress = kubernetes.networking.v1.Ingress("ingress",
 metadata={
 "namespace": ns.metadata.name,
 "annotations": {
 "alb_ingress_kubernetes_io_scheme": "internet-facing",
 "alb_ingress_kubernetes_io_target_type": "ip",
 },
 },
 spec={
 "ingress_class_name": ingress_class.metadata.name,
 "rules": [{
 "http": {
 "paths": [{
 "path": "/",
 "path_type": "Prefix",
 "backend": {
 "service": {
 "name": service.metadata.name,
 "port": {
 "number": 80,
 },
 },
 },
 }],
 },
 }],
 },
 opts = pulumi.ResourceOptions(provider=k8s_provider))

pulumi.export("url", ingress.status.load_balancer.ingress[0].hostname)

package main

import (
 "github.com/pulumi/pulumi-awsx/sdk/v2/go/awsx/ec2"
 "github.com/pulumi/pulumi-eks/sdk/v3/go/eks"
 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes"
 appsv1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/apps/v1"
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/core/v1"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/meta/v1"
 networkingv1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/networking/v1"
 storagev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/storage/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // ...
 k8sProvider, err := kubernetes.NewProvider(ctx, "k8sProvider", &kubernetes.ProviderArgs{
 Kubeconfig: cluster.Kubeconfig,
 })
 if err != nil {
 return err
 }
 ns, err := corev1.NewNamespace(ctx, "ns", &corev1.NamespaceArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("nginx"),
 },
 }, pulumi.Provider(k8sProvider))
 if err != nil {
 return err
 }
 indexContent, err := corev1.NewConfigMap(ctx, "index-content", &corev1.ConfigMapArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 Data: pulumi.StringMap{
 "index.html": pulumi.String("<html><body><h1>Hello, Pulumi!</h1></body></html>"),
 },
 }, pulumi.Provider(k8sProvider))
 if err != nil {
 return err
 }
 deployment, err := appsv1.NewDeployment(ctx, "deployment", &appsv1.DeploymentArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 Spec: &appsv1.DeploymentSpecArgs{
 Selector: &metav1.LabelSelectorArgs{
 MatchLabels: pulumi.StringMap{
 "app": pulumi.String("nginx"),
 },
 },
 Replicas: pulumi.Int(3),
 Template: &corev1.PodTemplateSpecArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Labels: pulumi.StringMap{
 "app": pulumi.String("nginx"),
 },
 },
 Spec: &corev1.PodSpecArgs{
 Containers: corev1.ContainerArray{
 &corev1.ContainerArgs{
 Name: pulumi.String("nginx"),
 Image: pulumi.String("nginx"),
 Ports: corev1.ContainerPortArray{
 &corev1.ContainerPortArgs{
 ContainerPort: pulumi.Int(80),
 },
 },
 VolumeMounts: corev1.VolumeMountArray{
 &corev1.VolumeMountArgs{
 Name: pulumi.String("nginx-index"),
 MountPath: pulumi.String("/usr/share/nginx/html"),
 },
 },
 },
 },
 Volumes: corev1.VolumeArray{
 &corev1.VolumeArgs{
 Name: pulumi.String("nginx-index"),
 ConfigMap: &corev1.ConfigMapVolumeSourceArgs{
 Name: indexContent.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 },
 },
 },
 },
 },
 }, pulumi.Provider(k8sProvider))
 if err != nil {
 return err
 }
 service, err := corev1.NewService(ctx, "service", &corev1.ServiceArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("nginx"),
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 Spec: &corev1.ServiceSpecArgs{
 Selector: pulumi.StringMap{
 "app": pulumi.String("nginx"),
 },
 Ports: corev1.ServicePortArray{
 &corev1.ServicePortArgs{
 Port: pulumi.Int(80),
 TargetPort: pulumi.Any(80),
 },
 },
 },
 }, pulumi.Provider(k8sProvider), pulumi.DependsOn([]pulumi.Resource{
 deployment,
 }))
 if err != nil {
 return err
 }
 ingressClass, err := networkingv1.NewIngressClass(ctx, "ingressClass", &networkingv1.IngressClassArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 Labels: pulumi.StringMap{
 "app.kubernetes.io/name": pulumi.String("LoadBalancerController"),
 },
 Name: pulumi.String("alb"),
 },
 Spec: &networkingv1.IngressClassSpecArgs{
 Controller: pulumi.String("eks.amazonaws.com/alb"),
 },
 }, pulumi.Provider(k8sProvider))
 if err != nil {
 return err
 }
 ingress, err := networkingv1.NewIngress(ctx, "ingress", &networkingv1.IngressArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 Annotations: pulumi.StringMap{
 "alb.ingress.kubernetes.io/scheme": pulumi.String("internet-facing"),
 "alb.ingress.kubernetes.io/target-type": pulumi.String("ip"),
 },
 },
 Spec: &networkingv1.IngressSpecArgs{
 IngressClassName: ingressClass.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 Rules: networkingv1.IngressRuleArray{
 &networkingv1.IngressRuleArgs{
 Http: &networkingv1.HTTPIngressRuleValueArgs{
 Paths: networkingv1.HTTPIngressPathArray{
 &networkingv1.HTTPIngressPathArgs{
 Path: pulumi.String("/"),
 PathType: pulumi.String("Prefix"),
 Backend: &networkingv1.IngressBackendArgs{
 Service: &networkingv1.IngressServiceBackendArgs{
 Name: service.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 Port: &networkingv1.ServiceBackendPortArgs{
 Number: pulumi.Int(80),
 },
 },
 },
 },
 },
 },
 },
 },
 },
 }, pulumi.Provider(k8sProvider))
 if err != nil {
 return err
 }

 ctx.Export("url", ingress.Status.ApplyT(func(status networkingv1.IngressStatus) (*string, error) {
 return &status.LoadBalancer.Ingress[0].Hostname, nil
 }).(pulumi.StringPtrOutput))
 return nil
 })
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Awsx = Pulumi.Awsx;
using Eks = Pulumi.Eks;
using Kubernetes = Pulumi.Kubernetes;

return await Deployment.RunAsync(() =>
{
 // ...
 var k8sProvider = new Kubernetes.Provider.Provider("k8sProvider", new()
 {
 KubeConfig = cluster.Kubeconfig,
 });

 var ns = new Kubernetes.Core.V1.Namespace("ns", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "nginx",
 },
 }, new CustomResourceOptions
 {
 Provider = k8sProvider,
 });

 var indexContent = new Kubernetes.Core.V1.ConfigMap("index-content", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 },
 Data =
 {
 { "index.html", "<html><body><h1>Hello, Pulumi!</h1></body></html>" },
 },
 }, new CustomResourceOptions
 {
 Provider = k8sProvider,
 });

 var deployment = new Kubernetes.Apps.V1.Deployment("deployment", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 },
 Spec = new Kubernetes.Types.Inputs.Apps.V1.DeploymentSpecArgs
 {
 Selector = new Kubernetes.Types.Inputs.Meta.V1.LabelSelectorArgs
 {
 MatchLabels =
 {
 { "app", "nginx" },
 },
 },
 Replicas = 3,
 Template = new Kubernetes.Types.Inputs.Core.V1.PodTemplateSpecArgs
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Labels =
 {
 { "app", "nginx" },
 },
 },
 Spec = new Kubernetes.Types.Inputs.Core.V1.PodSpecArgs
 {
 Containers = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.ContainerArgs
 {
 Name = "nginx",
 Image = "nginx",
 Ports = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.ContainerPortArgs
 {
 ContainerPortValue = 80,
 },
 },
 VolumeMounts = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.VolumeMountArgs
 {
 Name = "nginx-index",
 MountPath = "/usr/share/nginx/html",
 },
 },
 },
 },
 Volumes = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.VolumeArgs
 {
 Name = "nginx-index",
 ConfigMap = new Kubernetes.Types.Inputs.Core.V1.ConfigMapVolumeSourceArgs
 {
 Name = indexContent.Metadata.Apply(metadata => metadata.Name),
 },
 },
 },
 },
 },
 },
 }, new CustomResourceOptions
 {
 Provider = k8sProvider,
 });

 var service = new Kubernetes.Core.V1.Service("service", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "nginx",
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 },
 Spec = new Kubernetes.Types.Inputs.Core.V1.ServiceSpecArgs
 {
 Selector =
 {
 { "app", "nginx" },
 },
 Ports = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.ServicePortArgs
 {
 Port = 80,
 TargetPort = 80,
 },
 },
 },
 }, new CustomResourceOptions
 {
 Provider = k8sProvider,
 DependsOn =
 {
 deployment,
 },
 });

 var ingressClass = new Kubernetes.Networking.V1.IngressClass("ingressClass", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 Labels =
 {
 { "app.kubernetes.io/name", "LoadBalancerController" },
 },
 Name = "alb",
 },
 Spec = new Kubernetes.Types.Inputs.Networking.V1.IngressClassSpecArgs
 {
 Controller = "eks.amazonaws.com/alb",
 },
 }, new CustomResourceOptions
 {
 Provider = k8sProvider,
 });

 var ingress = new Kubernetes.Networking.V1.Ingress("ingress", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 Annotations =
 {
 { "alb.ingress.kubernetes.io/scheme", "internet-facing" },
 { "alb.ingress.kubernetes.io/target-type", "ip" },
 },
 },
 Spec = new Kubernetes.Types.Inputs.Networking.V1.IngressSpecArgs
 {
 IngressClassName = ingressClass.Metadata.Apply(metadata => metadata.Name),
 Rules = new[]
 {
 new Kubernetes.Types.Inputs.Networking.V1.IngressRuleArgs
 {
 Http = new Kubernetes.Types.Inputs.Networking.V1.HTTPIngressRuleValueArgs
 {
 Paths = new[]
 {
 new Kubernetes.Types.Inputs.Networking.V1.HTTPIngressPathArgs
 {
 Path = "/",
 PathType = "Prefix",
 Backend = new Kubernetes.Types.Inputs.Networking.V1.IngressBackendArgs
 {
 Service = new Kubernetes.Types.Inputs.Networking.V1.IngressServiceBackendArgs
 {
 Name = service.Metadata.Apply(metadata => metadata.Name),
 Port = new Kubernetes.Types.Inputs.Networking.V1.ServiceBackendPortArgs
 {
 Number = 80,
 },
 },
 },
 },
 },
 },
 },
 },
 },
 }, new CustomResourceOptions
 {
 Provider = k8sProvider,
 });

 return new Dictionary<string, object?>
 {
 ["url"] = ingress.Status.Apply(status => status?.LoadBalancer?.Ingress[0]?.Hostname),
 };
});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.awsx.ec2.Vpc;
import com.pulumi.awsx.ec2.VpcArgs;
import com.pulumi.awsx.ec2.inputs.SubnetSpecArgs;
import com.pulumi.eks.Cluster;
import com.pulumi.eks.ClusterArgs;
import com.pulumi.eks.inputs.AutoModeOptionsArgs;
import com.pulumi.kubernetes.Provider;
import com.pulumi.kubernetes.ProviderArgs;
import com.pulumi.kubernetes.core_v1.Namespace;
import com.pulumi.kubernetes.core_v1.NamespaceArgs;
import com.pulumi.kubernetes.meta_v1.inputs.ObjectMetaArgs;
import com.pulumi.kubernetes.core_v1.ConfigMap;
import com.pulumi.kubernetes.core_v1.ConfigMapArgs;
import com.pulumi.kubernetes.apps_v1.Deployment;
import com.pulumi.kubernetes.apps_v1.DeploymentArgs;
import com.pulumi.kubernetes.apps_v1.inputs.DeploymentSpecArgs;
import com.pulumi.kubernetes.meta_v1.inputs.LabelSelectorArgs;
import com.pulumi.kubernetes.core_v1.inputs.PodTemplateSpecArgs;
import com.pulumi.kubernetes.core_v1.inputs.PodSpecArgs;
import com.pulumi.kubernetes.core_v1.Service;
import com.pulumi.kubernetes.core_v1.ServiceArgs;
import com.pulumi.kubernetes.core_v1.inputs.ServiceSpecArgs;
import com.pulumi.kubernetes.networking.k8s.io_v1.IngressClass;
import com.pulumi.kubernetes.networking.k8s.io_v1.IngressClassArgs;
import com.pulumi.kubernetes.networking.k8s.io_v1.inputs.IngressClassSpecArgs;
import com.pulumi.kubernetes.networking.k8s.io_v1.Ingress;
import com.pulumi.kubernetes.networking.k8s.io_v1.IngressArgs;
import com.pulumi.kubernetes.networking.k8s.io_v1.inputs.IngressSpecArgs;
import com.pulumi.kubernetes.storage.k8s.io_v1.StorageClass;
import com.pulumi.kubernetes.storage.k8s.io_v1.StorageClassArgs;
import com.pulumi.kubernetes.core_v1.PersistentVolumeClaim;
import com.pulumi.kubernetes.core_v1.PersistentVolumeClaimArgs;
import com.pulumi.kubernetes.core_v1.inputs.PersistentVolumeClaimSpecArgs;
import com.pulumi.kubernetes.core_v1.inputs.VolumeResourceRequirementsArgs;
import com.pulumi.resources.CustomResourceOptions;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 // ...
 var k8sProvider = new Provider("k8sProvider", ProviderArgs.builder()
 .kubeconfig(cluster.kubeconfig())
 .build());

 var ns = new Namespace("ns", NamespaceArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("nginx")
 .build())
 .build(), CustomResourceOptions.builder()
 .provider(k8sProvider)
 .build());

 var indexContent = new ConfigMap("indexContent", ConfigMapArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .namespace(ns.metadata().applyValue(metadata -> metadata.name()))
 .build())
 .data(Map.of("index.html", "<html><body><h1>Hello, Pulumi!</h1></body></html>"))
 .build(), CustomResourceOptions.builder()
 .provider(k8sProvider)
 .build());

 var deployment = new Deployment("deployment", DeploymentArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .namespace(ns.metadata().applyValue(metadata -> metadata.name()))
 .build())
 .spec(DeploymentSpecArgs.builder()
 .selector(LabelSelectorArgs.builder()
 .matchLabels(Map.of("app", "nginx"))
 .build())
 .replicas(3)
 .template(PodTemplateSpecArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .labels(Map.of("app", "nginx"))
 .build())
 .spec(PodSpecArgs.builder()
 .containers(ContainerArgs.builder()
 .name("nginx")
 .image("nginx")
 .ports(ContainerPortArgs.builder()
 .containerPort(80)
 .build())
 .volumeMounts(VolumeMountArgs.builder()
 .name("nginx-index")
 .mountPath("/usr/share/nginx/html")
 .build())
 .build())
 .volumes(VolumeArgs.builder()
 .name("nginx-index")
 .configMap(ConfigMapVolumeSourceArgs.builder()
 .name(indexContent.metadata().applyValue(metadata -> metadata.name()))
 .build())
 .build())
 .build())
 .build())
 .build())
 .build(), CustomResourceOptions.builder()
 .provider(k8sProvider)
 .build());

 var service = new Service("service", ServiceArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("nginx")
 .namespace(ns.metadata().applyValue(metadata -> metadata.name()))
 .build())
 .spec(ServiceSpecArgs.builder()
 .selector(Map.of("app", "nginx"))
 .ports(ServicePortArgs.builder()
 .port(80)
 .targetPort(80)
 .build())
 .build())
 .build(), CustomResourceOptions.builder()
 .provider(k8sProvider)
 .dependsOn(deployment)
 .build());

 var ingressClass = new IngressClass("ingressClass", IngressClassArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .namespace(ns.metadata().applyValue(metadata -> metadata.name()))
 .labels(Map.of("app.kubernetes.io/name", "LoadBalancerController"))
 .name("alb")
 .build())
 .spec(IngressClassSpecArgs.builder()
 .controller("eks.amazonaws.com/alb")
 .build())
 .build(), CustomResourceOptions.builder()
 .provider(k8sProvider)
 .build());

 var ingress = new Ingress("ingress", IngressArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .namespace(ns.metadata().applyValue(metadata -> metadata.name()))
 .annotations(Map.ofEntries(
 Map.entry("alb.ingress.kubernetes.io/scheme", "internet-facing"),
 Map.entry("alb.ingress.kubernetes.io/target-type", "ip")
 ))
 .build())
 .spec(IngressSpecArgs.builder()
 .ingressClassName(ingressClass.metadata().applyValue(metadata -> metadata.name()))
 .rules(IngressRuleArgs.builder()
 .http(HTTPIngressRuleValueArgs.builder()
 .paths(HTTPIngressPathArgs.builder()
 .path("/")
 .pathType("Prefix")
 .backend(IngressBackendArgs.builder()
 .service(IngressServiceBackendArgs.builder()
 .name(service.metadata().applyValue(metadata -> metadata.name()))
 .port(ServiceBackendPortArgs.builder()
 .number(80)
 .build())
 .build())
 .build())
 .build())
 .build())
 .build())
 .build())
 .build(), CustomResourceOptions.builder()
 .provider(k8sProvider)
 .build());

 ctx.export("url", ingress.status().applyValue(status -> status.loadBalancer().ingress()[0].hostname()));
 }
}

name: eks-auto-mode-yaml
description: A Pulumi YAML program to deploy a Kubernetes cluster on AWS
runtime: yaml
resources:
 # ...
 k8sProvider:
 type: pulumi:providers:kubernetes
 properties:
 kubeconfig: ${cluster.kubeconfig}
 ns:
 type: kubernetes:core/v1:Namespace
 properties:
 metadata:
 name: nginx
 options:
 provider: ${k8sProvider}
 index-content:
 type: kubernetes:core/v1:ConfigMap
 properties:
 metadata:
 namespace: ${ns.metadata.name}
 data:
 "index.html": "<html><body><h1>Hello, Pulumi!</h1></body></html>"
 options:
 provider: ${k8sProvider}
 deployment:
 type: kubernetes:apps/v1:Deployment
 properties:
 metadata:
 namespace: ${ns.metadata.name}
 spec:
 selector:
 matchLabels:
 app: nginx
 replicas: 3
 template:
 metadata:
 labels:
 app: nginx
 spec:
 containers:
 - name: nginx
 image: nginx
 ports:
 - containerPort: 80
 volumeMounts:
 - name: nginx-index
 mountPath: /usr/share/nginx/html
 volumes:
 - name: nginx-index
 configMap:
 name: ${index-content.metadata.name}
 options:
 provider: ${k8sProvider}
 service:
 type: kubernetes:core/v1:Service
 properties:
 metadata:
 name: nginx
 namespace: ${ns.metadata.name}
 spec:
 selector:
 app: nginx
 ports:
 - port: 80
 targetPort: 80
 options:
 provider: ${k8sProvider}
 dependsOn:
 - ${deployment}
 ingressClass:
 type: kubernetes:networking.k8s.io/v1:IngressClass
 properties:
 metadata:
 namespace: ${ns.metadata.name}
 labels:
 app.kubernetes.io/name: LoadBalancerController
 name: alb
 spec:
 controller: eks.amazonaws.com/alb
 options:
 provider: ${k8sProvider}
 ingress:
 type: kubernetes:networking.k8s.io/v1:Ingress
 properties:
 metadata:
 namespace: ${ns.metadata.name}
 annotations:
 alb.ingress.kubernetes.io/scheme: internet-facing
 alb.ingress.kubernetes.io/target-type: ip
 spec:
 ingressClassName: ${ingressClass.metadata.name}
 rules:
 - http:
 paths:
 - path: /
 pathType: Prefix
 backend:
 service:
 name: ${service.metadata.name}
 port:
 number: 80
 options:
 provider: ${k8sProvider}
outputs:
 url: ${ingress.status.loadBalancer.ingress[0].hostname}

After deploying this with pulumi up, you can access your application with curl once the Load Balancer has been provisioned:

~ curl $(pulumi stack output url)
Hello, Pulumi!

Deploying a Stateful Workload

EKS Auto Mode simplifies persistent storage management by automatically handling EBS volume provisioning for your stateful workloads. To leverage this feature, you can create a StorageClass that uses the EBS CSI driver (ebs.csi.eks.amazonaws.com) as its provisioner. EKS Auto Mode will then automatically manage the lifecycle of your persistent volumes, including provisioning, attaching, and cleanup.

This is how you’d configure persistent storage for a sample application:

// ...
const storageClass = new k8s.storage.v1.StorageClass("storage-class", {
 metadata: {
 name: "auto-ebs-sc",
 annotations: {
 "storageclass.kubernetes.io/is-default-class": "true",
 },
 },
 provisioner: "ebs.csi.eks.amazonaws.com",
 volumeBindingMode: "WaitForFirstConsumer",
 parameters: {
 type: "gp3",
 encrypted: "true",
 },
}, {
 provider: cluster.provider,
});
const pvc = new k8s.core.v1.PersistentVolumeClaim("pvc", {
 metadata: {
 name: "auto-ebs-claim",
 namespace: ns.metadata.apply(metadata => metadata.name),
 },
 spec: {
 accessModes: ["ReadWriteOnce"],
 storageClassName: storageClass.metadata.apply(metadata => metadata.name),
 resources: {
 requests: {
 storage: "8Gi",
 },
 },
 },
}, {
 provider: cluster.provider,
});
const statefulWorkload = new k8s.apps.v1.Deployment("stateful-workload", {
 metadata: {
 name: "inflate-stateful",
 namespace: ns.metadata.apply(metadata => metadata.name),
 },
 spec: {
 replicas: 1,
 selector: {
 matchLabels: {
 app: "inflate-stateful",
 },
 },
 template: {
 metadata: {
 labels: {
 app: "inflate-stateful",
 },
 },
 spec: {
 terminationGracePeriodSeconds: 0,
 nodeSelector: {
 "eks.amazonaws.com/compute-type": "auto",
 },
 containers: [{
 name: "bash",
 image: "public.ecr.aws/docker/library/bash:4.4",
 command: ["/usr/local/bin/bash"],
 args: [
 "-c",
 "while true; do echo $(date -u) >> /data/out.txt; sleep 60; done",
 ],
 resources: {
 requests: {
 cpu: "1",
 },
 },
 volumeMounts: [{
 name: "persistent-storage",
 mountPath: "/data",
 }],
 }],
 volumes: [{
 name: "persistent-storage",
 persistentVolumeClaim: {
 claimName: pvc.metadata.name,
 },
 }],
 },
 },
 },
}, {
 provider: cluster.provider,
});

export const url = ingress.status.loadBalancer.ingress[0].hostname.apply(h => `http://${h}:80`);

# ...
storage_class = kubernetes.storage.v1.StorageClass("storage-class",
 metadata={
 "name": "auto-ebs-sc",
 "annotations": {
 "storageclass_kubernetes_io_is_default_class": "true",
 },
 },
 provisioner="ebs.csi.eks.amazonaws.com",
 volume_binding_mode="WaitForFirstConsumer",
 parameters={
 "type": "gp3",
 "encrypted": "true",
 },
 opts = pulumi.ResourceOptions(provider=k8_s_provider))
pvc = kubernetes.core.v1.PersistentVolumeClaim("pvc",
 metadata={
 "name": "auto-ebs-claim",
 "namespace": ns.metadata.name,
 },
 spec={
 "access_modes": ["ReadWriteOnce"],
 "storage_class_name": storage_class.metadata.name,
 "resources": {
 "requests": {
 "storage": "8Gi",
 },
 },
 },
 opts = pulumi.ResourceOptions(provider=k8_s_provider))
stateful_workload = kubernetes.apps.v1.Deployment("stateful-workload",
 metadata={
 "name": "inflate-stateful",
 "namespace": ns.metadata.name,
 },
 spec={
 "replicas": 1,
 "selector": {
 "match_labels": {
 "app": "inflate-stateful",
 },
 },
 "template": {
 "metadata": {
 "labels": {
 "app": "inflate-stateful",
 },
 },
 "spec": {
 "termination_grace_period_seconds": 0,
 "node_selector": {
 "eks_amazonaws_com_compute_type": "auto",
 },
 "containers": [{
 "name": "bash",
 "image": "public.ecr.aws/docker/library/bash:4.4",
 "command": ["/usr/local/bin/bash"],
 "args": [
 "-c",
 "while true; do echo $(date -u) >> /data/out.txt; sleep 60; done",
 ],
 "resources": {
 "requests": {
 "cpu": "1",
 },
 },
 "volume_mounts": [{
 "name": "persistent-storage",
 "mount_path": "/data",
 }],
 }],
 "volumes": [{
 "name": "persistent-storage",
 "persistent_volume_claim": {
 "claim_name": pvc.metadata.name,
 },
 }],
 },
 },
 },
 opts = pulumi.ResourceOptions(provider=k8_s_provider))

package main

import (
 "github.com/pulumi/pulumi-awsx/sdk/v2/go/awsx/ec2"
 "github.com/pulumi/pulumi-eks/sdk/v3/go/eks"
 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes"
 appsv1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/apps/v1"
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/core/v1"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/meta/v1"
 networkingv1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/networking/v1"
 storagev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/storage/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // ...
 storageClass, err := storagev1.NewStorageClass(ctx, "storage-class", &storagev1.StorageClassArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("auto-ebs-sc"),
 Annotations: pulumi.StringMap{
 "storageclass.kubernetes.io/is-default-class": pulumi.String("true"),
 },
 },
 Provisioner: pulumi.String("ebs.csi.eks.amazonaws.com"),
 VolumeBindingMode: pulumi.String("WaitForFirstConsumer"),
 Parameters: pulumi.StringMap{
 "type": pulumi.String("gp3"),
 "encrypted": pulumi.String("true"),
 },
 }, pulumi.Provider(k8SProvider))
 if err != nil {
 return err
 }
 pvc, err := corev1.NewPersistentVolumeClaim(ctx, "pvc", &corev1.PersistentVolumeClaimArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("auto-ebs-claim"),
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 Spec: &corev1.PersistentVolumeClaimSpecArgs{
 AccessModes: pulumi.StringArray{
 pulumi.String("ReadWriteOnce"),
 },
 StorageClassName: storageClass.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 Resources: &corev1.VolumeResourceRequirementsArgs{
 Requests: pulumi.StringMap{
 "storage": pulumi.String("8Gi"),
 },
 },
 },
 }, pulumi.Provider(k8SProvider))
 if err != nil {
 return err
 }
 _, err = appsv1.NewDeployment(ctx, "stateful-workload", &appsv1.DeploymentArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("inflate-stateful"),
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 Spec: &appsv1.DeploymentSpecArgs{
 Replicas: pulumi.Int(1),
 Selector: &metav1.LabelSelectorArgs{
 MatchLabels: pulumi.StringMap{
 "app": pulumi.String("inflate-stateful"),
 },
 },
 Template: &corev1.PodTemplateSpecArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Labels: pulumi.StringMap{
 "app": pulumi.String("inflate-stateful"),
 },
 },
 Spec: &corev1.PodSpecArgs{
 TerminationGracePeriodSeconds: pulumi.Int(0),
 NodeSelector: pulumi.StringMap{
 "eks.amazonaws.com/compute-type": pulumi.String("auto"),
 },
 Containers: corev1.ContainerArray{
 &corev1.ContainerArgs{
 Name: pulumi.String("bash"),
 Image: pulumi.String("public.ecr.aws/docker/library/bash:4.4"),
 Command: pulumi.StringArray{
 pulumi.String("/usr/local/bin/bash"),
 },
 Args: pulumi.StringArray{
 pulumi.String("-c"),
 pulumi.String("while true; do echo $(date -u) >> /data/out.txt; sleep 60; done"),
 },
 Resources: &corev1.ResourceRequirementsArgs{
 Requests: pulumi.StringMap{
 "cpu": pulumi.String("1"),
 },
 },
 VolumeMounts: corev1.VolumeMountArray{
 &corev1.VolumeMountArgs{
 Name: pulumi.String("persistent-storage"),
 MountPath: pulumi.String("/data"),
 },
 },
 },
 },
 Volumes: corev1.VolumeArray{
 &corev1.VolumeArgs{
 Name: pulumi.String("persistent-storage"),
 PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSourceArgs{
 ClaimName: pvc.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return &metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 },
 },
 },
 },
 },
 }, pulumi.Provider(k8SProvider))
 if err != nil {
 return err
 }
 return nil
 })
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Awsx = Pulumi.Awsx;
using Eks = Pulumi.Eks;
using Kubernetes = Pulumi.Kubernetes;

return await Deployment.RunAsync(() =>
{
 // ...
 var storageClass = new Kubernetes.Storage.V1.StorageClass("storage-class", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "auto-ebs-sc",
 Annotations =
 {
 { "storageclass.kubernetes.io/is-default-class", "true" },
 },
 },
 Provisioner = "ebs.csi.eks.amazonaws.com",
 VolumeBindingMode = "WaitForFirstConsumer",
 Parameters =
 {
 { "type", "gp3" },
 { "encrypted", "true" },
 },
 }, new CustomResourceOptions
 {
 Provider = k8SProvider,
 });

 var pvc = new Kubernetes.Core.V1.PersistentVolumeClaim("pvc", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "auto-ebs-claim",
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 },
 Spec = new Kubernetes.Types.Inputs.Core.V1.PersistentVolumeClaimSpecArgs
 {
 AccessModes = new[]
 {
 "ReadWriteOnce",
 },
 StorageClassName = storageClass.Metadata.Apply(metadata => metadata.Name),
 Resources = new Kubernetes.Types.Inputs.Core.V1.VolumeResourceRequirementsArgs
 {
 Requests =
 {
 { "storage", "8Gi" },
 },
 },
 },
 }, new CustomResourceOptions
 {
 Provider = k8SProvider,
 });

 var statefulWorkload = new Kubernetes.Apps.V1.Deployment("stateful-workload", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "inflate-stateful",
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 },
 Spec = new Kubernetes.Types.Inputs.Apps.V1.DeploymentSpecArgs
 {
 Replicas = 1,
 Selector = new Kubernetes.Types.Inputs.Meta.V1.LabelSelectorArgs
 {
 MatchLabels =
 {
 { "app", "inflate-stateful" },
 },
 },
 Template = new Kubernetes.Types.Inputs.Core.V1.PodTemplateSpecArgs
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Labels =
 {
 { "app", "inflate-stateful" },
 },
 },
 Spec = new Kubernetes.Types.Inputs.Core.V1.PodSpecArgs
 {
 TerminationGracePeriodSeconds = 0,
 NodeSelector =
 {
 { "eks.amazonaws.com/compute-type", "auto" },
 },
 Containers = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.ContainerArgs
 {
 Name = "bash",
 Image = "public.ecr.aws/docker/library/bash:4.4",
 Command = new[]
 {
 "/usr/local/bin/bash",
 },
 Args = new[]
 {
 "-c",
 "while true; do echo $(date -u) >> /data/out.txt; sleep 60; done",
 },
 Resources = new Kubernetes.Types.Inputs.Core.V1.ResourceRequirementsArgs
 {
 Requests =
 {
 { "cpu", "1" },
 },
 },
 VolumeMounts = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.VolumeMountArgs
 {
 Name = "persistent-storage",
 MountPath = "/data",
 },
 },
 },
 },
 Volumes = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.VolumeArgs
 {
 Name = "persistent-storage",
 PersistentVolumeClaim = new Kubernetes.Types.Inputs.Core.V1.PersistentVolumeClaimVolumeSourceArgs
 {
 ClaimName = pvc.Metadata.Apply(metadata => metadata.Name),
 },
 },
 },
 },
 },
 },
 }, new CustomResourceOptions
 {
 Provider = k8SProvider,
 });
});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.awsx.ec2.Vpc;
import com.pulumi.awsx.ec2.VpcArgs;
import com.pulumi.awsx.ec2.inputs.SubnetSpecArgs;
import com.pulumi.eks.Cluster;
import com.pulumi.eks.ClusterArgs;
import com.pulumi.eks.inputs.AutoModeOptionsArgs;
import com.pulumi.kubernetes.Provider;
import com.pulumi.kubernetes.ProviderArgs;
import com.pulumi.kubernetes.core_v1.Namespace;
import com.pulumi.kubernetes.core_v1.NamespaceArgs;
import com.pulumi.kubernetes.meta_v1.inputs.ObjectMetaArgs;
import com.pulumi.kubernetes.core_v1.ConfigMap;
import com.pulumi.kubernetes.core_v1.ConfigMapArgs;
import com.pulumi.kubernetes.apps_v1.Deployment;
import com.pulumi.kubernetes.apps_v1.DeploymentArgs;
import com.pulumi.kubernetes.apps_v1.inputs.DeploymentSpecArgs;
import com.pulumi.kubernetes.meta_v1.inputs.LabelSelectorArgs;
import com.pulumi.kubernetes.core_v1.inputs.PodTemplateSpecArgs;
import com.pulumi.kubernetes.core_v1.inputs.PodSpecArgs;
import com.pulumi.kubernetes.core_v1.Service;
import com.pulumi.kubernetes.core_v1.ServiceArgs;
import com.pulumi.kubernetes.core_v1.inputs.ServiceSpecArgs;
import com.pulumi.kubernetes.networking.k8s.io_v1.IngressClass;
import com.pulumi.kubernetes.networking.k8s.io_v1.IngressClassArgs;
import com.pulumi.kubernetes.networking.k8s.io_v1.inputs.IngressClassSpecArgs;
import com.pulumi.kubernetes.networking.k8s.io_v1.Ingress;
import com.pulumi.kubernetes.networking.k8s.io_v1.IngressArgs;
import com.pulumi.kubernetes.networking.k8s.io_v1.inputs.IngressSpecArgs;
import com.pulumi.kubernetes.storage.k8s.io_v1.StorageClass;
import com.pulumi.kubernetes.storage.k8s.io_v1.StorageClassArgs;
import com.pulumi.kubernetes.core_v1.PersistentVolumeClaim;
import com.pulumi.kubernetes.core_v1.PersistentVolumeClaimArgs;
import com.pulumi.kubernetes.core_v1.inputs.PersistentVolumeClaimSpecArgs;
import com.pulumi.kubernetes.core_v1.inputs.VolumeResourceRequirementsArgs;
import com.pulumi.resources.CustomResourceOptions;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 // ...
 var storageClass = new StorageClass("storageClass", StorageClassArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("auto-ebs-sc")
 .annotations(Map.of("storageclass.kubernetes.io/is-default-class", "true"))
 .build())
 .provisioner("ebs.csi.eks.amazonaws.com")
 .volumeBindingMode("WaitForFirstConsumer")
 .parameters(Map.ofEntries(
 Map.entry("type", "gp3"),
 Map.entry("encrypted", "true")
 ))
 .build(), CustomResourceOptions.builder()
 .provider(k8SProvider)
 .build());

 var pvc = new PersistentVolumeClaim("pvc", PersistentVolumeClaimArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("auto-ebs-claim")
 .namespace(ns.metadata().applyValue(metadata -> metadata.name()))
 .build())
 .spec(PersistentVolumeClaimSpecArgs.builder()
 .accessModes("ReadWriteOnce")
 .storageClassName(storageClass.metadata().applyValue(metadata -> metadata.name()))
 .resources(VolumeResourceRequirementsArgs.builder()
 .requests(Map.of("storage", "8Gi"))
 .build())
 .build())
 .build(), CustomResourceOptions.builder()
 .provider(k8SProvider)
 .build());

 var statefulWorkload = new Deployment("statefulWorkload", DeploymentArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("inflate-stateful")
 .namespace(ns.metadata().applyValue(metadata -> metadata.name()))
 .build())
 .spec(DeploymentSpecArgs.builder()
 .replicas(1)
 .selector(LabelSelectorArgs.builder()
 .matchLabels(Map.of("app", "inflate-stateful"))
 .build())
 .template(PodTemplateSpecArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .labels(Map.of("app", "inflate-stateful"))
 .build())
 .spec(PodSpecArgs.builder()
 .terminationGracePeriodSeconds(0)
 .nodeSelector(Map.of("eks.amazonaws.com/compute-type", "auto"))
 .containers(ContainerArgs.builder()
 .name("bash")
 .image("public.ecr.aws/docker/library/bash:4.4")
 .command("/usr/local/bin/bash")
 .args(
 "-c",
 "while true; do echo $(date -u) >> /data/out.txt; sleep 60; done")
 .resources(ResourceRequirementsArgs.builder()
 .requests(Map.of("cpu", "1"))
 .build())
 .volumeMounts(VolumeMountArgs.builder()
 .name("persistent-storage")
 .mountPath("/data")
 .build())
 .build())
 .volumes(VolumeArgs.builder()
 .name("persistent-storage")
 .persistentVolumeClaim(PersistentVolumeClaimVolumeSourceArgs.builder()
 .claimName(pvc.metadata().applyValue(metadata -> metadata.name()))
 .build())
 .build())
 .build())
 .build())
 .build())
 .build(), CustomResourceOptions.builder()
 .provider(k8SProvider)
 .build());
 }
}

name: eks-auto-mode-yaml
description: A Pulumi YAML program to deploy a Kubernetes cluster on AWS
runtime: yaml
resources:
 # ...
 storage-class:
 type: kubernetes:storage.k8s.io/v1:StorageClass
 properties:
 metadata:
 name: auto-ebs-sc
 annotations:
 storageclass.kubernetes.io/is-default-class: "true"
 provisioner: ebs.csi.eks.amazonaws.com
 volumeBindingMode: WaitForFirstConsumer
 parameters:
 type: gp3
 encrypted: "true"
 options:
 provider: ${k8sProvider}
 pvc:
 type: kubernetes:core/v1:PersistentVolumeClaim
 properties:
 metadata:
 name: auto-ebs-claim
 namespace: ${ns.metadata.name}
 spec:
 accessModes:
 - ReadWriteOnce
 storageClassName: ${storage-class.metadata.name}
 resources:
 requests:
 storage: 8Gi
 options:
 provider: ${k8sProvider}
 stateful-workload:
 type: kubernetes:apps/v1:Deployment
 properties:
 metadata:
 name: inflate-stateful
 namespace: ${ns.metadata.name}
 spec:
 replicas: 1
 selector:
 matchLabels:
 app: inflate-stateful
 template:
 metadata:
 labels:
 app: inflate-stateful
 spec:
 terminationGracePeriodSeconds: 0
 nodeSelector:
 eks.amazonaws.com/compute-type: auto
 containers:
 - name: bash
 image: public.ecr.aws/docker/library/bash:4.4
 command: ["/usr/local/bin/bash"]
 args: ["-c", "while true; do echo $(date -u) >> /data/out.txt; sleep 60; done"]
 resources:
 requests:
 cpu: "1"
 volumeMounts:
 - name: persistent-storage
 mountPath: /data
 volumes:
 - name: persistent-storage
 persistentVolumeClaim:
 claimName: ${pvc.metadata.name}
 options:
 provider: ${k8sProvider}
outputs:
 url: ${ingress.status.loadBalancer.ingress[0].hostname}

Summary

EKS Auto Mode represents a significant step forward in simplifying Kubernetes operations on AWS. Pulumi’s support for this feature allows you to define EKS Auto Mode clusters using your favorite programming language and integrate them with your existing infrastructure code.

To get started with EKS Auto Mode in Pulumi:

Update your AWS providers to the latest versions
Check out our EKS documentation
Join our Community Slack for support and discussions

We are excited to see what you build with EKS Auto Mode and Pulumi!

Pulumi ESC and External Secrets Operator: The Perfect Solution for Today's Cloud-Native Secret Management

Engin Diri — Fri, 04 Oct 2024 00:00:00 +0000

Managing secrets in a cloud-native environment can be challenging, but it is crucial for ensuring the security and integrity of any application or infrastructure. We encounter a lot of different types of secrets, from API keys, database passwords, and certificates to tokens and passwords. These secrets need to be stored securely and accessed by different services in a secure way without exposing any sensitive information to unauthorized users.

Here is where Pulumi ESC and External Secrets Operator come into play by providing a secure and efficient solution for cloud-native secret management.

Pulumi ESC (Environments, Secrets, and Configuration)

Pulumi ESC is our new secrets management & orchestration service for environments, secrets, and configurations. It can be used as part of your Pulumi IaC (Infrastructure as Code) projects using pulumiConfig to expose the stored values to your Pulumi stacks.

But it can also be used as a standalone service to manage secrets and configurations. Pulumi ESC provides support for applications through its dedicated SDKs built for different programming languages. It can also be managed from the command line with the pulumi env commands in the Pulumi CLI, which can inject secrets and configuration as a context for your CLI tools (e.g. terraform).

For more information on Pulumi ESC, you can check out the official documentation.

External Secrets Operator (ESO)

The Open Source project External Secrets Operator (ESO) is a Kubernetes operator that helps manage secrets in a highly secure and scalable way. ESO synchronizes secrets stored in external secret management systems like Pulumi ESC, HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, and more into Kubernetes secrets. This approach allows us to control secret sprawl and centralize the management of secrets in a single place while providing a secure and controlled way to access them.

Why Kubernetes secrets are not enough?

Kubernetes secrets are a great way to store sensitive information in a Kubernetes cluster. Kubernetes secrets are created and stored as a Kubernetes resource object and then used by pods or other resources inside the cluster.

Here is an example of a Kubernetes secret manifest file:

apiVersion: v1
kind: Secret
metadata:
 name: mysecret
type: Opaque
data:
 username: YWRtaW4= # admin
 password: YWRtaW4= # admin

or using kubectl:

kubectl create secret generic mysecret --from-literal=username=admin --from-literal=password=admin

Kubernetes secrets have several limitations. Here are the top ones:

They are stored by default in etcd in base64 encoded format, which is not secure.
They are created either manually by using kubectl commands or by using a manifest file, making them hard to manage at scale.
Hard to manage and synchronize secrets across different environments and clusters.
There is no default way to rotate secrets automatically.

ESO shines by tackling all the challenges listed and providing a way to manage secrets in a more secure and efficient manner. Benefits include:

Mitigate security risks as secrets are stored in a secure external secret management system like Pulumi ESC, which provides more robust security features like encryption, access control, and auditing.
Eliminate manual management of secrets by synchronizing them from external secret management systems into the Kubernetes cluster, reducing the need for manual intervention and minimizing the risk of human error.
ESO provides a unified interface, making it easy to manage secrets across clusters and multi-cloud environments.
ESO integrates with external secret management systems which ensure that access to secrets is tracked and an audit trail is maintained. This is crucial for security compliance and adhering to regulatory standards.
Automate secrets rotation when the external secret management system has support it.

External Secrets Operator architecture

The central part of the ESO architecture is the extension of the Kubernetes API with a new Custom Resource Definition (CRD), which defines where the secrets are stored and how they are synchronized into Kubernetes secrets. The ESO controller then retrieves the secrets from the external secret management system and creates or updates the Kubernetes secrets. Every time a secret is updated in the external secret management system, the ESO controller will synchronize the secret in the Kubernetes cluster and update the secret accordingly.

Core resources of ESO

(Cluster)SecretStore: Defines the external secret management system where the secrets are stored. It contains the necessary information to connect to the external secret management system, such as the endpoint, authentication credentials, and other configuration options. The SecretStore is namespaced while the ClusterSecretStore is cluster wide, which allows you cross-namespaced access to the secrets.
(Cluster)ExternalSecret: Defines the secret that needs to be synchronized into the Kubernetes cluster. It contains the reference to the SecretStore and the secret key and how the data should be transformed and saved as a Kubernetes secret.
PushSecret: With the PushSecret resource, you can push secrets from the Kubernetes cluster to the external secret management system. The Pulumi ESC provider for ESO supports this feature, which allows you to push secrets from the cluster into Pulumi ESC. This feature is useful when you create secrets through other means and want to synchronize them into Pulumi ESC to manage them centrally.

How to Use Pulumi ESC and ESO Together

To demonstrate how to use Pulumi ESC and ESO together, we will store some secrets and configuration values in Pulumi ESC, then synchronize them into a Kubernetes cluster using ESO.

To follow along, you’ll need the following prerequisites:

A Kubernetes cluster (I will be using a local KinD cluster, but you can use any Kubernetes cluster)
Pulumi CLI installed. You can use the pulumi env commands to manage secrets and configurations.
kubectl CLI installed for some debugging

Step 1: Install ESO and Link to Pulumi ESC

We will use Pulumi with the pulumi-kubernetes provider to deploy ESO into the Kubernetes cluster. The power of Pulumi ESC integration comes in two ways: First, to supply the Pulumi access token to ESO via Pulumi IaC, allowing the new ESO instance to access Pulumi ESC later. Second, to provide configuration and secrets from Pulumi ESC, via ESO, to the PodInfo application within the Kubernetes cluster.

pulumi new kubernetes-typescript --name eso-pulumi-esc

You should see the following output:

This command will walk you through creating a new Pulumi project.

Enter a value or leave blank to accept the (default), and press <ENTER>.
Press ^C at any time to quit.

Project description (A minimal Kubernetes TypeScript Pulumi program):
Created project 'eso-pulumi-esc'

Please enter your desired stack name.
To create a stack in an organization, use the format <org-name>/<stack-name> (e.g. `acmecorp/dev`).
Stack name (dev):
Created stack 'dev'

The package manager to use for installing dependencies npm
Installing dependencies...


added 310 packages, and audited 311 packages in 12s

44 packages are looking for funding
 run `npm fund` for details

found 0 vulnerabilities
Finished installing dependencies

Your new project is ready to go! ✨

To perform an initial deployment, run `pulumi up`

Head to Pulumi Cloud Console and create a new Pulumi ESC project with the name eso-esc-demo:

And the environment eso-esc-dev:

In the editor add the following yaml into the Environment definition:

values:
 pulumiConfig:
 pulumi-pat:
 fn::secret: <your-pulumi-pat>

Don’t know how to create a PAT? Check out the official Pulumi documentation.

If you prefer to use the Pulumi CLI, you can create the environment by running:

pulumi env init <your-org>/eso-esc-demo/eso-esc-dev

And set the configuration by running the env edit command and copy the above YAML into the editor:

pulumi env edit <your-org>/eso-esc-demo/eso-esc-dev

Now, we need to link the Pulumi ESC project to the Pulumi IaC project. To do this, we need to add the following to your Pulumi.dev.yaml:

environment:
- eso-esc-demo/eso-esc-dev

Now that all the wiring is done, we can deploy the ESO instance into the Kubernetes cluster. Open the index.ts file and add the following code:

import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";

const config = new pulumi.Config();

// Create a Kubernetes namespace
const ns = new k8s.core.v1.Namespace("external-secrets", {
 metadata: {
 name: "external-secrets",
 },
});

// Deploy a Helm release into the namespace
const externalSecrets = new k8s.helm.v3.Release("external-secrets", {
 chart: "external-secrets",
 version: "0.10.4", // Specify the version of the chart
 namespace: ns.metadata.name,
 repositoryOpts: {
 repo: "https://charts.external-secrets.io",
 },
});

// Deploy a secret into the namespace
const mySecret = new k8s.core.v1.Secret("my-secret", {
 metadata: {
 namespace: ns.metadata.name,
 name: "pulumi-access-token",
 },
 stringData: {
 "PULUMI_ACCESS_TOKEN": config.require("pulumi-pat"),
 },
 type: "Opaque",
});

// Export the namespace name
export const namespaceName = ns.metadata.name;

// Export the Helm release name
export const helmReleaseName = externalSecrets.name;

Deploy the stack by running:

pulumi up

And you should see that the secret was created in the Kubernetes cluster and the ESO instance was deployed successfully.

kubectl get secret -n external-secrets pulumi-access-token -o jsonpath='{.data.PULUMI_ACCESS_TOKEN}' | base64 -d

Step 2: Create an External Secret

Now, we will create a secret in the Pulumi ESC project and synchronize it into the Kubernetes cluster by creating an ESO reference to the external secret.

Create a new ESC environment called esp-esc-app in the eso-esc-demo ESC project.

Add the following YAML into the Environment definition:

values:
 app:
 hello: world
 hello-secret:
 fn::secret: world

If you prefer to use the Pulumi CLI, you can create the environment by running:

pulumi env init <your-org>/eso-esc-demo/esp-esc-app

And set the configuration by running the env edit command and copy the above YAML into the editor:

pulumi env edit <your-org>/eso-esc-demo/esp-esc-app

Either way, you should see following environment configuration in the Pulumi Cloud Console:

We can now create ClusterExternalSecret in the index.ts and reference the ESC secret.

import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";

// cut for brevity

const crd = new k8s.apiextensions.CustomResource("cluster-secret-store", {
 apiVersion: "external-secrets.io/v1beta1",
 kind: "ClusterSecretStore",
 metadata: {
 name: "secret-store",
 },
 spec: {
 provider: {
 pulumi: {
 organization: pulumi.getOrganization(),
 project: "eso-esc-demo",
 environment: "esp-esc-app",
 accessToken: {
 secretRef: {
 name: mySecret.metadata.name,
 key: "PULUMI_ACCESS_TOKEN",
 namespace: mySecret.metadata.namespace,
 },
 },
 },
 },
 },
});

We can check that the secret was successfully synchronized by running:

kubectl get clustersecretstore secret-store -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}'

Step 3: Deploy an Application and Reference the Secret

Now, we can deploy an application that references the secret from the Kubernetes cluster. I am going to deploy the app called PodInfo, which is an application that exposes the pod information as an API. Add the following code to the index.ts:

import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";

// cut for brevity
const podInfo = new k8s.helm.v3.Release("podinfo", {
 chart: "podinfo",
 version: "6.7.0",
 namespace: "podinfo",
 createNamespace: true,
 repositoryOpts: {
 repo: "https://stefanprodan.github.io/podinfo",
 },
 values: {
 extraEnvs: [
 {
 name: "FROM_ESC_VIA_ESO",
 valueFrom: {
 secretKeyRef: {
 name: "esc-secret-store",
 key: "hello-secret",
 }
 }
 },
 {
 name: "FROM_ESC_VIA_ESO_2",
 valueFrom: {
 secretKeyRef: {
 name: "esc-secret-store",
 key: "hello",
 }
 }
 }
 ]
 },
});

const externalSecretPodInfo = new k8s.apiextensions.CustomResource("external-secret-podinfo", {
 apiVersion: "external-secrets.io/v1beta1",
 kind: "ExternalSecret",
 metadata: {
 name: "esc-secret-store",
 namespace: podInfo.namespace,
 },
 spec: {
 dataFrom: [
 {
 extract: {
 conversionStrategy: "Default",
 key: "app",
 }
 }
 ],
 refreshInterval: "1h",
 secretStoreRef: {
 kind: clusterSecretStore.kind,
 name: clusterSecretStore.metadata.name,
 }
 },
});

This will deploy the PodInfo application and create an ExternalSecret resource that references the secret from the Pulumi ESC project. We mount the secret into the pod as an environment variable by using the extraEnvs field in the Helm chart.

In a real-world application, you might use alternate ways to reference the secret, like mounting it as a volume or using fromEnv field in the Helm chart. It depends on your application and how you want to consume the secret.

After deploying the stack, you should see that the PodInfo application is running. To access the application, we can port-forward the service by running:

kubectl port-forward svc/podinfo 9898:9898 -n podinfo

Our PodInfo demo application provides a large selection of different endpoints to interact with. We are interested in the /env endpoint to check if the secret was successfully mounted into the pod. Let’s try that:

curl -s localhost:9898/env | grep FROM
 "FROM_ESC_VIA_ESO=world",
 "FROM_ESC_VIA_ESO_2=world",

And here we see secret values from the ESC project, proxied by ESO, and passed to the PodInfo application as environment variables. That means the secret was successfully mounted into the pod and the application is running as expected.

Step 4: Clean Up

After you are done with the demo, you can clean up the resources by running:

pulumi destroy

Conclusion

The combination of Pulumi ESC and External Secrets Operator provides a secure and efficient way to manage secrets in a cloud-native environment, improving security, efficiency, and compliance when consuming secrets in your Kubernetes cluster. By using Pulumi ESC, you can manage secrets and configurations in a centralized way, and with ESO, you now have a streamlined process to synchronize secrets into your Kubernetes cluster.

Start using Pulumi ESC and ESO today by creating an account on the Pulumi Cloud Console and begin managing your secrets in a secure and efficient way.

Create an Account

Uploading documents to your AI Slackbot in real-time using S3, SQS and Pulumi on AWS

Tyler Mulligan — Mon, 03 Jun 2024 17:21:02 +0100

In the introductory blog post, we learned to Create an AI Slack Bot to Chat with Your Data Using Embedchain, Pulumi on AWS, and continued with Adding data to Pinecone using S3, Embedchain, and Pulumi on AWS for an AI Slack bot.

For reference, here’s what our architecture looked like at the end of the second blog post.

To follow along, clone the project, git clone https://github.com/catmeme/arti.git or view it on GitHub.

S3 Events and SQS

AWS provides a few ways to integrate with AWS S3 Events, for this project, we’re going to focus on the latest and most straight-forward, S3 Event Notifications. Depending on your use case, integrating with EventBridge may make more sense.

While we can integrate S3 events directly with Lambda, what happens when we add 100’s of files at the same time? Potentially, hundreds of Lambdas could be invoked simultaneously, which could become an issue depending on other factors. We’ll solve for this by Using Lambda with Amazon SQS, and reserved concurrency.

Our Pulumi program is updated to add a Queue and integrate it with our new Lambda SQS handler (policies left out for brevity, you can view them in the deploy/pulumi/__main__.py in the project).

# Add a queue
sqs_queue = aws.sqs.Queue(...)

# Add a docker image runtime for our Lambda SQS handler
docker_image_lambda_sqs_handler = docker.Image(...)

# Add Lambda SQS handler
app_sqs_lambda = aws.lambda_.Function(
 resource_name=f"{app_name}-sqs-lambda",
 environment=lambda_environment, # We've added the sqs_queue.url here as well
 image_uri=docker_image_lambda_sqs_handler.repo_digest,
 opts=pulumi.ResourceOptions(depends_on=[docker_image_lambda_sqs_handler]),
 **common_lambda_options,
)

# Add event source mapping
event_source_mapping = aws.lambda_.EventSourceMapping(
 f"{app_name}-lambda-sqs-event-source-mapping",
 event_source_arn=sqs_queue.arn,
 function_name=app_sqs_lambda.name,
 enabled=True,
 batch_size=10,
)

Add the application code responsible for processing S3 events coming from SQS.

def handler(event, context):
 for record in event.get("Records", []):
 s3_event = json.loads(record.get("body", "{}"))
 for s3_record in s3_event.get("Records", []):
 object_key = s3_record.get("s3", {}).get("object", {}).get("key")
 if object_key:
 logger.info("Processed %s in bucket %s.", object_key, app_bucket_name)
 load_data(asset_location=f"{app_bucket_name}/{object_key}")

 return {"statusCode": 200, "body": json.dumps("Successfully processed S3 event(s) from SQS messages.")}

We also have a scheduled event that we set up for ingestion in the last post, which we’re going to disable in favor of real-time events.

That’s pretty much it on the infrastructure side, here’s the updated diagram.

Slack to S3 file upload bridge

So what do we do with this new functionality? How do we expose it to the user? The most obvious approach is to reuse our existing Slack integration.

We’ll start by creating a Slack file upload bridge to S3, the bot watches for file_shared events on Slack and re-uploads files to our S3 bucket.

Using Slack Bolt, we add a new function handle_file_upload with the @app.event("file_shared") decorator, binding it to said event. When that event occurs, it’s processed, and the details of our file are retrieved, a new payload is constructed using requests, and the file gets uploaded to S3, this is our bridge.

@app.event("file_shared")
def handle_file_uploads(event, say, client, logger):
 file_id = event['file']['id']
 channel_id = event['channel_id']

 # Fetch file details using the Slack API
 file_info = client.files_info(token=slack_bot_token, file=file_id)['file']
 file_url = file_info['url_private_download']

 # Download file content from Slack
 headers = {'Authorization': f"Bearer {slack_bot_token}"}
 response = requests.get(file_url, headers=headers, stream=True)

 if response.status_code == 200:
 # Upload the file to S3
 s3_client.upload_fileobj(
 Fileobj=response.raw,
 Bucket=os.getenv("APP_BUCKET_NAME"),
 Key=file_info['name']
 )

 # Fetch recent messages to find the correct timestamp
 messages = client.conversations_history(channel=channel_id, limit=10)['messages']
 for msg in messages:
 if msg.get('files') and any(f['id'] == file_id for f in msg.get('files', [])):
 # Found the message with the correct file
 message_ts = msg['ts']
 client.reactions_add(channel=channel_id, name="white_check_mark", timestamp=message_ts)
 # Respond in a thread to the message containing the file
 say(text=f"File `{file_info['name']}` uploaded to S3 successfully! :tada:", thread_ts=message_ts)
 return

 logger.error("No valid message found for reaction.")

Keep in mind, we’re not doing any validation here yet. We’re blindly copying files from Slack to S3 and having our data loader process them. It’s probably not a good idea to deploy this bot to a public Slack channel without more moderation/validation in place.

Here’s what it looks like when we drag/drop a file into Slack now. On the right, a file was shared with the bot in private message. The bot acknowledged this upload with the checkmark reaction, the job is placed on the queue. Shortly after, on the left, the SQS Lambda handler has finished processing the uploaded file. We’re now able to use the bot to talk about the uploaded files.

Slack interactivity

In minutes, we were able to move from a scheduled task for processing files in a bucket, to real-time file processing integrated with Slack. We have our /arti command we can use to interrogate the documents we just uploaded, but we’re going to have a little more fun.

Thanks to Slack Block Kit Builder, we can build a form that exposes all of the configuration options for the query method in Embedchain’s app, providing the user with more options for interacting with the RAG model.

Our code to utilize the JSON we’ve crafted in the builder looks something like (we’ll call this /smarti):

@app.command("/smarti")
def open_smarti_modal(ack, body, client):
 ack()
 print("Opening SMARTI modal...")
 modal_file_path = Path(config.project_root) / 'config' / 'slack_modals' / 'chat.json'

 with open(modal_file_path, 'r') as json_file:
 modal_view = json.load(json_file)

 client.views_open(
 trigger_id=body["trigger_id"],
 view=modal_view
 )

And the code for handling that submission looks something like:

def handle_smarti_submission(client, logger, body, view):
 values = view["state"]["values"]
 question = values["question"]["input"]["value"]
 model = values["model"]["input"]["selected_option"]["value"]
 temperature = float(values["temperature"]["input"]["value"])
 max_tokens = int(values["max_tokens"]["input"]["value"])
 top_p = float(values["top_p"]["input"]["value"])
 prompt = values["prompt"]["input"]["value"]
 system_prompt = values["system_prompt"]["input"]["value"]
 citation_options = values["citations"]["input"].get("selected_options", [])
 citations = any(option['value'] == 'true' for option in citation_options)

 ai_response = ask_ai(
 question=question,
 model=model,
 temperature=temperature,
 max_tokens=max_tokens,
 top_p=top_p,
 prompt=prompt,
 system_prompt=system_prompt,
 citations=citations
 )
 ...

Submitting that form results in a response such as below. The bot honored our prompts, talking like Shakespeare, a thread is created, and citations were noted, showing where the data was retrieved from.

Lastly, if we want interact with the bot more publicly, then we can use the app_mention event.

@app.event("app_mention")
def handle_app_mention(event, say, client):
 text = event['text']
 bot_user_id = client.auth_test()["user_id"]

 # Strip the mention of the bot to isolate the command and arguments
 command_text = text.split(f'<@{bot_user_id}>')[1].strip() if f'<@{bot_user_id}>' in text else text

 # Parse the command and arguments
 parts = command_text.split(maxsplit=1)
 if not parts:
 say("I need a command to do something useful!")
 return

 # Get the command and the remainder of the message as arguments
 command = parts[0].lower()
 args = parts[1] if len(parts) > 1 else ""

 if command == "ask":
 if args:
 response = ask_ai(question=args)
 say(response)
 else:
 say("Please provide a question to ask.")
 elif command == "list":
 list_data_sources(say)
 else:
 say(f"Sorry, I don't recognize the command `{command}`.")

Resulting in an interaction such as

Of course, this same methodology can be used on any of the events exposed by Slack.

To learn more about extending this bot, see Slack Bolt’s documentation.

Conclusion

In this blog post, we took our existing Slack bot which has a data loading process that loads documents from an S3 bucket into a vector database on a schedule, and we wired it up to behave in real time.

To expose this functionality to the end user, we created a bridge between file uploads to Slack and S3, allowing drag/drop capabilities of files in conversational chat.

Furthering our interactions with our application, we leveraged Slack’s Block Kit to create a custom modal, and interact with events.

As always, we welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is easy and free. Get started today!

Happy AI building!

Next-level IaC: Breakpoint Debugging for Pulumi Programs

Troy Howard — Fri, 31 May 2024 00:00:00 +0000

For many of us, life is suspended between presses of the F5 key. Our IDE is the only place where everything is organized and makes sense. And while we know that the likelihood of a code hole-in-one is very rare (that magical moment where you run your build and everything just works the first time), we always hope that this time, this will be the run where you see the output that lets you know you’ve gotten it right. And in between? Debugging.

Debugging is a complicated topic. But what is not complicated is understanding the cost of debugging. Debugging is the biggest unknown-unknown in software. It makes you miss your sprint goals, it holds back scaling and company growth, it slows your time to market. Debugging is what makes estimating so difficult, because until you know what the problem is, you don’t know how long it will take to fix, or indeed if it is even possible to fix at all. Which is why savvy developers invest a lot of time into refining their debugging process and seek out the best-in-class tools to streamline this process.

In the infrastructure world, debugging is a pretty big problem. Some of the most popular tools have nothing more than simple unstructured text output logs. While tools like Terraform call themselves “Infrastructure as Code” the “code” is a proprietary DSL, not a general purpose programming language, and so there is little-to-no IDE support. There is no F5. Even worse, there is no F9 for setting a breakpoint. There’s no way to inspect state mid-run, or more importantly to explore in an unstructured way, because this is where the “aha!” moments usually happen.

Pulumi gives you a rich IaC debugging experience by enabling breakpoint debugging in your favorite IDE. Let’s walk through an example of how to enable a full-featured debugging workflow in VS Code for TypeScript.

How Breakpoint Debugging Works

Using a debugger can feel like that moment in The Matrix where Neo first sees the green streaming code raining down. A debugger lifts the veil from your otherwise opaque executable code. Tools like VS Code provide a user experience where you set breakpoints, i.e. certain lines of code that are designated as points where you want to pause execution and inspect state. Hitting the F9 key in VS Code sets a breakpoint on the current line of code. Later, you run your code in debug mode. This is a special execution state where the underlying runtime (e.g. Node’s V8) exposes a TCP/IP port hosting a debugging protocol, and when run, it will pause execution until something attaches to it (i.e. connects to the port to start the debugging session).

At this point a debugging client (such as VS Code) will control the flow of the program, moving forward step-by-step, one line of code at a time. But how does it know which line of code it’s executing? When you run in debug mode, the binary code is instrumented with metadata about the line of code it represents. The debugger will tell the runtime, via the debugging protocol, “run until you reach line 109 in file data.ts, then pause and wait for a signal from me before continuing”.

While paused, the protocol also allows you to inspect the values of any variables that are in scope at that moment. A more sophisticated debugger will provide a REPL environment where you can execute ad-hoc code in the scope of the line where you are paused. This can be very powerful to inspect the values of various objects and to see in real-time what the effects of executing variations of code would be. After a debugging session you can take what you’ve learned and edit the code accordingly.

Example: Attaching a debugger to a TypeScript Pulumi program in VS Code

For this example, we will set up a new Pulumi project from the command-line and then set up VS Code to run that program in a debugger.

Step 1: Create a Pulumi project from a template

For this example, we will create a new Pulumi project using the TypeScript AWS static website template. Make sure you’ve set up Pulumi, logged into Pulumi Cloud, and configured your AWS credentials. Follow the prompts in the pulumi CLI to configure your project.

$ mkdir good-morning && cd good-morning
$ pulumi new static-website-aws-typescript

Then launch this project to make sure it works as expected:

$ pulumi up
...

Updating (dev)

View in Browser (Ctrl+O): https://app.pulumi.com/troy-pulumi-corp/good-morning/dev/updates/1

 Type Name Status
 + pulumi:pulumi:Stack good-morning-dev created (200s)
 + ├─ aws:s3:Bucket bucket created (1s)
 + ├─ aws:s3:BucketPublicAccessBlock public-access-block created (0.57s)
 + ├─ aws:s3:BucketOwnershipControls ownership-controls created (1s)
 + ├─ aws:cloudfront:Distribution cdn created (196s)
 + └─ synced-folder:index:S3BucketFolder bucket-folder created (0.41s)
 + ├─ aws:s3:BucketObject error.html created (0.46s)
 + └─ aws:s3:BucketObject index.html created (0.46s)

Outputs:
 cdnHostname : "##############.cloudfront.net"
 cdnURL : "https://##############.cloudfront.net"
 originHostname: "bucket-#######.s3-website-us-west-2.amazonaws.com"
 originURL : "http://bucket-#######.s3-website-us-west-2.amazonaws.com"

Resources:
 + 8 created

Great! You should be able to navigate to the website using the URL from the cdnURL output value.

$ open $(pulumi stack output cdnURL)

If all looks good, let’s move on to the next step by opening your project directory in VS Code:

$ code .

Step 2: Lets break stuff!

Now that we have a perfectly running Pulumi program, in order to have something to debug, we will need to break it!

For this example, we are going to change our indexDocument from index.html to good-morning.html.

$ cp ./wwww/index.html ./www/good-morning.html

Next, let’s edit the contents to say “Good morning, world!” instead of “Hello, world!”:

<!DOCTYPE html>
<html lang="en">
<head>
 <meta charset="utf-8">
 <title>Good morning, world!</title>
</head>
<body>
 <h1>Good morning, world! 👋</h1>
 <p>Deployed with 💜 by <a href="https://pulumi.com/">Pulumi</a>.</p>
</body>
</html>

Once that’s edited and saved, go ahead let’s redeploy the website:

$ pulumi up

Everything runs successfully… but if we navigate to the website, we don’t see our updated content?! Oh no! What went wrong? Why isn’t there an error message?! Time to put on our debugging hats and figure out what is going on. First, let’s set up VS Code to enable breakpoint debugging right in the IDE.

Step 3: Create a debug wrapper script

VS Code can run an arbitrary command when you hit F5. Typically this would be a one-liner that runs your language runtime like node foo.js or python foo.py. In our case, we need to run the pulumi CLI and need to do more than one step. The easiest way to do this is to create a small wrapper script and call that from VS Code.

In the root of the directory, create a file called pulumi-debug.sh:

#!/bin/bash

pulumi login
pulumi stack select dev
pulumi up -f

Let’s walk through these lines to explain what’s going on here:

pulumi login to set up credentials for the pulumi backend
pulumi stack select dev to run this against our dev stack
pulumi up -f to run our program update. The -f flag skips the preview phase and also skips the need to interactively confirm that we want to run the update.

Open the VS Code terminal panel and first set it to be executable, then try running the script:

$ chmod +x pulumi-debug.sh
$ ./pulumi-debug.sh

Ok now let’s set up VS Code to run that in debug mode when we hit F5!

Step 4: Configure VS Code `launch.json` and `tasks.json`

To run our Pulumi program in debug mode we will need to create a couple of VS Code configuration files, launch.json and tasks.json, in a hidden directory called .vscode within your project.

Create the .vscode directory and then create a new file called tasks.json file in that directory, with the following contents:

{
 "version": "2.0.0",
 "tasks": [
 {
 "label": "pulumi-debug",
 "type": "shell",
 "command": ". ./pulumi-debug.sh",
 "isBackground": true,
 "options": {
 "cwd": "${workspaceFolder}"
 }
 }
 ]
}

This creates a VS Code task named pulumi-debug which will run the wrapper script we just created from the root of our project.

Next, create a file named launch.json in the same directory. This will configure the behavior of the F5 key!

{
 "version": "0.2.0",
 "configurations": [
 {
 "name": "Launch Pulumi Program (debug)",
 "type": "node",
 "port": 9292,
 "request": "attach",
 "preLaunchTask": "pulumi-debug",
 "continueOnAttach": true,
 "restart": {
 "delay": 5000,
 "maxAttempts": 5
 },
 "skipFiles": [
 "${workspaceFolder}/node_modules/**/*.js",
 "${workspaceFolder}/lib/**/*.js",
 "<node_internals>/**/*.js"
 ]
 }
 ]
}

Let’s break this down a bit:

type: node tells VS Code that we are using the Node runtime
port: 9292 tells VS Code to attach the debugger to port 9292
request: attach tells VS Code that we want to attach to the debugger at launch
preLaunchTask: pulumi-debug tells VS Code to run the task we configured in tasks.json before we connect the debugger. This will run our pulumi-debug.sh script and then attach to the Node.js runtime
continueOnAttach: true tells VS Code to start running our code as soon as we attach to the waiting Node.js runtime (until we reach our first breakpoint that is, where it will pause and return control to the debugger client, aka VS Code)
restart: {...} configures VS Code to wait 5 seconds and then retry connecting, up to 5 times. This is here in case it takes a while for the Node.js runtime to start running. It’s unlikely, but possible!
skipFiles: {...} sets up some filters of files not to try to debug. This includes everything that is part of node’s internals, 3rdparty node modules, and other libraries. That way we don’t end up deep in someone else’s code unexpectedly and can just focus on our Pulumi program.
name sets a human-readable name which shows up in the drop down box in VS Code when you choose which [launch configuration][vs-code-launch-configs] to run.

Finally, we need to make one more edit to the pulumi-debug.sh file we created. Here we will pass the necessary options to the Node.js runtime to set it to run in debug mode.

Add the following line to the beginning of the file:

#!/bin/bash

export NODE_OPTIONS='--inspect-brk=127.0.0.1:9292'

pulumi login
pulumi stack select dev
pulumi up -f

The NODE_OPTIONS environment variable passes options to the Node.js V8 runtime. In this case, we are passing --inspect-brk which tells the Node.js runtime to run in debug mode, which will cause it to pause before running the code, until a debugging client connects to localhost on port 9292.

Ok, now we should be ready to run our Pulumi program from VS Code using the F5 key! Let’s give it a try. You should see Pulumi running inside of the “Terminal” tab. You will also see output indicating that it is waiting for a debugger to attach:

Debugger listening on ws://127.0.0.1:9292/########-####-####-####-############
For help, see: https://nodejs.org/en/docs/inspector
Debugger attached.
Waiting for the debugger to disconnect...

Step 5: Setting a debugging breakpoint

Now we can try setting a breakpoint in our code. Getting back to the problem at hand – our content doesn’t seem to be updating, despite no errors. Let’s start at the beginning, by investigating the inputs to our program, aka our configuration… at runtime!

Navigate to line 12 within the code and hit the F9 key. You’ll see a small red dot appear next to the line indicating that a breakpoint has been set. This line defines the bucket our program creates and seems like a good place to stop if we want to check the input configuration before we run the AWS S3 provider call.

Figure: A breakpoint set on line 12.

You can then run the program again using F5. When the execution of the code reaches the line it will pause there. At this point you can navigate around the code inspecting the state of the local variables, like you would in any other debugging session.

Let’s inspect the configuration variables. On the left side of the screen you’ll find a “Variables” panel listing all the local variables that are in scope.

Figure: Local variables displayed in the Variables side panel.

There’s also the “Debug” console which provides a REPL environment to navigate those variables and anything else that is in scope. You can run arbitrary functions here and see their output. In this case we can simply type the variable name at the prompt there, hit enter, and see the value. What’s great about the debug console is that Intellisense (e.g. using TAB to auto-complete) works here, so you can quickly navigate possible options.

Figure: Debug console tab completion and inspecting variable values.

In our case, we’re interested in the value of indexDocument variable. We can see that it is getting set to index.html… not good-morning.html. The Pulumi program didn’t error because index.html still exists. So we are still publishing our old index.html. That makes sense. A quick fix here is to modify index.ts and set the default to good-morning.html:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as synced_folder from "@pulumi/synced-folder";

// Import the program's configuration settings.
const config = new pulumi.Config();
const path = config.get("path") || "./www";
const indexDocument = config.get("indexDocument") || "good-morning.html";
const errorDocument = config.get("errorDocument") || "error.html";

...

Ok, we edited the document, which fixes the program for future runs, but for the current run that we are debugging, indexDocument is still set to index.html. Let’s use the debug console to set the value to good-morning.html before we continue the run.

> indexDocument="good-morning.html`

The output in the debug window will show the new value, and if you hover over the inputs being passed to the bucket creation call, it will now show as good-morning.html:

Figure: Screenshots of debug console and editor window after changing the indexDocument variable

When you’re ready for the program to continue on, hit F5 once again and the program will resume operation, either until the program exits or until it reaches the next breakpoint. Optionally you can step through line-by-line using the F10 key. VS Code provides a variety of debug actions once the session is established.

Most importantly, check the published website and make sure we fixed the issue! You can find the cdnURL output in the Terminal tab and click on it from there to open the link.

Considerations and Limitations

Breakpoint debugging is a powerful tool for inspecting the state of your Pulumi program. However, there are some important limitations to consider:

Pulumi programs are only part of the story: This technique will let you inspect the program you authored, but it doesn’t give access to what’s happening with the Pulumi backend, the providers, or anything else outside of the program itself. You might find yourself needing to dig deeper to resolve a problem with the stack as a whole. Luckily there are other techniques we can use for that, which we will delve into in an upcoming post.
Inputs and Outputs are asynchronous futures: If you’ve used Pulumi before you will have encountered Input<T> and Output<T> values, which are a core function of the tool. However, at the time that the debugger hits a breakpoint on those values, they are likely not populated with their final value. That means you won’t be able to inspect them in the debugger very well. Instead, you will need to add an apply(...) or all(..) block to the code, and place the breakpoint on the materialized value within that block. This follows all the same rules as accessing those variables in code in a Pulumi program.
```
bucket.id.apply(id => {
 // NOTE: The `debugger` keyword automatically sets a breakpoint here
 // so you don't need to set it via the IDE.
 debugger
});
```
Limited integration with VS Code: Unfortunately our technique of running Pulumi in wrapper script as a backgrounded pre-launch task in VS Code leaves some things to be desired. It is different enough from the use case that VS Code had in mind for these features (e.g. interacting with a long-running tsc-watch or similar task) that making things “just work” is a little difficult. For example, we can’t easily report on errors in the “Problems” tab, and VS Code complains that a problem matcher is missing the first time you run it. VS Code can’t tell that the program finished and so you’ll have to manually stop the debugging session after Pulumi has completed (or wait for the retry/timeout to expire).

Despite these limitations, Pulumi with VS Code as your IDE provides a rich experience compared to trawlling through unstructured logs, or adding in console.log(...) calls to echo out ad-hoc debugging information (aka. printf debugging). Also, you can enable this for any language that Pulumi supports, and most popular IDEs support breakpoint debugging (e.g. Visual Studio, Webstorm, neovim, etc). Python, Go, and C# all have popular debuggers and IDEs that can be used in the same manner. Whatever your preferred IDE or language is, you can use their full range of breakpoint debugging features for your Pulumi programs.

Pulumi also offers detailed logging which can be used to look into the other aspects that breakpoint debugging doesn’t cover. In an upcoming article we’ll show how to use these tools as well. Until then, have fun spelunking through your code with your favorite IDE!

Next Steps

If you haven’t already, install Pulumi today, and follow our self-directed Getting Started guides to learn more about making the most of Pulumi’s next-level infrastructure management features at your organization.

To learn more, you can watch the following video which provides a high level overview of how Pulumi works:

*{padding:0;margin:0;overflow:hidden}html,body{height:100%}img{position:absolute;width:100%;top:0;bottom:0;margin:auto}">

Pulumi Cloud

The Pulumi Cloud is a fully managed service that helps you adopt Pulumi’s open source SDK with ease. It provides built-in state and secrets management, integrates with source control and CI/CD, and offers a web console and API that make it easier to visualize and manage infrastructure. It is free for individual use, with features available for teams.

Create an Account

Next-level IaC: How Pulumi Supports Your API Economy Strategy

Engin Diri — Wed, 15 May 2024 00:00:00 +0000

When I am talking with community members, who are not using Pulumi yet, I often get asked what would be a good way to include their Infrastructure as Code (IaC) into existing software like a REST API. And my answer is always the same: Use our Pulumi Automation API.

With the Pulumi Automation API you can include Pulumi IaC into your existing software, and this for any of the Pulumi supported programming languages like TypeScript, Python, Go, or C#. This gives you a greater flexibility and control, which you will not have with other IaC tools like CloudFormation or Terraform.

This opens up many doors of opportunity to expose your services as an API, whether internally to your organization or externally to your potential customers. This opportunity for innovation fuels what is known as the API Economy.

What is API Economy?

As a matter of fact: — APIs enable innovation.

Over 40% of large organizations have 250+ APIs. And 71% plan to use even more APIs this year.

API Economy is a term that is used to describe the way that organizations are using APIs to create new business models and revenue streams. APIs are the building blocks of modern software development, and they allow developers to build on top of existing services and create new services that are greater than the sum of their parts.

Major international corporations leveraging the API economy through robust API strategies include Google, Amazon, Stripe, Twilio, Slack, and many other renowned brands. Google’s initial Maps service, for instance, has evolved into a comprehensive platform offering Maps, Routes, and Places APIs and SDKs. These tools enable companies to incorporate both dynamic and static maps into their applications and websites, offer navigational guidance, access 360° Street View images, utilize geolocation services, and more. This transformation exemplifies the concept of value multiplication in the API economy, as described by Gartner Vice President Kristin Moyer:

“The API economy is an enabler for turning a business or organisation into a platform. Platforms multiply value creation because they enable business ecosystems inside and outside of the enterprise to consummate matches among users and facilitate the creation and exchange of goods, services and social currency so that all participants are able to capture value.”

What is Pulumi Automation API in a nutshell?

Pulumi Automation API is a set of SDKs that allow you to interact with your Pulumi stacks programmatically. This means that you can use the Automation API to create, update, and delete stacks, as well as to query their current state. The Automation API is built on top of the Pulumi engine, which means that it is fully compatible with all of the existing Pulumi providers and resources.

And it is this capability that allows you to build your own Service API on top of Pulumi. We can now build a REST or gRPC API endpoint that, depending on the request parameters, will create, update, or delete a Pulumi stack. This API can then be exposed to the rest of the organization, allowing developers to interact with their infrastructure in a programmatic way.

How to build your own Service API with Pulumi Automation API?

Let’s take a look at how you can build your own Service API with Pulumi Automation API.

For this purpose, we will build two basic REST services written in TypeScript and in Python.

import express from "express"
import * as dotevnv from "dotenv"
import cors from "cors"
import helmet from "helmet"
import {fullyQualifiedStackName, RemoteWorkspace} from "@pulumi/pulumi/automation";

dotevnv.config()

if (!process.env.PORT) {
 console.log(`No port value specified...`)
}

const PORT = parseInt(process.env.PORT as string, 10)

const app = express()

app.use(express.json())
app.use(express.urlencoded({extended: true}))
app.use(cors())
app.use(helmet())

// Define the payload interface
interface ServicePayload {
 name: string;
 org: string;
 project: string;
 stack: string;
 region: string;
}

// Define a helper function to set up the Pulumi program
async function setUpPulumiProgram(payload: ServicePayload) {
 const org = payload.org;
 const project = payload.project;
 const stackName = fullyQualifiedStackName(org, project, payload.stack);
 const awsRegion = payload.region;

 // Using the Pulumi Automation API to create or select a stack in the remote workspace
 // and set the stack's configuration and secrets. This will use Pulumi Deployments to
 // deploy the stack to the specified AWS region.
 return await RemoteWorkspace.createOrSelectStack({
 stackName,
 url: "https://github.com/pulumi/examples.git",
 branch: "refs/heads/master",
 projectPath: project,
 }, {
 envVars: {
 AWS_REGION: awsRegion,
 AWS_ACCESS_KEY_ID: process.env.AWS_ACCESS_KEY_ID ?? "",
 AWS_SECRET_ACCESS_KEY: {secret: process.env.AWS_SECRET_ACCESS_KEY ?? ""},
 AWS_SESSION_TOKEN: {secret: process.env.AWS_SESSION_TOKEN ?? ""},
 },
 });
}

// Define the API endpoint for creating a new stack in the remote workspace.
app.post('/', async (req, res) => {
 const payload: ServicePayload = req.body;

 if ((payload.name) || !payload.org || !payload.project || !payload.stack || !payload.region) {
 res.status(400).send('Name, org, project, stack and region are required');
 return;
 }
 // Create a new stack in the remote workspace and set the stack's configuration and secrets.
 const stack = await setUpPulumiProgram(payload);

 // Deploy the stack to the specified AWS region. We'll print the stack's outputs to the console.
 const upRes = await stack.up({onOutput: console.log});
 res.status(200).send(`url: ${upRes.outputs.websiteUrl.value}`);
});

// Define the API endpoint for destroying a stack in the remote workspace.
app.delete('/', async (req, res) => {
 const {name, org, project, stack, region} = req.query;

 if (!name || !org || !project || !stack || !region) {
 res.status(400).send('Name, org, project, stack, and region are required');
 return;
 }

 const payload: ServicePayload = {
 name: name as string,
 org: org as string,
 project: project as string,
 stack: stack as string,
 region: region as string
 }

 // Define the stack and set the stack's configuration and secrets.
 const prog = await setUpPulumiProgram(payload);
 // Destroy the stack in the remote workspace. We'll print the stack's outputs to the console.
 await prog.destroy({onOutput: console.log});
 res.status(200).send('Stack destroyed');
});


app.listen(PORT, () => {
 console.log(`Server is listening on port ${PORT}`)
})

import os
import pulumi.automation as auto
from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()


# Define the payload that the API will receive. This payload will contain the name of the stack
# to create, the organization and project to use, the stack name, and the AWS region to deploy the stack to.
class ServicePayload(BaseModel):
 name: str
 org: str
 project: str
 stack: str
 region: str


# Define a helper function that will create a new stack in the remote workspace and set the stack's configuration
def setup_pulumi_program(payload: ServicePayload):
 org = payload.org
 project = payload.project
 stack_name = auto.fully_qualified_stack_name(org, project, payload.stack)
 aws_region = payload.region

 # Using the Pulumi Automation API to create or select a stack in the remote workspace
 # and set the stack's configuration and secrets. This will use Pulumi Deployments to
 # deploy the stack to the specified AWS region.
 return auto.create_or_select_remote_stack_git_source(
 stack_name=stack_name,
 url="https://github.com/pulumi/examples.git",
 branch="refs/heads/master",
 project_path=project,
 opts=auto.RemoteWorkspaceOptions(
 env_vars={
 "AWS_REGION": aws_region,
 "AWS_ACCESS_KEY_ID": os.environ["AWS_ACCESS_KEY_ID"],
 "AWS_SECRET_ACCESS_KEY": auto.Secret(os.environ["AWS_SECRET_ACCESS_KEY"]),
 "AWS_SESSION_TOKEN": auto.Secret(os.environ["AWS_SESSION_TOKEN"]),
 },
 ),
 )


# Define the API endpoints for creating and deleting a stack in the remote workspace.
@app.post("/")
async def create(payload: ServicePayload):
 # Create a new stack in the remote workspace and set the stack's configuration and secrets.
 stack = setup_pulumi_program(payload)
 # Deploy the stack to the specified AWS region. We'll print the stack's outputs to the console.
 up_res = stack.up(on_output=print)
 return {"url": up_res.outputs['websiteUrl'].value}


# Define the API endpoint for deleting a stack in the remote workspace.
@app.delete("/")
async def delete(name: str, org: str, project: str, stack: str, region: str):
 payload = ServicePayload(name=name, org=org, project=project, stack=stack, region=region)
 # Define the stack and set the stack's configuration and secrets.
 stack = setup_pulumi_program(payload)
 # Destroy the stack in the remote workspace. We'll print the stack's outputs to the console.
 stack.destroy(on_output=print)
 return {"message": "Stack destroyed"}

Let us run the above Python example in a FastAPI server to see how the Automation API works in practice.

Let’s start by creating a new Python file called main.py and pasting the above code into it. Next, we need to install the required dependencies:

python3 -m venv ven
source venv/bin/activate

Create a new file called requirements.txt and paste the following dependencies into it:

pulumi
fastapi
pydantic

We need to install the dependencies using the following command:

pip install -r requirements.txt --force-reinstall

Finally, we can run the FastAPI server using the following command:

fastapi dev main.py

This will start the FastAPI server on port 8000.

INFO Using path main.py
INFO Resolved absolute path /private/tmp/automation-api-example/python/main.py
INFO Searching for package file structure from directories with __init__.py files
INFO Importing from /private/tmp/automation-api-example/python

 ╭─ Python module file ─╮
 │ │
 │ 🐍 main.py │
 │ │
 ╰──────────────────────╯

INFO Importing module main
INFO Found importable FastAPI app

 ╭─ Importable FastAPI app ─╮
 │ │
 │ from main import app │
 │ │
 ╰──────────────────────────╯

INFO Using import string main:app

 ╭────────── FastAPI CLI - Development mode ───────────╮
 │ │
 │ Serving at: http://127.0.0.1:8000 │
 │ │
 │ API docs: http://127.0.0.1:8000/docs │
 │ │
 │ Running in development mode, for production use: │
 │ │
 │ fastapi run │
 │ │
 ╰─────────────────────────────────────────────────────╯

INFO: Will watch for changes in these directories: ['/private/tmp/automation-api-example/python']
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO: Started reloader process [55849] using WatchFiles
INFO: Started server process [55851]
INFO: Waiting for application startup.
INFO: Application startup complete.

To test the REST services, you can use a tool like curl or Postman. Here is an example of how you can create a new stack using curl:

curl -X POST -H "Content-Type: application/json" -d '{"name": "myexample", "org": "myorg", "project": "myproject", "stack": "mystack", "region": "us-west-2"}' http://localhost:3000

In the logs of the FastAPI server, you should see following output from the Automation API:

INFO: Create stack
Preparing deployment...

View Live: https://app.pulumi.com/dirien/aws-ts-s3-folder/dev/deployments/3

Setup:
Preparing environment
Pulling container image "pulumi/pulumi:3.115.2"
3.115.2: Pulling from pulumi/pulumi
Digest: sha256:a7b9022d7c1a40d81adeca30a21eb1cd49add474e3de923ade663db046e7ee1a
Status: Downloaded newer image for pulumi/pulumi:3.115.2
Starting container
Container created (6b92928bdcdfe6e54286dcdcb622480004f18c98b23b5b60775f71959a423708)
Container started (6b92928bdcdfe6e54286dcdcb622480004f18c98b23b5b60775f71959a423708)

Download deployment executor:
$ curl --retry 5 -H "Authorization: token $PULUMI_ACCESS_TOKEN" "$PULUMI_SERVICE_URL/deployments/executor" -o /pulumi-deploy-executor.gz && gzip -d /pulumi-deploy-executor.gz && chmod +x /pulumi-deploy-executor
 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
100 15.9M 0 15.9M 0 0 42.2M 0 --:--:-- --:--:-- --:--:-- 42.4M

Get source:
$ /pulumi-deploy-executor getsource git --repoURL="https://github.com/pulumi/examples.git" --workDir="/deployment" --branch="refs/heads/master" --repoDir="aws-ts-s3-folder"
Successfully cloned: https://github.com/pulumi/examples.git

Download dependencies:
$ /pulumi-deploy-executor dependencies download --workDir="/deployment/aws-ts-s3-folder"
npm WARN deprecated @opentelemetry/api-metrics@0.32.0: Please use @opentelemetry/api >= 1.3.0

added 294 packages, and audited 295 packages in 14s

43 packages are looking for funding
 run `npm fund` for details

found 0 vulnerabilities
npm notice
npm notice New major version of npm available! 9.6.7 -> 10.7.0
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.7.0>
npm notice Run `npm install -g npm@10.7.0` to update!
npm notice

pulumi update:
$ /pulumi-deploy-executor pulumi update --stackIdentity="dirien/aws-ts-s3-folder/dev" --workDir="/deployment/aws-ts-s3-folder"
Updating (dev)

View Live: https://app.pulumi.com/dirien/aws-ts-s3-folder/dev/updates/3


 + pulumi:pulumi:Stack aws-ts-s3-folder-dev creating (0s)
 + aws:s3:Bucket s3-website-bucket creating (0s)
 + aws:s3:Bucket s3-website-bucket created (4s)
 + aws:s3:BucketObject favicon.png creating (0s)
 + aws:s3:BucketObject index.html creating (0s)
 + aws:s3:BucketPublicAccessBlock public-access-block creating (0s)
 + aws:s3:BucketObject index.html created (0.98s)
 + aws:s3:BucketObject favicon.png created (1s)
 + aws:s3:BucketPublicAccessBlock public-access-block created (1s)
 + aws:s3:BucketPolicy bucketPolicy creating (0s)
 + aws:s3:BucketPolicy bucketPolicy created (0.83s)
 + pulumi:pulumi:Stack aws-ts-s3-folder-dev created (18s)
Outputs:
 bucketName: "s3-website-bucket-d55e9bd"
 websiteUrl: "s3-website-bucket-d55e9bd.s3-website.eu-central-1.amazonaws.com"

Resources:
 + 6 created

Duration: 20s

Result: succeeded
INFO: 127.0.0.1:49220 - "POST / HTTP/1.1" 200 OK

You should see the URL of the created infrastructure in the response from the FastAPI server. If you head over to the Pulumi console, you should see the created stack and the resources that were created and also that Pulumi Deployments were used to deploy the stack to the specified AWS region.

And here is an example of how you can delete a stack using curl:

curl -X DELETE "http://localhost:8000/?name=myexample&org=myorg&project=myproject&stack=mystack&region=us-west-2"

In the logs of the FastAPI server, you should see the following output from the Automation API:

INFO: Deleting stack
Preparing deployment...

View Live: https://app.pulumi.com/dirien/aws-ts-s3-folder/dev/deployments/4

Setup:
Preparing environment
Pulling container image "pulumi/pulumi:3.115.2"
3.115.2: Pulling from pulumi/pulumi
Digest: sha256:a7b9022d7c1a40d81adeca30a21eb1cd49add474e3de923ade663db046e7ee1a
Status: Downloaded newer image for pulumi/pulumi:3.115.2
Starting container
Container created (26af871d4e26a6dab5656a1c8dc8a7e87bf247f42bde77e2269c269f15054e73)
Container started (26af871d4e26a6dab5656a1c8dc8a7e87bf247f42bde77e2269c269f15054e73)

Download deployment executor:
$ curl --retry 5 -H "Authorization: token $PULUMI_ACCESS_TOKEN" "$PULUMI_SERVICE_URL/deployments/executor" -o /pulumi-deploy-executor.gz && gzip -d /pulumi-deploy-executor.gz && chmod +x /pulumi-deploy-executor
 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
100 15.9M 0 15.9M 0 0 43.5M 0 --:--:-- --:--:-- --:--:-- 43.5M

Get source:
$ /pulumi-deploy-executor getsource git --repoURL="https://github.com/pulumi/examples.git" --workDir="/deployment" --branch="refs/heads/master" --repoDir="aws-ts-s3-folder"
Successfully cloned: https://github.com/pulumi/examples.git

Download dependencies:
$ /pulumi-deploy-executor dependencies download --workDir="/deployment/aws-ts-s3-folder"
npm WARN deprecated @opentelemetry/api-metrics@0.32.0: Please use @opentelemetry/api >= 1.3.0

added 294 packages, and audited 295 packages in 14s

43 packages are looking for funding
 run `npm fund` for details

found 0 vulnerabilities
npm notice
npm notice New major version of npm available! 9.6.7 -> 10.7.0
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.7.0>
npm notice Run `npm install -g npm@10.7.0` to update!
npm notice

pulumi destroy:
$ /pulumi-deploy-executor pulumi destroy --stackIdentity="dirien/aws-ts-s3-folder/dev" --workDir="/deployment/aws-ts-s3-folder"
Destroying (dev)

View Live: https://app.pulumi.com/dirien/aws-ts-s3-folder/dev/updates/4


 - aws:s3:BucketPolicy bucketPolicy deleting (0s)
 - aws:s3:BucketPolicy bucketPolicy deleted (1s)
 - aws:s3:BucketPublicAccessBlock public-access-block deleting (0s)
 - aws:s3:BucketObject favicon.png deleting (0s)
 - aws:s3:BucketObject index.html deleting (0s)
 - aws:s3:BucketObject index.html deleted (0.61s)
 - aws:s3:BucketObject favicon.png deleted (0.73s)
 - aws:s3:BucketPublicAccessBlock public-access-block deleted (0.77s)
 - aws:s3:Bucket s3-website-bucket deleting (0s)
 - aws:s3:Bucket s3-website-bucket deleted (0.79s)
 - pulumi:pulumi:Stack aws-ts-s3-folder-dev deleting (0s)
 - pulumi:pulumi:Stack aws-ts-s3-folder-dev deleted (0.10s)
Outputs:
 - bucketName: "s3-website-bucket-d55e9bd"
 - websiteUrl: "s3-website-bucket-d55e9bd.s3-website.eu-central-1.amazonaws.com"

Resources:
 - 6 deleted

Duration: 7s

The resources in the stack have been deleted, but the history and configuration associated with the stack are still maintained.
If you want to remove the stack completely, run `pulumi stack rm dev`.
Result: succeeded
INFO: 127.0.0.1:51509 - "DELETE /?name=my-test&org=dirien&project=aws-ts-s3-folder&stack=dev&region=eu-central-1 HTTP/1.1" 200 OK

Next Steps

Now that we have our basic REST services, we can deploy them on a server (using a service like AWS Lambda, Google Cloud Functions, or Azure Functions) and expose them to the rest of the organization via an API Gateway. In parallel you can publish release notes and documentation to your organization, so that they can start using your new Service API.

One way could be to create an OpenAPI specification for your Service API and use a tool like Swagger to generate a client library for your organization. This way, developers can interact with your Service API in a programmatic way, without having to write any code.

Here is an example of an OpenAPI specification for our Service API:

openapi: 3.0.0
info:
 title: My Organization Service API
 description: API for creating and destroying a specific piece of infrastructure
 version: 1.0.0
servers:
- url: 'http://localhost:8000/'
 description: Development server
paths:
 /:
 post:
 summary: Create Infrastructure
 operationId: createInfrastructure
 requestBody:
 required: true
 content:
 application/json:
 schema:
 $ref: '#/components/schemas/ServicePayload'
 responses:
 '200':
 description: Infrastructure successfully created
 content:
 application/json:
 schema:
 type: object
 properties:
 url:
 type: string
 description: URL of the created infrastructure
 '400':
 description: Bad request if the input payload is incorrect
 '500':
 description: Internal server error
 delete:
 summary: Destroy Infrastructure
 operationId: destroyInfrastructure
 parameters:
 - in: query
 name: name
 schema:
 type: string
 required: true
 description: Name of the service
 - in: query
 name: org
 schema:
 type: string
 required: true
 description: Organization in Pulumi
 - in: query
 name: project
 schema:
 type: string
 required: true
 description: Project name in Pulumi
 - in: query
 name: stack
 schema:
 type: string
 required: true
 description: Stack name in Pulumi
 - in: query
 name: region
 schema:
 type: string
 required: true
 description: AWS region where the service will be deployed
 responses:
 '200':
 description: Infrastructure successfully destroyed
 content:
 application/json:
 schema:
 type: object
 properties:
 message:
 type: string
 description: Confirmation message of the destruction
 '400':
 description: Bad request if the required parameters are missing
 '500':
 description: Internal server error
components:
 schemas:
 ServicePayload:
 type: object
 required:
 - name
 - org
 - project
 - stack
 - region
 properties:
 name:
 type: string
 description: Name of the service
 org:
 type: string
 description: Organization in Pulumi
 project:
 type: string
 description: Project name in Pulumi
 stack:
 type: string
 description: Stack name in Pulumi
 region:
 type: string
 description: AWS region where the service will be deployed

Navigate to the Swagger Editor and paste the above OpenAPI specification to see how it looks.

Conclusion

In this blog post, we’ve explored how Pulumi’s Automation API can be used to build and expose a Service API within your organization. By embedding Pulumi as a library in any application, you can drive your Infrastructure as Code (IaC) deployments directly through code. This sets Pulumi apart from other tools like Terraform, which relies on static configuration files and a command-line interface.

Additionally, we touched on the concept of the API economy, where APIs are leveraged to connect services, accelerate product delivery, and drive business value through the possibilities of creating new touchpoints. The Pulumi Automation API is the perfect piece of technology, which makes it now possible to define, deploy, and manage infrastructure programmatically as part of your API strategy.

I hope this blog post has inspired you with ideas on how to take your IaC to the next level, making it more accessible and practical for your organization.

Next Steps

To learn more, you can watch the following video which provides a high level overview of how Pulumi works:

*{padding:0;margin:0;overflow:hidden}html,body{height:100%}img{position:absolute;width:100%;top:0;bottom:0;margin:auto}">

Pulumi Cloud

Create an Account

Adding data to Pinecone using S3, Embedchain and Pulumi on AWS for an AI Slack bot

Tyler Mulligan — Wed, 10 Apr 2024 17:00:00 +0100

In the introductory blog post, we learned to Create an AI Slack Bot to Chat with Your Data Using Embedchain, Pulumi on AWS. However, we made a few concessions in the application logic to illustrate the broader picture of what we were able to achieve combining these three technologies. Now that we have a solid foundation for deploying our Slack bot and querying our data, lets begin moving from proof-of-concept to production-ready, iteratively.

For reference, here’s what our architecture looked like at the end of the first iteration.

To follow along, clone the project, git clone https://github.com/catmeme/arti.git

Configuration

In our proof-of-concept, we defined our Embedchain config in our Python code. Best practices dictate strict separation of config from code, so the first thing we must address is getting the embedchain configuration out of our code.

A config/config.yaml made from our dict, is much easier to read, maintain and share.

app:
 config:
 id: arti
 collect_metrics: False
 log_level: INFO
llm:
 provider: openai
 config:
 model: 'gpt-3.5-turbo-1106'
 temperature: 0.5
 max_tokens: 1000
 top_p: 1
embedder:
 provider: openai
chunker:
 chunk_size: 2000
 chunk_overlap: 0
 length_function: len
vectordb:
 provider: chroma
 config:
 collection_name: arti-ai
 dir: /tmp/db
 allow_reset: True

Our goal is to replace our big inline dict, and read a YAML file instead, following this configuration specification.

def ask_ai(question):
 """Ask the AI a question."""
 print(f"Processing question: {question}")

 app_config = {
 # hardcoded configuration we're moving out to YAML
 }

 loader = DirectoryLoader(config={"recursive": True})
 app = App.from_config(config=app_config)
 app.add("assets", loader=loader)

 return app.query(question)

To do that, though, we don’t want to implement it directly as we already have a config module, src/arti_ai/config.py. Let’s define a new method to read that YAML file, and pass it to our application at startup.

class Config:
 def __init__(self, config_file="config/config.yaml", secrets_manager_client=None):
 self.secrets_manager_client = secrets_manager_client or boto3.client("secretsmanager")
 self.project_root = Path(__file__).resolve().parents[2]
 self.config_file = self.project_root / config_file
 load_dotenv()

 def get_embedchain_config(self):
 try:
 with open(self.config_file, 'r') as file:
 return yaml.safe_load(file)
 except FileNotFoundError as e:
 raise FileNotFoundError(f"Configuration file not found: {self.config_file}") from e
 except yaml.YAMLError as e:
 raise yaml.YAMLError(f"Error parsing YAML configuration: {self.config_file}") from e

Our app.py function simplifies, becoming more readable and flexible, we’ve even fixed a bug, making our application more portable with config.project_root, now arti can be run from any directory.

def ask_ai(question: str):
 """Ask the AI a question."""
 print(f"Processing question: {question}")

 from embedchain import App
 from embedchain.loaders.directory_loader import DirectoryLoader

 loader = DirectoryLoader(config={"recursive": True})
 app = App.from_config(config=config.get_embedchain_config()) # inline dict has become this
 app.add(f"{config.project_root}/assets", loader=loader)

 return app.query(question)

Now that we’ve separated out our configuration, we can have multiple scenarios preconfigured. In the repository, there are a few examples, including a private ai configuration, so that embedchain can be run 100% locally on your machine using gpt4all. The configuration difference boils down to updating the LLM and Embedder to open-source models.

llm:
 provider: gpt4all
 config:
 model: 'orca-mini-3b-gguf2-q4_0.gguf'
 max_tokens: 1000
 top_p: 1
embedder:
 provider: huggingface
 config:
 model: 'sentence-transformers/all-MiniLM-L6-v2'

Now that we have a dynamic approach to loading configuration from YAML, lets move on to improving the application architecture, which was not built to scale beyond relatively small data sets.

Application architecture

Knowing your limits

The current application architecture uses a single process with the expectation that a response is generated within 29 seconds, before the API Gateway timeout.

Slack also has an expectation, that we must acknowledge (or ack) a request within 3 seconds.

Currently, when the application receives a request, the API Gateway Lambda integration starts the application. The lifecycle of this single process approach is diagrammed below.

If we break this into two processes, Data Processing, and API Handling, we give our API handler more dedicated time within that 29 seconds. Simultaneously, we overcome another limitation, large amounts of data being loaded will take more than 29 seconds, by breaking out data processing into its own process, we can now have up to 15 minutes to load data using Lambda.

We’re going to use the same docker image for both Lambdas by adding an entrypoint script that can take an argument to select the handler, essentially deploying the same application twice as different Lambda Functions.

Our entrypoint.sh

#!/usr/bin/env sh
set -e

. /app/venv/bin/activate

if [ -n "$LAMBDA_HANDLER" ]; then
 /app/venv/bin/python -m awslambdaric $LAMBDA_HANDLER
else
 arti "$@"
fi

Here’s what the two process lifecycles look like separated

We have, however, introduced a new issue, and that is persistence. Our single process approach always loads the data, and warm requests will have the data cached, so it was okay that the Chroma database was ephemeral, it could always be recreated on the fly.

When our corpus is many gigabytes or terabytes, losing and recreating those embeddings for the vector database is a costly endeavor, both in time, and very likely monetarily.

How do we solve for that?

Multi-tier architecture and data persistence

By putting our vector database into another tier, we don’t couple its lifecycle with our application’s. It’s pretty cool that we can have an ephemeral vector database in Lambda for demonstration purposes, but it’s not an optimal configuration for production.

One option, is to deploy Chroma ourselves using a service such as EC2 or ECS, for example, but that incurs costs. For the ease of this demonstration, we’re going to use Pinecone, which Pulumi conveniently has a provider for. Though, there are many alternative options.

Navigate to Pinecone’s homepage, and click “Get started.” After confirming your account, login to Pinecone, and click “API Keys” in the sidebar. Create a new API key for Arti.

After configuring our Pinecone API key in AWS Secrets Manager, we are able to define our Pinecone index in our __main__.py Pulumi program.

import pulumi
import pinecone_pulumi as pinecone

pinecone_index = pinecone.PineconeIndex(
 f"{app_name}-pinecone-index",
 name="main-index",
 dimension=1536,
 metric=pinecone.IndexMetric.COSINE,
 spec=pinecone.PineconeSpecArgs(
 pod=pinecone.PineconePodSpecArgs(
 environment="gcp-starter",
 pod_type="starter",
 replicas=1,
 )
 ),
)

pulumi.export("output", {"value": pinecone_index.host})

Handling data load events

In the proof-of-concept, we are abusing docker to distribute our data sources, this isn’t optimal for a few reasons, the size of the docker image increases, which also increases build times. Do our documents really belong in the container, what about privacy, triggering builds, lead time to get new data added, etc.?

The obvious next place to put our documents is AWS S3 (Simple Storage Service). We don’t want to regress our existing implementation, which works locally as well as deployed to Lambda, so we abstract the concept of loading a path of assets, whether it’s a local path or S3. In the presence of an S3 bucket name, assume we intend to use that loader, otherwise use the directory loader.

def get_loader_and_asset_root():
 """Select and configures the appropriate loader based on AWS S3 bucket presence."""
 app_bucket_name = os.getenv("APP_BUCKET_NAME")
 root_path = os.getenv("ASSETS_ROOT_PATH", "assets")
 asset_location = root_path

 # pylint: disable=import-outside-toplevel
 if app_bucket_name:
 from arti_ai.s3_loader import S3BucketLoader # type: ignore

 loader = S3BucketLoader()
 asset_location = f"{app_bucket_name}/{root_path}"
 else:
 from embedchain.loaders.directory_loader import DirectoryLoader # type: ignore

 loader = DirectoryLoader(config={"recursive": True})

 # pylint: enable=import-outside-toplevel

 return loader, asset_location


def load_data():
 """Load data from data sources."""
 print("Loading data")

 # pylint: disable=import-outside-toplevel
 from embedchain import App

 # pylint: enable=import-outside-toplevel

 app = App.from_config(config=config.get_embedchain_config())

 loader, asset_location = get_loader_and_asset_root()
 response = app.add(asset_location, loader=loader)

 return response

With our data loading logic figured out, we’ll pass the APP_BUCKET_NAME in our Pulumi program to our Lambda Function, so it can retrieve the data.

app_bucket = aws.s3.Bucket(
 f"{app_name}-bucket",
 bucket=f"{app_name}-bucket",
 # ...
)

lambda_environment = pulumi.Output.all(
 app_bucket.id,
).apply(
 lambda args: {
 "variables": {
 # ...
 "APP_BUCKET_NAME": args[0],
 # ...
 }
 }
)

app_event_lambda = aws.lambda_.Function(
 f"{app_name}-lambda",
 architectures=["x86_64"],
 environment=lambda_environment,
 image_uri=docker_image_lambda_event_handler.repo_digest,
 # ...
)

If you’re new to Pulumi, you might be wondering what’s going on with the lambda_environment, why isn’t app_bucket.id referenced directly, and what is pulumi.Output.all? Simply put, this tells Pulumi to wait until the bucket is created before trying to use its name in a string we’re building. This is an important core feature of Pulumi that largely happens by nesting Outputs. To learn more, see Inputs & Outputs.

Triggering the data loader

When deployed, we’re going to need something that doesn’t require a local machine. One method for loading data is to use a scheduled task. AWS EventBridge makes that task simple, our Pulumi resource declarations are below. We define a rule/schedule (pulled from stack configuration), add our target (the event handler Lambda), and setup permissions to invoke the Lambda function.

event_rule_schedule = aws.cloudwatch.EventRule(
 f"{app_name}-event-rule-schedule",
 schedule_expression=data_load_event_rule_schedule,
 state=data_load_event_rule_state,
)

event_rule_target = aws.cloudwatch.EventTarget(
 f"{app_name}-event-target-schedule",
 rule=event_rule_schedule.name,
 arn=app_event_lambda.arn,
)

event_rule_target_permission = aws.lambda_.Permission(
 f"{app_name}-lambda-event-target-permissions-schedule",
 action="lambda:InvokeFunction",
 function=app_event_lambda.name,
 principal="events.amazonaws.com",
 source_arn=event_rule_schedule.arn,
)

Our event lambda handler looks something like

import json

from arti_ai.app import load_data


def handler(event, context):
 print("Received event: " + json.dumps(event, indent=2))
 print("Context AWS request ID: " + json.dumps(context, indent=2))

 load_data()

 return {"statusCode": 200, "body": json.dumps("Event processed successfully!")}

Locally, we can use our command-line interface (CLI), defined in __main__.py

if args.command == "ask":
 handle_ask(args.question)
elif args.command == "load":
 load_data()
elif args.command == "reset":
 reset_data()
else:
 parser.print_help()

Any of our YAML configuration files should work with the following commands:

arti ask "what can you tell me about fruits and vegetables?"
arti load
arti reset
arti -h

You could manage your live Pinecone data this way, whether you should is highly dependent on the use case.

Wrapping up

We reflected on the project that was presented in the original post, and reviewed short-comings with the proof-of-concept, working through improvements that touched upon architecture, application design, and infrastructure. Specifically, we have moved application configuration out of code into YAML, decoupled data-loading from querying, broken out our vector database to another tier, and integrated our application with AWS EventBridge to trigger scheduled tasks.

Our delivery on the concept was expedited by utilizing services that aligned with our improvements, and Pulumi helped us achieve our architecture changes using familiar patterns in just a few lines of code.

In the next post in this series, we’ll continue to make iterative improvements to both our applications and infrastructure, and start getting deeper into RAG.

As always, we welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is easy and free. Get started today!

Happy AI building!