Pulumi Blog: Python

Deploy a Private Hermes Agent on Render Securely with Pulumi, Modal, and Tailscale

Engin Diri — Wed, 24 Jun 2026 00:00:00 +0000

Personal AI agents had their breakout this year. OpenClaw crossed 100,000 GitHub stars within months of launching, and self-hosting your own assistant went from a hobbyist trick to something a lot of developers actually do. I wrote up how to deploy that lobster to AWS or Hetzner back when it was everywhere.

The one people are switching to now is Hermes, the open-source runtime from Nous Research, and it caught on just as quickly. The reason shows up in every “I ditched OpenClaw for Hermes” thread: it actually learns, building up memory and writing its own skills as it goes instead of running off a static, human-written list.

Here is the part the launch videos skip. Hermes writes and runs its own code, with no human approving the commands. A model that can write code will eventually write a bad one, and the only thing between that command and your credentials is the sandbox it runs in. That is the box you do not want on the public internet. Researchers found 175,000 exposed Ollama servers sitting open in early 2026, and attackers hijack the ones they find for compute. The fix is not a better lock on the front door. It is to have no front door at all.

So this post deploys a private Hermes agent as one Pulumi program across Render, Modal, and Tailscale. The agent and its chat UI run as Render private services with no public URL, Tailscale puts the UI on your tailnet, and Modal runs the agent’s code in throwaway sandboxes. One pulumi up to stand it up, one pulumi destroy to tear it down, around $50 a month. No Makefile, no CLI escape hatches, and an honest look at the few places “nothing on the public internet” still leaks.

What is Hermes?

Hermes is the open-source, MIT-licensed agent runtime from Nous Research. It runs continuously on a server rather than living in a browser tab, and you talk to it through a chat UI or a messaging platform like Telegram, Discord, Slack, or Signal. A few things make it more than a chatbot:

It is model-agnostic. You point it at any provider. This deployment uses OpenRouter, one key that fronts 400-plus models across providers, with automatic failover when one is down.
It remembers across restarts. Memory persists on disk, so it builds up context over time instead of resetting every session.
It writes its own skills. When it works through a task, it can save that approach as a reusable skill and reach for it next time.
It schedules itself. A built-in cron runs automations on a schedule, driven in plain language.
It runs code in a sandbox. When the agent needs to execute code, it hands that off to an isolated, ephemeral container instead of running it next to its own process.

The difference from a cloud-hosted assistant is the usual self-hosting trade: it runs on your infrastructure with your keys, and your conversation history and memory stay on your disk.

“Hermes” here is the agent runtime from Nous Research, not the Hermes family of language models that share the name. The agent is model-agnostic, and the model it talks to is a separate choice you make through OpenRouter.

Prerequisites

Before getting started, ensure you have:

Pulumi CLI installed and configured
A Pulumi Cloud account
A Render account, an API key, and your workspace owner ID
A Modal account and a token pair (token ID and secret)
An OpenRouter API key with credit on the account
A container registry you can push to; this post uses GitHub Container Registry (a GHCR username and token)
Docker running locally (the image build rides on a local buildx engine), or a Docker-enabled CI runner
A Tailscale account with HTTPS enabled and a reusable auth key
Node.js 18+ (for TypeScript) or Python 3.9+ (for Python), depending on the language you pick

This guide routes models through OpenRouter, but Hermes works with other providers too. The model is just a routing string you pass (gpt-5.5 here), so any model OpenRouter supports works by changing that one value.

Understanding the Hermes architecture

Strip away the dashboard clicks and the architecture is four pieces. One rule governs all of them: what the public internet is allowed to touch.

flowchart LR
User([Your devices]) -->|"tailnet (WireGuard)"| TS["Tailscale<br/>private mesh"]
TS -->|"open-webui.&lt;tailnet&gt;.ts.net"| WebUI["Open WebUI<br/>(Render private service,<br/>joined to your tailnet)"]
WebUI -->|"private network<br/>&lt;hermes-slug&gt;:8642"| Hermes["Hermes gateway<br/>(Render private service)"]
Hermes -->|"Modal SDK at runtime"| Modal["Modal sandbox<br/>isolated code execution"]
Hermes -->|model calls| OR["OpenRouter"]

Component	Port	Description
Hermes gateway	8642	Holds secrets and memory, talks to the model, creates Modal sandboxes. The most dangerous box, so it has no public URL.
Open WebUI	8080	The chat front end. Joins your tailnet and reaches Hermes over Render’s private network.
Modal sandbox	-	Isolated, ephemeral container for running the agent’s code.
Tailscale	-	A private WireGuard mesh that makes the UI reachable to your devices only.

The agent that can run code and read secrets is never routable from the internet, and neither is the UI in front of it. There is no inbound ingress on any box (the outbound calls to the model, to Modal, and to the image registry still cross the internet, as they have to). Expressing “which box can reach which” in code is what makes the boundary auditable: the rule lives in the program rather than in a dashboard.

The four pieces split cleanly in two: two you declare as resources, and two you hand a credential and let run.

Layer	How you provision it	Status
Render services, disks, secrets	official `render-oss` Terraform provider, bridged	Declarative, via one `pulumi package add`.
Container images	`@pulumi/docker-build`	Built and pushed during `pulumi up`. No `docker` CLI.
Modal sandboxes	nothing	No Terraform provider exists. A runtime credential.
Tailnet access (Tailscale)	nothing	No provider needed. A runtime credential, like Modal.

Two of these four layers are credentials, not resources. That distinction often gets missed, and it is worth getting right: a provider can only manage what it can provision, and spinning up a Modal sandbox is a runtime act, not a provisioned resource. We come back to it at each layer.

Setting up ESC for secrets management

Deploying Hermes means handling a pile of credentials: a Render API key, registry credentials, the OpenRouter key, the Modal token pair, and the Tailscale auth key. You don’t want these hardcoded or scattered across environment variables. Pulumi ESC (Environments, Secrets, and Configuration) stores them securely and passes them directly to your Pulumi program.

Create a new ESC environment:

pulumi env init <your-org>/hermes-secrets

Add your secrets to the environment:

values:
 renderApiKey:
 fn::secret: "rnd_xxxxx"
 renderOwnerId: "tea-xxxxx"
 ghcrUser: "your-github-username"
 ghcrToken:
 fn::secret: "ghp_xxxxx"
 openrouterApiKey:
 fn::secret: "sk-or-xxxxx"
 modalTokenId:
 fn::secret: "ak-xxxxx"
 modalTokenSecret:
 fn::secret: "as-xxxxx"
 tailscaleAuthKey:
 fn::secret: "tskey-auth-xxxxx"
 tailnetDnsName: "tailxxxxx.ts.net"
 pulumiConfig:
 renderApiKey: ${renderApiKey}
 ownerId: ${renderOwnerId}
 ghcrUser: ${ghcrUser}
 ghcrToken: ${ghcrToken}
 openrouterApiKey: ${openrouterApiKey}
 modalTokenId: ${modalTokenId}
 modalTokenSecret: ${modalTokenSecret}
 tailscaleAuthKey: ${tailscaleAuthKey}
 tailnetDnsName: ${tailnetDnsName}

To find your tailnet DNS name, go to the Tailscale admin console, look under the DNS section, and find your tailnet name (e.g., tailxxxxx.ts.net). This is the domain suffix used for all machines in your Tailscale network.

Then create a Pulumi.dev.yaml file in your project to reference the environment:

environment:
 - <your-org>/hermes-secrets

This keeps your secrets out of your codebase and hands them to the program at deploy time. The Modal token pair and the Tailscale auth key never become resources; they ride straight through as encrypted env vars on the services that need them.

Generating the Render SDK

Render has no first-party Pulumi provider. What it has is an official, maintained Terraform provider, and Pulumi can turn any Terraform provider into a typed local SDK with Any Terraform Provider:

pulumi package add terraform-provider render-oss/render 1.8.0

That one command generates a render SDK you import like any other package, in whatever language your stack is written in. There is no published @pulumi/render package and no vendor to wait on.

This is also where the provider map shows its edges. The bridge wraps a Terraform provider that already exists, so it works for Render. But Modal has no Terraform provider anywhere, so pulumi package add terraform-provider modal-labs/modal has nothing to fetch. That is not a gap to work around. Modal is not a provisioned resource here; Hermes creates sandboxes at runtime through the Modal SDK, and Pulumi’s only job is to hold the token pair and hand it over. Tailscale lands in the same bucket: a Tailscale provider exists, but it manages your tailnet’s settings, not the act of putting a container on the network, which is a runtime job done inside the image.

Securing with Tailscale

The usual way to make a self-hosted UI reachable is to give it a public hostname and put an identity gate in front: an SSO provider, an access policy, a public DNS record, and a certificate to keep renewing. Tailscale collapses all of it into nothing you declare. There is no gate to bolt in front of a public URL, because there is no public URL. Open WebUI joins your tailnet and answers only to your own devices.

This is a stronger position than a public UI behind a login. Because there is no public URL, there is no login to harden and no certificate to renew, and Open WebUI answers only to your own devices.

Tailscale lives in the Open WebUI image rather than the Pulumi program. The image adds Tailscale and an entrypoint that brings up tailscaled in userspace mode (a container has no /dev/net/tun), joins your tailnet, and runs tailscale serve to expose Open WebUI on it. serve publishes over HTTPS to authenticated members of your tailnet only, where its sibling tailscale funnel would put the same port on the public internet, the one thing this design avoids.

FROM ghcr.io/open-webui/open-webui:main-slim
RUN apt-get update && apt-get install -y --no-install-recommends curl ca-certificates \
 && curl -fsSL https://tailscale.com/install.sh | sh
COPY entrypoint.sh /usr/local/bin/ts-entrypoint
ENTRYPOINT ["/usr/local/bin/ts-entrypoint"]

#!/usr/bin/env bash
set -e
PORT="${PORT:-8080}"
mkdir -p /app/backend/data/tailscale
# Userspace networking: a container has no /dev/net/tun. Keep node state on the disk
# so it survives restarts, and accept-dns=false so Render's DNS still resolves the
# private Hermes hostname.
tailscaled --tun=userspace-networking --statedir=/app/backend/data/tailscale &
n=0
until tailscale up --authkey="${TS_AUTHKEY}" --hostname="${TS_HOSTNAME}" --accept-dns=false; do
 n=$((n + 1)); [ "${n}" -ge 30 ] && { echo "tailnet join failed; check TS_AUTHKEY"; exit 1; }
 sleep 2
done
tailscale serve --bg "${PORT}" # serve Open WebUI on the tailnet over HTTPS
cd /app/backend && exec bash start.sh

The TS_AUTHKEY env var on the service, pulled from ESC, is the only thing Pulumi contributes; everything else is the image doing its own networking.

There is one prerequisite, the same one any Tailscale node needs for serve: enable HTTPS for your tailnet once in the admin console so it can get a certificate. Use a reusable auth key, since the service re-authenticates on every redeploy, and the join loop has a ceiling so a dead key fails the deploy in a minute instead of hanging it.

To generate the reusable auth key:

Go to the Tailscale admin console.
Click Generate auth key.
Enable Reusable, since the service re-authenticates on every redeploy.
Copy the key into your ESC environment as tailscaleAuthKey.

Deploying the agent

This is the part a Makefile usually handles. Two private services, two disks, every secret an encrypted env var, and both images built and pushed during the same pulumi up that creates the services. No CLI runs anywhere, though the build still rides on a local Docker engine through buildx: the docker call is gone, the daemon it talks to is not, so pulumi up needs Docker running.

Create a new Pulumi project (pick the language you prefer; the companion repo has all three):

mkdir hermes-agent-pulumi && cd hermes-agent-pulumi
pulumi new typescript

Install the dependencies and generate the Render SDK:

npm install @pulumi/docker-build @pulumi/random
pulumi package add terraform-provider render-oss/render 1.8.0

Both services need the 2 GB Standard plan. The gateway and Open WebUI each exhaust memory on the 512 MB Starter and OOM on boot.

Two facts make this clean. Render env var values are encrypted at rest, so a secret is only an env var, with no separate resource. And @pulumi/docker-build builds and pushes an image through an embedded buildx during pulumi up, so the image pipeline is declarative too. Render then pulls the pushed image by digest.

The complete program

Hermes comes first: a private Render service with no public URL, a disk for its memory, and every secret as an encrypted env var. The shared bearer key is generated in state with random.RandomBytes, the clean replacement for openssl rand -base64 32; the Modal token pair and the OpenRouter key come from config or Pulumi ESC. Render has no port field, so Hermes binds 0.0.0.0:8642 and Open WebUI reaches it at the service’s read-only slug. Open WebUI is almost the mirror image, private too, except its image joins your tailnet (the tailscaled entrypoint from the previous section) and it carries the operational env vars that keep it lean: no local embedding models, and ENABLE_PERSISTENT_CONFIG=false so your config always wins over a stale copy in its database. One render.Provider carries your renderApiKey and ownerId into every Render resource.

Here is the whole program, copy-paste ready as your index.ts (or __main__.py, or Pulumi.yaml):

import * as pulumi from "@pulumi/pulumi";
import * as dockerbuild from "@pulumi/docker-build";
import * as random from "@pulumi/random";
// Local SDK generated by: pulumi package add terraform-provider render-oss/render 1.8.0
// (no @pulumi/render is published; the SDK is bridged from the official Render provider).
import * as render from "@pulumi/render";

const cfg = new pulumi.Config();

// ---------------------------------------------------------------------------
// Configuration. Non-secret values via `pulumi config set`, secrets via
// `pulumi config set --secret` or a Pulumi ESC environment (see README).
// The Render provider reads renderApiKey/renderOwnerId (or RENDER_API_KEY/RENDER_OWNER_ID).
// ---------------------------------------------------------------------------
const region = cfg.get("renderRegion") ?? "oregon";
const modelDefault = cfg.get("modelDefault") ?? "gpt-5.5";

// Container registry the images are pushed to and Render pulls from (GHCR here).
const ghcrUser = cfg.require("ghcrUser");
const ghcrToken = cfg.requireSecret("ghcrToken"); // GHCR PAT with write:packages

// Long-lived secrets supplied by you (dashboard-minted). Hold in config or ESC.
const modalTokenId = cfg.requireSecret("modalTokenId");
const modalTokenSecret = cfg.requireSecret("modalTokenSecret");
const openrouterApiKey = cfg.requireSecret("openrouterApiKey");

// Tailscale: the only "gateway". Open WebUI joins your tailnet and is reachable
// at https://<hostname>.<tailnet>.ts.net by your devices only — never public.
// Both values are already in your ESC. Enable HTTPS once in the Tailscale admin console.
const tailscaleAuthKey = cfg.requireSecret("tailscaleAuthKey");
const tailnetDnsName = cfg.require("tailnetDnsName"); // e.g. tailc6fb4e.ts.net
const webuiHostname = "open-webui";
const webuiTailnetUrl = `https://${webuiHostname}.${tailnetDnsName}`;

// Render provider auth (from ESC: pulumiConfig.renderApiKey / pulumiConfig.renderOwnerId).
const renderProvider = new render.Provider("render", {
 apiKey: cfg.requireSecret("renderApiKey"),
 ownerId: cfg.require("renderOwnerId"),
});

// ---------------------------------------------------------------------------
// Secrets generated in state (the in-IaC replacement for `openssl rand -base64 32`).
// The shared bearer authenticates Open WebUI -> Hermes; it lands on both services.
// ---------------------------------------------------------------------------
const sharedKey = new random.RandomBytes("hermes-shared-key", { length: 32 });
const webuiSecretKey = new random.RandomBytes("webui-secret-key", { length: 32 });

// ---------------------------------------------------------------------------
// Build and push both images during `pulumi up` (no docker CLI; the provider
// embeds buildx). Render then pulls them by their digest-pinned ref.
// ---------------------------------------------------------------------------
const registryAuth = [{ address: "ghcr.io", username: ghcrUser, password: ghcrToken }];

const hermesImage = new dockerbuild.Image("hermes-image", {
 context: { location: "./hermes" },
 dockerfile: { location: "./hermes/Dockerfile" },
 platforms: ["linux/amd64"],
 tags: [pulumi.interpolate`ghcr.io/${ghcrUser}/hermes-agent:latest`],
 push: true,
 registries: registryAuth,
});

const webuiImage = new dockerbuild.Image("webui-image", {
 context: { location: "./openwebui" },
 dockerfile: { location: "./openwebui/Dockerfile" },
 platforms: ["linux/amd64"],
 tags: [pulumi.interpolate`ghcr.io/${ghcrUser}/open-webui:latest`],
 push: true,
 registries: registryAuth,
});

// Credential Render uses to pull the private GHCR images.
const ghcrCredential = new render.RegistryCredential("ghcr", {
 name: "ghcr",
 registry: "GITHUB",
 username: ghcrUser,
 authToken: ghcrToken,
}, { provider: renderProvider });

// ---------------------------------------------------------------------------
// Hermes: a PRIVATE Render service. No public URL; reachable only over Render's
// private network. A disk holds its memory and workspace. Env values are
// encrypted at rest, which is how secrets ride on Render (no separate flag).
// ---------------------------------------------------------------------------
const hermes = new render.PrivateService("hermes-agent", {
 name: "hermes-agent",
 plan: "standard", // 2 GB; the gateway OOMs on starter's 512 MB
 region: region,
 runtimeSource: {
 // Render wants the bare repo in imageUrl; the tag/digest go in their own field.
 image: {
 imageUrl: pulumi.interpolate`ghcr.io/${ghcrUser}/hermes-agent`,
 digest: hermesImage.digest,
 registryCredentialId: ghcrCredential.id,
 },
 },
 disk: { name: "hermes-data", mountPath: "/opt/data", sizeGb: 5 },
 envVars: {
 PORT: { value: "8642" },
 API_SERVER_ENABLED: { value: "true" },
 API_SERVER_HOST: { value: "0.0.0.0" },
 API_SERVER_PORT: { value: "8642" },
 API_SERVER_MODEL_NAME: { value: modelDefault },
 // Auto-approve the agent's shell/terminal hooks (no TTY in a container),
 // so it can run code in the Modal sandbox without an interactive prompt.
 HERMES_ACCEPT_HOOKS: { value: "1" },
 // Secrets (encrypted at rest on Render):
 API_SERVER_KEY: { value: sharedKey.base64 },
 OPENROUTER_API_KEY: { value: openrouterApiKey },
 // Modal is a runtime credential, not a provisioned resource: Hermes creates
 // sandboxes at runtime with this token pair. No Modal provider, no Modal CLI.
 MODAL_TOKEN_ID: { value: modalTokenId },
 MODAL_TOKEN_SECRET: { value: modalTokenSecret },
 },
}, { provider: renderProvider });

// Open WebUI reaches Hermes at its Render-assigned internal hostname (the read-only
// slug), not the bare service name. Same region + same workspace share the network.
const hermesApiBaseUrl = pulumi.interpolate`http://${hermes.slug}:8642/v1`;

// ---------------------------------------------------------------------------
// Open WebUI: also a PRIVATE Render service, but the container joins your tailnet
// (tailscaled in userspace + `tailscale serve`), so you reach it at
// https://open-webui.<tailnet>.ts.net from your devices. Never public; the gate
// is tailnet membership. It still reaches Hermes over Render's private network.
// ---------------------------------------------------------------------------
const webui = new render.PrivateService("open-webui", {
 name: "open-webui",
 plan: "standard", // 2 GB; Open WebUI OOMs on starter's 512 MB
 region: region, // must match Hermes for private networking
 runtimeSource: {
 image: {
 imageUrl: pulumi.interpolate`ghcr.io/${ghcrUser}/open-webui`,
 digest: webuiImage.digest,
 registryCredentialId: ghcrCredential.id,
 },
 },
 disk: { name: "openwebui-data", mountPath: "/app/backend/data", sizeGb: 5 },
 envVars: {
 PORT: { value: "8080" }, // tailscale serve points at this; Open WebUI binds it
 OPENAI_API_BASE_URL: { value: hermesApiBaseUrl },
 ENABLE_OPENAI_API: { value: "true" },
 ENABLE_OLLAMA_API: { value: "false" },
 DEFAULT_MODELS: { value: modelDefault },
 WEBUI_AUTH: { value: "true" },
 WEBUI_URL: { value: webuiTailnetUrl },
 // Reached only over the tailnet, so the first sign-up is the admin.
 ENABLE_LOGIN_FORM: { value: "true" },
 ENABLE_SIGNUP: { value: "true" },
 DEFAULT_USER_ROLE: { value: "admin" },
 // Always apply env config; otherwise Open WebUI persists the first run's config
 // to its DB and ignores later changes.
 ENABLE_PERSISTENT_CONFIG: { value: "false" },
 // Don't load local embedding/RAG models (a big memory saver; the agent is the brain).
 RAG_EMBEDDING_ENGINE: { value: "openai" },
 BYPASS_EMBEDDING_AND_RETRIEVAL: { value: "true" },
 OFFLINE_MODE: { value: "true" },
 HF_HUB_OFFLINE: { value: "1" },
 // Secrets (encrypted at rest):
 OPENAI_API_KEY: { value: sharedKey.base64 }, // shared bearer to call Hermes
 WEBUI_SECRET_KEY: { value: webuiSecretKey.base64 },
 // Tailscale joins this container to your tailnet and serves Open WebUI on it.
 TS_AUTHKEY: { value: tailscaleAuthKey },
 TS_HOSTNAME: { value: webuiHostname },
 },
}, { dependsOn: hermes, provider: renderProvider });

export const hermesInternalUrl = hermesApiBaseUrl;
// Open this from any device on your tailnet:
export const openWebUiTailnetUrl = webuiTailnetUrl;

import pulumi
import pulumi_docker_build as docker_build
import pulumi_random as random

# Local SDK generated by: pulumi package add terraform-provider render-oss/render 1.8.0
# (no pulumi_render is published; the SDK is bridged from the official Render provider).
import pulumi_render as render

cfg = pulumi.Config()

# ---------------------------------------------------------------------------
# Configuration. Non-secret values via `pulumi config set`, secrets via
# `pulumi config set --secret` or a Pulumi ESC environment (see README).
# The Render provider reads renderApiKey/renderOwnerId (or RENDER_API_KEY/RENDER_OWNER_ID).
# ---------------------------------------------------------------------------
region = cfg.get("renderRegion") or "oregon"
model_default = cfg.get("modelDefault") or "gpt-5.5"

# Container registry the images are pushed to and Render pulls from (GHCR here).
ghcr_user = cfg.require("ghcrUser")
ghcr_token = cfg.require_secret("ghcrToken") # GHCR PAT with write:packages

# Long-lived secrets supplied by you (dashboard-minted). Hold in config or ESC.
modal_token_id = cfg.require_secret("modalTokenId")
modal_token_secret = cfg.require_secret("modalTokenSecret")
openrouter_api_key = cfg.require_secret("openrouterApiKey")

# Tailscale: the only "gateway". Open WebUI joins your tailnet and is reachable
# at https://<hostname>.<tailnet>.ts.net by your devices only -- never public.
# Enable HTTPS once in the Tailscale admin console so `tailscale serve` can get a cert.
tailscale_auth_key = cfg.require_secret("tailscaleAuthKey")
tailnet_dns_name = cfg.require("tailnetDnsName") # e.g. tailc6fb4e.ts.net
webui_hostname = "open-webui"
webui_tailnet_url = f"https://{webui_hostname}.{tailnet_dns_name}"

# Render provider auth (from ESC: pulumiConfig.renderApiKey / pulumiConfig.renderOwnerId).
render_provider = render.Provider(
 "render",
 api_key=cfg.require_secret("renderApiKey"),
 owner_id=cfg.require("renderOwnerId"),
)

# ---------------------------------------------------------------------------
# Secrets generated in state (the in-IaC replacement for `openssl rand -base64 32`).
# The shared bearer authenticates Open WebUI -> Hermes; it lands on both services.
# ---------------------------------------------------------------------------
shared_key = random.RandomBytes("hermes-shared-key", length=32)
webui_secret_key = random.RandomBytes("webui-secret-key", length=32)

# ---------------------------------------------------------------------------
# Build and push both images during `pulumi up` (no docker CLI; the provider
# embeds buildx). Render then pulls them by their digest-pinned ref.
# ---------------------------------------------------------------------------
registry_auth = [{"address": "ghcr.io", "username": ghcr_user, "password": ghcr_token}]

hermes_image = docker_build.Image(
 "hermes-image",
 context={"location": "./hermes"},
 dockerfile={"location": "./hermes/Dockerfile"},
 platforms=["linux/amd64"],
 tags=[pulumi.Output.concat("ghcr.io/", ghcr_user, "/hermes-agent:latest")],
 push=True,
 registries=registry_auth,
)

webui_image = docker_build.Image(
 "webui-image",
 context={"location": "./openwebui"},
 dockerfile={"location": "./openwebui/Dockerfile"},
 platforms=["linux/amd64"],
 tags=[pulumi.Output.concat("ghcr.io/", ghcr_user, "/open-webui:latest")],
 push=True,
 registries=registry_auth,
)

# Credential Render uses to pull the private GHCR images.
ghcr_credential = render.RegistryCredential(
 "ghcr",
 name="ghcr",
 registry="GITHUB",
 username=ghcr_user,
 auth_token=ghcr_token,
 opts=pulumi.ResourceOptions(provider=render_provider),
)

# ---------------------------------------------------------------------------
# Hermes: a PRIVATE Render service. No public URL; reachable only over Render's
# private network. A disk holds its memory and workspace. Env values are
# encrypted at rest, which is how secrets ride on Render (no separate flag).
# ---------------------------------------------------------------------------
hermes = render.PrivateService(
 "hermes-agent",
 name="hermes-agent",
 plan="standard", # 2 GB; the gateway OOMs on starter's 512 MB
 region=region,
 runtime_source={
 "image": {
 "image_url": pulumi.Output.concat("ghcr.io/", ghcr_user, "/hermes-agent"),
 "digest": hermes_image.digest,
 "registry_credential_id": ghcr_credential.id,
 },
 },
 disk={"name": "hermes-data", "mount_path": "/opt/data", "size_gb": 5},
 env_vars={
 "PORT": {"value": "8642"},
 "API_SERVER_ENABLED": {"value": "true"},
 "API_SERVER_HOST": {"value": "0.0.0.0"},
 "API_SERVER_PORT": {"value": "8642"},
 "API_SERVER_MODEL_NAME": {"value": model_default},
 # Auto-approve the agent's shell/terminal hooks (no TTY in a container).
 "HERMES_ACCEPT_HOOKS": {"value": "1"},
 # Secrets (encrypted at rest on Render):
 "API_SERVER_KEY": {"value": shared_key.base64},
 "OPENROUTER_API_KEY": {"value": openrouter_api_key},
 # Modal is a runtime credential: Hermes creates sandboxes at runtime with this pair.
 "MODAL_TOKEN_ID": {"value": modal_token_id},
 "MODAL_TOKEN_SECRET": {"value": modal_token_secret},
 },
 opts=pulumi.ResourceOptions(provider=render_provider),
)

# Open WebUI reaches Hermes at its Render-assigned internal hostname (the read-only slug).
hermes_api_base_url = pulumi.Output.concat("http://", hermes.slug, ":8642/v1")

# ---------------------------------------------------------------------------
# Open WebUI: also a PRIVATE Render service, but the container joins your tailnet
# (tailscaled in userspace + `tailscale serve`), so you reach it at
# https://open-webui.<tailnet>.ts.net from your devices. Never public.
# ---------------------------------------------------------------------------
webui = render.PrivateService(
 "open-webui",
 name="open-webui",
 plan="standard", # 2 GB; Open WebUI OOMs on starter's 512 MB
 region=region, # must match Hermes for private networking
 runtime_source={
 "image": {
 "image_url": pulumi.Output.concat("ghcr.io/", ghcr_user, "/open-webui"),
 "digest": webui_image.digest,
 "registry_credential_id": ghcr_credential.id,
 },
 },
 disk={"name": "openwebui-data", "mount_path": "/app/backend/data", "size_gb": 5},
 env_vars={
 "PORT": {"value": "8080"}, # tailscale serve points at this; Open WebUI binds it
 "OPENAI_API_BASE_URL": {"value": hermes_api_base_url},
 "ENABLE_OPENAI_API": {"value": "true"},
 "ENABLE_OLLAMA_API": {"value": "false"},
 "DEFAULT_MODELS": {"value": model_default},
 "WEBUI_AUTH": {"value": "true"},
 "WEBUI_URL": {"value": webui_tailnet_url},
 # Reached only over the tailnet, so the first sign-up is the admin.
 "ENABLE_LOGIN_FORM": {"value": "true"},
 "ENABLE_SIGNUP": {"value": "true"},
 "DEFAULT_USER_ROLE": {"value": "admin"},
 # Always apply env config; otherwise Open WebUI persists the first run's config.
 "ENABLE_PERSISTENT_CONFIG": {"value": "false"},
 # Don't load local embedding/RAG models (a big memory saver; the agent is the brain).
 "RAG_EMBEDDING_ENGINE": {"value": "openai"},
 "BYPASS_EMBEDDING_AND_RETRIEVAL": {"value": "true"},
 "OFFLINE_MODE": {"value": "true"},
 "HF_HUB_OFFLINE": {"value": "1"},
 # Secrets (encrypted at rest):
 "OPENAI_API_KEY": {"value": shared_key.base64},
 "WEBUI_SECRET_KEY": {"value": webui_secret_key.base64},
 # Tailscale joins this container to your tailnet and serves Open WebUI on it.
 "TS_AUTHKEY": {"value": tailscale_auth_key},
 "TS_HOSTNAME": {"value": webui_hostname},
 },
 opts=pulumi.ResourceOptions(depends_on=[hermes], provider=render_provider),
)

pulumi.export("hermesInternalUrl", hermes_api_base_url)
# Open this from any device on your tailnet:
pulumi.export("openWebUiTailnetUrl", webui_tailnet_url)

name: hermes-agent-pulumi
runtime: yaml
description: Deploy a Hermes agent (Render + Modal + Tailscale) as one declarative Pulumi program.

# The Render SDK is bridged from the official Terraform provider. Run once:
# pulumi package add terraform-provider render-oss/render 1.8.0
# Then `pulumi install` regenerates it on a fresh checkout.
packages:
 render:
 source: terraform-provider
 version: 1.1.4
 parameters:
 - render-oss/render
 - 1.8.0

config:
 renderRegion:
 type: string
 default: oregon
 modelDefault:
 type: string
 default: gpt-5.5
 ghcrUser:
 type: string
 ghcrToken:
 type: string
 secret: true
 modalTokenId:
 type: string
 secret: true
 modalTokenSecret:
 type: string
 secret: true
 openrouterApiKey:
 type: string
 secret: true
 tailscaleAuthKey:
 type: string
 secret: true
 tailnetDnsName:
 type: string
 renderApiKey:
 type: string
 secret: true
 renderOwnerId:
 type: string

variables:
 registryAuth:
 - address: ghcr.io
 username: ${ghcrUser}
 password: ${ghcrToken}
 # Open WebUI reaches Hermes at its private slug; you reach Open WebUI on the tailnet.
 hermesApiBaseUrl: http://${hermes-agent.slug}:8642/v1
 webuiTailnetUrl: https://open-webui.${tailnetDnsName}

resources:
 # Render provider auth (from config or ESC).
 render:
 type: pulumi:providers:render
 properties:
 apiKey: ${renderApiKey}
 ownerId: ${renderOwnerId}

 # Secrets generated in state: the replacement for `openssl rand -base64 32`.
 hermes-shared-key:
 type: random:RandomBytes
 properties:
 length: 32
 webui-secret-key:
 type: random:RandomBytes
 properties:
 length: 32

 # Build + push both images during `pulumi up` (no docker CLI).
 hermes-image:
 type: docker-build:Image
 properties:
 context:
 location: ./hermes
 dockerfile:
 location: ./hermes/Dockerfile
 platforms: [linux/amd64]
 tags:
 - ghcr.io/${ghcrUser}/hermes-agent:latest
 push: true
 registries: ${registryAuth}
 webui-image:
 type: docker-build:Image
 properties:
 context:
 location: ./openwebui
 dockerfile:
 location: ./openwebui/Dockerfile
 platforms: [linux/amd64]
 tags:
 - ghcr.io/${ghcrUser}/open-webui:latest
 push: true
 registries: ${registryAuth}

 # Credential Render uses to pull the private GHCR images.
 ghcr:
 type: render:RegistryCredential
 properties:
 name: ghcr
 registry: GITHUB
 username: ${ghcrUser}
 authToken: ${ghcrToken}
 options:
 provider: ${render}

 # Hermes: a PRIVATE Render service. No public URL; encrypted env vars are the secret mechanism.
 hermes-agent:
 type: render:PrivateService
 properties:
 name: hermes-agent
 plan: standard  # 2 GB; the gateway OOMs on starter's 512 MB
 region: ${renderRegion}
 runtimeSource:
 image:
 imageUrl: ghcr.io/${ghcrUser}/hermes-agent
 digest: ${hermes-image.digest}
 registryCredentialId: ${ghcr.id}
 disk:
 name: hermes-data
 mountPath: /opt/data
 sizeGb: 5
 envVars:
 PORT: { value: "8642" }
 API_SERVER_ENABLED: { value: "true" }
 API_SERVER_HOST: { value: "0.0.0.0" }
 API_SERVER_PORT: { value: "8642" }
 API_SERVER_MODEL_NAME: { value: "${modelDefault}" }
 HERMES_ACCEPT_HOOKS: { value: "1" }
 API_SERVER_KEY: { value: "${hermes-shared-key.base64}" }
 OPENROUTER_API_KEY: { value: "${openrouterApiKey}" }
 MODAL_TOKEN_ID: { value: "${modalTokenId}" }
 MODAL_TOKEN_SECRET: { value: "${modalTokenSecret}" }
 options:
 provider: ${render}

 # Open WebUI: also a PRIVATE Render service, joined to your tailnet by its image.
 open-webui:
 type: render:PrivateService
 properties:
 name: open-webui
 plan: standard  # 2 GB; Open WebUI OOMs on starter's 512 MB
 region: ${renderRegion}  # must match Hermes for private networking
 runtimeSource:
 image:
 imageUrl: ghcr.io/${ghcrUser}/open-webui
 digest: ${webui-image.digest}
 registryCredentialId: ${ghcr.id}
 disk:
 name: openwebui-data
 mountPath: /app/backend/data
 sizeGb: 5
 envVars:
 PORT: { value: "8080" }
 OPENAI_API_BASE_URL: { value: "${hermesApiBaseUrl}" }
 ENABLE_OPENAI_API: { value: "true" }
 ENABLE_OLLAMA_API: { value: "false" }
 DEFAULT_MODELS: { value: "${modelDefault}" }
 WEBUI_AUTH: { value: "true" }
 WEBUI_URL: { value: "${webuiTailnetUrl}" }
 ENABLE_LOGIN_FORM: { value: "true" }
 ENABLE_SIGNUP: { value: "true" }
 DEFAULT_USER_ROLE: { value: "admin" }
 ENABLE_PERSISTENT_CONFIG: { value: "false" }
 RAG_EMBEDDING_ENGINE: { value: "openai" }
 BYPASS_EMBEDDING_AND_RETRIEVAL: { value: "true" }
 OFFLINE_MODE: { value: "true" }
 HF_HUB_OFFLINE: { value: "1" }
 OPENAI_API_KEY: { value: "${hermes-shared-key.base64}" }
 WEBUI_SECRET_KEY: { value: "${webui-secret-key.base64}" }
 TS_AUTHKEY: { value: "${tailscaleAuthKey}" }
 TS_HOSTNAME: { value: "open-webui" }
 options:
 provider: ${render}
 dependsOn:
 - ${hermes-agent}

outputs:
 hermesInternalUrl: ${hermesApiBaseUrl}
 # Open this from any device on your tailnet:
 openWebUiTailnetUrl: ${webuiTailnetUrl}

Neither box is published to the internet. The same code, plus the Dockerfiles and the Tailscale entrypoint, is in the companion repo:

github.com/dirien/hermes-agent-pulumi

Cost

Two details shape the bill. The host starts on Render’s free Hobby workspace, but a private service with a persistent disk runs on paid compute, so you need a card on file the moment you add them. There is no free-tier version of this architecture. Both services also need the 2 GB Standard plan, since the gateway and Open WebUI each OOM on the 512 MB Starter.

	Hermes gateway	Open WebUI
Plan	Standard (2 GB)	Standard (2 GB)
Compute	~$25/mo	~$25/mo
Disk	5 GB	5 GB
Public URL	none	none (tailnet only)

That lands the host around $50/month, in line with what a comparable managed setup runs. Modal and OpenRouter bill by usage on top, so the total tracks how hard the agent works rather than a flat number. Tailscale is free for a personal tailnet, so the private gateway adds nothing to that.

The persistent disk pins each service to a single instance, so there is no autoscaling, and a redeploy drops a few seconds of connectivity while the old instance lets go of the disk. For a single-user agent that is a fine trade, and it is why “stand it up in a second region” is a copy-the-config move and not a live failover.

The OpenRouter Activity dashboard: 56 requests and 738K tokens of gpt-5.5 usage, all routed through one key.

Running the deployment

With your ESC environment referenced in Pulumi.dev.yaml, deploy with:

pulumi up

This generates the Render SDK’s resources, builds and pushes both images through buildx, and creates the two private services with their disks and encrypted env vars, all in one run, with no docker CLI and no secret import hiding in a script. When it finishes, both services show up in the Render dashboard with no public URL.

The hermes-agent and open-webui private services in the Render dashboard, both deployed with no public URL.

Access the agent

Give it a few minutes after pulumi up finishes. The images still have to be pulled and the containers booted, and Open WebUI has to join your tailnet before the URL resolves. Refresh until it loads.

Open WebUI is reachable only from a device on your tailnet, at https://open-webui.<your-tailnet>.ts.net, with no domain to buy, no DNS to point, and no certificate to manage. Open it, sign up so the first account becomes the admin, and ask the agent to run a Python command. It should fire up a Modal sandbox and return the result, with nothing exposed to the public internet at any point.

Open WebUI on the tailnet URL: the gpt-5.5 agent runs sum(i*i for i in range(1, 1001)) in a Modal sandbox and returns 333833500.

When the agent runs that code, Modal builds a throwaway sandbox specifically for it. There is no Modal resource to declare, no Modal CLI to call, and no Modal provider to bridge. The Hermes service carries MODAL_TOKEN_ID and MODAL_TOKEN_SECRET as two encrypted env vars, and that is the whole footprint.

The Modal dashboard: the per-task sandboxes the Hermes agent spun up at runtime to execute code.

Verify the deployment (optional)

If the chat UI does not load, give the services a few minutes to finish their first boot, then check two things:

Render dashboard logs for the hermes-agent and open-webui services, to confirm both started and pulled their images.
Tailscale admin console (Machines), to confirm the open-webui node joined your tailnet. If it did not, the TS_AUTHKEY is likely expired or not reusable.

Keep your AI agent private

Deploy your Hermes agent across Render, Modal, and Tailscale as one Pulumi program, with secrets held in Pulumi ESC and nothing exposed to the public internet.

Get started

Security considerations

Self-hosted AI servers get found fast: scanners like Shodan and Censys enumerate a freshly exposed one within hours to days, and the code-capable ones are already under active attack. Running an always-on agent on a machine you also use for everything else invites prompt injection on top of that. The answer here is to keep the agent unroutable from the internet and to isolate the code it runs. That defeats the scanner: nobody finds an open port to attack. It does less against the agent being turned against you from the inside, which is the harder problem.

One detail makes that isolation essential. In a headless container there is no one at the terminal to approve anything, and the same is true of its cron and CI runs, so the deployment runs Hermes with HERMES_ACCEPT_HOOKS=1 to auto-accept the shell hooks it would otherwise pause on. The practical result is that the agent’s code runs with no human in the loop, which is exactly why the Modal sandbox has to carry the weight: when the model writes a bad command, the throwaway container it runs in is what stands between that command and the gateway’s credentials, not a confirmation prompt.

Concern	Typical self-host	This deployment
Gateway reachability	Public port, or behind a login	Private service, no public URL
Chat UI reachability	Public hostname + auth	Tailnet only (`tailscale serve`)
Code execution	Runs next to the gateway	Isolated Modal sandbox, torn down after
Command approval	Human approves dangerous commands	Headless: isolation replaces the prompt
Secrets in transit	Exposed if accessed over HTTP	Encrypted env vars; tailnet-encrypted access

This is private, which is not the same as fully trusted. A few places the boundary still leaks:

The shared bearer never rotates. Hermes is unroutable from the internet, but it is wide open to Open WebUI, and the two share one static bearer key. Whoever owns the front end owns the gateway behind it. Rotate it periodically.
The tailnet policy is click-ops. The ACLs, the node’s tags, and the one-time HTTPS toggle live in the Tailscale admin console, not in pulumi up, and the reusable key has no tag scoping its blast radius. The Tailscale provider can manage that policy declaratively if you want it in code.
Signup defaults to admin. Fine on a single-user tailnet, the first thing to lock down (ENABLE_SIGNUP, DEFAULT_USER_ROLE) once more than one person can reach the URL.
Every messaging client is another entry point. Each platform you wire up adds another way in, which is why the tailnet is the baseline for securing this, not the full extent of it.
A compromised agent can still reach out. The private core stops external discovery, but it does not contain the agent itself. The gateway holds the Modal and OpenRouter tokens as env vars and can call out freely, so a prompt-injected agent could read its own environment and send them somewhere; the Modal sandbox isolates the code the agent runs, not the credentials it holds. Cap and rotate the OpenRouter key, keep the Modal token off the gateway, and allowlist outbound traffic, which on Render means a forward-proxy sidecar (Render gives you dedicated outbound IPs and inbound rules, but no destination egress filter).

My recommendations:

Keep both services private; never give Hermes a public URL
Run code in Modal sandboxes, since HERMES_ACCEPT_HOOKS=1 means no human approves commands
Rotate the shared bearer key and the Tailscale auth key periodically
Cap and rotate the OpenRouter key; a leaked credential is the likelier failure mode than an open port
Use Pulumi ESC for secrets instead of hardcoding
Lock down Open WebUI signup once more than one person can reach it
Manage Tailscale ACLs and tags deliberately, in the admin console or with the Tailscale provider

What’s next?

The deployment is the foundation the agent builds on. A private agent becomes useful when it remembers earlier context, runs code without a human at the terminal, and initiates contact on its own. The disk is what makes this persist: its memory and its schedule live on that disk, so they survive every restart.

It schedules itself. Hermes ships a first-class cron you drive in plain language, either by asking the agent in the chat window or through its CLI. The schedule takes intervals, five-field cron, or one-shots:

hermes cron create "every 1h" \
 "Scrape <listing-url> for the price, compare it with seen.json in the sandbox, message me on Telegram only if it dropped, and otherwise reply exactly [SILENT]." \
 --deliver telegram

The [SILENT] reply suppresses delivery when nothing changed, so the watcher only speaks up when there is something to say. Scheduled runs skip memory by default, so polling does not pollute what the agent has learned.

It connects where you work, with the same secret pattern. Messaging clients attach to the same gateway, and their tokens ride as encrypted env vars on the Hermes service, the exact shape you used for the Modal pair:

// alongside MODAL_TOKEN_ID on the hermes service:
TELEGRAM_BOT_TOKEN: { value: cfg.requireSecret("telegramBotToken") },

plus a platform block you add to config.yaml (the image ships only the model and terminal blocks; Hermes supports the messaging platforms on top, per its docs):

gateway:
 platforms:
 telegram:
 require_mention: true

With those two pieces, a few things are genuinely doable today:

A price or uptime watcher that only speaks up when something changed, using the [SILENT] and persistent-sandbox dedup pattern above.
A daily briefing that ranks and dedups your feeds and delivers the top items each morning, the documented daily-briefing bot. If you want it to adapt to your taste over time, that is the optional Honcho memory provider’s job; it reads your preferences out of the conversation itself. The built-in file memory has no version of that.
A PR-review teammate that reviews open pull requests against your repo’s AGENTS.md conventions and saves recurring patterns as a skill, so the fifth review needs less hand-holding than the first.

A few limits are worth stating plainly, since demos tend to overstate the capabilities. Hermes does not ship Gmail, Calendar, or market integrations out of the box; those are wired up as MCP servers or skills, which is more work than a launch video implies. It curates its own memory and improves its own skills, but it does not retrain the underlying model or autonomously change its high-level strategy, so “self-learning” here means whatever it writes into memory and skills each run.

Conclusion

Deploying a Hermes agent with infrastructure as code means you can reproduce the setup anytime, version control it, and tear it down with a single pulumi destroy. Keeping both services private, with no public URL and no exposed ports, means the code-executing core never sits on the internet for someone to find.

The part worth carrying to the next service you deploy: not every layer has a provider, and that is fine. The task is to tell a real resource apart from a runtime dependency wearing a costume. Render and the images are resources you declare; Modal and Tailscale are credentials you hand over. Draw that map first, and the program follows from it.

To generate that map instead of writing it by hand, Pulumi Neo can take a target architecture like this one and produce a first draft of the program. The agent side of this shift is covered in How Building AI Agents Has Changed in 2026 and the platform side in Agent Sprawl Is Here. Your IaC Platform Is the Answer..

If you run into issues or have questions, drop by the Pulumi Community Slack or GitHub Discussions. New to Pulumi? Get started here.

Use Your Mac for AI Agents: Self-Host Gemma 4 12 B with Pulumi and Tailscale

Pablo Seibelt — Thu, 04 Jun 2026 00:00:00 +0000

If you run AI tools and agents, you’ve probably accepted three tradeoffs: your data leaves your network, you can’t work offline, and your bill scales with usage.

Open-weight models now run well on consumer hardware. Once the model is on your machine, your data stays local, inference works offline, and tokens cost nothing. If you own a modern Mac, you can run a high-quality model yourself.

Gemma 4 is an open-weights model family from Google. This post focuses on Gemma 4 12 B, released in June 2026, using Unsloth’s Q8_0 GGUF. The 12 B model fits comfortably on a modern Mac while leaving enough headroom for local llama.cpp and a chat UI.

We’ll use llama.cpp for host-native inference, k3d for a local Kubernetes cluster, Pulumi for infrastructure as code, and Tailscale for secure access.

Prerequisites

This setup was validated on the following hardware:

macOS 26 Tahoe, version 26.5
MacBook Pro with Apple M3 Max
36 GB RAM

On this machine, llama.cpp reported about 20 output tokens per second for a 160-token validation response with unsloth/gemma-4-12b-it-GGUF, gemma-4-12b-it-Q8_0.gguf, and a 131,072-token context. Sustained throughput varies by prompt length, thermal state, and llama.cpp settings.

You’ll need brew, docker, pulumi, and tailscale installed. We’ll also install k3d during the process.

Run Gemma 4 with host-native llama.cpp

We use llama.cpp directly on macOS to leverage Apple Metal acceleration. Running the LLM on the host is more efficient than trying to pass GPU access into a local Kubernetes VM.

Install the build tools:

brew install cmake git

Then build llama.cpp from source and download the multimodal projector. In validation, Homebrew llama.cpp 9430 could run text inference, but it could not load the new Gemma 4 12 B projector and failed with unknown projector type: gemma4uv. Building current llama.cpp from source fixed that.

llm_home="$HOME/pulumi-gemma4-llm"
mkdir -p "$llm_home/models" "$llm_home/logs"

if [ ! -d "$llm_home/llama.cpp/.git" ]; then
 git clone --depth 1 https://github.com/ggml-org/llama.cpp.git "$llm_home/llama.cpp"
fi

cmake -S "$llm_home/llama.cpp" \
 -B "$llm_home/llama.cpp/build" \
 -DGGML_METAL=ON \
 -DGGML_BLAS=ON \
 -DCMAKE_BUILD_TYPE=Release

cmake --build "$llm_home/llama.cpp/build" --target llama-server -j 10

curl -L --fail \
 --output "$llm_home/models/mmproj-F16.gguf" \
 https://huggingface.co/unsloth/gemma-4-12b-it-GGUF/resolve/main/mmproj-F16.gguf

Then download and run the model with this command:

"$HOME/pulumi-gemma4-llm/llama.cpp/build/bin/llama-server" \
 --hf-repo unsloth/gemma-4-12b-it-GGUF \
 --hf-file gemma-4-12b-it-Q8_0.gguf \
 --mmproj "$HOME/pulumi-gemma4-llm/models/mmproj-F16.gguf" \
 --host 127.0.0.1 \
 --port 18080 \
 --ctx-size 131072 \
 --parallel 1 \
 --jinja \
 --reasoning off

We use port 18080 because 8080 is commonly used and is likely to conflict with another service you may already have running locally. If your port 8080 is free, you can use it and adjust the Pulumi config later.

The model file is about 12.65 GB, and the projector is about 116 MB. Gemma 4 12 B advertises a 131,072-token context, and this Mac loaded that full context with --parallel 1. llama.cpp projected about 15.1 GiB of Apple Metal device memory for the text model and about 258 MiB worst-case memory for the projector, leaving enough headroom for Open WebUI and the rest of the local stack. The --reasoning off flag keeps OpenAI-compatible chat responses visible in clients that do not read separate reasoning fields.

With --mmproj, /v1/models advertised capabilities: ["completion","multimodal"]. In local validation, Open WebUI accepted an uploaded Pulumi logo image and Gemma 4 described it correctly. A small WAV file also worked through the OpenAI-compatible input_audio request shape, though llama.cpp logs still mark audio input as experimental.

Verify the LLM API

Open a new terminal and check if llama.cpp is responding:

curl http://127.0.0.1:18080/v1/models

The /v1/models endpoint should return the model ID unsloth/gemma-4-12b-it-GGUF. Now try a chat completion:

curl http://127.0.0.1:18080/v1/chat/completions \
 -H "Content-Type: application/json" \
 -d '{
 "model": "unsloth/gemma-4-12b-it-GGUF",
 "messages": [{"role": "user", "content": "Reply with exactly: OK"}],
 "temperature": 0,
 "max_tokens": 32
 }'

The chat prompt Reply with exactly: OK should return content OK. In validation, llama.cpp reported an output token velocity of about 20 tokens per second for a longer 160-token response.

Keep llama.cpp running after reboot

For a permanent setup, put the llama.cpp startup script and logs under a folder in your home directory and let launchd restart it when you sign in:

llm_home="$HOME/pulumi-gemma4-llm"
mkdir -p "$llm_home/logs" "$HOME/Library/LaunchAgents"

cat > "$llm_home/start-llama-server.sh" <<'EOF'
#!/bin/zsh
set -euo pipefail

export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"

exec "$HOME/pulumi-gemma4-llm/llama.cpp/build/bin/llama-server" \
 --hf-repo unsloth/gemma-4-12b-it-GGUF \
 --hf-file gemma-4-12b-it-Q8_0.gguf \
 --mmproj "$HOME/pulumi-gemma4-llm/models/mmproj-F16.gguf" \
 --host 127.0.0.1 \
 --port 18080 \
 --ctx-size 131072 \
 --parallel 1 \
 --jinja \
 --reasoning off
EOF

chmod +x "$llm_home/start-llama-server.sh"

cat > "$HOME/Library/LaunchAgents/com.pulumi.gemma4.llama-server.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
 <key>Label</key>
 <string>com.pulumi.gemma4.llama-server</string>
 <key>ProgramArguments</key>
 <array>
 <string>$llm_home/start-llama-server.sh</string>
 </array>
 <key>WorkingDirectory</key>
 <string>$llm_home</string>
 <key>RunAtLoad</key>
 <true/>
 <key>KeepAlive</key>
 <true/>
 <key>StandardOutPath</key>
 <string>$llm_home/logs/llama-server.out.log</string>
 <key>StandardErrorPath</key>
 <string>$llm_home/logs/llama-server.err.log</string>
</dict>
</plist>
EOF

launchctl bootout gui/$(id -u)/com.pulumi.gemma4.llama-server 2>/dev/null || true
launchctl bootstrap gui/$(id -u) "$HOME/Library/LaunchAgents/com.pulumi.gemma4.llama-server.plist"
launchctl kickstart -k gui/$(id -u)/com.pulumi.gemma4.llama-server

Check the launchd service and llama.cpp logs:

launchctl print gui/$(id -u)/com.pulumi.gemma4.llama-server
tail -f "$HOME/pulumi-gemma4-llm/logs/llama-server.err.log"

If you want to stop llama.cpp later, unload the launchd service:

launchctl bootout gui/$(id -u)/com.pulumi.gemma4.llama-server

Deploy Open WebUI with Pulumi and k3d

Now we’ll deploy Open WebUI into a local Kubernetes cluster. This provides a polished chat interface that connects to our host-native LLM.

First, install k3d if you haven’t already:

brew install k3d

Create a new cluster for this project:

k3d cluster create pulumi-gemma4-blog-qa

We’ll use the Pulumi program in pulumi/examples. This program defaults to runtimeMode=host, which creates a Kubernetes ExternalName service pointing to your host machine.

Why not run the LLM inside Kubernetes on this Mac? Pulumi can do that, and the example supports it with runtimeMode=cluster, but that path is meant for Linux hosts with NVIDIA or AMD GPU device plugins.

On macOS, llama.cpp enables Metal by default, and Metal acceleration is available to native macOS processes. k3d runs Linux containers through Docker Desktop, so those pods do not get direct access to the Mac’s Metal device. Docker’s own vLLM Metal announcement calls out the same boundary: Metal-backed inference runs natively on the host because there is no Metal GPU passthrough for containers. That is why this setup keeps inference host-native and lets Pulumi manage the Kubernetes UI, service wiring, and optional Tailscale access around it.

Clone the examples repo, navigate to the program directory, and initialize a new stack:

git clone https://github.com/pulumi/examples.git
cd examples/kubernetes-py-self-host-gemma4-llm
pulumi stack init gemma4-local

Configure the program to match your local setup:

pulumi config set hostLlmPort 18080
pulumi config set llmBaseUrl http://llm-server:18080/v1

The Kubernetes service named llm-server maps to host.k3d.internal. In our validation, we confirmed that a disposable k3d pod could reach the Mac’s llama.cpp API at http://llm-server:18080/v1/models after a CoreDNS restart.

kubectl rollout restart deployment coredns -n kube-system

Run pulumi up to deploy Open WebUI and connect it to host-native llama.cpp:

pulumi up

In our validation environment, this command successfully reached the resource synthesis phase without requiring Tailscale credentials because Tailscale exposure is opt-in.

Access Open WebUI through Tailscale

Tailscale allows you to access your private Open WebUI instance from any device on your tailnet. Note that we only expose the web interface, not the raw LLM API, to keep the system secure.

The base Open WebUI deployment works without Tailscale credentials. To expose the web UI on your tailnet, enable Tailscale resources and provide an explicit api_key or OAuth/identity token:

pulumi config set enableTailscale true
pulumi config set tailscale:apiKey YOUR_API_KEY --secret

Once configured, Pulumi will create a Tailscale device or proxy that routes traffic to your Open WebUI service.

Use the model with Pi

Open WebUI gives you a browser-based chat interface, but local models are also useful from coding agents. Pi is the local coding agent used for this validation; if you do not use Pi, treat this section as an example of how any OpenAI-compatible client can point at the same local endpoint. Pi can point at the same OpenAI-compatible llama.cpp endpoint and use the model running on your Mac.

For a fresh Pi config, create ~/.pi/agent/models.json with a local provider that points at the llama.cpp API:

{
 "providers": {
 "local-llama": {
 "baseUrl": "http://127.0.0.1:18080/v1",
 "api": "openai-completions",
 "apiKey": "local",
 "compat": {
 "supportsDeveloperRole": false,
 "supportsReasoningEffort": false
 },
 "models": [
 {
 "id": "unsloth/gemma-4-12b-it-GGUF",
 "name": "Gemma 4 12B Q8 (local llama.cpp)",
 "reasoning": false,
 "input": ["text"],
 "contextWindow": 131072,
 "maxTokens": 1024,
 "cost": {
 "input": 0,
 "output": 0,
 "cacheRead": 0,
 "cacheWrite": 0
 }
 }
 ]
 }
 }
}

Then set Pi to use that provider and model by default in ~/.pi/agent/settings.json:

{
 "defaultProvider": "local-llama",
 "defaultModel": "unsloth/gemma-4-12b-it-GGUF",
 "defaultThinkingLevel": "off",
 "hideThinkingBlock": true
}

If you already have Pi configuration files, merge the local-llama provider and defaults into your existing JSON instead of replacing the files.

Manage your local AI stack as code

Let Pulumi provision the Kubernetes cluster, service wiring, and Tailscale access around your self-hosted model, so you can rebuild the whole setup with pulumi up.

Get started

Advanced: Linux GPU in-cluster serving

If you’re running on a Linux host with an NVIDIA or AMD GPU, you can run the LLM directly inside the Kubernetes cluster. This requires the NVIDIA or ROCm device plugins.

The Pulumi program supports this through runtimeMode=cluster. In this mode, it deploys a LlmServer pod that manages the llama.cpp process within the cluster, using GPU resource requests to ensure hardware acceleration.

Cleanup

When you’re done, you can tear down the resources:

pulumi destroy
k3d cluster delete pulumi-gemma4-blog-qa
# Stop llama.cpp using the PID from your terminal
kill <PID>

How We Load Data into Snowflake in Seconds with Pulumi

Pablo Seibelt — Mon, 23 Feb 2026 00:00:00 +0000

When you manage dozens of data-loading pipelines, copying and pasting IaC configurations between them is a recipe for mishap. IAM policies can drift, naming conventions diverge, and every new source is a new opportunity to make a mistake — not to mention compound the problem of duplication. In this post, we’ll show you how you can identify and encapsulate common patterns into composable components and walk through the production lessons we’ve learned running 25+ pipelines for over three years.

What we’ll cover

If you’re loading data into Snowflake and want reusable, composable infrastructure, this post is for you. Here’s what we’ll cover:

Handling and validating GitHub webhooks with AWS Lambda
Streaming webhook payloads directly into Snowflake with Amazon Data Firehose
Wiring it all up with a reusable Pulumi ComponentResource

The companion template also includes S3 auto-ingest and batch loading patterns, which we’ll cover in upcoming posts. We also use Pulumi ESC to handle authentication to both AWS and Snowflake using OpenID Connect.

Our own Josh Kodroff wrote an excellent introduction to Snowpipe with Pulumi. This post builds on his work using the newest Snowflake and AWS provider APIs and the direct Firehose-to-Snowflake destination, which wasn’t available when Josh wrote his post. Some resource names and grant patterns will also differ if you’re comparing the two.

Architecture overview

The following diagram shows the architecture in more detail:

GitHub sends webhook events to a Lambda Function URL.
Lambda validates the HMAC (Hash-based Message Authentication Code) signature and forwards the payload to Amazon Data Firehose.
Firehose streams records directly into Snowflake via the Snowpipe Streaming API. Data appears in Snowflake within seconds.
S3 is used only as a backup destination for failed records.

The direct Firehose-to-Snowflake destination is an AWS-native feature that works with any Snowflake account.

Project setup

Start a new Pulumi Python project and choose uv for dependency management when prompted:

mkdir snowpipe-data-loading && cd snowpipe-data-loading
pulumi new python

Notice Pulumi.yaml shows uv as your selected toolchain:

name: snowpipe-data-loading
runtime:
 name: python
 options:
 toolchain: uv

Add the provider dependencies for AWS, Snowflake, GitHub, and the Random and TLS providers:

uv add pulumi-aws pulumi-snowflake pulumi-github pulumi-random pulumi-tls

That’s it. uv creates the virtual environment and lockfile automatically, and Pulumi uses uv run under the hood to execute your program.

All examples in this post are in Python, but Pulumi supports multiple languages. You can implement the same components in TypeScript, Go, .NET, Java, or YAML.

Managing credentials with Pulumi ESC

This project needs credentials for AWS, Snowflake, and GitHub. Rather than managing long-lived secrets locally, we can use Pulumi ESC to obtain dynamic, short-lived OIDC credentials for AWS and Snowflake at runtime. When you run pulumi up, ESC exchanges a Pulumi-issued OIDC token for temporary credentials from each provider and injects them into your stack config automatically. If you prefer not to use ESC, you can set credentials directly with pulumi config set --secret.

Here is a single ESC environment that handles all three providers:

values:
 aws:
 login:
 fn::open::aws-login:
 oidc:
 duration: 1h
 roleArn: arn:aws:iam::123456789012:role/pulumi-esc-oidc
 sessionName: pulumi-snowpipe-demo
 snowflake:
 login:
 fn::open::snowflake-login:
 oidc:
 account: <your-snowflake-account>
 user: ESC_SERVICE_USER
 organizationName: <your-org-name>
 accountName: <your-account-name>
 environmentVariables:
 AWS_ACCESS_KEY_ID: ${aws.login.accessKeyId}
 AWS_SECRET_ACCESS_KEY: ${aws.login.secretAccessKey}
 AWS_SESSION_TOKEN: ${aws.login.sessionToken}
 SNOWFLAKE_USER: ${snowflake.login.user}
 SNOWFLAKE_TOKEN: ${snowflake.login.token}
 pulumiConfig:
 aws:region: us-west-2
 snowflake:organizationName: ${snowflake.organizationName}
 snowflake:accountName: ${snowflake.accountName}
 snowflake:authenticator: OAUTH
 snowflake:role: PULUMI_DEPLOYER
 github:token:
 fn::secret: <your-github-pat>
 github:owner: <your-github-org-or-user>

Then reference the environment from your stack config:

# Pulumi.<stack>.yaml
environment:
 - <project>/my-snowpipe-env
config:
 snowpipe-data-loading:database: LANDING_ZONE_WEBHOOKS
 snowpipe-data-loading:environment: dev
 snowpipe-data-loading:webhook-repo: <your-test-repo-name>
 snowflake:previewFeaturesEnabled:
 - snowflake_table_resource

Depending on your preferences, you can split credentials into separate per-provider environments and compose them with imports and reuse across stacks.

To set up OIDC trust for each provider, see the AWS OIDC guide and the Snowflake OIDC login guide. For GitHub authentication options (fine-grained PATs, classic PATs, or GitHub Apps), see the pulumi-github provider docs.

Shared infrastructure

The direct streaming pipeline needs an S3 bucket for backup/errors, a Snowflake database, and a schema.

Building the direct ingestion ComponentResource

Amazon Data Firehose supports Snowflake as a native destination via the Snowpipe Streaming API. Firehose streams records directly into Snowflake.

The Lambda handler

The Lambda function is the entry point for GitHub webhooks. It validates the HMAC-SHA256 signature, wraps the payload in an envelope with the event type, and forwards it to Firehose. Create this as lambda/webhook_handler.py:

import hashlib
import hmac
import json
import os

import boto3

firehose = boto3.client("firehose")

STREAM_NAME = os.environ["FIREHOSE_STREAM_NAME"]
WEBHOOK_SECRET = os.environ["WEBHOOK_SECRET"]


def handler(event, context):
 body = event.get("body", "")
 signature = (event.get("headers") or {}).get("x-hub-signature-256", "")

 # Validate HMAC-SHA256 signature
 expected = "sha256=" + hmac.new(
 WEBHOOK_SECRET.encode(), body.encode(), hashlib.sha256
 ).hexdigest()

 if not hmac.compare_digest(expected, signature):
 return {"statusCode": 401, "body": "Invalid signature"}

 github_event = (event.get("headers") or {}).get("x-github-event", "unknown")

 # Wrap in envelope - newline-delimited for S3 backup where Firehose concatenates records
 record = json.dumps({
 "github_event": github_event,
 "payload": json.loads(body),
 }) + "\n"

 firehose.put_record(
 DeliveryStreamName=STREAM_NAME,
 Record={"Data": record.encode()},
 )

 return {"statusCode": 200, "body": "OK"}

The envelope format {"github_event": "<type>", "payload": {...}}\n is important. The github_event field (e.g., push, pull_request, star) comes from the x-github-event header and lets downstream queries filter by event type. The trailing newline delimits records in the S3 backup destination, where Firehose concatenates them into files.

Another interesting part of this snippet is that instead of manually creating a secret which would have to be copy-pasted into both your webhook configuration and your Lambda environment, we are using random.RandomPassword to generate and store it securely it in Pulumi state.

The secret is automatically wired to both the Lambda env var and the GitHub webhook config, and it rotates cleanly if you ever need to replace it.

Why direct Firehose to Snowflake?

Data like this is normally written to and loaded from Amazon S3. But with an S3 intermediate path, you must wait for Firehose to buffer records (60 seconds), then for Snowpipe to detect the new file and load it. Total latency: about two minutes. With the direct Snowflake destination, Firehose uses the Snowpipe Streaming API to insert records as soon as they arrive, in seconds.

The direct path also removes the need for S3 event notifications, SQS queues, external stages, and pipe resources. S3 is still used, but only as a backup destination for failed records.

The DirectSnowflakeIngestion component

The component creates everything needed for the direct path: a TLS key pair for Snowflake authentication, a Snowflake service user with least-privilege grants, the landing table, a Firehose delivery stream with destination="snowflake", and a Lambda function with a public URL. Create an empty components/__init__.py so that Python treats the directory as a package.

The full component lives in components/direct_snowflake_ingestion.py:

import json
from dataclasses import dataclass

import pulumi
import pulumi_aws as aws
import pulumi_snowflake as snowflake
import pulumi_tls as tls

# In the example repository, you will find this class imported from a common library file instead
@dataclass
class ColumnDef:
 """Column definition for a Snowflake table."""

 name: str
 type: str
 nullable: bool = True

def strip_pem_headers(pem: str) -> str:
 """Remove PEM header/footer lines, returning only the base64 content."""
 lines = pem.strip().split("\n")
 return "".join(lines[1:-1])


@dataclass
class DirectSnowflakeIngestionArgs:
 bucket_arn: pulumi.Input[str]
 bucket_name: pulumi.Input[str]
 database: pulumi.Input[str]
 schema_name: pulumi.Input[str]
 table_name: str
 table_columns: list[ColumnDef]
 snowflake_account_url: pulumi.Input[str]
 snowflake_role_name: str
 lambda_code: pulumi.Archive
 lambda_handler: str
 lambda_environment: dict[str, pulumi.Input[str]]
 table_comment: str = ""
 s3_prefix: str = "direct-webhooks"
 s3_backup_mode: str = "FailedDataOnly"
 buffering_interval: int = 0
 buffering_size: int = 1
 retry_duration: int = 60
 data_loading_option: str = "VARIANT_CONTENT_AND_METADATA_MAPPING"
 content_column_name: str = "CONTENT"
 metadata_column_name: str = "METADATA"

class DirectSnowflakeIngestion(pulumi.ComponentResource):
 function_url: pulumi.Output[str]
 firehose_stream_name: pulumi.Output[str]
 snowflake_user_name: pulumi.Output[str]

 def __init__(
 self,
 name: str,
 args: DirectSnowflakeIngestionArgs,
 opts: pulumi.ResourceOptions | None = None,
 ):
 super().__init__(
 "snowpipe:direct:DirectSnowflakeIngestion", name, {}, opts
 )

 # --- TLS key pair for Snowflake auth ---
 key_pair = tls.PrivateKey(
 f"{name}-keypair",
 algorithm="RSA",
 rsa_bits=2048,
 opts=pulumi.ResourceOptions(parent=self),
 )

 # --- Snowflake role, user, and grants ---
 sf_role = snowflake.AccountRole(
 f"{name}-sf-role",
 name=args.snowflake_role_name,
 opts=pulumi.ResourceOptions(parent=self),
 )

 user_name = f"FIREHOSE_{name.upper().replace('-', '_')}_USER"
 sf_user = snowflake.ServiceUser(
 f"{name}-sf-user",
 name=user_name,
 login_name=user_name,
 default_role=sf_role.name,
 rsa_public_key=key_pair.public_key_pem.apply(strip_pem_headers),
 opts=pulumi.ResourceOptions(parent=self),
 )

 # Landing table
 table = snowflake.Table(
 f"{name}-table",
 name=args.table_name,
 database=args.database,
 schema=args.schema_name,
 comment=args.table_comment,
 columns=[
 snowflake.TableColumnArgs(
 name=col.name, type=col.type, nullable=col.nullable,
 )
 for col in args.table_columns
 ],
 opts=pulumi.ResourceOptions(parent=self),
 )

 # Grants: DB USAGE, schema USAGE, table INSERT+SELECT
 snowflake.GrantPrivilegesToAccountRole(
 f"{name}-grant-db-usage",
 account_role_name=sf_role.name,
 privileges=["USAGE"],
 on_account_object=snowflake.GrantPrivilegesToAccountRoleOnAccountObjectArgs(
 object_type="DATABASE",
 object_name=args.database,
 ),
 opts=pulumi.ResourceOptions(parent=self),
 )

 snowflake.GrantPrivilegesToAccountRole(
 f"{name}-grant-schema-usage",
 account_role_name=sf_role.name,
 privileges=["USAGE"],
 on_schema=snowflake.GrantPrivilegesToAccountRoleOnSchemaArgs(
 schema_name=pulumi.Output.all(
 args.database, args.schema_name
 ).apply(lambda parts: f'"{parts[0]}"."{parts[1]}"'),
 ),
 opts=pulumi.ResourceOptions(parent=self),
 )

 table_name = args.table_name
 snowflake.GrantPrivilegesToAccountRole(
 f"{name}-grant-table",
 account_role_name=sf_role.name,
 privileges=["INSERT", "SELECT"],
 on_schema_object=snowflake.GrantPrivilegesToAccountRoleOnSchemaObjectArgs(
 object_type="TABLE",
 object_name=pulumi.Output.all(
 args.database, args.schema_name
 ).apply(
 lambda parts: f'"{parts[0]}"."{parts[1]}"."{table_name}"'
 ),
 ),
 opts=pulumi.ResourceOptions(parent=self, depends_on=[table]),
 )

 # --- Firehose IAM role (S3 backup write) ---
 firehose_role = aws.iam.Role(
 f"{name}-firehose-role",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Action": "sts:AssumeRole",
 "Principal": {"Service": "firehose.amazonaws.com"},
 }],
 }),
 opts=pulumi.ResourceOptions(parent=self),
 )

 aws.iam.RolePolicy(
 f"{name}-firehose-s3-policy",
 role=firehose_role.id,
 policy=args.bucket_arn.apply(
 lambda arn: json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Action": [
 "s3:AbortMultipartUpload",
 "s3:GetBucketLocation",
 "s3:GetObject",
 "s3:ListBucket",
 "s3:ListBucketMultipartUploads",
 "s3:PutObject",
 ],
 "Resource": [arn, f"{arn}/*"],
 }],
 })
 ),
 opts=pulumi.ResourceOptions(parent=self),
 )

 # --- Firehose delivery stream (Snowflake destination) ---
 stream = aws.kinesis.FirehoseDeliveryStream(
 f"{name}-firehose",
 destination="snowflake",
 snowflake_configuration=aws.kinesis.FirehoseDeliveryStreamSnowflakeConfigurationArgs(
 account_url=args.snowflake_account_url,
 database=args.database,
 schema=args.schema_name,
 table=args.table_name,
 role_arn=firehose_role.arn,
 user=sf_user.name,
 private_key=key_pair.private_key_pem_pkcs8.apply(
 strip_pem_headers
 ),
 data_loading_option=args.data_loading_option,
 content_column_name=args.content_column_name,
 metadata_column_name=args.metadata_column_name,
 s3_backup_mode=args.s3_backup_mode,
 buffering_size=args.buffering_size,
 buffering_interval=args.buffering_interval,
 retry_duration=args.retry_duration,
 snowflake_role_configuration=aws.kinesis.FirehoseDeliveryStreamSnowflakeConfigurationSnowflakeRoleConfigurationArgs(
 enabled=True,
 snowflake_role=args.snowflake_role_name,
 ),
 s3_configuration=aws.kinesis.FirehoseDeliveryStreamSnowflakeConfigurationS3ConfigurationArgs(
 bucket_arn=args.bucket_arn,
 role_arn=firehose_role.arn,
 prefix=f"{args.s3_prefix}/backup/",
 error_output_prefix=f"{args.s3_prefix}/errors/",
 ),
 ),
 opts=pulumi.ResourceOptions(parent=self, depends_on=[table]),
 )

 # --- Lambda function + Function URL ---
 lambda_role = aws.iam.Role(
 f"{name}-lambda-role",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Action": "sts:AssumeRole",
 "Principal": {"Service": "lambda.amazonaws.com"},
 }],
 }),
 opts=pulumi.ResourceOptions(parent=self),
 )

 aws.iam.RolePolicyAttachment(
 f"{name}-lambda-basic-execution",
 role=lambda_role.name,
 policy_arn="arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole",
 opts=pulumi.ResourceOptions(parent=self),
 )

 aws.iam.RolePolicy(
 f"{name}-lambda-firehose-policy",
 role=lambda_role.id,
 policy=stream.arn.apply(
 lambda arn: json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Action": ["firehose:PutRecord"],
 "Resource": [arn],
 }],
 })
 ),
 opts=pulumi.ResourceOptions(parent=self),
 )

 env_vars = {
 **args.lambda_environment,
 "FIREHOSE_STREAM_NAME": stream.name,
 }

 fn = aws.lambda_.Function(
 f"{name}-handler",
 runtime="python3.11",
 handler=args.lambda_handler,
 role=lambda_role.arn,
 timeout=30,
 code=args.lambda_code,
 environment=aws.lambda_.FunctionEnvironmentArgs(
 variables=env_vars,
 ),
 opts=pulumi.ResourceOptions(parent=self),
 )

 fn_url = aws.lambda_.FunctionUrl(
 f"{name}-function-url",
 function_name=fn.name,
 authorization_type="NONE",
 opts=pulumi.ResourceOptions(parent=self),
 )

 aws.lambda_.Permission(
 f"{name}-function-url-permission",
 action="lambda:InvokeFunctionUrl",
 function=fn.name,
 principal="*",
 function_url_auth_type="NONE",
 opts=pulumi.ResourceOptions(parent=self),
 )

 # --- Outputs ---
 self.function_url = fn_url.function_url
 self.firehose_stream_name = stream.name
 self.snowflake_user_name = sf_user.name

 self.register_outputs({
 "function_url": self.function_url,
 "firehose_stream_name": self.firehose_stream_name,
 "snowflake_user_name": self.snowflake_user_name,
 })

A few things to note:

TLS key pair for authentication. The component generates an RSA key pair using the pulumi-tls provider. The public key is assigned to the Snowflake service user; the private key (PKCS#8 format, base64-encoded) is passed to Firehose. No passwords or OAuth tokens are stored.
ServiceUser instead of User. Snowflake service users can’t log in interactively. They authenticate only via key pair, which is exactly what Firehose needs.
destination="snowflake" on Firehose. This tells Firehose to use the Snowpipe Streaming API rather than writing to S3. The s3_configuration block is still required, but only for backup/error records.
Immediate flushing. buffering_interval=0 and buffering_size=1 ensure records are sent to Snowflake as soon as they arrive, minimizing latency. Tune according to your needs.

Amazon Data Firehose does not connect from fixed IP addresses, so you cannot use Snowflake network policies to restrict access by IP. If your Snowflake account uses network policies, you have three options: use AWS PrivateLink (requires Snowflake Business Critical edition), allow public internet access for the Firehose service user, or switch to S3 auto-ingest via Snowpipe which does not require direct network access to Snowflake from Firehose.

Wiring it together

With the component defined, __main__.py wires the direct ingestion pipeline:

import pulumi
import pulumi_aws as aws
import pulumi_github as github
import pulumi_random as random
import pulumi_snowflake as snowflake
from components.direct_snowflake_ingestion import (
 DirectSnowflakeIngestion, DirectSnowflakeIngestionArgs,
)
from components.snowpipe_pipeline import ColumnDef

config = pulumi.Config()
database_name = config.get("database") or "LANDING_ZONE_WEBHOOKS"
environment = config.get("environment") or "dev"

# --- Shared infrastructure ---

# S3 bucket for backup/errors
bucket = aws.s3.Bucket(
 "data-landing-bucket",
)

# Snowflake database and schema
database = snowflake.Database("demo-database", name=database_name)
schema = snowflake.Schema(
 "demo-schema",
 name="GITHUB",
 database=database.name,
)

# --- Direct ingestion pipeline ---

webhook_repo = config.require("webhook-repo")

# Snowflake account URL for Firehose configuration
snowflake_config = pulumi.Config("snowflake")
snowflake_account_url = (
 f"https://{snowflake_config.require('organizationName')}"
 f"-{snowflake_config.require('accountName')}"
 f".snowflakecomputing.com"
)

# Landing table columns: CONTENT (webhook JSON) + METADATA (Firehose metadata)
DIRECT_COLUMNS = [
 ColumnDef(name="CONTENT", type="VARIANT", nullable=True),
 ColumnDef(name="METADATA", type="VARIANT", nullable=True),
]

# Step 1: Generate webhook secret for HMAC validation
direct_webhook_secret = random.RandomPassword(
 "github-direct-webhook-secret", length=32, special=False
)

# Step 2: Direct ingestion pipeline - Lambda validates, Firehose streams to Snowflake
direct = DirectSnowflakeIngestion(
 "github-webhooks-direct",
 DirectSnowflakeIngestionArgs(
 bucket_arn=bucket.arn,
 bucket_name=bucket.bucket,
 database=database.name,
 schema_name=schema.name,
 table_name="REPOSITORY_EVENTS_DIRECT",
 table_columns=DIRECT_COLUMNS,
 table_comment="GitHub webhook events loaded via direct Firehose to Snowflake",
 snowflake_account_url=snowflake_account_url,
 snowflake_role_name="FIREHOSE_DIRECT_LOADER",
 lambda_code=pulumi.AssetArchive({
 "webhook_handler.py": pulumi.FileAsset("lambda/webhook_handler.py"),
 }),
 lambda_handler="webhook_handler.handler",
 lambda_environment={"WEBHOOK_SECRET": direct_webhook_secret.result},
 ),
)

# Step 3: GitHub webhook - sends events to the Lambda Function URL
github.RepositoryWebhook(
 "github-direct-webhook",
 repository=webhook_repo,
 configuration=github.RepositoryWebhookConfigurationArgs(
 url=direct.function_url,
 content_type="json",
 secret=direct_webhook_secret.result,
 ),
 events=["push", "pull_request", "issues", "star"],
)

# Exports
pulumi.export("webhook_url", direct.function_url)
pulumi.export("firehose_stream", direct.firehose_stream_name)

That’s the entire pipeline. One component, one GitHub webhook, one secret. The DirectSnowflakeIngestion component handles the TLS key pair, Snowflake service user, landing table, Firehose stream, and Lambda function internally, and now you can reuse this component for as many pipelines as you need.

The full code for this example is available on GitHub:

github.com/pulumi-demos/examples/tree/main/python/aws-snowflake-data-loading-real-time

Testing the pipeline

Deploy the stack:

pulumi up

The entire stack deploys in about two minutes. Immediately after deployment, you’ll start seeing GitHub events flowing into Snowflake.

Before querying, grant the DATA_READER role to your Snowflake user:

GRANT ROLE DATA_READER TO USER <your-user>;

In production, you can manage this grant through Pulumi, manually, or automatically via SCIM provisioning from your identity provider.

No need to craft test payloads. Just interact with the test repo. Star it, push a commit, or open an issue, then wait about 30 seconds and query Snowflake using the least-privilege reader role:

USE ROLE DATA_READER;

SELECT CONTENT:github_event::STRING AS event_type,
 CONTENT:payload:repository:full_name::STRING AS repo,
 METADATA:IngestionTime::TIMESTAMP AS ingested_at
FROM LANDING_ZONE_WEBHOOKS.GITHUB.REPOSITORY_EVENTS_DIRECT
ORDER BY ingested_at DESC;

You should see rows with event types like star, push, or issues, real GitHub events flowing through the entire pipeline. The METADATA column includes Firehose metadata like IngestionTime, which you can use to track end-to-end latency.

Other loading patterns

Direct streaming is the fastest path, but two other patterns are available in the companion template for different requirements:

S3 auto-ingest via Snowpipe. Firehose buffers to S3, and Snowpipe auto-ingests new files. Latency is about two minutes. Best when you need S3 as the system of record or can’t use direct Snowpipe Streaming.
Batch loading. Your orchestrator (Airflow, Prefect, cron, etc.) runs COPY INTO on a schedule. Best for full control over timing and deduplication.

We’ll walk through both patterns in detail in upcoming posts.

Stamp out data pipelines with components

Encapsulate your Snowflake loading logic in a Pulumi component, then create new pipelines with a handful of configuration values instead of copied files.

Get started

Publishing as reusable components

Once your components are battle-tested, you can share them across teams and projects instead of copying files around.

The most straightforward approach: push your components to a Git repository with a PulumiPlugin.yaml file at the root:

runtime: python
name: snowpipe-components
version: 1.0.0

Consumers add the package to their project with pulumi package add:

pulumi package add github.com/your-org/pulumi-snowpipe@v1.0.0

Pulumi downloads the package and generates typed SDKs automatically. The consumer’s __main__.py imports your components as if they were local, but they’re versioned and pinned.

Pulumi Cloud Private Registry

For organization-level discoverability, publish to the Pulumi Cloud Private Registry with pulumi package publish:

pulumi package publish ./schema.json

This gives you auto-generated API docs, usage tracking across teams, and cross-language SDK generation. Your Python components become usable from TypeScript, Go, and C# without any extra work. Teams browse available components in the Pulumi Cloud console, see who’s using what, and get notified when new versions are published. As an additional benefit, Neo will be able to use these components and build new pipelines in minutes from a natural language request.

Conclusion

ComponentResource is the key abstraction that makes this architecture scale. Instead of copying and pasting resources for each new data source, you instantiate a component with a handful of configuration parameters.

The DirectSnowflakeIngestion component in this post delivers data from GitHub webhooks into Snowflake in seconds: Lambda validates the HMAC signature, Firehose streams directly to Snowflake via the Snowpipe Streaming API, and the TLS key pair is managed entirely within Pulumi. No S3 intermediate, no SQS queues.

The component accepts pluggable Lambda handlers, so swapping GitHub for Stripe webhooks or any other source is just a matter of providing different lambda_code and lambda_environment arguments. We’ve been running this pattern in production for over three years across dozens of pipelines without significant changes to the infrastructure code.

You’ll find the complete example in the GitHub repository.

Host your Python app for $1.12 a month

Adam Gordon Bell — Wed, 29 Jan 2025 00:00:00 +0000

TL;DR (2026 pricing): Packaging a Flask app as a container and running it on AWS Lambda behind an HTTP API Gateway costs ~$1.12/month worst-case, or $0 when idle. The breakdown: $0.04 for API Gateway requests, $0.07 for Lambda compute (fully absorbed by the always-free tier), and $1.08 for 12 GB of egress at $0.09/GB (covered by AWS’s 100 GB/month free egress allowance unless other services in the account have already used it). Pulumi handles the entire infrastructure. Verified against AWS pricing as of April 2026.

How cheap can you host a Python app in 2026? For a low-traffic Flask API (say, 40,000 requests per month at 512 MB of memory), the answer is roughly $1.12/month on AWS worst-case, dropping to $0 when idle. The trick is to stop thinking of AWS Lambda as “one function per endpoint” and instead package your entire web framework as a container, deploy it to Lambda, and put it behind an HTTP API Gateway. Your code stays standard Flask. Your bill stays in the loose-change zone.

This post walks through the whole setup with Pulumi, then compares the resulting cost against Google Cloud Run, Fly.io, Railway, and Vercel using current 2026 prices.

Most developers maintain at least one low-traffic service that still needs to be reliably available: an internal reporting API, a webhook receiver, a side project with occasional use. Traditional hosting means paying for 24/7 server time, even when the service sits idle. AWS Lambda’s container support flips that: pay only when your API is being called, and any web framework that handles HTTP requests (Flask, FastAPI, Express) works without modification.

How does the pricing break down? (2026)

Plugging 40,000 requests per month at 512 MB memory and ~200 ms average execution into 2026 AWS pricing:

Lambda compute: 40,000 × 0.5 GB × 0.2 s = 4,000 GB-seconds × $0.0000166667 = $0.067, fully covered by the always-free tier (400,000 GB-seconds free per month).
Lambda requests: 40,000 × $0.20 / 1,000,000 = $0.008, also covered by the free tier (1M requests free per month).
HTTP API Gateway: 40,000 × $1.00 / 1,000,000 = $0.04.
Data transfer out: 12 GB × $0.09/GB = $1.08 if billed, but AWS’s always-free tier (expanded to 100 GB/month outbound in late 2021) covers this row outright unless other services in the same account have already exhausted that allowance.
Total: ~$0.04/month (egress in the free tier, only API Gateway bills) to ~$1.12/month (12 GB egress billed in full: $1.08 + $0.04 API Gateway), with Lambda compute and requests always fully covered by the free tier.

The underlying math hasn’t changed since this post was first published in early 2025; Lambda and API Gateway pricing has been remarkably stable, and AWS has only expanded free-tier allowances since.

Cheap Python hosting compared (2026)

Provider	Approx. cost / month	2026 notes
AWS Lambda + HTTP API	~$1.12	~$1.08 for 12 GB egress at $0.09/GB + $0.04 for 40k API Gateway requests at $1/M; Lambda compute ($0.067) is fully covered by the always-free tier (1M req/400k GB-s).
Google Cloud Run	~$1.44	2M requests, 360,000 GiB-seconds of memory, and 180,000 vCPU-seconds are free every month — compute is $0. Egress dominates: Premium tier is ~~$0.12/GB, so 12 GB ≈ $1.44. Standard tier (~~$0.085/GB) lands near $1.02.
Fly.io	~$2.18	Cheapest persistent `shared-cpu-1x@256MB` machine is ~$1.94/mo; 12 GB outbound at $0.02/GB (NA/EU) adds ~$0.24 (free tier was retired in late 2024).
Railway	~$6.00	Hobby plan starts at $5/mo and includes $5 of usage; a 512 MB always-on container plus 12 GB egress lands around $5.50–6.50 total.
Vercel (Pro)	$20+	Pro plan has a $20/mo floor; 40k function invocations are well under the included quotas, so you mostly pay for the seat.

Approximate 2026 hosting costs for a 512 MB Python API at 40,000 requests/month with 12 GB egress. Egress dominates at this scale; even 1 million requests/month stays under $30 on Lambda.

What we’ll cover:

Why use AWS Lambda for low-traffic Python apps?

“But Lambda means one function per REST endpoint, right?” While that’s how serverless was initially positioned, the Lambda monolith approach here is a more straightforward pattern. Take any REST API service, package it as a container, put it in Lambda, and give it an entire HTTP route behind an IP, domain name, or API Gateway prefix. Done.

This approach gives you several benefits:

Minimal Serverless Knowledge Required: You don’t need to be an expert in Lambda architecture or serverless patterns. If you can build a REST API and containerize it, you’re ready to go.
Standard Development Experience: Write your API exactly as you would for any other hosting platform: no special frameworks, no Lambda-specific patterns to learn.
Simple Local Development: Run your container locally, and it will behave exactly like it will in production. No complex serverless emulation is needed.
Zero Cost at Zero Usage: Get all the cost benefits of serverless without restructuring your application.

The trade-off? Cold starts when Lambda spins up a new container instance. But this is often a worthwhile exchange for operational simplicity and zero-cost idle periods for low-traffic services.

Let me show you how to set this up. With some Pulumi Python code for setting up the Infra and some containerization code, you get an easy hosting solution.

Why containers?

Containers let us package everything—code, system dependencies, runtime—into one unit. Any HTTP service that fits in a container will work: Flask, Express, Rails, plus whatever system tools you need. Local testing? Just start the container. No Lambda-specific configuration is required.

How do I build a Flask app for Lambda?

Let’s start with the fun part - writing some Python code, but I will keep it really simple. We’ll build a simple Flask application that demonstrates why running a full web framework in Lambda makes sense.

Here’s our starting point in scripts/simple_server.py

from typing import Dict
from flask import Flask, jsonify, request
from mangum import Mangum

# Create Flask app
app = Flask(__name__)

@app.route("/hello")
def hello() -> Dict[str, str]:
 return jsonify({"message": "Hello from Flask!"})

@app.route("/echo", methods=['POST'])
def echo() -> Dict[str, str]:
 data = request.get_json()
 return jsonify({"reversed": data['message'][::-1]})

# Create Lambda handler
lambda_handler = Mangum(app)

if __name__ == "__main__":
 app.run(host="0.0.0.0", port=3000)

Notice a few key things about this code:

It’s Just Flask: No Lambda-specific code, no special frameworks. If you know Flask, you know how to modify this.
Multiple Endpoints: We have several routes demonstrating different HTTP methods and URL patterns.

How do I containerize a Flask app for Lambda?

To containerize our Flask app for Lambda, we’ll start with a Dockerfile that meets AWS’s requirements. The key is using the right base image and adapting our application to Lambda’s runtime interface.

First, let’s create our Dockerfile:

FROM public.ecr.aws/lambda/python:3.12

# Set up working directory
WORKDIR /app
COPY scripts/simple_server.py scripts/
COPY scripts/simple_entrypoint.sh scripts/

# Install dependencies
RUN pip install --no-cache-dir \
 flask>=3.0.0 \
 mangum>=0.17.0

# Make entrypoint script executable
RUN chmod +x scripts/simple_entrypoint.sh

# Set environment variables
ENV FLASK_APP=scripts/simple_server.py
ENV PYTHONPATH=/app
ENV RUN_MODE=lambda
EXPOSE 3000

# Use our entrypoint script
ENTRYPOINT ["/app/scripts/simple_entrypoint.sh"]

Notice that:

We’re using AWS’s official Lambda Python base image. You can use your own base image and your Python package manager of choice, but starting with this base and pip install is the easiest.
The ${LAMBDA_TASK_ROOT} is a special directory where Lambda expects to find your code
We’re introducing a new file: simple_entrypoint.sh

Do I need to learn Lambda-specific patterns?

The beauty of everything covered so far is that it’s just a container. You can work with it locally with Docker run, put it in Kubernetes, or run it in one of the 1000 places you can run containers. Lambda has some odd requirements about function calling that get in the way of this, but simple_entrypoint.sh saves the day:

#!/bin/bash

if [ "$RUN_MODE" = "local" ]; then
 echo "Starting in local mode..."
 exec python scripts/simple_server.py
else
 echo "Starting in Lambda mode..."
 exec python -m awslambdaric scripts.simple_server.lambda_handler
fi

All this means is that locally, you don’t need to know about lambdas at all. If you supply RUN_MODE=local it runs the standard way you are used to:

#!/bin/bash

# Build the container
docker build -t simple-flask -f infra/Dockerfile .
docker run -d -p 3000:3000 -e RUN_MODE=local simple-flask

# Test the endpoints:
# 1. Test GET /hello
curl http://localhost:3000/hello

# 2. Test POST /echo with a message
curl -X POST "http://localhost:3000/echo" \
 -H "Content-Type: application/json" \
 -d '{"message": "hello world"}'

This local testing setup is crucial because it helps catch issues before deployment.

The beauty of this setup is that once it works locally, it’ll work the same way in AWS.

And since this is just a standard Python application in a container, you can use all your favorite Python tools and patterns – your preferred package manager (pip, poetry, uv), pytest for testing, standard logging, type hints, linting, and pre-commit hooks.

( I’m skipping those details here to focus on deployment, but the container approach means there are no serverless-specific limitations on your Python workflow. )

How do I deploy a Lambda container with Pulumi?

Now let’s get our containerized Flask app into AWS. While you could click around in the AWS console, we’re going to do this properly with infrastructure as code. You are welcome to use Cloudformation or manual setup, but it’s nice to be able to keep everything in Python, and this is the Pulumi blog. Using Python to define our AWS resources means we get type-checking and proper IDE support.

Here’s our infrastructure code:

Don’t let the infrastructure code below intimidate you. It’s just typed Python code, hooking up the Lambda, that you can version control and modify easily.

from typing import Any
import pulumi
import pulumi_aws as aws
import pulumi_awsx as awsx
import json

# First, create an API Gateway
api_gateway: aws.apigatewayv2.Api = aws.apigatewayv2.Api("flask-api",
 name="Flask Lambda API",
 protocol_type="HTTP",
 route_selection_expression="$request.method $request.path"
)

# Create IAM role for Lambda
lambda_role: aws.iam.Role = aws.iam.Role("flask-lambda-role",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": "sts:AssumeRole",
 "Principal": {
 "Service": "lambda.amazonaws.com"
 },
 "Effect": "Allow",
 "Sid": ""
 }]
 })
)

# Attach basic execution policy - Lambda needs this to write logs
lambda_role_policy: aws.iam.RolePolicyAttachment = aws.iam.RolePolicyAttachment("lambda-role-policy",
 role=lambda_role.name,
 policy_arn="arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
)

# Create ECR repository for our container
repository: aws.ecr.Repository = aws.ecr.Repository("flask-app-repo",
 name="flask-app-repo",
 force_delete=True, # Makes cleanup easier for testing
 image_scanning_configuration=aws.ecr.RepositoryImageScanningConfigurationArgs(
 scan_on_push=True,
 )
)

# Build and push the Docker image to ECR
image: awsx.ecr.Image = awsx.ecr.Image("flask-app-image",
 repository_url=repository.repository_url,
 path=".", # Path to your Dockerfile
 platform="linux/amd64" # Important for M1/M2 Mac users
)

# Create the Lambda function
lambda_function: aws.lambda_.Function = aws.lambda_.Function("flask-app",
 name="flask-app",
 package_type="Image",
 image_uri=image.image_uri,
 role=lambda_role.arn,
 timeout=30,
 memory_size=512,
 environment={
 "variables": {
 "FLASK_ENV": "production",
 "PYTHONUNBUFFERED": "1"
 }
 }
)

# Create API Gateway stage
stage: aws.apigatewayv2.Stage = aws.apigatewayv2.Stage("api-stage",
 api_id=api_gateway.id,
 name="$default",
 auto_deploy=True
)

# Connect API Gateway to Lambda
integration: aws.apigatewayv2.Integration = aws.apigatewayv2.Integration("lambda-integration",
 api_id=api_gateway.id,
 integration_type="AWS_PROXY",
 integration_uri=lambda_function.arn,
 integration_method="POST",
 payload_format_version="2.0"
)

# Create catch-all route
route: aws.apigatewayv2.Route = aws.apigatewayv2.Route("catch-all-route",
 api_id=api_gateway.id,
 route_key="ANY /{proxy+}",
 target=integration.id.apply(lambda id: f"integrations/{id}")
)

# Allow API Gateway to invoke Lambda
lambda_permission: aws.lambda_.Permission = aws.lambda_.Permission("api-lambda-permission",
 action="lambda:InvokeFunction",
 function=lambda_function.name,
 principal="apigateway.amazonaws.com",
 source_arn=api_gateway.execution_arn.apply(lambda arn: f"{arn}/*/*")
)

# Export the API Gateway URL
pulumi.export("url", api_gateway.api_endpoint)

Let’s break down what’s happening here:

API Gateway: We create an HTTP API (v2) that will receive all web requests and forward them to our Lambda.
IAM Role: Lambda needs permissions to execute and write logs. We create a role with the minimum necessary permissions.
Container Registry: We create an ECR repository and use Pulumi’s awsx package to build and push our Docker image automatically.
Lambda Function: We create the Lambda function using our container image. Notice how we set memory and timeout - these are important for a web application.
Integration: We connect API Gateway to Lambda using the AWS_PROXY integration type. ANY /{proxy+} means we route everything through with the full path.

Need a custom domain, that’s easy to use as well. Setup some Route53 records, CNAME a domain name appropriately and you’ll have a container backed by a custom domain name.

Why is this approach worth it?

The cool thing about using Pulumi and lambdas is that they make complex infrastructure changes safe and repeatable.

Want to increase the Lambda memory? Change one number and run pulumi up.
You get complete AWS Lambda and API Gateway management for your low-traffic REST APIs
And Infrastructure testing capabilities mean you can validate your serverless configuration before deployment

You can find the full code in the service status monitor repo from my Python devops article. A larger version with rolling updates is coming soon.

Alternative approaches

There are other ways to run containers that scale to zero. Google Cloud Run offers similar functionality on GCP, and AWS App Runner is another AWS service that can do this. Both have similar pricing models—very cheap for low-volume services. And SST is a similar solution for getting TypeScript / JavaScript solutions into an AWS Lambda.

The beauty of the container approach used here is that switching between these services is straightforward. Since we’re just running a standard container with a REST endpoint, there’s nothing AWS-specific in our application code. If you need to move to a different platform later, you can take your container with you.

Conclusion

Running Flask applications in Lambda containers represents a sweet spot between development simplicity and operational efficiency. You get:

True zero-cost periods when your application isn’t being used
Familiar development patterns - it’s just Flask
Infrastructure as code with Pulumi
Automatic scaling without managing servers

The trade-offs - primarily cold starts and some AWS-specific configuration - are worth it for many applications, especially:

APIs with sporadic traffic
Cost-sensitive projects

The key is understanding that this isn’t just about saving money - it’s about reducing operational complexity. No more worrying about scaling policies, server maintenance, or capacity planning. Your application scales automatically, and you only pay for what you use.

Is this approach right for every application? No, but this combination of familiar local development with serverless operations is under utilized.

I’d love to hear about your experiences with monolithic serverless applications. What hosting solutions have you tried? What challenges have you encountered?

Get started with Pulumi on AWS: the official quickstart for the tooling used in this post.
AWS Lambda function reference: every property on the resource we used above.
Code, deploy, and manage a serverless REST API on AWS with Pulumi: the function-per-route alternative to the monolith pattern.
Controlling AWS costs with Lambda and Pulumi: patterns for keeping your bill predictable as traffic grows.
The cursed container iceberg: every other place you can run the same container if you outgrow Lambda.
Python for DevOps: the companion post that ships the full source code for this example.

Changelog

2026-04-30: Refreshed for 2026. Re-verified Lambda/API Gateway/egress math, replaced the comparison table with current Cloud Run, Fly.io, Railway, and Vercel pricing, restructured H2s as user questions, added an answer-first intro and TL;DR, added a worked cost example and a HowTo schema for the deployment steps.
2025-03-13: Minor copy edits.
2025-01-29: Original publication.

Top 15 Python Tools for DevOps

Adam Gordon Bell — Mon, 06 Jan 2025 00:23:24 -0500

Have you ever had one of those moments: Elasticsearch is crashing, logs are filling up too fast, or a deployment needs data from three different APIs. The big enterprise solutions aren’t quite right, and your team estimates six months to build something proper. But you know that with Python and a few choice libraries, you could hack something together by tomorrow.

Let's build a little web dashboard in Python.

That’s where Python really shines. It lets you be the scrappy problem-solver who can travel up and down the stack. But Python isn’t just for quick fixes. The same language that helps you hack tomorrow’s solution can also build your company’s long-term infrastructure.

This guide covers both ends of that spectrum. I’ll show you python packages for “we need it yesterday” moments, and then move into solutions that can grow with your team. And along the way I’ll build a little example service monitor and package capturer to demonstrate the python packages I’m covering. I’ll wrap it in a CLI, in a web dashboard and deploy it to AWS, all while staying in Python.

So here’s my list of Python libraries for DevOps work, starting from #15 to the most important last at number #1.

Dashboards / monitoring

Let’s start with building web dashboards. Long-term, it’s usually better if there’s an off-the-shelf tool for that specific dashboard you need. But if you can create a quick visual on a Django endpoint, you can move on to the next issue.

15 and 14. Django and Flask: (Django, Flask) While tools like Grafana excel at metrics visualization, sometimes you need custom dashboards that integrate with internal systems. These frameworks let you build dashboards to track and visualize various aspects of your systems. I find Django a great fit here.
13. Prometheus: (prometheus-client) This library is a versatile tool for metrics. Want to expose custom metrics to Prometheus? Use the client to create counters, gauges, or histograms. Need to pull data from a weird API and expose it as metrics? Write a quick exporter that fetches from your API every few minutes and translates the responses into Prometheus metrics.

from flask import Flask, render_template_string
import requests

app = Flask(__name__)

@app.route("/")
def index():
 try:
 response = requests.get("https://www.githubstatus.com/api/v2/status.json", timeout=5)
 if response.status_code == 200:
 data = response.json()
 status = data["status"]["indicator"]
 description = data["status"]["description"]
 else:
 status = "critical"
 description = "Unable to reach GitHub API"
 except Exception as e:
 status = "critical"
 description = f"Error: {e}"

 color = {
 "none": "bg-green-500",
 "minor": "bg-yellow-500",
 "major": "bg-red-500",
 "critical": "bg-red-700",
 }.get(status, "bg-gray-500")

 return render_template_string(
 """
 ...
 """,
 color=color,
 description=description,
 )

Code for a simple Dashboard

You might also like:

Task scheduling and orchestration

Sometimes you need to run things on a schedule. And while I love cron, sometimes you need more. Maybe you need to coordinate multiple tasks, handle retries, or manage complex job dependencies. Here’s where Python can help:

12. Schedule: (schedule) When cron is too heavy but you need more than a bash script. I use it for quick automation tasks like “check this API every hour and alert if something’s wrong.” It runs in your Python process - no system configuration needed.
11. RQ (Redis Queue): (rq) Perfect for local development queues. Need to prototype a GitHub webhook handler that’ll eventually use SQS? With Redis in Docker and RQ, you can have a working queue in minutes. No AWS needed until production.
10. Airflow: (airflow) The big gun of scheduling. Yes, it’s complex, but sometimes you need that power. I’ve seen teams replace entire ETL systems with a few Airflow DAGs. When you need to coordinate multiple steps, handle failures gracefully, and have everything visible in a nice UI, it delivers.

def schedule_url_checks():
 for url in urls.values():
 schedule.every(30).seconds.do(lambda url=url: check_url_status(url))

def run_scheduler():
 while True:
 schedule.run_pending()
 time.sleep(1)

if __name__ == "__main__":
 schedule_url_checks()
 threading.Thread(target=run_scheduler, daemon=True).start()

Adding Schedule to our code, so we can request in the background.

Network analysis and security

When your network issues go beyond what tcpdump can tell you, or when you need to automate security checks, Python is here to help:

9. Scapy: (scapy) The Python alternative to Wireshark for programmatic network traffic analysis. Need to figure out why your microservices aren’t talking to each other? Want to check your load balancer? Use it to capture and analyze packets with a few lines of code.
8. Bandit: (bandit) Security scanning for your CI pipeline. Point it at your Python codebase and it’ll flag common issues like hardcoded credentials or unsafe deserialization.

def packet_callback(packet):
 global packet_stats
 packet_stats["total_packets"] += 1
 if packet.haslayer("HTTPRequest"):
 packet_stats["http_requests"] += 1

def start_packet_sniffer():
 sniff(prn=packet_callback, store=False, timeout=60) # Capture packets every 60 seconds

if __name__ == "__main__":
 threading.Thread(target=start_packet_sniffer, daemon=True).start()

Adding packet sniffing to our code, mainly just because we can.

Containerization and cloud interaction

Sometimes you need more control over your containers than Dockerfiles can provide. Maybe you need to dynamically build images based on environment variables, or automate container management:

7. Docker SDK for Python: (docker-py) When you need to automate Docker operations programmatically. I use it to clean up old containers and images, check container health, and build images dynamically from configuration files. More powerful than chaining Docker CLI commands.
6. Dagger: (dagger-io) Write container build logic in Python instead of Dockerfiles. Need to build multiple similar containers with slight variations? Want to pull in files from different sources based on environment? It lets you programmatically define container contents and build steps. (Bonus: this same code can later be used in CI/CD pipelines.)

import dagger

async def main():
 # Connect to the Dagger Engine
 async with dagger.Connection() as client:
 container = (
 client.container()
 .from_("python:3.11-slim")
 .with_exec(["python", "-c", "print('Hello from Dagger!')"])
 )

 # Execute the container and get the output
 output = await container.stdout()
 print(output)

if __name__ == "__main__":
 import asyncio
 asyncio.run(main())

Using Dagger makes it easy to stay in Python while building containers.

Building command-line tools

Let's build a CLI version of our service status dashboard in Python.

Every useful script eventually needs to become a proper tool. When your team starts asking “can I use that script too?” it’s time to enhance your CLI skills:

5/4. Click/Typer: (Click, Typer) Turn your quick scripts into proper CLI tools. Instead of remembering python cleanup.py --older-than 7 --dry-run, you get tab completion, help text, and proper argument handling. Typer is particularly nice if you’re using modern Python - it uses type hints to build the CLI interface.
3. Rich: (rich) Make your terminal output actually readable. Need to display tables of data? Want progress bars for long-running tasks? Rich makes it easy to build user-friendly CLIs. I use it to turn wall-of-text logs into colored, formatted output that helps spot problems quickly.

2. The essential toolkit

The foundation of any DevOps tooling consists of these essential libraries:

Requests (requests) - For making HTTP requests
Pytest (pytest) - For testing your code
logging (Python logging) - For structured logging
YAML/JSON/TOML parsers (PyYAML, json, toml) - For config file handling

While not flashy, these libraries form the backbone of every tool I build. And so I update my little tool to have some tests, some logging and store the api status urls in a toml file.

Now, #1 and it’s IaC time. I can get my dashboard onto AWS in a lambda using Pulumi.

# Build and push the Docker image to ECR
image = awsx.ecr.Image("app-server-image",
 repository_url=repository.repository_url,
 context="..",
 dockerfile="Dockerfile",
 platform="linux/amd64"
)

Push my Image to ECR

1. Pulumi

While most tools in this list excel at quick solutions, infrastructure demands a more robust approach. This is where Pulumi stands apart. It’s not just another scrappy tool—it’s an enterprise-grade Infrastructure as Code platform that happens to harness Python’s power and flexibility.

With support for real Python code instead of configuration languages, you can define, deploy, and manage your cloud resources using familiar programming concepts and practices.

With Pulumi, you get:

Complete programming language support for using loops, functions, and classes in your infrastructure code
Infrastructure testing capabilities to validate your deployments before they go live
Container deployment solutions for managing Docker images and Lambda functions
Rich AWS integration for comprehensive cloud resource management
Infrastructure automation to streamline your deployment workflows

# Create a Lambda function
lambda_function = aws.lambda_.Function("custom-lambda",
 name="custom-lambda",
 package_type="Image",
 image_uri=image.image_uri,
 role=lambda_role.arn,
 timeout=30,
 memory_size=512,
 environment={
 "variables": {
 "FLASK_ENV": "production"
 }
 }
)

Start it up in a Lambda

Manage your cloud in Python

Define AWS Lambda, ECR, and the rest of your infrastructure with real Python instead of YAML, and reuse the same loops, functions, and tests you already know.

Get started

Bringing it all together

Dashboard is still pretty basic, but now it's on AWS with a little Python infrastructure code.

Here’s where Python excels - combining these tools to solve real problems. For example:

Need a dashboard for infrastructure costs? Combine Flask with the AWS SDK and Pulumi Insights.
Want to automate container cleanup? Mix Docker SDK with Schedule for regular maintenance.
Building a custom GitHub webhook processor? Use Flask to receive webhooks, RQ for asynchronous handling, and Rich for readable logs.

Start simple (maybe just a script with Requests and PyYAML), then gradually add more tools as needed. That’s the beauty of Python in DevOps - you can start small and grow your solution naturally.

The art of getting things done

The tools I’ve covered span the full spectrum of DevOps needs—from quick scripts to enterprise-grade solutions. Python’s versatility makes it uniquely suited for both ends of this spectrum:

For urgent needs, tools like Schedule, Rich, and Click let you rapidly prototype solutions.
For long-term infrastructure, Pulumi and Airflow provide the robustness needed for production systems.
For everything in between, Python’s extensive library ecosystem helps you gradually evolve quick fixes into mature tools.

The key is knowing when to use each tool.

Pulumi + uv: Fast Python Package and Project Management

Adam Gordon Bell — Wed, 27 Nov 2024 12:43:45 +0100

Continuing our work to bring the best of modern Python to Infrastructure as Code, we are excited to announce built-in support for uv in Pulumi. uv is an extremely fast Python package manager that can install dependencies up to 100x faster than traditional tools, providing one of the fastest ways to manage your Python dependencies and virtual environments.

Why uv?

Listen in on this discussion between Adam Gordon Bell and Julien Poissonnier as they discuss Pulumi + uv and why uv is so fast!

uv brings several key advantages to your Python development workflow:

Blazing Fast Performance: Written in Rust, uv can install packages 10-100x faster than pip
Reliable Dependency Resolution: Advanced resolution algorithm ensures consistent environments
Drop-in Replacement: Works seamlessly with existing requirements.txt and pyproject.toml files
Built-in Virtual Environment Management: Simplified environment handling out of the box

Using uv in Pulumi

To use uv to manage your Python virtual environment and dependencies, set the toolchain option to uv in your Pulumi.yaml file:

name: uv-goes-brrrr
runtime:
 name: python
 options:
 toolchain: uv
 virtualenv: .venv

If you have uv installed, you can run pulumi new python and select uv as the toolchain to use for installing dependencies and running the program.

You can find more information on how to use uv in Pulumi in our Python documentation.

Pulumi + Python: Bringing the Best of Modern Python to IaC

Julien Poissonnier — Wed, 03 Jul 2024 14:17:38 +0200

One of our most important goals at Pulumi is to bring the very best of software engineering tools and practice to your cloud infrastructure. Pulumi embraces existing popular programming languages as a way to tap into everything that these language ecosystems offer developers – rich programming models, familiar syntax, IDE productivity, package management and versioning, componentization and reuse, testing, and so much more.

Python in particular has been one of the most popular languages used across the Pulumi platform, and the fastest growing over the last 2 years. Over the last few months, we’ve worked on a set of improvements that allow Pulumi users to take advantage of the latest and greatest tooling and features from the Python ecosystem. The key new features include:

More Pythonic resource APIs: A new dictionary-based API design which embraces a simple and familiar Pythonic API design along with access to great type checking and tooling building on recent enhancements to TypedDict support in the Python programming language and tools.
Native support for Poetry: Built-in support for working with Poetry in place of pip, making it easy to integrate Pulumi into projects and repositories leveraging the popular Poetry package management toolchain for Python.
Type checker integration: New ability to run your favorite Python type checkers directly within the Pulumi deployment process.

Pulumi + Python

Python was one of the first languages we ever supported in Pulumi, and has grown rapidly for defining and building Infrastructure as Code across any cloud. This growth has accelerated over the last year, driven by a few different trends:

DevOps practitioners with existing Python scripting experience who can naturally extend that experience into their IaC development
AI and Data-focused organizations with deep Python focus broadly within their organization leaning on IaC development which leverages the same language and tools
Broad growth in the Python developer ecosystem

The result has been strong and steady growth of Python Pulumi usage. This has been especially pronounced among some of the largest and most advanced Pulumi users, who are pushing the boundaries on managing complexity at scale using the best of the Python ecosystem.

New Features for Pulumi Python Users

Native Support for Poetry

Pulumi has long had support to create and manage virtual environments using Python’s builtin package manager pip. With the latest release of Pulumi, we are excited to announce that we now support Poetry as well. Poetry is a popular Python dependency management tool that allows you to declare your dependencies in a simple and concise way and manage your virtual environment with ease.

When creating a new Pulumi project, the Pulumi CLI will now ask you if you want to use Poetry to manage your dependencies. If you choose to use Poetry, the Pulumi CLI will automatically create a new Poetry project for you and install the necessary dependencies. To opt-in to using Poetry for an existing Pulumi project, set the toolchain runtime option to poetry and run pulumi install.

name: python-and-poetry-are-best-friends
runtime:
 name: python
 options:
 toolchain: poetry

Built-in type checking

Pulumi Python SDKs include type hints compatible with type checkers such as MyPy and Pyright. In the latest release of Pulumi, you can ask Pulumi to run your typechecker of choice for you as part of Pulumi operations and fail if there are any type errors. This can help you catch type errors earlier and ensure that your Pulumi programs are type-safe.

Add the typechecker of your choice to your dependencies and set the typechecker runtime option in your project file to mypy or pyright to enable automatic type checking.

name: python-with-typechecking
runtime:
 name: python
 options:
 typechecker: pyright

To enable live type checking in Visual Studio Code set the typeCheckingMode setting of the Pylance extension to basic or higher.

"python.analysis.typeCheckingMode": "basic"

Extensions for other editors typically have this already enabled.

We have updated the NGINX on AWS ECS Fargate example to use builtin typechecking and Poetry to manage dependencies. Check it out to see how you can use these new features in your Pulumi projects.

Pythonic input types

Pulumi has always had a focus on making Infrastructure as Code feel like idiomatic code and the latest release of Pulumi makes it easier to work with input types in Python.

Pulumi supports two ways of specifying inputs for resources in Python. You can use strongly typed, but verbose, argument classes, or you can use dictionaries, which are more idiomatic, but lack type hints. Leveraging improvements to Python’s TypedDict types, we can offer the best of both worlds – dictionary inputs now have type hints, providing you with type safety and the flexibility and simplicity of dictionaries.

This really pays off with large nested data structures, such as those you might use when configuring Kubernetes resources. Consider this program that uses argument classes to configure a deployment specification:

from pulumi_kubernetes.apps.v1 import Deployment
from pulumi_kubernetes.apps.v1 import Deployment, DeploymentSpecArgs
from pulumi_kubernetes.meta.v1 import LabelSelectorArgs, ObjectMetaArgs
from pulumi_kubernetes.core.v1 import ContainerArgs, PodSpecArgs, PodTemplateSpecArgs

app_labels = {"app": "nginx"}
deployment = Deployment(
 "nginx",
 spec=DeploymentSpecArgs(
 selector=LabelSelectorArgs(match_labels=app_labels),
 replicas=1,
 template=PodTemplateSpecArgs(
 metadata=ObjectMetaArgs(labels=app_labels),
 spec=PodSpecArgs(containers=[ContainerArgs(name="nginx", image="nginx")]),
 ),
 ),
)

With type hinted dictionaries, we can write the following, which is much more concise and just as type safe!

from pulumi_kubernetes.apps.v1 import Deployment

app_labels = {"app": "nginx"}

deployment = Deployment(
 "nginx",
 spec={
 "selector": {"match_labels": app_labels},
 "replicas": 1,
 "template": {
 "metadata": {"labels": app_labels},
 "spec": {"containers": [{"name": "nginx", "image": "nginx"}]},
 },
 },
)

The new Pythonic input types are available in the latest release of the Pulumi SDKs for Kubernetes, AWS and GCP, with more provider SDKs to follow soon.

Due to a performance issue in MyPy, type checking of TypedDict-based inputs is currently disabled when using MyPy as a type checker. We are working on helping resolve this issue. In the meantime, we recommend using Pyright as a type checker to get the most complete type checking possible for Dict-based Pulumi Python programs.

Build infrastructure in Python

Write your cloud infrastructure in Python with Poetry, built-in type checking, and idiomatic input types. Deploy your first stack, and manage it in Pulumi Cloud.

Get started

Summary

We hope you enjoy using these new features and we look forward to hearing your feedback. If you have any questions or need help, please don’t hesitate to reach out to us on Slack or GitHub.

Next-level IaC: Bridging the Declarative Gap

Troy Howard — Fri, 07 Jun 2024 00:00:00 +0000

Pulumi stands out in the world of infrastructure-as-code (IaC) for its flexibility and ease of use. This is due to its unique approach of using general purpose programming languages to describe system configuration. It accomplishes this through a small but brilliant implementation detail that bridges the worlds of declarative and standard programming using asynchronous semantics.

In Pulumi, you describe your infrastructure in code – real code, not a DSL or YAML – using your preferred general purpose programming language. You don’t have to become a specialist in a niche proprietary declarative language, like Terraform’s HCL. Rather, you can reuse your existing programming skills, writing in standard imperative, object-oriented, and even functional language styles, while still gaining all of the benefits of the declarative style that other tools emphasize.

Pulumi provides a unique mix of a declarative model embedded and implemented inside of a standard programming language, allowing all the flexibility of custom imperative code, while still enabling Pulumi’s deployment engine to infer opportunities for parallel asynchronous execution and to converge a partially-realized system.

The secret sauce that lets Pulumi achieve this is the apply function, and the way it handles inputs and outputs via asynchronous programming constructs, similar to futures or promises. We’ll go deeper on that in this article, but first let’s talk a little bit about the flow of operations in Pulumi.

Pulumi’s architecture and flow of operations

Pulumi is a system that is made up of a number of components. At a high level, those are:

Pulumi Programs: The code you write to describe the infrastructure you need.
Deployment Engine: A process that executes requests to create/modify infrastructure.
Resource Providers: Plugins that interface with cloud services like AWS, Azure, and others.

These components interact together to manage your infrastructure across multiple cloud providers using a consistent programming and execution model.

Figure: Pulumi’s Program, Deployment Engine, and Resource Provider architecture

Let’s get into an example of how information flows back and forth between these components. Suppose you want to write a Pulumi program to create an AWS S3 Bucket to host a static website. When that bucket is created, it will be given a unique identifier that you need to use in other parts of your infrastructure. Every time you create the bucket the identifier will change, and it’s generated by the AWS API, so we can’t know it ahead of time. We need to handle this as a variable, not a fixed configuration value.

To do that in a Pulumi program, you can create a bucket and then refer to its id property later, like this:

import * as pulumi from '@pulumi/pulumi';
import * as aws from '@pulumi/aws';

// Create an S3 bucket
const bucket = new aws.s3.Bucket("bucket");

// Export the name of the bucket
export const bucketName = bucket.id;

import pulumi
import pulumi_aws as aws

# Create an S3 bucket
bucket = aws.s3.Bucket('bucket')

# Export the name of the bucket
pulumi.export('bucket_name', bucket.id)

When this program runs, the call to create the bucket object will issue a request to the deployment engine. The deployment engine will load the resource provider for AWS, and ask it to create the S3 bucket. The AWS resource provider then issues the request to the AWS API, which will create the bucket and then return the id of the newly created resource. The provider will return that to the deployment engine, which will then set the realized value on your Pulumi program’s bucket.id property and then move on to creating anything that depends on that property.

This all seems pretty straight-forward, except that Pulumi programs don’t wait and imperatively do each operation, one at a time. Instead the deployment engine will run them in parallel, only waiting in cases where the output of one operation is needed to run the next operation. This can create complex structures with some operations needing to wait for inputs from other operations, and some that don’t, all finishing at different times, depending on how long the cloud service takes to process that request.

Inputs and Outputs as asynchronous futures

Infrastructure automation tasks can be long-running and can result in intermittent errors and may need to be retried before moving on to the next step. A robust IaC solution will need to run concurrently, be able to restart, retry, and resume itself, and understand that a set of tasks could happen in any order, not necessarily in the order you defined them in code.

Back in our Pulumi program code, that means that some of the time the bucket object from our earlier example has its id property set, and sometimes it doesn’t. We wouldn’t want to run any operation that needed that variable before it was set, but we also wouldn’t want to wait to start operations that don’t need that value.

To address this, Pulumi models the input and output values as futures. This allows you to describe a complex set of needs using a familiar value-like syntax. Instead of using a string result directly, which would only exist some of the time, you instead work with an Output<string>, that Pulumi understands as a reference to a value that currently doesn’t exist, but will exist when the operation completes. Similarly, Input<string> describes a dependency on an input string that will at some point be provided as the result of another operation.

This notion of future completion is essential to understanding why apply blocks are necessary within Pulumi programs. The apply function is called by the deployment engine only when the operation has completed and the fully realized value is available. By using Input<T> and Output<T> to build the connections between different parts of your infrastructure, the code is leveraging asynchronous value semantics not just to wrap and provide access to a future value, but also to describe a relationship between two components. This allows Pulumi programs to be very natural to author and as readable as traditional code, while also enabling Pulumi’s deployment engine to execute the entirety of your graph of operations in parallel, only waiting when necessary. This maximizes the efficiency of interacting with potentially slow and unreliable external cloud services.

Realizing values using the `apply` function

An important distinction though, is that Input<T>/Output<T> values are just declarative placeholders for the values that will eventually be present. At a certain point you will need to access the values directly, for example in the context of a transformation. The output of one operation might produce a string, which you need to transform into another data structure before passing to the next operation. Pulumi really shines in this context!

Using the apply function we can pass an Input<T> or Output<T> to a transformation function which will accept the realized value as its input (and only run when that value is available). Within that function you can handle the string as a normal string, then return it back as an Output<string>. This allows you to use the full power of your programming language to perform a custom operation inline with the rest of your infrastructure automation.

Here’s a simple example of using apply to transform an output to something new using your programming language’s libraries. In this case, Base64 encoding the id of our bucket object.

Example: Using `apply` to Base64 encode an Output

import * as pulumi from '@pulumi/pulumi';
import * as aws from '@pulumi/aws';

// Create an S3 bucket
const bucket = new aws.s3.Bucket("bucket");

// Export the name of the bucket
export const bucketName = bucket.id;

// Export the name of the bucket, in Base64 encoding
export const bucketNameBase64 = bucket.id.apply(id=> {
 // Transform the bucket name into a Base64 encoded string
 Buffer.from(id).toString('base64')
})

import base64
import pulumi
import pulumi_aws as aws

# Create an S3 bucket
bucket = aws.s3.Bucket('bucket')

# Export the name of the bucket
pulumi.export('bucket_name', bucket.id)

# A Base64 encoder function to use in the apply function's lambda
def base64_encode(data):
 data_bytes = data.encode("ascii")
 base64_bytes = base64.b64encode(data_bytes)
 base64_string = base64_bytes.decode("ascii")

# Export the name of the bucket, in Base64 encoding
pulumi.export('bucket_name', bucket.id.apply(lambda data: base64_encode))

Our example shows how to do Base64 encoding, but this technique can be used for any kind of transformation function. These can be as elaborate as your use case requires, and leverage the full power of your programming language and its entire ecosystem. Doing custom transformations like this in other popular tools is often not possible. This leaves you stuck using external coordination (e.g. gluing things together with Bash scripts) which breaks their declarative-only model (because you cannot describe everything you need to do in it) and interrupts these tools’ ability to fully understand your infrastructure’s dependency graph.

While Pulumi’s asynchronous semantics may seem somewhat complex for basic use cases, they unlock abilities in more complex use cases, which is when you really need your IaC tooling to show up for you. Because real life is never as simple as demo code.

Next steps

If you haven’t already, install Pulumi today, and follow our self-directed Getting Started guides to learn more about making the most of Pulumi’s next-level infrastructure management features at your organization.

To learn more, you can watch the following video which provides a high level overview of how Pulumi works:

*{padding:0;margin:0;overflow:hidden}html,body{height:100%}img{position:absolute;width:100%;top:0;bottom:0;margin:auto}">

Pulumi Cloud

The Pulumi Cloud is a fully managed service that helps you adopt Pulumi’s open source SDK with ease. It provides built-in state and secrets management, integrates with source control and CI/CD, and offers a web console and API that make it easier to visualize and manage infrastructure. It is free for individual use, with features available for teams.

Create an Account

Next-level IaC: Powering Up Pulumi with Package Ecosystems

Troy Howard — Mon, 13 May 2024 00:00:00 +0000

Every experienced tech professional I know has a programming language they love. But is it the syntax and symbols that make it so loveable? Not really. It’s the community and package ecosystem surrounding the language that makes a real impact on your heart… and on your productivity!

If we look at some of the biggest success stories in tech — Python, Node.js, Ruby, Perl, and Go — the common thread between all of them is an extensive ecosystem of packages, libraries, modules (or whatever you decide to call them… ahem, Gems?!). A great language will allow you to build anything you can imagine, but a great ecosystem will have already written it for you, and made it available in a convenient to install-and-use package.

One of the amazing things about Pulumi is that it is built around general-purpose programming languages, and that means your Pulumi programs have access to the entire ecosystem of packages that come with each language. This is a stark difference between Pulumi and other infrastructure automation tools that use proprietary domain-specific languages with not much in the way of community around them. Some tools might allow you to write custom code, but they certainly don’t make it convenient, and still… you have to write it yourself, which just adds so much overhead to a project where your core concern isn’t writing that custom function, but rather shipping your own product on the infrastructure you are trying to automate.

The flexibility that Pulumi gives you, and the access to these powerful libraries of code, means your productivity can skyrocket when you get to write in your favorite language, using your preferred tools and the package ecosystem you already know, love, and rely on every day.

Let’s have a look at a concrete example where this package ecosystem can get you out of a tough spot.

Use Case: Determining S3 MIME types on-the-fly

A popular example of cloud infrastructure automation is creating an S3 bucket and pushing some files to it. This is a common task that many of us have had to do, and most infrastructure automation tools have a way to accomplish this.

Here’s one way to do that in Pulumi:

import * as pulumi from '@pulumi/pulumi';
import * as aws from '@pulumi/aws';

// Create an S3 bucket where we will upload the file.
const bucket = new aws.s3.Bucket("my-bucket", {
 // You can apply additional bucket settings here.
});

const filePath = "path/to/your/local/file.txt"; // Replace with the path to the file you want to upload

// Upload the file to the S3 bucket.
const bucketObject = new aws.s3.BucketObject("my-bucket-object", {
 bucket: bucket, // Reference to the bucket created above.
 key: "file.txt", // This will be the file's name in the bucket.
 source: pulumi.FileAsset(filePath), // Contents of the file.
 contentType: "text/plain", // MIME-type of the file. Adjust if you upload a different type of file.
});

// Export the name of the bucket
export const bucketName = bucket.id;

// Export the URL of the file in the S3 bucket
export const objectUrl = pulumi.interpolate`s3://${bucket.id}/${bucketObject.key}`;

import pulumi
import pulumi_aws as aws

# Create an S3 bucket
bucket = aws.s3.Bucket('my-bucket')

# The path to the file to upload to S3
file_path = "path/to/your/local/file.txt" # Replace with the path to the file you want to upload

# Upload the file to the S3 bucket
bucket_object = aws.s3.BucketObject('my-bucket-object',
 bucket=bucket.id, # Reference to the bucket created above.
 key='file.txt', # This will be the file's name in the bucket.
 source=pulumi.FileAsset(file_path),
 content_type='text/plain') # MIME-type of the file. Adjust if you upload a different type of file.

# Export the name of the bucket
pulumi.export('bucket_name', bucket.id)

# Export the URL of the file in the S3 bucket
pulumi.export('object_url', pulumi.Output.concat('s3://', bucket.id, '/', bucket_object.key))

However, something that stands out as a weak link here is the line:

contentType: "text/plain", // MIME-type of the file. Adjust if you upload a different type of file.

content_type='text/plain') # MIME-type of the file. Adjust if you upload a different type of file.

Yeah, hard-coding the MIME type for every file I need to upload is a not going to scale. In fact, I’m probably going to get a random directory full of files I need to iterate over, each with different file types, and it will change every time it runs, and we won’t know what type of file it is ahead of time.

Packages to the rescue!! Turns out that determining the MIME type of a file only involves importing a community-created module like mime-types in TypeScript, or using the mimetypes module from Python’s standard library, and a single function call to guess the MIME type:

import * as mime from 'mime-types'

// Detect the MIME type of the file
const mimeType = mime.lookup('path/to/your/local/file.txt');

import mimetypes

# Detect the MIME type of the file
mime_type, _ = mimetypes.guess_type('path/to/your/local/file.txt')

So, to work that into our Pulumi program from above, we only need to make a few small edits:

import * as pulumi from '@pulumi/pulumi';
import * as aws from '@pulumi/aws';
import * as mime from 'mime-types';

// Create an S3 bucket where we will upload the file.
const bucket = new aws.s3.Bucket("my-bucket", {
 // You can apply additional bucket settings here.
});

const filePath = "path/to/your/local/file.txt"; // Replace with the path to the file you want to upload

// Upload the file to the S3 bucket.
const bucketObject = new aws.s3.BucketObject("my-bucket-object", {
 bucket: bucket, // Reference to the bucket created above.
 key: "file.txt", // This will be the file's name in the bucket.
 source: pulumi.FileAsset(filePath), // Contents of the file.
 contentType: mime.lookup(filePath) || "application/octet-stream", // MIME-type of the file
});

// Export the name of the bucket
export const bucketName = bucket.id;

// Export the URL of the file in the S3 bucket
export const objectUrl = pulumi.interpolate`s3://${bucket.id}/${bucketObject.key}`;

import pulumi
import pulumi_aws as aws
import mimetypes

# Create an S3 bucket
bucket = aws.s3.Bucket('my-bucket')

# The path to the file to upload to S3
file_path = "path/to/your/local/file.txt" # Replace with the path to the file you want to upload

# Detect the MIME type of the file
mime_type, _ = mimetypes.guess_type(file_path)

# Upload the file to the S3 bucket
bucket_object = aws.s3.BucketObject('my-bucket-object',
 bucket=bucket.id, # Reference to the bucket created above.
 key='file.txt', # This will be the file's name in the bucket.
 source=pulumi.FileAsset(file_path),
 content_type=mime_type or 'application/octet-stream') # MIME-type of the file

# Export the name of the bucket
pulumi.export('bucket_name', bucket.id)

# Export the URL of the file in the S3 bucket
pulumi.export('object_url', pulumi.Output.concat('s3://', bucket.id, '/', bucket_object.key))

Well that was pretty straightforward, wasn’t it? Trying to do this in another tool might take all day to figure out the right way to implement it, but in Pulumi it worked exactly like you would expect it to, in no time at all.

It’s a real game-changer when you can just quickly grab something complicated like MIME typing from a package. But when you can’t, you’re going to be stuck figuring out how to write that functionality yourself, from scratch, using things like Bash and primitive data types, and then find yourself copy-pasting that stuff all over the place. All the while fully aware of how much time you’re wasting by having to do so. That’s a real pain! It’s the kind of pain most people managing infrastructure have been through before.

The possibilities are endless

What other packages can you imagine using in your infrastructure solutions? Maybe you could use the text-to-image library to generate an image badge for your website? Maybe use the crypto module to encrypt a file before uploading it to S3? Or possibly using octokit to check on the state of something in git before taking the next step in your automation?

Whatever you can do with code in your favorite language, you can do in Pulumi as part of your infrastructure automation. I’m not saying you should use cowsay to generate custom MOTD banners for your ssh logins, I’m just saying you could.

Next Steps

To learn more, you can watch the following video which provides a high level overview of how Pulumi works:

*{padding:0;margin:0;overflow:hidden}html,body{height:100%}img{position:absolute;width:100%;top:0;bottom:0;margin:auto}">

Pulumi Cloud

Create an Account

Next-level IaC: Drop those wrapper scripts and let your language do that for you

Christian Nunciato — Tue, 30 Apr 2024 00:00:00 +0000

Our users are always telling us (particularly the ones who come to Pulumi from other IaC tools) that being able to use general-purpose languages to manage their infrastructure was a game changer for them.

I know it was for me. As a JavaScript developer, when I discovered Pulumi and saw that I could do pretty much everything I was doing with Terraform but with TypeScript, I was immediately hooked; that’s all it took. Just being able to write my resource declarations in a language I knew well (and that my IDE understood) was huge.

But it’s easy to forget, even when you’ve been using Pulumi for a while, that you aren’t just getting the language alone — you’re getting the full capability of the language’s runtime environment also. Not just JavaScript, but all of Node.js. Not just C#, but everything the .NET framework, runtime, and ecosystem have to offer.

This is quite a big deal, actually. Whereas with other tools, you’re often bound by the limits of the tool’s DSL, with Pulumi, you have no such constraints; you can do just about anything your chosen language can do. That means much more than just declaring long lists of cloud resources — it means unlocking all kinds of capabilities that can help bring your infrastructure-management practice to a whole new level.

In this post, the first of a new series we’re calling Next-level IaC, we’ll show you a few small but useful ways you can take advantage of this reality — first, by spotting some opportunities to simplify your deployments.

Run pre-deployment tasks with Pulumi

Fundamentally, a Pulumi program is just a regular program. That means you can take actions programmatically at runtime — before the Pulumi engine gets going.

Suppose you were tasked with writing the code to deploy and manage a static website. Conceptually, you might break this process up into a few separate steps:

Build the website
Deploy the supporting infrastructure (e.g, the cloud storage, maybe a content-delivery network)
Copy the content of the website into the infrastructure

This you might lead you to reach for Bash to write up a shell script like the one below, which uses Hugo (a popular static-site generator) to build the website, Pulumi to provision the infrastructure, and the AWS CLI to push the built website up into the AWS cloud:

#!/bin/bash

# Build the website.
hugo --destination ./public

# Provision the infrastructure.
pulumi up --yes

# Copy the website content into the infrastructure.
aws s3 sync ./public s3://$(pulumi stack output bucketName)

And this would definitely work. But it’s also unnecessary. You can avoid both the Bash wrapper and the dependency on the AWS CLI by using your chosen language’s built-in support for spawning shell processes:

import * as aws from "@pulumi/aws";
import * as childProcess from "child_process";

// Build the website.
const result = childProcess.execSync("hugo --destination ./public", { stdio: "pipe", cwd: "./www" });

// Provision a storage bucket for the website.
const bucket = new aws.s3.Bucket("bucket", {
 website: {
 indexDocument: "index.html",
 },
});

// Copy the website home page into the bucket.
const homepage = new aws.s3.BucketObject(
 "index.html",
 {
 bucket: bucket.id,
 content: fs.readFileSync("./www/public/index.html", "utf-8"),
 contentType: "text/html",
 acl: "public-read",
 },
);

import pulumi_aws as aws
import subprocess

# Build the website.
result = subprocess.run(
 ["hugo", "--destination", "./public"],
 stdout=subprocess.PIPE,
 cwd="./www",
 check=True,
 shell=True,
)

# Provision a storage bucket for the website.
bucket = aws.s3.Bucket(
 "bucket", website=aws.s3.BucketWebsiteArgs(index_document="index.html")
)

# Copy the website home page into the bucket
homepage = aws.s3.BucketObject(
 "index.html",
 bucket=bucket.id,
 content=open("./www/public/index.html", "r").read(),
 content_type="text/html",
 acl="public-read",
)

Running processes like these synchronously, as above, means they’ll complete before Pulumi starts registering resources — important here because the BucketObject resource needs that public folder to exist for the deployment to succeed. Fortunately it does, so all is well:

$ pulumi up
Updating (dev)
Type Name Plan Info
+ pulumi:pulumi:Stack example-dev create 15 messages
+ ├─ aws:s3:Bucket bucket create
+ └─ aws:s3:BucketObject index.html create
Diagnostics:
pulumi:pulumi:Stack (example-dev):
Start building sites ...
| EN
-------------------+-----
Pages | 4
Paginator pages | 0
Non-page files | 0
Static files | 0
Processed images | 0
Aliases | 0
Sitemaps | 1
Cleaned | 0
Total in 11 ms
Resources:
+ 3 created

Also notice how the example captures the output of the process in result and writes it to the terminal. Standard logging like this tends not to be possible with static languages (at least not without some hackery) because there’s no real way to do it programmatically. Similarly, those that do support spawning processes tend to require them to return strictly formatted JSON strings — which, as the example shows, not all processes do. (Forcing you to have to wrap them with yet more shell scripts.)

Simple stuff, but flexible and powerful. With one line, you’ve kept your deployment process simple, dropping two tools you don’t need thanks to the built-in capabilities of Pulumi and your language of choice.

Fetch data to generate resources dynamically

To build on the static-website scenario, suppose you wanted to retrieve some data from an external source and use it to provision some cloud resources dynamically. For example, you might want to let your marketing team create short URLs on their own for social media posts. For this, you might once again turn to Bash and the AWS CLI to pull those URLs from your corporate CMS (e.g., with curl), either before or after running pulumi, and create the corresponding redirects imperatively using a follow-up shell command of some sort.

Again, a totally reasonable approach. But again, why bother, when you have the full capability of Node.js Node.js Python Go .NET Java Pulumi YAML .NET .NET at fingertips? Just add a few lines to your program to fetch the data and be on your way:

// ...

// Fetch some redirects from a hypothetical CMS.
const redirects = fetch(`${process.env.CMS_ENDPOINT}/redirects.json`)
 .then(response => response.json())
 .then(items =>
 items.forEach((redirect: any, i: number) => {
 // Create an S3 website redirect for each one.
 new aws.s3.BucketObject(
 `redirect-${i}`,
 {
 bucket: bucket.id,
 key: redirect.from,
 websiteRedirect: redirect.to,
 acl: "public-read",
 },
 { dependsOn: [ownershipControls, publicAccess] },
 );
 }),
 );

#...
import json
import os
import requests

# ...

# Fetch some redirects from a hypothetical CMS.
response = requests.get(f"{os.environ['CMS_ENDPOINT']}/redirects.json")
redirects = json.loads(response.text)

# Create an S3 website redirect for each one.
for i, redirect in enumerate(redirects):
 aws.s3.BucketObject(
 f"redirect-{i}",
 bucket=bucket.id,
 key=redirect["from"],
 website_redirect=redirect["to"],
 acl="public-read",
 )

Notice the example parses the response using the language’s built-in JSON support. If your CMS happened to return data in some other format, or required some additional credentials for authorization, no problem — your language and its package ecosystem are going to be there to help you.

Also notice the example reads the source URL directly from the environment. This is typically more cumbersome (if even possible) with other tools, requiring you to name your environment variables in tool-specific ways and then tack on additional blocks of configuration before you’re able to use them. As you can see, Node.js Node.js Python Go .NET Java Pulumi YAML .NET .NET makes this much more convenient.

Finally, notice the redirects are translated into cloud resources using a familiar JavaScript TypeScript Python Go C# Java Pulumi YAML F# VB looping construct. We’ll cover this topic in much more depth in an upcoming post. (Stay tuned!)

Hook the event loop to run tasks after deployment

Finally, you can do nifty things like run tasks programmatically after a Pulumi deployment completes — once again, without ever having to leave the Pulumi program.

If your hypothetical website were sitting behind a CloudFront CDN, for example, you might want some way to clear the CDN’s cache after a deployment. This generally requires some imperative action, like using the AWS CLI to run aws cloudfront create-invalidation sometime later. How might you do something like this with Pulumi, given Pulumi programs exit automatically when their operations complete?

One way — again, since a Pulumi program is just a regular Node.js Node.js Python Go .NET Java Pulumi YAML .NET .NET program — would be to write a JavaScript TypeScript Python Go C# Java Pulumi YAML F# VB function to create an invalidation request, and then ask the Node.js Node.js Python Go .NET Java Pulumi YAML .NET .NET runtime to run that function before exiting.

The mechanics of this vary slightly by language, but it works, and it’s a very handy way to keep all of your deployment logic in one place — again avoiding the need to relegate anything to a Bash script:

// ...
import * as cloudfront from "@aws-sdk/client-cloudfront";

const config = new pulumi.Config("aws");
const region = config.require("region");

// Create a CloudFront distribution for the website.
const cdn = new aws.cloudfront.Distribution("cdn", {
 enabled: true,
 defaultRootObject: "index.html",
 origins: [{
 originId: bucket.arn,
 domainName: bucket.websiteEndpoint,
 // ...
 }],
 // ...
});

function createInvalidation(id: string) {
 // Only invalidate after a deployment.
 if (pulumi.runtime.isDryRun()) {
 console.log("This is a Pulumi preview, so skipping cache invalidation.");
 return;
 }

 process.on("beforeExit", () => {
 const client = new cloudfront.CloudFrontClient({ region });
 const command = new cloudfront.CreateInvalidationCommand({
 DistributionId: id,
 InvalidationBatch: {
 CallerReference: `invalidation-${Date.now()}`,
 Paths: {
 Quantity: 1,
 Items: ["/*"],
 },
 },
 });

 client
 .send(command)
 .then(result => {
 console.log(`Invalidation status for ${id}: ${result.Invalidation?.Status}.`);
 process.exit(0);
 })
 .catch(error => {
 console.error(error);
 process.exit(1);
 });
 });
}

// Register a function to be invoked before the program exits.
cdn.id.apply(id => createInvalidation(id));

The relevant code is the call to process.on("beforeExit"), which registers a function to be invoked just before the program exits using the resolved ID of the distribution. (For more on how the apply() method works, see Inputs and Outputs.) The function returns early for Pulumi previews — no sense clearing the cache if the site hasn’t changed — and uses the AWS SDK for JavaScript to submit the invalidation request to CloudFront, naming it uniquely with a timestamp and logging the result to the console:

# ...
import boto3
import asyncio

def create_invalidation(id):
 # Don't bother invalidating unless it's an actual deployment.
 if pulumi.runtime.is_dry_run():
 print("This is a Pulumi preview, so skipping cache invalidation.")
 return

 client = boto3.client("cloudfront")
 result = client.create_invalidation(
 DistributionId=id,
 InvalidationBatch={
 "CallerReference": f"invalidation-{time.time()}",
 "Paths": {
 "Quantity": 1,
 "Items": ["/*"],
 },
 },
 )

 print(
 f"Cache invalidation for distribution {id}: {result['Invalidation']['Status']}."
 )


# Register a function to be invoked before the program exits.
cdn.id.apply(lambda id: atexit.register(lambda: create_invalidation(id)))

The relevant code is the call to atexit.register(), which registers a function to be invoked just before the program exits using the resolved ID of the distribution. (For more on how the apply() method works, see Inputs and Outputs.) The function returns early for Pulumi previews — no sense clearing the cache if the site hasn’t changed — and uses Boto3 to submit the invalidation request to CloudFront, naming it uniquely with a timestamp and logging the result to the console:

$ pulumi preview
Previewing update (dev)
Type Name Info
pulumi:pulumi:Stack example-dev 16 messages
Diagnostics:
pulumi:pulumi:Stack (example-dev):
...
This is a Pulumi preview, so skipping cache invalidation.

$ pulumi up
Updating (dev)
Type Name Status Info
pulumi:pulumi:Stack example-dev 16 messages
Diagnostics:
pulumi:pulumi:Stack (example-dev):
...
Cache invalidation of distribution EAJOVC1QNGT13: InProgress.

Techniques like these are a great way to eliminate unnecessary scripts, glue code, and other dependencies while keeping your deployment logic clear, well contained, and amenable to change.

Wrapping up

Hopefully this first post has given you a few new ways to think about how to do more with Pulumi and your language of choice. Over the next few weeks, as this series unfolds, we’ll share a lot more, so keep an eye on this space!

You’ll find the full source for the examples above on GitHub:

Happy coding!

Next Steps

To learn more, you can watch the following video which provides a high level overview of how Pulumi works:

*{padding:0;margin:0;overflow:hidden}html,body{height:100%}img{position:absolute;width:100%;top:0;bottom:0;margin:auto}">

Pulumi Cloud

Create an Account

Deploy AI Models on Amazon SageMaker using Pulumi Python IaC

Kat Morgan — Mon, 11 Sep 2023 00:00:00 +0000

Running models from Hugging Face on Amazon SageMaker is a popular deployment option for AI/ML services. While the SageMaker console allows for provisioning these cloud resources, this deployment pattern is labor intensive to document and vulnerable to human errors when reproducing as a regular operations practice. Infrastructure as Code (IaC) offers a reliable and easy to duplicate deployment practice. By developing this IaC with Pulumi, practitioners can choose to write their infrastructure code in Python and seamlessly develop both AI application code and IaC code in the same language.

In this short tutorial we will deploy a publicly available Meta AI LlaMa 2 based model from Hugging Face, on Amazon SageMaker. Then we will test it with a natural language prompt using a short Python script.

Pulumi makes AI/ML Easy?

We will use the sagemaker-aws-python Pulumi Template to bootstrap our Python SageMaker IaC. Templates allow you to quickly bootstrap new Pulumi projects from a working scaffold which you can then customize further to your needs. Out of the box, this template provisions Amazon IAM Roles to assign SageMaker privileges, CloudWatch alarms to alert in case of latency or error spikes for the endpoint, and of course a Meta AI LlaMa 2 based LLM (NousResearch/Llama-2-7b-chat-hf) hosted on Hugging Face.

Requirements

Instructions

Let’s begin by logging into Pulumi Cloud:

# There are many ways to store Pulumi state, here we use Pulumi Cloud
# Other state backends include S3, local file, and more
pulumi login

If you’re unsure of which Pulumi state backend you’re using, you can check by running pulumi whoami (or pulumi whoami -v for more info).

2. Prepare a new Pulumi project

Here we create our new Pulumi project directory and populate it from the sagemaker-aws-python Pulumi template

# Create a new directory & change directories into it
mkdir newSageMaker && cd newSageMaker

# Start your project from the sagemaker-aws-python template
# Follow along with the prompts to create your new project and initialize a stack
pulumi new sagemaker-aws-python

While creating a new project from the sagemaker-aws-python Pulumi template, you will be prompted for a project name, description, stack name, and Amazon Web Service Region. You can proceed with defaults, or supply your own.

After responding to all prompts, the pulumi new command will proceed to setup a Python Virtual Environment (venv) and download all dependencies into the venv.

3. Deploy your model as a new SageMaker endpoint

This step may take between 10 and 20 minutes while Amazon builds your infrastructure and deploys the configured model. You can follow along in the console as resources are provisioned, or open the link displayed in terminal to view the deployment status and other stack information in Pulumi Cloud.

pulumi up

4. Try your new SageMaker endpoint

Once your stack has finished deploying, use this rudimentary Python snippet to test the deployed SageMaker endpoint.

NOTE: Notice that we are using us-east-1 in this script. Be sure to change the region in Python to match the region you deployed the SageMaker endpoint into.

First, save the following python snippet as test.py:

import json, boto3, argparse

def main(endpoint_name):
 client = boto3.client('SageMaker-runtime', region_name='us-east-1')
 payload = json.dumps({"inputs": "In 3 words, name the biggest mountain on earth?"})
 response = client.invoke_endpoint(EndpointName=endpoint_name, ContentType="application/json", Body=payload)
 print("Response:", json.loads(response['Body'].read().decode()))

if __name__ == "__main__":
 parser = argparse.ArgumentParser()
 parser.add_argument("endpoint_name")
 main(parser.parse_args().endpoint_name)

Once you have the test.py script created, source the Python virtual environment named venv (created automatically by Pulumi) and run the script using the name of your new endpoint, taken directly from the Pulumi stack output:

# Source the venv
# On Linux & MacOS
source venv/bin/activate

# Execute test.py
python3 test.py $(pulumi stack output EndpointName)

5. Cleanup all resources

Finally, when you’re finished with testing your Hugging Face model on SageMaker, you can easily clean up un-used resources with one easy command.

pulumi destroy

Conclusion

To recap, in a few commands, we created a new Pulumi Python project from a ready-to-roll template, deployed an LLM endpoint on Amazon SageMaker, and tested it with a short Python script to generate a response from our model!

AI and ML is rapidly becoming a necessity with every new day. It may appear daunting or out of reach at first glance, but with the power of IaC written as Pulumi Python programs, getting started has never been easier.

If you followed along then tell us how it worked out for you! We would love to know what you are looking forward to, or if you have ideas for future installments of the Pulumi Python #PulumiMLOps series!

Join us on Twitter, and on the Pulumi Community Slack to decide what #PulumiMLOps we take on next!

The Real AI Challenge is Cloud, not Code!

Kat Morgan — Wed, 30 Aug 2023 00:00:00 +0000

The AI industry is stealing the show as tech’s goldrush of the ’20s. Just looking at ChatGPT’s record setting user growth, and rapid 3rd party integration by top brands, it is not surprising the hype suggests this is the beginning of a major digital transformation.

However, using AI/ML in your own products has some major challenges and obstacles. Below is a diagram of the end to end workflow of building and using an AI model: preparing the data, training a model, fine-tuning a model, hosting and running a model, building a backend service to serve the model, and building the user interface that interacts with the model. Most AI engineers are only involved in a few steps of the process. However, there is one challenge that is common across the entire workflow: creating and managing the cloud infrastructure is hard.

Figure 1. A sneak peak at the map of ML development and product path

Training a model requires spinning up hyper scale GPU clusters. Preparing and consuming the training or fine-tuning data requires managing data warehouses and buckets of object storage. Building an AI backend service requires packaging the model, deploying and scaling the model across compute infrastructure, creating and managing the network infrastructure like load balancing, and managing the vector databases. Building great frontend user experiences requires provisioning and managing CDNs and web application services. Suffice to say: The most difficult challenge that we face with AI today is a problem of cloud orchestration.

Most of this cloud orchestration today is done manually or with complex scripts. This is usually fraught with errors and simply doesn’t scale. Too much time is spent making cloud infrastructure work instead of focusing on actual AI/ML tasks. Infrastructure as code (IaC) is a key component to solving the cloud orchestration challenge present in AI/ML. By applying the rigor and precision of software development practices to cloud operations, IaC offers AI/ML professionals an efficient, reliable, and predictable way to develop at the highest velocity. Pulumi is an open source infrastructure as code platform that allows engineers to use any programming languages, including Python, to provision and manage AI infrastructure.

This blog post is the first installment of a series on AI Infrastructure as Python. We will start this journey with building an AI backend service for a chatbot. In this post, we will deploy a LLM model behind an API backend running on compute infrastructure in the cloud. This entire infrastructure pipeline will be deployed using IaC written in Python. The series will follow this AI backend service through its evolution into a robust and resilient cloud native service.

The most difficult challenge that we face with AI today is a problem of cloud orchestration

Before we get into the details of using Pulumi to deploy a private LlaMa 2 API chatbot service—in 30 minutes or less, of course—let’s start with some quick terminology/context:

Katwalk Server is a demo AI application written to host your choice of LLM either locally or in the cloud, and serve it as an OpenAI API compatible service. As a practical, hands-on introduction to IaC for the MLOps space, this project aims to demystify the concepts and showcase the benefits Pulumi can bring to your AI/ML projects.
Large Language Models or LLMs (like ChatGPT) are machine learning models trained to generate human-like conversational text which have already become essential tools for businesses and individuals, transforming every day tasks and how we interact with technology.
LlaMa is an LLM created by Meta, available for free for research and commercial use.

Figure 2. Or just let ChatGPT describe itself

Deploy a private LlaMa 2 API chatbot service … in 30 minutes or less!

If you are ready to try the #MLOpsChallenge along with us, then let’s double check a few requirements before starting the clock on our 30 minute chatbot challenge. While Pulumi supports many languages, we chose Python for this project, as Python is a familiar language in the AI industry.

Requirements

Choose one of the following deployment platforms. (You can actually choose more than one, if you’d like!)

Now, for the real fun, let’s start the timer and deploy our own chatbot! Note that the instructions below are for Runpod; instructions for the other two deployment platforms are found in the GitHub repository.

1. Prepare your Pulumi IaC directory

First we need to clone our IaC repository from GitHub and prepare our Python environment.

# Clone the repository and cd to the pulumi iac directory
git clone https://github.com/pulumiverse/katwalk && cd katwalk/pulumi

# Create and initialize the python virtual env
python3 -m venv venv && source venv/bin/activate

# Install python dependencies
python3 -m pip install -r requirements.txt

Next, let’s setup our state and secrets store in Pulumi Cloud.

# There are many ways to store Pulumi state, here we use Pulumi Cloud
# Other state backends include S3, local file, and more
pulumi login

# Initialize your stack
# Here we name the stack "dev"
pulumi stack init --stack dev

3. Configure required credentials

Configure your Huggingface credentials. If you open the Pulumi.${stackName}.yaml file, you can see the encrypted secrets in this plain text YAML configuration.

# Set Huggingface.co username
pulumi config set hfUsername <huggingface_username>

# Configure token as secret
pulumi config set --secret hfToken <huggingface_api_token>

# Set the model that you want to download
pulumi config set hfModel "meta-llama/Llama-2-7b-chat-hf"

4. Deploy in the cloud on Runpod.io

While there are three providers already supported in this codebase, this post illustrates deploying to Runpod.io, a GPU and ML dedicated cloud catering to AI workloads and development tasks.

# This enables the IaC to deploy Katwalk Server
pulumi config set deploy True

# Set the deploy runtime to runpod to deploy on the Runpod cloud
pulumi config set runtime runpod

# Configure your Runpod API Token as a secure Pulumi secret
# API token available at https://www.runpod.io/console/user/settings
pulumi config set --secret runpodToken <your_runpod_api_token>

# Finally, run `pulumi up` to deploy!
pulumi up

# When done, you can 'destroy' the stack to deprovision your deployment
pulumi destroy

There’s currently a bug in the code that does not return the fully-qualified domain name (FQDN) from Runpod for your chatbot instance. To get the FQDN (referenced later in this post as RUNPOD_FQDN), you’ll need to log into Runpod and retrieve that information.

5. Build the Katwalk container

While there are many other ways to orchestrate ML services in the cloud and locally, in this project we are taking ownership of the full application and infrastructure story. In fact, all three supported providers use the same container image to serve the LlaMa LLM API Service, and you can build it yourself just like this:

# Set image build to True
pulumi config set imageBuild True

# Let's build this without pushing to a registry for now
pulumi config set imagePush False

# Give your katwalk image a project name
pulumi config set registryProject $(whoami)

# Now build the image!
pulumi up

6. Talk to your chatbot!

Once your Katwalk LlaMa server is running, you can interact with the API by sending prompts as a JSON payload. In future posts we will orchestrate a chat frontend for our chatbot but for now we’ll use a curl command. You can also substitute an API developer tool such as Postman or Insomnia!

# send a prompt with curl
curl -s -X 'POST' 'http://${RUNPOD_FQDN}/v1/chat' -H 'accept: application/json' -H 'Content-Type: application/json' -d '{"prompt": "Write a short story about a robot."}'
{
 "text": [
 "\nOnce upon a time, there was a little robot named R2. R2 was a friendly and curious robot who lived in a big city. One day, R2 decided to go on an adventure. He set out to explore the city and learn about all the different things he could see and do.\nAs R2 explored the city, he met all kinds of people. Some were kind and welcoming, while others were scared or suspicious of him. Despite this, R2 continued to be friendly and curious, always asking questions and trying to learn more about the world around him.\nOne day, while R2 was exploring a busy market, he saw a group of people gathered around a little girl who was"
 ]
}

Figure 3. A prompt and response from meta-llama/Llama-2-7b-chat-hf hosted on Katwalk Server

In a world where “data is the new black gold” and tokens are money, deploying a chatbot in 30 minutes is not just interesting, it is a novel capability quickly becoming a necessity.

Platform engineering, DevOps, or MLOps may be the next hardest step in building the AI future. Pulumi offers a reliable path to higher velocity AI App development and MLOps. Together with Python-based IaC, it empowers the MLOps community to build equity in common AI platform code, and share IaC in a familiar language.

If you followed along then tell us how it worked out for you! We would love to know what you are looking forward to, or if you have ideas for future installments of the Pulumi Python + MLOps series!

Join us in the Pulumi Community Slack or in the Pulumiverse Community Katwalk Github repository to decide what #aiChallenge we tackle next!

Figure 4. Midjourney generated image of an imagined machine learning research engineer working among cables, computers, keyboards, and screens, in the glow of electronic lights. Image was generated from a prompt ~80% produced by ChatGPT based on this blog post content and a few superficial details about the author

Advanced AWS Networking: Part 2

Josh Kodroff — Thu, 20 Apr 2023 00:00:00 +0000

In this blog series, you will learn how to create a hub-and-spoke network architecture in AWS with centralized egress and traffic inspection. In this second installment, we’ll show you how to create spoke VPCs to run your workloads, verify centralized egress is working, and then add centralized traffic inspection using Pulumi, the infrastructure as code tool that enables you to manage infrastructure with real programming languages!

Introduction

In AWS Advanced Networking Part 1 you:

Learned about the benefits of a hub-and-spoke network architecture with centralized egress.
Created the shared AWS Transit Gateway resources to be shared among our spoke VPCs and inspection VPC.
Defined your inspection VPC as a Pulumi component resource.

Now, to finish the architecture, you will need to:

Create a spoke VPC as a Pulumi component resource.
Test the network architecture by verifying that centralized egress is working.
Add traffic inspection with AWS Network Firewall.

In subsequent sections we will be showing code samples. Because this is a more advanced Pulumi program, we will be focusing on the most important parts of the architecture rather than covering every single line in the codebase. The complete code for this project is available on GitHub, which is a great place to go for full context.

Spoke VPC

Like your inspection VPC, we will implement the spoke VPC as a Pulumi component resource. Component resources allow you to logically group resources to form reusable components. Because your network will contain multiple spoke VPCs (at least 2, possibly more), and the spoke VPC will contain both the basic VPC and connectivity to Transit Gateway, the spoke VPC is a natural fit for a component resource.

First, add a file spoke.py to your Pulumi program. In Python, it’s good practice to put each component resource along with its associated arguments class in a separate file from your main program as it makes for intuitive code organization:

@dataclass
class SpokeVpcArgs:
 vpc_cidr_block: str
 tgw_id: pulumi.Input[str]
 tgw_route_table_id: pulumi.Input[str]


class SpokeVpc(pulumi.ComponentResource):
 def __init__(self, name: str, args: SpokeVpcArgs, opts: pulumi.ResourceOptions = None) -> None:
 super().__init__("awsAdvancedNetworking:index:SpokeVpc", name, None, opts)

Then, create a VPC with two sets of subnets, both isolated (with no pathway to the internet). One set of subnets will hold our spoke VPC workloads, and the other will hold our Transit Gateway attachment. We enable DNS hostnames and support so that we can later use VPC endpoints to enable AWS Systems Manager access to verify that centralized egress is working as expected without requiring direct host access via SSH:

self.vpc = awsx.ec2.Vpc(
 f"{self._name}-vpc",
 awsx.ec2.VpcArgs(
 cidr_block=args.vpc_cidr_block,
 subnet_specs=[
 awsx.ec2.SubnetSpecArgs(
 name="private",
 cidr_mask=28,
 type=awsx.ec2.SubnetType.ISOLATED,
 ),
 awsx.ec2.SubnetSpecArgs(
 name="tgw",
 cidr_mask=28,
 type=awsx.ec2.SubnetType.ISOLATED,
 ),
 ],
 nat_gateways=awsx.ec2.NatGatewayConfigurationArgs(
 strategy=awsx.ec2.NatGatewayStrategy.NONE,
 ),
 enable_dns_hostnames=True,
 enable_dns_support=True,
 )
)

Then, attach the Transit Gateway to the appropriate subnets:

self.tgw_attachment = aws.ec2transitgateway.VpcAttachment(
 f"{name}-tgw-vpc-attachment",
 aws.ec2transitgateway.VpcAttachmentArgs(
 transit_gateway_id=args.tgw_id,
 subnet_ids=tgw_subnets.apply(lambda x: x.ids),
 vpc_id=self.vpc.vpc_id,
 transit_gateway_default_route_table_association=False,
 transit_gateway_default_route_table_propagation=False,
 tags={
 "Name": f"{name}",
 },
 ),
 pulumi.ResourceOptions(
 delete_before_replace=True,
 depends_on=[self.vpc],
 parent=self,
 )
)

Add VPC endpoints to allow Systems Manager access to open an SSH connection from the AWS console without having to allow network access to the hosts. Because Pulumi uses real programming languages, you can use a Python for loop to create these resources with fewer lines of code using familiar syntax:

for service in ["ec2messages", "ssmmessages", "ssm"]:
 aws.ec2.VpcEndpoint(
 f"{self._name}-endpoint-{service}",
 aws.ec2.VpcEndpointArgs(
 vpc_id=self.vpc.vpc_id,
 service_name=f"com.amazonaws.{aws.config.region}.{service}",
 private_dns_enabled=True,
 security_group_ids=[vpc_endpoint_sg.id],
 vpc_endpoint_type="Interface",
 tags={
 "Name": f"{self._name}-{service}"
 },
 subnet_ids=subnet_ids
 )
 )

Add routes to the spoke VPC’s Transit Gateway route table that direct all non-local traffic to the Transit Gateway:

for subnet_id in private_subnet_ids:
 route_table = aws.ec2.get_route_table(
 subnet_id=subnet_id,
 )

 aws.ec2.Route(
 f"spoke{self._name}-tgw-route-{subnet_id}",
 aws.ec2.RouteArgs(
 route_table_id=route_table.id,
 destination_cidr_block="0.0.0.0/0",
 transit_gateway_id=self._args.tgw_id,
 ),
 pulumi.ResourceOptions(
 depends_on=[self.tgw_attachment],
 parent=self,
 ),
 )

Finally, add some outputs for the component. You’ll use these to place workloads in the proper subnets:

self.register_outputs({
 "vpc": self.vpc,
 "workload_subnet_ids": self.workload_subnet_ids
})

You can instantiate a spoke VPC like the following in __main__.py:

spoke1_vpc = SpokeVpc(
 "spoke1",
 SpokeVpcArgs(
 vpc_cidr_block="10.0.0.0/16",
 tgw_id=tgw.id,
 tgw_route_table_id=spoke_tgw_route_table.id,
 ),
)

Subsequent spoke VPCs should use a separate CIDR block:

spoke2_vpc = SpokeVpc(
 "spoke2",
 SpokeVpcArgs(
 vpc_cidr_block="10.1.0.0/16",
 tgw_id=tgw.id,
 tgw_route_table_id=spoke_tgw_route_table.id,
 ),
)

Now that you have your spoke VPCs added, it’s time to test that the architecture is working as expected. To do this, we’ll create a sample workload and deploy it to one of our spoke VPCs.

Creating a Sample Spoke VPC Workload

To test that internet traffic from your spoke VPC is going over the inspection VPC’s NAT gateway, you’ll add a workload (i.e., an EC2 instance running Amazon Linux) to your spoke VPC and verify that its public IP is that of the Elastic IP attached to the NAT gateway in the inspection VPC. Like the spoke VPC workload, the spoke VPC itself is a natural fit for a Pulumi component resource.

Your component should go in its own file spoke_workload.py, and will begin much like the spoke VPC component:

@dataclass
class SpokeWorkloadArgs:
 spoke_vpc_id: pulumi.Input[str]
 spoke_instance_subnet_id: str


class SpokeWorkload(pulumi.ComponentResource):
 def __init__(self, name: str, args: SpokeWorkloadArgs, opts: pulumi.ResourceOptions = None) -> None:
 super().__init__("awsAdvancedNetworking:index:SpokeWorkload", name, None, opts)

Start by creating a security group to allow outbound traffic. Note that this security group’s egress rules are overly permissive, but this is ok because the spoke workload is an ephemeral resource that only exists to verify network connectivity. In order for SSM access to work though, you must at least enable port 443:

sg = aws.ec2.SecurityGroup(
 f"{name}-instance-sg",
 aws.ec2.SecurityGroupArgs(
 description="Allow all outbound traffic",
 vpc_id=args.spoke_vpc_id,
 egress=[
 aws.ec2.SecurityGroupEgressArgs(
 cidr_blocks=["0.0.0.0/0"],
 description="Allow everything",
 protocol="-1",
 from_port=0,
 to_port=0
 ),
 ]
 ),
 opts=pulumi.ResourceOptions(
 parent=self
 ),
)

In addition to the security group which provides the necessary network access, you will also need to assign an EC2 instance role in order to complete the prerequisites for Systems Manager access to an EC2 instance which you’ll create shortly:

ec2_role = aws.iam.Role(
 f"{name}-instance-role",
 aws.iam.RoleArgs(
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": {
 "Effect": "Allow",
 "Principal": {
 "Service": "ec2.amazonaws.com",
 },
 "Action": "sts:AssumeRole",
 },
 })
 )
)

aws.iam.RolePolicyAttachment(
 f"{name}-role-policy-attachment",
 aws.iam.RolePolicyAttachmentArgs(
 role=ec2_role.name,
 policy_arn="arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
 )
)

instance_profile = aws.iam.InstanceProfile(
 f"{name}-instance-profile",
 aws.iam.InstanceProfileArgs(
 role=ec2_role.name,
 )
)

Finally, create an EC2 instance with the instance profile you created:

aws.ec2.Instance(
 f"{name}-instance",
 aws.ec2.InstanceArgs(
 ami=amazon_linux_2.id,
 instance_type="t3.micro",
 vpc_security_group_ids=[sg.id],
 subnet_id=args.spoke_instance_subnet_id,
 tags={
 "Name": f"{name}-instance",
 },
 iam_instance_profile=instance_profile.name,
 ),
 opts=pulumi.ResourceOptions(
 parent=self
 ),
)

Instantiate the workload in __main__.py:

spoke1_workload = SpokeWorkload(
 "spoke1",
 SpokeWorkloadArgs(
 spoke_instance_subnet_id=spoke1_vpc.workload_subnet_ids[0],
 spoke_vpc_id=spoke1_vpc.vpc.vpc_id,
 )
)

And deploy:

pulumi up

Now you have a workload with which you can verify that centralized egress is working!

Testing Centralized Egress

You can now test that centralized egress is working as expected. First, log into the AWS console and go the EC2 service homepage. Find your instance in the list of EC2 instances:

First, take note of the NAT gateway IP address which you included as a Pulumi stack output:

Then, in the AWS console, locate an instance running in a spoke VPC which you created as part of the workload component resource, and select Connect from the drop-down menu:

Connect to the instance to start an SSH session (note that the IAM principal must have permissions to perform SSM Systems Manager actions in order to connect):

Finally, verify your public IP address from the host in the spoke VPC:

If your public IP address for the instance in the spoke VPC matches the public IP address of the NAT gateway in the inspection VPC, you’ve successfully verified that centralized egress is working!

Adding AWS Network Firewall

AWS Network Firewall incurs significant costs (hundreds of USD per month). Be sure to tear down AWS Network Firewall and all other resources in this architecture with the pulumi destroy command to avoid incurring unwanted AWS charges.

Now that you’ve verified centralized egress is working, you can add centralized traffic inspection with AWS Network Firewall. In a new file called firewall_rules.py, add a rule to drop all traffic not destined for amazon.com:

allow_amazon = aws.networkfirewall.RuleGroup(
 "allow-amazon",
 aws.networkfirewall.RuleGroupArgs(
 capacity=100,
 name="allow-amazon",
 type="STATEFUL",
 rule_group=aws.networkfirewall.RuleGroupRuleGroupArgs(
 rules_source=aws.networkfirewall.RuleGroupRuleGroupRulesSourceArgs(
 rules_string='pass tcp any any <> $EXTERNAL_NET 443 (msg:"Allowing TCP in port 443"; flow:not_established; sid:892123; rev:1;)\n' +
 'pass tls any any -> $EXTERNAL_NET 443 (tls.sni; dotprefix; content:".amazon.com"; endswith; msg:"Allowing .amazon.com HTTPS requests"; sid:892125; rev:1;)'
 ),
 stateful_rule_options={
 "rule_order": "STRICT_ORDER",
 },
 )
 )
)

And add the rule to a firewall policy:

policy = aws.networkfirewall.FirewallPolicy(
 "firewall-policy",
 aws.networkfirewall.FirewallPolicyArgs(
 firewall_policy=aws.networkfirewall.FirewallPolicyFirewallPolicyArgs(
 stateless_default_actions=["aws:forward_to_sfe"],
 stateless_fragment_default_actions=["aws:forward_to_sfe"],
 stateful_default_actions=[
 "aws:drop_strict", "aws:alert_strict"],
 stateful_engine_options={
 "rule_order": "STRICT_ORDER"
 },
 stateless_rule_group_references=[{
 "priority": 10,
 "resource_arn": drop_remote.arn
 }],
 stateful_rule_group_references=[
 {
 "priority": 10,
 "resource_arn": allow_icmp.arn,
 },
 {
 "priority": 20,
 "resource_arn": allow_amazon.arn,
 },
 ]
 )
 )
)

Then, in inspection_vpc.py, you’ll need to add some additional network resources to create the firewall and route your egress traffic through the firewall. First create some subnets to contain AWS Network Firewall:

for i, inspection_subnet in enumerate(inspection_subnets):
 resource_name = f"{self.name}-inspection-{i+1}"
 subnet = aws.ec2.Subnet(
 resource_name,
 aws.ec2.SubnetArgs(
 vpc_id=self.vpc.vpc_id,
 availability_zone=inspection_subnet["az"],
 cidr_block=inspection_subnet["cidr"],
 tags={
 "Name": resource_name
 }
 ),
 opts=pulumi.ResourceOptions(
 parent=self,
 delete_before_replace=True,
 ),
 )

After creating route tables and associating with your subnets, create routes that send internal traffic to the TGW and external traffic to the NAT gateway:

aws.ec2.Route(
 f"{self.name}-insp-supernet-to-tgw-{i+1}",
 aws.ec2.RouteArgs(
 route_table_id=route_table.id,
 destination_cidr_block=self.args.supernet_cidr_block,
 transit_gateway_id=self.args.tgw_id
 ),
 opts=pulumi.ResourceOptions(
 parent=route_table,
 ),
)

aws.ec2.Route(
 f"{self.name}-insp-default-to-nat-{i+1}",
 aws.ec2.RouteArgs(
 route_table_id=route_table.id,
 destination_cidr_block="0.0.0.0/0",
 nat_gateway_id=self.nat_gateway.id
 ),
 opts=pulumi.ResourceOptions(
 parent=route_table,
 ),
)

Then create the AWS Network Firewall itself and deploy it to the subnets you created. AWS Network Firewall creates a VPC endpoint in each subnet to which it’s attached:

self.firewall = aws.networkfirewall.Firewall(
 f"{self.name}-firewall",
 aws.networkfirewall.FirewallArgs(
 vpc_id=self.vpc.vpc_id,
 firewall_policy_arn=self.args.firewall_policy_arn,
 subnet_mappings=subnet_mappings,
 ),
 opts=pulumi.ResourceOptions(
 parent=self,
 ),
)

You’ll also need to add some routes. In the public subnets of the inspection VPC, you’ll need to route all traffic bound for the spoke VPCs (i.e., the return traffic from the internet) through AWS Network Firewall’s generated VPC endpoints:

for subnet_id in public_subnet_ids:
 # ...

 aws.ec2.Route(
 f"{self.name}-{subnet_id}-to-firewall",
 aws.ec2.RouteArgs(
 route_table_id=route_table.id,
 destination_cidr_block=self.args.supernet_cidr_block,
 vpc_endpoint_id=subnet_attachments[0]["endpoint_id"],
 ),
 pulumi.ResourceOptions(
 parent=self,
 ),
 )

In the inspection VPC subnets with the Transit Gateway attachment, you’ll need to route all traffic bound for the internet through AWS Network Firewall. Previously, these routes went directly to the NAT gateway:

for subnet_id in tgw_subnet_ids:
 # ...

 aws.ec2.Route(
 f"{self.name}-{subnet_id}-to-firewall",
 aws.ec2.RouteArgs(
 route_table_id=route_table.id,
 destination_cidr_block="0.0.0.0/0",
 vpc_endpoint_id=subnet_attachments[0]["endpoint_id"],
 ),
 pulumi.ResourceOptions(
 parent=self,
 ),
 )

In order to test that our firewall rules are in effect, connect to one of your spoke EC2 instances like before. Traffic to amazon.com should be allowed:

But traffic to other domains should be blocked:

Conclusion

The hub-and-spoke architecture gives us several key advantages when our organization has multiple VPCs: cost savings via centralized egress and the ability to provide traffic inspection and apply network policies in a single place. While this architecture can be difficult to implement due to the resources needed, Pulumi’s approach of enabling practitioners to use real programming languages makes it easier to define complex, real-world infrastructure.

If you enjoyed this content, sign up for an upcoming workshop to learn more about how Pulumi can help you tame the complexity of the modern cloud, and join our community in the Pulumi Community Slack!

Advanced AWS Networking: Part 1

Josh Kodroff — Wed, 05 Apr 2023 00:00:00 +0000

In this blog series you will learn how to create a hub-and-spoke network architecture in AWS with centralized egress and traffic inspection. In this first installment, we’ll talk about the benefits of this architecture and begin to lay out some of its parts in Python with Pulumi, the infrastructure as code tool that enables you to manage infrastructure with real programming languages!

The complete code for this project is available on GitHub, which is a great place to go for full context.

What is a hub-and-spoke network?

A hub-and-spoke network is a common architecture for creating a network topology that provides isolation and security for your workloads. The hub-and-spoke architecture you’ll be creating on AWS has three main components: an inspection VPC, AWS Transit Gateway, and a series of spoke VPCs:

The inspection VPC provides centralized egress. It is the only VPC that has a route to the internet, so all other VPCs in the architecture must route their traffic through the inspection VPC. In part II of this blog post, you’ll add AWS Firewall to the inspection VPC to allow all traffic to the internet to be governed by a single set of policies.
Network connectivity between VPCs is accomplished via AWS Transit Gateway. The Transit Gateway maintains a central routing table that is used to route traffic from the spoke VPCs to the internet. We also need to maintain routes so that return traffic from the internet can be routed back to the correct spoke VPC.
The spoke VPCs are where we run our application workloads. They are isolated from each other and cannot communicate with each other unless we explicitly allow a network path. They will be able to communicate with the internet by default, but only through the inspection VPC’s NAT gateways.

A fully fleshed-out version of the architecture, with multiple spoke VPCs follows:

We could also enable communication between the spoke VPCs by adding routes to the Transit Gateway routing table. Traffic between spoke VPCs would not be desirable for security reasons if those VPCs represent different environments of the same workload (e.g., QA, UAT, and production). Traffic between spoke VPCs would potentially be desirable if those VPCs represent multiple workloads in the same environment, for example, our production online store and our production ERP software.

For simplicity, we will not be enabling routes between spoke VPCs in this architecture.

For a full discussion of all the considerations of building multi-VPC networks on AWS, see the excellent AWS whitepaper Building a Scalable and Secure Multi-VPC AWS Network Infrastructure.

Why use a hub-and-spoke network with centralized egress?

There’s several advantages to this architecture:

Cost savings: We save costs by allowing all of our spoke VPCs to share a single set of highly-available NAT gateways in the inspection VPC. There are costs to sending our traffic over AWS Transit Gateway, but in most scenarios Transit Gateway costs are significantly less than the cost of running NAT gateways in each spoke VPC.
Stable public IP addresses: Because all of our egress traffic to the internet originate from the same set of stable, public elastic IP addresses (attached to the NAT gateways in the inspection VPC), we can more easily interface with systems that use high-trust, network-based security models (i.e., IP allowlists). Network administrators for the systems we want to interface with only need allow a limited number of IPs to establish connectivity once and up-front, as opposed to having to allow new IPs for every VPC we create. These high-trust systems are often based in the data center and may have small and infrequent change windows, so only having to go through the change management process once, not for every spoke VPC we add, is a big win.
Centralized traffic inspection: When all traffic to the internet flows through our inspection VPC, we only need to apply a firewall and associated rules in a single place. This way, we can easily ensure that all network traffic to the internet complies with our organization’s security policies. (In the part of this post, we’ll add in AWS Firewall to our inspection VPC to add centralized traffic inspection capabilities to our architecture.)

Creating the first Transit Gateway resources

After initializing your Pulumi project and adding the necessary references to the AWS and AWSX, you can add the resources relating to your Transit Gateway.

You’ll need to configure a CIDR block for your network’s supernet. All spoke VPCs in this codebase must fall under the supernet CIDR block. You will use the supernet CIDR to create a route in the inspection VPC that will route traffic returning from the internet to the Transit Gateway. The Transit Gateway will, in turn, route the traffic back to the specific originating spoke VPC.

You can get the supernet CIDR from the Pulumi configuration. If the configuration value is not set, you can default to a CIDR block of 10.0.0.0/8:

config = pulumi.Config()
supernet_cidr = config.get("supernet-cidr") or "10.0.0.0/8"

Now you can add some of your initial Transit Gateway resources. You’ll start with the Transit Gateway itself along with some route tables for your inspection and spoke VPCs, respectively. You will add routes to this route table later to allow traffic from our spoke VPCs to the internet, as well as routes from the internet back to the originating spoke VPC:

tgw = aws.ec2transitgateway.TransitGateway(
 "tgw",
 aws.ec2transitgateway.TransitGatewayArgs(
 description=f"Transit Gateway - {project}",
 default_route_table_association="disable",
 default_route_table_propagation="disable",
 tags={
 "Name": "Pulumi"
 }
 )
)

inspection_tgw_route_table = aws.ec2transitgateway.RouteTable(
 "post-inspection-tgw-route-table",
 aws.ec2transitgateway.RouteTableArgs(
 transit_gateway_id=tgw.id,
 tags={
 "Name": "post-inspection",
 }
 ),
 # Adding the TGW as the parent makes the output of `pulumi up` a little
 # easier to understand as it groups these resources visually under the TGW
 # on which they depend.
 opts=pulumi.ResourceOptions(
 parent=tgw,
 ),
)

spoke_tgw_route_table = aws.ec2transitgateway.RouteTable(
 "spoke-tgw-route-table",
 aws.ec2transitgateway.RouteTableArgs(
 transit_gateway_id=tgw.id,
 tags={
 "Name": "spoke-tgw",
 }
 ),
 opts=pulumi.ResourceOptions(
 parent=tgw,
 ),
)

Creating the inspection VPC

The code samples in this section are not exhaustive. That is, they do not contain every line of code required so that we can focus on the most illustrative portions. See the GitHub repo for full context.

Now that you’ve added some foundational Transit Gateway resources, you can create your inspection VPC.

The inspection VPC is implemented as a Pulumi component resource. While we do not need to create multiple inspection VPCs in this codebase, implementing the inspection VPC as a component resource enhances the readability of both the code and the output by grouping all resources that belong to the inspection VPC in the output of pulumi up. You can also improve the component’s readability by placing its code in a separate file (in this case, inspection.py).

When you create a component resource in Python, it’s helpful to create a class that contains the arguments to the component resource. This allows you to pass the arguments to the component resource as a single object, which makes the code easier to read and understand. You can create a terse and easily readable arguments class in Python with the @dataclass decorator:

@dataclass
class InspectionVpcArgs:
 supernet_cidr_block: str
 vpc_cidr_block: str
 tgw_id: pulumi.Input[str]
 spoke_tgw_route_table_id: pulumi.Input[str]
 inspection_tgw_route_table_id: pulumi.Input[str]
 firewall_policy_arn: pulumi.Input[str]

Then, you can create the component resource class itself. The component resource class must inherit from pulumi.ComponentResource. The constructor for the component resource class must call the constructor of the base class, passing in the resource type (a string that uniquely identifies the component type, in the form of {project}:{module}:{component_name}), the resource name, and the resource options:

class InspectionVpc(pulumi.ComponentResource):
 def __init__(self, name: str, args: InspectionVpcArgs, opts: pulumi.ResourceOptions = None) -> None:
 super().__init__("awsAdvancedNetworking:index:InspectionVpc", name, None, opts)

You’ll start by creating a VPC with the AWSX VPC component. We’ll leverage the sensible defaults the component provides to create a VPC across 3 availability zones for high availability. You can either create a single NAT gateway in our inspection VPC to save costs, or create 3 NAT gateways, one in each availability zone, to provide high availability. See the code for details.

After creating the VPC, you’ll then need to attach the inspection VPC to the Transit Gateway. This is how the inspection VPC will be able to utilize the Transit Gateway and accept traffic from the spoke VPCs:

self.tgw_attachment = aws.ec2transitgateway.VpcAttachment(
 f"{name}-tgw-vpc-attachment",
 aws.ec2transitgateway.VpcAttachmentArgs(
 transit_gateway_id=self.args.tgw_id,
 subnet_ids=self.vpc.isolated_subnet_ids,
 vpc_id=self.vpc.vpc_id,
 transit_gateway_default_route_table_association=False,
 transit_gateway_default_route_table_propagation=False,
 appliance_mode_support="enable",
 tags={
 "Name": f"{name}",
 },
 ),
 # We can only have one attachment per VPC, so we need to tell Pulumi
 # explicitly to delete the old one before creating a new one:
 pulumi.ResourceOptions(
 delete_before_replace=True,
 depends_on=[self.vpc],
 parent=self,
 )
)

Then add a route to the Transit Gateway route table for the spoke VPCs that routes all traffic outside of the VPC to the inspection VPC (or to be precise, the Transit Gateway attachment in the inspection VPC). This route only needs to be added once as it will apply to all spoke VPCs, so we add it here in the inspection VPC since that’s where we create the Transit Gateway attachment:

aws.ec2transitgateway.Route(
 f"{name}-default-spoke-to-inspection",
 aws.ec2transitgateway.RouteArgs(
 destination_cidr_block="0.0.0.0/0",
 transit_gateway_attachment_id=self.tgw_attachment.id,
 transit_gateway_route_table_id=args.spoke_tgw_route_table_id,
 ),
 opts=pulumi.ResourceOptions(
 parent=self,
 )
)

Next, add routes within the inspection VPC’s isolated subnets that route traffic to the internet through the NAT gateway. The Transit Gateway:

for subnet_id in isolated_subnet_ids:
 route_table = aws.ec2.get_route_table(
 subnet_id=subnet_id
 )

 aws.ec2.Route(
 f"{self.name}-route-{subnet_id}-to-tgw",
 aws.ec2.RouteArgs(
 route_table_id=route_table.id,
 destination_cidr_block="0.0.0.0/0",
 nat_gateway_id=self.nat_gateway.id,
 ),
 pulumi.ResourceOptions(
 parent=self,
 ),
 )

Finally, you’ll need to add routes within the inspection VPC’s public subnets (where the NAT gateways live) that route return traffic from the internet back through the Transit Gateway. This is how the inspection VPC will be able to route traffic to the spoke VPCs:

for subnet_id in public_subnet_ids:
 route_table = aws.ec2.get_route_table(
 subnet_id=subnet_id
 )

 aws.ec2.Route(
 f"{self.name}-route-{subnet_id}-to-tgw",
 aws.ec2.RouteArgs(
 route_table_id=route_table.id,
 destination_cidr_block=self.args.supernet_cidr_block,
 transit_gateway_id=self.args.tgw_id,
 ),
 pulumi.ResourceOptions(
 depends_on=[self.tgw_attachment],
 parent=self,
 ),
 )

You need to create both sets of these routes within an apply because of the need to access the raw values of each of the subnet ids. Apply can be conceptually difficult for beginners in Pulumi to grasp, but it essentially means “once you know all of the values that needed, pass them on to this function”.

Final steps

Now that you’ve created our inspection VPC component, you can reference it in the main project file:

from inspection import InspectionVpc, InspectionVpcArgs

and you can create an instance of the component:

insp_vpc = InspectionVpc(
 "inspection",
 InspectionVpcArgs(
 supernet_cidr_block=supernet_cidr,
 vpc_cidr_block="10.129.0.0/24",
 tgw_id=tgw.id,
 inspection_tgw_route_table_id=inspection_tgw_route_table.id,
 spoke_tgw_route_table_id=spoke_tgw_route_table.id,
 firewall_policy_arn=None, # We will add this in the next post
 )
)

At this point, we do not have have a fully functional architecture - we still need to create the spoke VPCs - but if we want to get some feedback on our architecture, we can run pulumi up and see what happens. We should see that Pulumi will create the VPC, the Transit Gateway attachment, and the routes.

In the next post, we’ll create our spoke VPCs and verify connectivity, then add centralized inspection of network traffic going to the internet. Stay tuned!

Managing NOAA Open Data across Multiple Clouds with Pulumi

Denis Willett, PhD — Tue, 08 Nov 2022 21:01:21 +0000

Denis Willett is a software engineer at the North Carolina Institute of Climate Studies who works on the NOAA Open Data Dissemination Program. His work focuses on leveraging cloud technologies for the development of data processing and machine learning pipelines. Denis did his PhD in Entomology and Nematology at University of Florida and his undergraduate and masters work in Earth Systems at Stanford University. You can read his full bio here.

NOAA Open Data Dissemination (NODD) makes environmental data freely and publicly accessible across Amazon Web Services (AWS), Microsoft Azure (Azure), and Google Cloud Platform (GCP). These data include near real-time satellite imagery, weather models, radar feeds, drought information, ocean databases, and a suite of climate data records among many others. This program supports more than 220 datasets and over 24PB of open data. Since its inception, the program has been growing rapidly, almost doubling in size over the past year.

Availability of the petabyte scale environmental data on the cloud is only one aspect of NODD. The program places a premium of performant data delivery and access. NODD data pipelines provide resilient and low, in many cases near real-time, latencies to NOAA’s core data offerings. In addition, NODD provides performant, parallel access to these data supporting billions of requests per day and accessions of individual datasets exceeding 1PB in a single day.

To support this program, the North Carolina Institute for Climate Studies (NCICS) through the Cooperative Institute for Satellite Earth System Studies (CISESS) engineers infrastructure on all three major public clouds ranging from object stores to messaging platforms, data ingest and transformation pipelines, and monitoring systems. These systems are deployed and managed by a very small team that, by necessity, must be agile, resilient and facilitate low latency data transfers at petabyte scale. Given the scale and diversity of resources managed through this initiative, we began looking for tools to facilitate multi-cloud management using infrastructure as code (IAC).

Before Pulumi

Prior to beginning to transition to Pulumi, our team built infrastructure primarily using the Command Line Interface (CLIs) and consoles of each cloud service provider. While this worked well, it required extensive Cloud Service Partner (CSP) specific training to be able to not only navigate the nuances of each provider’s offerings, but also the unique interface to deploy resources. Changing and updating infrastructure in this manner resulted in long development cycles where upgrades were mostly manual and composed of repetitive tasks.

Because of the manual nature of these deployments across three different cloud service providers, infrastructure began to sprawl. A clear snapshot of what resources were deployed where became difficult to compile and management of legacy and deprecated systems depended on manual intervention for clean-up. Infrastructure also began to drift. What started as uniform deployment of resources across multiple cloud platforms morphed as manual updates to each instance resulted in unique environments that were hard to replicate if they had to be deleted and redeployed.

As an example, our program relies on Apache NiFi deployments on AWS, GCP, and Azure to coordinate and manage petabyte scale data transfers while managing data provenance. Drift in these deployments makes it especially difficult to redeploy identical configurations (even though we were using Ansible and Containerization) if they go down.

An additional consequence of this approach is knowledge silos. The team member who deployed a group of resources on AWS had trouble (even with extensive documentation) communicating reusable architectures to another team member who was working on similar situations in Azure. While there are obvious nuances relevant to each cloud environment, our need to replicate similar infrastructure in a multi-cloud environment at scale necessitated extensive detailed communication in a team environment. Questions like: “What is the ARN of the most recent version of this service?” and “What queues do we have on GCP?” required communication and time taken by the individual responsible to answer those questions.

How and why we chose Pulumi

As our team grew and our platform scaled (more than doubling in storage size over the past year by more than 10PB), we began to explore tooling to help us better navigate and manage our multi-cloud and multi-account environment. The nature of the NOAA Open Data Dissemination program agreements with each cloud service provider (CSP) meant that we had multiple accounts with different permissions, funding, and configuration on each CSP, resulting in flexible options.

We began looking at infrastructure as code (IAC) solutions and initially considered many of the cloud-specific options: CloudFormation on AWS, Deployment Manager on Google, and Resource Manager on Azure. While these were nice for automation, we quickly discarded these as options because each had an individual learning curve for that environment; they did not translate well in a multi-cloud environment. These tools also did not have the control features of a full-fledged programming language; looping and if-then statements were challenging to implement.

We also considered other Infrastructure as Code providers. While these solutions would meet a lot of our needs, and one of our team members had extensive experience with one of these tools, the ability to use a common tool seamlessly in a multi-cloud environment was attractive. However, the learning curve associated with a domain specific language was cause for reticence. In addition, we really wanted to be able to use the features of a full-fledged programming language.

After experimenting with a few IAC options, our research institute began using Pulumi in late 2021 and immediately began implementing it to expand our infrastructure.

Benefits of using Pulumi for multi-cloud infrastructure as code

Pulumi had all the benefits we were looking for in a multi-cloud infrastructure as code platform. In terms of automation, manual checklists could be converted to code reducing both deployment time and human errors in deployment. This automation with infrastructure as code (and accompanying state management) allowed us to adopt an immutable infrastructure paradigm where old resources are replaced with new updated infrastructure. While these benefits are accessible via other IAC tools, we particularly appreciated the Pulumi interface to be able to quickly access and inspect infrastructure deployments.

We saw immediate wins for our teams in the following areas:

Drift was eliminated. Single source of truth, version controlled (and tagged) code documented exactly what was deployed where. When updates were needed, the code was updated and redeployed without manually tweaking individual resources through the console.
Management of legacy and deprecated resources were obviated. With IAC managed deployment that would automatically clean up old resources, we saved many future hours cleaning up unused cloud resources.
Team collaboration was enhanced. With a single source of truth and a visual snapshot of resources and deployments, the number of ‘What was the ARN of that resource again?” type questions dropped considerably. Team members could simply look at the deployment in one location and see all of the resources across all three major clouds. This type of friction-reduction really enhances team productivity.

In addition to these IAC benefits, we saw a number of key advantages in implementing Pulumi. First and foremost was the ability to use full-featured programming languages to deploy IAC solutions in a multi-cloud environment. This not only allowed us to fully take advantage of programming control structures (e.g., need 100 different rules on AWS? Now you do not have to manually script each one, just put it in a loop), but also flattened the learning curve for adoption. Our team already used Python extensively in our day-to-day work. Adopting Pulumi was almost as simple as importing a package. Accessing Pulumi through Python made it much easier to embrace.

This accessibility through Python also meant that learning how to create, deploy, and share reusable templates was trivial as all team members already knew how to build, share, and import modules in Python. The ability to rapidly share these templates accelerated our development and allowed for standardization around best practices. For example, we use serverless capabilities on all three CSPs. Once we had a template for deploying our function as a service (FAAS) code on one platform, we did not have to reinvent the wheel or remember to go through a checklist; it was as simple as importing our internal FAAS module and pointing it to the correct code to deploy.

Next steps

Pulumi has become an automation and collaboration tool that is accelerating our ability to provide petabyte scale data to the public in a multi-cloud environment. As we continue to scale our program, Pulumi is allowing us to leverage a small team to manage increasingly larger, more complex deployments in a dynamic environment. In addition, it is rapidly becoming a key player in a cross-cloud fabric that we are building to facilitate working across multiple platforms. By providing a similar, intuitive interface through our language of choice, Python, it is allowing us to more quickly adapt and reproducibly deploy infrastructure across AWS, Azure, and GCP.

Please reach out to us with your thoughts, reactions, and comments at NODD@NOAA.GOV.

Data Science in the Cloud

Sophia Parafina — Thu, 18 Mar 2021 00:00:00 +0000

Data science has advanced because tools like Jupyter Notebook hide complexity by running high level code for the specific problem they are trying to solve. Increasing the level of abstraction lets a data scientist be more productive by reducing the effort to try multiple approaches to near zero, which encourages experimentation and better results.

Data scientists typically work locally, but they often store data for analyses and models in the cloud. There are clear advantages to using cloud resources for these tasks:

Data scientists generally don’t want to manage their storage and databases.
They need to be able to store large data sets cheaply.
They need large capacity swings available on-demand.

SDKs like AWS’ Python library, boto3, can create resources, but they still require domain expertise to manage and properly architect a solution. The Pulumi Automation API improves on raw SDKs by providing high-level abstractions for creating and managing cloud services, letting data scientists concentrate on analyses and models without being well-versed in cloud APIs.

How Pulumi Automation API Improves on Cloud SDKs

Pick any cloud provider, and you will find a cloud SDK in various languages, for example AWS SDKs, Azure SDKs, and Google Cloud SDKs. These SDKs directly access cloud resources, letting data scientists provision resources as required. The drawback to using these libraries is that the data scientist is also responsible for providing resources reliably and securely and performing modeling and analysis.

Pulumi reduces the burden of creating and maintaining repeatable and robust deployments by providing the latest language runtimes, concurrency management, support for diffing updates, and maintaining state with robust checkpoints. Your infrastructure will always be in a well-defined and recoverable state.

Automation API takes this one step further by providing an API overlay to Pulumi’s Infrastructure as Code engine. A data scientist can build infrastructure programmatically within Jupyter to automate building and creating infrastructure as functions and call them as needed. Let’s take a look at an example using a Jupyter notebook.

Infrastructure as a function

As a data scientist, you might want to share data, charts, or graphs with remote team members and make them available through a shared website. You want to keep iterating on your model but don’t want to update the site manually, or worse, let it get out of date.

This example is a simple overview of Automation API that demonstrates a basic deployment of a static website from within a Jupyter notebook. This example uses an inline program to define infrastructure within a function alongside your other code. The function called s3_static_site creates an s3 bucket, sets it up as a basic static website, and exports the URL. This is the equivalent of creating a Pulumi stack using the CLI.

We’ve added code to the example to upload files written to the www directory. Every time you want to update the website, you can write a file to the directory and call the function to upload it to the s3 bucket. Pulumi maintains state, so it will only upload new files when the function is called.

from pulumi_aws import s3

def s3_static_site():
 # Create a bucket and expose a website index document
 site_bucket = s3.Bucket("s3-website-bucket", website=s3.BucketWebsiteArgs(index_document="index.html"))
 index_content = """
 <html>
 <head><title>Hello S3</title><meta charset="UTF-8"></head>
 <body>
 <p>Hello, world!</p>
 <p>Made with ❤️ with <a href="https://pulumi.com">Pulumi</a></p>
 </body>
 </html>
 """

 # Write our index.html into the site bucket
 s3.BucketObject("index",
 bucket=site_bucket.id, # reference to the s3.Bucket object
 content=index_content,
 key="index.html", # set the key of the object
 content_type="text/html; charset=utf-8") # set the MIME type of the file

 # Set the access policy for the bucket so all objects are readable
 s3.BucketPolicy("bucket-policy", bucket=site_bucket.id, policy={
 "Version": "2012-10-17",
 "Statement": {
 "Effect": "Allow",
 "Principal": "*",
 "Action": ["s3:GetObject"],
 # Policy refers to bucket explicitly
 "Resource": [pulumi.Output.concat("arn:aws:s3:::", site_bucket.id, "/*")]
 },
 })

 # Export the website URL
 pulumi.export("website_url", site_bucket.website_endpoint)

We can write additional functions to deploy and destroy the S3 static website. We deploy the stack by calling the deploy_project function and pass it the s3_static_site function, the Pulumi AWS plugin, and the AWS region.

s3_site = deploy_project("my_first_project",
 s3_static_site,
 plugins=[("aws", "v3.23.0")],
 config={"aws:region": auto.ConfigValue(value="us-west-2")})

You can view the resources that the function created in two ways. The standard method is to open the AWS console to see your resources.

You can also view them with the Pulumi Service as a resource graph.

The Pulumi Service also displays updates to the static site and the changes made.

Now that the static site has been deployed, we want to get the site URL. The s3_static_site function exports the website url. Pulumi returns the s3_site as an object with outputs that we use to retrieve the URL. There are several ways to check if the site is working, but a simple way is to open the URL in a web browser.

import webbrowser

outputs = s3_site.outputs()

webbrowser.open(url)

Data Science Your Way

In this simple example, we demonstrated how to create a static website from an S3 bucket, write content into the bucket, and set the access policy as a Python function. We called the function, which created a stack for the website. The stack is returned as a Python object with the site URL as an output. You can manage the site and view updates using the Pulumi Service.

Because it’s code, you can create a separate module containing all the functions to create your infrastructure, import it into your notebooks, and reuse it. Pulumi lets you perform analysis and build models within a familiar environment while reducing the burden of managing infrastructure. You can find the complete example on GitHub including a Jupyter notebook for database migration.

Be sure to watch our video that explains the code in greater detail.

Zero Downtime InfluxDB Migration with Pulumi and Aiven

Trevor Kennedy — Thu, 25 Feb 2021 00:00:00 +0000

In this article, I’ll show how Pulumi can be used with Aiven’s services to create infrastructure that can be migrated from cloud to cloud with no downtime.

This tutorial will use Python, Pulumi, Grafana, and an AWS Lambda function to simulate recording temperature data in an InfluxDB database.

Register for Multicloud OSS Database Deployments With Zero Downtime - Pulumi and Aiven and learn how to build robust, multi-cloud applications using the language, open source database, and cloud of your choice.

A quick introduction to Pulumi, Aiven, and InfluxDB

Pulumi enables you to create, deploy, and manage infrastructure on any cloud using your favorite languages. Aiven augments this language agnosticism by giving developers and operators the ability to be cloud-agnostic for a variety of popular databases such as MySQL, PostgreSQL, Cassandra, Redis, and more.

Aiven offers a range of fully-managed open-source cloud database services. By using Aiven, you can accelerate application development on the open source data infrastructure and quickly deploy to public clouds (including AWS, GCP, and Azure).

InfluxDB is one of the most popular open source time-series databases for storing metrics and events. InfluxDB also has a number of client libraries in a variety of languages, including Python. Aiven makes this database available as a fully managed, single-tenant service in a variety of resource sizes.

I want to demonstrate how to migrate your services using Pulumi from one cloud to another with absolutely zero downtime in this tutorial. Data keeps streaming in the entire time, which you can visualize in Grafana. This gives you an added security factor in fault situations and great flexibility in terms of migrating services for any reason.

Demo outline

This demo will deploy the following infrastructure:

Grafana instance on Aiven (hosted on GCP)
InfluxDB database instance on Aiven (hosted on GCP)
Service integration between InfluxDB and Grafana
AWS Lambda Layer for the Python requests library
AWS Lambda function and associated role
AWS EventBridge trigger to execute the Lambda function every 60s

We will then migrate our services from a hybrid cloud (GCP + AWS) to a single cloud (AWS alone) with a single command and experience no downtime.

The lambda function code (located in the file lambda_code.py) is very basic. It only writes a randomly generated value (outside_temp) to InfluxDB using the InfluxDB Python Client library.

Requirements

Accounts for AWS, Pulumi & Aiven
Python 3.7+
AWS CLI 2.0+
Pulumi 2.16+

Cost estimate

This tutorial is about as cheap as they come; all you need is some time to perform all the steps.

$0 for AWS Lambda charges
Free Pulumi trial account
Free credits for new Aiven users (less than $1/hr for existing customers)
20-minute time estimate

Set up the environment

Your first step is to install Python. Follow the directions on this page to install Python on your local system.

Next, get the Aiven examples by cloning this repo to your local machine.

Run the setup script in the repo to activate the Python virtual environment and install the required dependencies:

$ cd aiven-examples/pulumi/influxdb_grafana_lambda
$ chmod +x setup.sh && ./setup.sh

Set your Aiven API token using the Pulumi CLI:

$ pulumi config set aiven:apiToken XXXXXXXXXXXXXXXXXXXX --secret

Note that you can generate an API Token on your profile page.

Install the Aiven and AWS plugins for Pulumi:

$ pulumi plugin install resource aws 3.22.0
$ pulumi plugin install resource aiven 3.3.2

To find out the current version numbers, go to Pulumi changelog or the Aiven changelog.

Set your preferred cloud regions and Aiven project name (these defaults are used in the setup script):

$ pulumi config set aws:region <desired AWS region> Example: us-west-2
$ pulumi config set aiven_cloud_region <desired cloud - region> Example: google-us-central1
$ pulumi config set aiven_project <Aiven project> Example: sa-demo

Example: google-us-central1

$ pulumi config set aiven_project <Aiven project> Example: sa-demo

Supported regions/clouds for each service are listed on the pricing page.
Project names are on the top left corner of the Aiven console.

Use Pulumi to create two services in Aiven

Now that you’re all set up, it’s time to create the services in the Aiven environment by using Pulumi. Issue the following commands to deploy the stack:

$ pulumi up

Select Yes to confirm the deployment.

After about 5 minutes or so, you should see that all of the resources have been created. The URLs and username/password combinations listed are needed to access the services. The influxdb_uri shown here is also added as an environment variable in the Lambda function so that the Python client can connect to InfluxDB.

Aiven console

To see the new services, log into the Aiven console. The resources pulumi-influxdb and pulumi-grafana are displayed.

AWS console

In the AWS console, the lambda function has also been created and will be executed by EventBridge every 60 seconds.

Grafana

Now things get exciting: you can see the real-world (for certain values of real, in any case) data streaming in by opening Grafana.

Open up the grafana_uri (shown in the console output) and log in with the provided username and password.

Click on the Explore panel and search for the outside_temp value written by the lambda function every minute (triggered by EventBridge).

Migrate your service from one region to another

Now that the services are running, it’s time to demonstrate the power of cloud services operating in multiple clouds: migrating to a different cloud without losing any incoming data and without any downtime. Let’s perform a zero-downtime migration between regions.

Start by changing your Aiven cloud region to a region in AWS (such as us-east-1). Then run the up command a second time.

$ pulumi config set aiven_cloud_region aws-us-east-1
$ pulumi up -y

The output shows that two resources have changed, and both Grafana and the InfluxDB service show as rebuilding. A quick check in the Aiven console shows that both services are indeed being migrated to AWS:

If we switch back to Grafana, you can see that outside_temp_value is still being written to InfluxDB even as we migrate to a different cloud!

Clean up

After we do projects, we need to tidy up after ourselves. Use the following commands to tear down the provisioned resources:

pulumi destroy
pulumi stack rm <my-stack-name-here>

Python code details

Lambda function

The lambda function is quite simple: a JSON payload is created for our randomly generated temperature reading, which is then passed to the InfluxDBClient for writing to the provisioned InfluxDB database. The automatically assigned Aiven service URI is added as an environment variable to the lambda function by Pulumi.

def lambda_handler(event, context):

 # Get the URL (including username and password) of the provisioned service instance
 uri = os.environ['SERVICE_URI']
 urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
 client = InfluxDBClient.from_dsn(uri, timeout=1.0, ssl=True)

 # Write our sample data
 json_body = [
 {
 "measurement": "outside_temp",
 "fields": {"value": random.uniform(-30, 130)}
 }
 ]

 try:
 client.write_points(json_body)
 return True
 except:
 print('Failed to send metrics to influxdb')
 return False

Lambda layer

Because this function depends on the influxdb-client-python library, a custom Lambda Layer is created and assigned to the function in the Pulumi script. Inside the layer, you will notice a python folder that allows Lambda to load the module automatically.

lambda_layer
 └───python
 └───influxdb-client-python

Aiven services

All of the Pulumi specific code resides in __main__.py. Even though creating Aiven services in Pulumi is very straightforward, I have created a wrapper function named create_service to make this even easier.

grafana = create_service("pulumi-grafana", "startup-4", "grafana")

Simply pass in the plan name, service type (Grafana in this case) and give your new service a name. The project name and cloud region are read in from the Pulumi config class.

Creating service integrations follows a similar paradigm:

create_integration("pulumi-grafana-influxdb", "dashboard", influxdb, grafana)

When creating an integration, you have to specify the integration type and both the source and sink resource names. Here, we want to visualize our metrics in Grafana,which is an integration of type dashboard.

Lambda function creation

Creating the Lambda function in AWS a tad more complicated, but this boils down to the methods below:

lambda_func = create_lambda(influxdb)
create_lambda_trigger(lambda_func)

These methods initiate the following logic:

Create an IAM role for the Lambda function
Create the Lambda Layer, which includes our library dependencies
Create the serverless function itself with environment variables, customer layer, and associated IAM policy
Create the CloudWatch event rule to run the Lambda function every minute
Sets the permissions of the function to allow it to be triggered by the CloudWatch event rule we just created
Sets the EventBridge target to the Lambda function

Wrapping up

In this article, I’ve demonstrated one way to deploy infrastructure as code in a manner that allows services to be migrated between clouds without any downtime. We also showed how InfluxDB could be used as an open source time-series backend for collecting temperature metrics.

This same approach can be utilized for any Aiven’s services, including managed Kafka and PostgreSQL. Sign up for a free 14-day trial and modify the lambda function to begin storing your own metrics data today.

About the Author

Trevor Kennedy is DataOps engineer who has spent 15+ years consulting in the public sector, telecommunications, financial services, insurance and energy industries. He has experience with all things data (data engineering, database development, data architecture, cloud migrations, and cloud architecture) on multiple platforms (on-premise, AWS, Azure and GCP) using both SQL, NoSQL & NewSQL technologies. Currently based in Austin, TX.

Automate Your Infrastructure with Automation API and Python

Sophia Parafina — Tue, 09 Feb 2021 00:00:00 +0000

General-purpose languages enable Infrastructure as Software – bringing tested toolchains and best practices to building infrastructure, e.g., languages, IDEs, testing, debugging, componentization, packaging, and versioning. Available in public preview, Pulumi’s Automation API is a robust programmatic layer on top of Pulumi’s infrastructure engine. It exposes Pulumi programs and stacks as strongly-typed and composable building blocks. Automation API allows you to embed the Pulumi engine inside your software projects so you can build software automation around entire infrastructure provisioning processes that normally require humans to operate.

Today, we are excited to announce Python support for this powerful feature, opening up a world of possibilities for Python developers.

Enabling Cloud Engineering

The Automation API is a subpackage in Pulumi’s language-specific SDKs that provides a programmable interface for creating and managing Stacks and performing infrastructure updates, refresh, previews, and destroy. You can define a Pulumi program as a function within your codebase and use methods to get and set configuration parameters programmatically. The Automation API uses a gRPC interface to execute programs that control and communicate with the core Pulumi engine.

To use Automation API, install the Pulumi CLI, which bundles and distributes the core engine. Today it’s available for Python, TypeScript/JavaScript and Go, with support for C# under active development.

Automation Use Cases

Let’s look at two use cases where Pulumi customers are applying the Automation API:

Platform APIs

Deploying infrastructure with shell scripts for online services creates operational challenges with observability and reliability, especially for SaaS scenarios. Automation API is just another package and can be instrumented, profiled, and monitored with your existing APM tools like Honeycomb, Prometheus, and Datadog.

Automation API runs inside your favorite frameworks and interacts with your other packages precisely as you’d expect. We can define custom declarative infrastructure and expose it to end-users behind a familiar REST interface, as in this example, that creates an S3 website with a POST request.

# an HTTP handler to create instances of our declarative infra
# each request creates an S3-backed static website with dynamic content from the POST body
@app.route("/sites", methods=["POST"])
def create_handler():
 """creates new sites"""
 stack_name = request.json.get('id')
 content = request.json.get('content')
 try:
 def pulumi_program():
 return create_pulumi_program(content)
 # create a new stack, generating our pulumi program on the fly from the POST body
 stack = auto.create_stack(stack_name=stack_name,
 project_name=project_name,
 program=pulumi_program)
 stack.set_config("aws:region", auto.ConfigValue("us-west-2"))
 # deploy the stack, tailing the logs to stdout
 up_res = stack.up(on_output=print)
 return jsonify(id=stack_name, url=up_res.outputs['website_url'].value)
 except auto.StackAlreadyExistsError:
 return make_response(f"stack '{stack_name}' already exists", 409)
 except Exception as exn:
 return make_response(str(exn), 500)

The handlers work with your favorite HTTP frameworks, including flask:

import Flask

# install necessary plugins once upon boot
ensure_plugins()

# initialize our Flask app
app = Flask(__name__)

# set up our routes
@app.route("/sites", methods=["POST"])
def create_handler():
 ...

@app.route("/sites", methods=["GET"])
def list_handler():
 ...

@app.route("/sites/<string:id>", methods=["GET"])
def get_handler(id: str):
 ...

@app.route("/sites/<string:id>", methods=["UPDATE"])
def update_handler(id: str):
 ...

@app.route("/sites/<string:id>", methods=["DELETE"])
def delete_handler(id: str):
 ...

Check out the full Pulumi over HTTP example in Python.

Complex Workflow Orchestration

The Automation API executes just like any other code in your application, meaning that a single process can deploy infrastructure and manage related activities such as database migrations and blue/green deployments.

In this example, we define an AWS RDS database:

cluster_instance = aws.rds.ClusterInstance(
 "db_instance",
 cluster_identifier=cluster.cluster_identifier,
 instance_class=aws.rds.InstanceType.T3_SMALL,
 engine=aws.rds.EngineType.AURORA_MYSQL,
 engine_version="5.7.mysql_aurora.2.03.2",
 publicly_accessible=True,
 db_subnet_group_name=subnet_group.name
)

pulumi.export("host", cluster.endpoint)
pulumi.export("db_name", db_name)
pulumi.export("db_user", db_user)
pulumi.export("db_pass", db_pass)

With Automation API, we can perform database migrations and preload rows in the same program:

# create (or select if one already exists) a stack that uses our inline program
stack = auto.create_or_select_stack(stack_name=stack_name,
 project_name=project_name,
 program=pulumi_program)

up_res = stack.up(on_output=print)
print(f"db host url: {up_res.outputs['host'].value}")

print("configuring db...")
with connect(
 host=up_res.outputs['host'].value,
 user=up_res.outputs['db_user'].value,
 password=up_res.outputs['db_pass'].value,
 database=up_res.outputs['db_name'].value) as connection:
 print("db configured!")

 # make sure the table exists
 print("creating table...")
 create_table_query = """CREATE TABLE IF NOT EXISTS hello_pulumi(
 id int(9) NOT NULL PRIMARY KEY,
 color varchar(14) NOT NULL);
 """
 with connection.cursor() as cursor:
 cursor.execute(create_table_query)
 connection.commit()

See the full database deployment and migration Automation API program in Python.

Automation API Resources

To get started with Automation API check out these links and watch our video.

Automation API is still in alpha. We maintain a list of known issues. Please file more as you find them!

🏭 🏭 🏭 Happy automating! 🏭 🏭 🏭

Create Amazon EKS clusters in your favorite language

Levi Blackstone — Thu, 04 Feb 2021 00:00:00 +0000

Check out version 3.0 of the Pulumi EKS Provider.

Pulumi’s infrastructure as code tooling combines the programming languages and tools you already know with the full power of cloud infrastructure. But until now, some Pulumi components for cloud infrastructure, like our popular EKS package for Amazon’s Elastic Kubernetes Service, were only available in a subset of the languages supported by Pulumi.

Now, you can use the EKS package–previously only available for TypeScript–in all four Pulumi languages: TypeScript, Python, .NET, and Go. Regardless of the language you choose, you can manage EKS clusters with Pulumi, starting with the v0.22.0 release. Check out our Modern Infrastructure Wednesday video to see it in action:

Making components available in every language

Pulumi’s EKS package is made up of Pulumi components. A component is a collection of Pulumi resources that provides an easy way to package best practices for using cloud resources, even if those best pratices span individual resources or even entire cloud providers. Until now though, components could only be used in Pulumi programs written in the same language as the component.

To change that, we’re creating “multi-language components”. With multi-language components, anyone will be able to author a component in one language and automatically compile it to all other languages supported by Pulumi. This means anyone in the worldwide Pulumi community will be able to take advantage of any component, even if it’s written in a different language. Right now, unless a component author is willing to write code in all four Pulumi languages, their component is only able to reach Pulumi users using the same language SDK.

In addition to expanding the library of publicly available components, this work will also enable organizations to write Pulumi components in the right programming language, or languages, for each team and each product. Infrastructure or platform teams will be able to write components in Go and share them with a webapp team using TypeScript, write components in .NET and share them with a machine learning team using Python, and any other combination that suits their business.

The next step in the story is a seamless authoring experience that enables component creators to write, build, and publish multi-language components. That will be in public preview soon, and we’re excited for you to try it out. If you’d like, subscribe to the multi-language components issue for updates.

Next steps

If you haven’t yet, try out the new pulumi-eks package with this walkthrough.

Announcing Cross-Language Enum Support

Komal Ali — Tue, 15 Dec 2020 00:00:00 +0000

Here at Pulumi, we believe in leveraging the best features of programming languages to create a delightful development experience for our users. Today, we continue our contributions in this area by announcing cross-language support for enum types in our provider SDKs, available in all Pulumi languages - Python, TypeScript, .NET and Go.

Enum types explained

An enum (short for enumerated) type is a data type consisting of a set of named values. In the Pulumi resource model, a property is an enum type when its value can only be set to a finite number of predetermined values.

For instance, consider a simple S3 bucket.

import * as pulumi from "@pulumi/pulumi";
import { s3 } from "@pulumi/aws";

const myBucket = new s3.Bucket("myBucket", { acl: "private" });

import pulumi
from pulumi_aws import s3

my_bucket = s3.Bucket("myBucket", acl="private")

using Pulumi;
using S3 = Pulumi.Aws.S3;

class MyStack : Stack
{
 public MyStack()
 {
 var myBucket = new S3.Bucket("myBucket", new S3.BucketArgs
 {
 Acl = "private",
 });
 }
}

package main

import (
 "github.com/pulumi/pulumi-aws/sdk/v3/go/aws/s3"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 myBucket, err := s3.NewBucket(ctx, "myBucket", &s3.BucketArgs{
 Acl: pulumi.String("private"),
 })
 if err != nil {
 return err
 }
 return nil
 })
}

In the above code, the S3 Bucket resource has a property called acl, where we pass in the string private to indicate that this is a private bucket.

If we look at the resource docs, we can see that the acl property can only be set to one of a few different values: private, public-read, public-read-write, aws-exec-read, authenticated-read, and log-delivery-write. The acl property is the perfect candidate for an enum type and is emitted as one, so you can use the following code instead.

import * as pulumi from "@pulumi/pulumi";
import { s3 } from "@pulumi/aws";

const myBucket = new s3.Bucket("myBucket", { acl: s3.CannedAcl.Private });

import pulumi
from pulumi_aws import s3

my_bucket = s3.Bucket("myBucket", acl=s3.CannedAcl.PRIVATE)

using Pulumi;
using S3 = Pulumi.Aws.S3;

class MyStack : Stack
{
 public MyStack()
 {
 var myBucket = new S3.Bucket("myBucket", new S3.BucketArgs
 {
 Acl = S3.CannedAcl.Private,
 });
 }
}

package main

import (
 "github.com/pulumi/pulumi-aws/sdk/v3/go/aws/s3"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 myBucket, err := s3.NewBucket(ctx, "myBucket", &s3.BucketArgs{
 Acl: s3.CannedAclPrivate,
 })
 if err != nil {
 return err
 }
 return nil
 })
}

IDE Superpowers

Enum properties provide discoverable and normalized constants that can be used in place of raw strings, allowing you to move faster with the added boosts of IDE tooling like type hints and autocomplete.

Using the provided constants, you can avoid referring back to the documentation to remember the valid values and save precious moments in the development cycle that would be lost to debugging errors caused by typos.

Optimized for flexibility

While some properties make sense as “strict” enums (i.e., the input value must be one of the enumerated values), there are times when it would be beneficial to have constants for commonly used values without restricting the input to only those values. We call this second category “relaxed” enums, and we model them a little differently in our SDKs.

“Strict” enums

A property is a “strict” enum when the input value must be one of the enumerated values. In this case, the property type is specified as the enum type.

We will use “strict” enums when we are sure that the enum will include all legal values, such as when a provider is auto-generated from a cloud provider specification (like our Azure-Native or Kubernetes providers).

export class StorageAccount extends pulumi.CustomResource {
 constructor(name: string, args: StorageAccountArgs, opts?: pulumi.CustomResourceOptions) {...}
}

export interface StorageAccountArgs {
 readonly accessTier?: pulumi.Input<enums.storage.latest.AccessTier>;
 ...
}

class StorageAccount(pulumi.CustomResource):
 def __init__(__self__,
 resource_name: str,
 access_tier: Optional[pulumi.Input['AccessTier']] = None,
 ...):
 ...

namespace Pulumi.AzureNative.Storage
{
 public partial class StorageAccount : Pulumi.CustomResource
 {
 public StorageAccount(string name, StorageAccountArgs args, ...)
 {
 }
 }

 public sealed class StorageAccountArgs : Pulumi.ResourceArgs
 {
 [Input("accessTier")]
 public Input<Pulumi.AzureNative.Storage.AccessTier>? AccessTier { get; set; }

 ...
 }
}

func NewStorageAccount(ctx *pulumi.Context,
name string, args *StorageAccountArgs, ...) (*StorageAccount, error) {
 ...
}

type StorageAccountArgs struct {
 AccessTier AccessTier
 ...
}

When using TypeScript, “strict” enums will accept both the constant (AccessTier.Cool) or the literal string ("Cool"). If there is an accidental spelling error in the literal, you are immediately alerted to the issue rather than having to wait until runtime.

“Relaxed” enums

When a property is a “relaxed” enum, the property type is specified as the Union of the enum type and the underlying primitive type. This means that you have the convenience of using the enum constants (i.e., s3.CannedAcl.Private), but you may also pass in the raw string (i.e., "private").

In the AWS provider (and other Terraform-based providers), we have opted for only using “relaxed” enums. The reasoning is twofold. Allowing the primitive type maintains backward compatibility and also allows users to use values that may not yet be represented in the Pulumi schema (e.g., a new Managed Policy ARN, EC2 Instance type, etc.).

export class Bucket extends pulumi.CustomResource {
 constructor(name: string, args?: BucketArgs, opts?: pulumi.CustomResourceOptions) {...}
}

export interface BucketArgs {
 readonly acl?: pulumi.Input<string | enums.s3.CannedAcl>;
 ...
}

class Bucket(pulumi.CustomResource):
 def __init__(__self__,
 resource_name: str,
 acl: Optional[pulumi.Input[Union[str, 'CannedAcl']]] = None,
 ...):
 ...

namespace Pulumi.Aws.S3
{
 public partial class Bucket : Pulumi.CustomResource
 {
 public Bucket(string name, BucketArgs? args = null, ...)
 {
 }
 }

 public sealed class BucketArgs : Pulumi.ResourceArgs
 {
 [Input("acl")]
 public InputUnion<string, Pulumi.Aws.S3.CannedAcl>? Acl { get; set; }

 ...
 }
}

func NewBucket(ctx *pulumi.Context,
name string, args *BucketArgs, ...) (*Bucket, error) {
 ...
}

// In Go, relaxed enums retain the type of the underlying primitive.
// However, the CannedAcl constants may still be passed in, as they are
// of the same underlying type.
type BucketArgs struct {
 Acl pulumi.StringPtrInput
 ...
}

Try them out!

You can find enum types integrated into v3.19.0 of the AWS provider and v0.7.1 of the Azure-Native provider, and we will be adding enums to other providers in the coming weeks and months.

Azure-Native: In the native Azure provider, all properties labeled as enums in the OpenAPI spec are represented as such. In all, there are over 1300 enums provided in the SDKs. The native Azure provider uses both “strict” and “relaxed” enums since the OpenAPI specification explicitly defines its properties as such.

AWS: The AWS provider enums are manually identified and maintained as part of the provider schema. We’ve already added many that you might find useful, such as Lambda Runtimes, EC2 Instance Types, and IAM Managed Policies, and will continue to add more in the coming months.

Take the new providers for a spin and let us know what you think in the Community Slack! If there are properties that you would like to see represented as enums, let us know. Or even better, submit a PR like this one to add them to the schema!

How to Deploy a Django Application to AWS

Vova Ivanov — Fri, 28 Aug 2020 00:00:00 +0000

In this blog post, we will finish swapping out the frontend and backend of our Python AWS application. Although Flask and Redis are different from Django and MySQL in many ways, the underlying infrastructure behind their deployment is nonetheless very similar, and can be effortlessly updated as we transition from one to the other.

We will be paying additional attention to security, and will be making use of Pulumi’s secret management tools to protect our passwords and private keys.

The first step is to create a new directory and initialize a Pulumi project with pulumi new aws-python.

$ mkdir aws-py-django-voting-app && cd aws-py-django-voting-app
$ pulumi new aws-python

Next, let’s create a folder to hold our application and start a Django project in it.

$ mkdir frontend && cd frontend
$ django-admin startproject mysite
$ cd mysite

This tutorial was written for the aws-py-django-voting-app example but will work with any other Django application. The most important file is ./mysite/settings.py, which we will modify to accept secrets and configuration parameters in the form of environment variables. A common mistake that programmers make is to submit files with important data to source code repositories. Even if it is a private repository, it is still not recommended to leave passwords and private keys in your files.

SECRET_KEY = os.environ['SECRET_KEY']

DATABASES = {
 'default': {
 'ENGINE': 'django.db.backends.mysql',
 'NAME': os.environ['DATABASE_NAME'],
 'USER': os.environ['USER_NAME'],
 'PASSWORD': os.environ['USER_PASSWORD'],
 'HOST': os.environ['DATABASE_ADDRESS'],
 'PORT': os.environ['DATABASE_PORT'],
 }
}

To perform database migrations as part of the deployment, we will create a setupDatabase.sh script inside the aws-py-django-voting-app/frontend/mysite folder. As a bonus, the script will create an admin account that we can use to log into our website. Make sure the file has exec permissions.

#!/bin/bash
set -exu
python3 /mysite/manage.py makemigrations
python3 /mysite/manage.py migrate
export DJANGO_SUPERUSER_PASSWORD=$DJANGO_PASSWORD
python3 /mysite/manage.py createsuperuser \
 --no-input \
 --username=$DJANGO_NAME \
 --email=$DJANGO_NAME@example.com

The next step is to go back into the aws-py-django-voting-app/frontend folder and containerize our application with Docker. First, we will list the libraries our application uses by creating a requirements.txt file with the following lines.

django==3.1
mysqlclient==1.4.0

And lastly, we can create the Dockerfile.

FROM ubuntu:18.04

WORKDIR /

EXPOSE 80

RUN apt-get update && \
 apt install -y gcc python3-dev python3-pip mysql-client-core-5.7 libmysqlclient-dev

ADD requirements.txt /

RUN pip3 install -r requirements.txt

ADD mysite /mysite

CMD [ "python3", "/mysite/manage.py", "runserver", "0.0.0.0:80" ]

Now that our Django application and Dockerfile are ready, we can return to the main aws-py-django-voting-app folder. The Pulumi project requires several configuration variables, which we set using pulumi config set. They are used to configure the MySQL admin account, a user account for initializing the table, and the Django website admin account. The private key that Django uses will also be passed in the same way.

$ pulumi config set sql-admin-name <NAME>
$ pulumi config set sql-admin-password <PASSWORD> --secret
$ pulumi config set sql-user-name <NAME>
$ pulumi config set sql-user-password <PASSWORD> --secret
$ pulumi config set django-admin-name <NAME>
$ pulumi config set django-admin-password <PASSWORD> --secret
$ pulumi config set django_secret_key = <VALUE> --secret

The requirements.txt file lists the libraries used by the project. We will need to add the following:

pulumi-docker>=2.0.0,<3.0.0
pulumi-mysql>=2.0.0,<3.0.0
mysql-connector-python>=1.0.0,<10.0.0

The first few lines of our __main__.py file indicate the libraries to import and describe the application’s configuration options.

import json
import base64
import pulumi
import pulumi_aws as aws
import pulumi_docker as docker
import pulumi_mysql as mysql

config = pulumi.Config()
sql_admin_name = config.require("sql-admin-name")
sql_admin_password = config.require_secret("sql-admin-password")
sql_user_name = config.require("sql-user-name")
sql_user_password = config.require_secret("sql-user-password")
availability_zone = pulumi.Config("aws").get("region")

django_admin_name = config.require("django-admin-name")
django_admin_password = config.require_secret("django-admin-password")

django_secret_key = config.require_secret("django-secret-key")

After setting up the imports and configurations, we create an ECS Cluster, VPC, Subnet, RDS instance, and several other items. All the components are identical to the ones in the first and second blog posts, and the full code can be seen in this example’s github repository.

app_cluster = aws.ecs.Cluster(...)

app_vpc = aws.ec2.Vpc(...)
app_vpc_subnet = aws.ec2.Subnet(...)

app_gateway = aws.ec2.InternetGateway(...)
app_routetable = aws.ec2.RouteTable(...)
app_security_group = aws.ec2.SecurityGroup(...)

app_exec_role = aws.iam.Role(...)
exec_policy_attachment = aws.iam.RolePolicyAttachment(...)
app_task_role = aws.iam.Role(...)
task_policy_attachment = aws.iam.RolePolicyAttachment(...)
task_policy_attachment = aws.iam.RolePolicyAttachment(...)

app_ecr_repo = aws.ecr.Repository(...)
app_lifecycle_policy = aws.ecr.LifecyclePolicy(...)

extra_rds_subnet = aws.ec2.Subnet(...)
app_database_subnetgroup = aws.rds.SubnetGroup(...)
mysql_rds_server = aws.rds.Instance(...)
mysql_provider = mysql.Provider(...)
mysql_database = mysql.Database(...)
mysql_user = mysql.User(...)
mysql_access_grant = mysql.Grant(...)

With the basic infrastructure and MySQL backend in place, all that’s left is to create the Django frontend.

A target group, balancer, and listener are created for the frontend.

django_targetgroup = aws.lb.TargetGroup("django-targetgroup",
 port=80,
 protocol="TCP",
 target_type="ip",
 stickiness= {
 "enabled": False,
 "type": "lb_cookie",
 },
 vpc_id=app_vpc.id)

django_balancer = aws.lb.LoadBalancer("django-balancer",
 load_balancer_type="network",
 internal=False,
 security_groups=[],
 subnets=[app_vpc_subnet.id])

django_listener = aws.lb.Listener("django-listener",
 load_balancer_arn=django_balancer.arn,
 port=80,
 protocol="TCP",
 default_actions=[{
 "type": "forward",
 "target_group_arn": django_targetgroup.arn
 }])

The application is built into a Docker image and pushed to our ECR repository we created earlier.

def get_registry_info(rid):
 creds = aws.ecr.get_credentials(registry_id=rid)
 decoded = base64.b64decode(creds.authorization_token).decode()
 parts = decoded.split(':')
 if len(parts) != 2:
 raise Exception("Invalid credentials")
 return docker.ImageRegistry(creds.proxy_endpoint, parts[0], parts[1])

app_registry = app_ecr_repo.registry_id.apply(get_registry_info)

django_image = docker.Image("django-dockerimage",
 image_name=app_ecr_repo.repository_url,
 build="./frontend",
 skip_push=False,
 registry=app_registry
)

To help with debugging our application, we will create a Cloudwatch instance to store all logs.

django_log_group = aws.cloudwatch.LogGroup("django-log-group",
 retention_in_days=1,
 name="django-log-group"
)

Our project is special because it uses two independent ECS services—one that sets up the MySQL database, and one that continuously runs and handles the website.

While it is possible to have a single ECS service set up both the database and the server, doing so presents the risk of exposing important information to the internet. To set up the database, the ECS service would need to know our MySQL admin credentials. However, since the service would also handle the website, it would mean that a successful attack on our Django website could allow the attacker to access the credentials for our backend.

First, let’s create the task definition that runs once and sets up the database.

django_database_task_definition = aws.ecs.TaskDefinition("django-database-task-definition",
 family="django_database_task_definition-family",
 cpu="256",
 memory="512",
 network_mode="awsvpc",
 requires_compatibilities=["FARGATE"],
 execution_role_arn=app_exec_role.arn,
 task_role_arn=app_task_role.arn,
 container_definitions=pulumi.Output.all(
 # We must do Output.all() to access Pulumi secrets
 django_image.image_name,
 django_secret_key,
 mysql_database.name,
 sql_admin_name,
 sql_admin_password,
 django_admin_name,
 django_admin_password,
 mysql_rds_server.address,
 mysql_rds_server.port).apply(lambda args: json.dumps([{
 "name": "django-container",
 "image": args[0],
 "memory": 512,
 "essential": True,
 "portMappings": [{
 "containerPort": 80,
 "hostPort": 80,
 "protocol": "tcp"
 }],
 "environment": [
 { "name": "SECRET_KEY", "value": args[1] },
 { "name": "DATABASE_NAME", "value": args[2] },
 { "name": "USER_NAME", "value": args[3] },
 { "name": "USER_PASSWORD", "value": args[4] },
 { "name": "DJANGO_NAME", "value": args[5] },
 { "name": "DJANGO_PASSWORD", "value": args[6] },
 { "name": "DATABASE_ADDRESS", "value": args[7] },
 { "name": "DATABASE_PORT", "value": str(int(args[8])) },
 ],
 "logConfiguration": {
 "logDriver": "awslogs",
 "options": {
 "awslogs-group": "django-log-group",
 "awslogs-region": "us-west-2",
 "awslogs-stream-prefix": "djangoApp-database",
 },
 },
 # We override the command in the Dockerfile with a new one
 "command": ["/mysite/setupDatabase.sh"]
 }])))

We can now launch our first service on Fargate.

django_database_service = aws.ecs.Service("django-database-service",
 cluster=app_cluster.arn,
 desired_count=1,
 launch_type="FARGATE",
 task_definition=django_database_task_definition.arn,
 wait_for_steady_state=False,
 network_configuration={
 "assign_public_ip": "true",
 "subnets": [app_vpc_subnet.id],
 "security_groups": [app_security_group.id]
 },
 load_balancers=[{
 "target_group_arn": django_targetgroup.arn,
 "container_name": "django-container",
 "container_port": 80,
 }],
 opts=pulumi.ResourceOptions(depends_on=[django_listener]),
)

Finally, let’s create the second task definition to run the website.

django_site_task_definition = aws.ecs.TaskDefinition("django-site-task-definition",
 family="django-site-task-definition-family",
 cpu="256",
 memory="512",
 network_mode="awsvpc",
 requires_compatibilities=["FARGATE"],
 execution_role_arn=app_exec_role.arn,
 task_role_arn=app_task_role.arn,
 container_definitions=pulumi.Output.all(
 django_image.image_name,
 django_secret_key,
 mysql_database.name,
 sql_user_name,
 sql_user_password,
 mysql_rds_server.address,
 mysql_rds_server.port).apply(lambda args: json.dumps([{
 "name": "django-container",
 "image": args[0],
 "memory": 512,
 "essential": True,
 "portMappings": [{
 "containerPort": 80,
 "hostPort": 80,
 "protocol": "tcp"
 }],
 "environment": [
 { "name": "SECRET_KEY", "value": args[1] },
 { "name": "DATABASE_NAME", "value": args[2] },
 { "name": "USER_NAME", "value": args[3] },
 { "name": "USER_PASSWORD", "value": args[4] },
 { "name": "DATABASE_ADDRESS", "value": args[5] },
 { "name": "DATABASE_PORT", "value": str(int(args[6])) },
 ],
 "logConfiguration": {
 "logDriver": "awslogs",
 "options": {
 "awslogs-group": "django-log-group",
 "awslogs-region": "us-west-2",
 "awslogs-stream-prefix": "djangoApp-site",
 },
 },
 }])))

Similarly, we will launch our second task definition on Fargate.

django_site_service = aws.ecs.Service("django-site-service",
 cluster=app_cluster.arn,
 desired_count=1,
 launch_type="FARGATE",
 task_definition=django_site_task_definition.arn,
 wait_for_steady_state=False,
 network_configuration={
 "assign_public_ip": "true",
 "subnets": [app_vpc_subnet.id],
 "security_groups": [app_security_group.id]
 },
 load_balancers=[{
 "target_group_arn": django_targetgroup.arn,
 "container_name": "django-container",
 "container_port": 80,
 }],
 opts=pulumi.ResourceOptions(depends_on=[django_listener]),

To connect to our application, we export the DNS name of the Django balancer and open it in a browser window.

pulumi.export("app-url", django_balancer.dns_name)

To add questions to the database, add “/admin/” to the end of the URL, and log in using your Django admin credentials.

The Administration panel allows you to manage the state of your application. Groups, users, choices, and questions can all be added easily.

Comparison between Flask and Django:

In this example, I described how to set up a basic Django voting application and deploy it to AWS. Although our Django and Flask applications use very different AWS services, we seamlessly replaced the previous infrastructure with what was needed.

Next week, we’ll explore PostgreSQL, Express, React, and Node.js, and use them to create a simple application.

The blog post’s full code and an in-depth explanation for each component are on Github.

Announcing Python Tooling Improvements

Justin Van Patten — Thu, 27 Aug 2020 00:00:00 +0000

Today we’re excited to announce some fairly significant improvements to the experience of writing Pulumi programs in Python. We’ve added type annotations to APIs and now allow passing nested data as strongly typed classes instead of raw dicts. This provides a much better editing experience in IDEs, improved type checking, and overall consistency.

Type Annotations

PEP 484 type annotations have been added to all Python APIs, allowing IDEs like PyCharm and VS Code and tools like mypy to catch errors earlier.

Consider the following example for retrieving an AMI, with an incorrect most_recent argument:

from pulumi_aws import get_ami

ami = get_ami(most_recent="true",
 owners=["137112412989"],
 filters=[{"name": "name", "values": ["amzn-ami-hvm-*"]}])

PyCharm shows the error:

As does VS Code with mypy:

And running mypy manually also reports the issue:

$ mypy __main__.py
__main__.py:3: error: Argument "most_recent" to "get_ami" has incompatible type "str"; expected "Optional[bool]"
Found 1 error in 1 file (checked 1 source file)

Data Classes

Consider an AWS bucket hosting a static website. Previously, the website and cors_rules inputs needed to be specified as raw dicts:

from pulumi_aws import s3

my_bucket = s3.Bucket("my-bucket",
 acl="public-read",
 website={
 "indexDocument": "index.html",
 "errorDocument": "error.html",
 },
 cors_rules=[{
 "allowedHeaders": ["*"],
 "allowedMethods": ["PUT", "POST"],
 "allowedOrigins": ["https://s3-website-test.mydomain.com"],
 "exposeHeaders": ["ETag"],
 "maxAgeSeconds": 3000,
 }],
)

You would likely have several questions if writing the above example from scratch. What keys can be set for website and cors_rules? Should the keys be camelCase or snake_case? Which keys are required? What are the types of values? To answer these questions, you’d need to refer to Bucket’s documentation.

We now allow optionally passing nested values using data classes:

from pulumi_aws import s3

my_bucket = s3.Bucket("my-bucket",
 acl="public-read",
 website=s3.BucketWebsiteArgs(
 index_document="index.html",
 error_document="error.html",
 ),
 cors_rules=[s3.BucketCorsRuleArgs(
 allowed_headers=["*"],
 allowed_methods=["PUT", "POST"],
 allowed_origins=["https://s3-website-test.mydomain.com"],
 expose_headers=["ETag"],
 max_age_seconds=3000,
 )],
)

IDEs can display argument names and types, reducing the need to refer to documentation. If a required argument is missing, or a value isn’t the right type, IDEs will immediately show the problem, allowing errors to be caught earlier.

The use of data classes makes setting nested values consistent with top-level values, with arguments that are always snake_case, for a more Pythonic feel.

Here’s what this looks like in VS Code using the new Pylance language server for Python:

Support for passing data classes is a fully compatible change. Nested data are passed as either raw dicts or data classes, so existing programs using dicts will continue to work as-is without modification.

Accessing Outputs

Nested outputs can now be accessed in a more strongly-typed way as well.

Consider accessing one of the website settings from the bucket. Previously, nested outputs were returned as raw dicts, and you’d get the value by looking it up in the dict by its key:

index_document = my_bucket.website.apply(lambda w: w["indexDocument"])

Now, nested outputs are returned as data classes, and values can be accessed via properties.

index_document = my_bucket.website.apply(lambda w: w.index_document)

Existing programs that lookup the values in the dict continue to work as-is because the new output data classes are subclasses of dict and continue to return the same data with the same keys as before.

Other improvements

Another improvement we’ve made is to hover tooltips in IDEs. Hovering over members in PyCharm and VS Code with Pylance now shows the member’s associated documentation:

We’ve additionally made some changes to reduce the amount of noise in IDE completion lists when “dotting” into individual modules.

Conclusion

All considered, these changes provide a major improvement to the Python experience.

The improvements are available in the latest releases of the AWS, Azure, AzureAD, Google Cloud, and Random providers. They will be rolling out to the rest of the providers this week, along with updates to examples and documentation.

Take the new providers for a spin and let us know what you think in the Community Slack!

Deploying a MySQL schema using Dynamic Providers

Vova Ivanov — Tue, 18 Aug 2020 00:00:00 +0000

In our previous post, we created a Python voting application using Flask and Redis. This blog post will explore creating a MySQL database and initializing it with a schema and data. What seems to be a simple step is much more interesting than it appears, because Pulumi’s MySQL provider does not support creating and populating tables. To do it, we will extend it with a Dynamic Provider.

The existing Pulumi MySQL provider allows us to create the MySQL server, database, and different users. However, it would be great to create the tables as part of the deployment, too, since database schema can be considered a type of infrastructure. Because creating tables requires admin credentials, deploying them along with the other infrastructure allows us to limit the sharing of admin credentials, and run the application under restricted permissions at all times.

A great advantage of Pulumi is its extensible and modular design. If support for something isn’t implemented, you can write it yourself easily. We will be writing a Dynamic Provider that connects to a MySQL server, initializes a table, and creates some starting data all during pulumi up.

The first step is to create a new directory and initialize a Pulumi project with pulumi new aws-python.

$ mkdir aws-py-dynamicresource && cd aws-py-dynamicresource
$ pulumi new aws-python

This project requires several configuration variables, which we set using pulumi config set. They are used to configure the MySQL admin account during deployment, and a user account for initializing the table.

$ pulumi config set sql-admin-name <NAME>
$ pulumi config set sql-admin-password <PASSWORD> --secret
$ pulumi config set sql-user-name <NAME>
$ pulumi config set sql-user-password <PASSWORD> --secret

The requirements.txt file lists the libraries used in the project. We will need to add the following:

pulumi-mysql>=1.0.0,<3.0.0
mysql-connector-python>=1.0.0,<10.0.0

We will write the code for our Dynamic Provider in a separate file to reduce clutter, for example, MySqlDynamicProvider.py. The first few lines of the file will indicate which libraries to import.

import mysql.connector as connector
from mysql.connector import errorcode
from pulumi import Input, Output, ResourceOptions
from pulumi.dynamic import *
from typing import Any, Optional
import binascii
import os

After setting up the imports, the next step is to write the code. The Dynamic Provider’s first component is a class with the arguments that the dynamic provider requires when created. These arguments are given a type Input[str] and automatically converted to regular str before being passed to the provider’s functions.

class SchemaInputs(object):
 creator_name: Input[str]
 creator_password: Input[str]
 server_address: Input[str]
 database_name: Input[str]
 creation_script: Input[str]
 deletion_script: Input[str]
 def __init__(self, creator_name, creator_password, server_address, database_name, creation_script, deletion_script):
 self.creator_name = creator_name
 self.creator_password = creator_password
 self.server_address = server_address
 self.database_name = database_name
 self.creation_script = creation_script
 self.deletion_script = deletion_script

The second step is to write the Dynamic Provider. The provider handles the create, read, update, and delete operations the resource needs. The create function instantiates a new resource and assigns it a unique ID. The delete function deletes an existing resource. The diff function determines if we can update the resource without entirely replacing it, and the update function performs the update.

class SchemaProvider(ResourceProvider):
 def create(self, args):
 connection = connector.connect(user=args["creator_name"],
 password=args["creator_password"],
 host=args["server_address"],
 database=args["database_name"])
 cursor = connection.cursor()
 cursor.execute(args["creation_script"])
 return CreateResult("schema-"+binascii.b2a_hex(os.urandom(16)).decode("utf-8"), outs=args)

 def delete(self, id, args):
 connection = connector.connect(user=args["creator_name"],
 password=args["creator_password"],
 host=args["server_address"],
 database=args["database_name"])
 cursor = connection.cursor()
 cursor.execute(args["deletion_script"])

 def diff(self, id, oldInputs, newInputs):
 replaces = []
 if (oldInputs["server_address"] != newInputs["server_address"]): replaces.append("server_address")
 if (oldInputs["database_name"] != newInputs["database_name"]): replaces.append("database_name")
 if (oldInputs["creation_script"] != newInputs["creation_script"]): replaces.append("creation_script")

 return DiffResult(
 changes=oldInputs != newInputs,
 replaces=replaces,
 stables=None,
 delete_before_replace=True)

 def update(self, id, oldInputs, newInputs):
 return UpdateResult(outs={**newInputs})

And lastly, we add the main Schema resource that we instantiate in our infrastructure code. We make all the inputs to our provider available as outputs, so that they can be accessed and exported.

class Schema(Resource):
 creator_name: Output[str]
 creator_password: Output[str]
 server_address: Output[str]
 database_name: Output[str]
 creation_script: Output[str]
 deletion_script: Output[str]
 def __init__(self, name: str, args: SchemaInputs, opts = None):
 super().__init__(SchemaProvider(), name, vars(args), opts)

With the dynamic provider finished, all that is left is to create it in our infrastructure. Like the previous post, the project uses a __main__.py file; the first few lines indicate the libraries to import and describe the application’s configuration options.

import json
import base64
import pulumi
import pulumi_aws as aws
import pulumi_mysql as mysql
from MySqlDynamicProvider import *

config = pulumi.Config()
admin_name = config.require("sql-admin-name")
admin_password = config.require_secret("sql-admin-password")
user_name = config.require("sql-user-name")
user_password = config.require_secret("sql-user-password")
availability_zone = pulumi.Config("aws").get("region")

To allow different tasks within our project to communicate, we create a Virtual Private Cloud and an associated subnet. Two subnets are required for the project, so the availability zone suffix is set to “a”.

app_vpc = aws.ec2.Vpc("app-vpc",
 cidr_block="172.31.0.0/16",
 enable_dns_hostnames=True)

app_vpc_subnet = aws.ec2.Subnet("app-vpc-subnet",
 cidr_block="172.31.0.0/20",
 availability_zone=availability_zone + "a",
 vpc_id=app_vpc)

A gateway and routing table are needed to allow the VPC to communicate with the Internet. Once created, we associate the routing table with our VPC.

app_gateway = aws.ec2.InternetGateway("app-gateway",
 vpc_id=app_vpc.id)

app_routetable = aws.ec2.RouteTable("app-routetable",
 routes=[
 {
 "cidr_block": "0.0.0.0/0",
 "gateway_id": app_gateway.id,
 }
 ],
 vpc_id=app_vpc.id)

app_routetable_association = aws.ec2.MainRouteTableAssociation("app_routetable_association",
 route_table_id=app_routetable.id,
 vpc_id=app_vpc)

To control traffic flow between applications running inside our VPC, we create a security group.

app_security_group = aws.ec2.SecurityGroup("security-group",
 vpc_id=app_vpc.id,
 description="Enables HTTP access",
 ingress=[{
 'protocol': 'tcp',
 'from_port': 0,
 'to_port': 65535,
 'cidr_blocks': ['0.0.0.0/0'],
 }],
 egress=[{
 'protocol': '-1',
 'from_port': 0,
 'to_port': 0,
 'cidr_blocks': ['0.0.0.0/0'],
 }])

Our MySQL database is created with an RDS instance. To create the instance, Amazon requires that it be given two subnets in different availability zones.

extra_rds_subnet = aws.ec2.Subnet("extra-rds-subnet",
 cidr_block="172.31.128.0/20",
 availability_zone=availability_zone + "b",
 vpc_id=app_vpc)

Both subnets are assigned to a SubnetGroup and belong to the RDS instance.

app_database_subnetgroup = aws.rds.SubnetGroup("app-database-subnetgroup",
 subnet_ids=[app_vpc_subnet.id, extra_rds_subnet.id])

mysql_rds_server = aws.rds.Instance("mysql-server",
 engine="mysql",
 username=admin_name,
 password=admin_password,
 instance_class="db.t2.micro",
 allocated_storage=20,
 skip_final_snapshot=True,
 publicly_accessible=True,
 db_subnet_group_name=app_database_subnetgroup.id,
 vpc_security_group_ids=[app_security_group.id])

Pulumi offers some additional tools to make handling MySQL easier.

mysql_provider = mysql.Provider("mysql-provider",
 endpoint=mysql_rds_server.endpoint,
 username=admin_name,
 password=admin_password)

We initialize the example database and create a user to manage it.

mysql_database = mysql.Database("mysql-database",
 name="votes-database",
 opts=pulumi.ResourceOptions(provider=mysql_provider))

mysql_user = mysql.User("mysql-standard-user",
 user=user_name,
 host="example.com",
 plaintext_password=user_password,
 opts=pulumi.ResourceOptions(provider=mysql_provider))

The user only needs the SELECT and UPDATE permissions to function.

mysql_access_grant = mysql.Grant("mysql-access-grant",
 user=mysql_user.user,
 host=mysql_user.host,
 database=mysql_database.name,
 privileges= ["SELECT", "UPDATE"],
 opts=pulumi.ResourceOptions(provider=mysql_provider))

Now, we use our Dynamic Provider. The provider takes creation_script as a parameter, connects to our MySQL server, and runs it during deployment.

creation_script = """
 CREATE TABLE votesTable (
 choice_id int(10) NOT NULL AUTO_INCREMENT,
 vote_count int(10) NOT NULL,
 PRIMARY KEY (choice_id)
 ) ENGINE=InnoDB;
 INSERT INTO votesTable(choice_id, vote_count) VALUES (0,0);
 INSERT INTO votesTable(choice_id, vote_count) VALUES (1,0);
 """

The deletion_script parameter drops the table and deletes stored data.

deletion_script = "DROP TABLE votesTable CASCADE"

When we create our resource, it behaves the same way as any other Pulumi resource but takes its arguments as a SchemaInputs object.

mysql_votes_table = Schema(name="mysql_votes_table",
 args=SchemaInputs(admin_name, admin_password, mysql_rds_server.address, mysql_database.name, creation_script, deletion_script))

We can export the ID of our new resource, and view it when deployed.

pulumi.export("dynamic-resource-id",mysql_votes_table.id)

In this example, I showed how straightforward it is to expand functionality with Pulumi by writing additional code. Dynamic Providers enable excellent flexibility in cloud architecture design and help break down barriers that would otherwise be challenging to overcome.

Next week, I’ll change the frontend from Flask to Django, and will show how to integrate it with our new MySQL server.

The blog post’s full code and an in-depth explanation for each component can be found on Github.

Creating a Python AWS Application Using Flask and Redis

Vova Ivanov — Thu, 13 Aug 2020 15:06:26 -0700

Meet Vova Ivanov—one of the Pulumi summer interns. He’ll be writing about his experiences learning Pulumi while modernizing a web app and its underlying infrastructure.

I’ve recently started developing with Pulumi, and have begun to explore its inner workings and mechanisms. I decided to construct a production-level application and to document each step that I take and my progress as I go along.

This blog post recreates the existing Typescript voting app example step by step in Python with Flask as the frontend and Redis as the backend. In future blog posts, we will explore how to change the front and backends, how to upgrade the app with additional AWS services, and migrating from one cloud provider to another.

The first step is to clone the Typescript voting app example. We’ll use a sparse clone to copy only the aws-ts-voting-app directory.

$ mkdir examples && cd examples
$ git init
$ git remote add origin -f https://github.com/pulumi/examples/
$ git config core.sparseCheckout true
$ echo aws-ts-voting-app >> .git/info/sparse-checkout
$ git pull origin master

The next step is to create a new directory and initialize a Pulumi project with pulumi new aws-python.

$ mkdir aws-py-voting-app && cd aws-py-voting-app
$ pulumi new aws-python

Since we won’t be modifying the application’s frontend, we simply copy the frontend/ folder from the example into our directory.

$ cp -r aws-ts-voting-app/frontend/ aws-py-voting-app/frontend

The requirements.txt file lists the libraries that the project depends on. We will need to add the following:

pulumi-docker>=2.0.0,<3.0.0

The biggest difference between our project and the example is the code describing the cloud infrastructure we want to create. Instead of an index.ts file, the project uses a __main__.py file. The first few lines of the file will indicate which libraries to import and describe a pair of configuration options used by the application.

import json
import pulumi
import pulumi_aws as aws
import pulumi_docker as docker

config = pulumi.Config()
redis_password = config.require("redis-password")
redis_port = 6379

After setting up the imports and configurations, we create an Elastic Container Service Cluster. A Cluster represents a group of tasks and services that work together for a certain purpose. In this instance, the purpose is to provide users with a voting application.

app_cluster = aws.ecs.Cluster("app-cluster")

To allow different tasks within our cluster to communicate, we create a Virtual Private Cloud and an associated subnet.

app_vpc = aws.ec2.Vpc("app-vpc",
 cidr_block="172.31.0.0/16",
 enable_dns_hostnames=True)

app_vpc_subnet = aws.ec2.Subnet("app-vpc-subnet",
 cidr_block="172.31.32.0/20",
 vpc_id=app_vpc)

A gateway and routing table are needed to allow the VPC to communicate with the Internet. Once created, we associate the routing table with our VPC.

app_gateway = aws.ec2.InternetGateway("app-gateway",
 vpc_id=app_vpc.id)

app_routetable = aws.ec2.RouteTable("app-routetable",
 routes=[
 {
 "cidr_block": "0.0.0.0/0",
 "gateway_id": app_gateway.id,
 }
 ],
 vpc_id=app_vpc.id)

app_routetable_association = aws.ec2.MainRouteTableAssociation("app_routetable_association",
 route_table_id=app_routetable.id,
 vpc_id=app_vpc)

To control traffic flow between applications running inside our VPC, we create a security group.

app_security_group = aws.ec2.SecurityGroup("security-group",
 vpc_id=app_vpc.id,
 description="Enables HTTP access",
 ingress=[{
 'protocol': 'tcp',
 'from_port': 0,
 'to_port': 65535,
 'cidr_blocks': ['0.0.0.0/0'],
 }],
 egress=[{
 'protocol': '-1',
 'from_port': 0,
 'to_port': 0,
 'cidr_blocks': ['0.0.0.0/0'],
 }])

To start our services, we need to create an Identity and Access Management (IAM) role, and attach execution permissions to it.

app_exec_role = aws.iam.Role("app-exec-role",
 assume_role_policy="""{
 "Version": "2012-10-17",
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Principal": {
 "Service": "ecs-tasks.amazonaws.com"
 },
 "Effect": "Allow",
 "Sid": ""
 }]
 }""")

exec_policy_attachment = aws.iam.RolePolicyAttachment("app-exec-policy", role=app_exec_role.name,
 policy_arn=aws.iam.ManagedPolicy.AMAZON_ECS_TASK_EXECUTION_ROLE_POLICY)

Likewise, our ECS service will need to have a task role to manage it, along with its own set of permissions.

app_task_role = aws.iam.Role("app-task-role",
 assume_role_policy="""{
 "Version": "2012-10-17",
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Principal": {
 "Service": "ecs-tasks.amazonaws.com"
 },
 "Effect": "Allow",
 "Sid": ""
 }]
 }""")

task_policy_attachment = aws.iam.RolePolicyAttachment("app-access-policy", role=app_task_role.name,
 policy_arn=aws.iam.ManagedPolicy.AMAZON_ECS_FULL_ACCESS)

An Elastic Container Registry Repository stores the application Docker images that we want to run. The life cycle policy automatically removes the oldest untagged image that we uploaded.

app_ecr_repo = aws.ecr.Repository("app-ecr-repo",
 image_tag_mutability="MUTABLE")

app_lifecycle_policy = aws.ecr.LifecyclePolicy("app-lifecycle-policy",
 repository=app_ecr_repo.name,
 policy="""{
 "rules": [
 {
 "rulePriority": 10,
 "description": "Remove untagged images",
 "selection": {
 "tagStatus": "untagged",
 "countType": "imageCountMoreThan",
 "countNumber": 1
 },
 "action": {
 "type": "expire"
 }
 }
 ]
 }""")

With the basic infrastructure in place, we can begin to set up the application itself. First, we will start with the Redis backend.

We need to create a target group, a balancer, and a listener to allow Redis to communicate with the other services in our VPC, w

redis_targetgroup = aws.lb.TargetGroup("redis-targetgroup",
 port=redis_port,
 protocol="TCP",
 target_type="ip",
 stickiness= {
 "enabled": False,
 "type": "lb_cookie",
 },
 vpc_id=app_vpc.id)

redis_balancer = aws.lb.LoadBalancer("redis-balancer",
 load_balancer_type="network",
 internal= False,
 security_groups=[],
 subnets=[app_vpc_subnet.id])

redis_listener = aws.lb.Listener("redis-listener",
 load_balancer_arn=redis_balancer.arn,
 port=redis_port,
 protocol="TCP",
 default_actions=[{
 "type": "forward",
 "target_group_arn": redis_targetgroup.arn
 }])

Every service running within ECS requires a task definition, specifying what hardware, internet, and container settings it will use.

redis_task_definition = aws.ecs.TaskDefinition("redis-task-definition",
 family="redis-task-definition-family",
 cpu="256",
 memory="512",
 network_mode="awsvpc",
 requires_compatibilities=["FARGATE"],
 execution_role_arn=app_exec_role.arn,
 task_role_arn=app_task_role.arn,
 container_definitions=json.dumps([{
 "name": "redis-container",
 "image": "redis:alpine",
 "memory": 512,
 "essential": True,
 "portMappings": [{
 "containerPort": redis_port,
 "hostPort": redis_port,
 "protocol": "tcp"
 }],
 "command": ["redis-server", "--requirepass", redis_password],
 }]))

Now that we have our task definition blueprint ready, we can launch a service using it. An endpoint is created to hold information needed to connect the backend with the frontend.

redis_service = aws.ecs.Service("redis-service",
 cluster=app_cluster.arn,
 desired_count=1,
 launch_type="FARGATE",
 task_definition=redis_task_definition.arn,
 wait_for_steady_state=False,
 network_configuration={
 "assign_public_ip": "true",
 "subnets": [app_vpc_subnet.id],
 "security_groups": [app_security_group.id]
 },
 load_balancers=[{
 "target_group_arn": redis_targetgroup.arn,
 "container_name": "redis-container",
 "container_port": redis_port,
 }],
 opts=pulumi.ResourceOptions(depends_on=[redis_listener]),
)

redis_endpoint = {"host": redis_balancer.dns_name, "port": redis_port}

The Redis backend is complete, and now all that’s left is to create the Flask frontend.

A similar set of a target group, balancer, and listener is created for the frontend. Unlike Redis, Flask will serve as the webpage and, as such, needs to receive its data from port 80.

flask_targetgroup = aws.lb.TargetGroup("flask-targetgroup",
 port=80,
 protocol="TCP",
 target_type="ip",
 stickiness= {
 "enabled": False,
 "type": "lb_cookie",
 },
 vpc_id=app_vpc.id)

flask_balancer = aws.lb.LoadBalancer("flask-balancer",
 load_balancer_type="network",
 internal=False,
 security_groups=[],
 subnets=[app_vpc_subnet.id])

flask_listener = aws.lb.Listener("flask-listener",
 load_balancer_arn=flask_balancer.arn,
 port=80,
 protocol="TCP",
 default_actions=[{
 "type": "forward",
 "target_group_arn": flask_targetgroup.arn
 }])

The application is built into a Docker image and pushed to our ECR repository we created earlier.

def get_registry_info(rid):
 creds = aws.ecr.get_credentials(registry_id=rid)
 decoded = base64.b64decode(creds.authorization_token).decode()
 parts = decoded.split(':')
 if len(parts) != 2:
 raise Exception("Invalid credentials")
 return docker.ImageRegistry(creds.proxy_endpoint, parts[0], parts[1])

app_registry = app_ecr_repo.registry_id.apply(get_registry_info)

flask_image = docker.Image("flask-dockerimage",
 image_name=app_ecr_repo.repository_url,
 build="./frontend",
 skip_push=False,
 registry=app_registry
)

A task definition is created for the frontend; this time, using the flask image that we built. The redis endpoint is passed in as a group of environment variables, informing our app of our database’s host and port values.

flask_task_definition = aws.ecs.TaskDefinition("flask-task-definition",
 family="frontend-task-definition-family",
 cpu="256",
 memory="512",
 network_mode="awsvpc",
 requires_compatibilities=["FARGATE"],
 execution_role_arn=app_exec_role.arn,
 task_role_arn=app_task_role.arn,
 container_definitions=pulumi.Output.all(flask_image.image_name, redis_endpoint).apply(lambda args: json.dumps([{
 "name": "flask-container",
 "image": args[0],
 "memory": 512,
 "essential": True,
 "portMappings": [{
 "containerPort": 80,
 "hostPort": 80,
 "protocol": "tcp"
 }],
 "environment": [
 { "name": "REDIS", "value": args[1]["host"] },
 { "name": "REDIS_PORT", "value": str(args[1]["port"]) },
 { "name": "REDIS_PWD", "value": redis_password },
 ],
 }])))

The service for our Flask frontend is launched. It can receive requests from the Internet, process them, and change the Redis database accordingly.

flask_service = aws.ecs.Service("flask-service",
 cluster=app_cluster.arn,
 desired_count=1,
 launch_type="FARGATE",
 task_definition=flask_task_definition.arn,
 wait_for_steady_state=False,
 network_configuration={
 "assign_public_ip": "true",
 "subnets": [app_vpc_subnet.id],
 "security_groups": [app_security_group.id]
 },
 load_balancers=[{
 "target_group_arn": flask_targetgroup.arn,
 "container_name": "flask-container",
 "container_port": 80,
 }],
 opts=pulumi.ResourceOptions(depends_on=[flask_listener]),
)

To connect to our application, we export the DNS name of the Flask balancer, and open it in a browser window.

pulumi.export("app-url", flask_balancer.dns_name)

In this example, I showed how straightforward it is to convert infrastructure code defined in TypeScript to one defined in Python. Teams often choose to standardize on one language and Pulumi expands that choice, allowing them to standardize on one language for the frontend, the backend, and the infrastructure.

Next week, I’ll show how to modernize the application by changing the frontend from Flask to Django, and swapping out the database from Redis to Amazon RDS—using Pulumi to stand up new resources as I go.

The full code for the blog post can be found on github.

Policy as Code with Python

Sophia Parafina — Tue, 16 Jun 2020 00:00:00 +0000

Policy as Code for Python is now GA in Pulumi 2.0. Policies written in code let you test, automate deployment, and enable version control. Python is a popular scripting language used for machine learning and artificial intelligence, data science, web development, and devops. It’s an ideal language for developers and operators to use in common.

Devops benefits greatly from policy as code because policies can use software development practices and verify resources without deployments that are costly in time and money. Organizations can also gain significant benefits through cost savings, fine-grained control over infrastructure, efficient deployments, improved compliance, and better use of cloud provider native resources.

How Policy as Code Works

Policies enforce specific criteria for a resource or a set of resources (stacks). For example, a common policy is to ensure that storage is not publicly accessible over the Internet or that virtual machines must have a firewall. Policies are enforced as either advisory, which prints a warning message the resource violates the policy; or mandatory, which prevents a resource deployment if it violates the policy.

Pulumi supports two types of policies:

ResourceValidationPolicy validates inputs of individual resources in a stack before the resource is created or modified.
StackValidationPolicy validates outputs of all resources in the stack after all resources have been created or modified.

ResourceValidationPolicy and StackValidationPolicy validations run during previews and updates. During previews, resources aren’t created or modified, so the ResourceValidationPolicy and StackValidationPolicy validations see a preview of what’s going to happen (as best we can determine). With stack validation policies, the validation happens after resources have been created or modified. Modifications to resources can’t be prevented, but using mandatory enforcement results in a deployment failing (despite the modified resources). If you run pulumi preview before an update and use a mandatory StackValidationPolicy, you can catch the problems before a real deployment occurs.

The difference is described in this table:

	Resource Validation	Stack Validation
What does it check?	Individual resources	All resources in the stack
When is the check performed?	Before resources are created/modified	After all stack resources have been created/modified
What information is available?	Resource input properties	Resource output properties (Note: inputs are propagated to outputs during preview)

Now that we have the basics of policy as code let’s take a look at example policies written in Python.

Examples

Policies are validation functions that validate resources in a Pulumi stack. In the examples below, we define a function that takes ResourceValidationArgs and ReportViolation as arguments. The function checks to see what type of resource and its arguments, these are used to determine if the resource violates the policy. If the resource is not compliant, the function reports the violation, and because the enforcement level is mandatory, the resource will not be deployed.

If you’d like to try out one of the examples, you can create a policy with pulumi policy new and run locally against a Pulumi program, specify pulumi up --policy-pack <path-to-policy-pack>.

from pulumi_policy import (
 EnforcementLevel,
 PolicyPack,
 ReportViolation,
 ResourceValidationArgs,
 ResourceValidationPolicy,
)

def s3_no_public_read_validator(args: ResourceValidationArgs, report_violation: ReportViolation):
 if args.resource_type == "aws:s3/bucket:Bucket" and "acl" in args.props:
 acl = args.props["acl"]
 if acl == "public-read" or acl == "public-read-write":
 report_violation(
 "You cannot set public-read or public-read-write on an S3 bucket. " +
 "Read more about ACLs here: https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html")

s3_no_public_read = ResourceValidationPolicy(
 name="s3-no-public-read",
 description="Prohibits setting the publicRead or publicReadWrite permission on AWS S3 buckets.",
 validate=s3_no_public_read_validator,
)

PolicyPack(
 name="aws-python",
 enforcement_level=EnforcementLevel.MANDATORY,
 policies=[
 s3_no_public_read,
 ],
)

from pulumi_policy import (
 EnforcementLevel,
 PolicyPack,
 ReportViolation,
 ResourceValidationArgs,
 ResourceValidationPolicy,
)

def storage_container_no_public_read_validator(args: ResourceValidationArgs, report_violation: ReportViolation):
 if args.resource_type == "azure:storage/container:Container" and "containerAccessType" in args.props:
 access_type = args.props["containerAccessType"]
 if access_type == "blob" or access_type == "container":
 report_violation(
 "Azure Storage Container must not have blob or container access set. " +
 "Read more about read access here: " +
 "https://docs.microsoft.com/en-us/azure/storage/blobs/storage-manage-access-to-resources")

storage_container_no_public_read = ResourceValidationPolicy(
 name="storage-container-no-public-read",
 description="Prohibits setting the public permission on Azure Storage Blob Containers.",
 validate=storage_container_no_public_read_validator,
)

PolicyPack(
 name="azure-python",
 enforcement_level=EnforcementLevel.MANDATORY,
 policies=[
 storage_container_no_public_read,
 ],
)

from pulumi_policy import (
 EnforcementLevel,
 PolicyPack,
 ReportViolation,
 ResourceValidationArgs,
 ResourceValidationPolicy,
)

def storage_bucket_no_public_read_validator(args: ResourceValidationArgs, report_violation: ReportViolation):
 if args.resource_type == "gcp:storage/bucketACL:BucketACL" and "predefinedAcl" in args.props:
 acl = args.props["predefinedAcl"]
 if acl == "public-read" or acl == "public-read-write":
 report_violation("Storage buckets acl cannot be set to public-read or public-read-write.")

storage_bucket_no_public_read = ResourceValidationPolicy(
 name="storage-bucket-no-public-read",
 description="Prohibits setting the publicRead or publicReadWrite permission on GCP Storage buckets.",
 validate=storage_bucket_no_public_read_validator,
)

PolicyPack(
 name="gcp-python",
 enforcement_level=EnforcementLevel.MANDATORY,
 policies=[
 storage_bucket_no_public_read,
 ],
)

from pulumi_policy import (
 EnforcementLevel,
 PolicyPack,
 ReportViolation,
 ResourceValidationArgs,
 ResourceValidationPolicy,
)

def no_public_services_validator(args: ResourceValidationArgs, report_violation: ReportViolation):
 if args.resource_type == "kubernetes:core/v1:Service" and "spec" in args.props:
 spec = args.props["spec"]
 if "type" in spec and spec["type"] == "LoadBalancer":
 report_violation(
 "Kubernetes Services cannot be of type LoadBalancer, which are exposed to " +
 "anything that can reach the Kubernetes cluster. This likely including the " +
 "public Internet.")

no_public_services = ResourceValidationPolicy(
 name="no-public-services",
 description="Kubernetes Services should be cluster-private.",
 validate=no_public_services_validator,
)

PolicyPack(
 name="kubernetes-python",
 enforcement_level=EnforcementLevel.MANDATORY,
 policies=[
 no_public_services,
 ],
)

Summary

Policy as Code is an important tool for building a secure and efficient cloud infrastructure. Pulumi lets you efficiently test resources before deployment and entire stacks when deployed. Learn more about what you can do with Policy as Code:

Testing Your Infrastructure as Code with Pulumi

Joe Duffy — Wed, 17 Apr 2019 00:00:00 +0000

Some parts of this blog post are out-of-date. Please refer to our Testing Guide for the updated overview and tutorials.

Using Pulumi and general purpose languages for infrastructure as code comes with many benefits: leveraging existing skills and knowledge, eliminating boilerplate through abstraction, and using the same ecosystem of tools like IDEs and linters that your team already knows and loves. In general, these are all attributes of software engineering, which not only make us more productive, but also improve the quality of our code. It’s only natural, therefore, that using general purpose languages unlocks another important software engineering practice: testing.

In this article, we will see the many ways in which Pulumi lets us test our infrastructure as code.

Why Test Your Infrastructure?

Before jumping into the details, it’s worth asking: Why test your infrastructure, anyway? There are many reasons to do this, however here are a few examples:

Unit testing individual functions or pieces of logic in your program
Ensuring that the program’s desired infrastructure state meets certain constraints
Catching common mistakes, like forgetting to encrypt your storage buckets, or exposing virtual machines unprotected to the open Internet
Ensuring that the provisioned infrastructure has certain desired attributes
Doing runtime testing of the application logic that is running inside of your compute infrastructure, to ensure that it is healthy after your infrastructure has been provisioned

As we see, there’s a broad spectrum of infrastructure testing options available. Pulumi has mechanisms available for testing at each point along this spectrum. Let’s dig in and see how.

Unit Testing

Pulumi programs are authored in a general purpose language like JavaScript, Python, TypeScript, or Go. The full power of each language is available, including access to tools and libraries for that runtime, including testing frameworks. Pulumi is also multi-cloud, meaning the full underlying platform for each cloud is available too, and therefore available for testing.

(Despite being multi-language and multi-cloud, for the purposes of this article let’s assume we are using JavaScript, with Mocha for testing, and targeting AWS. Python unittest, Go’s test framework, or any of your favorite unit test frameworks in your language of choice would do, and of course Pulumi works great with Azure, Google Cloud, and Kubernetes, among others.)

As we’ve seen, there are several reasons you may want to test your infrastructure code. One of them is basic unit testing. Because your code may have functions – say, to compute CIDR blocks, dynamically compute names and tags, etc. – you probably want to test them. Doing this is the same as writing ordinary unit tests against any program in your language as choice.

Going one step further, you may want to test the way your program allocates resources. To illustrate, let’s imagine we want to create a simple EC2-based webserver, but want to ensure:

Instances have a Name tag.
Instances must not use an inline userData script – we must use an AMI (image).
Instances must not have SSH open to the Internet.

Our running example is loosely based on our aws-js-webserver example:

index.js:

let aws = require("@pulumi/aws");

let group = new aws.ec2.SecurityGroup("web-secgrp", {
    ingress: [
 { protocol: "tcp", fromPort: 22, toPort: 22, cidrBlocks: ["0.0.0.0/0"] },
 { protocol: "tcp", fromPort: 80, toPort: 80, cidrBlocks: ["0.0.0.0/0"] },
    ],
});

let userData =
`#!/bin/bash echo "Hello, World!" > index.html nohup python -m SimpleHTTPServer 80 &`;

let server = new aws.ec2.Instance("web-server-www", {
    instanceType: "t2.micro",
    securityGroups: [ group.name ], // reference the group object above
    ami: "ami-c55673a0" // AMI for us-east-2 (Ohio),
    userData: userData // start a simple web server
});

exports.group = group;
exports.server = server;
exports.publicIp = server.publicIp;
exports.publicHostName = server.publicDns;

This is a basic Pulumi program: it simply allocates an EC2 security group and instance. Notice, however, that we are violating all three of the rules stated above – let’s write some tests!

Writing the Tests

The overall structure and scaffolding of our tests will look like any ordinary Mocha testing:

ec2tests.js

let assert = require("assert");
let mocha = require("mocha");
let pulumi = require("@pulumi/pulumi");
let infra = require("./index");

describe("Infrastructure", function() {
    let server = infra.server;
    describe("#server", function() {
 // TODO(check 1): Instances have a Name tag.
 // TODO(check 2): Instances must not use an inline userData script.
    });
    let group = infra.group;
    describe("#group", function() {
 // TODO(check 3): Instances must not have SSH open to the Internet.
    });
});

Now let’s implement our first test: ensuring that instances have a Name tag. To verify this we simply need to grab hold of the EC2 instance object, and check the relevant tags property:

// check 1: Instances have a Name tag.
it("must have a name tag", function(done) {
 pulumi.all([server.urn, server.tags]).apply(([urn, tags]) => {
     if (!tags || !tags["Name"]) {
     done(new Error(`Missing a name tag on server ${urn}`));
     } else {
      done();
     }
    });
});

This looks like a normal test, with a few noteworthy pieces:

Since we’re querying resource state before we’ve done a deployment, our tests are always effectively running in “plan” (or “preview”) mode. Thus, there are many properties whose values will simply not resolve yet or will be undefined. This includes any output properties computed by your cloud provider. That’s fine for these tests – we’re checking for valid inputs anyway. We’ll revisit this later when it comes to integration testing.
Because all Pulumi resource properties are “outputs” – since many of them are computed asynchronously – we need to use the apply method to get access to the values. This is a lot like promises and the then function.
Because we are using multiple properties, mostly just so we can show the resource URN in the error message, we need to use the pulumi.all function to join all of them.
Finally, since these outputs are resolved asynchronously, we need to use Mocha’s built-in asynchronous test capability, with the done callback, or by returning a promise.

After we’ve gotten through that setup, we get access to the raw inputs as plain JavaScript values. The tags property is a map, so we just make sure it is (1) not falsey, and (2) not missing an entry for the Name key in the map. This is very basic, but we can check anything!

Now let’s write our second check. It’s even easier:

// check 2: Instances must not use an inline userData script.
 it("must not use userData (use an AMI instead)", function(done) {
 pulumi.all([server.urn, server.userData]).apply(([urn, userData]) => {
 if (userData) {
 done(new Error(`Illegal use of userData on server ${urn}`));
 } else {
 done();
 }
    });
});

And finally, let’s write our third check. It’s a bit more complex because we’re searching for ingress rules associated with a security group – of which there may be many – and CIDR blocks within those ingress rules – of which there may also be many. But it’s still easy:

// check 3: Instances must not have SSH open to the Internet.
it("must not open port 22 (SSH) to the Internet", function(done) {
 pulumi.all([ group.urn, group.ingress ]).apply(([ urn, ingress ]) => {
 if (ingress.find(rule =>
 rule.fromPort == 22 && rule.cidrBlocks.find(block => block === "0.0.0.0/0"))) {
     done(new Error(`Illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0) on group ${urn}`));
 } else {
     done();
 }
    });
});

That’s it – now let’s actually run the tests!

Running the Tests

For the most part, we can run the tests in the usual way in your test framework of choice. There is one thing specific to Pulumi we need to cover first, however.

Usually you run your Pulumi programs with the pulumi CLI. That configures the language runtime, monitors execution from the engine so that resource operations can be recorded and turned into a plan, and so on. There is one problem, however: When running under the watchful eye of your test framework, this connection to the CLI, and Pulumi’s engine, will not exist.

To work around this problem, we simply have to mock three pieces of information:

Your project name, which can be supplied with the PULUMI_NODEJS_PROJECT environment variable (or more generally PULUMI_<LANGUAGE>_PROJECT in other languages).
Your stack name, which can be supplied with the PULUMI_NODEJS_STACK environment variable (or more generally PULUMI_<LANGUAGE>_STACK in other languages).
Your stack configuration variables. These can be supplied using the PULUMI_CONFIG environment variable, and its format is a JSON map of configuration key/value pairs.

The program will emit some warning messages indicating the connection to the CLI/engine is not available at runtime. This is important to know, because your program won’t actually be deploying anything – which can come as a surprise if that’s not what you meant to do! To tell Pulumi that’s what you intended, you can set PULUMI_TEST_MODE to true.

Imagine we want a project name of my-ws, stack name of dev, and AWS region of us-west-2. The command line to run your Mocha tests would therefore be:

$ PULUMI_TEST_MODE=true \
 PULUMI_NODEJS_STACK="my-ws" \
 PULUMI_NODEJS_PROJECT="dev" \
 PULUMI_CONFIG='{ "aws:region": "us-west-2" }' \
 mocha tests.js

Running this will tell us that we have three failing tests, as we had hoped!

 Infrastructure
    #server
1) must have a name tag      2) must not use userData (use an AMI instead)
    #group
3) must not open port 22 (SSH) to the Internet
  0 passing (17ms)
  3 failing
 1) Infrastructure
       #server
 must have a name tag:
Error: Missing a name tag on server
urn:pulumi:my-ws::my-dev::aws:ec2/instance:Instance::web-server-www
 2) Infrastructure
       #server
 must not use userData (use an AMI instead):
Error: Illegal use of userData on server
urn:pulumi:my-ws::my-dev::aws:ec2/instance:Instance::web-server-www
 3) Infrastructure
       #group
 must not open port 22 (SSH) to the Internet:
Error: Illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0) on group

Let’s fix our program to comply:

"use strict";

let aws = require("@pulumi/aws");

let group = new aws.ec2.SecurityGroup("web-secgrp", {
    ingress: [
 { protocol: "tcp", fromPort: 80, toPort: 80, cidrBlocks: ["0.0.0.0/0"] },
    ],
});

let server = new aws.ec2.Instance("web-server-www", {
    tags: { "Name": "web-server-www" },
    instanceType: "t2.micro",
    securityGroups: [ group.name ], // reference the group object above
    ami: "ami-c55673a0" // AMI for us-east-2 (Ohio),
});

exports.group = group;
exports.server = server;
exports.publicIp = server.publicIp;
exports.publicHostName = server.publicDns;

And then rerun our tests:

 Infrastructure
    #server
      ✓ must have a name tag
      ✓ must not use userData (use an AMI instead)
    #group
      ✓ must not open port 22 (SSH) to the Internet
 3 passing (16ms)

They all passed – huzzah! ✓✓✓

Deployment Testing

This style of testing is powerful and lets us do white box testing of the innards of how our infrastructure code works. This style of testing is somewhat limited in what we can check, however. The tests run on an in-memory representation of Pulumi’s deployment plan – before actually performing the deployment – and so cannot test anything the deployment itself. For that, Pulumi has an integration test framework. The two approaches work great together!

Pulumi’s integration test framework is written in Go and is how we test much of our internal code. If the earlier unit test approach took more of a white box testing approach, the integration test framework is very much black box. (There are also options for thorough internals-oriented testing.) This framework has been built to take a directory containing a full Pulumi program and drive various lifecycle operations against it – such as deploying a new stack from scratch, updating it with variations, and tearing it down afterwards, potentially multiple times. We run these in checkin suites, on a recurring basis (such as nightly), and as stress tests.

(We have an open work item which will bring similar integration testing capabilities to each native language SDK. You can use the Go integration test framework no matter the language your Pulumi program is written in, however.)

By running a program through this integration test framework, you can ensure:

Your project’s code is syntactically well formed and runs without errors.
Your stack’s configuration and secrets settings work and are interpreted correctly.
Your project can be successfully deployed to your cloud provider of choice.
Your project can be successfully updated from its starting state to N other states.
Your project can be successfully destroyed and removed from your cloud provider.

As we will see soon, you can also leverage this framework to perform runtime validation.

A Basic Integration Test

To see this in action, we’ll look from the pulumi/examples repo, since our team and community here at Pulumi use that to validate our own examples in pull requests, checkins, and nightlies.

The following is a simplified test of our example that provisions an S3 bucket and some objects in Pulumi:

example_test.go:

package test

import (
    "os"
    "path"
    "testing"

    "github.com/pulumi/pulumi/pkg/testing/integration"
)

func TestExamples(t *testing.T) {
    awsRegion := os.Getenv("AWS_REGION")
    if awsRegion == "" {
 awsRegion = "us-west-1"
    }
    cwd, _ := os.Getwd()
    integration.ProgramTest(t, &integration.ProgramTestOptions{
 Quick: true,
 SkipRefresh: true,
 Dir: path.Join(cwd, "..", "..", "aws-js-s3-folder"),
 Config: map[string]string{
     "aws:region": awsRegion,
 },
    })
}

This test runs through a basic lifecycle of stack creation, updating, and destruction, for the aws-js-s3-folder example. It takes about a minute or so to report success:

$ go test .
PASS
ok ... 43.993s

There are many, many options to control the behavior of these tests. For a full set of options, see the ProgramTestOptions data structure. For instance, you can configure a Jaeger endpoint for tracing (Tracing), tell the harness to expect failure for negative testing (ExpectFailure), apply a series of “edits” to the program for a sequence of update state transitions (EditDirs), and more. Let’s see how to use these to do validate that we deployed what we expected.

Validating the Shape of Resources

All of the above integration ensures that our program “works” – as in, it does not crash. But what if we want to validate properties of the resulting stack? For example, that certain kinds of resources did (or did not) get provisioned, and that they have certain attributes.

The ExtraRuntimeValidation option for ProgramTestOptions allows us to look at the post-deployment state that Pulumi has captured, so that we can do extra validation on it. This includes a full snapshot of the resulting stack’s state, including configuration, exported outputs, all resources and their property values, and all inter-resource dependencies.

To see a basic example of this, let’s check that our program creates a single S3 Bucket:

integration.ProgramTest(t, &integration.ProgramTestOptions{
 // as before...
 ExtraRuntimeValidation: func(t *testing.T, stack integration.RuntimeValidationStackInfo) {
     var foundBuckets int
     for _, res := range stack.Deployment.Resources {
 if res.Type == "aws:s3/bucket:Bucket" {
      foundBuckets++
 }
     }
     assert.Equal(t, 1, foundBuckets, "Expected to find a single AWS S3 Bucket")
 },
})

Now when we run go test, it will not only run through the battery of lifecycle tests but will also, after successfully standing up the stack, perform the extra validation against the resulting state.

Runtime Testing

All of the testing thus far has been exclusively about deployment behavior and the Pulumi resource model. What if you want to test that your provisioned infrastructure is actually working, however? For example, that our VM is alive, our S3 bucket contains what we expect, and so on.

You may have already guessed how to accomplish some of this: the ExtraRuntimeValidation option for ProgramTestOptions is the perfect opportunity to do this. At this point, you are running an arbitrary Go test, with access to the full state of your program’s resources. This state includes information such as VM IP addresses, URLs, and everything you need to actually interact with your resulting cloud applications and infrastructure to perform verification.

For instance, our test program above exports the bucket’s websiteEndpoint property, as the stack output named websiteUrl, which is a full URL that we can use to fetch the bucket’s configured index document. Although we could just as well have dug through the state file to find the bucket and read this property directly, many times our stacks will export useful properties like this that we can use conveniently for verification:

    integration.ProgramTest(t, &integration.ProgramTestOptions{
     // as before ...
 ExtraRuntimeValidation: func(t *testing.T, stack integration.RuntimeValidationStackInfo) {
     url := "http://" + stack.Outputs["websiteUrl"].(string)
     resp, err := http.Get(url)
     if !assert.NoError(t, err) {
 return
     }
     if !assert.Equal(t, 200, resp.StatusCode) {
 return
     }
     defer resp.Body.Close()
     body, err := ioutil.ReadAll(resp.Body)
     if !assert.NoError(t, err) {
 return
     }
     assert.Contains(t, string(body), "Hello, Pulumi!")
 },
})

Like our previous runtime validation checks, this will run in the harness right after the stack is stood up, all in response to a simple go test invocation. This just scratches the surface of what’s possible – anything you can write in code using Go’s test capabilities is within reach!

Continuous Infrastructure Integration

It’s wonderful to be able to easily run these on demand, on a laptop, when you’re iterating on infrastructure changes and want to verify them before submitting for code review. But we and many of our customers run infrastructure testing at various points in the development lifecycle:

In each pull request opened, to validate before allowing it to be merged.
In response to each commit, to double check it was merged correctly.
On a recurring basis, such as nighty or even weekly for extended testing.
As part of more exotic and thorough test programs, such as performance or stress testing, which typically run over a longer period of time, execute many things in parallel, and/or deploy the same program multiple times.

For each of these, Pulumi supports integration with your favorite continuous integration system. This gives you similar continuous integration testing coverage for your infrastructure as you might already enjoy for your application software or in other programming domains.

Pulumi supports your existing CI systems. Here are a few of those supported:

Please refer to the Continuous Delivery documentation for a more comprehensive guide.

Ephemeral Environments

A very powerful capability this unlocks is the ability to spin up ephemeral environments solely for purposes of acceptance testing. Pulumi’s concept of projects and stacks is designed to make it very easy to stand up entirely isolated and independent environments, and to tear them down, all in either a few easy CLI gestures, or by using the integration testing framework.

If you are using GitHub, Pulumi offers a GitHub App that helps to glue together your Pull Request workflow with this sort of acceptance testing run inside of your CI pipelines. Simply install the App into your GitHub repos, and Pulumi in your CI, and your Pull Requests will light up with infrastructure previews, updates, and test results seamlessly:

By leveraging Pulumi for your core acceptance test workflow, you’ll unlock new automation capabilities that improve your team’s productivity and confidence in the quality of changes.

In Summary

In this post, we saw that by using general purpose programming languages, we unlock many software engineering practices we’ve benefited from in our application development. This includes unit testing, integration testing, and even the intersection of both to do extensive runtime testing. It’s easy to run these on demand and harness them in your CI system of choice.

Pulumi is open source, free to use, and works with your favorite languages and clouds – give it a try today!

Programming the Cloud with Python

Sean Gilespie — Thu, 04 Apr 2019 00:00:00 +0000

Across the industry, the popularity of Python is exploding. Amongst our own customers at Pulumi, who automate their infrastructure using Python, we’ve seen the same. Stack Overflow wrote about the astounding growth of Python:

The term “fastest-growing” can be hard to define precisely, but we make the case that Python has a solid claim to being the fastest-growing major programming language.

– David Robinson, Stack Overflow

Since Python is not a new language, what could be driving this incredible adoption curve?

Python is, and always has been, particularly amenable to the automation of previously-manual tasks. So much so, in fact, that books have been written on how to automate common tasks with Python. In recent years, Python’s penchant for task automation have led to its use at the foundation of tools now ubiquitous in the DevOps space: tools like Ansible, SaltStack, and OpenStack are all authored primarily in Python. It would not be a stretch to claim that Python is the language of automation.

At Pulumi we are passionate about automating cloud infrastructure. However, we see a major open opportunity to leverage our most powerful, fundamental tool - the programming language - to automate our infrastructure tasks. We love the idea of “infrastructure of code”, but we struggle with the industry’s definition of “code”; it’s hardly code at all, often consisting of YAML documents, DSLs, or other, domain-specific solutions to the task at hand. Since Python is such an incredible tool for task automation and since its use is rapidly pervading the industry, wouldn’t it make sense to use code, to use Python, to automate tasks as critical as the provisioning of our cloud infrastructure and delivery of our applications?

Delivery is not a detail, it is our job. Now is the time to apply our core skills to our own work. Now is the time to engineer our delivery. We divide our work between ourselves and computers: humans for decisions, and automation for tasks.

– The Software Defined Delivery Manifesto

That sounds great, but what is this “Pulumi” thing?

Pulumi is an infrastructure as code tool for provisioning cloud infrastructure and deploying applications. The key difference between Pulumi and other tools in this space (such as Ansible, Terraform, and AWS CloudFormation) is that Pulumi aggressively reclaims the word “code”: users of Pulumi write programs in general purpose programming languages that, when run, deploy and run applications in cloud environments, while still retaining all the known benefits of infrastructure as code that our industry has grown accustomed to (repeatability, resilience to failure, history and auditing, and so on).

Pulumi is a rebellion against YAML and domain-specific non-Turing-complete languages. We firmly believe that your infrastructure isn’t just code: your infrastructure is software. By applying the same engineering discipline to your infrastructure code that you apply to your application code, we can iterate quickly and with confidence, leaning on things such as automated tests, code analysis tools, and a common language. Pulumi does not generate YAML; it instead brings the full API surface area of your cloud environment into Python, powered by an infrastructure as code engine.

It is hard to overstate how liberating it is to write infrastructure as code with a full general-purpose programming language at your disposal. Below is a code snippet of a Pulumi program, written in Python, that deploys a static website to S3:

import mimetypes
import os

from pulumi import export, FileAsset
from pulumi_aws import s3

web_bucket = s3.Bucket('s3-website-bucket', website={
    "index_document": "index.html"
})

content_dir = "www"
for file in os.listdir(content_dir):
    filepath = os.path.join(content_dir, file)
    mime_type, _ = mimetypes.guess_type(filepath)
    obj = s3.BucketObject(file,
        bucket=web_bucket.id,
        source=FileAsset(filepath),
        content_type=mime_type)

There are only 18 lines of Python code here, but this program does a ton!

First, we create an AWS S3 Bucket that we’ll use to host our website from, setting the index document to be “index.html”.
Next, we iterate over all files in the “www” directory (using the standard Python for loop!
Finally, for every file in the “www” directory, we guess its MIME type using the Python standard library and then create an S3 BucketObject inside the website bucket, which uploads the file to S3.

This is a simple example, but it’s hard to overstate how awesome it is to be able to reach into Python’s deep standard library whenever we need to. In this example we’re using both the “os” and “mimetypes” packages in the Python standard library: the first to list the files in a particular directory and the second to guess a particular file’s MIME type based on its extension and contents, which we eventually pass directly to S3.

Remember, this code isn’t performing imperative commands — like, say, your cloud provider’s Python SDK or Boto would — and instead creates a declarative resource graph that the Pulumi engine understands how to act upon. This works for the first deployment, but more importantly, for all subsequent incremental updates too. If we were to change the contents of one of the files on disk, Pulumi would recognize that one of the files has changed and re-upload it, without changing the bucket or any of the other already existing bucket objects!

Where can I learn more?

Don’t take our word for it, though! We’d love for you to check it out and see for yourself how great it is to reclaim the “code” in “infrastructure as code”.

If you want to know more about Pulumi and the things it can do for you, check out the Getting Started page and Documentation for more information.

If you’d like to try the above example out for yourself, we’ve put the full code on GitHub. There are also dozens of other Python examples across AWS, Azure, GCP, Kubernetes and more you can explore.

Python is the language of automation today and for the future. Pulumi is the infrastructure as code automation tool of the future. Using them both together is an incredible way to automate a crucial job while staying entirely within a language you already know and love!

Pulumi Blog: Python

Deploy a Private Hermes Agent on Render Securely with Pulumi, Modal, and Tailscale

What is Hermes?

Prerequisites

Understanding the Hermes architecture

Setting up ESC for secrets management

Generating the Render SDK

Securing with Tailscale

Deploying the agent

The complete program

Cost

Running the deployment

Access the agent

Verify the deployment (optional)

Security considerations

What’s next?

Conclusion

Use Your Mac for AI Agents: Self-Host Gemma 4 12 B with Pulumi and Tailscale

Prerequisites

Run Gemma 4 with host-native llama.cpp

Verify the LLM API

Keep llama.cpp running after reboot

Deploy Open WebUI with Pulumi and k3d

Access Open WebUI through Tailscale

Use the model with Pi

Advanced: Linux GPU in-cluster serving

Cleanup

How We Load Data into Snowflake in Seconds with Pulumi

What we’ll cover

Architecture overview

Project setup

Managing credentials with Pulumi ESC

Shared infrastructure

Building the direct ingestion ComponentResource

The Lambda handler

Why direct Firehose to Snowflake?

The DirectSnowflakeIngestion component

Wiring it together

Testing the pipeline

Other loading patterns

Publishing as reusable components

Git-based sharing

Pulumi Cloud Private Registry

Conclusion

Host your Python app for $1.12 a month

How does the pricing break down? (2026)

Cheap Python hosting compared (2026)

Why use AWS Lambda for low-traffic Python apps?

Why containers?

How do I build a Flask app for Lambda?

How do I containerize a Flask app for Lambda?

Do I need to learn Lambda-specific patterns?

How do I deploy a Lambda container with Pulumi?

Why is this approach worth it?

Alternative approaches

Conclusion

Related reading

Changelog

Top 15 Python Tools for DevOps

Dashboards / monitoring

Task scheduling and orchestration

Network analysis and security

Containerization and cloud interaction

Building command-line tools

2. The essential toolkit

1. Pulumi

Bringing it all together

The art of getting things done

Pulumi + uv: Fast Python Package and Project Management

Why uv?

Using uv in Pulumi

Pulumi + Python: Bringing the Best of Modern Python to IaC

Pulumi + Python

New Features for Pulumi Python Users

Native Support for Poetry

Built-in type checking

Pythonic input types

Summary

Next-level IaC: Bridging the Declarative Gap

Pulumi’s architecture and flow of operations

Realizing values using the `apply` function

Example: Using `apply` to Base64 encode an Output