Pulumi Blog: Adam Gordon Bell

Kubernetes Agent Sandbox: What It Is and How to Deploy It with Pulumi

Adam Gordon Bell — Tue, 21 Jul 2026 00:00:00 +0000

When you use a coding agent, it can seem like there’s a trade-off between autonomy and permissions. If you approve every command, it’s safe but slow. Let it do whatever it likes and it works more autonomously, but as the nx supply-chain attack showed, that can go badly.

The fix is to give the agent a sandbox: a box it’s allowed to wreck, with limited permissions and scoped network access. The only files are the checkout you handed it, the only credentials are the task’s own, and trashing the machine just means a disposable pod gets garbage-collected early. Pulumi Neo works this way, and if you want to scale that pattern up inside your own organization, the Kubernetes project Agent Sandbox is a great path to building your own. This post is what it is and how to deploy it on GKE with Pulumi.

What is Agent Sandbox?

Agent Sandbox is a Kubernetes SIGs project that gives AI agents isolated, disposable environments as Kubernetes resources: a Sandbox custom resource, backed by gVisor or Kata Containers for kernel-level isolation.

You could build Agent Sandbox yourself. You’d need gVisor support, a userspace kernel that sits between the agent’s code and your host. With that in place, you could approximate a sandbox for every agent by stringing together a StatefulSet of size one, a headless Service, and a PersistentVolumeClaim, plus some lifecycle machinery to keep warm pools of nodes around. Agent Sandbox wraps all of that up as a CRD, so you can run a Kubernetes cluster where each sandbox is a disposable, kernel-isolated environment a coding agent runs in.

apiVersion: agents.x-k8s.io/v1beta1
kind: Sandbox
metadata:
 name: demo-sandbox
spec:
 podTemplate:
 spec:
 runtimeClassName: gvisor
 containers:
 - name: agent
 image: ubuntu:24.04  # swap in your coding-agent image
 command: ["sleep", "infinity"]

The Sandbox CRD, the whole idea in one manifest.

There are two common patterns for using it. In the first, every coding agent session in the organization maps to its own pod with a persistent volume, basically Claude Code running across a whole fleet of pods, each one an individual user’s session.

In the second, you’re building your own Neo or Devin: a productized agent harness that needs to spin up execution environments. The harness lives outside the sandboxes and hands each task a disposable box, and Agent Sandbox is what spins those boxes up and down, suspending and resuming them.

Why not just run agents in Docker?

So why do we need gVisor or Kata Containers at all? The agent already runs inside a container. Isn’t that the box?

Not really. A container isn’t much of a security barrier. Every container on a host shares one kernel, and the Linux kernel exposes a huge surface area, 450+ syscalls, to every one of them. CVE-2019-5736 is the canonical example: a malicious container tricks runc into overwriting the host’s own runc binary, and after that every docker run on the host runs attacker code as root. Put a prompt-injectable AI agent in that container, and the risk is obvious.

gVisor, the runtime underneath Agent Sandbox’s default path, puts a userspace kernel written in Go between your agent and the host, which limits the possible security surface area. A kernel bug the agent can reach becomes, mostly, a crash in a userspace process rather than a root shell on your node. That is what makes an Agent Sandbox secure.

What is GKE Sandbox?

GKE Sandbox is GKE’s gVisor feature: a RuntimeClass named gvisor that runs a pod under the userspace kernel described above. It predates Agent Sandbox by years and is a per-pod isolation primitive, not an agent tool. Agent Sandbox is the lifecycle layer that sits on top of it: the CRD’s runtime selection points down at gVisor (or Kata) for the actual kernel isolation. They compose. On GKE, Agent Sandbox with runtimeClassName: gvisor is literally using GKE Sandbox underneath.

Google engineers in Kubernetes SIG Apps maintain Agent Sandbox, Janet Kuo and Justin Santa Barbara (of kOps) among them, and it launched at KubeCon NA in November 2025. It’s also one layer of a broader cloud-native agent stack taking shape: agent-sandbox for isolation, kagenti (IBM) for identity, kagent (Solo.io, a CNCF Sandbox project) for agent logic, agent-substrate for density. As Solo.io’s Lin Sun puts it, “Sandboxing your agents is necessary, but not sufficient.”

The rung above gVisor: hardware microVMs

gVisor filters syscalls in a userspace kernel. The heavier option gives the workload its own guest kernel behind hardware virtualization (KVM), so an escape has to cross a CPU-enforced boundary. Firecracker, the VMM AWS built for Lambda (and the isolation layer under Bedrock AgentCore), keeps that boundary cheap: roughly 50,000 lines of Rust against QEMU’s ~1.4 million lines of C. Google’s kvmCTF pays $250,000 for a KVM escape, which is how the market rates that boundary. Agent Sandbox reaches this rung through Kata Containers via the same runtime selection, and the tradeoff isn’t linear: Kata can beat gVisor on I/O-heavy work, because its guest runs a real kernel servicing syscalls natively.

The one-second problem

Isolation is the hard part, but it isn’t the only one. If you’re in scenario 2, using Agent Sandbox pods as the backing instances for your own agent harness, startup time is a challenge too. People abandon chat sessions, and when they come back they expect the agent to pick up quickly, not wait on a machine to boot.

Booting a fresh Kubernetes pod costs about a second of overhead. That’s nothing for a rolling deployment, but enough that the maintainers say it “breaks the continuity” of an interaction. So Agent Sandbox avoids paying that cost twice: suspend a sandbox and its pod goes away while its workspace and identity persist, and warm pools keep pre-booted pods ready for new sandboxes to adopt. The managed GKE version goes further and restores from memory snapshots, the same snapshot-restore move E2B (~150ms) and Fly’s Sprites (~300ms) use to pull the effective cold start well under a second¹.

Deploying it on GKE with Pulumi

If you already have a cluster and want to try Agent Sandbox, the kubectl apply from the project’s quickstart will probably get you going. But productionized use takes a bit more. Let’s walk through setting up a cluster with a sane access policy, named individuals, and a dedicated environment per user. The full program is at pulumi/examples/gcp-ts-agent-sandbox.

Move 1: the cluster. First we stand up a GKE cluster with a gVisor node pool. The one line that matters is the sandboxConfig, with sandboxType: "gvisor". As far as I’m aware, this can only be done on GKE:

const gvisorPool = new gcp.container.NodePool("gvisor-pool", {
 cluster: cluster.name,
 nodeConfig: {
 machineType: "n2-standard-4",
 imageType: "COS_CONTAINERD", // gVisor requires COS + containerd
 sandboxConfig: { sandboxType: "gvisor" },
 },
});

GKE installs the gvisor RuntimeClass, labels the nodes, and taints them so only pods that opt in land there.

That auto-install is the GKE convenience, not a requirement. Agent Sandbox itself is just CRDs and a controller, so on any other cluster you can install gVisor yourself (or Kata), register a RuntimeClass, and everything below works the same.

Move 2: the install, pinned. Next we install the controller and the CRDs, using the all-in-one sandbox-with-extensions.yaml manifest that shipped in last week’s v0.5.2²:

const base = `https://github.com/kubernetes-sigs/agent-sandbox/releases/download/v0.5.2`;

const agentSandbox = new k8s.yaml.v2.ConfigGroup("agent-sandbox", {
 files: [`${base}/sandbox-with-extensions.yaml`],
});

Pinning matters here: releases land every week or two, and the API group already graduated once (v1alpha to v1beta1).

Move 3: sandboxes are a loop, not a manifest. This is where Pulumi really shows its advantages. In practice you don’t create sandboxes by hand, because they’re per-user or per-task. So the demo reads a list and maps over it:

const developers = config.requireObject<Developer[]>("developers");

// Who may open which box, derived from that same list.
const acl = sandboxAcl(developers);

export const sandboxes = developers.map(dev => new AgentSandbox(`sbx-${dev.name}`, {
 owner: dev.name,
 credentials, // this owner's Claude Code creds, per box, not per cluster
 prompt: BOOT_PROMPT,
 // ...
}, { dependsOn: [agentSandbox, operator, acl] }));

That developers array is read at runtime, so it could just as easily be a GitHub team or whoever currently has a session open, none of which kubectl apply -k can loop over. (The developers list and the per-box credentials are stack config; the example’s README has the exact pulumi config set commands.)

Nothing in the sandbox is Claude-specific, either. The agent is just what the image installs: swap in Codex CLI, or point one of the Claude Code orchestration frameworks at it, and the isolation story doesn’t change.

Move 4: the egress policy. Create a Sandbox directly, as we do here, and no NetworkPolicy applies³: kernel isolation, but wide-open egress. (The SandboxTemplate extension manages a default policy for template-created sandboxes; a raw Sandbox gets nothing.) We set a policy that lets the agent reach api.anthropic.com and npm but not any private IPs:

egress: [{
 to: [{ ipBlock: {
 cidr: "0.0.0.0/0",
 except: ["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "169.254.169.254/32"],
 }}],
}, /* + a DNS rule to kube-system:53 */]

We also exclude 169.254.169.254 to keep the agent off the node metadata server.

Move 5: private access with Tailscale. The sandbox runs an IDE (code-server) on port 13337. Instead of a public load balancer with a password, we put each sandbox on a tailnet with the Tailscale Kubernetes operator. Three annotations on the Service do it:

"tailscale.com/expose": "true",
"tailscale.com/hostname": name, // sbx-adam
"tailscale.com/tags": `tag:${name}`, // tag:sbx-adam

Move 6: lock down each sandbox. Finally, we generate the tailnet access policy from the same developer list that creates the boxes, so each person can reach only their own box:

acls: developers.map(d => ({
 action: "accept",
 src: [d.email], // this person
 dst: [`tag:sbx-${d.name}:13337`], // this box, this port. Nothing else.
})),

Because the boxes and the grants come from one array, a developer’s sandbox and their permission to open it can’t drift apart.

Move 7: run it. With all of that declared, pulumi up builds the whole environment in one command: cluster, gVisor pool, controller, per-developer sandboxes, egress policy, and private access. Get the stack’s URL and open it:

$ pulumi stack output sandboxUrls
["http://sbx-adam:13337"]

Open it on any device signed into your tailnet and you’re in a full VS Code (code-server), running inside the sandbox pod, on the gVisor node pool, behind the egress policy set above.

Each box also boots with a task. The prompt in the demo asks the agent to figure out where it is: read /proc/version, decide whether it’s in a container or a VM, and write up the evidence in a REPORT.md. An agent doing forensics on its own jail is a decent smoke test that the isolation is real.

Here’s the finished result:

Recorded a little demo: Here's how every developer gets their own kernel-isolated dev box on Kubernetes: VS Code + Claude Code in a pod behind Tailscale with gVisor underneath for security isolation. write up coming
— Adam Gordon Bell (@adamgordonbell) July 20, 2026

Wrapping up

Agent Sandbox makes a kernel-isolated, disposable agent environment a first-class Kubernetes object, and Pulumi is how you stand one up as one program instead of a runbook.

The full program, everything in this post, deploy to teardown, is at pulumi/examples/gcp-ts-agent-sandbox. Clone it, point it at a GCP project, and you have your own dedicated agent cluster in about twelve minutes. If you’re new to Pulumi, get started here.

github.com/pulumi/examples

Warm pools plus snapshot restore are how the managed version keeps this fast at scale: Google’s GKE Agent Sandbox launch cites 300 sandboxes per second at sub-second latency. See Bringing you Agent Sandbox on GKE and Agent Substrate. ↩︎
The all-in-one manifest is new in v0.5.2 (July 17): earlier releases split the install into manifest.yaml plus extensions.yaml, and v0.5.2 renamed the core file — one more reason to pin the version you deploy. ↩︎
Verified against the v0.5.2 release manifests: sandbox-with-extensions.yaml installs no NetworkPolicy objects, and networkPolicyManagement defaults to Managed only for SandboxTemplate-created sandboxes. A directly-created Sandbox gets neither, so restricting egress is left to you. See the v0.5.2 release. ↩︎

Six Live Kubernetes Recommendations: AKS, Cilium, Rate Limiting, and More

Adam Gordon Bell — Mon, 15 Jun 2026 12:00:00 +0000

On June 10th, Engin and I ran a live workshop building an AKS cluster, an Azure Container Registry, and a random-cat web app from scratch in C#. This is the writeup, including the parts we didn’t get to live.

Live demos keep you honest. On June 10th my AKS workshop went a little sideways. Partway through, Docker Hub rate-limited my image pull and we had to adapt the content on the fly. The original plan was to stand up an AKS cluster with Cilium, an Azure Container Registry with the cluster’s pull permission wired in code, and a random-cat web app, then split the infrastructure from the workload into separate Pulumi stacks. Live, we didn’t make it through all of that, but we had some fun tangents and it turned out to be a great session. So here are my six recommendations for working with Kubernetes on Azure from this recent workshop.

1. Pick the language your team already uses

I opened the session by asking the room what language they prefer to work in, with one caveat from me: “Hopefully, it’s not PowerShell because this is not in PowerShell.” We got some C# answers and zero requests for PowerShell, which works out, because the whole workshop is in ~~PowerShell~~ C#.

I chose C# because, first, I genuinely love the language. I spent the early part of my career as a C# developer, and on Azure it just makes sense. Second, an Azure workshop tends to draw a big turnout of C# devs, and if they aren’t C# devs themselves, they often work somewhere the backend team uses C#. Pulumi lets you bring that language to your infrastructure rather than learn a new DSL.

AI coding assistants also came up. LLMs have substantially more C# and Go in their training data than HCL or Bicep. As I put it live: “If you’re working in HCL, well, there’s not that much out there in the training set, but there’s lots of C#. There’s lots of Go.” I co-authored parts of the workshop code with Claude.

Pick the language your team already writes.

Pulumi runs on the language your backend engineers know, and that language has more AI assistant training data behind it too.

Aside: everyone’s path ran through Docker Swarm

We also had some fun historical conversations about orchestrators. I asked Engin: “Did you use anything before Kubernetes became popular? Like Mesosphere or any of those other interesting solutions of the heyday?”

Engin had walked the same road most of us did.

Engin: “We used the classical Ansible powered provisioning of VMs… And then we switched to Consul to have a little bit more control… And then we had a short intermezzo with Docker Swarm… So Docker Swarm with Portainer was really goated. And Kubernetes then took over.”

My own history rhymed with his.

Adam: “I worked at a place where we used a bunch of shell scripts to have like JVM services, like a red green deploy… And then it was Docker Swarm for a little bit. And then it was Kubernetes.”

No single tool won. Teams picked what fit their existing skills and shifted gradually as the ecosystem matured.

2. Use Cilium on AKS if you can

By default, Kubernetes on AKS doesn’t turn on Cilium. My strong recommendation is to turn it on. The option is called Azure CNI Powered by Cilium, and it takes one config block:

NetworkProfile = new ACI.ContainerServiceNetworkProfileArgs
{
 NetworkDataplane = "cilium",
 NetworkPlugin = "azure",
 NetworkPluginMode = "overlay",
 NetworkPolicy = "cilium",
 PodCidr = "192.168.0.0/16",
},

This buys you eBPF networking instead of kube-proxy and iptables. eBPF lets you run sandboxed programs inside the Linux kernel itself, so packet handling and NetworkPolicy enforcement happen down in the kernel instead of up in userspace. You also get Hubble flow visibility built in, the same technology as GKE’s Dataplane V2. As I said live, “this is very easy to set up on Azure. You just network data plane, set it to Cilium. Network policy, set it to Cilium.”

Self-hosted, Cilium is helm install cilium with a dozen flags and a lifecycle you own forever. On AKS, it’s these five lines and Azure runs it. On a getting-started cluster, I’ll take the managed version every time. Owning that lifecycle yourself buys you nothing here.

Full code: 01-cluster/Program.cs

On managed Kubernetes, complexity you’d self-host becomes a field you set.

AKS gives you the managed version of eBPF networking. Pulumi makes the configuration code instead of portal clicks.

3. Debugging Kubernetes is easier when your tools have the full context

The rate limit wasn’t even the first failure of the hour. That honor went to Azure, which refused to provision the cluster on the first pulumi up. I was running in Canada Central, and Pulumi Neo read the error and diagnosed it:

Neo: “Capacity constraints in the selected region. We might need to change our location.”

Adam: “Maybe this Canadian Azure instance is not all it’s cracked up to be.”

Engin: “East US is always a good bet.”

We switched the region to East US, and the cluster came up. That is the kind of diagnosis a stack-aware assistant is good at, reading the actual provisioning error instead of leaving me to guess.

Then the image step hit. The workshop’s original flow pulled the cat app image into ACR with az acr import --source docker.io/agbell/my-random-cat, and Docker Hub rate-limited me, even though I was the only person in the room running the code. So how do you blow through an anonymous pull limit all by yourself? The trick is that az acr import runs server-side. Azure does the pull from Docker Hub, not my laptop, and it goes out over an egress IP shared by who-knows-how-many other Azure tenants. Docker Hub counts anonymous pulls per IP address, so all of that shared traffic lands in one bucket. It flagged me at the worst possible moment: “I just got flagged by Docker Hub. That is awesome. That will be a challenge for this demo.”

Nothing focuses a workshop like watching your own demo get rate-limited in front of a live audience.

Live, I asked Neo to route around it: “we’re using Neo. And Neo is going to switch us out to a nice Hello World project.” Neo could see the whole stack, down to the failing image reference, and it swapped the workload over to a standard hello-world image hosted on Azure, so the deploy no longer depended on Docker Hub. A few minutes later: “we should have a Hello World container running here. It’s way less fun than my image… just a bunch of random cats. But here we can see that this is working.” I won’t oversell it. The cat was gone, and we didn’t get through every planned stage. But the cluster was up with a workload deployed, and the session kept moving.

That’s the case for a stack-aware assistant over a chatbot. A general chatbot can tell you that Docker Hub has rate limits and that you should host images closer to home. Neo was working with the actual program state, so the workaround it proposed was already wired to my resources. Engin put it this way: “for me, Pulumi Neo, the subcommand is the new Pulumi app.” When something breaks mid-demo, the tool that can read your stack is the one that helps.

The long-term fix was switching to az acr build, building the image in ACR from source, so the app image never has to come from Docker Hub at all. Every stage in the repo now builds the cat image this way, from source shipped in each stage’s app/ folder:

az acr build --registry <your-acr> --image my-random-cat:latest app

No local Docker, and the app image never touches Docker Hub. The only Docker Hub dependency left is the base image (python:3.9-slim), pulled server-side by ACR’s build instead of by an import you re-run every time.

Full code: 02-app/Program.cs

An assistant that can read your stack beats one that can’t.

A general chatbot knows Docker Hub has rate limits. Neo could see the deployment and the failing image reference, and that context is what turned advice into a working fix mid-session.

4. You don’t have to abandon Terraform to start

One attendee, Martin, had been using Terraform for eight-plus years. His question, put directly: “I’m using TF for eight plus years. Why should I consider Pulumi? Where is the strength?… It seems unnecessary to TF for me. At least from what I’m seeing now.”

Fair question, and the honest answer is you don’t have to abandon your modules. Pulumi has first-class Terraform module support, so you can reference existing modules from a Pulumi program. Python ML teams can import infrastructure modules, and platform teams can wrap them in typed abstractions. The portfolio doesn’t change. The programs that use it just get more capable.

The sharper reason is about edges. HCL and Bicep are purpose-built DSLs, and when you hit what they can’t express, there’s no escape hatch inside the language. With a general-purpose language you can always drop to the SDK or shell out. The edges still exist. They’re just further out, and that’s the whole difference once a project gets past the basics.

You don’t have to rip and replace.

Pulumi can reference existing Terraform modules. Move what makes sense now. The DSL/language tradeoff shows up at the edges, not the basics.

5. Split slow infrastructure from fast workloads

Once we had the cluster up and the app running, the single program had grown: cluster, ACR, AcrPull, image build, deployment, service. Every new workload meant editing the file that also declares the cluster. We had planned this turn in the session in advance. I posed the problem and Engin had the recommendation ready:

Adam: “What do you think the problem is with my code here, Engin? … The problem is like every time I want to add a new service to my Kubernetes cluster, I have to go into this file that declares the cluster and all its infrastructure. I think I would prefer to split it.”

Engin: “Create other stacks which are then referencing each other to get the values out. Actually, I highly recommend.”

Adam: “Yes, I totally agree with you, Engin. We should split these out into two separate stacks.”

So that’s what we did. The infrastructure stack exports the kubeconfig and registry URL. The workload stack references those outputs via StackReference. The slow-moving infra and the fast-moving app become independent deployment units. The infra team and the app team can ship on their own cadences, without fighting over one file.

// workload/Program.cs — pulls cluster outputs, never touches cluster resources
var clusterStack = new StackReference(cfg.Require("clusterStack"));
var kubeconfig = clusterStack.GetOutput("kubeconfig");
var acrLoginServer = clusterStack.GetOutput("acrLoginServer");

The tricky part is one we didn’t cover live, which is how you split a running workload without taking down the running services. The real answer is migrating the stack state, moving resources between stacks instead of destroying and recreating them. As I noted in the session: “now we are just removing the service from our project. If this was actual production stuff, I don’t think I’d be just shutting down the service… You can actually just make changes to the state rather than tearing them down.”

For the workshop, a simpler move kept the Kubernetes cluster up and the demo moving: keep the project name. Pulumi identifies stacks as org/project/stack, so if you move the cluster code into aks-cluster/ while keeping the project name kube-kitties, Pulumi still sees org/kube-kitties/dev. Same stack, nothing recreated. Rename the project and you get a second cluster.

In stage four of the demo, I showed how to extend this pattern to work with existing YAML manifests: ConfigGroup drives raw Kubernetes YAML through the Pulumi provider, giving you previews and dependency ordering without rewriting files. One transformation swaps the Docker Hub image reference for the ACR copy, so manifests stay portable and pods pull from your private registry.

Code: 03-split/ and 04-split-yaml/

Keep the project name when you split, and the cluster doesn’t move.

Pulumi identifies stacks as org/project/stack. Same project name means the same stack. Rename it and you spin up a new cluster.

Run Kubernetes on AKS with code

Provision your AKS cluster, container registry, and workloads in C#, then split infrastructure from apps into separate stacks so each ships on its own cadence.

Get started

6. GitOps is the way

The final stage of the workshop is left as homework, and it’s a great one to pick up. GitOps is the standard way to run Kubernetes at this point, the One True Way™ depending on who you ask. My position from the call: “I think I’m on team GitOps.”

The take-home is the 05-gitops folder. We didn’t demo it live, but the code is real and verified: Pulumi stands up the cluster, installs Argo CD, and registers the cat as an Argo Application. From there, changes are git push.

To see Kubernetes and GitOps worked through end to end, watch the video above. Engin also recently ran a GCP workshop with great examples of GitOps with Pulumi, using Flux.

Let Pulumi stand up the cluster, then let GitOps run the apps.

Pulumi installs Argo CD or Flux as part of the infrastructure. From there, workload changes are a git push. Stage 5 of the workshop repo is a complete Argo CD example.

Try it yourself

That’s the whole workshop, six recommendations and two live failures that ended with a cat app running on AKS. To run it yourself, clone the repo. There’s a DEMO.md with every step of this session, and the stage folders mirror the arc above. 01-cluster is the infrastructure checkpoint, 02-app adds the cat app built into your own registry, 03-split and 04-split-yaml separate infrastructure from workload, and 05-gitops is the homework.

Pick a stage and bring it up:

git clone https://github.com/pulumi/workshops
cd workshops/az-getting-started-aks/02-app
pulumi up
curl http://$(pulumi stack output catServiceIp)/

Expect 5 to 8 minutes waiting on cluster create. That’s just how long AKS takes. And since each folder is a complete, runnable checkpoint with its own stack, run pulumi destroy before moving to the next stage, or you’ll have several clusters billing at once.

You’ll need an Azure subscription, the Pulumi CLI, and the .NET SDK. Basic Kubernetes familiarity helps. For more on the building blocks, see the Azure Native provider and the Kubernetes provider in the Pulumi Registry.

The full workshop code lives at github.com/pulumi/workshops under az-getting-started-aks. Check out our upcoming sessions on the workshop page, or subscribe to the newsletter to hear what’s coming up.

If there’s a seventh recommendation hiding in here, it’s the one I learned the hard way: never pull from Docker Hub live again. I’ll be keeping my images on a paid cloud registry from now on.

Engin and I will be doing more of these, and we’d love to see you at the next one.

Parts of the workshop code were authored by Claude and Pulumi Neo, and parts of this writeup got help from both as well.

Seven Rules for Building an AI-Native Software Factory

Ewan Dawson — Thu, 21 May 2026 06:00:00 -0700

Ewan Dawson is CTO of Compostable AI, where five engineers run an AI-native software factory: nineteen clients, custom AWS deployments, most of them shipped within a day of contract signing. This article is adapted from his recent Pulumi webinar, and covers rules in more depth than we had time for on stage.

For the past twenty years, I’ve viewed software development as a craft. The best engineers drew on decades of experience to get every function right.

But two years into the agentic AI revolution, I realised software is going to look more like a factory than a craft. The economics have changed. We can’t treat code as bespoke anymore. To scale, we have to think industrial — use the tools to ship more value with fewer engineers.

I joined Compostable AI soon after it was founded 2.5 years ago, and I built the engineering org AI-native from day one. The technology has come a long way since then, and so has my understanding of what AI-native actually means. Here are seven rules I keep coming back to.

An AI software factory is a software operation where autonomous agents write and ship most of the code. The engineers stop writing it by hand and spend their time deciding what gets built and talking to customers. The rules below are our rules for building and running an AI-native software factory.

1. Transform, don’t enhance

Going AI-native isn’t an upgrade to your existing process. If you treat AI as a way to hand your developers smarter tools, you leave most of the value on the table. You get the leverage by rebuilding how you write software — and the culture and processes around it.

I know that’s a tall order for a large, mature engineering org. My advice: start small. Pick one team or one business area and run it as a fully AI-native function. Take what you learn and roll it out from there. And do the political work early, especially with your Governance, Risk, and Compliance function. Get GRC on your side early. Otherwise AI becomes a compliance fight instead of a structural advantage.

Don’t bolt AI onto your existing workflow. Redesign the workflow around what agents can do.

Most of the leverage in this technology comes from rebuilding around it. The tool change is the small part.

2. Remove the problem, don’t solve it

Going AI-native flips which problems are hard and which are easy. The right move often isn’t to engineer a solution. It’s to reframe the problem so it goes away.

Here’s an example. Serving multiple clients with agents writing the code, blast radius wasn’t a hypothetical. One bad agent run could trash a customer’s database, or leak one client’s data into another’s. Our instinct was to build a secure multi-tenant sandbox with guardrails, approvals, rollback. But every version we tried still had agents loose in a shared environment, one bug away from making one customer’s data visible to another’s. So we removed the problem: every client gets two dedicated AWS accounts, one for production and one “digital twin” staging account. Agents iterate on staging until the work checks out. Only then does it ship to production. We have nineteen accounts now, one per client.

Managing nineteen AWS accounts with five engineers used to be an administrative nightmare. When code is cheap, infrastructure-as-code tools like AWS Control Tower and Pulumi make it the easier path.

Remove the problem before you try to solve it.

It’s cheaper to reframe the problem than to engineer your way through it.

3. Pick tools your agents can drive

Removing problems is the process side. The other side is tooling. If you want an automated factory, your tech stack has to be something agents can drive. This overlaps a lot with tools that have great developer experience. If a tool has a robust API plus a clean CLI, agents can drive it. If it’s heavy click-ops around a web UI, agents stop there.

We didn’t get there first try. Our first IaC tool worked fine when we had a couple of clients. As we added more, accounts drifted, deployments slowed, retries got complicated. We needed something built for where we were heading.

I went looking, and Pulumi fit. We express infrastructure as type-safe code — TypeScript, in our case, rather than HCL — and agents are good at writing it. Pair that with Pulumi Neo — pre-loaded with domain-specific Pulumi skills — and we ship infrastructure that follows best practices. One of my colleagues put it: “The scary thing about Neo is it just seems to know everything about what we do.” Pulumi IaC plus Pulumi ESC for configuration beats stitching tools together. And TypeScript lets us build higher-level abstractions that keep the AWS account fleet tractable.

“I don’t actually care if it’s HCL or TypeScript, as long as my software development agents can write it. And they do a better job with TypeScript than HCL.”

Tools have to share your AI-native mindset. If they don’t integrate deeply, the human becomes the glue.

If part of your stack still requires a human to click through a web UI to provision an account, your agents stop there.

4. Don’t let one agent do everything

When I first started with agents, I reached for a god prompt: one massive system prompt meant to guide a single agent through the whole software lifecycle. It didn’t work. Agents struggle when you give them multiple goals. The writer is lenient on its own work — it won’t catch what it just shipped. You don’t want it reviewing the code, checking for security flaws, or hunting bugs.

We get better results from a constellation of specialized agents, each handling one part of the line. Pulumi Neo handles infrastructure. Alongside it sit agents specialized in:

Code implementation
Code review and testing
Security auditing
Internal standards compliance
Documentation updates

Tasks pass down the line. Clean code comes out the other end, with almost no human involved.

Don’t let any agent mark its own homework. Specialize by job.

Treat agents the way you’d treat a team. The one who writes the code shouldn’t be the one signing it off.

5. Measure human hours per unit of value

Once we had agents writing and agents reviewing, throughput went up — but the bottleneck moved past the PR. Engineering hours were still the most expensive thing in the building, so my core metric is human hours per unit of value produced. Minimize that.

That means hunting for every step that still goes through a person — especially the mid-pipeline steps between ideation and production. Automate the human touchpoints along that line, and the factory runs 24/7.

Pushing automation this hard also forces good engineering. A chaotic, undocumented process is impossible to automate. Good engineering is still good engineering, AI or not. Agents won’t fix a weak process.

Measure human hours per unit of value. Treat every one as a bottleneck to remove.

You can’t automate what you can’t describe. Every human in the pipeline marks a piece that hasn’t been described yet.

6. Design for convergence, not one-shot correctness

Even with the human touchpoints removed, the agents don’t ship right the first try. Once you embrace the factory pipeline, you stop needing them to. We design for convergence instead — a system that lands on the right answer through automated iteration.

The loop we run looks like this:

Refinement: agents iterate on the Product Requirements Document until the problem is clear.
Planning: agents draft multiple technical approaches, and evaluation agents pick the best one.
Implementation: coding agents write the software.
Review: specialized checking agents look for bugs, API misuse, and security flaws.

If the checkers find a problem, they hand it back to the implementation agent. The loop repeats until the tests pass and the agents agree on a clean PR. Once it converges, we merge and deploy to staging.

Two things have to be true. You need a way to evaluate the output. Without that, you don’t know when to stop. And the loop has to converge — each pass has to get closer. A checker that fails every PR for a different reason isn’t helping — it just keeps the work going in circles. The feedback has to narrow the search, not widen it.

Once it converges, the question moves on. How cheap can we make it? Lower the time to PR, reduce token count, drop the overall cost. The optimization never really ends.

Don’t aim for one-shot correctness. Design for convergence.

It doesn’t matter how many tries it takes, as long as the loop closes without a human in it. Get convergence first. The optimization comes after.

7. Run the factory in the cloud, not on a laptop

Even a converged factory has to live somewhere. Try running a fully automated factory on individual developers’ laptops, and it falls apart. Laptops are highly trusted machines. Put autonomous agents on them and your security posture drops, fast. And the factory has to run 24/7. Events come from elsewhere — PR comments, Slack threads, errors in test environments.

Cloud also kills configuration drift across a dozen developer machines. The same prompts run against different model versions, and env vars sit half-set on half the laptops. The thing you’re trying to optimize lives in different states across the team. Cloud isn’t just where the factory runs; it’s the only place a team can iterate on it together. Keep everything in one place — AWS, Pulumi Cloud, GitHub. The specific stack matters less than the principle of one place.

And the part that matters most: the factory keeps running, testing, and deploying long after we’ve closed our laptops and gone to sleep — the dark factory pattern, where the line keeps producing with the lights off.

Build the factory somewhere you can work on it — not just somewhere it can run.

A factory scattered across laptops can’t be improved as a system. Cloud keeps it in one shape, 24/7, and lets the team iterate together.

How to build an AI-native software factory

There’s no single blueprint for how to build a software factory, but these seven rules are the software factory model we run at Compostable AI: transform the workflow rather than bolt AI onto it, remove problems instead of engineering around them, pick tools your agents can drive, split work across specialized agents, measure human hours per unit of value, design for convergence over one-shot correctness, and run the factory in the cloud so it operates 24/7.

Closing thought

I’ve shipped more code in the last two years than I did in the fifteen before that. Most of it in languages I couldn’t write by hand. And that’s after a stretch in leadership where I wrote almost none.

If you’re where I was two years ago: don’t ask how AI fits into what you already do. The factory is built one rule at a time, and it’s not a template — it’s the practice of finding where you’re taking advantage of the new economics and where you’re not, where your practices still need an update. The leverage is in finding these places and improving them.

Frequently asked questions

What is a software factory?

A software factory runs software delivery as a repeatable industrial process instead of hand-crafting each feature, the way a physical factory turns out products on a line. The idea predates AI. An AI-native software factory is the version where agents do most of the building and the engineers steer.

How is a software factory different from traditional software development?

Traditional development treats code as a craft, each function written and reviewed by hand. A software factory treats software delivery as an industrial process: agents handle the writing and checking through an automated, converging loop, and the expensive human hours move to defining the work and deciding what to build. The goal is to minimize human hours per unit of value shipped.

How do you keep AI agents from breaking things across customers?

Rather than build a shared multi-tenant sandbox, Compostable AI removes the problem: every client gets two dedicated AWS accounts, one for production and a “digital twin” staging account. Agents iterate on staging until the work checks out, and only then does it ship to production. Infrastructure-as-code tools like AWS Control Tower and Pulumi make running that account fleet tractable for a small team.

What tools does an AI-native software factory need?

Tools your agents can actually drive, anything with a solid API and a clean CLI, rather than click-ops around a web UI. Compostable AI expresses infrastructure as type-safe TypeScript with Pulumi, pairs it with Pulumi Neo for domain-specific infrastructure skills, and uses Pulumi ESC for configuration. If part of your stack still requires a human to click through a UI, your agents stop there.

Watch the original Pulumi webinar. Read the Compostable AI case study, and learn more about Compostable AI and Pulumi Neo.

Ten More Things You Can Do With Pulumi Neo

Adam Gordon Bell — Tue, 19 May 2026 05:00:00 -0700

Last fall, after launching Pulumi Neo, we wrote up 10 things you could do with it. In the months that followed, as platform teams handed Neo more real work, we watched and listened, shipping a steady stream of features like plan mode, read-only mode, AGENTS.md, an integration catalog, cross-cloud migration, and task sharing. With today’s release, Neo extends beyond the Pulumi Cloud console into the Pulumi CLI, GitHub, and Slack.

So here are 10 more things you can do with Neo.

1. Deploy your app to AWS without writing IaC

Hand Neo a repo. Neo picks the right services — ECS, AWS Fargate, ALB — writes the Pulumi, and opens a PR.

The cloud infrastructure part of getting a new service running, especially one in a new language, is always a few hours of boilerplate: a VPC and subnets, an IAM role, security groups, a load balancer, DNS, and a TLS cert.

With Neo, that work collapses into a prompt. Point Neo at a repo and ask:

Deploy this app to AWS as a publicly accessible service.

Plan mode comes back with the resources Neo will create, named and sized: ECS running on AWS Fargate, an ALB, and the VPC wiring. Approve, and Neo writes the Pulumi program, runs a preview, and opens a PR. You, the human in the loop, merge it after review.

Neo planning a PR and deploying an app to AWS.

Start a Neo task Ask Neo to deploy your app to AWS and make a PR

2. Diagnose a slow API from metrics, logs, and code

Slow endpoints live at the seam between runtime metrics and the stack that runs them. Neo reads both and proposes a fix with the metric evidence as the rationale.

Production incidents often involve multiple tools. When the checkout endpoint’s p95 latency climbs from 200ms to 1.2s, the metric is in Datadog, but the cause might be somewhere in your AWS account: maybe RDS is out of IOPS, maybe the connection pool is too small, maybe the autoscaler isn’t keeping up. Connecting “this metric looks bad” to a recent backend change and then to a one-line fix in your Pulumi program is an exercise in detective work.

Neo’s integration catalog bridges this gap. With built-in Datadog APM, PagerDuty, and Honeycomb integrations sitting alongside your Pulumi state, Neo can read traces and metrics from the tools your team already uses and take action.

Ask Neo:

Find the scaling bottleneck on /checkout from the last 7 days of metrics and propose a fix.

Neo pulls the metric history, matches the Datadog tag db.cluster=checkout-rds to the RDS instance in your prod-checkout Pulumi stack, and opens a PR with a Pulumi diff that bumps the storage IOPS and raises the connection-pool ceiling. You review and roll out the fix.

Toggle on the Honeycomb integration so Neo can read traces and metrics alongside your Pulumi stacks.

3. Automate PagerDuty incident response from Slack

A page comes in. You paste it into your on-call channel and tag Neo, and Neo replies with the cross-system view you’d otherwise spend the first 20 minutes assembling.

On-call triage is often about getting up to speed quickly. You get paged because something is in the red, and you don’t know why.

You mention Neo in the on-call Slack channel:

@neo, what’s going on with this alert?

Neo starts querying metrics and traces. With PagerDuty and Datadog in the integration catalog, it correlates the alert with every deploy and stack change tagged with the alert’s service in the last hour, and finds the change that lines up:

Two deploys in the last hour touched services tagged service:checkout: checkout-api@a3f9c2 (12 min ago, app-layer deploy) and Pulumi stack prod-checkout-rds (45 min ago, decreased max_connections from 200 → 100). p99 latency inflection at 14:03 lines up with the stack change. Likely cause: the connection-pool reduction is starving the API under current load.

You ask a couple of clarifying questions in-thread, then ask Neo to open a rollback PR against the Pulumi stack.

Authorize PagerDuty and Datadog in Neo's settings. Neo can then read alerts in your on-call Slack channel, find the change that correlates, and open a PR when you ask.

4. Implement a Jira or Linear ticket end-to-end

Hand Neo a ticket number from Linear, Jira, or GitHub Issues. Neo reads the description and acceptance criteria, plans against your stack, and opens a PR.

Tickets often pile up not because they’re unimportant, but because they’re not urgent. Ongoing maintenance quietly accumulates. Bumping a provider version, centralizing secret management, working through small policy violations: each one matters, but none of them ever moves to the top of the queue. Explaining each one to an agent is its own overhead.

The fix is letting Neo read the ticket itself. Connect Linear integration or Jira automation through the integration catalog (GitHub Issues works too), and Neo pulls the ticket the same way an engineer would: title, description, acceptance criteria.

Ask Neo:

Implement CAD-1234 in our payments stack.

Neo reads the ticket, plans against your existing stack, opens a PR, and drops a comment back on the ticket. The ticket and the PR end up linked, and your backlog shrinks.

Neo running locally in the Pulumi CLI: fielding a Linear issue, analyzing the codebase, and producing a PR that upgrades multiple projects to the latest Pulumi and AWS provider versions.

Start a Neo task Implement a Linear ticket end-to-end

5. Audit and tighten over-privileged IAM roles

Neo audits each role against what your stack code actually does, and proposes scoped policies that improve your security posture.

IAM cleanup is the kind of work nobody has the time to prioritize. Production has 40 roles. Half of them started with s3:* because nobody had time to scope them, and the cleanup slips quarter to quarter.

Ask Neo:

Audit IAM permissions across my accounts and propose narrower policies for over-privileged stack-managed roles.

Neo cross-references each role’s policy against what the stack code actually calls, and opens a PR per role. The PR body lists the API calls Neo found in the stack code, like s3:GetObject on audit-logs-* and s3:PutObject on audit-logs-staging, as the justification for the scoped policy. The evidence sits next to the diff.

If you’re unclear about which roles count as in-scope or what your team considers over-privileged, start in plan mode and agree on that with Neo first.

Neo auditing an over-privileged IAM role and proposing a narrower policy, with the actually-used permissions as evidence.

Start a Neo task Audit IAM and tighten over-privileged roles

6. Migrate from AWS CDK onto your platform’s golden paths

Neo reads your existing AWS CDK app and lands a PR that swaps AWS’s defaults for your team’s published components.

CDK’s L2 constructs encode AWS’s defaults. s3.Bucket with encryption: BucketEncryption.S3_MANAGED is a sane choice, but it’s AWS’s idea of sane, not yours. A platform team that’s published its own components to the Pulumi Private Registry has already decided what your bucket defaults look like: encryption with the right KMS key, tagging by cost center.

Ask Neo:

Migrate the payments-vpc CDK stack to Pulumi using our published components.¹

Neo reads the source CDK app and your registry side by side. It maps each CDK construct to its closest team-published equivalent, clarifying with you where the mapping is ambiguous.

// Before (AWS CDK, AWS's defaults)
const bucket = new s3.Bucket(this, "Assets", {
 bucketName: "payments-assets",
 encryption: s3.BucketEncryption.S3_MANAGED,
 versioned: true,
});

// After (Pulumi, your team's published component)
import * as platform from "@payments/platform";

const bucket = new platform.Bucket("assets", {
 bucketName: "payments-assets",
 classification: "internal",
});

Start a Neo task Migrate CDK onto your golden paths

Neo handles Terraform and Azure ARM migrations the same way.

7. Containerize a service and migrate it to Kubernetes from a runbook

Write the containerization pattern down once. Every service after that is a prompt away.

Containerizing an application and moving it to Kubernetes involves several small decisions: which Docker image, what labels go on deployments, how ingress is wired, and how secrets reach the pod. But after a team has moved two or three services, the pattern is set. The decisions get written down in a runbook, and every subsequent migration is mostly the same shape.

Ask Neo:

Containerize the billing-api service and write its Kubernetes manifests, following our K8s migration runbook in Confluence.

Neo reads the source repo and the runbook in Confluence via the integration catalog and starts working on your request.

You can save this as a Neo skill that splits the work into multiple PRs — Dockerfile first, ECR config next, Deployment/Service/Ingress manifests after — and link back to each runbook convention for ease of review. The output reflects your conventions: the labels you actually use, the ingress class you’ve standardized on, and the External Secrets Operator config your team prefers.

You’re still the one reviewing the PRs and deciding what the cutover looks like in production. Neo follows your internal standards, so the new service ends up shaped like the last one you migrated.

Neo migrating a VM-based service to Kubernetes step by step, following the team's Confluence runbook.

Once you’ve delegated something a few times, the next move is to automate it. The remaining three tasks are the kind Neo doesn’t need to be asked for. Drift, deps, compliance: they’re the operations you put on a schedule via Neo Automations.

8. Schedule daily configuration drift detection across your cloud infrastructure

Schedule a daily drift check across your cloud. Wake up to PRs that fix what changed overnight.

Configuration drift is an ongoing challenge. The security team rotated an IAM role at 04:47 UTC. Someone changed a security group in the AWS console three weeks ago. Left alone, drift turns into security gaps, into compliance issues, and into the kind of “wait, who changed that?” confusion nobody wants to chase down.

Pulumi Cloud is already good at configuration drift detection. Neo takes it a step further.

Ask Neo:

Every morning at 6 AM, check all production infrastructure for drift and create PRs to fix any issues you find.

From then on, the task runs on its own, and you wake up to a PR per drifted resource. The description spells out what happened (iam_role.audit-reader had inline policy AllowReadAuditLogs added at 04:47 UTC) and cites the section of infra/runbooks/drift.md Neo followed.

Some drift gets encoded into the Pulumi program, like the IAM rotation above. Some gets reverted, like the security group rule added from the console. Some gets ignored entirely, like autoscaler-managed Lambda concurrency reservations the runbook tells Neo to skip. You write the runbook once; Neo follows it every morning to decide what to do.

Neo's morning drift PR. The body names the resource, the change, when it happened, and the section of the runbook Neo followed to decide what to do.

Start a Neo task Schedule a daily drift check

9. Schedule weekly upgrades for outdated Lambda runtimes and providers

Lambda runtimes and container base images age out. Schedule the upgrade pass; review the PRs Neo opens.

AWS Lambda end-of-life notices come out months ahead. Node 20 stopped receiving updates as an AWS Lambda runtime at the end of April. Python 3.9 reached end-of-support last December. After the deadline, AWS blocks new deploys and eventually stops invoking the function. Each one needs to move to a supported runtime before the cutoff.²

Schedule it:

Every Sunday night at 10 PM, check our Lambda functions for runtimes nearing end-of-support and open PRs to upgrade them.

Neo reads the AWS Lambda runtime deprecation page, matches the end-of-support runtimes against every Lambda function in your stacks, and opens one PR per stack.

If Python 3.9 is reaching end-of-support, the upgrade is to Python 3.12, and datetime.utcnow() calls need to move to datetime.now(datetime.UTC). Neo can make all of those replacements in the same PR.

The same task can catch container base images with critical CVEs and bump them too.

Setting up a weekly task in the Scheduled Tasks UI. Once saved, Neo runs the prompt every Sunday night and opens PRs you review on Monday.

Start a Neo task Schedule a weekly runtime upgrade check

10. Fix AWS CIS Benchmark failures with daily PRs

Run the AWS CIS Benchmark on a schedule. Wake up to PRs that fix every failure.

The CIS AWS Foundations Benchmark, available through AWS Security Hub, is something every team should be keeping an eye on. The benchmark finds issues like S3 buckets that allow public read access (S3.1), root user access keys that shouldn’t exist (IAM.4), or CloudTrail not being enabled (CloudTrail.1). Scanning for these issues is a solved problem, but closing and addressing them is not. They pile up between audits because each one is a code change in a different stack, and nobody owns the cross-stack cleanup.³

Schedule the cleanup:

Every morning, read CIS Benchmark failures from Security Hub. For every failure on an IaC-managed resource, open a PR with the fix.

Neo opens one PR per failure. A bucket failing S3.1 arrives as a Pulumi diff that adds blockPublicAccess to the bucket in your prod-checkout stack. The PR body lists the CIS rule number, the resource ID, the diff, and a clean pulumi preview against the live infrastructure.

The runbook is where your security team writes down what each control means for your stacks. Block public S3 buckets, except the ones tagged public-content=true for CloudFront origins. Don’t auto-touch the break-glass IAM roles; page a human instead. Multi-region CloudTrail stays on, no exceptions. Neo reads that file, checks each Security Hub finding against it, and only opens a PR for the ones you’ve said are safe to fix. The rest get routed or ignored, the way your team already handles them.

A PR raised by Neo to fix a CIS Benchmark failure, with the failing rule, the resource, and the runbook decision laid out in the body.

Start a Neo task Schedule daily AWS CIS Benchmark scans

Neo: your newest platform engineer

Over the past year, many product teams have stopped treating AI as a request-by-request assistant and started delegating to it outright.⁴ Agents open pull requests, investigate issues, and iterate on review feedback.

But platform engineers have held back because a bad infrastructure change doesn’t just fail, it can take production down. Coding agents benefit from fast, forgiving feedback loops, but infrastructure recovery is rarely as simple as reverting a commit.

What was missing wasn’t the appetite. It was an agent with enough organizational context and grounding to plan reliably, enough guardrails to feel safe and contain mistakes, and enough discipline to keep working without being asked.

The theme across these tasks is clear. A thing platform engineers used to keep in their heads becomes a task you delegate, then becomes work that runs without you. Neo isn’t generating infrastructure from a template. It’s a teammate who knows your code, your providers, your conventions, your production metrics, and can raise PRs for you to review.

Neo now lives in your terminal, in your pull requests, in your Slack workspace, and in Pulumi Cloud. Pick one of these workflows and give it a try.

The observant reader will notice Terraform-to-Pulumi was covered in the original post. ↩︎
Also covered in the original post. Last year you could ask Neo to do it once. This year you can put it on a schedule. ↩︎
Also covered in the original post. Last year Neo could remediate violations on demand. This year Security Hub feeds findings to a scheduled task that knows your runbook’s interpretation of each control. ↩︎
For a concrete example, see Seven Rules for Building an AI-Native Software Factory: Ewan Dawson, CTO of Compostable AI, runs nineteen client deployments with five engineers, using Pulumi Neo to handle most of the infrastructure work. ↩︎

CDKTF is deprecated: What's next for your team?

Adam Gordon Bell — Thu, 18 Dec 2025 10:00:00 -0800

In July, 2020, CDK for Terraform (CDKTF) was introduced, and last week, on December 10, it was officially deprecated. Support for CDKTF has stopped, the organization and repository have been archived, and HashiCorp/IBM will no longer be updating or maintaining it, leaving a lot of teams out there without a clear path forward.

For most teams, that means it’s time to start looking for a replacement.

It’s an unfortunate situation to suddenly find yourself in as a user of CDKTF, but you do have options, and Pulumi is one of them. In this post, we’ll help you understand what those options are, how Pulumi fits into them, and what it’d look like to migrate your CDKTF projects to Pulumi.

What are the alternatives to CDKTF?

Teams migrating away from CDKTF generally have three options:

Option 1: Fall back to HCL

HashiCorp’s official recommendation is to export your projects to HashiCorp Configuration Language (HCL) and manage them with Terraform. CDKTF even has a command that makes this fairly simple:

cdktf synth --hcl

Of course, if you’re using CDKTF, you probably chose it specifically to avoid HCL. So while possible, this probably isn’t the choice most teams would make unless they had to.

Option 2: Migrate to AWS CDK

If your team is all-in on AWS, another option would be to migrate to AWS CDK. It’s widely used, officially supported, the programming model is similar to CDKTF’s, and both CDK and CDKTF transpile to an intermediate format (CloudFormation YAML and Terraform JSON, respectively) that gets passed on to their underlying tools for deployment.

But while their programming and deployment models are conceptually similar, their resource models and APIs are entirely different. Here’s the code for an S3 bucket written in AWS CDK, for example:

import * as s3 from 'aws-cdk-lib/aws-s3';

const bucket = new s3.Bucket(this, 'my-bucket', {
 bucketName: 'my-example-bucket',
 versioned: true,
 publicReadAccess: false,
});

And here’s the code for a similarly configured bucket in CDKTF:

import { S3Bucket } from '@cdktf/provider-aws/lib/s3-bucket';

const bucket = new S3Bucket(this, 'my-bucket', {
 bucket: 'my-example-bucket',
 versioning: {
 enabled: true,
 },
 acl: 'private',
});

Notice how different these APIs are — and this is just one simple resource with only a few properties; imagine having to rewrite dozens or hundreds of them. Beyond that, there’s also the problem of state: How would you go about translating the contents of a Terraform state file containing hundreds of resources into the equivalent CloudFormation YAML or JSON?

Despite their surface similarities, CDKTF and AWS CDK have little in common. Migration would essentially mean a ground-up rewrite that’d also leave you without the multi-cloud support you already have with CDKTF. For most teams, that makes this option a practical non-starter.

Option 3: Migrate to Pulumi

This is where we should acknowledge our obvious bias — but we genuinely believe that for most users of CDKTF, Pulumi really is the simplest and most broadly compatible alternative.

Like CDKTF, Pulumi lets you build and manage your infrastructure with general-purpose languages like TypeScript, Python, Go, C#, and Java, and it supports organizing your code into higher-level abstractions called components, which you can think of like CDKTF constructs. Both organize cloud resources into stacks (think dev, prod), and both track deployment state similarly, with local, remote, and cloud-hosted options available.

Many of Pulumi’s most popular providers (e.g., the AWS provider) are also built from open-source Terraform schemas, which means their resource models will be nearly identical to what you’re used to with CDKTF. Here’s what an S3 bucket looks like in Pulumi, for example:

import * as aws from '@pulumi/aws';

const bucket = new aws.s3.Bucket('my-bucket', {
 bucket: 'my-example-bucket',
 versioning: {
 enabled: true,
 },
 acl: 'private',
});

You can also use any Terraform provider with Pulumi, and you can even reference Terraform modules directly from within your Pulumi code.

Pulumi is also different from CDKTF in several ways. One is that rather than transpile your source code to a format like JSON as CDKTF does (and then deploying it separately later), Pulumi uses its own declarative deployment engine that resolves the resource graph at runtime and provisions cloud resources directly, which is much faster and more flexible. You can learn more about the deployment model in How Pulumi Works.

Given the API similarities, the support for all Terraform providers and modules, the ability to coexist alongside Terraform-managed projects, and the built-in support for conversion (which we’ll cover next), we think Pulumi is the best alternative for most teams looking to migrate.

Migrate from CDKTF to Pulumi

Keep writing infrastructure in TypeScript, Python, Go, C#, or Java, and use Pulumi’s built-in convert and import commands to bring your CDKTF projects and state across.

Get started

What migrating to Pulumi looks like

Migrating a CDKTF project to Pulumi generally happens in three steps:

Conversion, which translates your CDKTF code into a new Pulumi program
Import, which reads the contents of your CDKTF state into a new Pulumi stack
Refactoring, which brings the code in the new program into alignment with the stack’s currently deployed resources

Conversion and import

Migration starts with exporting your CDKTF project to HCL with cdktf synth. From there, Pulumi’s built-in convert and import commands handle creating the new program and importing your state:

# Export your project to HCL.
cdktf synth --hcl

# Convert the HCL into a new Pulumi project.
pulumi convert --from terraform --language typescript

# Create a new Pulumi stack.
pulumi stack init dev

# Import your CDKTF stack's resources into your new Pulumi stack.
pulumi import --from terraform ./terraform.dev.tfstate

The converter automatically translates Terraform input variables, data sources, resources, and outputs into their Pulumi equivalents. You can read more about how this works in Converting Terraform HCL to Pulumi.

Refactoring

Once you’ve imported your state, you’ll often have to make some adjustments to the code to bring it in line with the new Pulumi stack. For instance, pulumi import marks new resources protected by default, to prevent them from being accidentally deleted — but since the code produced by pulumi convert doesn’t include the protect resource option, you’ll need to add it yourself. Fortunately the import step also emits code that you can copy into your program to make this process a little easier.

Refactoring can get a bit more complicated when custom logic and higher-level abstractions are involved, as fidelity to the original CDKTF code is often lost in the translation to HCL. In these situations, having the help of an LLM to recapture that original logic or translate your CDKTF constructs into Pulumi components can be a big time-saver.

An end-to-end example

The best way to get a feel for how this works, though, is to try it yourself.

The pulumi/cdktf-to-pulumi-example repository on GitHub contains a CDKTF project with multiple stacks written in TypeScript, along with a guide that walks you through the process of migrating that project to Pulumi. The guide covers everything we’ve discussed here so far, including:

Converting the CDKTF project into a new Pulumi project
Importing its actively running resources into Pulumi stacks
Modifying the generated code to align with imported state
Performing an initial deployment with Pulumi to complete the migration process

The walkthrough takes only a few minutes to complete, and it’s a great way to stand up an example of your own to get more familiar with Pulumi.

github.com/pulumi/cdktf-to-pulumi-example

What’s next?

If you’re moving on from CDKTF and looking for an alternative, there are a few possible paths forward. For teams that want to keep using real languages and avoid a ground-up rewrite, Pulumi offers the clearest way forward.

To learn more about how Pulumi works, how it differs from CDKTF and from Terraform, how to handle additional conversion scenarios, and more, we recommend:

Diving into the Pulumi docs to get familiar with core concepts and features of the platform
Reading Migrating from Terraform or CDKTF to Pulumi for more detailed, Terraform-specific migration guidance
Joining us in the Pulumi Community Slack to ask questions and learn from others who’ve successfully made the leap from Terraform and CDKTF to Pulumi
Checking out Pulumi for All Your IaC — Including Terraform and HCL to learn more about Pulumi’s native support for Terraform and HCL

And of course, feel free to reach out! We’d love to help in any way we can.

AWS built an integrated AI Agent training pipeline and they want you to rent it

Adam Gordon Bell — Wed, 10 Dec 2025 00:00:00 +0000

AWS re:Invent 2025 delivered a myriad of announcements across AI, silicon, and cloud infrastructure. AWS unveiled the expanded Nova model family, introduced Nova Forge for custom model training, launched Trainium3 UltraServers, and added major production features to AgentCore. It was a lot, and taken at face value, it looks like another scattershot year of big releases.

But if you look past the firehose, a pattern emerges. These announcements fit together into a single bet about how enterprise AI will be built.

AWS built a vertically integrated agent-training pipeline; it’s expensive, ambitious, and not for everyone.

The clearest place to see that pattern is in how AWS talked about the Nova models. AWS dropped four new foundation models (Lite, Pro, Sonic, Omni) spanning text, multimodal, and speech. And AWS downplayed benchmarks entirely. A hint that the models aren’t the real story.

What AWS was really foregrounding wasn’t the models themselves, but the system used to shape them: Nova Forge.

Try it with Neo: Set up Nova 2 with Pulumi

Rent the lab: Nova Forge

Nova Forge is a managed way to run continued pretraining, fine-tuning, and reward-based alignment on Amazon’s Nova models using your data and your reinforcement loops. Instead of a finished, frozen model plus a thin fine-tuning API, you feed your data into earlier training stages while AWS handles the ugly parts: large-scale training runs, cluster management, and hosting. Access is $100,000 per year¹, plus compute costs!

If you can afford that, you bring big proprietary datasets (code, tickets, logs, documents) and they keep doing next-token pretraining on a mix of their data and yours, then instruction tuning (SFT), then “RL-style” preference optimization, but with your data and your reward signals mixed in.

Why does this exist? Because the kind of training this enables is usually out of reach. Training a GPT-4-class frontier model from scratch runs tens of millions to $100M+ in compute alone². You don’t own the weights and you’re locked into their stack, but you get frontier-level capabilities with your data baked in, without building datacenters or staffing ML teams.

Think of it as frontier-lab-as-a-service. No one else offers anything this close to a public, end-to-end training pipeline. And the only reason AWS can offer it is the next announcement.

The margin weapon: Trainium

The idealized Trainium flywheel: each generation should decrease training costs.

AWS built its own AI accelerator so it doesn’t have to live entirely on NVIDIA. Trainium is that chip. You don’t buy it; you rent it as a cloud box. This year: their third-gen chip (Trainium3) and new rack-scale Trn3 UltraServers are out, with 4× the performance and big energy/cost gains over the previous gen, positioned as a serious alternative to high-end GPUs for training and serving big models.

I’m sure one reason for Trainium’s is that AWS wants to stop handing NVIDIA half its AI training revenue. But the real story is bigger than cost-cutting. Trainium is the quiet machinery that makes AWS’s model-factory ambitions economically viable. You can only rent a frontier training pipeline if you can afford it, and Trainium makes it cheaper (if that word applies to six-figure entry costs).

Trainium is what turns Forge from a one-off experiment into an actual development pipeline. By compressing the marginal cost of each training cycle, AWS is trying to make iterative specialization economically viable. You can tune, test, and retrain until you converge on something useful.

AWS is clearly positioning Trainium3 to anchor a fully vertical stack

Spinning up a Trainium instance with Pulumi (where available):

import pulumi_aws as aws

# Assumes: ami, subnet, security_group already configured
trn1_instance = aws.ec2.Instance("trn1-instance",
 instance_type="trn1.2xlarge",
 ami=ami.id,
 subnet_id=subnet.id,
 vpc_security_group_ids=[security_group.id],
 associate_public_ip_address=True,
 tags={"Name": "trn1-training-instance"},
)

Try it with Neo: Provision Trainium instances

The data moat play

For most companies, this whole stack is overkill. If your AI roadmap is “add a chatbot and maybe summarize some tickets,” you don’t need Nova Forge, and you definitely don’t need Trainium. Hosted models plus RAG and an Agentic loop will get you 90% of the way there.

But this type of training is powerful, and it’s never been in reach to so many. If LLMs behave like the distributions they’re trained on, then getting your proprietary mess (logs, incident reports, claims histories, deal flows, call transcripts) into the core training loop means the model doesn’t just know your docs; it behaves like someone who’s lived inside your systems. That’s qualitatively different from “we stuffed a PDF into the context window.”

Latency and cost at scale matter too. For high-volume workflows like support triage, routing, code review, and fraud checks, “generic frontier model + giant prompt + RAG + tools” is slow and expensive. A smaller model that has your world baked into the weights can run with smaller contexts, simpler prompts, and fewer tool calls. And then there is reinforcement learning, which I’ll get to shortly.

But even if you get that far, a custom Nova model sitting in Bedrock is only half the story. You still need somewhere for it to act: a runtime, tools, policies, and an audit trail. That’s the gap AgentCore is meant to fill.

Where the models work: AgentCore

AgentCore: building blocks so you don't have to wire agents from scratch.

If Nova is the brain and Trainium is the muscle to build it, AgentCore is the nervous system.

AgentCore is a managed runtime for AI agents: instead of you wiring LLMs, tools, memory, auth, and logging together on Lambda or Fargate, AWS gives you a sticky per-session microVM, a standard way to call tools (Gateway), built-in long- and short-term memory, identity/permissions, and observability/evals. You package your agent, deploy it as an AgentCore runtime, and AWS handles the ugly parts: session isolation, scaling, policy guardrails, and tracing. You pay Fargate-ish per-vCPU/GB-hour pricing for the runtime plus normal Bedrock token and tool-call costs.

At re:Invent 2025, AgentCore picked up the missing “production” pieces: Policy, Evaluations, and episodic Memory. These handle guardrails, quality checks, and per-session state, so you don’t have to build them yourself.

Deploying an AgentCore runtime with Pulumi:

import pulumi_aws as aws

# Assumes: role, ecr_repo already configured
agent_runtime = aws.bedrock.AgentcoreAgentRuntime("my-agent",
 agent_runtime_name="my-agent-runtime",
 role_arn=role.arn,
 agent_runtime_artifact={
 "container_configuration": {
 "container_uri": f"{ecr_repo.repository_url}:latest",
 },
 },
 network_configuration={
 "network_mode": "PUBLIC",
 })

Try it with Neo: Deploy an AgentCore runtime — Requires a container image in ECR.

How does this come together? AWS shipped a use case.

The proof of concept: Nova Act

The AWS AI stack: vertically integrated from silicon to agent runtime. Nova Act uses the full stack.

Nova Act is the concrete example of this whole thing coming together. It handles browser-based UI automation: form filling, search-and-extract, QA testing. Amazon claims ~90% reliability. It deploys directly to AgentCore Runtime.

It’s not “an LLM plus Playwright.” Nova Act uses a specialized Nova 2 Lite variant trained on synthetic “web gym” environments: browser simulations that mirror enterprise UIs and provide an automatic reward signal when tasks are completed correctly. Instead of judging output quality, this model was trained on an RL loop that asks: Did the workflow succeed?

That specialized model is wrapped in AgentCore. The platform handles isolation, scaling, logging, and guardrails, so Nova Act behaves like a production automation system rather than a brittle demo.

Seen this way, Nova Act is Amazon’s reference implementation for a certain class of enterprise agents: start with a strong general model, specialize it through domain-specific RL in a controlled environment, and run it on AgentCore with tools and policies around it. It’s the pattern AWS expects customers to adopt.

One stack to rule them all

So Nova Forge, Trainium, AgentCore, and Nova Act connect. Trainium lowers the cost of big training runs. Nova Forge lets enterprises plug their own data and rewards into those runs. AgentCore is where the resulting models act, with tools, memory, and policy guardrails. Nova Act shows the pattern in action: a domain-specialized Nova model, trained in a controlled loop, running as a production agent.

Most enterprises still won’t choose this path. They don’t have the data, the reward loops, or the operational maturity to make early-stage training worthwhile.

But AWS’s bet is that enterprise AI is moving past stock foundation models and generic chatbots. AWS is expecting a world of agents shaped by proprietary data and domain feedback. Most companies won’t build the infrastructure to train and operate those agents, and so AWS is offering to rent them the whole pipeline.

Try it with Neo: Deploy a Bedrock-powered API with Pulumi

CNBC reporting on Nova Forge pricing. Source ↩︎
Sam Altman stated GPT-4 cost “more than $100 million” to train. Source ↩︎

How to Implement Robust Security Guardrails Using Policy as Code

Adam Gordon Bell — Tue, 30 Sep 2025 00:00:00 +0000

Welcome to the third post in our IDP Best Practices series, where we explore how to implement policy as code with Pulumi CrossGuard to create deployment guardrails that make self-service infrastructure both powerful and safe.

Platform engineering presents a fundamental tension: we want to enable developer velocity while maintaining security and compliance. Every platform team faces the same question: how do you give teams the freedom to deploy infrastructure quickly without compromising on safety, security, or organizational standards? The answer isn’t to choose between speed and safety, but rather to embrace automated guardrails powered by policy as code that make both possible simultaneously.

Want hands-on experience? Access the complete demo code and policy examples from this workshop.

The Platform Engineering Challenge: Speed vs. Safety

Let me tell you a story that perfectly captures the platform engineering dilemma. Statsig, a fast-growing feature flag platform processing an incredible 2 trillion events daily, had one infrastructure engineer named Jason who handled all infrastructure requests. Everything worked smoothly until Jason went on parental leave. Suddenly, the entire engineering organization ground to a halt. Infrastructure requests piled up, deployments slowed to a crawl, and the team faced a stark realization: their entire infrastructure capability depended on a single person.

This crisis forced them to confront a fundamental question that every growing engineering team eventually faces: How do you enable self-service infrastructure without sacrificing security, compliance, or operational stability? The answer wasn’t to hire more Jasons or to lock down infrastructure even tighter. Instead, they discovered that deployment guardrails with policy as code could transform their infrastructure conversation from “talk to the infrastructure person” to “ship with confidence, knowing guardrails will catch any issues.”

Think of it like building a system of roads with guardrails. When you create well-designed infrastructure components with proper safety barriers, teams can drive fast and confidently, knowing they’re protected from going off the cliff. The guardrails don’t slow them down; they enable speed by removing fear.

Understanding Platform Engineering Layers

Before we dive into implementing guardrails, we need to understand the architecture of a modern platform and where policies fit within it. Think of your platform as having three distinct layers, each with different security and access requirements.

At the foundation, you have your Layer 1: Foundational Infrastructure, which includes security controls, shared networking, and identity management systems like OIDC and IAM. Platform teams typically manage this layer with strict access controls because it forms the security backbone of your entire infrastructure.

Building on that foundation is Layer 2: Shared Infrastructure, which encompasses VPCs, compute platforms, and load balancers. These are standardized components that teams can use but with some room for customization based on specific needs.

The most dynamic layer is Layer 3: The Workloads Layer, where deployable artifacts, pipelines, and operability tools live. This is where the rubber meets the road, where most self-service activity happens and, consequently, where guardrails become absolutely critical. Developers need maximum freedom at this layer to innovate and ship quickly, but this freedom also introduces the most risk. It’s precisely here that policy-driven guardrails prove their worth, enabling that freedom while automatically preventing dangerous configurations.

What Are Deployment Guardrails?

Deployment guardrails are automated policies that act as your infrastructure’s safety net. Rather than relying on manual reviews or hoping developers remember all the security requirements, guardrails automatically prevent misconfigurations before they reach production. They enforce security standards without human intervention, guide developers toward best practices through immediate feedback, and enable safe self-service by catching issues at the earliest possible moment.

A helpful analogy is to think of guardrails like type checking in programming languages. Just as TypeScript doesn’t restrict your ability to write JavaScript but rather catches type errors before runtime, deployment guardrails don’t limit your infrastructure creativity. They simply ensure you’re following secure patterns and catch potentially dangerous configurations before they cause problems in production.

Introducing Pulumi CrossGuard: Policy as Code

Pulumi CrossGuard is Pulumi’s policy as code framework that brings the same engineering rigor to compliance and security that you apply to your application code. Instead of maintaining policy documents in wikis or relying on manual reviews, you can write policies in familiar programming languages like Python, TypeScript, or Go. These policies then enforce themselves across all your cloud resources and providers, running at different stages of the deployment lifecycle and integrating seamlessly with your CI/CD pipelines for automated enforcement.

Key Policy Types

CrossGuard supports two fundamental types of policies, each serving different validation needs:

Resource Policies: Validate individual resources

def restrict_dangerous_ports(args: ResourceValidationArgs, report_violation: ReportViolation):
 if args.resource_type == "aws:lb/targetGroup:TargetGroup":
 port = args.props.get("port")
 dangerous_ports = ["22", "23", "3389"]
 if str(port) in dangerous_ports:
 report_violation("Dangerous port detected. Avoid using SSH, Telnet, or RDP ports.")

Stack Policies: Validate relationships across resources

def validate_microservice_encryption(args: StackValidationArgs, report_violation: ReportViolation):
 microservice_components = []
 s3_buckets = []

 for resource in args.resources:
 if resource.resource_type == "custom:infrastructure:Microservice":
 microservice_components.append(resource)
 elif resource.resource_type == "aws:s3/bucket:Bucket":
 s3_buckets.append(resource)

 if microservice_components and s3_buckets:
 for bucket in s3_buckets:
 encryption = bucket.props.get("serverSideEncryptionConfiguration")
 if not encryption:
 report_violation("S3 bucket must have encryption enabled when used with microservice components.")

Building Practical Guardrails: Real-World Examples

Let’s walk through building guardrails for a microservice platform, starting with the component from our previous workshop.

Example 1: Port Security Policy

Problem: Developers might accidentally expose services on dangerous ports like SSH (22) or RDP (3389).

Solution: A policy that blocks dangerous port configurations.

import pulumi_policy as policy

def restrict_dangerous_ports_validation(args, report_violation):
 """Prevent services from running on dangerous ports."""
 if args.resource_type == "aws:lb/targetGroup:TargetGroup":
 port = args.props.get("port")
 dangerous_ports = ["22", "23", "3389", "5432", "3306"]

 if str(port) in dangerous_ports:
 report_violation(
 f"Port {port} is not allowed. This port is commonly used for "
 f"administrative services and should not be exposed via load balancer."
 )

restrict_dangerous_ports = policy.ResourceValidationPolicy(
 name="restrict-dangerous-ports",
 description="Prevent services from using dangerous ports",
 validate=restrict_dangerous_ports_validation,
 enforcement_level=policy.EnforcementLevel.MANDATORY
)

Example 2: Resource Limits Policy

Problem: Teams might request oversized resources, leading to cost overruns.

Solution: A policy that enforces reasonable resource limits with advisory warnings.

def limit_memory_validation(args, report_violation):
 """Limit memory allocation for microservices."""
 if args.resource_type == "custom:infrastructure:Microservice":
 # Extract memory from component tags or properties
 memory_str = args.props.get("memory", "512Mi")

 # Parse memory value (assuming format like "2Gi", "1024Mi")
 if memory_str.endswith("Gi"):
 memory_gb = float(memory_str[:-2])
 if memory_gb > 1:
 report_violation(
 f"Memory allocation of {memory_str} exceeds recommended limit of 1Gi "
 f"for microservices. Consider optimizing your application or "
 f"contact the platform team for exceptions."
 )

limit_memory = policy.ResourceValidationPolicy(
 name="limit-microservice-memory",
 description="Enforce reasonable memory limits for microservices",
 validate=limit_memory_validation,
 enforcement_level=policy.EnforcementLevel.ADVISORY
)

Example 3: Cross-Resource Security Policy

Problem: When microservices use S3 buckets, encryption should be mandatory.

Solution: A stack policy that validates encryption across related resources.

def microservice_s3_encryption_validation(args, report_violation):
 """Ensure S3 buckets used with microservices have encryption enabled."""
 microservice_resources = []
 s3_buckets = []

 # Collect relevant resources
 for resource in args.resources:
 if resource.resource_type == "custom:infrastructure:Microservice":
 microservice_resources.append(resource)
 elif resource.resource_type == "aws:s3/bucket:Bucket":
 s3_buckets.append(resource)

 # If we have both microservices and S3 buckets, check encryption
 if microservice_resources and s3_buckets:
 for bucket in s3_buckets:
 sse_config = bucket.props.get("serverSideEncryptionConfiguration")
 if not sse_config:
 report_violation(
 f"S3 bucket '{bucket.name}' must have server-side encryption "
 f"enabled when used with microservice components."
 )

microservice_s3_encryption = policy.StackValidationPolicy(
 name="microservice-s3-encryption",
 description="Ensure S3 buckets used with microservices are encrypted",
 validate=microservice_s3_encryption_validation,
 enforcement_level=policy.EnforcementLevel.MANDATORY
)

Policy Enforcement Models

Pulumi CrossGuard supports multiple enforcement models to fit different workflows, and understanding when to use each model is crucial for effective policy implementation.

The Preventative Model

The most common approach is the preventative model, where policies run before resources are created. This gives you fast feedback and prevents bad deployments from ever reaching the cloud. When you run pulumi preview, policies are evaluated immediately, and if violations are found during pulumi up, the deployment is blocked. This model works particularly well for validating port configurations, resource limits, and other settings that you can check without needing to see the actual deployed resource.

# Policies run automatically during preview and deployment
pulumi preview # Policies evaluated here
pulumi up # Policies block deployment if violations found

The Detective Model

Sometimes you need to validate the actual cloud state, including auto-generated values like ARNs or IP addresses. That’s where the detective model comes in. These policies run after resources are created, making them perfect for compliance scanning and drift detection. While they can’t prevent initial deployment of non-compliant resources, they excel at catching configuration drift and validating properties that only exist after deployment.

CI/CD Integration

The third model integrates policies directly into your deployment pipeline. This ensures consistent enforcement across all teams and creates natural deployment gates. For example, you might configure GitHub Actions to run policy validation on every pull request, blocking merges if violations are found. This approach combines the best of both worlds: early feedback during development and guaranteed enforcement before production.

# GitHub Actions example
- name: Run Policy Validation
 run: |
 pulumi preview --policy-pack ./policies
 if [ $? -ne 0 ]; then
 echo "Policy violations found. Deployment blocked."
 exit 1
 fi

Policy Remediation: Beyond Detection

Modern policy frameworks don’t just detect violations; they can automatically fix them:

def auto_tag_resources(args, report_violation):
 """Automatically add required tags to resources."""
 if args.resource_type == "aws:s3/bucket:Bucket":
 tags = args.props.get("tags", {})

 if "Department" not in tags:
 # Instead of just reporting, fix the issue
 if not hasattr(args, 'tags'):
 args.props["tags"] = {}
 args.props["tags"]["Department"] = "Engineering"
 return args.props # Return modified properties

 return None # No changes needed

auto_tag_policy = policy.ResourceValidationPolicy(
 name="auto-tag-resources",
 description="Automatically add required tags",
 validate=auto_tag_resources,
 enforcement_level=policy.EnforcementLevel.REMEDIATE
)

Server-Side Policy Enforcement

For enterprise deployments, Pulumi provides server-side policy enforcement that ensures policies can’t be bypassed. The process starts by publishing your policies to your Pulumi organization with pulumi policy publish ./my-policies. Once published, you can create policy groups that combine multiple policies with specific enforcement levels, targeting particular stacks or environments while configuring exceptions for special cases. The beauty of this approach is that policies run automatically without requiring CLI flags, providing consistent governance across your entire organization without relying on developers to remember to include policy packs in their commands.

Compliance-Ready Policies

While custom policies address your specific organizational needs, compliance requirements often follow industry standards. Pulumi provides hundreds of pre-built policies for common compliance frameworks:

import { PolicyManager } from "@pulumi/policy";
import { policyManager } from "@pulumi/compliance-ready-policies/policy-manager";

new PolicyPack("aws-compliance-ready-policies-typescript", {
 policies: [
 ...policyManager.selectPolicies({
 vendors: ["aws"],
 services: ["ec2", "s3"],
 severities: ["medium", "high", "critical"],
 topics: ["encryption"],
 frameworks: ["pcidss"],
 }, "mandatory"),
 ],
});

This automatically includes policies for major compliance frameworks like PCI DSS for payment card industry standards, SOC 2 for security and compliance controls, ISO 27001 for information security management, and CIS Benchmarks for security configuration standards.

Add guardrails with policy as code

Write policies in Python or TypeScript, enforce them across every deployment, and give teams self-service infrastructure that stays secure and compliant.

Get started

Best Practices for Policy Implementation

After implementing policies at dozens of organizations, we’ve learned that successful policy adoption follows predictable patterns. The key is to adopt a product mindset: think of your policies as a product serving your development teams, not as bureaucratic rules imposed from above. Start small and iterate based on real feedback rather than trying to implement a comprehensive policy framework from day one.

Start Small and Iterate

Begin with just two or three critical policies that address your most pressing risks. Use advisory enforcement initially, which warns developers about violations but doesn’t block deployments. This gives your team time to understand and adapt to the policies. Only after gathering feedback and refining the policies should you graduate to mandatory enforcement.

Provide Clear, Actionable Error Messages

Nothing frustrates developers more than cryptic policy violations. Your error messages should explain not just what’s wrong, but how to fix it:

def good_error_message(args, report_violation):
 if violation_detected:
 report_violation(
 "S3 bucket encryption is required for compliance. "
 "Add serverSideEncryptionConfiguration to fix this. "
 "See: https://docs.company.com/s3-encryption"
 )

Embrace Progressive Enforcement

Think of enforcement levels as a dial, not a switch. Start with advisory mode to warn about issues, move to mandatory to block deployments, and eventually implement remediation to automatically fix common issues. This progression gives teams time to adapt while gradually raising the security bar.

Test Your Policies Thoroughly

Policies are code, and like any code, they need testing. Write unit tests for your policies to ensure they catch violations correctly and don’t produce false positives:

# Test policies with unit tests
def test_dangerous_port_policy():
 # Mock resource with port 22
 args = create_mock_args("aws:lb/targetGroup:TargetGroup", {"port": 22})
 violations = run_policy(restrict_dangerous_ports, args)
 assert len(violations) == 1
 assert "dangerous port" in violations[0].message.lower()

Document Escape Hatches

No matter how well-designed your policies are, there will always be edge cases. Rather than forcing teams to work around policies in dangerous ways, provide clear, documented escape hatches. Create processes for requesting exceptions, handling emergency exemptions, and appealing for policy changes. As one platform engineer put it, you need “guardrails outside the guardrails” because platform adoption will never be 100%, and that’s okay. This transparency builds trust and ensures policies remain effective enablers rather than bureaucratic obstacles to circumvent.

Real-World Success: Statsig’s Transformation

Remember Statsig’s infrastructure crisis when Jason went on parental leave? Let’s look at how they transformed their platform using the guardrails approach we’ve been discussing. After implementing policy-driven guardrails, their infrastructure story completely changed.

As Tyrone Wong, Infrastructure Engineer at Statsig, explains: “We had a single infra owner. Parental leave forced us to build self-service, and that’s when everything clicked. Developers open a PR, Pulumi previews the change right in the PR, and CI blocks risky changes. That’s how you move fast safely. The magic was turning ’talk to the infra person’ into ‘ship with guardrails.’”

The results speak for themselves. Deployment times dropped from 1.5 weeks to mere minutes. The infrastructure ticket queue that once haunted their Slack channels disappeared entirely. Security actually improved through automated policy enforcement, eliminating the human error factor. Most importantly, developer satisfaction soared as teams gained the autonomy to ship infrastructure changes without friction or fear.

When asked if developers were actually using the self-service platform, Tyrone’s response was telling: “Yeah, people are actually using it because there’s changes they wanted to get done. Previously it became this giant effort with lots of meetings, people being concerned, and Google docs about what the changes would be. Now people can just put together a PR, we have our discussions there, see if anything gets flagged, and move forward.”

Building Your Policy Strategy

Implementing deployment guardrails isn’t a big-bang transformation; it’s a journey that unfolds in phases. Based on patterns we’ve seen across successful implementations, the first week or two should focus on assessment. Start by understanding your current state, auditing existing infrastructure patterns to identify what teams are actually deploying. Look for common misconfigurations that have caused incidents or near-misses, and document your security and compliance requirements as specific, enforceable rules. Most importantly, survey your developers to understand their pain points with the current infrastructure process, as this assessment forms the foundation for policies that solve real problems rather than creating new ones.

In weeks three and four, build your foundation by implementing three to five core policies that address your most critical risks. Set up CI/CD integration so policies run automatically on every pull request, starting with advisory enforcement to gather feedback without blocking deployments. Create clear documentation and runbooks that explain not just what the policies do, but why they exist and how to work with them.

By the second month, you’re ready to expand. Add compliance-specific policies for regulatory requirements and implement server-side enforcement to ensure policies can’t be bypassed. Create formal processes for policy exemptions and exceptions, and begin measuring policy effectiveness through metrics like violation rates and remediation times.

Remember that policy implementation is never “done.” Continuously monitor violation patterns to identify areas where policies might be too strict or too lenient. Refine policies based on developer feedback and incident data, add automated remediation for common violations to reduce manual fixes, and gradually expand coverage to new services and teams using lessons learned from early adopters.

Measuring Policy Success

You can’t improve what you don’t measure, and policy effectiveness is no exception. Successful teams track metrics across three critical dimensions to validate their policy strategy.

From a security perspective, start by monitoring your policy violation rate: the percentage of deployments that trigger policy warnings or blocks. This tells you whether your policies are catching real issues or creating unnecessary friction. Track time to remediation to understand how quickly teams fix violations when they occur. Most importantly, measure the reduction in security incidents caused by misconfigurations. If your policies aren’t reducing incidents, they’re not addressing the right risks.

Security is only half the equation, though. The developer experience dimension is equally important. Monitor deployment velocity to ensure policies aren’t slowing teams down. Regular developer satisfaction surveys reveal whether teams see policies as helpful guardrails or annoying obstacles. Track support ticket volume; successful policy implementation should reduce infrastructure-related requests as teams become more self-sufficient.

Finally, don’t forget to measure the operational health of your policy system itself. Policy coverage shows what percentage of your resources are protected by policies, while the exemption rate reveals whether policies are practical or if teams constantly need exceptions. Track remediation automation to see how many violations are fixed automatically versus requiring manual intervention. Together, these metrics paint a complete picture of your policy program’s health and help you continuously improve your approach.

Common Pitfalls and How to Avoid Them

Even with the best intentions, policy implementations can go wrong. Through years of helping organizations implement policy as code, we’ve identified four critical pitfalls that can derail your efforts.

The first and most common trap is over-engineering policies. It’s tempting to create sophisticated policies that handle every edge case, but overly complex policies are hard to understand and maintain. Teams won’t trust what they don’t understand. Instead, start with simple, clear policies that address specific risks. Use descriptive naming and provide examples of both compliant and non-compliant configurations. Remember, a simple policy that everyone follows is better than a perfect policy that everyone circumvents.

Equally dangerous is falling into “policy theater”: implementing policies because you think you should, not because they address real risks. This creates friction without improving security. Every policy should trace back to an actual incident, security requirement, or compliance need. If you can’t explain why a policy exists with a concrete example, it probably shouldn’t exist.

Poor developer experience is another common failure point. Nothing kills policy adoption faster than cryptic error messages and unclear remediation steps. When a developer hits a policy violation at 5 PM on a Friday, they need clear guidance on how to fix it. Invest heavily in documentation, helpful error messages, and examples. Consider your policies’ error messages as part of your product’s user experience.

Finally, inadequate testing can erode trust quickly. Policies that break legitimate use cases will be worked around or disabled. Before enforcing any policy, test it thoroughly with real scenarios from your production workloads. Run policies in advisory mode first to catch false positives, and have teams attempt real deployments with policies enabled to ensure they don’t block valid configurations.

The Future of Policy as Code

As infrastructure becomes increasingly complex and distributed, policy as code is evolving beyond simple rule enforcement to become an intelligent layer that understands context and intent. Three major trends are shaping this evolution.

First, we’re seeing the emergence of AI-enhanced policies with smart detection systems that use machine learning to identify anomalies that rule-based policies might miss. These systems learn from your infrastructure patterns and can provide contextual recommendations that adapt based on actual usage. Even more exciting is predictive enforcement: imagine policies that can identify risky patterns before they become violations, guiding developers away from problems they haven’t encountered yet.

Second, the multi-cloud reality is driving the need for universal governance. Organizations need policies that work seamlessly across AWS, Azure, and GCP, with federated enforcement that maintains consistency across multiple cloud accounts and regions. Compliance automation is also maturing, with systems that automatically collect evidence for audits and generate compliance reports without manual intervention.

Finally, policy enforcement is moving closer to where developers actually work. IDE integration will soon provide real-time policy feedback as you write infrastructure code, catching issues before you even attempt to deploy. Self-service exemption workflows will let developers request and receive policy exceptions through automated approval processes. Perhaps most intriguingly, we’re seeing the development of learning policies: systems that improve and adapt based on developer feedback and usage patterns, becoming more helpful over time rather than more restrictive.

Conclusion: Enabling Safe Self-Service at Scale

We started this post with a fundamental tension in platform engineering: the need for both speed and safety. Through the lens of Statsig’s transformation and the technical deep-dive into Pulumi CrossGuard, we’ve seen that this isn’t actually a tension that needs resolving. It’s a false dichotomy that policy as code eliminates entirely.

The key insight from successful platform teams like Statsig is that guardrails don’t restrict freedom; they enable it. When developers know that automated policies will catch dangerous configurations, they gain the confidence to move fast and experiment. When platform teams know that policies automatically enforce security and compliance standards, they can focus on building better platforms instead of reviewing every change. This is the magic of policy as code: it transforms infrastructure governance from a bottleneck into an accelerator.

But perhaps the most important lesson is that policy as code isn’t about saying “no” to developers. It’s about intelligent automation that makes the secure path the path of least resistance. It’s about catching mistakes before they become incidents. It’s about encoding your organization’s hard-won knowledge into systems that help every developer benefit from that experience.

As you embark on your own journey to implement deployment guardrails, remember that perfection isn’t the goal; progress is. Start small, iterate based on feedback, and gradually expand your coverage. Your developers will thank you for the clarity and confidence that comes with well-designed guardrails, and your security team will sleep better knowing that policies are enforced automatically and consistently.

The path from manual reviews to automated guardrails is well-traveled and well-documented. Our complete policy examples provide real-world implementations you can adapt to your needs, while the CrossGuard documentation offers deep technical details for advanced use cases. If you’re on AWS, AWSGuard’s pre-built policies offer immediate value, and our compliance-ready policy catalog addresses specific regulatory requirements.

The future of infrastructure management isn’t about choosing between developer autonomy and operational control. It’s about using policy as code to achieve both, creating platforms that are simultaneously powerful and safe, flexible and compliant, fast and secure.

In our next post, we’ll explore Day 2 Platform Operations, diving into how to maintain infrastructure compliance after deployment and automatically remediate configuration drift. Because getting to production is just the beginning; keeping your infrastructure secure and compliant over time is where the real challenge lies.

Get Started with Pulumi

Use Pulumi's open-source SDK to create, deploy, and manage infrastructure on any cloud.

Governance as an Enabler: Scaling Safely and Confidently

Adam Gordon Bell — Tue, 17 Jun 2025 00:00:00 +0000

In previous articles in this series, we’ve explored how platform engineering transforms infrastructure chaos into consistent provisioning, empowers engineering teams through self-service infrastructure, optimizes workflows, embeds security directly into your platform, and provides observability as a superpower. Each pillar builds upon the previous ones, creating a cohesive foundation that accelerates innovation and productivity.

But as your platform scales, new challenges inevitably emerge. You’ve empowered engineering teams with self-service infrastructure, streamlined workflows, and embedded security directly into your platform. But as your platform scales, new challenges emerge: How do you ensure consistency, compliance, and cost control without slowing your teams down?

In this article, we’ll explore how Platform Engineering transforms governance from a manual, bureaucratic process into an automated, built-in enabler, helping your organization scale safely and confidently. By embedding governance directly into your platform, you can maintain control, ensure compliance, and manage costs effectively, all while preserving the autonomy and speed your engineering teams have come to expect.

The Problem: Governance as a Manual Bottleneck

Dealing with manual compliance checks and red tape

With increased team autonomy and self-service capabilities, how do you ensure consistency, compliance, and cost control across your entire organization?

Governance often feels like a necessary evil: manual, bureaucratic, and slow. Application teams see it as red tape, while operations teams struggle to maintain control. Manual compliance checks, lengthy audits, and unclear or inconsistent policies create friction and frustration. Teams may bypass governance processes entirely, leading to shadow IT, inconsistent resource configurations, and hidden risks.

The consequences of manual, bureaucratic governance are clear:

Increased compliance risks and audit failures: Without automated enforcement, compliance becomes reactive and error-prone, increasing the likelihood of regulatory violations and audit findings.
Unpredictable cloud costs and budget overruns: Without clear guardrails, self-service infrastructure can lead to resource sprawl, wasted resources, and unexpected cloud bills.
Reduced team autonomy and slower innovation: Manual governance processes reintroduce bottlenecks, slowing down deployments and undermining the agility your platform was designed to achieve.

The Solution: Embedding Governance into Your Platform

Successfully scaling safely with embedded governance

Governance should live inside your platform, not off to the side as a separate process. To make that happen, build these four capabilities into your IDP:

Policy-as-Code for Automated Compliance: Declare rules (like approved regions or required tags) as code. The platform enforces them whenever infrastructure is created or updated, so compliance happens automatically.
Platform-Level RBAC for Permission Boundaries: Decide who can act on projects, stacks, and templates before any cloud credentials run. This early check prevents unauthorized requests from ever reaching cloud provider.
Audit Logs and Drift Detection for Real-Time Visibility: Record every deployment, who ran it, and what changed. Continuously compare live infrastructure to the desired state in code. If someone bypasses approved processes, the platform flags it and alerts the team.
Resource Lifecycle and Deployment Controls: Automatically retire idle environments after a set time (Resource TTLs) so forgotten test clusters don’t rack up bills. Also if needed, gate production changes behind lightweight approval workflows. Routine dev or staging updates roll out instantly, but high impact production changes wait until a reviewer signs off.

Let’s dive into each of these.

A. Policy-as-Code for Compliance and Operational Standards: Automating Trust and Consistency

An engineering team is ready to launch a new service. They’ve tested it and everything looks good, until the deployment fails. Not because of a bug. Because it’s targeting an unapproved cloud region.

Now they’re stuck. A compliance review kicks off. Slack threads fly. A ticket gets filed. What should’ve been a smooth release turns into a delay, all because of a policy someone missed.

Policy-as-code prevents this.

When teams deploy something that breaks the rules (like using an unapproved region), the platform blocks it automatically. The error shows up right away, with a clear message. Nothing gets provisioned, and nobody has to file a ticket.

If you’re already using intent-based components (“I need a Java service with Kafka and PostgreSQL”), most details are handled for you: tags, regions, naming. But people still override things. That’s why policy-as-code matters.

Think of it as a safety net. A menu of components handle the defaults. Policies catch anything that slips through. Together, they keep your platform consistent without slowing anyone down.

B. Role-Based Access Control (RBAC): Balancing Autonomy and Control

Stopped by lengthy approval processes

As your platform grows, managing permissions manually gets messy. If an engineer needs to fix a production issue but doesn’t have access, they file a ticket and wait, sometimes for days. Give developers too many rights, and they might change production by accident. Both options slow teams down and increase risk.

The fix is an RBAC model built into your platform. First, the platform decides who can deploy, who can publish components, and who can manage templates. This check runs before any cloud credentials are used, so invalid requests get blocked early. Second, the cloud IAM layer controls which API calls are allowed, like creating an EC2 instance or updating a database.

This pairs well with a two-level intent-based approach. Teams describe what they need (“I want a Python Lambda with an SQS queue”), and the platform enforces access only for users with the right scopes. Everyone gets just enough access to do their job, no more, no less.

A Platform with RBAC makes permissions clear, reduces mistakes, and keeps everything auditable. Devs move fast, spinning up resources as needed, while strong guardrails stay in place. The result is a scalable, least-privilege model that balances autonomy and control, so your organization can grow safely.

C. Auditability, Traceability, and Drift Detection: Ensuring Visibility and Trust

Anxious about audit failures and compliance risks

An ops engineer spots a production database misbehaving. A quick check shows someone changed its configuration outside the approved workflow. Without an audit trail or drift detection, the team scrambles to figure out who made the change and when. Meanwhile, the incorrect setting stays active, posing a security and compliance risk. No one can fix it without guessing.

A platform with audit logs records every action: who deployed, when, and what changed. Drift detection watches live infrastructure and compares it to the desired state in code. If someone bypasses the workflow (say, editing a database setting in the console), the platform flags it and alerts: “User Alice changed max_connections on prod-db-01 at 3:42 PM, which no longer matches the expected state.” Now the team can pinpoint the change, talk to the right person, and revert or update the code, restoring consistency in minutes, not hours.

Together, audit logs and drift detection give you real-time visibility into every change. You stop playing detective. You see who did what, when, and how it deviated from code, all in one place. That transparency speeds audits, catches unauthorized changes fast, and builds trust across teams. With automatic traces of every change, your platform scales without surprises.

D. Resource Lifecycle and Deployment Controls: Scaling Responsibly and Safely

An engineer spins up a test environment, then walks away. Another pushes a change straight to production without review. The abandoned test cluster runs up cloud costs; the unreviewed prod tweak risks an outage. Without automation, both lead to wasted spend and stressful cleanups.

A modern platform handles this with ephemeral environments where possible and approval gates where it matters.

In dev and staging, engineers can move quickly. They can create test or preview environments, often tied to users or pull requests, that shut down automatically after a set time. TTL rules keep things tidy without manual cleanup.

Production, by contrast, is gated. High-impact changes, like modifying a database schema or adjusting a load balancer, require approval. Before anything is provisioned, sign-off is required. Every approval (or denial) is logged: who, when, and why.

This setup keeps development fast and flexible, while making production changes deliberate and auditable. Your platform stays clean, cost-effective, and safe without getting in the way.

Drowning in resource sprawl and unexpected costs

Real-World Example: Governance Enablement in Action

An engineering team opens a pull request for a new customer-facing service. The platform spins up a preview environment using a template with secure defaults, pre-approved modules, and policy checks. CI runs tests and policy checks in preview (names, regions, encryption, compliance) so the team catches issues early. If a rule fails (say, an unencrypted database), the PR fails before it reaches main. After the PR merges to main, it deploys to production, confident all policy validations have passed.

Idle QA environments shut down after 48 hours, so forgotten clusters don’t rack up bills. Sensitive production changes, like updating a load balancer or altering a critical schema, are carefully reviewed via pull request. Once approved, the platform deploys automatically and logs every action. Drift detection flags console edits, letting the team revert or update code in minutes.

Result: Governance becomes an invisible safety net. Engineers move fast, knowing policy-as-code, RBAC, TTL cleanup, approval gates, and change tracking catch mistakes. Ops stays in control without firefighting or chasing orphan resources. The platform scales safely, balancing freedom with built-in guardrails.

Metrics: Measuring Governance Enablement

To ensure your governance practices truly empower your organization, it’s essential to track clear, actionable metrics. These metrics help you understand the effectiveness of your governance processes, identify areas for improvement, and ensure governance remains frictionless and enabling:

Time Spent on Manual Compliance Checks and Audits: Measure how much time your teams spend manually verifying compliance or performing audits. Effective governance automation should significantly reduce this overhead, freeing teams to focus on higher-value tasks.
Number of Compliance Violations or Audit Findings: Track how frequently compliance violations or audit issues occur. Effective governance should reduce these incidents, demonstrating that automated policies and guardrails are working as intended.
Cloud Cost Predictability and Budget Adherence: Monitor how accurately your cloud spending aligns with forecasts and budgets. Good governance practices, such as automated tagging, resource lifecycle management, and cost controls, should improve predictability and reduce unexpected cost overruns.
Engineering Team Satisfaction with Governance Processes: Regularly survey engineering teams to gauge their satisfaction with governance processes. High satisfaction indicates that governance is enabling rather than hindering their workflows.

Tracking these metrics helps you continuously improve your platform’s governance practices, ensuring they remain effective and frictionless.

Working within clear, automated guardrails

Pulumi and Governance Enablement

Pulumi provides built-in governance features that help you scale safely and confidently, embedding compliance, consistency, and control directly into your platform:

CrossGuard (Policy as Code) Define and enforce compliance and operational policies automatically. CrossGuard checks every resource against your organization’s standards before deployment, preventing non-compliant resources and reducing manual audits.
Role-Based Access Control (RBAC) and Teams Manage permissions with precision. Pulumi’s RBAC ensures teams get exactly the access they need, no more, no less, so developers can move quickly within clear boundaries and ops can reduce risk.
Audit Logs and Drift Detection Capture a full history of every change and compare live infrastructure to the desired state in code. Audit logs simplify compliance reviews, drift detection spots unauthorized edits, and teams can fix issues in minutes.
Time-to-Live (TTL) Stacks / Ephemeral Environments Spin up short-lived environments for testing or previews. You can assign a TTL to any stack so it shuts down automatically after a set period. That keeps forgotten test resources from racking up costs and ensures your platform stays clean.

By leveraging Pulumi’s governance features, including CrossGuard, RBAC, audit logs, and TTL stacks, your platform becomes a powerful enabler. You automate compliance, maintain consistency, and empower engineering teams to innovate quickly and safely.

Conclusion: Governance as a Platform Feature

Having autonomy with built-in controls

Governance doesn’t have to slow you down. By embedding governance directly into your platform, you empower engineering teams to innovate quickly while ensuring compliance, consistency, and control. Instead of manual checks, governance becomes automatic, transparent, and frictionless, enabling your organization to scale safely and confidently.

Your engineering teams gain autonomy and speed, your operations teams gain visibility and control, and your organization gains the confidence to innovate at scale.

You’ve now seen all six pillars of a modern internal developer platform—provisioning, self-service, developer experience, security, observability, and governance. If you’d like to see how Pulumi makes building and running a platform like this simpler, check out Pulumi IDP.

Observability as a Developer Superpower

Adam Gordon Bell — Tue, 10 Jun 2025 00:00:00 +0000

Frustratedly trying to figure out what's actually happening

In previous articles in this series, we’ve shown how platform engineering turns infrastructure chaos into consistency, gives teams self-service tools, smooths developer workflows, and bakes security into the platform. Each pillar builds on the last. Together, they create an internal developer platform that cuts friction and speeds innovation.

Even so, teams still face a big challenge: seeing what’s really happening. Whether things go wrong or run smoothly, engineering teams need clear, actionable insights into their systems. Without observability, you end up guessing, reacting slowly, and hunting through scattered data.

This article shows how observability can be a superpower, giving teams the visibility, insights, and confidence to build better software. Embedding observability into your platform lets teams spot, understand, and fix problems fast, turning reactive firefighting into proactive innovation.

The Problem: Data Overload Without Insights

Teams drown in metrics, logs, and traces but lack useful insights. In practice, this shows up in three common friction points:

Tool sprawl: Teams use separate tools for metrics, logs, and traces. They waste time flipping between dashboards and stitching data together.
Alert fatigue: Teams get hit with noisy, context-free alerts. With no clear priority or context, key alerts get lost, causing missed issues or slow responses.
Reactive debugging: Troubleshooting turns into a late-night fire drill. Teams spend hours digging through logs and metrics after users have already noticed the problem.

When observability is limited to post-mortems and fragmented dashboards, your team wastes time reacting instead of preventing problems, and innovation grinds to a halt.

The Solution: Observability as an Engineering Superpower

Exhausted from hours of reactive debugging when the problem could have been caught earlier

The solution isn’t just about bolting on more monitoring tools. it’s about baking visibility, context, and guidance into your platform. To do this, embrace three key principles:

Centralized Service Dashboards & Service List Surface every running service (or database, function, etc.) in one “Services” portal, complete with health badges (CPU, error rate), on-call owner info, and one-click links to that service’s metrics, logs, and traces. By unifying all telemetry behind a single service card, you eliminate context-switching and help engineers find exactly what they need in seconds, not minutes.
Actionable Alerts and Insights Replace vague, noisy notifications with context-rich, prioritized alerts that include severity, correlated root-cause data, and recommended next steps (“Database latency jumped 200% since last deploy: rollback or scale up replicas”). Group and surface only the most critical issues first to reduce alert fatigue and speed up resolution.
Embedding Observability into Engineering Workflows Ship every new microservice with built-in logging, metrics, and tracing by including those hooks in your platform’s service templates and CI/CD pipelines. When instrumentation is automatic, “oops, I forgot to add a span” moments disappear, and teams gain immediate visibility into performance and errors from day one.

When observability becomes a superpower, engineering teams gain the visibility and insights they need to confidently build, deploy, and operate software. Instead of drowning in data, they proactively identify and resolve issues, optimize performance, and innovate with confidence.

A. Centralized Service Dashboards & Service List

Reading the Service Catalog

Imagine you’re paged at 2 AM because “OrderService” is failing, but you don’t know where to look. Metrics live in Grafana, logs are in Elasticsearch, traces in Jaeger, and you still have to hunt down who’s on call. You spend precious minutes clicking through multiple UIs and Slack channels just to figure out who owns the service and where its telemetry lives.

A centralized service list solves this by surfacing every running microservice (or database, or function) in one place. In your platform’s web portal, you land on a “Services” page that shows OrderService alongside CPU and error‐rate badges, an on-call owner, and links to its real-time dashboard, filtered logs, and trace waterfall. No matter which team spun it up, you know exactly where to click: metrics, logs, traces, deployment history, and contact info all live behind a single service card.

By embedding a service list into your platform, you eliminate context switching and reduce onboarding friction. If a service isn’t listed, it isn’t properly instrumented, so gaps stand out immediately. In practice, this “one‐pane‐of‐glass” approach means engineers spend seconds finding the right dashboard and the right person, instead of minutes piecing together fragments across disconnected tools.

B. Actionable Alerts and Insights: Reducing Noise and Accelerating Response

Digging through alert noise

It’s 4 AM and your phone buzzes with three simultaneous alerts, each with vague descriptions like “High CPU usage detected” or “Error rate increased.” Without clear context or recommended actions, you must manually investigate each alert, digging through logs and metrics to determine severity and root cause. This manual triage process is slow, frustrating, and error-prone, increasing the risk of missing critical issues or delaying resolution.

Engineering teams often face a constant barrage of alerts, many of which lack clear context, severity, or actionable next steps. This flood of noisy, ambiguous notifications creates alert fatigue, causing your team to overlook critical issues or waste valuable time investigating false positives.

A platform approach should focus on actionable metrics. Instead of vague notifications, clearly state the issue (“Database latency increased by 200%”), provide relevant context (“Latency spike correlated with recent deployment”), and suggest immediate actions (“Rollback recent deployment or scale database resources”). Additionally, alerts are automatically grouped and prioritized based on severity and impact, ensuring you focus on the most critical issues first.

By holistically approaching alerts and insights, you significantly reduce alert fatigue and noise, accelerate incident response and resolution, and empower engineering teams with greater confidence and autonomy. Your team spends less time manually triaging alerts and more time proactively resolving issues, improving reliability, productivity, and overall team satisfaction.

C. Embedding Observability into Engineering Workflows: Visibility from Day One

At 2 AM, where are the logs for this service.

You’ve just deployed a brand-new microservice to production, only to discover performance issues or unexpected behavior. Sure, you should remember to add tracing, logging, and metrics by hand, but in practice, things slip through the cracks. It isn’t until real-world traffic hits that you realize you forgot to instrument X or Y, and now you’re scrambling to retroactively add code, redeploy, and wait for data to appear, delaying resolution and frustrating your team.

If your platform’s service templates already include all the necessary logging, metrics gathering, and tracing out of the box, it makes life a lot easier. Embedding observability into those templates and engineering workflows ensures every new microservice ships with built-in instrumentation.

This proactive approach reduces “oops, I forgot” moments, accelerates troubleshooting, and increases team productivity and satisfaction, ultimately improving the reliability and quality of your software.

Real-World Example: Observability Superpower in Action

Applying an Actional Metric

At 3:15 AM, your pagerduty alert goes off: “CheckoutService latency spiked 150%.” You log into your platform’s Services portal and immediately see CheckoutService highlighted with a red latency badge and Todd Rivera listed as the on-call owner. Rather than scouring multiple dashboards, you click its service card to jump straight to the metrics, logs, and trace views.

The alert itself is remarkably precise: “CheckoutService latency rose 150% at 3:10 AM following the v2.3.1 deployment. PaymentGatewayService upstream error rate jumped from 0.2% to 2.3%. Recommendation: rollback v2.3.1 or scale PaymentGateway pods. Contact Todd Rivera.” Instantly, you know where the problem lies, which upstream service is impacted, and what the next step should be. In the trace waterfall, you spot a 200 ms delay on CheckoutService calls to PaymentGateway. The logs, automatically instrumented by your service template, filter to TimeoutException entries all timestamped at 3:10 AM. Opening the “Ask Platform” AI widget, you type, “Why did CheckoutService latency spike at 3:10 AM?” The AI responds: “Likely cause: v2.3.1 added index idx_created_at to PaymentGateway’s transactions table, causing an 80 ms delay per request. Rollback v2.3.1 or patch queries to remove the new index.”

Armed with this precise diagnosis, you open a rollback pull request and, after Todd’s ok, deploy it within minutes.

CheckoutService latency and PaymentGateway errors immediately return to baseline. By moving from alert to resolution entirely within the platform (thanks to built-in instrumentation, actionable alerts, and AI-driven analysis, you’ve squashed a major incident before most users ever noticed.

Metrics: Measuring Observability Enablement

To ensure your observability practices truly empower engineering teams, it’s essential to track clear, actionable metrics. These metrics help you understand the effectiveness of your observability tools and processes, identify areas for improvement, and demonstrate the tangible impact observability has on your organization.

Key metrics to measure observability enablement include:

Mean Time to Detection (MTTD):
How quickly are issues identified after they occur? Effective observability should significantly reduce the time it takes to detect problems, enabling faster responses and minimizing user impact.
Mean Time to Resolution (MTTR):
How quickly are issues resolved once detected? With clear, actionable insights and unified observability, your teams should resolve issues faster, reducing downtime and improving reliability.
Engineering Team Satisfaction with Observability Tools:
Regularly survey your teams to gauge their satisfaction with observability tools and workflows. Higher satisfaction indicates that observability is effectively embedded into engineering workflows, reducing friction and increasing productivity.
Adoption Rate of Observability Tools and Dashboards:
Track how widely observability tools and dashboards are adopted across teams. Increased adoption indicates that your teams find these tools valuable, intuitive, and helpful in their daily work.
Reduction in Alert Noise and False Positives:
Measure the volume and accuracy of alerts over time. Effective observability should reduce noisy, irrelevant alerts, ensuring your teams focus on meaningful, actionable notifications.

Tracking these metrics helps you continuously improve your observability practices, ensuring they remain effective and empowering. By regularly reviewing and acting on these insights, you can proactively enhance team productivity, reliability, and overall satisfaction.

Pulumi and Observability Enablement

Platform insights

Pulumi’s platform features let you explore observability without bolting on extra tools. Key Pulumi features that enable observability include:

Pulumi Insights:
Provides unified visibility and powerful search across all your cloud resources. Your teams can quickly discover, explore, and understand their infrastructure, eliminating manual searches and reducing cognitive load.
Centralized Service List: Pulumi IDP’s Services portal gives you a single place to register each microservice, database, or cloud resource and link to its dashboards, logs, and traces.
Pulumi Co-Pilot:
Delivers AI-powered troubleshooting and insights directly within your workflows. Your teams can ask natural-language questions about their infrastructure (such as “What infrastructure changed yesterday?”) and receive immediate, actionable answers.
Built-In Instrumentation via IDP Components:
When you author components and templates in Pulumi IDP, you can bake in standard logging, metrics, and tracing hooks. Every service spun up from those templates ships with consistent instrumentation on day one.

With Pulumi, observability can become an integrated part of your platform, accelerating innovation, improving reliability, and empowering engineering teams to confidently build, deploy, and operate software.

Conclusion: Observability as a Platform Feature

Happy resolving an incident in minutes.

Observability isn’t just about plugging in more tools. It’s about baking-in consistent instrumentation, measurement, and context so every engineer (platform, DevOps/SRE, or application) knows exactly where to look and how to act.

Service-Templates with Built-In Telemetry By providing service templates that already include logging, metrics gathering, and tracing, you eliminate “Oops, I forgot to instrument X” moments. Every new microservice inherits a standard setup, so you never have to retroactively add code or scramble when traffic first hits production.
Consistent Service Dashboards & Centralized Service List Instead of hunting across eight different dashboards, engineers always start from a single “Service List” page. From there, one click takes them to that service’s metrics overview, log stream, or trace waterfall. This unified entry point reduces cognitive load and cuts straight to “Where’s the problem?”
Measuring Alert Quality and Actionability A truly mature platform doesn’t just send alerts. It tracks whether those alerts are helpful or noise. By measuring “ratio of actionable alerts vs. false positives,” you continuously fine-tune thresholds and eliminate alert fatigue. The result? Engineers trust their notifications and respond faster to real incidents.
AI-Driven Context and Natural-Language Troubleshooting On top of unified telemetry and alert quality metrics, AI can instantly correlate recent deployments, configuration changes, and error spikes. Engineers can ask, “Why did latency jump at 3 AM?” or “What changed in production last night?” in plain English, and the platform provides a clear, context-enriched answer. This additional layer turns reactive firefighting into proactive problem prevention.

When you combine these elements (components and templates, a single service dashboard, alert quality measurement, and AI/natural-language querying), you transform observability into a genuine superpower. Issues are spotted, triaged, and fixed before customers even notice.

Next time, we’ll dive into the final pillar, Platform Governance, showing how to enforce policy, manage costs, and keep your platform secure and compliant as it scales.

From Configuration Chaos to Programming Languages

Daniel Ward — Tue, 03 Jun 2025 00:00:00 +0000

This post is based on our video interview with Daniel Ward, a Software Developer/Consultant at Lean TECHniques and Microsoft MVP. Daniel shares his strategies for moving teams from configuration chaos to programming languages for infrastructure, including the 10% rule for change, viral adoption, and why Pulumi beats YAML for complex systems. Watch the video below or read on. - Adam Gordon Bell

Imagine your infrastructure codebase has evolved from a few files to thousands of lines across dozens of configuration files. Your team spends hours deciphering CloudFormation templates, your AWS CDK deployments take 30+ minutes, and introducing any change becomes a multi-day effort. This is the reality many organizations face when their infrastructure-as-code approach outgrows their tooling.

As a consultant and developer who helps organizations improve their software delivery processes, I’ve seen firsthand how the right technologies—specifically using actual programming languages like C#—can transform infrastructure management instead of forcing teams to use declarative configuration files.

My Background: Coaching Teams Through Technical Transformation

My background is primarily as a .NET developer—I’m now a Microsoft MVP in my first year—though I work with TypeScript and other languages as needed. About six years ago, I moved into consulting, or what we call “coaching” at Lean Techniques.

The idea isn’t just to write code for clients or tell them what to do and leave. Our goal is that once we leave, they don’t need us anymore. We’re essentially trying to put ourselves out of a job.

Organizations bring me in for various reasons—from implementing automated testing after costly failures to migrating from mainframes to the cloud. Sometimes it’s about product delivery challenges, like month-long sprints that never complete on time.

It’s about bringing places up to a modern standard. As we say at Lean Techniques, turning IT from a cost center into a competitive advantage.

If you’re ready to move from sprawling configuration files to code-centric infrastructure, here are eight practical steps to guide you:

1. Follow the 10% Rule for Sustainable Change

One of my guiding principles comes from Gerald Weinberg’s book “The Secrets of Consulting”: the 10% rule. When entering a new team or organization, don’t try to improve more than 10% at a time.

There are a few reasons for this. One is that you can actually end up making the client look bad. If you improve everything at once, their boss might ask why this consultant fixed everything in six months when the team couldn’t fix it in five years.

But there’s a deeper reason related to social dynamics. Any social group, in programming or outside of work—if you come in new and try to change how the group works before familiarizing yourself with it, people will be resistant. They’ll ask “Who do you think you are? You just got here!” And they’re right—you could be wrong about some things. You have to understand how they work to understand what the problems are.

This applies directly to introducing new technologies like infrastructure as code. Rather than overhauling everything at once, I recommend starting small.

If a place is completely new to infrastructure as code, I’ll tell them to start small. None of their engineers know how to do this yet, and I’m not going to do all the work for them and leave, because then none of their engineers will know how to maintain it.

2. Quantify the Cost of Inaction to Overcome Resistance

Even with a measured approach, resistance is inevitable. I’ve experienced teams that are exceptionally open to improvement, often coming from difficult situations and eager for change. I’ve also worked with teams that resist most suggestions, where engineers feel the consultant was forced upon them by management.

It’s always easier if you can do pull-based versus push-based change—where the team has asked for your help instead of the manager telling them “I’m inflicting this help on you.”

For those looking to influence change within their organization, I’ve found that influencing upper-level management comes down to numbers. They deal with too many teams to get into specifics—they want numbers. If you can show how many hours your team spends each week manually pushing code or waiting on another team, and multiply that by engineer salaries, you’re showing how much money is being wasted.

This approach is far more effective than what can come across as “Here’s a new tool I want to play with”—an anti-pattern that reinforces the stereotype of technical people just wanting to use new technologies.

3. Build Consensus with Nemawashi Conversations

I’ve found a Japanese concept called “nemawashi” (literally “turning the roots”) provides another effective approach to organizational change:

Before introducing an idea to a manager or the whole team, individually go to every person on the team first and get their thoughts. If most people respond positively, you can then approach the decision-maker saying, “Everyone on the team wants this—what do you think?”

This isn’t just about collecting checkmarks but genuinely gathering feedback. If you’re getting pushback from every single person, maybe it’s not as great an idea as you thought. Either way, you learn something from it.

4. Drive Viral Adoption Through Early Wins

Rather than forcing change on resistant teams, I prefer what I call “viral adoption”:

We try to work with teams that ask for our help instead of being pushed onto teams. People are way less resistant if you can show them you’re making their job easier. We work with one team who wants our help, make them really effective, and then other teams notice: “How come they’re releasing so many times a week? How do they have so few defects? How is their pipeline so fast?”

This success creates natural curiosity, leading other teams to either request similar help or study and adopt the successful team’s practices. Either way, I’m happy that things are improving.

5. Align Dev and Ops with Infrastructure as Code

When it comes to infrastructure as code specifically, I’ve noticed it’s becoming table stakes for many organizations, though implementation quality varies widely.

It’s hard to have actual continuous delivery without infrastructure as code. Without it, developers can’t define their own infrastructure and have to wait on another team to do it. By continuous, I mean teams should be able to deploy every 30 minutes if they want. If you have to wait one to three business days for the operations team to create your resources, that’s not continuous.

Resistance to infrastructure as code typically comes from two directions:

From developers: “I already have my plate full—now I have to learn cloud and deployment too?”
From operations: “That’s my job—I don’t want to lose it.”

I see this as a win-win opportunity for both sides. For operations teams, the job shifts from mundane tasks like “create a lambda for me” or “replace this code bundle” to more creative platform engineering work. For developers, the benefit is clear: no more waiting on operations to proceed with their work.

6. Invest in Platform Engineering for Reusable Components

Platform engineering has emerged as a solution to these challenges. Instead of every team reinventing solutions for common needs like logging, monitoring, or deployment pipelines, platform teams create standardized, reusable components.

It seems wasteful if every single team has to manage their own side of the Splunk instance or Dynatrace. These are solved problems that every team needs. A platform team can look across the organization, see what can be standardized, and remove those problems from individual teams.

This approach is particularly powerful with tools like Pulumi, which allows creating reusable component resources for common patterns: “It’s very common for teams to create a Lambda-backed API Gateway or a Lambda hooked up to SQS. These are super common things, so here’s a component resource for that so you don’t have to write it yourself.”

The key difference in mindset is that platform teams aren’t enforcing rules—they’re building tools that make developers’ lives easier. Instead of saying “You must do logging this way,” it’s “Here’s an endpoint you can use—you’re good to go.”

7. Avoid YAML Pitfalls: Embrace Programming Languages

One of my pain points is the limitations of configuration languages like YAML for complex infrastructure. I recently gave a presentation about a GitHub Actions workflow that grew to over 1,000 lines, calling shared actions that call other shared actions, making it nearly impossible to understand what’s happening.

I couldn’t use breakpoints or the normal libraries I would use. I’ve seen a 69,000-line YAML file for Prometheus and Kubernetes—that’s an auto-generated example, but even a thousand lines calling several other files is too much.

The fundamental issue is abstraction. Programming languages allow loops, conditionals, and functions to remove repetition and build larger structures. Configuration languages often lack these capabilities, leading to copy-pasting and unmanageable growth.

With declarative approaches, you end up using a lot of magic strings. It’s easy to forget one when you change values. The tooling is also an issue—I’m familiar with IDEs and libraries for programming languages, but the equivalent tools for YAML just aren’t as mature.

8. Choose Pulumi for Flexible, Code-First IaC

When comparing infrastructure-as-code options, I see three major players: AWS CDK, Terraform, and Pulumi. While AWS CDK offers similar capabilities to Pulumi for AWS resources, it relies on CloudFormation under the hood:

CloudFormation is just a black box—it’s proprietary. If something goes wrong, you have to hope someone’s already asked about it on GitHub. When deployments take 15-30 minutes, there’s not much you can do—it’s just how CloudFormation works.

Compared to Terraform, which uses HCL (HashiCorp Configuration Language), Pulumi offers the flexibility of actual programming languages:

There’s nothing wrong with declarative approaches in some cases, but at a certain point—when you have big systems and chunks of resources you want to share between teams—it becomes more cumbersome than helpful. That’s why Terraform is working on their own CDK now.

What sets Pulumi apart is its flexibility: Even if you have platform teams who aren’t developers and want to set standards, they can use YAML with Pulumi and then transform those component resources into other languages for dev teams to use. You get the best of all worlds.

Start Small, Think Big: Small Changes, Big Impact

Whether you’re introducing infrastructure as code or any significant technical change, my advice remains consistent: start small, focus on solving real problems, and let success spread naturally.

By understanding the human elements of change—the 10% rule, nemawashi, and viral adoption—you can transform resistance into enthusiasm. And by choosing tools like Pulumi that bridge the gap between developers and operations, you can create infrastructure that’s both powerful and maintainable.

I’m currently writing a blog series on Pulumi at daninacan.com. I also speak at conferences occasionally, so check out my speaking page if you’re interested.

Ready to escape configuration chaos? Start small, focus on solving real problems, and get started with Pulumi to leverage the full power of programming languages for infrastructure. Join our community Slack to connect with engineers making the same transformation.

Platform Engineering Buffet at SEITENBAU

Nico Thomas — Tue, 27 May 2025 00:00:00 +0000

This post is based on my video interview with Nico Thomas, a platform engineer at SEITENBAU. In this article, Nico shares how he and his team built a flexible, reusable platform to support 20+ diverse projects using Pulumi. Watch the video below or read on for Nico’s take on building an infrastructure buffet. - Adam Gordon Bell

Imagine your company isn’t Spotify with one product pipeline, but 20 independent government projects running on Ansible-provisioned VMs—some on Kubernetes, some delivered into other customer data centers, some operated by clients themselves. Each team is building its own CI/CD, secrets management, and integrations. You’re left with a million reinvented wheels.

That was the situation my team and I faced at SEITENBAU, a German software company specializing in custom development for both public and private sectors. Unlike the typical platform engineering stories—where teams build infrastructure for a single product with multiple microservices—we had a fundamentally different challenge: How do you build a platform that serves dozens of completely independent projects, each with their own technology choices, deployment targets, and operational models?

The answer? Stop trying to force everyone onto the same plate. Instead, build a buffet.

The Buffet Philosophy

Think of a traditional platform as a set menu—also known as prix fixe—where everyone gets the same courses, prepared the same way, in the same order. But what if your diners include vegans, carnivores, and people with various dietary restrictions? What if some want a quick snack while others need a seven-course meal?

That was exactly the situation we faced. We had:

Government projects that needed to run on-premises due to data sovereignty requirements.
Modern cloud-native applications leveraging Kubernetes and GitOps.
Legacy systems still operating on virtual machines managed with Ansible.
Projects we ran as managed services and others we simply delivered to customers.
Teams using various CI/CD tools, artifact repositories, and deployment strategies.

A one-size-fits-all platform would have been a disaster. Instead, my team built a platform offering a rich selection of pre-configured, production-ready components. Teams could mix and match according to their specific needs, creating their own unique infrastructure combinations.

The Challenge: Beyond Microservices

Most platform engineering resources assume you’re building infrastructure for a single product – Spotify’s Backstage, for example. But what happens when you’re dealing with over 20 completely independent projects, each with distinct requirements?

Each project is distinct, featuring its own:

Technology stack (Java predominates, but we support various frameworks and languages).
Infrastructure requirements (cloud, on-premises, or hybrid setups).
Security and compliance constraints.
Operational models (managed by us, the client, or jointly).
Release cadences and deployment strategies.

Initially, this diversity seemed chaotic, but we quickly noticed that underneath these variations, common patterns emerged. Every project consistently required source control, CI/CD pipelines, artifact storage, secrets management, and deployment targets. The critical realization was that we didn’t need to standardize what teams chose—we needed to standardize how they connected these building blocks.

That’s where Pulumi became essential. Its flexibility allowed us to construct an infrastructure buffet that genuinely accommodated the diverse appetites and needs of every team.

Building the Kitchen: Architecture with Pulumi

Behind every great buffet is a well-organized kitchen. For our infrastructure, that kitchen is powered by Pulumi and Python, orchestrating a complex web of infrastructure services, configurations, and deployments. Using Pulumi together with Python has proven extremely valuable—it makes our day-to-day tasks easier to maintain and understand.

The Multi-Stack Approach

Rather than creating a single monolithic infrastructure program, we adopted a modular approach with multiple Pulumi stacks. This decision was essential for managing complexity and empowering team autonomy.

Our architecture comprises several specialized Pulumi programs:

GitLab System: Manages installations and configurations of GitLab.
Project Configuration: Sets up team-specific GitLab groups, CI/CD runners, and agents.
Cluster Management: Provisions Kubernetes clusters complete with standard add-ons.
VM Infrastructure: Manages traditional VM-based deployments utilizing Ansible.
Platform Services: Configures shared services such as artifact repositories and identity providers.

Each program can maintain multiple stacks (dev, staging, prod), allowing us to manage various environments independently while leveraging shared infrastructure code.

From Ingredients to Ready-Made Dishes: Reusable Components

The real magic happens in our component library. Just as a buffet offers complete dishes rather than raw ingredients, we’ve built high-level components that encapsulate complex infrastructure patterns.

Example: The Kubernetes Storage Component

Consider our Longhorn storage component. Instead of requiring teams to understand Helm charts, storage classes, and snapshot configurations, they simply declare they need persistent storage.

The component handles all the complexity: installing Longhorn, configuring appropriate replication for test vs. production environments, setting up snapshot classes, and integrating with our internal certificate authority.

Two Years Later: The Buffet is Open

After two years of running this platform in production, we’ve learned what works and what we’d do differently.

Building a platform for diverse, independent projects requires a different mindset than traditional platform engineering. Instead of forcing standardization, we’ve embraced flexibility while maintaining consistency where it matters.

The buffet approach—powered by Pulumi’s flexibility and Python’s expressiveness—has allowed us to serve 20+ different projects efficiently while giving teams the autonomy they need. It’s not about having one perfect dish; it’s about having a kitchen flexible enough to satisfy everyone.

If you’re facing similar challenges with diverse project requirements, consider building a buffet instead of a prix fixe menu. Your teams will thank you for it.

Want to build your own infrastructure buffet? Get started with Pulumi and join our community Slack to connect with engineers solving similar challenges.

Platform Engineering: Evolution or Rebranding?

Adam Gordon Bell — Mon, 12 May 2025 00:00:00 +0000

2 Months ago, user deacon91, after years of working in the industry has declared DevOps to be a dead end on Reddit’s r/devops. And I’ve been thinking about his thoughts on the industry since then.

His point was that DevOps, the breaking down of silos between development and operations, had been tried. We learned the lessons of this approach and need to do something better: Platform Engineering.

And platform engineering has certainly emerged as a new hot area. But is it truly an evolutionary step toward a greater way of developing and operating software? Or is it, as others in that same Reddit group declare, merely a fad and a rebranding of existing ideas?

That’s what I want to explore: is platform engineering a rebrand or a genuine evolution?

Let’s start by summarizing the cynical case - that it’s merely a rebrand.

The Rebrand Case

Title Drift

This title drift happens in many fields. Just as the garbage man becomes the sanitation worker so does the sysadmin becomes the DevOps engineer.

This creates title inflation in our competitive industry. People who job-hop regularly update their LinkedIn profiles from “business analyst” to “product manager.” Meanwhile, the person who stays put for years might still be called a “programmer” while everyone around them becomes a “Staff-plus Senior Backend Engineer.”

That ops wizard who’s been at the same company for eight years? Still an “Operations Engineer.” Their friend who switches jobs every 18 months? Now a “Senior SRE” making 50% more money.

Title drift becomes a self-reinforcing cycle. When the recruiter discovers that the best DevOps candidates now call themselves “Platform Engineers,” your company’s job postings change overnight—even if no one plans to build an actual platform.

This is how you end up with people like u/hajimenogio92 doing the exact same sysadmin work they’ve always done, just with “Platform Engineer” on their LinkedIn profile.

This is one aspect of what people mean why they say “platform engineering is just a rebrand.” The other thing they sometimes mean is that it’s a fad everyone is chasing.

Hype Cycle Reboot

Tech movements decay in predictable ways. Remember Extreme Programming? It began as a revolution to empower developers. Then it morphed into Agile, and in some places, into the 12 steps, 7 competencies, and many processes of SAFe.

The Scaled Agile Framework® (SAFe®) is a set of organization and workflow patterns for implementing agile practices at enterprise scale.

Maybe SAFe makes sense for your work place, but it’s certainly a lot more complex than it’s root and this pattern is what skeptics see in platform engineering.

DevOps started at grassroots conferences with a simple message: “Let’s tear down the wall between developers and operations.” But once it peaked, everyone scrambled to slap “DevOps” on whatever they were already doing. Just like companies today adding “AI” to their products.

Eventually, the original movement gets so diluted that we need a fresh start. New terminology. New conferences. New keynote speakers recycling old ideas with updated slides. New vendors selling solutions to problems you didn’t know you had.

In this view, platform engineering isn’t revolutionary—it’s just the next square on the buzzword bingo card. DevOps plateaued. SRE plateaued. The Kubernetes experts ran out of things to say. So the industry needed something new to hype.

There’s some truth to this cynicism, but it blinds us to what platform engineering actually offers. More importantly, this view misses something crucial: where a platform approach succeeds, it reveals fundamental flaws in the original DevOps vision.

The Case for Evolution

The original DevOps movement introduced valuable ideas about breaking down silos and encouraging collaboration. However, its purest vision—where “everybody does everything”—faces practical limitations at scale.

Totally cross-functional teams work well at small organizations but struggle when engineering teams grow beyond 20 people because:

People naturally have different specializations and preferences
Few individuals want to perform both development and operations roles
Staying current on everything in your Ops and Dev stack is often unrealistic

But most importantly, as organizations grow, specializations natuarally emerge. Governance and corporate policies often force the issue. A platform team is one such specialization and a powerful one if done right.

Instead of focusing on eliminating silos, it focuses on creating productive abstractions and interfaces between application teams and operating services. The platform functions as an abstraction layer (similar to how a database abstracts data storage), with dedicated teams treating it as a product with clear boundaries.

This approach contradicts the original DevOps vision but represents a necessary evolution. Platform Engineering acknowledges inevitable organizational divisions while still enabling collaboration. It leverages skills and organizational workflows from application development and product management to build a way to operate software effectively.

Why not both?

So yeah, title drift is a thing. And the hype cycle is real. But the idea of a dedicated team removing frictions by building tools and reusable abstractions is an idea that’s here to stay.

In fact, it’s just a way to scale the principles inherent in the original ideas of DevOps. It may technically diverge from the literal principles of DevOps, but it is most definitely true to the spirit.

At Pulumi, we see this evolution firsthand. Our most forward-looking users aren’t chasing titles—they’re building internal platforms that empower teams, reduce friction, and let developers focus on what matters most. That’s why we’re investing in platform team features.

Not because it’s trendy—but because, done well, it works. And I think that’s the insight deacon91 was getting at.

Get a Free Platform Engineering Guide →

Security as an Enabler: Building Trust into Your Platform

Adam Gordon Bell — Fri, 11 Apr 2025 00:00:00 +0000

In previous articles, we looked at how platform engineering fixes infrastructure chaos, enables self-service, and improves developer workflows. These pillars work together to boost both developer productivity and organizational speed.

But there’s still one critical element that can make or break all this progress: security.

Traditional security efforts — even “shift-left” initiatives — often create friction instead of clearing the way for innovation. Embedding security directly into your platform changes that. By weaving in policy-as-code, centralized secrets management, and identity-based authentication, you turn security from a blocker into an enabler. And with the right metrics, you can measure how well your platform balances protection and speed.

When security is baked into your internal developer platform, it stops being a gatekeeper. Developers move faster — and with more confidence.

Done right, your teams gain autonomy without compromising safety. Security people become partners instead of blockers. Your organization builds trust that speeds up innovation rather than slowing it down.

The Problem: Security as a Gatekeeper

Traditional security practices often create friction rather than enable innovation. Developers run into several pain points: last-minute security reviews that delay deployments, unclear policies that feel arbitrary, and security teams that act more like blockers than partners.

This friction has real consequences. When security becomes a bottleneck, developers find ways around it. They take shortcuts. All of this increases risk through inconsistent security practices.

The end result is that innovation slows down. Developer autonomy shrinks. And the agility your platform was supposed to deliver disappears.

Many organizations try to fix this by “shifting security left” - moving security checks earlier in development. But just dumping security responsibilities onto developers isn’t the answer. It overwhelms them and can still create inconsistencies.

The real solution isn’t just about changing when security happens. It’s about changing how it works.

Key Security Functions within a Platform

Effective platform security involves several key functions, regardless of how they are distributed across teams or individuals in your organization:

Defining Security Strategy & Policy:
- Developing and maintaining security policies, standards, and architectural best practices tailored to the organization.
- Providing security guidance, education, and support across engineering teams.
- Performing threat modeling, security architecture reviews, and proactive security assessments.
Monitoring & Responding to Threats:
- Actively monitoring systems to detect security events and potential threats in real-time.
- Managing security tooling (like SIEMs, vulnerability scanners) and orchestrating incident response procedures.
- Investigating incidents, coordinating remediation efforts, and managing security automation.
Integrating Security into Development & Infrastructure:
- Embedding security controls directly into the development lifecycle and infrastructure provisioning processes.
- Implementing secure-by-default configurations, automated security checks (SAST, DAST, SCA), and policy-as-code guardrails.
- Ensuring security considerations are built into CI/CD pipelines, service templates, and infrastructure modules from the start.

Collaboration between these functions is crucial. A successful platform ensures that policy definition, operational monitoring, and practical implementation work together seamlessly to create a secure environment without hindering velocity.

The Solution: Embedding Security into Developer Workflows

The key to solving security friction isn’t just adding more checkpoints earlier. It’s weaving security directly into your platform and your developers’ daily work.

Adding automated scans to your CI/CD pipeline is a good start. Tools like Dependabot, Snyk, and Trivy catch problems early and alert developers quickly. But these “shift-left” checks alone aren’t enough. Developers still need to interpret findings and fix issues, often without adequate support.

Platform engineering takes a different approach. It builds security in from the start, rather than treating it as a final hurdle or dumping complex responsibilities on developers. It uses secure-by-default components and automated guardrails that work continuously.

Let’s see what this looks like in practice.

Policy-as-Code for Security: Automating Trust

If you’ve implemented the intent-based approach we covered earlier, your developers already use secure-by-default modules. They simply request what they need (“I need a Java service with Kafka and PostgreSQL”), and the platform handles security details like IAM roles, encryption, and permissions.

But secure defaults aren’t enough. You still need automated guardrails. Policy-as-Code adds this enforcement layer, keeping security standards consistent—even when someone tries to bypass or change the modules. It’s a backup system: secure modules provide your first defense, and Policy-as-Code catches anything that might slip through.

For Example:

IAM Roles with Least Privilege:
- Secure Default (Module): The platform team’s IAM module automatically generates roles with least-privilege permissions. Developers don’t specify IAM roles directly.
- Policy-as-Code (Additional Guardrail): A policy explicitly checks all IAM roles provisioned by the platform, ensuring no role grants overly broad permissions (e.g., no wildcard permissions). If a module is accidentally modified or misconfigured, the policy catches it immediately.
Mandatory Encryption at Rest:
- Secure Default (Module): The platform team’s database module automatically provisions databases with encryption at rest enabled. Developers don’t specify encryption settings directly.
- Policy-as-Code (Additional Guardrail): A policy explicitly checks all storage resources to ensure encryption at rest is always enabled. If a developer or platform engineer accidentally modifies the module or bypasses it, the policy immediately flags the issue.

When you add Policy-as-Code to your platform, security becomes automatic and clear. Developers get instant feedback they can trust, and security teams maintain control without slowing things down. Your teams can move fast without taking shortcuts that create risk.

Protecting Sensitive Credentials by Default

Policy-as-Code secures your infrastructure, but you still need to protect how applications access sensitive resources like databases and APIs.

Here’s what often happens: A developer copies database passwords into a local .env file. They accidentally commit this file to Git, exposing the credentials to anyone with repository access. Or they inadvertently bake these secrets into a Docker image that others can download. Worse yet, if their laptop gets hacked, attackers can use these credentials to break into production systems.

When these breaches happen, everything stops. Teams scramble to rotate credentials, investigate the damage, and fix systems—creating delays and increasing risk.

Centralized secrets management provides a robust solution to credential security. It stores sensitive information in a secure vault and offers controlled access methods at runtime. Developers work with the secrets management solution instead of handling credentials directly, which significantly reduces exposure risk.

Platform teams build these secure patterns into their service templates. They create clear pathways for accessing credentials that guide developers toward secure practices in both development and production environments.

With this approach, your deployments become both faster and safer. Developers follow simpler, more consistent security procedures, while security teams gain confidence that credentials are properly protected throughout your systems.

Identity and Access Management: Frictionless Security through Identity

Your platform must also solve another security problem: how people and systems safely access cloud resources.

Developers hate juggling multiple credentials—cloud provider keys, passwords, or tokens. These credentials can expire, get lost, or leak, causing headaches and security risks.

Consider a common scenario: A developer trying to fix a broken production system finds their credentials have expired. Now they’re stuck hunting for new credentials instead of fixing the problem.

Identity-based authentication fixes this. Developers just use their regular login. When developers log in normally, they automatically get secure access to cloud resources through OIDC or SAML. No more managing credentials. Access stays secure and current.

This approach simplifies access and removes barriers. Developers get secure access to resources when they need them, speeding up work without weakening security.

Identity-based authentication, combined with policy-as-code guardrails and centralized secrets management, creates a comprehensive security system that protects without impeding work. To see how these practices function together, let’s look at a real example.

Real-World Example: Security Enablement from Build to Incident Response

A developer deploys a new microservice. They select a secure template from the service catalog. Policy-as-code instantly checks that encryption and permissions are set correctly, and credentials are setup in a centralized secrets management environment.

As the service builds, automatic scans check dependencies and container images for vulnerabilities. The system creates a dependency list (SBOM), tracking every component used. Build pipelines verify and sign code, so only trusted code reaches production.

Once deployed, security monitoring begins automatically, showing the service’s security status in real time. Later, someone discovers a vulnerability in a library used by the microservice. Thanks to the dependency list, the security team immediately knows which services are vulnerable. Monitoring spots suspicious activity and sends clear alerts to the team’s dashboards.

Clear incident response plans help teams contain and fix the problem. The team patches the vulnerable component, using the same secure pipelines to quickly roll out the fix.

In this scenario, security helps rather than blocks at every stage. From building to responding to incidents, your platform helps teams build secure software quickly.

But how do you know if your security practices are truly enabling your teams? To answer that, you need clear, actionable metrics.

Metrics: Measuring Security Enablement

To check if your security approach works, track two things:

Security Incident Rate:
Count how many security problems reach production. Fewer incidents means your security is working.
Developer Security Friction Score:
Include security questions in your regular developer surveys. Ask how much security processes slow their work or cause frustration. Improving scores show that security fits naturally into their workflow.

These two measures combined tell you your security works and at what cost, so your team can build good software fast.

Pulumi and Security Enablement

Achieving these goals, a decreasing incident rate coupled with a low developer friction score, requires a platform built with the right foundations. Pulumi helps you construct such a platform by providing built-in capabilities that directly embed security into your infrastructure workflows:

Policy as Code (CrossGuard): Automatically enforce security and compliance standards.
Secure Secrets Management (Pulumi ESC): Centralize and securely inject secrets without manual handling.
Identity-Based Authentication: Simplify secure access to cloud resources using existing identities.

With Pulumi, security becomes an integrated, frictionless part of your platform—accelerating innovation while building trust.

Security as a Platform Feature

Security can help, not hinder. By building security into your platform, you help developers work faster and safer. Security becomes a feature, not an obstacle.

Security completes the platform foundation we’ve built in previous articles on Provisioning, Self-Service, and Developer Experience, creating a platform that speeds up development.

But developers still need to see what’s happening in their systems. Next time, we’ll explore how Observability helps developers spot and fix problems quickly.

Improve Developer Experience: Increase Dev Productivity with Internal Developer Platforms

Adam Gordon Bell — Thu, 13 Mar 2025 00:00:00 +0000

In the last article in this Platform Engineering Pillars series, we explored how self-service infrastructure frees developers from bottlenecks and dependency gates. By providing reusable infrastructure modules and intent-based configurations, platform teams dramatically reduce infrastructure friction. This self-service model powers faster deployments, increased autonomy, and fewer delays.

However, infrastructure provisioning alone isn’t enough to improve developer experience. Even with efficient provisioning, developers can still face inconsistent local setups, sluggish CI/CD pipelines, poor documentation, and fragmented tooling. These obstacles quietly reduce developer productivity, slow developer velocity, and increase operational overhead.

Your platform’s ability to support daily workflows directly impacts core success metrics like time to first commit, time to production, and developer satisfaction.

When improving developer experience as part of a platform engineering initiative, measure:

Time to First Commit – How quickly a developer can become productive in a new environment or project. Pay particular attention to new hires.
Time to Production – How fast code moves from commit to successful production release.
Developer Satisfaction – Regularly survey friction points and prioritize fixing the top issues.

Friction logs are also a powerful way to identify roadblocks, recording every obstacle encountered during common workflows. This reveals hidden friction and provides a roadmap for improvement.

A great developer experience accelerates iteration, improves problem-solving, and drives better products. An internal developer platform (IDP) designed for DevEx is a fundamental competitive advantage.

The Service Catalog: Foundation for Developer Experience

Developer friction often starts with something simple: not knowing what’s already available or who owns it. Without a single, reliable source of truth, developers reinvent the wheel, duplicate work, and waste time chasing information.

A well-structured service catalog in your internal developer platform addresses this by providing a central repository for all services and applications. At its most basic, it’s just a web page with each service’s README, service metadata, and operational information.

But, properly structured, this delivers some large benefits:

Context at a Glance: Every catalog entry follows a standardized template, capturing core information such as purpose, interfaces (REST, gRPC, events, etc.), language/runtime, and environment requirements. New developers can quickly grasp a service’s role and easily jump into productive work.
Discoverability and Reuse: Without a catalog, teams often recreate services they don’t realize already exist. Duplicate services not only waste resources but lead to fragmented knowledge and inconsistencies. With a catalog, developers quickly find and leverage existing solutions, increasing reusability and reducing churn.
Ownership Transparency: Each entry in the catalog explicitly names team ownership, CODEOWNERS files, and pertinent contact details like Slack channels or on-call rotations. Clear ownership streamlines communication, issue resolution, and collaboration, saving precious time and avoiding frustrations.

A service catalog supports autonomy, reduces wasted cycles, and increases developer velocity by making essential information instantly accessible.

But a Service Catalog alone isn’t enough. To fully streamline and enhance your development workflow, you must pair it with consistent local environments and reliable CI/CD processes. Both of which can be serviced by standardized service templates.

Service Templates: Golden Paths for Improving Developer Productivity

Earlier in this series, we introduced service templates as a way to scaffold consistent, ready-to-go services. In an internal developer platform, templates aren’t just for infrastructure, they’re the golden paths that define how developers start and succeed.

If 90% of new services are Go GRPC services, then a carefully crafted, opinionated go GRPC service blueprint will remove a lot of friction. It will give developers everything they need to begin delivering real business value immediately. A few select curated options of common starting points will help guide developers to generate projects complete with standardized structure, built-in quality checks, documentation, and clear guidelines for contribution.

A great service template provides:

Project Bootstrapping

The template quickly scaffolds an entire service—including all necessary files and directories. Everything essential for your organization’s standards should be included, from CI/CD configurations and dependency management to directory structures and environment-specific configuration files.

Built-in Quality Tools

Templates embed guardrails that maintain consistency and high-quality standards across teams:

Linting rules and formatters: Tools like ESLint, flake8, or Checkstyle that ensure code style and consistency across your codebase.
Testing frameworks: Unit tests, integration tests, and contract testing tools set up and pre-configured (JUnit, pytest, Jest, Pact), helping teams maintain high quality right out of the gate.

Documentation and Contribution Guides

Many platform teams overlook how clear, consistent documentation is crucial for initial and ongoing developer productivity. Every new service should arrive out-of-the-box with:

README: Describing at a glance what the project does, why it exists, and how to run it.
Documentation: Service template establishes a place for service-specific documentation that will be surfaced in the Service catalog. Markdown explanations, example input/output, and real-world use-cases.
Contribution guidelines and coding standards: Clearly documenting the expectations, conventions, and standards every developer can follow with confidence.

Embedding this documentation into templates ensures that even brand-new services are discoverable, understandable, and maintainable — right from their first commit.

A great template means the service catalog becomes more than just an organizational listing. These thoughtfully designed templates can directly power it. Every new service entering your catalog aligns neatly with your organization’s best practices in code quality, documentation, and process standards.

When paired with your service catalog, templates ensure every new service aligns with best practices — boosting developer productivity and developer velocity from day one.

Streamlining Local Development to Reduce Friction

One of the fastest ways to improve time to first commit is to standardize local development workflows through your internal developer platform.

Your service templates provide the ideal mechanism for simplifying how quickly developers can get projects running locally. By embedding standardized local-development tooling into these templates, you ensure they’re available consistently across your service catalog. Good templates typically contain:

A preconfigured, containerized environments: Using Dev Containers, Docker Compose, or similar tools lets developers launch an environment that closely resembles production with minimal friction, completely avoiding dependency drift or local environment inconsistencies.
Simple, standardized command runner: Makefiles, just scripts, package.json scripts—whatever works as they’re consistently documented and easy to run commands such as make build, make test, or make lint. Teams might diverge on preferred tooling over time; if so, simply record clearly in the project’s readme (and thus in your service catalog) exactly what’s needed.
Built-in documentation: Each service template includes clear instructions in a README outlining step-by-step processes for setup, running tests, and other everyday developer tasks.

With these pieces fully integrated into your catalog and templates, you significantly reduce friction and speed up development cycles, moving you meaningfully closer to improving your core developer-experience metrics.

The next critical area we need to address is how smoothly and quickly code moves from a local commit to a production release.

CI/CD Integration: Fast, Reliable Pipelines for Experienced Developers

Experienced developers cite unreliable or slow pipelines as major productivity killers. Their frustrations often revolve around flaky builds, unreliable tests, and cumbersome environment setups. Slow or unpredictable pipelines break developer flow, causing frustration, delays, and lower productivity.

A strong internal developer platform integrates fast, stable, and predictable CI/CD processes:

Stable, Fast Feedback:
Every new service comes pre-configured with build acceleration strategies like intelligent caching (Gradle, Bazel, Docker layers) and automatic parallel test execution. Stable and speedy pipelines help teams iterate quickly and confidently.
Ephemeral, Self-service Environments:
Instead of battling shared staging environments – which get blocked by other teams— developers spin up short-lived testing environments directly from pull requests. Need to test your payment-service changes against the latest user-authentication service? Create a dedicated, temporary environment on-the-fly, validate interactions, then shut it down automatically when finished.
Test Reliability:
Strictly enforce policies to quarantine flaky tests, quickly escalate notifications to responsible developers, and provide clear paths for fixing instabilities. Proactive flakiness management ensures credibility and reliability for test pipelines over time.

Reliable CI/CD pipelines protect developer focus, reduce cognitive load, maintain developer velocity, and prevent wasted time, increasing developer productivity.

Developer experience is a vast, nuanced topic, and there’s much more a comprehensive platform can include than we’ve been able to explore fully here. Even just within your Service Catalog and developer workflows, valuable additions might include:

Onboarding Guides: Step-by-step guides rapidly bringing new developers up to speed on teams, tools, and processes.
Frictionless Product Backlog Integration: Quick access from codebases to the product backlog to connect code directly with the business context.
Code Coverage & Quality Dashboards: Easy-to-use visualizations offering fast insights into quality metrics and test coverage.
Deprecation Notices & Change Management: Automated notifications about changes, upcoming maintenance, or decommission plans to keep developers informed and prepared.
Pre-built Common Components: Shared libraries and components solving everyday needs like logging, authentication, validation, and configuration.

And that’s truly just scratching the surface. But remember: Platform engineering is an iterative journey, you don’t need to tackle every single area all at once. If you start with friction logs and surveys to regularly identify and address pain points affecting daily workflows, you’ll already be heading confidently in the right direction.

Conclusion: IDPs as a DevEx Multiplier

A great developer experience is not optional — it’s a force multiplier for developer productivity and developer velocity. An internal developer platform with service catalogs, golden path templates, streamlined local dev, and integrated CI/CD removes friction at every step of the developer journey.

The result? Faster onboarding, quicker delivery, and happier teams.

With Pulumi, platform teams can automate consistent environments, implement golden paths, and enable self-service infrastructure that scales with your organization.

Organization Templates enables standardized service creation and consistent developer experiences
Stack References for managing dependencies between environments and services
Pulumi Deployments to streamline CI/CD workflows and enable self-service infrastructure
Review Stacks for creating ephemeral, on-demand testing environments
Automation API for programmatically managing infrastructure and implementing platform workflows

By prioritizing developer experience in your platform engineering strategy, you empower teams to move faster, build better, and maintain momentum, making your internal developer platform one of the most valuable assets in your organization.

Self-Service Infrastructure: From Tickets to Tools

Adam Gordon Bell — Thu, 06 Mar 2025 00:00:00 +0000

Previous articles in this series explored platform engineering principles and how Infrastructure as Code creates a solid foundation. But there’s still an important challenge to address: the infrastructure provisioning process itself. Without proper modularity and a clear separation between intent and infrastructure details, things get messy—leading to friction, delays, and unnecessary complexity.

The Missing Layer: Abstraction Through Modularity

The root issue here is how teams define and work with infrastructure. Without good abstraction layers, infrastructure code typically becomes unwieldy and complex.

You end up with

Highly detailed specifications that require deep domain knowledge
Environment-specific configurations with subtle variations across resources
Complex interdependencies that are difficult to trace and verify

This approach burdens developers with low-level details they don’t need to worry about, while platform teams become gatekeepers, reviewing intricate configurations. Alternatively, a traditional ticketing model takes over, discarding the benefits of modern practices.

Solution: A Platform Engineering Approach

What’s missing is a proper abstraction layer between raw infrastructure code and developer needs. That layer is the platform. A Platform for self-service infrastructure. Without modular, reusable infrastructure patterns, every new application deployment becomes an exercise in starting from scratch and defining every detail instead of just stating what the application requires and deferring details to the platform.

Common Anti-Patterns Without Proper Abstraction

Without this modular platform approach to cloud infrastructure, organizations typically fall into one of two problematic patterns.

The Copy-Paste Anti-Pattern

Developers copy infrastructure code from previous projects. They find something similar, duplicate large chunks of configuration, make a few changes, and hope everything is right, and submit it for review. This leads to a brittle system with siloed knowledge. Security fixes must be applied manually across every instance – no one has a full picture of the infrastructure – and Day 2 Operations become error prone.

And this isn’t limited to infrastructure: CI scripts get copied and modified, Dockerfiles and Kubernetes manifests get copied and modified.

It feels like control, but it’s not. Developers still rely on platform specialists to validate changes and catch mistakes.

The Ticketing Anti-Pattern

Alternatively, some organizations rely on rigid ticketing systems for infrastructure changes. A platform specialist applies writes and applies all changes, ensuring consistency — but also creating bottlenecks. Developers might wait days or weeks for simple modifications, leaving the process entirely unfinished.

This guarantees expert oversight but strips developers of autonomy, making the platform team a bottleneck for every deployment.

Neither anti-pattern strikes a balance between standardization and autonomy, leading to either sloppy infrastructure (copy-paste) or slow deployments (ticketing).

The Resulting Problems

Day 1 Bottleneck: Slow, Manual Infrastructure = Developer Gridlock

When provisioning becomes a bottleneck, developers with finished code wait days for infrastructure. This disrupts focus and forces frequent context switching. They juggle tasks to stay productive and then have to rebuild focus when the requested infrastructure is ready.

These delays slow the entire delivery pipeline. Teams add buffer time for infrastructure delays, projects slip, and the business sees technology as slow-moving. Meanwhile, the real culprit is slow provisioning that kills momentum.

Day 2 And Beyond Chaos: Sprawling, Unpredictable Infrastructure = Operational Nightmare

Day 1 bottlenecks frustrate developers, but the real long-term cost is patching hell. Copy-paste infrastructure creates hundreds of snowflakes, each needing manual attention for every security fix or version bump. One-off configurations and custom environments pile up, creating technical debt no one fully understands.

Operations teams are overwhelmed by the basic, critical task of simply keeping the lights on across a sprawling, inconsistent landscape.

The Solution: A Platform of Options and Self Service

A Self-service platform is the solution to these problems. The platform is this curated list of modules, providing application developers with a set of pre-built options that let them work at a higher level of abstraction. By choosing from these modules, developers express their intent for infrastructure, leaving the platform to handle the implementation details. Let me explain.

A strong self-service approach divides responsibilities into two clear layers:

Level 1: Platform Team (Module Definition)

Builds and updates reusable infrastructure modules
Bakes in security, compliance, and best practices
Hides complexity so developers don’t need to worry about copy and pasting low-level details
Sets guardrails for scalability, security, and compliance
Updates modules as technologies and needs change

Level 2: Application Developers (Module Consumption)

Browse a catalog of ready-made modules
Compose pre-built modules to set up application infrastructure
Spend time on code, not cloud configurations
Operate within built-in guardrails
Deploy applications faster using standardized components

This setup keeps things fast and flexible. Developers get the autonomy to build without reinventing infrastructure, while platform teams maintain control where it matters.

The Power of Intent-Based Specification

Layer of Abstraction to the Rescue

The two-level approach works by replacing over-specification with intent-based design.

Over-specification happens when developers specify too many infrastructure details that:

Require expertise they might not have
Tightly coupled infrastructure to specific implementations
Make future changes hard or even impossible

Intent-based specification means developers state what their app needs, not how to implement it, meaning:

Platform teams can update underlying infrastructure without breaking applications
New capabilities can be added without disrupting existing applications

Focusing on what developers need, not how it’s built, creates a system that’s easier to maintain and evolve for both platform teams and developers.

Let’s walk through an example.

Practical Example: Java Application with Kafka, PostgreSQL, and Redis

The platform team provides reusable modules:

Java Application Module

This module handles everything needed to run a Java app. It covers the Java runtime, memory settings, JVM options, scaling rules, health checks, monitoring, and Kubernetes namespacing. It also manages containers, security scans, resource limits, and logging integration—so developers don’t have to.

Kafka Integration Module

Similarly, this module simplifies Kafka integration, handling broker connections, authentication, topics, and security. It takes care of SASL/SSL authentication, replication factors, ACLs, and encryption. Developers just specify topics and schemas.

And so on for common infrastructure components used in the org.

Level 2: Developer Implementation

Developers use a simpler interface. To deploy a new Java service with Kafka, PostgreSQL, and Redis, they only need to specify the required input parameters in the platform-defined modules:

# Developer's intent-based configuration
application:
 name: order-processing-service
 module: java-application
 version: 17
 memory: 2GB
 scaling:
 min: 2
 max: 10

integrations:
 - module: kafka
 topics:
 - name: incoming-orders
 partitions: 5
 - name: processed-orders
 partitions: 3
 consumer_group: order-processors

 - module: postgresql
 database: orders_db
 size: medium
 backup: enabled

 - module: redis
 size: small
 ttl: 3600

( Note: YAML not required. This pseudo-code could be UI driven, or it could be written in your programming language of choice. The point is it specifies intent and defers specifics to the platform. )

Solving Day 1 and Day 2 Challenges

The intent-based specification solves immediate and long-term problems:

The Self-Service Transformation in Action:

Developer Experience	Platform Response
Submits simple intent-based configuration	Expands into complete infrastructure
Focuses on application needs, not infrastructure details	Provisions resources with appropriate sizing and security controls
Specifies “what” is needed, not “how” to configure it	Handles networking, monitoring, backups, and credentials
Limited expertise required in Kubernetes, networking, or security	Implements best practices automatically across all deployments
	Maintains and updates the underlying implementation over time

This approach transforms infrastructure from a developer burden into an organizational asset that becomes more valuable over time, rather than decaying into technical debt.

Developer Interaction Models: Choosing Your Platform Interface

Hopefully, now you are bought in on the value of separate layers of abstraction and building a platform to expose those.

The next key decision is how developers will actually use this platform. The ideal interface balances ease of use with the necessary flexibility. Organizations and requirements vary, so a spectrum of interaction models exists, each with its own trade-offs. Tree common approaches are: CLI-Driven Infrastructure as Code, the Self-Service Portal, and a Hybrid model.

Aspect	CLI-Driven	Self-Service Portal	Hybrid Approach
Developer Experience	Code-based, using IaC modules and components directly	Click-based UI with forms and catalogs	Portal for common tasks, code for customization
Code Visibility	Full visibility and control	Limited or no visibility	Visible but with varying levels of access
Governance	Trust-based or PR reviews	Enforced through portal constraints	Guided paths with managed exceptions
Learning Curve	Higher (requires IaC knowledge)	Lower (minimal infrastructure knowledge)	Moderate (basic UI with optional advanced use)
Best For	Experienced teams needing flexibility	Teams prioritizing speed and standardization	Organizations with diverse skill levels

1. CLI-Driven Infrastructure as Code: Empowering Developers with Familiar Tools

In this approach, platform teams build reusable infrastructure modules while developers consume them directly through code. Developers use familiar IaC tools like Pulumi CLI to instantiate these modules, with changes flowing through version control like any other code.

The key strength is transparency and familiarity. Developers maintain visibility and ownership of their intent-based definitions. This code-centric workflow integrates naturally with existing development practices.

However, this flexibility creates governance challenges and requires either strong trust or approval processes that might reintroduce bottlenecks. The approach also assumes developers have some IaC knowledge, making it best suited for organizations with mature DevOps practices and experienced teams.

2. Self-Service Portal: Click-Ops Simplicity for Rapid Provisioning

The self-service portal approach creates a web interface where developers provision infrastructure without writing code. They simply browse a catalog, fill out forms, and click buttons to deploy standardized resources—all pre-configured by the platform team.

This approach keeps infrastructure code completely centralized and hidden from developers. The portal translates simple selections (“Add Database: Medium Size”) into the appropriate underlying configurations, ensuring perfect compliance with organizational standards.

The result is dramatically reduced friction and learning curve, enabling even infrastructure novices to deploy resources in minutes. However, this simplicity comes at the cost of flexibility. Customizations and unique needs typically require platform team involvement, making this model ideal for organizations with predictable workloads or where standardization outweighs customization needs.

3. Hybrid Approach: Code-Backed Self-Service

The hybrid approach generates and modifies infrastructure code via pull requests from UI interactions. When developers configure resources through the portal, the system commits code to repositories automatically, creating a transparent workflow where changes are visible, reviewable, and auditable through standard Git processes.

This model provides both simplicity and flexibility. Routine tasks remain point-and-click while complex scenarios allow direct code editing within the same PR workflow. The approach maintains a complete audit trail through commit history while giving platform teams visibility into all changes.

A significant tradeoff is implementation complexity: building a system that generates commits and PRs based on version control integration requires substantial investment. Organizations must weigh this upfront cost against the long-term benefits of combining UI simplicity with code transparency.

Lifecycle Management

More effective lifecycle management is an advantage of a platform approach. Consider migrating from Kafka to AWS Kinesis. Traditionally, this would involve significant application developer effort, including updating connection details, authentication, and application logic. However, with the two-level abstraction and intent-based approach described earlier, this migration becomes simpler. Platform teams introduce a new Kinesis module matching the existing Kafka module’s intent-based interface. Data is mirrored, and developers can plan a switchover. Migration will not be free, but efforts and coordination will be reduced.

Life-Cycle management is a huge topic in Day 2 operations and outside the scope of what can be covered here, but introducing abstractions and a platform of modules will pay off long term.

Looking Ahead: Operational Excellence and Beyond

The two-level abstraction model does not just simplify provisioning. It also lays a strong foundation for operational excellence. By standardizing infrastructure through reusable modules and intent-based specifications, we create consistency and visibility that significantly simplify critical operational concerns such as:

Drift Detection and Reconciliation: Standardized modules make it easier to detect and correct deviations from desired state.
Continuous Compliance and Security: Centralized module definitions allow security and compliance controls to be consistently enforced and audited.
Cost Management and FinOps: Intent-based specifications enable clearer cost attribution and optimization opportunities.
Observability and Incident Response: Consistent infrastructure patterns simplify monitoring, alerting, and troubleshooting.

Later articles in this series will explore each of these areas in greater depth, demonstrating how a platform approach provides a powerful foundation for addressing them effectively.

Self-Service with Pulumi

The two-level architecture of IDP self-service creates a powerful interface between platform teams and application developers. By embracing intent-based specification, this approach enables developers to clearly express what they need with minimal complexity while giving platform teams the flexibility to implement and evolve the underlying infrastructure.

As you build out your self-service approach, consider how Pulumi might fit in. With Pulumi, you get:

Component resources for creating reusable infrastructure modules that encapsulate best practices.
Organization templates for building a catalog of infrastructure templates.
Packages in your language of choice for a code-first self-service workflow.
A Platform-in-the-box experience for those who want a portal-first experience for self-service infrastructure.
A Backstage plugin for those who prefer to use Backstage for an IDP.

A great self-service workflow and abstraction layer are key to making platform engineering work. Next in this series, we’ll tackle developer experience directly.

Provisioning: From Chaos to Control

Adam Gordon Bell — Thu, 27 Feb 2025 00:00:00 +0000

Provisioning is the first pillar of platform engineering. Without consistent infrastructure provisioning – the automated creation and management of the underlying cloud resources – the rest of the platform suffers. Self-service, governance, and streamlined developer workflows all depend on it. Ultimately, a self-service layer on top of your cloud infrastructure is the goal, enabling developers to quickly and safely provision the resources they need, while adhering to organizational best practices and policies. But before self-service, the foundation of a good IDP is a robust and reliable provisioning system.

By defining cloud resources as code and automating deployments, platform engineering teams ensure every environment – development, staging, and production – stays consistent and maintainable. This cuts down on configuration drift, reduces manual work, and supports auditable, collaborative workflows for every change.

Let’s explore how platform engineering teams can achieve this by version-controlling infrastructure, automating deployments, separating environments properly, and limiting console interventions. By applying these principles, teams can create a platform where developers can move fast without breaking things, and where infrastructure supports innovation rather than slowing it down.

Top 5 IaC anti-patterns

Console-Only Changes: Making “quick fixes” in production without updating IaC leads to drift, confusion, and hidden risks.
Hard-Coded Secrets: Sensitive values in code or config files are a security liability. Use encrypted state or a secrets manager.
Copy-Paste Config: Reusing half-baked snippets across projects creates cruft. Break out shared modules or templates.
No Clear File Structure: Throwing all configs into a single file or folder obscures dependencies. Organize by environment, service, or module.
Skipping Peer Reviews: Infrastructure mistakes can be costly. A simple pull request process can catch errors before deployment.

💡 Tip: Read our list of IaC Best Practices 💡

Building a reliable IaC foundation

It’s not enough to simply define your resources in code. A reliable IaC foundation should ensure that infrastructure changes move smoothly from idea to production, without surprising developers or straining operations. Here’s how to structure these workflows so they’re both transparent and resilient:

Version control as the single source of truth

Whether you’re using GitHub, GitLab, or something else, every infrastructure definition needs to live in version control. This is just the basics – and hopefully, you are already doing this, as you have been for years. Version-controlled changes allow you to track history, compare revisions, and peer review. A pull request might catch a misconfigured subnet or a database that was accidentally left publicly accessible, mistakes that could be catastrophic if they slip into production. This also means anyone can propose improvements or bug fixes, expanding the pool of expertise beyond a single “infrastructure guru.”

Automating changes with CI/CD

Next, consider how to apply infrastructure modifications after they’re merged into main. A CI pipeline can automatically run a “plan” or “preview,” then only proceed with the actual update if no blockers appear. Policy checks can also be added, enforcing rules such as “no public S3 buckets” or “use encrypted volumes only.” By codifying these conditions, you avoid manual verification steps and reduce the chance of risky configurations slipping past manual review.

Trunk-based infrastructure workflow

Protected main Branch: No direct commits
Pull Request Merges: Integrate changes via Pull Requests
Automated PR Checks: Previews and Policy-as-Code run on every PR
Merge: Merge to main triggers automated deployments

Distinguish each environment

Not all environments have the same requirements or budgets, so it’s wise to separate dev, staging, and production configurations. You might share core modules across these environments, but keep each environment’s parameters and secrets in independent config files or stack definitions. This deliberate separation ensures that dev changes are tested in a realistic but contained environment before they can impact production. As a result, teams can experiment more confidently without risking live workloads.

Modularity and reusability

A key principle of effective IaC, and a cornerstone of any good Internal Developer Platform, is reusability. You shouldn’t be copying and pasting large chunks of infrastructure code. Instead, you should build reusable components (or modules, or constructs, depending on the tool’s terminology) that encapsulate best practices and can be easily consumed by developers. This achieves several critical goals:

Consistency: Ensures that similar resources are provisioned in the same way, reducing configuration drift.
Abstraction: Hides complexity from developers. They don’t need to know how a secure, compliant database is created, just that it is.
Maintainability: Changes to the underlying implementation of a component can be rolled out to all consumers of that component.
Speed: Developers can provision infrastructure faster by using pre-built components.
Reduced Errors: Standard components means that there will be less opportunities to make mistakes.

Think of these reusable components as the “building blocks” of your IDP. Your platform team creates and maintains them, and developers use them to assemble their applications.

Regardless of the specific IaC tool you choose, reusable components generally share these characteristics:

Parameterized Inputs: Components accept inputs (parameters, properties, configuration – the term varies) that allow users to customize their behavior. For example, a “database” component might accept a database_size parameter.
Defined Outputs: Components expose outputs that other parts of the infrastructure can consume. For example, a “database” component might output the database’s connection string.
Encapsulation: The internal implementation details of the component are hidden from the consumer. The consumer interacts with a well-defined interface (inputs and outputs).
Versioned: Components should be versioned (e.g., using semantic versioning) to allow for controlled updates and rollbacks.
Discoverable: Components should be easily discoverable by developers, often through a central catalog or registry.

Example: An IAM roles component

IAM roles are one of the most commonly misconfigured AWS resources, making them a perfect candidate for abstraction into a reusable component. Developers frequently make mistakes when setting up roles, leading to overly permissive access, broken permissions, or operational headaches.

Common IAM role mistakes

Overly Broad Permissions – Developers often grant * permissions instead of following the principle of least privilege.
Missing Assume Role Policies – Without a proper trust policy, services may not be able to use the role at all.
Directly Attaching Policies to Users – Instead of assigning roles to groups, users might manually attach policies, leading to hard-to-audit permission structures.
Lack of Naming Conventions & Tagging – Unstructured role names and missing tags make governance difficult.

How a reusable IAM module fixes this

A platform team can provide a self-service IAM role module that enforces best practices while still allowing customization. This module would:

✅ Require a Least Privilege Policy – The module only allows explicitly defined permissions, preventing wildcards.
✅ Auto-Generate Trust Policies – Ensures correct settings, reducing misconfigurations.
✅ Enforce Naming Standards & Tagging – Ensures roles are easily identifiable and trackable.

By using a reusable IAM role module, developers can self-service their permissions while the platform team ensures security, consistency, and maintainability. This removes a major source of security risk and operational overhead.

No console interventions

No matter how carefully IaC is managed, emergencies may occasionally necessitate direct console changes. However, such interventions are strictly forbidden except under an official “break glass” procedure. If a quick fix is applied in the cloud console, it’s critical that the change is immediately documented and returned to code. Regular drift detection can help with this: once a day or once a week, automated scans can compare actual cloud resources to your declared infrastructure. When it detects a mismatch, you decide whether the code or the running resource is the correct version. Either way, human “hotfixes” never linger as hidden surprises for long.

By combining version control, automation pipelines, environment-based configs, and drift checks, you build a solid provisioning foundation. Once you have that in place, the next step is making these patterns easy to use, so teams can start new services without reinventing the wheel.

But first, let’s cover some metrics and roadblocks.

Metrics that matter

The classic DORA metrics focus on software delivery. Adapting and modifying them for infrastructure provisioning can help measure how effectively your platform delivers reliable, self-service cloud resources:

Lead Time for Infrastructure Changes: How fast does a merged PR turn into running infrastructure? Shorter indicates efficient automation and fewer manual steps.
Infrastructure Drift Frequency: Are developers constantly making console tweaks, or does your infrastructure stay in sync with code? A low drift rate means your code and actual resources stay aligned.
Infrastructure Change Failure Rate: What percentage of infrastructure changes result in incidents or rollbacks? A low failure rate indicates a robust, well-tested infrastructure pipeline, reducing wasted effort and potential outages.
Platform Self-Service Rate: What percentage of provisioning changes are implemented by the requesting team (without needing to involve a central operations team)? A high self-service rate shows that your platform empowers developers and reduces bottlenecks.

Common roadblocks

Adopting Infrastructure as Code helps teams automate provisioning, but common pitfalls can slow progress before they see results. Left unaddressed, problems hidden configuration tweaks, scattered code, or reliance on a single “infrastructure hero” make it hard to keep environments consistent and secure.

By spotting these roadblocks early, teams can build a stronger foundation for a repeatable, collaborative IaC workflow.

Knowledge silos: Relying on just one person (“Go ask Sarah”) for all infrastructure tasks creates a risky bottleneck. The broader team can’t replicate or improve the process, slowing new initiatives and leaving you vulnerable if that individual is out of office or leaves the company. Moving to a collaborative, version-controlled IaC model ensures that no single engineer is a gatekeeper for new environments or configuration changes.
Overly complex or unstructured IaC: Bloated or disorganized infrastructure code leads to confusion, duplication, and frequent errors. Engineers might copy-paste configurations from previous projects or cram all resources into a single file. By refactoring IaC into smaller modules, adopting a clear folder structure, and removing dead code, teams gain consistency and accelerate ramp-up times for new services.
Learning curves: Teams new to IaC often encounter a steep learning curve. There is initially a lot to learn, and resistance from people used to quickly working through cloud UI can slow down full adoption. But time invested in learning tools and setting cultural norms pay off in agility and a platform strategy with guidance and golden paths can help reduce the burden.
Cultural inertia: The adoption of a platform engineering approach is a shift in how organizations manage infrastructure, and like all, change is not merely technical but also organizational. Resistance to change and organizational inertia can make it difficult to implement new ways of getting work done. Expecting, understanding, and working through this resistance is key to overcoming it.
Lack of comprehensive approach: IaC tools are great for provisioning, but without a full platform approach, covering monitoring, logging, and the full life cycle of cloud resources, provisioning efforts may fall short. We will see in this series how the pillars of a comprehensive approach strengthen themselves.

These roadblocks highlight why a thoughtful, well-structured approach to provisioning is essential. But that’s not enough because the next step is to make these consistent patterns easily accessible so teams can spin up entire services without starting from scratch. And that’s where “golden path” templates come in.

Platform team provisioning responsibilities

Setting up the initial IaC foundation
Creating and maintaining templates
Automating workflows
Monitoring usage and compliance
Internal Platform Evangelism

Service templates

Even with a solid IaC foundation and reusable modules, spinning up a brand-new service can still require developers to understand how to pull in the underlying infrastructure components and how to use them. That’s where templates come in.

These are pre-vetted, ready-to-use blueprints that capture your organization’s best practices for a complete service or application, including infrastructure, application, and security components. They are the key to enabling true self-service.

Think of service templates as the next step up from the reusable components we discussed earlier. While a component might represent a single “building block” (like a secure S3 bucket or a database), a service template represents an entire structure built from those blocks (like a complete web application with a database, load balancer, and containerized application). They combine those building blocks with sensible default configurations and best practices for things like logging, Kubernetes setup, and security baked in.

Beyond bundling your organization’s best practices into a single repository, these service templates can also include a preconfigured CI/CD pipeline.

This means that when a team spins up a new service from the template, they inherit not just the infrastructure modules and best-practice configurations, but a fully operational CI/CD workflow. No extra setup is required. Developers get immediate feedback on their changes, and the merge to main triggers automatic deployments that keep every environment consistent.

Instead of cobbling together snippets from old repos, teams can scaffold out a service by simply selecting the template that fits their need – whether it’s a containerized web app, a serverless function, or a data-processing pipeline – and providing a few key parameters.

These curated paths cut onboarding time, reduce configuration missteps, and ensure that the IaC modules are used in new services. If developers don’t have to remember which tags to apply or how to route logs, or even where to import infrastructure modules from, they can focus on delivering features rather than wrestling with infrastructure details. And as your platform matures, and new versions of your modules are introduced, you can update these templates, automatically sharing improvements with every new project.

Bringing it all together: The impact of effective provisioning

With reusable modules and service templates in place, and a deployment workflow that takes changes from git commits all the way to your cloud account, your platform engineering team creates a foundation that transforms how developers interact with infrastructure. This transformation yields measurable benefits:

Lower Infrastructure Change Failure Rate: Automated validations and peer reviews catch issues early, reducing the number of failed deployments.
Increased Platform Self-Service Efficiency: Developers can provision resources in minutes instead of days, boosting productivity and reducing bottlenecks.
Reduced Lead Time for Infrastructure Changes: Streamlined automation shortens the time from code commit to deployed infrastructure.
Minimized Infrastructure Drift: Fewer manual console interventions mean your actual infrastructure stays aligned with what’s declared in code.

(In the upcoming article, we’ll dig deeper into how governance guards these golden paths, ensuring that new projects comply with security and corporate policies right from day one.)

Managing infrastructure provisioning with Pulumi

As you build out your platform engineering plan, consider how Pulumi might fit in. With Pulumi, you get:

Infrastructure as Code in the language of your choice.
Component resources for creating reusable infrastructure modules that encapsulate best practices.
Pulumi Deployments for a fully managed infrastructure deployment solution.
And a Platform in the box experience.

Reliable infrastructure provisioning is the cornerstone that supports your entire platform engineering initiative. By implementing the practices outlined in this article – version-controlled IaC, automated CI/CD workflows, environment separation, and golden-path templates – you establish a foundation that enables consistency, speed, and trust.

But provisioning alone isn’t enough. In upcoming articles, we’ll explore how policy-as-code and broader governance tie into this foundation, ensuring that all those newly created services meet security and compliance standards from the moment they’re deployed. But first, we’ll look at expanding the provisioning foundation covered here into a true self-service platform, so teams can quickly spin up services without risking the organizational chaos that often accompanies speed.

Provisioning is the foundation – next in this series we’ll build on it.

Platform Pillars: Build Platforms, Not Infrastructure

Adam Gordon Bell — Wed, 26 Feb 2025 00:00:00 +0000

Software drives innovation. Development teams face pressure to ship features faster. But speed collides with infrastructure complexity. Developers struggle with cloud setups, juggle scattered tools, and wait on operations teams for resources. The result is friction and slower innovation.

This is where Platform Engineering comes in. It helps developers move faster by creating tools that actually work. A good internal platform lets teams self-serve infrastructure, find documentation, follow best practices, and focus on what they do best: writing useful software.

Building a platform isn’t about finding one perfect tool. It’s about assembling the right pieces, or pillars. These pillars define what every successful internal developer platform needs.

This series explores these key pillars of Platform Engineering, offering a practical guide to building platforms that remove barriers to developer speed. Each pillar addresses a specific challenge organizations face when scaling developer productivity. The first challenge is overcoming infrastructure chaos.

Problem: Infrastructure chaos and bottlenecks

Developers face many pain points with manual provisioning, which is why Infrastructure as Code has become so essential:

Inconsistent environments: Dev, staging, and production drift apart. “Works on my machine” becomes everyone’s excuse.
Slow processes: Console clicking and waiting on ops creates bottlenecks. Developers wait days for basic infrastructure.
Configuration drift: Manual configs lead to errors. Quick production fixes become undocumented mysteries that cause problems later.
Lack of repeatability: Spinning up new environments or replicating existing ones is a complex, time-consuming, and often unreliable process.

That’s why provisioning is the first pillar to master if you want to build a successful developer platform.

Problem: Dependency bottlenecks and limited developer autonomy

Even with automated provisioning, another challenge emerges - developers still struggle when tools are provided without considering usability.

Hard to find what you need: Developers waste time hunting for the right infrastructure modules. They end up searching docs, bothering colleagues, or rebuilding something that already exists.
Too much complexity: Most developers aren’t infrastructure experts. Learning all the options and parameters creates hesitation and mistakes.
Lack of real autonomy: Developers have the tools but not the confidence to use them. They still need approvals or fear breaking production.
Inconsistent workflows: Teams create their own ways of using infrastructure, making collaboration harder.

By implementing self-service infrastructure provisioning, you transform your platform from a centrally controlled system to a developer-centric environment. You unlock developer autonomy, accelerate development cycles, and free up operations teams to focus on higher-value activities.

Problem: Inefficient and fragmented development processes

Infrastructure is just one piece of the puzzle. Developers still face daily friction moving code to production.

A typical developer’s day is full of frustrations:

Inconsistent local environments: Variations in local setups, dependency conflicts, and operating system differences lead to debugging headaches and wasted time getting started.
Slow CI/CD pipelines: Long builds, flaky tests, and mysterious pipeline configs block progress. Developers stare at progress bars instead of writing code.
Cumbersome testing and staging setups: Provisioning environments for testing and staging can be a manual process. Environments may not accurately mirror production, leading to surprises and integration issues late in the development cycle.
Lack of discoverability and reusability: Nobody knows what code already exists, so everyone rebuilds it. Duplication becomes the norm.
Developer toil and context switching: Developers spend more time on environment setup and troubleshooting than on writing features.

These inefficiencies add up, creating significant friction in the developer workflow. Developers spend more time wrestling with processes and less time focused on building and innovating. A Platform approach can help solve these challenges.

Problem: Security as a bottleneck and source of friction

Speed and efficiency matter, but security can’t be compromised. Traditionally, security acts as a gate – a necessary hurdle before releasing software. Platform Engineering aims to transform security from a bottleneck into an enabler.

For many developers, security feels like a roadblock, leading to common friction points:

Last-minute security: Checks happen at the end of development, causing costly rework when issues are found.
Restrictive policies: Security rules feel arbitrary and unclear, slowing down deployments.
The “No” department: Security teams are seen as blockers who enforce policies without offering solutions.
Speed vs. security mindset: Everyone believes you must choose between going fast and being secure.
Security is “Someone else’s job”: Developers think security is only the security team’s responsibility.

This “security as a gate” approach not only slows things down but leads to workarounds as developers try to bypass security to meet deadlines.

By integrating security into your platform, you transform security from a gatekeeper into a built-in feature of the development process. Security becomes an enabler of trust and speed, letting developers innovate with confidence.

Problem: Data overload without insights

Developers in modern cloud environments often find themselves in a paradox: they are drowning in data, yet starving for actionable information. Monitoring systems pump out metrics, logs, and traces, but finding what matters feels impossible. An ideal internal developer platform shifts the focus from data collection to actionable insights.

Here’s what developers typically experience:

Tool sprawl: Teams use different tools for metrics, logs, and traces. Developers waste time jumping between dashboards and manually connecting the dots.
Alert noise: Developers get bombarded with alerts lacking context or clear next steps. Important signals get buried in the noise.
Dashboard overload: Existing dashboards overwhelm developers with granular metrics that are hard to connect to application behavior.
Reactive debugging: Troubleshooting becomes a time-consuming fire drill. Developers spend hours sifting through logs to pinpoint issues.
Missing proactive insights: Observability is only used for firefighting. Developers lack insights to optimize performance or prevent issues before they affect users.
Disconnected from production: Many developers feel blind to how their code actually runs in production, limiting their ability to improve it.

This data flood without clear insights leaves developers reactive rather than proactive, hampering their ability to build reliable applications.

By improving observability in your platform, you transform it from an operations function into a developer superpower. Developers gain the insights they need to build better applications with greater confidence, using AI to focus on what truly matters.

Problem: Balancing autonomy with control and cost efficiency

Empowering developers with self-service is essential, but unchecked autonomy leads to chaos. As platform adoption grows, organizations need to maintain control without creating bottlenecks. This is where Governance as Code becomes critical, establishing guardrails while preserving speed.

Self-service creates new challenges:

Configuration drift: In decentralized environments, infrastructure setups diverge from standards, creating security risks.
Cloud cost explosion: Without proper controls, self-service can lead to wasted resources and budget overruns. Teams over-provision or forget to clean up unused environments.
Compliance headaches: Meeting regulatory requirements becomes harder when infrastructure is created in an uncontrolled way.
Audit gaps: Without clear governance, tracking who created what and when becomes nearly impossible.
Manual governance slows everything: If governance relies on approvals and tickets, it defeats the purpose of self-service.

These challenges highlight the need for automated governance that scales with your platform, ensuring control without slowing developers down.

By implementing Governance as Code, you ensure that self-service doesn’t conflict with security, compliance, and cost efficiency. Governance becomes an automated part of your platform, enabling responsible scaling.

Conclusion

These six challenges represent the core obstacles standing between your organization and true developer velocity. When properly addressed, each transforms into a capability that empowers developers and accelerates innovation.

The upcoming articles in this series will explore each area in depth, with practical strategies and real-world examples to help you build an effective internal developer platform.

Ready to implement these principles in your organization? Check out “Platform Engineering: Recommended Practices for Infrastructure as Code”. It provides detailed guidance on building scalable internal developer platforms.

How to structure infrastructure code for scalability
Managing development environments with developer stacks and Git branches
Designing modular and reusable Pulumi projects
Implementing cross-stack dependencies with stack references
Setting up RBAC, security, and governance at scale
Automating infrastructure management with stack orchestration

Get the Free Platform Engineering Ebook →

And don’t miss the next article in this series, where we’ll explore provisioning as the foundation of your developer platform journey.

Host your Python app for $1.12 a month

Adam Gordon Bell — Wed, 29 Jan 2025 00:00:00 +0000

TL;DR (2026 pricing): Packaging a Flask app as a container and running it on AWS Lambda behind an HTTP API Gateway costs ~$1.12/month worst-case, or $0 when idle. The breakdown: $0.04 for API Gateway requests, $0.07 for Lambda compute (fully absorbed by the always-free tier), and $1.08 for 12 GB of egress at $0.09/GB (covered by AWS’s 100 GB/month free egress allowance unless other services in the account have already used it). Pulumi handles the entire infrastructure. Verified against AWS pricing as of April 2026.

How cheap can you host a Python app in 2026? For a low-traffic Flask API (say, 40,000 requests per month at 512 MB of memory), the answer is roughly $1.12/month on AWS worst-case, dropping to $0 when idle. The trick is to stop thinking of AWS Lambda as “one function per endpoint” and instead package your entire web framework as a container, deploy it to Lambda, and put it behind an HTTP API Gateway. Your code stays standard Flask. Your bill stays in the loose-change zone.

This post walks through the whole setup with Pulumi, then compares the resulting cost against Google Cloud Run, Fly.io, Railway, and Vercel using current 2026 prices.

Most developers maintain at least one low-traffic service that still needs to be reliably available: an internal reporting API, a webhook receiver, a side project with occasional use. Traditional hosting means paying for 24/7 server time, even when the service sits idle. AWS Lambda’s container support flips that: pay only when your API is being called, and any web framework that handles HTTP requests (Flask, FastAPI, Express) works without modification.

How does the pricing break down? (2026)

Plugging 40,000 requests per month at 512 MB memory and ~200 ms average execution into 2026 AWS pricing:

Lambda compute: 40,000 × 0.5 GB × 0.2 s = 4,000 GB-seconds × $0.0000166667 = $0.067, fully covered by the always-free tier (400,000 GB-seconds free per month).
Lambda requests: 40,000 × $0.20 / 1,000,000 = $0.008, also covered by the free tier (1M requests free per month).
HTTP API Gateway: 40,000 × $1.00 / 1,000,000 = $0.04.
Data transfer out: 12 GB × $0.09/GB = $1.08 if billed, but AWS’s always-free tier (expanded to 100 GB/month outbound in late 2021) covers this row outright unless other services in the same account have already exhausted that allowance.
Total: ~$0.04/month (egress in the free tier, only API Gateway bills) to ~$1.12/month (12 GB egress billed in full: $1.08 + $0.04 API Gateway), with Lambda compute and requests always fully covered by the free tier.

The underlying math hasn’t changed since this post was first published in early 2025; Lambda and API Gateway pricing has been remarkably stable, and AWS has only expanded free-tier allowances since.

Cheap Python hosting compared (2026)

Provider	Approx. cost / month	2026 notes
AWS Lambda + HTTP API	~$1.12	~$1.08 for 12 GB egress at $0.09/GB + $0.04 for 40k API Gateway requests at $1/M; Lambda compute ($0.067) is fully covered by the always-free tier (1M req/400k GB-s).
Google Cloud Run	~$1.44	2M requests, 360,000 GiB-seconds of memory, and 180,000 vCPU-seconds are free every month — compute is $0. Egress dominates: Premium tier is ~~$0.12/GB, so 12 GB ≈ $1.44. Standard tier (~~$0.085/GB) lands near $1.02.
Fly.io	~$2.18	Cheapest persistent `shared-cpu-1x@256MB` machine is ~$1.94/mo; 12 GB outbound at $0.02/GB (NA/EU) adds ~$0.24 (free tier was retired in late 2024).
Railway	~$6.00	Hobby plan starts at $5/mo and includes $5 of usage; a 512 MB always-on container plus 12 GB egress lands around $5.50–6.50 total.
Vercel (Pro)	$20+	Pro plan has a $20/mo floor; 40k function invocations are well under the included quotas, so you mostly pay for the seat.

Approximate 2026 hosting costs for a 512 MB Python API at 40,000 requests/month with 12 GB egress. Egress dominates at this scale; even 1 million requests/month stays under $30 on Lambda.

What we’ll cover:

Why use AWS Lambda for low-traffic Python apps?

“But Lambda means one function per REST endpoint, right?” While that’s how serverless was initially positioned, the Lambda monolith approach here is a more straightforward pattern. Take any REST API service, package it as a container, put it in Lambda, and give it an entire HTTP route behind an IP, domain name, or API Gateway prefix. Done.

This approach gives you several benefits:

Minimal Serverless Knowledge Required: You don’t need to be an expert in Lambda architecture or serverless patterns. If you can build a REST API and containerize it, you’re ready to go.
Standard Development Experience: Write your API exactly as you would for any other hosting platform: no special frameworks, no Lambda-specific patterns to learn.
Simple Local Development: Run your container locally, and it will behave exactly like it will in production. No complex serverless emulation is needed.
Zero Cost at Zero Usage: Get all the cost benefits of serverless without restructuring your application.

The trade-off? Cold starts when Lambda spins up a new container instance. But this is often a worthwhile exchange for operational simplicity and zero-cost idle periods for low-traffic services.

Let me show you how to set this up. With some Pulumi Python code for setting up the Infra and some containerization code, you get an easy hosting solution.

Why containers?

Containers let us package everything—code, system dependencies, runtime—into one unit. Any HTTP service that fits in a container will work: Flask, Express, Rails, plus whatever system tools you need. Local testing? Just start the container. No Lambda-specific configuration is required.

How do I build a Flask app for Lambda?

Let’s start with the fun part - writing some Python code, but I will keep it really simple. We’ll build a simple Flask application that demonstrates why running a full web framework in Lambda makes sense.

Here’s our starting point in scripts/simple_server.py

from typing import Dict
from flask import Flask, jsonify, request
from mangum import Mangum

# Create Flask app
app = Flask(__name__)

@app.route("/hello")
def hello() -> Dict[str, str]:
 return jsonify({"message": "Hello from Flask!"})

@app.route("/echo", methods=['POST'])
def echo() -> Dict[str, str]:
 data = request.get_json()
 return jsonify({"reversed": data['message'][::-1]})

# Create Lambda handler
lambda_handler = Mangum(app)

if __name__ == "__main__":
 app.run(host="0.0.0.0", port=3000)

Notice a few key things about this code:

It’s Just Flask: No Lambda-specific code, no special frameworks. If you know Flask, you know how to modify this.
Multiple Endpoints: We have several routes demonstrating different HTTP methods and URL patterns.

How do I containerize a Flask app for Lambda?

To containerize our Flask app for Lambda, we’ll start with a Dockerfile that meets AWS’s requirements. The key is using the right base image and adapting our application to Lambda’s runtime interface.

First, let’s create our Dockerfile:

FROM public.ecr.aws/lambda/python:3.12

# Set up working directory
WORKDIR /app
COPY scripts/simple_server.py scripts/
COPY scripts/simple_entrypoint.sh scripts/

# Install dependencies
RUN pip install --no-cache-dir \
 flask>=3.0.0 \
 mangum>=0.17.0

# Make entrypoint script executable
RUN chmod +x scripts/simple_entrypoint.sh

# Set environment variables
ENV FLASK_APP=scripts/simple_server.py
ENV PYTHONPATH=/app
ENV RUN_MODE=lambda
EXPOSE 3000

# Use our entrypoint script
ENTRYPOINT ["/app/scripts/simple_entrypoint.sh"]

Notice that:

We’re using AWS’s official Lambda Python base image. You can use your own base image and your Python package manager of choice, but starting with this base and pip install is the easiest.
The ${LAMBDA_TASK_ROOT} is a special directory where Lambda expects to find your code
We’re introducing a new file: simple_entrypoint.sh

Do I need to learn Lambda-specific patterns?

The beauty of everything covered so far is that it’s just a container. You can work with it locally with Docker run, put it in Kubernetes, or run it in one of the 1000 places you can run containers. Lambda has some odd requirements about function calling that get in the way of this, but simple_entrypoint.sh saves the day:

#!/bin/bash

if [ "$RUN_MODE" = "local" ]; then
 echo "Starting in local mode..."
 exec python scripts/simple_server.py
else
 echo "Starting in Lambda mode..."
 exec python -m awslambdaric scripts.simple_server.lambda_handler
fi

All this means is that locally, you don’t need to know about lambdas at all. If you supply RUN_MODE=local it runs the standard way you are used to:

#!/bin/bash

# Build the container
docker build -t simple-flask -f infra/Dockerfile .
docker run -d -p 3000:3000 -e RUN_MODE=local simple-flask

# Test the endpoints:
# 1. Test GET /hello
curl http://localhost:3000/hello

# 2. Test POST /echo with a message
curl -X POST "http://localhost:3000/echo" \
 -H "Content-Type: application/json" \
 -d '{"message": "hello world"}'

This local testing setup is crucial because it helps catch issues before deployment.

The beauty of this setup is that once it works locally, it’ll work the same way in AWS.

And since this is just a standard Python application in a container, you can use all your favorite Python tools and patterns – your preferred package manager (pip, poetry, uv), pytest for testing, standard logging, type hints, linting, and pre-commit hooks.

( I’m skipping those details here to focus on deployment, but the container approach means there are no serverless-specific limitations on your Python workflow. )

How do I deploy a Lambda container with Pulumi?

Now let’s get our containerized Flask app into AWS. While you could click around in the AWS console, we’re going to do this properly with infrastructure as code. You are welcome to use Cloudformation or manual setup, but it’s nice to be able to keep everything in Python, and this is the Pulumi blog. Using Python to define our AWS resources means we get type-checking and proper IDE support.

Here’s our infrastructure code:

Don’t let the infrastructure code below intimidate you. It’s just typed Python code, hooking up the Lambda, that you can version control and modify easily.

from typing import Any
import pulumi
import pulumi_aws as aws
import pulumi_awsx as awsx
import json

# First, create an API Gateway
api_gateway: aws.apigatewayv2.Api = aws.apigatewayv2.Api("flask-api",
 name="Flask Lambda API",
 protocol_type="HTTP",
 route_selection_expression="$request.method $request.path"
)

# Create IAM role for Lambda
lambda_role: aws.iam.Role = aws.iam.Role("flask-lambda-role",
 assume_role_policy=json.dumps({
 "Version": "2012-10-17",
 "Statement": [{
 "Action": "sts:AssumeRole",
 "Principal": {
 "Service": "lambda.amazonaws.com"
 },
 "Effect": "Allow",
 "Sid": ""
 }]
 })
)

# Attach basic execution policy - Lambda needs this to write logs
lambda_role_policy: aws.iam.RolePolicyAttachment = aws.iam.RolePolicyAttachment("lambda-role-policy",
 role=lambda_role.name,
 policy_arn="arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
)

# Create ECR repository for our container
repository: aws.ecr.Repository = aws.ecr.Repository("flask-app-repo",
 name="flask-app-repo",
 force_delete=True, # Makes cleanup easier for testing
 image_scanning_configuration=aws.ecr.RepositoryImageScanningConfigurationArgs(
 scan_on_push=True,
 )
)

# Build and push the Docker image to ECR
image: awsx.ecr.Image = awsx.ecr.Image("flask-app-image",
 repository_url=repository.repository_url,
 path=".", # Path to your Dockerfile
 platform="linux/amd64" # Important for M1/M2 Mac users
)

# Create the Lambda function
lambda_function: aws.lambda_.Function = aws.lambda_.Function("flask-app",
 name="flask-app",
 package_type="Image",
 image_uri=image.image_uri,
 role=lambda_role.arn,
 timeout=30,
 memory_size=512,
 environment={
 "variables": {
 "FLASK_ENV": "production",
 "PYTHONUNBUFFERED": "1"
 }
 }
)

# Create API Gateway stage
stage: aws.apigatewayv2.Stage = aws.apigatewayv2.Stage("api-stage",
 api_id=api_gateway.id,
 name="$default",
 auto_deploy=True
)

# Connect API Gateway to Lambda
integration: aws.apigatewayv2.Integration = aws.apigatewayv2.Integration("lambda-integration",
 api_id=api_gateway.id,
 integration_type="AWS_PROXY",
 integration_uri=lambda_function.arn,
 integration_method="POST",
 payload_format_version="2.0"
)

# Create catch-all route
route: aws.apigatewayv2.Route = aws.apigatewayv2.Route("catch-all-route",
 api_id=api_gateway.id,
 route_key="ANY /{proxy+}",
 target=integration.id.apply(lambda id: f"integrations/{id}")
)

# Allow API Gateway to invoke Lambda
lambda_permission: aws.lambda_.Permission = aws.lambda_.Permission("api-lambda-permission",
 action="lambda:InvokeFunction",
 function=lambda_function.name,
 principal="apigateway.amazonaws.com",
 source_arn=api_gateway.execution_arn.apply(lambda arn: f"{arn}/*/*")
)

# Export the API Gateway URL
pulumi.export("url", api_gateway.api_endpoint)

Let’s break down what’s happening here:

API Gateway: We create an HTTP API (v2) that will receive all web requests and forward them to our Lambda.
IAM Role: Lambda needs permissions to execute and write logs. We create a role with the minimum necessary permissions.
Container Registry: We create an ECR repository and use Pulumi’s awsx package to build and push our Docker image automatically.
Lambda Function: We create the Lambda function using our container image. Notice how we set memory and timeout - these are important for a web application.
Integration: We connect API Gateway to Lambda using the AWS_PROXY integration type. ANY /{proxy+} means we route everything through with the full path.

Need a custom domain, that’s easy to use as well. Setup some Route53 records, CNAME a domain name appropriately and you’ll have a container backed by a custom domain name.

Why is this approach worth it?

The cool thing about using Pulumi and lambdas is that they make complex infrastructure changes safe and repeatable.

Want to increase the Lambda memory? Change one number and run pulumi up.
You get complete AWS Lambda and API Gateway management for your low-traffic REST APIs
And Infrastructure testing capabilities mean you can validate your serverless configuration before deployment

You can find the full code in the service status monitor repo from my Python devops article. A larger version with rolling updates is coming soon.

Alternative approaches

There are other ways to run containers that scale to zero. Google Cloud Run offers similar functionality on GCP, and AWS App Runner is another AWS service that can do this. Both have similar pricing models—very cheap for low-volume services. And SST is a similar solution for getting TypeScript / JavaScript solutions into an AWS Lambda.

The beauty of the container approach used here is that switching between these services is straightforward. Since we’re just running a standard container with a REST endpoint, there’s nothing AWS-specific in our application code. If you need to move to a different platform later, you can take your container with you.

Conclusion

Running Flask applications in Lambda containers represents a sweet spot between development simplicity and operational efficiency. You get:

True zero-cost periods when your application isn’t being used
Familiar development patterns - it’s just Flask
Infrastructure as code with Pulumi
Automatic scaling without managing servers

The trade-offs - primarily cold starts and some AWS-specific configuration - are worth it for many applications, especially:

APIs with sporadic traffic
Cost-sensitive projects

The key is understanding that this isn’t just about saving money - it’s about reducing operational complexity. No more worrying about scaling policies, server maintenance, or capacity planning. Your application scales automatically, and you only pay for what you use.

Is this approach right for every application? No, but this combination of familiar local development with serverless operations is under utilized.

I’d love to hear about your experiences with monolithic serverless applications. What hosting solutions have you tried? What challenges have you encountered?

Get started with Pulumi on AWS: the official quickstart for the tooling used in this post.
AWS Lambda function reference: every property on the resource we used above.
Code, deploy, and manage a serverless REST API on AWS with Pulumi: the function-per-route alternative to the monolith pattern.
Controlling AWS costs with Lambda and Pulumi: patterns for keeping your bill predictable as traffic grows.
The cursed container iceberg: every other place you can run the same container if you outgrow Lambda.
Python for DevOps: the companion post that ships the full source code for this example.

Changelog

2026-04-30: Refreshed for 2026. Re-verified Lambda/API Gateway/egress math, replaced the comparison table with current Cloud Run, Fly.io, Railway, and Vercel pricing, restructured H2s as user questions, added an answer-first intro and TL;DR, added a worked cost example and a HowTo schema for the deployment steps.
2025-03-13: Minor copy edits.
2025-01-29: Original publication.

Top 15 Python Tools for DevOps

Adam Gordon Bell — Mon, 06 Jan 2025 00:23:24 -0500

Have you ever had one of those moments: Elasticsearch is crashing, logs are filling up too fast, or a deployment needs data from three different APIs. The big enterprise solutions aren’t quite right, and your team estimates six months to build something proper. But you know that with Python and a few choice libraries, you could hack something together by tomorrow.

Let's build a little web dashboard in Python.

That’s where Python really shines. It lets you be the scrappy problem-solver who can travel up and down the stack. But Python isn’t just for quick fixes. The same language that helps you hack tomorrow’s solution can also build your company’s long-term infrastructure.

This guide covers both ends of that spectrum. I’ll show you python packages for “we need it yesterday” moments, and then move into solutions that can grow with your team. And along the way I’ll build a little example service monitor and package capturer to demonstrate the python packages I’m covering. I’ll wrap it in a CLI, in a web dashboard and deploy it to AWS, all while staying in Python.

So here’s my list of Python libraries for DevOps work, starting from #15 to the most important last at number #1.

Dashboards / monitoring

Let’s start with building web dashboards. Long-term, it’s usually better if there’s an off-the-shelf tool for that specific dashboard you need. But if you can create a quick visual on a Django endpoint, you can move on to the next issue.

15 and 14. Django and Flask: (Django, Flask) While tools like Grafana excel at metrics visualization, sometimes you need custom dashboards that integrate with internal systems. These frameworks let you build dashboards to track and visualize various aspects of your systems. I find Django a great fit here.
13. Prometheus: (prometheus-client) This library is a versatile tool for metrics. Want to expose custom metrics to Prometheus? Use the client to create counters, gauges, or histograms. Need to pull data from a weird API and expose it as metrics? Write a quick exporter that fetches from your API every few minutes and translates the responses into Prometheus metrics.

from flask import Flask, render_template_string
import requests

app = Flask(__name__)

@app.route("/")
def index():
 try:
 response = requests.get("https://www.githubstatus.com/api/v2/status.json", timeout=5)
 if response.status_code == 200:
 data = response.json()
 status = data["status"]["indicator"]
 description = data["status"]["description"]
 else:
 status = "critical"
 description = "Unable to reach GitHub API"
 except Exception as e:
 status = "critical"
 description = f"Error: {e}"

 color = {
 "none": "bg-green-500",
 "minor": "bg-yellow-500",
 "major": "bg-red-500",
 "critical": "bg-red-700",
 }.get(status, "bg-gray-500")

 return render_template_string(
 """
 ...
 """,
 color=color,
 description=description,
 )

Code for a simple Dashboard

You might also like:

Task scheduling and orchestration

Sometimes you need to run things on a schedule. And while I love cron, sometimes you need more. Maybe you need to coordinate multiple tasks, handle retries, or manage complex job dependencies. Here’s where Python can help:

12. Schedule: (schedule) When cron is too heavy but you need more than a bash script. I use it for quick automation tasks like “check this API every hour and alert if something’s wrong.” It runs in your Python process - no system configuration needed.
11. RQ (Redis Queue): (rq) Perfect for local development queues. Need to prototype a GitHub webhook handler that’ll eventually use SQS? With Redis in Docker and RQ, you can have a working queue in minutes. No AWS needed until production.
10. Airflow: (airflow) The big gun of scheduling. Yes, it’s complex, but sometimes you need that power. I’ve seen teams replace entire ETL systems with a few Airflow DAGs. When you need to coordinate multiple steps, handle failures gracefully, and have everything visible in a nice UI, it delivers.

def schedule_url_checks():
 for url in urls.values():
 schedule.every(30).seconds.do(lambda url=url: check_url_status(url))

def run_scheduler():
 while True:
 schedule.run_pending()
 time.sleep(1)

if __name__ == "__main__":
 schedule_url_checks()
 threading.Thread(target=run_scheduler, daemon=True).start()

Adding Schedule to our code, so we can request in the background.

Network analysis and security

When your network issues go beyond what tcpdump can tell you, or when you need to automate security checks, Python is here to help:

9. Scapy: (scapy) The Python alternative to Wireshark for programmatic network traffic analysis. Need to figure out why your microservices aren’t talking to each other? Want to check your load balancer? Use it to capture and analyze packets with a few lines of code.
8. Bandit: (bandit) Security scanning for your CI pipeline. Point it at your Python codebase and it’ll flag common issues like hardcoded credentials or unsafe deserialization.

def packet_callback(packet):
 global packet_stats
 packet_stats["total_packets"] += 1
 if packet.haslayer("HTTPRequest"):
 packet_stats["http_requests"] += 1

def start_packet_sniffer():
 sniff(prn=packet_callback, store=False, timeout=60) # Capture packets every 60 seconds

if __name__ == "__main__":
 threading.Thread(target=start_packet_sniffer, daemon=True).start()

Adding packet sniffing to our code, mainly just because we can.

Containerization and cloud interaction

Sometimes you need more control over your containers than Dockerfiles can provide. Maybe you need to dynamically build images based on environment variables, or automate container management:

7. Docker SDK for Python: (docker-py) When you need to automate Docker operations programmatically. I use it to clean up old containers and images, check container health, and build images dynamically from configuration files. More powerful than chaining Docker CLI commands.
6. Dagger: (dagger-io) Write container build logic in Python instead of Dockerfiles. Need to build multiple similar containers with slight variations? Want to pull in files from different sources based on environment? It lets you programmatically define container contents and build steps. (Bonus: this same code can later be used in CI/CD pipelines.)

import dagger

async def main():
 # Connect to the Dagger Engine
 async with dagger.Connection() as client:
 container = (
 client.container()
 .from_("python:3.11-slim")
 .with_exec(["python", "-c", "print('Hello from Dagger!')"])
 )

 # Execute the container and get the output
 output = await container.stdout()
 print(output)

if __name__ == "__main__":
 import asyncio
 asyncio.run(main())

Using Dagger makes it easy to stay in Python while building containers.

Building command-line tools

Let's build a CLI version of our service status dashboard in Python.

Every useful script eventually needs to become a proper tool. When your team starts asking “can I use that script too?” it’s time to enhance your CLI skills:

5/4. Click/Typer: (Click, Typer) Turn your quick scripts into proper CLI tools. Instead of remembering python cleanup.py --older-than 7 --dry-run, you get tab completion, help text, and proper argument handling. Typer is particularly nice if you’re using modern Python - it uses type hints to build the CLI interface.
3. Rich: (rich) Make your terminal output actually readable. Need to display tables of data? Want progress bars for long-running tasks? Rich makes it easy to build user-friendly CLIs. I use it to turn wall-of-text logs into colored, formatted output that helps spot problems quickly.

2. The essential toolkit

The foundation of any DevOps tooling consists of these essential libraries:

Requests (requests) - For making HTTP requests
Pytest (pytest) - For testing your code
logging (Python logging) - For structured logging
YAML/JSON/TOML parsers (PyYAML, json, toml) - For config file handling

While not flashy, these libraries form the backbone of every tool I build. And so I update my little tool to have some tests, some logging and store the api status urls in a toml file.

Now, #1 and it’s IaC time. I can get my dashboard onto AWS in a lambda using Pulumi.

# Build and push the Docker image to ECR
image = awsx.ecr.Image("app-server-image",
 repository_url=repository.repository_url,
 context="..",
 dockerfile="Dockerfile",
 platform="linux/amd64"
)

Push my Image to ECR

1. Pulumi

While most tools in this list excel at quick solutions, infrastructure demands a more robust approach. This is where Pulumi stands apart. It’s not just another scrappy tool—it’s an enterprise-grade Infrastructure as Code platform that happens to harness Python’s power and flexibility.

With support for real Python code instead of configuration languages, you can define, deploy, and manage your cloud resources using familiar programming concepts and practices.

With Pulumi, you get:

Complete programming language support for using loops, functions, and classes in your infrastructure code
Infrastructure testing capabilities to validate your deployments before they go live
Container deployment solutions for managing Docker images and Lambda functions
Rich AWS integration for comprehensive cloud resource management
Infrastructure automation to streamline your deployment workflows

# Create a Lambda function
lambda_function = aws.lambda_.Function("custom-lambda",
 name="custom-lambda",
 package_type="Image",
 image_uri=image.image_uri,
 role=lambda_role.arn,
 timeout=30,
 memory_size=512,
 environment={
 "variables": {
 "FLASK_ENV": "production"
 }
 }
)

Start it up in a Lambda

Manage your cloud in Python

Define AWS Lambda, ECR, and the rest of your infrastructure with real Python instead of YAML, and reuse the same loops, functions, and tests you already know.

Get started

Bringing it all together

Dashboard is still pretty basic, but now it's on AWS with a little Python infrastructure code.

Here’s where Python excels - combining these tools to solve real problems. For example:

Need a dashboard for infrastructure costs? Combine Flask with the AWS SDK and Pulumi Insights.
Want to automate container cleanup? Mix Docker SDK with Schedule for regular maintenance.
Building a custom GitHub webhook processor? Use Flask to receive webhooks, RQ for asynchronous handling, and Rich for readable logs.

Start simple (maybe just a script with Requests and PyYAML), then gradually add more tools as needed. That’s the beauty of Python in DevOps - you can start small and grow your solution naturally.

The art of getting things done

The tools I’ve covered span the full spectrum of DevOps needs—from quick scripts to enterprise-grade solutions. Python’s versatility makes it uniquely suited for both ends of this spectrum:

For urgent needs, tools like Schedule, Rich, and Click let you rapidly prototype solutions.
For long-term infrastructure, Pulumi and Airflow provide the robustness needed for production systems.
For everything in between, Python’s extensive library ecosystem helps you gradually evolve quick fixes into mature tools.

The key is knowing when to use each tool.

AI Engineering Lessons from Building Pulumi Copilot

Artur Laksberg — Thu, 12 Dec 2024 13:56:13 -0500

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

Building AI-powered developer tools comes with unique challenges, and now that we’ve launched our REST API, we want to share some lessons we’ve learned building Pulumi Copilot, an AI assistant for cloud infrastructure.

One of the big challenges was determining what ‘working’ really meant. So when a message landed in our feedback channel after months of rigorous testing - ‘Your tool doesn’t know anything!’ - it caused some mild panic. We’d just made some changes, so we braced for the worst. But our evals were still looking strong, so what was going on?

The user was attempting to force-delete a stack that still contained resources. But when we dug deeper, we found something fascinating: Copilot had confidently suggested a ‘–force’ flag, which would have been a logical solution… except this flag doesn’t exist in Pulumi. Our AI was hallucinating exactly what the user wanted. But this wasn’t just a bug - it was the first of many insights that would reshape how we approach AI-powered tools.

To understand how we got here - and why this “error” actually taught us something valuable about our product - let’s start with the core challenge we faced: balancing traditional software engineering with this new world of prompt engineering.

Engineering for Reality: Prompt Engineering vs Software Engineering

When building LLM-powered applications, it’s tempting to throw every task at the model. Modern LLMs can generate code, format text, and create clickable links. But this approach carries hidden costs.

Working on Copilot taught us a key lesson: let the LLM do what it does best and use good old imperative code for everything else.

Take a seemingly simple feature: listing a user’s Pulumi stacks with clickable links based on data from a backend API. Our first implementation used a complex prompt instructing the LLM to construct URLs in the format app.pulumi.com/org/project/stack. The prompt explained the format, provided examples, and asked the LLM to generate these links from JSON data it had.

It worked - almost all of the time. But we were seeing occasional malformed URLs and more importantly, this was burning input tokens (and money) on a complicated prompt that made AI construct strings that could be deterministically generated.

The solution was straightforward: generate the full links in the backend service and include them directly in the context. The LLM then needs no instructions on how to create them. Simple stuff, but it gave faster responses and perfect URLs at a lower cost.

When you find yourself writing elaborate prompts to handle structured data transformations, stop and ask: Could traditional code do this better? Could this be decomposed so that the LLM does less?

To validate this approach, we tested Copilot ourselves to see what worked.

Copilot in Action: Real-World Dogfooding

The internal testing phase taught us invaluable lessons about how people would actually use the tool. We watched our team try Copilot in their daily work, and three common use cases emerged:

Debugging Deployments: LLMs clearly excel at summarization. One of the first questions our internal users asked was, ‘Why did my latest infrastructure deployment fail?’ Using Copilot to extract a clear natural language explanation requests like these has been a clear win.

Understanding Complex Infrastructure: Copilot helped our engineers gain insights into Pulumi’s own infrastructure. Asking, ‘How many resources are in production?’ ‘What expensive compute is running’ or ‘What version are the EKS clusters in EU?" shows the value of allowing users to express infrastructure questions in natural language.

Generating Code: One of the first queries logged was, ‘I want a static website on AWS behind a CloudFront CDN.’ Another came from a Solutions Engineer, tasked with demonstrating Pulumi’s CrossGuard policy engine to a prospect, asking Copilot to generate policy code.

These early experiences showed the value of Copilot. But they also revealed the need for a systematic approach to handling diverse user queries. This led to the development of what we call skills.

Skillful Slicing: Modular Mastery

As Copilot grew, we broke it into smaller pieces we call skills. Each skill does one specific job. The Insights skill handles queries about resource usage and configuration (“How many S3 buckets do I have?”), the Cloud Skill interacts with the Pulumi Service API to manage infrastructure (“Show me my stacks.”), the Code Skill generates Pulumi code snippets (“Write a Typescript program…”), and the Docs Skill retrieves information from Pulumi documentation (“How do I use update plans?”).

When you ask Copilot something, it figures out what you need and picks the right skill for the job – like a manager deciding which expert to send your question to. This function-calling approach, orchestrated by a component we call the “outer loop,” allows Copilot to access and process information beyond its internal knowledge base.

Fortunately, Pulumi Cloud already exposes a rich API - in fact, this is what powers the Pulumi Console and the Pulumi CLI - so all we had to do is to build a skill that maps the user query to the appropriate Pulumi Cloud REST API. A question like “Show me my stacks” translates into the List Stacks API call. A question like ‘Show me my untagged EC2 instances’ is a bit more complex but it breaks down into clear components - resource type (EC2), filter condition (untagged) - that route to the Insights skill. This mapping helped us handle the many ways users phrase the same technical request.

Refining this routing system revealed another opportunity: streamlining the Debug button workflow.

Debug Dispatch

Originally, when a user clicked ‘Debug with Copilot’, the system would send a text query like “Analyze this update and explain any errors.” Copilot would then:

Determines the user wants to analyze an update
Identifies which API to call
Calls the API
Summarize the result

Having the LLM figure out what the user wants is the right approach in general, but in this case, we already know the user’s intent - they clicked a debug button. So, we can directly call the analysis API to get the results and use the LLM solely for what it does best: summarizing technical output into clear, actionable explanations.

This is another small win for the “Software Engineering over Prompt Engineering” approach. Traditional code handles the predictable parts, while AI focuses on the human-facing explanations.

But while minimizing the LLM workload helped with efficiency, we soon faced an even trickier challenge: the deceptive polish of AI-generated outputs.

The Illusion of Correctness

Large language models excel at generating well-structured, grammatically correct output. They make neat tables, tell good stories, and generally sound confident. That’s what makes them dangerous because this polished presentation can mask underlying flaws in the information itself, creating a false sense of confidence for users.

One of our early testers, Pablo, a data engineer at Pulumi, encountered this firsthand. He posed a query to Pulumi Copilot, asking for a summary of resources within a specific project. The response he received was impeccably formatted, neatly categorizing resources by type and providing counts for each. It looked right, and for us humans sometimes looking right carries a lot of weight.

However, a closer inspection revealed the numbers were way off. Copilot had asked for the wrong data and then summarized it beautifully - but incorrectly. This highlighted our next challenge: how do you systematically test a system that can be confidently wrong while sounding completely right?

Testing the Untestable: Validating LLM Outputs

When testing traditional code, we expect consistent, predictable outputs. With LLMs, even correct answers can vary. Here’s how we tackle this challenge.

Our first approach was simple: keyword checks. For example, when testing the update analysis feature, we checked if the LLM’s response included the word “security” and described the error. This worked for straightforward cases but broke down quickly. Take a question like, “How many Lambdas am I running?” The LLM might give the right numbers but skip the word “running,” failing the test even though the answer was correct.

These early failures revealed the limitations of keyword-based validation and underscored the need for a more nuanced approach. Inspired by platforms like LangSmith and Promptfoo, we began leveraging LLMs themselves as evaluators. For deterministic tasks, simple keyword checks suffice, but for more complex scenarios—like assessing whether a response answers a specific question—we rely on an “LLM Judge.” This approach balances efficiency and flexibility, reserving LLM evaluation for cases where it truly adds value. The test suite now integrates both methods, running against every code change to validate response content, accuracy, and format.

The eval suite keeps getting more robust, which means that when new AI models drop, we can quickly catch any weirdness before it hits production. The generative AI space moves crazy fast and the code changes a lot, but the evals are a safety net - catching hallucinations, maintaining quality, and making sure we don’t ship anything that’ll annoy our users.

So, while hallucinations are now much rarer, what about that one with the --force flag? Yes, it’s “just another bug,” but it taught us something fascinating about these AI errors.

You might also like:

LLMs think like humans (sort of)

The --force hallucination wasn’t totally wrong - it was revealing what users intuitively expect from the CLI, and the LLM accidentally showed us what was missing. Force deletion is a common pattern across developer tools, and the LLM, trained on vast amounts of documentation and code, simply reflects these established conventions.

This has fundamentally changed how we view hallucinations. While the team works constantly to minimize them – and our eval work means they happen way less frequently – some of them are clearly product signals. The LLM, in this light, becomes an unexpected source of user research, drawing on its training across thousands of developer tools and experiences.

This insight is one of the key lessons of building Copilot:

Minimize LLM Usage: Let traditional code handle deterministic tasks, reserve LLMs for natural language work
Decompose into Skills: Break complex tasks into modular units that combine LLM and traditional code appropriately
Test Rigorously: Use multiple validation approaches, including LLMs testing LLMs
Learn from Hallucinations: Sometimes incorrect outputs reveal user expectations
Learn from Users Continuously: User interactions improve our AI systems - from training better skills to catching hallucinations and revealing product opportunities.

These lessons helped shape our latest release: the Pulumi Copilot REST API, now available in preview. You can integrate these same capabilities and skills into your own tools and workflows. Whether you’re building CLI extensions, chat integrations, or automated deployment checks, the API provides a contextual understanding of Copilot. Try it out.

We can’t wait to see what you build!

105 Ways to Run Containers: The Cloud Container Iceberg

Adam Gordon Bell — Tue, 03 Dec 2024 10:29:54 -0500

If a computer can be connected to the internet, someone has tried to run a container on it. From quantum computers to smart toasters, from phones in AWS racks to CI pipelines, there’s many ways to deploy containers. While most people only know about the mainstream cloud providers at the surface, there’s actually a vast world of increasingly unusual and specialized options beneath.

This guide serves two purposes: to showcase the surprising breadth of container options available today, and to help you understand the full spectrum of choices – from practical to very experimental. Whether you’re looking for production-ready solutions or just curious about what’s out there in the dark depths, you’ll find something interesting here.

Constraints

But first, we need ground rules for our container adventures. All listed items should:

Technically possible: Must work today with existing tech, even if the setup makes you question your life choices.
Runs real containers: OCI containers only. No JavaScript “containers” pretending to be the real thing.
Actually doable: You can set this up right now with a laptop and a credit card. No theoretical schemes allowed.

Above the iceberg – The usual suspects

These are the familiar faces of container orchestration—the ones you’d introduce to your parents. Reliable, well-documented, and about as exciting as watching paint dry.

Amazon Elastic Kubernetes Service (EKS)
Amazon Elastic Container Service (ECS)
AWS Fargate
Azure Kubernetes Service (AKS)
Google Kubernetes Engine (GKE)
Google Cloud Run
Amazon EC2
AWS App Runner
Azure App Service for Containers
Google Compute Engine with Containers

On the iceberg – The slightly less obvious

As we dip just below the surface, we find services like a rare B-side acoustic release from a big band. It’s not as mainstream but still a respectable choice for running your containers.

AWS Elastic Beanstalk – Deploy containers, debug YAML
Amazon Lightsail Containers – AWS with the guard rails on
Azure Container Apps – Serverless, but complicated
Google App Engine Flexible Environment – PaaS for Containerized Apps
Azure Container Instances (ACI) – Kubernetes without the chaos

The kubernetes multiverse – Because one Universe of k8s wasn’t enough

Diving deeper, we discover there’s an entire multiverse of regional and specialty K8s providers, helpfully cataloged by the CNCF. Think parallel universes, but instead of evil twins, you get slightly different container runtime implementations.

The Global Titans – The international tech giants couldn’t resist joining the K8s party: Huawei Cloud Container Engine, Tencent, and Baidu bring their own flavor of cloud-native orchestration to the mix. Add in Samsung, China Mobile KCS, JD Cloud, Inspur, and QingCloud, and you have enough options to start a diplomatic incident.
The European Union of Containers – Europeans take their container privacy seriously: IONOS and plusserver bring German engineering to K8s, OVH adds French flair, Exoscale offers Swiss precision, and STACKIT, SysEleven, WIIT, Gcore, and Previder ensure your containers comply with every possible EU regulation. Conoa Proact throws in some Swedish design principles for good measure.
The Asia-Pacific Pod Party – The APAC region isn’t missing out: NHN brings Korean optimization to your clusters, while ARDC Nectar and Catalyst let you run containers upside down in Australia and New Zealand. TKS Cloud Service ensures your containers can run anywhere from Tokyo to Tasmania.
The Specialists’ Society – For when regular K8s isn’t quirky enough: OKESTRO VIOLA orchestrates your containers with musical flair, while Petasus, De Novo, Aruba, and Taikun provide enough specialty K8s options to ensure every cluster is a unique snowflake.

Below the water – Kubernetes everywhere – Let a thousand K8s bloom

Let’s plunge deeper. Because one Kubernetes cluster is never enough, why not sprinkle them everywhere? On-premises, in the cloud, under your desk—there’s a K8s solution for all that.

Amazon EKS Anywhere – AWS container but in your own data center
Amazon ECS Anywhere – Like EKS Anywhere but ECS
Azure Arc – Hybrid cloud for masochists
Google Anthos – Hybrid cloud something something
AWS Outposts – AWS, now with hardware shipping delays
Azure Red Hat OpenShift – Azure but with a fedora
VMware Tanzu – Because VMware heard you like Kubernetes
Red Hat OpenShift – Fedoras on AWS
IBM Cloud Kubernetes Service – Kubernetes with a touch of enterprise nostalgia
Oracle Container Engine for Kubernetes (OKE) – For the brave souls trusting Oracle
Alibaba Cloud Elastic Container Service (ACK) – K8s with Chinese characteristics
DigitalOcean Kubernetes (DOKS) – K8s in the ocean, hopefully no sharks
Linode Kubernetes Engine (LKE) – Lightweight K8s for the minimalist
Vultr Kubernetes Engine (VKE) – Yet another place to run your pods
Scaleway Kapsule – French elegance meets Kubernetes
Hetzner Cloud Managed Kubernetes – German engineering for your clusters
Nutanix Karbon - Because regular Kubernetes wasn’t converged enough
Metal Stack Cloud Kubernetes - For when virtual machines are too virtual

You might also like:

Middle of the iceberg – Serverless and function containers – Now you see me, now you don’t

As we descend to the depths, we find containers with commitment issues. They appear, do their job, and disappear.

AWS Batch – Containers that only work the night shift
AWS Lambda – Lambda’s containers for when a 250 MB zip file wasn’t enough baggage.
Azure Functions – Roll your own functions in containers and hope for the best.
Google Cloud Functions – Containers pretending to be functions, but now on Google Cloud.
Alibaba Function Compute – You get the idea.

Bottom of the iceberg – Specialized and hybrid platforms – The swiss army knife of container hosting

Down here, containers run in ways that make people question your life choices. These platforms let you host containers in environments many fear to tread.

AWS Proton – Automate your deployments and run a container
Azure Virtual Machines with Docker – When you miss managing servers but love containers
IBM Cloud Code Engine – Run anything, anywhere, if you can figure it out
Alibaba Elastic Container Instance (ECI) – Containers that disappear faster than you can say “Jack Ma”

Below the iceberg – Containers on the edge of reason

In these lightless depths, you want your containers to live life on the edge—literally. Because nothing says “enterprise-ready” like running production workloads on a smart toaster.

AWS IoT Greengrass – Run AWS Lambda and Docker containers on everything from factory equipment to smart fridges. Nothing says “digital transformation” like your smart fridge running machine learning models.
Azure IoT Edge – Like Greengrass, but only when your toaster prefers Windows updates.
AWS Snowball Edge – For when ‘cloud migration’ means paying FedEx to move your containers. Yes, it’s literally a box they mail you that can run containers.
AWS Snowcone – An even smaller box that runs just enough containers to make you appreciate your data center. Perfect for when you need AWS in places where AWS doesn’t want to build a region.
BalenaCloud IoT Device Management – A platform that lets you remotely deploy and manage containers on fleets of Raspberry Pis. Because nothing says “enterprise IoT” like running your production stack on $35 computers.
Fly.io – Global Edge Container Deployment, because apparently just picking one region was too simple.
Edge Node Service (Tencent Cloud) – Edge deployment behind great firewall.
StackPath Edge Computing – More edge containers.
Koyeb – Serverless Edge containers.
Platform.sh – Multi-Cloud Edge containers.
StarlingX - Edge computing platform for when 5G and IoT buzzwords aren’t enough
ZStack Edge - Edge computing with Chinese characteristics, now at the edge of the network, too

The abyss – Where containers fear to tread

And now we reach the darkest depths. For the bravest souls who look at conventional container platforms and think, “Not edgy enough.” Here lie the most unusual ways to run containers in production.

Amazon GameLift – Run your containerized services disguised as game servers. High scores may apply.
AWS Device Farm – Deploy containers on hundreds of actual phones in AWS racks. Perfect for when “mobile-first” goes too far.
AWS AppStream 2.0 – Stream Windows to run Docker to stream containers. It’s virtualization all the way down.
Google Cloud Dataflow – Hide containerized services in pipeline steps named “daily_data_normalize_37”. Hope your SRE enjoys debugging ETL.
Google Cloud Composer – Your containerized microservice masquerades as an Airflow DAG. Because nothing says reliability like cron.
Google Colab & Jupyter Notebooks – Run containers in data_preprocessing_step.ipynb. Watch ML teams debug production.
Azure Quantum – Package quantum containers with az quantum job submit. Now, your uptime is truly uncertain.
Azure Cognitive Services in Containers – Offline AI containers that promise not to become self-aware. Terms and conditions may apply.
Azure DevTest Labs – Run containerized test environments that self-destruct like Mission Impossible tapes. Perfect for when temporary becomes permanent.

Beyond the abyss

There you have it, the container iceberg. And I didn’t even have room to add the extra cursed ways you can run containers. Here is some of what didn’t make it:

Cloud IDEs and Workspaces – Want to turn your development environment into a sneaky container platform? Take your pick: GitHub Codespaces, GitPod for the GitLab enthusiasts, or Eclipse Che for the masochists who miss Eclipse crashing locally. JetBrains Space or Replit? All this lets you run a container until you close your browser tab.
GitHub Actions et al. – Ever wanted to run a production service disguised as a CI job? Just label your production API start-up script as “integration_test_step_42”, add a sleep 21600 and a call to retrigger itself, and hope nobody notices your “test” has been running for months. Apply this same trick to AWS CodeBuild, GitLab CI/CD, CircleCI, Azure Pipelines, Travis CI, Bitbucket Pipelines, Drone CI, Jenkins Cloud, TeamCity Cloud, and Google Cloud Build for Bonus points.

Managing your container zoo with pulumi

Whether you’re deploying containers to mainstream cloud providers or attempting to run them on quantum computers (really, please let us know how that goes), Pulumi helps you manage it all with real code instead of config file soup. Our Kubernetes support means you can wrangle your containers wherever they might roam – from AWS to Azure, GCP to Kubernetes, or even that smart toaster you’re eyeing for production deployment.

With Pulumi, you get:

Cloud-native container orchestration across all major platforms and Kubernetes distributions
Infrastructure testing to ensure your containers deploy correctly (even in the weird places)
Provider templates for when you invent yet another way to run containers
Stack references for managing multiple environments without losing your mind
Automation API for programmatically herding your container fleets

Ready to bring some sanity to your container deployment strategy? Get started with Pulumi:

Try Pulumi for Free

The final word

At this point, 105 options in, you might wonder: “Has cloud container hosting gone too far?” Well, I’ve a solution to propose:

At Pulumi, we’ve got providers for all the usual suspects and, surprisingly, many more in this container circus. And if you’ve figured out how to run containers on a quantum computer inside a data pipeline in New Zealand, you can write a provider for that, too.

So, use Pulumi and we will figure out how RedHat-Tanzu-Greengrass-Icecore-Device-Farm works and provide you a sane interface to it. You’ve got other things to worry about.

Pulumi + uv: Fast Python Package and Project Management

Adam Gordon Bell — Wed, 27 Nov 2024 12:43:45 +0100

Continuing our work to bring the best of modern Python to Infrastructure as Code, we are excited to announce built-in support for uv in Pulumi. uv is an extremely fast Python package manager that can install dependencies up to 100x faster than traditional tools, providing one of the fastest ways to manage your Python dependencies and virtual environments.

Why uv?

Listen in on this discussion between Adam Gordon Bell and Julien Poissonnier as they discuss Pulumi + uv and why uv is so fast!

uv brings several key advantages to your Python development workflow:

Blazing Fast Performance: Written in Rust, uv can install packages 10-100x faster than pip
Reliable Dependency Resolution: Advanced resolution algorithm ensures consistent environments
Drop-in Replacement: Works seamlessly with existing requirements.txt and pyproject.toml files
Built-in Virtual Environment Management: Simplified environment handling out of the box

Using uv in Pulumi

To use uv to manage your Python virtual environment and dependencies, set the toolchain option to uv in your Pulumi.yaml file:

name: uv-goes-brrrr
runtime:
 name: python
 options:
 toolchain: uv
 virtualenv: .venv

If you have uv installed, you can run pulumi new python and select uv as the toolchain to use for installing dependencies and running the program.

You can find more information on how to use uv in Pulumi in our Python documentation.

Fargate vs EC2

Adam Gordon Bell — Mon, 18 Nov 2024 08:42:34 -0500

Building an EKS cluster requires choosing how your containers will actually run - either on EC2 instances you manage or through AWS Fargate’s pod-by-pod approach. The differences can be pretty dramatic in practice. I’m setting up a demo cluster right now using Pulumi, so let me show you what I mean.

Here is my Fargate cluster:

import pulumi
import pulumi_awsx as awsx
import pulumi_eks as eks
import pulumi_kubernetes as k8s

# Create VPC with default CIDR
eks_vpc = awsx.ec2.Vpc("eks-vpc",
 enable_dns_hostnames=True)

# Create minimal EKS cluster
eks_cluster = eks.Cluster("eks-cluster",
 vpc_id=eks_vpc.vpc_id,
 private_subnet_ids=eks_vpc.private_subnet_ids,
 fargate=True,
)

pulumi.export("kubeconfig", eks_cluster.kubeconfig)

And when I look in k9s I see this:

You can see that each pod is on its own node. Even the core-dns pods in the system namespace are each on their own nodes. That is a lot of nodes, and it’s going to keep growing with my cluster.

If I change my cluster to be backed by EC2 nodes, I get something different.

eks_cluster = eks.Cluster("eks-cluster",
 vpc_id=eks_vpc.vpc_id,
 private_subnet_ids=eks_vpc.private_subnet_ids,
- fargate=True,
+ instance_type="t3.medium",
+ desired_capacity=2,
+ min_size=1,
+ max_size=2,
)

Now, there are only two nodes, with the pods distributed among them. Those are my two t3.mediums.

So when should you use Fargate, and when EC2 for EKS? Well, it really all relates to what we are seeing here with the number and size of nodes.

Let me explain.

Bin Packing

Normally, the Kubernetes scheduler has to solve a bin-packing problem—fitting M pods across N nodes based on available resources. This is a straight-up resource optimization problem of balancing CPU, memory, and other resource demands across the cluster.

This can be surprisingly complex in practice. Teams must balance selecting the right instance types for different workload needs while managing multiple node groups. Adding to this complexity are scheduling rules and resource quotas that need careful configuration. Perhaps most challenging is striking the right balance between high resource utilization and maintaining enough headroom for spikes in demand.

With Fargate, AWS sidesteps this challenge by providing a correctly sized bin for each pod. Each Fargate pod runs on its own dynamically provisioned, right-sized mini-environment, where the “bin” (the Fargate instance) is sized to match the pod’s requested resources. This means there’s no need for Kubernetes to optimize resource usage across a pool of shared nodes, as each pod effectively has its own “container” provided by Fargate that fits it’s needs precisely.

In other words, AWS effectively pushes this bin-packing responsibility to itself, so you don’t have to worry ( as much ) about it.

While Fargate abstracts away node management, it’s important to understand that Fargate pods still run on EC2 instances behind the scenes. And each Fargate pod gets its own ENI (Elastic Network Interface) that can sometimes limit your scaling because pods aren’t sharing a network namespace.

Pros and Cons

So yeah, Fargate abstracts away the complexity of managing nodes and scaling clusters by shifting bin-packing to AWS. With its extensive infrastructure, AWS is arguably better positioned to optimize placements at scale. They can monitor their entire cloud landscape, placing Fargate instances dynamically across available resources. This allows them to potentially leverage underutilized resources in ways that would be challenging or impractical at an individual account’s scale.

Responsibility	EC2	Fargate
Instance Types	You pick size and provision nodes	AWS provisions per pod request
Pod Isolation	You prevent noisy neighbors	AWS isolates each pod
Resource Allocation	You find space on nodes	AWS creates dedicated space

Customer vs AWS Responsibility EC2 vs Fargate

Ideally, this frees you to focus on the services rather than on the underlying infrastructure, but in practice, it’s a bit more complex.

Since each pod is treated as a standalone instance, you lose out on the cost efficiencies of co-locating multiple pods on a single node. Even though AWS is “solving” the bin-packing problem globally, to do so, they need to introduce hard barriers, and while noisy neighbors, therefore, aren’t a problem, sharing resources is now impossible. In a way, you are throwing away the run time benefit of containers, the shared kernel virtualization, for something more like VMs. And just like VMs, Fargate pods take longer to start than pods on existing EC2 nodes (typically 30-60 seconds vs near-immediate starts).

A Note on Scaling

Both EC2 and Fargate work with Kubernetes’ Horizontal Pod Autoscaling (HPA), which adds or removes pods based on metrics like CPU usage. The difference? With EC2, you need enough nodes available to handle new pods. With Fargate, AWS automatically provisions resources for each new pod - though you’ll face that 30-60 second startup delay and potential VPC limits since each pod needs its own network interface.

Feature	Fargate	EC2
Cost	Higher per pod	Lower cost per pod
Scalability	Scales on-demand, isolated resources	Requires manual node scaling
Resource Sharing	Not possible	Efficient resource sharing among pods
Startup Time	30-60 seconds	Near instant on existing nodes
Use Cases	Burst workloads, isolated pods	Cost-sensitive workloads, Resource sharing workloads

Maybe this will make more sense with an example.

Workload Example: Static Analysis

Have you ever had to run CPU-intensive batch jobs alongside lighter web services?

In a previous role, when my team and I were tasked with putting our static container image analyzer into production, we went the typical Kubernetes route, throwing it into an EC2-backed K8s cluster. The analyzer was built in Scala, running on the JVM, and was designed to take in work from a Kafka topic. Each analysis ultimately led to the results being sent to a database. We also had scheduled daily rescans of existing images whenever vulnerability lists were updated.

Then, a web app connected to this database and showed all your results. It was stateless and did far less work per request, mainly just answering an HTTP request by talking to a database.

The hard part though is that each analysis job involved uncompressing and analyzing Docker layers from S3 and could use a lot of CPU. Memory needs were around 2 GB per instance, which kept the JVM happy, but long sustained bursts in CPU usage were common, especially during intensive parts of the analysis.

In our K8s setup, the biggest pain point was handling this analysis workload’s burstiness and keeping a bunch of stateless web apps up and answering requests. We ran multiple instances of the static analyzer. Still, the analyzer was the worst type of noisy neighbor to other services in the cluster, so we’d end up over-provisioning to ensure we had enough headroom, which led to wasted resources and extra cost. The setup worked, but was not great.

Thinking back, I can see how Fargate could’ve streamlined things here. With Fargate, each static analyzer instance would run in its own isolated environment, so there’d be no resource contention or need for pre-provisioned node capacity. Each pod would be fully provisioned with the exact CPU and memory needed, and we could do some sort of on-demand scaling to deal with backlogs of analysis work in the Kafka topic.

Fargate on EKS could be a great solution for this type of chunky, resource-heavy macro-services. There are probably other ways to handle this situation, and we worked on decomposing the analysis into smaller chunks, but Fargate might have been a good solution.

But there are lots of situations where Fargate is less of a fit.

You might also like:

Example: Go Services for E-commerce

Ok, another workload that comes to mind is this one from a friend - He worked on an e-commerce backend with many lightweight Go services, each handling specific tasks—user profiles, catalog browsing, order processing, and notifications. These services were network-heavy, handling gRPC calls and database requests, but they didn’t need much CPU or memory per instance.

Fargate wouldn’t have made sense here—it would have isolated each pod, preventing resource sharing and driving up costs without real benefit. For lightweight services like these, EC2-backed Kubernetes is perfect, keeping costs low and maximizing resources by sharing nodes across small, efficient services.

So, for lightweight micro-services that can share resources, EC2 nodes with many pods each are probably the way to go.

Other Fargate Cases

Fargate can excel with long-running event-driven and bursty workloads, where you need compute resources briefly or unpredictably. Think file processing jobs that trigger on uploads, or batch processing that runs sporadically. When jobs are intermittent, Fargate’s ability to scale from zero and its isolated resources can outweigh its higher per-pod cost and startup delay. Just ensure your workload can tolerate the 30-60 second cold start.

Fargate has some hard limitations that might rule it out for your use case:

No DaemonSets
Maximum of 4 vCPU and 30GB memory per pod
Storage limitations: Only ephemeral storage up to 20GB per pod, no persistent volumes
No GPU support

Fargate vs EC2 Pricing

Ok, EC2 costs less than Fargate on a pure cost-of-compute basis. If my t3.medium ($0.0416/hr) can typically run 6-8 pods reliably, and this works out to under $0.01/hr per pod. Comparable Fargate pods (0.5 vCPU/1GB) cost about $0.025/hr each, so 2.5x.

These calculations can change based on how much headroom you need on EC2 and how many containers you can run on a given EC2 instance. Still, Fargate is just more expensive unless all your containers fit the exact computing sizes offered by Fargate. And don’t forget, AWS is likely incurring costs maintaining the warm compute capacity for launching Fargate pods, so it shouldn’t be surprising that Fargate is more expensive.

But really, pure compute cost calculations are an insufficient metric for making this decision - you need to factor in your team’s operational capacity, scaling patterns, and isolation requirements.

( Also, if Fargate makes sense because of your workload but the cost is a concern, then look into Fargate Spot instances, which can be up to 70% cheaper, but be aware that AWS can terminate them at any point to reclaim capacity. )

Misconceptions About Fargate

Fargate saves you from managing servers With containerized workloads and immutable infrastructure, your EC2 nodes are minimal, standardized, and automatically replaced. You’re not logging in, installing packages, or managing configuration - everything runs in containers. When a node needs updates, you replace it entirely.
Fargate handles all scaling challenges While it handles node scaling, you still face ENI limits, longer pod startup times (30-60s), and potential networking bottlenecks.
It’s simpler and more reliable It may be, but it may not be. The complexity shifts to networking, pod configuration, and working within Fargate’s constraints. You trade node management for new operational challenges like slower startups and possible networking complexity.

Managing Container Orchestration with Pulumi

Whether you choose EC2 or Fargate, Pulumi makes it easy to manage in your favorite programming languages. With its rich AWS support and infrastructure as code capabilities, you can define, deploy, and manage your entire Kubernetes infrastructure using familiar tools and practices. Pulumi’s AWS provider helps you implement the right solution for your needs.

With Pulumi, you get:

Complete control over EKS cluster configuration, including node types, scaling policies, and networking setup
Built-in support for both EC2 node groups and Fargate profiles within the same cluster
Infrastructure testing capabilities to validate your cluster configuration before deployment
Automation APIs for programmatic infrastructure management and custom tooling

Ready to streamline your EKS infrastructure management?

Try Pulumi for Free

Why Not Both

I hope my examples point to a pragmatic approach: use EC2 as your foundation for efficient microservices that can share resources, like our Go e-commerce setup. Add Fargate when you need isolation or flexible scaling - like for the static analysis workload. Better yet, combine both approaches in the same cluster, for cost-efficiency and isolation where needed.

If you’re interested in using Pulumi to manage EKS infra, here’s a practical EKS setup guide guide to get your started.

Now, back to getting this demo cluster up and working!

When to Use Cosmos DB

Adam Gordon Bell — Mon, 11 Nov 2024 17:39:54 -0500

When should you use Cosmos DB on Azure? It’s a hard question to answer.

Azure Cosmos DB tries to do it all - which makes it both powerful and, honestly, kind of confusing. The marketing doesn’t help either. Yeah, it’s a flexible database, but before you jump in, you need to understand how it actually works under the hood, what it’ll cost you, and what trade-offs you’re making by choosing it.

I’m going to break all that down. Here’s a roadmap:

TL;DR

The short version is: you should use it on Azure when you’d use DynamoDB on AWS. That’s to say, you should use it when you’re OK with being cloud-specific, when you want hands-off scalability, and when you know the query pattern and potential cost ramifications ahead of time.

But let’s start at the beginning.

What is Cosmos DB? Features & Architecture Explained

Cosmos DB is Microsoft high availability distributed database. First released as DocumentDB in 2014 and rebranded to Cosmos DB in 2017, its used internally at MS for Bing and Office 365 and available on Azure in all regions.

Types Of Databases

Different databases are good at different things. Relational databases focus on strong consistency and complex relationships. Key-value stores are optimized for speed and simplicity. Document databases trade schema rigidity for flexibility. NewSQL systems attempt to combine consistency and scale.

Type	Example	Best For
Relational	PostgreSQL	Traditional business applications with complex transactions
Key-Value	Redis	Caching and session management
Document	MongoDB	Applications with evolving data structures
NewSQL	Spanner, CockroachDB	Global applications requiring both scale and consistency
Dynamo-Inspired	DynamoDB, Cassandra	High write throughput, eventually consistent applications

Cosmos DB can look like each of those categories. This is because it’s a multi-model DB, with a number of APIs that correspond to each model type. Despite it having a SQL interface (often recognized as the most used and recommended interface), and despite it purporting to be multi-model, at its lowest layer, it is actually backed by a document-based storage layer. It is a powerful document database, one with tunable consistency, distribution by logical and physical partition, and more, but it’s helpful to know how the storage layer works to understand everything else. I’m going to show some examples, but first let’s clear up what Cosmos DB is not.

If you are going to use Cosmos DB, you should probably be using the Core API, also called the “API for NoSQL”. It’s the quickest to get new features. Confusingly, despite being called a NoSQL API, data is queried and added using SQL.

Cosmos DB Vs. SQL Server ( Azure SQL Database ) or Postgres

This is a personal opinion, but if you have a use case that is well served by a traditional relational database, you should choose that. Postgres and SQL Server are both great and both available on Azure ( Postgres as Azure Database for PostgreSQL ). They both have great query planners; they have great tooling for migration and backups, as well as schema management. The SQL API from Cosmos should not be considered an alternative to a relational database unless you have specific needs because it’s a different type of beast designed for different use cases. But let me show you.

If you grab our CosmoDB how to guide and pulumi up, you’ll see from that, the first hint of its document roots in that the pulumi namespace is Pulumi.AzureNative.DocumentDB, but it goes deeper because although I can insert and select with sql in Cosmos DB, there is no schema.

How to Insert & Query Data in Cosmos DB NoSQL

If I insert this customer:

CosmosDatabase database = await client.
 CreateDatabaseIfNotExistsAsync(databaseId);
CosmosContainer container = await database.
 CreateContainerIfNotExistsAsync(containerId, "/customerId");

// Insert a customer document
var customer = new Customer
{
 Id = Guid.NewGuid().ToString(),
 CustomerId = "12345", // Cosmos DB Partition key
 Name = "John Doe",
 Email = "john.doe@example.com"
};

await InsertCustomer(container, customer);

I can do selects against it like SELECT * FROM c WHERE c.customerId = '12345" but when I look in that container in Azure Data Explorer, I see something like this:

{
 "id": "1d4e26b3-7a02-4e97-8a9d-eccc0b2fd537",
 "customerId": "12345",
 "name": "John Doe",
 "email": "john.doe@example.com",
 "_rid": "<resource id>",
 "_self": "<document URI>",
 "_etag": "\"00000000-0000-0000-0000-000000000000\"",
 "_attachments": "attachments/",
 "_ts": 1609459200
}

As with any document database, rather than a schema, it’s just a JSON document. That’s all good, but because there is no schema, if I needed to do something like migrate name to fullname, how do I do it? Well, there are a bunch of strategies, but they all involve writing some application code to go record by record and move the value from name to full name.

So don’t be fooled by the SQL usage in the API. Cosmos DB is not a relational database; before you use a document database, you should be clear of the trade-offs.

Document Database / NoSQL Trade-Offs

Joins and Relationships: Cosmos DB lacks support for complex joins and foreign key constraints. Cosmos DB typically requires denormalization or application-level management of relationships.
ACID Transactions and Consistency: Cosmos DB supports ACID transactions but only within single partitions (more on partitions soon). Cross-partition operations in Cosmos DB may require manual consistency handling.
Schema Evolution and Migrations: Relational databases enforce a schema and support migration tools. Cosmos DB’s schemaless design offers flexibility but requires custom logic for data migrations, which can increase operational overhead with frequent schema changes.
Query Optimization and Indexing: Postgres, SQL Server, and any relational database will have a query optimization layer with a sophisticated planner and manual indexing control. Cosmos DB auto-indexes all fields, simplifying queries but potentially increasing costs. Custom indexing policies offer less control than you would see in SQL Server or Postgres.
Cost Structure: Relational database costs are tied to compute and storage resources. Scaling might be a challenge past some point, but costs are fixed. Cosmos DB uses Request Units (RUs) for billing and charging per read, write, and query. This can lead to unpredictable costs if not carefully tuned.
Operational Maturity: Relational databases are mature with mature solutions for backup, replication, and monitoring. Cosmos DB is fully managed but has fewer third-party tools. Postgres or another relational DB may be easier to integrate into existing workflows requiring mature operational support.

If your use case aligns well with relational databases, consider using PostgreSQL or SQL Server. You have multiple Azure options, including Azure Database for PostgreSQL and Azure SQL Database. There’s even Azure Cosmos DB for PostgreSQL, which confusingly uses native PostgreSQL combined with the Citus extension for distributed capabilities. It’s not based on Cosmos DB’s NoSQL DocumentDB storage layer and is a Postgres database.

Cosmos DB is best for other use cases, so on we go to ‘NoSQL’ document databases.

Azure offers various services under the “Cosmos DB” brand like Azure Cosmos DB for PostgreSQL, Azure Cosmos DB for MongoDB (VCore), and a managed instance for Cassandra. These are not based on the core Cosmos DB engine, they are not an API into Cosmos, they are a whole different things - hosted instances of specific database types.

Cosmos DB Vs MongoDB

So, let’s talk MongoDB then. Cosmos DB is multi-model in a specific way. It has a single storage layer, the DocumentDB, and then various APIs. One of these is a MongoDB interface. This Mongo interface, on top of Cosmos DB, might be a great choice if you are used to Mongo, because you can often just change your query string on your existing mongodb code and start using cosmos.

using MongoDB.Driver;
using MongoDB.Bson;

// Connect to Cosmos DB using MongoDB API
var client = new MongoClient("mongodb://<account>:<key>@<account>.mongo.cosmos.azure.com:10255/?ssl=true");
var db = client.GetDatabase("customersdb");
var collection = db.GetCollection<BsonDocument>("customers");

var docs = await collection.Find(
 new BsonDocument("customerId", "12345")
).ToListAsync();

That will totally work. You can query using MQL, assuming you avoid Mongo’s advanced features. Still, you will quickly need to understand Cosmos-specific things, like RUs and partitions, or else you might end up with unexpected performance or costs.

Reddit Azure users report some challenges with using the Mongo API, and Microsoft also recommends using the SQL-based “API for NoSQL” if you can. So Mongo fans may want to consider not choosing Cosmos’ Mongo API.

However, the big advantage of Cosmos over Mongo is distribution and scaling up. But to understand that, we need to explain partitions.

With MongoDB, you’re setting up shards, configuring keys, and managing replica sets. Sure, you get control, but you’re also the one staying up late managing infrastructure as your data grows.

In Cosmos DB, scaling and partitioning are more hands-off – you just have to enable ‘automatic scaling’ – but you need the correct partition key from the start. So you need to understand your access patterns and be aware that ‘automatic scaling’ does what it says, but costs will increase as you scale.

How Cosmos DB Partitions & Scales Data Efficiently

In MongoDB, a shard is a collection of data stored on a specific server, used to distribute data across multiple servers for horizontal scaling. Partitioning is really an improvement on this idea. Cosmos separates partitions into logical and physical partitions. You choose the logical partitions only and leave the DB to decide the best way to group these onto specific physical servers. The physical groupings are physical partitions, and the system manages scaling by handling data distribution across physical partitions without user intervention.

Cosmos DB Partitioning

Logical Partitions: Like MongoDB’s shards, logical partitions group documents with the same partition key. However, each logical partition can store up to 20 GB of data and is automatically managed by Cosmos DB.
Physical Partitions: Cosmos DB hides the complexity of managing the physical servers. Physical partitions store and serve one or more logical partitions, and Cosmos DB handles how data is distributed across physical partitions for you. This allows Cosmos DB to automatically scale and balance the load as needed.

Comparison To MongoDB

Feature	MongoDB Sharding	Cosmos DB Partitioning
Data Distribution	Manual sharding with shard key and replica sets	Automatic partitioning with partition key
Scaling	Manual scaling (add shards)	Automatic scaling (Azure-managed)
Operational Complexity	Requires manual setup and maintenance	Minimal operational overhead
Global Distribution	Requires manual setup of replica sets	Built-in global distribution
Consistency Models	Eventual (with strong for replica sets)	5 consistency models, including strong
Query Performance	Requires index management, can result in scatter-gather queries	Auto-indexing, but cross-partition queries can be costly
Cost Model	Based on instance, storage, and IO usage	Request Units (RUs) per operation, with potential for high costs

MongoDB Vs. Cosmos DB Cost Comparison

Your exact numbers may vary, but by my back-of-the-napkin math, Cosmos DB is cheaper more than MongoDB Atlas for similar workloads. It also has SLAs on read speed and scaling is much easier.

Service	Cost (500 reads/sec, 50 writes/sec, 10 KB document)	Pricing Model
MongoDB Atlas	~$387.62/month	Based on instance size (M30), fixed pricing
Cosmos DB	~$227.28/month	Request Units (RUs) based, scales with document size and operations

Cosmos DB Vs Cassandra

After learning about partitioning and seeing the Cassandra API, you might think Cosmos DB is basically managed by Cassandra. And there are some key similarities. In many ways, Cosmos DB is Cassandra-inspired, but there are key differences.

You might also like:

How Cosmos DB and Cassandra Relate

Partitioning: Both use logical partition keys for scalability to distribute data across nodes. Cassandra requires manual management, while Cosmos DB automates partitioning and scaling, reducing operational burdens, though at the cost of flexibility.
Tunable Consistency: Like Cassandra, Cosmos DB offers tunable consistency but expands on it with five levels: Strong, Bounded Staleness, Session, Consistent Prefix, and Eventual—allowing finer control over consistency vs. performance.

Key Differences

Cassandra API: Though Cosmos DB offers a Cassandra API, it’s not running Cassandra under the hood. So, while the API is familiar, the system behaves differently, especially in terms of performance.
Operational Complexity: Cassandra gives complete control over nodes and replication but requires hands-on management. Cosmos DB automates these tasks, offering hands-off operations while maintaining high availability, making it easier to manage at scale.

API for Apache Cassandra

If you have existing Apache Cassandra usage, in theory, the API for Apache Cassandra lets you use CosmosDB equivalently by just changing the query string. As with the Mongo API above, if possible you are probably better to use Core NoSQL API, but that will required rewriting dataaccess code.

Cosmos DB CQL Querying

using Cassandra;

// Connect to Cosmos DB's Cassandra API
var cluster = Cluster.Builder()
 .WithCredentials("<username>", "<password>") // From Azure Portal
 .WithPort(10350)
 .AddContactPoint("<account>.cassandra.cosmos.azure.com")
 .WithSSL()
 .Build();

var session = await cluster.ConnectAsync();

var cql = "SELECT * FROM customersdb.customers WHERE customer_id = ?";
var results = await session.ExecuteAsync(cql, "12345");

foreach (var row in results)
{
 Console.WriteLine($"Customer: {row["name"]}");
}

When to Use Cassandra Vs Cosmos DB

Cassandra: Use when you need control and can manage partitioning and scaling manually.
Cosmos DB: Use when you want hands-off scalability and a fully managed service with the option to use Cassandra-like APIs.

So use Cassandra when you need complete control over partitioning and scaling and are prepared to manage infrastructure manually. Opt for Cosmos DB when you want a hands-off managed service that handles the operational complexity for you, even if you’re using the familiar Cassandra API.

Dynamo DB Vs Cosmos DB

After comparing Cosmos DB to Cassandra, it’s natural to think of DynamoDB next. Like Cosmos DB, DynamoDB is fully managed and Dynamo-inspired, but being part of AWS, it operates differently from both Cassandra and Cosmos DB, especially in terms of operational simplicity and pricing.

Managed Operations

DynamoDB and Cosmos DB remove the ops overhead you’d get with Cassandra. If Cassandra gives you full control over nodes and partitioning, DynamoDB and Cosmos DB automate much of the complexity behind the scenes. With DynamoDB, you don’t have to manually manage shards or replication—AWS handles that for you, just like Cosmos DB on Azure.

Both databases also rely heavily on partition keys for data distribution. In DynamoDB, choosing the right partition key is critical to avoid hot spots, just like you need a well-thought-out partition key in Cosmos DB to balance the load across physical partitions. But where DynamoDB allows you to toggle between on-demand and provisioned capacity to manage scaling, Cosmos DB simply auto-scales as your traffic grows, offering a more seamless experience if you’re willing to pay for it.

Cosmos DB Consistency Levels

While Cassandra offers tunable consistency, DynamoDB simplifies this down to just two options: eventual consistency (default) or strong consistency. It’s enough for many use cases, but Cosmos DB follows Cassandra in covering many levels of consistency ranging from strong to eventual. One level, Session level consistency, seems both unique to Cosmos and valuable. It ensures you are always able to read your writes in the same session without impacting performance as much as higher consistency levels can. This makes Cosmos DB pretty unique if you to fine-tune consistency for globally distributed applications.

Cosmos DB vs Dynamo DB Costs

Again, using some back-of-the-napkin math, DynamoDB in provisioned Mode is more cost-effective than Cosmos DB for the same workload with 10 kb document sizes, but the numbers could swing the other way based on your use case. This is with estimating 500 reads / second ( 400 point reads or get-time, 100 more complex queries) and 50 writes.

Service	Workload (500 reads/sec, 50 writes/sec)	3 KB Document Size	10 KB Document Size	Key Difference
DynamoDB	500 reads/sec, 50 writes/sec	$137.67/month	$54.84/month
Cosmos DB	500 reads/sec, 50 writes/sec	$227.28/month	$212.92/month

Tangent: Azure Table Storage vs. Cosmos DB

Cosmos DB also has a Azure Table API and marketing material saying it’s a better choice than Azure Table Storage. My take is Azure Table and Cosmos DB’s Table API both provide key-value storage with distinct trade-offs. Azure Table Storage is cost-effective and ideal for simpler, predictable workloads, requiring minimal complexity. Cosmos DB’s Table API adds flexibility, optional global distribution, and tunable consistency, but also brings more complexity and cost.

While both offer similar functionality, Cosmos DB’s advanced features will cost you. If you know you aren’t going to need scale read and writes or need multi-region writes or other features, Azure Table Storage’s simpler, predictable pricing is the way to go.

Provision Azure databases as code

Whether you land on Cosmos DB, Azure SQL, or PostgreSQL, Pulumi lets you define and manage your entire Azure data layer in the language you already use.

Get started

When Cosmos DB Makes Sense

OK, I think all these comparisons give us a solid grounding to talk about trade-offs. So, with all this in mind, when should you use Cosmos DB?

Document Database with Global Distribution: Cosmos DB is a strong choice if you need a document database that supports global distribution with multiple consistency models. Its automatic scaling and geo-replication mean you can reach a global audience and a high level of throughput with minimal operational overhead.
Hands-Off Scalability: This came up again and again. When you want to focus more on development and less on database administration, Cosmos DB offers hands-off scalability. It automatically handles partitioning, indexing, and replication, which is great for teams that prefer highly managed services.
Schema Flexibility: When your data structure is not fixed and might evolve over time, Cosmos DB’s schema-less nature is an advantage. This flexibility is helpful in cases where your application data needs to adapt frequently without the constraints of a rigid schema. Also, there is a taste aspect to this. While I like schemas and structure, some teams have extensive experience working with document-based NoSQL solutions, and they will find CosmosDB a great fit.

When to Avoid Cosmos DB

Cost Sensitivity: Cosmos DB can be expensive, when you start working multi-region or start writing a lot of data per partition. If cost is a primary concern, especially for smaller projects, it might be better to consider alternatives.
Complex Joins and Transactions: If your application requires complex joins, foreign key relationships, or cross-partition transactions, Cosmos DB might not be the best fit. A relational database like PostgreSQL or SQL Server would serve these requirements more effectively.
Predictable Workloads with Lower Complexity: For applications that have predictable workloads and do not need the complexity of multi-region consistency or scaling to varying demand levels, a simpler solution like a traditional relational database might be more cost-effective and easier to manage.
Need for Strong Schema Enforcement: Long-term, there is a lot to be said for having a scheme and an easy way to evolve your data. If your application benefits from strict schema enforcement to ensure data integrity and reduce developer errors, a relational database like PostgreSQL or SQL Server will be a better choice than Cosmos DB. Cosmos DB’s schema-less nature can introduce challenges if your data model needs strict consistency and structure.
Avoiding Vendor Lock-In: If you want to maintain portability and avoid vendor lock-in, Cosmos DB might not be ideal for you. A highly managed database service, whether CosmosDB, Dynamo, or Spanner, will be hard to replace on another cloud vendor. A more portable database like PostgreSQL or Cassandra might be preferable for multi-cloud or cloud-agnostic strategies.

More Reasons for Cosmos DB ( Update )

Since this article came out, I’ve had a chance to talk to several Cosmos DB users inside and outside Microsoft. I spoke with Mark Brown on the Cosmos DB team, and he highlighted two additional reasons why teams choose Cosmos DB, that I had originally under emphasized:

High Availability: Cosmos DB is designed for mission-critical applications requiring always-on availability. It’s the only Azure database with five 9s availability and active-active multi-region writes. This high availability is why services like Azure Active Directory and Teams use it.
Guaranteed Low Latency: Cosmos DB offers SLAs on read and write latency (<10ms P99), a guarantee few other databases can match. This speed is achieved by co-locating compute and storage on the same cluster, minimizing data access times.

Managing Database Infrastructure with Pulumi

Whether you choose Cosmos DB, PostgreSQL, or Azure Table Storage Pulumi simplifies the process of provisioning and managing database infrastructure on Azure. With infrastructure as code in your favorite programming languages, you can define, deploy, and maintain your entire database infrastructure using familiar development practices. Pulumi’s Azure Native provider helps you implement the right database solution for your specific needs.

With Pulumi, you get:

Rich Azure database provisioning capabilities for Cosmos DB, PostgreSQL, and Azure SQL
Automated configuration management to handle connection strings and secrets securely
Infrastructure testing capabilities to validate your database configuration before deployment
Stack references for managing multiple database environments
Custom components for creating reusable database infrastructure patterns

Ready to streamline your database infrastructure management?

Try Pulumi for Free

Summary

If you need something that is document-based, scales globally, and handles different consistency levels well, Cosmos DB could be a great choice - especially if you’re already using Azure. Azure takes it seriously - it’s in every data center they have, and they treat it as mission-critical. But here’s the thing - while it’s easier to manage than the alternatives we looked at, you’ll pay more for that convenience.

Many times, that cost is not the variable being optimized for. Data is paramount, and Cosmos DB works well for many. James Thurley, Co-founder of Canopy Simulations, found it a perfect fit for their needs:

We’ve been using Cosmos DB as a document database since it was called DocumentDB, and it’s been fast, reliable, and straightforward for our needs. We store large, arbitrarily structured JSON documents, and Cosmos DB’s full indexing has handled that effortlessly. For us, the simplicity works—we don’t need to scale to millions of users, and with auto-scaling, each customer’s database runs smoothly without any partitioning.

For help provisioning Cosmos DB, check out our Azure Cosmos DB How to guide and if you decide to go with Azure Tables instead, our Azure Native provider is there to help you all the same.

And if you have a Cosmos DB experience to share ( or I got something wrong ): reach out. And, if you haven’t already, give Pulumi IaC a try:

Get Started with Pulumi

Use Pulumi's open-source SDK to create, deploy, and manage infrastructure on any cloud.