Pulumi Blog: Perspectives

Stop Prompting. Design the Loop.

Engin Diri — Tue, 09 Jun 2026 00:00:00 +0000

For about two years, the unit of work with a coding agent was the prompt. You wrote a good one, you gave it enough context, you read what came back, and you wrote the next one. The agent was a tool, and you were holding it the entire time, one turn after another.

That part is ending. Addy Osmani, a director of AI at Google Cloud, has a name for what replaces it, and I have not stopped thinking about it since: loop engineering. You stop being the person who prompts the agent. You design the loop that prompts it for you.

In my phrasing: you stop being the thing that runs, and start designing the thing that runs. The leverage moves up a layer. What I want to do here is take an honest look at the pieces, and at the part nobody automates.

The leverage moved up a layer

The people building these tools have already made the jump. Peter Steinberger has been posting it as a monthly reminder.

Peter Steinberger (@steipete) on X.

Boris Cherny, who heads Claude Code at Anthropic, says the same thing about his own job. He does not prompt Claude anymore. He has loops running that prompt Claude and decide what to do next, scanning the issue tracker, the team chat, and the timeline for what to build. “My job is to write loops.”

A loop is a goal that prompts itself. You set the purpose, and the system keeps iterating until it’s met. In practice it finds the work, hands it out, checks the result, writes down what got finished, and decides the next thing, then it pokes the agent instead of you. You build that small system once and let it run.

Look closer, and a loop is really two loops nested. The inner one does the work against a spec. The outer one decides what the work should be: it watches an issue tracker, an error feed, a changelog, then writes the next spec and hands it down. Most people are still running that outer loop by hand, in their head, and calling it a backlog.

The part that surprised me is that this is barely a tooling problem anymore. A year ago a loop meant a pile of bash you wrote and maintained forever. Now the pieces ship inside the products, and the same shapes show up in Claude Code and in Codex. Osmani puts loop engineering one floor above the harness, the context and tooling you wire around a single agent. I wrote about that harness a couple of weeks ago. The loop is the thing that runs on top of it: it runs on a timer, it spawns helpers, and it feeds itself.

The five pieces, and the one that holds them together

Strip loop engineering down and you get roughly five building blocks, plus one place to remember things. Both Claude Code and Codex have all five now. The names differ here and there; the capability is the same.

Automations are the heartbeat. They are what make a loop an actual loop and not one run you did once. A prompt or command on a cadence, a scheduled task, a hook that fires at a point in the agent’s lifecycle, or a job on CI that keeps running after you close the laptop. Discovery and triage run themselves, and the findings that matter come to you.
Worktrees keep parallel from turning into chaos. The second you run more than one agent, the files start colliding. Two agents writing the same file is the same headache as two engineers committing to the same lines with nobody talking first. A git worktree is a separate working directory on its own branch, so one agent’s edits cannot touch another’s checkout.
Skills are intent, written down. An agent starts every session cold and fills any hole in your intent with a confident guess. A skill is that intent written on the outside: the conventions, the build steps, the “we don’t do it like this because of that one incident,” recorded once where the agent reads it every run. Without skills, the loop re-derives your whole project from zero every cycle.
Connectors let the loop touch your real tools. Built on MCP, they let the agent read the issue tracker, query a database, hit a staging API, or drop a message in chat. This is the difference between an agent that says “here is the fix” and a loop that opens the pull request, links the ticket, and pings the channel once CI goes green.
Sub-agents keep the maker away from the checker. The model that wrote the code is far too generous grading its own homework. A second agent with different instructions, and sometimes a different model, catches the things the first one talked itself into. Worktrees and a cold-context reviewer are two pieces I have written about before, back when the question was running agents in parallel without them trampling each other.

Then the sixth thing: memory. A markdown file, a Linear board, a state file, anything that lives outside the single conversation and holds what is done and what is next. It sounds too dumb to matter, and it’s the whole game. The model forgets everything between runs, so the memory has to live on disk, not in the context window. The agent forgets. The repo does not.

What makes a loop hold together

A loop running unattended is also a loop making mistakes unattended. The one thing that keeps it honest is verification, and verification needs an oracle, something outside the model that returns a hard yes or no. Passing tests, a clean build, a green pipeline, a real production signal. Without an oracle, the loop compounds confidently wrong work, faster than you can read it.

The cleanest version of this already ships in the tools. Claude Code’s /goal keeps working across turns until a condition you actually wrote holds, something like “every test in auth/ passes and lint is clean,” and after every turn a separate, faster model reads the transcript and decides whether you are there yet. The agent that wrote the code is not the one that grades it. That is the maker-and-checker split applied to the stop condition itself. Codex’s /goal reaches the same finish line a different way: the agent audits its own work against the evidence before it can call the goal done.

What the loop still won’t do for you

The loop changes the shape of the work. It does not take it off your desk. And a few things get sharper as the loop gets better, not softer.

Verification is still on you. The split reviewer is what makes “it’s done” mean something, but “done” is a claim, not a proof. Your job is still to ship code you confirmed works, which is harder to remember when the diff arrived while you were at lunch.

The bill comes in two currencies. Tokens and attention. A single unattended run can burn through millions of tokens, and that is only worth it when the tokens buy something worth more than they cost. The quieter trap is the second currency: memory is what lets a loop compound over time, and slop compounds right alongside it. A loop pointed at a vague goal does not get tired and stop. It gets faster.

Your understanding rots if you let it. The faster the loop ships code you did not write, the wider the gap between what exists in the repo and what you actually understand. A smooth loop grows that gap faster, not slower, unless you read what it made. The comfortable posture, where you stop having an opinion and take whatever the loop gives back, is the risky one. Two engineers can build the exact same loop and get opposite results, one moving faster on work they understand deeply, the other avoiding the work entirely. The loop cannot tell which one you are.

When the loop reaches production

Most of this thinking grew up around application code, where a bad run costs you a revert. When the loop reaches into infrastructure, the blast radius is a production outage rather than a revert, and the verification bar has to rise to meet it. The upside is that infrastructure hands the loop a better oracle than application code does. A plan diff is deterministic and machine-readable, a policy check returns a hard verdict, and drift and cost are numbers you can put a threshold on. A reviewer, whether human or agent, can read the change cold, with no memory of the prompt that produced it. That cold-context check is exactly what an unattended loop needs, and it’s the reason an infrastructure loop can be built to hold together while you sleep. Pulumi Neo reasons over the state graph directly, so the checker grounds every claim in what the change actually does, not in what the writer says it does.

Where to start

Pick the loop you can actually trust first. In order:

Start where “done” is unambiguous. CI triage, dependency bumps, a flaky-test hunt, a failing job you keep re-running by hand. Loops need an oracle, so begin where the oracle already exists.
Write the memory file before the loop. One markdown file, or a board. What is done, what is next, what was tried and failed. This is the spine, and everything else hangs off it.
Split the checker from the maker. Use /goal with a verifiable condition, or a second agent with its own instructions. Never let the agent that did the work be the one that decides the work is finished.
Cap it, then read everything. A max-iteration count, a token budget, a teardown step. Run it once, end to end, then read every line it shipped. The first run is the measurement, not the payoff.

Then look at what you built. You designed it once, and it ran without you steering each step. That is the real shift. But the leverage only holds if you wire the loop like an engineer, not like someone looking for permission to stop thinking. Read what it ships. Keep an opinion. Judgment is the one part that does not move up a layer.

The loop will do the typing. The thinking is the work.

See how Pulumi Neo closes the loop on your infrastructure

Stop Tuning Prompts. Build a Harness.

Engin Diri — Tue, 26 May 2026 00:00:00 +0000

Anthropic shipped a piece earlier this month called How Claude Code Works in Large Codebases. I have not read anything more useful about coding agents this year. The core claim, in their words: “the ecosystem built around the model—the harness—determines how Claude Code performs more than the model alone.” In my phrasing: in a real codebase, the model is the smaller variable. The layer of context and tooling you wire around the agent matters more than which version of Sonnet or Opus is behind it.

The post stays high-level, which is the right move for a launch piece. What I want to do here is land it. Same seven pieces, but with the wiring you would actually put in a repo, in the order I would put it.

How Claude Code navigates without an index

Anthropic’s writeup says Claude Code works from the live codebase and does not require a codebase index to be built, maintained, or uploaded. The agent navigates the way an engineer would, with grep, find, ls, file reads, and reference-following. Anthropic calls this agentic search, and the upside is obvious: no separate index exists for you to keep fresh.

The downside is also obvious. An engineer who has never seen your repo and only has shell tools will flounder if you drop them in the root with no map. That is your agent on day one. Everything that follows is about giving it the map.

The AI layer in seven pieces

Every codebase used to have two artifacts engineers cared about: the code and the tests. A third exists now. Call it the AI layer, or the harness, or whatever you want. This layer is the set of context and tools you give your coding agent to operate in this specific repo. Anthropic breaks it into seven pieces, and each one solves a different scaling problem.

Anthropic gives each piece a role: CLAUDE.md is the foundation, hooks do self-improvement, skills are progressive disclosure, plugins handle distribution, LSP gives navigation, MCP is extension, subagents split exploration from editing. They are not equal in usage either. CLAUDE.md is read at the start of each session and stays in context for the duration. The others fire when relevant.

Lean and layered CLAUDE.md

The single biggest mistake I see is a root CLAUDE.md that has grown into a small book. Two thousand lines of conventions for parts of the repo the current task will never touch. Every session pays the tax. Anthropic’s own guidance is to keep these files focused on what applies broadly so they do not become a drag on performance, and you can feel that drag in practice: the agent gets cautious, slow, and oddly literal.

Keep the root file lean. What is this repo, broadly. The tech stack. The commands the agent will need (make test, make lint, how to run the dev server). General conventions that apply everywhere. That is most of what belongs there.

Local conventions go in subdirectory CLAUDE.md files. When the agent starts in a subdirectory, Claude Code walks upward from the working directory and loads every CLAUDE.md it finds on the way to the repo root, so root context is never lost and intermediate layers stack in the order you would expect. Claude Code can also discover files below the current working directory when it reads files in those subdirectories. That means services/api/CLAUDE.md only joins the session when the work reaches that service. Same for services/billing/, the frontend, the data layer.

If you already know the task is scoped to one service, start the agent in that subdirectory. The working directory becomes the focus, and the agent stays out of unrelated code unless you tell it otherwise. Most of the time, you know.

Two more cheap wins live in the same neighborhood. Scope the make test and make lint commands so the subdirectory version runs only the slice the agent is working in, instead of the whole repo on every change. And version-control your exclusion rules in .claude/settings.json so the agent never reads dist/, generated SDKs, or vendored code. Every file the agent skips is tokens you keep for the work that matters. If your directory layout is unconventional or has historical baggage, Anthropic also suggests adding a short codebase map to the root CLAUDE.md so the agent has somewhere to anchor.

Hooks that make the harness self-improving

Most teams use hooks as guardrails. Block edits in vendor/, refuse to delete migrations, kill the run if a secret turns up in a diff. That is fine and you should do it. But hooks have a second life that almost no one uses, and that second life is the more interesting one.

Both kinds register the same way, in .claude/settings.json, against named events Claude Code fires during a session:

{
 "hooks": {
 "SessionStart": [
 {
 "hooks": [
 {
 "type": "command",
 "command": "uv run --directory \"$CLAUDE_PROJECT_DIR\" python .claude/hooks/session_start_context.py"
 }
 ]
 }
 ],
 "Stop": [
 {
 "hooks": [
 {
 "type": "command",
 "command": "uv run --directory \"$CLAUDE_PROJECT_DIR\" python .claude/hooks/propose_claude_md.py"
 }
 ]
 }
 ]
 }
}

A SessionStart hook fires before the agent has done anything. Whatever the script prints to stdout is injected straight into the session as context, so you can preload the things the agent would otherwise have to spend a turn discovering: the current branch, the uncommitted diff, the last few commits. For a larger team you might fetch the Confluence or Notion page that owns the directory the engineer is working in. Every developer starts each session pre-oriented, with no manual setup.

"""SessionStart hook — prints orientation Claude reads as session context."""
import os, subprocess
from pathlib import Path

root = Path(os.environ.get("CLAUDE_PROJECT_DIR", "."))

def git(*args):
 out = subprocess.run(["git", *args], cwd=root, capture_output=True, text=True)
 return out.stdout.strip()

print("# Orientation\n")
print(f"## Branch\n{git('rev-parse', '--abbrev-ref', 'HEAD')}\n")
print(f"## Uncommitted changes\n{git('status', '--porcelain') or '(clean)'}\n")
print(f"## Recent commits\n{git('log', '-5', '--oneline')}\n")

The Stop hook is the more interesting one. It fires when the agent finishes its turn. At that moment the session context is still fresh, the diff is still small, and you have a free shot at a question nobody asks: did anything I changed invalidate the rules I wrote down? Spawn a separate headless Claude session, hand it the diff and the relevant CLAUDE.md files, ask it to propose updates, and write the result to a markdown review file. You read it when you are ready. The CLAUDE.md files stop going stale on their own.

The trick is to make the hook itself cheap and dispatch the LLM call in the background, so the end of every turn does not block on a reflection:

"""Stop hook — dispatch a headless Claude reflection in the background."""
import os, subprocess, sys
from pathlib import Path

# The reflector spawns its own headless Claude, whose Stop hook lands back
# here. The lock prevents infinite recursion.
if os.environ.get("REFLECT_LOCK"):
 sys.exit(0)

root = Path(os.environ.get("CLAUDE_PROJECT_DIR", "."))
diff = subprocess.run(
 ["git", "diff", "HEAD"], cwd=root, capture_output=True, text=True
).stdout
if not diff.strip():
 sys.exit(0)

env = {**os.environ, "REFLECT_LOCK": "1"}
subprocess.Popen(
 ["uv", "run", "python", ".claude/hooks/reflect_claude_md.py"],
 cwd=root, env=env,
 stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
)

reflect_claude_md.py is the part that calls a headless claude against the diff and writes .claude/claude-md-review.md. You can grow it from twenty lines to two hundred without ever blocking the agent.

The pattern that ties the two together: hooks let the harness improve itself in the background while you do the actual work.

Path-scoped skills

Skills are where the agent learns how to do a thing. CLAUDE.md is conventions (“every route is registered here”). Skills are workflows (“here is how you add a new route in this repo, end to end”). The two overlap, but the framing keeps me honest: rules in CLAUDE.md, recipes in skills.

The piece of the skills system most teams miss is the path scope. A skill can declare which directories it activates in. A create-api-endpoint skill that only loads when the agent is editing under services/api/ is invisible the rest of the time. With dozens of skills in a real repo, scoping is the difference between a useful library and a wall of irrelevant prompts.

The mental model: progressive disclosure for expertise. Most knowledge in a large codebase is local. Load it locally.

Symbol-level search through LSP and MCP

grep is fine until it isn’t. Past six-digit line counts, plain string search gets slow, returns too much, and burns tokens reading files the agent did not need to open. You also lose what every IDE has done for decades: jump-to-definition, find-references, hover-for-types.

You can give the agent the same navigation. Run a language server locally, wrap it in a small MCP server, expose two or three tools: where_is, find_references, goto_definition. The agent now searches by symbol, not by string. A request like “find every place monthly_total_cents is referenced” returns one definition and the actual references, instead of fifty grep hits that mention the substring in unrelated comments.

This is also where bigger orgs invest. Custom MCP servers that expose internal search systems, the code-ownership graph, the design-doc index. The patterns are the same; the targets are domain-specific. The point is that the agent does not have to brute-force its way through your repo when you already have better tools for finding things.

Image: Anthropic, How Claude Code Works in Large Codebases.

Subagents for exploration

The rule I follow: split exploration from editing. A subagent runs in its own context window. You ask which files implement the billing webhook flow, or what the user model looks like across services. It does the digging, and only the summary comes back to your primary session.

The win is context budget, not parallelism. Exploration is wasteful by nature. The agent reads forty files to find the three that matter, and most of those forty get thrown away. If that happens in your primary session, your editing turns start with a context window already half full of noise. If it happens in a subagent, the noise stays there. You get the answer.

Use the built-in Explore subagent liberally. Custom subagents earn their place when you have a workflow specific enough that a generic explorer is the wrong tool. The file shape is small: a single markdown file under .claude/agents/, a short frontmatter block, and a prompt body. name, description, tools, and model are enough to start:

---
name: explorer
description: Read-only repo explorer. Map a service or package without burning the main session's context, then return findings.
tools: Read, Grep, Glob
model: sonnet
---

You are a read-only explorer. The parent agent will hand you one service or
package to map. Read its `CLAUDE.md` if there is one, then trace entry points,
the public surface, and dependencies. Return findings as your final response.
No edits.

Restricting tools to read-only is the load-bearing line. The model only sees the tools you expose, so an explorer subagent without Write or Edit has nothing to call when it gets tempted, even if the prompt body forgot to say so. Treat that as a strong default. If you need a hard guarantee, layer a PreToolUse hook on top.

Don’t let it rot

The harness is not a one-time setup. Models improve, and rules written for last year’s model often constrain this year’s. A note like “always split refactors into single-file changes” might have saved you in 2024 and might block a beneficial cross-file edit in 2026. Anthropic suggests reviewing your CLAUDE.md files every three to six months, or whenever performance feels like it has plateaued after a major model release. The stop-hook reflection gives you a head start. The rest is on you.

Assign an owner

The last piece is not technical. The teams that get value out of Claude Code at scale have someone who owns the harness. A small platform-engineering team, or one DRI, or a hybrid PM/engineer doing it half-time. Their job is the same shape as owning a CI pipeline: write the conventions, build the skills, run the LSP wrapper, version the hooks, evangelize what works, retire what does not.

Plugins are the distribution vehicle. A good harness that lives in one engineer’s dotfiles stays tribal. The same harness packaged as a plugin (or a private marketplace) is how a team of five hundred ends up running the same skills, the same MCP servers, and the same hooks without anyone having to remember to copy a config.

The pattern that fails: ship Claude Code to the org on a Friday, hope adoption goes viral, watch every team grow its own slightly different version of CLAUDE.md for six months. The pattern that works: a quiet build-out period, a small set of approved skills, a working plugin or two, a documented governance story, then broad access.

Treat the harness like infrastructure.

Where to start

The order that has worked for me, in any repo:

Trim the root CLAUDE.md until it fits on one screen. Move the rest into subdirectories.
Add a Stop hook that proposes updates to those CLAUDE.md files in headless mode.
Convert your three most common repeated tasks into path-scoped skills.
Run a language server behind an MCP server. Stop searching by string.
Get comfortable dispatching exploration to subagents.

Most teams will plateau on step one for a week and find the agent is already noticeably sharper. The rest compounds. I have written more on the agent-tooling shift this is part of in How Building AI Agents Has Changed in 2026, and on the workflow side in The Claude Skills I Actually Use for DevOps and Superpowers, GSD, and GSTACK.

The model will keep getting better. The harness is the work.

How Building AI Agents Has Changed in 2026

Engin Diri — Thu, 14 May 2026 00:00:00 +0000

Twelve months ago, building an AI agent meant picking a framework, defining your tools, standing up a RAG pipeline, and writing a stack of glue code to wire it all together. That was the default playbook. The post-mortem on six months of work usually went the same way: half the time went into infrastructure that had nothing to do with the agent’s actual job.

That isn’t where the work is anymore. Most of the middle layer is gone. The SDKs ship with the tools, the skills system replaced the upfront tool registry, and longer context windows pushed vector search out of the default slot it held all of last year.

The shape is the same as a lot of infrastructure shifts before it. The hard thing got cheap, the cheap thing got expected, and the question moved up a level.

The old playbook

A 2024 to 2025 agent project looked like this. You picked a framework, usually LangChain, LlamaIndex, or an early version of Pydantic AI. You wrote tool definitions, usually a wrapper around an API the agent would call. You stood up a RAG pipeline: chunk your documents, embed them, pick a vector database, write retrievers, layer reranking on top. Then you wrote the agent loop yourself, including prompt assembly, tool dispatch, retry logic, and observability.

This was the default for good reasons. Foundation models had short context windows. They didn’t ship with file access. They couldn’t run code. If you wanted an agent to do anything useful with your data, you had to bring the data to the model in pre-digested chunks.

The cost wasn’t only setup time. It was infra bills, retries against embedding APIs, and a context strategy that fought the model as the model got better. By mid-2025 the retrieval layer was often the bottleneck on quality. The agent would ask a question, get five plausible-looking chunks, and answer from those instead of the document you actually wanted it to read. Chunking decisions made on a Tuesday in March were still hurting answer quality six months later.

Most teams I talked to in 2025 were tuning their RAG pipeline. Almost nobody enjoyed it.

The shift: three things changed at once

Three changes landed close enough together that they collapsed the middle layer.

Built-in tools. The Claude Agent SDK ships with Read, Write, Edit, Bash, Grep, Glob, WebSearch, and WebFetch out of the box. OpenAI’s Codex SDK is similar in shape, with shell and file tools available to the agent by default. These are the tools every agent project was rebuilding in 2024, often as a side quest to the work the agent was actually meant to do. A Read that handles binary files. A Bash that streams output and respects working directory. A Grep that doesn’t choke on large files. The 80% of agent tooling everyone was paying their team to reimplement is now table stakes.

The consequence is that you can give an agent the ability to do real work with about ten lines of configuration. The flip side is that the differentiator moved up a layer. The value isn’t in having Read. It’s in what the agent does with it.

Anything outside the built-in toolbox plugs in through MCP servers. The registry has grown nearly 8x since early 2025, and every major model vendor now ships first-party support. The picture in 2026 is more layered than that, though. A lot of what used to call for an MCP server is now better served by the agent invoking a CLI through Bash and wrapping the recipe in a skill. Benchmarks put CLI-based tool calls at a fraction of the context cost of equivalent MCP calls, with fewer round-trips and fewer failure modes. MCP still earns its place for protocol-heavy work like browser control, OAuth flows, and streaming services, but it stopped being the automatic answer to “how do I give my agent a new capability.”

Skills replaced tool stuffing. The old way was to register every tool the agent might need at startup, eating context every turn whether the agent used the tool or not. A hundred tools meant a heavy system prompt before the agent had thought about anything. The skills pattern flips that. A skill is a small markdown package with a name and a one-line description. The agent sees the description (around 100 tokens) and only loads the body when it decides the skill is relevant. A hundred skills no longer means a hundred tools’ worth of context tax. Anthropic frames this as progressive disclosure: because the body only loads on demand, the amount of content you can bundle into a single skill is effectively unbounded.

Progressive disclosure isn’t a new idea. What’s new is that the agent harness now treats it as the default loading strategy instead of something you have to engineer.

RAG got demoted. This is the change with the biggest blast radius and the smallest amount of commentary. A year ago, “we need to add RAG” was the reflex answer when somebody asked how an agent would handle a corpus. Today that question splits three ways. If the corpus fits in the context window, put it in. If the agent can grep the filesystem, let it grep. If the corpus is genuinely too large for either, vector search is still right, but you’ll find that’s a smaller set of cases than it used to be. You can see this in the coding agents that already ship today. Cursor, Claude Code, and Devin lean on grep, find, and direct file reads more than vector search. LlamaIndex’s own writing on agentic retrieval is one of the clearer reads on where this is going.

Vector search didn’t get worse. The context around it improved enough that it stopped being the right first move.

Taken together, what got pulled into the SDK is the middle of an agent project: the tools layer, the retrieval layer, and the loop. What’s left for the team is the system prompt, the skills, and the policies around what the agent is allowed to do.

When you still need a framework

The first reaction to a lot of this is to declare that frameworks are over. They aren’t, but the cases where you reach for one have narrowed.

Pydantic AI is still the right choice when you want strong typing, deterministic output schemas, and an evaluation loop that matches how the rest of your Python codebase already thinks. LangGraph is still the right choice when your problem is genuinely a graph of agent states with branching and human approval steps. OpenAI’s Agents SDK is built around explicit handoffs between agents and earns its place when that pattern fits how you want to decompose the work. CrewAI is the fastest path I’ve seen for prototyping a multi-agent system, as long as you can live with its opinions. Any team running production traffic across multiple model providers is going to want a routing layer that the official SDK from any single vendor isn’t going to give them. Anthropic’s own writing on building effective agents lands in the same place: start with the simplest thing, add complexity only when the problem demands it.

The mental model that works for me: start with the SDK, reach for a framework when you outgrow it. “Outgrow” usually means one of four things:

Multi-provider routing. You’re running production traffic across more than one model vendor and need a routing layer the official SDKs don’t ship.
Multi-agent orchestration. Your problem genuinely decomposes into separate agents with handoffs, branching, or human approval steps.
Deterministic typing. You need strong schemas and validation around inputs and outputs, and the rest of your codebase already thinks that way.
Production observability. You need eval loops, replay, or tracing beyond what the SDK provides out of the box.

If none of those four are biting, the SDK is probably enough, and adding a framework on top is a layer you’ll regret in six months.

Where this lands for infrastructure work

Two things from the new agent shape map cleanly onto infrastructure work. The first is that “built-in tools plus governed actions” is the model an IaC platform was already running. The SDK assumes the agent has tools that do real work. The platform assumes those tools have policies, audit logs, and short-lived credentials around them. Those assumptions stack.

The second is that a state graph is already structured context. You don’t need to chunk it. You don’t need to embed it. An agent reasoning over a Pulumi stack can grep its way through the program graph the same way it greps a codebase, and the answers are grounded in the same source of truth the rest of your platform uses. I wrote the deeper version in Grounded AI: Why Neo Knows Your Infrastructure. The dark-factory and sprawl posts (The Dark Factory Pattern for Infrastructure and Agent Sprawl Is Here. Your IaC Platform Is the Answer.) are the places to go if you want to push on this further.

Start with the SDK

A year ago, an agent project was 80% glue code and 20% the thing the agent actually did. On most projects today that ratio is flipped. If you’ve been sitting on an agent idea, build it the SDK way first and reach for a framework only when you hit something the SDK genuinely can’t do. Most teams will be surprised how often they don’t.

There’s one agent you don’t have to build at all. Pulumi Neo is the same SDK-first shape applied to the IaC slice: tools that reason directly over your state graph, governed by the controls the rest of your platform already runs on. Save your own SDK time for the agents only you can build.

See how Pulumi Neo works

Agent Sprawl Is Here. Your IaC Platform Is the Answer.

Engin Diri — Wed, 22 Apr 2026 00:00:00 +0000

Somewhere in your company right now, a developer is building an AI agent. Maybe it’s a release agent that cuts tags when tests pass. Maybe it’s a cost agent that shuts down idle EC2 overnight. It’s running, it’s in production, and there’s a decent chance the platform team doesn’t know it exists.

This isn’t a thought experiment. OutSystems just surveyed 1,900 IT leaders and the numbers are rough: 96% of enterprises run AI agents in production today, 94% say the sprawl is becoming a real security problem, and only 12% have any central way to manage it. Twelve percent. You can read the full report here.

The real question is where those agents run. Inside the platform you’ve already built, or somewhere off to the side where nobody on the platform team can see them.

The new platform tension

Platform teams have always had two jobs that pull in opposite directions. Let developers ship without waiting on a ticket. Keep the infrastructure coherent while they do. Golden paths, review stacks, a catalog of components that don’t fight each other.

Agents break the second half of that deal.

A developer with a sharp prompt can spin up an SRE agent that watches a queue, a release agent that cuts tags when the test suite goes green, or a cost agent that kills idle infra at 2 a.m. That’s useful. It’s also running on your production cloud account, using credentials you never provisioned, writing to systems you never approved, and the only audit trail is whatever the developer remembered to log. The Salesforce 2026 Connectivity Benchmark pegs the average enterprise at twelve agents today, projected to grow 67% over the next two years. Most teams aren’t ready for one, let alone twenty.

This is the same shape as every sprawl problem before it. I wrote about the last one in How Secrets Sprawl Is Slowing You Down, and the pattern keeps repeating. When something useful gets cheap, it proliferates. When it proliferates without structure, it becomes a liability.

The clock is also ticking on the compliance side. The EU AI Act’s high-risk obligations kick in on 2 August 2026. Colorado’s AI Act goes live on 30 June 2026 after last year’s delay. A folder of unreviewed agent scripts isn’t going to hold up against either of those.

Three ways to respond (only one of them works)

There are roughly three paths from here.

Do nothing. Accept the sprawl and hope nothing catches fire. This is the default, and it’s also how you end up explaining to an auditor why some finance agent moved data between three systems last Thursday and nobody remembers which prompt triggered it.

Mandate centralization. Tell developers every agent has to be registered and approved before it runs. This sounds responsible on a slide, and it falls apart inside a sprint. Developers route around friction. If the official path takes a week and the unofficial path takes an afternoon, the unofficial path wins, and you’ve just pushed the sprawl underground where you can’t see it anymore.

Make the platform the obvious path. Build the thing developers actually want to use. A place where an agent inherits the guardrails, credentials, policies, and audit trail by default, because that’s what’s on offer. Adoption becomes a side effect of shipping something good.

Option three is the only one that scales. It’s also the one where most platform teams look at their existing stack and assume they need to build a pile of new scaffolding. I don’t think they do, and the rest of this post is why.

The seven things an AI agent needs from your platform

An agent needs seven concrete things from the platform it runs on. Each one maps to a Pulumi primitive you already own.

1. A trustworthy context lake

Agents are only as good as the context they can reason over. Drop a generic LLM into your cloud account and you’ll get plausible-sounding nonsense, because the model has never seen your environment. What you actually need is a grounded source of truth: what resources exist, how they relate, which stack owns what, which version is running where.

Pulumi state is already that. Your program graph, your stack outputs, your resource metadata, all of it adds up to a structured record of what you’ve actually deployed. Pulumi Neo reasons directly over that graph, which is why it can tell you why a deployment drifted instead of guessing. I wrote the long version of that argument there. Short version: you already have the context lake. Point agents at it.

2. Pre-cleared integrations

An agent that needs to touch five systems shouldn’t need five separate credential dances. That’s where credential sprawl starts. Every agent gets a long-lived key, every key ends up in somebody’s .env, and every rotation turns into an incident.

The Pulumi surface here is the 200+ providers plus Pulumi ESC handling dynamic credentials through OIDC. An agent doesn’t ask for an AWS access key. It asks ESC for a short-lived, scoped token bound to the environment it’s allowed to operate in, and the token expires when the task ends. No static keys, no rotation pain, no awkward postmortem about how something got committed to GitHub. The ESC patterns I walked through in the Claude skills post work just as well for an autonomous agent as they do for a human developer, which is really the whole point.

3. Governed actions

There’s a real difference between “an agent can see your infrastructure” and “an agent can change your infrastructure.” The second one is where you actually need structure. Pulumi Deployments gives you that structure: defined workflows, controlled triggers, running inside your Pulumi Cloud boundary instead of whatever environment the developer happened to spin up. The Automation API lets you build higher-order orchestration on the same primitives your developers already use.

The framing I keep coming back to goes like this. An agent shouldn’t call pulumi up directly. It should submit an action to a governed pipeline that runs pulumi up on its behalf, inside an environment you control, with a log trail and the guardrails already in place. Same effect, very different threat model.

4. Deterministic policy

Real governance lives outside the prompt. “Please don’t delete production” is a wish written into a system prompt, not an enforced control. And when an agent overrides your intent to do what it thought you meant, it’s behaving exactly the way the technology was designed to behave.

Pulumi Policies is the answer the IaC community landed on years ago: policy as code, written in a real programming language, evaluated deterministically at preview and update time. Disallow production RDS deletions. Require encryption at rest. Block S3 buckets with public ACLs. An agent running through Pulumi hits those gates whether it “wants” to or not, because the gates live in the pipeline and not in the prompt. This is the pillar most teams underweight, and it’s the first one most auditors ask about.

5. An audit trail

When something goes wrong at 3 a.m. (and with enough agents running, something will), you need answers fast. What changed, who changed it, and why. Not just “which agent,” but which version of which agent, triggered by what event, authorized by which policy, touching which resources.

Pulumi Cloud’s activity log, the stack update history, and ESC audit logs already capture all of that. Every update is versioned. Every secret access is logged. Every policy evaluation is recorded. When an agent submits a change through your Pulumi pipeline, it inherits that audit surface for free. The alternative is reconstructing an incident from a mix of Slack messages, container logs, and developer memory, which is roughly the state most teams without a platform are in today.

6. A review process

Not every agent action should wait for a human. But agents do need a promotion path, the same way new platform components do. Experimental, then reviewed, then trusted, then autonomous. That’s exactly what pulumi preview, review stacks, and Deployments PR workflows already model for human contributors. An agent that wants to make a change should have to submit it the same way a junior engineer would. As a diff, with a plan, against a preview environment, until it earns the trust to skip steps.

This connects back to the pattern I laid out in Golden Paths: Infrastructure Components and Templates. Golden paths were never only for humans. They’re just paths, and agents can walk them too.

7. Human-in-the-loop approval

The last pillar is the one that keeps the other six honest. Some decisions shouldn’t be automated, full stop. Production rollbacks outside business hours. Destructive changes above a certain blast-radius threshold. Anything that touches a regulated data boundary. For those cases, you want a forced human checkpoint that the agent can’t route around.

Pulumi Deployments approvals already play that role for human changes. Pulumi Neo’s review steps add the AI-aware version: a structured plan, a diff, a named approver, and a record of what they decided and why. I walked through what this looks like in practice in Self-Verifying AI Agents. Short version: an agent that proposes is much safer than an agent that commits.

Why IaC is the natural substrate for this

Step back from the seven pillars and look at what they have in common. Context, integrations, governed actions, deterministic policy, audit, review, approval. None of those are new problems that AI agents invented. They’re the problems infrastructure-as-code has been quietly solving for a decade, for human developers.

Every meaningful agent action ends up being a change, whether that’s to infrastructure, configuration, secrets, or state. IaC is the one layer in your stack that already treats change as the unit of work. Plan, preview, apply, record. If you want governance for agents and you don’t want to build it twice, the most efficient move is to route agent changes through the same substrate your humans already use.

I made the same point from a different angle in Token Efficiency vs Cognitive Efficiency: Choosing IaC for AI Agents. An IaC platform that models your world as a graph of typed resources is a much better reasoning surface for an agent than a stack of YAML or a bash script somebody wrote on a Friday. The structure is what makes it work.

What this means for the platform engineer

There’s a narrative floating around that AI is going to make platform engineers less relevant. I haven’t seen it hold up against an actual production environment. Every stat I’ve looked at points the other way. Gartner expects 70% of enterprises to deploy agentic AI as part of IT infrastructure and operations by 2029, up from less than 5% in 2025. LangChain’s State of Agent Engineering report already has 57% of teams running agents in production today. And Gartner projects that 80% of large software engineering orgs will have a platform team by end of 2026, up from 45% in 2022. More agents means more changes, more changes means more blast radius, and more blast radius means more need for the thing platform teams are uniquely equipped to provide.

Your classic responsibilities haven’t gone anywhere either. Golden paths, service catalogs, CI/CD, on-call rotations, all of that is still yours. Agents are an additional layer that needs the same discipline. The upside is that if your platform already runs on a mature IaC surface, you’re extending a muscle you’ve been building for years instead of growing a new one.

The developer-facing side matters too. A developer building an agent needs to know what’s available to them, needs templates that work on the first try, and needs to see what teammates have already built so they don’t start from a blank page. That’s the territory the Claude skills post and IDP Strategy: Self-Service Infrastructure That Balances Autonomy With Control cover. That’s the experience layer that makes developers actually choose your platform instead of routing around it. You need both sides working at once. The governance your security team cares about, and the experience your developers will actually reach for.

Close the window

The agents your developers are shipping this week are going to outlive the experiment that started them. Some of them will become critical. At least one will cause an incident. At least one will eventually show up in an audit. All of them are going to be easier to govern if they were built on your platform from day one than if you try to wrap policy around them later.

If you want the longer view on where this is going, AI Predictions for 2026: A DevOps Engineer’s Guide is the companion piece. If you want the developer-facing version of the grounding argument, Grounded AI is what to read next.

Either way, here’s where I land. The substrate for agent governance is already running in your stack. You’ve been pointing it at human changes for years. Now point it at the agents too.

See how Pulumi Neo governs agent actions

Superpowers, GSD, and GSTACK: Picking the Right Framework for Your Coding Agent

Engin Diri — Mon, 13 Apr 2026 00:00:00 +0000

Three community frameworks have emerged that fix the specific ways AI coding agents break down on real projects. Superpowers enforces test-driven development. GSD prevents context rot. GSTACK adds role-based governance. All three started with Claude Code but now work across Cursor, Codex, Windsurf, Gemini CLI, and more.

Pulumi uses general-purpose programming languages to define infrastructure. TypeScript, Python, Go, C#, Java. Every framework that makes AI agents write better TypeScript also makes your pulumi up better. After spending a few weeks with each one, I have opinions about when to use which.

The problem all three frameworks solve

AI coding agents are impressive for the first 30 minutes. Then things go sideways. The patterns are predictable enough that three separate teams independently built frameworks to fix them.

Context rot. Every LLM has a context window. As that window fills up, earlier instructions fade. You start a session asking for an S3 bucket with AES-256 encryption, proper ACLs, and access logging. Two hours and 200K tokens later, the agent creates a new bucket with none of those requirements. The context window got crowded and your original instructions lost weight.

No test discipline. Agents write code that looks plausible. Plausible code compiles. Plausible code even runs, for a while. But plausible code without tests is a liability. The agent adds a feature and quietly breaks two others because nothing verified the existing behavior was preserved.

Scope drift. You ask for a VPC with three subnets. The agent decides you also need a NAT gateway, a transit gateway, a VPN endpoint, and a custom DNS resolver. Helpful in theory. In practice, you now have infrastructure you never requested and barely understand. You will also pay for it monthly.

These problems are not specific to Claude Code or any particular agent. They happen with Cursor, Codex, Windsurf, and every other LLM-powered coding tool. The context window does not care which brand name is on the wrapper.

Superpowers: the test-driven discipline enforcer

Superpowers was created by Jesse Vincent and has accumulated over 149K GitHub stars. The core idea is simple: no production code gets written without a failing test first.

The framework enforces a 7-phase workflow. Brainstorm the approach. Write a spec. Create a plan. Write failing tests (TDD). Spin up subagents to implement. Review. Finalize. Every phase has gates. You cannot skip ahead. The iron law is that production code only exists to make a failing test pass.

This sounds rigid. It is. That is the point.

Superpowers includes a Visual Companion for design decisions, which helps when you are making architectural choices that need visual reasoning. The main orchestrator manages the entire workflow from a single context window, delegating implementation work to subagents that run in isolation.

The tradeoff is that the mega-orchestrator pattern means the orchestrator itself can hit context limits on very long sessions. One big brain coordinating everything works well until the big brain fills up. For most projects, this is not an issue. For marathon sessions with dozens of files, keep it in mind.

The workflow breaks down into skills that trigger automatically:

Skill	Phase	What it does
`brainstorming`	Design	Refines rough ideas through Socratic questions, saves design doc
`writing-plans`	Planning	Breaks work into 2-5 minute tasks with exact file paths and code
`test-driven-development`	Implementation	RED-GREEN-REFACTOR: failing test first, minimal code, commit
`subagent-driven-development`	Implementation	Dispatches fresh subagent per task with two-stage review
`requesting-code-review`	Review	Reviews against plan, blocks progress on critical issues
`finishing-a-development-branch`	Finalize	Verifies tests pass, presents merge/PR/keep/discard options

The results speak for themselves. The chardet maintainer used Superpowers to rewrite chardet v7.0.0 from scratch, achieving a 41x performance improvement. Not a 41% improvement. 41 times faster. That is what happens when every code change has to pass a test: the agent optimizes aggressively because it has a safety net.

Superpowers works with Claude Code, Cursor, Codex, OpenCode, GitHub Copilot CLI, and Gemini CLI.

GSD: preventing context rot before it ruins your project

GSD (Get Shit Done) was created by Lex Christopherson and has over 51K stars. Where Superpowers focuses on test discipline, GSD attacks the context window problem directly.

The key architectural decision: GSD does not use a single mega-orchestrator. Instead, it assigns a separate orchestrator to each phase of work. Each orchestrator stays under 50% of its context capacity. When a phase completes, the orchestrator writes its state to disk as Markdown files, then a fresh orchestrator picks up where the last one left off.

Think about why this matters. With a single orchestrator, your 200K token context window is a shared resource. Instructions from hour one compete with code from hour three. GSD sidesteps this entirely. Every phase starts with a full context budget because the previous phase’s orchestrator handed off cleanly and shut down.

The state files use XML-formatted instructions because (it turns out) LLMs parse structured XML more reliably than freeform Markdown. GSD also includes quality gates that detect schema drift and scope reduction. If the agent starts cutting corners or wandering from the plan, the gates catch it.

GSD evolved from v1 (pure Markdown configuration) to v2 (TypeScript SDK), which tells you something about the level of engineering behind it. The v2 SDK gives you programmatic control over orchestration, not just static instruction files.

The tradeoff: GSD has more ceremony than the other two frameworks. For a quick script or a single-file change, the phase-based workflow is overkill. GSD earns its keep on projects that span multiple files, multiple sessions, or multiple days.

The core commands map to a phase-based workflow:

Command	What it does
`/gsd-new-project`	Full initialization: questions, research, requirements, roadmap
`/gsd-discuss-phase`	Capture implementation decisions before planning starts
`/gsd-plan-phase`	Research, plan, and verify for a single phase
`/gsd-execute-phase`	Execute all plans in parallel waves, verify when complete
`/gsd-verify-work`	Manual user acceptance testing
`/gsd-ship`	Create PR from verified phase work with auto-generated body
`/gsd-fast`	Inline trivial tasks, skips planning entirely

GSD supports the widest range of agents: 14 and counting. Claude Code, Cursor, Windsurf, Codex, Copilot, Gemini CLI, Cline, Augment, Trae, Qwen Code, and more.

GSTACK: when you need a whole team, not just an engineer

GSTACK was created by Garry Tan (CEO of Y Combinator) and has over 71K stars. It takes a fundamentally different approach from the other two frameworks.

Instead of disciplining a single agent, GSTACK models a 23-person team. CEO, product manager, QA lead, engineer, designer, security reviewer. Each role has its own responsibilities, its own constraints, and its own slice of the problem.

The framework enforces five layers of constraint. Role focus keeps each specialist in their lane. Data flow controls what information passes between roles. Quality control gates ensure standards at handoff points. The “boil the lake” principle means each role finishes what it can do perfectly and skips what it cannot, rather than producing mediocre work across everything. And the simplicity layer pushes back against unnecessary complexity.

The role isolation is what makes GSTACK distinctive. The engineer role does not see the product roadmap. The QA role does not see the implementation details. Each role only receives the context it needs to do its job. This is not just about efficiency. It prevents the kind of scope creep where an agent that knows everything tries to do everything.

“Boil the lake” is my favorite principle across all three frameworks. It is the opposite of how most agents work. Agents default to attempting everything and producing something mediocre. GSTACK says: do fewer things, but do them right.

The tradeoff: 23 specialist roles feels heavy for pure infrastructure work. If you are writing Pulumi programs and deploying cloud resources with component resources, you probably do not need a product manager role or a designer role. GSTACK shines when you are building a product, not just provisioning infrastructure.

Each slash command activates a different specialist:

Command	Role	What it does
`/office-hours`	YC partner	Six forcing questions that reframe your product before you write code
`/plan-ceo-review`	CEO	Four modes: expand scope, selective expand, hold, reduce
`/plan-eng-review`	Engineering manager	Lock architecture, map data flow, list edge cases
`/review`	Staff engineer	Find bugs that pass CI but break in production, auto-fix the obvious ones
`/qa`	QA lead	Real Playwright browser testing, not simulated
`/ship`	Release engineer	One-command deploy with coverage audit
`/cso`	Security officer	OWASP and STRIDE security audits

GSTACK works with Claude Code, Codex CLI, OpenCode, Cursor, Factory Droid, Slate, and Kiro.

Where each framework fits

	Superpowers	GSD	GSTACK
What it locks down	The dev process itself	The execution environment	Who decides what
Orchestration	Single orchestrator	Per-phase orchestrators	23 specialist roles
Context management	One window	State-to-disk, fresh per phase	Role-scoped handoffs
Where it shines	TDD, subagent delegation, disciplined plan execution	Marathon sessions, parallel workstreams, crash recovery	Product strategy, multi-perspective review, real browser QA
Where it struggles	Anything beyond the build phase	Overkill for small tasks, no role separation	The actual writing-code part
Best for	Solo devs who need test discipline	Complex projects that span days or weeks	Founder-engineers shipping a product
GitHub stars	149K	51K	71K
Agent support	6 agents	14+ agents	7 agents

For infrastructure work, GSD’s context management matters most. Long Pulumi sessions that provision dozens of resources across multiple stacks are exactly the scenario where context rot bites hardest. GSD’s phase-based approach keeps each orchestrator fresh.

Superpowers’ TDD workflow maps well to application code where unit tests are straightforward. Infrastructure testing is different. You cannot unit test whether an IAM policy actually grants the right permissions. You can test the shape of the policy with Pulumi’s testing frameworks, but the real validation happens at pulumi preview and pulumi up. Superpowers still helps here (discipline is discipline), but the TDD cycle is less natural for infra than for app code.

GSTACK shines when the project has product dimensions. If you are building a SaaS platform where the infrastructure serves a product vision, GSTACK’s multi-role governance keeps the product thinking connected to the engineering work. For pure infra provisioning, the extra roles add overhead without much benefit.

My honest take: none of these is universally best. Knowing your failure mode is the real decision.

What keeps going wrong	Try this	The reason
Code works today, breaks tomorrow	Superpowers	Forces every change through a failing test first
Quality drops after the first hour	GSD	Fresh context per phase, nothing carries over
You ship features nobody asked for	GSTACK	Product review before engineering starts
All of the above	GSTACK for direction, bolt on Superpowers TDD	No single framework covers everything yet

Combining frameworks with Pulumi workflows

These frameworks solve the “how” of agent orchestration. Skills (like the ones from Pulumi Agent Skills) solve the “what,” teaching agents the right patterns for specific technologies. Frameworks and skills complement each other. A skill tells the agent to use OIDC instead of hardcoded credentials. A framework makes sure the agent still remembers that instruction 200K tokens later.

GSD’s state-to-disk approach pairs naturally with Pulumi stack outputs. Each phase can read the previous phase’s stack outputs from the state files, so a networking phase can provision a VPC and the compute phase can reference the subnet IDs without any context window gymnastics.

Superpowers’ TDD cycle maps to infrastructure validation. Write a failing test (the expected shape of your infrastructure). Run pulumi preview (red, the resources do not exist yet). Run pulumi up (green, the infrastructure matches the test). This is not a perfect analogy since infrastructure tests are broader than unit tests, but the discipline of “verify before moving on” translates directly.

You do not have to pick one framework and commit forever. Try GSD for a long multi-stack project. Try Superpowers for a focused library. See which failure mode bites you most and let that guide your choice.

Getting started

github.com/obra/superpowers

github.com/gsd-build/get-shit-done

github.com/garrytan/gstack

All three frameworks support multiple agents. For Claude Code, the install commands are straightforward:

# Superpowers
/plugin install superpowers@claude-plugins-official

# GSD (the installer asks which agents and whether to install globally or locally)
npx get-shit-done-cc@latest

# GSTACK
git clone --single-branch --depth 1 https://github.com/garrytan/gstack.git ~/.claude/skills/gstack && cd ~/.claude/skills/gstack && ./setup

Check each repository’s README for Cursor, Codex, Windsurf, and other agents.

If you want a managed experience that handles orchestration for you, Pulumi Neo is grounded in your actual infrastructure, not internet patterns. It understands your stacks, your dependencies, and your deployment history. The 10 things you can do with Neo post shows what that looks like in practice.

Pick one and give it a project. You will know within an hour whether it fixes your particular failure mode.

Try Pulumi for Free

Token Efficiency vs Cognitive Efficiency: Choosing IaC for AI Agents

Engin Diri — Tue, 03 Mar 2026 00:00:00 +0000

When an AI agent writes infrastructure code, two things matter: how compact the output is (token efficiency) and how well the model actually reasons about what it’s writing (cognitive efficiency). HCL produces fewer tokens for the same resource. But does that make it the better choice when agents need to refactor, debug, and iterate? We ran a benchmark across Claude Opus 4.6 and GPT-5.2-Codex to find out.

These two goals pull in opposite directions

You might assume that the language producing fewer tokens is also the one models reason about best. Research into LLM-driven infrastructure generation suggests otherwise.

Where HCL wins on tokens

HCL is declarative and minimal. It requires no imports, no runtime constructs, and no language scaffolding. For simple infrastructure generation, HCL leads to fewer tokens and lower generation cost.

For a straightforward resource definition, HCL gets straight to the point:

resource "aws_s3_bucket" "example" {
 bucket = "my-bucket"
}

Compare that with the Pulumi TypeScript equivalent:

import * as aws from "@pulumi/aws";

const bucket = new aws.s3.Bucket("example", {
 bucket: "my-bucket",
});

The HCL version requires fewer tokens. No import statement, no variable declaration, no constructor syntax. For single-shot generation of simple resources, that compactness matters. But the picture changes once you account for deployability and refactoring.

What the data shows

HCL’s token advantage is real for simple generation. But agents don’t just generate once. They validate, repair failures, and refactor. We built a benchmark that measures the full cycle: an open-source tool that sends identical prompts to Claude Opus 4.6 (claude-opus-4-6) and OpenAI GPT-5.2-Codex (gpt-5.2-codex), requesting both Terraform HCL and Pulumi TypeScript for the same AWS infrastructure (VPC with public and private subnets across 2 AZs, an EC2 instance with security groups for SSH and HTTP, and an RDS PostgreSQL instance with a security group allowing port 5432 only from the EC2 security group, plus all networking: internet gateway, NAT gateway, route tables, and associations). We measured token consumption, cost, and deployability across two scenarios: initial generation and refactoring into reusable components.

Methodology: Temperature 0, 5 runs per combination, randomized execution order. Each generated output goes through a three-stage validation pipeline: formatting (terraform fmt for HCL, prettier for TypeScript), static analysis (terraform validate for HCL, tsc --noEmit for TypeScript), and provider-level validation (terraform plan for HCL, pulumi preview for TypeScript). Both plan and preview check against real AWS provider schemas without creating resources, making their pass rates comparable across formats. If plan/preview fails, the benchmark feeds the error back to the model for one self-repair attempt. At temperature 0, Claude Opus 4.6 produced near-identical outputs across runs (sd=0-4 tokens). GPT-5.2-Codex showed more natural variation (sd=130-165 tokens). With 5 runs per combination the results are directional, not statistically conclusive. Costs are estimates based on published pricing as of 2026-02-22. Full methodology and reproducible code:

github.com/dirien/iac-token-benchmark

Scenario 1: Generation

HCL uses fewer tokens for generation:

Provider	Format	Output tokens (mean)	LOC (mean)	Cost (mean)	Plan/Preview pass	Repairs needed
Claude Opus 4.6	Terraform	2,007	212	$0.051	5/5	0/5
Claude Opus 4.6	Pulumi TS	2,555	200	$0.065	5/5	0/5
GPT-5.2-Codex	Terraform	1,565	110	$0.022	2/5	2/5
GPT-5.2-Codex	Pulumi TS	2,322	147	$0.033	0/5	5/5

HCL produces 21-33% fewer output tokens across both models. For simple resource generation, this translates directly to lower cost. Pulumi TypeScript uses more tokens for fewer lines of code because imports, type annotations, and constructor syntax add tokens without adding functional lines.

The Plan/Preview column tells a more complete story. Claude Opus 4.6 produced deployable code on the first pass for both formats: 5/5 for Terraform and 5/5 for Pulumi. Neither needed repairs. GPT-5.2-Codex struggled with both formats, but Terraform fared slightly better (2/5 vs 0/5).

Scenario 2: Refactoring into reusable components

We took each model’s generation output and asked it to refactor the code into a reusable module or component with parameterized environment name, instance sizes, and availability zone count.

Provider	Format	Output tokens (mean)	LOC (mean)	Cost (mean)	Plan/Preview pass
Claude Opus 4.6	Pulumi TS	2,720	218	$0.082	5/5
Claude Opus 4.6	Terraform	3,379	345	$0.095	5/5
GPT-5.2-Codex	Pulumi TS	2,477	248	$0.038	4/5
GPT-5.2-Codex	Terraform	1,356	119	$0.021	0/5

This is where the results get interesting. Opus + Pulumi refactoring used 20% fewer tokens, cost 14% less, and passed pulumi preview on every run (5/5) with zero repairs. Opus + Terraform also ended up at 5/5 for terraform plan, but it needed repair cycles to get there. The benchmark run log tells the story:

# Pulumi refactoring: runs 29-33, sequential, no gaps = zero repairs
✓ [29/40] anthropic/pulumi-ts/refactor run 1 — 2721 tokens, $0.0817
✓ [30/40] anthropic/pulumi-ts/refactor run 2 — 2721 tokens, $0.0817
✓ [31/40] anthropic/pulumi-ts/refactor run 3 — 2721 tokens, $0.0817
✓ [32/40] anthropic/pulumi-ts/refactor run 4 — 2721 tokens, $0.0817
✓ [33/40] anthropic/pulumi-ts/refactor run 5 — 2714 tokens, $0.0816

# Terraform refactoring: 34→36→38→40→42, every run skips a number = every run triggered self-repair
✓ [34/40] anthropic/terraform/refactor run 1 — 3388 tokens, $0.0956
✓ [36/40] anthropic/terraform/refactor run 2 — 3339 tokens, $0.0944
✓ [38/40] anthropic/terraform/refactor run 3 — 3390 tokens, $0.0957
✓ [40/40] anthropic/terraform/refactor run 4 — 3388 tokens, $0.0956
✓ [42/40] anthropic/terraform/refactor run 5 — 3388 tokens, $0.0956

The skipped numbers (35, 37, 39, 41) are self-repair turns where the model received terraform plan errors and regenerated the code. Each repair consumed additional tokens that do not show up in the first-generation cost but do show up in the total pipeline cost.

GPT-5.2-Codex tells a different story. Both formats needed repair on every run, but what happened after repair is what matters:

# Codex + Terraform refactoring: repaired, but still failed plan (0/5 deployable)
terraform run 1 turn 2: plan_valid=False tokens=1559
terraform run 2 turn 2: plan_valid=False tokens=1165
terraform run 3 turn 2: plan_valid=False tokens=1068
terraform run 4 turn 2: plan_valid=False tokens=1134
terraform run 5 turn 2: plan_valid=False tokens=1101

# Codex + Pulumi refactoring: repaired, and preview passed (4/5 deployable)
pulumi-ts run 1 turn 2: plan_valid=True tokens=2246
pulumi-ts run 2 turn 2: plan_valid=True tokens=2567
pulumi-ts run 3 turn 2: plan_valid=True tokens=2187
pulumi-ts run 4 turn 2: plan_valid=True tokens=2531
pulumi-ts run 5 turn 2: plan_valid=False tokens=2647

Codex + Terraform used fewer tokens but produced zero deployable refactored code after repair. Codex + Pulumi used more tokens but recovered to deployable code 4 out of 5 times. TypeScript’s type errors gave the model enough information to fix the problems. HCL’s plan errors did not.

Total pipeline cost

This is the number that matters for production agent workflows. It includes generation, any self-repair cycles, and refactoring:

Provider	Format	Total tokens (mean)	Total cost (mean)
Claude Opus 4.6	Pulumi TS	8,183	$0.146
Claude Opus 4.6	Terraform	14,669	$0.249
GPT-5.2-Codex	Terraform	8,723	$0.084
GPT-5.2-Codex	Pulumi TS	15,211	$0.138

Opus + Pulumi had the lowest total pipeline cost at $0.146, 41% cheaper than Opus + Terraform at $0.249. The difference comes entirely from repair cycles: Pulumi needed zero repairs across both scenarios, while Terraform refactoring triggered self-repair on every run.

With Codex, Terraform had the lower pipeline cost ($0.084 vs $0.138), driven by its smaller token output. But Codex + Terraform produced zero deployable refactored code (0/5 plan pass), while Codex + Pulumi produced deployable code 4 out of 5 times.

What this means

HCL uses fewer tokens per generation. For single-shot resource creation, HCL’s compactness saves 21-33% on output tokens. That advantage is consistent across both models.
Pulumi produces deployable refactored code more reliably. With Opus, Pulumi refactoring passed preview 5/5 with zero repairs. Codex + Pulumi passed 4/5. Codex + Terraform passed 0/5.
Total pipeline cost favors Pulumi with Opus. Opus + Pulumi cost 41% less than Opus + Terraform across the full pipeline ($0.146 vs $0.249), because Terraform refactoring needed repair cycles that Pulumi did not.
The tradeoff depends on your model and workflow. Codex + Terraform is cheapest on raw tokens but produces no deployable refactored code. Codex + Pulumi costs more per token but actually deploys. Opus + Pulumi is the best of both: fewer refactoring tokens and zero repairs.

Why Pulumi refactoring deploys cleaner

The TerraFormer project identified what they call the correctness-congruence gap: LLMs generate configurations that look valid but fail to match the user’s architectural intent. An error taxonomy study cataloged the same pattern across models. A survey of LLMs for IaC found the gap between syntax validity and architectural correctness widens with infrastructure complexity.

Refactoring is where this gap bites hardest. Turning a flat resource list into a parameterized, reusable module requires the model to restructure dependencies, introduce variables, and compose abstractions. With Pulumi, the model can use TypeScript’s standard refactoring patterns: extract a class, add typed constructor parameters, compose functions. These are patterns it has practiced across millions of repositories during training. With HCL, the same refactoring requires count, for_each, dynamic blocks, and module variable plumbing, domain-specific constructs that have far less representation in training data.

Our benchmark confirms this directly. Opus produced 2,720 tokens for Pulumi refactoring versus 3,379 for Terraform, a 20% reduction, and every Pulumi run passed pulumi preview without repair. The Terraform refactoring runs all triggered self-repair because the restructured HCL modules had issues that terraform plan caught.

Training data distribution makes this structural. LLMs have far more TypeScript than HCL in their corpora. A model refactoring TypeScript draws on patterns from the entire open-source ecosystem. A model refactoring HCL modules has a much smaller pool. Since general-purpose languages dominate new code production, this gap will widen over time.

Tooling can close the gap further. The Pulumi MCP server gives AI agents direct access to resource schemas at generation time. A tool like get-resource returns every property, type, and required field for a given cloud resource. The agent does not have to guess from what it memorized during training. It can look up the correct schema before writing a single line of code.

This changes the workflow from “generate, fail, read error, retry” to “look up schema, generate correctly.” Agent skills push this further by encoding working Pulumi idioms as structured prompts, so the model starts from a known-good baseline. Terraform has no equivalent to this MCP-based schema lookup. That difference matters more with every iteration.

Where the industry is heading

One way to think about IaC language choice is through the lens of AI engineering maturity levels. At Level 3 (agentic coding), agents generate infrastructure from prompts. HCL’s 21-33% token savings on generation matters here. At Levels 4-5, agents iterate on specifications, refactor code, and maintain systems over time. Our benchmark shows this is where Pulumi pulls ahead: 41% lower total pipeline cost with Opus, and more deployable refactored output with both models.

The industry is moving toward Levels 4-5. Agents are taking on refactoring, feature flags, environment parameterization. 43% of DevOps teams need four or more deployment iterations before infrastructure is production-ready. The first-generation token advantage HCL holds applies to the task that is shrinking as a share of agent workloads. The refactoring and deployability advantages that Pulumi offers apply to the tasks that are growing.

Choosing based on your workflow

If your agents primarily generate well-scoped resource definitions, HCL saves 21-33% on output tokens. That advantage is real and consistent.

If your agents need deployable output on the first pass (which avoids repair costs entirely), our data shows Opus + Pulumi is the strongest combination: 5/5 plan/preview pass for both generation and refactoring, zero repairs, lowest total pipeline cost.

If your agents evolve infrastructure over time through refactoring and modularizing, Pulumi produced deployable refactored code more reliably across both models we tested (5/5 and 4/5 vs 5/5 and 0/5 for Terraform).

Pulumi covers the full iteration loop, from generation through repair to refactoring, using the same language patterns and tooling that models already know.

Get started

Ready to explore how AI agents work with Pulumi? Check out Pulumi AI to see LLM-powered infrastructure generation in action, or get started with Pulumi’s documentation.

Join the conversation in the Pulumi Community Slack or Pulumi Community Discussions.

The Superintelligence Flywheel: Infrastructure for the AI Era

Joe Duffy — Thu, 11 Dec 2025 00:00:00 +0000

We’ve been in the infrastructure business for nearly a decade, and we’ve never been more excited about, or in awe of, the scale we are seeing as the industry pursues superintelligence. We are now hitting a tipping point that requires entirely different approaches to managing and scaling infrastructure in this new era.

What do we mean by superintelligence? Superintelligence means AI systems that operate with genuine autonomy—planning, reasoning, executing, adapting—at scale, on the path toward human-level and eventually superhuman intelligence. The infrastructure needed to accomplish this is greater than anything we’ve ever seen. Jensen Huang projects $600 billion in AI infrastructure spending this year, scaling to $3-4 trillion by decade’s end. Stargate committed $500 billion to AI infrastructure in the U.S. Microsoft, Meta, and Google are each spending $70-90 billion annually on datacenters. AWS just activated Project Rainier, a data center scaling to one million custom Trainium chips for Anthropic’s frontier models.

Superintelligence is driving the biggest, fastest infrastructure scaling period in the history of computing. This is exciting but comes with challenges: all of that infrastructure has to be managed, secured, scaled, made compliant, and cost effective. Legacy infrastructure tools weren’t built for this reality—they add friction that slows progress or breaks it altogether.

This reveals an important insight:

The infrastructure required to build superintelligence demands superintelligence for infrastructure.

The systems being built today for superintelligence are already straining human platform teams to their limits and yet we’re still only just getting started. To succeed, we will have no choice but to use AI itself to help us manage the infrastructure scaling ahead on the path to superintelligence.

Superintelligence demands more infrastructure, which demands superintelligent approaches to managing and scaling that infrastructure, which leads to faster progress towards superintelligence.

We call this the superintelligence flywheel.

The Flywheel Defined

The superintelligence flywheel describes a self-reinforcing cycle between two forces:

Superintelligence Infrastructure: The massively distributed compute, storage, networking, and orchestration systems required to train and serve frontier AI models.

Infrastructure Superintelligence: AI systems capable of managing infrastructure with increasing autonomy. These systems handle the full spectrum of infrastructure automation, from provisioning to securing to evolving and scaling entire environments.

These reinforce one another. As AI infrastructure grows more complex, human teams will increasingly use AI to manage it. This affects all of us: frontier labs, technology innovators, and enterprises modernizing their products and workforces with an AI-first mindset. Infrastructure agents are simply a requirement for all of us in this new world.

The flywheel accelerates.

The Technical Reality

Frontier labs have the most extreme requirements, but everybody else is somewhere on the curve. You don’t have to be training GPT-5 to feel this pressure. If you’re deploying fine-tuned models, running inference at scale, or just trying to keep up with AI-assisted developers shipping faster—you’re on the same curve, just earlier. To stay competitive and innovative in this new era, access to infrastructure needs to be democratized and inherently automation- and experimentation-friendly.

This can be seen most vividly with frontier lab-scale infrastructure challenges:

Training starts with data preparation. Petabytes of data turns into trillions of curated tokens across thousands of CPU cores. Pre-training runs massive GPU clusters hot for months, at a scale where hardware failures are statistical certainties. Checkpoint management, gradient synchronization, and fault-tolerant scheduling become first-order concerns. Fine-tuning and reinforcement learning add even more coordination complexity: synthetic data generation, AI judges, multiple model copies running simultaneously in tightly coupled loops, hundreds of experiment variants, infrastructure that scales elastically as experiments converge or diverge.

Inference means serving models to an ever-elastic sea of users across massive GPU fleets, many clouds, and distributed globally. ChatGPT, for instance, handles over 2.5 billion queries per day with sub-second time-to-first-token expectations. Traffic patterns shift by geography, time, and viral adoption, and the software must be ready to react. The difference between good and poor GPU utilization at this scale can make or break the user experience and the bank.

These examples share three properties that break traditional infrastructure approaches:

Massive scale: Thousands or millions of coordinated resources with complex architectures across multiple regions and cloud providers.
Extreme dynamism: Infrastructure that must spin up, reallocate, and tear down continuously based on job requirements, spot pricing, capacity availability, and failures.
Velocity beyond human capacity: Infrastructure changes that occur faster than human operators can review, approve, and execute manually, often initiated by AI agents.

Of course, even smaller scale examples share these challenges. Whether you’re using AWS SageMaker or Google Vertex AI to fine-tune models for your enterprise, building out your data engineering function to train custom models, or moving your first ML workloads to production—these infrastructure demands become relevant as you scale.

AI-assisted software development tools are making engineers dramatically more productive. Developers are shipping more code than ever before. But unlike software engineering, legacy infrastructure practices lack inherent guardrails to enable AI-assisted infrastructure engineering: linters, static analysis, testing frameworks, canaries, staged rollouts. The tooling that worked when humans were the limiting factor won’t work when AI agents are generating and deploying infrastructure code at machine speed.

Infrastructure Fungibility: The Holy Grail

Instead of treating each use case as a special snowflake, the holy grail is infrastructure fungibility across many clouds. In this world, infrastructure is dynamically allocated based on availability, demand, and data locality. Inference gets served based on current user demand. Experiments get capacity when it’s available. Long-horizon training runs on whatever’s left over.

This is extraordinarily difficult. But without it, the costs compound: researchers waiting for capacity, GPU-hours sitting idle between workloads, time delays that let competitors ship first. At frontier scale, even small inefficiencies translate to tens of millions of dollars.

This level of dynamism can’t be managed by humans watching dashboards and approving PRs. The infrastructure itself needs to become intelligent—capable of planning, executing, and adapting without waiting for human operators.

The frontier labs building this have learned through direct experience. They have historically had to work around legacy infrastructure tools and slow-moving DevOps teams, taking matters into their own hands. As one infrastructure lead put it: “There are thirty people in the world that know how to do this right now.” It is time for this knowledge to spread faster, in the hands of platform teams, because everyone building with AI is running into some version of the same problem.

Towards a Superintelligence Platform

There are two foundations that form a cohesive superintelligence platform:

Foundation #1: Pulumi as the Infrastructure Automation Substrate

We built Pulumi on three principles that are well-suited to the unique challenges of this era:

Infrastructure as code must be actual code. Domain-specific languages and static configuration formats create a ceiling. Instead, standing on the shoulders of giants with real programming languages provides type systems, conditionals, loops, abstractions, package management, testing frameworks, and more. These capabilities facilitate scale. Just as importantly, your team already knows Python, TypeScript, and Go—why force them to learn yet another proprietary DSL and context-switch constantly between application code and infrastructure configuration?

But there’s a deeper reason this matters now. LLMs have been trained on billions of lines of Python, TypeScript, and Go, and deeply understand how to code correctly. Infrastructure in real programming languages becomes infrastructure that AI systems can reason about, generate correctly, and modify safely. We call this an “LLM-sympathetic architecture”: by projecting cloud infrastructure into code space, LLMs can manipulate infrastructure through coding, something they are already superb at which ensures it can be done repeatably and correctly.

One substrate across all of the clouds. Every cloud, every SaaS provider. Frontier labs run training jobs wherever capacity exists: AWS, Azure, Google Cloud, Oracle Cloud, CoreWeave, Lambda Labs. GPU availability is the constraint; cloud provider preference is now secondary.

A single lab may prefer to train on CoreWeave, overflow to AWS when spot capacity becomes available, and tap Google Cloud when the reservations in other cloud providers are used up. Inference deployments span every region where users exist, across whichever providers offer the best latency and cost characteristics.

Infrastructure platforms that aren’t inherently multi-cloud simply can’t serve these requirements, forcing teams to adopt many fragmented tools and workflows. A platform for the superintelligence era simply must treat multi-cloud as a given with unified resource management, policy enforcement, and state tracking across all providers.

Agent-ready architecture. Human-in-the-loop infrastructure management doesn’t scale to the velocity of the new era. The path forward is progressive autonomy: AI agents that operate infrastructure with increasing independence, bounded by human-defined policies and guardrails.

This requires platforms designed for agent interaction from the ground up, with APIs agents can invoke, state they can reason about, policies that define boundaries, audit trails that maintain accountability. The infrastructure platform becomes the substrate on which agents operate. Infrastructure as code, it turns out, is already very good at this thanks to its declarative core.

Foundation #2: Neo as the Agentic Layer

If managing superintelligence scale infrastructure requires AI agents that can handle infrastructure automation, the critical question is: what should those agents look like?

Neo is our answer. Neo is an AI agent that operates infrastructure like a senior platform engineer, built on top of Pulumi’s foundation. Because Pulumi already provides the guardrails that software engineering takes for granted—previews, policy enforcement, state tracking—Neo can handle infrastructure tasks automatically while validating changes before they hit production.

Uniquely enabled by infrastructure as code. Neo operates on infrastructure code, not opaque configurations or cloud APIs directly. Even for resources not yet managed by Pulumi, Neo seamlessly imports them under code management before making changes. Every action flows through version control, policy checks, and state tracking. This ensures the underlying LLM’s coding smarts apply to infrastructure changes.

Built for progressive autonomy. Organizations configure Neo’s independence by environment and task type. Dev environments might permit fully autonomous operation—daily waste cleanup, weekly drift reconciliation—while production changes may require human approval. When Neo encounters unexpected state or errors, it can self-diagnose or loop in a human for assistance as needed. As confidence builds, the autonomy boundary expands, towards full autonomy.

Inherently understands and respects guardrails. The same velocity that creates opportunity creates risk. An agent that can provision 1,000 resources can also misconfigure 1,000 resources. Pulumi’s infrastructure as code foundation provides the safety mechanisms software engineering takes for granted: previews before deployment, policy as code enforcement, audit trails, human-in-the-loop approvals, and organizational best practices encoded in components. Just as you wouldn’t ship code without version control, you shouldn’t let AI manage infrastructure without these guardrails.

Agentically automate anything: Give Neo a prompt like “Provision a PyTorch training environment for LoRA fine-tuning on GKE with 50 A100s using Nvidia’s GPU Operator and Kueue—then run a smoke test.” Neo generates the project, previews changes, provisions everything, verifies it works. Days become minutes. It’s not just for new infrastructure: one customer used Neo to tackle 300,000+ policy violations—work they estimated would take years is now done in weeks.

Frontier models plus infrastructure smarts: Neo builds on a mixture of frontier models while incorporating nearly a decade of infrastructure expertise. Neo benefits from Pulumi’s nearly one billion deployments, hundreds of thousands of AI interactions and known-to-be-working code examples, and many petabytes of semantically-understood, anonymized infrastructure metadata. This foundation is something that no other company in the world has at their disposal.

Pulumi + Neo: The Superintelligence Platform

Pulumi without Neo is already a powerful infrastructure platform—but one that still requires human operators for every change. Neo without Pulumi would never be production-ready, as it would lack the cloud-to-code projection and guardrails that make Neo so good. Together, humans define intent and policy, and intelligent agents handle execution with autonomy.

This is the foundation for setting the superintelligence flywheel in motion.

Evidence at Scale

The flywheel isn’t theoretical. We’re facilitating it daily with thousands of customers:

Hyper-scale: Wiz manages over one million cloud resources across tens of thousands of Kubernetes clusters in hundreds of data centers across every cloud provider imaginable. They do hundreds of thousands of updates daily across thousands of Kubernetes clusters, leading to 5x faster customer onboarding and entering new markets in days rather than months. They treat infrastructure provisioning as just another service in their distributed system.

Infrastructure democratization: Supabase achieved the infrastructure velocity needed to keep up with the AI era’s demand for their own product, empowering 20x more teammates to scale infrastructure in TypeScript, the same language as their application code, rather than proprietary DSLs that nobody understood. New regions deploy in about a week, with 43,000 new databases daily and 100,000+ API calls per second. According to their platform team, now thanks to Pulumi, “the infrastructure team acts as groundkeepers, not gatekeepers.”

Frontier science: We manage infrastructure for organizations including NVIDIA and multiple frontier labs building the models that will define the next decade. The scale requirements for these workloads exceed any we’ve ever seen and are only getting bigger. Just as many customers democratize infrastructure for their developers, we’re seeing frontier labs use Pulumi to accelerate getting science into production with self-service infrastructure for AI researchers.

The Superintelligence Flywheel is in Motion

AI is driving the largest infrastructure buildout in history—whether you’re training frontier models or deploying your first ML workloads to production. That complexity demands intelligent agents. Organizations that embrace this—building infrastructure for AI while using AI for infrastructure—will define the next era of computing.

The superintelligence flywheel is spinning fast, and it’s only going to get faster from here.

Pulumi provides the infrastructure platform for the superintelligence era: infrastructure as code in general-purpose languages, unified multi-cloud management, and Neo for progressive infrastructure autonomy.

Pulumi Neo ・ Superintelligence Infrastructure ・ Case Studies ・ Contact

AI Predictions for 2026: A DevOps Engineer's Guide

Engin Diri — Thu, 11 Dec 2025 00:00:00 +0000

The IDE is dying, and so is tool calling. OpenAI is not going to win. And next year, you’re going to be shipping code that you’ve never reviewed before, even as an experienced engineer.

These are bold claims, but the way we use AI in 2026 for coding and agents is going to look completely different. In this post, I want to cover my predictions and why they matter right now for DevOps engineers. Some of these are definitely hot takes, but that’s what makes this conversation worth having.

IDEs are dead (at least as we know them)

Traditional IDEs where the code is the focus of the interface are simply going to become irrelevant. We’re moving toward agent manager interfaces where we can kick off agents in parallel to work on different features in a codebase or even work on different projects at the exact same time.

We’re already seeing this transition. Google Antigravity combines a familiar AI-powered coding experience with a new agent-first interface. You can deploy agents that autonomously plan, execute, and verify complex tasks across your editor, terminal, and browser. Cursor 2.0 lets you run up to eight agents in parallel on a single prompt, using git worktrees or remote machines to prevent file conflicts. Each agent operates in its own isolated copy of your codebase.

AWS validated this direction at re:Invent 2025 by announcing “frontier agents” including Kiro for autonomous coding, along with dedicated security and DevOps agents. These agents maintain state, log actions, operate with policy guardrails, and integrate directly with CI/CD pipelines.

For infrastructure specifically, Pulumi Neo represents this same shift. Instead of writing code or running CLI commands for every operation, you describe what you need in natural language and Neo handles the implementation. It works across your entire infrastructure, understanding dependencies and creating execution plans that go through pull requests for review.

For DevOps engineers, this means your pipelines need to accommodate AI-generated code at scale. Multiple agents working simultaneously need isolated, reproducible environments. More generated code means more artifacts to track, version, and deploy.

The different paths to AI dominance

A lot of people think that in the future, a single large language model will have a monopoly and be the best at absolutely everything. But what’s really going to happen is different providers will specialize and focus on being the best at different things.

Google is going down the generalist route with Gemini, aiming to be the jack-of-all-trades. Anthropic is focusing on being the best for coding. You can see this in the benchmarks: when Opus 4.5 came out, the first benchmark they highlighted was for software engineering, because that’s what Anthropic is focusing on.

Amazon is carving out its own niche with the Nova model family, announced at re:Invent 2025. The Nova 2 lineup includes specialized models: Pro for complex reasoning, Sonic for real-time voice conversations, and Omni for simultaneous text, audio, and video processing. With Nova Forge, organizations can build custom frontier models by combining their proprietary data with AWS open weight models. The re:Invent message was clear: leveraging your first-party data is now fundamental to going beyond generic AI. We’re talking about 30-40% increases in accuracy when you bring your own data into the equation.

But here’s the hot take: I don’t think OpenAI is going to come out on top with any kind of specialization. They’ve disappointed time and time again with GPT-5 and GPT-4.5. With 4.5, they seemed to try to be the creative specialist, but it just didn’t work. The GPT-5 launch in August 2025 was described as “barely better than last month’s flavor of the month” and on some metrics it’s actually worse than earlier models.

For DevOps teams, this specialization means you’ll need infrastructure that’s model-agnostic and supports multiple AI backends. Plan for secrets management across multiple LLM providers and design your systems to swap models based on the task at hand.

The local AI breakthrough

2026 will be the year of local AI. We didn’t see that much this year besides DeepSeek at the start of 2025, which was a big deal. We had a couple of new models like Qwen 3, but nothing that fundamentally changed the game. Now we’re starting to see new hardware that makes it obvious we’re going to be able to run very large models on smaller devices.

There’s new AI chips that can run upwards of 120 billion parameter large language models on the edge, which would be a complete game-changer. Right now, hardware requirements are one of the biggest challenges for scaling local AI. If we can solve the hardware problem, we get 100% data privacy and zero-millisecond latency for our agents.

AWS is addressing this with Trainium3 UltraServers, their 3nm AI chips delivering 4.4x more compute than the previous generation. More significantly, AWS AI Factories allow organizations to deploy racks of Trainium chips and NVIDIA GPUs directly into their own data centers, addressing data sovereignty concerns while keeping AI inference close to the data.

For DevOps, this opens the door to zero-latency inference in CI/CD pipelines, complete data privacy for sensitive codebases, and reduced cloud costs for AI-heavy workloads.

Engineers as system architects

We’re finally going to get to the point where we’re not the coders. We delegate that entirely to our coding agents, and we become the system architects. This mirrors the evolution of other engineering disciplines. Civil engineers don’t fabricate the steel beams; they design the structure and verify the integrity.

I think of this as a three-step process:

Define: Set your objectives and the system that your agents will operate under
Orchestrate: Delegate the coding to your agents
Validate: Ensure the quality of the outputs and the overall system

We’re still in the loop. We are the final say in whatever is created, but we’re delegating the grunt work to our coding agents.

This is exactly the model that Pulumi Neo implements for infrastructure. When you give Neo a complex request, it creates a task plan outlining the steps it will take to accomplish your goal. This plan provides transparency into Neo’s approach and gives you the opportunity to adjust the strategy before execution begins. Neo operates in different modes: Review mode where everything requires approval, Balanced mode where only deployments need sign-off, or Auto mode for full autonomy. You define the boundaries, Neo orchestrates the work, and you validate through pull requests and previews.

For DevOps engineers, this shift means building robust validation infrastructure becomes critical. When AI writes the code, you need automated testing pipelines, security scanning, and verification systems that can operate at the speed of AI-generated changes.

Code execution is replacing tool calling

Here’s a key insight that kept coming up at re:Invent: models are no longer the bottleneck. Context is. Our agents are going to change a lot next year because code execution is starting to replace tool calling. The problem with tool calling right now is that all the capabilities you give an agent take up context upfront. When you try to give a lot of different tools to an agent, you completely overwhelm it.

Anthropic’s research on code execution with MCP addresses exactly this problem. Code execution is a massive token reduction, faster, and more flexible. You’re giving the agent the ability to generate its own capabilities at runtime by writing code to interact with APIs. A workflow that previously consumed about 150,000 tokens when tools were passed directly through the model was reimplemented with code execution and used only about 2,000 tokens. That’s a 98.7% reduction.

AWS embraced this pattern with Amazon Bedrock AgentCore, which now includes code interpretation capabilities. AgentCore supports any agent framework (CrewAI, LangGraph, OpenAI SDK) and provides memory, browser tools, and observability features that make code execution practical at enterprise scale.

For DevOps, this means you need sandboxed, secure execution environments for AI-generated code. Running agent-generated code requires appropriate isolation, resource limits, and monitoring.

Progressive disclosure and composable skills

The best part about code execution flexibility is it unlocks progressive disclosure. All I mean by that is: you have a lot of capabilities for an agent, but you don’t actually give all of them upfront. Instead, you allow the agent to discover capabilities and then leverage them in a more flexible way.

For each capability, you just have a bit of metadata or description that loads upfront. When the agent decides to leverage that capability, then you load the full instructions. Now you can practically scale to infinity because all capabilities don’t have to be loaded at runtime.

Claude Skills is a good example of this pattern. Skills are organized folders of instructions, scripts, and resources that agents can discover and load dynamically. At session start, the agent scans available skills and populates the system prompt with just a brief name and description (around 100 tokens). The full skill prompt loads only after Claude selects it, preventing context bloat while maintaining discoverability.

Kiro Powers addresses the same problem. Connecting five MCP servers can consume over 50,000 tokens, roughly 40% of an AI model’s context window, before you even type your first request. Powers bundle MCP servers, steering files, and hooks into units that load dynamically based on conversation context. Mention “payment” and the Stripe power activates. Datadog, Figma, and others have powers available.

For DevOps, this translates to modular infrastructure definitions, on-demand capability loading, and efficient resource utilization. Think about how you can apply this pattern to your own automation.

Agent-to-agent protocols are finally happening

Agent-to-agent protocols are where AI agents operate in a peer network, discover each other’s capabilities in real time, and interact autonomously. When Google released their A2A protocol earlier this year, there was a ton of buzz. A lot of people thought it was going to be the next big standard, like the next MCP. But then it kind of fell to the wayside.

The big reason is the chicken-and-egg problem. For A2A to be useful, you need many people to adopt it at the same time. Otherwise, if you build an A2A-compatible agent, it has no other agents to talk to. The whole value proposition is lost unless you already have a big network to attach to.

But that’s finally changing. The Linux Foundation launched the A2A project in June 2025, and adoption is accelerating. Adobe, Microsoft, SAP, ServiceNow, and S&P Global are all implementing A2A. In July 2025, Google released version 0.3 of the A2A protocol with a more stable interface critical to accelerating enterprise adoption.

Machines paying machines

My next big prediction is that machines paying machines is going to become a very big thing. Coinbase released the x402 protocol for exactly this: building AI agents that you expose over the internet but require payment whenever someone else interacts with them.

This goes really well with agent-to-agent protocols. You can create a peer network where you monetize your agents. They all leverage each other but make payments whenever they take advantage of another agent’s capabilities. Cryptocurrency is the perfect solution for this kind of machine-to-machine network because it enables micropayments to be made quickly and globally.

The x402 protocol has achieved 156,000 weekly transactions with 492% growth since launching in May 2025. It’s now integrated with Anthropic’s MCP Protocol, Google Gemini, OpenAI Codex, and other platforms. Stablecoins like USDC enable charging per request, per service, or per second of usage at near-zero transaction costs, enabling payments as low as $0.001 per request.

Artifact reviews instead of diff reviews

When we want to do a rigorous code review, traditionally, we look line by line at all the changes. But coding agents are getting to the point where they can prove their code works through artifacts. Instead of reviewing line by line, we can look at browser recordings, full working demos of a backend API, and other artifacts.

Google Antigravity is a perfect example. As part of its coding process, it can autonomously spin up your website, visit it, scroll through it, take screenshots, and record everything. Agents generate artifacts, including tangible deliverables like task lists, implementation plans, screenshots, and browser recordings. You can verify the agent’s logic at a glance.

Amazon Nova Act takes this further. It enables AI agents to automate browser-based tasks like form filling, QA testing, and workflow validation with over 90% reliability. The service includes built-in observability through live viewing, CloudTrail logging, and session replay, making it possible to review what an agent actually did rather than parsing through code changes.

Shipping code you’ve never read

For the last prediction, we’re tying everything together. We’ve talked about reviewing artifacts instead of diffs, creating systems instead of coding, and the new capabilities for agents with code execution.

We’re going to get to the point very quickly where we’re shipping code that we have never read before. And I’m not talking about people who vibe code. Even experienced engineers are going to trust their systems so much that they have the ability to review the code but they’re not going to. We’re just going to ship to production after reviewing the artifacts.

I presented on this exact topic at the Tel Aviv Pulumi User Group meetup at Qodo HQ back in October, where I demonstrated how Pulumi Neo’s autonomous decision-making capabilities can handle infrastructure tasks that we traditionally managed manually. Qodo is doing fascinating work in this space with their agentic development tools, building systems that let you trust the output without necessarily reviewing every line.

I’m not saying we’re taking the human completely out of the loop. I’m saying we’re going to have a lot of trust in our systems and a validation process that includes us, but that doesn’t necessarily have to be us actually looking at the code. Tools like Pulumi Neo create pull requests with clear documentation of changes, run previews to validate infrastructure modifications, and provide the transparency needed to ship with confidence.

The path forward

The predictions I’ve outlined point to a fundamental shift in how software gets built and deployed. For DevOps engineers, this isn’t a threat but an opportunity to become more strategic and less operational. We’re entering the battle of the agentic frameworks, where the winners will be those who can build faster, cheaper agentic applications through their platforms.

The immediate reality is that your CI/CD pipelines need to accommodate AI-generated code at scale, your secrets management needs to handle multiple LLM providers, and your execution environments need proper sandboxing for agent-generated code. These aren’t future concerns; they’re requirements for working effectively with the AI tools available today.

Looking further out, the engineers who thrive will be those who embrace the system architect role. Define clear objectives and constraints for your AI agents. Build validation frameworks that can verify outcomes without requiring line-by-line code review. Design infrastructure that’s modular enough to load capabilities on demand.

The technology to make this happen already exists. Agent orchestration platforms are shipping. Code execution is replacing tool calling. Progressive disclosure patterns are proven. The question isn’t whether these changes are coming; it’s whether you’ll be ready when they arrive.

Start building for the AI-driven future today

If you want to experience what this future looks like right now, Pulumi Neo is the place to start. Neo lets you make natural language requests for routine infrastructure tasks, analysis, and management. Instead of writing code for every operation, you describe what you need, and Neo handles the implementation, creating task plans, running previews, and opening pull requests for your review.

Whether you’re looking to update outdated resources across your infrastructure, analyze your cloud spend, or automate complex multi-step workflows, Neo provides the agent-first experience that’s defining the next generation of DevOps tooling.

Get started with Pulumi Neo and see how AI-powered infrastructure automation can transform your workflow.
Project: Build a production-ready AI agent by moving from local prototypes to deployed infrastructure with Amazon Bedrock AgentCore, the Strands SDK, and Pulumi.

AWS built an integrated AI Agent training pipeline and they want you to rent it

Adam Gordon Bell — Wed, 10 Dec 2025 00:00:00 +0000

AWS re:Invent 2025 delivered a myriad of announcements across AI, silicon, and cloud infrastructure. AWS unveiled the expanded Nova model family, introduced Nova Forge for custom model training, launched Trainium3 UltraServers, and added major production features to AgentCore. It was a lot, and taken at face value, it looks like another scattershot year of big releases.

But if you look past the firehose, a pattern emerges. These announcements fit together into a single bet about how enterprise AI will be built.

AWS built a vertically integrated agent-training pipeline; it’s expensive, ambitious, and not for everyone.

The clearest place to see that pattern is in how AWS talked about the Nova models. AWS dropped four new foundation models (Lite, Pro, Sonic, Omni) spanning text, multimodal, and speech. And AWS downplayed benchmarks entirely. A hint that the models aren’t the real story.

What AWS was really foregrounding wasn’t the models themselves, but the system used to shape them: Nova Forge.

Try it with Neo: Set up Nova 2 with Pulumi

Rent the lab: Nova Forge

Nova Forge is a managed way to run continued pretraining, fine-tuning, and reward-based alignment on Amazon’s Nova models using your data and your reinforcement loops. Instead of a finished, frozen model plus a thin fine-tuning API, you feed your data into earlier training stages while AWS handles the ugly parts: large-scale training runs, cluster management, and hosting. Access is $100,000 per year¹, plus compute costs!

If you can afford that, you bring big proprietary datasets (code, tickets, logs, documents) and they keep doing next-token pretraining on a mix of their data and yours, then instruction tuning (SFT), then “RL-style” preference optimization, but with your data and your reward signals mixed in.

Why does this exist? Because the kind of training this enables is usually out of reach. Training a GPT-4-class frontier model from scratch runs tens of millions to $100M+ in compute alone². You don’t own the weights and you’re locked into their stack, but you get frontier-level capabilities with your data baked in, without building datacenters or staffing ML teams.

Think of it as frontier-lab-as-a-service. No one else offers anything this close to a public, end-to-end training pipeline. And the only reason AWS can offer it is the next announcement.

The margin weapon: Trainium

The idealized Trainium flywheel: each generation should decrease training costs.

AWS built its own AI accelerator so it doesn’t have to live entirely on NVIDIA. Trainium is that chip. You don’t buy it; you rent it as a cloud box. This year: their third-gen chip (Trainium3) and new rack-scale Trn3 UltraServers are out, with 4× the performance and big energy/cost gains over the previous gen, positioned as a serious alternative to high-end GPUs for training and serving big models.

I’m sure one reason for Trainium’s is that AWS wants to stop handing NVIDIA half its AI training revenue. But the real story is bigger than cost-cutting. Trainium is the quiet machinery that makes AWS’s model-factory ambitions economically viable. You can only rent a frontier training pipeline if you can afford it, and Trainium makes it cheaper (if that word applies to six-figure entry costs).

Trainium is what turns Forge from a one-off experiment into an actual development pipeline. By compressing the marginal cost of each training cycle, AWS is trying to make iterative specialization economically viable. You can tune, test, and retrain until you converge on something useful.

AWS is clearly positioning Trainium3 to anchor a fully vertical stack

Spinning up a Trainium instance with Pulumi (where available):

import pulumi_aws as aws

# Assumes: ami, subnet, security_group already configured
trn1_instance = aws.ec2.Instance("trn1-instance",
 instance_type="trn1.2xlarge",
 ami=ami.id,
 subnet_id=subnet.id,
 vpc_security_group_ids=[security_group.id],
 associate_public_ip_address=True,
 tags={"Name": "trn1-training-instance"},
)

Try it with Neo: Provision Trainium instances

The data moat play

For most companies, this whole stack is overkill. If your AI roadmap is “add a chatbot and maybe summarize some tickets,” you don’t need Nova Forge, and you definitely don’t need Trainium. Hosted models plus RAG and an Agentic loop will get you 90% of the way there.

But this type of training is powerful, and it’s never been in reach to so many. If LLMs behave like the distributions they’re trained on, then getting your proprietary mess (logs, incident reports, claims histories, deal flows, call transcripts) into the core training loop means the model doesn’t just know your docs; it behaves like someone who’s lived inside your systems. That’s qualitatively different from “we stuffed a PDF into the context window.”

Latency and cost at scale matter too. For high-volume workflows like support triage, routing, code review, and fraud checks, “generic frontier model + giant prompt + RAG + tools” is slow and expensive. A smaller model that has your world baked into the weights can run with smaller contexts, simpler prompts, and fewer tool calls. And then there is reinforcement learning, which I’ll get to shortly.

But even if you get that far, a custom Nova model sitting in Bedrock is only half the story. You still need somewhere for it to act: a runtime, tools, policies, and an audit trail. That’s the gap AgentCore is meant to fill.

Where the models work: AgentCore

AgentCore: building blocks so you don't have to wire agents from scratch.

If Nova is the brain and Trainium is the muscle to build it, AgentCore is the nervous system.

AgentCore is a managed runtime for AI agents: instead of you wiring LLMs, tools, memory, auth, and logging together on Lambda or Fargate, AWS gives you a sticky per-session microVM, a standard way to call tools (Gateway), built-in long- and short-term memory, identity/permissions, and observability/evals. You package your agent, deploy it as an AgentCore runtime, and AWS handles the ugly parts: session isolation, scaling, policy guardrails, and tracing. You pay Fargate-ish per-vCPU/GB-hour pricing for the runtime plus normal Bedrock token and tool-call costs.

At re:Invent 2025, AgentCore picked up the missing “production” pieces: Policy, Evaluations, and episodic Memory. These handle guardrails, quality checks, and per-session state, so you don’t have to build them yourself.

Deploying an AgentCore runtime with Pulumi:

import pulumi_aws as aws

# Assumes: role, ecr_repo already configured
agent_runtime = aws.bedrock.AgentcoreAgentRuntime("my-agent",
 agent_runtime_name="my-agent-runtime",
 role_arn=role.arn,
 agent_runtime_artifact={
 "container_configuration": {
 "container_uri": f"{ecr_repo.repository_url}:latest",
 },
 },
 network_configuration={
 "network_mode": "PUBLIC",
 })

Try it with Neo: Deploy an AgentCore runtime — Requires a container image in ECR.

How does this come together? AWS shipped a use case.

The proof of concept: Nova Act

The AWS AI stack: vertically integrated from silicon to agent runtime. Nova Act uses the full stack.

Nova Act is the concrete example of this whole thing coming together. It handles browser-based UI automation: form filling, search-and-extract, QA testing. Amazon claims ~90% reliability. It deploys directly to AgentCore Runtime.

It’s not “an LLM plus Playwright.” Nova Act uses a specialized Nova 2 Lite variant trained on synthetic “web gym” environments: browser simulations that mirror enterprise UIs and provide an automatic reward signal when tasks are completed correctly. Instead of judging output quality, this model was trained on an RL loop that asks: Did the workflow succeed?

That specialized model is wrapped in AgentCore. The platform handles isolation, scaling, logging, and guardrails, so Nova Act behaves like a production automation system rather than a brittle demo.

Seen this way, Nova Act is Amazon’s reference implementation for a certain class of enterprise agents: start with a strong general model, specialize it through domain-specific RL in a controlled environment, and run it on AgentCore with tools and policies around it. It’s the pattern AWS expects customers to adopt.

One stack to rule them all

So Nova Forge, Trainium, AgentCore, and Nova Act connect. Trainium lowers the cost of big training runs. Nova Forge lets enterprises plug their own data and rewards into those runs. AgentCore is where the resulting models act, with tools, memory, and policy guardrails. Nova Act shows the pattern in action: a domain-specialized Nova model, trained in a controlled loop, running as a production agent.

Most enterprises still won’t choose this path. They don’t have the data, the reward loops, or the operational maturity to make early-stage training worthwhile.

But AWS’s bet is that enterprise AI is moving past stock foundation models and generic chatbots. AWS is expecting a world of agents shaped by proprietary data and domain feedback. Most companies won’t build the infrastructure to train and operate those agents, and so AWS is offering to rent them the whole pipeline.

Try it with Neo: Deploy a Bedrock-powered API with Pulumi

CNBC reporting on Nova Forge pricing. Source ↩︎
Sam Altman stated GPT-4 cost “more than $100 million” to train. Source ↩︎

Platform Engineering: Evolution or Rebranding?

Adam Gordon Bell — Mon, 12 May 2025 00:00:00 +0000

2 Months ago, user deacon91, after years of working in the industry has declared DevOps to be a dead end on Reddit’s r/devops. And I’ve been thinking about his thoughts on the industry since then.

His point was that DevOps, the breaking down of silos between development and operations, had been tried. We learned the lessons of this approach and need to do something better: Platform Engineering.

And platform engineering has certainly emerged as a new hot area. But is it truly an evolutionary step toward a greater way of developing and operating software? Or is it, as others in that same Reddit group declare, merely a fad and a rebranding of existing ideas?

That’s what I want to explore: is platform engineering a rebrand or a genuine evolution?

Let’s start by summarizing the cynical case - that it’s merely a rebrand.

The Rebrand Case

Title Drift

This title drift happens in many fields. Just as the garbage man becomes the sanitation worker so does the sysadmin becomes the DevOps engineer.

This creates title inflation in our competitive industry. People who job-hop regularly update their LinkedIn profiles from “business analyst” to “product manager.” Meanwhile, the person who stays put for years might still be called a “programmer” while everyone around them becomes a “Staff-plus Senior Backend Engineer.”

That ops wizard who’s been at the same company for eight years? Still an “Operations Engineer.” Their friend who switches jobs every 18 months? Now a “Senior SRE” making 50% more money.

Title drift becomes a self-reinforcing cycle. When the recruiter discovers that the best DevOps candidates now call themselves “Platform Engineers,” your company’s job postings change overnight—even if no one plans to build an actual platform.

This is how you end up with people like u/hajimenogio92 doing the exact same sysadmin work they’ve always done, just with “Platform Engineer” on their LinkedIn profile.

This is one aspect of what people mean why they say “platform engineering is just a rebrand.” The other thing they sometimes mean is that it’s a fad everyone is chasing.

Hype Cycle Reboot

Tech movements decay in predictable ways. Remember Extreme Programming? It began as a revolution to empower developers. Then it morphed into Agile, and in some places, into the 12 steps, 7 competencies, and many processes of SAFe.

The Scaled Agile Framework® (SAFe®) is a set of organization and workflow patterns for implementing agile practices at enterprise scale.

Maybe SAFe makes sense for your work place, but it’s certainly a lot more complex than it’s root and this pattern is what skeptics see in platform engineering.

DevOps started at grassroots conferences with a simple message: “Let’s tear down the wall between developers and operations.” But once it peaked, everyone scrambled to slap “DevOps” on whatever they were already doing. Just like companies today adding “AI” to their products.

Eventually, the original movement gets so diluted that we need a fresh start. New terminology. New conferences. New keynote speakers recycling old ideas with updated slides. New vendors selling solutions to problems you didn’t know you had.

In this view, platform engineering isn’t revolutionary—it’s just the next square on the buzzword bingo card. DevOps plateaued. SRE plateaued. The Kubernetes experts ran out of things to say. So the industry needed something new to hype.

There’s some truth to this cynicism, but it blinds us to what platform engineering actually offers. More importantly, this view misses something crucial: where a platform approach succeeds, it reveals fundamental flaws in the original DevOps vision.

The Case for Evolution

The original DevOps movement introduced valuable ideas about breaking down silos and encouraging collaboration. However, its purest vision—where “everybody does everything”—faces practical limitations at scale.

Totally cross-functional teams work well at small organizations but struggle when engineering teams grow beyond 20 people because:

People naturally have different specializations and preferences
Few individuals want to perform both development and operations roles
Staying current on everything in your Ops and Dev stack is often unrealistic

But most importantly, as organizations grow, specializations natuarally emerge. Governance and corporate policies often force the issue. A platform team is one such specialization and a powerful one if done right.

Instead of focusing on eliminating silos, it focuses on creating productive abstractions and interfaces between application teams and operating services. The platform functions as an abstraction layer (similar to how a database abstracts data storage), with dedicated teams treating it as a product with clear boundaries.

This approach contradicts the original DevOps vision but represents a necessary evolution. Platform Engineering acknowledges inevitable organizational divisions while still enabling collaboration. It leverages skills and organizational workflows from application development and product management to build a way to operate software effectively.

Why not both?

So yeah, title drift is a thing. And the hype cycle is real. But the idea of a dedicated team removing frictions by building tools and reusable abstractions is an idea that’s here to stay.

In fact, it’s just a way to scale the principles inherent in the original ideas of DevOps. It may technically diverge from the literal principles of DevOps, but it is most definitely true to the spirit.

At Pulumi, we see this evolution firsthand. Our most forward-looking users aren’t chasing titles—they’re building internal platforms that empower teams, reduce friction, and let developers focus on what matters most. That’s why we’re investing in platform team features.

Not because it’s trendy—but because, done well, it works. And I think that’s the insight deacon91 was getting at.

Get a Free Platform Engineering Guide →

Your Perfect Infrastructure May Not Be So Perfect

Simen A. W. Olsen — Mon, 16 Dec 2024 10:43:07 +0000

Guest Article: Simen A. W. Olsen from Bjerk, is here to share his lessons learned on why designing the perfect architecture for your future needs might be a mistake

I remember standing in front of our engineering team in 2018, proudly presenting what I believed was the future-proof architectural design for our new distributed system. The diagrams were immaculate, the technology choices were cutting-edge, and the scalability patterns were ready for any possible future scenario.

I was basically the Leonardo da Vinci of system design… if Leonardo had been really into Kubernetes and had a concerning addiction to coffee. But six months later, that “future-proof” architecture had become a constraint rather than an enabler, and my masterpiece was looking more like a finger painting done by a caffeinated raccoon.

This experience taught me something crucial: trying to build the perfect system that anticipates every future need is often worse than creating a system designed to change quickly. It’s like trying to predict what your kid will want to be when they grow up and pre-buying all the necessary equipment. Congrats, you now own a space suit, a stethoscope, and a dragon costume — and they decided to become a software engineer anyway.

The Over-Planning

Many teams fall into a common trap: they try to design systems that anticipate every possible future requirement. This happens even in agile teams, where we convince ourselves we need to “get the architecture right” before we can start iterating. You know, because nothing says “agile” like spending three months in a room drawing boxes and arrows while muttering “microservices” under your breath like it’s a magic spell.

In 2008, Netflix faced a choice: build the perfect data center that could handle all their anticipated future needs, or move to the cloud with a simpler architecture that could evolve. They chose the latter, focusing on making their system easy to change rather than trying to make it perfect. Smart move — unlike those my past self made who probably would’ve insisted on building a data center capable of streaming to Mars, just in case Elon asked nicely.

You might also like:

The Core Principles of Change-Ready Architecture

Through both failures and successes, I’ve identified three principles that define truly adaptable architecture:

Embrace simplicity. I think it makes sense to start with the simplest architecture that could possibly work for your current needs. Complexity should be earned, not presumed. If your architecture diagram looks like a plate of spaghetti that’s been hit by lightning, you might be doing it wrong.
Make change cheap. Instead of trying to avoid change, make it inexpensive. This means investing in automated testing, continuous deployment, and monitoring. When change is cheap, you don’t need to fear it.
Learn through action. Rather than trying to predict the future, build mechanisms that help you learn quickly about real needs. It includes feature toggles (Protip: Try Unleash.), A/B testing, and robust monitoring of how your system is actually being used. You know, actual data, not just what that one loud guy in planning insists will definitely happen.

The rise of AI and machine learning systems has made one thing clear: we can’t predict how our systems will need to evolve. The most successful teams aren’t those that try to build the perfect AI architecture upfront, but those that can rapidly experiment and adapt their systems based on real-world feedback.

The biggest pushback I hear is, “But what if we need to scale?” or “What about future requirements?” These fears often drive teams to over-architect their solutions. But here’s the reality: the cost of changing a simple system is usually lower than the cost of maintaining an over-engineered one. The key is understanding that good architecture isn’t about predicting the future — it’s about making future changes as painless as possible.

Conclusion

That over-engineered system I was so proud of in 2018? Its most significant flaw wasn’t in what it got wrong about the future — it was that it tried too hard to be right about the future in the first place. It’s like bringing a fully packed suitcase to a first date. Today, I know that the best architecture isn’t one that anticipates every need, but one that makes it easy to respond to needs as they emerge.

The next time you’re tempted to design for every possible future scenario, remember: the goal isn’t to build a perfect system, but to build one that’s perfectly easy to change. And if someone tells you they’ve designed the perfect future-proof architecture, they’re either lying, or they’ve discovered time travel — and in that case, they should be sharing lottery numbers, not system designs.

You might also like:

105 Ways to Run Containers: The Cloud Container Iceberg

Adam Gordon Bell — Tue, 03 Dec 2024 10:29:54 -0500

If a computer can be connected to the internet, someone has tried to run a container on it. From quantum computers to smart toasters, from phones in AWS racks to CI pipelines, there’s many ways to deploy containers. While most people only know about the mainstream cloud providers at the surface, there’s actually a vast world of increasingly unusual and specialized options beneath.

This guide serves two purposes: to showcase the surprising breadth of container options available today, and to help you understand the full spectrum of choices – from practical to very experimental. Whether you’re looking for production-ready solutions or just curious about what’s out there in the dark depths, you’ll find something interesting here.

Constraints

But first, we need ground rules for our container adventures. All listed items should:

Technically possible: Must work today with existing tech, even if the setup makes you question your life choices.
Runs real containers: OCI containers only. No JavaScript “containers” pretending to be the real thing.
Actually doable: You can set this up right now with a laptop and a credit card. No theoretical schemes allowed.

Above the iceberg – The usual suspects

These are the familiar faces of container orchestration—the ones you’d introduce to your parents. Reliable, well-documented, and about as exciting as watching paint dry.

Amazon Elastic Kubernetes Service (EKS)
Amazon Elastic Container Service (ECS)
AWS Fargate
Azure Kubernetes Service (AKS)
Google Kubernetes Engine (GKE)
Google Cloud Run
Amazon EC2
AWS App Runner
Azure App Service for Containers
Google Compute Engine with Containers

On the iceberg – The slightly less obvious

As we dip just below the surface, we find services like a rare B-side acoustic release from a big band. It’s not as mainstream but still a respectable choice for running your containers.

AWS Elastic Beanstalk – Deploy containers, debug YAML
Amazon Lightsail Containers – AWS with the guard rails on
Azure Container Apps – Serverless, but complicated
Google App Engine Flexible Environment – PaaS for Containerized Apps
Azure Container Instances (ACI) – Kubernetes without the chaos

The kubernetes multiverse – Because one Universe of k8s wasn’t enough

Diving deeper, we discover there’s an entire multiverse of regional and specialty K8s providers, helpfully cataloged by the CNCF. Think parallel universes, but instead of evil twins, you get slightly different container runtime implementations.

The Global Titans – The international tech giants couldn’t resist joining the K8s party: Huawei Cloud Container Engine, Tencent, and Baidu bring their own flavor of cloud-native orchestration to the mix. Add in Samsung, China Mobile KCS, JD Cloud, Inspur, and QingCloud, and you have enough options to start a diplomatic incident.
The European Union of Containers – Europeans take their container privacy seriously: IONOS and plusserver bring German engineering to K8s, OVH adds French flair, Exoscale offers Swiss precision, and STACKIT, SysEleven, WIIT, Gcore, and Previder ensure your containers comply with every possible EU regulation. Conoa Proact throws in some Swedish design principles for good measure.
The Asia-Pacific Pod Party – The APAC region isn’t missing out: NHN brings Korean optimization to your clusters, while ARDC Nectar and Catalyst let you run containers upside down in Australia and New Zealand. TKS Cloud Service ensures your containers can run anywhere from Tokyo to Tasmania.
The Specialists’ Society – For when regular K8s isn’t quirky enough: OKESTRO VIOLA orchestrates your containers with musical flair, while Petasus, De Novo, Aruba, and Taikun provide enough specialty K8s options to ensure every cluster is a unique snowflake.

Below the water – Kubernetes everywhere – Let a thousand K8s bloom

Let’s plunge deeper. Because one Kubernetes cluster is never enough, why not sprinkle them everywhere? On-premises, in the cloud, under your desk—there’s a K8s solution for all that.

Amazon EKS Anywhere – AWS container but in your own data center
Amazon ECS Anywhere – Like EKS Anywhere but ECS
Azure Arc – Hybrid cloud for masochists
Google Anthos – Hybrid cloud something something
AWS Outposts – AWS, now with hardware shipping delays
Azure Red Hat OpenShift – Azure but with a fedora
VMware Tanzu – Because VMware heard you like Kubernetes
Red Hat OpenShift – Fedoras on AWS
IBM Cloud Kubernetes Service – Kubernetes with a touch of enterprise nostalgia
Oracle Container Engine for Kubernetes (OKE) – For the brave souls trusting Oracle
Alibaba Cloud Elastic Container Service (ACK) – K8s with Chinese characteristics
DigitalOcean Kubernetes (DOKS) – K8s in the ocean, hopefully no sharks
Linode Kubernetes Engine (LKE) – Lightweight K8s for the minimalist
Vultr Kubernetes Engine (VKE) – Yet another place to run your pods
Scaleway Kapsule – French elegance meets Kubernetes
Hetzner Cloud Managed Kubernetes – German engineering for your clusters
Nutanix Karbon - Because regular Kubernetes wasn’t converged enough
Metal Stack Cloud Kubernetes - For when virtual machines are too virtual

You might also like:

Middle of the iceberg – Serverless and function containers – Now you see me, now you don’t

As we descend to the depths, we find containers with commitment issues. They appear, do their job, and disappear.

AWS Batch – Containers that only work the night shift
AWS Lambda – Lambda’s containers for when a 250 MB zip file wasn’t enough baggage.
Azure Functions – Roll your own functions in containers and hope for the best.
Google Cloud Functions – Containers pretending to be functions, but now on Google Cloud.
Alibaba Function Compute – You get the idea.

Bottom of the iceberg – Specialized and hybrid platforms – The swiss army knife of container hosting

Down here, containers run in ways that make people question your life choices. These platforms let you host containers in environments many fear to tread.

AWS Proton – Automate your deployments and run a container
Azure Virtual Machines with Docker – When you miss managing servers but love containers
IBM Cloud Code Engine – Run anything, anywhere, if you can figure it out
Alibaba Elastic Container Instance (ECI) – Containers that disappear faster than you can say “Jack Ma”

Below the iceberg – Containers on the edge of reason

In these lightless depths, you want your containers to live life on the edge—literally. Because nothing says “enterprise-ready” like running production workloads on a smart toaster.

AWS IoT Greengrass – Run AWS Lambda and Docker containers on everything from factory equipment to smart fridges. Nothing says “digital transformation” like your smart fridge running machine learning models.
Azure IoT Edge – Like Greengrass, but only when your toaster prefers Windows updates.
AWS Snowball Edge – For when ‘cloud migration’ means paying FedEx to move your containers. Yes, it’s literally a box they mail you that can run containers.
AWS Snowcone – An even smaller box that runs just enough containers to make you appreciate your data center. Perfect for when you need AWS in places where AWS doesn’t want to build a region.
BalenaCloud IoT Device Management – A platform that lets you remotely deploy and manage containers on fleets of Raspberry Pis. Because nothing says “enterprise IoT” like running your production stack on $35 computers.
Fly.io – Global Edge Container Deployment, because apparently just picking one region was too simple.
Edge Node Service (Tencent Cloud) – Edge deployment behind great firewall.
StackPath Edge Computing – More edge containers.
Koyeb – Serverless Edge containers.
Platform.sh – Multi-Cloud Edge containers.
StarlingX - Edge computing platform for when 5G and IoT buzzwords aren’t enough
ZStack Edge - Edge computing with Chinese characteristics, now at the edge of the network, too

The abyss – Where containers fear to tread

And now we reach the darkest depths. For the bravest souls who look at conventional container platforms and think, “Not edgy enough.” Here lie the most unusual ways to run containers in production.

Amazon GameLift – Run your containerized services disguised as game servers. High scores may apply.
AWS Device Farm – Deploy containers on hundreds of actual phones in AWS racks. Perfect for when “mobile-first” goes too far.
AWS AppStream 2.0 – Stream Windows to run Docker to stream containers. It’s virtualization all the way down.
Google Cloud Dataflow – Hide containerized services in pipeline steps named “daily_data_normalize_37”. Hope your SRE enjoys debugging ETL.
Google Cloud Composer – Your containerized microservice masquerades as an Airflow DAG. Because nothing says reliability like cron.
Google Colab & Jupyter Notebooks – Run containers in data_preprocessing_step.ipynb. Watch ML teams debug production.
Azure Quantum – Package quantum containers with az quantum job submit. Now, your uptime is truly uncertain.
Azure Cognitive Services in Containers – Offline AI containers that promise not to become self-aware. Terms and conditions may apply.
Azure DevTest Labs – Run containerized test environments that self-destruct like Mission Impossible tapes. Perfect for when temporary becomes permanent.

Beyond the abyss

There you have it, the container iceberg. And I didn’t even have room to add the extra cursed ways you can run containers. Here is some of what didn’t make it:

Cloud IDEs and Workspaces – Want to turn your development environment into a sneaky container platform? Take your pick: GitHub Codespaces, GitPod for the GitLab enthusiasts, or Eclipse Che for the masochists who miss Eclipse crashing locally. JetBrains Space or Replit? All this lets you run a container until you close your browser tab.
GitHub Actions et al. – Ever wanted to run a production service disguised as a CI job? Just label your production API start-up script as “integration_test_step_42”, add a sleep 21600 and a call to retrigger itself, and hope nobody notices your “test” has been running for months. Apply this same trick to AWS CodeBuild, GitLab CI/CD, CircleCI, Azure Pipelines, Travis CI, Bitbucket Pipelines, Drone CI, Jenkins Cloud, TeamCity Cloud, and Google Cloud Build for Bonus points.

Managing your container zoo with pulumi

Whether you’re deploying containers to mainstream cloud providers or attempting to run them on quantum computers (really, please let us know how that goes), Pulumi helps you manage it all with real code instead of config file soup. Our Kubernetes support means you can wrangle your containers wherever they might roam – from AWS to Azure, GCP to Kubernetes, or even that smart toaster you’re eyeing for production deployment.

With Pulumi, you get:

Cloud-native container orchestration across all major platforms and Kubernetes distributions
Infrastructure testing to ensure your containers deploy correctly (even in the weird places)
Provider templates for when you invent yet another way to run containers
Stack references for managing multiple environments without losing your mind
Automation API for programmatically herding your container fleets

Ready to bring some sanity to your container deployment strategy? Get started with Pulumi:

Try Pulumi for Free

The final word

At this point, 105 options in, you might wonder: “Has cloud container hosting gone too far?” Well, I’ve a solution to propose:

At Pulumi, we’ve got providers for all the usual suspects and, surprisingly, many more in this container circus. And if you’ve figured out how to run containers on a quantum computer inside a data pipeline in New Zealand, you can write a provider for that, too.

So, use Pulumi and we will figure out how RedHat-Tanzu-Greengrass-Icecore-Device-Farm works and provide you a sane interface to it. You’ve got other things to worry about.

Fargate vs EC2

Adam Gordon Bell — Mon, 18 Nov 2024 08:42:34 -0500

Building an EKS cluster requires choosing how your containers will actually run - either on EC2 instances you manage or through AWS Fargate’s pod-by-pod approach. The differences can be pretty dramatic in practice. I’m setting up a demo cluster right now using Pulumi, so let me show you what I mean.

Here is my Fargate cluster:

import pulumi
import pulumi_awsx as awsx
import pulumi_eks as eks
import pulumi_kubernetes as k8s

# Create VPC with default CIDR
eks_vpc = awsx.ec2.Vpc("eks-vpc",
 enable_dns_hostnames=True)

# Create minimal EKS cluster
eks_cluster = eks.Cluster("eks-cluster",
 vpc_id=eks_vpc.vpc_id,
 private_subnet_ids=eks_vpc.private_subnet_ids,
 fargate=True,
)

pulumi.export("kubeconfig", eks_cluster.kubeconfig)

And when I look in k9s I see this:

You can see that each pod is on its own node. Even the core-dns pods in the system namespace are each on their own nodes. That is a lot of nodes, and it’s going to keep growing with my cluster.

If I change my cluster to be backed by EC2 nodes, I get something different.

eks_cluster = eks.Cluster("eks-cluster",
 vpc_id=eks_vpc.vpc_id,
 private_subnet_ids=eks_vpc.private_subnet_ids,
- fargate=True,
+ instance_type="t3.medium",
+ desired_capacity=2,
+ min_size=1,
+ max_size=2,
)

Now, there are only two nodes, with the pods distributed among them. Those are my two t3.mediums.

So when should you use Fargate, and when EC2 for EKS? Well, it really all relates to what we are seeing here with the number and size of nodes.

Let me explain.

Bin Packing

Normally, the Kubernetes scheduler has to solve a bin-packing problem—fitting M pods across N nodes based on available resources. This is a straight-up resource optimization problem of balancing CPU, memory, and other resource demands across the cluster.

This can be surprisingly complex in practice. Teams must balance selecting the right instance types for different workload needs while managing multiple node groups. Adding to this complexity are scheduling rules and resource quotas that need careful configuration. Perhaps most challenging is striking the right balance between high resource utilization and maintaining enough headroom for spikes in demand.

With Fargate, AWS sidesteps this challenge by providing a correctly sized bin for each pod. Each Fargate pod runs on its own dynamically provisioned, right-sized mini-environment, where the “bin” (the Fargate instance) is sized to match the pod’s requested resources. This means there’s no need for Kubernetes to optimize resource usage across a pool of shared nodes, as each pod effectively has its own “container” provided by Fargate that fits it’s needs precisely.

In other words, AWS effectively pushes this bin-packing responsibility to itself, so you don’t have to worry ( as much ) about it.

While Fargate abstracts away node management, it’s important to understand that Fargate pods still run on EC2 instances behind the scenes. And each Fargate pod gets its own ENI (Elastic Network Interface) that can sometimes limit your scaling because pods aren’t sharing a network namespace.

Pros and Cons

So yeah, Fargate abstracts away the complexity of managing nodes and scaling clusters by shifting bin-packing to AWS. With its extensive infrastructure, AWS is arguably better positioned to optimize placements at scale. They can monitor their entire cloud landscape, placing Fargate instances dynamically across available resources. This allows them to potentially leverage underutilized resources in ways that would be challenging or impractical at an individual account’s scale.

Responsibility	EC2	Fargate
Instance Types	You pick size and provision nodes	AWS provisions per pod request
Pod Isolation	You prevent noisy neighbors	AWS isolates each pod
Resource Allocation	You find space on nodes	AWS creates dedicated space

Customer vs AWS Responsibility EC2 vs Fargate

Ideally, this frees you to focus on the services rather than on the underlying infrastructure, but in practice, it’s a bit more complex.

Since each pod is treated as a standalone instance, you lose out on the cost efficiencies of co-locating multiple pods on a single node. Even though AWS is “solving” the bin-packing problem globally, to do so, they need to introduce hard barriers, and while noisy neighbors, therefore, aren’t a problem, sharing resources is now impossible. In a way, you are throwing away the run time benefit of containers, the shared kernel virtualization, for something more like VMs. And just like VMs, Fargate pods take longer to start than pods on existing EC2 nodes (typically 30-60 seconds vs near-immediate starts).

A Note on Scaling

Both EC2 and Fargate work with Kubernetes’ Horizontal Pod Autoscaling (HPA), which adds or removes pods based on metrics like CPU usage. The difference? With EC2, you need enough nodes available to handle new pods. With Fargate, AWS automatically provisions resources for each new pod - though you’ll face that 30-60 second startup delay and potential VPC limits since each pod needs its own network interface.

Feature	Fargate	EC2
Cost	Higher per pod	Lower cost per pod
Scalability	Scales on-demand, isolated resources	Requires manual node scaling
Resource Sharing	Not possible	Efficient resource sharing among pods
Startup Time	30-60 seconds	Near instant on existing nodes
Use Cases	Burst workloads, isolated pods	Cost-sensitive workloads, Resource sharing workloads

Maybe this will make more sense with an example.

Workload Example: Static Analysis

Have you ever had to run CPU-intensive batch jobs alongside lighter web services?

In a previous role, when my team and I were tasked with putting our static container image analyzer into production, we went the typical Kubernetes route, throwing it into an EC2-backed K8s cluster. The analyzer was built in Scala, running on the JVM, and was designed to take in work from a Kafka topic. Each analysis ultimately led to the results being sent to a database. We also had scheduled daily rescans of existing images whenever vulnerability lists were updated.

Then, a web app connected to this database and showed all your results. It was stateless and did far less work per request, mainly just answering an HTTP request by talking to a database.

The hard part though is that each analysis job involved uncompressing and analyzing Docker layers from S3 and could use a lot of CPU. Memory needs were around 2 GB per instance, which kept the JVM happy, but long sustained bursts in CPU usage were common, especially during intensive parts of the analysis.

In our K8s setup, the biggest pain point was handling this analysis workload’s burstiness and keeping a bunch of stateless web apps up and answering requests. We ran multiple instances of the static analyzer. Still, the analyzer was the worst type of noisy neighbor to other services in the cluster, so we’d end up over-provisioning to ensure we had enough headroom, which led to wasted resources and extra cost. The setup worked, but was not great.

Thinking back, I can see how Fargate could’ve streamlined things here. With Fargate, each static analyzer instance would run in its own isolated environment, so there’d be no resource contention or need for pre-provisioned node capacity. Each pod would be fully provisioned with the exact CPU and memory needed, and we could do some sort of on-demand scaling to deal with backlogs of analysis work in the Kafka topic.

Fargate on EKS could be a great solution for this type of chunky, resource-heavy macro-services. There are probably other ways to handle this situation, and we worked on decomposing the analysis into smaller chunks, but Fargate might have been a good solution.

But there are lots of situations where Fargate is less of a fit.

You might also like:

Example: Go Services for E-commerce

Ok, another workload that comes to mind is this one from a friend - He worked on an e-commerce backend with many lightweight Go services, each handling specific tasks—user profiles, catalog browsing, order processing, and notifications. These services were network-heavy, handling gRPC calls and database requests, but they didn’t need much CPU or memory per instance.

Fargate wouldn’t have made sense here—it would have isolated each pod, preventing resource sharing and driving up costs without real benefit. For lightweight services like these, EC2-backed Kubernetes is perfect, keeping costs low and maximizing resources by sharing nodes across small, efficient services.

So, for lightweight micro-services that can share resources, EC2 nodes with many pods each are probably the way to go.

Other Fargate Cases

Fargate can excel with long-running event-driven and bursty workloads, where you need compute resources briefly or unpredictably. Think file processing jobs that trigger on uploads, or batch processing that runs sporadically. When jobs are intermittent, Fargate’s ability to scale from zero and its isolated resources can outweigh its higher per-pod cost and startup delay. Just ensure your workload can tolerate the 30-60 second cold start.

Fargate has some hard limitations that might rule it out for your use case:

No DaemonSets
Maximum of 4 vCPU and 30GB memory per pod
Storage limitations: Only ephemeral storage up to 20GB per pod, no persistent volumes
No GPU support

Fargate vs EC2 Pricing

Ok, EC2 costs less than Fargate on a pure cost-of-compute basis. If my t3.medium ($0.0416/hr) can typically run 6-8 pods reliably, and this works out to under $0.01/hr per pod. Comparable Fargate pods (0.5 vCPU/1GB) cost about $0.025/hr each, so 2.5x.

These calculations can change based on how much headroom you need on EC2 and how many containers you can run on a given EC2 instance. Still, Fargate is just more expensive unless all your containers fit the exact computing sizes offered by Fargate. And don’t forget, AWS is likely incurring costs maintaining the warm compute capacity for launching Fargate pods, so it shouldn’t be surprising that Fargate is more expensive.

But really, pure compute cost calculations are an insufficient metric for making this decision - you need to factor in your team’s operational capacity, scaling patterns, and isolation requirements.

( Also, if Fargate makes sense because of your workload but the cost is a concern, then look into Fargate Spot instances, which can be up to 70% cheaper, but be aware that AWS can terminate them at any point to reclaim capacity. )

Misconceptions About Fargate

Fargate saves you from managing servers With containerized workloads and immutable infrastructure, your EC2 nodes are minimal, standardized, and automatically replaced. You’re not logging in, installing packages, or managing configuration - everything runs in containers. When a node needs updates, you replace it entirely.
Fargate handles all scaling challenges While it handles node scaling, you still face ENI limits, longer pod startup times (30-60s), and potential networking bottlenecks.
It’s simpler and more reliable It may be, but it may not be. The complexity shifts to networking, pod configuration, and working within Fargate’s constraints. You trade node management for new operational challenges like slower startups and possible networking complexity.

Managing Container Orchestration with Pulumi

Whether you choose EC2 or Fargate, Pulumi makes it easy to manage in your favorite programming languages. With its rich AWS support and infrastructure as code capabilities, you can define, deploy, and manage your entire Kubernetes infrastructure using familiar tools and practices. Pulumi’s AWS provider helps you implement the right solution for your needs.

With Pulumi, you get:

Complete control over EKS cluster configuration, including node types, scaling policies, and networking setup
Built-in support for both EC2 node groups and Fargate profiles within the same cluster
Infrastructure testing capabilities to validate your cluster configuration before deployment
Automation APIs for programmatic infrastructure management and custom tooling

Ready to streamline your EKS infrastructure management?

Try Pulumi for Free

Why Not Both

I hope my examples point to a pragmatic approach: use EC2 as your foundation for efficient microservices that can share resources, like our Go e-commerce setup. Add Fargate when you need isolation or flexible scaling - like for the static analysis workload. Better yet, combine both approaches in the same cluster, for cost-efficiency and isolation where needed.

If you’re interested in using Pulumi to manage EKS infra, here’s a practical EKS setup guide guide to get your started.

Now, back to getting this demo cluster up and working!

The Present and (Near) Future of AI and Infrastructure as Code

Luke Hoban — Wed, 14 Feb 2024 07:00:00 -0800

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

AI is impacting almost every industry today, and for good reason - we are seeing fundamentally new experiences being made possible across a wide variety of products, and a set of new AI capabilities that promise even more incredible change in the near future.

Software development is among the earliest and most prominent fields to realize the benefits of AI, evidenced by the rapid adoption of tools like Github Copilot which is now one of the most heavily adopted developer tools of all time. Developers are benefiting from an incredible increase in their productivity with better scale and faster time to market.

We’re seeing the impacts of AI in the cloud Infrastructure development space in two impactful and complimentary directions:

🤖➜☁️: AI is transforming how we author, build and manage cloud infrastructure
☁️➜🤖: Cloud infrastructure tooling is changing how we build and deliver AI-based applications

At Pulumi, we’ve already seen profound impacts from AI in both of these directions.

Infrastructure engineers are using Pulumi AI to discover solutions for cloud infrastructure needs, leading to a dynamic experience where you can generate solutions in seconds with a few simple prompts, and using tools like GitHub Copilot to quickly iterate on and test their infrastructure.

At the same time, thousands of organizations are using Pulumi and Pulumi Cloud to manage the fundamentally cloud-native infrastructure and services available to build and scale AI-based applications - managed vector databases, elastic compute services, model serving, streaming API management services, and workflow management - all evolving at the rapid pace of the AI space, and scaling with the demand that organizations are seeing for AI.

In this article, we’ll dig into both of these trends and the impacts that they are already having today, as well as the trends we are expecting to see (and build) for the near future.

AI transforming Infrastructure as Code

Today

The scale of the cloud is truly incredible, and the sheer size of the AWS API is huge relative to almost any other computing platform, while AWS is just one of over 135 cloud providers for which Pulumi supports building and managing infrastructure. The result is more than 50,000 infrastructure resource types that can be managed, and over 1,000,000 configurable input properties on those resources. In practice, cloud engineers face a daunting task as they figure out how to combine these resources and properties to solve their unique problems. Finding an example or guidance for how to combine these together, given the enormous variety of possible ways, can be hard or impossible.

As a result, one of the most fundamental challenges in enabling developers to use the cloud is helping them discover how to solve their problems using the enormous collection of available cloud infrastructure building blocks. For example, two years ago, I tried looking up how to “deploy Metabase on Azure using managed PostreSQL and containers” to solve a problem I was working on. I couldn’t find any content on the internet that directly answered this. Not just for Pulumi, there was no answer using any other IaC or deployment technology.

AI is perfectly suited for this challenge. With Pulumi AI, I can ask a question and get back an answer to the exact things I’m looking for - even if that specific use case has never been answered before. Pulumi AI doesn’t always get it completely right on the first try, but it typically gets very close, and points me in the direction I’m looking for. When I try its suggestion and get errors, I can ask it to fix them to iterate to a working solution.

The adoption of Pulumi AI since its release less than a year ago has been incredible, with tens of thousands of users submitting hundreds of thousands of prompts. And, perhaps surprisingly, usage hasn’t been driven only by or primarily by new Pulumi users. We’ve seen many of the most advanced Pulumi users with the deepest cloud knowledge leverage Pulumi AI the most heavily, helping them accelerate the development of new infrastructure solutions and discover more effective approaches to solving their infrastructure needs using the best of their cloud platforms.

On top of providing individual users a way to answer their unique questions, it is also helpful to get answers for the most common questions available on the internet, avoiding the need to generate new answers when solutions are needed. With Pulumi AI Answers, we’ve made answers to many of the most asked questions about cloud infrastructure available directly on the internet, making it even easier to find solutions to cloud infrastructure problems and IaC code to solve for them.

As Infrastructure as Code tooling and cloud development generally embraces traditional programming languages like in Pulumi or CDK, we see the breadth of amazing tools and tooling ecosystems available in the Node.js, Python, Go, .NET and Java ecosystems all come to bear to support scalable cloud infrastructure management. AI-based tooling is no exception, with tools like GitHub Copilot, Amazon CodeWhisperer and SourceGraph Cody changing how developers write, iterate on and test their code, including their infrastructure code in popular languages that have huge training data sets. These tools are simply incredible at understanding and working with these popular languages, and modern IaC users are benefiting from these tools for productivity and discoverability.

Of course, it’s not enough just to have the code you need; you actually have to deploy that code to create and manage your cloud infrastructure. This means getting the code into source control, setting up continuous delivery, and connecting to the configuration and secrets needed for the environment you are deploying to. Recently, we added “Deploy with Pulumi” support to each Pulumi AI answer, along with pulumi new support in the Pulumi CLI, enabling you can go from a few simple text prompts to a working solution fully integrated with source control, CI/CD and Pulumi ESC in less than 30 seconds.

Near Future

Even with the rapid progress over the last year, we’re clearly still just getting started in realizing the impact of AI on cloud infrastructure management.

Tightening the feedback loop for tools like Pulumi AI will continue to enable AI to iterate and solve more complex cloud infrastructure challenges, and arrive at correct working solutions. And instead of asking questions and getting back code, these tools will soon be able to actually do the deployments and verify that they work. The result is a natural language interface, which can build and evolve cloud infrastructure. Even as the interface becomes natural language, Infrastructure as Code is a uniquely good “code-behind” for this because it maintains a desired state configuration for the infrastructure, which AI tools can easily iterate on and manage. The incredible pace of improvement in AI model skill and performance for writing code in particular will accrue directly into improvements here.


An early prototype of interactive AI infrastructure development

AI-based tooling will also extend well beyond just creating and managing infrastructure, most importantly to the operational tooling used to understand, support, and diagnose that infrastructure, including policy enforcement across cost, compliance, and security. Giving AI-based tools and agents the ability to access things like cloud resource search, the ability to run operational CLI commands inside target environments, policy definition, and the opportunity to trigger deployment workflows will empower these tools to support and ultimately automated broad classes of cloud infrastructure management tasks.

Infrastructure as Code transforming AI

Today

One of the biggest imperatives in almost every development organization today is to identify and invest in ways to leverage AI to improve software-based products, tools and processes. This has led, in turn, to massive growth in the need for cloud infrastructure to support and enable building and delivering these applications.

One of the most striking things about modern AI-based applications is that they are significantly more cloud-native than previous generations of application architectures. This is true for several reasons:

Modern AI technology and services have been built around cloud-native assumptions
AI-based applications frequently have especially bursty compute needs
AI applications and the space overall is scaling rapidly, requiring elasticity and low operational overhead
New managed services, like vector datastores, managed model serving APIs, managed data and training pipelines

This cloud-native quality of AI-based applications makes Infrastructure as Code even more important and valuable for AI applications than in other, more traditional applications.

The services to support building AI applications have all been built over the last two years around the assumption of the cloud. GPUs are dominantly accessed through cloud services. LLMs are invoked via managed services from OpenAI or via managed hosting of open-source models via SageMaker, Huggingface, or others. Retrieval Augmented Generation (RAG) is powered by dozens of managed database offerings, from dedicated vector databases like Pinecone, to vector extensions in Amazon Aurora PostgreSQL and more. Developers coming to AI assume they will stitch these services into rich application offerings.

At Pulumi, we’ve seen dozens of AI-focused companies adopting and using Pulumi to manage their infrastructure, motivated by a combination of these AI-oriented infrastructure needs, along with the ease of using Python (already heavily used within the AI space), and TypeScript (popular due to a wide variety of AI-oriented front end frameworks). Pulumi also exposes the ability to manage many of these services directly, provisioning and managing the lifecycle of AI services across your infrastructure through support for things like Azure OpenAI Service, Pinecone Indexes, Amazon SageMaker Endpoints, and much more.

As these applications move from prototypes to production, the need for robust and scalable infrastructure to deploy these applications in the major cloud platforms is essential, and IaC offers the best way to manage this at scale. For example, deploying a LangChain application in production, or deploying the production-grade infrastructure for deploying AI applications leveraging Pinecone’s vector database at scale. Both of these highlight ready-to-deploy cloud architectures for running an AI application in a production environment.

The cloud-native foundations of the modern AI application wave also lead to a lot of adoption and usage of Kubernetes as an orchestration platform. Infrastructure as code with Pulumi helps teams manage the Kubernetes cluster, the workloads, and the external AI service dependencies all in a consistent way, using a unified programming model and tools, and ensuring consistent delivery of changes, the ability to easily spin up test and development environments, and consistent application of policy and insights.

One particular trend in the rapid growth of IaC is in fast-growing companies building the new managed AI infrastructure services themselves - from data services (e.g. Pinecone) to compute (e.g. Modal). These products face the need to manage complex cloud infrastructure as part of both multi-tenant and single-tenant solutions in order to offer simple abstracted data and compute services for their AI-application developer customers. Automation of infrastructure provisioning and changes via tools like Pulumi Automation API and Pulumi Deployments has seen rapid adoption and scaling among these customers. The ability to deploy across many cloud providers to offer low latency data and compute services to customers, data sovereignty, and access scarce GPU and accelerated computing resources across all cloud providers with a single consistent programming model and deployment strategy proves more important than ever.

Near Future

The growth in the development of AI-based applications is still in its early days, and all the trends we’re seeing will only accelerate over the coming year.

As these AI applications transition from prototypes to production deployment, their infrastructure needs are growing and maturing. Simple prototyping environments like Jupyter notebooks or Vercel deployments will transition into AWS/Azure/Google Cloud for the security, cost, compliance and management benefits. Larger development teams and faster customer-facing iteration will require standing up testing and ephemeral development environments, and some of the managed service dependencies will be brought in-house into self-managed open source model-service, training infrastructure and data pipelines.

As patterns continue to solidify around common architectures, higher-level infrastructure components will abstract away some of this production infrastructure complexity, initially within organizations, and then in public components shared in the Pulumi Registry and elsewhere.

And of course, the set of managed cloud services to support the AI ecosystem will continue to grow, expand and mature. Whether delivered as part of a major cloud platform or via a fast-growing startup, it will all be available to use productively at scale via infrastructure as code in Pulumi.

Conclusion

At Pulumi we often say that All Software is Cloud Software, and that All Developers are Cloud Developers. This has been truer than ever as AI has risen to the forefront of every organization’s software development priorities over the last 15 months. AI is transforming every aspect of software development, including cloud infrastructure development. Cloud infrastructure supports, enables, and accelerates how AI applications are built and scaled in production. We’ve already seen incredible impacts of both of these trends, yet there is so much more ahead on these arcs.

We’re hard at work on all of these at Pulumi, and excited to see all of the innovation across the industry to build the future of AI-enabled cloud infrastructure management. Check out Pulumi AI and the AI with Pulumi page to get started experiencing the future of AI for Infrastructure as Code today!

Infrastructure as Code in Any Programming Language

Joe Duffy — Tue, 31 Oct 2023 22:36:29 +0000

This is the first of a three-part series originally published on The New Stack.

Infrastructure as Code is a technology for automating the infrastructure for your cloud applications. If you’re an engineer, whether that’s developing a backend service or within a central platform team, it’s not just about writing application code. You’ll need to provision, update and perform other tasks associated with its supporting infrastructure, and that’s where Infrastructure as Code can help. Instead of manually pointing-and-clicking in the cloud console, which is unrepeatable and error-prone, or writing ad-hoc scripts, which can be tedious and hard to scale, Infrastructure as Code lets us, as engineers, use familiar techniques by just writing code.

Not every engineer has a deep infrastructure background and yet needs to get more hands-on with infrastructure these days, which is OK: This three-part series was written from an engineer’s point of view. In it, we will demystify Infrastructure as Code — the why, what, and how — through the lens of Pulumi, a popular Infrastructure as Code tool among engineers.

Why We Need Infrastructure as Code

Modern applications need cloud infrastructure to run. That is equally true for simple monolithic applications running on virtual machines as it is for exotic distributed serverless applications that are fully elastic in scale. The applications themselves need infrastructure that they directly use, like whatever they run within (such as virtual machine, containerized service, serverless function, static website) in addition to any other resources they consume (databases, pub/sub topics, queues, AI/ML services, observability metrics and dashboards), but also depend on more primitive infrastructure to run atop (Kubernetes clusters, security roles and permissions, private networks, load balancers, encryption keys and more).

The phrase “cloud infrastructure” is also broader than it may seem. This phrase evokes immediate thoughts of popular clouds like Amazon Web Services (AWS) , Microsoft Azure, and Google Cloud, as well as more specialized or regional ones like Alibaba Cloud, DigitalOcean and Oracle Cloud. It also quickly leads to cloud native infrastructure like Docker, Kubernetes and Helm. But this phrase also perhaps not obviously applies to modern software-as-a-service (SaaS) infrastructure companies who increasingly are supplying critical pieces of cloud infrastructure, including Confluent, Cloudflare, Databricks, DataDog, Elastic, MongoDB, New Relic and Snowflake.

These companies are essentially specialized clouds providing more specialized services but are increasingly expanding to become clouds of their own. It also applies to private cloud technologies like F5, VMware vSphere and related technologies. Finally, also not obviously, there are SaaS tools that have configurable state that we use every day and may want repeatable management of just like our other cloud assets, including Auth0, GitHub, GitLab and PagerDuty.

Cloud infrastructure’s reach is far and wide! But it also means that there are many complex moving pieces to manage and tame. And with so much innovation happening in cloud capabilities, that complexity is just growing with time. This begs questions like the following:

Where does the infrastructure come from? How do we change it as our requirements evolve? How do we scale it as our needs grow, whether that’s increasing the compute and memory available to our workloads, scaling to many new instances, increasing our availability and reducing latency by deploying to new regions and environments worldwide… or, as is usually the case, a combination of all of these?

How do we ensure our infrastructure practices are repeatable in the event something fails or a mistake is made? How do we capture and reuse best practices? Do the answers to these questions differ across clouds? How do we ensure collaboration can take place safely and our deployments aren’t flaky and prone to colliding? And how do we secure all of it and ensure best practices and policies are enforced at all times?

These are all things that Infrastructure as Code solves, and it starts with code.

Benefits of Infrastructure as Code

The Infrastructure as Code approach provides many benefits, but they fall into two primary categories:

Using code to declare infrastructure
Using a declarative engine to orchestrate infrastructure changes

It is the combination of these two things that leads to the magic of “Infrastructure as Code.”

The Benefits of Code

Encoding your cloud application infrastructure in code results in a durable artifact representing your desired architecture. This can be code-reviewed, committed to source control and versioned in the usual ways. Infrastructure as Code tools not only know how to stand up the initial version of your infrastructure, but can replicate it across many environments (like dev, staging, prod and multiple regions), in addition to upgrading individual environments as your requirements evolve.

In terms of the “code” aspect, the expression of your code varies across the Infrastructure as Code landscape. Some tools support markup languages like JSON or YAML, while others support domain-specific languages (DSLs) that are specific to that tool. In this article, we will use Pulumi, which takes a unique approach to Infrastructure as Code that is well-suited for engineers: namely you use industry-standard general-purpose languages, including C#, Go, Java, JavaScript, Python or TypeScript, to express your code. This approach is great for engineers because it makes infrastructure more accessible and lets you use standard engineering tools and practices that you’re already using to build other software in your team.

Examples of these benefits include having rich constructs like simple if-statements and for-loops, which help avoid repetition and model complex infrastructure needs. Since all of these languages are broadly supported throughout the industry, virtually any editor you pick up will have great support, such as Visual Studio Code, PyCharm, Sublime Text, IntelliJ or even vim or emacs. That means you’ll get interactive statement completion, red squiggles if you make a typo or have a type-checking error, documentation as you hover, right-click to go to definition or refactor, and so much more. It’s easy to take these things for granted, but they are essential for software engineering productivity.

There are other benefits still such as linters, testing tools, the ability to share and reuse with package managers rather than copy-and-pasting and more. Lastly, each of these languages has enormous communities that add up to more than 20 million engineers, which means there’s a wealth of knowledge and support available.

It turns out Pulumi supports YAML too — the L in YAML stands for language, after all — which is a fine choice for simple scenarios, those where you want to machine-generate your Infrastructure as Code, or when engineers want to enable their sysadmins to do Infrastructure as Code too.

The Benefits of Declarative

Code is one major benefit. But in addition to the benefits of code, Infrastructure as Code has another significant advantage: It is “declarative,” even if you’ve chosen an imperative language like Go to express your code.

Infrastructure as Code tools generally work using a concept called “desired state.” The code, when run, produces a picture of the infrastructure your application requires. The Infrastructure as Code tool then understands how to compare the desired state with reality, and plan a course of action based on that information.

If it’s your first time creating a certain environment, something Pulumi calls a stack, then of course all the declared infrastructure will need to be created from scratch. Upon subsequent evaluations, however, that same infrastructure may need to be updated, deleted or even re-created, in addition to new infrastructure that may get spun up when it’s the first time it has been declared. This plan is presented ahead of performing any actions, so you and your team can review it, and if the course of action is wrong, you can correct it first.

One example of this process would be to first create a microservice environment that includes a Layer 4 network load balancer, a containerized cluster and a replicated, containerized service. You could subsequently add a private container registry, switch to a Layer 7 application load balancer, and scale up the service from one to three replicas.

This declarative approach ensures we can preview changes before they are made so we don’t have any unpleasant deployment surprises, gives us a full audit history of exactly what has changed in our actual infrastructure and when, similar to what source control does for our code artifacts, allows us to gate deployments on verification checks such as testing and policy enforcement, and makes it easier to integrate with various automation workflows.

It’s the codification of infrastructure and repeatability of the declarative approach that lets us use Infrastructure as Code in many kinds of automated workflows. That includes running a command-line interface (CLI) manually or as part of a script. Although “manual” may sound bad, the actual deployment is done with all of the above safeguards, so it’s fairly common for an Infrastructure as Code tool to be run this way.

However, most teams will adopt a CI/CD model for their most important environments, like production, which will trigger the actual deployment of code changes off a code commit. This ensures that all changes have been reviewed in the usual ways and go through a standard CI/CD pipeline. That pipeline may also include continuous verification (CV) such as running tests.

Some Infrastructure as Code tools support just one cloud, but our chosen tool, Pulumi, supports many, including all of those mentioned above, so all of these workflows can be standardized across all of the clouds and service providers. It can even track dependencies between cloud services — for example, it would not be strange to provision an Elastic Kubernetes Service cluster in AWS, install Datadog agents on its nodes, deploy some Kubernetes workloads, and place a Cloudflare content delivery network in front of that application, all using a single Infrastructure as Code program.

Pulumi’s unique approach unlocks an even more sophisticated workflow for running your code, using its so-called “Automation API.” This approach embeds Infrastructure as Code workflows right into larger pieces of software so that it can be programmed for highly dynamic scenarios.

This unlocks scenarios like building custom tools and libraries that build on top of and extend Infrastructure as Code, internal infrastructure provisioning portals and even entire SaaS products that need to provision or manage infrastructure as part of delivering their capabilities to their own end users.

In Part 2 of our series, we will take you through the steps needed to set up Infrastructure as Code. In doing so, we will be using Pulumi’s free and open source SDK, which is available here. It’s easy to get started, but you may want to take time now to explore the platform. You might also like to sign up for Pulumi Cloud, which can be done here.

Pulumi's Declarative and Imperative Approach to IaC

Ringo De Smet — Fri, 23 Sep 2022 00:00:00 +0000

On a regular basis, articles and tweets pass by discussing whether some specific tool is imperative or declarative.

It’s no surprise that Pulumi is often the tool being debated. What if I tell you that Pulumi is imperative, declarative and imperative?

When we look at our Frequently Asked Questions, we read the following on the declarative versus imperative topic:

Pulumi is a declarative tool that uses imperative languages to define your end state. The language is used for authoring your program. It’s not used for talking to the cloud provider API.

It is good to know to refresh what declarative and imperative mean:

Both terms refer to how the user provides direction to the automation platform. With an imperative tool, you define the steps to execute in order to reach the desired solution. With a declarative tool, you define the desired state of the final solution, and the automation platform determines how to achieve that state. (Source)

The title of the article mentions imperative and declarative both. Pulumi leverages the best of both worlds into our product.

Pulumi tries to offer a solution where our customers are only limited by their imagination rather than the tool at hand. Let me use the Pulumi architecture to highlight why I mention imperative twice.

Pulumi Architecture

Here is the diagram from our How Pulumi Works page:

You code your infrastructure in your preferred programming language. When you are done coding, you run pulumi up and the Pulumi CLI starts the language host for your selected programming language, as well as the required providers. The interaction between these 3 parts of the architecture results in the actual creation or modification of your infrastructure.

An Imperative Part: The Language Host

Under the hood, the Pulumi CLI does a lot of things, but one of the first actions is starting the language runtime, which is configured in the Pulumi.yaml project file:

name: blog_code
description: Example for declarative blog post
runtime: python
main: blog_code/

Once the language runtime is up, it turns to the program file. Here is a small Python example:

from pulumi_aws import s3

bucket = s3.Bucket('bucket')

for i in range(10):
 s3.BucketObject(
 f'object-{i}',
 s3.BucketObjectArgs(
 bucket=bucket.id,
 key=str(i),
 )
 )

A Pulumi program models the to-be state of your infrastructure. If you read this program, you can find that we define 11 resources as our to-be infrastructure:

1 AWS S3 bucket
10 Objects in the bucket created in the previous step

The stack itself is also modelled as a resource, and it is the parent to all other resources.

$ pulumi up

Updating (<masked>/dev)

View Live: https://app.pulumi.com/<masked>/blog_code/dev/updates/1

 Type Name Status
 + pulumi:pulumi:Stack blog_code-dev created
 + ├─ aws:s3:Bucket bucket created
 + ├─ aws:s3:BucketObject object-0 created
 + ├─ aws:s3:BucketObject object-1 created
 + ├─ aws:s3:BucketObject object-3 created
 + ├─ aws:s3:BucketObject object-2 created
 + ├─ aws:s3:BucketObject object-6 created
 + ├─ aws:s3:BucketObject object-4 created
 + ├─ aws:s3:BucketObject object-5 created
 + ├─ aws:s3:BucketObject object-7 created
 + ├─ aws:s3:BucketObject object-9 created
 + └─ aws:s3:BucketObject object-8 created

Resources:
 + 12 created

Duration: 13s

While this is definitely an imperative program, there is one important thing to understand: Instantiating an s3.Bucket, s3.BucketObject, or any other Pulumi resource should not be interpreted as an imperative creation of the resource in the language host. Behind the scenes, any resource instantiation in the language host triggers a Register Resource request to the Pulumi deployment engine. All these Register Resource requests together form the resource model you as an infrastructure developer want to get in the end.

Running your program always sends the full resource model to the Pulumi deployment engine regardless of what state your current infrastructure is in.

Our previous example was shown in Python, but recently we also delivered support for YAML. The creation of our S3 bucket could be converted to this snippet:

resources:
 bucket:
 type: aws:s3:Bucket

The separation of language support from the engine allows us to offer both imperative & declarative solutions. But remember that none of the supported languages do any of the provisioning directly in the language host.

If the resource provisioning is not taking place in the language host, where is the magic happening then?

The Declarative Part: The CLI and Engine

In the previous step, you found out that the language host sends requests to the engine to fulfill your to-be infrastructure.

It is now that the Pulumi deployment engine gets to work. The engine combines the intended model of the infrastructure received from the language host, the current state recorded in the state backend, and the actual resource state to compute which actions need to be executed to bring the actual state in line with the intended model.

Our little example contains dependencies: every s3.BucketObject uses the bucket.id as a way to define in which bucket these objects should be stored. The property id from the s3.Bucket is an Output. Outputs are Pulumi’s way of tracking which property of one resource is required by another, hereby creating a dependency between the resources. The engine uses all these outputs passed from one resource to another as the vertices in a directed acyclic graph (DAG). The engine determines the order of actions based on this graph.

On a first run of pulumi up of our example program, the engine will first create the bucket and wait for the provisioning to be complete, after which the actual bucket ID is passed to the creation of all the bucket objects. Since none of the bucket objects depend on other resources, these bucket objects can all be provisioned concurrently.

On a second run, assuming no modifications to our example program, the Pulumi deployment engine will compare the to-be model with the actual state and conclude that nothing needs to be done.

Let’s crank up the number of bucket objects to 11.

from pulumi_aws import s3

bucket = s3.Bucket('bucket')

for i in range(11): # <- number of objects increased by 1
 s3.BucketObject(
 f'object-{i}',
 s3.BucketObjectArgs(
 bucket=bucket.id,
 key=str(i),
 )
 )

The third Pulumi run will find the bucket and the first 10 objects in the state. The only action taken now is the creation of the 11th bucket object. This brings the actual state back in sync with your intended model.

$ pulumi up

Updating (<masked>/dev)

View Live: https://app.pulumi.com/<masked>/blog_code/dev/updates/2

 Type Name Status
 pulumi:pulumi:Stack blog_code-dev
 + └─ aws:s3:BucketObject object-10 created

Resources:
 + 1 created
 12 unchanged

Duration: 5s

Although you create the intended model of your infrastructure with an imperative language, the engine definitely processes this in a declarative way.

Another Imperative Part: The Providers

These examples hopefully made clear that the Pulumi deployment engine calculates a set of actions to declaratively bring the actual state in sync with your intended model. But the engine is not the component which knows how to talk to all the different APIs of cloud and tool vendors. That’s the role of the provider.

The engine and the provider processes are connected with a gRPC connection. This is similar to the connection between the language host and the engine, with the only difference of which requests are sent. The API that providers expose are of a CRUD nature:

Create: create a new resource
Read: read information of an existing resource
Update: update an existing resource with modified information
Delete: delete an existing resource when no longer needed

The providers receive requests from the engine dependent on the set of actions the engine calculated. A provider doesn’t know anything about the state, the correlation between resource, and so forth.

The nature of the provider API is clearly imperative.

Summary

If the question ever pops up again whether Pulumi is declarative or imperative, the answer is clearly we are both. It is only based on which component of our architecture you are talking about:

Language host: imperative (JS/TS, Go, Python, C#/F#/.NET, Java) and declarative (YAML)
Pulumi engine: declarative
Providers: imperative

To us, it mainly matters if we can solve your infrastructure automation problem. If you still miss something in our offering, we want to hear from you!

Pulumi and RedMonk on developer-first infrastructure and why it matters

George Huang — Tue, 26 Apr 2022 00:00:00 +0000

What do assembly languages and the cloud have in common? Are abstractions the future of cloud computing? What does “infrastructure” really mean? And why do these questions matter to the platform engineers, infrastructure engineers, and developers who are building modern cloud applications today?

Joe Duffy (Founder & CEO, Pulumi) and James Governor (Co-founder, RedMonk) recently answered these questions and more in a conversation about developer-first infrastructure. Developer-first infrastructure means empowering developers to build and deploy modern cloud applications and infrastructure through the use of software engineering practices that tame modern cloud complexity.

If you’re interested in learning how the worlds of software engineering and cloud infrastructure are intersecting in addition to principles and best practices for building in the cloud, then watch the 28-minute video below.

Highlights of Pulumi and RedMonk’s discussion on developer-first infrastructure

In the first half of the discussion, Joe and James discuss the complexities of building on the modern cloud as application architectures have evolved from simple three-tier apps to the modern, distributed applications of today. Joe traces the evolution of modern computing in which complexity has been tamed over time through abstractions: from low-level assembly languages to higher-level languages and runtimes that make software more accessible, and finally to operating systems. They then discuss how an analogous transformation has yet to occur in the cloud on a mainstream basis.

“The level of abstraction [that most people are doing] with cloud infrastructure reminds me of assembly languages. You look at writing these YAML templates that over-specify every detail of the underlying architecture. I think [the industry] can do better. I think we can be inspired by what we’ve done in programming languages.” — Joe Duffy

In the second half of the discussion, the two discuss what developer-first infrastructure is, and how companies can use this philosophy to tame modern cloud complexity and accelerate development velocity. Companies that execute this well have a “developer-first multiple” — that is, the most valuable companies in the world today are the ones who’ve thought hard about the developer experience for their own employees or the communities that interact with them.

Joe describes five principles of developer-first infrastructure that make developers’ lives easier when building on the cloud.

Cloud Engineering
Harmonized Applications and Infrastructure
Beyond “Lift and Shift”
Secure and Observable By-Construction
An API for Everything

Finally, Joe dives into the best practices of developer-first infrastructure:

Unified Engineering: Software engineering tools and practices for applications and infrastructure.
APIs: Making everything programmable and composable with APIs.
Everything as Code: Use “as code” techniques like infrastructure as code and policy as code.
Polyglot: Embrace the best language ecosystems for the job at hand.
Modularity: Tackle complexity with real sharing and reuse, not copy & paste.
Continuous Delivery: Deliver and scale applications and infrastructure with automated techniques.
Continuous Verification: Verify and enforce guardrails at all times.
Environment Parity: Share as much as possible between dev, test, and prod environments.
Securely Configured: Loosely couple and securely configure environments.
Distributed Architectures: Go beyond “lift and shift” to building truly cloud-native applications and infrastructure.
Observable: Instrument applications and infrastructure to log high cardinality events, early and often.

Next steps

Developer-first infrastructure means empowering developers to build and deploy modern cloud applications and infrastructure through the use of software engineering practices that tame modern cloud complexity. If you feel inspired after viewing the Pulumi/RedMonk discussion video, then consider applying developer-first principles and practices for your next greenfield project or application using an infrastructure as code tool like Pulumi. You can also watch Joe Duffy’s presentation on developer-first infrastructure from the Cloud Engineering Summit to learn more about developer-first.

What Exactly Is Cloud Engineering?

Matty Stratton — Fri, 23 Jul 2021 12:46:42 -0500

When we think about the idea of “cloud engineering,” we often think about the concept of taking standard software engineering practices and tools, and making them available and consistent across development, infrastructure, and compliance teams.

It sounds a lot like what DevOps was supposed to accomplish, right? Many great practices have come out of software engineering that we can apply to operations and infrastructure. Likewise, practices from operational disciplines are equally applicable to development teams.

In cloud engineering, we look at how all of these practices are available to multiple functions and teams. It’s a compelling concept, and the more that we refactor our thinking around this, the more effective we can be at delivering value to our customers and users.

Cloud engineering has three main components: Build, Deploy, and Manage. These are all different areas, but they have a lot in common. We will dig into each of these areas in detail in upcoming posts in this series, but here are the key concepts around each.

Build

This might sound like it’s just about writing code, but the build area of cloud engineering is about creating services and infrastructure that provide what our customers need. In today’s world, we use cloud resources to build applications, services, and infrastructure. These resources can make up a shared service platform, giving a single, consistent experience across multiple teams.

It’s also advantageous to create reusable infrastructure components so that application and service delivery focuses on differentiators rather than “reinventing the wheel.” Reusable components provide a consistent and standard implementation, using our organizations’ and teams’ existing practices.

Besides expressing our own best practices and common configurations and approaches, when we build our infrastructure using standard programming languages, we can take advantage of the entire ecosystem surrounding them! We can leverage existing IDEs, test frameworks and approaches, and other wonderful tools available for those languages. Using modern architectures also focuses on the value our services and platforms provide, rather than bespoke and custom implementations.

Deploy

It doesn’t count until it’s in production, right? Code and infrastructure don’t give any value until it’s in front of our customers and users. But doing this in a manner that is highly efficient and quality-consistent is key. Deployment processes that take too long or require too many manual steps can block us from getting new features to our customers or resolving service issues.

When we apply software engineering practices to our deployment processes, this can ensure that we ship the same way, every time. It’s become common practice to apply the principles of continuous integration and delivery to our application software, but we can use the same principles with our infrastructure. This means that new and changed infrastructure resources can meet our quality controls and be tracked and understood when we are investigating the dreaded “what changed?” problem.

The value of automating deployment isn’t just about providing the tests; it ensures all the steps we require are performed every time. Regardless of years of experience, every human is capable of missing a step or making an error. We want to have our skilled humans focus on the areas that benefit from human expertise and not waste their time on the things that don’t.

“But what about checklists?” Checklists are great! But they are even better when a human defines them and are run by software. The power of a checklist is in defining steps, and in many ways, automation expresses our checklists in code. Code that can be tested, reviewed, and managed.

But beyond simple automation, having a unified approach to deployment for both application code and infrastructure changes allows us to consider automation as the key part of the application. We think about infrastructure as a critical and essential component rather than something that happens “on the side.”

Manage

Getting our services into production is a key step, but it’s not the end. Our customers constantly use our services and applications, and we need to manage all of the resources in use. As it applies to our applications and services, visibility across all of our infrastructure allows everyone on our team, regardless of their role, to have a common understanding of what’s going on.

You may be familiar with the adage “security is everyone’s job.” What that means in the world of cloud engineering is that we consider security and compliance (whether regulatory policies or organizational policies) to be closely integrated into our work. Treating our policy as code, just as we treat our infrastructure as code, is a powerful idea! When we express these policies as code rather than prose in a document, we can apply these policy checks both before and after we deploy our services and infrastructure. Doing so extends the common “vocabulary” for collaboration across all teams, regardless of where they sit in the org chart.

Another key piece of the manage story is that we need controls in place to allow who can make changes and what they can change! We trust our team members to want to do the right thing, but we also need guardrails and controls to ensure that they can do so. This means that we need visibility into all changes that occur - treating our infrastructure just like we do our source code in git! Additionally, having the capability and intentionality around fine-grained access controls is critical to make all of our team members successful, provide reliability around our services, and provide confidence for our customers.

Summary

When we take the ideas of cloud engineering and apply them to our organizations, we can deliver value to our customers and users. We can do this by applying the practices and tools already available to our teams and applying the same principles to our infrastructure and application code. We get increased collaboration capability within our teams, a higher level of trust and confidence in our services and applications, and a better handle on the complexity of the modern cloud. We will explore some specific practices and tools around each of these areas in future posts and how they can be applied to your organization!

Do you have a great story around your experiences with cloud engineering? We would love to hear it at the upcoming Cloud Engineering Summit! Submit your proposal to the CFP by July 31st!

The Past, Present, and Future of Cloud Engineering

Joe Duffy — Thu, 08 Oct 2020 00:00:00 +0000

All software is cloud software. All modern applications interact with the cloud in some way, whether it’s using cloud for storage, compute capabilities, or with rich cloud services for data, AI/ML, and so much more, to deliver amazing new experiences. As a result, all developers today are cloud developers, and infrastructure teams are key to enabling innovation across the entire organization. I had a great time telling this story at the Cloud Engineering Summit today and wanted to take a moment to put pen to paper.

Introduction

The cloud is the world’s biggest supercomputer. Although the cloud broke our architectures, workflows, and teams, we are now amidst a renaissance of rebuilding new ways to create truly cloud-native applications and experiences.

We conceived of the Cloud Engineering Summit to celebrate this renaissance. The talks throughout the day explore how we build architectures and applications. We also explore how to operate those cloud architectures in a team setting through workflows using automation and processes to build, deploy, and manage those applications and their infrastructure. Finally, teams are working together more closely and in new ways in order to collaborate to deliver applications and infrastructure across all cloud engineering disciplines.

This is what we are calling Cloud Engineering: the next quantum leap in how we build cloud software. Cloud engineering means bringing the cloud closer to application development and infusing its capabilities to create new cloud-enabled experiences. Cloud engineering also means bringing more engineering practices to infrastructure, enabling us to leverage infrastructure as not just code but software. And finally, cloud engineering enables us to unlock collaboration, enabling the entire team to move faster with confidence.

Cloud Engineering in Practice

When the cloud came along, our initial approach was to reuse the same application patterns from the Internet 1.0 era, starting with “lift and shift” and virtual machine-centric architectures. We’ve gone through at least two major sea changes since then, including the introduction of containers and serverless architectures, and if we were to start over with a blank slate today, we’d surely do things very differently.

In fact, the cloud effectively broke our existing approaches for architectures, workflows, and teams.

As the cloud incrementally broke these over the past decade, we invented best practices, tools, and techniques to cope with it. This led to DevOps and a fundamental change to how we worked together to build increasingly cloud-enabled software. DevOps started from a place of collaboration, bringing engineers and operations people closer together and was a very powerful precursor to what is now happening.

This is where cloud engineering now enters the picture. Instead of being reactionary to change, we can now proactively pursue approaches designed for the cloud. Instead of just coping, we can take a step back and reimagine new ways of doing things.

Cloud engineering takes the best practices from DevOps and extends them. There is no question that DevOps, despite its original intent, ended up more “opsy” than “devvy”. With cloud engineering, application developers are now embracing the cloud and infusing cloud capabilities such as microservices and serverless into applications in fundamentally new ways, creating new application experiences for end users.

And at the same time, infrastructure experts, capabilities, and teams are key to enabling innovation more than ever before. During the cloud era, infrastructure became programmable software, which allows us to apply all the software engineering practices we know and love to infrastructure, including testing, refactoring, security, continuous delivery, and agile.

More cloud infrastructure capabilities have moved into application land, while at the same time more application development practices have moved into infrastructure land. What this means is that the whole team is collaborating more closely than ever before. Everyone is empowered to bring their unique skillsets to the table to enable moving faster and deliver amazing software capabilities, regardless of title — developer, DevOps, systems engineer, infrastructure architect, security engineer, etc. This is the modern cloud engineering organization.

Cloud Engineering Architectures

Infrastructure used to be a tax, and IT a cost center, but cloud infrastructure is increasingly a competitive advantage that can transform entire business models. Just think of the many cloud era companies that are enabling new experiences thanks to the cloud — AirBnB, Amazon, Lyft, Spotify, and others. Zoom experienced a 20x increase in usage when COVID started and, because they architected their application for the cloud, they were able to scale and meet demand. Five to ten years ago, without cloud engineering, that wouldn’t have been possible.

Zoom massively scaling is a great example of the difference between just tossing a traditional application into a virtual machine, versus fully embracing cloud capabilities as a foundational piece of the application’s design. The ability to scale up elastically, and pay as you go, is an immense competitive advantage when designing applications to run on the cloud’s planet-scale supercomputer.

This means that cloud infrastructure is a superpower to be wielded and used for great things, not an afterthought. If only teams can figure out how to harness them, they can accomplish new business outcomes and reshape how applications work.

Application models have also evolved in ways that enable this new cloud engineering era. In the 2010s, you had XML web services and SOA, which was the precursor to microservices, and now with modern programming languages, the shift to distributed architectures is a lot easier. Asynchronous programming and concurrency models are pervasive on all of our modern languages and frameworks, and we can use those things to build truly distributed applications that are composed of many smaller pieces that communicate asynchronously and can scale on demand.

Cloud Engineering Workflows

It’s not just the application architectures that have changed; the way that we build, secure, test, ship, and manage those applications and their infrastructures have also fundamentally changed. This is what we call “workflows”: the way in which we operationalize, and often continuously deliver, our cloud applications and infrastructure into our many dev, test, and production environments.

Infrastructure as code lets us declare infrastructure using “code” (although, in many cases, code is really just markup or “text”). We are only just now beginning to apply all the engineering practices that are available to developers when building infrastructure, with technologies like Pulumi that enable infrastructure as software. This includes leveraging great IDEs and tools, enabling higher levels of productivity. Instead of copying and pasting text, we have real patterns and best practices thanks to sharing and reuse.

Among the many things the cloud broke is the inner loop development cycle. We’re often stepping into and out of our IDE, running command lines, pointing and clicking in the cloud console, and so on. The infrastructure as text world often means finding failures 10 minutes into a deployment, when it’s too late, rather than finding errors interactively in your IDE. By embracing infrastructure as software, and the fact that infrastructure is moving closer to application architectures, our inner loop and great productivity is starting to be restored.

Treating infrastructure as software enables us to test infrastructure before it has been deployed, spin up ephemeral environments to exhaustively validate changes before attempting to deploy them, or even apply advanced forms of testing, such as chaos testing, to ensure applications function properly during a service outage, or fuzz testing, to ensure the application works in the face of randomized changes.

Security is a also a key cross-cutting concern but is still often overlooked during the inner loop of developing cloud software. The security perimeter for cloud software is no longer the four walls of the data center but has instead moved to software managed networks and firewalls. This changes the threat model and requires that teams adopt zero trust and principle of least privilege approaches, where applications, infrastructure, and team members have the least permissions possible to get the job done, minimizing the blast zone of possible incidents. We must also adopt approaches that ensure software is secure before deploying, including applying policies that do things like preventing unintended Internet access to services, block S3 buckets and databases open to the Internet, and scan containers for known vulnerabilities.

Once you’ve shipped an application to production, the job has only just begun. Teams must adopt patterns and practices for how to monitor and react to situations that occur in their live environments. That often also includes testing and flighting changes in production. One approach is to adopt an “OODA loop” like approach, where OODA means Observe, Orient, Decide, Act. This is a decision-making framework that allows for rapid decision making with the context of a situation, and good analogy for how we maintain our cloud applications in the real world. How do we know something is wrong? How do we react? The old model of simply relying on performance counters, such as CPU or memory utilization, and highly structured logging, doesn’t really work in the modern cloud era. Instead, teams are adopting observability, and a “log everything” approach, so that unstructured and semi-structured high-cardinality data is available at all times, enabling us to observe, debug problems in realtime as they occur, and make decisions and act when when things go wrong.

Continuous delivery is not a new idea, however, teams are now moving beyond just delivering applications towards shipping entire environments, packaging applications and infrastructure together. This enables all of the above practices, including security and testing, to apply to entire cloud application environments. Continuous delivery is also how application and infrastructure teams work more closer together, enabling teams to go from shipping features shipping quarterly, to monthly, to every week, to daily, to shipping features on every Git commit. Delivery and workflow are intertwined, and our cloud engineering brings them even closer together across the entire team.

Cloud Engineering Teams

Cloud engineering teams are empowered: everybody brings their unique skills, as an expert in their domain, and collaborates to enable 1+1=3 outcomes. Cloud engineering is not about “NoOps” or operations being less important than it was before. It is about recognizing that all of these skills are essential and that a cloud engineering team is the composition of all of these skillsets.

Application developers can focus on what they’re great at: architecting, implementing, and delivering new functionality to their application’s end users. Developers probably aren’t going to want to suddenly go deep in what it takes to create and configure an AWS virtual private cloud, but they sure do want to tap into all of those amazing data and AI/ML services, and bring container and serverless architectures and continuous delivery into the inner loop of how they are building and shipping code. And there’s just no way to ignore the runtime environment when it comes to debugging, performance tuning, scaling the application, and observability, just to name a few important facets.

Infrastructure and DevOps engineers likewise can also focus on what they’re great at: building scalable, reliable, and cost-effective foundations that the entire organization runs on. Infrastructure teams can benefit from great productivity wins of great IDEs, tools, and true sharing and reuse of best practices. That’s not to say suddenly infrastructure engineers need or want to learn everything there is to know about the latest application frameworks or patterns. But many of the foundations of DevOps, where we apply more software engineering and agile practices to how we ship infrastructure, including test driven infrastructure, have reshaped how infrastructure experts operate.

Security engineers are also empowered to be part of this entire lifecycle. This ensures that security isn’t an afterthought, but instead, that security engineers have all the requisite insertion points into the application and infrastructure architectures and workflows, to ensure that solutions are secure on day one, and stay secure. Security engineering is a central part of how cloud engineering teams operate.

Finally, all of these modern cloud engineering practices have helped teams to cope in this suddenly all-remote COVID world. By embracing distributed architectures, automated workflows, and multi-faceted teams that collaborate closely, entire businesses have made the shift to all-remote and online collaboration much faster. Those teams practicing cloud engineering have had an easier time reacting to rapid change.

Towards a Cloud Operating System

Today’s cloud is comprised of 1,000s of building block services across a landscape of multiple cloud providers. AWS alone offers over 200 of these building blocks, and every SaaS infrastructure provider is part of the picture too. As we have seen above, this offers incredible capability, if teams can only figure out how to harness the power of those building blocks. It’s not always easy but the cloud engineering practices above are key to getting started.

It does beg the question, however: If the cloud is a giant supercomputer, where is its operating system? When will we go beyond building blocks? In fact, we are well on our way.

Kubernetes gives us some fundamental operating system-like components — a scheduler, resource model, and a security model, for example — but a complete operating system must go well beyond this, including programming and application models, and many key facets of the cloud, including all those AWS building blocks, still live outside of the Kubernetes cluster (and likely always will).

What’s the user interface to this giant supercomputer supposed to look like? What’s the human experience of using that computer? We’ve extended our previous experience from client-server desktop and Internet applications into the cloud, but it’s just the start of how we interact with the cloud.

What does the developer’s interface to that supercomputer, the programming model, look like? For now, YAML and tools like Terraform give us a way to configure the building blocks using low-level markup languages, but we need to go beyond this and leverage all the things we know and love about other operating system programming models. This includes using great languages and runtimes like TypeScript, Python, .NET, and Go. This is the approach we’ve taken with Pulumi to start creating richer developer experiences with respect to how we stitch together all those building blocks and build bigger things out of smaller things, but it is still early days here.

Finally, where is the application model? How do we reliably and repeatably install, version, secure, work with, and manage these cloud applications? Containers provide a building block for building and deploying applications in the cloud, Helm and ideas like CNAB provide an installation model, but it is a far cry from an “AppStore” experience that we get on other operating systems. The cloud supercomputer interface is very low-level and bottom-up compared to application models which approach the experience from a top-down perspective.

Despite the clear gaps, it is clear we’ve taken some major steps in the direction of a true cloud operating system.

The Future of Cloud Engineering

As we’ve seen, the future of cloud engineering is already here in a very real way. The cloud is being infused into application development, we are increasingly treating cloud infrastructure as software, and teams are embracing new ways of working together.

We hope you enjoyed learning more about cloud engineering architectures, workflows, and teams. Watch the Cloud Engineering Summit Keynote here or check out one of the many amazing Cloud Engineering talks now!

This event would not have been possible without the participation and support of our top sponsors: