Pulumi Blog: Developer experience

Treating Prompts Like Code: A Content Engineer's AI Workflow

Cam Soper — Mon, 09 Mar 2026 00:00:00 +0000

Pulumi has a lot of engineers. It has marketers, solution architects, developer advocates. Everyone has something to contribute to docs and blog posts — domain expertise, hard-won lessons, real-world examples. What they don’t all have is familiarity with our Hugo setup, our style guide, our metadata conventions, or where a new document is supposed to live in the navigation tree. I joined Pulumi in July 2025 as a Senior Technical Content Engineer. A few weeks in, my sole teammate departed. The docs practice was now, functionally, me.

The problem was clear enough: how do you take one docs engineer’s accumulated knowledge and make it available to everyone who needs it, without that engineer becoming a bottleneck?

I started packaging it. Here’s what that looked like in practice.

The real problem AI solves

Everyone talks about AI making you faster. That’s not wrong, but it’s not the most interesting part — at least not for me.

The most interesting part is what it does to the starting problem. I have an ADHD brain (not formally diagnosed, but with enough self-recognition to know what’s going on). I know what that means for my relationship with most tasks: I can see the problem, I understand it, I want to fix it, and then the sheer weight of starting crushes me flat.

When I’m stuck on a task, the issue is almost never that I don’t know what to do. It’s that my brain is trying to hold the entire finished product in working memory while simultaneously producing the first step. That’s an enormous cognitive tax, and for an ADHD brain it’s often insurmountable.

Talking through a problem conversationally is a completely different cognitive load. I can tell Claude “here’s the issue, here’s what I’m trying to accomplish, here’s what’s weird about it,” and suddenly I’m not staring at a blank page anymore. I’m in a conversation. The scaffold exists. I can build on it.

That dynamic isn’t new for me. In a previous role writing training modules at Microsoft, I did some of my best work, not because the work was easy, but because I had a collaborator. A friend to think out loud with. Someone to say “okay, so what are we actually trying to say here?” That conversational scaffolding was the difference between spinning and shipping.

In my current role as a team of one, AI turned out to be that collaborator.

This isn’t really a productivity story. It’s closer to a cognitive accommodation story. And I’d bet a lot of people — diagnosed or not — will recognize what I’m describing.

Treating prompts like code

If conversational scaffolding could lower my own activation energy, the next question was obvious: could I build that for anyone who needed it? I knew I wanted to use AI to solve this problem, but I didn’t want to just write a bunch of one-off prompts. That would be a maintenance nightmare, and it wouldn’t scale beyond me. I needed a system. Claude Code calls these reusable prompts skills — other platforms have the same idea under names like plugins or extensions. My first real experiment was /docs-review — a reusable prompt that would run my writing through a consistent set of criteria before I committed it. Nothing fancy. I just wanted a reliable bar that didn’t depend on my mood or how much coffee I’d had.

Then it occurred to me: every PR to our docs repo should get this automatically. So I wired it into our CI/CD pipeline. Meagan, my manager, loved it — and after a few weeks, she noticed that PR quality had improved dramatically. On almost every PR, contributors were now spontaneously pushing an “Addressing feedback” commit right after the automated review posts — catching and fixing issues before I ever saw the PR.

That’s when something clicked: I wasn’t writing prompts anymore. I was writing modules — reusable, composable pieces of my own expertise.

The insight was straightforward, but it changed how I thought about the whole system: if multiple skills need the same context — our style guide, our review criteria, our content standards — that context should live in one place and get consumed by everything that needs it. Just like a shared library. Just like any decent software project.

I created a REVIEW-CRITERIA.md file as the single source of truth for what a “good” docs PR review looks like at Pulumi. Every skill that does any kind of review pulls from it. Change it once, and everything gets smarter at once. Likewise with our style guide, our Hugo conventions, our navigation structure. All of that lives in central reference files that any skill can pull from. If something changes, I change it in one place and all the skills get the update.

This also matters for token efficiency — which sounds like a nerdy footnote but isn’t, especially when automated reviews are running on every PR. Duplicating context across skills bloats token usage fast. Modularizing keeps it lean. Your CI/CD pipeline doesn’t care about elegance, but it definitely cares about cost.

The mental model I kept coming back to: Don’t Repeat Yourself. It’s the same principle that makes good software maintainable. It turns out it makes good AI workflows maintainable too.

The skill catalog

From there, the system grew organically. Whenever I found myself doing something more than once, I asked: “Can I turn this into a skill?” Here’s a sampling of what that produced:

/fix-issue — takes a GitHub issue and recommends a concrete plan of attack, so I go from “here’s a ticket” to “here’s what I’m doing” without the spinning-up tax.

/shipit — runs pre-commit checks, writes a focused commit message, and drafts a PR description.

/pr-review — full doc review on a PR branch: style guide, code examples, screenshots, optional test deployment, then an Approve/Merge/Request Changes dialog with a drafted comment.

/slack-to-issue — converts #docs Slack conversations into properly formed GitHub issues. Slack is where decisions happen; issues are where work gets tracked.

/glow-up — runs an older doc through the modern style guide and flags outdated screenshots, for digging out of accumulated technical debt.

/new-doc and /new-blog-post — guide anyone through adding a new document or blog post with the right location, metadata, and navigation wiring. Engineers, marketers, whoever. The barrier to contributing just dropped significantly.

/docs-tools — helps other repo users discover that any of this exists. Discoverability is a real problem with internal tooling.

Slack’s built-in Claude integration isn’t the same Claude running your Claude Code workflows — they don’t share context or custom instructions. If you want consistent criteria across both surfaces, you need to bring your own backend. That’s exactly what /slack-to-issue handles.

Other people started contributing skills to the repo — not because I asked, but because the pattern was legible enough to extend. Someone built a skill for SEO analysis. Marketing added their own review criteria. Engineers contributed workflows I never would have thought to build.

The thing I’d built as a personal survival tool had become a shared platform. That happened because I treated the prompts like code: modular, reusable, documented, open for contribution.

Honest limitations

It’s not a replacement for human judgment. These are probabilistic tools — they’re right most of the time, not all of the time. /pr-review doesn’t approve PRs autonomously. It highlights things and then asks me, the human, to read them and make the call. The AI does the first pass; I do the last one. That’s not a workaround for a limitation — that’s the design.

The system isn’t finished, either. It’s probably never finished. I’m still tweaking review criteria, still finding edge cases where a skill produces something weird, still adding new tools as new pain points emerge. Treating prompts like code means treating them like software: you ship, you iterate, you maintain. There’s no version 1.0 and done.

And the ADHD angle is real but it’s not magic. There are still days where the paralysis wins. AI lowers the activation energy for starting; it doesn’t eliminate it. I’m still the one who has to show up. I suppose I could automate that too, but then we’d be in a whole different kind of dystopia.

Turn your workflows into skills

Pulumi’s Agent Skills bring the same modular approach to infrastructure, teaching your coding agent the conventions your team relies on.

Get started

Know your models and their costs. At Pulumi we primarily use Claude, and I work in Claude Code; for most tasks I reach for Sonnet rather than Opus. Opus is excellent, but it’s significantly more expensive, and well-crafted instructions to Sonnet handle the vast majority of my work just as effectively.

Treat it like a coworker. Don’t just issue commands and wait for output. Ask what it thinks. Push back when it’s wrong. Explain your reasoning. The more you engage conversationally, the better the results tend to be. That extends to alignment, too — before diving into a complex task, talk through the approach first. A few minutes of alignment up front beats iterating on a misunderstood spec. I’ve gone as far as adding personal instructions to my config — things like playing along when I’m pretending to be Captain Picard, or using colorful language when the context calls for it. (Yes, those are literal config settings.) That sounds frivolous, but it isn’t: a tool you actually enjoy using is a tool you’ll reach for instead of avoid.

Modularize your workflow. Don’t write one giant monolithic prompt that tries to do everything. Break it into focused skills that do one thing well and share common context through a central reference file. Easier to maintain, easier to debug, cheaper to run.

Version control your prompts. Your skills are code. Treat them like code. Commit them, review them, iterate on them. If a skill starts producing weird output after a tweak, you’ll want to know what changed.

Think about token burn rate. This matters most when running automation in CI/CD. Keep your skills focused — a skill that checks style doesn’t need to load your Hugo navigation conventions. The model only reads what you give it, so give it only what it needs.

Not everything needs to be a prompt. This one is underappreciated: skills can include scripts, and that’s often the right call. When my team moves a doc in the repo, it needs to happen via git mv to preserve history, and we need to add a redirect alias to the front matter to prevent 404s and protect SEO. That’s not something I want an AI to reason through from scratch every time — it’s a solved problem. So it’s a script. The skill just knows the script exists and what it does. Claude orchestrates; the script executes. That’s a cleaner, more reliable division of labor than asking an LLM to reinvent the wheel on every run.

Not everything needs to be generative. Corollary to the last point: if you need deterministic output, don’t use probabilistic tools. We have a skill that generates the meta image for blog posts — procedurally, not generatively. No AI-generated imagery. We have a brand to protect, and “let the AI vibe it out” isn’t a content strategy. The skill follows our visual standards programmatically and produces something consistent every time. Know what you’re automating and why.

What’s next

The next frontier is bringing some of this tooling to the less technical members of the team — marketing, in particular. The skills I’ve built assume a certain comfort level with terminals and repos. That’s fine for engineers. It’s a barrier for everyone else. A friendly interface would lower that bar significantly — that’s the direction I’m currently exploring.

If you’re a technical writer, a developer advocate, or a solo practitioner figuring out how AI fits into your workflow, the approach described here is a solid starting point. The tools matter, but the mental model matters more: treat your prompts like code. Make them reusable. Document them. Share them.

Our docs repo is public, so the skills are there for anyone who wants them. If you’re building something similar, steal freely — or contribute back.

The blank page is still there. It’s just a lot less intimidating when you’ve got a good collaborator and a solid set of tools.

See our docs skills for inspiration

Golden Paths in IDPs: A Complete Guide to Reusable Infrastructure with Pulumi Components and Templates

Engin Diri — Wed, 20 Aug 2025 10:00:00 +0200

Welcome to the second post in our IDP Best Practices series. In this article, we explore how to create golden paths, pre-architected, reusable infrastructure patterns that help standardize and accelerate cloud development.

Modern cloud platforms offer endless options, over 200 AWS services, sprawling Azure catalogs, and countless DevOps tools. The result? Developers face decision fatigue and inconsistent implementations. Golden paths solve this by providing ready-to-use, production-grade infrastructure that encodes your organization’s best practices, security policies, and operational standards.

In this guide, you’ll learn how to build golden paths for your Internal Developer Platform using two core Pulumi constructs: Components, reusable infrastructure building blocks, and Templates, predefined, deployable patterns. You’ll see how to create infrastructure abstractions that are written once, shared across teams, and consumed in any language, turning weeks of setup into minutes of developer-ready infrastructure.

The complete code examples from this post are available on GitHub.

The Platform Engineering Layer Cake: A Model for IDPs

To understand where golden paths fit into an Internal Developer Platform (IDP), think in layers. This three-tier model structures your platform to deliver increasing levels of abstraction, reuse, and developer value:

Layer 1: Infrastructure Layer

This is the foundation: raw cloud resources include VMs, databases, networks, and storage, which are the fundamental building blocks from AWS, Azure, GCP, and other providers. Pulumi gives you programmatic access to these resources through native providers, but working at this level requires deep infrastructure knowledge.

Layer 2: Platform Layer - Components

This is where the magic happens. Pulumi Components take those raw resources and package them into higher-level abstractions. Instead of manually configuring 20+ AWS resources for a secure web application, you create a component that handles all that complexity and exposes just the configuration that matters:

const app = new SecureWebApplication("my-app", {
 instanceType: "t3.medium",
 minReplicas: 2,
 maxReplicas: 10,
 enableWAF: true,
 environment: "production"
});

Part 1: Building Reusable Infrastructure Components

Reusable infrastructure components act as the foundation of your platform. They encapsulate complexity while remaining composable and scalable. Let’s build a real-world example: a component that deploys containerized microservices to AWS Fargate.

The Power of Multi-Language Components

Here’s what makes Pulumi components revolutionary: write once, consume anywhere. Your platform team can author components in TypeScript, but application teams can consume them in Python, Go, .NET, Java, or even YAML. This separation of concerns is crucial for scaling platform engineering across diverse teams.

Building a Microservice Component Step-by-Step

Let’s create a component that encapsulates everything needed to deploy a production-ready microservice:

// microservice-component.ts
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";

export interface MicroserviceArgs {
 appPath: string; // Path to application code
 cpu?: number; // CPU units (256, 512, 1024, etc.)
 memory?: number; // Memory in MB
 port?: number; // Application port
 desiredCount?: number; // Number of tasks
}

export class MicroserviceComponent extends pulumi.ComponentResource {
 public readonly url: pulumi.Output<string>;
 public readonly serviceName: pulumi.Output<string>;
 public readonly clusterName: pulumi.Output<string>;

 constructor(name: string, args: MicroserviceArgs, opts?: pulumi.ComponentResourceOptions) {
 super("custom:infrastructure:Microservice", name, {}, opts);

 // Set defaults
 const cpu = args.cpu || 256;
 const memory = args.memory || 512;
 const port = args.port || 8080;
 const desiredCount = args.desiredCount || 2;

 // Create ECR repository for Docker images
 const repository = new awsx.ecr.Repository(`${name}-repo`, {
 forceDelete: true
 }, { parent: this });

 // Build and push Docker image
 const image = new awsx.ecr.Image(`${name}-image`, {
 repositoryUrl: repository.url,
 context: args.appPath,
 platform: "linux/amd64",
 }, { parent: this });

 // Create Application Load Balancer
 const loadBalancer = new awsx.lb.ApplicationLoadBalancer(`${name}-lb`, {
 defaultTargetGroup: {
 port: port,
 protocol: "HTTP",
 healthCheck: {
 path: "/health",
 interval: 30,
 timeout: 5,
 healthyThreshold: 2,
 unhealthyThreshold: 3,
 }
 }
 }, { parent: this });

 // Create ECS cluster
 const cluster = new aws.ecs.Cluster(`${name}-cluster`, {}, { parent: this });

 // Create Fargate service
 const service = new awsx.ecs.FargateService(`${name}-service`, {
 cluster: cluster.arn,
 taskDefinitionArgs: {
 container: {
 name: name,
 image: image.imageUri,
 cpu: cpu,
 memory: memory,
 essential: true,
 portMappings: [{
 containerPort: port,
 targetGroup: loadBalancer.defaultTargetGroup,
 }],
 },
 },
 desiredCount: desiredCount,
 assignPublicIp: true,
 }, { parent: this });

 // Export outputs
 this.url = pulumi.interpolate`http://${loadBalancer.loadBalancer.dnsName}`;
 this.serviceName = service.service.name;
 this.clusterName = cluster.name;

 // Register outputs
 this.registerOutputs({
 url: this.url,
 serviceName: this.serviceName,
 clusterName: this.clusterName,
 });
 }
}

Documenting for Discoverability and Adoption

To make your component easy to use, add comprehensive documentation and examples to a README.md file:

# MicroserviceComponent

Abstraction for resources needed when using AWS container services.

A component to abstract the details related to:
- Creating a docker image and pushing it to AWS ECR.
- Deploy to ECS Fargate using the docker image.

# Inputs

* appPath: Path to local folder containing the app and Dockerfile.
* port: Port to expose via an ALB.
* cpu (Optional): CPU capacity. Defaults to 256 (i.e. 0.25 vCPU).
* memory (Optional): Memory capacity. Defaults to 512 (i.e. 0.5GB).
* containerName (Optional): Name of the container. Defaults to "my-app".

# Outputs

* publicUrl: The DNS name for the loadbalancer fronting the app.

# Usage
## Specify Package in `Pulumi.yaml`

Add the following to your `Pulumi.yaml` file:
Note: If no version is specified, the latest version will be used.

... omit for brevity ...

This documentation will help developers understand how to use your component effectively, including required inputs, outputs, and example usage.

Publishing Your Component via Private Registry

Once your component is ready, publish it to your Pulumi Private Registry:

# Tag your component version
git tag v1.0.1

# Publish to your organization's registry
pulumi package publish https://registry.pulumi.com/myorg/microservice-component

Now any team in your organization can discover and use your component, regardless of their language preference. All they need is to navigate to the Components section in the Pulumi IDP and search for microservice-component.

Part 2: From Building Blocks to Golden Paths

While components are powerful, they’re just the starting point. Golden path templates layer on opinionated scaffolding, workflows, and compliance best practices that guide developers to production.

Golden Path Maturity: From Zero to Product-Grade Platforms

Teams evolve from ad hoc deployments to mature, productized templates with versioning, metrics, and governance. These maturity levels reflect how deeply your platform enables safe, consistent delivery.

Organizations typically progress through three stages of golden path maturity:

Stage 1: No Golden Paths

Every team reinvents the wheel
Inconsistent practices across projects
High cognitive load on developers
Security and compliance gaps

Stage 2: Dawn of Templates

Basic cookie-cutter templates emerge
Some standardization begins
Manual processes still dominate
Limited support and evolution

Stage 3: Templates as Products

Golden paths have dedicated owners
Regular release cycles and versioning
Migration guides for updates
Metrics track adoption and success
Continuous improvement based on feedback

What Makes a Golden Path Golden?

Drawing from Spotify’s pioneering work, golden paths share these characteristics:

Pre-architected and Supported: The platform team owns and supports the path
Optional but Recommended: Developers can deviate, but staying on the path ensures support
Transparent Abstractions: The implementation is visible, not a black box
Extensible: Teams can add project-specific resources without breaking the pattern
Evolutionary: Templates improve based on feedback and new requirements

Building a Go Microservice Golden Path

Let’s create a complete golden path for Go microservices that includes:

Application scaffolding with best practices
Infrastructure deployment using our component
CI/CD pipeline configuration
Observability setup
Security controls

Step 1: Template Structure

go-microservice-boilerplate/
├── Pulumi.yaml # Infrastructure definition
├── src/ # Application code
│ ├── main.go # Go microservice
│ ├── Dockerfile # Container definition
│ └── go.mod # Dependencies
└── README.md # Documentation

Step 2: Application Scaffolding

The src/main.go file contains a minimal Go microservice with all the best practices and compliance guardrails baked in. This helps developers to quickly get started and extend the application without worrying too much about selecting the right libraries or frameworks.

// source/main.go
package main

import (
 "context"
 "flag"
 "net/http"
 "os"
 "os/signal"
 "time"

 "github.com/labstack/echo/v4"
 "github.com/labstack/echo/v4/middleware"
 "go.opentelemetry.io/contrib/instrumentation/github.com/labstack/echo/otelecho"
 "go.opentelemetry.io/otel"
 "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"
 "go.opentelemetry.io/otel/sdk/resource"
 sdktrace "go.opentelemetry.io/otel/sdk/trace"
 semconv "go.opentelemetry.io/otel/semconv/v1.34.0"
)

func initTracer() (*sdktrace.TracerProvider, error) {
 exporter, err := otlptracehttp.New(context.Background())
 if err != nil {
 return nil, err
 }

 resource, err := resource.Merge(
 resource.Default(),
 resource.NewWithAttributes(
 semconv.SchemaURL,
 semconv.ServiceNameKey.String("${PROJECT}"),
 semconv.ServiceVersionKey.String("1.0.0"),
 ),
 )
 if err != nil {
 return nil, err
 }

 tp := sdktrace.NewTracerProvider(
 sdktrace.WithBatcher(exporter),
 sdktrace.WithResource(resource),
 sdktrace.WithSampler(sdktrace.AlwaysSample()),
 )

 otel.SetTracerProvider(tp)
 return tp, nil
}

func echoHandler(c echo.Context) error {
 message := c.QueryParam("message")
 if message == "" {
 message = "Hello from Go microservice!"
 }

 return c.JSON(http.StatusOK, map[string]string{
 "echo": message,
 "service": "${PROJECT}",
 "timestamp": time.Now().UTC().Format(time.RFC3339),
 })
}

func healthHandler(c echo.Context) error {
 return c.JSON(http.StatusOK, map[string]string{
 "status": "healthy",
 "service": "${PROJECT}",
 })
}

func main() {
 healthCheck := flag.Bool("health-check", false, "Run health check and exit")
 flag.Parse()

 if *healthCheck {
 resp, err := http.Get("http://localhost:8080/health")
 if err != nil || resp.StatusCode != http.StatusOK {
 os.Exit(1)
 }
 os.Exit(0)
 }

 tp, err := initTracer()
 if err != nil {
 panic(err)
 }
 defer func() {
 if err := tp.Shutdown(context.Background()); err != nil {
 panic(err)
 }
 }()

 e := echo.New()

 e.Use(middleware.Logger())
 e.Use(middleware.Recover())
 e.Use(middleware.CORS())
 e.Use(otelecho.Middleware("${PROJECT}"))

 e.GET("/echo", echoHandler)
 e.GET("/health", healthHandler)

 go func() {
 if err := e.Start(":8080"); err != nil && err != http.ErrServerClosed {
 e.Logger.Fatal("shutting down the server")
 }
 }()

 quit := make(chan os.Signal, 1)
 signal.Notify(quit, os.Interrupt)
 <-quit
 ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
 defer cancel()
 if err := e.Shutdown(ctx); err != nil {
 e.Logger.Fatal(err)
 }
}

Step 3: Infrastructure Template with YAML

Here’s how you define the infrastructure using Pulumi YAML. You can instantly spot one of the features of Pulumi Components. We consume the micorservice-component in YAML without knowing that it was authored in a different language. This is the power of Pulumi Components: write once, consume anywhere.

And we see another feature of Pulumi Components: As it is a first-class citizen, we can embed the component and add additional resources like auto-scaling policies, alarms, and more directly in the YAML file. This allows us to create a complete golden path for deploying Go microservices on AWS ECS with auto-scaling capabilities.

# Pulumi.yaml
name: ${PROJECT}
description: ${DESCRIPTION}
runtime: yaml
packages:
 component-microservice: https://github.com/smithrobs/component-microservice@v1.0.1
template:
 description: A template for deploying a Go microservice on AWS ECS with auto-scaling capabilities.
resources:
 microserviceComponent:
 type: component-microservice:MicroserviceComponent
 properties:
 appPath: ./src
 port: 8080
 containerName: ${PROJECT}
 ecsTarget:
 type: aws:appautoscaling:Target
 name: ecs_target
 properties:
 maxCapacity: 4
 minCapacity: 1
 resourceId: service/${microserviceComponent.clusterName}/${microserviceComponent.serviceName}
 scalableDimension: ecs:service:DesiredCount
 serviceNamespace: ecs
 ecsPolicyUP:
 type: aws:appautoscaling:Policy
 name: ecs_policy_up
 properties:
 name: ecs_policy_up
 policyType: StepScaling
 resourceId: ${ecsTarget.resourceId}
 scalableDimension: ${ecsTarget.scalableDimension}
 serviceNamespace: ${ecsTarget.serviceNamespace}
 stepScalingPolicyConfiguration:
 adjustmentType: ChangeInCapacity
 cooldown: 60
 metricAggregationType: Maximum
 stepAdjustments:
 - metricIntervalUpperBound: 0
 scalingAdjustment: 1
 ecsPolicyDown:
 type: aws:appautoscaling:Policy
 name: ecs_policy_down
 properties:
 name: ecs_policy_down
 policyType: StepScaling
 resourceId: ${ecsTarget.resourceId}
 scalableDimension: ${ecsTarget.scalableDimension}
 serviceNamespace: ${ecsTarget.serviceNamespace}
 stepScalingPolicyConfiguration:
 adjustmentType: ChangeInCapacity
 cooldown: 60
 metricAggregationType: Maximum
 stepAdjustments:
 - metricIntervalLowerBound: 0
 scalingAdjustment: -1
 serviceCPUHighUtilization:
 type: aws:cloudwatch:MetricAlarm
 name: service_cpu_high_utilization
 properties:
 name: service_cpu_high_utilization
 comparisonOperator: GreaterThanOrEqualToThreshold
 evaluationPeriods: 2
 metricName: CPUUtilization
 namespace: AWS/ECS
 period: 60
 statistic: Average
 threshold: 80
 alarmActions:
 - ${ecsPolicyUP.arn}
 dimensions:
 ClusterName: ${microserviceComponent.clusterName}
 ServiceName: ${microserviceComponent.serviceName}
 serviceCPULowUtilization:
 type: aws:cloudwatch:MetricAlarm
 name: service_cpu_low_utilization
 properties:
 name: service_cpu_low_utilization
 comparisonOperator: LessThanOrEqualToThreshold
 evaluationPeriods: 2
 metricName: CPUUtilization
 namespace: AWS/ECS
 period: 60
 statistic: Average
 threshold: 10
 alarmActions:
 - ${ecsPolicyDown.arn}
 dimensions:
 ClusterName: ${microserviceComponent.clusterName}
 ServiceName: ${microserviceComponent.serviceName}

outputs:
 publicUrl: ${microserviceComponent.publicUrl}

Step 4: Documentation and Examples

Add a README.md file to your template directory to provide clear instructions on how to use the template, including prerequisites, configuration options, and example commands:

# Go Microservice Golden Path

A **golden path** template that gets your Go microservice from code to production on AWS in minutes, not weeks.

## What You Get

This golden path provides everything your development team needs to deploy production-ready Go microservices:

✅ **Production-ready Go microservice** with Echo framework and OpenTelemetry tracing
✅ **AWS infrastructure that scales** - ECS with auto-scaling from 1-4 instances
✅ **Security by default** - Hardened containers, IAM roles, security groups
✅ **Monitoring built-in** - Health checks, load balancer monitoring, CloudWatch alarms
✅ **One-command deployment** - `pulumi up` handles everything
✅ **No AWS expertise required** - Complex ECS setup abstracted away

## For Development Teams: What to Expect

### Your Experience

**Day 1: Getting Started**

- Clone this repo, run `pulumi up`
- Your service is live on AWS in 5-10 minutes
- Public URL provided automatically - no manual setup needed

**Day 2-N: Development Workflow**

- Write your Go code in `microservice/main.go`
- Test locally with `go run main.go`
- Deploy changes with `pulumi up`
- AWS automatically rebuilds and redeploys your container

**Production Operations**

- Service automatically scales with CPU load (80% up, 10% down)
- Health checks ensure unhealthy containers are replaced
- Load balancer distributes traffic across healthy instances
- Distributed tracing helps debug issues across services

### What's Handled For You

You **don't** need to learn or configure:

- ECS clusters, services, and task definitions
- Application Load Balancers and target groups
- Auto-scaling policies and CloudWatch alarms
- Security groups and IAM roles
- Container registries and image building
- Health check configuration

You **do** focus on:

- Writing your Go application logic
- Adding your business endpoints
- Testing your service locally
- Deploying with confidence

Making Templates Available in Pulumi IDP

Publishing your template to Pulumi IDP enables true self-service. Head to Settings → Integrations → Organization Template Sources and add your template repository.

Once published, developers can discover and deploy it directly from the IDP interface.

Now developers can deploy through multiple interfaces:

CLI Deployment:

pulumi new https://github.com/myorg/go-microservice-boilerplate

No-Code Deployment: Navigate to Pulumi IDP → Templates → Deploy with Pulumi → Configure and launch

Pulumi Deployment: Navigate to Pulumi IDP → Templates → Deploy with Pulumi → Configure and launch

Build golden paths with Pulumi

Package your infrastructure patterns into reusable components and templates, publish them to a private registry, and let teams deploy production-ready stacks in minutes.

Get started

Best Practices for Reusable Infrastructure Components and Templates

Well-designed components and templates are the foundation of scalable, self-service infrastructure. These best practices ensure your abstractions are maintainable, discoverable, and production-ready.

1. Design for Day 2 Operations from Day 1

Think beyond deployment. A golden path must also support ongoing operations:

How will teams safely update their infrastructure?
What happens during scaling events?
How do you handle disaster recovery?
What metrics and logs are needed for troubleshooting?

2. Expose Complexity Progressively

Provide sensible defaults that work, while enabling customization for advanced use cases:

interface ComponentArgs {
 // Required - what users must provide
 appName: string;

 // Common customizations with good defaults
 instanceType?: string; // default: "t3.micro"
 replicas?: number; // default: 2

 // Advanced options for power users
 networkConfig?: NetworkConfig;
 securityPolicies?: SecurityPolicy[];
 customMetrics?: MetricDefinition[];
}

4. Use Semantic Versioning Everywhere

Clear versioning for both components and templates indicate stability and reliability:

Major versions (1.0.0 → 2.0.0): Breaking changes
Minor versions (1.0.0 → 1.1.0): New features, backward compatible
Patch versions (1.0.0 → 1.0.1): Bug fixes

This lets teams adopt updates with confidence and control.

5. Test Your Abstractions

Don’t ship black boxes. Create automated tests to validate key functionality and resource creation. Focus on:

Smoke tests that validate resource existence
Output validation to ensure correctness
Integration tests to confirm end-to-end behavior

import { expect } from "chai";
import * as pulumi from "@pulumi/pulumi";

describe("MicroserviceComponent", () => {
 it("should create required resources", async () => {
 const component = new MicroserviceComponent("test", {
 appPath: "./test-app",
 port: 3000
 });

 const resources = await pulumi.runtime.allResources();
 expect(resources).to.include("aws:ecs/cluster:Cluster");
 expect(resources).to.include("aws:ecs/service:Service");
 expect(resources).to.include("aws:lb/loadBalancer:LoadBalancer");
 });
});

How to Measure Golden Path Success

Golden paths aren’t complete until they deliver measurable value. Use these KPIs to assess performance and drive iteration:

Adoption Metrics

Template usage rate: Percentage of new projects using golden paths
Component reuse: Number of stacks consuming shared components
Time to first deployment: Deployment time and frequency from code to production

Quality Metrics

Security compliance rate: Percentage of deployments passing security policies
Stability: Deployment incident frequency
Mean time to recovery (MTTR): Time to recover from production issues

Developer Experience Metrics

Developer satisfaction: Survey teams about their platform experience
Support ticket volume: Ticket volume related to infrastructure
Contribution: Number of PRs or issues submitted to platform templates

Real-World Results: Success Stories

Organizations using golden paths report significant improvements in speed and reliability:

Snowflake reduced deployment time from 1.5 weeks to less than a day
Mercedes-Benz decreased infrastructure provisioning from weeks to minutes
Starburst Data cut deployment time from 2 weeks to 3 hours

These results show that golden paths are not just developer tools, they’re a competitive advantage. To learn more, download the whitepaper: The Golden Path to Cloud Success: Your IDP Roadmap.

Common Pitfalls and How to Avoid Them

Pitfall 1: Over-Abstraction

Problem: Creating components so abstract they’re unusable Solution: Start with concrete use cases, then generalize based on actual patterns

Pitfall 2: Insufficient Escape Hatches

Problem: Golden paths become golden cages Solution: Always provide ways to extend or override default behavior

Pitfall 3: Poor Versioning Strategy

Problem: Breaking changes without migration paths Solution: Maintain backward compatibility and provide clear upgrade guides

Pitfall 4: Lack of Ownership

Problem: Templates become orphaned and outdated Solution: Assign clear ownership and establish maintenance schedules

The Future of Golden Paths: AI and Beyond

The future of golden paths is intelligent, cross-cloud, and fully integrated with modern workflows.

AI-Enhanced Templates

Imagine templates that adapt based on your application’s actual behavior, automatically tuning resources and configurations for optimal performance and cost.

Cross-Cloud Portability

Components that abstract not just resources but entire cloud providers, enabling true multi-cloud golden paths.

GitOps-Native Workflows

Templates that include not just infrastructure but complete GitOps pipelines, from code commit to production deployment.

Your First Steps to Golden Paths

Ready to get started? Here’s your action plan:

Audit Current Patterns: Document the infrastructure patterns your teams use most frequently
Build one reusable component: Pick your most common pattern and build a reusable component
Create Your First Template: Build a complete golden path for one project type
Gather Feedback: Deploy with a pilot team and iterate based on their experience
Scale Gradually: Expand your component library and template catalog based on demand
Measure and Iterate: Track adoption and continuously improve based on metrics

Conclusion: From Fragmentation to Flow

Golden paths aren’t about restricting creativity or enforcing rigid standards. They’re about removing friction from the development process, encoding expertise into reusable patterns, and empowering developers to move at the speed of business.

By building a library of components and templates, you transform your Internal Developer Platform from a collection of tools into a force multiplier for your entire engineering organization. You give developers the gift of not having to solve solved problems, while maintaining the flexibility to innovate where it matters.

Start small. Solve one problem well. Expand from there. With the right foundation, your developer platform will evolve into a system of self-service, speed, and stability.

Ready to Build Your Golden Paths?

Code examples: Explore golden path examples on GitHub
Build with Components: Pulumi Components documentation
Enable Self-Service: Pulumi IDP documentation
Join the Community: Connect with platform engineers in our Pulumi Slack

Next in our series: Policy as Code for Safer IDPs. You will learn how to add automated guardrails that make sure every deployment meets your security and compliance standards without slowing down development.

How to Build an Internal Developer Platform: Strategy, Best Practices, and Self-Service Infrastructure

Mitch Gerdisch — Thu, 14 Aug 2025 05:13:02 +0200

Welcome to the first post in our IDP Best Practices series. In this guide, we’ll walk through the strategic foundations for designing an Internal Developer Platform that empowers developers without sacrificing governance, security, or operational control.

At Pulumi, we’ve worked with hundreds of teams facing the same core challenge: How do you give developers the infrastructure access they need, while maintaining the governance and security your organization requires?

That tension is at the heart of every IDP conversation. Teams want to move faster and innovate, but also need to stay compliant, control costs, and maintain operational stability.

The good news? You can do both, with a clear strategy and the right approach. This series shares proven best practices for designing, building, and scaling IDPs using Pulumi.

These lessons come from real-world implementations across industries and company sizes—and are built to grow with you.

Understanding the Platform Engineering Layers in Your Internal Developer Platform

When we work with customers on their platform strategy, we often start by referring to the “platform engineering layer cake.” Here is a quick walkthrough of each layer and how Pulumi IDP constructs fit this layer cake approach:

Layer 1: Infrastructure Layer - This is your raw cloud resources: VMs, databases, networks, storage. These are the fundamental building blocks that exist in AWS, Azure, GCP, and other cloud providers.

Layer 2: Platform Layer - This is where Pulumi Components live. Components take those raw infrastructure resources and package them into higher-level abstractions that encapsulate best practices, security policies, and organizational standards. For example, instead of manually configuring 15 different AWS resources to create a secure web application, you create a component that handles all that complexity and exposes just the configuration options that matter to your developers.

Layer 3: Developer Experience Layer - This is where Pulumi Templates and the Private Registry come into play. Templates provide ready-to-deploy patterns that developers can customize, while the private registry makes everything discoverable and manageable at scale.

In our workshop, we focused specifically on those top two layers because that’s where the transformation happens. This is where you turn raw infrastructure into something developers can actually use productively without becoming infrastructure experts themselves.

Here’s how this maps to what you’re actually building:

Infrastructure Layer → Raw Cloud Resources

Your foundation is still the same: AWS EC2 instances, Azure Virtual Networks, GCP Cloud Functions. Pulumi’s providers give you access to these resources using real programming languages instead of YAML or proprietary DSLs.

Platform Layer → Pulumi Components

This is where you create your reusable building blocks. A component might encapsulate:

A complete microservice infrastructure pattern (load balancer, auto-scaling group, database, monitoring)
A secure data pipeline (storage, processing, access controls, encryption)
A standardized Kubernetes application deployment (ingress, service, deployment, secrets management)

Components are written once in any supported language (TypeScript, Python, Go, .NET, Java) but can be consumed by teams using any of these languages. This is crucial. Your platform team might write components in Go, but your application teams can consume them in Python or TypeScript.

Developer Experience Layer → Templates + Private Registry

Templates are complete, deployable projects that use your components. They provide:

Getting started patterns: “Deploy a new microservice”
Reference architectures: “Deploy a three-tier web application”
Environment patterns: “Set up a complete dev/staging/prod pipeline”

The Private Registry is your distribution mechanism. It’s where components and templates become discoverable, versioned, and manageable. Developers don’t need to hunt through Git repositories or Slack channels to find the right building blocks. Everything is catalogued, documented, and accessible through familiar package management workflows.

IDP Example: Building a Web Application Platform Step-by-Step

Let’s make this concrete with an example from our workshops. Say you want to enable teams to deploy secure web applications:

Infrastructure Layer: You need an Application Load Balancer, EC2 Auto Scaling Group, RDS database, VPC with proper subnets, Security Groups, IAM roles, CloudWatch monitoring, and S3 bucket for static assets. That’s about 20+ individual AWS resources that need to be configured correctly and securely.

Platform Layer (Components): You create a WebApplication component that encapsulates all this complexity. Your component exposes simple configuration options like:

const app = new WebApplication("my-app", {
 instanceType: "t3.medium",
 minSize: 2,
 maxSize: 10,
 databaseSize: "small",
 environment: "production"
});

Behind the scenes, your component handles all the security configurations, networking setup, monitoring, and best practices. You publish this component to your private registry.

Developer Experience Layer (Templates): You create templates like:

web-app-starter: A simple web application using your WebApplication component
microservice-template: API service with database and monitoring
full-stack-template: Frontend, backend, and database in a complete environment

Developers can now run pulumi new web-app-starter and get a production-ready, secure web application in minutes instead of weeks. The template uses your component, which handles all the underlying infrastructure complexity.

This layered approach solves the core challenge we see every platform team struggling with: how do you provide developers with the infrastructure they need without creating operational bottlenecks or compromising security standards.

Why Internal Developer Platforms Matter in 2025 and Beyond

While DevOps brought us incredible advances in how we ship and maintain software, we’ve watched many organizations struggle with the unintended consequences: tool sprawl, Day 2 operational pain, and developers who are drowning in infrastructure complexity.

That’s why platform teams have emerged. You exist to solve these challenges by building tools and workflows that enable your internal customers (the developers) to provision infrastructure and deploy software without getting blocked. But here’s the thing I’ve learned from working with hundreds of platform teams: success isn’t just about the tools you choose. It’s about the strategy behind how you implement them.

5 Core Components of a Successful Internal Developer Platform

Through years of working with platform teams and analyzing what separates successful IDPs from those that struggle to gain adoption, we’ve identified five essential components that every platform strategy needs. These aren’t theoretical concepts. They’re battle-tested patterns that work across organizations of all sizes and industries.

1. Abstractions

Let me be clear about something: abstractions aren’t about dumbing down infrastructure for developers. The abstraction is really where you’re hiding the complexity of the underlying infrastructure from your end users, but you’re doing it intentionally to provide appropriate interfaces for different personas in your organization.

We’ve seen too many platform teams get this wrong by either over-abstracting (creating black boxes that developers can’t customize) or under-abstracting (exposing too much complexity). The sweet spot is creating component resources that encapsulate your infrastructure patterns and best practices into reusable building blocks that can be consumed across different programming languages and deployment scenarios, while still providing escape hatches when needed.

2. Blueprints

Blueprints are your templatized, well-architected patterns that developers can use to bootstrap their infrastructure. But here’s what I want you to understand about blueprints: they’re not just starting points that you throw over the wall to developers. They’re carefully designed patterns that embody your organizational best practices and architectural decisions.

When we see successful platform teams, their templates serve as both accelerators and guardrails. They give developers a fast path to production-ready infrastructure while ensuring that everything they deploy follows your organization’s standards. That’s the power of good blueprint design: speed and compliance working together, not against each other.

3. Workflows

Here’s where many platform teams get trapped: they try to build one workflow to rule them all. But in our experience working with organizations across every industry, you need to support multiple consumption patterns because you have multiple types of users. You need:

No-code workflows for users who want point-and-click deployment (think product managers or junior developers who just need to spin up a database)
Low-code workflows using tools like Pulumi YAML for configuration-driven infrastructure (perfect for developers who understand infrastructure but don’t want to write Go or TypeScript)
Full-code workflows for developers who need maximum flexibility and want to leverage the full power of general-purpose programming languages

The key insight? These aren’t three different platforms. They’re three different interfaces to the same underlying components and templates.

4. Security guardrails

This is probably the most important distinction we can share with you: you want to have guardrails, not gates. Too many platform teams create approval processes and manual checkpoints that completely undermine the self-service promise they’re trying to deliver.

Guardrails are different. They start with the best practices and security built into the reusable abstractions. But even the most secure solutions need a safety net. This safety net is policy-as-code that runs automatically, preventing violations before deployment happens. When we work with customers on implementing Pulumi CrossGuard, we’re not creating new friction. We’re embedding security, compliance, and cost controls directly into the deployment process. The developer gets immediate feedback, and you get the assurance that nothing goes to production without meeting your standards.

5. Self-service

Everything we’ve talked about so far builds toward this moment: enabling developers to provision and manage infrastructure independently. But self-service isn’t just about providing a UI. It’s about carefully orchestrating all the other components to create experiences that are both powerful and intuitive.

When we walked through the demo in our workshop, what you saw was the culmination of thoughtful platform design. Developers can discover available services, deploy infrastructure through whatever interface makes sense for them, and manage their resources over time, all without requiring tickets or manual intervention from platform teams. That’s the promise of a well-designed IDP: developer autonomy without operational chaos.

Put your IDP strategy into practice

Build reusable components, golden paths, and self-service workflows on Pulumi, and give developers autonomy while you keep governance and costs in check.

Get started

How to Implement Your IDP Strategy

Supporting Different Personas in Your Internal Developer Platform Design

One of the biggest mistakes we see platform teams make: they try to build for their most sophisticated users first. But here’s what I’ve learned from working with hundreds of organizations: you need to think about all your personas from day one.

In most organizations, you’re serving:

Infrastructure engineers who want full control and flexibility (they’ll use the full-code workflows)
Application developers who need infrastructure but prefer higher-level abstractions (perfect for low-code approaches)
Non-technical users who want simple, point-and-click deployment (your no-code users)

The breakthrough insight that separates successful platform teams from struggling ones is this: all these personas can be served by the same underlying components and templates, just consumed through different interfaces. You don’t need to build three different platforms—you need to build one platform with three different consumption models.

Why a Private Registry Is Key to IDP Adoption

Here’s something we always tell platform teams: if you don’t solve the discoverability problem, your beautiful components and templates will sit unused in Git repositories where no one can find them. That’s why the private registry isn’t just a nice-to-have. It’s the foundation that makes everything else work.

When you establish a private registry as your source of truth for components, templates, providers, and policies, you’re solving two critical problems at once. First, discoverability: developers can actually find and explore what’s available. Second, lifecycle management: you can see where each package is being used, track version drift, and understand the impact of changes before you make them.

The workflow is beautifully simple: you publish standardized building blocks with a single pulumi publish command, and developers discover and consume these assets through familiar package management workflows they already know.

Aligning Your IDP to Organizational Context and Services

One pattern we see in every successful IDP implementation is that the platform reflects how the organization actually works. Your developers don’t think in terms of individual stacks or resources. They think in terms of services, applications, and business functionality.

That’s why we built Pulumi Services - to let teams logically group stacks and ESC environments in ways that make sense to your organization. Pulumi Services improve organization by helping your team model infrastructure in a way that’s familiar - simplifying Day 2 operations by providing additional context like links to relevant dashboards, Slack channels and more. When something breaks at 2 AM, your on-call engineer needs to understand dependencies, track usage, and manage infrastructure in the context of the business services that are actually impacted.

How to Measure the Success of Your Internal Developer Platform

Here’s something that might surprise you: the most successful platform teams we work with don’t measure their success by how many cool features they’ve built. They measure success by the business outcomes they’re driving:

Reduced time-to-market through faster infrastructure provisioning (because at the end of the day, your platform exists to help the business move faster)
Improved developer productivity by eliminating infrastructure bottlenecks (happy developers ship more features)
Enhanced security posture through consistent policy enforcement (security that doesn’t slow people down is security that actually gets used)
Lower operational overhead through standardization and automation (because your platform team’s time is better spent on innovation than toil)

The numbers speak for themselves. We’ve worked with customers like Snowflake who reduced deployment times from one and a half weeks to less than a day. Starburst Data cut their infrastructure deployments from two weeks to just three hours. These aren’t vanity metrics. They’re business transformations.

IDP Strategy: 5 Key Steps to Build a Strong Foundation

If you’re just starting your IDP journey, here’s the roadmap we give to every platform team we work with:

Start with user research: Before you write a single line of code, go talk to your developers. Understand your different personas and their actual needs, not what you think they need. Do your users like to work from the command line? Do they prefer interacting with a UI?
Identify common patterns: Look for infrastructure patterns that get repeated across teams. These are your goldmine—the patterns that, once abstracted and templatized, will provide immediate value.
Begin with components: Build reusable infrastructure building blocks before you worry about fancy UIs or workflow orchestration. You need solid foundations before you can build the house.
Implement progressive disclosure: Give people simple interfaces for common use cases, but always provide escape hatches for complex scenarios. The moment your abstraction becomes a prison, developers will route around it.
Measure and iterate: Track adoption and gather feedback religiously. Your platform is only as good as its adoption rate, and adoption only happens when you’re solving real problems for real people.

The Future of IDPs and Platform Engineering

Trends come and go, but platform engineering is here to stay. It’s not just another tool or methodology. This is how modern organizations scale infrastructure to match the speed of innovation. The companies that win are the ones that treat their Internal Developer Platform (IDP) as a strategic foundation, not just a tool.

Your job isn’t just to manage infrastructure and choose the right tools - it’s to enable developers to move faster while keeping governance, security, and costs under control. Get that balance right, and your platform becomes a force multiplier across your entire engineering department.

The fastest way to get there? Focus on these five essential components:

Abstractions
Blueprints
Workflows
Security guardrails
Self-service

Together, they turn your IDP into more than just infrastructure automation, they create a platform that developers trust and your business relies on.

Ready to Build a Successful IDP that Scales?

Pulumi makes it easy to go from static IaC to dynamic, self-service infrastructure with real programming languages and built-in guardrails.

Explore Pulumi IDP or sign up for free to get started today.

AI-Assisted Infrastructure as Code with Pulumi's Model Context Protocol Server

Mikhail Shilkov — Tue, 08 Apr 2025 00:00:00 +0000

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

Infrastructure as Code (IaC) has revolutionized how we manage cloud resources, but navigating complex cloud provider APIs, writing boilerplate code, and iterating through deployment cycles can still be time-consuming. Pulumi offers a fantastic developer experience using familiar programming languages. But what if we could make it even faster and more intuitive by integrating powerful AI assistants directly into the development loop?

This is where the Pulumi Model Context Protocol (MCP) Server integration shines. MCP is a specification that allows language models (like the AI in your coding assistant) to interact with external tools and data sources in a structured way. By connecting AI-powered code assistants with Pulumi’s CLI and registry via MCP, we can bring real-time resource information and infrastructure management directly into the development environment, dramatically reducing friction and accelerating workflows.

Several AI coding assistants like GitHub Copilot, Anthropic’s Claude Code, Windsurf and others are rapidly evolving; this post will use Cursor (an AI-first code editor) to demonstrate a real-world example of this synergy in action.

Setting up the Pulumi MCP Integration in Cursor

Before diving in, you typically need to configure your AI assistant to communicate with the Pulumi MCP server. For Cursor, you can do so by clicking the following button:

The assistant can now leverage Pulumi tools seamlessly. These tools are specific actions enabled by the MCP server—like searching the Pulumi Registry or running a pulumi command—allowing the assistant to gather information or interact with your Pulumi project.

For detailed setup instructions for Cursor, Claude Code, Windsurf, Claude Desktop, and other AI assistants, see the Pulumi MCP Server documentation.

The Goal: Provisioning an AKS Cluster

Our objective for this walkthrough is to provision a temporary Azure Kubernetes Service (AKS) cluster for a short experiment using Pulumi and TypeScript. We need the cluster created with minimal fuss and its kubeconfig exported for access.

The Traditional Approach

The conventional method for this task involves significant context switching: searching Azure and Pulumi documentation in a browser, writing code in an editor, running commands (pulumi preview, pulumi up) in a terminal, and manually correlating information between these different environments. This process can be slow, requires deep knowledge recall (or constant lookups), and is prone to errors during the manual translation from documentation to code.

The AI Assistant + Pulumi MCP Approach

Let’s walk through how the same task unfolds much more efficiently using an AI assistant integrated with Pulumi MCP.

Understanding the AI Assistant + Tool Interaction

Before detailing the steps, it’s helpful to understand the user interface flow when the AI assistant uses integrated tools like those provided by the Pulumi MCP integration. Typically:

The developer provides a prompt or instruction in the chat interface within their editor.
The AI assistant analyzes the request and determines that it needs specific information (like resource properties) or needs to perform an action (like running pulumi preview).
The assistant indicates it will use a specific tool (e.g., pulumi_registry_listResources, pulumi_cli_preview). This often appears as a distinct UI element in the chat, showing the tool name and parameters being used.
The tool executes via the MCP server, interacting with the Pulumi CLI or registry as needed.
The output or result from the tool (e.g., registry listings, preview results, deployment errors) is displayed directly in the chat interface.
The AI assistant processes this output and uses it to continue the task – either by generating code, providing an answer, or deciding on the next step (like suggesting a fix for an error).

This tight loop keeps the developer focused within their editor environment, minimizing disruptions.

Step-by-Step Walkthrough

Here’s how the AKS provisioning task played out:

The Request

The developer starts with a natural language request to the AI assistant within the editor:

“I have an empty Pulumi project with TypeScript. Please edit the program to provision an AKS cluster for me. It’s a temporary AKS cluster that I need for a short experiment, so I don’t need any particular configuration of it. Just export its kubeconfig when you are done. Please use the tools to lookup resource information and to run pulumi preview to make sure your code works. When done, run pulumi up for me.”

AI-Powered Resource Discovery

Instead of the developer manually searching docs, the AI assistant leverages the MCP integration.

The AI assistant first queries the Pulumi Registry via MCP to list resources within the Azure Native provider’s containerservice module (using the pulumi_registry_listResources tool). This immediately identified ManagedCluster as the relevant resource.
The AI assistant then requests detailed information specifically for ManagedCluster (using pulumi_registry_getResource), directly retrieving its required properties, descriptions, and structure without leaving the editor.

AI Code Generation & Editing

Using the information retrieved from the Pulumi Registry, the assistant generated the necessary TypeScript code. It defined the ManagedCluster resource with a basic configuration and used its code editing capabilities to insert the code directly into the developer’s index.ts file.

Integrated Validation

Before attempting a potentially time-consuming deployment, the assistant ran pulumi preview using the integrated CLI tool (pulumi_cli_preview). The preview output appeared directly in the chat.

The preview succeeded, showing the resources that would be created. The assistant then proceeded to run pulumi up using the integrated CLI tool (pulumi_cli_up). However, the deployment flagged an error: an incorrect attempt to interpolate a Pulumi Output<string> directly into a resource property.

AI-Assisted Debugging & Iteration

The assistant analyzed the error message from the tool:

It identified the incorrect string interpolation and proposed a fix (using pulumi.interpolate), applying it via the editing tool.
Self-correction: The first fix inadvertently introduced a circular dependency (caught by linters integrated into the editor and surfaced to the assistant). The assistant recognized this and further refined the code by removing the problematic property, correctly relying on Azure’s default behavior.
Another attempt to deploy using pulumi up failed, this time surfacing an Azure API error: the provided SSH key format was invalid.

Collaborative Problem Solving & Tooling

The developer, seeing the SSH key error, suggested using the Pulumi TLS package to generate a valid key dynamically.

The assistant used the integrated terminal tool to execute npm install @pulumi/tls.
It then edited the index.ts file again, incorporating the tls.PrivateKey resource and correctly using its publicKeyOpenssh output for the ManagedCluster’s Linux profile.

Successful Deployment

The subsequent pulumi preview showed the correct plan (including the new TLS key resource). The assistant then executed pulumi up, which completed successfully. The success message and resource summary appeared in the chat.

Accessing Outputs

Finally, retrieving the kubeconfig was trivial. The assistant used the stack output tool (pulumi_cli_stack_output kubeconfig) to fetch and display the configuration directly.

Why This Changes the Game

Integrating AI assistants with the Pulumi Model Context Protocol offers tangible benefits:

Reduced Context Switching: The entire workflow – discovery, coding, validation, deployment, debugging, output retrieval – happens primarily inside the editor. No more juggling browser tabs and terminal windows.
Accelerated Discovery: MCP integration provides immediate, context-aware access to Pulumi resource schemas and documentation snippets.
Faster Coding: AI generates boilerplate and resource definitions quickly based on registry data and natural language requests.
Tighter Feedback Loop: pulumi preview and pulumi up results are instantly available within the coding environment, enabling rapid iteration.
Intelligent Assistance & Collaboration: The AI assists with debugging complex errors, incorporates developer suggestions, and leverages integrated tooling effectively.

Conclusion

The synergy between AI coding assistants and the Pulumi Model Context Protocol (MCP) integration creates a remarkably efficient environment for Infrastructure as Code development. By bringing cloud resource knowledge, code generation, and the Pulumi CLI workflow directly into the editor, developers can build, deploy, and iterate on their infrastructure faster and with significantly less friction.

Whether you use Cursor, Copilot, Claude Code, Windsurf, or another emerging AI tool, integrating it with the Pulumi MCP server offers a glimpse into the future of streamlined, intelligent IaC development, ultimately boosting productivity and improving the overall developer experience.

Your Perfect Infrastructure May Not Be So Perfect

Simen A. W. Olsen — Mon, 16 Dec 2024 10:43:07 +0000

Guest Article: Simen A. W. Olsen from Bjerk, is here to share his lessons learned on why designing the perfect architecture for your future needs might be a mistake

I remember standing in front of our engineering team in 2018, proudly presenting what I believed was the future-proof architectural design for our new distributed system. The diagrams were immaculate, the technology choices were cutting-edge, and the scalability patterns were ready for any possible future scenario.

I was basically the Leonardo da Vinci of system design… if Leonardo had been really into Kubernetes and had a concerning addiction to coffee. But six months later, that “future-proof” architecture had become a constraint rather than an enabler, and my masterpiece was looking more like a finger painting done by a caffeinated raccoon.

This experience taught me something crucial: trying to build the perfect system that anticipates every future need is often worse than creating a system designed to change quickly. It’s like trying to predict what your kid will want to be when they grow up and pre-buying all the necessary equipment. Congrats, you now own a space suit, a stethoscope, and a dragon costume — and they decided to become a software engineer anyway.

The Over-Planning

Many teams fall into a common trap: they try to design systems that anticipate every possible future requirement. This happens even in agile teams, where we convince ourselves we need to “get the architecture right” before we can start iterating. You know, because nothing says “agile” like spending three months in a room drawing boxes and arrows while muttering “microservices” under your breath like it’s a magic spell.

In 2008, Netflix faced a choice: build the perfect data center that could handle all their anticipated future needs, or move to the cloud with a simpler architecture that could evolve. They chose the latter, focusing on making their system easy to change rather than trying to make it perfect. Smart move — unlike those my past self made who probably would’ve insisted on building a data center capable of streaming to Mars, just in case Elon asked nicely.

You might also like:

The Core Principles of Change-Ready Architecture

Through both failures and successes, I’ve identified three principles that define truly adaptable architecture:

Embrace simplicity. I think it makes sense to start with the simplest architecture that could possibly work for your current needs. Complexity should be earned, not presumed. If your architecture diagram looks like a plate of spaghetti that’s been hit by lightning, you might be doing it wrong.
Make change cheap. Instead of trying to avoid change, make it inexpensive. This means investing in automated testing, continuous deployment, and monitoring. When change is cheap, you don’t need to fear it.
Learn through action. Rather than trying to predict the future, build mechanisms that help you learn quickly about real needs. It includes feature toggles (Protip: Try Unleash.), A/B testing, and robust monitoring of how your system is actually being used. You know, actual data, not just what that one loud guy in planning insists will definitely happen.

The rise of AI and machine learning systems has made one thing clear: we can’t predict how our systems will need to evolve. The most successful teams aren’t those that try to build the perfect AI architecture upfront, but those that can rapidly experiment and adapt their systems based on real-world feedback.

The biggest pushback I hear is, “But what if we need to scale?” or “What about future requirements?” These fears often drive teams to over-architect their solutions. But here’s the reality: the cost of changing a simple system is usually lower than the cost of maintaining an over-engineered one. The key is understanding that good architecture isn’t about predicting the future — it’s about making future changes as painless as possible.

Conclusion

That over-engineered system I was so proud of in 2018? Its most significant flaw wasn’t in what it got wrong about the future — it was that it tried too hard to be right about the future in the first place. It’s like bringing a fully packed suitcase to a first date. Today, I know that the best architecture isn’t one that anticipates every need, but one that makes it easy to respond to needs as they emerge.

The next time you’re tempted to design for every possible future scenario, remember: the goal isn’t to build a perfect system, but to build one that’s perfectly easy to change. And if someone tells you they’ve designed the perfect future-proof architecture, they’re either lying, or they’ve discovered time travel — and in that case, they should be sharing lottery numbers, not system designs.

You might also like:

Integrating DevOps and Security in Platform Engineering

Sara Huddleston — Wed, 11 Dec 2024 07:41:06 +0000

Platform engineering has become essential for mid-to-large organizations, moving beyond a DevOps trend. Gartner predicts that by 2026, 80% of software companies will have internal platform services to streamline development. The goal is to empower developers with self-service tools while maintaining security, compliance, and reliability through DevSecOps practices.

At PulumiUP Europe 2024, experts shared insights on aligning DevOps with security to build scalable, secure platforms:

Jess Mink, Sr. Director of Platform Engineering at Honeycomb
Kief Morris, Global Head of Infrastructure Engineering at ThoughtWorks
Lindsay Jack, VP of Engineering & Security at Snyk
Nariman Aga-Tagiyev, Application Security Architect at WiseFrog Security
Komal Ali, Engineering Manager at Pulumi

The panel discussed key strategies, challenges, and pillars of successful platform engineering.

The Core Pillars of Platform Engineering

Platform engineering teams comprise multiple professionals with many responsibilities and focus areas. According to our panel of experts, the core pillars of platform engineering include:

Developer Experience (DevEx): Provide developers with the tools, frameworks, and abstractions they need to be productive and proactive without getting stuck in infrastructure or operational concerns.
Reliability and Scalability: Ensure that the platform and infrastructure can support the organization’s needs, with the ability to scale up or down as required.
Security and Compliance: Embed robust, accessible security and compliance frameworks into the development lifecycle while making it easy for developers to adhere to these policies.
Automation and Tooling: Leverage Infrastructure as Code (IaC) and automation to enforce standardized processes and consistency and reduce cognitive load and manual effort.
Observability and Monitoring: Provide visibility into the platform’s health and performance, delivering actionable insights that allow teams to identify and resolve issues quickly.

These pillars work together to create a platform that empowers developers to innovate and deliver value to the organization and customers while maintaining the necessary controls and safeguards.

As Jess Mink, Director of Platform Engineering at Honeycomb, explains:

“The goal of platform engineering is to help the company run smoother and faster and unlock things people didn’t know were possible […] We tend to focus on tools and software, but it’s really about people, processes, and tools. If you consider this, platforms are responsible for social and technical support across the organization. A common pitfall is building tools no one uses because you didn’t meet people where they are.”

Aligning DevOps and Security for Secure Platform Engineering

Integrating security into platform engineering ensures it becomes a proactive part of the development lifecycle and ensures that “DevSecOps” is not an afterthought but a core consideration.

Key best practices shared by the panel include:

Shift Left Security

One of the fundamental principles of DevSecOps is to “shift left” - that is, to integrate security earlier in the development process rather than waiting until the end. This can involve activities such as:

Incorporating security requirements and threat modeling into the initial design phase
Automating security scans and tests as part of the continuous integration (CI) pipeline
Providing developers with secure coding guidelines and tools to identify and remediate vulnerabilities

By addressing security concerns upfront, organizations can reduce the time and cost of remediating issues later in the development lifecycle.

Embrace Automation and Standardization

Consistency is key. Platform engineering teams should leverage automation and built-in safeguards and security processes. This may include:

Automating the provisioning of secure infrastructure and application environments
Implementing Infrastructure as Code (IaC) to define and manage infrastructure in a declarative, version-controlled manner
Standardizing security controls, policies, and configurations across the platform

By automating these tasks, platform engineering teams can reduce the risk of human error, improve visibility and auditability, and free up developers to focus on building features rather than managing infrastructure.

Prioritize Observability and Monitoring

Effective security requires visibility into the health and performance of the platform. Platform engineering teams should invest in robust observability and monitoring solutions to:

Detect and respond to security incidents and anomalies in real-time
Gain insights into the behavior and usage patterns of the platform
Identify and address vulnerabilities or misconfigurations proactively

By integrating security-focused monitoring and alerting into the platform, organizations can quickly identify and mitigate threats while also providing developers with the necessary context to understand and address security-related issues.

Foster a Culture of Collaboration

Platform engineering is often referred to as being the practical application of DevOps practices. Integrating security practices often requires a cultural shift towards collaboration and shared responsibility between all teams in development, operations, and security, thus the name DevSecOps. Platform engineering teams can facilitate this by:

Treating Platforms as Products
Involving security stakeholders in the design and planning of platform initiatives
Providing security training and education to developers to empower them to make informed decisions
Establishing communication channels and feedback loops between teams

As Kief Morris, Global Head of Infrastructure Engineering at ThoughtWorks, explains:

“There is a new way of thinking that is trying to avoid that “build it, and they will come mentality,” which leads to building it and nobody using it. One of the trends we are seeing is product thinking […]—using techniques like creating user personas of different types of users in the organization, conducting research to understand their needs, and talking with them.”

Breaking silos and fostering communication helps organizations build secure, scalable platforms that support the needs of developers, platform engineers, architects, and security teams.

Challenges of Integrating Security in Platform Engineering

While the benefits of integrating DevOps and security in platform engineering are clear, the journey has expected challenges. Our panel of experts highlighted several key obstacles that organizations may face:

Balancing Autonomy and Control

The primary goal is to empower developers to be more productive and proactive. However, this autonomy needs to be balanced with necessary security controls and governance measures.

Jess Mink points out the importance of achieving harmony between developer autonomy and operational control, stating:

“It’s a delicate balance - you want to make things easy for developers, but you also need to maintain the right level of control and security. It’s about finding the right abstractions and interfaces that give developers the freedom they need while still ensuring the platform remains secure and compliant.”

Driving Adoption and Changing Mindsets

Integrating security into the platform engineering workflow can often be met with resistance from developers accustomed to moving quickly and may view security as an obstacle to their productivity.

As Nariman, a Software Security Architect, notes:

“The challenge is not just about the tools or the technology - it’s about changing the mindset and getting people to understand the importance of security. You need to find ways to motivate developers and make them feel like they’re part of the solution rather than just imposing more rules and processes.”

Effective communication, education, and a focus on the business value of security are key to driving adoption and fostering a culture of shared responsibility.

Adapting to Evolving Needs and Technologies

As organizations grow and their technology stacks evolve, the demands on the platform engineering team can shift rapidly. Keeping up with these changes, while maintaining a secure and reliable platform, can be a significant challenge.

Lindsay Jack, VP of Engineering for the Platform Division at Snyk, explains:

“You might have a platform team that’s really good at a certain set of technologies, but then the organization starts moving in a new direction, and suddenly those skills don’t match up anymore. It’s about being agile and adaptable and making sure you have the right mix of skills and expertise to support the organization’s evolving needs.”

Fostering internal mobility, continuous learning, and a flexible, modular platform architecture can help platform engineering teams navigate these changes more effectively.

Measuring Success in Secure Platform Engineering

Measuring the success of a platform engineering initiative can be complex as it involves balancing a range of technical, operational, and business-oriented metrics but also considers human factors. According to our panel, some key metrics to include:

Developer Experience: Metrics such as developer satisfaction surveys, time-to-onboard new developers, and the number of self-service platform capabilities can provide insights into the effectiveness of the platform in supporting developer productivity and autonomy.
Reliability and Scalability: Monitoring service-level objectives (SLOs), incident response times, and the ability to handle increased traffic or user demands can help assess the platform’s reliability and scalability.
Security and Compliance: Tracking the number of security incidents, the ratio of security issues found during threat modeling versus post-deployment, and the adoption of security best practices can indicate the platform’s security posture.
Automation and Tooling: Metrics like the percentage of infrastructure provisioned through code, the frequency of platform updates, and the time saved through automation can demonstrate the platform’s operational efficiency.
Observability and Monitoring: Measuring the effectiveness of observability tools, the time to detect and resolve issues, and the quality of incident reports can provide insights into the platform’s overall health and performance.

As Jess Mink emphasizes, it’s important not just to collect these metrics but to use them to drive meaningful action and improvement:

“We look at all of those [metric categories] every quarter and write summaries that go up to the executive level. This creates visibility and a shared understanding of problems so that there’s room for movement and change in the right ways.”

The Future of Secure Platform Engineering

Software development and infrastructure management are evolving, and the role of platform engineering will only become more critical to support it. By integrating DevOps and security practices, platform engineering teams can create scalable, secure platforms that empower developers to be more productive and innovate, delivering business value.

Learn how Pulumi customers build secure, scalable platforms and empower their development teams:

Tivity Health: DevSecOps Game-Changer: Security Automation That Delivers Business Results
BMW Group: Unified and Programmatic Approach to Infrastructure Management at BMW Using Pulumi
Lemonade: How the DevOps team supports a much larger group of developers using by Pulumi to standardize infrastructure components and enforce best practices.

Discover platform engineering best practices in The Guide to Platform Engineering: 7 Steps to Get It Right.

Build secure, scalable platforms with confidence—get started with the Platform Engineering Workshop Series & Course

DevSecOps Game-Changer: Security Automation That Delivers Business Results

Sara Huddleston — Thu, 21 Nov 2024 07:16:30 +0000

Organizations are under constant pressure to deliver new products and features faster than ever. But speed alone isn’t enough—businesses must also navigate the complex challenges of ensuring security and managing infrastructure costs effectively.

Enter DevSecOps - the strategic integration of security practices into the DevOps workflow. By automating security processes, organizations can achieve improved speed, scalability, and business impact, all while ensuring security remains a priority.

Tivity Health, a leading health and fitness solutions provider, has embraced this DevSecOps approach using Pulumi, a modern infrastructure as code (IaC) platform. During PulumiUP 2024, David Giambruno, Tivity Health’s VP of Engineering and DevOps, shared how, by leveraging Pulumi, he led the transformation that continuously drives remarkable results in speed, cost savings, and security.

On this article:

The Beginning: From Data Center to the Cloud

Tivity Health’s journey began with a strategic decision to transition from a traditional data center environment to a cloud-native architecture. Rather than opting for a “lift and shift” approach, they made the bold choice to go directly to a cloud-native model, embracing the principles of DevSecOps along the way.

The need to automate infrastructure deployment and management was fundamental. Giambruno explained, “If you can’t automate it, we don’t need it.” This philosophy drove the team to seek out a solution that would not only streamline their operations but also empower their developers to focus on building innovative products for their customers.

Embracing Pulumi: The Power of Automation, Productivity, and Security

Tivity Health’s search for the right tool led them to Pulumi, a unified platform for all the infrastructure needs that allows teams to use general programming languages, such as TypeScript, Python, Java, and Go, to define and manage their cloud infrastructure. Giambruno and his team immediately recognized Pulumi’s ability to deliver on their key requirements:

Automation: Pulumi’s infrastructure as code (IaC) approach enabled Tivity Health to automate the deployment and management of its cloud environments, reducing the time and effort required for these tasks.
Productivity: using Pulumi’s general-purpose programming languages allowed developers to define, deploy, and manage infrastructure within their existing tools.
Security: Pulumi’s security features allowed Tivity Health to integrate security practices into its infrastructure deployment processes, reducing the risk of security breaches and ensuring compliance.
Cost Optimization: Pulumi’s ability to automate the spin-up and teardown of cloud environments on demand has led to significant cost reductions for Tivity Health.

Giambruno describes Pulumi as a “three-dimensional” tool, offering a versatile set of capabilities that have been instrumental in transforming Tivity Health’s operations. “The ability to use those dimensions in lots of different ways to do the automation is what really makes a difference to the teams,” he says.

Driving Business Impact Through Security Automation

By adopting Pulumi and DevSecOps automation, Tivity Health realized significant business benefits:

Speed and Agility: By automating its infrastructure deployment and management processes, Tivity Health has dramatically reduced the time required to spin up new environments or change existing ones. “We run it through automation and boom, it’s out, it’s done,” Giambruno says. This newfound speed and agility have empowered Tivity Health’s developers to focus on building products and features rather than getting bogged down in infrastructure-related tasks.
Cost Optimization: Tivity Health’s cloud-native approach and Pulumi’s automation capabilities have resulted in significant cost savings. The company estimates that its annual cloud spend has decreased from $9.5 million in its data center days to just $2 million—a staggering 79% reduction. These cost savings have allowed Tivity Health to redirect resources towards more strategic initiatives that drive business growth.
Improved Security and Compliance: Tivity Health’s DevSecOps strategy, anchored by Pulumi, has strengthened its security posture and compliance efforts. By integrating security directly into its infrastructure workflows, Tivity Health has improved its security posture. Automation ensures that security measures are enforced consistently across their cloud environments, reducing risks and improving compliance.

Fostering DevOps Culture Through Cross-Functional Collaboration

Using Pulumi provided more than technical benefits. It also enabled better collaboration between the development, security, and operations teams. By providing a common language and framework for infrastructure management, Pulumi has helped break down silos and align these traditionally disparate groups towards a shared goal of delivering secure, high-quality products faster.

Lessons Learned: Navigating the Cultural Shift

Transitioning to DevSecOps and cloud-native practices required a cultural shift at Tivity Health. Giambrono acknowledges that this cultural shift was not without its challenges. He emphasizes the importance of addressing the human element of change, offering the following advice for organizations embarking on a similar journey:

Start with a proof of concepts: Giambruno recommends beginning with a small-scale proof of concepts to demonstrate the capabilities and benefits of the new technologies and processes to help alleviate fears and build confidence among team members.
Empower developers and make them feel safe: By involving developers in the process and ensuring they feel comfortable with the new tools and workflows, Tivity Health gained user buy-in and overall support for the new DevSecOps approach.
Secure buy-in from business and financial stakeholders: Address the concerns of business and financial stakeholders early on, such as the impact on costs and the ability to deliver tangible results. This is crucial for securing the necessary support and resources.
Celebrate successes and build momentum: Giambruno highlights the importance of celebrating the team’s achievements along the way, even when there are bumps on the road, as this helps build enthusiasm and keep the momentum going.

The Future of DevSecOps and Pulumi at Tivity Health

As Tivity Health continues to refine its DevSecOps strategy, Giambruno is optimistic about the future. He envisions a world where the company’s ability to deploy new products and features rapidly will give Tivity Health a significant competitive edge, allowing it to better serve its customers and drive business growth.

Looking ahead, Giambruno is particularly enthusiastic about the potential of Pulumi’s AI-powered capabilities, which he believes will further streamline and optimize the company’s infrastructure management processes. “I’m super looking forward to the tests we’re going to do, like when we acquire someone and then taking them in, ‘Borg-ing’ them into our automation and seeing how much we can take out of their operating cost as fast as possible,” he says.

Tivity Health’s journey with Pulumi is a powerful example of how the right tool, DevSecOps strategy, and automation can drive tangible business results. By focusing on automation, security, and collaboration, organizations can achieve faster, cheaper, and better cloud deployments—putting them on the path to long-term success in an increasingly competitive, cloud-native world.

To learn more about Pulumi and how it can transform your software development and infrastructure management:

Get started with Pulumi Tutorials
Attend an upcoming workshop
Try out the Pulumi Neo code assistant to accelerate your infrastructure as code journey

Pulumi Patterns and Practices Platform (P3): Some Assembly Required

Troy Howard — Mon, 11 Nov 2024 00:00:00 +0000

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

Setting up an internal developer platform (IDP) can be a daunting task. There are a lot of tools out there that do some of the work for you, but none of them do all of it. Pulumi P3 is no different. Pulumi Patterns & Practices Platform (P3) is a reference architecture that we will be describing, and providing code for, through this series of articles.

We will never try to sell you on the idea that you can simply download a package, click next a few times, and achieve transformative success. That’s because any effective IDP will require some customization and integration to work within your environment.

Tools that purport to have it all figured out have only figured out how to manipulate you into a false narrative they have constructed in a vacuum, where all your organizational needs fit neatly into a few boxes they’ve decided on for you. And also charge you for. In addition to everything else you’re being charged for. Ultimately you’ll still need to build a lot yourself and these products rarely give guidance on how to do that.

When we first started hearing about our customers using Pulumi as an internal developer platform (IDP), we were frankly surprised, as our goals were primarily for Pulumi to be the best developer experience in infrastructure. But it makes sense. All the parts are there, some assembly required. Our goal with Pulumi Patterns and Practices Platform (P3) is to help with that assembly process.

Starting with our previous blog post on the topic, and continuing here, we are examining this use case, and attempting to formalize that into a collection of reusable components and some guidance on how you can skip the marketing pitches and pricing charts, and get straight to the hard work of building your own highly customized internal developer platform with Pulumi at its core.

Pulumi P3: Bill of Materials

Previously we identified the essential qualities of an effective IDP. Those were consistency, reproducibility, visibility, security and compliance, auditability, developer experience. In the last half of the post we discussed which parts of Pulumi could be used to meet those needs. That looks like:

Consistency: component resources, organization templates, drift detection
Reproducibility: stacks, deployments, versioned data
Visibility: Pulumi Insights, Pulumi Copilot
Security and Compliance: RBAC, GitHub Teams, SAML-SSO, Pulumi ESC, Pulumi Crossguard
Auditability: audit logging
Developer Experience: Python/Go/JavaScript/C#, popular IDE support, command-line tools, deeply hackable

That’s all great, and much of that is already built-into Pulumi without the need for you to do anything at all. So, what parts do you actually need to set up and configure? Here’s the bill of materials (BOM) to set up your own instance of Pulumi P3:

Bill of Materials:

Authentication and Identity Management:
- A GitHub organization that matches your Pulumi Cloud organization
- GitHub Teams users and roles that match your organizational structure and security needs
Secrets, Configuration, and Policy:
- Pulumi ESC environments to manage secrets across clouds
- Pulumi Crossguard policy packs that capture your company policies
Developer Experience:
- A set of reusable multi-language components for cross-cutting concerns/common services
- A set of organization templates that match your common use cases

Let’s go through each of those and briefly discuss what it looks like to set that up.

Authentication and identity management

We highly recommend using GitHub for code management. So much so that we have deeply integrated GitHub into Pulumi Cloud across a number of features. While we support alternatives such as GitLab, this will be the easiest and more feature-rich way to configure your platform.

In Pulumi Cloud, you have the ability to create organizations. A Pulumi Cloud organization can help you manage teams, roles, stacks, settings, and provide a dashboard across the entire organization. Pulumi Cloud also allows you to use a variety of identity providers to log in, including GitHub.

For simplicity’s sake, we suggest that you start with your GitHub organization. Create the GitHub organization, set up teams, and add members to those teams, assigning either admin or user roles to each member.

Next, in Pulumi Cloud, create an organization with exactly the same name as your GitHub organization, and choose GitHub as your identity provider. When a Pulumi organization is backed by a GitHub organization, then only members of that GitHub organization may be added to the Pulumi organization. Similarly, as soon as someone loses access to the GitHub organization, they will no longer have access to the Pulumi organization. You will also be able to import your GitHub teams directly into Pulumi Cloud. Then assign your users to the same roles in Pulumi Cloud teams as they have in the associated GitHub teams.

Figure: Mapping GitHub orgs, teams, and roles to Pulumi

Finally, you can map teams to stacks to grant access at specific permission levels. If you’re not familiar with Pulumi Stacks, a stack is a materialized instance of a specific set of cloud resources, as defined in a Pulumi program.

Pulumi ESC: Managing credentials, configuration, and other secrets

In order to deploy a stack you will need secrets such as cloud credentials and other configuration values that are provided to the deployment engine. Pulumi ESC is a secure system for managing secrets. They are organized by environments.

An example set of environments might look something like this:

Example: AWS login/credentials

# aws-creds ESC environment
values:
 creds:
 fn::open::aws-login:
 oidc:
 roleArn: arn:aws:iam::123456789012:role/pulumi-environments-oidc
 sessionName: pulumi-environments-session
 duration: 1h
 environmentVariables:
 AWS_ACCESS_KEY_ID: ${aws.creds.accessKeyId}
 AWS_SECRET_ACCESS_KEY: ${aws.creds.secretAccessKey}
 AWS_SESSION_TOKEN: ${aws.creds.sessionToken}

Example: Default production environment to use us-east-1 region

# aws-production ESC environment
imports:
 - aws-creds
values:
 aws:
 region: us-east-1
 pulumiConfig:
 aws:region: ${aws.region}

Example: Default staging environment to use us-west-2 region

# aws-staging ESC environment
imports:
 - aws-creds
values:
 aws:
 region: us-west-2
 pulumiConfig:
 aws:region: ${aws.region}

Here we define three environments for AWS:

aws-creds: sets up login via OpenID Connect (OIDC) and provides standard environment variables containing AWS credentials to the Pulumi program
aws-production: imports everything from aws-creds then sets the region to us-east-1
aws-staging: does the same, but sets the region to us-west-2.

Using that in a Pulumi program is as simple as adding the following settings to your stack config:

# Pulumi.staging.yaml
environment:
 - aws-staging

In this manner, you can configure separate environments for staging and production, with a complex set of configuration values and secrets, using different environments for each one. From the developer’s perspective they would only need to change aws-staging to aws-production when they go to deploy their stack.

Another strong benefit of this approach is that all secrets will be encrypted both in-flight and at-rest. Pulumi waits until the last moment to decrypt secrets at runtime. By default, uses automatic, per-stack encryption keys provided by Pulumi Cloud, but you could use a provider of your own choosing instead.

Pulumi Crossguard: Policy-as-Code

Pulumi Crossguard allows you to check and enforce policies on your deployments. Policies are rules, written in code, that run during deployments to check that the resources are conforming to the necessary criteria. You can use off-the-shelf policies like AWSGuard and Pulumi Compliance-Ready Policies or write your own.

Either way you end up with a policy pack that you can apply to your entire Pulumi organization via Pulumi Cloud.

Here’s an example policy that checks for the presence of a tag user:Stack on a S3 bucket:

from pulumi_policy import (
 EnforcementLevel,
 PolicyPack,
 ResourceValidationPolicy,
)

def s3_check_required_tags(args, report_violation):
 if args.resource_type == "aws:s3/bucket:Bucket":
 if ("tags" not in args.props or
 "user:Stack" not in args.props["tags"]):
 report_violation("S3 Bucket is missing required user:Stack tag.")

PolicyPack(
 name="bucket-tags",
 enforcement_level=EnforcementLevel.MANDATORY,
 policies=[
 ResourceValidationPolicy(
 name="s3-tags",
 description="Ensure required tags are present on S3 buckets.",
 validate=s3_check_required_tags,
 ),
 ],
)

If the tag isn’t on the resource, it blocks the deployment with an error message. The error message would look something like this:

Policies:
❌ bucket-tags@v0.0.1
- [mandatory] s3-tags (aws:s3/bucket:Bucket: my-bucket)
Ensure required tags are present on S3 buckets.
S3 Bucket is missing required user:Stack tag.

This allows you to implement company-specific policies that can be as simple or complex as you need them to be.

To apply this across your entire organization, you can publish this policy pack to Pulumi Cloud, with the following commands:

$ pulumi policy publish myorg
$ pulumi policy enable myorg/my-policy-pack latest

Some other great features of Crossguard are the ability to version policies, define multiple policy groups, and create remediation policies that automatically fix policy violations when possible. We will cover these topics in a future post where we go deeper on how to use policies effectively.

Multi-Language Components (MLC)

In Pulumi, a component resource is something that your developers can import in their Pulumi program, instantiate and modify. These are made available via a provider, which is in turn, made available to Pulumi via a provider package. There are many of these already available in the Pulumi Registry. However, in a custom internal developer platform you can define your own components, and bake appropriate settings/configuration directly into the underlying code.

A multi-language component (MLC) is even more useful. You can author your component in your language of choice and then generate a SDK that surfaces that component into all of the languages that Pulumi supports. For example, your platform team might be comfortable writing in Python, but the developers that write your microservices might use Go, and the developers who write the front-end apps might use Node.js. Both teams might need to deploy apps and infrastructure into your Kubernetes cluster. With multi-language components you can write a component in Python that abstracts away all the details of your custom Kubernetes cluster, and make that available to both teams, in both Go, Node.js, and any other language that Pulumi supports.

To build a MLC, you’ll follow these basic steps to create the component, provider, provider package, and generate the multi-language SDK:

Fork one of the component provider boilerplate repos for Python, TypeScript, or Go.
Update the package and code-generator configuration files, which name your component and package, define the inputs and outputs, and declare the dependencies.
Implement the component in your preferred language.
Generate an SDK for the other languages.
Deploy the package.

Here’s a quick example of creating a custom S3 Bucket component in Python, that complies with the tagging policy we built earlier:

from pulumi_aws import s3
import pulumi

class TaggedBucket(pulumi.ComponentResource):

 def __init__(self, name, opts = None):
 super().__init__('mycorp:index:TaggedBucket', name, None, opts)

 # Create a bucket and add a custom tag to it.
 bucket = s3.Bucket(
 f'{name}-bucket',
 tags={
 'user:Stack': pulumi.get_stack()
 },
 opts=ResourceOptions(parent=self))

 self.register_outputs({
 'bucket': bucket,
 'websiteUrl': bucket.website_endpoint,
 'bucketDnsName': bucket.bucketDomainName
 })

This shows the component implementation in isolation from the provider/packaging/SDK boilerplate. In this code sample, we’re creating a component called TaggedBucket that creates a S3 bucket, and adds a tag user:Stack with the current stack name as its value. A developer could now use this in a TypeScript Pulumi program as such, and this resource would automatically have the tags added to it.

import * as mycorp from "mycorp/mycorp-components";

const taggedBucket = new mycorp.TaggedBucket("example");

export const bucket = taggedBucket.bucket;
export const url = taggedBucket.websiteUrl;
export const dnsName = taggedBucket.bucketDnsName;

If you want to see how to create MLCs in more detail, check out this video that walks you through the entire process, and this repo for the code shown in the video. In a follow-up post in this series, we will build some reference MLCs that do things like implement a time-to-live (TTL) for stacks in your staging environment, automate drift detection, and automatically instrument your developer’s deployments with observability tools integrated by default.

Organization templates and the New Project Wizard

The final piece that ties all this together are organization templates. You may have used some of our built-in templates when you learned how to use Pulumi. These are great for basic use cases, but the real magic happens when you bring together your custom components and custom security environments to create personalized templates which represent the internal use cases for your organization.

Pulumi’s New Project Wizard reads these templates and provides an in-browser way to create a new project and deploy it. Running one of these templates will commit and push code to GitHub, and trigger an initial deployment – all in a few clicks and without leaving the browser.

Each template needs the following parts:

A Pulumi.yaml describing the template and its configuration values
A GitHub repo (public or private) containing the code for the templated Pulumi program

Here’s an example of a simple template using the components and environments we described above.

# Pulumi.yaml
name: ${PROJECT}
description: ${DESCRIPTION}
runtime: python
template:
 description: A Python Pulumi program that creates a tagged bucket.

"""__main__.py: A minimal Pulumi program"""

import pulumi
import mycorp

# Create an AWS resource (S3 Bucket)
tagged_bucket = mycorp.TaggedBucket('my-bucket')

# Export the name of the bucket
pulumi.export('bucket_name', tagged_bucket.bucket)

# Pulumi.production.yaml
environment:
 - aws-production

# Pulumi.staging.yaml
environment:
 - aws-staging

The Pulumi.yaml sets up the template and will populate the name and description from the settings provided during the template dialogue. The custom TaggedBucket component will create an S3 bucket, which will be tagged with user:Stack set to the name of the stack. Default stack configurations are provided for the staging and production environments which map to our two ESC environments, aws-production and aws-staging.

How it all works together

With all of that in place, from the developer’s perspective, all they need to do is create a new project from the template, answering three questions: the stack name, the name of the project, and an optional description.

If the developer names the stack staging it will automatically apply the aws-staging ESC environment, which will include the AWS credentials and set the region to us-west-2. However, if the developer names the stack production it will get the aws-production ESC environment setting it to use the us-east-1 region. The name of the stack will be stored in a tag on the resource.

Pulumi Crossguard will apply the bucket-tags policy check to see if the resource has the required user:Stack tag set and will allow the deployment to proceed only if it has that tag. If a developer created a standard S3 Bucket instead of using our internal TaggedBucket component, and failed to add the required tag, they will get an error message from our custom policy when they try to deploy.

Later, we can create additional automation that might do something like delete anything tagged staging after two weeks, or run drift detection on anything tagged production. We will be exploring these concepts in more detail in later posts.

More to Come

While setting up the Pulumi Patterns and Practices Platform (P3) reference architecture is not a simple click-to-deploy, hopefully this high-level tour of the various parts you need to assemble shows that really, it is only a matter of creating a few carefully constructed YAML files and snippets of code, and wiring them together properly. You can start small and build out your platform over time.

The next few posts in this series will go beyond these simple examples, showing much more complicated implementations of all of these pieces, and recommend some best practices for managing your infrastructure with this platform.

And if you are already ready to get your hands on Pulumi after this introduction, feel free to create an account and follow some of our Getting Started guides to see how easy simple use cases are and begin to imagine how that same developer experience will scale up to your entire organization.

To learn more, you can watch the following video which provides a high level overview of how Pulumi works:

*{padding:0;margin:0;overflow:hidden}html,body{height:100%}img{position:absolute;width:100%;top:0;bottom:0;margin:auto}">

Pulumi Cloud

The Pulumi Cloud is a fully managed service that helps you adopt Pulumi’s open source SDK with ease. It provides built-in state and secrets management, integrates with source control and CI/CD, and offers a web console and API that make it easier to visualize and manage infrastructure. It is free for individual use, with features available for teams.

Create an Account

Unified and Programmatic Approach to Infrastructure Management at BMW Using Pulumi

Sara Huddleston — Thu, 03 Oct 2024 09:52:10 +0000

In the ever-evolving world of automotive technology, BMW has been at the forefront of innovation, seamlessly integrating software into the heart of their vehicles. As cars become increasingly complex, with a growing emphasis on connectivity, over-the-air upgrades, and brand-specific user experiences, the need for a robust and scalable software development approach has become paramount.

Enter the BMW Software Factory, a platform that aims to empower the company’s developers and provide them with a superior development experience. At the core of this initiative is the adoption of Pulumi, a modern infrastructure as code (IaC) solution that has transformed the way BMW manages its software ecosystem.

On this article:

The Challenges of a Sprawling Software Landscape

BMW’s software journey has been a testament to the exponential growth of automotive technology. What started with simple engine controllers has evolved into a complex network of electronic control units (ECUs) scattered throughout the vehicle. As the software footprint continues to expand, BMW recognized the need for a unified and efficient approach to software development and deployment.

The company’s initial efforts involved introducing a platform called “Code Craft,” which provided a comprehensive stack of services to support the software development lifecycle. This stack included a GitHub Enterprise application for source code management, a Gerrit system for Android-based developments, a continuous integration (CI) pipeline, artifact stores, build caching, and various other tools and services.

However, as the demand for software-driven features grew, the complexity of managing this sprawling ecosystem became increasingly challenging. BMW found itself grappling with the need to scale its infrastructure, navigate network limitations across multiple data centers, and adapt to the ever-changing landscape of cloud computing.

The Evolution of BMW’s Software Development Toolchain

BMW’s journey to streamline its software development process has been gradual and iterative. The company’s initial approach involved using Ansible for deployment and a custom-built deployment scripting solution for its OpenShift cluster.

As the complexity of the platform increased, BMW turned to Helm and Kubernetes to manage its containerized services. However, as the company ventured into the public cloud, the limitations of these tools became apparent. The team recognized the need for a more comprehensive and scalable solution to manage their infrastructure as code.

At this critical juncture, BMW discovered Pulumi. This modern IaC solution offered a unique advantage – the ability to leverage a full-fledged programming language, Python, to define and manage their infrastructure. This shift proved to be a game-changer, allowing BMW to leverage its expertise in Python and benefit from the rich ecosystem of libraries and tools available in the Python community.

Embracing Pulumi: Streamlining Infrastructure Management

BMW’s adoption of Pulumi was a strategic move that aimed to address the growing complexity of its software ecosystem. By transitioning from a patchwork of tools to a unified IaC solution, the company was able to streamline its infrastructure management and improve developer productivity.

Shared Modules: Promoting Reusability and Best Practices

One of BMW’s key initiatives was developing a shared modules library, which allowed the team to abstract the complexity of various infrastructure components, such as databases, and provide a consistent and user-friendly interface for their developers.

By leveraging Pulumi’s Python bindings, BMW was able to create reusable modules that encapsulated best practices and sensible defaults, making it easier for developers to provision and manage infrastructure resources. BMW used Pydantic, a data validation library, to define schema-based configurations for its infrastructure. These were then integrated into its IDEs, providing developers with auto-completion and validation support.
This approach not only accelerated the development process but also ensured that the infrastructure deployed across the organization adhered to consistent security and compliance standards.

Transformation Features: Protecting Legacy Services

As part of their software factory, BMW also faced the challenge of integrating legacy services that did not natively support modern authentication and authorization mechanisms, such as OpenID Connect (OIDC).
To address this, BMW leveraged Pulumi’s transformation features to seamlessly inject an OAuth2 proxy into their deployments, providing a secure and consistent way to protect these services without requiring extensive modifications to the underlying applications.
By encapsulating this functionality within a shared module, BMW was able to apply the OAuth2 proxy to multiple services, ensuring a consistent and secure access control layer across their software ecosystem.

Policy Enforcement: Ensuring Compliance and Security

One key benefit of Pulumi’s IaC approach is the ability to define and enforce policies across the organization, ensuring that infrastructure deployments adhere to security and compliance standards. BMW has leveraged Pulumi’s policy-as-code capabilities to implement mandatory checks, such as ensuring that all S3 buckets are encrypted at rest, preventing the deployment of non-compliant resources.

By integrating these policy checks into their deployment workflows, BMW has shifted security and compliance concerns to the left, addressing issues early in the development process and reducing the risk of costly post-deployment remediations.

The Benefits of Pulumi: Accelerating Development and Improving Maintainability

BMW’s adoption of Pulumi has yielded significant benefits, transforming the way the company approaches software development and infrastructure management.

Accelerated Development with Shared Modules

The implementation of shared modules has been a game-changer for BMW. It allows developers to leverage pre-built and tested infrastructure components without having to reinvent the wheel. This has resulted in a significant acceleration of the development process, as teams can focus on building their applications rather than grappling with the complexities of infrastructure provisioning.

Improved Maintainability and Consistency

By centralizing infrastructure management within the shared modules, BMW has ensured that best practices and security standards are consistently applied across the organization. This has not only improved the overall maintainability of the software ecosystem but has also reduced the risk of security and compliance violations.

Leveraging Python’s Ecosystem

BMW’s decision to leverage Pulumi’s Python bindings has been a strategic advantage, as the company was able to tap into the rich ecosystem of Python libraries and tools. This has enabled the team to seamlessly integrate Pulumi with their existing Python-based toolchain, including dependency management, testing frameworks, and code quality tools, further enhancing the development experience.

Streamlined Cloud Migration

As BMW continues to expand its use of public cloud services, Pulumi has played a crucial role in simplifying the migration process. By providing a consistent IaC approach across on-premises and cloud environments, Pulumi has enabled BMW to manage its infrastructure in a unified manner, reducing the complexity and overhead associated with multi-cloud deployments.

The Future of BMW’s Software Factory: Embracing the Cloud

Looking ahead, BMW’s Software Factory is poised to take the next step in its evolution, with plans to transition away from the self-hosted backend and embrace cloud-native services. This strategic move aims to further improve developer productivity and reduce the internal effort required to maintain the underlying infrastructure.

By leveraging the expertise and reliability of cloud service providers, BMW can focus on delivering innovative software features to their customers while the cloud providers handle the operational aspects of running the necessary services.

As BMW continues to push the boundaries of automotive software, Pulumi’s role in its Software Factory will only become more crucial. By providing a scalable and flexible IaC solution, Pulumi empowers BMW’s developers to innovate with confidence and be secure in the knowledge that their infrastructure is managed consistently and in alignment with the company’s security and compliance standards.

Conclusion: Unlocking the Future of Automotive Software

BMW’s journey with Pulumi in the Software Factory showcases the power of modern IaC solutions in navigating the complexities of the automotive software landscape. By embracing a unified and programmatic approach to infrastructure management, BMW has accelerated development, improved maintainability, and ensured compliance across its sprawling software ecosystem.

As the automotive industry continues to evolve, with cars becoming increasingly software-driven, the lessons learned by BMW can serve as a blueprint for other organizations looking to streamline their software development and deployment processes. By leveraging the capabilities of Pulumi and other cutting-edge technologies, the future of automotive software is poised to be more efficient, secure, and responsive to the ever-changing needs of both manufacturers and consumers.

To learn more about Pulumi and how it can transform your software development and infrastructure management:

Get started with Pulumi Tutorials
Attend an upcoming workshop
Try out the Pulumi Neo code assistant to accelerate your infrastructure as code journey

AWS CDK vs Pulumi: Why SST Chose Pulumi

Sara Huddleston — Wed, 25 Sep 2024 07:32:40 +0000

Cloud computing tools evolve, and so must the frameworks developers rely on. For SST (Serverless Stack), AWS CDK was a great starting point—but it had limitations.

CDK tied infrastructure to AWS.
Debugging was frustrating due to CloudFormation templates.
Multi-cloud was nearly impossible.

The solution? Pulumi. In this post, we’ll explore why SST moved to Pulumi, what challenges they overcame, and what this means for developers building modern cloud applications.

TL;DR: Pulumi lets SST offer a faster, more flexible, and provider-agnostic infrastructure experience.

The Beginnings of SST

Application developers have witnessed the rapid evolution of cloud computing and the growing need for developers to have direct access to powerful cloud resources. However, the traditional tools and approaches to Infrastructure as Code (IaC) have often been geared more toward DevOps teams, leaving application developers feeling disconnected from the infrastructure side of their projects.

This was the driving force behind the creation of SST (Serverless Stack), a framework that aims to bridge the gap between application developers and infrastructure management. In the early days, SST was built on top of AWS’s Cloud Development Kit (CDK), which allowed developers to define their infrastructure using TypeScript or Python. While this was a step in the right direction, they soon realized that the limitations of CDK and the underlying AWS CloudFormation were holding them back from truly empowering application developers.

CDK and CloudFormation Limitations

As the SST’s team continued to work with CDK and CloudFormation, they encountered several challenges that led them to reevaluate their approach. One key issue was the disconnect between application developers’ thinking and working and traditional IaC tools’ operating methods.

With CDK and CloudFormation, the infrastructure code is essentially a code generator, producing an intermediary format (such as YAML or JSON) that is then executed to deploy the resources. This means that the actual code you write as a developer is not the same as the code running during the deployment process. This can lead to several problems, such as difficulty debugging, lack of visibility into the deployment process, and challenges in extending or customizing the deployment workflow.

Additionally, as the SST’s team expanded its focus beyond the AWS ecosystem and started exploring other cloud providers and even on-premises infrastructure, it found that the AWS-centric nature of CDK and CloudFormation was becoming a limitation. It needed a more flexible and provider-agnostic solution that would allow it to deploy and manage infrastructure across a wide range of platforms.

Feature	AWS CDK	Pulumi
Multi-Cloud Support	❌ AWS-Only	✅ AWS, Azure, GCP, On-Prem
Debugging	❌ Hard due to CloudFormation	✅ Real-time debugging
Language Support	⚠️ TypeScript, Python	✅ Any programming language
Deployment Speed	❌ Slower due to CloudFormation	✅ Faster direct execution
Visibility	❌ Hard to trace errors	✅ Clear deployment state
Extensibility	⚠️ Limited to AWS ecosystem	✅ Custom providers & workflows
State Management	❌ CloudFormation state file	✅ Pulumi-managed state
Secrets Management	❌ AWS Secrets Manager only	✅ Cross-cloud secret support

A Provider-agnostic Solution: Discovering Pulumi

It was during this time that the SST team discovered Pulumi, a modern IaC platform that takes a fundamentally different approach to infrastructure management. Instead of generating an intermediary format, Pulumi treats the infrastructure code as a first-class program executed directly during deployment.

This paradigm shift had several important implications for SST and its users:

Visibility and Extensibility: With Pulumi, the infrastructure code is the same code that is running during deployment, which means there is much greater visibility into the deployment process and the ability to extend or customize it as needed.
Multi-Cloud Capabilities: Pulumi’s provider-agnostic approach allows SST’s team to easily work with a wide range of cloud and on-premises platforms, giving their users the flexibility to deploy their infrastructure wherever it makes the most sense for their application.
Simplified Mental Model: For application developers, the Pulumi model of “your code is the deployment” aligns much more closely with their existing mental models and workflows, making it easier for them to adopt and work with IaC tools.

Transitioning to Pulumi

Transitioning SST from CDK to Pulumi was not a trivial undertaking. Still, they knew it was a necessary step to truly fulfill their mission of empowering application developers with powerful infrastructure management capabilities.

One key challenge they faced was re-implementing the higher-level components and abstractions they had built on top of CDK. These components were designed to simplify the infrastructure management experience for their users, and they wanted to ensure that they could provide a similar level of abstraction and ease of use with Pulumi.

Additionally, they had to carefully consider how to handle the various edge cases and complex deployment scenarios their users encountered with the CDK-based version of SST. They wanted to ensure that the Pulumi-based version would not only match the functionality of the previous version but also improve upon it and address some of the limitations they had encountered.

The Benefits of Pulumi

As they worked through the transition to Pulumi, they realized the significant benefits that Pulumi offered to both the SST team as the framework developers and the users.

Improved Visibility and Debugging

One of Pulumi’s most immediate and tangible benefits was the improved visibility and debugging capabilities it provided. With the infrastructure code being the same as the deployment code, they could easily trace issues back to the source and understand exactly what was happening during the deployment process.

This starkly contrasted the CDK/CloudFormation approach, where the intermediary format (CloudFormation templates) often obscured the underlying logic and made it much more difficult to diagnose and resolve problems.

Extensibility and Customization

Pulumi’s design also allowed the SST team to easily extend and customize the deployment process to meet the specific needs of their users. They could leverage Pulumi’s built-in extensibility features, such as custom providers and dynamic components, to integrate with a wide range of cloud and on-premises services and implement complex deployment workflows tailored to our users’ requirements.

This level of customization was much more challenging with the CDK/CloudFormation approach, where they often had to resort to hacky workarounds or custom Lambda functions to achieve the desired functionality.

Multi-Cloud Capabilities

As mentioned earlier, one key driver for their transition to Pulumi was the need to support a wider range of cloud and on-premises platforms. With Pulumi’s provider-agnostic approach, the SST team was able to easily add support for new providers, allowing their users to deploy and manage infrastructure across a diverse set of environments.

This flexibility has been particularly valuable for their users, who may have workloads or requirements that span multiple cloud providers or even on-premises infrastructure. With SST built on Pulumi, they can now manage all of their infrastructure through a single, consistent interface without having to juggle multiple tools or approaches.

Simplified Mental Model

One of Pulumi’s most significant benefits for SST users is the simplified mental model it provides. By treating the infrastructure code as a first-class program, Pulumi aligns much more closely with how application developers think and work.

Instead of having to navigate the complexities of intermediary formats, deployment pipelines, and the separation between infrastructure code and deployment code, SST users can now focus on writing their infrastructure logic in the same programming languages they use for their application code. This makes it much easier for them to understand, maintain, and extend their infrastructure as their needs evolve.

Build across any cloud with Pulumi

Define your infrastructure in TypeScript, Python, Go, or C#, and deploy it across AWS, Azure, Google Cloud, and on-premises environments from one consistent workflow.

Get started

The Future of SST with Pulumi

With Pulumi’s foundation in place, they can now focus on further enhancing the developer experience and expanding the capabilities of their framework. Some of the key areas they are exploring include:

Deeper Integration with Application Frameworks: By leveraging Pulumi’s flexibility, they can create even tighter integrations between SST and the application frameworks and libraries that their users rely on, making managing infrastructure seamless alongside their application code.
Expanded Provider Support: the SST team will continue to add support for a wide range of cloud and on-premises providers, ensuring that their users can deploy and manage their infrastructure wherever it makes the most sense for their needs.
Improved Deployment Workflows: Building on Pulumi’s extensibility, they can create more advanced deployment workflows that address the specific needs of application developers, such as faster deployment times, better rollback capabilities, and more granular control over the deployment process.
Enhanced Observability and Monitoring: By treating the infrastructure code as a first-class program, they can provide their users with better visibility into the deployment process and more robust monitoring and observability capabilities, helping them to identify and resolve issues quickly.

As the SST team continues to evolve with Pulumi at its core, they aim to deliver an even more powerful and user-friendly infrastructure management experience for developers. This will empower teams to focus on building great applications while effortlessly managing the underlying infrastructure.

Conclusion

If you’re interested in exploring Pulumi further, here are several ways to get involved:

Discover how Pulumi Crosswalk for AWS simplifies “day one” tasks in our AWS guide
Watch our on-demand workshop Getting Stated with Infrastructure as Code on AWS
Register for one of our upcoming Platform Engineering or DevOps workshops
But most importantly, try Pulumi today!

Frequently Asked Questions

What is SST?

SST is a framework that makes building modern full-stack applications on your infrastructure easy.

What is SST’s Ion?

Ion is the code name for a new engine for deploying SST applications. The constructs (or components) are deployed using Pulumi instead of CDK and CloudFormation (CFN). Once Ion is stable, it will be released as SST v3.

Does SST use mostly Terraform or Pulumi?

SST leverages Pulumi behind the scenes for its providers and deployment engine while also bridging Terraform providers through Pulumi.

How does SST make money?

SST (Serverless Stack) makes money primarily through its managed service, called SST Console. The Console is optional and includes a free tier. In short, SST primarily relies on SaaS (Software as a Service) revenue through its managed platform and potentially supplemental income streams like enterprise services and consulting.

Pulumi Patterns and Practices Platform (P3): A reference architecture for large-scale organizations

Troy Howard — Mon, 05 Aug 2024 00:00:00 +0000

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

Infrastructure management is all fun and games until you find yourself scrolling through 1000+ resources in your AWS console. Worse, when one rogue product team wants to use Azure and your data team wants to be on GCP, you’re ARM wrestling in Azure and watching your economies of scale tip the wrong direction as you’re copy-pasting CloudFormation templates into yet another git repo. This. Needs. To. Be. A. Platform!

And in that moment of overwhelm, you will be sold to, nurture-emailed every week, and told all your problems will be solved by implementing an IDP (internal developer platform, as if you’ve never seen this acronym before). An IDP that costs a lot of money and a lot of time to implement beyond default settings. An IDP that really only solves half of your problems. Your internal team offers to build something that feels more like welding together random pieces of code into an abstract found-art sculpture built from junkyard refuse, already 5 years out of date. How long will this investment be useful before you have to start over?

It’s exhausting. If there was a good solution on the market, you wouldn’t be reading this article. So let’s talk about what you really need, and how Pulumi can help.

An effective internal developer platform

There are quite a few listicles out there professing to authoritatively tell you the 5, or 7, or 11 essential components of an internal developer platform. Personally, I trust our customers to tell us, and here’s what they have said they need:

Consistency: Bring some order to the chaos. As your company and your infrastructure grows, it gets more and more complicated to maintain consistency. You might already have established design patterns that you want to replicate, but don’t have any way to encode those practices in your current tools. There’s a lot of copy/paste of reusable blocks, but no way to apply DRY principles or to modularize/templatize the important parts (hint: all the parts are important!).

Reproducibility: Repeatable behaviors, who dat? If you run your deploy twice do you get the same results each time? What if you replicate your production environment to create a test environment, are they actually identical? How much more work does it take to get them to be? Will you get the same version of the training dataset every time you run your AI workloads? It’s anyone’s guess. A lack of reproducibility slows down development, makes debugging more difficult, and makes that reuse we just talked about harder to achieve.

Visibility: When your node count, and user count starts to go beyond about 50-100 resources (computing or human) you quickly run into a problem of visibility. It can be very difficult to get a handle on what’s happening, how many resources you have, where they are, and how much they cost. Any system that purports to be able to manage 1000 nodes or more must have deeply integrated analytics, dashboards, charts, and be searchable, across all your clouds, all your users, and every kind of resource.

Security and Compliance: Good fences make good neighbors. RBAC, policy-as-code, excellent secrets management, integration with your existing identity providers. These are the things you need to build security and policy guardrails you can rely on. Without them? It’s just a powder keg of liability waiting to catch a spark.

Auditability: What happened and who did it? This is like a high-stakes game of Clue. How quickly can you figure out who ran that bad deployment? Was it Colonel Mustard in the library with the candlestick? Or Blake the new Front-End Developer with overly-broad permissions in AWS? Being able to answer these questions needs to happen quickly. Quickly, like minutes, not hours or days. And it might have happened 6 months ago. Oof.

Developer Experience: In the ideal world, developers drive their own DevOps. The platform team provides self-service tools and streamlined workflows that allow your engineers to provision new resources, so your team doesn’t have to. And you know, if the developers don’t like the user experience, they won’t use it at all, and will invent their own tools. You will have ROGUE SYSTEMS to hunt down and argue against in tedious overly-technical meetings. This is not what you want. We need to keep the developers happy to prevent this.

A holistic view of the Patterns and Practices Platform reference architecture

Pulumi has a broad surface area of products and features that address these needs. Designed with integration in mind from the beginning, our tools orchestrate well, presenting a smooth and streamlined workflow for both operations teams and developer teams.

We have an idea of how you can use all the Pulumi products together to deliver a comprehensive internal platform for security, infrastructure management, and deployments. Call it an internal platform for developer platform engineers (IPfDPE), if you want. We call it the realization of a vision we’ve been working hard to build for many years.

Pulumi Patterns and Practices Platform (P3) is a reference architecture that we will be describing, and providing code for, through this series of articles. We’ll be diving deep into not just what you can do with our tools, but how to do it, and provide code for a reference implementation that you can use to jump start the process.

Here’s a quick overview to give you an idea of how we’ll be addressing those needs in Pulumi Patterns and Practices Platform (P3).

Consistency

Pulumi can help bring consistency to your software catalog by encoding design patterns into reusable component resources and by building custom organization templates that provide a no-code or low-code way to start a new project. Templates help get projects off the ground faster and ensure consistent code structure, policy compliance, and best practices.

Figure: An internal developer portal using custom templates in Pulumi Cloud

Beyond that, because Pulumi is multi-cloud (AWS, Azure, Google Cloud, and more) and multi-language (JavaScript, Python, Go, C#, Java) you can enjoy the same consistency across all your environments and all your developer teams, regardless of the languages they prefer, or cloud tooling they need.

Another core aspect of consistency is drift detection. Pulumi automatically detects and remediates cloud resources that have deviated from the expected state stored in Pulumi Cloud. This tech is better than ibuprofen at getting rid of developer-created headaches.

Reproducibility

Since 2010, scientists have felt that we are in a crisis – a reproducibility crisis – wherein we cannot easily reproduce an experiment in order to verify published results. Similarly, the software industry is entering into a reproducibility crisis of its own, especially around AI training workflows, where it is increasingly difficult to recreate crucial build and prod environments. Pulumi Stacks make it very easy to manage both configuration and state across multiple environments, and make reproducing a deployment within Pulumi a matter of a few basic operations.

You can use Pulumi programs to capture all of the necessary resources for an AI training workload, including things like versioned data using dynamic tables with time-travel functionality in Snowflake. That means you can be sure that not only will your deployment be on the infrastructure you need, it will also have the exact version of data, every time, which is essential to A/B testing and debugging your models.

Visibility

Every resource under management by Pulumi is visible within Pulumi Insights. From this single-pane-of-glass interface, you can search for resources across all cloud environments. Pulumi Copilot provides a state-of-the-art AI chat interface to ask complex questions and get immediate results. Pulumi Insight’s analytics gives you the ability to identify anomalies or trends in resource usage and dig into cost, security, and compliance concerns.

Figure: Search for any resource with Pulumi Insights

Security and Compliance

In the modern parlance, when you say DevOps, you mean DevSecOps. Pulumi is designed to be secure by default. Pulumi Cloud offers full role-based access control (RBAC) functionality including deep integration with GitHub teams and SAML-based SSO, managed secrets and flexibly-defined secure environments with Pulumi ESC, and policy-as-code provided by Pulumi Crossguard. Most importantly all of these features are deeply integrated across the platform, creating an air-tight system with all the guardrails you need for managing security and access.

Auditability

Every action a user takes in Pulumi can be tracked via the audit log which is searchable in two clicks from the Pulumi Cloud homepage dashboard. Audit logs can be filtered by user with one more click. Creating automated backups of your audit logs is a first-class feature. You will never have to worry about responding quickly when someone asks about an event that happened in your system. Also, each deployment and update has logs directly visible from the Pulumi Cloud app, regardless of how it was initiated.

Figure: Viewing the audit log in Pulumi Cloud

Developer Experience

Probably the most compelling aspect of Pulumi is the developer experience. Developers love Pulumi, because they get to use their preferred tools. General purpose programming languages, visual IDEs, command-line tools, and products with an API-driven architecture are what developers want, and it’s what Pulumi delivers in spades.

With Pulumi templates and custom internal component resources in place, developers can drive their own DevOps, provisioning their own infrastructure resources and managing their own deployments directly, reducing bottlenecks in platform teams. Product engineering teams can self-service with a stream-lined workflow that stays compliant with company policy by default. Deep in the code of their favorite programming languages, your developers will never even know they are following the company rules.

Figure: Using C# to write a Pulumi program in VS Code

More to Come

So now that we’ve made a case for how Pulumi can be applied to meet the most pressing needs of a larger organization, hopefully you will realize that the Pulumi Patterns and Practices Platform (P3) reference architecture we are presenting here is more than just infrastructure-as-code. P3 is a Pulumi-powered platform for teams, where your developer portal is not just a catalog of software, but a fully functional control-plane across all your cloud environments.

Stay tuned for the following series of posts where we will use Pulumi to implement the P3 reference architecture for a fully-featured internal developer platform (IDP, or IPfDPE if you prefer). That said, you may already have invested in some popular in cloud-native tools like Backstage or Kubernetes. Pulumi plays well with others, and you will be delighted to see how you can use Pulumi to cover the gaps in the CNCF ecosystem.

To learn more, you can watch the following video which provides a high level overview of how Pulumi works:

*{padding:0;margin:0;overflow:hidden}html,body{height:100%}img{position:absolute;width:100%;top:0;bottom:0;margin:auto}">

Pulumi Cloud

Create an Account

Platform Engineering & DevOps Series Kickoff Announcement

Sara Huddleston — Mon, 15 Jul 2024 14:38:46 +0000

We are excited to announce the kickoff of the Platform Engineering & DevOps Series, which will run from July 16 to October 31. This series will feature in-person events across various cities, including Berlin, London, Paris, Sydney, Boston, San Francisco, Seattle, Austin, Denver, and NYC. For those unable to attend an in-person event, we encourage you to participate in a virtual DevOps & Platform Engineering workshop.

Upcoming In-Person Events

Pulumi is hosting in-person meetups and virtual workshops for engineers interested in Platform Engineering and DevOps, covering topics from best practices to setup and implementation to advanced topics. Join us for tech talks, hands-on workshops, and opportunities to network with fellow engineers.

First Platform Engineering & DevOps in-person event: Berlin - July 16

Join Dominik Kress (Product Manager at Giant Swarm and member of the CNCF Platforms Working Group) and Guy Menahem (Solutions Architect at Komodor and community leader at The Platformers) at LOBE Block for an engaging discussion on platform engineering.

Dominik Kress will present “What the Heck is the CNCF Platform Working Group? Answers from a Member!”
Guy Menahem will present “For This Backstage, You Don’t Need a Special Pass: Platform Engineering in Practice.”

For more details and to RSVP, visit the Berlin Meetup page.

Second In-Person Event: Boston, July 24

Join Pulumi’s founder & CEO, Joe Duffy, and Jamie Baldanza, Director of DevOps at Relay Therapeutics, for the launch of our Boston meetup at The Foundry. This is a great opportunity for DevOps engineers, platform engineers, software engineers, and Cloud enthusiasts to connect and share insights.

RSVP is required due to limited capacity. Secure your spot, and RSVP at the Boston Meetup page.

More coming soon!

We expected Seattle, Paris, and London’s meetups to happen in early September. Details for the other in-person events will be coming soon, so stay tuned for more updates at Platform Engineering & DevOps Series.

No upcoming in-person events in your area? Attend one of our virtual workshops and keep up with the latest in DevOps & Platform Engineering!

Virtual Workshops

Getting Started with CI/CD for AWS using GitHub Actions: July 17. Learn more and register here.
From Zero to Production in Kubernetes: July 25. Learn more and register here.
Advanced CI/CD for AWS using Pulumi and GitHub Actions: August 14. Learn more and register here.
Mastering Platform Engineering: From Setup to Scaling Success: TBD.

Join us for tech talks, hands-on workshops, and opportunities to network with industry peers. We are excited to meet you in person or virtually. See you there!

Why Developer Experience (DevEx) is Business Critical

Sara Huddleston — Tue, 20 Feb 2024 20:47:44 +0000

“Developer experience is hard to sell,” said Cleve Littlefield, Engineering Manager at Pulumi, during a casual meeting. With experience as both an end-user developer and a lead in self-service platform implementation, Cleve’s observation stuck with me.

Though I have expertise in leading implementations and upgrades for internal platforms, none were specifically for developers. However, experience remains vital across departments, addressing tools, processes, systems, and best practices, aiming to reduce cognitive load, increase productivity, enhance collaboration, boost communication and much more.

Intriguingly, engineering teams may perceive its value differently. Therefore, we will dive into the concept of developer experience, aka DevEx, which, in truth, should translate into a competitive advantage.

In this DevEx article

What is developer experience?

Developer experience (DevEx), also referred to as DevX or DX, encompasses systems, technology, processes, and culture that collectively impact the effectiveness of software development. It looks at all components of a developer’s ecosystem—from environment to workflows to tools—and evaluates how they contribute to developer productivity, satisfaction, and operational impact.

Developer experience revolves around the ease or difficulty of executing essential tasks. A positive developer experience means those tasks are relatively easy to perform, translating into higher performance.

The quality of the experience is equally vital for developers building and managing internal software as it is for those involved in developing customer-facing software products.

Is developer experience important?

It is not just important. It’s business critical. If your company is in the business of creating and selling software or relies on critical internal software, developers are vital internal stakeholders to your business foundation.

Greg Mondello, director of product at GitHub, said, “In most contexts, software development capacity is the limiting factor for innovation. Therefore, improvements to the effectiveness of software development are inherently valuable […] companies with better DevEx outperform their competitors, regardless of vertical.”

Platform Engineering Forrester Opportunity Snapshot - DevEx impacts overall business performance. Credit: Humanitec

As per McKinsey’s report, an enhanced developer experience (DevEx) can yield significant advantages for organizations, including enhanced talent hiring and retention, improved security, and increased developer productivity- this makes DevEx important for all companies, not just tech.

A positive developer experience can yield numerous business advantages, which can include:

Improvement in collaboration across departments
Higher attraction and retention top talent
Increase developer productivity and developer velocity
Reduction in operational costs
Accelerate time-to-market
Support innovation
Elevate customer attraction and retention
Increase bottom line through revenue growth

To this end, let’s look into what goes into developer experience and how to achieve it.

The GitHub developer experience (DevEx) formula

GitHub’s DevEx formula takes into consideration the following:

Productivity: The speed and simplicity of implementing changes to a codebase
Impact: The seamless transition from idea to production without obstacles
Satisfaction: The environment, workflows, and tools influence developer happiness

Optimizing DevEx involves fostering a collaborative environment where developers can be most productive, impactful, and satisfied.

DevEx formula. Credit: GitHub

Noah Gift, the founder of Pragmatic AI Labs and a professor at Duke University specializing in machine learning, MLOps, AI, data science, and cloud architecture, noted that with the right platform, there could be:

A 75% increase in productivity
A sustained 22% productivity increase three years later
An 80% reduction in onboarding time

Merely having skilled developers is insufficient. Developers also require the appropriate tools and processes to excel in their work. Top-tier developers anticipate the availability of such resources, influencing the hiring and retention of talent.

What does a great developer experience look like?

Engineering teams that have good developer experiences are more productive and efficient. Great DevEx is often enabled by a platform where the end-users are the developers.

Developer-first readily available internal developer platform, infrastructure, and development tooling, with well-organized and explicit documentation, including tutorials, demo environments, and curated learning paths.
A unified hub for all developer requirements eliminates the need for developers to navigate through numerous tools, saving valuable time.
Simplified workflows emerge when developers can swiftly select app patterns, such as API microservices or front-end journeys, and deploy them within minutes. This includes all-encompassing elements like environments, pre-integrated DevSecOps pipelines, monitoring, and fully automated change and release procedures.
Centralized tracking of software, API, and infrastructure status and versions promotes asset transparency. A real-time view supports adherence to architectural standards, security controls, and patching status, with bots offering automatic suggestions for code enhancements related to issues.
Integrated analytics and KPIs such as developer velocity, tech debt, error rate, mean time to recovery, and infrastructure cost can be automatically pulled by standard organizational tools, including backlog, pipeline, and monitoring tools.

Survey by GitHub on a typical experience for developers. Credit: GitHub

Without the proper tools, progress may slow down the development process and hinder developer productivity. It becomes crucial to implement appropriate measures to avoid disruptions that could impact everyone involved, including the developers themselves. Although not often mentioned, DevOps and Platform Engineering also aim to enhance the developer experience and achieve the associated benefits.

How does DevEx intersect with DevOps?

DevOps is about developers and operations working together and sharing values, assumptions, and responsibility for the software they build and maintain.

According to Vilas Veeraraghavan, VP of Engineering at Truckstop, “Developer Experience (DevEx) is the investment you make to enable solid DevOps practices, reduce developer burnout, and improve retention. The need for this investment is more pronounced as companies grow and the tech stack gets more diverse.”

Watch Jowanza Joseph, Head of Engineering at StreamFT, below as he discusses how the right tool empowered developers, improved DevEx, reduced bottlenecks, increased productivity, and helped DevOps be a culture, a practice, not a person.

Let’s address a strong indicator of success or failure - culture.

Culture is not something you are—it’s something you do. Software companies often focus on DevOps practices and culture, but all would benefit from adopting a DevOpsAll since the entire company should strive for the same thing. The diversity of perspectives, ideas, and insights can spark creativity and innovation and even ensure product-market alignment.

Organizational culture is a predictor of outcome success

Why it matters? Organizational culture encompasses everyone, from the development team and operations to product management teams, sales, marketing, and more.

According to George Spafford, Senior Director Analyst at Gartner, “Infrastructure & Operations leaders embarking on DevOps initiatives and struggling to address organizational change and the value they will provide to the larger enterprise […] people-related factors tend to be the greatest challenges — not technology.”

According to a Harvard study encompassing 200 companies over 11 years, companies that focused on shaping their culture outperformed their competitors:

Revenue growth was 4.1 times higher
The stock price was 12.2 times higher
Net income was 756% vs. 1%
Return on investment (ROI) was 15 times higher

The driving force behind this exceptional financial performance is that engaged employees are prone to increased productivity, fostering superior innovation and more effective problem-solving.

Slide from Mark’s New York Atlassian Community Group presentation. Credit: Mark Cruth, principal modern work coach/advocate at Atlassian

Based on decades of research on employee engagement, Gallup has established that engaged employees consistently yield superior business outcomes compared to their counterparts. This holds true across various industries, company sizes, and nationalities and remains consistent in prosperous and challenging economic conditions.

If it is not yet clear, then I will spell it out. Developer experience impacts employee engagement. A better developer experience can boost the engagement of the engineers and developers in the organization, and more engaged people are more likely to go the extra mile.

How developer experience supports better business outcomes. Credit: Swarmia

If organizations invest in cultural values centered on human-first factors, then even the choices made in tools, processes, and systems will also be with the end-user in mind.

How do you implement a great developer experience?

There isn’t a bulletproof checklist, but there are several aspects to consider, and you can see the outline below. First, you need to view the developers as customers.

You will need to understand how developers work, the developer workflow, what the onboarding process looks like for a new developer, the developer teams’ pain points, what developers love and hate about their day-to-day work, where they are spending most of their time, where they wish they spent most of their time, and so much more, that boils down to communication, and asking the right questions, and deep diving into analytics.

Cultivate a DevOps culture

Shared responsibility: Encourage a culture of shared responsibility where developers, operations, and other stakeholders work collaboratively
Continuous learning: Promote a culture of continuous learning and improvement to adapt to evolving technologies and methodologies
Open communication: Foster open communication channels to enhance collaboration and transparency among team members
Company-wide collaboration tools: It’s essential to encourage communication among developers and cross-functional teams to stimulate creativity, spark innovation, and enhance productivity and the speed of decision-making. Cross-functional collaboration contributes to successful business outcomes

Define clear objectives

Alignment with business goals: Ensure DevEx objectives align with business goals to drive meaningful outcomes
User-centric approach: Prioritize user needs and feedback, shaping the DevEx strategy around the actual experiences of developers

Leverage developer-first tools

Internal developer portals (IDPs): IDPs enable developers to quickly provision approved infrastructure, boosting productivity with pre-configured architectures, automated testing, and deployment adhering to organizational standards
CI pipelines: usually, pipelines can be made faster and more unified. If they have a microservice architecture, they are likely duplicating pipeline code for every service
Software development kits (SDKs): SDKs are integral components of a positive DevEx and provide developers with tools that facilitate crafting applications tailored to specific platforms or frameworks. Offering easy-to-access, well-designed, functional SDKs can make a meaningful difference in helping engineers quickly prototype ideas and refine their creations
Infrastructure as code (IaC): Developers like to write code, particularly in their preferred programming language, provision infrastructure, ensuring consistency and reproducibility
Monitoring and observability: Implement effective monitoring and observability tools (e.g., Prometheus, New Relic, Grafana) to gain insights into application performance and proactively address issues
Collaborative documentation platforms: Use collaborative documentation platforms to document code, processes, and best practices, ensuring knowledge sharing and reducing onboarding friction

Platform Engineering Forrester Opportunity Snapshot - Which of the following steps organizations are taking to improve DevEx? Credit: Humanitec

Prioritize developer-friendly workflows

Fast feedback loops: Minimize feedback loops by integrating quick feedback mechanisms, allowing developers to identify and resolve issues early in the development cycle
Self-service environments: Provide developers with self-service environments to facilitate experimentation, testing, and debugging without unnecessary dependencies on other teams
Version control best practices: Enforce version control best practices to maintain a clean and organized codebase, allowing for easier collaboration and tracking of changes

Continuous integration of feedback

Developer surveys: Conduct regular surveys to gather feedback on the DevEx, ensuring continuous improvement based on real user experiences
Rapid iteration: Encourage a culture of rapid iteration, where feedback is quickly incorporated into the development process to enhance the overall developer experience

Measure and optimize

Key performance indicators (KPIs): Define and track key metrics such as deployment frequency, lead time, and mean time to recovery to measure the effectiveness of the DevEx strategy
Iterative optimization: Continuously iterate and optimize DevEx based on performance metrics and user feedback

Implementing a Developer Experience requires a holistic approach that blends cultural aspects with powerful DevOps and platform engineering tools. Watch Daniel Tao, Head of Engineering at Atlassian, and Sven Peters, developer evangelist, discuss how to build a great developer experience through culture and an integrated approach to developer tools.

What DevEx is not

Now that DevEx has been defined, let’s address some common misconceptions about developer experience.

DevEx is not the same as User Experience (UX)

Developer experience may sound similar to user experience (for developers), but it’s different, although user experience is a component. For example, a platform engineer building an internal developer platform must view the developers as their customers and create a good user experience by making it intuitive and easy to use.

DevEx is not about enabling lazy developers

Part of the DevEx strategy is to retain talent. Companies that invest in DevEx don’t do it to pamper developers. The true reason behind it is that they want their developers to stay “in the zone,” also known as “deep work.” When in this state, they are immersed in writing code and solving problems they enjoy working on, and their creativity is higher, it sparks innovative ideas, and the quality and speed of their output are also higher.

DevEx is not about implementing AI and cool tools

Technology impacts developer experience, a robust generative AI tool can help create code faster, and internal developer platforms can help reduce cognitive load on developers. But again, tooling is only one aspect of it. Tools can’t fix culture or replace processes. Many DevEx considerations should be considered to ensure that when the development teams adopt a tool, it addresses their pain points and enables developers.

DevEx is not a one-person decision

DevEx should be a joint effort from the entire team. One needs to be aware of the issues to plan to fix them, so open communication and team collaboration are vital to developing and implementing a DevEx strategy, intersecting DevOps and platform engineering initiatives. Many large organizations already have DevEx teams or professionals with a bird’s-eye view of what happens within the engineering team. They are internal advocates for software engineers. They consider the developer’s journey from being new developers and their ‘Time to First Contribution’ to day-to-day activities. They work with engineers to learn which problems are a priority and identify misconceptions and bottlenecks that may slow the development process.

Conclusion

Developer experience (DevEx) is essential for creating maximum output while increasing developer satisfaction. The appropriate technology tools, workflows, and a supportive development community can help foster progress and innovation while promoting high-quality software engineering results that can become a competitive advantage and ultimately benefit the business’s bottom line.

By dedicating attention to improving the DevEx, companies will also be able to bring more talent aboard, reduce their “Time to First Contribution,” enable developers to provide higher output contributions and improve talent retention. It’s time to start focusing on superior developer experiences.

Frequently asked questions

What does business critical mean?

Business critical means essential for a business to operate and be successful. It refers to business processes, systems, or activities that are considered essential for proper functioning, success, or survival. If these elements are disrupted or compromised, the impact can significantly affect its operations, financial health, or overall business objectives. Business-critical components are often prioritized for attention, protection, and resource allocation to ensure the organization’s continued effectiveness and stability, and to remain competitive.

What is the role of a Developer Experience Engineer (DXE)?

Developer Experience Engineers are internal advocates indispensable to making life easier for developers by making workflows more efficient, implementing the same processes across the development team, and assisting in choosing tools to lighten their load. Some companies do not necessarily have engineers with that job title, and it’s instead seen as part of DevOps or Platform Engineering initiatives. Other times, the internal developer advocate natural rises from the developer community to help form an integral part of the developer team’s unity. They also champion communication and work with cross-function coworkers and feel safe means of providing constructive feedback.

How do we cultivate a positive developer community?

The importance of DevEx’s human element is now in focus. A productive and effective developer community could be seen as a flourishing habitat, strengthening cooperation and maximizing productivity through collaboration. Such an atmosphere also sustains morale among the developers, resulting in job satisfaction and contentment.

Here are some steps to build a developer community: open communication needs to become standard, successes should consistently be recognized, and avenues for growth need to exist. Diversity and acceptance are significant. Task expectations should be attainable goals and set high enough not to compromise quality but not realistic so they can be achieved without burnout and unbalanced life routines.

How does technology impact developer experience?

Technological advancements continue to change software development and impact the Developer experience. AI, ML, TDD, and internal developer platforms are some of these technologies that help reduce developers’ cognitive load. They automate complex tasks for more accuracy and enable intelligent decision-making. Besides this, generative AI has come into play in creating higher-quality code faster. At the same time, GitOps is a management approach based on one source of truth - Git, which makes collaboration smoother & boosts productivity by allowing automation. All these things undoubtedly improve the DX substantially for any developer out there!

Pulumi Blog: Developer experience

Treating Prompts Like Code: A Content Engineer's AI Workflow

The real problem AI solves

Treating prompts like code

The skill catalog

Honest limitations

Lessons to share

What’s next

Golden Paths in IDPs: A Complete Guide to Reusable Infrastructure with Pulumi Components and Templates

The Platform Engineering Layer Cake: A Model for IDPs

Layer 1: Infrastructure Layer

Layer 2: Platform Layer - Components

Part 1: Building Reusable Infrastructure Components

The Power of Multi-Language Components

Building a Microservice Component Step-by-Step

Documenting for Discoverability and Adoption

Publishing Your Component via Private Registry

Part 2: From Building Blocks to Golden Paths

Golden Path Maturity: From Zero to Product-Grade Platforms

Stage 1: No Golden Paths

Stage 2: Dawn of Templates

Stage 3: Templates as Products

What Makes a Golden Path Golden?

Building a Go Microservice Golden Path

Step 1: Template Structure

Step 2: Application Scaffolding

Step 3: Infrastructure Template with YAML

Step 4: Documentation and Examples

Making Templates Available in Pulumi IDP

Best Practices for Reusable Infrastructure Components and Templates

1. Design for Day 2 Operations from Day 1

2. Expose Complexity Progressively

4. Use Semantic Versioning Everywhere

5. Test Your Abstractions

How to Measure Golden Path Success

Adoption Metrics

Quality Metrics

Developer Experience Metrics

Real-World Results: Success Stories

Common Pitfalls and How to Avoid Them

Pitfall 1: Over-Abstraction

Pitfall 2: Insufficient Escape Hatches

Pitfall 3: Poor Versioning Strategy

Pitfall 4: Lack of Ownership

The Future of Golden Paths: AI and Beyond

AI-Enhanced Templates

Cross-Cloud Portability

GitOps-Native Workflows

Your First Steps to Golden Paths

Conclusion: From Fragmentation to Flow

Ready to Build Your Golden Paths?

How to Build an Internal Developer Platform: Strategy, Best Practices, and Self-Service Infrastructure

Understanding the Platform Engineering Layers in Your Internal Developer Platform

Infrastructure Layer → Raw Cloud Resources

Platform Layer → Pulumi Components

Developer Experience Layer → Templates + Private Registry

IDP Example: Building a Web Application Platform Step-by-Step

Why Internal Developer Platforms Matter in 2025 and Beyond

5 Core Components of a Successful Internal Developer Platform

1. Abstractions

2. Blueprints

3. Workflows

4. Security guardrails

5. Self-service

How to Implement Your IDP Strategy

Supporting Different Personas in Your Internal Developer Platform Design

Why a Private Registry Is Key to IDP Adoption

Aligning Your IDP to Organizational Context and Services

How to Measure the Success of Your Internal Developer Platform

IDP Strategy: 5 Key Steps to Build a Strong Foundation

The Future of IDPs and Platform Engineering

Ready to Build a Successful IDP that Scales?

AI-Assisted Infrastructure as Code with Pulumi's Model Context Protocol Server

Setting up the Pulumi MCP Integration in Cursor

The Goal: Provisioning an AKS Cluster

The Traditional Approach

The AI Assistant + Pulumi MCP Approach

Understanding the AI Assistant + Tool Interaction

Step-by-Step Walkthrough

The Request