Pulumi Blog: IDP Best Practices Series

Day 2 Operations: Drift Detection and Remediation

Mitch Gerdisch — Fri, 24 Oct 2025 00:00:00 +0000

Welcome to the fourth post in our IDP Best Practices series. Today we’re diving into the world of drift detection and remediation, those critical day 2 operations that keep your infrastructure aligned with its intended configuration long after the initial deployment.

You’ve built a beautiful platform with robust guardrails, comprehensive templates, and well-defined golden paths. Your developers are productive, deployments are smooth, and everything seems perfect. Then reality hits. An on-call engineer makes an emergency change through the AWS console during a 3 AM incident. A team member tweaks a security group rule to debug a connection issue and forgets to revert it. Auto-scaling adjusts capacity based on load patterns. Before you know it, your actual infrastructure has quietly diverged from what your code describes.

This post continues our journey through the IDP Best Practices series. We’ve covered strategy and self-service infrastructure, built golden paths with components and templates, and established deployment guardrails through policy as code. Now we’re tackling what happens after deployment when the real world starts making changes to your carefully crafted infrastructure.

Series roadmap:

How to Build an Internal Developer Platform: Strategy, Best Practices, and Self-Service Infrastructure
Build Golden Paths with Infrastructure Components and Templates
Deployment Guardrails with Policy as Code
Day 2 Operations: Drift Detection and Remediation (you are here)
Extend Your IDP for AI Applications: GPUs, Models, and Cost Controls
Next-Gen IDPs: How to Modernize Legacy Infrastructure with Pulumi

The Reality of Infrastructure Drift

Let me paint you a picture that might feel uncomfortably familiar. It’s Saturday afternoon, and you’re enjoying a barbecue with friends when your phone buzzes with that dreaded alert: production is down. You rush to your laptop, but you’re not on the corporate VPN. The AWS console is accessible, but you can’t reach the application itself to properly diagnose the issue.

Faced with a growing incident and impatient stakeholders, you make a quick decision. You modify the security group directly in the console, temporarily allowing access from anywhere (0.0.0.0/0) just so you can get in and fix the problem. Twenty minutes later, the issue is resolved, everyone’s relieved, and you return to your barbecue. That temporary security group change? It stays there, a silent time bomb waiting to be discovered in your next security audit.

This scenario plays out in organizations every day. Your infrastructure has now drifted from its intended state, and it will continue drifting, accumulating more changes, until someone or something notices and fixes it. Each untracked change increases the gap between what you think you have and what actually exists in production.

Understanding the Prevention vs. Detection Paradigm

When building a robust platform, you need to think in layers of defense. Your first instinct might be to prevent all drift from happening in the first place, and that’s absolutely the right starting point. But prevention alone isn’t enough in the messy reality of production operations.

Prevention: The First Line of Defense

Prevention is what we’ve been building throughout this series. You establish code reviews where teammates scrutinize infrastructure changes before they go live. You implement automated testing that validates configurations before deployment. You deploy CrossGuard policies that act as guardrails, blocking misconfigurations before they can cause damage. You create standardized components that encode best practices, making it easier to do the right thing than the wrong thing.

These preventative measures work brilliantly for changes that flow through your normal IaC pipeline. They catch typos, enforce standards, and maintain consistency across your infrastructure. But they have a critical blind spot: they can’t see changes that bypass your pipeline entirely.

Detection and Remediation: The Safety Net

This is where drift detection enters the picture. Think of it as continuous verification, constantly comparing what should exist with what actually exists. While prevention blocks most issues at the gate, detection catches everything that finds another way in.

What kinds of changes slip past prevention? Emergency modifications made during incidents when there’s no time for a proper code review. Console changes made by well-meaning engineers debugging production issues. External factors you can’t control, like AWS changing service limits or deprecating API endpoints. Even accidental modifications by team members who thought they were in the staging environment.

The truth is, no matter how robust your preventative controls, you need detective controls to maintain infrastructure integrity. Prevention and detection work together, creating a comprehensive defense against infrastructure chaos.

What Is Infrastructure Drift?

Infrastructure drift sounds technical, but the concept is surprisingly simple. Drift occurs when your actual infrastructure state doesn’t match your declared state in code. It’s the gap between intention and reality, between what you coded and what’s actually running.

To understand drift properly, you need to grasp three distinct states that Pulumi manages. First, there’s the desired state, which is what your Pulumi program declares in code. This is your intention, your blueprint for how things should be. Then there’s the current state, stored in Pulumi’s state file, which tracks what Pulumi believes exists based on its last interaction with your cloud provider. Finally, there’s the actual state, the ground truth of what really exists in your cloud provider right now.

The journey to drift typically follows a predictable pattern. You start by running pulumi up to deploy your resources, and everything is in perfect harmony. Pulumi creates the resources, records their configuration in its state file, and all three states align perfectly. Time passes, and your resources continue running in the cloud, serving traffic and doing their job.

Then someone makes a manual change. Maybe they modify a resource directly in the console, or an external system alters a configuration. Suddenly, your actual state has diverged from both your current state (what Pulumi thinks exists) and your desired state (what your code says should exist). You now have drift, and it will persist until you detect and address it.

Pulumi’s Drift Detection and Remediation Workflow

Pulumi provides a complete workflow for handling drift that’s both powerful and surprisingly straightforward. Rather than treating drift as an exceptional case, Pulumi makes it a routine part of infrastructure management.

1. Drift Detection with `pulumi refresh`

The foundation of drift detection is the pulumi refresh command. When you run this command, Pulumi springs into action, querying your cloud providers to discover the actual state of your resources. It then compares this reality against what it has recorded in its state file:

# Run drift detection
pulumi refresh --preview-only

# This command:
# 1. Reads your current Pulumi state
# 2. Queries cloud providers for actual resource configurations
# 3. Identifies any differences
# 4. Reports what has drifted

The --preview-only flag is particularly useful here. It shows you exactly what’s drifted without making any changes to your state file. You get a clear report of the differences, allowing you to review and understand the drift before deciding how to handle it.

2. State Reconciliation with `pulumi refresh`

After reviewing the drift, you face a decision. If the changes are legitimate and should be preserved, you can update your state file to match reality:

# Update state file to match cloud reality
pulumi refresh

# This command:
# 1. Detects drift (as above)
# 2. Updates state file to match actual cloud state
# 3. Prepares for remediation via pulumi up

This reconciliation step is crucial. It acknowledges the current reality and updates Pulumi’s understanding of your infrastructure without making any actual changes to your resources. You’re essentially telling Pulumi, “This is what actually exists right now.”

3. Drift Remediation with `pulumi up`

With your state file now reflecting reality, you can use pulumi up to remediate the drift. This command compares the updated state against your code and determines what changes are needed to restore your infrastructure to its intended configuration:

# Remediate drift - restore to desired state
pulumi up

# This command:
# 1. Compares updated state with your Pulumi program
# 2. Identifies required changes
# 3. Applies changes to restore infrastructure

This three-step dance of detect, reconcile, and remediate gives you complete control over how you handle drift. You can review changes, decide whether to keep or revert them, and ensure your infrastructure returns to its intended state.

Automating Drift Detection with Pulumi Deployments

Running pulumi refresh manually works great for small teams and simple infrastructures, but it doesn’t scale. As your platform grows to dozens or hundreds of stacks across multiple environments, manual drift detection becomes impossible. This is where Pulumi Deployments transforms drift detection from a manual chore into an automated safeguard.

Setting Up Automated Drift Detection

The journey to automated drift detection starts with connecting your infrastructure code to Pulumi’s deployment system. Let’s walk through the setup process.

Step 1: Configure Deployment Settings

Your first step is connecting Pulumi to your source control system using Deployment Settings. This integration allows Pulumi to access your infrastructure code and run automated operations:

Pulumi offers native integrations with all major version control systems. If you’re using GitHub, you get full app integration with PR previews, making it easy to see the impact of changes before merging. GitLab users enjoy similar functionality with merge request automation. For other Git providers or self-hosted solutions, you can use raw Git integration with direct repository access using credentials.

Step 2: Create Drift Detection Schedules

With your source control connected, you can now configure automated drift detection to run on whatever schedule makes sense for your organization. Some teams check hourly for production environments, while others might run daily checks for development stacks:

For environments where you’re confident in your automation, you can take it a step further and enable automatic remediation:

With auto-remediation enabled, Pulumi not only detects drift but automatically runs pulumi up to restore your infrastructure to its intended state. This is powerful but should be used carefully, especially in production environments.

Step 3: Configure Webhooks for Notifications

Detecting drift is only valuable if the right people know about it. Pulumi’s webhook system ensures your team stays informed when drift occurs:

You can route notifications wherever your team actually pays attention. Send alerts directly to Slack channels where your ops team congregates. Push notifications to Microsoft Teams if that’s your collaboration platform. Use custom webhooks to integrate with PagerDuty, Datadog, or any other monitoring system. You can even use deployment triggers to automatically run dependent stacks when drift is detected and fixed.

Handling Common Drift Scenarios

Every organization faces similar drift scenarios. Understanding how to handle these common cases will prepare you for real-world operations.

Scenario 1: Emergency Hotfix

The most common drift scenario occurs during incidents. Someone makes an emergency change to fix production, and now you need to decide whether to keep or revert that change:

# 1. Acknowledge the drift
pulumi refresh --stack production

# 2. Review the changes
pulumi preview --stack production

# 3. Either incorporate or revert
# Option A: Accept the change
pulumi refresh --stack production --yes

# Option B: Revert to code
pulumi up --stack production --yes

The key here is having a clear process. First, acknowledge the drift by running refresh to update your state. Then review the changes carefully. Finally, make an explicit decision: either accept the change by updating your code to match, or revert it by running pulumi up to restore the original configuration.

Scenario 2: Legitimate External Changes

Not all drift is bad. Auto-scaling groups adjust capacity based on load. Managed services update configurations automatically. Some changes are legitimate and should be preserved:

// Mark resources that can change externally
const autoScalingGroup = new aws.autoscaling.Group("app-asg", {
 minSize: 2,
 maxSize: 10,
 // ... other config
}, {
 ignoreChanges: ["desiredCapacity"] // Ignore scaling changes
});

The ignoreChanges option tells Pulumi to ignore specific properties when detecting drift. Use it for fields that legitimately change outside your control, like auto-scaling capacity or last-modified timestamps.

Scenario 3: Configuration Drift

Sometimes drift occurs in configuration values rather than resource structure. IP allowlists, feature flags, and environment-specific settings often drift. Handle these with Pulumi’s configuration system:

# Use Pulumi's configuration system
config = pulumi.Config()
allowed_ips = config.get_object("allowed_ips") or ["10.0.0.0/8"]

security_group = aws.ec2.SecurityGroup("app-sg",
 ingress=[{
 "from_port": 443,
 "to_port": 443,
 "protocol": "tcp",
 "cidr_blocks": allowed_ips # Managed via configuration
 }]
)

By externalizing configuration values, you make them easier to update without code changes. This pattern works particularly well for values that change frequently or differ between environments.

The Value of Automated Drift Detection

After implementing drift detection for hundreds of stacks, patterns emerge that tell a compelling story about its value. The transformation begins subtly but grows profound as teams discover what they’ve been missing.

The first change most organizations notice is in their security posture. Where once unauthorized changes might lurk undetected for weeks, only surfacing during quarterly audits or worse, during incident investigations, now they’re caught within minutes. Security teams who previously dreaded audit season find themselves with continuous compliance data at their fingertips. Every change, whether legitimate or not, creates an audit trail that transforms compliance reporting from a scrambling fire drill into a routine export of already-collected data.

As teams settle into this new reality, operational improvements become apparent. Incidents that used to take hours to diagnose now resolve in minutes because engineers can immediately check whether drift contributed to the problem. One team reported that their mean time to recovery dropped by 40% simply because they could eliminate drift as a cause within seconds rather than spending hours manually comparing configurations. Change management, once a bureaucratic nightmare of spreadsheets and approval chains, transforms into an automated system where every modification is tracked, timestamped, and traceable to its source.

Then come the unexpected discoveries. Drift detection becomes a cost optimization tool that nobody anticipated. It uncovers orphaned resources spinning away forgotten in some corner of your cloud account, created during a debugging session six months ago. It catches that instance someone manually upgraded to a larger size during a performance investigation and forgot to downsize. One organization discovered they were spending $15,000 monthly on resources that weren’t even supposed to exist, all caught by their drift detection system in its first week of operation.

Perhaps most importantly, drift detection changes how teams work. Engineers stop wasting time on manual infrastructure audits. Debugging accelerates when you have clear reports showing exactly what changed and when. The constant anxiety about whether production matches your code disappears, replaced by confidence that any discrepancies will be caught and reported immediately. Teams report feeling more empowered to move quickly, knowing they have a safety net that will catch configuration drift before it causes problems.

The cumulative effect is transformative. Organizations running drift detection report not just fewer incidents and lower costs, but a fundamental shift in how they think about infrastructure management. It becomes proactive rather than reactive, confident rather than anxious, automated rather than manual. What starts as a simple scheduled check evolves into a cornerstone of operational excellence.

Getting Started with Drift Detection

Ready to implement drift detection for your IDP? Here’s a practical action plan that you can execute today.

Quick Start Checklist

Begin with the simplest possible drift detection: a manual check on your production stack. Run pulumi refresh --preview-only --stack production to see if you have any existing drift. This takes just 15 minutes and often reveals surprising discrepancies.

Next, invest 30 minutes in setting up automated detection. Configure Pulumi Deployments, create an hourly detection schedule for your most critical stack, and set up Slack notifications so you’ll know immediately when drift occurs.

Spend the next week monitoring and learning. Review the drift patterns that emerge. Identify common causes. Are certain resources drifting repeatedly? Are changes happening at predictable times? Document which changes are legitimate and which represent problems.

After a week of observation, you’re ready to implement remediation. Start with non-production environments where mistakes have lower impact. Enable auto-remediation gradually, monitoring success rates and adjusting your configuration based on what you learn.

Finally, optimize and scale your drift detection. Adjust schedules based on the patterns you’ve observed. Implement custom workflows for complex scenarios. Expand coverage to all your environments and stacks. What starts as a simple check becomes a comprehensive system protecting your entire infrastructure.

Conclusion: Day 2 Operations as a Competitive Advantage

Drift detection and remediation represent more than just good hygiene; they’re competitive advantages in a world where infrastructure complexity keeps growing.

Teams with robust day 2 operations move faster because they have confidence in their infrastructure state. They recover quickly from incidents because they can immediately identify what changed. They maintain security without slowing development by automatically detecting and reverting unauthorized modifications. Most importantly, they scale operations without scaling headcount, automating work that would otherwise require armies of engineers manually checking infrastructure.

The math is compelling. A single undetected security group change could lead to a breach costing millions. One overlooked configuration drift might cause hours of downtime. Yet implementing comprehensive drift detection takes just hours of setup and minutes of ongoing maintenance. The ROI is immediate and substantial.

Our IDP journey continues with “Extend Your IDP for AI Applications: GPUs, Models, and Cost Controls”. As AI workloads become central to modern applications, we’ll explore how to adapt your platform for machine learning workflows. You’ll learn about GPU orchestration, model deployment pipelines, and the unique cost management challenges that AI infrastructure presents.

But don’t wait for the next post to get started with drift detection. Even a simple hourly check can prevent major incidents. Set up basic detection today, and your future self will thank you the next time you’re debugging a production issue at 3 AM. More importantly, your on-call team will appreciate not having to debug issues caused by accumulated drift that could have been prevented.

Ready to implement drift detection? Start your free Pulumi Cloud trial and set up your first drift detection schedule in minutes. No credit card required.

How to Implement Robust Security Guardrails Using Policy as Code

Adam Gordon Bell — Tue, 30 Sep 2025 00:00:00 +0000

Welcome to the third post in our IDP Best Practices series, where we explore how to implement policy as code with Pulumi CrossGuard to create deployment guardrails that make self-service infrastructure both powerful and safe.

Platform engineering presents a fundamental tension: we want to enable developer velocity while maintaining security and compliance. Every platform team faces the same question: how do you give teams the freedom to deploy infrastructure quickly without compromising on safety, security, or organizational standards? The answer isn’t to choose between speed and safety, but rather to embrace automated guardrails powered by policy as code that make both possible simultaneously.

This post is part of our IDP Best Practices series:

How to Build an Internal Developer Platform: Strategy, Best Practices, and Self-Service Infrastructure
Build Golden Paths with Infrastructure Components and Templates
Deployment Guardrails with Policy as Code (you are here)
Day 2 Operations: Drift Detection and Remediation
Extend Your IDP for AI Applications: GPUs, Models, and Cost Controls
Next-Gen IDPs: How to Modernize Legacy Infrastructure with Pulumi

Want hands-on experience? Access the complete demo code and policy examples from this workshop.

The Platform Engineering Challenge: Speed vs. Safety

Let me tell you a story that perfectly captures the platform engineering dilemma. Statsig, a fast-growing feature flag platform processing an incredible 2 trillion events daily, had one infrastructure engineer named Jason who handled all infrastructure requests. Everything worked smoothly until Jason went on parental leave. Suddenly, the entire engineering organization ground to a halt. Infrastructure requests piled up, deployments slowed to a crawl, and the team faced a stark realization: their entire infrastructure capability depended on a single person.

This crisis forced them to confront a fundamental question that every growing engineering team eventually faces: How do you enable self-service infrastructure without sacrificing security, compliance, or operational stability? The answer wasn’t to hire more Jasons or to lock down infrastructure even tighter. Instead, they discovered that deployment guardrails with policy as code could transform their infrastructure conversation from “talk to the infrastructure person” to “ship with confidence, knowing guardrails will catch any issues.”

Think of it like building a system of roads with guardrails. When you create well-designed infrastructure components with proper safety barriers, teams can drive fast and confidently, knowing they’re protected from going off the cliff. The guardrails don’t slow them down; they enable speed by removing fear.

Understanding Platform Engineering Layers

Before we dive into implementing guardrails, we need to understand the architecture of a modern platform and where policies fit within it. Think of your platform as having three distinct layers, each with different security and access requirements.

At the foundation, you have your Layer 1: Foundational Infrastructure, which includes security controls, shared networking, and identity management systems like OIDC and IAM. Platform teams typically manage this layer with strict access controls because it forms the security backbone of your entire infrastructure.

Building on that foundation is Layer 2: Shared Infrastructure, which encompasses VPCs, compute platforms, and load balancers. These are standardized components that teams can use but with some room for customization based on specific needs.

The most dynamic layer is Layer 3: The Workloads Layer, where deployable artifacts, pipelines, and operability tools live. This is where the rubber meets the road, where most self-service activity happens and, consequently, where guardrails become absolutely critical. Developers need maximum freedom at this layer to innovate and ship quickly, but this freedom also introduces the most risk. It’s precisely here that policy-driven guardrails prove their worth, enabling that freedom while automatically preventing dangerous configurations.

What Are Deployment Guardrails?

Deployment guardrails are automated policies that act as your infrastructure’s safety net. Rather than relying on manual reviews or hoping developers remember all the security requirements, guardrails automatically prevent misconfigurations before they reach production. They enforce security standards without human intervention, guide developers toward best practices through immediate feedback, and enable safe self-service by catching issues at the earliest possible moment.

A helpful analogy is to think of guardrails like type checking in programming languages. Just as TypeScript doesn’t restrict your ability to write JavaScript but rather catches type errors before runtime, deployment guardrails don’t limit your infrastructure creativity. They simply ensure you’re following secure patterns and catch potentially dangerous configurations before they cause problems in production.

Introducing Pulumi CrossGuard: Policy as Code

Pulumi CrossGuard is Pulumi’s policy as code framework that brings the same engineering rigor to compliance and security that you apply to your application code. Instead of maintaining policy documents in wikis or relying on manual reviews, you can write policies in familiar programming languages like Python, TypeScript, or Go. These policies then enforce themselves across all your cloud resources and providers, running at different stages of the deployment lifecycle and integrating seamlessly with your CI/CD pipelines for automated enforcement.

Key Policy Types

CrossGuard supports two fundamental types of policies, each serving different validation needs:

Resource Policies: Validate individual resources

def restrict_dangerous_ports(args: ResourceValidationArgs, report_violation: ReportViolation):
 if args.resource_type == "aws:lb/targetGroup:TargetGroup":
 port = args.props.get("port")
 dangerous_ports = ["22", "23", "3389"]
 if str(port) in dangerous_ports:
 report_violation("Dangerous port detected. Avoid using SSH, Telnet, or RDP ports.")

Stack Policies: Validate relationships across resources

def validate_microservice_encryption(args: StackValidationArgs, report_violation: ReportViolation):
 microservice_components = []
 s3_buckets = []

 for resource in args.resources:
 if resource.resource_type == "custom:infrastructure:Microservice":
 microservice_components.append(resource)
 elif resource.resource_type == "aws:s3/bucket:Bucket":
 s3_buckets.append(resource)

 if microservice_components and s3_buckets:
 for bucket in s3_buckets:
 encryption = bucket.props.get("serverSideEncryptionConfiguration")
 if not encryption:
 report_violation("S3 bucket must have encryption enabled when used with microservice components.")

Building Practical Guardrails: Real-World Examples

Let’s walk through building guardrails for a microservice platform, starting with the component from our previous workshop.

Example 1: Port Security Policy

Problem: Developers might accidentally expose services on dangerous ports like SSH (22) or RDP (3389).

Solution: A policy that blocks dangerous port configurations.

import pulumi_policy as policy

def restrict_dangerous_ports_validation(args, report_violation):
 """Prevent services from running on dangerous ports."""
 if args.resource_type == "aws:lb/targetGroup:TargetGroup":
 port = args.props.get("port")
 dangerous_ports = ["22", "23", "3389", "5432", "3306"]

 if str(port) in dangerous_ports:
 report_violation(
 f"Port {port} is not allowed. This port is commonly used for "
 f"administrative services and should not be exposed via load balancer."
 )

restrict_dangerous_ports = policy.ResourceValidationPolicy(
 name="restrict-dangerous-ports",
 description="Prevent services from using dangerous ports",
 validate=restrict_dangerous_ports_validation,
 enforcement_level=policy.EnforcementLevel.MANDATORY
)

Example 2: Resource Limits Policy

Problem: Teams might request oversized resources, leading to cost overruns.

Solution: A policy that enforces reasonable resource limits with advisory warnings.

def limit_memory_validation(args, report_violation):
 """Limit memory allocation for microservices."""
 if args.resource_type == "custom:infrastructure:Microservice":
 # Extract memory from component tags or properties
 memory_str = args.props.get("memory", "512Mi")

 # Parse memory value (assuming format like "2Gi", "1024Mi")
 if memory_str.endswith("Gi"):
 memory_gb = float(memory_str[:-2])
 if memory_gb > 1:
 report_violation(
 f"Memory allocation of {memory_str} exceeds recommended limit of 1Gi "
 f"for microservices. Consider optimizing your application or "
 f"contact the platform team for exceptions."
 )

limit_memory = policy.ResourceValidationPolicy(
 name="limit-microservice-memory",
 description="Enforce reasonable memory limits for microservices",
 validate=limit_memory_validation,
 enforcement_level=policy.EnforcementLevel.ADVISORY
)

Example 3: Cross-Resource Security Policy

Problem: When microservices use S3 buckets, encryption should be mandatory.

Solution: A stack policy that validates encryption across related resources.

def microservice_s3_encryption_validation(args, report_violation):
 """Ensure S3 buckets used with microservices have encryption enabled."""
 microservice_resources = []
 s3_buckets = []

 # Collect relevant resources
 for resource in args.resources:
 if resource.resource_type == "custom:infrastructure:Microservice":
 microservice_resources.append(resource)
 elif resource.resource_type == "aws:s3/bucket:Bucket":
 s3_buckets.append(resource)

 # If we have both microservices and S3 buckets, check encryption
 if microservice_resources and s3_buckets:
 for bucket in s3_buckets:
 sse_config = bucket.props.get("serverSideEncryptionConfiguration")
 if not sse_config:
 report_violation(
 f"S3 bucket '{bucket.name}' must have server-side encryption "
 f"enabled when used with microservice components."
 )

microservice_s3_encryption = policy.StackValidationPolicy(
 name="microservice-s3-encryption",
 description="Ensure S3 buckets used with microservices are encrypted",
 validate=microservice_s3_encryption_validation,
 enforcement_level=policy.EnforcementLevel.MANDATORY
)

Policy Enforcement Models

Pulumi CrossGuard supports multiple enforcement models to fit different workflows, and understanding when to use each model is crucial for effective policy implementation.

The Preventative Model

The most common approach is the preventative model, where policies run before resources are created. This gives you fast feedback and prevents bad deployments from ever reaching the cloud. When you run pulumi preview, policies are evaluated immediately, and if violations are found during pulumi up, the deployment is blocked. This model works particularly well for validating port configurations, resource limits, and other settings that you can check without needing to see the actual deployed resource.

# Policies run automatically during preview and deployment
pulumi preview # Policies evaluated here
pulumi up # Policies block deployment if violations found

The Detective Model

Sometimes you need to validate the actual cloud state, including auto-generated values like ARNs or IP addresses. That’s where the detective model comes in. These policies run after resources are created, making them perfect for compliance scanning and drift detection. While they can’t prevent initial deployment of non-compliant resources, they excel at catching configuration drift and validating properties that only exist after deployment.

CI/CD Integration

The third model integrates policies directly into your deployment pipeline. This ensures consistent enforcement across all teams and creates natural deployment gates. For example, you might configure GitHub Actions to run policy validation on every pull request, blocking merges if violations are found. This approach combines the best of both worlds: early feedback during development and guaranteed enforcement before production.

# GitHub Actions example
- name: Run Policy Validation
 run: |
 pulumi preview --policy-pack ./policies
 if [ $? -ne 0 ]; then
 echo "Policy violations found. Deployment blocked."
 exit 1
 fi

Policy Remediation: Beyond Detection

Modern policy frameworks don’t just detect violations; they can automatically fix them:

def auto_tag_resources(args, report_violation):
 """Automatically add required tags to resources."""
 if args.resource_type == "aws:s3/bucket:Bucket":
 tags = args.props.get("tags", {})

 if "Department" not in tags:
 # Instead of just reporting, fix the issue
 if not hasattr(args, 'tags'):
 args.props["tags"] = {}
 args.props["tags"]["Department"] = "Engineering"
 return args.props # Return modified properties

 return None # No changes needed

auto_tag_policy = policy.ResourceValidationPolicy(
 name="auto-tag-resources",
 description="Automatically add required tags",
 validate=auto_tag_resources,
 enforcement_level=policy.EnforcementLevel.REMEDIATE
)

Server-Side Policy Enforcement

For enterprise deployments, Pulumi provides server-side policy enforcement that ensures policies can’t be bypassed. The process starts by publishing your policies to your Pulumi organization with pulumi policy publish ./my-policies. Once published, you can create policy groups that combine multiple policies with specific enforcement levels, targeting particular stacks or environments while configuring exceptions for special cases. The beauty of this approach is that policies run automatically without requiring CLI flags, providing consistent governance across your entire organization without relying on developers to remember to include policy packs in their commands.

Compliance-Ready Policies

While custom policies address your specific organizational needs, compliance requirements often follow industry standards. Pulumi provides hundreds of pre-built policies for common compliance frameworks:

import { PolicyManager } from "@pulumi/policy";
import { policyManager } from "@pulumi/compliance-ready-policies/policy-manager";

new PolicyPack("aws-compliance-ready-policies-typescript", {
 policies: [
 ...policyManager.selectPolicies({
 vendors: ["aws"],
 services: ["ec2", "s3"],
 severities: ["medium", "high", "critical"],
 topics: ["encryption"],
 frameworks: ["pcidss"],
 }, "mandatory"),
 ],
});

This automatically includes policies for major compliance frameworks like PCI DSS for payment card industry standards, SOC 2 for security and compliance controls, ISO 27001 for information security management, and CIS Benchmarks for security configuration standards.

Best Practices for Policy Implementation

After implementing policies at dozens of organizations, we’ve learned that successful policy adoption follows predictable patterns. The key is to adopt a product mindset: think of your policies as a product serving your development teams, not as bureaucratic rules imposed from above. Start small and iterate based on real feedback rather than trying to implement a comprehensive policy framework from day one.

Start Small and Iterate

Begin with just two or three critical policies that address your most pressing risks. Use advisory enforcement initially, which warns developers about violations but doesn’t block deployments. This gives your team time to understand and adapt to the policies. Only after gathering feedback and refining the policies should you graduate to mandatory enforcement.

Provide Clear, Actionable Error Messages

Nothing frustrates developers more than cryptic policy violations. Your error messages should explain not just what’s wrong, but how to fix it:

def good_error_message(args, report_violation):
 if violation_detected:
 report_violation(
 "S3 bucket encryption is required for compliance. "
 "Add serverSideEncryptionConfiguration to fix this. "
 "See: https://docs.company.com/s3-encryption"
 )

Embrace Progressive Enforcement

Think of enforcement levels as a dial, not a switch. Start with advisory mode to warn about issues, move to mandatory to block deployments, and eventually implement remediation to automatically fix common issues. This progression gives teams time to adapt while gradually raising the security bar.

Test Your Policies Thoroughly

Policies are code, and like any code, they need testing. Write unit tests for your policies to ensure they catch violations correctly and don’t produce false positives:

# Test policies with unit tests
def test_dangerous_port_policy():
 # Mock resource with port 22
 args = create_mock_args("aws:lb/targetGroup:TargetGroup", {"port": 22})
 violations = run_policy(restrict_dangerous_ports, args)
 assert len(violations) == 1
 assert "dangerous port" in violations[0].message.lower()

Document Escape Hatches

No matter how well-designed your policies are, there will always be edge cases. Rather than forcing teams to work around policies in dangerous ways, provide clear, documented escape hatches. Create processes for requesting exceptions, handling emergency exemptions, and appealing for policy changes. As one platform engineer put it, you need “guardrails outside the guardrails” because platform adoption will never be 100%, and that’s okay. This transparency builds trust and ensures policies remain effective enablers rather than bureaucratic obstacles to circumvent.

Real-World Success: Statsig’s Transformation

Remember Statsig’s infrastructure crisis when Jason went on parental leave? Let’s look at how they transformed their platform using the guardrails approach we’ve been discussing. After implementing policy-driven guardrails, their infrastructure story completely changed.

As Tyrone Wong, Infrastructure Engineer at Statsig, explains: “We had a single infra owner. Parental leave forced us to build self-service, and that’s when everything clicked. Developers open a PR, Pulumi previews the change right in the PR, and CI blocks risky changes. That’s how you move fast safely. The magic was turning ’talk to the infra person’ into ‘ship with guardrails.’”

The results speak for themselves. Deployment times dropped from 1.5 weeks to mere minutes. The infrastructure ticket queue that once haunted their Slack channels disappeared entirely. Security actually improved through automated policy enforcement, eliminating the human error factor. Most importantly, developer satisfaction soared as teams gained the autonomy to ship infrastructure changes without friction or fear.

When asked if developers were actually using the self-service platform, Tyrone’s response was telling: “Yeah, people are actually using it because there’s changes they wanted to get done. Previously it became this giant effort with lots of meetings, people being concerned, and Google docs about what the changes would be. Now people can just put together a PR, we have our discussions there, see if anything gets flagged, and move forward.”

Building Your Policy Strategy

Implementing deployment guardrails isn’t a big-bang transformation; it’s a journey that unfolds in phases. Based on patterns we’ve seen across successful implementations, the first week or two should focus on assessment. Start by understanding your current state, auditing existing infrastructure patterns to identify what teams are actually deploying. Look for common misconfigurations that have caused incidents or near-misses, and document your security and compliance requirements as specific, enforceable rules. Most importantly, survey your developers to understand their pain points with the current infrastructure process, as this assessment forms the foundation for policies that solve real problems rather than creating new ones.

In weeks three and four, build your foundation by implementing three to five core policies that address your most critical risks. Set up CI/CD integration so policies run automatically on every pull request, starting with advisory enforcement to gather feedback without blocking deployments. Create clear documentation and runbooks that explain not just what the policies do, but why they exist and how to work with them.

By the second month, you’re ready to expand. Add compliance-specific policies for regulatory requirements and implement server-side enforcement to ensure policies can’t be bypassed. Create formal processes for policy exemptions and exceptions, and begin measuring policy effectiveness through metrics like violation rates and remediation times.

Remember that policy implementation is never “done.” Continuously monitor violation patterns to identify areas where policies might be too strict or too lenient. Refine policies based on developer feedback and incident data, add automated remediation for common violations to reduce manual fixes, and gradually expand coverage to new services and teams using lessons learned from early adopters.

Measuring Policy Success

You can’t improve what you don’t measure, and policy effectiveness is no exception. Successful teams track metrics across three critical dimensions to validate their policy strategy.

From a security perspective, start by monitoring your policy violation rate: the percentage of deployments that trigger policy warnings or blocks. This tells you whether your policies are catching real issues or creating unnecessary friction. Track time to remediation to understand how quickly teams fix violations when they occur. Most importantly, measure the reduction in security incidents caused by misconfigurations. If your policies aren’t reducing incidents, they’re not addressing the right risks.

Security is only half the equation, though. The developer experience dimension is equally important. Monitor deployment velocity to ensure policies aren’t slowing teams down. Regular developer satisfaction surveys reveal whether teams see policies as helpful guardrails or annoying obstacles. Track support ticket volume; successful policy implementation should reduce infrastructure-related requests as teams become more self-sufficient.

Finally, don’t forget to measure the operational health of your policy system itself. Policy coverage shows what percentage of your resources are protected by policies, while the exemption rate reveals whether policies are practical or if teams constantly need exceptions. Track remediation automation to see how many violations are fixed automatically versus requiring manual intervention. Together, these metrics paint a complete picture of your policy program’s health and help you continuously improve your approach.

Common Pitfalls and How to Avoid Them

Even with the best intentions, policy implementations can go wrong. Through years of helping organizations implement policy as code, we’ve identified four critical pitfalls that can derail your efforts.

The first and most common trap is over-engineering policies. It’s tempting to create sophisticated policies that handle every edge case, but overly complex policies are hard to understand and maintain. Teams won’t trust what they don’t understand. Instead, start with simple, clear policies that address specific risks. Use descriptive naming and provide examples of both compliant and non-compliant configurations. Remember, a simple policy that everyone follows is better than a perfect policy that everyone circumvents.

Equally dangerous is falling into “policy theater”: implementing policies because you think you should, not because they address real risks. This creates friction without improving security. Every policy should trace back to an actual incident, security requirement, or compliance need. If you can’t explain why a policy exists with a concrete example, it probably shouldn’t exist.

Poor developer experience is another common failure point. Nothing kills policy adoption faster than cryptic error messages and unclear remediation steps. When a developer hits a policy violation at 5 PM on a Friday, they need clear guidance on how to fix it. Invest heavily in documentation, helpful error messages, and examples. Consider your policies’ error messages as part of your product’s user experience.

Finally, inadequate testing can erode trust quickly. Policies that break legitimate use cases will be worked around or disabled. Before enforcing any policy, test it thoroughly with real scenarios from your production workloads. Run policies in advisory mode first to catch false positives, and have teams attempt real deployments with policies enabled to ensure they don’t block valid configurations.

The Future of Policy as Code

As infrastructure becomes increasingly complex and distributed, policy as code is evolving beyond simple rule enforcement to become an intelligent layer that understands context and intent. Three major trends are shaping this evolution.

First, we’re seeing the emergence of AI-enhanced policies with smart detection systems that use machine learning to identify anomalies that rule-based policies might miss. These systems learn from your infrastructure patterns and can provide contextual recommendations that adapt based on actual usage. Even more exciting is predictive enforcement: imagine policies that can identify risky patterns before they become violations, guiding developers away from problems they haven’t encountered yet.

Second, the multi-cloud reality is driving the need for universal governance. Organizations need policies that work seamlessly across AWS, Azure, and GCP, with federated enforcement that maintains consistency across multiple cloud accounts and regions. Compliance automation is also maturing, with systems that automatically collect evidence for audits and generate compliance reports without manual intervention.

Finally, policy enforcement is moving closer to where developers actually work. IDE integration will soon provide real-time policy feedback as you write infrastructure code, catching issues before you even attempt to deploy. Self-service exemption workflows will let developers request and receive policy exceptions through automated approval processes. Perhaps most intriguingly, we’re seeing the development of learning policies: systems that improve and adapt based on developer feedback and usage patterns, becoming more helpful over time rather than more restrictive.

Conclusion: Enabling Safe Self-Service at Scale

We started this post with a fundamental tension in platform engineering: the need for both speed and safety. Through the lens of Statsig’s transformation and the technical deep-dive into Pulumi CrossGuard, we’ve seen that this isn’t actually a tension that needs resolving. It’s a false dichotomy that policy as code eliminates entirely.

The key insight from successful platform teams like Statsig is that guardrails don’t restrict freedom; they enable it. When developers know that automated policies will catch dangerous configurations, they gain the confidence to move fast and experiment. When platform teams know that policies automatically enforce security and compliance standards, they can focus on building better platforms instead of reviewing every change. This is the magic of policy as code: it transforms infrastructure governance from a bottleneck into an accelerator.

But perhaps the most important lesson is that policy as code isn’t about saying “no” to developers. It’s about intelligent automation that makes the secure path the path of least resistance. It’s about catching mistakes before they become incidents. It’s about encoding your organization’s hard-won knowledge into systems that help every developer benefit from that experience.

As you embark on your own journey to implement deployment guardrails, remember that perfection isn’t the goal; progress is. Start small, iterate based on feedback, and gradually expand your coverage. Your developers will thank you for the clarity and confidence that comes with well-designed guardrails, and your security team will sleep better knowing that policies are enforced automatically and consistently.

The path from manual reviews to automated guardrails is well-traveled and well-documented. Our complete policy examples provide real-world implementations you can adapt to your needs, while the CrossGuard documentation offers deep technical details for advanced use cases. If you’re on AWS, AWSGuard’s pre-built policies offer immediate value, and our compliance-ready policy catalog addresses specific regulatory requirements.

The future of infrastructure management isn’t about choosing between developer autonomy and operational control. It’s about using policy as code to achieve both, creating platforms that are simultaneously powerful and safe, flexible and compliant, fast and secure.

In our next post, we’ll explore Day 2 Platform Operations, diving into how to maintain infrastructure compliance after deployment and automatically remediate configuration drift. Because getting to production is just the beginning; keeping your infrastructure secure and compliant over time is where the real challenge lies.

Get Started with Pulumi

Use Pulumi's open-source SDK to create, deploy, and manage infrastructure on any cloud.

Golden Paths in IDPs: A Complete Guide to Reusable Infrastructure with Pulumi Components and Templates

Engin Diri — Wed, 20 Aug 2025 10:00:00 +0200

Welcome to the second post in our IDP Best Practices series. In this article, we explore how to create golden paths, pre-architected, reusable infrastructure patterns that help standardize and accelerate cloud development.

Modern cloud platforms offer endless options, over 200 AWS services, sprawling Azure catalogs, and countless DevOps tools. The result? Developers face decision fatigue and inconsistent implementations. Golden paths solve this by providing ready-to-use, production-grade infrastructure that encodes your organization’s best practices, security policies, and operational standards.

In this guide, you’ll learn how to build golden paths for your Internal Developer Platform using two core Pulumi constructs: Components, reusable infrastructure building blocks, and Templates, predefined, deployable patterns. You’ll see how to create infrastructure abstractions that are written once, shared across teams, and consumed in any language, turning weeks of setup into minutes of developer-ready infrastructure.

This post is part of our IDP Best Practices series:

How to Build an Internal Developer Platform: Strategy, Best Practices, and Self-Service Infrastructure
Build Golden Paths: Guide to Reusable Infrastructure with Pulumi Components and Templates (you are here)
Policy as Code for Safer IDPs: Enabling Developer Self-Service with Guardrails
Day 2 Operations: Drift Detection and Remediation
Extend Your IDP for AI Applications: GPUs, Models, and Cost Controls
Next-Gen IDPs: How to Modernize Legacy Infrastructure with Pulumi

The complete code examples from this post are available on GitHub.

The Platform Engineering Layer Cake: A Model for IDPs

To understand where golden paths fit into an Internal Developer Platform (IDP), think in layers. This three-tier model structures your platform to deliver increasing levels of abstraction, reuse, and developer value:

Layer 1: Infrastructure Layer

This is the foundation: raw cloud resources include VMs, databases, networks, and storage, which are the fundamental building blocks from AWS, Azure, GCP, and other providers. Pulumi gives you programmatic access to these resources through native providers, but working at this level requires deep infrastructure knowledge.

Layer 2: Platform Layer - Components

This is where the magic happens. Pulumi Components take those raw resources and package them into higher-level abstractions. Instead of manually configuring 20+ AWS resources for a secure web application, you create a component that handles all that complexity and exposes just the configuration that matters:

const app = new SecureWebApplication("my-app", {
 instanceType: "t3.medium",
 minReplicas: 2,
 maxReplicas: 10,
 enableWAF: true,
 environment: "production"
});

Part 1: Building Reusable Infrastructure Components

Reusable infrastructure components act as the foundation of your platform. They encapsulate complexity while remaining composable and scalable. Let’s build a real-world example: a component that deploys containerized microservices to AWS Fargate.

The Power of Multi-Language Components

Here’s what makes Pulumi components revolutionary: write once, consume anywhere. Your platform team can author components in TypeScript, but application teams can consume them in Python, Go, .NET, Java, or even YAML. This separation of concerns is crucial for scaling platform engineering across diverse teams.

Building a Microservice Component Step-by-Step

Let’s create a component that encapsulates everything needed to deploy a production-ready microservice:

// microservice-component.ts
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";

export interface MicroserviceArgs {
 appPath: string; // Path to application code
 cpu?: number; // CPU units (256, 512, 1024, etc.)
 memory?: number; // Memory in MB
 port?: number; // Application port
 desiredCount?: number; // Number of tasks
}

export class MicroserviceComponent extends pulumi.ComponentResource {
 public readonly url: pulumi.Output<string>;
 public readonly serviceName: pulumi.Output<string>;
 public readonly clusterName: pulumi.Output<string>;

 constructor(name: string, args: MicroserviceArgs, opts?: pulumi.ComponentResourceOptions) {
 super("custom:infrastructure:Microservice", name, {}, opts);

 // Set defaults
 const cpu = args.cpu || 256;
 const memory = args.memory || 512;
 const port = args.port || 8080;
 const desiredCount = args.desiredCount || 2;

 // Create ECR repository for Docker images
 const repository = new awsx.ecr.Repository(`${name}-repo`, {
 forceDelete: true
 }, { parent: this });

 // Build and push Docker image
 const image = new awsx.ecr.Image(`${name}-image`, {
 repositoryUrl: repository.url,
 context: args.appPath,
 platform: "linux/amd64",
 }, { parent: this });

 // Create Application Load Balancer
 const loadBalancer = new awsx.lb.ApplicationLoadBalancer(`${name}-lb`, {
 defaultTargetGroup: {
 port: port,
 protocol: "HTTP",
 healthCheck: {
 path: "/health",
 interval: 30,
 timeout: 5,
 healthyThreshold: 2,
 unhealthyThreshold: 3,
 }
 }
 }, { parent: this });

 // Create ECS cluster
 const cluster = new aws.ecs.Cluster(`${name}-cluster`, {}, { parent: this });

 // Create Fargate service
 const service = new awsx.ecs.FargateService(`${name}-service`, {
 cluster: cluster.arn,
 taskDefinitionArgs: {
 container: {
 name: name,
 image: image.imageUri,
 cpu: cpu,
 memory: memory,
 essential: true,
 portMappings: [{
 containerPort: port,
 targetGroup: loadBalancer.defaultTargetGroup,
 }],
 },
 },
 desiredCount: desiredCount,
 assignPublicIp: true,
 }, { parent: this });

 // Export outputs
 this.url = pulumi.interpolate`http://${loadBalancer.loadBalancer.dnsName}`;
 this.serviceName = service.service.name;
 this.clusterName = cluster.name;

 // Register outputs
 this.registerOutputs({
 url: this.url,
 serviceName: this.serviceName,
 clusterName: this.clusterName,
 });
 }
}

Documenting for Discoverability and Adoption

To make your component easy to use, add comprehensive documentation and examples to a README.md file:

# MicroserviceComponent

Abstraction for resources needed when using AWS container services.

A component to abstract the details related to:
- Creating a docker image and pushing it to AWS ECR.
- Deploy to ECS Fargate using the docker image.

# Inputs

* appPath: Path to local folder containing the app and Dockerfile.
* port: Port to expose via an ALB.
* cpu (Optional): CPU capacity. Defaults to 256 (i.e. 0.25 vCPU).
* memory (Optional): Memory capacity. Defaults to 512 (i.e. 0.5GB).
* containerName (Optional): Name of the container. Defaults to "my-app".

# Outputs

* publicUrl: The DNS name for the loadbalancer fronting the app.

# Usage
## Specify Package in `Pulumi.yaml`

Add the following to your `Pulumi.yaml` file:
Note: If no version is specified, the latest version will be used.

... omit for brevity ...

This documentation will help developers understand how to use your component effectively, including required inputs, outputs, and example usage.

Publishing Your Component via Private Registry

Once your component is ready, publish it to your Pulumi Private Registry:

# Tag your component version
git tag v1.0.1

# Publish to your organization's registry
pulumi package publish https://registry.pulumi.com/myorg/microservice-component

Now any team in your organization can discover and use your component, regardless of their language preference. All they need is to navigate to the Components section in the Pulumi IDP and search for microservice-component.

Part 2: From Building Blocks to Golden Paths

While components are powerful, they’re just the starting point. Golden path templates layer on opinionated scaffolding, workflows, and compliance best practices that guide developers to production.

Golden Path Maturity: From Zero to Product-Grade Platforms

Teams evolve from ad hoc deployments to mature, productized templates with versioning, metrics, and governance. These maturity levels reflect how deeply your platform enables safe, consistent delivery.

Organizations typically progress through three stages of golden path maturity:

Stage 1: No Golden Paths

Every team reinvents the wheel
Inconsistent practices across projects
High cognitive load on developers
Security and compliance gaps

Stage 2: Dawn of Templates

Basic cookie-cutter templates emerge
Some standardization begins
Manual processes still dominate
Limited support and evolution

Stage 3: Templates as Products

Golden paths have dedicated owners
Regular release cycles and versioning
Migration guides for updates
Metrics track adoption and success
Continuous improvement based on feedback

What Makes a Golden Path Golden?

Drawing from Spotify’s pioneering work, golden paths share these characteristics:

Pre-architected and Supported: The platform team owns and supports the path
Optional but Recommended: Developers can deviate, but staying on the path ensures support
Transparent Abstractions: The implementation is visible, not a black box
Extensible: Teams can add project-specific resources without breaking the pattern
Evolutionary: Templates improve based on feedback and new requirements

Building a Go Microservice Golden Path

Let’s create a complete golden path for Go microservices that includes:

Application scaffolding with best practices
Infrastructure deployment using our component
CI/CD pipeline configuration
Observability setup
Security controls

Step 1: Template Structure

go-microservice-boilerplate/
├── Pulumi.yaml # Infrastructure definition
├── src/ # Application code
│ ├── main.go # Go microservice
│ ├── Dockerfile # Container definition
│ └── go.mod # Dependencies
└── README.md # Documentation

Step 2: Application Scaffolding

The src/main.go file contains a minimal Go microservice with all the best practices and compliance guardrails baked in. This helps developers to quickly get started and extend the application without worrying too much about selecting the right libraries or frameworks.

// source/main.go
package main

import (
 "context"
 "flag"
 "net/http"
 "os"
 "os/signal"
 "time"

 "github.com/labstack/echo/v4"
 "github.com/labstack/echo/v4/middleware"
 "go.opentelemetry.io/contrib/instrumentation/github.com/labstack/echo/otelecho"
 "go.opentelemetry.io/otel"
 "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"
 "go.opentelemetry.io/otel/sdk/resource"
 sdktrace "go.opentelemetry.io/otel/sdk/trace"
 semconv "go.opentelemetry.io/otel/semconv/v1.34.0"
)

func initTracer() (*sdktrace.TracerProvider, error) {
 exporter, err := otlptracehttp.New(context.Background())
 if err != nil {
 return nil, err
 }

 resource, err := resource.Merge(
 resource.Default(),
 resource.NewWithAttributes(
 semconv.SchemaURL,
 semconv.ServiceNameKey.String("${PROJECT}"),
 semconv.ServiceVersionKey.String("1.0.0"),
 ),
 )
 if err != nil {
 return nil, err
 }

 tp := sdktrace.NewTracerProvider(
 sdktrace.WithBatcher(exporter),
 sdktrace.WithResource(resource),
 sdktrace.WithSampler(sdktrace.AlwaysSample()),
 )

 otel.SetTracerProvider(tp)
 return tp, nil
}

func echoHandler(c echo.Context) error {
 message := c.QueryParam("message")
 if message == "" {
 message = "Hello from Go microservice!"
 }

 return c.JSON(http.StatusOK, map[string]string{
 "echo": message,
 "service": "${PROJECT}",
 "timestamp": time.Now().UTC().Format(time.RFC3339),
 })
}

func healthHandler(c echo.Context) error {
 return c.JSON(http.StatusOK, map[string]string{
 "status": "healthy",
 "service": "${PROJECT}",
 })
}

func main() {
 healthCheck := flag.Bool("health-check", false, "Run health check and exit")
 flag.Parse()

 if *healthCheck {
 resp, err := http.Get("http://localhost:8080/health")
 if err != nil || resp.StatusCode != http.StatusOK {
 os.Exit(1)
 }
 os.Exit(0)
 }

 tp, err := initTracer()
 if err != nil {
 panic(err)
 }
 defer func() {
 if err := tp.Shutdown(context.Background()); err != nil {
 panic(err)
 }
 }()

 e := echo.New()

 e.Use(middleware.Logger())
 e.Use(middleware.Recover())
 e.Use(middleware.CORS())
 e.Use(otelecho.Middleware("${PROJECT}"))

 e.GET("/echo", echoHandler)
 e.GET("/health", healthHandler)

 go func() {
 if err := e.Start(":8080"); err != nil && err != http.ErrServerClosed {
 e.Logger.Fatal("shutting down the server")
 }
 }()

 quit := make(chan os.Signal, 1)
 signal.Notify(quit, os.Interrupt)
 <-quit
 ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
 defer cancel()
 if err := e.Shutdown(ctx); err != nil {
 e.Logger.Fatal(err)
 }
}

Step 3: Infrastructure Template with YAML

Here’s how you define the infrastructure using Pulumi YAML. You can instantly spot one of the features of Pulumi Components. We consume the micorservice-component in YAML without knowing that it was authored in a different language. This is the power of Pulumi Components: write once, consume anywhere.

And we see another feature of Pulumi Components: As it is a first-class citizen, we can embed the component and add additional resources like auto-scaling policies, alarms, and more directly in the YAML file. This allows us to create a complete golden path for deploying Go microservices on AWS ECS with auto-scaling capabilities.

# Pulumi.yaml
name: ${PROJECT}
description: ${DESCRIPTION}
runtime: yaml
packages:
 component-microservice: https://github.com/smithrobs/component-microservice@v1.0.1
template:
 description: A template for deploying a Go microservice on AWS ECS with auto-scaling capabilities.
resources:
 microserviceComponent:
 type: component-microservice:MicroserviceComponent
 properties:
 appPath: ./src
 port: 8080
 containerName: ${PROJECT}
 ecsTarget:
 type: aws:appautoscaling:Target
 name: ecs_target
 properties:
 maxCapacity: 4
 minCapacity: 1
 resourceId: service/${microserviceComponent.clusterName}/${microserviceComponent.serviceName}
 scalableDimension: ecs:service:DesiredCount
 serviceNamespace: ecs
 ecsPolicyUP:
 type: aws:appautoscaling:Policy
 name: ecs_policy_up
 properties:
 name: ecs_policy_up
 policyType: StepScaling
 resourceId: ${ecsTarget.resourceId}
 scalableDimension: ${ecsTarget.scalableDimension}
 serviceNamespace: ${ecsTarget.serviceNamespace}
 stepScalingPolicyConfiguration:
 adjustmentType: ChangeInCapacity
 cooldown: 60
 metricAggregationType: Maximum
 stepAdjustments:
 - metricIntervalUpperBound: 0
 scalingAdjustment: 1
 ecsPolicyDown:
 type: aws:appautoscaling:Policy
 name: ecs_policy_down
 properties:
 name: ecs_policy_down
 policyType: StepScaling
 resourceId: ${ecsTarget.resourceId}
 scalableDimension: ${ecsTarget.scalableDimension}
 serviceNamespace: ${ecsTarget.serviceNamespace}
 stepScalingPolicyConfiguration:
 adjustmentType: ChangeInCapacity
 cooldown: 60
 metricAggregationType: Maximum
 stepAdjustments:
 - metricIntervalLowerBound: 0
 scalingAdjustment: -1
 serviceCPUHighUtilization:
 type: aws:cloudwatch:MetricAlarm
 name: service_cpu_high_utilization
 properties:
 name: service_cpu_high_utilization
 comparisonOperator: GreaterThanOrEqualToThreshold
 evaluationPeriods: 2
 metricName: CPUUtilization
 namespace: AWS/ECS
 period: 60
 statistic: Average
 threshold: 80
 alarmActions:
 - ${ecsPolicyUP.arn}
 dimensions:
 ClusterName: ${microserviceComponent.clusterName}
 ServiceName: ${microserviceComponent.serviceName}
 serviceCPULowUtilization:
 type: aws:cloudwatch:MetricAlarm
 name: service_cpu_low_utilization
 properties:
 name: service_cpu_low_utilization
 comparisonOperator: LessThanOrEqualToThreshold
 evaluationPeriods: 2
 metricName: CPUUtilization
 namespace: AWS/ECS
 period: 60
 statistic: Average
 threshold: 10
 alarmActions:
 - ${ecsPolicyDown.arn}
 dimensions:
 ClusterName: ${microserviceComponent.clusterName}
 ServiceName: ${microserviceComponent.serviceName}

outputs:
 publicUrl: ${microserviceComponent.publicUrl}

Step 4: Documentation and Examples

Add a README.md file to your template directory to provide clear instructions on how to use the template, including prerequisites, configuration options, and example commands:

# Go Microservice Golden Path

A **golden path** template that gets your Go microservice from code to production on AWS in minutes, not weeks.

## What You Get

This golden path provides everything your development team needs to deploy production-ready Go microservices:

✅ **Production-ready Go microservice** with Echo framework and OpenTelemetry tracing
✅ **AWS infrastructure that scales** - ECS with auto-scaling from 1-4 instances
✅ **Security by default** - Hardened containers, IAM roles, security groups
✅ **Monitoring built-in** - Health checks, load balancer monitoring, CloudWatch alarms
✅ **One-command deployment** - `pulumi up` handles everything
✅ **No AWS expertise required** - Complex ECS setup abstracted away

## For Development Teams: What to Expect

### Your Experience

**Day 1: Getting Started**

- Clone this repo, run `pulumi up`
- Your service is live on AWS in 5-10 minutes
- Public URL provided automatically - no manual setup needed

**Day 2-N: Development Workflow**

- Write your Go code in `microservice/main.go`
- Test locally with `go run main.go`
- Deploy changes with `pulumi up`
- AWS automatically rebuilds and redeploys your container

**Production Operations**

- Service automatically scales with CPU load (80% up, 10% down)
- Health checks ensure unhealthy containers are replaced
- Load balancer distributes traffic across healthy instances
- Distributed tracing helps debug issues across services

### What's Handled For You

You **don't** need to learn or configure:

- ECS clusters, services, and task definitions
- Application Load Balancers and target groups
- Auto-scaling policies and CloudWatch alarms
- Security groups and IAM roles
- Container registries and image building
- Health check configuration

You **do** focus on:

- Writing your Go application logic
- Adding your business endpoints
- Testing your service locally
- Deploying with confidence

Making Templates Available in Pulumi IDP

Publishing your template to Pulumi IDP enables true self-service. Head to Settings → Integrations → Organization Template Sources and add your template repository.

Once published, developers can discover and deploy it directly from the IDP interface.

Now developers can deploy through multiple interfaces:

CLI Deployment:

pulumi new https://github.com/myorg/go-microservice-boilerplate

No-Code Deployment: Navigate to Pulumi IDP → Templates → Deploy with Pulumi → Configure and launch

Pulumi Deployment: Navigate to Pulumi IDP → Templates → Deploy with Pulumi → Configure and launch

Best Practices for Reusable Infrastructure Components and Templates

Well-designed components and templates are the foundation of scalable, self-service infrastructure. These best practices ensure your abstractions are maintainable, discoverable, and production-ready.

1. Design for Day 2 Operations from Day 1

Think beyond deployment. A golden path must also support ongoing operations:

How will teams safely update their infrastructure?
What happens during scaling events?
How do you handle disaster recovery?
What metrics and logs are needed for troubleshooting?

2. Expose Complexity Progressively

Provide sensible defaults that work, while enabling customization for advanced use cases:

interface ComponentArgs {
 // Required - what users must provide
 appName: string;

 // Common customizations with good defaults
 instanceType?: string; // default: "t3.micro"
 replicas?: number; // default: 2

 // Advanced options for power users
 networkConfig?: NetworkConfig;
 securityPolicies?: SecurityPolicy[];
 customMetrics?: MetricDefinition[];
}

4. Use Semantic Versioning Everywhere

Clear versioning for both components and templates indicate stability and reliability:

Major versions (1.0.0 → 2.0.0): Breaking changes
Minor versions (1.0.0 → 1.1.0): New features, backward compatible
Patch versions (1.0.0 → 1.0.1): Bug fixes

This lets teams adopt updates with confidence and control.

5. Test Your Abstractions

Don’t ship black boxes. Create automated tests to validate key functionality and resource creation. Focus on:

Smoke tests that validate resource existence
Output validation to ensure correctness
Integration tests to confirm end-to-end behavior

import { expect } from "chai";
import * as pulumi from "@pulumi/pulumi";

describe("MicroserviceComponent", () => {
 it("should create required resources", async () => {
 const component = new MicroserviceComponent("test", {
 appPath: "./test-app",
 port: 3000
 });

 const resources = await pulumi.runtime.allResources();
 expect(resources).to.include("aws:ecs/cluster:Cluster");
 expect(resources).to.include("aws:ecs/service:Service");
 expect(resources).to.include("aws:lb/loadBalancer:LoadBalancer");
 });
});

How to Measure Golden Path Success

Golden paths aren’t complete until they deliver measurable value. Use these KPIs to assess performance and drive iteration:

Adoption Metrics

Template usage rate: Percentage of new projects using golden paths
Component reuse: Number of stacks consuming shared components
Time to first deployment: Deployment time and frequency from code to production

Quality Metrics

Security compliance rate: Percentage of deployments passing security policies
Stability: Deployment incident frequency
Mean time to recovery (MTTR): Time to recover from production issues

Developer Experience Metrics

Developer satisfaction: Survey teams about their platform experience
Support ticket volume: Ticket volume related to infrastructure
Contribution: Number of PRs or issues submitted to platform templates

Real-World Results: Success Stories

Organizations using golden paths report significant improvements in speed and reliability:

Snowflake reduced deployment time from 1.5 weeks to less than a day
Mercedes-Benz decreased infrastructure provisioning from weeks to minutes
Starburst Data cut deployment time from 2 weeks to 3 hours

These results show that golden paths are not just developer tools, they’re a competitive advantage. To learn more, download the whitepaper: The Golden Path to Cloud Success: Your IDP Roadmap.

Common Pitfalls and How to Avoid Them

Pitfall 1: Over-Abstraction

Problem: Creating components so abstract they’re unusable Solution: Start with concrete use cases, then generalize based on actual patterns

Pitfall 2: Insufficient Escape Hatches

Problem: Golden paths become golden cages Solution: Always provide ways to extend or override default behavior

Pitfall 3: Poor Versioning Strategy

Problem: Breaking changes without migration paths Solution: Maintain backward compatibility and provide clear upgrade guides

Pitfall 4: Lack of Ownership

Problem: Templates become orphaned and outdated Solution: Assign clear ownership and establish maintenance schedules

The Future of Golden Paths: AI and Beyond

The future of golden paths is intelligent, cross-cloud, and fully integrated with modern workflows.

AI-Enhanced Templates

Imagine templates that adapt based on your application’s actual behavior, automatically tuning resources and configurations for optimal performance and cost.

Cross-Cloud Portability

Components that abstract not just resources but entire cloud providers, enabling true multi-cloud golden paths.

GitOps-Native Workflows

Templates that include not just infrastructure but complete GitOps pipelines, from code commit to production deployment.

Your First Steps to Golden Paths

Ready to get started? Here’s your action plan:

Audit Current Patterns: Document the infrastructure patterns your teams use most frequently
Build one reusable component: Pick your most common pattern and build a reusable component
Create Your First Template: Build a complete golden path for one project type
Gather Feedback: Deploy with a pilot team and iterate based on their experience
Scale Gradually: Expand your component library and template catalog based on demand
Measure and Iterate: Track adoption and continuously improve based on metrics

Conclusion: From Fragmentation to Flow

Golden paths aren’t about restricting creativity or enforcing rigid standards. They’re about removing friction from the development process, encoding expertise into reusable patterns, and empowering developers to move at the speed of business.

By building a library of components and templates, you transform your Internal Developer Platform from a collection of tools into a force multiplier for your entire engineering organization. You give developers the gift of not having to solve solved problems, while maintaining the flexibility to innovate where it matters.

Start small. Solve one problem well. Expand from there. With the right foundation, your developer platform will evolve into a system of self-service, speed, and stability.

Ready to Build Your Golden Paths?

Code examples: Explore golden path examples on GitHub
Build with Components: Pulumi Components documentation
Enable Self-Service: Pulumi IDP documentation
Join the Community: Connect with platform engineers in our Pulumi Slack

Next in our series: Policy as Code for Safer IDPs. You will learn how to add automated guardrails that make sure every deployment meets your security and compliance standards without slowing down development.

How to Build an Internal Developer Platform: Strategy, Best Practices, and Self-Service Infrastructure

Mitch Gerdisch — Thu, 14 Aug 2025 05:13:02 +0200

Welcome to the first post in our IDP Best Practices series. In this guide, we’ll walk through the strategic foundations for designing an Internal Developer Platform that empowers developers without sacrificing governance, security, or operational control.

At Pulumi, we’ve worked with hundreds of teams facing the same core challenge: How do you give developers the infrastructure access they need, while maintaining the governance and security your organization requires?

That tension is at the heart of every IDP conversation. Teams want to move faster and innovate, but also need to stay compliant, control costs, and maintain operational stability.

The good news? You can do both, with a clear strategy and the right approach. This series shares proven best practices for designing, building, and scaling IDPs using Pulumi.

These lessons come from real-world implementations across industries and company sizes—and are built to grow with you.

This post is part of our IDP Best Practices series. You can explore the full series below:

How to Build an Internal Developer Platform: Strategy, Best Practices, and Self-Service Infrastructure (you are here)
Build Golden Paths: Guide to Reusable Infrastructure with Pulumi Components and Templates
Policy as Code for Safer IDPs: Enabling Developer Self-Service with Guardrails
Day 2 Operations: Drift Detection and Remediation
Extend Your IDP for AI Applications: GPUs, Models, and Cost Controls
Next-Gen IDPs: How to Modernize Legacy Infrastructure with Pulumi

Understanding the Platform Engineering Layers in Your Internal Developer Platform

When we work with customers on their platform strategy, we often start by referring to the “platform engineering layer cake.” Here is a quick walkthrough of each layer and how Pulumi IDP constructs fit this layer cake approach:

Layer 1: Infrastructure Layer - This is your raw cloud resources: VMs, databases, networks, storage. These are the fundamental building blocks that exist in AWS, Azure, GCP, and other cloud providers.

Layer 2: Platform Layer - This is where Pulumi Components live. Components take those raw infrastructure resources and package them into higher-level abstractions that encapsulate best practices, security policies, and organizational standards. For example, instead of manually configuring 15 different AWS resources to create a secure web application, you create a component that handles all that complexity and exposes just the configuration options that matter to your developers.

Layer 3: Developer Experience Layer - This is where Pulumi Templates and the Private Registry come into play. Templates provide ready-to-deploy patterns that developers can customize, while the private registry makes everything discoverable and manageable at scale.

In our workshop, we focused specifically on those top two layers because that’s where the transformation happens. This is where you turn raw infrastructure into something developers can actually use productively without becoming infrastructure experts themselves.

Here’s how this maps to what you’re actually building:

Infrastructure Layer → Raw Cloud Resources

Your foundation is still the same: AWS EC2 instances, Azure Virtual Networks, GCP Cloud Functions. Pulumi’s providers give you access to these resources using real programming languages instead of YAML or proprietary DSLs.

Platform Layer → Pulumi Components

This is where you create your reusable building blocks. A component might encapsulate:

A complete microservice infrastructure pattern (load balancer, auto-scaling group, database, monitoring)
A secure data pipeline (storage, processing, access controls, encryption)
A standardized Kubernetes application deployment (ingress, service, deployment, secrets management)

Components are written once in any supported language (TypeScript, Python, Go, .NET, Java) but can be consumed by teams using any of these languages. This is crucial. Your platform team might write components in Go, but your application teams can consume them in Python or TypeScript.

Developer Experience Layer → Templates + Private Registry

Templates are complete, deployable projects that use your components. They provide:

Getting started patterns: “Deploy a new microservice”
Reference architectures: “Deploy a three-tier web application”
Environment patterns: “Set up a complete dev/staging/prod pipeline”

The Private Registry is your distribution mechanism. It’s where components and templates become discoverable, versioned, and manageable. Developers don’t need to hunt through Git repositories or Slack channels to find the right building blocks. Everything is catalogued, documented, and accessible through familiar package management workflows.

IDP Example: Building a Web Application Platform Step-by-Step

Let’s make this concrete with an example from our workshops. Say you want to enable teams to deploy secure web applications:

Infrastructure Layer: You need an Application Load Balancer, EC2 Auto Scaling Group, RDS database, VPC with proper subnets, Security Groups, IAM roles, CloudWatch monitoring, and S3 bucket for static assets. That’s about 20+ individual AWS resources that need to be configured correctly and securely.

Platform Layer (Components): You create a WebApplication component that encapsulates all this complexity. Your component exposes simple configuration options like:

const app = new WebApplication("my-app", {
 instanceType: "t3.medium",
 minSize: 2,
 maxSize: 10,
 databaseSize: "small",
 environment: "production"
});

Behind the scenes, your component handles all the security configurations, networking setup, monitoring, and best practices. You publish this component to your private registry.

Developer Experience Layer (Templates): You create templates like:

web-app-starter: A simple web application using your WebApplication component
microservice-template: API service with database and monitoring
full-stack-template: Frontend, backend, and database in a complete environment

Developers can now run pulumi new web-app-starter and get a production-ready, secure web application in minutes instead of weeks. The template uses your component, which handles all the underlying infrastructure complexity.

This layered approach solves the core challenge we see every platform team struggling with: how do you provide developers with the infrastructure they need without creating operational bottlenecks or compromising security standards.

Why Internal Developer Platforms Matter in 2025 and Beyond

While DevOps brought us incredible advances in how we ship and maintain software, we’ve watched many organizations struggle with the unintended consequences: tool sprawl, Day 2 operational pain, and developers who are drowning in infrastructure complexity.

That’s why platform teams have emerged. You exist to solve these challenges by building tools and workflows that enable your internal customers (the developers) to provision infrastructure and deploy software without getting blocked. But here’s the thing I’ve learned from working with hundreds of platform teams: success isn’t just about the tools you choose. It’s about the strategy behind how you implement them.

5 Core Components of a Successful Internal Developer Platform

Through years of working with platform teams and analyzing what separates successful IDPs from those that struggle to gain adoption, we’ve identified five essential components that every platform strategy needs. These aren’t theoretical concepts. They’re battle-tested patterns that work across organizations of all sizes and industries.

1. Abstractions

Let me be clear about something: abstractions aren’t about dumbing down infrastructure for developers. The abstraction is really where you’re hiding the complexity of the underlying infrastructure from your end users, but you’re doing it intentionally to provide appropriate interfaces for different personas in your organization.

We’ve seen too many platform teams get this wrong by either over-abstracting (creating black boxes that developers can’t customize) or under-abstracting (exposing too much complexity). The sweet spot is creating component resources that encapsulate your infrastructure patterns and best practices into reusable building blocks that can be consumed across different programming languages and deployment scenarios, while still providing escape hatches when needed.

2. Blueprints

Blueprints are your templatized, well-architected patterns that developers can use to bootstrap their infrastructure. But here’s what I want you to understand about blueprints: they’re not just starting points that you throw over the wall to developers. They’re carefully designed patterns that embody your organizational best practices and architectural decisions.

When we see successful platform teams, their templates serve as both accelerators and guardrails. They give developers a fast path to production-ready infrastructure while ensuring that everything they deploy follows your organization’s standards. That’s the power of good blueprint design: speed and compliance working together, not against each other.

3. Workflows

Here’s where many platform teams get trapped: they try to build one workflow to rule them all. But in our experience working with organizations across every industry, you need to support multiple consumption patterns because you have multiple types of users. You need:

No-code workflows for users who want point-and-click deployment (think product managers or junior developers who just need to spin up a database)
Low-code workflows using tools like Pulumi YAML for configuration-driven infrastructure (perfect for developers who understand infrastructure but don’t want to write Go or TypeScript)
Full-code workflows for developers who need maximum flexibility and want to leverage the full power of general-purpose programming languages

The key insight? These aren’t three different platforms. They’re three different interfaces to the same underlying components and templates.

4. Security guardrails

This is probably the most important distinction we can share with you: you want to have guardrails, not gates. Too many platform teams create approval processes and manual checkpoints that completely undermine the self-service promise they’re trying to deliver.

Guardrails are different. They start with the best practices and security built into the reusable abstractions. But even the most secure solutions need a safety net. This safety net is policy-as-code that runs automatically, preventing violations before deployment happens. When we work with customers on implementing Pulumi CrossGuard, we’re not creating new friction. We’re embedding security, compliance, and cost controls directly into the deployment process. The developer gets immediate feedback, and you get the assurance that nothing goes to production without meeting your standards.

5. Self-service

Everything we’ve talked about so far builds toward this moment: enabling developers to provision and manage infrastructure independently. But self-service isn’t just about providing a UI. It’s about carefully orchestrating all the other components to create experiences that are both powerful and intuitive.

When we walked through the demo in our workshop, what you saw was the culmination of thoughtful platform design. Developers can discover available services, deploy infrastructure through whatever interface makes sense for them, and manage their resources over time, all without requiring tickets or manual intervention from platform teams. That’s the promise of a well-designed IDP: developer autonomy without operational chaos.

How to Implement Your IDP Strategy

Supporting Different Personas in Your Internal Developer Platform Design

One of the biggest mistakes we see platform teams make: they try to build for their most sophisticated users first. But here’s what I’ve learned from working with hundreds of organizations: you need to think about all your personas from day one.

In most organizations, you’re serving:

Infrastructure engineers who want full control and flexibility (they’ll use the full-code workflows)
Application developers who need infrastructure but prefer higher-level abstractions (perfect for low-code approaches)
Non-technical users who want simple, point-and-click deployment (your no-code users)

The breakthrough insight that separates successful platform teams from struggling ones is this: all these personas can be served by the same underlying components and templates, just consumed through different interfaces. You don’t need to build three different platforms—you need to build one platform with three different consumption models.

Why a Private Registry Is Key to IDP Adoption

Here’s something we always tell platform teams: if you don’t solve the discoverability problem, your beautiful components and templates will sit unused in Git repositories where no one can find them. That’s why the private registry isn’t just a nice-to-have. It’s the foundation that makes everything else work.

When you establish a private registry as your source of truth for components, templates, providers, and policies, you’re solving two critical problems at once. First, discoverability: developers can actually find and explore what’s available. Second, lifecycle management: you can see where each package is being used, track version drift, and understand the impact of changes before you make them.

The workflow is beautifully simple: you publish standardized building blocks with a single pulumi publish command, and developers discover and consume these assets through familiar package management workflows they already know.

Aligning Your IDP to Organizational Context and Services

One pattern we see in every successful IDP implementation is that the platform reflects how the organization actually works. Your developers don’t think in terms of individual stacks or resources. They think in terms of services, applications, and business functionality.

That’s why we built Pulumi Services - to let teams logically group stacks and ESC environments in ways that make sense to your organization. Pulumi Services improve organization by helping your team model infrastructure in a way that’s familiar - simplifying Day 2 operations by providing additional context like links to relevant dashboards, Slack channels and more. When something breaks at 2 AM, your on-call engineer needs to understand dependencies, track usage, and manage infrastructure in the context of the business services that are actually impacted.

How to Measure the Success of Your Internal Developer Platform

Here’s something that might surprise you: the most successful platform teams we work with don’t measure their success by how many cool features they’ve built. They measure success by the business outcomes they’re driving:

Reduced time-to-market through faster infrastructure provisioning (because at the end of the day, your platform exists to help the business move faster)
Improved developer productivity by eliminating infrastructure bottlenecks (happy developers ship more features)
Enhanced security posture through consistent policy enforcement (security that doesn’t slow people down is security that actually gets used)
Lower operational overhead through standardization and automation (because your platform team’s time is better spent on innovation than toil)

The numbers speak for themselves. We’ve worked with customers like Snowflake who reduced deployment times from one and a half weeks to less than a day. Starburst Data cut their infrastructure deployments from two weeks to just three hours. These aren’t vanity metrics. They’re business transformations.

IDP Strategy: 5 Key Steps to Build a Strong Foundation

If you’re just starting your IDP journey, here’s the roadmap we give to every platform team we work with:

Start with user research: Before you write a single line of code, go talk to your developers. Understand your different personas and their actual needs, not what you think they need. Do your users like to work from the command line? Do they prefer interacting with a UI?
Identify common patterns: Look for infrastructure patterns that get repeated across teams. These are your goldmine—the patterns that, once abstracted and templatized, will provide immediate value.
Begin with components: Build reusable infrastructure building blocks before you worry about fancy UIs or workflow orchestration. You need solid foundations before you can build the house.
Implement progressive disclosure: Give people simple interfaces for common use cases, but always provide escape hatches for complex scenarios. The moment your abstraction becomes a prison, developers will route around it.
Measure and iterate: Track adoption and gather feedback religiously. Your platform is only as good as its adoption rate, and adoption only happens when you’re solving real problems for real people.

The Future of IDPs and Platform Engineering

Trends come and go, but platform engineering is here to stay. It’s not just another tool or methodology. This is how modern organizations scale infrastructure to match the speed of innovation. The companies that win are the ones that treat their Internal Developer Platform (IDP) as a strategic foundation, not just a tool.

Your job isn’t just to manage infrastructure and choose the right tools - it’s to enable developers to move faster while keeping governance, security, and costs under control. Get that balance right, and your platform becomes a force multiplier across your entire engineering department.

The fastest way to get there? Focus on these five essential components:

Abstractions
Blueprints
Workflows
Security guardrails
Self-service

Together, they turn your IDP into more than just infrastructure automation, they create a platform that developers trust and your business relies on.

Ready to Build a Successful IDP that Scales?

Pulumi makes it easy to go from static IaC to dynamic, self-service infrastructure with real programming languages and built-in guardrails.

Explore Pulumi IDP or sign up for free to get started today.

Pulumi Blog: IDP Best Practices Series

Day 2 Operations: Drift Detection and Remediation

The Reality of Infrastructure Drift

Understanding the Prevention vs. Detection Paradigm

Prevention: The First Line of Defense

Detection and Remediation: The Safety Net

What Is Infrastructure Drift?

Pulumi’s Drift Detection and Remediation Workflow

1. Drift Detection with pulumi refresh

2. State Reconciliation with pulumi refresh

3. Drift Remediation with pulumi up

Automating Drift Detection with Pulumi Deployments

Setting Up Automated Drift Detection

Step 1: Configure Deployment Settings

Step 2: Create Drift Detection Schedules

Step 3: Configure Webhooks for Notifications

Handling Common Drift Scenarios

Scenario 1: Emergency Hotfix

Scenario 2: Legitimate External Changes

Scenario 3: Configuration Drift

The Value of Automated Drift Detection

Getting Started with Drift Detection

Quick Start Checklist

Conclusion: Day 2 Operations as a Competitive Advantage

How to Implement Robust Security Guardrails Using Policy as Code

The Platform Engineering Challenge: Speed vs. Safety

Understanding Platform Engineering Layers

What Are Deployment Guardrails?

Introducing Pulumi CrossGuard: Policy as Code

Key Policy Types

Building Practical Guardrails: Real-World Examples

Example 1: Port Security Policy

Example 2: Resource Limits Policy

Example 3: Cross-Resource Security Policy

Policy Enforcement Models

The Preventative Model

The Detective Model

CI/CD Integration

Policy Remediation: Beyond Detection

Server-Side Policy Enforcement

Compliance-Ready Policies

Best Practices for Policy Implementation

Start Small and Iterate

Provide Clear, Actionable Error Messages

Embrace Progressive Enforcement

Test Your Policies Thoroughly

Document Escape Hatches

Real-World Success: Statsig’s Transformation

Building Your Policy Strategy

Measuring Policy Success

Common Pitfalls and How to Avoid Them

The Future of Policy as Code

Conclusion: Enabling Safe Self-Service at Scale

Get Started with Pulumi

Golden Paths in IDPs: A Complete Guide to Reusable Infrastructure with Pulumi Components and Templates

The Platform Engineering Layer Cake: A Model for IDPs

Layer 1: Infrastructure Layer

Layer 2: Platform Layer - Components

Part 1: Building Reusable Infrastructure Components

The Power of Multi-Language Components

Building a Microservice Component Step-by-Step

Documenting for Discoverability and Adoption

Publishing Your Component via Private Registry

Part 2: From Building Blocks to Golden Paths

Golden Path Maturity: From Zero to Product-Grade Platforms

Stage 1: No Golden Paths

Stage 2: Dawn of Templates

Stage 3: Templates as Products

What Makes a Golden Path Golden?

Building a Go Microservice Golden Path

Step 1: Template Structure

Step 2: Application Scaffolding

Step 3: Infrastructure Template with YAML

Step 4: Documentation and Examples

Making Templates Available in Pulumi IDP

Best Practices for Reusable Infrastructure Components and Templates

1. Design for Day 2 Operations from Day 1

2. Expose Complexity Progressively

4. Use Semantic Versioning Everywhere

5. Test Your Abstractions

1. Drift Detection with `pulumi refresh`

2. State Reconciliation with `pulumi refresh`

3. Drift Remediation with `pulumi up`