Pulumi Cloud orchestrates a growing number of workflow types: Deployments, Insights discovery scans, and policy evaluations. Some of that work runs on Pulumi’s infrastructure, and some of it runs on yours via customer-managed workflow runners. We needed a scheduling system that could handle all of these workflow types reliably across both environments. In this post, we’ll take a look at the system we built.
When you manage dozens of data-loading pipelines, copying and pasting IaC configurations between them is a recipe for mishap. IAM policies can drift, naming conventions diverge, and every new source is a new opportunity to make a mistake — not to mention compound the problem of duplication. In this post, we’ll show you how you can identify and encapsulate common patterns into composable components and walk through the production lessons we’ve learned running 25+ pipelines for over three years.
You can now control what happens when a resource fails during create, update, or delete—retry with backoff, fail fast, or handle errors in custom code. Last year, Pulumi IaC introduced the resource hooks feature, allowing you to run custom code at different points in the lifecycle of resources. Today we’re adding the onError hook so you can react when operations fail.
Getting started with GitOps can feel like trying to herd cats through a YAML factory while the factory is on fire. It’s one of those things that seems like it ought to be simple (just use Git!), but in practice is much more complex — and you may not realize how much more complex until you’re weeks or more into a project. After years of running GitOps workflows in production across dozens of clusters, I’ve collected a list of best practices that I’m hoping can save you from having to make many of the mistakes I’ve made. Think of it as the GitOps cheat sheet I wish I’d had from Day 1.
Many organizations have years of infrastructure built and managed with Terraform. Outputs such as VPC IDs, subnet lists, database endpoints, and cluster names are the connective tissue between infrastructure layers. Getting those values into other tools and workflows often means manual copy-paste, wrapper scripts, or brittle glue code.
The terraform-state provider for Pulumi ESC helps bridge that gap.
It reads outputs directly from your Terraform state files and makes them available as first-class values in your ESC environments — no scripts, no duplication, no drift.
Any output marked as sensitive in your Terraform state is automatically treated as a secret in ESC.
If you’ve used pulumi-stacks to read outputs from Pulumi stacks, this is the same idea for Terraform.
Managing database credentials is one of the persistent challenges in cloud infrastructure. Passwords need to be rotated, secrets need to be stored securely, and access needs to be carefully controlled. AWS IAM authentication for RDS offers a better way: instead of managing long-lived passwords, your applications authenticate using short-lived tokens generated from IAM credentials. This approach is more secure, eliminates password rotation overhead, and integrates seamlessly with your existing IAM policies. With Pulumi, you can set up this entire system using reusable components that make IAM authentication a standard part of your infrastructure.
Pulumi ESC environments can now validate configuration values against JSON Schema with the new fn::validate built-in function. Invalid configurations are caught immediately when you save, preventing misconfigurations from reaching your deployments.
Running multiple providers with different credentials in the same Pulumi program has always been tricky.
Providers expect fixed environment variable names like AWS_ACCESS_KEY_ID or ARM_CLIENT_SECRET, so if you need two AWS providers targeting different accounts, you couldn’t configure them both via environment variables.
Pulumi v3.220.0 introduces envVarMappings, a new resource option that solves this problem by letting you remap provider environment variables to custom keys.
Platform teams need visibility into package adoption at scale. Responding to security advisories, planning deprecations, and tracking version sprawl all require knowing which stacks run which package versions across your organization.
Before Platybot, our #analytics Slack channel was a support queue. Every day, people from every team would ask questions: “Which customers use feature X?”, “What’s our ARR by plan type?”, “Do we have a report for template usage?” Our two-person data team was a bottleneck.
