Pulumi Blog: Announcements

New: Versioned CLI and SDK Docs

Cam Soper — Tue, 07 Jul 2026 00:00:00 +0000

Pinned to an older Pulumi CLI or SDK version and finding that the docs describe a newer release? The Pulumi CLI command reference and the SDK API docs now include a version selector, so the documentation you’re reading matches the version you’re actually running.

How it works

When you open the CLI command reference, you’ll see a version dropdown near the top of the page, below the title. The SDK API docs carry the same dropdown in the upper-right corner. Choose a release, and the page loads the documentation generated for that exact version.

What’s available

Alongside the latest release, we keep immutable snapshots of previous versions going back to v3.150.0 (early 2025). The CLI command reference and the Node.js, Python, .NET, and Java SDK API docs are all covered, so the docs you need are only a dropdown away. (The Go SDK is versioned on pkg.go.dev, so its documentation lives there rather than in the Pulumi docs.)

Get started

Head to the CLI command reference or the SDK API docs and try the version dropdown. Selections are sticky, so when you navigate into a version, you’ll be able to click around within that same version without having to choose it again.

Have feedback? Let us know in the Pulumi Community Slack or by opening an issue on GitHub.

Browse the CLI command reference

Trigger Deployments on Git Tags

Michael Fallihee — Fri, 05 Jun 2026 00:00:00 +0000

A git tag is how many teams mark a release as ready. Pulumi Deployments can now act on that signal directly: configure a tag-based trigger, push a version tag like v1.2.0, and Pulumi automatically runs pulumi up for your stack. No extra pipeline glue, no manual click — your release tag is the deployment.

Why tags?

Push to Deploy has long let you preview changes on a pull request and update a stack when commits merge to a branch. That branch-based model is a great fit for continuous delivery to shared development and QA environments, where every merge should flow straight through.

But promotion to production is often deliberate, not continuous. You merge throughout the day, then decide — separately — that a particular commit is the release. The conventional way to record that decision is a git tag: v1.2.0, 2026.06.0, release-2026-06-04. Tagging is already part of most teams’ release rituals.

Tag-based triggers connect that ritual to your infrastructure. Instead of wiring up a separate CI job to call the Pulumi Deployments REST API on a tag event, you configure the trigger once in your stack’s deployment settings and let Pulumi handle the rest.

How it works

Tag triggers are controlled by two settings on your stack’s deployment configuration:

Run updates for pushed tags — a toggle that enables running pulumi up when a matching tag is pushed.
Tag filters — a list of glob patterns that decide which tag names qualify.

Tag filters use the same model as the path filters you may already know, except the patterns match against the tag name rather than changed file paths. A few examples:

v* — deploy on any tag beginning with v, such as v1.0.0 and v2.3.1.
v* plus !*-rc* — deploy on release tags but skip release candidates like v1.2.0-rc1.
2026.* — deploy on calendar-versioned releases such as 2026.06.0.

Filters prefixed with ! are exclusions, and an exclusion always wins over an include. With no filters configured and the toggle on, every tag push deploys. Deleting a tag never triggers a deployment.

When a tag push kicks off a deployment, Pulumi sets the PULUMI_CI_TAG_NAME environment variable to the tag name. Your pre-run commands or your Pulumi program can read it — for example, to stamp the release version onto a resource tag or an application config value.

Works across every VCS integration

Tag triggers are available across all five version control integrations: GitHub, GitLab, Bitbucket, Azure DevOps, and Custom VCS.

Get started

You can configure tag triggers wherever you manage deployment settings today — the Pulumi Cloud console, the REST API, or as code with the pulumiservice.DeploymentSettings resource.

To try it out:

Open a stack’s Settings > Deploy tab in the Pulumi Cloud console.
Enable Run updates for pushed tags and add a tag filter such as v*.
Push a tag — git tag v1.0.0 && git push origin v1.0.0 — and watch the deployment run.

For the full details, see the deployment triggers and tag filtering documentation. We’d love to hear how you put tag-based deployments to work.

Best AI Infrastructure Tools in 2026

Alex Leventer — Mon, 25 May 2026 00:00:00 +0000

“AI infrastructure tools” covers two distinct markets: infrastructure for AI (GPU clouds like CoreWeave, MLOps platforms like Weights & Biases) and AI for infrastructure (agentic platforms like Pulumi Neo that generate, deploy, and govern cloud resources for you). Most teams need tools from both categories, and picking the wrong one wastes budget and adoption goodwill.

The pressure to get this right is real. McKinsey research puts the productivity lift from generative AI in software development at 20–45%, which is great for application teams and a problem for platform teams trying to keep up with the resulting feature flow. Infrastructure investment is climbing on both fronts: more spend on the compute that trains and serves models, more spend on AI tools that manage everything else.

This guide covers both categories: the compute and MLOps stack in Part 1, and AI-powered infrastructure management in Part 2, where the more interesting product shift is happening.

AI infrastructure tools overview

Tools for building AI infrastructure

CoreWeave: GPU cloud built for AI workloads
Lambda Labs: straightforward GPU cloud for research and startups
Modal: serverless GPU compute
Weights & Biases: ML experiment tracking and model management
MLflow: open-source ML lifecycle platform
Hyperscaler AI platforms: AWS SageMaker, Google Vertex AI, Azure ML

AI-powered infrastructure management tools

Pulumi Neo: agentic AI with policy automation
Firefly AIaC: asset codification and IaC generation
env0 Cloud Compass: multi-IaC insights and analysis
Spacelift AI: run explanation and troubleshooting
Crossplane with Upbound: Kubernetes-native infrastructure
General-purpose code assistants: Copilot, Claude Code, Cursor, Gemini
AWS Application Composer: visual serverless builder

Quick picks

If you only have two minutes:

Enterprise compliance: Pulumi Neo. Executes changes (not only suggestions), ships with policy packs for CIS, HITRUST, NIST, and PCI DSS, and works with Terraform, CloudFormation, and resources created by hand.
Serious GPU compute: CoreWeave. Purpose-built for AI workloads, deep NVIDIA partnership, and prices that generally undercut the hyperscalers.
Best developer experience for ML: Modal. Decorate a Python function, get a GPU, pay by the second.
Open-source MLOps: MLflow. No vendor lock-in, runs anywhere, plays well with everything.

What is AI infrastructure?

The term covers two distinct categories that share almost no vendors.

Infrastructure for AI is the compute, storage, and orchestration that AI workloads run on. Training a large model is not a normal cloud workload: it wants thousands of GPUs talking to each other over fat, low-latency networks for weeks at a time. Inference is different again: lower latency, smarter batching, different hardware. General-purpose cloud was not designed for either case, which is why specialized GPU clouds and MLOps platforms exist.

AI-powered infrastructure management is the inverse: AI tools that manage cloud infrastructure. They generate infrastructure as code, run deployments, detect drift, and remediate policy violations. The pitch is that modern infrastructure (multi-cloud, containers, microservices, regulated workloads) has gotten too complex for humans to manage by hand and too varied for scripted automation to keep up with.

Most organizations end up needing both: somewhere to run their ML workloads, and something to keep the rest of the cloud sane.

Part 1: Tools for building AI infrastructure

These are the platforms you run AI and ML workloads on: GPU clouds for raw compute, MLOps platforms for the lifecycle around them.

CoreWeave

CoreWeave is the GPU cloud that broke out of the AI hype cycle into a real public company. They went public in 2025, signed a multi-billion-dollar capacity deal with OpenAI, and acquired Weights & Biases. Their thesis from day one was that AI workloads deserve infrastructure designed for AI workloads, not a GPU SKU bolted onto a general-purpose cloud.

License: Proprietary
Best for: Large-scale training and high-throughput inference; teams that need dedicated GPU capacity with first access to new NVIDIA hardware
Strengths: GPU infrastructure designed for AI; Kubernetes-native; direct NVIDIA partnership; handles distributed training at scale
Watch out for: Smaller global footprint than AWS/GCP/Azure; not a general-purpose cloud, so if you need RDS, S3, and a managed Kafka in the same provider, this isn’t it

Lambda Labs

Lambda has been the approachable GPU cloud for a long time. Environments come pre-configured with PyTorch and TensorFlow, and you can be running on an H100 in about as long as it takes to copy your SSH key.

License: Proprietary
Best for: Research teams, startups, and individual practitioners who want GPUs without a configuration tax
Strengths: Straightforward to start on; pre-configured deep learning environments; competitive on-demand pricing; strong learning resources
Watch out for: Smaller scale than CoreWeave or the hyperscalers; availability gets tight during demand spikes

Modal’s pitch is that you write a Python function, decorate it, and Modal handles the GPU. No capacity planning, no idle instances burning money overnight, no Dockerfile to maintain.

License: Proprietary
Best for: Variable ML workloads where reserved capacity would sit idle; data scientists who’d rather not learn Kubernetes
Strengths: Strong developer experience; serverless GPUs with automatic scaling; pay-per-second pricing; cold starts are fast for what they are
Watch out for: You give up infrastructure control. Not ideal for long training jobs that need reserved hardware or strict configuration requirements.

Weights & Biases

Weights & Biases is the de facto standard for ML experiment tracking and model management, integrated with essentially every framework and cloud you’d plausibly use. CoreWeave acquired the company in 2025, which has accelerated the joint roadmap but raised some neutrality questions for teams that prefer their tooling cloud-agnostic.

License: Proprietary with a free tier
Best for: ML teams that need shared experiment tracking, model versioning, and reporting
Strengths: Industry-leading experiment tracking and visualization; comprehensive model registry; strong team collaboration; broad integration surface
Watch out for: Costs scale quickly past the free tier; some teams self-host alternatives for data residency reasons

MLflow

MLflow is the leading open-source MLOps platform: experiment tracking, packaging, registry, and serving, with no lock-in. Originally built at Databricks, it’s now a broad open-source ecosystem with managed offerings from multiple vendors (including Databricks and the major clouds).

License: Apache 2.0
Best for: Teams that want MLOps without a vendor; or want the option to start managed and self-host later
Strengths: Open source; covers the full ML lifecycle; runs locally, on-prem, or managed; broad framework support
Watch out for: Self-hosting carries the usual operational tax; commercial alternatives have stronger collaboration UX out of the box

Hyperscaler AI platforms

The major clouds all sell end-to-end ML platforms. Each leads on the dimensions that line up with its parent cloud (Vertex for Google’s models and TPUs, SageMaker for AWS-native data pipelines, Azure ML for Microsoft-stack integration), but the wider integration with the rest of the cloud is the deciding factor.

AWS SageMaker: end-to-end ML on AWS, deeply integrated with S3 and Glue, with first-class connections to Lambda for serverless inference and to the rest of the AWS data stack. The default pick if your data already lives in AWS.
Google Vertex AI: Google’s ML stack, including TPUs for workloads that need them, plus access to Google’s foundation models. Strongest when paired with BigQuery.
Azure Machine Learning: the natural choice when the rest of your stack is Microsoft; first-party MLOps integrations across GitHub Actions, Azure DevOps, and Microsoft Fabric for downstream reporting. The right choice if you’re already an Azure shop with Microsoft compliance requirements.

The shared tradeoff: hyperscaler GPU compute typically runs 2–3x the per-hour price of specialized providers, and the platforms work best when you commit to them top to bottom. For organizations already inside one cloud, the unified billing and single support contract usually justifies the premium. For a new ML team starting from scratch, it rarely does.

Part 2: AI-powered infrastructure management tools

This is where the more interesting product shift is happening. Instead of running AI on infrastructure, these tools point AI at your infrastructure and let it do work. They’re a newer, AI-native layer on top of the broader infrastructure as code tooling landscape.

From code generation to agentic execution

Before the tool list, one distinction matters more than any feature comparison: whether the tool generates code or executes changes.

Code generation tools like GitHub Copilot suggest infrastructure code based on context. You review it, maybe edit it, run it yourself. The AI helps, but you’re still the one doing the work.

Agentic platforms generate the code and run it, with the guardrails you define. They understand your environment, handle multi-step workflows, and enforce policies on the way through. You describe the outcome; the platform makes it happen.

Capability	Code generation	Agentic execution
Generates infrastructure code	Yes	Yes
Understands infrastructure context	Limited	Deep
Executes changes	No	Yes
Handles multi-step workflows	No	Yes
Enforces policies automatically	No	Yes
Remediates drift and violations	No	Yes

Where you want to land on this spectrum is mostly a governance question, not a productivity one.

Pulumi Neo

Pulumi Neo is Pulumi’s agentic AI for infrastructure. The distinguishing claim is execution: Neo doesn’t only suggest a Terraform snippet, it figures out the right resources, generates the code, and runs the deployment inside whatever guardrails you’ve set.

License: Proprietary (Pulumi Cloud)
Best for: Platform engineering teams that want AI automation with real policy controls, especially in regulated industries

A few things that set it apart in practice:

Policy automation and compliance. Neo is integrated with Pulumi Insights and Governance, which ships pre-built policy packs for CIS benchmarks, HITRUST CSF, NIST SP 800-53, and PCI DSS. Detection and remediation run in the same loop: Neo finds a violation, generates a fix, and (subject to approvals) applies it. You can batch-remediate across stacks and accounts with prompts like “find and fix all unencrypted S3 buckets across our AWS accounts.”

Works with infrastructure you didn’t create with Pulumi. Neo’s governance applies to Pulumi-managed resources, Terraform state, CloudFormation stacks, and resources someone clicked together in the AWS console. That matters because the realistic adoption path is to point Neo at what you have, audit it, and gradually bring it under management, not to migrate everything first.

Progressive autonomy. Trust levels are configurable. Start with human approval for everything; loosen it for well-defined, low-risk operations as confidence builds; keep production and sensitive resources behind strict approvals. This is the part that tends to determine whether enterprises actually deploy agentic AI in anger, versus letting it sit as a sandbox toy.

IDE and CI/CD integration. The Pulumi MCP Server brings Neo into Cursor, Claude Code, Claude Desktop, Windsurf, and any other MCP-compatible client. The Pulumi Cloud UI is the home base for approvals, history, and remediation status. Neo also slots into CI/CD pipelines for pre-merge policy remediation.

Case studies:

Werner Enterprises reduced infrastructure provisioning time from 3 days to 4 hours using Pulumi.
Spear AI cut their Authority to Operate (ATO) timeline from an expected 1.5 years to roughly 3 months by using policy-as-code to evidence compliance controls for auditors.

Tradeoff to be honest about: Neo gets more valuable the deeper you are in the Pulumi ecosystem. If you’re running IaC, ESC, and policy packs already, Neo has a lot of context to draw on. If you’re kicking the tires, it’s still useful, but the differentiating capability (context-aware, policy-respecting agentic execution) is harder to feel.

Firefly AIaC

Firefly is an asset management platform with AI features bolted on top of a strong core. The core capability is asset codification: it discovers cloud resources you already have and generates the IaC for them.

License: Proprietary
Best for: Teams that need to codify existing cloud footprints or generate IaC from natural language

Strengths: solid asset discovery, multi-cloud coverage, natural-language IaC generation, drift detection with remediation hooks. Caveat: AI features here are supplementary to the asset management product, not the main event, and Firefly is less focused on agentic execution than on inventory and policy.

env0 Cloud Compass

env0’s Cloud Compass adds AI to env0’s IaC automation platform, focusing on analysis rather than autonomous execution.

License: Proprietary
Best for: Multi-IaC shops that want AI-generated PR summaries, drift explanations, and cost insights

Strengths: multi-tool support across Terraform, OpenTofu, Pulumi, and Terragrunt; AI-generated PR summaries; drift cause analysis; cost estimation. Caveat: this is analysis and explanation, not action: Cloud Compass complements an agentic tool rather than replacing one.

Spacelift AI

Spacelift’s AI work is focused on the post-run experience: explaining what happened in a deployment and helping troubleshoot failures.

License: Proprietary
Best for: GitOps shops that want AI assistance reading complex runs and diagnosing failed deployments

Strengths: AI-powered run explanation; troubleshooting guidance for failures; broad IaC tool support; mature CI/CD integration. Caveat: like Spacelift as a whole, this is observation and explanation, not generation or execution. Pair with something that writes the code.

Crossplane with Upbound

Crossplane brings Kubernetes-style declarative management to cloud resources. Upbound is the company that commercializes it, and is layering AI-native control-plane capabilities into the 2.0 generation.

License: Apache 2.0 (Crossplane); proprietary (Upbound)
Best for: Teams already deep in Kubernetes that want to manage cloud resources the same way

Strengths: Kubernetes-native model; native GitOps fit; very active OSS community; AI control-plane work emerging from Upbound. Caveat: the learning curve is real if you’re not already living in Kubernetes; the commercial AI features are still maturing.

General-purpose code assistants

General-purpose AI coding assistants are the tools your developers already have open: GitHub Copilot, Claude Code, Cursor, and Google’s Gemini and Antigravity. They write Terraform HCL, Pulumi programs, and CloudFormation templates competently, about as well as they write anything else.

License: Proprietary (subscription), varies by tool
Best for: Developers who want broad code assistance, including infrastructure code, inside their existing editor

Strengths: excellent line-by-line code completion; broad language support; first-class editor integration; trained on huge corpora. Caveat: no infrastructure context. They don’t know what’s in your account, what your policies are, or which subnet you should pick. Treat their IaC suggestions as first-pass scaffolding, not production output.

AWS Application Composer

Application Composer is AWS’s visual builder for serverless applications. Drag services onto a canvas, get a CloudFormation template out, with AI suggestions for service configuration along the way.

License: Proprietary (AWS, included)
Best for: Teams building AWS serverless apps who prefer a visual workflow

Strengths: visual development for serverless; direct AWS integration; AI suggestions for service configuration; emits CloudFormation. Caveat: AWS-only, CloudFormation-only, and best suited to serverless rather than general infrastructure.

Comparison tables

Infrastructure for AI

Tool	Category	Key strength	Limitation	Pricing	Best for
CoreWeave	GPU cloud	Purpose-built GPU infra, NVIDIA partnership	Not a general-purpose cloud	Per-GPU-hour	Large-scale AI training
Lambda Labs	GPU cloud	Approachable, pre-configured environments	Smaller scale	Per-GPU-hour	Research teams, startups
Modal	Serverless GPU	Developer experience, pay-per-second	Less infrastructure control	Pay-per-use	Variable ML workloads
Weights & Biases	MLOps	Industry-standard experiment tracking	Costs scale quickly	Free tier + paid	ML team collaboration
MLflow	MLOps	Open source, no lock-in	Self-hosting overhead	Free (self-hosted)	Flexible ML lifecycle
AWS SageMaker	Hyperscaler	AWS ecosystem integration	Higher cost, lock-in	Per-use	AWS-native orgs
Google Vertex AI	Hyperscaler	Google models, TPU access	Lock-in	Per-use	Google Cloud users
Azure ML	Hyperscaler	Microsoft integration, enterprise features	Lock-in	Per-use	Microsoft ecosystem

AI-powered infrastructure management

Tool	Approach	Key strength	Limitation	Pricing	Best for
Pulumi Neo	Agentic AI	Execution + policy automation	Best within Pulumi ecosystem	Pulumi Cloud tiers	Enterprise platform teams
Firefly AIaC	Asset management	Asset codification, IaC generation	AI is supplementary	Proprietary	Codifying existing infra
env0 Cloud Compass	Multi-IaC platform	Multi-tool support, PR analysis	Analysis, not execution	Proprietary	Multi-IaC environments
Spacelift AI	CI/CD platform	Run explanation, troubleshooting	Observation, not action	Proprietary	GitOps workflows
Crossplane / Upbound	Kubernetes-native	K8s patterns for infra	Requires K8s expertise	Open source + commercial	Kubernetes-native teams
Code assistants	Code assistant	Broad language support, IDE	No infrastructure context	Subscription	General code assistance
AWS Composer	Visual builder	Visual serverless development	AWS- and CFN-only	Included with AWS	AWS serverless apps

See agentic infrastructure in action

Pulumi Neo generates, deploys, and governs cloud infrastructure across the resources you already have, with the policy guardrails you define.

Explore Pulumi Neo

How to choose

There’s no universal best tool. Five questions sort the field quickly:

Cloud strategy. Multi-cloud means tools like Pulumi Neo, Firefly, env0, or Crossplane. Single-cloud commitment means hyperscaler-native tools may integrate more deeply (AWS Composer, SageMaker, and so on).
Team expertise. Programmers gravitate to tools that use real languages (Pulumi Neo, Pulumi IaC). Kubernetes teams find Crossplane natural; everyone else finds it steep. Teams that prefer visual workflows should look at AWS Composer or env0’s UI.
Compliance. Regulated industries (healthcare, finance, government) get the most value from tools with pre-built compliance packs and audit trails. Pulumi Neo’s CIS/HITRUST/NIST/PCI packs are the most direct fit. If preventative policy enforcement matters, prefer tools that block non-compliant deployments rather than flag them after the fact.
Existing footprint. Greenfield projects can use anything. Brownfield is where it gets interesting: Pulumi Neo works against Terraform, CloudFormation, and manually-created resources, which lets you adopt incrementally instead of migrating first. Mixed-IaC shops should also look at env0.
Budget. Open source first: MLflow for MLOps, Crossplane for Kubernetes-native infra. Open source is not free, though: self-hosting carries a real total cost of ownership in hosting, maintenance, and the expertise to operate it. Commercial tools (Pulumi Cloud, env0, Spacelift) fold that operational cost into the price, on top of support, SLAs, and the enterprise-tier features open source can lack.

Before adopting anything, get visibility into what you have today, pilot on staging where mistakes are cheap, and define success metrics up front: time to provision, policy violation rates, mean time to remediate. The best AI infrastructure tool is the one your team will actually use, which means meeting developers where they already work.

Key trends and outlook

From copilots to agents. “AI suggests code” and “AI runs the deploy” are different products with different governance implications. The teams getting value from agentic tools have figured out which tasks to delegate fully, which to keep human-in-the-loop, and which to leave alone.

Progressive autonomy. Enterprise adoption follows a predictable shape: visibility → recommendations → human-approved execution → autonomous execution for well-understood scenarios. Tools that support that graduation will see stronger enterprise traction than tools that force an all-or-nothing choice.

Policy as the control plane. As AI takes on more infrastructure tasks, policy frameworks become the primary control plane. Done well, policy becomes an enabler (guardrails that let you safely expand automation) rather than a brake on velocity.

MCP standardization. The Model Context Protocol is becoming the integration standard between AI assistants and infrastructure tools. The practical upshot is that the IDE is increasingly a viable surface for managing infrastructure, with AI mediating between natural language and the underlying APIs.

Consolidation. CoreWeave acquiring Weights & Biases and NVIDIA acquiring Run:ai both point toward integrated platforms across the AI infrastructure stack. For tool selection today, that’s an argument for picking vendors with clear strategic direction over point solutions likely to be acquired or out-competed.

Frequently asked questions

What is the best AI agent for cloud infrastructure management?

For enterprise governance plus true agentic capability, Pulumi Neo is currently the most complete offering: it executes changes (not just suggests them), integrates with pre-built compliance frameworks, and works with infrastructure regardless of how it was provisioned. For Kubernetes-native shops, Crossplane with Upbound’s emerging AI features is worth tracking.

How can I use generative AI to manage cloud infrastructure?

Start by identifying the repetitive, time-consuming infrastructure work in your team. The highest-value early use cases tend to be:

Code generation: write IaC from natural-language descriptions, then review.
Documentation: explain unfamiliar configurations and reduce onboarding time.
Troubleshooting: analyze logs, errors, and configs to suggest likely causes.
Security and compliance: scan for violations and generate fixes.
Full automation: for shops that want it, agentic platforms like Pulumi Neo execute provisioning workflows end-to-end with governance controls intact.

What is agentic AI for infrastructure?

Agentic AI for infrastructure means AI systems that autonomously execute infrastructure tasks, not just generate code suggestions. The difference from a code assistant is action: an agent understands your environment, respects your policies, and performs multi-step work (provisioning, configuration, security controls) within the boundaries you’ve defined.

How do AI agents improve DevOps workflows?

By automating the repetitive parts (provisioning, drift remediation, policy enforcement), reducing context-switching, and catching issues earlier. Teams that have rolled out agentic tools well report faster provisioning, fewer policy violations slipping into production, and quicker compliance remediation. The compounding effect (engineers freed for higher-value work as the agent absorbs the routine) is the actual point.

What’s the difference between AI code generation and agentic execution?

Code generation suggests IaC for a human to review and run. Agentic execution generates the code and runs it, with policy and governance enforced along the way. It’s the difference between a knowledgeable colleague who suggests an approach and a knowledgeable colleague who also ships the change with appropriate oversight.

Can AI generate Terraform or Pulumi programs?

Yes. Most general-purpose AI assistants (Copilot, Claude, Gemini, ChatGPT, Cursor) can produce Terraform HCL, Pulumi programs in TypeScript / Python / Go, and CloudFormation. Quality varies. Generic assistants lack environment context and will happily emit syntactically correct but operationally wrong code. Infrastructure-specific tools like Pulumi Neo generate code that’s aware of your existing resources, policies, and provider constraints.

Can AI help with infrastructure compliance and policy automation?

Yes, and this is one of the highest-leverage uses of AI in infrastructure. Tools like Pulumi Neo detect policy violations across your footprint (including resources created outside IaC), generate compliant remediation, and apply it with the approvals you require. Pre-built frameworks for CIS, HITRUST, NIST, and PCI DSS shorten what would otherwise be a long manual compliance project.

Are AI infrastructure tools secure for enterprise use?

Enterprise-grade ones are. Look for RBAC, full audit logging of AI actions, preventative policy enforcement (not just detection), and human-in-the-loop approvals for sensitive operations. SOC 2, data residency options, and configurable autonomy levels are table stakes. The risk to avoid is wiring a consumer AI assistant directly into a production cloud account without those controls.

How do I choose between different AI infrastructure tools?

Match the tool to your context: existing clouds and IaC, team skills, compliance requirements, budget. Enterprise platform teams with governance needs should evaluate Pulumi Neo first. MLOps-focused teams should look at Weights & Biases or MLflow. For general code assistance inside the editor, a general-purpose assistant like Copilot, Cursor, or Gemini is the default. Most organizations end up with more than one: a code assistant for daily development and an agentic platform for production infrastructure.

What are the best tools for machine learning infrastructure?

For GPU compute, CoreWeave leads at scale, Modal wins for variable workloads and developer experience, and the hyperscalers are the default pick if you’re already inside one of them. For experiment tracking and model management, Weights & Biases is the leading commercial platform; MLflow is the leading open-source one. Most teams pick on the deploy model and pricing fit rather than capability gap. For the cloud infrastructure underneath the ML workloads, the same infrastructure management story applies: Pulumi Neo can provision and govern ML infrastructure the same way it handles everything else.

Conclusion

Two categories, two problems. GPU clouds and MLOps platforms (CoreWeave, Lambda, Modal, hyperscaler trio, W&B, MLflow) solve the compute and lifecycle problem for running AI workloads. AI-powered infrastructure tools (Neo, Firefly, env0, Spacelift, Crossplane, code assistants, Composer) solve the management problem for everything else.

For GPU workloads, the choice mostly comes down to scale and where you already are. For infrastructure management, the real question is how much you actually want AI to do. Code assistants help you write IaC faster, but you’re still running it. Agentic platforms like Pulumi Neo execute changes and enforce policy on the way through, with the guardrails you control.

The pattern from teams getting real value: treat AI as a force multiplier on routine work (provisioning, drift, compliance) and keep human judgment in the loop for the architecture and the edge cases.

If you want to see agentic infrastructure management running against real resources, start with Pulumi Neo.

The Agentic Infrastructure Era

Joe Duffy — Tue, 19 May 2026 03:00:00 -0700

The first frontier agents excelled at was coding. The reason is evident: we have billions of lines of self-documenting code available on the internet for the LLMs to learn from. We can measure their performance on coding thanks to linters, type checkers, compilers, and test suites. The most advanced agentic systems to hit product/market fit have been coding-oriented, and it has resulted in an intense velocity increase in how much and how fast code we can write.

But as the AI tsunami whips up reams of code, what happens to it becomes just as critical. As an industry, we’ve moved beyond just coding to engineering, which includes documentation, tests, automation, and, yes, managing the very infrastructure our applications need to run. The deeper into production you go, however, the less good agents naturally are at helping. At Pulumi, we live and breathe infrastructure, and have seen this firsthand. But we’ve also been hard at work building the platform this new era runs on. In this post, I’ll share our point of view, what we’ve built, what we’re launching today, and why all infrastructure is about to be agentic.

See Pulumi Neo in action → Neo is the agent at the center of everything below: it previews changes, checks them against policy, and opens PRs for your review.

LLMs are natural coders

It is remarkable to look back and note that frontier models, less than two years ago, in August 2024, scored just 33% on SWE-bench Verified. Present-day models score 86%, which represents a 4x reduction in the errors models will make when coding. This enables models to solve increasingly difficult coding problems, and humans can lean more heavily on them to offload tasks. Anthropic’s new Mythos model scores 94% and, although it isn’t generally available at the time of this article, there’s no question we’ll close in on 95% by the end of 2026. That is another 2.3x reduction in error rates. This very naturally puts us onto the last mile of fully agentic coding.

This has been the result of code being highly in-distribution combined with the relentless pursuit of solving coding problems from the frontier labs, especially with Anthropic’s Claude Code, but now with OpenAI’s Codex, driving a tight feedback loop that turns into an improvement flywheel.

Given that LLMs are natural coders, most of us simply assumed that the breakout success we’ve seen with agents for coding would automatically translate into new problem domains. And for sure, we have seen some success. But perhaps not as much as we’d like. Not all problem domains are documented equally well so that the models can naturally learn about them.

Andrej Karpathy noted nearly a year ago that, “Building a modern app is a bit like assembling IKEA furniture,” observing that, though writing the code was easy, fun, and fast, the next mile of actually getting the application running in production entailed many things the LLM wasn’t naturally good at, including “services, API keys, configurations, dev/prod deployments.”

At the same time, we’re seeing something magical happen here at Pulumi: LLMs are now doing over 20% of the infrastructure deployments, up from virtually zero a year ago. We expect this to grow to over 50% before the end of this year and well beyond afterwards.

The agentic infrastructure era is here. Today we’re announcing several new platform capabilities to accelerate it further.

Before getting to what’s new, however, why are we seeing this happening in reality?

Turning infrastructure problems into coding problems

We began our journey with our open-source infrastructure-as-code project nearly ten years ago. Having spent much of my career working on programming languages and compilers, I had a strong conviction that the right substrate for infrastructure was the languages developers already knew and loved, not yet another DSL and certainly not piles of YAML. So we focused first on great ergonomics for humans, and for us, that meant letting you use programming languages, tools, and ecosystems that humans already know and love. Languages like Python, TypeScript, Go, C#, Java, and more. Infrastructure as code also, importantly, comes with guardrails to make infrastructure deployments dependable, reviewable, and auditable. Over time we’ve built out an entire platform with security, compliance, and governance capabilities.

One way of thinking about this is we modeled the realm of cloud infrastructure inside the realm of programming languages: cloud resources become objects, configurations are just variables, dependencies between resources are just references, standard blueprints become classes. Doing so turns the cloud into something that is suddenly programmable, and allows us to apply real software engineering patterns and practices to infrastructure.

Here’s the twist we’ve realized this past year. It is a happy accident that the combination of this plus the verifiability of every change is the exact combination that empowers agents to do infrastructure. LLMs are natural coders. By mapping infrastructure space in code space, agents can do what they’re great at – code – and the infrastructure as code engine is the link that maps those code edits back down into infrastructure space. And as the SWE-bench curve plays out, every domain expressed in code rides it. Infrastructure, thanks to the bet we made nine years ago, is now one of them.

This isn’t just about volume of in-distribution code. Most production infrastructure code lives in private repositories. In contrast, public Python and TypeScript include vast amounts of genuinely production-grade open source, demonstrating real patterns at scale. Public infrastructure DSL corpora skew heavily toward tutorial-grade content that doesn’t scale. By modeling infrastructure in real languages, Pulumi inherits at-scale in-distribution directly. Patterns in Python or TypeScript generalize from what the model has practiced, while DSL patterns are idiosyncratic by definition and don’t compound the same way. Code can express everything an engineer can do, which is exactly why models trained on it have become prolific engineers.

What does this look like in practice? For example, an agent can define reusable infrastructure:

import pulumi
import pulumi_coreweave as coreweave
import pulumi_nvidia_aicr as aicr

class TrainingCluster(pulumi.ComponentResource):
 """A reusable training cluster resource."""
 kubeconfig: pulumi.Output[str]
 def __init__(self, name: str, *, nodes: int, accelerator: str):
 super().__init__("training:Cluster", name)
 self.cluster = coreweave.KubernetesCluster(f"{name}-k8s",
 accelerator=accelerator, # gb200/b200/h100
 count=nodes)
 self.stack = aicr.ClusterStack(f"{name}-aicr",
 kubeconfig=self.cluster.kubeconfig,
 accelerator=accelerator,
 intent="training",
 platform="kubeflow")

And then flexibly consume it – all using languages, like Python, it is already fluent in:

import pulumi
from infra import TrainingCluster
from orchestration import RLFleet

# Spin up an RL stack with 1 learner and 8 rollout workers.
learner = TrainingCluster("learner",
 nodes=4, accelerator="gb200")
rollouts = [
 TrainingCluster(f"rollout-{i}",
 nodes=2, accelerator="b200")
 for i in range(8)
]

fleet = RLFleet("ppo",
 learner=learner, workers=rollouts)

pulumi.export("login", fleet.head_node)

Beyond just being in-distribution, infrastructure as code also gives us a critical piece: the ability to diff changes at the infrastructure layer. Just as we wouldn’t vibe code without git showing us the source changes, we shouldn’t vibe infrastructure without a tool that shows what it will do before it does it, and what it has already done in the past. It’s like git diff for your infrastructure.

The combination of languages, type systems, linting, testing, and infrastructure diffs gives agents enough of what they need to create a fully closed, verifiable, self-correcting loop. Verifiability is the thing that ultimately matters. Every action an agent takes can be previewed, policy-checked, reviewed, and audited. By humans, by other agents, by CI/CD automation. Guardrails guide the agent and verifiability proves the end result.

When I say “infrastructure” by the way, I mean something quite broad. It includes public cloud hyperscalers, of course, like AWS, Azure, Google Cloud, and CoreWeave. It includes cloud native technologies like Kubernetes and Helm. It also includes private clouds powered by VMware and other data center technologies. It includes broad cloud platforms like Cloudflare. But, subtly, also any hosted managed infrastructure service, such as Snowflake, MongoDB, Databricks, and more. And increasingly, the observability, security, and operational tooling that wraps everything else, like Datadog, Sentry, PagerDuty, and other systems that give agents the context to reason about what’s actually happening in production. We support thousands of providers with a consistent programming model and agents are increasingly writing those too.

What we’re shipping today

The foundation of expressing infrastructure as code, combined with verifiable changes, makes a 100% agentic infrastructure future even conceivable. But we still have work to realize this vision.

Meeting agents where they work (the CLI)

We know real code is the right substrate for agentic infrastructure – but we also know agents want to start with the smallest possible commitment and progressively level up. So we’ve designed the Pulumi experience as a progression: an agent should be able to sign up, run its first command, and grow into a full infrastructure as code workflow without ever hitting a wall or manual intervention. Here’s what we’re launching today to make that journey work end-to-end:

Agent accounts. Now agents can use free, ephemeral Pulumi Cloud accounts straight from Claude Code, Codex, OpenCode, Copilot, Cursor, and others. This lets an agent do anything a human could with the Pulumi product without a manual signup step, removing friction while letting agents benefit from robust infrastructure management. A human can claim that account at any time to make it permanent and to share with their team.

One-command execution. Today an agent needs to know how to find, download, and install Pulumi, and manage subsequent versioning and upgrades. Anything that adds friction is discouraging to an agent. We now have an npm package that enables npx pulumi <anything>-style commands so agents can run any Pulumi command anywhere without needing to worry about installation or version pinning.

Imperative infrastructure operations. The new pulumi do command enables direct create, read, update, delete, list, and API operations with a single command. For example, pulumi do create eks:Cluster spins up an AWS EKS cluster with built-in best practices, and pulumi do update cloudflare:r2:Bucket --tags '{ "Foo": "Bar" }' updates a bucket’s tags. All thousands of providers and hundreds of thousands of resource types in the Pulumi ecosystem are supported across clouds. As the complexity of your scenario outgrows singular commands, it is easy to eject out into a full-blown infrastructure-as-code project.

Pulumi Cloud in the CLI. Over the years, we’ve added tons of great capabilities to Pulumi Cloud – things like change history, time-to-live stacks, drift detection, resource discovery and search, private registries, IDP, audit logs, secrets management, team-wide policy enforcement – but didn’t add the respective ergonomic commands to the CLI. These are the sort of features that matter at scale. Now it’s all there in your terminal where agents can use them. Over 30 new commands, you can think of this as the equivalent of the gh CLI which agents really like.

Read the blog: Better CLI Interactions for Agents and Humans →

Neo, everywhere you work

Once an agent (or a team) has graduated through these capabilities, what comes next is asynchronous infrastructure work and, increasingly, autonomy. That’s where Neo comes in: Pulumi’s own infrastructure agent, now upgraded with the surfaces, integrations, and cadences teams actually need to run agentic infrastructure in production. We are shipping:

Neo in the CLI. The new pulumi neo command lets you run the same agent that is already in Pulumi Cloud. It even shares the same agentic loop and uses a sophisticated architecture where the agent workstation is your local computer, allowing it to more seamlessly access source context and tools. This allows you to do agentic infrastructure wherever you prefer: Neo in the cloud for asynchronous, autonomous tasks, or your workstation for more intensive deep-dive paired sessions.

Neo GitHub and Slack Apps. Now you can @-mention Neo from GitHub pull requests, and/or straight from Slack, to kick off agentic infrastructure workflows wherever it is most convenient, complete with whatever guardrails you’ve configured already.

Neo Integration Catalog. A new integration catalog lets you configure connectors to other systems that bring valuable infrastructure management context, including Atlassian, Datadog, Honeycomb, Linear, PagerDuty, and Supabase. This lets you tap into additional planning, specification, observability, and live site incident data, expanding what agents can do on Days 0, 1, and especially 2.

Scheduled Tasks and Read-Only Sessions. Now you can automate recurring infrastructure tasks, including confining Neo to read-only operations for extra safety. This opens up scenarios like reporting on infrastructure patterns weekly, automatically cleaning up waste in your dev accounts daily, or scheduled maintenance and upgrades.

Read the blog: 10 More Things You Can Do with Pulumi Neo →

Partnering with the frontier of AI infrastructure

We’re investing in deep partnerships with the companies building the frontier of AI infrastructure, and are shipping two new partner providers in close collaboration with the teams behind them:

CoreWeave provider. This new provider brings the full power of the CoreWeave platform, the leader in GPU infrastructure, to help teams provision AI workloads with ease. This includes support for all of the CoreWeave services including CoreWeave Kubernetes Service (CKS) and includes several examples and templates out of the box for training and inference architectures written in AI-native languages.

NVIDIA AI Cluster Runtime (AICR) provider. This new provider delivers fully functional NVIDIA software stacks atop the underlying cloud provider’s GPU infrastructure. This packages out-of-the-box components like NVIDIA GPU Operator, Kubeflow, NIM Operator, and dozens more otherwise tricky-to-configure cluster components. We’re excited to showcase the new AICR provider in tandem with our new CoreWeave provider: you can now spin up a CKS cluster and configure it with AICR in one program.

Educating and measuring agentic infrastructure intelligence

We’ve also invested in making Pulumi maximally legible to agents, because the best tools only matter if agents can find them, read them, and know which one to reach for and when:

Agent-friendly docs. We now serve our docs website in markdown to agents and did a pass over our documentation and CLI text to ensure it’s maximally useful to agents.

Agent-friendly CLI ergonomics. We have added --json and structured errors across the CLI to help agents parse and react to outputs appropriately.

New skills. We’ve done a pass over our skills and added a new uber-skill that describes how an agent should leverage the full suite of capabilities we now offer, from Level 0 (agent accounts), to Level 1 (pulumi do), to Level 2 (full IaC and pulumi up), to Level 3 (Pulumi Cloud governance and full autonomy with Neo), and beyond. It’s handy for humans too.

InfraBench. We have created a new benchmark to measure how well an agent performs on a wide array of representative infrastructure tasks. This is the infrastructure equivalent to SWE-bench and lets us measure stock agents, agents plus the new tools and skills, and Neo, and how they’re improving over time. Although we are still hard at work on building out the comprehensive suite, and will keep it internally for now, we will begin publishing our progress against InfraBench as we continue tackling new problems and improving agent performance.

What comes next

We’re primarily focused on two pillars as we go forward:

First, we are making agentic infrastructure a reality with increasingly autonomous workflows. The pace and scale of infrastructure in an AI era are beyond anything we’ve seen before, and we need to ensure infrastructure doesn’t become the bottleneck to velocity. We will be shipping steady improvements that help agents score better on InfraBench as a measure of their abilities. This will begin with new building blocks and in-context improvements, but we’re really excited that every improvement the frontier labs ship in general-purpose coding ability is automatically an improvement in agentic infrastructure on Pulumi, at zero cost to us or to our users. The result is that Pulumi-equipped agents will consistently and measurably outperform raw agents on real infrastructure work, and that gap will compound as the substrate and the models both improve.

Next, a world in which agents are provisioning, updating, monitoring, and managing all of our infrastructure changes the human/agent/infrastructure interface greatly. We need better change management, policy enforcement, and observability tools. Going from a single human doing a single infrastructure task at a time, to a single agent doing that task, to teams of agents doing lots of autonomous infrastructure tasks at once, will reveal new bottlenecks needed to fully govern your infrastructure. We have a lot of those facilities in Pulumi Cloud already but we will continue pushing here to improve them and ensure they scale up as fast as they need to. This is the layer that matters even as agents themselves get great at infrastructure. The smartest agent in the world still needs guardrails, audit trails, and policy enforcement to be trusted with production systems at scale, and that layer gets more valuable as agents get more capable, not less.

The combination of these two things will lead to a world where agents are able to manage more infrastructure, and to do so with increasing autonomy, while human operators retain full visibility, understanding, and control over what their teams of agents are doing at all times. This is the full engineering loop applied to infrastructure. Not just code generation, but design, review, deployment, and operation, with agents and humans collaborating across every step.

It’s a new era, and we’re just getting started

It was remarkable to see in practice and at scale that what’s good for humans is good for agents: ergonomic infrastructure as code. We’ve seen every point along the spectrum, ranging from Wiz managing 1 million resources and doing 100k daily deployments, to Compostable.ai already living the 100% agentic infrastructure dream, to a frontier lab who grew their infrastructure footprint by 982% in a year, all using infrastructure modeled in true code. All very different scales and teams, but with one thing in common: it was code all the way down.

Languages and verifiability already provide strong foundations, however, there are still clear areas we need to improve, beginning with making the full suite of infrastructure automation, security, and governance tools more accessible to agents. That’s what today is all about. To deep dive on details of the launches, check out the companion blog posts released today.

Agentic infrastructure is a super exciting one-way door for our industry. We’re pushing hard to take this beyond 20% and toward 100% in the years ahead. This is uniquely enabled by infrastructure as code in languages the models already speak combined with built-in verifiability that makes agentic work automatically safe. There’s more to be done, however, today’s launch will bend the curve even further upwards. We’re still only just getting started.

Get started with Pulumi free → Install the CLI and provision your first agentic-ready stack in minutes.

Happy clouding,

-Joe

Vibe Clouding: Give In, Forget That Cloud Infrastructure Even Exists

Joe Duffy — Tue, 01 Apr 2025 00:00:00 +0000

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

By this point in 2025, we’ve all heard about “vibe coding”: the AI-fueled craze that enables even my 2 year old nephew to build new applications by simply giving into the vibes, embracing exponentials, and forgetting that the code even exists. Vibe coding enables anybody who can type on a keyboard or speak to a computer to build IPO-worthy software businesses overnight. Today we are excited to introduce vibe coding’s similarly revolutionary close cousin: “vibe clouding”. By giving into the vibes, you can now spin up cloud infrastructure anywhere and everywhere, all by just saying stuff, copy and pasting stuff, and vibing. And even better, it mostly works! Read on to learn more, or just watch the video below.

The Code Vibes

In just two months, vibe coding has transformed the entire coding profession. If you aren’t familiar with the concept, this post by Andrej Karpathy introduces the idea:

Enter the Cloud Vibes

Today we’re announcing an entirely new way of managing your infrastructure, vibe clouding. The vibes are no longer limited to application developers; as a platform engineer, you too can now vibe like the best of them. The combination of natural language support thanks to LLMs and an improved pulumi watch command put you in the flow and the cloud at your fingertips.

Infrastructure as Code in Any Language. No, Really

Today we’re introducing the ability to write declarative IaC in natural language. No code necessary.

IaC is now table stakes for anybody doing anything in the cloud. Pulumi’s unique approach (which is open source, by the way) allows you to use any programming language to declare cloud infrastructure, such as Python or TypeScript, and its declarative model brings you desired goal state into existence as if by magic.

Declaring Cloud Infrastructure in Natural Language

Now you can express infrastructure in any language, programming or otherwise. For example, this IaC “program” written in English creates a Python webserver in AWS that’s accessible over the Internet:

An AWS EC2 instance running a Python web server.
It accepts traffic over the Internet on port 80 and simply responds "Hello, World."
Capture and export its auto-assigned IP address and DNS name.

We save this in a *.pai file (pai = Pulumi AI), and then deploy it with pulumi up like normal. The new language host asks Pulumi’s AI Copilot to figure out the infrastructure we’ve requested and then shows us a preview of what it will do:

$ pulumi up
Previewing update (dev)

 Type Name Plan Info
 + pulumi:pulumi:Stack vibeclouding-dev create 2 messages
 + ├─ aws:ec2:SecurityGroup webSecurityGroup create
 + └─ aws:ec2:Instance webServerInstance create

Diagnostics:
 pulumi:pulumi:Stack (vibeclouding-dev):
 🧠 Thinking...
 🧠 I am happy with this, let's go!

Outputs:
 InstanceDNS: output<string>
 InstanceIP : output<string>

Resources:
 + 3 to create

Do you want to perform this update? [Use arrows to move, type to filter]
 yes
> no
 details

We can select “yes” and it will deploy our cloud stuff:

Updating (dev)
Type Name Status
+ pulumi:pulumi:Stack vibeclouding-dev created (17s)
+ ├─ aws:ec2:SecurityGroup webSecurityGroup created (2s)
+ └─ aws:ec2:Instance webServerInstance created (12s)
Outputs:
publicDns: "ec2-54-189-144-3.us-west-2.compute.amazonaws.com"
publicIP : "54.189.144.3"
Resources:
+ 3 created
Duration: 18s

Seconds later, after it completes, we’ve got a Python webserver on the Internet:

$ curl 54.189.144.3
Hello, World.

Not Just English

This is a new language runtime just like Pulumi’s other supported languages (python, go, java, yaml, etc). We almost called this new language english, except that we can use other languages:

Spanish:

Una instancia de AWS EC2 que ejecuta un servidor web Python.
Acepta tráfico de internet en el puerto 80 y simplemente responde "Hola a todos".
Captura y exporta su dirección IP y nombre DNS autoasignados.

Mandarin Chinese:

运行 Python Web 服务器的 AWS EC2 实例。
它通过端口 80 接受 Internet 上的流量，并简单地响应“Hello, World”。
捕获并导出其自动分配的 IP 地址和 DNS 名称。

Hawaiian:

ʻO kahi hiʻohiʻona AWS EC2 e holo ana i kahi kikowaena pūnaewele Python.
ʻAe ʻo ia i ke kaʻa ma luna o ka Pūnaewele ma ke awa 80 a pane wale ʻo "Aloha, World."
Hopu a hoʻokuʻu aku i kāna helu IP i hāʻawi ʻia a me ka inoa DNS.

The LLM is happy to accept any language known to us humans. So we are simply calling it ai. You just specify runtime: ai in your Pulumi.yaml and all other Pulumi features work as normal:

name: py-webserver
runtime: ai

It Is Feature Rich and Works for Real Workflows

If you change your mind and want it to use a Node.js web server … or listen on port 8080 … or run on Azure instead of AWS … with a different response … Simply change your prompt:

A virtual machine in Azure running a Node.js web server.
It accepts traffic over the Internet on port 8080 and simply responds "Hello, Vibe Coders!"
Capture and export its auto-assigned IP address and DNS name.

We then rerun pulumi up and it knows how to evolve your infrastructure towards the new goal state.

Your prompts can be as imprecise or specific as you’d like. For example, “Run a joke of the day application on Google” needn’t tell the LLM what sorts of infrastructure you want – it will just intelligently decide between a GCE VM, Docker container in Google Cloud Run, GKE app, etc.

You can also spread your prompts across any number of *.pai files: so you can stick your webserver into webserver.pai, your database in database.pai, and so on, and you can even cross-references resources between files, and the LLM will figure it out. Just vibe and out pops cloud infrastructure!

Watch Mode for Rapid Iteration and … Vibing

This is already amazingly cool, but we haven’t begun vibing just yet. We are halfway there.

Today we’re also introducing a vastly improved watch command to properly unlock the vibes.

The little-known pulumi watch command sits there waiting for changes to any parts of your Pulumi IaC and in response to you hitting save will redeploy the changes. It turns out that this combined with native LLM language support yields the magical combination that unlocks vibe clouding.

Let’s take our original example above but use the pulumi watch command to trigger the deployments:

Just write specifications fluently and hit save whenever it’s convenient. Pulumi incrementally deploys what it can, when it can, reports on status interactively, and summarizes what it did afterwards.

If your thought is half-baked, exploratory in nature, or you hit an unexpected hiccup, thanks to the declarative nature of IaC, it can always pick up safely where it left off. Don’t worry, just vibe.

This is Just the Beginning

By now, I hope you’ve realized what date it is today: April 1st. April Fool’s Day. We just couldn’t help ourselves. So this is all just a joke. Sort of.

Much of this is actually becoming reality thanks to the intersection of multiple factors:

Pulumi use of “code” pairs well with the LLMs’ ability to produce correct code.
Pulumi language extensibility enables adding “natural language” as a language provider.
Pulumi Copilot, launched less than a year ago, gives us skills that make the LLMs even better at infrastructure tasks.
The LLM models are getting so darned good … and will get even better still.

The improvements to pulumi watch are real and already in the latest Pulumi release. The pulumi-language-ai language host is, of course, just an early prototype of what’s possible, but we look forward to hearing your feedback if you have the chance to kick the tires. Even more exciting things are on their way throughout 2025 and they won’t be launched on April 1st next time.

Get Started Vibing

If you want to get started, simply:

Download Pulumi
Download the pulumi-language-ai host and ensure it’s on your PATH
Set OPENAI_API_KEY to a valid OpenAI key
Run pulumi watch, begin typing, copy and pasting, and vibing

We highly recommend playing this video on repeat in the background as you do so:

Happy vibe clouding.

Pulumi - Automate, Secure, and Manage Everything You Run in the Cloud

Joe Duffy — Wed, 18 Sep 2024 09:00:00 +0000

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

At our fourth annual PulumiUP conference, we unveiled a new vision for Pulumi, going beyond infrastructure as code to a comprehensive product suite. Pulumi’s platform now consists of three core products: Pulumi IaC is open source infrastructure as code in any programming language; Pulumi ESC is centralized secrets management that scales; and Pulumi Insights delivers instant visibility, intelligence, and control over all of your infrastructure. Each product can be adopted independently but is better together. In this post, we will share more about how Pulumi is your trusted partner to automate, secure, and manage everything you run in the cloud.

Pulumi - One Unified Platform For All Your Infrastructure Needs

The Pulumi platform solves a broad set of problems that engineering leaders face in the modern cloud era. This spans cloud automation, security, and management, which aligns to our three products: Pulumi IaC, Pulumi ESC, and Pulumi Insights. Our unified approach ensures developers, infrastructure experts, and security teams can collaborate closely and ship faster with high confidence.

In addition to our three products, Pulumi’s platform incorporates three core capabilities that span the entire product suite. Pulumi CrossGuard is our policy as code engine which ensures security, compliance, and best practices; Pulumi Deployments is our deployment workflow technology that lets us orchestrate infrastructure-oriented tasks; and Pulumi Copilot is our generative AI technology that deeply understands the cloud and infrastructure management and operations.

You can see this visually depicted here:

We’ve now worked with 3,000 customers to help with their cloud journey and have listened to and solved the problems we regularly hear about. That includes Nvidia who is at the forefront of the AI revolution as well as BMW who is reimagining their entire engineering mindset in a cloud-first world, both leveraging Pulumi. Unity Games now ships 5x faster from code to cloud, and Tivity Health saved 71% of their cloud spend, automating everything and saving $6.5M annually.

Now let’s dive into the three products and what we announced at PulumiUP:

Pulumi IaC - Infrastructure as Code in Any Programming Language

Pulumi IaC is our flagship open source technology, and it powers virtually everything we do, even the new products. We have seen accelerated adoption of Pulumi IaC. We’ve had over 100M downloads, are about to crack 21K GitHub stars (give us a star and bump us over!), and have seen 167% the contributions of Terraform this past month and 300% of OpenTofu. Not only is Pulumi the most powerful IaC technology, it is becoming more popular and vibrant too.

We are always improving Pulumi IaC and today is no different. We launched a number of new and exciting IaC features. That includes using any Terraform provider, beyond the nearly 200 providers that Pulumi already supports. We also shipped a new Pulumi VS Code Extension that enables rich debugging and easier editing of your IaC programs. Pulumi Copilot can now diagnose and fix your IaC errors. And finally, we’ve improved our Kubernetes support with powerful controls over await logic for more robust cloud native deployments. And we’re not slowing down anytime soon.

Pulumi is open source, of course, but works best with Pulumi Cloud, the easiest, most secure, and most reliable way to adopt Pulumi IaC. Starburst saved 100 days per year by moving from DIY state management to Pulumi Cloud. By using Pulumi Cloud, you also gain access to the other products in this post. Try IaC now and choose the option that suits you best.

Pulumi ESC - Centralized Secrets Management and Orchestration that Scales

Pulumi ESC is a new approach to secrets and configuration management that is scalable, secure automatically with easy dynamic credentials, and tames sprawl with engineering best practices. ESC is not just a feature of our IaC product — it can be adopted entirely independently of it for application and infrastructure secrets. We initially launched ESC in preview nearly a year ago, and since then have had hundreds of customers and thousands of users try it out – many at production scale – who have given us amazing feedback. We are excited to announce that ESC is generally available.

Being generally available is a big deal to us, and means it is ready for robust and secure enterprise-grade workloads. New capabilities available at GA include a new projects system which lets you group your environments into familiar hierarchies. We have added versioning, tagging, and pinning, so that you can institute more robust change management practices. ESC can manage encrypted secrets directly, but we have always supported pulling secrets from other secret stores, like AWS Secrets Manager, Azure KeyVault, Google Secret Manager, and HashiCorp Vault. We have now added 1Password to the list. We now also support secrets sync, which lets you still edit your secrets straight from Pulumi ESC, but have them automatically mirrored back to the backing secrets store. Flexible webhooks now support custom workflows triggered by environment changes. The new Pulumi VS Code Extension helps you edit your environments in code. We’ve launched application SDKs for Node.js, Python, and Go, as well as a Kubernetes Operator to inject secrets into Kubernetes workloads.

We aim for you to “fall into the pit of success” with security, so that it is built-in and automatic, and ESC is a major enabler of that. Try out ESC now, or read more in the launch post.

Pulumi Insights - Visibility, Intelligence, and Controls Over All Infrastructure

Pulumi Insights already delivers visibility, intelligence, and controls over your infrastructure resources. We began with Insights a year and a half ago as a collection of IaC features. We added resource search with structured and natural language search syntax, to help you find your IaC infrastructure no matter where it is running. We added the ability to export your infrastructure data and slice and dice it from data tools like Snowflake. We also added new dashboards to help visualize your infrastructure at scale.

The big news with Pulumi Insights 2.0 is that we’re bringing these capabilities to all of your cloud infrastructure, no matter where it is, and no matter how it was created. You point Pulumi at your cloud accounts for any of nearly 200 supported clouds, and you get an instant asset inventory of everything you have. This is true even for resources created by Terraform, CloudFormation, Azure Resource Manager, cloud CLIs and SDKs, or even by clicking in your cloud console. An improved resource view gives you pivot-table-like capabilities so you can understand everything you have, and new graph visualizations with inferred dependencies help you to make sense of how things relate. Search and Pulumi Copilot work over discovered cloud resources, letting you explore and ask questions interactively. Pulumi Insights leverages Pulumi CrossGuard, our policy as code technology, and can tap into any existing IaC policies to find issues pertaining to security, compliance, cost, and best practices. Pulumi’s compliance-ready policy packs just work. Insights can also leverage CrossGuard’s ability to automate remediations so that Insights not only tells you about your issues, but helps you fix them with the click of a button. Finally, Insights is better together with Pulumi IaC, and enables you to migrate any resources in the cloud to Pulumi IaC – improving the robustness and built-in security of your operations – with a fully interactive import experience.

This marks the start of an ambitious long-term vision for Pulumi Insights. Some of what was announced is actively shipping while other pieces will ship in the coming weeks. Learn more in the launch blog and, get started with Pulumi Insights 2.0.

Thank You

I am incredibly thankful to our community, 3,000 customers, and the partners and speakers who made PulumiUP 2024 a reality. Your feedback and continued belief in Pulumi has fueled the innovations we’re shipping today and into the future.

All of the talks are streaming live here and will be up on YouTube afterwards.

We can’t wait to hear what you think about the new vision for Pulumi’s platform as well as the new IaC, ESC, and Insights capabilities announced at PulumiUP 2024. As always, you can try Pulumi for free here.

Happy cloud spelunking.

-Joe

PulumiUP 2024: Dive Into the Future of Cloud, Platform Engineering, and AI/ML

Sara Huddleston — Wed, 11 Sep 2024 07:04:40 +0000

PulumiUP 2024 is just around the corner! It will be held on September 18th, starting at 8 AM PT | 15:00 UTC +0, and with over 5,500 engineers from all over the world already registered, this is shaping up to be the must-attend event for cloud professionals, platform engineers, and AI/ML enthusiasts alike. From entry-level engineers to tech executives, this event brings together professionals from companies of all sizes to explore the latest innovations and best practices in Cloud and IaC, Platform Engineering & DevOps, and AI/ML.

If you haven’t registered yet, now’s the time! Start building your schedule today, select the talks you want to watch live and on-demand and add them to your schedule.

PulumiUP at Glance

Main Track: Keynote, Cloud Culture Talks, and More

In the Main Track, you’ll hear directly from the Pulumi leadership team about exciting product innovations. Don’t miss the Cloud Culture talks focused on the human side of technology and collaboration.

Keynote and New Product Announcements - Joe Duffy (Co-founder and CEO), Luke Hoban (CTO), and other key Pulumi leaders will unveil new products and innovations shaping the next wave of cloud development. Expect to see live demos showcasing how these tools can revolutionize your workflows, from cloud infrastructure to security automation.
"Why Care About Building a Cloud Center of Excellence?" - Alex Radu (VP Product & Marketing Manager, Public Cloud, J.P. Morgan) discusses why building a Cloud Center of Excellence is essential for long-term cloud strategy success.
"What Is ‘HugOps’ and Why Is It Important?" - Rees Pozzi (Senior Platform Engineer, Kainos) explores the meaning and importance of “HugOps,” a practice promoting empathy and collaboration in the tech world but also by all those impacted by it.

AI/ML Track Highlights

Panel: AI for Cloud Development

AI is transforming how we approach development and infrastructure, and PulumiUP offers an incredible lineup of talks and panels on the topic.

"Data Privacy Challenges with Large Language Models" - Aditi Godbole (Data Science, AI & ML Leader, SAP) will address the critical question of managing data privacy, addressing LLM basics, AI data privacy fundamentals, and specific privacy issues in LLMs.
"Pulumi Powered AI/ML on Kubernetes" - Jason Smith (App Modernization Specialist, Google Cloud) will show you how to serve an open-source LLM with a RAG on Kubernetes and set up a Kubernetes environment for AI/ML workloads with Pulumi.
"AI Tools for Developers" - Jim Clark (Principal Software Engineer, Docker) will explore how Tools can enhance the capabilities of AI assistants and show how the combination of AI, Tools, and containerized runtimes, can enhance interactions between developers and their tools.
"The AI Governance Challenge" - Patty O’Callaghan (Technical Director, Charles River) will explore the complexities of AI governance and how to navigate them.
Panel Discussion: “AI for Cloud Development” - Luke Hoban (CTO, Pulumi), Phillip Carter (Principal Product Manager, Honeycomb), Giri Sreenivas (CPO, Docker), Clare Liguori (Senior Principal Software Engineer, AWS) and Meagan Cojocar (Principal Product Manager, Pulumi) will provide diverse perspectives on how AI is shaping cloud infrastructure.

Platform Engineering and DevOps Track Highlights

Panel: Secrets and Policies—Automating Cybersecurity

The Platform Engineering & DevOps track at PulumiUP is perfect for anyone looking to automate infrastructure, secure their systems, and scale their environments with ease. This track features talks and panels on cutting-edge technologies and practices.

"Moving Mountains: How IaC Unlocks Massive Refactoring Possibilities" - Shaun Verch (Infrastructure Engineer, Oso) will show you how infrastructure as code can transform large-scale refactoring projects.
"Using Pulumi to Empower Kubernetes Fleet Management" - Blake Romano (Senior Software Engineer, Imagine Learning) will show you how Imagine Learning leverages Pulumi to manage a fleet of Kubernetes clusters and other platform resources quickly and reliably, and how it revolutionized their workflow.
"Streamline Your Infrastructure Deployment with GitOps and Pulumi Operator" - Sam Cogan (Solutions Architect & Azure MVP, WTW) will highlight how you can define your infrastructure in Git, and have Pulumi automatically reconcile it into deployed infrastructure.
"Is This a Platform? Platform Engineering Before PMF" - Jk Jensen (Team Lead, Staff Software Engineer, MystenLabs) will share how they built their service platform around Pulumi to help engineers and researchers move from prototypes to deployed environments, enabling their teams to iterate more quickly and develop an increasing number of systems without additional complexity.
"Security Automation—Faster. Cheaper. Better." - David Giambruno (VP of Engineering and DevOps, Tivity Health) and Joe Duffy (Founder/CEO, Pulumi) will discuss security automation and how these efforts impacted speed and agility, cost optimization and improved security and compliance.
"Meet Devs Where They Are: Why IaC Must Be Real Code" - Jeremy Adams (Head of Ecosystems, Dagger) will present why real code (that teams already use for applications) is essential for both IaC and CI/CD pipelines, and to meet modern DevOps best practices.
"Security of IaC Pipelines and Infrastructure Governance With Policies-As-Code" - Marina Novikova (Senior Partner Solutions Architect, AWS) and Andy Taylor (Senior Networking Specialist/ Solutions Architect, AWS) will explore security best practices for IaC pipelines and infrastructure governance using policies-as-code, cover best practices for handling sensitive information like access keys, tokens and encryption keys, and scaling and automating your DevSecOps.
Panel Discussion: “Secrets and Policies—Automating Cybersecurity” - Arun Loganathan (Senior Product Manager, Pulumi), Maya Kaczorowski (PM of Security), Jason Meller (VP, Product, 1Password), and Ofir Cohen (CTO, Container Security, Wiz) will explore the intersection of innovation and security, offering insights into how AI is reshaping the threat landscape and how organizations can stay ahead.

Cloud and IaC Track Highlights

Panel: Infrastructure as Code - Can we do better?

The Cloud and IaC track is ideal for those wanting to deepen their understanding of infrastructure as code and its impact on modern cloud computing. Learn from industry pioneers and see where the future of IaC is headed.

"From CDK to Pulumi: Evolution of SST" - Dax Raad (Founding Engineer, SST) and Jay V (Founder, SST) will share insights into the early development of their product, initially built on AWS Cloud Development Kit (CDK), the challenges they encountered and the groundbreaking decision to transition to Pulumi.
"Unified Software Development at BMW Software Factory with Pulumi" - Jan-Peter Alten (Expert Software Engineer, DevOps, BMW Group) will showcase how BMW uses Pulumi to unify their software development processes, resulting in an increase in developer productivity, streamlined infrastructure management, and enhanced scalability, security, and compliance across their complex software ecosystem.
"Stacking Accounts: Multi-Account Deployments in a Single Stack for Monitoring, Management, and More" - Denis Willett (Software Engineer, North Carolina Institute for Climate Studies) will explain how they use Pulumi for managing 60+PB of environmental data across AWS, GCP, and Azure and leverage multi-account deployments in a single stack via the Pulumi Automation API.
"Industrialize the Configuration of Your GitHub Repositories With IaC" - Alexandre Nédélec (Software Engineer, Avanade) will live code using the GitHub, Azure native, and Azure AD Pulumi providers to create a GitHub repository and configure its pipeline to deploy to Azure using OpenID Connect.
Panel Discussion: “Infrastructure as Code — Can We Do Better?” - Joe Duffy (Co-Founder and CEO, Pulumi), Brian Grant (CTO, Stealth), Elad Ben-Israel (Co-Founder and CEO, Winglang), Adam Jacob (Co-Founder and CEO, System Initiative), and Luke Hoban (CTO, Pulumi) will discuss the past, the present, and the future of Infrastructure as Code and the overall infrastructure technologies.

Workshops and Hands-On Learning

Want to get hands-on experience? Our live workshops are designed to deepen your understanding of trending topics, Pulumi, and its applications.

"Managing Team Secrets with 1Password & Pulumi ESC" on September 25, led by Phil Johnston (1Password) and Diana Esteves (Pulumi).
"Securing IaC Pipelines in Regulated Industries" on September 26, led by Josh Kodroff (Pulumi) and Marina Novikova (AWS).
"Deploy Tailscale infrastructure in AWS with Pulumi" on October 15, led by Diana Esteves (Pulumi) and Lee Briggs (Tailscale).
"Observability as Code for AI Apps with New Relic and Pulumi" on October 30, led by Diana Esteves (Pulumi) and Harry Kimpel (New Relic).

Amazing Partners and Sponsors

Thank you to our amazing customers, sponsors, and partners for supporting PulumiUP!

AWS The world’s most comprehensive and broadly adopted cloud, offering over 200 fully featured services from data centers globally.
Google Cloud Help developers build quickly, securely, and cost effectively with the next generation of modern infrastructure designed to meet specific workload and industry needs.
New Relic Data for engineers to monitor, debug, and improve their entire stack.
Docker Accelerate how you build, share, and run applications.
1Password Streamline how you manage SSH keys, API tokens, and other infrastructure secrets throughout the entire software development life cycle.
Tailscale Make creating software-defined networks easy: securely connecting users, services, and devices.
Pinecone Build remarkable GenAI applications fast, with lower cost, better performance, and greater ease of use at any scale.
Wiz Secure everything you build and run in the cloud.
Honeycomb Observability that helps solve problems you couldn’t before.
Mysten Labs Building critical infrastructure to enable a more decentralized internet.
Imagine Learning Empowering educators to inspire breakthrough moments in every student’s unique learning journey with digital-first, K–12 education solutions.
Dagger Powerful, programmable open source CI/CD engine that runs your pipelines in containers.
Oso Authorization as a service.

Build Your Schedule Now

With so many compelling topics across AI/ML, Platform Engineering, DevOps, and Cloud and IaC, now is the time to build your PulumiUP schedule. Select the sessions that match your goals and interests to maximize your experience. Plus, you’ll receive a link to watch your chosen panel discussions and tech talks on-demand!

Haven’t registered yet? It’s not too late! Join 5,500 engineers worldwide in this groundbreaking event and gain the insights you need to drive innovation at your company. Register now->

We can’t wait to see you there!

Introducing `pulumi state move`: Move Resources Between Stacks or Projects

Thomas Gummerer — Thu, 25 Jul 2024 00:00:00 +0000

Today we’re announcing the pulumi state move command, which can be used to move resources that are managed by Pulumi between different stacks and/or projects. With the pulumi state move command, you can refactor your Pulumi Infrastructure as Code without any disruption to your deployed cloud infrastructure, enabling you to evolve and scale with confidence.

When refactoring a Pulumi project from a monolithic structure to micro stacks, you might need to move resources between different projects or stacks, without recreating them. While this has been possible with some significant manual effort to hand-modify Pulumi state files (our users came up with automated solutions for this!), doing so can be error prone and time consuming. Having a simpler, integrated solution for this problem was a much requested feature. The new pulumi state move command, combined with the aliases command for refactoring resources within stacks, offers the richest collection of tools for refactoring IaC at scale available in the industry today.

The pulumi state move command works as follows:

$ pulumi state move --help
Move resources from one stack to another
This command can be used to move resources from one stack to another. This can be useful when
splitting a stack into multiple stacks or when merging multiple stacks into one.
Usage:
pulumi state move [flags] <urn>...
Flags:
--dest string The name of the stack to move resources to
-h, --help help for move
--include-parents Include all the parents of the moved resources as well
--source string The name of the stack to move resources from
-y, --yes Automatically approve and perform the move

Both dest and source can be either stacks in the current project, or stacks in a different project, using the fully qualified stack names. Note that this works only for stacks within the same backend, it is currently not possible to move a resource between different backends (though you can move stacks between backends using other existing tools).

The resources being moved have to be specified by their full URN, and multiple URNs can be passed at once. For each resource being moved, all the children of that resource will also be moved, and the relationships between all resources being moved is preserved. Resources with other types of dependencies will however not be moved to the target stack by default. The easiest way to get the full URN of the resources is to use pulumi stack --show-urns. Note that URNs can contain characters that get interpreted by the shell, so it is always a good idea to wrap them in single quotes (') when passing them as arguments.

The pulumi state move command won’t modify the code of your program directly, it will only modify the state of the source and destination stacks. So following running the command, you should ensure that you modify both programs to match the changes you have made. This can typically be accomplished by copy/pasting source code for the resources and/or components between the two codebases. Inputs and outputs of resources that were moved may need to be adjusted as part of this process. This can be done either by using stack references, or recreating the inputs in the program.

Example

To demonstrate how the command works in practice, let’s go through an example. Let’s assume we have the following program:

const randomPet = new random.RandomPet("a-random-pet");

const b = new aws.s3.Bucket("b");

const index = new aws.s3.BucketObject("index.html", {
 bucket: b.bucket,
 content: "Thanks for using Pulumi!",
}, {
 parent: b,
});

const randomSite = new aws.s3.BucketObject("random.html", {
 bucket: b.bucket,
 content: randomPet.id,
}, {
 parent: b,
});

We create a bucket, and two bucket objects, both of them having the bucket as their parent. One of the objects has a random pet name as its content. At some point we decide we’d rather group all the AWS resources in a separate program, because this one grew too large. We can do that using the pulumi state move command, and then adjusting the code.

First we can find out which stacks already exist and we can move the resources to:

$ pulumi stack ls --all
NAME LAST UPDATE RESOURCE COUNT URL
v-thomas-pulumi-corp/pulumi-demo-move-aws/dev 23 minutes ago 0 <url>
v-thomas-pulumi-corp/pulumi-demo-move/dev 6 minutes ago 7 <url>

Next we can find the URN of the resources using pulumi stack --show-urns:

$ pulumi stack --show-urns
Current stack is dev:
 Owner: v-thomas-pulumi-corp
 Last updated: 13 seconds ago (2024-07-23 13:52:17.58884965 +0200 CEST)
 Pulumi version used: 3.125.1-dev.0
Current stack resources (7):
 TYPE NAME
 pulumi:pulumi:Stack pulumi-demo-move-dev
 │ URN: urn:pulumi:dev::pulumi-demo-move::pulumi:pulumi:Stack::pulumi-demo-move-dev
 ├─ random:index/randomPet:RandomPet a-random-pet
 │ URN: urn:pulumi:dev::pulumi-demo-move::random:index/randomPet:RandomPet::a-random-pet
 ├─ aws:s3/bucket:Bucket b
 │ │ URN: urn:pulumi:dev::pulumi-demo-move::aws:s3/bucket:Bucket::b
 │ ├─ aws:s3/bucketObject:BucketObject random.html
 │ │ URN: urn:pulumi:dev::pulumi-demo-move::aws:s3/bucket:Bucket$aws:s3/bucketObject:BucketObject::random.html
 │ └─ aws:s3/bucketObject:BucketObject index.html
 │ URN: urn:pulumi:dev::pulumi-demo-move::aws:s3/bucket:Bucket$aws:s3/bucketObject:BucketObject::index.html
 ├─ pulumi:providers:random default_4_16_3
 │ URN: urn:pulumi:dev::pulumi-demo-move::pulumi:providers:random::default_4_16_3
 └─ pulumi:providers:aws default_6_45_0
 URN: urn:pulumi:dev::pulumi-demo-move::pulumi:providers:aws::default_6_45_0

Next we can actually move the resources. Since we’re moving the resources from the currently selected stack, we can omit the --source argument:

$ pulumi state move --dest v-thomas-pulumi-corp/pulumi-demo-move-aws/dev 'urn:pulumi:dev::pulumi-demo-move::aws:s3/bucket:Bucket::b'
Planning to move the following resources from v-thomas-pulumi-corp/pulumi-demo-move/dev to v-thomas-pulumi-corp/pulumi-demo-move-aws/dev:

 - urn:pulumi:dev::pulumi-demo-move::aws:s3/bucket:Bucket::b
 - urn:pulumi:dev::pulumi-demo-move::aws:s3/bucket:Bucket$aws:s3/bucketObject:BucketObject::random.html
 - urn:pulumi:dev::pulumi-demo-move::aws:s3/bucket:Bucket$aws:s3/bucketObject:BucketObject::index.html

The following resources being moved to v-thomas-pulumi-corp/pulumi-demo-move-aws/dev have dependencies on resources in v-thomas-pulumi-corp/pulumi-demo-move/dev:

 - urn:pulumi:dev::pulumi-demo-move::aws:s3/bucket:Bucket$aws:s3/bucketObject:BucketObject::random.html has a dependency on urn:pulumi:dev::pulumi-demo-move::random:index/randomPet:RandomPet::a-random-pet
 - urn:pulumi:dev::pulumi-demo-move::aws:s3/bucket:Bucket$aws:s3/bucketObject:BucketObject::random.html (content) has a property dependency on urn:pulumi:dev::pulumi-demo-move::random:index/randomPet:RandomPet::a-random-pet

If you go ahead with moving these dependencies, it will be necessary to create the appropriate inputs and outputs in the program for the stack the resources are moved to.

Do you want to perform this move? yes
Successfully moved resources from v-thomas-pulumi-corp/pulumi-demo-move/dev to v-thomas-pulumi-corp/pulumi-demo-move-aws/dev

There’s a few things worth explaining above:

We’re moving the resources to a different project, so we need to use the fully qualified stack name. If the resources are being moved between different stacks in the same project, the argument can be only the stack name.
Before actually moving the resources, the command gives a list of resources that is being moved. Since the bucket objects are children of the bucket, they will be included in the move. However the random pet is not a child resource, so it will not be moved.
The command also warns us about dependencies that will not be transferred to the destination stack. Since we didn’t want to include the pet, but the bucket has a dependency on it, pulumi will warn us about that.
Before going ahead with the move, we get prompted to confirm whether we want to go ahead with it. In this example we do want to go ahead with the move, so we confirm the move, and the resources will be moved for us.

Now that the resources are moved, we need to adjust the program to take advantage of that. The source program will now look as follows:

const randomPet = new random.RandomPet("a-random-pet");
export const randomPetName = randomPet.id;

Only the random pet remains here. We also export its name, so we can continue using it in the destination program. Meanwhile the destination program is mostly just a copy of the original AWS resources, but there is an important change:

const stackRef = new pulumi.StackReference(`v-thomas-pulumi-com/pulumi-demo-move/dev`)

const b = new aws.s3.Bucket("b");

const index = new aws.s3.BucketObject("index.html", {
 bucket: b.bucket,
 content: "Thanks for using Pulumi!",
}, {
 parent: b,
});

const randomSite = new aws.s3.BucketObject("random.html", {
 bucket: b.bucket,
 content: stackRef.getOutput("randomPetName"),
}, {
 parent: b,
});

Note how we now need to specify the content of random.html, since we no longer have the random pet in the same program. In this case we use a stack reference to reference the output from the source program. It is of course up to the user how to re-create the output. It could also come from config, or be hardcoded depending on the use-case.

Refactor your infrastructure safely

Use pulumi state move to split a growing project into focused stacks without recreating your cloud resources. Organize your infrastructure as code and manage every stack in Pulumi Cloud.

Learn more

As always, we would love to hear your feedback in the community slack. If you encounter any issues with the command, please open an issue.

Celebrating 20,000 Stars: A Milestone for the Pulumi Community

Aaron Kao — Thu, 23 May 2024 18:48:50 +0000

Today, we’re excited to announce that the Pulumi open source project has crossed the incredible milestone of 20,000 stars on GitHub. 🎉 This is a huge achievement, and it wouldn’t have been possible without y’all - our incredible global community of developers.

Since its inception, Pulumi’s vision has been rooted in enabling teams to build and manage modern cloud infrastructure using familiar languages and tools. Over the years, your feedback, contributions, and passion have been invaluable in shaping Pulumi into what it is today. Seeing so many developers embrace our modern approach to cloud infrastructure management is really exciting. We are deeply grateful for the trust y’all have placed in us, and we will continue to push the boundaries in how teams manage cloud infrastructure.

Fueled by Community Contributions

Over the past 7 years, the Pulumi project has had 4,400+ contributors that made 75,000+ pull requests. Pulumi is depended upon by 6,600+ GitHub projects and supports over 160+ packages in the registry. It has been downloaded more than 100 million times by 170,000+ developers. In just the last month, Pulumi has been the most active Infrastructure as Code (IaC) open-source project compared to Terraform or OpenTofu. Pulumi has had 117 merged pull requests (67 TF, 83 OpenTofu), 82 closed issues (49 TF, 54 OpenTofu), and 53 new issues (27 TF, 29, OpenTofu).

Notable Features and Milestones

We also want to take this time to reflect on some of the notable features we shipped that got us here. The Pulumi roadmap is public so we can be transparent about the features being worked on and to encourage deeper collaboration with the community. Here are some of the most popular features that were added to Pulumi over the last 7 years.

Mar 2017 - Pulumi founded
Jun 2018 - Pulumi open source project launched
Aug 2018 - Kubernetes native provider
Oct 2018 - Series A fundraise
Oct 2018 - Pulumi Cloud launched
Nov 2018 - Stack Reference to allow access of outputs of one stack from another
Jun 2019 - AWSX as higher-level components for the AWS platform
Sep 2019 - Pulumi 1.0
Sep 2019 - Transformations to modify the properties and resource options for child resource of a component or stack
Nov 2019 - .NET support
Dec 2019 - CrossGuard for policy as code
Apr 2020 - Pulumi 2.0
May 2020 - Golang support
Sep 2020 - Azure Native Provider for same day access to new Azure features
Oct 2020 - Series B fundraise
Oct 2020 - Automation API exposes IaC through a programmatic interface
Oct 2020 - Import command to import resources not managed by Pulumi into a stack
Apr 2021 - First PulumiUP
Apr 2021 - Pulumi 3.0
Apr 2021 - Pulumi Packages to provide multi-language support for components and providers
Oct 2021 - Pulumi Kubernetes Operator
May 2022 - Java support
Nov 2022 - Pulumi Deployments to automate infrastructure deployments
Nov 2022 - YAML support
Dec 2022 - Hierarchical config
Apr 2023 - Pulumi Insights for intelligence, search, and analytics over any infrastructure
Apr 2023 - First Pulumi User Group (PUG)
Jul 2023 - Terraform conversion improvement as an easy way to migrate TF to Pulumi
Sep 2023 - Pulumi AI added to pulumi CLI
Sep 2023 - Go generics support added
Oct 2023 - Series C fundraise
Oct 2023 - Pulumi ESC to manage secrets and configuration complexity
Oct 2023 - Pulumi for Platform Teams through developer portals and more
Apr 2024 - Drift detection, TTL stacks, and scheduled deployments

We also shipped many other features that didn’t make this post, so please see our blog for all those details.

A Heartfelt Thank You

We are immensely grateful to our community for helping us reach this incredible milestone and making Pulumi a thriving ecosystem. Your contributions, feedback, and support have been invaluable in shaping Pulumi’s journey. We also want to thank the Puluminaries - our community leaders and advocates whose contributions have been pivotal to our success.

If you’re passionate about cloud infrastructure, consider joining the Puluminaries. As a Puluminary, you’ll gain direct access to our engineering team, preview roadmaps, and help contribute to code, content, and community support.

We look forward to continuing our innovations to democratize the cloud and empower developers worldwide. We encourage you to join our Slack community to engage with fellow developers and be part of the ongoing conversation about the future of how teams manage cloud infrastructure.

Thank y’all for being part of this journey. Here’s to the next 20,000 stars and beyond!

Pulumi + Azure Deployment Environments: Better Together for Enterprise Developers

Mikhail Shilkov — Tue, 21 May 2024 00:00:00 +0000

We are excited to announce the support for authoring Azure Deployment Environments (ADE) environment definitions in Pulumi Infrastructure as Code (IaC) empowering developers to self-serve app infrastructure required to deploy and test cloud-based applications. With Pulumi support, you can now manage your Azure resources in these environments using the same familiar programming model and the full power of our IaC platform.

Azure Deployment Environments is a service that enables developers to quickly spin up app infrastructure with project-based templates, all while maintaining centralized management and governance. Azure Deployment Environments also provides developers with an intuitive self-service portal where they can choose a curated, project-specific template to deploy new environments with just a few clicks. Thanks to the ADE extensibility model, you can now turn any Pulumi program into an environment definition and use it directly to securely provision application infrastructure in Azure. This post will guide you through creating ADE templates and deploying resources from the Microsoft developer portal.

How It Works

Azure Deployment Environments natively supports the Azure Resource Manager (ARM) and Bicep IaC frameworks. In addition, ADE provides an extensibility model that enables customers to create custom container images to author environment definitions.

The Azure Deployment Environments and Pulumi teams collaborated to publish a standard container image that our joint customers can use directly to run Pulumi programs in ADE: pulumi/azure-deployment-environments. The image leverages ADE’s extensibility model to run Pulumi programs in the container.

When used with Pulumi, each environment definition consists of two components:

An environment.yaml file that describes the environment by referencing the Pulumi Docker image and pointing to a Pulumi program. Here is an example of such a file:

name: MyPulumiEnvironment
version: 1.0.0
summary: My First Pulumi-Enabled Environment
description: Deploys a Pulumi stack into ADE
runner: pulumi/azure-deployment-environments:latest
templatePath: Pulumi.yaml

A Pulumi program that defines the resources to deploy. This program can use any of the Pulumi Azure providers to define resources in the environment. Here is a very simple Pulumi program defined in YAML:

name: ade-pulumi
runtime: yaml
description: My first Pulumi program in ADE
config:
resource-group-name:
 type: string
resources:
sa:
 type: azure-native:storage:StorageAccount
 properties:
 kind: Storage
 resourceGroupName: ${resource-group-name}
 sku:
 name: Standard_LRS

This is all you need to define a Pulumi environment in ADE. If you prefer to use a programming language like C#, TypeScript, Python, or Go, you will also need to add the code next to the Pulumi.yaml file.

Once the environment definition is added to the catalog, a user simply chooses an applicable environment in their developer portal. After accepting parameter values, ADE will run the Pulumi program in the container to provision Azure resources. In the case of the above program, it will create a new Storage Account in the environment, and the user will be able to use the resources once they are provisioned by Pulumi.

Getting Started

To get started, you must have the Azure Deployment Environments service configured. You can manually create one in the Azure Portal by following the instructions or using our sample Pulumi program to provision all the required resources.

Once you have an ADE instance, you can start authoring your environment definitions. You can use the Pulumi Azure Deployment Environments samples as a starting point. The folder contains four simple Pulumi programs in four different languages, each creating a Storage Account in the environment. The programs also show how to use parameters to integrate with ADE’s configuration system.

Once you have the definitions, you will publish them to a GitHub or Azure DevOps repository and attach the repository as a Catalog in Azure Deployment Environments. You can do so manually using the Azure Portal or configure it within your Pulumi project, as illustrated in our sample.

Finally, navigate to the Microsoft developer portal to create a new environment. You will see the list of available templates, including the ones you just published.

Bring a Custom Container Image

The Pulumi container image pulumi/azure-deployment-environments is a great starting point for your environment definitions. However, if you want to customize the behavior to your needs, you can also bring your own container image to run Pulumi programs in ADE. To do so, refer to the Configure a container image to execute deployments with Pulumi article on Microsoft Learn.

One benefit of using a custom image is the ability to provide a Pulumi Cloud access token to the image via the PULUMI_ACCESS_TOKEN environment variable. This allows the image to authenticate with the Pulumi Cloud backend without requiring additional configuration. Pulumi Cloud securely encrypts and stores your infrastructure state, manages secrets, provides search and clear visibility across all your cloud resources, runs remote deployments, integrates with CI/CD pipelines, detects configuration drift, and enables centralized policy enforcement. Additional features such as RBAC and audit logging enable your team to collaborate easily, ship faster, more securely and with confidence for every deployment.

If you have a suggestion for improving the standard Pulumi container image, please open an issue on our GitHub repository. We are always looking for ways to improve the experience for our Azure users.

Conclusion

Together with the Azure Deployment Environments team, Pulumi is excited to enable our shared customers to take advantage of the ADE extensibility model, Pulumi IaC platform, and empower developers with self-service of Azure App infrastructure, within enterprise guardrails. We hope that this new capability will help you streamline your development processes and make it easier for your teams to provision new environments in Azure quickly.

If you want to learn more:

Attend our workshop Platform Engineering with Microsoft Azure and Pulumi that takes place on June 20
Read the announcement on the Azure blog
Explore the code at pulumi/azure-deployment-environments

Continue on Error in the Pulumi CLI

Thomas Gummerer — Thu, 09 May 2024 00:00:00 +0000

When managing many resources with Pulumi, a pulumi up or pulumi destroy can lead to a complex graph of resource operations being performed as concurrently as possible relative to the dependencies in your program. If one of those operations fails, you may have noticed that Pulumi takes the most conservative approach, letting already in-flight operations complete, but not starting any new operations. Most often, that’s what you want – there might be no point in bringing up more infrastructure if a resource fails.

However, in some cases it can be useful to keep going to try to bring resources that are independent from the failed one into the requested state, be that resources being created or destroyed. For example, when doing a pulumi destroy, you may want to have Pulumi destroy as many resources as it can, without stopping when the first error occurs.

You can now do exactly that with the new --continue-on-error flag for pulumi up and pulumi destroy.

Using --continue-on-error means that resources that are not in the same dependency tree as the failed resource will still continue to be updated or destroyed, as they would normally. To make sure dependencies are still correctly respected, resources that depend on a successful update or destroy of the failed resource will not continue to be updated. This means that this flag is always safe to use, as Pulumi will continue to manage the failed resources, and they can be updated or destroyed in subsequent runs of Pulumi.

When the execution finishes, Pulumi will report the resource failures as you would currently expect from a failure, and exit with a non-zero exit code. This indicates that even though we continued to update resources independent from the failed one, there was an error during the deployment.

What do I need to do?

pulumi destroy --continue-on-error was introduced in Pulumi v3.112.0. After upgrading to this version or later of the Pulumi CLI you can go ahead and use this feature.

pulumi up --continue-on-error was introduced in Pulumi v3.114.0. However, it also requires updated language SDKs to work best. Since we now have resources that fail to create, but the Pulumi program continues to execute, the SDK has to deal with the outputs of these resources. Currently during pulumi up there are no “unknown” values expected by the SDK. Since we may now have a failed resource, the outputs for that resource might still be unknown, and the SDK will have to deal with that. This means that to fully support this feature some SDK changes were necessary. So in addition to upgrading the Pulumi CLI to v3.114.0, you will also have to upgrade the Pulumi SDK to v3.114.0 or later for Go, Python and Node.js, and 3.63.0 or later for .NET. For Pulumi YAML programs, you’ll need to use v3.115.0 or later of the Pulumi CLI.

When --continue-on-error is used with older SDK versions, the Pulumi engine will return an error to your program, which then will need to be handled, or the rest of the program might not be executed, and thus some resources may not be updated.

This feature was a highly requested feature in the Pulumi open source project, and we’re constantly working to address these highly requested features across the open source projects, so jump in and give features you’d like to see us add a 👍.

Give --continue-on-error a try and let us know what you think!

A recap of October 2023 - A big month at Pulumi!

Joe Duffy — Mon, 16 Oct 2023 00:00:00 +0000

We’ve been hard at work and are having the biggest month ever at Pulumi. That includes two new products, Pulumi ESC and Pulumi for Platform Teams, and news of our Series C. That’s a boatload of fresh infrastructure as code innovation. And October isn’t even over yet!

Two New Products!

Here’s a recap of the news:

Pulumi ESC is a new approach to cloud environments, secrets, and configuration. It works for applications and infrastructure and cuts down on sprawl while making everything more secure. This tech spans developer desktop scenarios and scales up beyond 1,000s of environments. Read the overview, blog, or get started with the docs. Oh, and don’t forget to star the GitHub repo!

Pulumi for Platform Teams takes what we’ve learned working with over 2,000 customers and 150,000 end users, many building internal developer platforms, and provides common patterns and practices right out of the box. This includes a developer portal, a Backstage plugin, advanced deployment workflows, and the ability to report on and automatically remediate compliance policies. Read the overview, blog, or get started with the docs.

Can we help you build an internal cloud platform? Although the new capabilities are easy to use, we’re happy to give you a demo, discuss architecture, or even help you build out your internal platform. Get in touch.

These launches build on our “multi-product” momentum from our Pulumi Insights product launch earlier this year. Insights applies the power of generative AI to all things cloud giving you an intelligent interface to your infrastructure. We are taking infrastructure as code to many new places. We’ve processed over 130,000 prompts since launch — it’s free to use, so give it a try today!

Series C Fundraise

Last but not least, we announced our Series C. This fundraise will enable us to keep building an incredible team and reimagining how developers and infrastructure teams program the cloud. We’re hiring and have endless fun hard/problems ahead check out and apply to the open positions here.

Let Us Know What You Think

We can’t wait to hear what you think. Get Started with Pulumi, join our Community Slack, and let us know how you’re using these new capabilities and any feedback you have.

From the entire Pulumi team, thank you!

Building the Best Infrastructure as Code with $41M Series C Funding

Joe Duffy — Tue, 03 Oct 2023 00:00:00 +0000

Today we announced a $41M Series C fundraise from Madrona Ventures, NEA, Tola Capital, and Strike Capital. These new funds will help us accelerate momentum, keep innovating with the best infrastructure as code technology on the market, and expand into new product areas to solve even more of our customers’ most pressing cloud challenges. We’ve had a year of exciting milestones – surpassing 150,000 end users in our community, 2,000 customers, and 100 employees – and look forward to many more to come. The future is full of cloud, and yet incredibly bright!

Incredible momentum

Over 2,000 customers have chosen Pulumi Cloud for their infrastructure as code needs. Many more companies build on our open source, including over half of the Fortune 50. And we estimate that our community recently surpassed 150,000 happy and productive end users.

This year we made the top 5% of the Inc. 5000 Fastest Growing Companies in America and were listed in Redpoint’s InfraRed 100, to highlight transformative cloud infrastructure companies. This momentum will continue to propel us even further on our growth trajectory.

The best infrastructure as code

Why have we seen such incredible growth? It’s a combination of things: an amazing, multi-billion-dollar fast-growing market – infrastructure as code – plus a product that is widely recognized as the best product and clearly leading the innovation agenda in that market.

Pulumi’s unique approach to infrastructure as code is to stand on the shoulders of giants by letting you use any language. At the time we began, this was not a popular idea! In fact, a good friend of mine, who is a legend in the cloud native space, bluntly told me, “YAML is good enough – infrastructure as code doesn’t need that good of a developer experience.” We disagreed.

Our thesis was that we went through a one way door with cloud software. We have entered into the era of real distributed applications. And that is super exciting! It means all developers are building cloud software and that infrastructure is central to innovating in the modern cloud.

By applying the industry’s best languages to the infrastructure as code space, we gave the cloud a programmable surface area. We can now treat cloud building blocks as programmable objects and build bigger things out of smaller things. The cloud is the new operating system and this foundational technology lays the groundwork for us to realize our long-term vision.

Fast forward 7 years, and Pulumi is the fastest growing infrastructure as code product and has inspired others, fundamentally reshaping the landscape. Our end users and customers tell us all the time they feel not just 10% more productive, but 10x more productive, thanks to rich, expressive languages, great tools and editors, sharing and reuse, and because infrastructure as code in any language also means infrastructure as code for anybody in the whole team.

We also bet big on open source. Open source is in our entire team’s DNA. We honestly felt it was table stakes that something so fundamental to how we write code and build platforms be open source. This builds trust, helps to foster an ecosystem and community of openness, and enables new scenarios that we couldn’t have even imagined. We paired our open source technology with a great Pulumi Cloud SaaS that 2/3rds of our end users elect to use simply because it’s the easiest, most reliable, and most secure way to adopt Pulumi at scale. Because we were very thoughtful, intentional, and smart about the relationship between our open source and commercial products, we can stand confidently by our open source heritage forever.

Building a platform

Over time, we have expanded beyond infrastructure as code, by solving one problem after another that we see our customers facing. Let there be no question, we are still very much all about infrastructure as code! But as we have worked with our over 2,000 customers, we recognize they have other challenges in areas adjacent to infrastructure as code.

This began with Policy as Code. Policy as code allows you to verify that infrastructure state follows cost, compliance, security, reliability, or best practice guidelines. I ran static analysis efforts at Microsoft for many years during the Trustworthy Computing era and saw just how important and powerful enforcing policy as close to development time as possible is. We are still very early days here but we built Pulumi’s engine to support policy as code right from the outset.

Next came many obvious adjacencies to the infrastructure as code workflow, including Testing and CI/CD Automation. Thanks to our choice of general purpose languages, we decided to tap into software engineering practices too; we sometimes call this “infrastructure as software” because it goes well beyond “just code.” We then built the Automation API to answer the question of, what if infrastructure as code were a library you could link with, and not just a command line interface? The things our customers have built with these capabilities have been mind blowing: self-serve portals, custom CLIs and frameworks, and even entire SaaS products.

Last year, we then began branching out into entirely new product lines:

Pulumi Deployments delivers server-side orchestration of deployment workflows. We frequently heard from customers that no product on the market solved this for them. You can point Pulumi Cloud at a Git repo and get automatic Pull Request validations and deployments upon merges. Or do one-click deployments straight from the Pulumi Cloud console. Pulumi Deployments is highly extensible with a REST API unlocks a world of possibilities for platform teams, including but not limited to drift detection and remediation, blue/green and multi-region rollouts, dependent stack updates, ephemeral stacks, and automatic infrastructure cleanup.
Pulumi Insights provides search, analytics, and intelligence over your infrastructure, across any clouds. Our customers frequently told us they had trouble taming, or even getting visibility into, cloud chaos. The headline feature here is our Pulumi AI coding assistant: if you can dream up an infrastructure as code configuration, it can build it for you in any language. Pulumi AI quickly surpassed 100K prompts within its first couple of months. Insights also enables you to search over any infrastructure anywhere, helping you gain understanding over your team’s use of cloud infrastructure, analyze it further, or even just to find that needle in the haystack.

Pulumi is gradually shifting from a single infrastructure as code product to a complete enterprise platform. As we build out the platform, we do so in a highly integrated way. Pulumi Cloud pulls it all together with a unified data model, REST API, identity model, and AI model. These are not a loosely related assortment of technologies – they are deeply integrated, and can increasingly be adopted independently, but are better together even more than the sum of the parts.

What’s next for Pulumi

It is still day one for us and we’re only just getting started. Our ambitious mission is to democratize the cloud so that every builder, no matter their background, can make the most out of the cloud. And although infrastructure as code in any language makes a giant step in that direction, there is so much more to be done. If the cloud is the new operating system, there are entire application and security models still missing. For our customers, the cloud is a competitive advantage and they’re always wanting to do more, and to do it faster, with the cloud.

As we look to the future, there are several key themes we are excited to tackle:

The best infrastructure as code. We hear all the time from users and customers that they think Pulumi is the best infrastructure as code technology on the market. We want to keep it that way. Best means the most productive, the most fun, the most flexible, and the most enterprise grade. We will continue investing substantially into making this technology even better and more accessible while also growing our community.
Reimagined cloud developer experiences. The modern cloud’s capabilities are endless–and yet that can be daunting. For engineers simply wanting to build a modern cloud application or architecture, the concept count and level of abstraction demanded by the cloud are still often an order of magnitude more complex than they should be. We will continue leveraging our infrastructure as code foundations to deliver revolutionary new developer experiences that make us more productive and impactful in the cloud.
Security and best practices by-construction. Simply put, security should not be an afterthought. No best practice should be. Yet for most cloud platform teams, both still are. We have chipped away at this problem but there is more to be done. Pulumi’s approach enables us to apply tried and true techniques from the programming languages domain to ensure that applications and infrastructure are increasingly correct by-construction. We need to shift from finding things after-the-fact towards stopping security issues dead in their tracks before they ever make their way to production.
AI powers the next wave of cloud innovation. We are big believers in AI. We have already extended large models to deeply understand the cloud infrastructure domain and have realized immense benefits of doing so. We have begun to add AI to our product in areas that improve the user experience or deliver unique capabilities. As we go forward, just as we always relentlessly pursue the best developer experiences for our end users, we expect to ask ourselves, “How can this be better in an AI-first world?”

In short, the new Series C funds will allow us to double down on innovation that is already working, while breaking new ground in pursuit of our mission. Most importantly, we look forward to working with you, our end users and community, to build the best product experiences that solve your hardest and most pressing problems.

Thank you

It is a privilege to count over 2,000 customers, and 150,000 end users, as part of our community. I still remember coding furiously in my basement with Eric and Luke, and going live with our first customer, like it was yesterday. We are just as passionate about our original dream today as we were back then – perhaps even more so, since it is coming true!

We are also so lucky to have such incredible investors at Madrona, NEA, Tola, and Strike who believe in the Pulumi dream too, and are helping us to build an iconic company.

To our end users, we would not be here if it weren’t for you. Whether you are a community member, an enterprise customer, or anywhere in between, your continued feedback, success stories, and belief in the Pulumi vision keep us going and make it all worthwhile.

From the entire Pulumi team, thank you!

Announcing the Speaker Lineup for PulumiUP 2023

George Huang — Wed, 24 May 2023 00:00:00 +0000

PulumiUP is our virtual user conference for the Pulumi community and anyone interested in Infrastructure as Code. Every year, we strive to build a program packed with technical talks, demos, and best practices with the goal of leaving you with new learnings and inspiration that will help you become a better cloud engineer. You’ll hear from industry leaders and experts about IaC, software engineering, DevOps, Platform engineering, and AI.

We hope you’ll join us virtually and take this opportunity to grow your knowledge and become infrastructure as code stars.

Get Ready for Amazing Launches and Demos

Learn about Pulumi’s newest features across the core IaC SDKs and Pulumi Cloud. And watch hands-on demos by the Pulumi team:

Joe Duffy, Co-founder and CEO
Luke Hoban, Co-creator of TypeScript and CTO
Meagan Cojocar, Principal Product Manager
Evan Boyle, Engineering Manager
Zaid Ajaj, Software Engineer

Final Speakers Announced

You’ll hear from innovators, leaders, and experts from multiple industries and learn about infrastructure as code, software engineering, and cloud engineering.

Sam Cogan - Solution Architect, WTW

Talk: Keeping your Infrastructure Code DRY (Don’t Repeat Yourself)
Sam is a Solution Architect with WTW and a Microsoft Azure MVP. Sam is focused on delivering applications into the cloud using IaaS, PaaS, Serverless, Containers, and Kubernetes. Sam is particularly focused on automation and DevOps, including Infrastructure as Code, Cloud automation tooling, PowerShell, and CI/CD tooling.

Dennis Sauvé - DevOps Engineer, Washington Trust Bank

Talk: How a Bank Modernized its Software Engineering with Infrastructure as Code Automation
Dennis is a development operations engineer at Washington Trust Bank in Spokane, WA. With years of experience in software development and cloud engineering, he works to streamline deployment pipelines and create solutions for developers.

Ala Shiban - Co-Founder & CEO, Klotho

Talk: Creating a ‘Cloud-Aware’ Code Solution
Ala is the co-founder of Klotho, a startup that transforms plain code to cloud-native code. He started his journey on Commodore64 with cassette tapes and (much) more recently worked on distributed systems, developer tools, and large-scale web, cloud, and microservice backends.

He’s led smaller teams and larger organizations at Microsoft’s Developer Division and Riot Games’ Infrastructure Platform organization.

Matt Stephenson - Sr. Principal Software Engineer, Starburst Data

Talk: Creating Infrastructure Automation Magic with Code
Matt is a Sr. Principal Software Engineer at Starburst Data. He’s responsible for Starburst’s infrastructure architecture. He’s worked at many tech companies, including Square, Oracle, Google, and Amazon. Aside from technology, Matt is a pilot and helps build The Man for Burning Man each year.

Tim Holm - Co-Founder & CTO, Nitric

Talk: Engineering a Multi-cloud Platform
Tim Holm is the Co-founder and CTO at Nitric, with 12 years of experience in software engineering and cloud development. In disciplines ranging from Server and Storage Virtualisation and Automation, Mobile Application development, Full-stack application development, and Cloud infrastructure automation. Before founding Nitric, Tim worked to deliver critical infrastructure projects in Government and Fintech.

In his spare time, Tim enjoys video games, building Lego, and mountain biking.

Tyler Scheuble - Head of Platform, People Data Labs

Talk: Using Monorepos to Level-up your Infrastructure as Code and Software Delivery
Tyler possesses a genuine enthusiasm for technology. At People Data Labs, he has designed petabyte-scale data pipelines and high-performance APIs with demanding availability requirements, which informed his approach to leading the development of the next generation of the startup’s development pipelines and environments.

Tyler is passionate about finding simple solutions to cross-cutting concerns such as security, observability, and development velocity.

Sefi Genis - Co-Founder & CTO, Firefly

Talk: Lessons Learned from Writing Thousands of Lines of IaC
Sefi is the CTO and Co-Founder at Firefly. He started his career in the 8200 Unit of the Israeli Intelligence Corps and then continued on to security and developer tooling companies.

Sefi was a founding engineer at Dome9 Security (later acquired by Checkpoint), where he led the Backend Infrastructure of the product post-acquisition. Before founding Firefly, he became Head of Engineering in an Israeli cybersecurity startup company. Today he is building Firefly to help engineers manage the new reality of cloud-native operations.

Panel Discussion - AI and the Future of Cloud Development

Prepare yourself for the future of cloud development! Gain valuable insights from industry experts in our panel discussion, where we explore the transformative impact of AI on the cloud infrastructure and software engineering industries. The panel discussion will be moderated by CEO Joe Duffy. These are the panelists:

Amanda Silver - CVP Developer Tools, Microsoft

Amanda is the CVP of PM for Microsoft’s Developer Division, including the Visual Studio family of products, .NET, TypeScript, and our developer platforms. She has been key to Microsoft’s transformation to contribute to open source with the introduction of TypeScript, Visual Studio Code, and the acquisition of both Xamarin and GitHub. She believes that a tight digital feedback loop with zero distance between end-users and engineering teams is a critical element of great product development.

Beyang Liu - Co-Founder & CTO, Sourcegraph

Beyang is the CTO and co-founder of Sourcegraph. Prior to Sourcegraph, Beyang was a software engineer at Palantir Technologies, where he developed new data analysis software on a small, customer-facing team working with Fortune 500 companies. Beyang studied Computer Science at Stanford, where he published research in probabilistic graphical models and computer vision at the Stanford AI Lab and thoroughly enjoyed his compilers course.

Luke Hoban - TypeScript Co-Creator and CTO, Pulumi

Luke is CTO at Pulumi. He has worked on developer tools his entire career. He started out on Visual Studio, C#, and .NET in the early 2000s, later joined the ECMAScript standards body as a representative of Microsoft and then became one of the co-founders of the TypeScript programming language.

Featured Partners

Thank you to our amazing partners for supporting PulumiUP! Visit these booths for tech talks, demos and Q&A about your favorite cloud platforms and developer tools:

AWS - Solutions Architects will be on hand to demo the latest AWS and Pulumi capabilities and help you get the most from the AWS platform.
Google Cloud - Customer Engineers will help you accelerate your cloud journey with helpful tips and Q&A.
Microsoft - Cloud Advocates will be available to show you how to kickstart your projects on Azure.
Equinix - Stop by the Equinix booth to learn all about the new Equinix Pulumi provider.
CircleCI - Stop by the CircleCI booth to learn best practices for adopting CI/CD in your organization.
New Relic - The New Relic team will demonstrate how to get started with Observability as Code with Pulumi and New Relic.
Firefly - If you have existing cloud resources you can convert them to Pulumi code easily with Firefly. Stop by the Firefly booth for a demo.
Nitric - The Nitric team will be on site to show you how to create cloud-agnostic infrastructure code from your application code.
Klotho - The Klotho team will show you how devops teams an application teams can easily collaborate to moderize and scale infrastructure without disruptions.
Port - Join the Port team to learn how to setup a self-service portal for your developers.

Want to attend PulumiUP 2023? Register now!

Join a Pulumi User Group (PUG) Meetup!

Sara Huddleston — Mon, 01 May 2023 00:00:00 +0000

Hi everyone,

After several great years on Meetup, the Pulumi Community is moving to a new home: Luma.

This wasn’t an easy decision. Together we’ve built 38 Pulumi User Groups across 14 countries, with nearly 10,000 members between them. Meetup is where that community took shape, and we’re grateful for everything that happened there.

We want to make it as easy as possible for you to show up. All upcoming events and workshops will be published on Luma going forward.

We have a global community made up of people from many different countries, but we also have many local communities. We want to enable our users to meet, share knowledge and ideas, answer questions, and learn about Pulumi and best practices. For that reason, we created the Pulumi User Groups, aka PUGs on Meetup.

Members of PUGs will have access to a global network of experts who can offer help, advice, and support on using Pulumi effectively. The PUGs will also feature regular in-person meetups, webinars, in-person and virtual workshops, and other events to help members stay up-to-date with the latest cloud infrastructure and Pulumi developments.

23 Pulumi User Groups (PUGs) Meetups in 10 countries

PUGs are led by Pulumi employees and Puluminaries, who are passionate community members that are experts in topics related to cloud infrastructure, Pulumi Cloud, cloud-native software development, and much more, and are willing to share their knowledge with the community.

United States Pulumi User Groups

South America Pulumi User Groups

São Paulo Pulumi User Group

European Pulumi User Groups

Middle East Pulumi User Groups

Tel Aviv Pulumi User Group

Asia Pulumi User Groups

Singapore Pulumi User Group

Oceania Pulumi User Groups

The First PUG in person

Our very first in-person PUG meetup took place in London, UK. The event was well-attended by developers, cloud architects, and other infrastructure enthusiasts. The attendees had the opportunity to network with other like-minded professionals and learn more about Pulumi’s tools and features from expert speakers.

During the meetup, attendees heard from several speakers, including Joe Duffy, Co-Founder & CEO at Pulumi, who was there to speak about Infrastructure as Code that writes itself after the release of Pulumi Insights, which happened that same day.

Other speakers included Liam Hampton, Sr. Regional Cloud Advocate at Microsoft, who spoke about Microsoft Azure and IaC, and Sam Cogan, Director of Solution Architecture at Willis Towers Watson, who spoke about building a library of reusable modules you can repeatedly reuse in your IaC projects.

Overall, the first PUG meetup was a huge success, and attendees left feeling inspired and energized about Pulumi’s tools and the community’s possibilities. With the launch of PUGs, Pulumi is demonstrating its commitment to building a strong and supportive community of professionals passionate about cloud infrastructure.

More PUGs and events to come

We are planning more in-person PUG events for the months to come. With PUGs, you’ll have the opportunity to connect with like-minded professionals, share knowledge and best practices, and stay up-to-date with the latest developments in cloud infrastructure and Pulumi.

Join the Pulumi User Groups in your local area, or if there isn’t one yet, you are welcome to join the Pulumi Slack community and meet people from all over the world.

Pulumi Insights: Intelligence for Cloud Infrastructure

Luke Hoban — Thu, 13 Apr 2023 06:00:00 -0800

Note: This post discusses Pulumi Copilot, which Pulumi Neo has replaced. Learn about Neo →

You can now get Insights capabilities on all your cloud infrastructure, even resources not managed by Pulumi IaC. Get started with Insights 2.0

There are new intelligent cloud management capabilities available in Pulumi Copilot. Learn More

We’ve seen incredible acceleration of cloud adoption over the past 5 years. Pulumi’s flagship open source IaC solution gives engineers great tools to scale up their cloud infrastructure using the same programming languages and tools they already know and love. As a result, thousands of companies of every size and scale have adopted Pulumi as a lynchpin of their cloud infrastructure strategy.

Today we’re excited to announce Pulumi Insights, the next major productivity enhancement for infrastructure as code. Pulumi Insights provides intelligence, search, and analytics over any infrastructure, in any cloud across your organization, leveraging the latest advances in generative AI and Large Language Models (LLMs). Whether you have an AWS VPC, a Kubernetes CRD, or a DataDog alarm definition, Pulumi Insights enables you to intelligently find and interact with all of your resources from within the Pulumi Cloud.

Pulumi Insights was born from partnering with our community and observing their struggles to make sense of millions of cloud resources and resource updates per month, across hundreds of clouds, regions, and accounts. With Pulumi Insights, we now help teams manage cloud footprints of any size with the following key components:

🧠 Intelligence: New AI-powered tool for learning, discovering and building cloud infrastructure with Pulumi, using the power of generative AI and Large Language Models (LLMs).
🔎 Search: Multi-cloud resource search, powered by the Pulumi resource supergraph, which surfaces rich metadata and relationships between cloud infrastructure.
📊 Analytics: Visualize Pulumi resource data with your favorite BI tools, including Excel, Tableau and others, using your own data warehouse for reporting on cost, compliance, and other operational use cases.

All of these features work across Pulumi’s 130+ supported cloud providers, including AWS, Azure, Google Cloud, Cloudflare, and Kubernetes, and can be defined in any programming language, including TypeScript, Python, .NET, Go, Java and YAML.

Pulumi Insights is available for all Pulumi Cloud users today.

Intelligence - Productivity with the Power of AI

The scale of Cloud APIs is often intimidating for new and even seasoned cloud engineers. At Pulumi, we provide access to hundreds of clouds, tens-of-thousands of resources and hundreds-of-thousands of resource properties. Pulumi has many thousands of examples showing how to solve specific problems, and yet developers still sometimes struggle to find the perfect code snippet that addresses their exact set of requirements.

Our new Pulumi Neo is an assistant purpose-built for creating cloud infrastructure using Pulumi. It builds on the power of Large Language Models and GPT to dramatically reduce the time it takes to discover, learn and use new cloud infrastructure APIs. Because Pulumi uses popular programming languages that GPT has already been trained on, Pulumi AI understands how to solve complex Cloud Infrastructure problems.

Using natural language, you can ask to accomplish a specific Cloud Infrastructure goal, and receive a Pulumi program in your favorite language. In addition, Pulumi AI supports iterating on your cloud infrastructure allowing the addition of new features, improving security and performance, fixing correctness issues and clarifying requirements.

The open source pulumi-ai CLI takes these ideas even further, by interactively deploying any cloud infrastructure you request using natural language prompts. You can build arbitrarily complex cloud infrastructure, without requiring detailed knowledge of specific APIs! Of course, since we’re using Pulumi’s Automation API under the hood, you can go to the Pulumi Cloud console to see everything that has been deployed, and even jump straight into your Cloud Console to inspect the results.

Pulumi AI will enhance your team’s ability to go from concept to product. This demo shows Pulumi AI taking natural language input and deploying cloud infrastructure and services which are exposed live on the internet, constantly updating as you refine your request.

Check out Pulumi Neo today at pulumi.com/neo.

Search - Find Anything in Any Cloud

Teams don’t use just a single cloud or a single cloud account. Indeed, many teams manage resources across dozens of clouds and hundreds of accounts! It can be almost impossible to aggregate and search your infrastructure across these accounts, since every cloud and every account is (by design!) a walled garden.

Pulumi Resource Search offers multi-cloud search and analytics across every environment in your organization. You can issue queries that find all of your AWS VPCs, or all of your VPCs in AWS and Azure, or all resources with the “production” tag across all cloud environments. You also have access to statistics about cloud usage, including a breakdown by cloud provider, resource type, and department. Resource search enables you to find the needle within your cloud haystack and visualize cloud consumption trends.

Infrastructure can be discovered interactively with a rich, structured query language, for example:

package:snowflake
data-warehouse stack:production package:aws modified:>=2023-03-21
stack:production type:”github:index/issuelabel:IssueLabel”
package:aws project:pulumi-cloud-import-aws stack:-staging-us-west-2

Search is powered by the Pulumi resource supergraph, which exposes rich metadata around cloud infrastructure resources and models relationships between resources, such as the project and stack they are deployed from, the components they are part of, the provider used to manage them. Pulumi enables faceted search, which allows users to specify criteria by applying multiple filters based on classification of resources.

AI Assist for Search

We are also making available today a private preview of AI Assist for resource search that enables natural language queries. You can provide a plain English (or Spanish, or French, or Japanese!) request, to express queries where you might not know the exact syntax, type tokens, or package names. AI Assist makes it easier to gain insight over infrastructure. You can join the waitlist now for access to this additional functionality for search.

Cloud Import

While all resources managed by Pulumi are part of the resource supergraph, you can also bring resources defined and managed outside of Pulumi into the graph to make them available for Insights. We’re also opening up a private preview of Pulumi Cloud Import, a feature that lets you apply Search and Insights to resources created outside of Pulumi, whether via ClickOps or through other tools like Terraform, CloudFormation, ARM or Kubernetes YAML. You can join the waitlist for access to these new features today.

Analytics - Deeper Insights for Cloud Infrastructure

We are also making the data in the Pulumi supergraph available for export into other Business Intelligence tools for deeper analysis in combination with other key business data. You can visualize resource data with your favorite BI tools, including Excel, Tableau, Looker, PowerBI and others, using your own data warehouse, such as Snowflake, Redshift, Databricks and more, and create reporting on cost, compliance, and many other operational use cases.

The ability to join Pulumi resource data with other key business data in your data warehouse enables users to address a broad range of key business needs - answering questions like:

What infrastructure is stale and possibly orphaned / contributing to our cost?
Where are we spending the most, broken down by department or environment?
Are all my primary data stores marked as protected to avoid unintentional deletion?
What are my most commonly used clouds? What are the most common resource types?
Who has performed the most updates across my organization?

For example, here is a dashboard in Metabase, created by combining Pulumi resource data with cost and department data in a Snowflake data warehouse.

Pulumi resource Data Export is available today both interactively in the console and via REST API . Learn more about these features in the Pulumi Data Export documentation.

Conclusion

We’d like to express our thanks to the entire community that helped us design, develop, test and ship this incredible release. From Intelligence to Search to Analytics, the new features in Pulumi Insights will make all of us more productive. We’re incredibly excited by the use cases we’ve already seen our customers build, and we can’t wait to see what you build on top of these foundations!

To get started with Pulumi Insights today:

Sign up or log in to Pulumi Cloud
Try out Pulumi Neo
Watch the videos for Pulumi Insights, Pulumi AI and Pulumi Search
Check out the Pulumi Insights Documentation
Join the Pulumi Community Slack

Preview of .NET resource providers

Fraser Waters — Wed, 18 Jan 2023 00:00:00 +0000

Today we are pleased to announce the Preview of .NET support for custom resource providers. This means you can build custom providers using your favorite .NET language, including C#, F#, and VB.NET.

This is a first look preview release of the provider interface for .NET. It has minimal documentation but is feature complete. We hope to gather early feedback to refine the API and quickly bring this to a production quality release.

You will need to use version 3.52.0 to access the new APIs. You can install this in your project with:

dotnet add package Pulumi --version 3.52.0

You can see an example of using the new provider interface in the dotnet integration tests directory. This has replaced the old test provider that was written in Go.

Please view the provider interface and raise issues and discussions on GitHub or our Community Slack in the #dotnet channel — we can’t wait to hear what you think!

Introducing KubeCrash: Cloud Native Crash Courses

Kat Cosgrove — Fri, 08 Apr 2022 00:00:00 +0000

Can’t make it to Valencia for KubeCon this year? Timezone doesn’t work for the virtual conference either? We can’t fix time, but if you’re feeling left out and still want some of that sweet cloud native content, you can still join us for KubeCrash, a new event hosting live crash courses and sessions on cloud native tech. Come hang out and learn directly from the maintainers of cloud native open source projects!

About KubeCrash

Five open source companies have teamed up to bring you top-notch, KubeCon-grade crash courses on cloud native tech. No vendor pitches, just awesome open source content on projects such as Linkerd, cert-manager, CockroachDB, Pulumi, Polaris, and Goldilocks.

With Kubernetes becoming the new standard for cloud-hosted application development, DevOps teams are driving the technology choices for enterprise-grade cloud native tooling. Freely available open source solutions are often the primary source for these tooling decisions.

KubeCrash provides a half-day knowledge sharing and virtual learning environment for developers, reliability engineers, cloud security specialists, and platform engineers. Learn directly from the maintainers of some of the most popular open source projects in this series of focused talks and workshops.

The KubeCrash program

Come prepared for a schedule packed with great content and actionable insights directly from the teams that maintain some of the ecosystem’s most popular open source projects. The program will cover the latest learnings on implementing scalable zero-trust, scanning workloads for improved cloud native security, using service mesh to ensure high availability across multi-cluster infrastructure, and delivering “serverless” for multi-cloud deployments.

Program overview

Using cert-manager to enable zero-trust identities for intra-pod communication — Jake Sanders, cert-manager maintainer

Modern cloud native architectures require the network to be considered untrustworthy and this is why internal workloads are rapidly driving the use of mTLS and private PKI. This workshop from Jetstack will demonstrate how to use cert-manager to issue, manage and rotate mTLS certs, allowing users to have strongly attested and verified Machine Identities between Kubernetes pods. All without the workload private keys leaving node memory! Think of this session as a precursor to implementing a service mesh solution, using cert-manager to establish zero trust environments, perhaps defined by trust domains, and enforce security for pod to pod traffic.

Multi-cluster failover using Linkerd — Eliza Weisman, Linkerd maintainer

Failover across clusters is a great way to improve the overall uptime and reliability of Kubernetes applications. While whole-cluster failover can be accomplished at the global ingress layer, failing over individual services is a little more difficult. During this session, Linkerd maintainer Eliza Weisman will walk you through how to use Linkerd, the CNCF graduated service mesh, to enable traffic failover for individual services across clusters. Attendees will learn how to combine service mesh metrics, traffic shifting, and cross-cluster communication in a cohesive and automated way using pure open source, while preserving fundamental security guarantees such as mutual TLS.

Optimizing and Securing Kubernetes Workloads with Polaris and Goldilocks — Rachel Sweeney, Fairwinds and Andy Suderman, Polaris and Goldilocks maintainer

Learn how to scan your Kubernetes workloads to improve your resource utilization and security using open source tools Polaris and Goldilocks. You will watch Andy Suderman, Director of R&D and Technology, and Rachel Sweeney, SRE at Fairwinds, as they show how to correctly configure your clusters based on Kubernetes’ best practices for security and efficiency.

Using Kubernetes to deliver a “serverless” service — Lisa-Marie Namphy and Jim Walker, Cockroach Labs

In this talk, Cockroach Labs team members will share how they leverage Kubernetes to deliver a “serverless” experience. Serverless promises to change the way we consume software. It allows us to potentially pay for what we use only and help drive down operational costs by minimizing resource consumption. Architecting for serverless requires a unique look at app logic and how it is deployed—a combination of the logical and physical worlds. An architectural pattern has emerged where we can scale ephemeral compute separate from services that need to persist.

Multi-cloud, single deploy: cloud engineering with Kubernetes and Pulumi — Aaron Friel and Guinevere Saenger, Pulumi

Business constraints and customer requests often require to stand up new Kubernetes environments across multiple cloud providers. This growing complexity in computing infrastructure will incur greater operational costs for organizations when coordinating across multiple teams. Pulumi engineers Aaron Friel and Guinevere Saenger will demonstrate standing up Kubernetes clusters, deploying applications, and automating ops tasks by building a CLI using the Pulumi Automation API. These tools empower every engineer—from application developers to site reliability engineers—to be a cloud engineer.

Join us on May 17

If you’re staying in the Americas (or are up for a late-night session) this KubeCon, join us on Tuesday, May 17th starting at 9 am PST/10 am CST/12 pm EST. Enjoy a specifically curated set of sessions, each led by a project maintainer from projects covering modern cloud native security to improving the developer experience. Register today!

Six Things You Might Not Know About the Pulumi Service

Chris Smith — Mon, 24 Jan 2022 00:00:00 +0000

As a reader of this blog, you’ve probably heard of the Pulumi Service, the default state-management backend of the Pulumi CLI. If that’s the case, there’s also a good chance you’ve heard of a number of the Service’s key features, like helping you organize your projects and stacks, collaborate with others with the help of organizations, or handle sensitive data securely with built-in support for encrypted secrets.

What you might not know, though, is that we’re adding new features to the Pulumi Service all the time, and that some of these features can be fairly easy to miss. So in this post, we’ll highlight a handful of the features you might not be aware of, and that we think make it even easier to manage your infrastructure with Pulumi.

Let’s get started!

Feature 1: Stack Tags

The first thing we’ll cover is the Pulumi stack tags feature. Stack tags are key/value pairs that are associated with your Pulumi stack. You can use them for any number of purposes, from keeping things organized to even flagging whether or not it’s okay for a script to reclaim the stack’s resources.

While you can create, edit, and remove stack tags using the Pulumi CLI (see pulumi stack tag), you can also do so in the Pulumi Console:

Once set, you can quickly filter stacks on your organization’s Projects tab as well.

Feature 2: Link Updates to your CI/CD pipeline

Another helpful thing the Pulumi Service can do is link the Pulumi stack update to the CI/CD job or run that was used to perform it, as well as to the specific source commit:

When Pulumi updates a stack, it will store some information about the local machine state, such the current git SHA if the stack resides in a git repository. It is with this data that the Pulumi Service links to relevant services where possible.

If you’re curious to know all of the information Pulumi has for a stack update, you can navigate to the Environment tab for that update:

Feature 3: Custom Update Messages

One of the most useful pieces of additional data stored with each update is the message property. By default, Pulumi will use whatever the latest git commit message is. But did you know you can customize that message from the command line?

By passing the --message (or -m) flag to pulumi up, you can provide a custom update message, which can help explain the context for why you made a particular infrastructure change:

$ pulumi up --message "Release the hounds\!"

Feature 4: Viewing Resource Changes

Another thing you might have missed is that the Pulumi Service supports multiple views for a stack update’s logs.

The standard view mimics the Pulumi CLI’s output. However, you can switch to the Diff or Diagnostic views, too.

The Diff view displays all of the properties of modified resources, along with their previous/updated values. This can be helpful when trying to understand exactly what happened for a stack update.

The Diagnostic view just outputs so-called “diagnostic” messages from the Pulumi update. Most commonly, this is where you will see the output of things like calls to JavaScript’s console.log(...) or Python’s print(...) functions. It’ll also include the output of dependent tools, like the output when building Docker containers.

Feature 5: Stack Resource Visualization

Another cool feature of the Pulumi Service is being able to visualize a stack’s resources and their relationships to one another. This is a really important feature!

One of the advantages of using an infrastructure as code tool like Pulumi is that you aren’t just creating a flat list of resources. Instead, by the virtue of how the code and resource graph is structured, you are building a graph of resources.

This means that Pulumi has a semantic understanding of not only what resources are in your stack but also how they relate to one another.

If you navigate to a stack’s Resources tab, by default you see them listed in a table. You can filter this list by resource type, or by providing a custom filter:

However, in order to see the expanded resource graph, click the Graph View button. This will switch to rendering the resources as an interactive graph.

Notice here that the resources that are part of a component are grouped together. Also, you can double click on those nodes in the graph to expand or collapse them!

Feature 6: Drilling into Resource Properties

But you don’t need to just settle for a “top-level view” of your resources. You can click on any specific resource on the Resources tab, and see all of the properties of that resource.

This allows you to see all of the properties that Pulumi is tracking for that resource. It will also show you any relationships to other resources in the same stack.

The Pulumi Service also links you to the resource’s cloud provider’s console where possible, so you can see even more details.

Is your resource missing a link to the cloud provider console? With new services being launched from the myriad providers Pulumi supports every day, it’s hard to keep up. So if you would like us to add cloud provider links, please file an issue over in the pulumi/console-requests GitHub repository so we can update our database.

There are plenty of other features to check out in the Pulumi Service, and we have exciting improvements on the way! As always, feel free to stop by the Pulumi Community Slack to learn more, ask questions, or share anything cool you’re up to!

2021 End of Year Review

Laura Santamaria — Fri, 31 Dec 2021 10:26:47 -0600

It’s the end of the 2021 calendar year here at Pulumi, and like everyone, we’re counting down until 2022 while looking back at our year. We’ve had a very exciting year! In case you missed anything from our past year, here’s a rundown of the top stories from Pulumi:

January

Pulumi became SOC2 certified in early January 2021.

February

We brought support for all four official Pulumi languages to EKS.

We released Python support for our powerful Automation API.

March

We immediately followed up with releasing C# support for the Automation API.

The Azure Native provider, which communicates directly with the Azure resource model, went GA (general availability) after a period of being in beta.

April

We had a bit of fun on April Fool’s Day with Pulumi Interstellar (if you don’t understand April Fool’s Day because it’s not something you have in your country, please know this is a joke for fun).

The biggest news for April, though, was our PulumiUP event, where we showcased new releases for the Pulumi platform. Here are the big announcements:

Pulumi 3.0: Automation API, native Providers, Packages and Components, improved SDKs for Python and Go, and new integrations and features.
Pulumi Packages and multi-language Components
The Google Cloud Native Provider

May

We took a bit of a break in May from the blog to be heads down working on the next features after the excitement of PulumiUP. We did talk a bit about integration with App Runner, though!

June

We started posting our release notes as regular updates to the blog.

We took some time to dive into testing infrastructure in one of our many educational articles.

We worked with the community for a hackathon on multi-language components.

We announced Pulumi Team Edition with usage-based pricing and a free tier to get you started and Pulumi Enterprise Edition with access control and more on the same usage-based pricing system.

July

July was a month of education, with articles on Cloud Engineering as a concept, more on testing, and a new series on Kubernetes:

August

We kicked off an entire series on Azure developers for this month. Start here.

September

September saw preparations for Cloud Engineering Summit, with blog posts talking about the upcoming event. Still, we had time for more features!

We announced a public preview of a Helm Release resource.

We launched a Provider for Snowflake.

We shipped the public release of the Pulumi REST API, which had been a behind-the-scenes powerhouse.

We started supporting AWS Lambdas on Gravitron2 processors.

We opened up our roadmap to the public to vote on and discuss upcoming projects.

And finally, just squeezing into the end of September, we launched the public preview of the AWS Native Provider, powered by AWS’s Cloud Control API, so you can use Pulumi to work directly with the AWS resource model.

October

Cloud Engineering Summit was our biggest event of the year, and you can enjoy all of the sessions over on PulumiTV. That being said, we still had a lot to share on the blog.

We introduced resource methods for Pulumi Packages.

We shipped the 1.0 release of the Kubernetes Operator.

We delivered the Pulumi Registry.

November

We started providing same-day support for Azure Container Apps.

We reworked functions to accept outputs.

December

We started some more educational series:

… and Beyond

As we look forward to 2022, we’re not done yet! We’ve done a lot of things in 2021, and we’re on course to do a lot more in the next year. I’m excited that we’re all along for the ride, and we can’t wait to share more features and launches with you. Thanks for being with us, and here’s to 2022.

By the way, did you know we’re hiring?

Announcing the Pulumi REST API

Joe Duffy — Mon, 20 Sep 2021 00:00:00 +0000

Pulumi was designed to be highly extensible from the outset. That includes core languages and cloud providers, of course, but our community is often using Pulumi as a central part of building and connecting their cloud engineering and automation systems, using features like the Automation API. Today we are happy to announce the next major step in enabling these kind of scenarios: the Pulumi REST API. This REST API offers functionality to manage projects and stacks, cloud resources, policies, and more. It has, in fact, been there all along, powering the Pulumi SDK, CLI, and Console behind the scenes, although we haven’t fully documented or supported it until now. That changes today! We’ve already seen some amazing things built with this API and we’re excited to see what you build with it too.

Overview

Pulumi provides a lot of functionality out of the box. But we understand that modern cloud engineering systems are highly interconnected and complex, and that many teams have unique needs. Therefore, we set out to design the Pulumi platform to be extensible in many ways. The Pulumi client (SDK, CLI, etc) was designed to interoperate with the Pulumi server and, though the SDK has a specific set of needs, this protocol has always been designed to be public and easy to consume for custom scenarios and integrations.

Manage Stacks, Updates, Teams, and More

The Pulumi REST API is available at https://api.pulumi.com and includes functionality to interact with and manipulate any kind of metadata managed by Pulumi. That includes:

Projects and Stacks
- List all projects and stacks for a user or organization
- Get metadata about a specific stack, including stack config and resource states
- Transfer a stack between a user and/or organization
- Delete a stack
- Manage stack tags
Previews and Updates
- List all stack previews and updates
- Get the status of an ongoing or past stack preview or update
- List the detailed events occurring during a stack’s update, such as the resource changes, policy checks, or stdout/stderr messages
Organizations
- List or manage users of an organization
- Manage RBAC teams and user roles within an organization
- List or manage access tokens within an organization
Audit Logs
- Get audit log events as JSON
- Export audit log events as CSV or CEF

Authentication and authorization is governed by the Pulumi identity and RBAC systems. As we add functionality to the Pulumi SDK and Service, we are committed to shipping and supporting the underlying REST APIs. If it’s used in our web console or CLI, we want it to be available to so you can add custom features and functionality.

Read complete details about the REST API or keep reading for examples and scenarios.

The REST API for Cloud Engineering

We have already seen Cloud Engineering teams using the Pulumi REST API to build all sorts of custom functionality. These new capabilities are especially powerful when used in combination with the Automation API, which lets you use infrastructure as code as a library embedded into a larger system, and Webhooks, which lets you invoke your own REST API in response to a Pulumi event such as a stack update.

Examples of scenarios we’ve seen include:

Custom dashboards, including “click to deploy”.
Programmatically getting stack outputs to integrate with other automation needs.
Making more metadata-driven Automation API projects that consume project, stack, config, resource, or other service state to make provisioning decisions.
Exposing a full set of projects and stacks into other systems, such as a portal for managed service providers.
Querying resource metadata to find stacks containing resources with certain properties (such as, for example, in response to a security incident).
Managing teams, users, and access tokens for specific automation needs.
Pulling audit logs into other systems (e.g., SIEM).
Enforcing certain policies such as ensuring that stack tags are applied consistently.
Performing automated resource drift and remediation.

With the Pulumi cloud engineering platform, if you can dream it, you can build it!

Example: Finding Stacks with RDS Databases

Let’s take a look at the Pulumi REST API in action. This example will demonstrate querying stacks and resource metadata across an entire organization.

Imagine that there is suddenly a MySQL vulnerability or EOL notice that we need to respond to. We use Amazon RDS and would like to query our entire organization to find projects and stacks that currently have RDS databases deployed using version 5.6 of MySQL. Thankfully the Pulumi REST API makes all metadata about all of my cloud projects and stacks — across all clouds — available for easy querying and inspection. So a dozen lines of bash will do the trick!

The first step is to generate an API access token. The Pulumi Service lets you manage many access tokens (the same as what’s used for pulumi login) and it’s easiest to just create one specific to this project’s purpose. This will let us revoke that token after we’re done and know that there is no lingering access.

To do that, go to https://app.pulumi.com/account/tokens (or click your avatar in the upper right and go to Settings, where you’ll see Access Tokens on the left-hand nav):

Click the Create Token button. That will pop a box that lets us enter a description:

This then gives us the access token text. We need to copy it and put it somewhere for safekeeping, because once we leave this page, we won’t be able to recover it:

We will use this token for all API calls and the token dictates what authorization we will have to perform actions within a given user or organization account. As we will see, all APIs pertain to given permission scopes, much like GitHub, which will determine the permissions granted. If we’re using curl, for example, we will pass the "Authorization: token XYZ" header, with -H, as described in the Authorization section of the REST API documentation.

If we forget to do that, we’ll receive a 4xx error. For instance, 401:

{"code":401,"message":"Unauthorized: No credentials provided or are invalid."}

To test that this is working, let’s curl the /api/user endpoint, which gives us information about the currently “logged in” user (i.e., the user to which the token we’re using belongs):

$ curl \
 -H "Authorization: token pul-fa..REDACTED..fa" \
 https://api.pulumi.com/api/user
{"githubLogin":"joeduffy","name":"Joe Duffy","email":"joe@pulumi.com",...}

OK, looks like everything’s working! Let’s get back to our scenario of finding unpatched RDS instances. To do that, well use a combination of bash, curl, and jq (payloads are JSON).

Here is the full script. We set up the auth header as noted above, enumerate all stacks in the acmecorp organization, export the stack’s metadata including its resources, and then check to see if there are any RDS instances with "mysql" version "5.7" that we need to patch:

# Set up our auth token info.
export AUTH_HEAD="Authorization: token $PAPI_TOKEN"
export ORG_NAME=acmecorp

# Enumerate all stacks in our organization.
for stack in $(
 curl -s -H "$AUTH_HEAD" \
 https://api.pulumi.com/api/user/stacks?organization=$ORG_NAME |
 jq -c '.stacks[]'); do
 ORG=$(echo $stack | jq -r '.orgName')
 PROJ=$(echo $stack | jq -r '.projectName')
 STACK=$(echo $stack | jq -r '.stackName')

 # Query each stack to see if we have RDS instances.
 RDS_INSTANCES=$(curl -s -H "$AUTH_HEAD" \
 https://api.pulumi.com/api/stacks/$ORG/$PROJ/$STACK/export |
 jq -c '.deployment?.resources[]? | select (.type == "aws:rds/instance:Instance")')
 if [[ ! -z $RDS_INSTANCES ]]; then
 for rds in $RDS_INSTANCES; do
 # For each RDS instance, warn on any instance that is 'mysql'
 # version '5.7', and error out so we can patch our instances!
 URN=$(echo $rds | jq -r '.urn')
 ENGINE=$(echo $rds | jq -r '.engine')
 ENGINE_VERSION=$(echo $rds | jq -r '.engineVersion')
 if [[ "$ENGINE"="mysql" && "$ENGINE_VERSION"="5.7" ]]; then
 >&2 echo -e "error: vulnerable MySQL 5.7 RDS instance found:"
 >&2 echo -e " URN=$URN"
 >&2 echo -e " ORG/PROJ/STACK=$ORG/$PROJ/$STACK"
 >&2 echo -e " DETAILS=$rds"
 fi
 done
 fi
done

Now all we need to do is generate a token, set our PAPI_TOKEN environment variable to contain it, and run this script — voila, all of our vulnerable RDS databases will be revealed. Now we can go update those stacks, perhaps programmatically with the Automation API, and then possibly even write a Policy as Code rule to prohibit new 5.6 instances from getting created in the future.

Give It a Try Today

As we have seen, the Pulumi REST API gives us easy access to projects, stacks, cloud resources, teams, and more. It is now generally available to all users of the Pulumi Service, both free and paying customers. To give it a try today, sign up for your free account now, give our Getting Started guide a try, and then check out the Pulumi REST API documentation.

We can’t wait to see the powerful automation you build for your team using the Pulumi REST API! Have fun spelunking in the cloud.

Pulumi Interstellar

Zack Chase — Thu, 01 Apr 2021 00:00:00 +0000

Earth is just the beginning. We are putting down the foundations of space so our children can build their future. At Pulumi, we are committed to making life multi-planetary. We are excited to announce Pulumi Interstellar, a collection of resource providers that will help us reach the future of a space-faring and multi-planet species.

For our first offering of space related resource providers, we have aligned closely with the current priorities in the space industry today:

Getting to New Planets
Making Planets Habitable

We believe that solving those two areas for our customers will enable people to build the future they imagine, on the planets they desire to inhabit. One challenging aspect of achieving our goals is that each new planet represents a different operational environment.

One might think the right solution for provisioning and managing resources in space would be a proprietary domain-specific language designed from the ground up to operate in these new environments. However 10 out of 10 leading rocket scientists confirm that the heterogeneous diversity of existing programming languages allows us to move faster and with less frustration.

In the following sections we will take a closer look at the two main areas defined above and how you too can begin building the future in a familiar programming language like JavaScript, TypeScript, Python, Go, or .NET.

Rockets As Code

Rockets are the first step to making humans a multi-planet species, they enable you and your resources to travel to new planets. Today, we are extremely excited to introduce the Pulumi Rocket Provider, which allows users to configure rocket fuel requirements, initiation sequences, configure deployment payloads, reserve seats, and much more.

In the initial release of the Rocket Provider we are happy to enable two key use cases: Resource Deployment and Planetary Travel.

Resource Deployment

import * as tyrell from "@pulumi/tyrell"

const engineId = tyrell.rocket.getEngine({
 owners: ["420"],
 mostRecent: true,
 filters: [{name: "SuperDraco", values: ["tyrell/engines/super-draco-15.03-*"]}],
});

const spaceship = new tyrell.spaceship.Instance("Edward Isreal", {
 instanceType: tyrell.spaceship.InstanceTypes.interplanetary,
 engine: engineId,
 capability: "kuiper_belt",
}];

from pulumi_tyrell import rocket, spaceship

engine_id = rocket.getEngine(
 owners=["420"],
 most_recent=true,
 filters="""[
 {name: "SuperDraco", values: ["tyrell/engines/super-draco-15.03-*"]}
 ]""",
)

spaceship = spaceship.Instance("Edward Isreal",
 instanceType=spaceship.instance_types.interplanetary,
 engine=engineId,
 capability="kuiper_belt"
)

using Pulumi;
using Tyrell = Pulumi.Tyrell;

class MyStack : Stack
{
 public MyStack()
 {
 var engineId = new Tyrell.Rocket.GetEngine(new Tyrell.Rocket.GetEngineArgs
 {
 Owners =
 {
 "420",
 }
 MostRecent = true,
 Filters = {
 { "name": "SuperDraco", "values": ["tyrell/engines/super-draco-15.03-*"] },
 }
 });

 var spaceship = new Tyrell.Spaceship.Instance("", new Tyrell.Spaceship.InstanceArgs{
 InstanceType = Tyrell.Spaceship.InstanceTypes.Interplanetary,
 EngineId = engineId,
 Capability = "kuiper_belt",
 });
 }

}

package main

import (
 "github.com/pulumi/pulumi-tyrell/sdk/v3/go"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 engineId, err := tyrell.Rocket.GetEngine(&tyrell.Rocket.GetEngineArgs{
 Owners: []string{"420"},
 MostRecent: true,
 Filters: []RocketEngineFilter{
 RocketEngineFilter{
 Name: "SuperDraco",
 Values: []string{"tyrell/engines/super-draco-15.03-*"},
 },
 },
 })
 if err != nil {
 return err
 }

 spaceship, err := tyrell.Spaceship.Instance("Edward Isreal", &tyrell.Spaceship.InstanceArgs{
 InstanceType: tyrell.Spaceship.InstanceTypes.Interplanetary,
 EngineId: engineId,
 Capability: "kuiper_belt",
 })
 if err != nil {
 return err
 }

 return nil
 })
}

On Day 1 you can deploy resources via reusable rockets. By declaring a payload on your Rocket, you tell Pulumi to deliver the payload to the location specified. Resource Deployments will be particularly useful when configuring a new planet in the initial stages, as you can deploy new resources and terraforming equipment in a consistent, repeatable, and scalable manner.

Once the rocket arrives at its destination, Pulumi’s Space AI Assistant, nicknamed IRWIN, will take control to ensure your resources are deployed efficiently and correctly. By using Resource Deployments you can ensure your infrastructure and workloads are up and running before arriving on your new planet.

Planetary Travel

const config = pulumi.Config();
const passengers = config.require("passengers");

const seats = tyrell.tyrell.PassengerGroup("Robinsons", {
 spaceshipId: spaceship.then(it => it.id),
 seats: passengers,
 departure: "Luna",
 destination: "Mars",
 berth: "Colonist",
}];

config = pulumi.Config()
passengers = config.require("passengers")

seats = tyrell.tyrell.PassengerGroup("Robinsons",
 spaceshipId=spaceship.id,
 seats=passengers,
 departure="Luna",
 destination="Mars",
 berth="Colonist",
)

var config = Pulumi.Config();
var passengers = config.require("passengers");

var seats = new Tyrell.Tyrell.PassengerGroup("Robinsons", new Tyrell.Tyrell.PassengerGroupArgs
{
 SpaceshipId = spaceship.id,
 Seats = passengers,
 Departure = "Luna",
 Destination = "Mars",
 Berth = "Colonist",
});

config := pulumi.Config()
passengers := config.require("passengers")

seats, err := tyrell.PassengerGroup("Robinsons", &tyrell.PassengerGroupArgs{
 SpaceshipId: spaceship.id,
 Seats: passengers,
 Departure: "Luna",
 Destination: "Mars",
 Berth: "Colonist",
})

Once a planet is habitable and configured to your specifications, you will need a way to travel to your new home. Simply adding a seats argument to any Rocket will reserve those seats (as they eventually become available).

When your program completes deployment, you will receive a reservation confirmation via the email you used to register for the Pulumi Service. If you are an open source user, you will need to provide a valid email address via the --confirmation-email flag to receive your confirmation email.

Terraforming As Code

One of the most challenging obstacles preventing humans from becoming a multi-planet species is that Earth is the only planet in the known solar system that supports life. While there are many options for making other planets liveable, the team at Pulumi believes in the two step model towards habitability.

First, we need to provision dome-based communities in a repeatable and scalable manner. Once we have established our dome communities, we can then terraform any planet using the latest technology, i.e., nuclear missiles.

Dome-Based Communities

import * as weyland from "@pulumi/weyland-yutani"

const domeId = weyland.dome.getDome({
 owners: ["099720109477"],
 mostRecent: true,
 filters: [{ name: "name", values: ["weyland-yutanu/terraforming/acheron-*"] }],
}).then(it => it.id);

const terradome = weyland.dome,Instance("Hadley's Hope", {
 instanceType: weyland.dome.InstanceType.LV_246,
 dome: domeId,
 type: "atmospheric-processor",
}];

from pulumi_weyland_yutani import dome

dome = dome.get_dome(
 owners=["099720109477"],
 most_recent=true,
 filters="""[
 {name: "BasicDome", values: ["weyland-yutanu/terraforming/acheron-*"]}
 ]""",
)

terradome = dome.Instance("Hadley's Hope",
 instance_type=dome.instance_type.lv_246,
 dome=dome.id,
 type="atmospheric-processor",
)

using Pulumi;
using WeylandYutani = Pulumi.WeylandYutani;

class MyStack : Stack
{
 public MyStack()
 {
 var dome = new WeylandYutani.Dome.GetDome(new WeylandYutani.Dome.GetDomeArgs
 {
 Owners =
 {
 "099720109477",
 }
 MostRecent = true,
 Filters = {
 { "name": "BasicDome", "values": ["weyland-yutanu/terraforming/acheron-*"] },
 }
 });

 var terradome = new WeylandYutani.Dome.Instance("Hadley's Hope", new Weyland.Yutani.Dome.InstanceArgs
 {
 InstanceType = Weyland.Dome.InstanceType.LV_246,
 Dome = dome.Id,
 Type = "atmospheric-processor",
 });
 }

}

package main

import (
 "github.com/pulumi/pulumi-weyland-yutani/sdk/v3/go"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 dome, err := weyland.Dome.GetDome(&weyland.Dome.GetDomeArgs{
 Owners: []string{"099720109477"},
 MostRecent: true,
 Filters: []RocketEngineFilter{
 RocketEngineFilter{
 Name: "BasicDome",
 Values: []string{"weyland-yutanu/terraforming/acheron-*"},
 },
 },
 })
 if err != nil {
 return err
 }

 terradome, err := weyland.Dome.Instance("", &weyland.Dome.InstanceArgs{
 InstanceType: weyland.Dome.InstanceType.LV_246,
 Dome: dome.Id,
 Type: "atmospheric-processor",
 })
 if err != nil {
 return err
 }

 return nil
 })
}

The first step to establish humanity on other planets is to provision dome-based communities and build critical infrastructure and resources while we work to terraform the planet. Out of the box, Pulumi provides boilerplate communities that are flexible and resilient enough to support any use case.

Nuclear Terraforming

While we will not be able to support nuclear terraforming on Day 1, we are working extremely diligently to ensure our customers have access to state of the art terraforming techniques. We hope to bring a preview of Nuclear Terraforming to customers in early Q3 of 2022. Be on the lookout for monthly updates as we develop this new groundbreaking technology.

How To Get Started

Getting started is as simple as getting started with any other Pulumi resource provider, you just need to download the package with your preferred language’s package manager, configure the provider, and then start building. All space resources are billed via Dogecoin, so you will need to provide a wallet address in your provider’s arguments.

$ pulumi config set --secret doge DilB3JrTyGm55FiMyRlhM7NZmfj0J9QVX

Don’t worry, we’ll encrypt your wallet address. Your Dogecoin wallet will go straight to your provider and Pulumi never receives or handles your secrets.

Summary

Today is one of the most exciting days in Pulumi’s short history and we are incredibly excited to partner with our customers and leading space companies to make our multi-planetary dreams a reality. We cannot wait to see what you’ll build!

*{padding:0;margin:0;overflow:hidden}html,body{height:100%}img{position:absolute;width:100%;top:0;bottom:0;margin:auto}" >

Pulumi raises Series B to build the future of Cloud Engineering

Joe Duffy — Tue, 27 Oct 2020 00:00:00 +0000

Today I’m thrilled to announce that we’ve raised $37.5 million in Series B funding led by NEA with participation from existing investors, Madrona Venture Group and Tola Capital. We will use this funding to continue serving our fast-growing community of developers and infrastructure practitioners, making Cloud Engineering the new reality for organizations embracing the modern cloud in all aspects of how they ship software.

All software is cloud software,
All developers are cloud developers,
Infrastructure teams enable innovation.

Our vision for the modern cloud era

All software is cloud software, all developers are cloud developers, and infrastructure teams enable innovation. This is Cloud Engineering. The cloud has become a competitive advantage for many companies, enabling new disruptive business models and end user experiences. Pulumi aims to enable this new way of building.

We set out over three years ago to fundamentally rethink how teams create cloud software, with a belief that by standing on the shoulders of giants—leveraging our industry’s best languages, tools, and ecosystems for infrastructure—we could build a platform that uniquely enables developers and infrastructure experts alike to leverage the cloud in new ways. Despite having created languages like C# and TypeScript in our past lives, it struck our founding team that the world didn’t need another programming language, DSL, or YAML templating solution; instead, we could unleash creativity and collaboration by embracing the very languages our industry already knows and loves.

This led to Pulumi’s unique approach to infrastructure as code, something we often call “infrastructure as software.” Bring your own language, fire up your favorite editor, get full access to any or many clouds of your choice, and code away. Use standard libraries, package up and share and reuse your own best practices, or benefit from the community’s shared knowledge. Tap into all the goodness surrounding that language including familiar engineering superpowers like refactoring, interactive documentation and error checking in your IDE, debugging, test frameworks, and tools like code formatters and linters. All the while still getting the belts and suspenders of infrastructure as code: plan deployments by previewing them before they happen, get a full history of who deployed what and when, enforce security, compliance, and cost controls using policy as code, and more, all open source with an easy to use, but optional, SaaS backend.

In addition to turbocharging infrastructure as code, this new approach has enabled new ways to program the cloud. When we were starting, we often heard that infrastructure is “too boring” or that general purpose languages instead of YAML was “too powerful.” On the contrary, the day that AWS put a REST API in front of spinning up a new virtual machine, the world changed irreversibly forever. By having an entire programming and resource model for interacting with cloud infrastructure, we can layer rich abstractions on top, and build bigger things out of smaller things, as we’ve always done with operating systems and the hardware/software interface. As a recent example and preview of things to come, check out our new Automation API, something that simply isn’t possible with traditional YAML, JSON, or DSL-based infrastructure as code approaches.

We also took an approach of integrating with many incredible partners. Pulumi now supports over 50 infrastructure providers including AWS, Azure, Google Cloud, Kubernetes, Alibaba, Auth0, Cloudflare, Datadog, Fastly, GitHub, GitLab, MongoDB, New Relic, and more; over a dozen CI/CD integrations including Azure DevOps, CircleCI, GitHub Actions, GitLab CI, Jenkins, Octopus Deploy, and Spinnaker, for seamless and continuous delivery; and many identity providers, including Atlassian, Azure Active Directory, G Suite, GitHub, GitLab, and Okta. We embraced the Kubernetes ecosystem from day one, with great support for the entire Kubernetes API, managed clusters in all the major clouds (AWS EKS, AKS, GKE, etc.), Helm 2 and 3, Kustomize, Open Policy Agent (OPA)’s Rego language for policy as code, and a Kubernetes Operator, to name a few key features that are helping customers go to production with Pulumi and Kubernetes.

Three years after starting on this journey, hundreds of customers and tens of thousands of end users have used Pulumi to leverage the cloud in transformative new ways, and we regularly hear from developers and infrastructure practitioners about how Pulumi has enabled them to do things previously thought impossible.

Cloud Engineering is here — and growing fast

Cloud Engineering is bigger than just Pulumi. Last month, thousands of us tuned in to learn from industry leaders and expert practitioners at our first annual Cloud Engineering Summit. It is clear that Cloud Engineering is already here in a big way and becoming a standard for the most forward-thinking software teams.

A lot has changed in only three years. At the time we conceived of Pulumi, there was significant fragmentation, uncertainty, and difficulty in adopting Kubernetes, Serverless, Docker, and continuous delivery for production workloads. Developers seldom touched infrastructure. Fast forward to today, and we now have managed container orchestrators, including Kubernetes, across all the major cloud providers, most organizations are shipping daily, and it’s evident that mixed workloads of containers, serverless, VMs, and data services are here to stay. And 6.5 million developers have gone cloud native. Teams are now looking for solutions that help them go from idea to production faster than ever before, and Cloud Engineering is emerging as the best way to do just that.

Cloud Engineering teams are also standardizing on platforms and workflows that go beyond ad-hoc “walls of YAML and mountains of Bash” approaches, taming cloud software complexity while also improving reliability and security with technologies like policy as code. Teams are also increasingly looking for cloud-agnostic platforms due the reality of multi-cloud, whether that’s because they need to ship code to multiple cloud providers, manage Kubernetes resources in addition to cloud infrastructure, manage hosted SaaS infrastructure services by vendors like Cloudflare, Datadog, MailChimp, or MongoDB alongside their cloud infrastructure, or all of the above.

Because all software is cloud software, every software organization in the world needs a Cloud Engineering Platform. We are happy to be that very platform for many of the world’s most innovative companies, and look forward to helping many more customers disrupt their own industries as the cloud continues to transform modern businesses.

What’s next?

We remain focused on Pulumi’s mission to enable Cloud Engineers of all backgrounds, developers and infrastructure experts alike, to collaborate and do their best work together. We have made great progress in three years, laying a foundation that helps to democratize access to cloud infrastructure. But we’re only just getting started.

So, where do we go from here?

We will continue ensuring Pulumi is unquestionably the best infrastructure as code platform available. Features like the Automation API are enabling groundbreaking ways to use cloud infrastructure, powering entire platforms that themselves are powering entire organizations and new cloud products. We are also hard at work creating libraries with built-in best practices, new workshops and educational material, and simpler user experiences, all with the aim of making the cloud easier to use than ever before.

We aren’t going to stop there. This foundation also enables new and exciting product experiences. Although all software is becoming cloud software in a meaningful way, the cloud’s capabilities aren’t yet as easy to harness in our applications as a standard operating system’s are. The next phase of our journey will tackle this problem.

Pulumi is and will continue to be the Cloud Engineering Platform powering the world’s most innovative companies and teams. All software is cloud software, all developers are cloud developers, and infrastructure teams are central to enabling this innovation—and this Series B fundraise will help to ensure we realize the full potential of this vision!

I’d like to take a moment to thank our customers. Our community has grown into an inclusive and collaborative place, reflecting our own company values. One of those values says, “our own success is directly enabled by the success of our customers,” which we remember every day by helping you transform and innovate. We could not have done it without your support and you will continue to be our guiding light for the future.

To the clouds and beyond!

Joe

Pulumi Wins 2020 Gartner Cool Vendor Award

Sophia Parafina — Thu, 28 May 2020 00:00:00 +0000

Pulumi is honored to be named as one of only three vendors in the 2020 Gartner Cool Vendor for Agile and DevOps report, published on May 28th, 2020. Being recognized in this way is a strong validation of Pulumi’s impact thanks to our more modern approach to Infrastructure as Code and approaches to building cloud software. Vendors can only be selected once and in only one category making this an exclusive award.

What Makes a Cool Vendor

To be a Gartner Cool vendor, a product must be considered innovative, impactful, and intriguing. Vendors are scored on a scale of 1-10 across the following areas: technology, use of technology, business model, transformative nature, and newness. Vendor claims are rigorously verified by Gartner analysts who speak to customers to confirm that the technology works.

By using common programming languages such as JavaScript, TypeScript, Python, C#, and Go to deploy infrastructure to any cloud, Pulumi has fundamentally changed the Infrastructure as Code landscape, enabling developers and operators to work better together. The impact on organizations is immediate. Instead of copying YAML or using a DSL, companies can write code to deploy infrastructure using industry best practices for development such as unit and integration testing, versioning, and error correction in IDEs. Moreover, infrastructure can be abstracted into reusable components to standardize on best practices while shipping faster with confidence.

Cool Features

Here is what we think makes Pulumi stand out from the rest.

Open-source meets SaaS. We do everything in the open, and the entire platform is available on GitHub. The Individual Edition includes the core provisioning engine, language and cloud providers, testing, Policy as Code, secrets management, and over a dozen CI/CD integrations. You can use Pulumi entirely offline or our SaaS for state management and team collaboration. We see strong community growth, and in fact, our community members are sharing and reusing best practices of their own!.
Any cloud, any language. Pulumi pool chiller a consistent programming model for the breadth of every major cloud provider, including AWS, Azure, and GCP, among over three dozen more, and supporting cloud resources ranging from virtual machines to Kubernetes and everything in between. You have the freedom to choose your language and deploy on one cloud provider or any combination of providers.
Testing. Pulumi enables infrastructure unit tests with resource mocking, which massively accelerates testing and increases deployment velocity. You can perform integration testing by quickly spinning up an ephemeral environment with only the resources you need to test or test long-lived environments in your CI/CD pipelines.
Policy as Code. Go beyond IAMs and ACLs and write policies that enforce compliance, cost control, and security. You can even secure your software supply chain by requiring the latest cloud resource versions to ensure vulnerabilities have been patched. Pulumi’s policy engine is open source, and the Enterprise Edition SaaS offers advanced features for organization-wide controls.

And we like to think that our mascot, Pulumipus, is also pretty cool.

Although Pulumi is cool, we have hundreds of organizations worldwide that have decided to run their production infrastructure using Pulumi. Check out some select stories here.

Be one of the cool kids today by giving Pulumi a try today – it’s not just cool, but the future of Infrastructure as Code and, we think, the way all developers and operations teams will collaborate in the future.

Announcing Pulumi 2.0, Now with Superpowers

Joe Duffy — Tue, 21 Apr 2020 00:00:00 +0000

This blog post announces Pulumi 2.0, which has been superseded by newer versions. For information about the latest Pulumi release, please refer to the current documentation and recent release notes.

Today we are excited to announce Pulumi 2.0, the next major stage in our journey as an open source project, company, and community. This release expands on our original vision of using your favorite languages and tools to do all things infrastructure as code, now with new cloud engineering superpowers that will help you and your team adopt modern cloud architectures.

Get Started Learn More

What is Pulumi?

We call Pulumi’s unique approach Modern Infrastructure as Code:

Familiar Languages. Use the best languages and tools to declare infrastructure and gain best-in-class productivity and engineering capabilities. Connect to existing communities and ecosystems and leverage world-class language and tooling innovation.
Multi-Cloud. Adopt consistent workflows across any cloud(s) — public, private, or hybrid — while still using the full breadth and depth of your target cloud’s services.
Modern Architectures. Although Pulumi works great for legacy workloads, it was born in a world of modern container, serverless, and cloud-native architectures. Feel free to mix and match these approaches as you modernize over time.
Cloud Engineering Teams. By choosing Pulumi, you’re betting on your team. The whole team, in fact: developers, infrastructure and operations teams, and security engineers alike. Tackle cloud solutions together without technology or team silos.

Introducing Superpowers!

The main theme of 2.0 is something we’ve taken to calling “superpowers.” This isn’t just a buzzword we came up with — it’s something we keep hearing from happy end users.

What is Pulumi 2.0?

After shipping 1.0 last year, we’ve been hard at work helping customers to succeed at their modern cloud projects. This has taken us beyond the basics to include:

Provisioning. All core language SDKs are now at parity, including all existing languages — Node.js (JavaScript, TypeScript) and Python — as well as the new additions to the family — .NET (C#, F#, etc) and Go. We’ve also significantly expanded upon the supported cloud resource providers in the Registry, totalling above three dozen, and rolled out entirely new API documentation.
Delivery. As we’ve worked with customers to go from development to production, we’ve added more CI/CD integrations as well as environment management capabilities to help at scale, such as project and stack tagging and filtering.
Architecture. Customers are building their own platforms that use abstraction and packaging mechanisms, including our component model, to codify their own best practices. New coexistence and migration options help to build larger systems out of many component parts, including those you haven’t been able to port to Pulumi yet.
Policy. Our new policy as code framework, CrossGuard, lets you define policies using familiar languages and then apply those policies at deployment-time. This prevents mistakes from ever getting out the door, including cost, security, and compliance. Configurable policy packs let you write a single policy and apply it flexibly across your projects.
Testing. Many customers are using a spectrum of techniques to validate their infrastructure in new ways. This includes unit testing using familiar frameworks and new mocking capabilities through integration testing for short- and long-lived environments.

Welcome to the Team, Pulumipus!

We’re also taking the opportunity to have a little fun with the launch and introducing a new mascot, The Pulumipus. You may have met them already at an event or at PulumiHQ, and we’re happy to welcome them as the official mascot going forward.

Expect The Pulumipus to pop up from time to time to help you out with your infrastructure superpowers!

Give it a Try Today

Today is an exciting day for us, our customers, and our community — your passionate support and feedback shaped this release, and we thank you.

To learn more, join us for our launch event next Wednesday, or just download Pulumi and give it a try. If you’re upgrading from Pulumi 1.0, please see our migration guide.

Get Started Learn More

Please tell your friends and colleagues — we will see you in the Community Slack, where we look forward to hearing what you’re doing with your new superpowers!

Joe

Pulumi’s Coronavirus Plan

Joe Duffy — Wed, 18 Mar 2020 00:00:00 +0000

Together, we’re facing an unprecedented situation with the COVID-19 pandemic. In this note, I’d like to tell you about the steps we’ve taken to ensure the health of our employees, community, and to ensure business continuity throughout.

To our customers

First and foremost, our thoughts are with anyone impacted by the virus either directly or indirectly. We are taking the utmost precautions necessary to ensure the health of our employees and the community.

At Pulumi, we are entirely customer and community-driven and have taken a number of steps to ensure business continuity. We know that stable infrastructure is critical to keeping your own business running smoothly and want you assure you that Pulumi is here for you as you focus on your own response to the situation.

Cloud infrastructure can be an essential component in the fight against this pandemic — something we are keenly aware of as we have government and healthcare customers that must scale up their response. If you or someone you know may be able to benefit from our technology, please contact us directly.

A cross-functional leadership team at Pulumi met early on and acted quickly, moving to remote work over 2 weeks ago. This same team regularly assesses recent developments and monitors local, state, federal, and international guidelines, to ensure that we react appropriately to the ever-changing situation.

Here are some of the concrete steps we have taken.

Remote work

The Pulumi team is fortunate enough to have been remote-friendly since our inception, with many employees already outside of our Seattle HQ, including international employees who are ready to help our customers. Over 2 weeks ago, we made the quick decision to move to 100% remote temporarily so that employees are safe and to support social distancing initiatives.

This shift ensures we are doing our utmost to support the containment of the disease spreading. Moving from remote-friendly to all-remote has required some changes, however, our own team has felt that it has overall pushed us to improve communication and knowledge sharing. GitLab’s all-remote guide has been very helpful in this transition.

Software as a service continuity

From day one, the Pulumi service was architected for continuous delivery without manual processes. We ship new features and improvements multiple times a week, every week, and this situation has not impacted that cadence. We have strong operations policies and procedures to respond to any situations that may arise.

Expect to see the same great dependable service you’ve come to know with a steady stream of new features and improvements in the weeks ahead as we get ready for our major product launch in early April.

Leveraging digital channels

The demand for Pulumi content and education continues to grow and, even in the face of restricted travel, we will ramp it up further. Although many events worldwide we’d planned to sponsor or otherwise participate in have been canceled, we have shifted to working with customers through digital channels wherever we can.

Our blog, Community Slack, and PulumiTV YouTube channel are already great online resources, but we have significant new efforts in the works. We’re starting a weekly live video stream called Modern Infrastructure as Code Wednesday featuring learnings from our deepest technical experts. We are having our first online Hackathon this Thursday. Our major upcoming launch on April 22nd has now turned into a day full of online infrastructure content from our team, customers, and partners. Learn more details about these digital events and more here.

Please drop us a note if you’d like us to participate in an online event, host a joint webinar, or want to see a specific topic covered.

In the weeks ahead

We will continue meeting regularly as a leadership team, and talking to our customers and partners, so that we can evolve our strategy and take new measures for health, safety, and continuity purposes as appropriate. We will keep you updated.

We are confident in what we are doing to keep our employees safe, continue delivering value and reliability in our products, and to keep Pulumi’s business functioning normally for our valued customers. Even as we deal with this pandemic, our entire team remains laser-focused on your success and, as always, if there is anything we can ever to do help with your own ever-changing needs, please get in touch with us.

An Update on our Roadmap

Joe Duffy — Mon, 09 Mar 2020 00:00:00 +0000

2020 is off to a big start for us! The .NET and Go SDKs are now more idiomatic and easier to use, Policy as Code is ready for prime-time, and many other useful features and foundational improvements have been released. I’d like to take a moment to highlight these improvements and ask for feedback — we want to make sure these new features work great for you!

.NET

Our .NET SDK has come a long way. We recently tidied up the API surface area to feel more familiar to .NET developers, added C# getting started guides for AWS, Azure, GCP, and Kubernetes, and added many new examples. We also closed the gap on core Pulumi features missing from the initial release, including aliases, transformations, StackReference, and .NET versions of the Kubernetes, Docker, and Terraform remote state packages.

The remaining areas to improve for the .NET SDK include making the API documentation available in C# (details are below in the “still to come” section), migrating to .NET Core 3.1, and enabling you to test your infrastructure using familiar C# tools more easily.

Go

We completely overhauled our Go SDK to be more idiomatic and easier to use. This blog post provides an overview of this new programming model.

The major change in the Go SDK is that we previously used map[string]interface{} types in many places instead of properly-typed data structures. No more! Instead, you now get strong typing for all Go resource APIs across all providers, including AWS, Azure, GCP, and Kubernetes. The improvements here are huge, as you can see in this before/after PR. Examples and documentation have been updated to this new style.

GoDocs are now here for all of our providers and we’ve also added Go versions to our getting started guides for AWS, Azure, and GCP, and have many more improvements on the way.

We’ve also added support for core Pulumi features, including aliases and transformations. A few remain to flesh out like adding Go support for our Kubernetes, Docker, and Terraform remote state packages. The biggest change to come is retiring Dep in favor of Go modules everywhere.

Policy as Code

CrossGuard, our Policy as Code framework, is now ready to take for a serious test drive. CrossGuard lets you to write policies in real code and enforce them during updates. Policies can check for anything; however, common checks include security, compliance, cost management, and general and team best practices. You can write your policy packages or use off-the-shelf policy packs like our AWSGuard package.

CrossGuard is open source and the functionality is behind the new pulumi policy command, as well as the new --policy-pack flag for the preview and up commands. You can use these features with any edition of Pulumi, without restriction, including the offline backends.

If you choose to use CrossGuard with the Enterprise Edition of Pulumi, however, you’ll get some added functionality. This includes server-side enforcement of policies and organizational policies, including “policy groups” which let you group and apply many policies together to your stacks, such as applying different policies to production than development environments.

We recently added the ability to tag policy packs with a semantic version so that you can easily manage and apply packs using user-friendly versions instead of auto-generated numbers.

The remaining areas of focus between now and launch include:

Enabling policy pack authoring in Python. Policies can already apply to stacks written in any language, no matter the language of the policy pack itself, but today policy packs are authored in Node.js. Python is coming soon, with Go and .NET following afterward.
Allowing policy packs to be configured. Configuring policy packs will let you selectively enable or disable specific rules, and/or set configuration variables on them that influence their behavior, on a per stack or policy group basis.
Making reference policy packs available for AWS, Azure, GCP, and Kubernetes. These packs will be usable out-of-the-box with a collection of some of the most important configurable rules we’ve seen, in addition to being a starting point for creating your packs.

CrossGuard is ready to try today, and we’ve begun blogging about some example use cases that you may find interesting.

Other Goodies

In addition to the major areas above, we’ve continued to ship many other improvements. Here are some of our favorite ones:

Stack export for the CLI and in the service, enabling you to export any past version of a stack’s state file. This helps you debug, restore, or roll back changes, or even just for historical reporting purposes.
Many quality-of-life improvements in the Pulumi SaaS Console, including stack tagging, grouping, and sorting; deep linking into CI/CD systems; infrastructure configuration pretty-printing; reverse stack membership lookups; performance improvements; and more.
Full audit logs for Pulumi Enterprise enable you to go back in time and figure out who, when, and why something about your organization, project, or stack changed.
Pulumi runtime mocking for infrastructure testing, developed in partnership with several of our customers leading the charge on “test-driven infrastructure.” Look for more end-to-end docs and guidance soon!
The ability to generate Kubernetes YAML gives you infrastructure authoring capabilities in your language of choice, while still letting you drive deployments using other YAML-based tools, such as kubectl.
We’ve been steadily adding more and more providers and now have over 30 of them, including Aiven, Cloudflare, Consul, Datadog, DigitalOcean, DNSimple, F5 BIG-IP, Fastly, GitLab, Kafka, Linode, MySQL, New Relic, Okta, Packet, PostgreSQL, RabbitMQ, SignalFX, Vault, and more.

Goodies Still to Come

Our API docs are getting a major overhaul! The new design uses a more resource-oriented view, which will make it much easier to navigate and find what you need. This will also ensure all languages are uniform in terms of how documentation is presented as well as coverage of code samples. Expect to see some of this land and evolve in the days to come.

We continue to make progress on our multi-language library architecture. This will enable us to bring your favorite packages like EKS and AWSX to new languages, including Python, C#, and Go, in addition to eventually letting you write your own. This has been a long journey as we figure out how to correctly build this capability, however we will be bringing EKS to Python first.

Our tf2pulumi tool, which converts any Terraform HCL to Pulumi code, is currently getting a fresh coat of paint. Look for HCL2 support and Python code-generation soon.

As of Pulumi CLI v3.71.0, tf2pulumi has been replaced with pulumi convert --from terraform. Converting Full Terraform Programs to Pulumi blog has more details.

We’ve made some significant performance improvements and have others still in the works. Recent changes have reduced overall network bandwidth used by an average update in some cases by 100x, as well as reducing startup times by multiple seconds. The net result is that you’ll see fewer delays during updates and an overall snappier CLI experience.

Next Steps

A lot of good work has already landed in 2020 — and there is a lot more on its way, including a few surprises!

We appreciate all the feedback you’ve given us that is helping to make sure our work in early 2020 is a major step function in overall features, quality, and capabilities. We’re always looking for feedback so please let us know what you think. Give Pulumi a try today, drop into our Community Slack to share your feedback, and make sure to file any suggestions or improvements we can make on GitHub.

2019 Year at a Glance

Joe Duffy — Tue, 31 Dec 2019 00:00:00 +0000

As we celebrate another incredible year in the books here at Pulumi, I wanted to share some reflections about our most exciting milestones over the past twelve months. The best part has been connecting with more customers worldwide, as we saw more than a 15x growth in our customer base, surely a sign of big things to come in 2020. We couldn’t have done it without our amazing community; thank you deeply for your continued support and passion around Pulumi’s bold mission to empower every engineer to program the cloud — you make it all worthwhile.

It’s impossible to highlight everything, but here are some of our favorite milestones:

Pulumi 1.0. After working with several major organizations to go to production with Pulumi, including multiple Fortune 500 and Global 2000 customers, we felt it was time to focus on a “1.0” release. To us, that meant significant improvements to quality, rounding out any open design issues, and sticking to compatibility. As such, we finished JavaScript, TypeScript, and Python support, made significant improvements to documentation and examples, and landed on a release we were proud to stand behind. Months later, we are up to version 1.8, and have stuck to our promise of “no breaking changes”: we understand how fundamental an infrastructure as code tool is to your organization, and how frustrating breaking changes would be. A solid 1.0 serves as a critical foundation for us going forward.

Global community. Our community of end users grew significantly during 2019, with thousands of end users sharing best practices on our Community Slack, and we expanded our physical presence globally. This included over two dozen events from big to small, including AWS re:Invent, Microsoft Build, Google Cloud Next, KubeCon EU and NA, and many Devopsdays, as well as several community workshops. We’ve also seen an uptick in community-led user groups and presentations, consulting and integration partnerships, and look forward to expanding our efforts here even more significantly next year. (If you ever need support/swag for your event, please contact us!)

Pulumi 2.0 roadmap. After 1.0, we published a roadmap to our next major release, Pulumi 2.0. The two themes for this release are to continue pushing the boundaries of productivity, while also giving you “belts and suspenders” for teams and enterprises. We feel this is the magic of Pulumi: the ability to simultaneously run fast and build out infrastructure productively and using the latest and greatest features in AWS, Azure, Google Cloud, and Kubernetes … while still ensuring that security, compliance, and cost-related requirements and best practices are adhered to.

Simpler, per-user pricing. When we launched, our pricing was based on the number of stacks you have. Although our hearts were in the right place with this approach, we realized after working with customers that this could force architectural choices based on price, not what was in the best interest of teams and their projects. This didn’t feel right — we don’t want to penalize teams for doing the right thing! We rolled out per-user pricing in response and have had success with teams of all sizes — from teams of less than 5 engineers all the way up to teams of hundreds.

Pulumi Crosswalk. We introduced the Pulumi Crosswalk brand for our collection of cloud-specific higher level frameworks that simplify many common infrastructure patterns, while also including built-in best practices. Throughout the year, we’ve rounded out that support, starting with AWS, then Kubernetes, as well as some initial steps towards frameworks for Azure and Google Cloud:

Pulumi Crosswalk for AWS simplifies common AWS patterns, including creating VPCs, ECS, EKS, and Fargate clusters, building and publishing Docker images to ECR registries, and more.
Pulumi Crosswalk for Kubernetes encapsulates battle-tested approaches to running Kubernetes clusters and related infrastructure in production. This includes playbooks for running Kubernetes in AWS, Azure, or Google Cloud, as well as kx, a highly productive library that eliminates the “walls of YAML” boilerplate from your Kubernetes applications configurations.
Although still works in progress, Pulumi Crosswalk for Azure will include simple Azure Functions callbacks and globally distributed applications that auto-scale alongside Cosmos DB, and Pulumi Crosswalk for Google Cloud will include simplified Google Functions.

Policy as Code (“CrossGuard”). We shipped a preview of our new policy as code support, code-named CrossGuard, which allows you to enforce policies at deployment time. CrossGuard brings the same “familiar language” approach we’ve used for infrastructure as code, and applies it to policy as code, enabling you to author your own rich rules to enforce security, compliance, cost, and team best practices. We open sourced previews of out-of-the-box rules by way of AWSGuard and OPA integration. CrossGuard is open source and Pulumi’s Team and Enterprise Editions support advanced organization-wide controls.

.NET. We have added initial preview support for writing your infrastructure as code in any .NET Core language, including C#, F#, and VB. This covers any existing Pulumi resource provider, including AWS, Azure, Google Cloud, as well as support for the entire Kubernetes object model. .NET, along with Go, will transition out of preview into being generally available as part of the 2.0 release.

More identity options. Out of the gate, we supported GitHub as an identity provider when authenticating with the Pulumi service. Since then, we added GitLab, Atlassian, Email, and SAML/SSO, with user guides for Azure ActiveDirectory, Google GSuite, and Okta. These all integrate with features like history/auditing, RBAC, and policy as code.

Enhanced, pluggable secrets. We heard that keeping secrets from leaking in plaintext to your start files is a challenge with current infrastructure as code tools. So, we deepend the way secrets integrate with Pulumi’s engine, including transitive encryption so that, no matter where your secret goes in your program, it won’t leak in plaintext. In addition, we added the ability to use your own secrets provider for encryption, including AWS KMS, Azure KeyVault, Google Cloud KMS, and HashiCorp Vault. This ensures you get great ease-of-use — falling into the “pit of success” with secrets — while still leveraging your existing organization’s approach.

More state storage options. Although the hosted Pulumi SaaS works great for most end users, offering the right mix of convenience and control, we understand it doesn’t work for everyone. It is now possible to store state locally using the pulumi login --local command, or to configure Pulumi to store state in AWS S3, Azure Blob Storage, or Google Cloud Storage. Additionally, Pulumi Enterprise offers a self-hosted option, so that you can still benefit from the Pulumi SaaS’s features around identity, policies, and controls, while hosting it in your own cloud account or behind your firewall.

More CI/CD integrations. In partnership with customers, and in some cases the CI/CD vendors themselves, we have continued adding more CI/CD integrations, to enable automated delivery of your infrastructure. This includes Jenkins, Azure DevOps Pipelines, Octopus Deploy, Codefresh, and revised support for GitHub Actions.

Coexistence and conversion tools. We know that many teams have existing infrastructure in place. We want to make it a easy as possible to choose Pulumi and, as we’ve worked with major customers to transition away from existing infrastructure tools, we’ve built tools to help ease the adoption process. This includes the following approaches:

Coexist with existing infrastructure, including referencing existing Terraform remote state, deploying Kubernetes YAML as-is, deploying Kubernetes Helm charts, deploying AWS CloudFormation stacks, or deploying Azure Resource Manager (ARM) templates — in each case, enabling you to consume output properties from infrastructure deployed via other tools, for either temporary or permanent coexistence of Pulumi with these other tools.
Convert your existing infrastructure so that it comes under the control of Pulumi. This includes adopting existing cloud resources, no matter how they were provisioned (even if done manually by pointing and clicking in your cloud’s UI), as well as the tf2pulumi tool, which converts existing HCL programs to Pulumi, preserving your existing Terraform project and code structure.

As of Pulumi CLI v3.71.0, tf2pulumi has been replaced with pulumi convert --from terraform. Converting Full Terraform Programs to Pulumi blog has more details.

Support for large-scale projects. As we’ve seen bigger and bigger Pulumi projects, we have put together guidance for multi-project approaches. This includes adding the ability for stacks to reference one another, an essential building block when, for instance, one project depends on a piece of infrastructure provisioned by another project.

Lots of providers! We’ve added more than 20 new providers for managing resources across many cloud and service providers. This includes the existing core cloud providers for AWS, Azure, Google Cloud, and Kubernetes, in addition to newly added providers DigitalOcean, Linode, OpenStack, Packet, and vSphere. Modern cloud infrastructures include components managed by SaaS or other service providers alongside and connected to your core cloud infrastructure, and for those, we have added support for Aiven, CloudAMQP, Cloudflare, Consul, Datadog, DNSimple, Docker, Fastly, F5 BigIP, GitLab, Kafka, MySQL, New Relic, Okta, PostgreSQL, RabbitMQ, SignalFX, Spotinst, and Vault.

It’s impossible to overstate how big an impact you, the community, have had on Pulumi. In fact, every feature on this list was delivered in response to a real community, customer, or partner need.

2019 has been a major year for the Pulumi community by any measure. As we look forward, we anticipate even more accelerated growth and capabilities. We wish you all a wonderful New Year, and want to again thank you for the love and support — here’s to an incredible 2020!

— Joe

Pulumi 2.0 Roadmap

Joe Duffy — Mon, 02 Dec 2019 00:00:00 +0000

Today we’ve published Pulumi’s 2.0 roadmap. 2.0 is the next major phase in Pulumi’s journey, and will include better productivity through languages, libraries, and tools, in addition to advanced features for teams in production. And, though we are excited to share our own thoughts, more than anything else, we’d love to hear your feedback to help make sure it’s right.

Since releasing Pulumi 1.0 in September, we have heard loud and clear that you appreciate the commitment to compatibility, as well as the completeness and stability of the platform, and we have been hard at work making sure we honor those promises.

As we look to 2.0, we are aiming to help you be more successful with modern cloud architectures, by implementing features that we’ve heard are missing or could be better, as well as doing better at the fundamentals. Although this is a “2.0” release, we do not plan any major breaking changes, so that we can continue honoring our promise to provide a stable platform to you.

Two Themes for 2.0

The two themes we have chosen for Pulumi 2.0 are:

Best in class productivity. Pulumi’s foundation of general-purpose languages has already enabled great productivity gains thanks to ecosystems of tools, sharing and reuse, and familiarity. But we have ideas for more. This includes more languages, features that bring deployments closer to the inner development loop like watch mode, more libraries of off-the-shelf patterns and practices, and the ability to test your infrastructure more easily.
Belts and suspenders for teams and enterprises. As we work with more teams and enterprises to deliver demanding many-cloud workloads to public, private, and hybrid clouds — increasingly involving Kubernetes, and often spanning developers and operations teams — we have heard about many features that would maximize Pulumi’s effectiveness and utility. This includes advanced security and compliance capabilities, more organization-wide controls and policies, and more flexible self- and on-premises hosting of the Pulumi service.

Several major 2.0 features are available to try out today and we’d love to hear what you think. The 2.0 roadmap is on GitHub so that you can easily comment or follow along as we make progress.

Trying Out 2.0 Preview Features

A number of new features are already available in “preview”; the preview tag simply means we expect possibly-significant changes to them between now and the 2.0 release.

Many features require you to set the PULUMI_EXPERIMENTAL environment variable to true before you can use them. This can be done by running export PULUMI_EXPERIMENTAL=true in your macOS or Linux shell, or by running $env:PULUMI_EXPERIMENTAL = "true" in your Windows PowerShell console.

Better Language Support

Support for Go and .NET Core (C#/F#/VB) languages is currently marked preview because there is known work before we’re comfortable calling them “done.” This includes improving the programming models to be more idiomatic for their respective ecosystems — such as stronger typing in Go instead of interface{} and StackReference support in .NET — as well as improving the documentation, availability of samples, templates, and tutorials, written in those languages.

Expect to see a steady stream of improvements between now and 2.0 being finished; to follow along, see these work items on GitHub: Go and .NET.

Policy as Code (“CrossGuard”)

Today we have released preview support for a new policy as code offering that we call CrossGuard. Just as you can write your infrastructure as code in your favorite language, so too can you now write your policies. These policies run whenever you do a Pulumi deployment, enabling CrossGuard to restrict deployments that violate security, compliance, cost, or stylistic requirements.

You can write custom policies, or use off-the-shelf policies authored by the community, including our preview AWS pack and Open Policy Agent (OPA) integration. Just as you can share your own infrastructure creations using your language’s package manager, you can share your policies too.

CrossGuard is 100% open source, enabled by pulumi up’s new --policy-pack flag, and advanced organization-wide controls are available in the Team and Enterprise Editions. To learn more, read the CrossGuard user guide; or, to dive straight in, try out the getting started tutorial.

Testing

We’ve blogged previously about how to test your infrastructure. However, we hear all the time about new scenarios and ways in which Pulumi could do a better job here, including removing many of the rough edges currently hit when testing Pulumi code.

Pulumi 2.0 will support a spectrum of testing scenarios, including unit testing, integration testing, and post-deployment runtime validation. The addition of CrossGuard also means you can validate your infrastructure configuration integrated into your existing CI/CD processes. As part of the 2.0 release, we will be publishing comprehensive guides and tutorials for testing your infrastructure.

Watch Mode

We recently added the ability to deploy changes to your applications and infrastructure continuously, straight from your IDE. This brings infrastructure as code to your core inner development loop in a major way and is guaranteed to change the way you think about the essential differences between the way you approach engineering modern cloud applications versus the requisite infrastructure.

To give watch mode a try, simply run pulumi watch. Or, to see a demo of watch mode in action, check out this video:

Self-Host and On-Premises

Although we’ve worked hard to make the Pulumi service easy and affordable to use by default, we understand that some customers can’t depend on a multi-tenanted SaaS website for their infrastructure deployments. For these customers, Pulumi already offers custom state hosting in AWS S3, Azure Blob Storage, GCP Cloud Storage, or even as a manually managed JSON file. However, by hosting the state this way, you lose out on what makes the Pulumi service so great: identity, teams, RBAC, policies, webhooks, and more.

Today we are releasing preview support for the self-hosted Pulumi service. This gives you the full capabilities of the Pulumi Enterprise SaaS, while also giving you the flexibility to run it anywhere. Self-hosted Pulumi Enterprise can run on-premises behind your firewall on a VM or in Kubernetes, in your AWS, Azure, or GCP account — virtually anywhere that can run VMs or containers. For more information, see the tutorial about how to configure self-hosted Pulumi Enterprise, or contact us for pricing information and a demo.

A Lot More …

This is just a sampling of what’s already ready to try out. Expect a lot more, including many features and stability improvements that are driven by ongoing customers and community feedback.

From Modern Infrastructure to Modern Architecture

Pulumi 1.0 gave us a solid foundation of modern infrastructure as code. As we see the things you are building with Pulumi, we understand that challenges don’t stop at infrastructure provisioning. Pulumi is also about the applications themselves, as modern cloud architectures demand infrastructure and applications to coexist in harmony and unique ways that Pulumi enables.

In addition to many great features, Pulumi 2.0 will also further advance the Pulumi Crosswalk family of offerings to bring more of these modern application architecture patterns to your fingertips. This includes support for additional languages beyond JavaScript and TypeScript, improved documentation, interactive tutorials, and more extensive libraries of patterns and practices.

Two Asks

Although a lot of 2.0 is already underway, we are still planning out the concrete roadmap and there is a lot of room for feedback. We have two asks for you:

Please give the preview features a try, and let us know what you think.
Please let us know what is missing from our list.

Please give us feedback by commenting on the GitHub tracking issue or in the Community Slack.

Next Steps

1.0 was a major milestone for us and 2.0 is going to be better. 2.0 will make modern cloud architectures even more approachable and productive to create, while also deepening support for modern teams and workloads running in production, often across many-clouds.

We want to thank you for your continued support and feedback — it’s what makes Pulumi tick!

Announcing CrossGuard Preview

Erin Krengel — Mon, 02 Dec 2019 00:00:00 +0000

This blog post discusses CrossGuard in “preview” status. CrossGuard is now fully released and integrated into Pulumi. For current information about Pulumi’s policy as code capabilities, please refer to the latest CrossGuard documentation.

Over the past few months, we have been hard at work on Pulumi CrossGuard, a Policy as Code solution. Using CrossGuard, you can express flexible business and security rules using code. CrossGuard enables organization administrators to enforce these policies across their organization or just on specific stacks. CrossGuard allows you to verify or enforce custom policies on changes before they are applied to your resources. CrossGuard is 100% open source and available to all users of Pulumi, including the Individual Edition. Advanced organization-wide policy management features are available to Enterprise customers.

You can run policies against any Pulumi stack, independent of the stack’s language. This means you can have policies written in TypeScript and run them against a Python-based stack. Policies are grouped in Policy Packs and run against a stack during both pulumi preview and pulumi up. Each resource in a stack is run through the Policy Pack prior to creation, meaning you can block the creation of out-of-compliance resources.

Why

Using Pulumi, developers and operators are empowered to self-provision their infrastructure and move quickly. One of our core goals is to allow you to codify best practices and share them with your team. We often hear from organization leaders that they are concerned about enforcing security and compliance best-practices. CrossGuard provides a way to automatically enforce compliance, allowing teams to operate self-sufficiently while still ensuring organization-wide requirements are met.

As an organization administrator, you may want to encourage use of smaller compute instances to optimize for cost. As a security engineer, you may want to ensure databases and other storage solutions are not publicly accessible. As a team lead, you may want to ensure your containers have reserved an appropriate amount of memory and CPU to meet your team’s requirements. Whether you’re an organization administrator, security engineer or team lead, CrossGuard provides you the capability to enforce best practice for cost, compliance, security, team practices, and much more.

Key Features

The key features available during the CrossGuard preview are:

Policy SDK to express custom policies using TypeScript or JavaScript
Run Policy Packs locally using the --policy-pack flag, available to all users
AWSGuard codifies best practices for AWS
Enforce Policy Packs across your organization or particular stacks in the Pulumi Service, for Team and Enterprise users
View policy errors and Policy Packs for an update in the Pulumi Service

Pulumi CrossGuard for Everyone

Pulumi CrossGuard has something to offer everyone in your organization.

For Organization Administrators

You want your organization to be able to move quickly and deliver business value. To do so, your team needs to be able to self-service their infrastructure. Pulumi CrossGuard gives you the peace of mind that you need to allow your team to work independently. Codify business and security policies to ensure your team meets the requirements you set forth.

For Security Engineers

There’s a lot of complexity to security and often the rest of your organization does not have all the knowledge they need to deliver software securely. Express security policies that allow you to share your knowledge and prevent issues from ending up in production.

For Developers and Operators

There are a ton of moving pieces in our ecosystems today. It is difficult to know everything about the cloud resources we use. Pulumi CrossGuard ensures you do not fall into the trap of insecure or unreasonable defaults.

CrossGuard in Action

With an understanding of what CrossGuard enables, let’s look at what that means in practice. The following is a policy pack definition that prohibits creating an AWS S3 bucket that is public readable. Our Policy Pack is not complicated. It uses a validateResourceOfType helper function to filter for the resource type we care about and reports a violation if present.

new PolicyPack("aws-typescript", {
 policies: [{
 name: "s3-no-public-read",
 description: "Prohibits setting the publicRead or publicReadWrite permission on AWS S3 buckets.",
 enforcementLevel: "mandatory",
 validateResource: validateResourceOfType(aws.s3.Bucket, (bucket, args, reportViolation) => {
 if (bucket.acl === "public-read" || bucket.acl === "public-read-write") {
 reportViolation(
 "You cannot set public-read or public-read-write on an S3 bucket. " +
 "Read more about ACLs here: https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html");
 }
 }),
 }],
});

We initially run pulumi preview for an S3 bucket that will pass the policy.

// Create an AWS resource (S3 Bucket)
const bucket = new aws.s3.Bucket("my-bucket");

We then modify the bucket to have its access control list (ACL) set to public-read, which will cause our bucket to be out-of-compliance.

// Create an AWS resource (S3 Bucket)
const bucket = new aws.s3.Bucket("my-bucket", {
 acl: "public-read",
});

While this is a simple demonstration of what you can do with Policy as Code, the framework is very flexible and supports much more complex policies. Our AWSGuard library is a great example of how you can use Policy as Code to write configurable policies.

Try it Today

Pulumi CrossGuard empowers everyone to build better, safer applications and infrastructure. Today, Pulumi CrossGuard is available to preview for all Pulumi users. For Enterprise organizations, administrators can opt-in to the preview via the “Policies” tab in the Pulumi Service. To get started with CrossGuard today, here are some resources:

If haven’t played around with Pulumi yet, here is Pulumi’s Getting Started tutorial.
Policy as Code Getting Started tutorial

We’ve initially released the capability to create policies with TypeScript or JavaScript (that work with Pulumi programs written in any supported language) and plan to add policy SDKs for other supported languages. We would love to hear any feedback you have! You can submit feedback via our Contact Us form or in our community slack.

A Year of Helping Build Production-Ready Kubernetes

Joe Duffy — Thu, 14 Nov 2019 00:00:00 +0000

Today we announced Pulumi Crosswalk for Kubernetes, a collection of open source tools, libraries, and playbooks to help developers and operators work together to bring Kubernetes into their organizations. They capture the lessons we learned this past year working with organizations to go from zero to Kubernetes in production for their infrastructure and application workloads. By releasing these as open source, we hope to help everybody be more successful with their Kubernetes projects — as we have learned through experience, it isn’t easy going!

If you want to dive straight in, browse to the documentation. Below is the backstory and why we’re so excited about this release.

Applying infrastructure as code to Kubernetes

A little over a year ago, we launched support for managing Kubernetes resources using Pulumi’s open source infrastructure as code tools. The entire Kubernetes object model across all supported versions is available in your language of choice — including JavaScript, TypeScript, and Python — in addition to the entire ecosystem of Helm Charts. This offers 100% API resource compatibility, and same-day support for newly released Kubernetes versions.

By leveraging general purpose languages, you get rich capabilities like for loops, functions, and classes, the ability to share and reuse best practices — eliminating copy and paste — as well as access to your favorite tools, including editors, test frameworks, and static analysis tools.

This is exciting for any of us who have struggled with “walls of YAML or JSON” — but this alone isn’t the whole story.

A unified approach to infrastructure as code delivers a robust delivery workflow, and enables managing configuration across not only just Kubernetes resources, but also any of the cluster’s underlying public or private cloud infrastructure, including AWS, Azure, Google Cloud, vSphere, and others.

At the time, we didn’t realize how powerful this unified approach would turn out to be. For instance, properly provisioning an Amazon EKS cluster requires not only orchestrating deployments across AWS, but also going back and forth with Kubernetes resources. We designed Pulumi to let you do this using a single consistent platform and workflow, replacing the “mountains of Bash” that are typically required to integrate multiple tools.

This was an exciting foundation to start from, and we had a number of end users and organizations who soon came to our doors to adopt Kubernetes, with our help.

Why Kubernetes in production is hard

If I had to summarize the challenges that we faced and which we worked to help solve, they are:

Kubernetes infrastructure is not an island. Although AWS, Azure, and GCP offer managed Kubernetes services, most organizations with existing infrastructure maturity find it difficult to provision and manage cluster infrastructure. This is often because going to production requires connecting a cluster to a lot of surrounding infrastructure — including IAM, networking, encryption services, private Docker registries, and more. Kubernetes-specific tools only help with a fraction of this problem and create silos.
Empowering an entire organization is tough. Most organizations adopting Kubernetes are attempting to harmonize the “two sides of the house” – developers and operators. The infrastructure operations team is responsible for managing the cluster and underlying infrastructure, while the developers are trying to deliver applications and services to the cluster (often also using other public cloud services, like object stores). Although the Kubernetes architecture is well-suited to this, teams struggle to standardize on tools, workflows, and practices around both infrastructure and application delivery.
Day Two is even harder than Day One. Many teams decide to go “all in” on Kubernetes … only to find out that securing, scaling, and operationalizing Kubernetes is far more of a full-time job than they imagined. Can you perform zero-downtime upgrades? How do you perform failover between regions? How fast can you promote new container images from dev, to staging, to dozens of production clusters?

Adding to the complexity, these challenges tend to differ depending on which cloud environment you’re targeting. Although Kubernetes has helped to standardize the container compute layer, and what it means to scale out a load balanced service, for example, it does not abstract away all of the infrastructure capabilities between the clouds, like IAM, container registries, and data services.

We experienced this first hand with many customers over the past year. There are three specific areas we’ve worked to develop into turnkey solutions for customers:

Infrastructure playbooks
Making Kubernetes more accessible to developers
Operationalizing delivery and observability insights at-scale

Let me tell you about each of these in turn.

Developing playbooks for cluster infrastructures

Most practitioners are left to discover solutions to the hardest problems with Kubernetes infrastructure on their own. After helping dozens of customers go to production with AWS EKS, Azure AKS, Google Cloud GKE, and on-premises Kubernetes infrastructures, we decided to develop best practices into a set of playbooks. This helps organizations to avoid reinventing the wheel, in addition to mitigating potential pitfalls around security, reliability, and maintainability.

These playbooks cover a lot of ground, including topics like:

Control planes. Each Kubernetes cluster has a control plane comprised of controllers, state management, and other centralized services. Each provider offers a distinct and unique way of creating, deploying, and managing the control plane.
Worker nodes. Each cluster also requires worker nodes to actually run compute. Depending on your compute, storage, and security needs, the provisioning and management of these nodes — often using multiple node pools — differs greatly. Getting this right is essential to attaining a good tradeoff between cost and performance.
Identity. All of the above also needs to integrate with your team’s security policies, including Identity and Access Management (IAM) and Role Based Access Controls (RBAC). If you’re in AWS, you undoubtedly want to connect to AWS IAM; if you’re in Azure, you want to leverage existing ActiveDirectory settings; if you’re in GCP, you’ll want to use GCP IAM; and if you’re using a custom configuration, it can be even more complicated. This is an essential part of deploying Kubernetes within an organization.
Cluster services. Each cluster needs cluster-wide services deployed into the cluster, including performance and monitoring services (AWS CloudWatch, Azure Log Analytics and Monitoring, Datadog, Prometheus, etc.), service meshes, container registries, CRDs, and operators.
Application delivery. Applications need to be Dockerized, packaged up and deployed to private container registries, and rolled out to the cluster, often continuously.
Application services. In addition to the applications themselves, often cloud native applications require additional services to come along for the ride. This includes ingress controllers, DNS and certificate management services, and more. Upgrading clusters. Last but not least, cluster upgrades are an essential part of the overall playbook. Kubernetes is fast-moving, so upgrades are a regular occurrence, and yet it’s not always easy to deploy upgrades with confidence, especially when you are worried about zero downtime and persistent workloads.

This layer cake of concerns can be visualized using the following diagram:

We are confident that anybody looking to create production-ready infrastructure in any of the public clouds, or even on-premises, will benefit from this material. It is also open source so we hope to improve and evolve it in partnership with the community in the weeks to come.

Making Kubernetes more accessible to developers

Besides having to navigate walls of YAML, there is a significant amount of repetition on most idiomatic Kubernetes configurations. In fact, it often feels like hand-authoring Kubernetes YAML is akin to writing software in assembly language, rather than higher level languages.

Pulumi exposes 100% of the Kubernetes API, which means you can use modern language practices to start taming some of this low-level chaos. But new library constructs available in Crosswalk take this to another level by eliminating common boilerplate patterns.

For example, this wall of YAML:

apiVersion: v1
kind: ConfigMap
data:
 config: very important data
metadata:
 name: app-config
---
apiVersion: v1
kind: Secret
data:
 app-password: JikkMz9mK2hIRDo3
 database-password: Y29uZmlnLmRhdGFiYXNlUGFzc3dvcmQ=
metadata:
 name: app-secrets
type: Opaque
---
apiVersion: apps/v1
kind: Deployment
metadata:
 name: nginx
spec:
 replicas: 3
 selector:
 matchLabels:
 app: nginx
 template:
 metadata:
 labels:
 app: nginx
 spec:
 containers:
 - env:
 - name: APP_CONFIG_PATH
 value: /app/config
 - name: APP_USER
 value: config.user
 - name: APP_PASSWORD
 valueFrom:
 secretKeyRef:
 key: app-password
 name: app-secrets
 - name: APP_DATABASE_PASSWORD
 valueFrom:
 secretKeyRef:
 key: database-password
 name: app-secrets
 image: nginx
 imagePullPolicy: Always
 name: nginx
 ports:
 - containerPort: 80
 name: http
 volumeMounts:
 - mountPath: /app/config
 name: app-config-rsg88x4g
 restartPolicy: Always
 volumes:
 - configMap:
 name: app-config
 name: app-config
---
apiVersion: v1
kind: Service
metadata:
 name: nginx
spec:
 ports:
 - name: http
 port: 80
 targetPort: 80
 selector:
 app: nginx
 type: LoadBalancer

Is distilled down to its essence, with far less ceremony and repetition

// Define the application configuration and secrets.
const configs = new kx.ConfigMap("app-config", {
 data: { "config": "very important data" }
});

const secrets = new kx.Secret("app-secrets", {
 stringData: {
 "app-password": new kx.RandomPassword("app-password"),
 "database-password": config.databasePassword
 }
});

// Define the application Pod.
const appConfigPath = "/app/config";
const appPod = new kx.PodBuilder({
 containers: [{
 image: "app:1.0.0",
 env: {
 "APP_CONFIG_PATH": appConfigPath,
 "APP_USER": config.user,
 "APP_PASSWORD": secrets.asEnvValue("app-password"),
 "APP_DATABASE_PASSWORD": secrets.asEnvValue("database-password"),
 },
 volumeMounts: [
 configs.mount(appConfigPath)
 ],
 }],
});

// Create a Kubernetes Deployment using the previous Pod definition.
const deployment = new kx.Deployment("nginx", {
 spec: appPod.asDeploymentSpec({ replicas: 3 }),
});

// Expose the Deployment with a load balancer using a Kubernetes Service.
const service = deployment.createService({
 type: kx.types.ServiceType.LoadBalancer,
});

These library extensions have been designed to live alongside your existing Kubernetes configuration, so that you can gradually reduce complexity, while the result still feels familiar and integrated.

In addition to making Kubernetes configuration more accessible in these ways, there are other benefits to application configuration that we have seen emerge. This includes:

Data Services. For many of us, managing persistent workloads in Kubernetes is not worth the hassle. Although it’s possible to manage the delicate dance of tainting, draining, and migration, many of our customers rely on hosted offerings for data stores — such as Amazon S3, RDS, Azure Cosmos DB or Google Cloud BigTable — and Kubernetes for stateless compute. Using managed services brings the advantage of intrinsic scaling, backups, and management, without the tedium. Pulumi lets you mix and match Kubernetes and public cloud infrastructure as code.
Container Registries. Most end users need to use private container registries to host application images. Furthermore, most of us want to use our cloud provider’s native offering, such as Amazon Elastic Container Registry, Azure Container Registry, or Google Container Registry, because it integrates with IAM. Using Pulumi, you don’t need separate deployment pipelines for containers compared to the supporting application infrastructure.
Serverless. If you want to use the best and most cost effective serverless capabilities, you likely want to use AWS Lambda, Azure Functions, or Google Cloud Functions. We see end users augmenting container-based applications with serverless extensions all the time.
Architecture as Code. Thanks to Pulumi’s use of general-purpose languages for infrastructure as code, end users have been able to abstract and encapsulate complexity into reusable forms, such as functions and classes. This helps you go beyond even the basic Kubernetes configuration challenges that Kx solves, to codifying recurring patterns in your cloud native architectures.

In addition to all of that, we’ve been hard at work on a new experimental “watch mode” feature which, in conjunction with the above, brings all of Kubernetes to your fingertips in a radically new way. We’re excited to show this off at our booth at KubeCon next week!

Operationalizing delivery at-scale

After you’ve mastered the infrastructure and application authoring challenges, the difficulties move to Day Two and beyond. At this stage, organizations need to figure out how to deliver changes continuously to an ever-increasing number of environments.

Since our initial launch last year, we have added many different CI/CD integrations, including support for GitLab, Codefresh, Azure DevOps, Octopus Deploy, GitHub Actions, and more. This enables delivery at global scale. In fact, one of our customers continuously deploys to 80 different environments across their development and infrastructure operations teams.

Finally, we are excited to release a new technology in preview today, a new query command, and library of Kubernetes queries, that leverages the Pulumi object model to ask operational queries about your cluster and its applications. This includes queries like “How many distinct versions of MySQL are running in my cluster?”, “Which Pods are exposed to the Internet via a load-balanced Service?”, and so on.

For instance, this query shows the distinct versions of MySQL inside your cluster:

import * as kq from "@pulumi/query-kubernetes";

// Find all distinct versions of MySQL running in your cluster.
const mySqlVersions = kq
 .list("v1", "Pod")
 .flatMap(pod => pod.spec.containers)
 .map(container => container.image)
 .filter(imageName => imageName.includes("mysql"))
 .distinct();

mySqlVersions.forEach(console.log);

CQL also supports live streaming queries, which the CLI will display results from in real time. Read more and get started with Pulumi Query for Kubernetes here.

Join a growing community

Everything above is open source and has been battle-tested with many end users and organizations over the last year — all the way from startups to ISVs to the largest enterprises in the world. We are excited to share these best practices in reusable forms, leveraging the foundation uniquely provided by Pulumi’s infrastructure as code platform using familiar languages.

To get started, check out the documentation, or get started with the open source SDK now.

We sincerely hope that your team can be successful in your Kubernetes journey as a result of these lessons learned from others. Additionally, Pulumi offers training and services, so if we can help in any way, please join our entire team and thousands of practitioners on our Community Slack — or simply contact us and we’ll be in touch.

Pulumi 💜 .NET Core

Joe Duffy — Mon, 11 Nov 2019 00:00:00 +0000

Today we are excited to announce the Preview of .NET Core support for all of your modern infrastructure as code needs. This means you can create, deploy, and manage your infrastructure, on any cloud, using your favorite .NET language, including C#, F#, and VB.NET.

This is a heartwarming milestone for the entire Pulumi team. It’s no secret that several of our founding members had a hand in shaping the early .NET project over a decade ago. In fact, before leaving Microsoft to found Pulumi, I had the privilege of helping to kick off the .NET Core effort to take .NET open source and add macOS and Linux support alongside Windows. It’s truly wonderful to build on top of this amazing and vibrant ecosystem, and reconnect with old friends!

Please head on over to the .NET blog to read more, or check out our new .NET Core landing page if you can’t wait to dive in.

True to the spirit of Pulumi, our .NET support and the associated libraries are open source on GitHub. We are releasing .NET support in preview first so we can gather feedback over the next few weeks and make sure everything is just right. Join us in our Community Slack — we can’t wait to hear what you think!

Pulumi 1.0

Joe Duffy — Thu, 05 Sep 2019 00:00:00 +0000

This blog post announces Pulumi 1.0, which has been superseded by newer versions. For information about the latest Pulumi release, please refer to the current documentation and recent release notes.

Today we are excited to announce the general availability of Pulumi 1.0. Pulumi is a modern infrastructure as code tool that lets you declare infrastructure using familiar, general-purpose languages, with a SaaS management console for configuring identities, organizations, and related policies. By using familiar languages, developers and operators are able to work better together, sharing and reusing best practices, accomplishing new levels of automation, and unlocking access to ecosystems of existing tools. The 1.0 release is a significant milestone for us, our community, and our customers, and signals completeness, stability, and compatibility.

What is Pulumi?

Using Pulumi, you:

Declare infrastructure in TypeScript, JavaScript, Python, or Go.
Leverage tools such as libraries, package managers, IDEs, and test frameworks from your language of choice.
Access all of the services from many clouds, including AWS, Azure, GCP, Kubernetes, and DigitalOcean.
Use the Pulumi CLI or CI/CD integrations to perform deployments predictably and reliably.
Build modern, reliable, and scalable applications using a multitude of cloud architectures, including containers, serverless functions, and static websites.
Use higher level frameworks that abstract away tedious aspects of cloud development.

This functionality is delivered through an open source SDK consisting of a CLI and assorted libraries. Pulumi also provides a hosted service for improved usability, reliability, security, and other management features.

Additional Features

In addition to the core infrastructure as code features, Pulumi’s SDK offers:

Stack management: deploy to and promote between multiple environments.
State storage: use the free Pulumi service or choose instead to manually manage state in AWS S3, Azure Storage, Google Cloud Storage, or the filesystem.
Configuration: easily manage different configuration between environments.
Encrypted secrets: use the free Pulumi KMS service to encrypt sensitive configuration, such as passwords or tokens, or plug in AWS KMS, Azure KeyVault, GCP KMS, or HashiCorp Vault.

Using Pulumi in conjunction with the Pulumi service delivers the following additional features:

Pulumi Service: a dashboard for gaining visibility into and managing your deployments.
Identity: user and organization identity provider integrations with GitHub, GitLab, Atlassian, or SAML/SSO, including Active Directory, Okta, Google G Suite, and others.
Role based access controls: team management, also integrated with your identity provider.
REST API and Webhooks: a fully programmable web API for advanced automation scenarios.
CI/CD Integrations: automated deployment integrations with Azure DevOps, AWS Code Services, CircleCI, Codefresh, GitHub, GitLab, Google Cloud Build, Jenkins, Travis, and others.

The Pulumi Service is available as a free Individual Edition as well as advanced Team and Enterprise SaaS editions for larger teams looking for robust enterprise capabilities.

Why Pulumi?

Using familiar languages offers significant advantages for declaring your infrastructure:

Control flow: use if statements, for loops, case statements, etc.
Abstractions and reuse: leverage classes and functions to avoid repeating the same boilerplate over and over again
Package managers: share libraries with your team or the community
IDEs: use your favorite IDE and get statement completion, live error checking, and interactive documentation
Project organization: use modules and well known techniques for managing complex projects
Testing: apply standard test tools, frameworks, and techniques to infrastructure
Code review: use existing code review practices for infrastructure changes

This approach unlocks access to entire ecosystems that aren’t available to markup DSLs. This approach also helps developers and operators work better together using a shared foundation.

How Pulumi Works

Although Pulumi uses general-purpose imperative languages, it is declarative. The program simply declares a set of resources and the Pulumi engine orchestrates all CRUD operations.

Packages in your language of choice provide full access to many clouds, including AWS, Azure, GCP, Kubernetes, DigitalOcean, and more. Each package projects the full set of services from the underlying cloud provider as classes and APIs. Declaring a new S3 bucket is as easy as:

let b = new aws.s3.Bucket("my-bucket", { acl: "private" });

Because of higher level components, even creating a new Kubernetes cluster is this easy:

let cluster = new eks.Cluster("my-cluster");
export const kubeconfig = cluster.kubeconfig;

The pulumi up CLI command runs this program using Pulumi’s multi-language engine, and presents a plan for review — including an overview of the resources to create, update, and/or delete. After you choose to apply those changes, the Pulumi engine coordinates those API calls in parallel to accomplish your deployment.

To see this workflow in action, watch this short 5 minute video:

How to Get It

The Pulumi 1.0 milestone signals completeness, stability, and compatibility, and is the result of two years of work helping 1,000s of customers take modern cloud architectures across many clouds into production. This includes startups and Global 2000s looking to innovate faster using the cloud.

If you’re already a Pulumi user, simply upgrade in the usual ways.

If this is your first time using Pulumi, give it a try today. It is open source and free to use.

For more information about Pulumi Team and Enterprise editions, visit our products page.

We’d like to sincerely thank our community and customers for coming along on this journey with us. Without you, we couldn’t have reached this milestone. We hope you enjoy Pulumi 1.0!

Happy Birthday to Pulumi open source!

Joe Duffy — Tue, 18 Jun 2019 00:00:00 +0000

One year ago today – on June 18, 2018 – we open sourced Pulumi, a new approach to multi-cloud infrastructure as code using your favorite languages. And what a year it has been!

Here are some highlights we’ve added in partnership with the community since launching:

Over 100 examples, tutorials, and a brand new Getting Started guide.
A native Kubernetes provider with 100% Kubernetes API/version coverage.
A steady stream of improvements across AWS, Azure, and Google Cloud providers.
Pulumi Crosswalk for AWS, a framework with built-in AWS infrastructure best practices.
Over 20 additional providers, including CloudFlare, Digital Ocean, and MySQL.
Brought our Python 3 SDK to parity with our Node.js-based JavaScript and TypeScript SDKs.
Team and Enterprise SaaS editions for teams managing infrastructure in production.
GitHub, GitLab, Atlassian, and SAML/SSO identity providers.
CI/CD integrations with GitHub, GitLab, Codefresh, CircleCI, major clouds, and more.
Pluggable secrets management and transitive state encryption.
Pluggable state backends for AWS S3, Azure Blob Store, and Google Cloud Store.
Tools for managing complex, multi-stack environments, including Terraform integration.
Numerous engine reliability and performance improvements, including parallelism.
75 blogs, increasingly focused on end to end solutions we see working with customers.

In addition to the steady stream of product improvements, the community has grown fast:

1,000s of cloud engineers trying out Pulumi across over 1,000 companies.
1,000 end users collaborating in our Community Slack – including emerging MVPs!
Open source contributions from over 100 community members.
Nearly 5,000 commits, across over 100 open source Git repositories.

What we’ve enjoyed most is hearing about the innovation you, the end user and customer, have accomplished thanks to Pulumi’s unique approach. These experiences range from enabling better collaboration between application and infrastructure engineers; vastly improved productivity; more secure, robust, and reliable delivery pipelines; and an improved ability to go to production in multiple clouds, including by leveraging the cloud native Kubernetes stack.

A few notable success stories include:

Mercedes-Benz R&D North America used Python infrastructure as code to break down team silos between developers and infrastructure engineers.
Tableau was able to get up and running with Kubernetes in AWS, using Amazon EKS, and make progress on their journey to bringing continuous delivery to their organization.
Learning Machine went from 25KLOC YAML to 500 lines of JavaScript, got containers running in AWS and Kubernetes, and halved their cloud bill along the way. Read more.
Mapbox built a scalable IoT tracking solution with serverless Lambdas. Read more.
Multiple early stage startups, including Jargon, a TechStars company, have used Pulumi to get up and running quickly in the cloud – doing in days what used to take weeks.
Dozens of more great stories that we can’t wait to share in the weeks to come!

We want to sincerely thank our community and our customers and partners – without you, the year wouldn’t have nearly been as successful. The best part is that we’re only just getting started. Expect another year of more innovation, growing community engagement, customer success, and more focus on end to end solutions borne out of experience.

Whether it’s adopting infrastructure as code on your favorite cloud, deploying containers to Kubernetes, or architecting and executing on your multi-cloud journey, Pulumi is here for you!

Sincerely,

Joe and the Pulumi Team

Introducing Pulumi Crosswalk for AWS: The Easiest Way to AWS

Luke Hoban — Mon, 10 Jun 2019 00:00:00 +0000

Some of the code in this post is out of date. See the AWS guides for an updated overview and examples.

Amazon Web Services provides an incredible platform for developers to build cloud-native applications, and is used by millions of customers of all sizes. The building block services that AWS offers enable teams to offload undifferentiated heavy-lifting to AWS. To maximally benefit from these services though, cloud engineering teams must learn how to compose all of these building blocks together to build and deliver their own applications. Today, this is still too hard. Getting from your laptop to a production-ready AWS deployment frequently takes days or weeks instead of minutes or hours. And AWS building block services frequently leave you to re-implement (and re-discover) best-practices instead of providing these as smart defaults.

Pulumi Crosswalk for AWS is a new open source library of infrastructure-as-code components that make it easier to get from zero to production on AWS, easier to adopt AWS best practices by default, and easier to evolve your AWS infrastructure as your application needs mature.

Crosswalk for AWS offers higher-level components for a wide range of the AWS platform - from serverless (Lambda, API Gateway and more) to containers (ECS, Fargate) to Kubernetes (EKS) and across networking (VPCs, SecurityGroups), monitoring (Cloudwatch Dashboards and Alarms) and load balancing (ALB and NLB). Because Crosswalk builds on the primitive building blocks of the AWS platform, you can use 100% of what AWS offers, and can seamlessly move between high-level components and low-level platform primitives, mixing-and-matching as needed.

In some of these domains, there have been successful tools that have helped developers more easily work with one specific aspect of AWS - Serverless Framework for serverless, Docker Compose for ECS, eksctl for EKS. But these tools have traditionally been limited to one specific kind of compute, limiting what users can accomplish, fragmenting development of complete cloud solutions, and leading to sharp cliffs when application needs evolve. Crosswalk for AWS offers the same kind of simpler and more productive developer interface as these tools, but applied uniformly across the entire AWS platform. Simple things are simple, and arbitrarily complex things are possible.

Whether you are starting a new SaaS project at a startup, migrating an enterprise application to cloud-native infrastructure, or delivering a new service as part of an established cloud application, Crosswalk for AWS provides the easiest way to get from zero to production, and to then evolve with your projects’ needs. You can get started with Crosswalk for AWS today!

Note: Crosswalk for AWS projects are authored using the Pulumi infrastructure-as-code tools. Pulumi allows you to define your infrastructure using code. As your infrastructure code evolves, Pulumi makes the minimal required changes to your cloud resources, ensuring that your infrastructure is updated without downtime (if possible).

From Zero to Production And Beyond with Crosswalk for AWS

Serverless (Lambda, API Gateway)

When getting started with a new cloud project, the starting point is often just serving a simple API on the internet. With just a few lines of code and tens of seconds to deploy, Crosswalk for AWS gives us our API.

import * as awsx from "@pulumi/awsx";

const api = new awsx.apigateway.API("api", {
 routes: [{
 method: "GET",
 path: "/",
 eventHandler: async (ev, ctx) => {
 return {
 statusCode: 200,
 body: JSON.stringify(ev),
 };
 }
 }],
})

export const endpoint = api.url;

By building on top of great AWS building blocks like Lambda, API Gateway, IAM and more, we avoid needing to worry about infrastructure, pay nearly zero fixed costs, and gain the ability to iterate quickly. With just a few more lines of code, we can wire up to our own domain, add authorization, provision a database, or chain together a more complex event-driven application.

Containers (ECS, Fargate)

As our needs grow, we might find we need to deploy containers - either custom applications or existing Docker images. Again, just a few lines of code gets our container running in production in AWS.

import * as awsx from "@pulumi/awsx";

// Create the cluster
const cluster = new awsx.ecs.Cluster("cluster");

// Configure a Load Balancer
const alb = new awsx.lb.ApplicationLoadBalancer("lb", {
 securityGroups: cluster.securityGroups,
});
const web = alb.createListener("web", { port: 80, external: true });

// Define the scaled-out Fargate service
const appService = new awsx.ecs.FargateService("app-svc", {
 cluster,
 taskDefinitionArgs: {
 container: {
 image: awsx.ecs.Image.fromPath("app", "./app"),
 portMappings: [ web ],
 },
 },
 desiredCount: 5,
});

export const url = web.endpoint.hostname;

Using ECS, Fargate, ECR and ELB, we get a robust production-ready container deployment - horizontally scaled out, load-balanced, and integrated with a private image repository. With just a few more lines of code, we can add autoscaling, customize our ECS cluster, or wire through advanced container configuration (volumes, environment variables, and more).

Networking (VPC)

To adopt best practices security posture, we may want to move our cluster into a custom VPC, and run our containers in private subnets. We can define and configure our VPC, and then redeploy our containers into this new VPC with ease:

import * as awsx from "@pulumi/awsx";

const vpc = new awsx.ec2.Vpc("vpc", {
 numberOfAvailabilityZones: 3,
});

const cluster = new awsx.ecs.Cluster("cluster", {
 vpc: vpc,
});

// ... same as before ...

The Vpc component builds on best-practices Virtual Private Cloud design patterns based on AWS guidance and documentation. By building in simple defaults for routing, subnet structure, NATs and multi-AZ you can get started quickly without having to re-discover these best practices. And as your needs grow, you can deeply customize and evolve your VPC structure - defining custom CIDR blocks, customizing Internet and NAT Gateways or additional additional private subnets.

Kubernetes (EKS)

As our project continues to evolve, we may want to move our container workloads into Kubernetes using EKS. We can stand up an EKS cluster with just a few lines of code, then customize our cluster and deploy Kubernetes workloads into our cluster.

import * as awsx from "@pulumi/awsx";
import * as eks from "@pulumi/eks";

// Create a VPC for our cluster.
const vpc = new awsx.ec2.Vpc("vpc");

// Create the EKS cluster itself.
const cluster = new eks.Cluster("cluster", {
 vpcId: vpc.id,
 subnetIds: vpc.privateSubnetIds,
 instanceType: "m5.large",
 desiredCapacity: 2,
 minSize: 1,
 maxSize: 2,
 deployDashboard: true,
});

// Export the cluster's kubeconfig.
export const kubeconfig = cluster.kubeconfig;

Building on EKS we get all the benefits of managed Kubernetes paired with the platform capabilities of AWS. With Crosswalk for AWS, we can get started quickly, and then evolve to take advantage of all of these platform features, like customized node groups, private networking, and even deploying Kubernetes YAML and Helm charts to our cluster.

Monitoring (CloudWatch)

Getting our infrastructure up and running in AWS is great, but we also want to be able to monitor it. We can add a few lines to any of the solutions above to define metrics, dashboards, alerting and more to monitor our AWS infrastructure.

import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";

// Create a bucket and a function to log all new object created events
const bucket = new aws.s3.Bucket("b");
const subscription = bucket.onObjectCreated("newObject", async (ev) => {
 console.log(JSON.stringify(ev));
});

// Create a CloudWatch Dashobard for our functions invocations
const dashboardName = "funcDashboard";
const dashboard = new awsx.cloudwatch.Dashboard(dashboardName, {
 widgets: [
 new awsx.cloudwatch.LineGraphMetricWidget({
 title: "Lambda invocations",
 width: 14,
 metrics: awsx.lambda.metrics.invocations({
 function: subscription.func, statistic: "Sum", period: 60, }), }), ], });

// Export the URL of the dashboard in the AWS console
export const dashboardUrl =
 `https://${aws.config.region}.console.aws.amazon.com/cloudwatch/home?` +
 `region=${aws.config.region}#dashboards:name=${dashboardName}`;

With CloudWatch being deeply integrated into all AWS services, we can easily build up robust logging, alerting and dashboarding solutions across our AWS infrastructure. With just a few more lines of code we can get logs for our functions and containers, create alarms when we cross critical thresholds, and create rich dashboard layouts all of which will version along with our infrastructure!

Early Feedback

We’ve been working on pieces of Crosswalk for AWS since before we launched Pulumi, and we’re excited to be launching Crosswalk for AWS today to bring these together for all AWS users. Over the last few months, users have built on Crosswalk for AWS for all of the scenarios above and more.

Tableau has used Crosswalk for AWS to accelerate their Kubernetes migration on AWS, building on the networking and EKS capabilities of Crosswalk for AWS.

“We’ve been happily using Pulumi’s EKS support for more than three months now. Our team was looking for an end-to-end solution to tame the complexity of Kubernetes on AWS and ensure we adhere to AWS best practices. Pulumi’s Crosswalk effort has equipped our team to scale far better, our delivery is now automated, and we can now deliver new application and infrastructure features with much faster turn-around, sometimes in just a few hours. Pulumi is a key contributor to our team’s improvement in productivity.” - Pankaj Dhingra, Sr Director, Cloud Engineering, Tableau

Compute Software has used Crosswalk for AWS to go from zero to production, and then to evolve their infrastructure in AWS, all without needing a dedicated cloud infrastructure engineering team.

“We use Crosswalk for AWS with Fargate. It’s been instrumental in getting us into production. Deploying to the cloud is necessarily complex. Crosswalk streamlines the process by exposing the controls you care about, always allowing you to dive deeper when needed.” - Kenny Williams, Co-Founder, Compute Software

We’re excited by the success these teams and many more have already had with Crosswalk for AWS, but even more excited about the potential to truly democratize access to the AWS platform and unlock highly productive cloud development for all kinds of engineering organizations.

What’s Next?

We’re still just scratching the surface of what’s possible with Crosswalk. We’re continuing to evolve the open source projects to simplify additional features of the AWS platform. And we’re also working to bring the same approach to other cloud platforms, including Azure, GCP and Kubernetes.

We believe that making it easier for cloud engineers to develop and deploy natively in the cloud will unlock creativity and empower developers to take advantage of more of the rich capabilities of AWS. We can’t wait to see what you build with Crosswalk!

Crosswalk for AWS is free and open source, and you can get started today.

For more on Crosswalk for AWS see:

Announcing Support for Email-based Identities

Praneet Loke — Thu, 21 Mar 2019 00:00:00 +0000

We have been hard at work the past few months providing our users with more ways to connect to Pulumi. Here are some our past announcements related to identities:

Today, we are pleased to announce that we are launching support for email-based identities. You no longer need to use a social identity to sign-up for an account on Pulumi. Just fill out the signup form, and you are ready to go.

Quickstart

Get started quickly by creating a new template-based project. This creates a project, and a stack ready to be used by the Pulumi CLI. The wizard also shows you the commands you need to run in the CLI.

Haven’t setup your CLI for a specific cloud yet? Check out this docs page on how to do that.

Once you’ve deployed your first stack, the next step is up to you. You can refer to our documentation on integrating Pulumi into your CI/CD pipeline, setting up chat-ops workflows via Webhooks, or taking your Kubernetes solutions to the next level!

We’d love to hear what you’re up to. Say 👋 in the Pulumi Community Slack or drop us a line.

2018 Year at a Glance

Joe Duffy — Mon, 31 Dec 2018 00:00:00 +0000

As we close out 2018, and enter into a New Year, I was reflecting on our progress here at Pulumi this past year and wanted to share some thoughts. It’s been an incredible year and we are hugely thankful to our passionate community, customers, and partners.

Here are some of the exciting things that happened:

Launching our open source community. After being hard at work for a little over a year, we launched our open source project, with the aim of making it considerably easier and more enjoyable to create and operate cloud software. This was a major moment for us. We had previously only shown Pulumi to a few select friends, family, and private beta users, and the reception was beyond our wildest expectations. We got passionate +1’s from cloud engineers of all roles and responsibilities – developers, infrastructure engineers, operators, and managers alike. The Pulumi community has since grown to thousands of users across hundreds of companies of all sizes spread across the globe. The community is now a warm, welcoming, and collaborative place with significant open source contributions and passionate community leaders emerging who are helping one another while also helping us to make Pulumi even better.

Support for major public, private, and hybrid clouds. We launched with support for the three major public clouds, AWS, Azure, and Google Cloud, and have since added capabilities across all of them, including adding serverless, containers, and infrastructure productivity libraries – it is here where we believe Pulumi’s unique approach of using general purpose languages truly shines. We immediately had a wave of inbound interest in applying the Pulumi approach to infrastructure as code in other areas, and quickly added other providers including OpenStack, VMWare vSphere, Alibaba Cloud, F5 BigIP, and more. In September, we unveiled a native Kubernetes provider with support for the entire API surface area across all versions, plus an operations tool, KubeSpy – both of which continue to lead to significant community growth. Many customers are loving that they can achieve a consistent engineering workflow across multi-cloud environments – often spanning public, private, and/or hybrid clouds.

Launching our commercial SaaS product. In October, we launched our commercial SaaS product, with reasonable pricing so that teams of all sizes are able to get their code to the cloud productively, securely, and collaboratively. If Pulumi is like Git – a local CLI and SDK – then the Pulumi SaaS is like GitHub – a hosted service to ensure you can use Pulumi in a team setting. In addition to the Team Edition, which is meant for the “Fortune 500,000,” we also launched the Enterprise Edition, for the Fortune 500. We already have multiple Fortune 500 customers moving into production with Pulumi, and expect this to be a significant area of future growth. Since launching, we’ve added support for more identity providers – including GitHub, GitLab, and SSO/SAML (for Okta, ADFs, and others) – as well as team management capabilities, such as RBAC, our GitHub App, GitHub Actions partnership, and more.

Closing our Series A funding. Also in October, we secured $15MM in additional funds, which will enable us to scale to meet the incredible opportunity in front of us. The funds will ensure that our open source projects are constantly improving and achieving production quality; that our commercial product can meet the most rigorous business requirements, small or large; and that we can grow our organization in areas necessary to meet our customers’ needs. When we set out to make it 10x easier for teams to do cloud engineering, we saw many facets in need of attention, and these funds will help us take concrete steps towards realizing the vision.

What an incredible year! It can still hardly believe it’s been just six months since launching.

As we look to 2019, we plan to continue engaging with and growing our open source community, built on the same tenets of openness and collaboration – the best part of my job. We have heard loud and clear that what customers need today are solutions, not just raw technologies, and we will continue to deliver commercial value in our Team and Enterprise Edition SaaS products that cloud engineering easier, more secure, and more collaborative.

We are just getting started and we can’t wait to continue on the journey of reimagining cloud engineering together. Thank you for your passionate support – and here’s to a wonderful 2019!

— Joe

Delivering Cloud Native Infrastructure as Code

Marc Holmes — Wed, 05 Dec 2018 00:00:00 +0000

Enterprise software has undergone a slow shift from containerless servers to serverless containers.

The evolution of the cloud, combined with the shift to increasingly ephemeral infrastructure, and the connection of application code and infrastructure code, demands a different view of cloud development and devops.

To a first approximation, all developers are cloud developers, all applications are cloud native, and all operations are cloud-first. Yet, there is a lack of a consistent approach to delivering cloud native applications and infrastructure. The tools and processes differ by technology generation, and even by cloud vendor, and so deny the full potential of cloud native application delivery.

In our latest white paper, Delivering Cloud Native Infrastructure as Code, we make the case for a consistent programming model for the cloud and examine:

How the cloud has already evolved three times as it increasingly moves toward stateless compute to deliver on the opportunities afforded by unprecedented economies of scope and scale.
How stateless compute has shifted infrastructure management concerns from ‘at rest’ to ‘in motion’, and moved these concerns up the stack to development.
How the growth of DSL-based tools has lead to complexity for DevOps teams, failed to deliver on the promised collaboration between development and operations functions, and does not satisfy the need for increasing delivery speed in the cloud.

You can read the paper in full here.

Let us know what you think!

Meet the Pulumi team at AWS re:Invent

Marc Holmes — Thu, 15 Nov 2018 00:00:00 +0000

Heading to AWS re:Invent? Concerned about how you’ll manage to get that much YAML into your carry on bag? Or maybe you just like purple.

Whatever the reason, the Pulumi team will be there all week at **Booth 316, Startup Central, Aria Quad, **and we’d love to chat with you about AWS and Pulumi.

Catch up with us on serverless functions, containers and Kubernetes, managed services and any other cloud native infrastructure as code, and see how you can more productively manage your AWS cloud resources with general purpose programming languages. We can even help you migrate your CloudFormation to Pulumi.

If you want to grab a specific time to talk through your needs, then use this link, otherwise we’ll just see you at the booth!

Building a future of cloud engineering

Joe Duffy — Mon, 22 Oct 2018 00:00:00 +0000

We founded Pulumi because of a deeply held belief that the cloud promises to change all aspects of software development and that there remains an incredible opportunity to reimagine the entire experience, from idea to creation to delivery to management, with one person in mind: you, the engineer.

Too often, we still think of the cloud as an afterthought, as though it were still 1998 with virtual machines and XML configuration. At Pulumi, we believe instead that the cloud is a first class application architecture concern. We still use thankless tools, markup languages, and brittle, homegrown delivery platforms. At Pulumi, we believe instead in a lovable and productive development experience, with excellent collaboration and reuse.

Since getting started just over a year and a half ago, we’ve stood on the shoulders of giants, tapping into decades of progress our industry has made in programming languages, runtimes, and frameworks, and marrying it with all the amazing innovation in cloud containers, orchestrators, managed data and AI services, and serverless capabilities.

The best part of this journey has been hearing from our passionate community. Thousands of engineers, and hundreds of companies, have created and deployed cloud applications and infrastructure using Pulumi. We are proud to be an open source company, and thrive on the daily collaboration and evolution of the platform.

And we’re just getting started! Today we are thrilled to announce that we’ve raised $15M in additional funding from our partners, Madrona Venture Group and Tola Capital, both here in Seattle. We are honored by their shared belief in our vision and our team. Read more in our press release.

How We Got Here, a Tale of Cloud Evolution

We have seen stunning advancement in cloud platform capabilities. It’s hard to believe that Docker was launched just five years, and AWS Lambda only three, ago! And yet, these technologies change all aspects of how we create, deliver, and manage cloud software. Meanwhile, the cloud has become more programmable with APIs everywhere.

At the same time, “DevOps” as an approach to organizing teams and developing software has matured greatly, and is now the default for many organizations. New practices such as infrastructure-as-code, immutable infrastructure, and Site Reliability Engineering have modernized these roles, giving us approaches that can leverage a more rigorous software engineering practices to tasks that used to be more ad-hoc, manual, and error prone.

But despite these fundamental changes, the way we engineer the cloud hasn’t fundamentally changed. As cloud engineers, we still think of configuration as an afterthought and a task best suited for JSON, YAML, or DSLs. Each individual resource we provision – even fine-grained ones like serverless functions – requires often an equal amount of configuration as there is actual application code. The lack of expressiveness has led us to resort to bolting on templating systems, writing mountains of bash, and overall find ourselves swimming in a sea of complexity.

The problem is, these are crude approximations of familiar, general-purpose languages. They lack real abstraction, sharing, and reuse. You can’t fire up your editor and refactor, test, or analyze your code to find bugs ahead of time. Instead, you find simple syntax errors when it’s too late, after production is on the floor. Developers hate to touch those configuration languages, and yet simultaneously operators want to be more productive and empower their teams to collaborate.

It’s safe to conclude: we have hit the limits of the “afterthought” approach.

Towards Cloud Engineering

We believe in a future where developers and operators alike can meet on common ground to work together. We like to call this combined discipline cloud engineering. It’s not Dev, it’s not DevOps, and it’s not SRE; cloud engineering is the best of all of them. In this new future, our teams will experience more joy, collaboration, and productivity than ever before.

Enter Pulumi.

Using your favorite general purpose programming language – whether it is JavaScript, TypeScript, Python, Go, or something else – you can express the entirety of your cloud software. As a cloud engineer, you can develop it in your favorite editor, with statement completion, built-in documentation, and interactive feedback when you’ve got something wrong. You can test things and benefit from static analysis and linters that enforce best practices. You can reuse packages, publish your own, and refactor your code just like with application code. In fact, after living in this world for long enough, the boundary between infrastructure and application code begins to blur.

This has led to a phrase we use a lot at Pulumi – program the cloud – our rally cry, if you will.

Just like code, we can collaborate with source control systems like Git – code reviewing changes, pushing and pulling code to perform deployments – using easy but robust continuous delivery workflows. Thanks to familiar languages, a cloud object model, and programmable cloud APIs, the system has true semantic understanding of your cloud software, its resources and their relationships, which can be used for rich deployment statuses and to organize and find information more easily. It’s never been such a breeze to go from idea to production.

We’ve become enamored by this new approach to creating software that is truly born in the cloud. But it’s not just our own belief – this new age of cloud engineering is working terrifically in practice. We have worked with organizations of all sizes – small ISVs to Fortune 500 Enterprises – to vastly improve their team efficiency and velocity. We almost always reduce configuration sprawl by an order of magnitude thanks to Pulumi, leading to a level of productivity that these teams previously hadn’t imagined was possible. Developers, operators, and managers alike are loving living in this future of cloud engineering.

Pulumi’s Next Chapter

We’ve only just begun. Pulumi supports AWS, Azure, Google Cloud Platform, and Kubernetes in a big way, with early support for VMWare vSphere and OpenStack. Our delivery platform works great with your favorite CI systems. Expect to see continued growth in all of these areas, in addition to new clouds, languages, and integrations.

Today we launched the Team Edition for our SaaS delivery platform. This includes many features that you, the cloud engineering community, told us you need to better operationalize Pulumi within your teams. We already have many happy customers using these features successfully for organizations big and small. Expect to see a steady stream of improvements, in addition to Enterprise Edition capabilities for the largest of organizations needing on-premises, advanced security, and custom identity. Please always let us know how we can better meet your needs.

This new round of funding will allow us to strengthen our commitment to open source and community, increase our R&D velocity, and also scale our business to enable organizations of all sizes to successfully achieve cloud engineering within their teams. We are hiring in all areas of the company as we enter this next exciting chapter of the company.

Download Pulumi – we can’t wait to see all the great things you build next with Pulumi!

Joe & Eric

Continuous Delivery to Any Cloud using GitHub Actions

Joe Duffy — Wed, 17 Oct 2018 00:00:00 +0000

Today we announced our partnership with GitHub on the new GitHub Actions feature. We are super excited about this bold and innovative technology, especially as it relates to Pulumi, and CI/CD more broadly. We truly believe that Pulumi plus GitHub Actions delivers the easiest, most capable, and friction-free way to achieve continuous delivery of cloud applications and infrastructure, no matter your cloud – AWS, Azure, Google Cloud, Kubernetes, or even on-premises. In this post, we’ll dig deeper to see why, and how to get up and running. It’s refreshingly easy!

GitHub Actions + Pulumi == ❤️

Pulumi lets you write cloud applications and infrastructure in your favorite language. This includes containers, serverless, and even VM-based infrastructure. Combined with GitHub Actions, we can get continuous deployment to any cloud – AWS, Azure, Google Cloud, Kubernetes, or even on-premises – all straight from GitHub.

GitHub Actions gives us a way to run compute in GitHub’s cloud, triggered by any GitHub event – commits, pull requests, and more. This enables us to run Pulumi in response to these events, giving us the benefits of infrastructure as code seamlessly integrated into a lovable workflow. We can edit any files – application- or infrastructure-related – to do end-to-end continuous delivery, without any of the YAML, tedious bash scripting, or manual, error-prone steps.

This capability enables some exciting scenarios:

Deploy cloud changes anytime a commit is made, including
- Provisioning entirely new cloud environments
- Updating existing infrastructure, like scaling things up or modifying config
- Rebuilding and redeploying app containers to your cluster
Preview deployments before they’re actually made, integrated with Pull Requests
Use Git branches to achieve GitOps-style promotion from test/staging to production

To see things in action, check out the video shown at the GitHub Universe keynote today:

In this short video, we see a Ruby on Rails application that is continuously deployed to our Kubernetes cluster (full code available here). It’s just a standard Rails application, with a Dockerfile, and a complete Pulumi infrastructure specification, which are just a collection of TypeScript modules. (We could have used Python or Go.) This includes a fully functioning Google Kubernetes cluster, hosted PostgreSQL database (for easier management!), Kubernetes definitions, and automatic building and publishing of our Rails Docker image to the Docker Hub, which then gets auto-deployed to our cluster. And it works across three entirely separate testing, staging, and production environments!

Notice that it’s all ordinary code, so that it works great with GitHub, in addition to our favorite IDEs and tools.

After committing our changes, Pulumi takes it from there. Deployments can be previewed, diffed, and are recorded so that you’ll always know who changed what, when, and why – all very “Git-like.” Pulumi’s GitHub App adds to this and enables “GitOps” so that teams can propose, approve, and promote code from “staging” to “production” using pull requests (more on that below).

We’ve chosen GKE here as an illustration because it’s amazing how powerful such a simple example can be, but this works just as well if you’re using AKS, EKS, on-premises clusters – and of course works for non-Kubernetes scenarios, like VMs, ECS, and Serverless applications.

All of this with just a handful of TypeScript and GitHub Actions set up to work with Pulumi – despite being quite capable, it’s super easy to get all of this up and running. Let’s see how!

Getting Up and Running

The full sequence of steps is available in our GitHub Actions Getting Started Guide.

In summary, using GitHub Actions with Pulumi is as easy as signing up for Pulumi (if you haven’t already), creating one or more workflow files in your repo (see here for examples), configuring secrets, and adding a Pulumi program. Afterwards, you can simply Commit and Pull Request away to deploy All The Things! GitHub Actions will fire and run Pulumi tasks appropriately; you’ll know it’s working if you see Pulumi actions show up alongside your PRs and commits:

Two sorts of actions, defined the examples referenced above, are

Pulumi Preview runs pulumi preview in response to a Pull Request, showing what would happen if the PR were merged into the target branch.
Pulumi Up runs pulumi up on the target branch, in response to a commit on that branch.

If either of these fail, you’ll see the standard GitHub Checks red “x”, whereas success will give a green “checkmark”; click on either to get the full Pulumi log output:

GitHub Actions’ Logs page, plus Pulumi’s Console, together give you total insight into a deployment’s status, including the Docker build logs, Kubernetes status updates, and more – so you’ll always have the full scoop on what’s going on, good or bad, in your cloud deployments.

Pulumi’s GitHub App

GitHub Actions work with Pulumi without needing any extra configuration beyond the workflow file and GitHub Secrets described above. That’s why it’s so easy!

If you are going to use this setup in a real team setting, you’ll probably also want to use Pulumi’s GitHub App. Simply by installing it into your repo, and combined with the above, you’ll instantly get improved GitHub Checks API integration and, more importantly, context added by the Pulumi bot to your Pull Requests about what a deployment will do before it’s even done, as shown above.

Note that this is optional with GitHub Actions, but worthwhile – it’s much easier for your teams to collaborate on deployments, including knowing whether a change might lead to downtime before it’s even triggered. “GitOps” ahoy!

Learning More

There is so much more fun to have, and we’re just getting started. Look for more in the weeks to come. In the meantime, here are some follow up links in case you want to learn more about GitHub Actions and Pulumi:

To learn more, install Pulumi and then check out our Getting Started guide.
In addition to the keynote video, we have two other videos that you might enjoy watching to learn more:
- In this video, we see the GitOps workflow for promoting from staging to production.
- In this longer-form video, we explore more nuances of how the entire experience works.
The Ruby on Rails App, deployed to Kubernetes, from the keynote, is available for easy forking here.

This is an exciting day for developers, DevOps engineers, and SREs alike. The combination of GitHub Actions and Pulumi gives us all an easy, automated solution for continuous deployment of cloud applications and infrastructure to any cloud, purely using code and Git. What’s best about it is that it’s fun in the same way programming is fun.

If you’re not yet in the GitHub Actions private beta, but want to try Pulumi, head on over to our Pulumi Getting Started guide. You’ll be running CLI commands, but we think that can be a lot of fun too!

We want to thank our GitHub partners for the bold and innovative work that went into GitHub Actions, and for letting us in on the action – we can’t wait to see what you, the Pulumi community, builds using this magical combination!

Introducing Pulumi, a Cloud Development Platform

Joe Duffy — Mon, 18 Jun 2018 00:00:00 +0000

Ahoy!

Today we launched Pulumi, an open source cloud development platform, providing the cloud’s first true programming model using real programming languages.

We believe that containers, serverless, and hosted cloud services change everything about how we design and build software and applications. All developers these days are cloud developers. We saw a big gap in the way people want to create this software, and how they are required to do it today with configuration YAML and templating languages, and sought to close that gap.

Pulumi brings all the things we know and love with programming languages – IDEs, abstraction, encapsulation, sharing and reuse, and more – to the creation of all aspects of cloud programs and applications, while retaining the full power of immutable infrastructure across Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP), as well as Kubernetes running in any public, private, or hybrid cloud. An open ecosystem of packages promises to eliminate copy-and-paste, and encourage sharing of best practices within teams and the community.

We started on Pulumi a little over a year ago, have built an incredible team, and I’m thrilled to announce its availability. Download or learn more about this unique cloud development platform here, or read more background on Pulumi’s motivation and world-view over on my personal blog.

The team can’t wait to see all the incredible things you will build using Pulumi.