Pulumi Blog: Levi Blackstone

Connect Your Cloud Accounts to Pulumi in Minutes

Levi Blackstone — Wed, 15 Jul 2026 00:00:00 +0000

Pulumi Insights gives you visibility and governance across your entire cloud footprint, but that visibility is only as complete as the set of accounts you’ve connected. Until now, connecting an account meant repeating a manual setup for each one: OIDC configuration, hand-written Pulumi ESC environments, and per-account scan and policy setup. For an organization with dozens or hundreds of AWS accounts, Azure subscriptions, or Google Cloud projects, that per-account friction was the biggest obstacle to getting a complete picture. Today, the new Connect cloud accounts wizard removes it: discover every account in your cloud organization, select the ones you want, and onboard them all in a single guided flow.

From hours of setup to a single flow

The time savings are real: connecting a batch of accounts across AWS, Azure, and Google Cloud takes about three minutes end to end. Setting up those same accounts with the manual per-account workflow would take hours to days.

The wizard handles the entire onboarding lifecycle for AWS, Microsoft Azure, and Google Cloud:

Bulk discovery: Authenticate once with your cloud organization and the wizard lists every account, subscription, or project you can access. Accounts that are already connected are recognized and skipped.
Automatic OIDC setup: The recommended flows configure each account with short-lived credentials based on OpenID Connect (OIDC) and workload identity federation. No long-lived cloud secrets are stored in Pulumi Cloud.
ESC environments, created for you: The wizard generates Pulumi ESC environments that follow the best practices from the manual OIDC guides — environments that previously had to be authored one by one.
Scans and policies from day one: By default, scheduled discovery scans and a pre-built policy pack are applied to every account as part of setup: the Pulumi Best Practices pack on the Team and Enterprise editions, or a compliance pack (CIS, or NIST 800-53 for AWS and Google Cloud) on Business Critical.

How it works

The wizard walks you through five steps: choose a provider, authenticate, select accounts, configure discovery and policy, and review the results. You can open it from Management > Accounts in the Pulumi Cloud console, or from the Get to know Pulumi card on the home dashboard.

Authentication uses each provider’s native federation mechanism: IAM Identity Center (SSO) for AWS, Microsoft Entra ID workload identity federation for Azure, and Workload Identity Federation for Google Cloud. After you sign in, the wizard discovers the accounts in your organization and pre-selects everything that isn’t already connected. You can search, select all, or toggle individual accounts.

Choose the right access level for your security posture

Not every team wants to grant write access on day one. The wizard offers two access levels, and you can set them per account:

Build & Manage (read and write): Enables the full platform: Pulumi Neo, infrastructure as code, deployments, and policies that remediate issues automatically (Business Critical).
Discovery & Policy (read-only): Limited to discovery scanning and inventory. Pulumi can’t modify your infrastructure.

If your security review requires it, start everything read-only and raise access for specific accounts later.

Everything set up, nothing hidden

When setup completes, the summary shows exactly what was created: the ESC environments grouped by access level, the state of discovery scanning, and the policy pack applied. If any account fails to connect, the summary lists it with the specific error so you can fix the cause and re-run the wizard. Accounts that connected successfully are skipped on retry.

For security reviewers, the docs include a full accounting of what the wizard creates in your cloud provider and in Pulumi Cloud: the IAM roles, app registrations, and service accounts on the cloud side, and the ESC environments and Insights accounts on the Pulumi side.

Get started

The Connect cloud accounts wizard is available now for all Pulumi Cloud organizations. To connect your first accounts:

Navigate to Management > Accounts in the Pulumi Cloud console and select Connect cloud accounts.
Follow the guided flow for AWS, Azure, or Google Cloud.
Explore your discovered resources and policy findings.

To learn more:

Connect cloud accounts documentation — prerequisites, each wizard step in detail, and troubleshooting
Insights & Governance overview — full documentation for discovery and policy capabilities
Pulumi ESC — how the generated environments manage cloud credentials with OIDC

From Kubernetes Gatekeeper to Full-Stack Governance with OPA

Levi Blackstone — Thu, 19 Mar 2026 00:00:00 +0000

Pulumi’s OPA (Open Policy Agent) support is now stable. The v1.1.0 release of pulumi-policy-opa makes OPA/Rego a first-class policy language for Pulumi with full feature parity alongside the native TypeScript and Python policy SDKs. Write Rego policies that validate any resource Pulumi manages, across AWS, Azure, GCP, Kubernetes, and the rest of the provider ecosystem. If you already have Kubernetes Gatekeeper constraint templates, a new compatibility mode lets you drop those .rego files directly into a Pulumi policy pack and enforce them against your Kubernetes resources without modification.

What’s in the stable release

OPA/Rego is now fully supported as a policy language for Pulumi Insights, with the same capabilities as the TypeScript and Python SDKs:

Resource and stack-level policies: Validate individual resources with deny and warn rules, or evaluate your entire stack at once with stack_deny and stack_warn for cross-resource checks like relationship validation and resource count limits.
Enforcement levels: Control how violations are handled. mandatory blocks deployments, advisory surfaces warnings, and disabled turns rules off without removing them. Enforcement levels can be overridden per policy without modifying Rego source.
Policy configuration: Pass custom parameters to policies via configuration files, with optional JSON schema validation. Configuration values are accessible in Rego as data.config.<policy_name>.<key>.
OPA metadata annotations: Use standard OPA # METADATA comments to provide titles, descriptions, and messages for your policies. These populate the policy metadata displayed in Pulumi Cloud.
Preventative and audit evaluation: OPA policies work with both preventative enforcement during pulumi up and audit policy scans for continuous compliance monitoring.

You can choose whichever language best fits your team. Organizations already using OPA across their toolchain can standardize on Rego for Pulumi policies, while teams preferring TypeScript or Python can continue to use those. All three languages work side by side in the same policy groups.

Kubernetes Gatekeeper compatibility

The headline feature of this release is native support for Kubernetes Gatekeeper constraint template rules. If you’re running Gatekeeper as an admission controller in your clusters, you likely have a library of .rego policies that enforce security and operational standards at admission time. With v1.1.0, those same rules can now run as Pulumi policies, catching violations during pulumi preview before resources ever reach the cluster.

To enable Gatekeeper compatibility, set inputFormat: kubernetes-admission in your PulumiPolicy.yaml:

description: Kubernetes Gatekeeper Policy Pack
runtime: opa
inputFormat: kubernetes-admission

With this setting, Pulumi automatically wraps Kubernetes resources in the Gatekeeper AdmissionReview structure (input.review.object, input.review.kind, etc.), so your existing rules work without modification. Non-Kubernetes resources are silently skipped.

Here’s an example that reuses standard Gatekeeper-style rules, requiring an app label and prohibiting the latest image tag:

package gatekeeper

import rego.v1

# METADATA
# title: Require App Label
# description: All Kubernetes resources must have an "app" label.
violation contains {"msg": msg} if {
 not input.review.object.metadata.labels["app"]
 msg := sprintf("%s '%s' is missing required label: app",
 [input.review.kind.kind, input.review.name])
}

# METADATA
# title: Disallow Latest Tag
# description: Container images must not use the "latest" tag.
deny contains msg if {
 container := input.review.object.spec.template.spec.containers[_]
 endswith(container.image, ":latest")
 msg := sprintf("container '%s' uses the 'latest' tag -- pin to a specific version",
 [container.name])
}

These rules are identical to what you’d write for Gatekeeper. Both rule head formats are supported and can coexist: the violation[{"msg": msg}] map format and the deny[msg] string format. Per-policy configuration via input.parameters also works as expected. You can take a .rego file from your Gatekeeper constraint templates, drop it into a Pulumi policy pack, and publish it to Pulumi Cloud to enforce automatically across your stacks.

This shifts policy enforcement left. Instead of waiting for the Kubernetes API server to reject a resource at admission time, you catch the violation during pulumi preview, before anything is deployed.

Walkthrough: Reusing policies from the gatekeeper-library

The OPA Gatekeeper Library is a community-maintained collection of constraint templates covering common Kubernetes guardrails like pod security, image provenance, and resource limits. You can use these policies directly with Pulumi. Here’s an end-to-end example using the allowedrepos policy to restrict which container image registries your deployments can use.

Create a new Kubernetes OPA policy pack:
```
pulumi policy new kubernetes-opa
```

Copy the Rego source from gatekeeper-library into your policy pack as allowedrepos.rego. No modifications are needed:

package k8sallowedrepos

violation[{"msg": msg}] {
 container := input.review.object.spec.containers[_]
 not strings.any_prefix_match(container.image, input.parameters.repos)
 msg := sprintf("container <%v> has an invalid image repo <%v>, allowed repos are %v",
 [container.name, container.image, input.parameters.repos])
}

violation[{"msg": msg}] {
 container := input.review.object.spec.initContainers[_]
 not strings.any_prefix_match(container.image, input.parameters.repos)
 msg := sprintf("initContainer <%v> has an invalid image repo <%v>, allowed repos are %v",
 [container.name, container.image, input.parameters.repos])
}

violation[{"msg": msg}] {
 container := input.review.object.spec.ephemeralContainers[_]
 not strings.any_prefix_match(container.image, input.parameters.repos)
 msg := sprintf("ephemeralContainer <%v> has an invalid image repo <%v>, allowed repos are %v",
 [container.name, container.image, input.parameters.repos])
}

Verify that your PulumiPolicy.yaml has Gatekeeper compatibility enabled:

description: Kubernetes Gatekeeper Policy Pack
runtime: opa
inputFormat: kubernetes-admission

Configure the allowed registries. Create a policy-config.json file to pass the repos parameter:

{
 "k8sallowedrepos": {
 "repos": ["gcr.io/my-company/", "docker.io/library/"]
 }
}

Test the policy locally against a stack:
```
pulumi preview --policy-pack . --policy-pack-config policy-config.json
```
Any Kubernetes deployment using an image outside the allowed registries will produce a violation at preview time, before it reaches the cluster.
Publish the pack and add it to a policy group to enforce it across your organization:
```
pulumi policy publish
```

The same approach works for any policy in the gatekeeper-library: containerlimits, requiredlabels, disallowedtags, and others. Copy the Rego, configure parameters, and publish.

Part of the Pulumi Insights governance story

OPA policy support is part of the broader Pulumi Insights governance platform. Insights gives you visibility and compliance across your entire cloud footprint, and OPA policies plug directly into that:

Audit policy scans continuously evaluate OPA policies against your Pulumi stacks and discovered cloud resources, providing a compliance baseline without redeploying anything.
Self-hosted execution lets you run policy evaluations on your own infrastructure using customer-managed workflow runners, keeping credentials and data within your network.
Pre-built compliance packs for CIS, NIST, PCI DSS, and other frameworks are available alongside your custom OPA policies in the same policy groups.

Whether you’re enforcing policy at deployment time, scanning existing infrastructure for drift, or running continuous compliance checks, OPA policies are a native participant.

Frequently asked questions

Do I need to modify my existing Gatekeeper `.rego` files?

No. Set inputFormat: kubernetes-admission in your PulumiPolicy.yaml and your existing Gatekeeper constraint template rules work as-is. Pulumi handles the AdmissionReview wrapping automatically.

What happens with non-Kubernetes resources?

When using inputFormat: kubernetes-admission, non-Kubernetes resources are silently skipped during evaluation. Your Gatekeeper rules only run against Kubernetes resources.

Do I need OPA installed locally?

No. The pulumi-policy-opa analyzer plugin embeds the OPA evaluation engine and is installed automatically by the Pulumi CLI (v3.227.0+). The standalone OPA CLI is only needed if you want to run opa test against your policies independently.

When should I use OPA vs. TypeScript or Python for policies?

If your team already writes Rego for other tools like Gatekeeper, writing Pulumi policies in Rego keeps your policy language consistent. If your team is more comfortable with general-purpose languages or needs auto-remediation, use the TypeScript or Python SDKs.

Gatekeeper constraint templates can be reused directly via the kubernetes-admission input format, but other OPA integrations use different input structures, so those policies would need to be adapted to Pulumi’s resource model. All three languages work together in the same policy groups.

Get started

Templates are available for kubernetes-opa, aws-opa, azure-opa, and gcp-opa via pulumi policy new. For more details, see the policy authoring guide and the Policy as Code overview.

Get started with OPA policies

github.com/pulumi/pulumi-policy-opa

Run Pulumi Insights on Your Own Infrastructure

Levi Blackstone — Mon, 02 Mar 2026 00:06:00 -0700

Pulumi Insights gives you visibility and governance across your entire cloud footprint: discovery scans catalog every resource in your cloud accounts, and policy evaluations continuously enforce compliance against those resources. Until now, Insights workflows ran exclusively on Pulumi-hosted infrastructure. That works well for many teams, but enterprises with strict data residency requirements, private network constraints, or regulatory obligations need to run this work in their own environments. Today, Pulumi Insights supports customer-managed workflow runners for both SaaS Pulumi Cloud and self-hosted Pulumi Cloud installations.

Insights at a glance

Insights provides two complementary capabilities that together form a governance lifecycle for your cloud infrastructure.

Discovery scans cloud accounts across AWS, Azure, GCP, and more to catalog every resource regardless of how it was provisioned: Pulumi, Terraform, CloudFormation, or manual creation. Once cataloged, you can search, filter, group, and export your resource data. You can also import unmanaged resources into Pulumi to bring them under IaC management.

Policy enforces compliance with policy-as-code written in TypeScript or Python. Pulumi ships pre-built compliance packs for CIS, NIST, PCI DSS, HITRUST, and other frameworks so you can start evaluating without writing any code. Audit policy groups continuously evaluate all discovered resources and IaC stacks, while preventative policies block non-compliant deployments before they reach production.

This enables you to map out your cloud estate, evaluate compliance, and then remediate any issues uncovered by policy.

Why self-hosted?

Running Insights on your own infrastructure with customer-managed workflow runners gives you:

Data residency: Scan execution and policy evaluation run entirely within your private network.
Private infrastructure access: Scan resources in VPCs and environments that are not accessible from the public internet.
Compliance: Cloud provider credentials can stay internal to your network, meeting regulatory requirements for credential handling.
Flexible hosting: Run workflow runners on any environment that meets your needs, including Linux, macOS, Docker, and Kubernetes.

How it works

Customer-managed workflow runners are lightweight agents that poll Pulumi Cloud for pending work, execute it locally, and report results back. You can configure runners to handle specific workflow types: discovery scans, policy evaluations, deployments, or all three.

This works identically whether you use SaaS Pulumi Cloud or a self-hosted installation. The runner communicates with the Pulumi Cloud API over HTTPS, so no inbound connectivity is required, making it well suited to run in restricted network environments.

Under the hood, this is powered by a distributed work scheduling system that routes activities to the right runner pool, handles lease-based execution, and recovers automatically from failures. For a deep dive on the architecture, see How We Built a Distributed Work Scheduling System for Pulumi Cloud.

If your team already uses customer-managed workflow runners for Pulumi Deployments, your existing runner pools can handle Insights workflows with no additional infrastructure.

Get started

Self-hosted Insights is available on the Business Critical edition of Pulumi Cloud. To learn more or get set up:

Self-hosted Insights documentation — configuration and setup for discovery scans and audit policy evaluations on your own infrastructure
Customer-managed workflow runners — runner installation, configuration reference, and pool management
Insights & Governance overview — full documentation for discovery and policy capabilities
Contact sales to enable self-hosted Insights for your organization

How We Built a Distributed Work Scheduling System for Pulumi Cloud

Levi Blackstone — Thu, 26 Feb 2026 00:00:00 +0000

Pulumi Cloud orchestrates a growing number of workflow types: Deployments, Insights discovery scans, and policy evaluations. Some of that work runs on Pulumi’s infrastructure, and some of it runs on yours via customer-managed workflow runners. We needed a scheduling system that could handle all of these workflow types reliably across both environments. In this post, we’ll take a look at the system we built.

Where we started

For our first workflow integration, Deployments, scheduling wasn’t too complicated. A deployment was queued, a worker picked it up, and it ran. The queue was purpose-built for deployments, and it worked well for that single use case. Over time, we added more sophisticated logic to handle retries, ordering, rate limiting, observability, and more.

With the launch of Insights, the number of workflow types grew. Now Pulumi Cloud manages discovery scans to catalog cloud resources and runs audit policy evaluations to continuously verify compliance. While these workflows share similarities, each type needed its own scheduling, retry logic, and failure handling.

Later we added the option for customers to run workflows on their own infrastructure using customer-managed workflow runners. As the complexity of these requirements grew, we knew that our initial approach for Deployments wasn’t going to scale. We needed a single system that could schedule any type of work, route it to the right place, and handle the messy reality of distributed execution: crashes, network failures, rate limits, and retries.

We call this the background activity system.

Why not use an off-the-shelf queue?

Why build this instead of using Amazon SQS, RabbitMQ, or one of the many existing queue libraries? We considered these options but chose to build our own for a few reasons.

Pulumi Cloud supports self-hosted installations, including air-gapped environments. We intentionally minimize external dependencies so that self-hosted customers don’t have to stand up additional infrastructure. A system built on an external queue works fine for our hosted service, but it means self-hosted customers would need to provide a compatible backend. By building on top of the database we already require, we avoid adding another system to maintain.

More importantly, queueing is only part of the problem. What we actually need is scheduling with durability. This means ensuring that remote workers don’t lose activities on restart, priority so that urgent work gets compute resources first, constraints like “only so many scans per org at a time,” structured logging for observability, and checkpointing so that long-running operations can resume after a failure.

These features can be layered onto a generic queue library but can require more code than implementing them directly. For example, priority queues are often implemented with multiple ranked queues, but this breaks single-activity-at-a-time constraints. A second queue wouldn’t see a job already running in the first one. There’s no way for producers in a distributed system to coordinate across the queues without support in the queuing system itself.

Capacity management is another area where generic queues fall short. Distributed systems need to respond dynamically to slowdowns, network interruptions, and rate limits from downstream services. These are common low-level details that every workflow type needs, and building them into the scheduling layer means individual handlers don’t have to solve them independently.

We also need structured logging that works everywhere, including on customer-managed runners behind firewalls where centralized logging services aren’t accessible.

Building this ourselves gave us a system that works with existing infrastructure and handles these requirements natively.

Design constraints

With that context, here are some of the constraints that shaped the design:

Pull-only agents. Customer-managed workflow runners live behind NATs, corporate proxies, and air-gapped networks. They can’t accept inbound connections, so all communication has to be agent-initiated.
Mixed execution environments. The same system needs to work for Pulumi-hosted workers (with direct access to internal systems) and customer-managed runners (communicating entirely over REST). We didn’t want to maintain two separate code paths.
Different workflow types. Deployments, Insights scans, and audit policy evaluations have different payloads and execution semantics, but they all need the same scheduling guarantees: exactly-once execution, automatic retries, failure recovery, and observability.
Automatic fault tolerance. Agents crash, networks drop, and machines get recycled by autoscalers. The system needs to detect these failures and recover without needing a person to step in.
Extensibility. We knew we’d keep adding workflow types. Adding a new one should mean writing a handler and registering it, not building new infrastructure.

The background activity

At the center of the system is the background activity, a persistent, typed work unit. Each activity includes:

A type discriminator that identifies what kind of work it represents (e.g., “insights-discovery” or “policy-evaluation”)
A payload specific to that type, containing whatever data the handler needs
A routing context that determines which runner pool should execute it
Scheduling metadata like priority, activation time, and retry configuration
A status tracking where the activity is in its lifecycle

The type discriminator makes this system polymorphic. The scheduling engine doesn’t need to know what’s inside the payload. It moves activities through their lifecycle and delegates the actual work to a type-specific handler.

The state machine

Every activity follows the same lifecycle regardless of type:

---
config:
flowchart:
curve: linear
---
graph LR
Start(( )) -->|Created| Ready
Ready -->|Leased| Pending
Pending -->|Started| Executing
Executing -->|Success| Completed
Completed --> End(( ))
Executing -->|Error| Failed
Failed --> End
Executing -->|Canceled| Canceled
Canceled --> End
Executing -->|Dependencies| Waiting
Waiting -->|Unblocked| Ready
Pending -->|Lease expired| Restarting
Executing -->|Lease expired| Restarting
Restarting -->|Re-lease| Ready
style Start fill:#000,stroke:#000,color:#000
style End fill:#000,stroke:#000,color:#000

The states fall into two groups:

Running states (work is in flight or can be resumed):

Ready: queued and eligible to be claimed by a worker
Pending: claimed by a worker, execution about to start
Executing: actively running on a worker
Waiting: parked, blocked on one or more dependency activities
Restarting: recovered after a worker failure, ready to be re-claimed

Terminal states (work is done):

Completed, Failed, Canceled

New workflow types get these features automatically: scheduling, retries, dependency management, and observability.

Leases: distributed execution without coordination

A central challenge of any distributed work queue is preventing double-execution. If two agents try to execute the same activity simultaneously, you get duplicate work and data corruption. A central coordinator can solve this, but it becomes a single point of failure.

We use lease-based optimistic concurrency instead. This is a well-known pattern, adapted here for long-running, stateful workflows.

How it works

When an agent is ready for new work, it asks the service to lease an activity. The service atomically selects the highest-priority ready activity, assigns a lease token with an expiration time, and transitions the activity to Pending. No other agent can claim the same activity.

sequenceDiagram
participant Agent
participant Service
Agent->>Service: Poll for work
Note right of Service: Select highest-priority<br/>Ready activity
Note right of Service: Atomically set lease<br/>token + expiration
Service->>Agent: Lease (token, expiration)
Agent->>Service: Begin execution
Note right of Service: Transition to Executing
Note over Agent: Work in progress...
Agent->>Service: Renew lease
Service->>Agent: New expiration
Note over Agent: Work continues...
Agent->>Service: Complete (token, result)
Note right of Service: Transition to Completed<br/>Archive activity
Service->>Agent: Acknowledged

While executing, the agent periodically renews its lease to signal that it’s still working. If the agent crashes, loses network connectivity, or is terminated, it stops renewing. Once the lease expires, the service transitions the activity to Restarting, making it available for another agent to claim.

sequenceDiagram
participant A as Agent A
participant S as Service
participant B as Agent B
A->>S: Lease activity
S->>A: Token + expiration
Note over A: Executing...
A--xS: Agent A crashes
Note right of S: Lease expires
Note right of S: Transition → Restarting
B->>S: Poll for work
Note right of S: Lease to Agent B<br/>Transition → Pending
S->>B: Token + expiration
Note over B: Agent B continues execution

The service doesn’t need to explicitly coordinate between workers because leases are acquired using atomic database operations. The lease expiration is the failure detector; if a lease expires, then the work needs to be rescheduled.

Routing work to the right runner pool

Pulumi Cloud supports multiple workflow runner pools. An organization might have one pool for production in us-east-1, another for staging in eu-west-1, and use Pulumi-hosted runners for development. Work needs to reach the right pool.

Each activity carries a routing context that identifies which runner pool should execute it. When a runner polls for work, it filters by its own pool identifier so that it only sees activities meant for it.

We use prefix matching for this filtering. A runner matches activities whose context starts with its pool’s identifier. This means the service can use hierarchical contexts (e.g., pool-abc/insights/scan-123) and runners will still match on the pool prefix. Cleanup is also straightforward; when a runner pool is deleted, all activities with that context prefix are bulk-canceled.

This routing mechanism works the same way regardless of workflow type, and adding a new workflow type doesn’t require changes to the routing layer.

Dependencies and multi-step workflows

Some workflows are naturally multi-step. An Insights discovery scan might discover resources that then need policy evaluation. Rather than building a separate orchestration engine, we built dependency management into the activity system.

An activity can declare a dependency set: a list of other activities that must complete before it can run. A dependent activity enters the Waiting state when created. As its dependencies complete, the system checks whether all prerequisites are satisfied. When the last one finishes, the waiting activity transitions to Ready and enters the scheduling queue.

graph TD
A["Insights Discovery<br/><b>Executing</b>"] -->|depends on| B["Policy Evaluation<br/><b>Waiting</b>"]
A -->|completes| C["Insights Discovery<br/><b>Completed</b>"]
C -->|triggers| D["Policy Evaluation<br/><b>Ready</b><br/>(auto-scheduled)"]

This gives us a lightweight DAG of work without requiring a separate workflow engine. Dependent activities get the same guarantees as any other activity: lease-based execution, automatic recovery, and observability.

Two execution modes, one interface

This is where the design really pays off for customer-managed runners. The system supports two execution modes:

Direct mode runs in-process alongside the Pulumi Cloud service. Workers have low-latency access to internal systems and can process activities with minimal overhead. This is what Pulumi-hosted runners use.
Remote mode communicates over REST APIs. The runner polls for activities, leases them, executes work locally, and reports results back over HTTP. This is what customer-managed runners use. No database access, no internal network access, no inbound connectivity required.

Both modes share the same handler interface so that a workflow handler doesn’t need to know where it’s running. Whether it’s running on Pulumi’s hosted infrastructure or on a customer’s Kubernetes cluster, the handler simply processes the payload and reports a result.

Putting it all together

Let’s walk through a concrete example. A user wants to run an Insights discovery scan on an AWS account using a customer-managed workflow runner.

sequenceDiagram
participant User
participant PC as Pulumi Cloud
participant Runner as Workflow Runner<br/>(Customer Infrastructure)
User->>PC: 1. Configure Insights scan<br/>for runner pool
Note right of PC: 2. Create background activity<br/>type: insights-discovery<br/>context: runner-pool-xyz<br/>status: Ready
Runner->>PC: 3. Poll for work (filtered by pool)
PC->>Runner: 4. Lease activity (token + expiration)
Runner->>PC: 5. Initialize workflow
PC->>Runner: Return cloud credentials + job token
Note over Runner: 6. Execute scan locally<br/>(talks directly to cloud APIs —<br/>credentials are used only on<br/>the runner)
Runner->>PC: 7. Renew lease
Note right of PC: Extend expiration
Runner->>PC: 8. Report completion
Note right of PC: 9. Mark completed<br/>Archive activity
Note right of PC: 10. Unblock dependent activities<br/>(e.g., policy evaluation)

A user configures an AWS account for Insights scanning in Pulumi Cloud and assigns it to a workflow runner pool.
Pulumi Cloud creates a background activity with the type set to insights discovery, the routing context set to the runner pool, and the payload containing the account configuration.
A customer-managed workflow runner polling that pool detects new work.
The runner leases the activity, acquiring an exclusive lock via the lease token.
The runner initializes the workflow, receiving any required cloud provider credentials (e.g., resolved from Pulumi ESC (Environments, Secrets, and Configuration)) and a job token from Pulumi Cloud.
The runner executes the scan locally on the customer’s infrastructure, talking directly to the cloud provider APIs.
During execution, the runner periodically renews its lease to signal liveness.
The scan completes, and the runner reports the result back to Pulumi Cloud.
The service marks the activity as completed and archives it.
If a dependent policy evaluation activity was waiting on this scan, it automatically transitions to Ready and enters the scheduling queue, where another runner in the pool can pick it up.

This flow works the same way whether the runner is hosted by Pulumi or by the customer. The only difference is whether the execution mode is direct or remote.

Retries and scheduling

Failures are expected in distributed systems. The background activity system handles them at several levels:

Lease expiration covers hard failures like agent crashes, network partitions, and machine terminations. If a lease expires, the activity moves to Restarting, and is available for another agent to pick up.
Handler-controlled retries cover soft failures like transient API errors and rate limits. A handler can request a reschedule with a delay, putting the activity back in Ready with a future activation time.
Automatic retries provide a configurable retry budget per activity. Each activity can specify how many times it should be retried and the delay between attempts, preventing runaway retry loops.
Priority scheduling ensures urgent work gets processed first. Higher-priority activities are leased before lower-priority ones, even if the lower-priority activity has been waiting longer.
Lease renewal during slowdowns keeps the activity alive without blocking other work, even if a downstream service is slow. The agent continues renewing its lease while it waits, and the scheduler remains free to assign other activities to other agents.

Observability

Every activity generates a structured log of its execution, including timestamps, severity levels, and code context. Logs are stored with the activity record and are accessible via an API and admin tooling.

This is especially useful for customer-managed runners, where the service can’t directly observe the execution environment. The structured log gives operators visibility into the execution context, even when the runner is behind a firewall. Handlers can also use these logs as a progress journal, encoding checkpoints that allow a restarted activity to pick up where it left off rather than starting from scratch.

Retention policies are configurable per organization and per workflow type. Completed activities can be retained for auditing or purged to manage storage, and failed activities are typically retained longer for debugging.

What we learned

A generic system pays off quickly. Our initial instinct was to build targeted solutions for each workflow type. Investing in a generic activity system required more upfront design work, but now adding a new workflow type requires a fraction of the effort it would take otherwise. New workflows ship with full scheduling, retry, and observability support from day one.

Leases handle many failure modes. We evaluated several approaches for distributed work coordination, including message queues with explicit acknowledgment and coordinator-based assignment. The lease model works well because all failure modes are handled through timeouts. If an agent is running as expected, it renews. If it isn’t, the lease expires.

Keeping the execution paths symmetric requires discipline. Making the hosted and self-hosted paths share the same handler interface was a deliberate choice. It would be easy to add shortcuts for the hosted path that bypass the remote API, but resisting that temptation means that features work for both cases automatically.

The hard part isn’t running the work. Running a scan or a deployment is straightforward once you have the right credentials. The real complexity is in everything around the execution: scheduling, routing, leasing, retrying, resolving dependencies, and cleaning up. These operational concerns aren’t visible to users, but they are essential to providing a reliable experience.

Wrapping it up

Today this system powers deployments, Insights discovery scans, and policy evaluations across both Pulumi Cloud and customer-managed infrastructure. The architecture is general enough that every new workflow type we add inherits the full scheduling, routing, retry, and observability stack without additional plumbing.

If you’re interested in running workflows on your own infrastructure, check out customer-managed workflow runners. To see how Insights can help you understand and manage your cloud infrastructure, get started with Pulumi Insights.

New Audit Policy Scans for Pulumi Stacks

Levi Blackstone — Wed, 05 Nov 2025 00:04:00 +0000

Audit Policy Scans for Pulumi Stacks is part of the next generation of Pulumi Policies. This capability uses policies to run compliance checks against the last successful deployment state of your stacks, providing continuous compliance monitoring without impacting your existing CI/CD workflows.

Until now, Pulumi’s preventative policies have served as a critical “shift-left” gate, blocking non-compliant changes during pulumi up. While essential, this created challenges for organizations wanting to roll out new governance across thousands of existing stacks. This new evaluation mode solves that problem, giving you a complete and continuous view of your IaC compliance posture without the friction.

A Complete Audit Story: Cloud Accounts and IaC Stacks

Pulumi’s audit philosophy is to provide complete visibility across your entire cloud footprint. We achieve this through two complementary audit modes:

Audit Scans for Cloud Accounts (Existing): This capability scans your live cloud environments (like an AWS account or Azure subscription). Its primary purpose is to give you a holistic view of your security posture by discovering all resources, including those not managed by Pulumi, and detecting configuration drift. This is how you find unmanaged, legacy, or manually-created resources that violate your policies.
Audit Scans for Pulumi Stacks (New): The feature extends this audit power to the source of truth for your managed infrastructure: your Pulumi stacks. It evaluates the last successfully deployed state of your IaC. This allows you to get an instant compliance baseline of all your managed infrastructure without having to redeploy anything, making it perfect for frictionless policy rollouts at scale.

Together, these two audit modes give you a comprehensive picture of your entire cloud estate, all feeding into one unified Policy Findings hub.

Key Scenarios for Audit Scans for Stacks

This new evaluation mode for IaC stacks is designed to unlock policy adoption at scale and accelerate developer velocity. Key use cases include:

Gain an instant compliance overview across all existing stacks: Apply policies centrally and get immediate visibility into your compliance posture without requiring redeployment of every stack.
Accelerate policy rollouts: By decoupling policy evaluation from the critical deployment path, deployments become faster and developers are unblocked. Teams still get immediate feedback on the compliance of the deployed infrastructure.
Ensure continuous compliance: Evaluation is automatically triggered whenever policy configurations change. This ensures the latest policies are always evaluated against the latest deployed infrastructure.

How It Works

This new mechanism runs against the last known successfully deployed state of your IaC Stacks as recorded by Pulumi Cloud.

Evaluation is triggered automatically after a successful pulumi up or when a policy pack attached to the stack is updated.
Results are unified into the same Policy Findings hub you use today, giving you a single pane of glass for all preventative and audit policy findings.

How to Get Started

You can enable audit scans for your IaC stacks by adding them to an Audit Policy Group.

Navigate to the Policies tab in the left navigation bar of the Pulumi Cloud console.
Create a new Audit Policy Group or select an existing one.
Select Pulumi Stacks and add the specific stacks you want to monitor.
Add Policy Packs to the group.
Save the policy group. That’s it!

The next time any of the selected stacks completes a pulumi up, a post-deployment evaluation will be automatically triggered, and the results will populate in the Policy Findings hub.

Audit policy scans consume workflow minutes from your Pulumi Cloud plan. Each scan counts toward your monthly quota based on the complexity and number of resources evaluated. See pricing documentation for details.

A Complete Governance Picture

With the addition of post-deployment evaluation for IaC stacks, you now have a complete, 360-degree view of your cloud environment. You can use audit scans for cloud accounts to get a handle on your entire live footprint, and use audit scans for stacks to easily assess your IaC-managed footprint without adding friction to your development process.

Try Pulumi Policies

Ready to try these features?

Sign up for Pulumi Cloud and start a Neo task
Read the Get Started guide to continuously scan resources and identify violations
Join the Community Slack to share feedback on the new features

For complete documentation, visit our Policies documentation.

Kubernetes-native Option for Pulumi Deployments Customer-Managed Agents

Levi Blackstone — Wed, 14 Aug 2024 08:08:31 -0700

We are excited to announce the availability of Kubernetes-native support for Pulumi Deployments Customer-Managed Agents, further enhancing the flexibility and control over your infrastructure deployments. This new feature allows you to self-host deployment agents within your Kubernetes environment, bringing the same power and flexibility of Pulumi-hosted deployments to isolated environments.

Customer Managed Agents: Flexibility and Control

Customer Managed Agents, announced earlier this year, allow you to self-host deployment agents, delivering the power and flexibility of Pulumi Deployments within your own infrastructure. Here’s how you can benefit:

Host Anywhere: Deploy Customer Managed Agents on any hardware or environment of your choice, including fully private VPCs.
Compliance and Security: Keep your cloud provider credentials within your private network, meeting stringent compliance requirements.
Scalability: Set up multiple agent pools and scale dynamically, supporting up to 150 concurrent deployments.
Versatility: Mix and match Pulumi-hosted and self-hosted deployments to suit different infrastructure needs.

Key Features of Pulumi Deployments

Customer-Managed agents support all the capabilities of Pulumi Deployments including:

Multiple Deployment Triggers: Supports all Pulumi Deployment triggers such as Drift Detection, Scheduled Deployments, click to deploy, REST API, Git push to deploy, Review Stacks, and remote Automation API.
OpenID Authentication: Fetch tokens dynamically for increased security.
Flexible Configuration: Configure through YAML files or environment variables, adaptable to various deployment environments.

Docker and Kubernetes Support

By default, Customer-Managed Agents are deployed on Virtual Machines using Docker. However, with this new release, you can now leverage Kubernetes to run these agents, enabling you to deploy them within your existing infrastructure. This integrates natively with Kubernetes pods and does not require Docker in Docker or privileged execution.

Getting Started with Kubernetes

To use Kubernetes for running Customer-Managed Agents:

Install the Pulumi GitHub App: Install the Pulumi GitHub App and ensure that it is integrated with your source control.
Set Up Deployment Pools:
- Navigate to Deployment Runners under Organization Settings in the left hand navigation bar.
- Create a new pool by clicking ‘Add new pool’
- You will get a new access token, copy and save the token.
Install Agents:
- Follow the instructions provided on the Deployment Runners page to install agents within your Kubernetes environment.
Configure Agents:
- All configuration for Customer-Managed Agents is done through the pulumi-deployment-agent.yamlfile. This file can be created manually or with the customer-managed-deployment-agent configure command.
- Or you can leverage the Pulumi program example repo.
- Set environment variables directly in your Kubernetes deployment.
Configure Stack:
- Go to Deploy settings, found under the Settings tab on a stack. Scroll down and select the pool you created under the Deployment Runner pool drop-down.

Verification

After setting up, verify the agent by doing a pulumi refresh through the Deploy Actions drop-down on your stack page.

To increase deployment concurrency, use the same token to set up multiple agents and they will be assigned to the same pool.

Wrapping It Up

Customer-Managed Agents with Kubernetes support empower you to manage deployments according to your specific needs. Whether it’s for security, compliance, or operational efficiency, this feature provides the flexibility and control necessary for modern infrastructure management.

For more details, check out our documentation or contact our sales team to get started with Customer-Managed Agents today.

Feel free to reach out with feedback or questions. We’re here to support you every step of the way.

Pulumi Launches Scheduled Deployments: Automate Recurring Workflows

Meagan Cojocar — Wed, 24 Apr 2024 07:00:02 -0800

We are excited to announce Scheduled Deployments for Pulumi Cloud. This new feature, which builds upon the robust foundation of Pulumi Deployments, is designed to streamline cloud management by automating the deployment and deactivation of resources according to customizable schedules. Two new Pulumi features also announced today, Drift Detection and Remediation and Time-to-Live Stacks, are built on top of the new Scheduled Deployments functionality. These three new features are available today on the Enterprise and Business Critical editions of Pulumi Cloud.

Simplifying Cloud Operations Through Automation

Scheduled Deployments enable precise automation of cloud operations, allowing users to schedule any Pulumi operation—such as pulumi up, pulumi refresh, pulumi destroy, or pulumi preview—for any stack with Pulumi Deployments. This feature is not just about automating routine tasks; it’s about enhancing operational efficiency, reducing costs, and deploying resources exactly when needed.

Example Use Cases

Automating Deployments outside of Git: Pulumi Deployments handles Git based Deployments well, but we have heard from customers there are times you want to intentionally diverge from Git triggered Deployments, such as wanting to deploy only at set times ahead of a large usage event or deploy only on non-peak traffic times.
Cost Optimization: Automatically power down non-critical development or test stacks during off-hours to cut unnecessary cloud spend. These stacks can be reactivated on a schedule or on-demand, ensuring resources are available precisely when teams need them without incurring costs during idle times.
Product Launches: Configure infrastructure to deploy right before a major release with pulumi up, ensuring that all resources are aligned and active precisely for the event, eliminating the need for last-minute manual updates.
Scheduled Processing: Perfect for tasks that require computing resources at specific times, such as nightly batch processing or periodic machine learning model training. Schedule these operations to maximize resource utilization and shut down upon task completion to optimize costs.

How Scheduled Deployments Work

Users can define schedules for any Pulumi operation, on any stack, using cron expressions, which provide the flexibility to specify exact times for operations. This granular control over deployment timing allows for effective management of infrastructure tasks, accommodating even the most complex scheduling needs. Pulumi Deployments Schedules can be set up using the Pulumi Cloud console, our REST API or managed in source control using the Pulumi Service Provider.

Integration with Existing Features

Scheduled Deployments integrate seamlessly with existing Pulumi Deployments. The concurrency limits of Pulumi Deployments apply and if deployments on a stack are paused, the scheduled deployments will queue, waiting for the pause to lift before executing. This integration ensures that all scheduled activities respect the operational parameters set within Pulumi Cloud.

Getting Started with Scheduled Deployments

Setting it up in the Pulumi Cloud Console

We’ll walk through how to get up and running with Scheduled Deployments in the Pulumi Cloud console, REST API and Pulumi Service Provider. For more details on how to implement and maximize the benefits of Scheduled Deployments, please visit our documentation.

Setting up Scheduled Deployments via the Pulumi Cloud console is straightforward:

In order to set up Scheduled Deployments in the Pulumi Cloud console, follow these steps:

Ensure Deployments Settings are configured on the stack see the docs
Navigate to the Stack > Settings > Schedules page
Select “Raw operation”
Select from the drop-down the type of operation you would like to schedule
Set the schedule using a cron expression
Save the Schedule

Now you have a Scheduled Deployment!

The Stack > Settings > Schedules page will show all of the schedules configured for your stack, including Drift, TTL or Raw Operation schedules.

Setting it up in the API

For those who prefer to automate and script their infrastructure tasks, Time-to-Live schedules can be configured programmatically using simple HTTP requests.

Create a schedule
Get a schedule for a stack
Update or delete a schedule
Pause or resume a schedule
List all schedules in the organization (includes Drift and Time-to-Live schedules)

Below is an example of creating a Schedule on a stack programmatically:

curl \
 -H "Accept: application/vnd.pulumi+8" \
 -H "Content-Type: application/json" \
 -H "Authorization: token $PULUMI_ACCESS_TOKEN" \
 --request POST \
 --data '{ "scheduleCron":"0 0 * * *", "request": { "operation": "update" } }' \
 https://api.pulumi.com/api/stacks/{organization}/{project}/{stack}/deployments/schedules

Refer to the Pulumi Deployments REST API documentation for more details on how to use the REST API to manage Scheduled Deployments.

Setting it up in the Pulumi Service Provider

Pulumi Service Provider for Creating a Raw Schedule

The Pulumi Service provider allows you to set up a raw schedule for operations like updates or deployments based on a specific time or a recurring schedule inside of source control.

import * as pulumi from "@pulumi/pulumi";
import * as pulumiservice from "@pulumi/pulumiservice";

const organizationName = "my-org";
const projectName = "my-project";
const stackName = "prod-stack";

const rawSchedule = new pulumiservice.DeploymentSchedule("rawSchedule", {
 organization: organizationName,
 project: projectName,
 stack: stackName,
 scheduleCron: "0 0 * * *", // Run an update daily at midnight
 pulumiOperation: pulumiservice.PulumiOperation.update
});

export const scheduleId = rawSchedule.scheduleId;

import pulumi
import pulumi_pulumiservice as pulumiservice

organization_name = "my-org"
project_name = "my-project"
stack_name = "prod-stack"

# Create a raw deployment schedule
raw_schedule = pulumiservice.DeploymentSchedule("rawSchedule",
 organization=organization_name,
 project=project_name,
 stack=stack_name,
 schedule_cron="0 0 * * *", # Run an update daily at midnight
 pulumi_operation=pulumiservice.PulumiOperation.update)

pulumi.export('scheduleId', raw_schedule.schedule_id)

package main

import (
 "github.com/pulumi/pulumi-pulumiservice/sdk/go/pulumiservice"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 rawSchedule, err := pulumiservice.NewDeploymentSchedule(ctx, "rawSchedule", &pulumiservice.DeploymentScheduleArgs{
 Organization: pulumi.String("my-org"),
 Project: pulumi.String("my-project"),
 Stack: pulumi.String("prod-stack"),
 ScheduleCron: pulumi.String("0 0 * * *"), // Run an update daily at midnight
 PulumiOperation: pulumiservice.PulumiOperationUpdate,
 })
 if err != nil {
 return err
 }

 ctx.Export("scheduleId", rawSchedule.ScheduleID)
 return nil
 })
}

using Pulumi;
using PulumiService = Pulumi.PulumiService;

class Program
{
 static Task<int> Main() => Deployment.RunAsync(() => {
 var rawSchedule = new PulumiService.DeploymentSchedule("rawSchedule", new PulumiService.DeploymentScheduleArgs
 {
 Organization = "my-org",
 Project = "my-project",
 Stack = "prod-stack",
 ScheduleCron = "0 0 * * *", // Run an update daily at midnight
 PulumiOperation = PulumiService.PulumiOperation.Update,
 });

 return new Dictionary<string, object?>
 {
 { "scheduleId", rawSchedule.ScheduleId }
 };
 });
}

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.pulumiservice.DeploymentSchedule;
import com.pulumi.pulumiservice.DeploymentScheduleArgs;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 private static to stack(Context ctx) {
 var rawSchedule = new DeploymentSchedule("rawSchedule", DeploymentScheduleArgs.builder()
 .organization("my-org")
 .project("my-project")
 .stack("prod-stack")
 .scheduleCron("0 0 * * *") // Run an update daily at midnight
 .pulumiOperation(com.pulumi.pulumiservice.PulumiOperation.update())
 .build());

 ctx.export("scheduleId", rawSchedule.name());
 }
}

name: raw-schedule-setup
runtime: yaml
description: Setup of a raw schedule for automatic operations with Pulumi

resources:
 rawSchedule:
 type: pulumiservice:index:DeploymentSchedule
 properties:
 organization: my-org
 project: my-project
 stack: prod-stack
 scheduleCron: "0 0 * * *" # Run an update daily at midnight
 pulumiOperation: update

outputs:
 scheduleId: ${rawSchedule.scheduleId}

See the Pulumi Service Provider documentation for more details on how to manage Scheduled Deployments in source control.

Wrapping it up

We are looking forward to unlocking this capability to enable an out of the box way to automate your reoccuring workflows.

Happy building!

Secure your Kubernetes toolchain with Pulumi ESC and OIDC

Levi Blackstone — Tue, 21 Nov 2023 00:00:00 -0700

Keeping long-lived kubeconfig around on disk is insecure and error-prone. You need a secure workflow that removes tedium. With Pulumi and ESC, we provide an automated workflow that generates a kubeconfig on-the-fly for every command using short-term credentials issued via OIDC. This makes it easy for your team to connect to a given Kubernetes environment, and it works well with Kubernetes tools such as kubectl and the Pulumi Kubernetes provider. Let’s take a look.

In this post, we will create a Kubernetes cluster, deploy an application to the created cluster, and then use kubectl to check on the deployed application. Pulumi gives you great flexibility to factor your infrastructure into reusable parts, and we’ll do that here. With ESC, we’ll define an environment that provides access to an AWS account, and use that environment to create the cluster. Then we’ll define an environment that encapsulates the kubeconfig needed to connect to that cluster.

Create a Kubernetes Cluster

First, we create a Kubernetes cluster using the pulumi-eks component.

import * as pulumi from "@pulumi/pulumi";
import * as eks from "@pulumi/eks";

const eksCluster = new eks.Cluster("eks-cluster", {});
export const kubeconfig = eksCluster.kubeconfig;

import pulumi
import pulumi_eks as eks

eks_cluster = eks.Cluster("eks-cluster")
pulumi.export("kubeconfig", eks_cluster.kubeconfig)

import (
 "github.com/pulumi/pulumi-eks/sdk/go/eks"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 eksCluster, err := eks.NewCluster(ctx, "eks-cluster", nil)
 if err != nil {
 return err
 }
 ctx.Export("kubeconfig", eksCluster.Kubeconfig)
 return nil
 })
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Eks = Pulumi.Eks;

return await Deployment.RunAsync(() =>
{
 var eksCluster = new Eks.Cluster("eks-cluster");

 return new Dictionary<string, object?>
 {
 ["kubeconfig"] = eksCluster.Kubeconfig,
 };
});

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.eks.Cluster;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 var eksCluster = new Cluster("eksCluster");

 ctx.export("kubeconfig", eksCluster.kubeconfig());
 }
}

name: eks-cluster
runtime: nodejs
description: EKS Kubernetes Cluster
resources:
 eks-cluster:
 type: eks:Cluster
outputs:
 kubeconfig: ${eks-cluster.kubeconfig}

This component includes an output value for the cluster’s kubeconfig that we can use to connect to the cluster. However, before we create the cluster, we need to configure AWS credentials so that Pulumi can deploy the requested changes. While it is possible to set this configuration directly on the stack, let’s see how it can be done with Pulumi ESC. First, we create the following ESC environment (the aws-sandbox environment):

values:
 aws:
 creds:
 fn::open::aws-login:
 oidc:
 duration: 1h
 roleArn: arn:aws:iam::0123456789:role/deploy-oidc
 sessionName: pulumi-environments-session
 environmentVariables:
 AWS_ACCESS_KEY_ID: ${aws.creds.keyId}
 AWS_SECRET_ACCESS_KEY: ${aws.creds.secretAccessKey}
 AWS_SESSION_TOKEN: ${aws.creds.sessionToken}
 pulumiConfig:
 aws:region: us-west-2

Notice that this environment uses the aws-login ESC provider to dynamically load short-lived credentials using OIDC. These credentials are then exposed as environment variables for consumers of this environment. We also set the AWS region in the pulumiConfig section to configure the region for pulumi to manage resources with the pulumi-aws provider.

To have Pulumi use this ESC environment when deploying the cluster, we set the following configuration for our stack:

environment:
 - aws-sandbox

The environment key accepts a list of ESC environments to import. When the stack is updated, pulumi automatically sets the configured environment variables and stack configuration based on the ESC environment. We run our update, and Pulumi creates our EKS cluster.

pulumi up --skip-preview
Updating (demo)
Type Name Status
+ pulumi:pulumi:Stack eks-cluster-demo created (577s)
+ └─ eks:index:Cluster eks-cluster created (561s)
+ ├─ eks:index:ServiceRole eks-cluster-eksRole created (2s)
+ │ ├─ aws:iam:Role eks-cluster-eksRole-role created (0.78s)
+ │ └─ aws:iam:RolePolicyAttachment eks-cluster-eksRole-4b490823 created (0.44s)
+ ├─ eks:index:ServiceRole eks-cluster-instanceRole created (3s)
+ │ ├─ aws:iam:Role eks-cluster-instanceRole-role created (0.85s)
+ │ ├─ aws:iam:RolePolicyAttachment eks-cluster-instanceRole-3eb088f2 created (0.46s)
+ │ ├─ aws:iam:RolePolicyAttachment eks-cluster-instanceRole-03516f97 created (0.59s)
+ │ └─ aws:iam:RolePolicyAttachment eks-cluster-instanceRole-e1b295bd created (0.75s)
+ ├─ eks:index:RandomSuffix eks-cluster-cfnStackName created (0.49s)
+ ├─ aws:ec2:SecurityGroup eks-cluster-eksClusterSecurityGroup created (1s)
+ ├─ aws:iam:InstanceProfile eks-cluster-instanceProfile created (0.48s)
+ ├─ aws:eks:Cluster eks-cluster-eksCluster created (441s)
+ ├─ aws:ec2:SecurityGroupRule eks-cluster-eksClusterInternetEgressRule created (0.81s)
+ ├─ aws:ec2:SecurityGroup eks-cluster-nodeSecurityGroup created (1s)
+ ├─ pulumi:providers:kubernetes eks-cluster-eks-k8s created (0.14s)
+ ├─ eks:index:VpcCni eks-cluster-vpc-cni created (2s)
+ ├─ kubernetes:core/v1:ConfigMap eks-cluster-nodeAccess created (0.60s)
+ ├─ aws:ec2:SecurityGroupRule eks-cluster-eksExtApiServerClusterIngressRule created (1s)
+ ├─ aws:ec2:SecurityGroupRule eks-cluster-eksClusterIngressRule created (1s)
+ ├─ aws:ec2:SecurityGroupRule eks-cluster-eksNodeInternetEgressRule created (1s)
+ ├─ aws:ec2:SecurityGroupRule eks-cluster-eksNodeClusterIngressRule created (2s)
+ ├─ aws:ec2:SecurityGroupRule eks-cluster-eksNodeIngressRule created (2s)
+ ├─ aws:ec2:LaunchConfiguration eks-cluster-nodeLaunchConfiguration created (1s)
+ ├─ aws:cloudformation:Stack eks-cluster-nodes created (108s)
+ └─ pulumi:providers:kubernetes eks-cluster-provider created (0.12s)
Outputs:
kubeconfig: {
apiVersion : "v1"
clusters : [
[0]: {
cluster: {
certificate-authority-data: "<redacted>"
server : "https://FB9978041F7B72B227B7E95F076822E4.gr7.us-west-2.eks.amazonaws.com"
}
name : "kubernetes"
}
]
contexts : [
[0]: {
context: {
cluster: "kubernetes"
user : "aws"
}
name : "aws"
}
]
current-context: "aws"
kind : "Config"
users : [
[0]: {
name: "aws"
user: {
exec: {
apiVersion: "client.authentication.k8s.io/v1beta1"
args : [
[0]: "eks"
[1]: "get-token"
[2]: "--cluster-name"
[3]: "eks-cluster-eksCluster-1268ff9"
]
command : "aws"
env : [
[0]: {
name : "KUBERNETES_EXEC_INFO"
value: (json) {
apiVersion: "client.authentication.k8s.io/v1beta1"
}
}
]
}
}
}
]
}
Resources:
+ 27 created
Duration: 9m38s

Connect to the cluster

Let’s create another Pulumi ESC environment called eks-sandbox to encapsulate the configuration needed to connect to the EKS cluster.

imports:
 - aws-sandbox
values:
 stacks:
 fn::open::pulumi-stacks:
 stacks:
 eks-cluster:
 stack: eks-cluster/demo
 kubeconfig: {'fn::toJSON': "${stacks.eks-cluster.kubeconfig}"}
 pulumiConfig:
 kubernetes:kubeconfig: ${kubeconfig}
 files:
 KUBECONFIG: ${kubeconfig}

This ESC environment uses another ESC provider to reference outputs from Pulumi stacks in your organization. In this case, we specify that we want to read outputs from the eks-cluster/demo stack, and then export the kubeconfig output from that stack as pulumiConfig and files. As we saw on the cluster environment, pulumiConfig sets configuration for Pulumi providers, so this allows us to run our stack and deploy the application to our cluster.

We import the aws-sandbox environment defined earlier, to provide the user with AWS credentials to authenticate to the EKS cluster.

Since the eks-sandbox environment defines a KUBECONFIG variable that kubectl understands, we can run kubectl in the context of this environment like this:

$ pulumi env run eks-sandbox -- kubectl cluster-info
Kubernetes control plane is running at https://2F3303BF9E03F8D61099AAB5ED6F31A4.gr7.us-west-2.eks.amazonaws.com
CoreDNS is running at https://2F3303BF9E03F8D61099AAB5ED6F31A4.gr7.us-west-2.eks.amazonaws.com/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

Deploy an application to the cluster

Now, we want to deploy an application to our new EKS cluster. While we could have included this in the same stack as the cluster, it is generally better to follow the separation of concerns principle, and manage applications separately from the underlying infrastructure. To achieve this goal, we create a new application stack with the following program:

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";

const nginxDeployment = new kubernetes.apps.v1.Deployment("nginx-deployment", {
 metadata: {
 name: "nginx",
 labels: {
 app: "nginx",
 },
 },
 spec: {
 replicas: 3,
 selector: {
 matchLabels: {
 app: "nginx",
 },
 },
 template: {
 metadata: {
 labels: {
 app: "nginx",
 },
 },
 spec: {
 containers: [{
 name: "nginx",
 image: "nginx",
 ports: [{
 containerPort: 80,
 }],
 }],
 },
 },
 },
});
const nginxService = new kubernetes.core.v1.Service("nginx-service", {
 metadata: {
 name: "nginx",
 },
 spec: {
 type: "LoadBalancer",
 selector: {
 app: "nginx",
 },
 ports: [{
 protocol: "TCP",
 port: 80,
 }],
 },
});

import pulumi
import pulumi_kubernetes as kubernetes

nginx_deployment = kubernetes.apps.v1.Deployment("nginx-deployment",
 metadata=kubernetes.meta.v1.ObjectMetaArgs(
 name="nginx",
 labels={
 "app": "nginx",
 },
 ),
 spec=kubernetes.apps.v1.DeploymentSpecArgs(
 replicas=3,
 selector=kubernetes.meta.v1.LabelSelectorArgs(
 match_labels={
 "app": "nginx",
 },
 ),
 template=kubernetes.core.v1.PodTemplateSpecArgs(
 metadata=kubernetes.meta.v1.ObjectMetaArgs(
 labels={
 "app": "nginx",
 },
 ),
 spec=kubernetes.core.v1.PodSpecArgs(
 containers=[kubernetes.core.v1.ContainerArgs(
 name="nginx",
 image="nginx",
 ports=[kubernetes.core.v1.ContainerPortArgs(
 container_port=80,
 )],
 )],
 ),
 ),
 ))
nginx_service = kubernetes.core.v1.Service("nginx-service",
 metadata=kubernetes.meta.v1.ObjectMetaArgs(
 name="nginx",
 ),
 spec=kubernetes.core.v1.ServiceSpecArgs(
 type="LoadBalancer",
 selector={
 "app": "nginx",
 },
 ports=[kubernetes.core.v1.ServicePortArgs(
 protocol="TCP",
 port=80,
 )],
 ))

import (
 appsv1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/apps/v1"
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/core/v1"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/meta/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err := appsv1.NewDeployment(ctx, "nginx-deployment", &appsv1.DeploymentArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("nginx"),
 Labels: pulumi.StringMap{
 "app": pulumi.String("nginx"),
 },
 },
 Spec: &appsv1.DeploymentSpecArgs{
 Replicas: pulumi.Int(3),
 Selector: &metav1.LabelSelectorArgs{
 MatchLabels: pulumi.StringMap{
 "app": pulumi.String("nginx"),
 },
 },
 Template: &corev1.PodTemplateSpecArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Labels: pulumi.StringMap{
 "app": pulumi.String("nginx"),
 },
 },
 Spec: &corev1.PodSpecArgs{
 Containers: corev1.ContainerArray{
 &corev1.ContainerArgs{
 Name: pulumi.String("nginx"),
 Image: pulumi.String("nginx"),
 Ports: corev1.ContainerPortArray{
 &corev1.ContainerPortArgs{
 ContainerPort: pulumi.Int(80),
 },
 },
 },
 },
 },
 },
 },
 })
 if err != nil {
 return err
 }
 _, err = corev1.NewService(ctx, "nginx-service", &corev1.ServiceArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("nginx"),
 },
 Spec: &corev1.ServiceSpecArgs{
 Type: pulumi.String("LoadBalancer"),
 Selector: pulumi.StringMap{
 "app": pulumi.String("nginx"),
 },
 Ports: corev1.ServicePortArray{
 &corev1.ServicePortArgs{
 Protocol: pulumi.String("TCP"),
 Port: pulumi.Int(80),
 },
 },
 },
 })
 if err != nil {
 return err
 }
 return nil
 })
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Kubernetes = Pulumi.Kubernetes;

return await Deployment.RunAsync(() =>
{
 var nginxDeployment = new Kubernetes.Apps.V1.Deployment("nginx-deployment", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "nginx",
 Labels =
 {
 { "app", "nginx" },
 },
 },
 Spec = new Kubernetes.Types.Inputs.Apps.V1.DeploymentSpecArgs
 {
 Replicas = 3,
 Selector = new Kubernetes.Types.Inputs.Meta.V1.LabelSelectorArgs
 {
 MatchLabels =
 {
 { "app", "nginx" },
 },
 },
 Template = new Kubernetes.Types.Inputs.Core.V1.PodTemplateSpecArgs
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Labels =
 {
 { "app", "nginx" },
 },
 },
 Spec = new Kubernetes.Types.Inputs.Core.V1.PodSpecArgs
 {
 Containers = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.ContainerArgs
 {
 Name = "nginx",
 Image = "nginx",
 Ports = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.ContainerPortArgs
 {
 ContainerPortValue = 80,
 },
 },
 },
 },
 },
 },
 },
 });

 var nginxService = new Kubernetes.Core.V1.Service("nginx-service", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "nginx",
 },
 Spec = new Kubernetes.Types.Inputs.Core.V1.ServiceSpecArgs
 {
 Type = "LoadBalancer",
 Selector =
 {
 { "app", "nginx" },
 },
 Ports = new[]
 {
 new Kubernetes.Types.Inputs.Core.V1.ServicePortArgs
 {
 Protocol = "TCP",
 Port = 80,
 },
 },
 },
 });

});

import java.security.Provider.Service;
import java.util.Map;

import javax.naming.Context;

import com.pulumi.Pulumi;
import com.pulumi.kubernetes.apps_v1.Deployment;
import com.pulumi.kubernetes.apps_v1.DeploymentArgs;
import com.pulumi.kubernetes.apps_v1.inputs.DeploymentSpecArgs;
import com.pulumi.kubernetes.core_v1.ServiceArgs;
import com.pulumi.kubernetes.core_v1.inputs.PodSpecArgs;
import com.pulumi.kubernetes.core_v1.inputs.PodTemplateSpecArgs;
import com.pulumi.kubernetes.core_v1.inputs.ServiceSpecArgs;
import com.pulumi.kubernetes.meta_v1.inputs.LabelSelectorArgs;
import com.pulumi.kubernetes.meta_v1.inputs.ObjectMetaArgs;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 var nginxDeployment = new Deployment("nginxDeployment", DeploymentArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("nginx")
 .labels(Map.of("app", "nginx"))
 .build())
 .spec(DeploymentSpecArgs.builder()
 .replicas(3)
 .selector(LabelSelectorArgs.builder()
 .matchLabels(Map.of("app", "nginx"))
 .build())
 .template(PodTemplateSpecArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .labels(Map.of("app", "nginx"))
 .build())
 .spec(PodSpecArgs.builder()
 .containers(ContainerArgs.builder()
 .name("nginx")
 .image("nginx")
 .ports(ContainerPortArgs.builder()
 .containerPort(80)
 .build())
 .build())
 .build())
 .build())
 .build())
 .build());

 var nginxService = new Service("nginxService", ServiceArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("nginx")
 .build())
 .spec(ServiceSpecArgs.builder()
 .type("LoadBalancer")
 .selector(Map.of("app", "nginx"))
 .ports(ServicePortArgs.builder()
 .protocol("TCP")
 .port(80)
 .build())
 .build())
 .build());

 }
}

name: yaml-example
runtime: yaml
description: Deployment + Service
resources:
 nginx-deployment:
 type: kubernetes:apps/v1:Deployment
 properties:
 metadata:
 name: nginx
 labels:
 app: nginx
 spec:
 replicas: 3
 selector:
 matchLabels:
 app: nginx
 template:
 metadata:
 labels:
 app: nginx
 spec:
 containers:
 - name: nginx
 image: "nginx"
 ports:
 - containerPort: 80
 nginx-service:
 type: kubernetes:core/v1:Service
 properties:
 metadata:
 name: nginx
 spec:
 type: LoadBalancer
 selector:
 app: nginx
 ports:
 - protocol: TCP
 port: 80

We need to set some configuration before we deploy this stack. In this case, we need a kubeconfig to allow the pulumi-kubernetes provider to connect to our EKS cluster.

We import the eks-sandbox environment with the following stack configuration:

environment:
 - eks-sandbox

Now we update our stack, and see that our application gets deployed to the cluster, without needing to manually configure any credentials – ESC automatically injects them for us!

pulumi up --skip-preview
Updating (demo)
Type Name Status
+ pulumi:pulumi:Stack yaml-example-demo created (0.31s)
+ ├─ kubernetes:core/v1:Service nginx-service created (10s)
+ └─ kubernetes:apps/v1:Deployment nginx-deployment created (2s)
Resources:
+ 3 created
Duration: 13s

Now that the application is deployed, let’s take a look using kubectl:

$ pulumi env run eks-sandbox -- kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 3/3 3 3 103s

ESC environments can be used with the pulumi CLI using the pulumi env command.

Export Files

As we see in the ESC environment definition named eks-sandbox, there’s a files section (under values) for defining values that are automatically exported as temporary files when you run the environment. You can use this feature to generate configuration files to support your preferred tools. Pulumi exports each key-value pair as a temporary file containing the actual value, and an associated environment variable pointing to it.

Earlier we exported an environment variable named KUBECONFIG with a dynamic value based on a stack output. Here’s a similar example with the configuration declared statically:

values:
 files:
 KUBECONFIG: |
 apiVersion: v1
 kind: Config
 clusters:
 - cluster:
 server: https://127.0.0.1:6443
 name: docker-desktop

Conclusion

Pulumi ESC makes it easier than ever to tame infrastructure complexity. For this demo, we created two reusable ESC environments to configure AWS and Kubernetes credentials, created an EKS cluster, deployed a load-balanced application to the cluster, and configured our local toolchain to connect to the cluster. Since we factored the shared configuration into ESC environments, it is easy to extend this example to deploy additional AWS or Kubernetes resources. Pulumi ESC supports dynamic credentials using OIDC across AWS, Azure, and Google Cloud, with additional providers on the roadmap. Check out the following links to learn more about Pulumi ESC today!

Pulumi Kubernetes 4.0: Even More Kubernetes-Native

Levi Blackstone — Wed, 19 Jul 2023 16:00:00 -0400

Since the very earliest days of the Pulumi project, Kubernetes has been a core part of the Pulumi platform. The initial Pulumi Kubernetes provider supported the entire API surface area of the Kubernetes platform, derived directly and automatically from the Kubernetes OpenAPI specifications, and available to all of Pulumi’s familiar programming languages. Since then, we have offered day one support for every new Kubernetes version, added support for Helm, YAML, Kustomize and CRDs, added tools for converting to Pulumi (kube2pulumi and crd2pulumi) and delivered the Pulumi Kubernetes Operator. During that same time, Kubernetes usage has continued to expand within the ecosystem and among Pulumi users, with the Kubernetes provider growing from the fourth most used to the second most used provider on the platform.

We are excited to release the next major version of our Kubernetes provider - Pulumi Kubernetes 4.0.

Join us for a free, on-demand webinar with Pulumi and the CNCF to learn how to use Pulumi’s Kubernetes Provider and other open-source tools to build internal developer platforms. Register →

Pulumi Kubernetes 4.0 uses Kubernetes Server-Side Apply by default, supports upserting on all resources, and enables new Patch resources for every resource type in the provider. It also brings improved diffs, removes dependence on kubectl annotations, and provides simpler access to outputs in the Python, Go and Java SDKs, supporting any self-hosted or managed Kubernetes cluster with version 1.13 or newer.

Together, these additions make Pulumi’s support for Kubernetes even more Kubernetes-native, allowing the best of both worlds between Kubernetes’ flexible resource model and Pulumi’s robust desired state infrastructure as code model, all while using your favorite languages (TypeScript, Python, Go, .NET and Java) to define and manage your Kubernetes Infrastructure. Pulumi programs that need to manage Kubernetes resources that are also modified by other systems (external controllers, managed Kubernetes service control planes, kubectl or other IaC solutions) can faithfully represent the modifications they want to make to existing resources within the Pulumi resource lifecycle. Pulumi accurately detects any differences between the current state of resources in the cluster, and the desired state specified by Pulumi IaC.

Server-Side Apply, Upserts and Patch Resources

Kubernetes Server-side Apply (SSA) mode is now default in the v4 provider. SSA mode enables several useful new features: Kubernetes resources can be safely managed by more than one controller. Any conflicting fields will produce a detailed error message which avoids accidentally overriding changes made by other controllers. Resources can be “upserted”; in other words, create the resource if it does not exist, or apply the specified configuration to an existing resource. Patch resources add a new way to manage shared Kubernetes resources. Patch resources can be used to make targeted changes, and do not require the entire resource to be specified. This is a convenient way to update shared resources; a common example is resources that are provisioned by the cloud provider rather than as part of the Pulumi stack. Deleting a Patch resource undoes the patched change, but does not remove the shared Kubernetes resource. See the SSA how-to guide for more details and usage examples for SSA mode.

Note: Users may opt into the previous default Client-side Apply (CSA) mode by setting the enableServerSideApply provider configuration to “false”.

In this example, we define a ConfigMap as an upsert which will create the resource if it doesn’t exist.

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";

const example = new kubernetes.core.v1.ConfigMap("example", {
 metadata: {
 annotations: {
 "pulumi.com/patchForce": "true",
 },
 name: "example",
 },
 data: {
 foo: "bar",
 },
});

import pulumi
import pulumi_kubernetes as kubernetes

example = kubernetes.core.v1.ConfigMap(
 "example",
 metadata=kubernetes.meta.v1.ObjectMetaArgs(
 annotations={
 "pulumi.com/patchForce": "true",
 },
 name="example",
 ),
 data={
 "foo": "bar",
 })

package main

import (
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/core/v1"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/meta/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err = corev1.NewConfigMap(ctx, "example", &corev1.ConfigMapArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Annotations: pulumi.StringMap{
 "pulumi.com/patchForce": pulumi.String("true"),
 },
 Name: pulumi.String("example"),
 },
 Data: pulumi.StringMap{
 "foo": pulumi.String("bar"),
 },
 })
 if err != nil {
 return err
 }
 return nil
 })
}

using System.Collections.Generic;
using Pulumi;
using Kubernetes = Pulumi.Kubernetes;

return await Deployment.RunAsync(() =>
{
 var example = new Kubernetes.Core.V1.ConfigMap("example", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Annotations =
 {
 { "pulumi.com/patchForce", "true" },
 },
 Name = "example",
 },
 Data =
 {
 { "foo", "bar" },
 },
 });
});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.kubernetes;
import com.pulumi.kubernetes.core_v1.ConfigMap;
import com.pulumi.kubernetes.core_v1.ConfigMapArgs;
import com.pulumi.kubernetes.meta_v1.inputs.ObjectMetaArgs;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 var example = new ConfigMap("example", ConfigMapArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .annotations(Map.of("pulumi.com/patchForce", "true"))
 .name("example")
 .build())
 .data(Map.of("foo", "bar"))
 .build());

 }
}

resources:
 example:
 type: kubernetes:core/v1:ConfigMap
 properties:
 metadata:
 annotations:
 pulumi.com/patchForce: "true"
 name: example
 data:
 foo: bar

Improved diff experience

The process for calculating what changes a provider needs to make to achieve the desired state is called “diffing”. This is a complex process that accounts for the current and previous state of the Pulumi program (the “inputs”), the stack’s state/checkpoint, and the actual current state of any resources managed by the provider. A major challenge of this process is detecting “drift”, where resources have been changed by something other than the Pulumi stack so that these changes are not reflected in Pulumi’s view of the world. The Server-side Apply (SSA) support available in v3.x versions does detect resource drift, but the implementation was slow for larger stacks. The v4 provider now detects resource drift in both Client-side Apply and Server-side Apply by running a refresh operation. This brings the behavior in line with other providers, and offers consistent performance that scales well to larger stacks.

As part of this work, we removed the use of the kubectl.kubernetes.io/last-applied-configuration annotation, which was previously used by the provider to track input values for Kubernetes resources. This change addresses many longstanding issues caused by the use of this annotation, including challenges working with resources that were modified by tools other than kubectl.

Migrating to Kubernetes 4.0

You can migrate your existing Pulumi Kubernetes projects to 4.0 today, or start a new project targeting 4.0 with any of the Pulumi templates.

For users migrating to 4.0, check out the v4 Provider Migration Guide with full details of the additional changes in this release.

Getting Started

Try Infrastructure as Code for Kubernetes by following this quick getting started guide.

Join us for a free, on-demand webinar with Pulumi and the CNCF to learn how to use Pulumi’s Kubernetes Provider and other open-source tools to build internal developer platforms.Register →

Manage Shared Kubernetes Resources Safely with Pulumi

Levi Blackstone — Wed, 27 Jul 2022 00:00:00 +0000

Kubernetes resources often have more than one controller making changes to them. These controllers can include kubectl, the Kubernetes control plane, custom operators, or infrastructure as code (IaC) tools like Pulumi. With the v3.20.1 release of the Kubernetes provider, you have some powerful new options for managing shared resources in Kubernetes. In this post, we show you how Pulumi can help you work with shared resources safely and effectively.

We first launched the Pulumi Kubernetes provider in 2018, and it has grown to be one of the most used providers across the Pulumi ecosystem. Many of our customers rely on this provider to manage critical production workloads, and a common request was for an easier way to manage shared Kubernetes resources with Pulumi. We’ve been listening, and are grateful to everyone who took the time to share their feedback with us!

A lot has changed since 2018, and our provider continues to evolve to support new use cases and the ever-growing richness of the Kubernetes ecosystem. One such development is Server-Side Apply (SSA), which is a resource management strategy that was introduced in Kubernetes v1.18. Kubernetes clients that use SSA can safely share the management of Kubernetes resources by making the API Server responsible for computing diffs and resolving conflicts.

We have kept a close eye on the SSA feature since it was announced and have been waiting patiently for its broad availability in production clusters. The wait is over, and we are excited to bring the power of SSA to Pulumi users!

What’s new in Pulumi’s Kubernetes provider?

Our Server-Side Apply support adds several new capabilities:

New Patch resource types corresponding to every Kubernetes resource kind. (e.g., NamespacePatch, DeploymentPatch)
“Upsert” support; create a resource if it does not exist, or update the existing resource with specified changes.
Diffs no longer depend on the last-applied-configuration annotation, which fixes a number of open issues in the provider.

For additional details on each of these changes, see the how-to guide for Server-Side Apply.

Now, let’s take a look at some practical examples that take advantage of these improvements.

Adding labels to an existing Namespace

Some Kubernetes resources, like Namespaces, are shared by all users of the cluster. Users often want to edit these resources, perhaps by adding a new label to them. In this example, we add a label to the kube-public Namespace using the new NamespacePatch resource:

Multiple Patch resources can modify the same Kubernetes resource.

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";

const provider = new kubernetes.Provider("provider", {enableServerSideApply: true});
const blogLabel = new kubernetes.core.v1.NamespacePatch("blog-label", {metadata: {
 name: "kube-public",
 labels: {
 "pulumi.com/blog-example": "true",
 },
 }}, {
 provider: provider,
});

import pulumi
import pulumi_kubernetes as kubernetes

provider = kubernetes.Provider("provider", enable_server_side_apply=True)
blog_label = kubernetes.core.v1.NamespacePatch("blog-label", metadata=kubernetes.meta.v1.ObjectMetaPatchArgs(
 name="kube-public",
 labels={
 "pulumi.com/blog-example": "true",
 },
), opts=pulumi.ResourceOptions(provider=provider))

package main

import (
 "github.com/pulumi/pulumi-kubernetes/sdk/v3/go/kubernetes"
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v3/go/kubernetes/core/v1"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v3/go/kubernetes/meta/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 provider, err := kubernetes.NewProvider(ctx, "provider", &kubernetes.ProviderArgs{
 EnableServerSideApply: pulumi.Bool(true),
 })
 if err != nil {
 return err
 }
 _, err = corev1.NewNamespacePatch(ctx, "blog-label", &corev1.NamespacePatchArgs{
 Metadata: &metav1.ObjectMetaPatchArgs{
 Name: pulumi.String("kube-public"),
 Labels: pulumi.StringMap{
 "pulumi.com/blog-example": pulumi.String("true"),
 },
 },
 }, pulumi.Provider(provider))
 if err != nil {
 return err
 }
 return nil
 })
}

using System.Collections.Generic;
using Pulumi;
using Kubernetes = Pulumi.Kubernetes;

return await Deployment.RunAsync(() =>
{
 var provider = new Kubernetes.Provider("provider", new()
 {
 EnableServerSideApply = true,
 });

 var blogLabel = new Kubernetes.Core.V1.NamespacePatch("blog-label", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaPatchArgs
 {
 Name = "kube-public",
 Labels =
 {
 { "pulumi.com/blog-example", "true" },
 },
 },
 }, new CustomResourceOptions
 {
 Provider = provider,
 });

});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.kubernetes;
import com.pulumi.ProviderArgs;
import com.pulumi.kubernetes.core_v1.NamespacePatch;
import com.pulumi.kubernetes.core_v1.NamespacePatchArgs;
import com.pulumi.kubernetes.meta_v1.inputs.ObjectMetaPatchArgs;
import com.pulumi.resources.CustomResourceOptions;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
public static void main(String[] args) {
Pulumi.run(App::stack);
}

 public static void stack(Context ctx) {
 var provider = new Provider("provider", ProviderArgs.builder()
 .enableServerSideApply(true)
 .build());

 var blogLabel = new NamespacePatch("blogLabel", NamespacePatchArgs.builder()
 .metadata(ObjectMetaPatchArgs.builder()
 .name("kube-public")
 .labels(Map.of("pulumi.com/blog-example", "true"))
 .build())
 .build(), CustomResourceOptions.builder()
 .provider(provider)
 .build());

 }
}

resources:
 provider:
 type: pulumi:providers:kubernetes
 properties:
 enableServerSideApply: true
 blog-label:
 type: kubernetes:core/v1:NamespacePatch
 properties:
 metadata:
 name: kube-public
 labels:
 pulumi.com/blog-example: "true"
 options:
 provider: ${provider}

After running an update with this program, the new label is applied to the shared Namespace.

$ pulumi up --skip-preview
Updating (dev)
View Live: https://app.pulumi.com/lblackstone/ssa/dev/updates/1
Type Name Status
+ pulumi:pulumi:Stack ssa-dev created
+ ├─ pulumi:providers:kubernetes provider created
+ └─ kubernetes:core/v1:NamespacePatch blog-label created
Resources:
+ 3 created
Duration: 2s
$ kubectl get ns kube-public -o yaml
apiVersion: v1
kind: Namespace
metadata:
creationTimestamp: "2022-05-17T22:44:22Z"
labels:
kubernetes.io/metadata.name: kube-public
pulumi.com/blog-example: "true"
name: kube-public
resourceVersion: "1571522"
uid: 1096ae2b-925e-4ac7-9b26-9cc6b7c6c4cb
spec:
finalizers:
- kubernetes
status:
phase: Active

Change the default StorageClass

If you want to change the default StorageClass for a cluster, the upstream documentation suggests using a kubectl patch command to make this change. We can accomplish the same thing declaratively with StorageClassPatch. In this case, we want to override any existing setting for this value, so we use the pulumi.com/patchForce annotation to overwrite the existing annotation. If the annotation does not already exist, then it will be created.

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";

const provider = new kubernetes.Provider("provider", {enableServerSideApply: true});
const storageClassHostpath = new kubernetes.storage.v1.StorageClassPatch("storage-class-hostpath", {metadata: {
 name: "hostpath",
 annotations: {
 "pulumi.com/patchForce": "true",
 "storageclass.kubernetes.io/is-default-class": "false",
 },
 }}, {
 provider: provider,
});
const storageClassPremium = new kubernetes.storage.v1.StorageClassPatch("storage-class-premium", {metadata: {
 name: "premium",
 annotations: {
 "pulumi.com/patchForce": "true",
 "storageclass.kubernetes.io/is-default-class": "true",
 },
 }}, {
 provider: provider,
});

import pulumi
import pulumi_kubernetes as kubernetes

provider = kubernetes.Provider("provider", enable_server_side_apply=True)
storage_class_hostpath = kubernetes.storage.v1.StorageClassPatch("storage-class-hostpath", metadata=kubernetes.meta.v1.ObjectMetaPatchArgs(
 name="hostpath",
 annotations={
 "pulumi.com/patchForce": "true",
 "storageclass.kubernetes.io/is-default-class": "false",
 },
), opts=pulumi.ResourceOptions(provider=provider))
storage_class_premium = kubernetes.storage.v1.StorageClassPatch("storage-class-premium", metadata=kubernetes.meta.v1.ObjectMetaPatchArgs(
 name="premium",
 annotations={
 "pulumi.com/patchForce": "true",
 "storageclass.kubernetes.io/is-default-class": "true",
 },
), opts=pulumi.ResourceOptions(provider=provider))

package main

import (
 "github.com/pulumi/pulumi-kubernetes/sdk/v3/go/kubernetes"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v3/go/kubernetes/meta/v1"
 storagev1 "github.com/pulumi/pulumi-kubernetes/sdk/v3/go/kubernetes/storage/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 provider, err := kubernetes.NewProvider(ctx, "provider", &kubernetes.ProviderArgs{
 EnableServerSideApply: pulumi.Bool(true),
 })
 if err != nil {
 return err
 }
 _, err = storagev1.NewStorageClassPatch(ctx, "storage-class-hostpath", &storagev1.StorageClassPatchArgs{
 Metadata: &metav1.ObjectMetaPatchArgs{
 Name: pulumi.String("hostpath"),
 Annotations: pulumi.StringMap{
 "pulumi.com/patchForce": pulumi.String("true"),
 "storageclass.kubernetes.io/is-default-class": pulumi.String("false"),
 },
 },
 }, pulumi.Provider(provider))
 if err != nil {
 return err
 }
 _, err = storagev1.NewStorageClassPatch(ctx, "storage-class-premium", &storagev1.StorageClassPatchArgs{
 Metadata: &metav1.ObjectMetaPatchArgs{
 Name: pulumi.String("premium"),
 Annotations: pulumi.StringMap{
 "pulumi.com/patchForce": pulumi.String("true"),
 "storageclass.kubernetes.io/is-default-class": pulumi.String("true"),
 },
 },
 }, pulumi.Provider(provider))
 if err != nil {
 return err
 }
 return nil
 })
}

using System.Collections.Generic;
using Pulumi;
using Kubernetes = Pulumi.Kubernetes;

return await Deployment.RunAsync(() =>
{
 var provider = new Kubernetes.Provider("provider", new()
 {
 EnableServerSideApply = true,
 });

 var storageClassHostpath = new Kubernetes.Storage.V1.StorageClassPatch("storage-class-hostpath", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaPatchArgs
 {
 Name = "hostpath",
 Annotations =
 {
 { "pulumi.com/patchForce", "true" },
 { "storageclass.kubernetes.io/is-default-class", "false" },
 },
 },
 }, new CustomResourceOptions
 {
 Provider = provider,
 });

 var storageClassPremium = new Kubernetes.Storage.V1.StorageClassPatch("storage-class-premium", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaPatchArgs
 {
 Name = "premium",
 Annotations =
 {
 { "pulumi.com/patchForce", "true" },
 { "storageclass.kubernetes.io/is-default-class", "true" },
 },
 },
 }, new CustomResourceOptions
 {
 Provider = provider,
 });
});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.kubernetes.Provider;
import com.pulumi.kubernetes.ProviderArgs;
import com.pulumi.kubernetes.storage.k8s.io_v1.StorageClassPatch;
import com.pulumi.kubernetes.storage.k8s.io_v1.StorageClassPatchArgs;
import com.pulumi.kubernetes.meta_v1.inputs.ObjectMetaPatchArgs;
import com.pulumi.resources.CustomResourceOptions;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 var provider = new Provider("provider", ProviderArgs.builder()
 .enableServerSideApply(true)
 .build());

 var storageClassHostpath = new StorageClassPatch("storageClassHostpath", StorageClassPatchArgs.builder()
 .metadata(ObjectMetaPatchArgs.builder()
 .name("hostpath")
 .annotations(Map.of("pulumi.com/patchForce", "true"))
 .annotations(Map.of("storageclass.kubernetes.io/is-default-class", "false"))
 .build())
 .build(), CustomResourceOptions.builder()
 .provider(provider)
 .build());

 var storageClassPremium = new StorageClassPatch("storageClassPremium", StorageClassPatchArgs.builder()
 .metadata(ObjectMetaPatchArgs.builder()
 .name("premium")
 .annotations(Map.of("pulumi.com/patchForce", "true"))
 .annotations(Map.of("storageclass.kubernetes.io/is-default-class", "true"))
 .build())
 .build(), CustomResourceOptions.builder()
 .provider(provider)
 .build());

 }
}

resources:
 provider:
 type: pulumi:providers:kubernetes
 properties:
 enableServerSideApply: true
 storage-class-hostpath:
 type: kubernetes:storage.k8s.io/v1:StorageClassPatch
 properties:
 metadata:
 name: hostpath
 annotations:
 pulumi.com/patchForce: "true"
 storageclass.kubernetes.io/is-default-class: "false"
 options:
 provider: ${provider}
 storage-class-premium:
 type: kubernetes:storage.k8s.io/v1:StorageClassPatch
 properties:
 metadata:
 name: premium
 annotations:
 pulumi.com/patchForce: "true"
 storageclass.kubernetes.io/is-default-class: "true"
 options:
 provider: ${provider}

Many applications use ConfigMap resources to configure shared data. Consider the case where more than one application depends on the same ConfigMap, but does not use all the same keys. Each application requires that the same ConfigMap exists, but may need to update rather than create it (also known as an “Upsert” operation). With Server-Side Apply support, this doesn’t require any special handling. Create a ConfigMap resource with the desired configuration, and the changes will be applied as required. Again, in the case of conflicts, we can set the pulumi.com/patchForce annotation to force an override for conflicting values.

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";

const provider = new kubernetes.Provider("provider", {enableServerSideApply: true});
const sharedConfig = new kubernetes.core.v1.ConfigMap("shared-config", {
 metadata: {
 name: "app-settings",
 },
 data: {
 backendUrl: "https://example.com",
 },
}, {
 provider: provider,
});

import pulumi
import pulumi_kubernetes as kubernetes

provider = kubernetes.Provider("provider", enable_server_side_apply=True)
shared_config = kubernetes.core.v1.ConfigMap("shared-config",
 metadata=kubernetes.meta.v1.ObjectMetaArgs(
 name="app-settings",
 ),
 data={
 "backendUrl": "https://example.com",
 },
 opts=pulumi.ResourceOptions(provider=provider))

package main

import (
 "github.com/pulumi/pulumi-kubernetes/sdk/v3/go/kubernetes"
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v3/go/kubernetes/core/v1"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v3/go/kubernetes/meta/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 provider, err := kubernetes.NewProvider(ctx, "provider", &kubernetes.ProviderArgs{
 EnableServerSideApply: pulumi.Bool(true),
 })
 if err != nil {
 return err
 }
 _, err = corev1.NewConfigMap(ctx, "shared-config", &corev1.ConfigMapArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("app-settings"),
 },
 Data: pulumi.StringMap{
 "backendUrl": pulumi.String("https://example.com"),
 },
 }, pulumi.Provider(provider))
 if err != nil {
 return err
 }
 return nil
 })
}

using System.Collections.Generic;
using Pulumi;
using Kubernetes = Pulumi.Kubernetes;

return await Deployment.RunAsync(() =>
{
 var provider = new Kubernetes.Provider("provider", new()
 {
 EnableServerSideApply = true,
 });

 var sharedConfig = new Kubernetes.Core.V1.ConfigMap("shared-config", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "app-settings",
 },
 Data =
 {
 { "backendUrl", "https://example.com" },
 },
 }, new CustomResourceOptions
 {
 Provider = provider,
 });
});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.kubernetes.Provider;
import com.pulumi.kubernetes.ProviderArgs;
import com.pulumi.kubernetes.core_v1.ConfigMap;
import com.pulumi.kubernetes.core_v1.ConfigMapArgs;
import com.pulumi.kubernetes.meta_v1.inputs.ObjectMetaArgs;
import com.pulumi.resources.CustomResourceOptions;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 var provider = new Provider("provider", ProviderArgs.builder()
 .enableServerSideApply(true)
 .build());

 var sharedConfig = new ConfigMap("sharedConfig", ConfigMapArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("app-settings")
 .build())
 .data(Map.of("backendUrl", "https://example.com"))
 .build(), CustomResourceOptions.builder()
 .provider(provider)
 .build());

 }
}

resources:
 provider:
 type: pulumi:providers:kubernetes
 properties:
 enableServerSideApply: true
 shared-config:
 type: kubernetes:core/v1:ConfigMap
 properties:
 metadata:
 name: app-settings
 data:
 backendUrl: https://example.com
 options:
 provider: ${provider}

Here is what it looks like to update an existing ConfigMap with this program.

$ kubectl get configmap app-settings -o yaml
apiVersion: v1
data:
appReplicas: "3"
kind: ConfigMap
metadata:
creationTimestamp: "2022-07-26T22:25:16Z"
name: app-settings
namespace: default
resourceVersion: "1603792"
uid: b366e6a5-703e-4897-ab5d-a99b19ba82d5
$ pulumi up --skip-preview
Updating (dev)
View Live: https://app.pulumi.com/lblackstone/pulumi-k8s-test/dev/updates/42
Type Name Status
+ pulumi:pulumi:Stack pulumi-k8s-test-dev created
+ ├─ pulumi:providers:kubernetes provider created
+ └─ kubernetes:core/v1:ConfigMap shared-config created
Resources:
+ 3 created
Duration: 2s
kubectl get configmap app-settings -o yaml
apiVersion: v1
data:
appReplicas: "3"
backendUrl: https://example.com
kind: ConfigMap
metadata:
creationTimestamp: "2022-07-26T22:25:16Z"
name: app-settings
namespace: default
resourceVersion: "1603817"
uid: b366e6a5-703e-4897-ab5d-a99b19ba82d5

Use the retainOnDelete option if you don’t want to delete the shared resource when a stack is destroyed.

Give it a try!

To get started with these improvements, check out our how-to guide for Server-Side Apply. If you run into any issues, you can reach out to us on Community Slack with your questions on the #kubernetes channel, or file an issue on the pulumi-kubernetes repo.

Create Amazon EKS clusters in your favorite language

Levi Blackstone — Thu, 04 Feb 2021 00:00:00 +0000

Check out version 3.0 of the Pulumi EKS Provider.

Pulumi’s infrastructure as code tooling combines the programming languages and tools you already know with the full power of cloud infrastructure. But until now, some Pulumi components for cloud infrastructure, like our popular EKS package for Amazon’s Elastic Kubernetes Service, were only available in a subset of the languages supported by Pulumi.

Now, you can use the EKS package–previously only available for TypeScript–in all four Pulumi languages: TypeScript, Python, .NET, and Go. Regardless of the language you choose, you can manage EKS clusters with Pulumi, starting with the v0.22.0 release. Check out our Modern Infrastructure Wednesday video to see it in action:

Making components available in every language

Pulumi’s EKS package is made up of Pulumi components. A component is a collection of Pulumi resources that provides an easy way to package best practices for using cloud resources, even if those best pratices span individual resources or even entire cloud providers. Until now though, components could only be used in Pulumi programs written in the same language as the component.

To change that, we’re creating “multi-language components”. With multi-language components, anyone will be able to author a component in one language and automatically compile it to all other languages supported by Pulumi. This means anyone in the worldwide Pulumi community will be able to take advantage of any component, even if it’s written in a different language. Right now, unless a component author is willing to write code in all four Pulumi languages, their component is only able to reach Pulumi users using the same language SDK.

In addition to expanding the library of publicly available components, this work will also enable organizations to write Pulumi components in the right programming language, or languages, for each team and each product. Infrastructure or platform teams will be able to write components in Go and share them with a webapp team using TypeScript, write components in .NET and share them with a machine learning team using Python, and any other combination that suits their business.

The next step in the story is a seamless authoring experience that enables component creators to write, build, and publish multi-language components. That will be in public preview soon, and we’re excited for you to try it out. If you’d like, subscribe to the multi-language components issue for updates.

Next steps

If you haven’t yet, try out the new pulumi-eks package with this walkthrough.

Easily Create and Manage AWS EKS Kubernetes Clusters

Luke Hoban — Thu, 28 Jan 2021 00:00:00 +0000

Provisioning, managing, and monitoring a Kubernetes cluster is not easy. AWS now offers EKS to reduce that burden – but it’s still difficult to get up and running. Pulumi’s infrastructure as code SDKs can help! We can provision an entire EKS cluster with a single CLI command, thanks to the @pulumi/eks package. Let’s see how.

Updated January 2021 to include Python, .NET, and Go support.

Updated October 2024 to Version 3.0

To get started, download the free and open source Pulumi SDK, which includes the pulumi CLI we’ll use below. If you don’t have the AWS CLI installed and configured, set that up too. You’ll need AWS credentials to access your AWS account and provision resources.

From here, there are two ways to proceed:

Provision a new cluster with a single CLI command
Use infrastructure as code to manage clusters … including Kubernetes resources themselves!

Installing Amazon EKS with Pulumi

With the Pulumi CLI installed and configured, a single command will provision your cluster:

$ pulumi new https://github.com/pulumi/apps/eks

This template will walk you through configuring a few settings such as the region to deploy, the instance type, and number of cluster nodes.

The CLI will first show you a preview of the AWS and Kubernetes resources that will be deployed to create the cluster, before actually performing any operation. Once you have reviewed and selected “Yes,” the cluster will begin deploying. The terminal output will look like this:

Following these steps will provide a complete managed Amazon EKS cluster, configured similarly to the steps outlined in the Amazon EKS Getting Started guide, including optional support for Amazon EBS-backed StorageClasses and access to the Kubernetes Dashboard. Pulumi replaces all of these tedious, manual steps with a straightforward description of your desired state, and then makes it so!

Once the deployment is complete – it will take 10-20 minutes for AWS to provision these resources – you can grab the kubeconfig.json needed to work with this cluster and connect to it with kubectl:

$ pulumi stack output kubeconfig > kubeconfig.json
$ KUBECONFIG=./kubeconfig.json kubectl get nodes

Reusable AWS EKS component

The Amazon EKS installer we just saw is built on top of a reusable eks.Cluster component that is available in Pulumi’s infrastructure as code solution. In fact, the command above created an instance of this component with reasonable defaults. If you prefer, you can do it by hand and programmatically. Here are examples in each of Pulumi’s supported languages:

import * as eks from "@pulumi/eks";

// Create an EKS cluster.
const cluster = new eks.Cluster("cluster", {
 instanceType: "t2.medium",
 desiredCapacity: 2,
 minSize: 1,
 maxSize: 2,
});

// Export the cluster's kubeconfig.
export const kubeconfig = cluster.kubeconfig;

import pulumi
import pulumi_eks as eks

# Create an EKS cluster.
cluster = eks.Cluster(
 "cluster",
 instance_type="t2.medium",
 desired_capacity=2,
 min_size=1,
 max_size=2,
)

# Export the cluster's kubeconfig.
pulumi.export("kubeconfig", cluster.kubeconfig)

using System;
using System.Threading.Tasks;
using Pulumi;
using Pulumi.Eks.Cluster;

class EksStack : Stack
{
 public EksStack()
 {
 // Create an EKS cluster.
 var cluster = new Cluster("cluster", new ClusterArgs
 {
 InstanceType = "t2.medium",
 DesiredCapacity = 2,
 MinSize = 1,
 MaxSize = 2,
 });

 // Export the cluster's kubeconfig.
 this.Kubeconfig = cluster.Kubeconfig;
 }

 [Output("kubeconfig")]
 public Output<string> Kubeconfig { get; set; }
}

class Program
{
 static Task<int> Main(string[] args) => Deployment.RunAsync<EksStack>();
}

package main

import (
 "github.com/pulumi/pulumi-eks/sdk/go/eks/cluster"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // Create an EKS cluster.
 cluster, err := cluster.NewCluster(ctx, "cluster",
 cluster.ClusterArgs{
 InstanceType: pulumi.String("t2.medium"),
 DesiredCapacity: pulumi.Int(2),
 MinSize: pulumi.Int(1),
 MaxSize: pulumi.Int(2),
 },
 )
 if err != nil {
 return err
 }

 // Export the cluster's kubeconfig.
 ctx.Export("kubeconfig", cluster.Kubeconfig)

 return nil
 })
}

Adding resources, and deploying Helm charts

You can extend or modify this program to change the details of the Amazon EKS cluster, or to install additional AWS or Kubernetes resources to associate with the cluster. For example, if applications running in the cluster need access to an Amazon S3 bucket, create a new Bucket resource:

const bucket = new aws.s3.Bucket("assets");

bucket = aws.s3.Bucket("assets")

var bucket = new Aws.S3.Bucket("bucket");

bucket, err := s3.NewBucket(ctx, "bucket", &s3.BucketArgs{})

Or, to install Kubernetes applications automatically into the Amazon EKS cluster (such as WordPress), add them using the Pulumi Kubernetes provider pointing at the newly provisioned EKS cluster:

const wordpress = new k8s.helm.v3.Chart("wpdev", {
 version: "9.6.0",
 chart: "wordpress",
 fetchOpts: {
 repo: "https://charts.bitnami.com/bitnami",
 },
}, { providers: { kubernetes: cluster.provider }});

wordpress = Chart(
 "wpdev",
 ChartOpts(
 chart="wordpress",
 version="9.6.0",
 fetch_opts=FetchOpts(
 repo="https://charts.bitnami.com/bitnami",
 ),
 ),
 opts=ResourceOptions(provider=cluster.provider),
)

var wordpress = new Chart("wpdev", new ChartArgs
{
 Chart = "wordpress",
 Version = "9.6.0",
 FetchOptions = new ChartFetchArgs
 {
 Repo = "https://charts.bitnami.com/bitnami"
 }
}, new CustomResourceOptions
{
 Provider = cluster.Provider,
});

wordpress, err := helm.NewChart(ctx, "wpdev", helm.ChartArgs{
 Chart: pulumi.String("wordpress"),
 Version: pulumi.String("9.6.0"),
 FetchArgs: helm.FetchArgs{
 repo: pulumi.String("https://charts.bitnami.com/bitnami"),
 },
}, pulumi.Provider(cluster.Provider))

Pulumi makes it possible to define the cluster, AWS resources, and Kubernetes objects needed to bootstrap your entire Kubernetes environment, all in one place. They can then be versioned together and exact replicas of the environment can be created independently for operations such as testing or disaster recovery.

As usual, by expressing these resources in code instead of in YAML, you get many software engineering benefits: improved tooling, the ability to refactor using your favorite IDE, the ability to create and share reusable components, and strong typing to validate correctness up front.

What’s happening under the hood?

To make this whole process buttery smooth, the Pulumi EKS component takes care of setting up all the pieces needed for an Amazon EKS cluster:

Creating an EKS Service Role
Optionally creating a new Virtual Private Cloud (VPC)
Creating an Amazon EKS cluster
Configuring a Kubernetes provider with access to the Amazon EKS cluster
Launching worker nodes in an Auto Scaling group to join the cluster
Installing a ConfigMap into the cluster to allow new Amazon EC2 worker nodes to join
Installing a StorageClass into the cluster to allow provisioning Amazon EBS-backed PersistentVolumes
Constructing the kubeconfig needed to access the cluster
Optionally installing additional Kubernetes YAML and/or Helm charts

This involves steps that provision resources in AWS, steps that provision resources in Kubernetes, and other custom computation to coordinate these steps. In addition, during the deployment Pulumi also provides complex coordination between these different steps.

Managing Pulumi Deployments

Once you have deployed your Amazon EKS cluster and Kubernetes resources to AWS, you can manage the deployment on app.pulumi.com, including deep links into the AWS console and Kubernetes dashboard to get real-time insights on your cluster.

All of the provisioned resources across AWS and Kubernetes are available from this single view, along with an audit history of deployments to this stack. The stack exports are also published in this same dashboard, so that key outputs, like your kubeconfig, are easily available and can be shared with other users in your organization.

Additional management features, including RBAC, CI/CD integrations, and rich resource graph visualization, are also available via the Pulumi app, including Team and Enterprise Editions for production teams of all sizes.

Get Started

We’ve seen how Pulumi takes care of the heavy lifting with AWS EKS so that you don’t have to. You can use Pulumi to easily deploy new clusters, managed AWS resources, and Kubernetes apps, all with the same powerful toolchain. Support is also available for Azure AKS, Google GKE, DigitalOcean’s Managed Kubernetes, and custom clusters (including Minikube).

For more information:

Introducing kube2pulumi: No More YAML

Levi Blackstone — Wed, 12 Aug 2020 00:00:00 +0000

Kubernetes users often joke about being “YAML engineers,” and the pile of YAML seems to get deeper every day. Today, we’re pleased to announce kube2pulumi, a tool to automatically convert Kubernetes manifests into modern code! Instead of manipulating YAML directly, you can take advantage of the rich ecosystem of programming language tools to supercharge your productivity.

Using programming languages to manage Kubernetes resources has many benefits: strong typing, linting, auto-completion, refactoring, testing, and the ability to use standard language features like variables, loops, conditions, and abstraction. Pulumi also supports Helm, Kustomize, and YAML, but these tools don’t take full advantage of the programming language. With kube2pulumi, you can realize these benefits without manual conversion work upfront.

If you want to try it out right now in your browser, head over to our Kubernetes to Pulumi Converter!

Otherwise, keep reading as we check out an example of kube2pulumi in action, and then take a closer look at the generated code.

Getting Started with kube2pulumi

Let’s start with a simple example: converting a Namespace. Here’s the YAML manifest:

apiVersion: v1
kind: Namespace
metadata:
 name: foo

kube2pulumi will generate any Pulumi-supported language, so choose whichever one you like!

Here, we choose TypeScript and select our YAML manifest.

$ kube2pulumi typescript -f namespace.yaml
Conversion successful! Generated File: namespace.ts

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";

const fooNamespace = new kubernetes.core.v1.Namespace("fooNamespace", {
 apiVersion: "v1",
 kind: "Namespace",
 metadata: {
 name: "foo",
 },
});

Here, we choose Python and select our YAML manifest.

$ kube2pulumi python -f namespace.yaml
Conversion successful! Generated File: `__main__.py`

import pulumi
import pulumi_kubernetes as kubernetes

foo_namespace = kubernetes.core.v1.Namespace("fooNamespace",
 api_version="v1",
 kind="Namespace",
 metadata={
 "name": "foo",
 })

Here, we choose C# and select our YAML manifest.

$ kube2pulumi csharp -f namespace.yaml
Conversion successful! Generated File: Program.cs

using Pulumi;
using Kubernetes = Pulumi.Kubernetes;

class MyStack : Stack
{
 public MyStack()
 {
 var fooNamespace = new Kubernetes.Core.V1.Namespace("fooNamespace", new Kubernetes.Types.Inputs.Core.V1.NamespaceArgs
 {
 ApiVersion = "v1",
 Kind = "Namespace",
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "foo",
 },
 });
 }

}

Here, we choose Go and select our YAML manifest.

$ kube2pulumi go -f namespace.yaml
Conversion successful! Generated File: main.go

package main

import (
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/core/v1"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/meta/v1"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err := corev1.NewNamespace(ctx, "fooNamespace", &corev1.NamespaceArgs{
 ApiVersion: pulumi.String("v1"),
 Kind: pulumi.String("Namespace"),
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("foo"),
 },
 })
 if err != nil {
 return err
 }
 return nil
 })
}

Multi-resource Example

That’s pretty cool but doesn’t show off the full power of kube2pulumi. Let’s try a more realistic conversion scenario. Our newly announced Pulumi Kubernetes Operator requires a few different resources to install: a Service Account, Role, RoleBinding, and Deployment, totaling about 130 lines of YAML. Let’s try converting that directory to some real code!

$ ls operator
operator.yaml role.yaml role_binding.yaml service_account.yaml

$ kube2pulumi typescript -d operator
Conversion successful! Generated File: operator/main.ts

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";

const pulumi_kubernetes_operatorDeployment = new kubernetes.apps.v1.Deployment("pulumi_kubernetes_operatorDeployment", {
 apiVersion: "apps/v1",
 kind: "Deployment",
 metadata: {
 name: "pulumi-kubernetes-operator",
 },
 spec: {
 replicas: 1,
 selector: {
 matchLabels: {
 name: "pulumi-kubernetes-operator",
 },
 },
 template: {
 metadata: {
 labels: {
 name: "pulumi-kubernetes-operator",
 },
 },
 spec: {
 serviceAccountName: "pulumi-kubernetes-operator",
 imagePullSecrets: [{
 name: "pulumi-kubernetes-operator",
 }],
 containers: [{
 name: "pulumi-kubernetes-operator",
 image: "pulumi/pulumi-kubernetes-operator:v0.0.2",
 command: ["pulumi-kubernetes-operator"],
 args: ["--zap-level=debug"],
 imagePullPolicy: "Always",
 env: [
 {
 name: "WATCH_NAMESPACE",
 valueFrom: {
 fieldRef: {
 fieldPath: "metadata.namespace",
 },
 },
 },
 {
 name: "POD_NAME",
 valueFrom: {
 fieldRef: {
 fieldPath: "metadata.name",
 },
 },
 },
 {
 name: "OPERATOR_NAME",
 value: "pulumi-kubernetes-operator",
 },
 ],
 }],
 },
 },
 },
});
const pulumi_kubernetes_operatorRole = new kubernetes.rbac.v1.Role("pulumi_kubernetes_operatorRole", {
 apiVersion: "rbac.authorization.k8s.io/v1",
 kind: "Role",
 metadata: {
 name: "pulumi-kubernetes-operator",
 },
 rules: [
 {
 apiGroups: [""],
 resources: [
 "pods",
 "services",
 "services/finalizers",
 "endpoints",
 "persistentvolumeclaims",
 "events",
 "configmaps",
 "secrets",
 ],
 verbs: [
 "create",
 "delete",
 "get",
 "list",
 "patch",
 "update",
 "watch",
 ],
 },
 {
 apiGroups: ["apps"],
 resources: [
 "deployments",
 "daemonsets",
 "replicasets",
 "statefulsets",
 ],
 verbs: [
 "create",
 "delete",
 "get",
 "list",
 "patch",
 "update",
 "watch",
 ],
 },
 {
 apiGroups: ["monitoring.coreos.com"],
 resources: ["servicemonitors"],
 verbs: [
 "get",
 "create",
 ],
 },
 {
 apiGroups: ["apps"],
 resourceNames: ["pulumi-kubernetes-operator"],
 resources: ["deployments/finalizers"],
 verbs: ["update"],
 },
 {
 apiGroups: [""],
 resources: ["pods"],
 verbs: ["get"],
 },
 {
 apiGroups: ["apps"],
 resources: [
 "replicasets",
 "deployments",
 ],
 verbs: ["get"],
 },
 {
 apiGroups: ["pulumi.com"],
 resources: ["*"],
 verbs: [
 "create",
 "delete",
 "get",
 "list",
 "patch",
 "update",
 "watch",
 ],
 },
 ],
});
const pulumi_kubernetes_operatorRoleBinding = new kubernetes.rbac.v1.RoleBinding("pulumi_kubernetes_operatorRoleBinding", {
 kind: "RoleBinding",
 apiVersion: "rbac.authorization.k8s.io/v1",
 metadata: {
 name: "pulumi-kubernetes-operator",
 },
 subjects: [{
 kind: "ServiceAccount",
 name: "pulumi-kubernetes-operator",
 }],
 roleRef: {
 kind: "Role",
 name: "pulumi-kubernetes-operator",
 apiGroup: "rbac.authorization.k8s.io",
 },
});
const pulumi_kubernetes_operatorServiceAccount = new kubernetes.core.v1.ServiceAccount("pulumi_kubernetes_operatorServiceAccount", {
 apiVersion: "v1",
 kind: "ServiceAccount",
 metadata: {
 name: "pulumi-kubernetes-operator",
 },
});

Superpowers

Now that we’ve got some Kubernetes resources defined in real code, we can start working on them like any other software project. First, we define a variable for the operator name and then swap out hardcoded string references to the other resource names. With these changes, it doesn’t require any thought to keep these resources in sync! You can later decide to change the name in one place, and every other reference will update automatically as well.

const operatorName = "pulumi-kubernetes-operator";

const pulumi_kubernetes_operatorServiceAccount = new kubernetes.core.v1.ServiceAccount("pulumi_kubernetes_operatorServiceAccount", {
 apiVersion: "v1",
 kind: "ServiceAccount",
 metadata: {
 name: operatorName,
 },
});
const pulumi_kubernetes_operatorRoleBinding = new kubernetes.rbac.v1.RoleBinding("pulumi_kubernetes_operatorRoleBinding", {
 kind: "RoleBinding",
 apiVersion: "rbac.authorization.k8s.io/v1",
 metadata: {
 name: operatorName,
 },
 subjects: [{
 kind: "ServiceAccount",
 name: pulumi_kubernetes_operatorServiceAccount.metadata.name,
 }],
 roleRef: {
 kind: "Role",
 name: pulumi_kubernetes_operatorRole.metadata.name,
 apiGroup: "rbac.authorization.k8s.io",
 },
});

This simple refactoring is just the tip of the iceberg. At this point, you could:

wrap these resources in a class to abstract away the low-level details
add configuration values to the stack to make it easy to tweak certain values
or even add custom logic for Pulumi to run after a particular resource is created

Pulumi gives you Kubernetes superpowers, and you decide how to use them!

Learn More

If you’d like to learn about Pulumi and how to manage your infrastructure and Kubernetes through code, get started today. Pulumi is open source and free to use.

For further examples on how to use Pulumi to create Kubernetes clusters, or deploy workloads to a cluster, check out the rest of the Kubernetes tutorials.

As always, you can check out our code on GitHub, follow us on Twitter, subscribe to our YouTube channel, or join our Community Slack channel if you have any questions, need support, or just want to say hello.

Introducing crd2pulumi: Typed CustomResources for Kubernetes

Levi Blackstone — Wed, 12 Aug 2020 00:00:00 +0000

CustomResources in Kubernetes allow users to extend the API with their types. These types are defined using CustomResourceDefinitions (CRDs), which include an OpenAPI schema. This extensibility is quite useful but comes at the cost of complex YAML definitions. Our new crd2pulumi tool takes the pain out of managing CustomResources by generating types in the Pulumi-supported language of your choice!

Pulumi already supports the management of CRDs and their associated CustomResources using the apiextensions package. However, these SDK resources are untyped since every schema is, well, custom. While this is fine for simple CRDs, it quickly becomes unwieldy for real-world CRDs like cert-manager or Istio. These CRDs contain thousands of lines of complex YAML schemas, making it difficult to write CustomResources that adhere to those specs. Programming languages offer a better path forward. Instead of wrangling error-prone YAML definitions, using types in a programming language lets you use IDE type checking and autocomplete features!

Getting Started with crd2pulumi

Let’s test crd2pulumi on the example CronTab CRD specified in the Kubernetes Documentation.

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
 # name must match the spec fields below, and be in the form: <plural>.<group>
 name: crontabs.stable.example.com
spec:
 # group name to use for REST API: /apis/<group>/<version>
 group: stable.example.com
 # list of versions supported by this CustomResourceDefinition
 versions:
 - name: v1
 # Each version can be enabled/disabled by Served flag.
 served: true
 # One and only one version must be marked as the storage version.
 storage: true
 schema:
 openAPIV3Schema:
 type: object
 properties:
 spec:
 type: object
 properties:
 cronSpec:
 type: string
 image:
 type: string
 replicas:
 type: integer
 # either Namespaced or Cluster
 scope: Namespaced
 names:
 # plural name to be used in the URL: /apis/<group>/<version>/<plural>
 plural: crontabs
 # singular name to be used as an alias on the CLI and for display
 singular: crontab
 # kind is normally the CamelCased singular type. Your resource manifests use this.
 kind: CronTab
 # shortNames allow shorter string to match your resource on the CLI
 shortNames:
 - ct

Copy this definition into a file called crontab.yaml and then run crd2pulumi to generate types for your language of choice.

$ crd2pulumi --nodejsPath ./crontabs crontabs.yaml

import * as crontabs from "./crontabs"
import * as pulumi from "@pulumi/pulumi"

// Register the CronTab CRD.
const cronTabDefinition = new crontabs.stable.CronTabDefinition("my-crontab-definition")

// Instantiate a CronTab resource.
const myCronTab = new crontabs.stable.v1.CronTab("my-new-cron-object",
{
 metadata: {
 name: "my-new-cron-object",
 },
 spec: {
 cronSpec: "* * * * */5",
 image: "my-awesome-cron-image",
 replicas: 3,
 }
})

$ crd2pulumi --pythonPath ./crontabs crontabs.yaml

import pulumi_kubernetes as k8s
import crontabs.pulumi_crds as crontabs


# Register the CronTab CRD.
crontab_definition = k8s.yaml.ConfigFile("my-crontab-definition", file="crontabs.yaml")

# Instantiate a CronTab resource.
crontab_instance = crontabs.stable.v1.CronTab(
 "my-new-cron-object",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name="my-new-cron-object"
 ),
 spec=crontabs.stable.v1.CronTabSpecArgs(
 cron_spec="* * * */5",
 image="my-awesome-cron-image",
 replicas=3,
 )
)

$ crd2pulumi --dotnetPath ./crontabs crontabs.yaml

using Pulumi;
using Pulumi.Kubernetes.Yaml;
using Pulumi.Kubernetes.Types.Inputs.Meta.V1;

class MyStack : Stack
{
 public MyStack()
 {
 // Register a CronTab CRD.
 var cronTabDefinition = new Pulumi.Kubernetes.Yaml.ConfigFile("my-crontab-definition",
 new ConfigFileArgs{
 File = "crontabs.yaml"
 }
 );

 // Instantiate a CronTab resource.
 var cronTabInstance = new Pulumi.Crds.Stable.V1.CronTab("my-new-cron-object",
 new Pulumi.Kubernetes.Types.Inputs.Stable.V1.CronTabArgs{
 Metadata = new ObjectMetaArgs{
 Name = "my-new-cron-object"
 },
 Spec = new Pulumi.Kubernetes.Types.Inputs.Stable.V1.CronTabSpecArgs{
 CronSpec = "* * * * */5",
 Image = "my-awesome-cron-image",
 Replicas = 3
 }
 });
 }
}

$ crd2pulumi --goPath ./crontabs crontab.yaml

package main

import (
 crontabsv1 "crds-go-final/crontabs/stable/v1"

 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/meta/v1"
 "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/yaml"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // Register the CronTab CRD.
 _, err := yaml.NewConfigFile(ctx, "my-crontab-definition",
 &yaml.ConfigFileArgs{
 File: "crontabs.yaml",
 },
 )
 if err != nil {
 return err
 }

 // Instantiate a CronTab resource.
 _, err = crontabsv1.NewCronTab(ctx, "my-new-cron-object", &crontabsv1.CronTabArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("my-new-cron-object"),
 },
 Spec: crontabsv1.CronTabSpecArgs{
 CronSpec: pulumi.String("* * * * */5"),
 Image: pulumi.String("my-awesome-cron-image"),
 Replicas: pulumi.IntPtr(3),
 },
 })
 if err != nil {
 return err
 }

 return nil
 })
}

As you can see, the v1.CronTab object is strongly-typed. So if you try to set replicas to a string or add an unsupported argument, your IDE will immediately warn you!

Cert Manager Example

Now let’s examine a real-world cert-manager example. In this case, the CRD is over 1200 lines of YAML, but crd2pulumi generates a nice interface so that we don’t have to worry about it. Here’s what it looks like to create a Certificate CustomResource using our new types.

$ crd2pulumi --nodejsPath ./certificates certificate.yaml

import * as certificates from "./certificates"

// Register the Certificate CRD.
new certificates.certmanager.CertificateDefinition("certificate");

// Instantiate a Certificate resource.
new certificates.certmanager.v1beta1.Certificate("example-cert", {
 metadata: {
 name: "example-com",
 },
 spec: {
 secretName: "example-com-tls",
 duration: "2160h",
 renewBefore: "360h",
 commonName: "example.com",
 dnsNames: [
 "example.com",
 "www.example.com",
 ],
 issuerRef: {
 name: "ca-issuer",
 kind: "Issuer",
 }
 }
});

$ crd2pulumi --pythonPath ./certificates certificate.yaml

import pulumi_kubernetes as k8s
import certmanager.pulumi_crds as certmanager


# Register the Certificate CRD.
_ = k8s.yaml.ConfigFile("my-certificate-definition", file="certificate.yaml")

# Instantiate a Certificate resource.
_ = certmanager.certmanager.v1beta1.Certificate(
 "example-cert",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name="example-com"
 ),
 spec=certmanager.certmanager.v1beta1.CertificateSpecArgs(
 secret_name="example-com-tls",
 duration="2160h",
 renew_before="360h",
 common_name="example.com",
 dns_names=[
 "example.com",
 "www.example.com"
 ],
 issuer_ref=certmanager.certmanager.v1beta1.CertificateSpecIssuerRefArgs(
 name="ca-issuer",
 kind="Issuer"
 )
 )
)

$ crd2pulumi --dotnetPath ./certificates certificate.yaml

using Pulumi;
using Pulumi.Kubernetes.Yaml;
using Pulumi.Kubernetes.Types.Inputs.Meta.V1;

class MyStack : Stack
{
 public MyStack()
 {
 // Register a Certificate CRD.
 var certificateDefinition = new Pulumi.Kubernetes.Yaml.ConfigFile("my-certificate-definition",
 new ConfigFileArgs{
 File = "certificate.yaml"
 }
 );

 // Instantiate a Certificate resource.
 var certificateInstance = new Pulumi.Crds.Certmanager.V1Beta1.Certificate("example-cert",
 new Pulumi.Kubernetes.Types.Inputs.Certmanager.V1Beta1.CertificateArgs{
 Metadata = new ObjectMetaArgs{
 Name = "example-com"
 },
 Spec = new Pulumi.Kubernetes.Types.Inputs.Certmanager.V1Beta1.CertificateSpecArgs{
 SecretName = "example-com-tls",
 Duration = "2160h",
 RenewBefore = "360h",
 CommonName = "example.com",
 DnsNames = {
 "example.com",
 "www.example.com"
 },
 IssuerRef = new Pulumi.Kubernetes.Types.Inputs.Certmanager.V1Beta1.CertificateSpecIssuerRefArgs{
 Name = "ca-issuer",
 Kind = "Issuer"
 }
 }
 }
 );
 }
}

$ crd2pulumi --goPath ./certificates certificate.yaml

package main

import (
 certv1b1 "crds-go-final/certificates/certmanager/v1beta1"

 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/meta/v1"
 "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/yaml"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // Register the Certificate CRD.
 _, err := yaml.NewConfigFile(ctx, "my-certificate-definition",
 &yaml.ConfigFileArgs{
 File: "certificate.yaml",
 },
 )
 if err != nil {
 return err
 }

 // Instantiate a Certificate resource.
 _, err = certv1b1.NewCertificate(ctx, "example-cert", &certv1b1.CertificateArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("example-com"),
 },
 Spec: certv1b1.CertificateSpecArgs{
 SecretName: pulumi.String("example-com-tls"),
 Duration: pulumi.String("2160h"),
 RenewBefore: pulumi.String("360h"),
 CommonName: pulumi.String("example.com"),
 DnsNames: pulumi.StringArray{
 pulumi.String("example.com"),
 pulumi.String("www.example.com"),
 },
 IssuerRef: certv1b1.CertificateSpecIssuerRefArgs{
 Name: pulumi.String("ca-issuer"),
 Kind: pulumi.String("Issuer"),
 },
 },
 })
 if err != nil {
 return err
 }

 return nil
 })
}

Kubernetes can be complex, but Pulumi gives you the tools you need to manage it successfully. With Pulumi superpowers at your fingertips, you can stop worrying about YAML indentation, and get back to solving the problems you care about!

Learn More

If you’d like to try crd2pulumi today, head to the release page and download the appropriate binary for your operating system.

If you’d like to learn about Pulumi and how to manage your infrastructure and Kubernetes through code, get started today. Pulumi is open source and free to use.

For further examples on how to use Pulumi to create Kubernetes clusters, or deploy workloads to a cluster, check out the rest of the Kubernetes tutorials.

Announcing Kustomize Support

Levi Blackstone — Tue, 07 Jul 2020 00:00:00 +0000

Kubernetes is complex, and there are many ways to manage Kubernetes resources. Pulumi supports many of these options, including native code SDKs, YAML, Helm, and now, Kustomize. There’s no need to rewrite your existing configurations to get started with Pulumi. You can efficiently adopt existing resources to deploy your modern application and save time and effort.

Over the past year, we’ve heard from many users that they’d like to manage Kustomize resources directly with Pulumi. The v2.4.0 release of our Kubernetes provider adds support for Kustomize to all of our SDKs (Go, TypeScript, .NET, Python). One of our primary goals at Pulumi is to meet users where they are. Our support for Kustomize, Helm, and YAML means that you don’t have to “boil the ocean” to get started with Pulumi. The following examples show you how to get started with Pulumi and Kustomize today!

Working with local files

Let’s see how to deploy a Kustomization from a local directory.

import * as k8s from "@pulumi/kubernetes";

new k8s.kustomize.Directory("helloWorldLocal", {
 directory: "./helloWorld",
});

import pulumi_kubernetes as k8s

k8s.kustomize.Directory("helloWorldLocal",
 directory="helloWorld")

using System;
using System.Threading.Tasks;
using Pulumi;
using Pulumi.Kubernetes.Kustomize;

class KustomizeStack : Stack
{
 public KustomizeStack()
 {
 var files = new Directory("helloWorldLocal", new DirectoryArgs
 {
 Directory = "./helloWorld"
 });
 }

}

class Program
{
 static Task<int> Main(string[] args) => Deployment.RunAsync<KustomizeStack>();
}

import (
 "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/kustomize"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err := kustomize.NewDirectory(ctx, "helloWorldLocal",
 kustomize.DirectoryArgs{
 Directory: pulumi.String("./helloWorld"),
 },
 )
 if err != nil {
 return err
 }

 return nil
 })
}

Working with git repos

We can also use Kustomizations directly from a git repository, local or remote, by specifying the path to the directory.

import * as k8s from "@pulumi/kubernetes";

new k8s.kustomize.Directory("helloWorldRemote", {
 directory: "https://github.com/kubernetes-sigs/kustomize/tree/v3.3.1/examples/helloWorld",
});

import pulumi_kubernetes as k8s

k8s.kustomize.Directory("helloWorldRemote",
 directory="https://github.com/kubernetes-sigs/kustomize/tree/v3.3.1/examples/helloWorld")

using System;
using System.Threading.Tasks;
using Pulumi;
using Pulumi.Kubernetes.Kustomize;

class KustomizeStack : Stack
{
 public KustomizeStack()
 {
 var files = new Directory("helloWorldRemote", new DirectoryArgs
 {
 Directory = "https://github.com/kubernetes-sigs/kustomize/tree/v3.3.1/examples/helloWorld"
 });
 }

}

class Program
{
 static Task<int> Main(string[] args) => Deployment.RunAsync<KustomizeStack>();
}

import (
 "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/kustomize"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err := kustomize.NewDirectory(ctx, "helloWorldRemote",
 kustomize.DirectoryArgs{
 Directory: pulumi.String("https://github.com/kubernetes-sigs/kustomize/tree/v3.3.1/examples/helloWorld"),
 },
 )
 if err != nil {
 return err
 }

 return nil
 })
}

A better way to Kustomize

One of my favorite parts of Pulumi’s Kubernetes SDK is the ability to apply transformations to resource definitions. This feature allows you to make changes to existing definitions before they are applied to a cluster. This saves the headache of forking an upstream repo just to make a few tweaks, and then trying to keep those changes in sync! Transformations are available for the Helm, Kustomize, and YAML SDKs.

Kustomize overlays can be used for a similar purpose, but require changing files in the directory. Transformations happen in memory, and are not persisted to disk. Keep both options in mind when you want to modify an existing configuration.

import * as k8s from "@pulumi/kubernetes";

new k8s.kustomize.Directory("helloWorldRemote", {
 directory: "https://github.com/kubernetes-sigs/kustomize/tree/v3.3.1/examples/helloWorld",
 transformations: [
 (obj: any) => {
 if (obj.kind === "Service") {
 obj.spec.type = "ClusterIP"
 }
 }
 ]
});

import pulumi_kubernetes as k8s

def change_service_type(obj):
 if obj.kind == "Service":
 obj.spec.type = "ClusterIP"

k8s.kustomize.Directory("helloWorldRemote",
 directory="https://github.com/kubernetes-sigs/kustomize/tree/v3.3.1/examples/helloWorld",
 transformations=[change_service_type])

using System;
using System.Threading.Tasks;
using Pulumi;
using Pulumi.Kubernetes.Kustomize;

class KustomizeStack : Stack
{
 public KustomizeStack()
 {
 // Make every service private to the cluster, i.e., turn all services into ClusterIP instead of
 // LoadBalancer.
 ImmutableDictionary<string, object> LoadBalancerToClusterIP(ImmutableDictionary<string, object> obj, CustomResourceOptions opts)
 {
 if ((string)obj["kind"] == "Service")
 {
 var spec = (ImmutableDictionary<string, object>) obj["spec"];
 if (spec != null && (string) spec["type"] == "LoadBalancer")
 {
 return obj.SetItem("spec", spec.SetItem("type", "ClusterIP"));
 }
 }

 return obj;
 }

 var files = new Directory("helloWorldRemote", new DirectoryArgs
 {
 Directory = "https://github.com/kubernetes-sigs/kustomize/tree/v3.3.1/examples/helloWorld",
 Transformations =
 {
 LoadBalancerToClusterIP,
 }
 });
 }
}

class Program
{
 static Task<int> Main(string[] args) => Deployment.RunAsync<KustomizeStack>();
}

import (
 "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/kustomize"
 "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err := kustomize.NewDirectory(ctx, "helloWorldRemote",
 kustomize.DirectoryArgs{
 Directory: pulumi.String("https://github.com/kubernetes-sigs/kustomize/tree/v3.3.1/examples/helloWorld"),
 Transformations: []yaml.Transformation{
 func(state map[string]interface{}, opts ...pulumi.ResourceOption) {
 if state["kind"] == "Service" {
 spec := state["spec"].(map[string]interface{})
 spec["type"] = "ClusterIP"
 }
 },
 },
 },
 )
 if err != nil {
 return err
 }

 return nil
 })
}

Learn More

If you’d like to learn about Pulumi and how to manage your infrastructure and Kubernetes through code, get started today. Pulumi is open source and free to use.

For further examples on how to use Pulumi to create Kubernetes clusters, or deploy workloads to a cluster, check out the rest of the Kubernetes tutorials.

Winning with Pulumi Superpowers and Kubernetes

Levi Blackstone — Wed, 03 Jun 2020 00:00:00 +0000

You’ve containerized your application, and it’s running great on your desktop using Docker Compose or Swarm. But now it’s time to test it locally with minikube and then put it into production with Kubernetes. Manifests are a bit like Compose files - it’s just YAML, right?

It’s Dangerous to Go Alone. Take this

Recently, Pulumi engineer Levi Blackstone posted a Twitter thread on why he thinks Pulumi is the best way to build and deploy modern applications on Kubernetes. Levi might be a bit biased, but he makes solid points for using Pulumi with Kubernetes, whether you’re a developer or operator.

For Devs

First, let’s take a look at Kubernetes from a dev’s point of view.

Use the full Kubernetes API surface – Each of Pulumi’s Kubernetes SDKs (TypeScript, Python, .NET and Go) are generated from the OpenAPI spec. This is critical because it means that resource definitions in any language are a 1:1 match with the upstream API. In the example below, the deployment manifest on the left corresponds to the TypeScript on the right.

Conditional logic – Another significant drawback of YAML is the lack of conditional logic. It’s common to change parameters depending on the environment, but this usually calls for separate tools to template your YAML. With Pulumi, this logic can go right into your resource definitions. It’s common to develop your application locally using minikube, which doesn’t natively support LoadBalancer Services. On the other hand, many cloud deployments do use a LoadBalancer Service to route traffic. The example below shows how you can deploy to either environment without having to rewrite the manifest by using a configuration value.

var frontendServiceType string
if isMinikube {
 frontendServiceType = “ClusterIP”
} else {
 frontendServiceType = “LoadBalancer”
}

frontendService, err := corev1.NewService(ctx, “frontent”, &corev1.ServiceArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Labels: frontendLabels,
 Name: pulumi.String(“frontend”),
 },
 Spec: &corev1.ServiceSpecArgs{
 Type: pulumi.String(frontendServiceType),
 Ports: corev1.ServicePortArray{
 corev1.ServicePortArg{
 Port: pulumi.Int(80),
 },
 },
 Selector: frontendLabels,
 },
})

Strongly typed resources – A tricky part about defining Kubernetes resources in YAML/JSON is the lack of typing. It’s hard to track typos, and you have to apply the resource to a cluster to get any feedback. Pulumi helps here by showing type information right in the SDK.

Readiness logic – Pulumi includes readiness logic to help you understand when your Kubernetes resources are ready. This makes it easy to integrate Kubernetes workflows into CI systems! Unlike kubectl apply, Pulumi doesn’t indicate success until all of the resources are ready to go. Let Pulumi worry about the bookkeeping so you can focus on more interesting matters.

Deprecation warnings – Pulumi warns you if you’re using Kubernetes resources with deprecated apiVersions, and gives you info about which versions remove them entirely. Don’t be caught by surprise when you upgrade your clusters!

For Operators

Pulumi is not just for developers; operators can leverage it to manage Kubernetes clusters, reuse existing YAML manifests and Helm charts, and conveniently manage common configurations like networking, storage, or RBAC. You can even export your Pulumi programs to YAML for portability.

Manage Kubernetes clusters – With Pulumi, you can manage every layer of your stack, from specifying a custom VPC to deploying Kubernetes clusters! Check out this example where we create clusters on AWS, AKS, and GKE, and then deploy an app to all of them.
Use existing YAML manifests – Many of us already have a pile of YAML Kubernetes definitions to manage. Manage these manifests directly; no conversion needed! Pulumi shows detailed previews and allows you to run arbitrary transformations on the YAML before it is applied.

def set_namespace(namespace):
 def f(obj):
 if "metadata" in obj:
 obj["metadata"]["namespace"] = namespace
 else:
 obj["metadata"] = {"namespace": namespace}
 return f

deployment = ConfigFile(
 "deployment",
 file_id="nginx-deployment.yaml",
 transformations=[set_namespace("test)],
)

Helm support – Pulumi lets you manage Helm charts similarly to our YAML support. It’s common for organizations to fork upstream charts to customize a few values, but this can often be avoided with the use of transformations in Pulumi.

Extension libraries – Our kubernetesx library makes it easier to manage Kubernetes resources by abstracting away a lot of the boilerplate. This dramatically cuts the amount of code you need to write, while still giving you access to the full API surface if you need it. Read about kubernetesx on our [blog] (/blog/introducing-kx).
YAML export – Pulumi lets you export YAML manifests from the resource definitions in your program. This can be useful for integrating with other systems that expect Kubernetes YAML.

Conclusion

If you’re just starting out with Kubernetes, Pulumi provides a great experience by letting you take advantage of conditional logic, strong typing, error checking, and full access to the Kubernetes API. When deploying infrastructure, you can avoid deploying deprecated versions and know precisely when your resources are ready.

You don’t have to start from scratch if you have existing YAML manifests and Helm charts. Pulumi gives you the flexibility to manage your stack in whatever way makes sense to you, on any cloud provider, including your own infrastructure. We make it easier to bridge the gap between developers and operators so you can put Kubernetes to work for your organization!

Learn how Pulumi can make your Kubernetes experience smoother and more efficient by working through tutorials or trying out our examples.

Deploy Kubernetes and Applications with Go

Levi Blackstone — Tue, 14 Apr 2020 00:00:00 +0000

We’re excited that Go is now a first-class language in Pulumi and that you can build your infrastructure with Go on AWS, Azure, GCP, and many other clouds. Users often ask, “Can I use Pulumi to manage Kubernetes infrastructure in Go today?” With the release of Pulumi 2.0., the answer is “Yes!”

Building your Kubernetes infrastructure with Infrastructure as Code offers several benefits:

Strongly-typed inputs and outputs for resources and invokes
First-class language support in editors/IDEs like vim, VS Code, GoLand, and atom
Reusable components and classes: Abstract standard functionality into classes and libraries for code reuse, and clean infrastructure design, instead of copy/pasting pages of YAML.
Embed Pulumi programs within other Go-based tools

These benefits provide a productive experience for Go developers using Kubernetes.

Tour of Kubernetes with Go

Let’s start with a basic example, e.g. deploying an Nginx pod into a Kubernetes cluster.

pod, err := corev1.NewPod(ctx, "pod", &corev1.PodArgs{
 Spec: corev1.PodSpecArgs{
 Containers: corev1.ContainerArray{
 corev1.ContainerArgs{
 Name: pulumi.String("nginx"),
 Image: pulumi.String("nginx"),
 }
 },
 },
})
if err != nil {
 return err
}

We can deploy this with pulumi up.

$ pulumi up
Previewing update (nginx):
Type Name Plan
+ pulumi:pulumi:Stack kubernetes-go-nginx create
+ └─ kubernetes:core:Pod pod create
Resources:
+ 2 to create
Do you want to perform this update? yes
Updating (nginx):
Type Name Status
+ pulumi:pulumi:Stack kubernetes-go-nginx created
+ └─ kubernetes:core:Pod pod created
Resources:
+ 2 created
Duration: 16s

Although programming languages are imperative, Pulumi describes the desired state of the infrastructure. If we change our program, Pulumi will compute the minimum delta and apply it to our Kubernetes cluster, which will transition to the new desired state. For example, we can modify our Kubernetes resources in place inside the cluster by adding a label to our Pod:

pod, err := corev1.NewPod(ctx, "pod", &corev1.PodArgs{
+ Metadata: &metav1.ObjectMetaArgs{
+ Labels: pulumi.StringMap{"app": pulumi.String("nginx")},
+ },
 Spec: corev1.PodSpecArgs{
 Containers: corev1.ContainerArray{
 corev1.ContainerArgs{
 Name: pulumi.String("nginx"),
 Image: pulumi.String("nginx"),
 }},
 },
})
if err != nil {
 return err
}

When we update the deployment with ‘pulumi up`, the preview shows that the pod is updated in place with the new label.

$ pulumi up
Previewing update (nginx):
Type Name Plan Info
pulumi:pulumi:Stack kubernetes-go-nginx
~ └─ kubernetes:core:Pod pod update [diff: ~metadata]
Resources:
~ 1 to update
1 unchanged
Do you want to perform this update? details
pulumi:pulumi:Stack: (same)
[urn=urn:pulumi:nginx::kubernetes-go::pulumi:pulumi:Stack::kubernetes-go-nginx]
~ kubernetes:core/v1:Pod: (update)
[id=default/pod-yxmgq7q2]
[urn=urn:pulumi:nginx::kubernetes-go::kubernetes:core/v1:Pod::pod]
~ metadata: {
~ labels: {
+ app: "nginx"
}
}
Do you want to perform this update? yes
Updating (nginx):
Type Name Status Info
pulumi:pulumi:Stack kubernetes-go-nginx
~ └─ kubernetes:core:Pod pod updated [diff: ~metadata]
Resources:
~ 1 updated
1 unchanged
Duration: 13s

The benefits of Go start to shine when we extract common code into a reusable component. For example, we can create a new ServiceDeployment component that combines both a Kubernetes Service and Deployment with opinionated defaults. Our ServiceDeployment component can describe entire Kubernetes applications (100s of lines of YAML), in a short snippet of Go:

// Initialize config
conf := config.New(ctx, "")

// Minikube does not implement services of type `LoadBalancer` so
// require the user to specify if we're running on minikube. If so,
// create only services of type ClusterIP.
isMinikube := conf.GetBool("isMinikube")

// Redis leader Deployment + Service
_, err := NewServiceDeployment(ctx, "redis-master", &ServiceDeploymentArgs{
 Image: pulumi.String("k8s.gcr.io/redis:e2e"),
 Ports: pulumi.IntArray{pulumi.Int(6379)},
})
if err != nil {
 return err
}

// Redis follower Deployment + Service
_, err = NewServiceDeployment(ctx, "redis-slave", &ServiceDeploymentArgs{
 Image: pulumi.String("gcr.io/google_samples/gb-redisslave:v3"),
 Ports: pulumi.IntArray{pulumi.Int(6379)},
})
if err != nil {
 return err
}

// Frontend Deployment + Service
frontend, err := NewServiceDeployment(ctx, "frontend", &ServiceDeploymentArgs{
 AllocateIPAddress: true,
 Image: pulumi.String("gcr.io/google-samples/gb-frontend:v4"),
 IsMinikube: pulumi.Bool(isMinikube),
 Ports: pulumi.IntArray{pulumi.Int(80)},
 Replicas: pulumi.Int(3),
})
if err != nil {
 return err
}

if isMinikube {
 ctx.Export("frontendIP", frontend.Service.Spec.ApplyT(
 func(spec *corev1.ServiceSpec) *string { return spec.ClusterIP }))
} else {
 ctx.Export("frontendIP", frontend.FrontendIP)
}

The complete implementation of the ServiceDeployment component is detailed in the Guestbook example in the Pulumi Examples repo on GitHub.

Building Docker Images for Kubernetes with Go

In the previous examples, we specified the Docker image to deploy as part of our Kubernetes Deployments by referring to an image in a registry. If we wanted to push our custom Docker image that uses our application’s source code, and deploy that in our Kubernetes Pod or Deployment, we can do it with the pulumi-docker package. For example, we can deploy our custom Nginx Docker image with the following:

image, err := docker.NewImage(ctx, "node-app", &docker.ImageArgs{
 ImageName: pulumi.String("my-username/my-nginx"),
 Build: &docker.DockerBuildArgs{
 Context: pulumi.String("./app"),
 },
})
if err != nil {
 return err
}

pod, err := corev1.NewPod(ctx, "pod", &corev1.PodArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Labels: pulumi.StringMap{"app": pulumi.String("nginx")},
 },
 Spec: corev1.PodSpecArgs{
 Containers: corev1.ContainerArray{
 corev1.ContainerArgs{
 Name: pulumi.String("nginx"),
 Image: image.ImageName,
 }},
 },
})
if err != nil {
 return err
}

We can use our Dockerfile from the app folder in our Pulumi project, instead of using the default Nginx image from DockerHub. For example, adding the following in our Dockerfile deploys customized files in the app/content folder to our Nginx server:

FROM nginx
COPY content /usr/share/nginx/html

When we deploy this with Pulumi, the Dockerfile is built locally, pushed to a registry, then made available by specifying the image name in the registry referenced from the Pod in Kubernetes. Our changes happen automatically, seamlessly deploying and versioning both the application code and infrastructure together with a simple pulumi up.

We can also push to another Docker container registry (like ACR, GCR, ECR, or others) by using additional parameters on the pulumi.docker.ImageArgs class.

Cloud + Kubernetes with Go

Pulumi works with both Kubernetes and cloud providers (AWS, Azure, GCP, and more.) Within the same Pulumi program, you can build a Kubernetes cluster and then deploy applications and services into that cluster.

For example, we can deploy a managed GKE cluster, and then deploy a Pod into it:

masterVersion := "1.14.10-gke.27"
cluster, err := container.NewCluster(ctx, "demo-cluster", &container.ClusterArgs{
 InitialNodeCount: pulumi.Int(2),
 MinMasterVersion: pulumi.String(masterVersion),
 NodeVersion: pulumi.String(masterVersion),
 NodeConfig: &container.ClusterNodeConfigArgs{
 MachineType: pulumi.String("n1-standard-1"),
 OauthScopes: pulumi.StringArray{
 pulumi.String("https://www.googleapis.com/auth/compute"),
 pulumi.String("https://www.googleapis.com/auth/devstorage.read_only"),
 pulumi.String("https://www.googleapis.com/auth/logging.write"),
 pulumi.String("https://www.googleapis.com/auth/monitoring"),
 },
 },
})
if err != nil {
 return err
}

ctx.Export("kubeconfig", generateKubeconfig(
 cluster.Endpoint, cluster.Name, cluster.MasterAuth))

k8sProvider, err := providers.NewProvider(
 ctx, "k8sprovider", &providers.ProviderArgs{
 Kubeconfig: generateKubeconfig(
 cluster.Endpoint, cluster.Name, cluster.MasterAuth),
}, pulumi.DependsOn([]pulumi.Resource{cluster}))
if err != nil {
 return err
}

namespace, err := corev1.NewNamespace(ctx, "app-ns", &corev1.NamespaceArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("demo-ns"),
 },
}, pulumi.Provider(k8sProvider))
if err != nil {
 return err
}

appLabels := pulumi.StringMap{
 "app": pulumi.String("demo-app"),
}
_, err = appsv1.NewDeployment(ctx, "app-dep", &appsv1.DeploymentArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Namespace: namespace.Metadata.Elem().Name(),
 },
 Spec: appsv1.DeploymentSpecArgs{
 Selector: &metav1.LabelSelectorArgs{
 MatchLabels: appLabels,
 },
 Replicas: pulumi.Int(3),
 Template: &corev1.PodTemplateSpecArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Labels: appLabels,
 },
 Spec: &corev1.PodSpecArgs{
 Containers: corev1.ContainerArray{
 corev1.ContainerArgs{
 Name: pulumi.String("demo-app"),
 Image: pulumi.String("jocatalin/kubernetes-bootcamp:v2"),
 }},
 },
 },
 },
}, pulumi.Provider(k8sProvider))
if err != nil {
 return err
}

This example first deploys a GKE Kubernetes Cluster into GCP, then deploys Kubernetes resources into that GKE Cluster. This example combines infrastructure and application deployment in a few dozen lines of declarative and strongly typed Go code.

Check out the full GKE in Go example on GitHub.

Conclusion

Kubernetes support is one of several significant new additions made available to Pulumi’s Go community and more improvements are on the way. Get started with Kubernetes and Go today, and let us know what you think!

Generate Kubernetes YAML with Familiar Programming Languages

Levi Blackstone — Tue, 03 Mar 2020 00:00:00 +0000

Stop writing Kubernetes YAML by hand, and start using the power of familiar programming languages! Pulumi can generate Kubernetes manifests that easily integrate into existing CI/CD workflows.

While Pulumi has excellent support for deploying and updating Kubernetes resources on a cluster, many users have asked for the option to render YAML that they can integrate into existing workflows. The v1.5.4 release of pulumi-kubernetes adds the renderYamlToDirectory option, which enables this feature. This option is available in every Pulumi-supported language, including TypeScript/JavaScript, Python, and .NET (Go support is coming soon!).

Aside from easily templating configuration across resources, using a familiar programming language allows you to write and consume libraries, and easily mix in infrastructure configuration (e.g., managed database endpoints, object storage, etc.), all in the same program.

Rendering YAML from TypeScript

First, choose a directory for the rendered manifests, and specify that path on a Provider.

import * as k8s from "@pulumi/kubernetes";
import * as kx from "@pulumi/kubernetesx";

// Instantiate a Kubernetes Provider and specify the render directory.
const provider = new k8s.Provider("render-yaml", {
 renderYamlToDirectory: "rendered",
});

Next, use that Provider for any Kubernetes resources you want to render as YAML.

// Create a Kubernetes PersistentVolumeClaim.
const pvc = new kx.PersistentVolumeClaim("data", {
 spec: {
 accessModes: [ "ReadWriteOnce" ],
 resources: { requests: { storage: "1Gi" } },
 }
}, { provider });

// Create a Kubernetes ConfigMap.
const cm = new kx.ConfigMap("cm", {
 data: { "config": "very important data" },
}, { provider });

// Create a Kubernetes Secret.
const secret = new kx.Secret("secret", {
 stringData: {
 "password": new random.RandomPassword("pw", {
 length: 12}).result,
 }
}, { provider });

// Define a Pod.
const pb = new kx.PodBuilder({
 containers: [{
 env: {
 CONFIG: cm.asEnvValue("config"),
 PASSWORD: secret.asEnvValue("password"),
 },
 image: "nginx",
 ports: {http: 8080},
 volumeMounts: [ pvc.mount("/data") ],
 }]
});

// Create a Kubernetes Deployment.
const deployment = new kx.Deployment("nginx", {
 spec: pb.asDeploymentSpec( { replicas: 3 } ),
}, { provider });

// Create a Kubernetes Service.
const service = deployment.createService({
 type: kx.types.ServiceType.LoadBalancer,
});

Now, run pulumi update, and Pulumi renders these resources to YAML. The update process resolves Outputs as usual, so that the manifests can include other infrastructure configuration specified in your program. The rendered manifests are kept in sync with changes to the program on each update.

Here’s what the resulting directory looks like:

Note that CustomResourceDefinition resources need to be applied first, so they are rendered in a separate subdirectory. (This example doesn’t include any CRDs, so the directory is empty). You could deploy the rendered manifests with kubectl like this:

kubectl apply -f "${RENDER_DIRECTORY}/0-crd"
kubectl apply -f "${RENDER_DIRECTORY}/1-manifest"

Voilà! From 33 (44 with whitespace and comments) lines of TypeScript code to 102 lines of YAML!

Here’s the rendered Deployment resource.

See this gist for the complete rendered output.

Caveats

There are two important caveats to note about YAML rendering support:

The YAML-rendered resources are not created on a Kubernetes cluster, so information that is computed server-side will not be available in your program. For example, a Service will not have IP assignments, so attempting to export these values will not work as usual (i.e., the value will be undefined).
Any Secret values will appear in plaintext in the rendered manifests. This includes any values marked as secret in Pulumi. A warning will be printed for any secret values being rendered to YAML, but it is your responsibility to protect the rendered files.

Learn More

If you’d like to learn about Pulumi and how to manage your infrastructure and Kubernetes through code, get started today. Pulumi is open source and free to use.

For further examples on how to use Pulumi to create Kubernetes clusters, or deploy workloads to a cluster, check out the rest of the Kubernetes tutorials.

Introducing kx: Kubernetes for Everyone

Levi Blackstone — Thu, 14 Nov 2019 00:00:00 +0000

Kubernetes provides a rich, standards-based API that works across cloud and on-premise infrastructure. However, many of the API fields are deeply nested and require users to specify the same values redundantly across different resources. While this explicit specification is necessary for Kubernetes to operate, this often leads users to copy-paste existing code to manage the boilerplate.

Today, as part of our Crosswalk for Kubernetes announcement, we’re introducing the Kubernetes Extensions (kx) library for Pulumi. kx is designed to simplify the declaration of Kubernetes resources, and make the API easier for everyone to use.

Kubernetes YAML: The cloud assembly language

With great power comes great complexity. The sheer number of Kubernetes configuration options can be overwhelming to users. YAML was not designed for programming, yet Kubernetes users often joke about being “YAML architects.”

Kubernetes YAML has been compared to an assembly language. We find this analogy to be particularly apt: it gives users complete control over the system, although it is closely tied to low-level implementation details.

Most Kubernetes applications require users to define several different resource types, and then manually tie them together with text string references. The API for each of these resource types is complex and deeply nested, so users often rely on copy-pasted examples and frequent examination of the API spec. Wouldn’t it be nice if there was a way to go above these low-level implementation details, and get back to thinking about your application?

The advent of high level programming languages completely changed the face of computing. Rather than being closely tied to the details of the underlying hardware, programmers could now focus on higher level concerns, like solving business problems.

Raising the Bar

kx mirrors the advantages of high-level programming languages, abstracting away the low-level details so that you can focus on more important things!

kx is designed to reduce complexity, with the ability to drop down to the raw API when necessary. This design provides a simple interface for the majority of cases, while still providing access to the full set of tools required for production apps.

Two primary goals for kx are terse resource declarations, and the ability to compose resources effortlessly as you build out applications on Kubernetes.

To illustrate this point, let’s check out a representative example of a real Kubernetes application, including a ConfigMap, Secret, Deployment, and Service.

Using kx	Equivalent YAML

This code is available here

Needless to say, it requires significantly more effort to understand and maintain the YAML version. Even including comments, the kx version is 50% shorter by only including information that is relevant to the application.

kx is available today in developer preview. We invite you to try it out and let us know what you think!

Learn more

If you’d like to learn about Pulumi and how to manage your infrastructure and Kubernetes through code, click here to get started today. Pulumi is open source and free to use.

Pulumi: A Better Way to Kubernetes

Levi Blackstone — Tue, 21 May 2019 00:00:00 +0000

Kubernetes is a powerful container orchestrator that is being adopted rapidly across the industry. At the same time, it is notoriously complex and presents a steep learning curve for newcomers. Nobody likes programming in YAML, and templates make it even harder. It’s difficult to understand the state of the cluster – Did my deployment succeed? Why isn’t my app working? And we often need to manage hosted cloud resources in addition to Kubernetes ones.

In this post, we will see how Pulumi can help you tame these issues and make Kubernetes more accessible, using familiar languages and your favorite tools. It’s simply Kubernetes made easy!

Clusters as code

Pulumi can seamlessly manage multiple layers of your stack, from the raw infrastructure to Kubernetes resources, and all the way up to serverless app code. Rather than gluing together yet another set of tools, you can create managed Kubernetes clusters with GKE, EKS or AKS and then reference them directly in your Pulumi code!

Apps as code (not YAML!)

Pulumi exposes the full API surface of Kubernetes as classes in your SDK, so you can manage any k8s resource (including CRDs and related CustomResources) in your Pulumi code. Stop trying to work around the limitations of YAML with templates, and reap the benefits of real software development practices: abstraction, conditionals, looping, library support, packaging, testing, debugging, and more! From the obvious (use variables to manage resource metadata, to the more esoteric (gate a canary deployment with Prometheus, Pulumi keeps the simple things simple, and makes the hard things possible.

Integrate with existing Helm charts or manifests

Most projects aren’t greenfield apps, and we know it’s critical to support an incremental transition when you switch tools. Our SDKs allow you to import Helm charts and YAML manifests and then mix and match these resources within your Pulumi program. While you can keep it simple and just import the resources, you have the full power of a programming language at your disposal. Why fight with a complicated YAML templating scheme when you can accomplish the same thing (and much more) with a real programming language? Take advantage of 60 years of software engineering know-how to make your deployments more reproducible and maintainable.

Make full use of your cloud

While you can run stateful workloads on Kubernetes using StatefulSets, it’s often better to use a managed service from your cloud provider. Since Pulumi can manage cloud resources as well as k8s resources it’s easy to bridge that gap! Try adding in a managed database, message queue or object store, and see how much simpler your k8s app can be. When your app is already running in the cloud, think outside of k8s, and use the power of the cloud! It’s far easier to maintain an app at scale if you strategically mix in managed cloud resources.

Debugging failures

It can be daunting to troubleshoot failures in Kubernetes, especially for new users. What happens after I run kubectl apply? Why is my app not working? Even with a detailed guide to follow, it can feel like playing 20 questions with kubectl to get the answers you need. Pulumi’s Kubernetes provider includes sophisticated logic to check resource readiness, and proactively surfaces errors during updates. With Pulumi, it’s far easier to understand the state of your k8s resources, and get the information you need to make changes when something goes wrong.

Integrating with CI/CD

Kubernetes uses an eventual consistency model that can be difficult to integrate with CI/CD systems. How do you know when your application is ready? Common workflows involve scripting kubectl calls and parsing JSON output in Bash. This approach is brittle, and the process is a little different for every Kubernetes resource type. Pulumi’s state reconciliation model is a natural fit for CI/CD systems: review changes with a preview, and then proceed with confidence once an update succeeds. This is great for GitOps and ChatOps workflows. You don’t have to be an expert on the inner workings of Kubernetes to be productive with Pulumi.

Since you can manage a full infrastructure stack with Pulumi, you can create infrastructure on demand (k8s cluster, databases, networking, object storage, etc.), spin up your application, run tests, and then tear the whole stack back down! This saves you money and builds confidence that you can recover from disaster scenarios.

Learn more

If you’d like to learn about Pulumi and how to manage your infrastructure and Kubernetes through code, click here to get started today. Pulumi is open source and free to use.

If you’d like to chat with our team, or get hands-on assistance with migrating your existing configuration code (including ksonnet programs) to Pulumi, please don’t hesitate to drop us a line.

Improving Kubernetes Management with Pulumi's Await Logic

Levi Blackstone — Tue, 05 Mar 2019 00:00:00 +0000

Pulumi enables customers to create, deploy, and manage modern applications and infrastructure in their preferred cloud environment using general purpose languages such as Javascript, Typescript and Python. For many businesses today, the use of modern technology is associated with Kubernetes, tools (command line or domain specific tools) to bring-up Kubernetes and a large pile of raw YAML files to deploy Kubernetes resources with.

Pulumi’s ready to use, language specific Kubernetes packages allow you to trade in the grab bag of tools and YAML files in exchange for the full expressive power of a general purpose language. In this blog post, we discuss “await logic”, which allows users to have better visibility into the state of Kubernetes resources as they are being deployed or created.

We’ve previously talked about how Pulumi tracks the status of Kubernetes resources to provide fine-grained status messages for both your infrastructure and applications. Pulumi uses a resource graph for orchestration, dependency management, differential updates, and cascading rollouts.

Kubernetes’ high availability design is based on an eventually consistent model. Pulumi augments this model with await logic to provide the notion of resource “readiness.” When you bring up a Kubernetes cluster with pulumi up, cluster resources are only marked ready after the resource status has been verified by the await logic.

You can see an example of what this looks like in practice in the screencast below:

Pulumi’s sophisticated await logic helps customers with a couple use cases:

Visualize fine-grained, live, single-view status updates for all of the Kubernetes cluster resources belonging to a Pulumi stack. Most applications deployed to Kubernetes have a variety of interacting components, including Pods, Deployments, Secrets, Persistent Volumes, Config Maps, Ingress and more. Rather than requiring multiple commands like kubectl get pod --all-namespaces, kubectl get pv --all-namespaces, kubectl get configmap, etc., Pulumi understands the complex dependencies in your application, and takes the guesswork out of deploying applications to Kubernetes. Pulumi can easily manage multiple stacks of cloud resources, simplifying the process of adding new applications, or sharing responsibilities between teams.
Reliably integrate with CI/CD systems for infrastructure and application deployments without requiring hardcoded timeouts, or scripting kubectl and parsing the resulting JSON/YAML to detect errors. If errors are encountered, Pulumi automatically surfaces the relevant messages, making it easy to understand why your deployment failed.

New Annotations to Customize Kubernetes Await Logic

Although this await logic works great for most workloads, the fast-moving Kubernetes ecosystem is vast and complex, and edge cases are inevitable. We have recently added two annotations that can be used to configure the behavior of the await logic.

In the 0.20.4 release, we added the “timeoutSeconds” parameter to allow users to override the default timeout values for resource creation. This is useful for special cases where a resource is expected to take a long time to be ready, but should not generally need to be set. To override the timeout for a selected resource, choose a timeout value, and add the following annotation to the resource’s metadata: pulumi.com/timeoutSeconds: "600".

In the 0.20.2 release, we added the ability to completely disable await logic for user-selected Kubernetes resources. Since the default await logic is so useful, we only recommend using this feature to work around currently unhandled edge cases while we work on longer term solutions. To disable await logic for a selected resource, add the following annotation to the resource’s metadata: pulumi.com/skipAwait: "true".

Here’s a TypeScript example that demonstrates both annotations:

If you do encounter an edge case and have to use this feature to work around it, it may be an oversight in the Pulumi Kubernetes provider - we would love to hear from you - please open a GitHub issue!

Special thanks to our awesome community, including Piotrek Bzdyl and Jamie Neil, who have helped us track down and improve the await logic for several interesting edge cases!

Learn More

If you’d like to learn about Pulumi and how to manage your infrastructure and Kubernetes through code, click here to get started today. Pulumi is open source and free to use.

If you’d like to go deeper on certain topics, here are some additional resources to check out:

Overview of Pulumi Kubernetes Scenarios
Tutorial: Create a Kubernetes cluster on a cloud provider Amazon EKS, Google GKE, or Azure AKS
Tutorial: Operate and deploy to a Kubernetes cluster
Docs: Pulumi docs, including an overview of the programming model
Video: Watch Joe Beda take Pulumi for a spin in last week’s TGIK

If you’d like to chat with our team, or get hands-on assistance with migrating your existing configuration code (including ksonnet programs) to Pulumi, please don’t hesitate to drop us a line.

Pulumi Blog: Levi Blackstone

Connect Your Cloud Accounts to Pulumi in Minutes

From hours of setup to a single flow

How it works

Choose the right access level for your security posture

Everything set up, nothing hidden

Get started

From Kubernetes Gatekeeper to Full-Stack Governance with OPA

What’s in the stable release

Kubernetes Gatekeeper compatibility

Walkthrough: Reusing policies from the gatekeeper-library

Part of the Pulumi Insights governance story

Frequently asked questions

Do I need to modify my existing Gatekeeper .rego files?

What happens with non-Kubernetes resources?

Do I need OPA installed locally?

When should I use OPA vs. TypeScript or Python for policies?

Get started

Run Pulumi Insights on Your Own Infrastructure

Insights at a glance

Why self-hosted?

How it works

Get started

How We Built a Distributed Work Scheduling System for Pulumi Cloud

Where we started

Why not use an off-the-shelf queue?

Design constraints

The background activity

The state machine

Leases: distributed execution without coordination

How it works

Routing work to the right runner pool

Dependencies and multi-step workflows

Two execution modes, one interface

Putting it all together

Retries and scheduling

Observability

What we learned

Wrapping it up

New Audit Policy Scans for Pulumi Stacks

A Complete Audit Story: Cloud Accounts and IaC Stacks

Key Scenarios for Audit Scans for Stacks

How It Works

How to Get Started

A Complete Governance Picture

Try Pulumi Policies

Kubernetes-native Option for Pulumi Deployments Customer-Managed Agents

Customer Managed Agents: Flexibility and Control

Key Features of Pulumi Deployments

Docker and Kubernetes Support

Getting Started with Kubernetes

Verification

Wrapping It Up

Pulumi Launches Scheduled Deployments: Automate Recurring Workflows

Simplifying Cloud Operations Through Automation

Example Use Cases

How Scheduled Deployments Work

Integration with Existing Features

Getting Started with Scheduled Deployments

Setting it up in the Pulumi Cloud Console

Setting it up in the API

Setting it up in the Pulumi Service Provider

Pulumi Service Provider for Creating a Raw Schedule

Wrapping it up

Secure your Kubernetes toolchain with Pulumi ESC and OIDC

Create a Kubernetes Cluster

Connect to the cluster

Deploy an application to the cluster

Export Files

Conclusion

Pulumi Kubernetes 4.0: Even More Kubernetes-Native

Server-Side Apply, Upserts and Patch Resources

Improved diff experience

Migrating to Kubernetes 4.0

Getting Started

Manage Shared Kubernetes Resources Safely with Pulumi

What’s new in Pulumi’s Kubernetes provider?

Adding labels to an existing Namespace

Change the default StorageClass

Share a ConfigMap resource

Do I need to modify my existing Gatekeeper `.rego` files?