Pulumi Blog: Helm

Best Kubernetes Infrastructure as Code Tools in 2026

Pulumi Content Team — Fri, 14 Aug 2026 00:00:00 +0000

There is no single best Kubernetes infrastructure as code tool, because “Kubernetes IaC” actually spans three different jobs. For provisioning the cluster and its cloud dependencies, Pulumi and Terraform (or OpenTofu) are the strongest general-purpose options. For templating and packaging workloads, Helm and Kustomize dominate. For continuous reconciliation once things are running, Argo CD and Flux lead the GitOps category. The right stack usually combines one tool from each layer, not a single tool that claims to do all three.

What counts as infrastructure as code for Kubernetes?

Kubernetes infrastructure as code work splits into three layers that get conflated constantly, and the confusion is where most tool comparisons go wrong.

The cluster and cloud layer provisions the things Kubernetes itself sits on top of: the managed control plane (EKS, GKE, AKS), node pools, the VPC and subnets, IAM roles, load balancers, and cluster add-ons. Terraform, Pulumi, and cloud-native tools like CloudFormation operate here.

The in-cluster workload layer defines what runs on the cluster once it exists: Deployments, Services, ConfigMaps, CustomResourceDefinitions, and the Helm charts or Kustomize overlays that template them. This is where Helm, Kustomize, and Crossplane’s custom resources live.

The delivery and reconciliation layer keeps what’s declared in Git in sync with what’s actually running on the cluster, continuously, rather than as a one-shot apply. Argo CD and Flux own this layer, and they consume the output of the other two rather than replacing them.

Most real Kubernetes platforms use tools from at least two of these layers together. A team might provision EKS with Terraform, package its application with Helm, and let Argo CD reconcile it continuously. Knowing which layer a tool actually addresses, rather than treating “Kubernetes IaC” as one shopping list, is the first decision that matters.

Pulumi provisions the cluster and the workloads on it in the same language

Pulumi is a general-purpose infrastructure as code platform that supports Python, TypeScript, Go, C#, Java, and YAML, backed by a dedicated Kubernetes provider that covers both the cluster layer and the workload layer in the same program. A single Pulumi stack can create an EKS cluster, its VPC and node groups, and the Deployments and Services that run inside it, with full dependency tracking between the two: Pulumi knows a Deployment depends on a cluster that does not exist yet, and sequences accordingly.

For workloads, Pulumi offers two distinct ways to work with Helm charts. kubernetes.helm.v4.Chart renders a chart client-side via helm template into individual resources, giving full Pulumi state, diffing, and policy visibility into every object a chart creates, at the cost of not producing a real Helm release. kubernetes.helm.sh/v3.Release installs through the embedded Helm SDK instead, producing an actual release with hooks, and can even import releases installed by the Helm CLI. Teams pick based on whether they need release semantics or resource-level visibility; Pulumi’s documentation is upfront that these are genuinely different tradeoffs, not that one supersedes the other. Kustomize overlays are supported directly through kubernetes.kustomize.v2.Directory.

As of the v4 provider, Server-Side Apply is the default for managing resources, which lets Pulumi share ownership of an object with other controllers safely instead of overwriting whatever they changed. Await and readiness logic (waiting for a Deployment’s rollout to finish before considering it “done”) is handled natively for common resource types, with skipAwait and waitFor escape hatches for anything unusual. As Pulumi engineer Bryce Lampe put it when the improved await logic shipped, “One of the advantages of using Pulumi to manage Kubernetes resources is that it natively and intuitively handles this problem of readiness and dependencies,” which matters more than it sounds: a naive apply-and-move-on model is a common source of flaky CI in Kubernetes pipelines.

Pulumi also plugs into an existing GitOps workflow rather than requiring you to replace it: the Kubernetes provider’s renderYamlToDirectory option renders manifests to disk instead of applying them directly, a pattern Pulumi’s own provider documentation calls out as an Argo CD Config Management Plugin use case. The provider currently labels this a developer-preview beta feature, disabled by default, so treat it as a bridge worth piloting rather than a settled default for a GitOps handoff. For CRDs, crd2pulumi generates strongly typed SDKs from CRD definitions, so a custom resource gets the same autocomplete and type-checking as a built-in one. Teams running Pulumi as a standing service inside the cluster can use the Pulumi Kubernetes Operator to drive stack updates from in-cluster custom resources.

The honest tradeoff: Pulumi is a commercial platform with an open-source core, and a team fully committed to a YAML-only, kubectl-native workflow will find a general-purpose language layer to be more machinery than it needs for a handful of static manifests. Materialize, which runs multi-region Amazon EKS clusters, adopted Pulumi from day one to manage that complexity in Python, after finding that tools like Terraform required developers to learn a proprietary configuration language and created unnecessary friction; new engineers reach their first meaningful contribution in a week rather than the month the team estimates it would otherwise take. A team with one cluster and no cross-region variation may not have that problem to solve.

Terraform and OpenTofu remain the default for the cluster layer, with real friction at the workload layer

Terraform (and its community-governed fork, OpenTofu, since HashiCorp’s Business Source License change) is still the most widely deployed way to provision the cluster itself: the AWS, Google, and Azure providers are mature, and the Kubernetes and Helm providers (currently at 3.2.1 and 3.2.0 respectively) let the same HCL codebase reach into the cluster afterward. That reach is also where the friction shows up. Terraform plans at plan time, before it has ever talked to a live cluster, so it frequently cannot know what a CRD’s schema actually looks like until it’s already been applied; this is a well-documented source of “plan differs from apply” surprises when CRDs and their consumers live in the same configuration. The common workaround is a two-stage apply, provisioning the cluster in one Terraform run and the workloads that depend on its CRDs in a second, which works but adds operational ceremony that a single-language, single-run tool doesn’t need. Our practical guide to Terraform and Kubernetes walks through where that friction shows up day to day and how a general-purpose language changes the testing and CRD story.

HCL itself is also a genuine limit at the workload layer: templating a Deployment’s environment variables across ten similar services means either heavy use of for_each and dynamic blocks, or accepting a lot of copy-pasted HCL, because HCL was designed as a configuration language rather than a general-purpose one with functions and reusable abstractions. None of this makes Terraform a poor choice for the cluster layer, where it remains a defensible default with a huge ecosystem of examples and modules. It’s a reason many Terraform shops hand workload templating off to Helm rather than fighting the Kubernetes provider for it.

Helm is the default packaging format, and now defaults to safer applies

Helm is the de facto package manager for Kubernetes: charts bundle a set of manifests with a templating layer and values files, and the public Artifact Hub ecosystem aggregates charts from vendors and the community, so most common software (databases, ingress controllers, observability stacks) has a chart available, official or otherwise. Helm 4 made Server-Side Apply the default instead of the old three-way merge against a stored last-applied-configuration annotation, which reduces a class of drift bugs that plagued Helm 3 installs sharing objects with other controllers. It also added WASM-based plugins and an experimental v3 chart format.

Helm’s real weakness is templating itself: charts are YAML run through Go’s text/template engine, which has no real type system, awkward conditionals, and error messages that point at rendered output rather than the source template. Values files also tend to sprawl as a chart tries to expose every possible customization, and a chart author’s defaults are a starting point you inherit rather than infrastructure you designed. Helm is excellent for consuming someone else’s well-maintained chart and mediocre as an authoring experience for your own complex, multi-service application, which is exactly why tools like Pulumi and Kustomize exist to sit on either side of it.

Kustomize is the built-in choice for environment-specific overlays

Kustomize ships inside kubectl and takes a patch-based approach instead of a templating one: you write a base set of plain YAML manifests, then layer environment-specific overlays (a different replica count for staging, a different image tag for production) that patch the base without touching it. That model avoids Helm’s templating-language problem entirely, at the cost of being far less expressive for anything beyond structural changes to existing YAML: generating a resource conditionally, or computing a value, is awkward or impossible.

The practical friction is version lag: the Kustomize binary bundled inside a given kubectl release trails the standalone Kustomize CLI’s releases, so a team relying on kubectl’s bundled version can be several minor versions behind what’s documented upstream. Teams that hit this usually install the standalone kustomize binary directly rather than depend on kubectl’s copy.

Crossplane turns cloud resources into Kubernetes-native APIs

Crossplane, which graduated to a CNCF top-level project on November 6, 2025, takes a fundamentally different approach: instead of a CLI that applies configuration, cloud resources become managed resources, native Kubernetes custom resources that a control-plane cluster reconciles continuously, overriding drift back to the desired state, the same way Kubernetes reconciles a Deployment. This is a genuinely different mental model from Terraform, Pulumi, or Helm, and it’s the right one for platform teams building a self-service layer where application teams request infrastructure (an S3 bucket, a Postgres instance) through the same kubectl apply and RBAC model they already use for everything else.

Crossplane v2 removed native patch-and-transform composition and introduced namespaced Composite Resources (XRs) alongside an alpha “Operations” feature for one-off imperative tasks. Claims, the v1 abstraction that let application teams request infrastructure without touching the (cluster-scoped) XR directly, were already namespaced; what changes in v2 is that they aren’t supported by the new namespaced or cluster-scoped XRs at all, and survive only when an XRD opts into the LegacyCluster scope for backward compatibility. Teams on Crossplane v1 migrating to v2 should plan to move application teams onto namespaced XRs directly rather than assume claims carry forward as-is. It also means Crossplane’s learning curve is steeper than a CLI-based tool: composing custom APIs out of managed resources requires understanding Crossplane’s own compositional model in addition to the cloud provider’s resource shapes. Teams that don’t need a self-service internal platform, and just want to provision infrastructure for their own team, usually find Terraform or Pulumi a faster path to the same cloud resources.

Argo CD and Flux reconcile what’s running, and assume something else already authored it

Argo CD (CNCF graduated December 2022, now at 3.5.1) and Flux (CNCF graduated November 2022, now at 2.9.4) are the two dominant GitOps controllers: both watch a Git repository and continuously compare the cluster’s actual state against what’s declared there, surfacing drift as it appears and correcting it automatically when self-healing is enabled, rather than waiting for the next scheduled apply. This is the delivery layer from the three-layer breakdown above, and it’s genuinely complementary to, not competitive with, Terraform, Pulumi, Helm, and Kustomize: Argo CD and Flux consume rendered manifests or chart references, they don’t generate cloud infrastructure or author application configuration themselves.

The CNCF’s 2025 annual survey found that GitOps adoption tracks organizational maturity closely: 58% of self-identified “cloud native innovators” use GitOps extensively, compared to 23% of “adopters” earlier in their journey, which suggests GitOps is a practice teams grow into rather than a day-one requirement. Neither Argo CD nor Flux replaces the need for a provisioning tool at the cluster layer; a common and reasonable pattern is Terraform or Pulumi for the cluster and cloud dependencies, then Argo CD or Flux reconciling the workloads that run on it, with Pulumi’s renderYamlToDirectory option (currently a developer-preview beta feature) designed to feed that second stage.

cdk8s and kro bring code and native APIs to workload definitions

cdk8s applies the AWS CDK model to Kubernetes manifests: you define Deployments, Services, and other objects in TypeScript, Python, Java, or Go, and cdk8s synthesizes plain YAML at build time, which plays nicely with existing GitOps pipelines since the output is still just manifests. It’s a narrower tool than Pulumi in scope, generating Kubernetes YAML only rather than also provisioning the cluster and cloud resources around it, and it has no equivalent to Pulumi’s state-tracked diffing or drift detection since it only emits manifests for something else to apply. The core cdk8s library is stable on the 2.x line; the higher-level cdk8s+ construct libraries are versioned and released separately per supported Kubernetes version.

kro takes Crossplane’s “infrastructure as a native API” idea and generalizes it: it composes any native Kubernetes resource as well as any CRD installed in the cluster, Crossplane’s managed resources included, into a new custom API that platform teams expose to their users. kro recently moved into the kubernetes-sigs organization as a SIG Cloud Provider subproject, which is a credibility signal, but it remains genuinely alpha (v1alpha1 APIs, releases still in the 0.7 to 0.9.3 range) and is not yet a CNCF top-level project. It’s worth watching and piloting, not yet a safe default for anything you can’t easily rebuild.

A handful of other projects are worth knowing about without needing a full section: Timoni applies CUE’s type system to Kubernetes packaging as a Helm alternative; KubeVela builds an application-delivery abstraction on top of the Open Application Model; Tanka uses Jsonnet for templated manifests in the Grafana Labs ecosystem; and Ansible’s Kubernetes modules remain common in shops that already standardized on Ansible for configuration management generally.

How the tools compare across cluster, workload, and delivery layers

Reconciliation model is the dimension most roundups leave out, and it matters as much as authoring language: a one-shot apply tool and a continuously reconciling controller solve different problems even when they touch the same YAML.

Tool	Authoring language	Provisions cloud + cluster?	Reconciliation model	Maturity / governance	Best fit
Pulumi	Python, TypeScript, Go, C#, Java, YAML	Yes	One-shot apply (`pulumi up`), continuous via Kubernetes Operator	Open-source core, commercial platform	Teams wanting one codebase for cluster + workloads
Terraform / OpenTofu	HCL	Yes	One-shot apply	Terraform: HashiCorp/IBM; OpenTofu: Linux Foundation	Cluster and cloud provisioning, HCL-standardized shops
Helm	YAML + Go templates	No (workload only)	One-shot install/upgrade	CNCF graduated	Packaging and consuming shared application charts
Kustomize	YAML (patch-based)	No (workload only)	One-shot apply	Built into `kubectl`, part of Kubernetes SIG-CLI	Environment overlays on plain manifests
Crossplane	Kubernetes YAML / CRDs	Yes (via managed resources)	Continuous reconciliation	CNCF graduated Nov 2025	Self-service internal platforms
Argo CD	Kubernetes YAML (consumed)	No (delivery only)	Continuous reconciliation	CNCF graduated	GitOps delivery, UI-driven ops
Flux	Kubernetes YAML (consumed)	No (delivery only)	Continuous reconciliation	CNCF graduated	GitOps delivery, GitOps Toolkit composability
cdk8s	TypeScript, Python, Java, Go	No (workload only)	One-shot synth to YAML	Stable 2.x core, CNCF Sandbox	Code-first manifest generation feeding existing GitOps
kro	Kubernetes YAML / CRDs	Yes (composes existing CRDs)	Continuous reconciliation	Alpha, `kubernetes-sigs` subproject	Composing custom platform APIs, early adopters

The same task in Terraform and Pulumi

The clearest way to see the practical difference is the same job in both: create a managed EKS cluster and deploy a single workload onto it.

Terraform provisions the cluster with the AWS provider, then reaches into it with the Kubernetes provider in the same configuration:

resource "aws_eks_cluster" "main" {
 name = "app-cluster"
 role_arn = aws_iam_role.eks.arn

 vpc_config {
 subnet_ids = aws_subnet.private[*].id
 }
}

provider "kubernetes" {
 host = aws_eks_cluster.main.endpoint
 cluster_ca_certificate = base64decode(aws_eks_cluster.main.certificate_authority[0].data)
 token = data.aws_eks_cluster_auth.main.token
}

resource "kubernetes_deployment" "app" {
 metadata {
 name = "app"
 }

 spec {
 replicas = 3

 selector {
 match_labels = { app = "app" }
 }

 template {
 metadata {
 labels = { app = "app" }
 }

 spec {
 container {
 name = "app"
 image = "myorg/app:1.4.0"
 }
 }
 }
 }
}

Pulumi, in TypeScript, does the same two-stage job with the dependency between them tracked automatically, rather than requiring a separate provider block wired up by hand:

import * as aws from "@pulumi/aws";
import * as eks from "@pulumi/eks";
import * as k8s from "@pulumi/kubernetes";

const cluster = new eks.Cluster("app-cluster", {
 instanceType: "t3.medium",
 desiredCapacity: 3,
});

const provider = new k8s.Provider("k8s-provider", {
 kubeconfig: cluster.kubeconfig,
});

const app = new k8s.apps.v1.Deployment("app", {
 spec: {
 replicas: 3,
 selector: { matchLabels: { app: "app" } },
 template: {
 metadata: { labels: { app: "app" } },
 spec: {
 containers: [{ name: "app", image: "myorg/app:1.4.0" }],
 },
 },
 },
}, { provider });

Both are legitimate. Terraform’s version separates concerns across a provider block and requires knowing the Kubernetes provider’s HCL shape for a Deployment resource; Pulumi’s version is fewer moving pieces because the cluster’s kubeconfig flows directly into the next resource as a normal object reference, and the whole thing can be unit tested with the same test framework used for the application code it deploys.

How to choose a Kubernetes IaC tool

Most teams end up combining tools across the three layers rather than picking one. Working through these five steps in order avoids the most common mistake, which is picking a workload-layer tool (like Helm) and then trying to stretch it to also provision the cluster.

Decide who provisions the cluster and its cloud dependencies (VPC, node pools, IAM), and pick a cluster-layer tool for that job: Pulumi or Terraform/OpenTofu for most teams, or a cloud-native tool if you’re single-cloud and want the tightest console integration.
Decide how workloads get packaged. If you’re consuming other people’s software (databases, ingress controllers, observability agents), Helm’s chart ecosystem is hard to beat. If you’re authoring your own application manifests, Kustomize overlays or a code-first approach (Pulumi, cdk8s) usually scale better than hand-templated Helm charts.
Decide whether you need a self-service internal platform. If application teams should be able to request infrastructure through kubectl apply without touching Terraform or Pulumi directly, Crossplane or kro are worth the steeper learning curve. If your platform team is the only one touching infrastructure, you likely don’t need this layer yet.
Decide whether you need continuous reconciliation or a one-shot apply is enough. A small team deploying a few times a week can live with terraform apply or pulumi up in CI. A team running many clusters, or one where configuration drift is a recurring incident cause, benefits from Argo CD or Flux catching and correcting drift automatically.
Match the result to your team’s actual skills, not the tool with the most GitHub stars. A team of application engineers who already write Python or TypeScript daily will move faster in a tool that speaks their language than in a new DSL, and a platform team that already lives in YAML and kubectl will get more value from Kustomize and Argo CD than from introducing a new language to learn.

Is Terraform or Pulumi better for Kubernetes?

Neither is universally better; the choice depends on what happens after the cluster exists. Terraform’s ecosystem of provider modules and examples is larger for pure cluster provisioning, and HCL is a smaller thing to learn than a full programming language. Pulumi’s advantage shows up once workloads enter the picture: because the cluster and its workloads live in the same general-purpose language, dependencies between them are tracked automatically, the same testing and CI tools used for application code apply to infrastructure code, and CRD-heavy workloads avoid the plan-time unknowns that come from HCL not having talked to a live cluster yet. Teams already standardized on HCL across their infrastructure will find less reason to switch; teams that want one codebase and one language for cluster and workloads together tend to prefer Pulumi.

What are the alternatives to Helm?

Kustomize is the most common alternative for teams that find Helm’s templating language awkward, trading some expressiveness for patch-based simplicity on plain YAML. cdk8s replaces Helm’s templating with a real programming language that synthesizes YAML at build time. Pulumi can render Helm charts directly through kubernetes.helm.v4.Chart for full state visibility, or install them as genuine Helm releases through kubernetes.helm.sh/v3.Release, without requiring a separate templating tool at all. Timoni applies CUE’s type system as a more strictly typed alternative to Helm’s Go templates. None of these fully replace Helm’s chart ecosystem for consuming third-party software; they’re strongest for authoring your own application’s manifests.

How do you manage a managed Kubernetes service like EKS, GKE, or AKS as code?

Managed Kubernetes services expose the control plane as a cloud resource like any other, so the tools that provision it are the same cluster-layer tools used for the rest of your cloud footprint: Terraform, OpenTofu, or Pulumi, using each cloud’s native provider (aws_eks_cluster, google_container_cluster, azurerm_kubernetes_cluster, or Pulumi’s equivalent resources and the higher-level @pulumi/eks package). The node pools, VPC, subnets, and IAM roles the control plane depends on get provisioned in the same run. What happens after the cluster exists, deploying and reconciling workloads, is a separate decision covered by the workload and delivery layers above, not something the cluster-provisioning tool needs to also own.

Is GitOps a replacement for infrastructure as code?

No. GitOps tools like Argo CD and Flux reconcile what’s already declared in Git against what’s running on the cluster; they don’t generate that declaration or provision the cloud infrastructure underneath it. A GitOps workflow still needs something upstream producing the manifests it reconciles, whether that’s Helm charts, Kustomize overlays, cdk8s output, or Pulumi’s renderYamlToDirectory rendering a stack to disk. GitOps is a delivery and drift-correction practice layered on top of infrastructure as code, not a substitute for the provisioning step itself.

Do you need Kubernetes-specific IaC tooling at all?

Not always. A team running a single small cluster with a handful of stable workloads can reasonably manage everything with plain kubectl apply against version-controlled YAML and skip templating tools entirely; the complexity Helm, Kustomize, and Crossplane solve only shows up once you have multiple environments, multiple similar services, or multiple teams needing self-service access. The honest signal that it’s time to adopt tooling is repetition: the same YAML copied and hand-edited across environments, or the same cluster-provisioning steps run manually more than once, is worth automating before it causes an incident.

Which Kubernetes IaC tool should you choose?

There’s no single winner because “Kubernetes IaC” spans three different jobs. Most production setups combine a cluster-layer tool (Pulumi or Terraform), a workload-layer tool (Helm, Kustomize, or a code-first option), and often a delivery-layer tool (Argo CD or Flux) once the team is large enough to need continuous reconciliation. The five-step framework above is a better starting point than any single ranked list, because the right combination depends on your team’s language preferences, how many environments you run, and whether you need to expose infrastructure to other teams as a self-service API.

Frequently asked questions

Does Pulumi replace Helm and Kustomize?

Not entirely. Pulumi can render Helm charts and apply Kustomize overlays directly from its Kubernetes provider, so a team can consume the existing Helm and Kustomize ecosystems from inside a Pulumi program rather than needing a separate CLI step. For authoring new manifests from scratch, Pulumi’s general-purpose language is an alternative to writing Helm templates or Kustomize bases, but plenty of teams use Pulumi for the cluster layer while still authoring workload manifests as Helm charts.

Can you use more than one of these tools together?

Yes, and most real deployments do. A common combination is a cluster-layer tool (Terraform or Pulumi) for the control plane and cloud dependencies, a workload-layer tool (Helm or Kustomize) for packaging what runs on it, and a delivery-layer tool (Argo CD or Flux) for continuous reconciliation. The three-layer breakdown at the top of this post exists precisely because these tools are usually complementary rather than competing for the same job.

Is Crossplane a replacement for Terraform or Pulumi?

Not for most teams. Crossplane’s strength is exposing infrastructure as native Kubernetes APIs so application teams can self-serve through kubectl, which is a platform-engineering pattern, not a general substitute for a cluster-provisioning tool. A platform team still typically uses Terraform or Pulumi to provision the Crossplane control-plane cluster itself and configure Crossplane’s provider credentials.

Why did the CNCF’s 2025 survey findings matter for this comparison?

The CNCF’s 2025 Annual Cloud Native Survey found 82% of container users now run Kubernetes in production, up from 66% in 2023, and 47% cited cultural change, more than training or security, as the top adoption challenge. Those numbers matter for tool choice because Kubernetes adoption has moved well past early adopters into mainstream production use, which means the tooling decision is increasingly about fitting an existing team’s skills and processes rather than picking a tool for a greenfield experiment.

Where to go next

For the cluster-provisioning layer specifically, the Pulumi Kubernetes getting-started guide walks through creating a cluster and deploying a first workload. The Pulumi vs. Helm comparison and Pulumi vs. Crossplane comparison go deeper on the workload- and platform-layer tradeoffs summarized here, and the Pulumi vs. Terraform comparison covers the cluster-layer decision in more depth than a roundup can. If your infrastructure spans more than Kubernetes, our broader infrastructure as code tools roundup compares the cloud-provisioning landscape beyond the cluster.

For running AI workloads specifically on Kubernetes, our companion piece on provisioning and governing infrastructure for AI agents covers GPU-aware scheduling, credentials, and where policy as code and Pulumi’s Neo agent fit into that picture. And if a self-service platform for other teams is the eventual goal, Pulumi’s IDP documentation covers the components, templates, and policy guardrails that turn infrastructure code into something other teams can safely consume without needing to understand Pulumi themselves.

Whatever combination you land on, the layer breakdown here is the durable part: know whether you’re solving a cluster problem, a workload problem, or a reconciliation problem before you compare tools, because most of the disagreement in “best Kubernetes IaC tool” debates comes from comparing tools that were never solving the same problem in the first place.

New: Helm Chart v4 resource with new features and languages

Eron Wright — Thu, 13 Jun 2024 00:00:00 -0700

Today we’re happy to announce a new “v4” version of the Chart resource, available now in v4.13 of the Pulumi Kubernetes provider. The new kubernetes.helm.sh/v4.Chart resource is provided side-by-side with the existing kubernetes.helm.sh/v3.Chart resource. We expect to deprecate v3 in the future.

When you need to install a third-party application into your Kubernetes cluster, you’re likely to find a Helm chart for that in Artifact Hub or other registry. Pulumi provides two ways to apply a Helm chart, as outlined in Choosing the Right Helm Resource For Your Use Case. The Chart resource offers deeper integration with Pulumi and better drift remediation. v4 brings a host of new features, including enhanced SDK support across all Pulumi SDKs, full OCI registry support, improved handling of chart values, better connectivity for cluster interactions, and improved resource ordering. Let’s dig in.

What’s new

Let’s look at what’s new with Chart v4.

New language support - Java SDK & YAML SDK

The Chart v4 resource is a Pulumi Component that works consistently across all Pulumi SDKs. Earlier versions were implemented separately for each SDK, leading to various inconsistencies and limited SDK support.

Here, for example, is a simple deployment of cert-manager, a well-known Kubernetes add-on:

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";

const certman = new kubernetes.helm.v4.Chart("cert-manager", {
 namespace: "cert-manager",
 chart: "oci://registry-1.docker.io/bitnamicharts/cert-manager",
 version: "1.3.1",
});

import pulumi
import pulumi_kubernetes as kubernetes

certman = kubernetes.helm.v4.Chart("cert-manager",
 namespace="cert-manager",
 chart="oci://registry-1.docker.io/bitnamicharts/cert-manager",
 version="1.3.1")

package main

import (
 helmv4 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/helm/v4"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err := helmv4.NewChart(ctx, "cert-manager", &helmv4.ChartArgs{
 Namespace: pulumi.String("cert-manager"),
 Chart: pulumi.String("oci://registry-1.docker.io/bitnamicharts/cert-manager"),
 Version: pulumi.String("1.3.1"),
 })
 if err != nil {
 return err
 }
 return nil
 })
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Kubernetes = Pulumi.Kubernetes;

return await Deployment.RunAsync(() =>
{
 var certman = new Kubernetes.Helm.V4.Chart("cert-manager", new()
 {
 Namespace = "cert-manager",
 Chart = "oci://registry-1.docker.io/bitnamicharts/cert-manager",
 Version = "1.3.1",
 });

});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.kubernetes.helm.sh_v4.Chart;
import com.pulumi.kubernetes.helm.sh_v4.ChartArgs;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 var certman = new Chart("cert-manager", ChartArgs.builder()
 .namespace("cert-manager")
 .chart("oci://registry-1.docker.io/bitnamicharts/cert-manager")
 .version("1.3.1")
 .build());

 }
}

resources:
 cert-manager:
 type: kubernetes:helm.sh/v4:Chart
 properties:
 namespace: cert-manager
 chart: oci://registry-1.docker.io/bitnamicharts/cert-manager
 version: "1.3.1"

OCI registry support

You can use container registries with OCI support such as Docker Hub to store and share Helm chart packages. The Chart v4 resource now has full support for OCI, bringing it to parity with the Release resource.

To use an authenticated OCI registry, you must first login using helm registry login or docker login.

Chart v4 also supports the use of Helm chart repositories, and adopts the same repositoryOpts API as was introduced in the Release resource.

Lock file support

Helm has support for lock files (Chart.lock) to control a chart’s dependencies. When deploying a chart from a local directory, Pulumi automatically rebuilds the chart’s dependencies if a lock file is present. See Helm Dependency Build for details.

Better handling of chart values

Chart v4 offers new ways to work with Chart values. It is now possible to use multiple values files and to use Pulumi Assets. Of course you can also use output values from other resources as chart values.

It is also possible to set a chart value to the contents of a text file, similarly to using Helm’s --set-file argument. To do that, simply use a Pulumi Asset as a value within the values map. This is useful for injecting large values into a chart, such as additional templates as supported by the extraDeploy parameter of some Bitnami charts.

Here’s an example of using a combination of values from different sources.

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";

const certManager = new kubernetes.helm.v4.Chart("cert-manager", {
 namespace: "cert-manager",
 chart: "oci://registry-1.docker.io/bitnamicharts/cert-manager",
 version: "1.3.1",
 valueYamlFiles: [new pulumi.asset.FileAsset("values.yaml")],
 values: {
 commonLabels: {
 "pulumi.com/stackName": pulumi.getStack(),
 },
 extraDeploy: [new pulumi.asset.FileAsset("manifest.yaml")],
 },
});

import pulumi
import pulumi_kubernetes as kubernetes

cert_manager = kubernetes.helm.v4.Chart("cert-manager",
 namespace="cert-manager",
 chart="oci://registry-1.docker.io/bitnamicharts/cert-manager",
 version="1.3.1",
 value_yaml_files=[pulumi.FileAsset("values.yaml")],
 values={
 "commonLabels": {
 "pulumi.com/stackName": pulumi.get_stack(),
 },
 "extraDeploy": [pulumi.FileAsset("manifest.yaml")],
 })

package main

import (
 helmv4 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/helm/v4"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err := helmv4.NewChart(ctx, "cert-manager", &helmv4.ChartArgs{
 Namespace: pulumi.String("cert-manager"),
 Chart: pulumi.String("oci://registry-1.docker.io/bitnamicharts/cert-manager"),
 Version: pulumi.String("1.3.1"),
 ValueYamlFiles: pulumi.AssetOrArchiveArray{
 pulumi.NewFileAsset("values.yaml"),
 },
 Values: pulumi.Map{
 "commonLabels": pulumi.Any(map[string]interface{}{
 "pulumi.com/stackName": ctx.Stack(),
 }),
 "extraDeploy": pulumi.Array{
 pulumi.NewFileAsset("manifest.yaml"),
 },
 },
 })
 if err != nil {
 return err
 }
 return nil
 })
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Kubernetes = Pulumi.Kubernetes;

return await Deployment.RunAsync(() =>
{
 var certManager = new Kubernetes.Helm.V4.Chart("cert-manager", new()
 {
 Namespace = "cert-manager",
 Chart = "oci://registry-1.docker.io/bitnamicharts/cert-manager",
 Version = "1.3.1",
 ValueYamlFiles = new[]
 {
 new FileAsset("values.yaml"),
 },
 Values =
 {
 { "commonLabels", new Dictionary<string, object?>
 {
 ["pulumi.com/stackName"] = Deployment.Instance.StackName,
 } },
 { "extraDeploy", new[]
 {
 new FileAsset("manifest.yaml"),
 } },
 },
 });

});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.deployment.Deployment;
import com.pulumi.kubernetes.helm.v4.Chart;
import com.pulumi.kubernetes.helm.v4.ChartArgs;
import com.pulumi.asset.FileAsset;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 var certManager = new Chart("cert-manager", ChartArgs.builder()
 .namespace("cert-manager")
 .chart("oci://registry-1.docker.io/bitnamicharts/cert-manager")
 .version("1.3.1")
 .valueYamlFiles(new FileAsset("values.yaml"))
 .values(Map.ofEntries(
 Map.entry("commonLabels", Map.of("pulumi.com/stackName", Deployment.getInstance().getStackName())),
 Map.entry("extraDeploy", List.of(new FileAsset("manifest.yaml")))
 ))
 .build());

 }
}

resources:
 cert-manager:
 type: kubernetes:helm.sh/v4:Chart
 properties:
 namespace: cert-manager
 chart: oci://registry-1.docker.io/bitnamicharts/cert-manager
 version: "1.3.1"
 valueYamlFiles:
 - fn::fileAsset: values.yaml
 values:
 commonLabels:
 pulumi.com/stackName: ${pulumi.stack}
 extraDeploy:
 - fn::fileAsset: manifest.yaml

Better connectivity

You may now use charts that use chart functions requiring a connection to the cluster, e.g. to:

Check the Kubernetes server version with .Capabilities.KubeVersion
Check if an API version or kind is available with .Capabilities.APIVersions.Has
Use the lookup function in your templates to get existing objects in your live cluster.

Note that the lookup function is executed in both the preview and the non-preview mode, and keep in mind that the expected object may not exist during a preview.

Improved resource ordering

It’s now easy to wait for a chart’s resources to be installed before installing other resources, simply by using the dependsOn option. In earlier versions, we relied on a ready output property.

The Chart resource automatically detects dependencies between resources in the manifest(s). For example, it knows to install namespaces and Custom Resource Definitions (CRDs) first.

Use the config.kubernetes.io/depends-on annotation to declare an explicit resource dependency. The annotation accepts a list of resource references, delimited by commas.

It consists of the group, kind, name, and optionally the namespace, delimited by forward slashes.

Resource Scope	Format
namespace-scoped	`<group>/namespaces/<namespace>/<kind>/<name>`
cluster-scoped	`<group>/<kind>/<name>`

For resources in the “core” group, the empty string is used instead (for example: /namespaces/test/Pod/pod-a).

New-style Pulumi transformations

Pulumi has a new way to transform component resources and their children, the transforms options. The older transformations option doesn’t work with multi-language components like Chart v4. See Resource Option: transforms for more details.

Note: you cannot change an object’s namespace or name using a Pulumi transformation, and you cannot add or discard an object.

Here’s an example of using the transforms option to add the pulumi.com/patchForce annotation to a chart’s resources.

const applyPatchForceAnnotation = async (args: pulumi.ResourceTransformArgs) => {
 switch(args.type) {
 case "kubernetes:helm.sh/v4:Chart":
 break;
 default:
 args.props.metadata.annotations = {
 "pulumi.com/patchForce": "true",
 ...args.props.metadata.annotations
 }
 }
 return {
 props: args.props,
 opts: args.opts,
 };
};

const ingressController = new kubernetes.helm.v4.Chart("ingresscontroller", {
 chart: "nginx-ingress",
 namespace: ingressNs.metadata.name,
 repositoryOpts: {
 repo: "https://helm.nginx.com/stable",
 },
 version: "0.14.1",
}, {transforms: [applyPatchForceAnnotation]});

def apply_patchforce_annotation(args: ResourceTransformArgs):
 if not args.type_ == "kubernetes:helm.sh/v4:Chart":
 if not 'metadata' in args.props:
 args.props['metadata'] = {}
 if not 'annotations' in args.props['metadata']:
 args.props['metadata']['annotations'] = {}
 args.props['metadata']['annotations']['pulumi.com/patchForce'] = 'true'

 return ResourceTransformResult(
 props=args.props,
 opts=args.opts)

ingresscontroller = kubernetes.helm.v4.Chart(
 "ingresscontroller",
 chart="nginx-ingress",
 namespace=ingress_ns.metadata.name,
 repository_opts=kubernetes.helm.v3.RepositoryOptsArgs(
 repo="https://helm.nginx.com/stable",
 ),
 version="0.14.1",
 opts=pulumi.ResourceOptions(transforms=[apply_patchforce_annotation])
)

applyPatchForceAnnotation := func(ctx context.Context, rta *pulumi.ResourceTransformArgs) *pulumi.ResourceTransformResult {
 transform := func(applier interface{}) {
 o := rta.Props.ToMapOutputWithContext(ctx).ApplyT(applier)
 r, err := internals.UnsafeAwaitOutput(ctx, o)
 if err != nil {
 panic(err)
 }
 rta.Props = r.Value.(pulumi.Map)
 }

 switch rta.Type {
 case "kubernetes:helm.sh/v4:Chart":
 // Do nothing for Helm charts
 default:
 transform(func(obj map[string]any) pulumi.Map {
 // note: obj is an ordinary Unstructured object at this point.
 unstructured.SetNestedField(obj, "true", "metadata", "annotations", "pulumi.com/patchForce")
 return pulumi.ToMap(obj)
 })
 }
 return &pulumi.ResourceTransformResult{
 Props: rta.Props,
 Opts: rta.Opts,
 }
}

// Use Helm to install the Nginx ingress controller
_, err = helmv4.NewChart(ctx, "ingresscontroller", &helmv4.ChartArgs{
 Chart: pulumi.String("nginx-ingress"),
 Namespace: ingressNs.Metadata.Name(),
 RepositoryOpts: &helmv4.RepositoryOptsArgs{
 Repo: pulumi.String("https://helm.nginx.com/stable"),
 },
 Version: pulumi.String("0.14.1"),
}, pulumi.Transforms([]pulumi.ResourceTransform{applyPatchForceAnnotation}))
if err != nil {
 return err
}

Post-rendering support

New to v4 is support for a post-rendering command, with optional arguments, to be applied to the rendered manifest. See Advanced Helm Techniques: Post Rendering for details.

“Keep” policy

The Chart v4 resource now understands Helm resource policies, specifically “keep” which instructs Pulumi not to delete a given object when the resource is destroyed. Simply apply the helm.sh/resource-policy: keep annotation to the object. See Tell Helm Not To Uninstall a Resource for details.

Release name

Charts typically generate object names based on the Helm release name. By default, Chart v4 uses its own resource name as the release name. To customize the release name, set the name property on the Chart.

Some charts also accept a fullnameOverride value to control object naming, use that if possible.

Limitations

Not supported: Kubernetes transformations

Chart v4 does not support the transformations argument as seen in Chart v3, that facilitates a Kubernetes-centric transformation and/or discarding of objects from the rendered manifest.

One alternative is to use use Pulumi transformations to transform the object and resource options. Another is to use post-rendering, which we’ll cover next.

Not supported: Helm hooks

Some charts use hooks to perform pre- and post-installation tasks, and we recommend using the Release resource to install such charts. Another option is to emulate the hook by performing the task directly, using other Pulumi resources such as kubernetes.batch/v1.Job and kubernetes.core/v1.Secret.

Note that Chart v3 does not support hooks either but, for historical reasons, does emit the hooks as ordinary child resources.

Evolving: Resource outputs

The child resources created by the chart are presented to your program via the resources output, as an array of Pulumi Resource objects. In some SDKs, you can cast the resource into the appropriate resource type, e.g. corev1.ConfigMap. Not all SDKs support this (yet); see pulumi/pulumi#15788.

Deploy Helm charts with Pulumi

Apply any Helm chart with the Chart v4 resource and manage the cloud resources it depends on, like IAM roles and storage buckets, in the same program.

Get started

Example: Argo CD

Here’s a real-world example of installing Argo CD into a Kubernetes cluster, and of using Argo CD’s Application custom resource to deploy the ‘guestbook’ example.

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";
import * as random from "@pulumi/random";

const ns = new kubernetes.core.v1.Namespace("ns", {metadata: {
 name: "argocd",
}});
const redisPasswordResource = new random.RandomPassword("redis-password", {length: 16});
const redisSecret = new kubernetes.core.v1.Secret("redis-secret", {
 metadata: {
 name: "argocd-redis",
 namespace: ns.metadata.apply(metadata => metadata.name),
 },
 type: "Opaque",
 stringData: {
 auth: redisPasswordResource.result,
 },
});
const argocd = new kubernetes.helm.v4.Chart("argocd", {
 chart: "argo-cd",
 version: "6.11.1",
 namespace: ns.metadata.apply(metadata => metadata.name),
 repositoryOpts: {
 repo: "https://argoproj.github.io/argo-helm",
 },
 values: {
 fullnameOverride: "",
 },
});
const guestbook = new kubernetes.yaml.v2.ConfigGroup("guestbook", {objs: [{
 apiVersion: "argoproj.io/v1alpha1",
 kind: "Application",
 metadata: {
 name: "guestbook",
 namespace: ns.metadata.apply(metadata => metadata.name),
 },
 spec: {
 project: "default",
 source: {
 repoURL: "https://github.com/argoproj/argocd-example-apps.git",
 targetRevision: "HEAD",
 path: "guestbook",
 },
 destination: {
 server: "https://kubernetes.default.svc",
 namespace: "default",
 },
 },
}]}, {
 dependsOn: [argocd],
});
export const redisPassword = redisPasswordResource.result;

import pulumi
import pulumi_kubernetes as kubernetes
import pulumi_random as random

ns = kubernetes.core.v1.Namespace("ns", metadata=kubernetes.meta.v1.ObjectMetaArgs(
 name="argocd",
))
redis_password_resource = random.RandomPassword("redis-password", length=16)
redis_secret = kubernetes.core.v1.Secret("redis-secret",
 metadata=kubernetes.meta.v1.ObjectMetaArgs(
 name="argocd-redis",
 namespace=ns.metadata.name,
 ),
 type="Opaque",
 string_data={
 "auth": redis_password_resource.result,
 })
argocd = kubernetes.helm.v4.Chart("argocd",
 chart="argo-cd",
 version="6.11.1",
 namespace=ns.metadata.name,
 repository_opts=kubernetes.helm.v4.RepositoryOptsArgs(
 repo="https://argoproj.github.io/argo-helm",
 ),
 values={
 "fullnameOverride": "",
 })
guestbook = kubernetes.yaml.v2.ConfigGroup("guestbook", objs=[{
 "apiVersion": "argoproj.io/v1alpha1",
 "kind": "Application",
 "metadata": {
 "name": "guestbook",
 "namespace": ns.metadata.name,
 },
 "spec": {
 "project": "default",
 "source": {
 "repoURL": "https://github.com/argoproj/argocd-example-apps.git",
 "targetRevision": "HEAD",
 "path": "guestbook",
 },
 "destination": {
 "server": "https://kubernetes.default.svc",
 "namespace": "default",
 },
 },
}],
opts=pulumi.ResourceOptions(depends_on=[argocd]))
pulumi.export("redisPassword", redis_password_resource.result)

package main

import (
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/core/v1"
 helmv4 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/helm/v4"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/meta/v1"
 yamlv2 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/yaml/v2"
 "github.com/pulumi/pulumi-random/sdk/v4/go/random"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 ns, err := corev1.NewNamespace(ctx, "ns", &corev1.NamespaceArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("argocd"),
 },
 })
 if err != nil {
 return err
 }
 redisPasswordResource, err := random.NewRandomPassword(ctx, "redis-password", &random.RandomPasswordArgs{
 Length: pulumi.Int(16),
 })
 if err != nil {
 return err
 }
 _, err = corev1.NewSecret(ctx, "redis-secret", &corev1.SecretArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("argocd-redis"),
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 Type: pulumi.String("Opaque"),
 StringData: pulumi.StringMap{
 "auth": redisPasswordResource.Result,
 },
 })
 if err != nil {
 return err
 }
 argocd, err := helmv4.NewChart(ctx, "argocd", &helmv4.ChartArgs{
 Chart: pulumi.String("argo-cd"),
 Version: pulumi.String("6.11.1"),
 Namespace: ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 RepositoryOpts: &helmv4.RepositoryOptsArgs{
 Repo: pulumi.String("https://argoproj.github.io/argo-helm"),
 },
 Values: pulumi.Map{
 "fullnameOverride": pulumi.Any(""),
 },
 })
 if err != nil {
 return err
 }
 _, err = yamlv2.NewConfigGroup(ctx, "guestbook", &yamlv2.ConfigGroupArgs{
 Objs: pulumi.Array{
 pulumi.Any(map[string]interface{}{
 "apiVersion": "argoproj.io/v1alpha1",
 "kind": "Application",
 "metadata": map[string]interface{}{
 "name": "guestbook",
 "namespace": ns.Metadata.ApplyT(func(metadata metav1.ObjectMeta) (*string, error) {
 return metadata.Name, nil
 }).(pulumi.StringPtrOutput),
 },
 "spec": map[string]interface{}{
 "project": "default",
 "source": map[string]interface{}{
 "repoURL": "https://github.com/argoproj/argocd-example-apps.git",
 "targetRevision": "HEAD",
 "path": "guestbook",
 },
 "destination": map[string]interface{}{
 "server": "https://kubernetes.default.svc",
 "namespace": "default",
 },
 },
 }),
 },
 }, pulumi.DependsOn([]pulumi.Resource{
 argocd,
 }))
 if err != nil {
 return err
 }
 ctx.Export("redisPassword", redisPasswordResource.Result)
 return nil
 })
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Kubernetes = Pulumi.Kubernetes;
using Random = Pulumi.Random;

return await Deployment.RunAsync(() =>
{
 var ns = new Kubernetes.Core.V1.Namespace("ns", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "argocd",
 },
 });

 var redisPasswordResource = new Random.RandomPassword("redis-password", new()
 {
 Length = 16,
 });

 var redisSecret = new Kubernetes.Core.V1.Secret("redis-secret", new()
 {
 Metadata = new Kubernetes.Types.Inputs.Meta.V1.ObjectMetaArgs
 {
 Name = "argocd-redis",
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 },
 Type = "Opaque",
 StringData =
 {
 { "auth", redisPasswordResource.Result },
 },
 });

 var argocd = new Kubernetes.Helm.V4.Chart("argocd", new()
 {
 Chart = "argo-cd",
 Version = "6.11.1",
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 RepositoryOpts = new Kubernetes.Types.Inputs.Helm.V4.RepositoryOptsArgs
 {
 Repo = "https://argoproj.github.io/argo-helm",
 },
 Values =
 {
 { "fullnameOverride", "" },
 },
 });

 var guestbook = new Kubernetes.Yaml.V2.ConfigGroup("guestbook", new()
 {
 Objs = new[]
 {
 new Dictionary<string, object?>
 {
 ["apiVersion"] = "argoproj.io/v1alpha1",
 ["kind"] = "Application",
 ["metadata"] = new Dictionary<string, object?>
 {
 ["name"] = "guestbook",
 ["namespace"] = ns.Metadata.Apply(metadata => metadata.Name),
 },
 ["spec"] = new Dictionary<string, object?>
 {
 ["project"] = "default",
 ["source"] = new Dictionary<string, object?>
 {
 ["repoURL"] = "https://github.com/argoproj/argocd-example-apps.git",
 ["targetRevision"] = "HEAD",
 ["path"] = "guestbook",
 },
 ["destination"] = new Dictionary<string, object?>
 {
 ["server"] = "https://kubernetes.default.svc",
 ["namespace"] = "default",
 },
 },
 },
 },
 }, new ComponentResourceOptions
 {
 DependsOn =
 {
 argocd,
 },
 });

 return new Dictionary<string, object?>
 {
 ["redisPassword"] = redisPasswordResource.Result,
 };
});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.kubernetes.core.v1.Namespace;
import com.pulumi.kubernetes.core.v1.NamespaceArgs;
import com.pulumi.kubernetes.meta.v1.inputs.ObjectMetaArgs;
import com.pulumi.random.RandomPassword;
import com.pulumi.random.RandomPasswordArgs;
import com.pulumi.kubernetes.core.v1.Secret;
import com.pulumi.kubernetes.core.v1.SecretArgs;
import com.pulumi.kubernetes.helm.v4.Chart;
import com.pulumi.kubernetes.helm.v4.ChartArgs;
import com.pulumi.kubernetes.helm.v4.inputs.RepositoryOptsArgs;
import com.pulumi.kubernetes.yaml.v2.ConfigGroup;
import com.pulumi.kubernetes.yaml.v2.ConfigGroupArgs;
import com.pulumi.resources.ComponentResourceOptions;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 public static void stack(Context ctx) {
 var ns = new Namespace("ns", NamespaceArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("argocd")
 .build())
 .build());

 var redisPasswordResource = new RandomPassword("redisPasswordResource", RandomPasswordArgs.builder()
 .length(16)
 .build());

 var redisSecret = new Secret("redisSecret", SecretArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("argocd-redis")
 .namespace(ns.metadata().applyValue(metadata -> metadata.name().get()))
 .build())
 .type("Opaque")
 .stringData(redisPasswordResource.result().applyValue(password -> Map.of("auth", password)))
 .build());

 var argocd = new Chart("argocd", ChartArgs.builder()
 .chart("argo-cd")
 .version("6.11.1")
 .namespace(ns.metadata().applyValue(metadata -> metadata.name().get()))
 .repositoryOpts(RepositoryOptsArgs.builder()
 .repo("https://argoproj.github.io/argo-helm")
 .build())
 .values(Map.of("fullnameOverride", ""))
 .build());

 var guestbook = new ConfigGroup("guestbook", ConfigGroupArgs.builder()
 .objs(Map.ofEntries(
 Map.entry("apiVersion", "argoproj.io/v1alpha1"),
 Map.entry("kind", "Application"),
 Map.entry("metadata", Map.ofEntries(
 Map.entry("name", "guestbook"),
 Map.entry("namespace", ns.metadata().applyValue(metadata -> metadata.name().get()))
 )),
 Map.entry("spec", Map.ofEntries(
 Map.entry("project", "default"),
 Map.entry("source", Map.ofEntries(
 Map.entry("repoURL", "https://github.com/argoproj/argocd-example-apps.git"),
 Map.entry("targetRevision", "HEAD"),
 Map.entry("path", "guestbook")
 )),
 Map.entry("destination", Map.ofEntries(
 Map.entry("server", "https://kubernetes.default.svc"),
 Map.entry("namespace", "default")
 ))
 ))
 ))
 .build(), ComponentResourceOptions.builder()
 .dependsOn(argocd)
 .build());

 ctx.export("redisPassword", redisPasswordResource.result());
 }
}

resources:
 ns:
 type: kubernetes:core/v1:Namespace
 properties:
 metadata:
 name: argocd

 redis-password:
 type: random:RandomPassword
 properties:
 length: 16

 redis-secret:
 type: kubernetes:core/v1:Secret
 properties:
 metadata:
 name: argocd-redis
 namespace: ${ns.metadata.name}
 type: Opaque
 stringData:
 auth: ${redis-password.result}
 options:
 retainOnDelete: true

 argocd:
 type: kubernetes:helm.sh/v4:Chart
 properties:
 chart: argo-cd
 version: "6.11.1"
 namespace: ${ns.metadata.name}
 repositoryOpts:
 repo: https://argoproj.github.io/argo-helm
 values:
 fullnameOverride: ""

 guestbook:
 type: kubernetes:yaml/v2:ConfigGroup
 properties:
 objs:
 - apiVersion: argoproj.io/v1alpha1
 kind: Application
 metadata:
 name: guestbook
 namespace: ${ns.metadata.name}
 spec:
 project: default
 source:
 repoURL: https://github.com/argoproj/argocd-example-apps.git
 targetRevision: HEAD
 path: guestbook
 destination:
 server: https://kubernetes.default.svc
 namespace: default
 options:
 dependsOn:
 - ${argocd}

outputs:
 redisPassword: ${redis-password.result}

The program creates the argocd namespace, installs the ArgoCD server, and then creates an ArgoCD Application resource. Observe how the program installs and uses a Custom Resource Definition (CRD) successfully, and uses dependsOn to ensure that the CRD is installed before using it.

The argo-cd chart normally makes use of a Helm hook to initialize a password for the redis server. Since the Chart v4 resource doesn’t support Helm hooks, this program creates the password directly.

Conclusion

Pulumi loves empowering developers to use the best tools for the job, and we recommend using Helm charts to install third-party Kubernetes applications. Pulumi complements Helm by handling the cloud resources that are often required, such as an IAM Role or cloud storage bucket. Such combinations make for great reusable componentry.

Check out the following links to learn more about Pulumi Kubernetes today!

Helm Release Resource for Kubernetes Generally Available

Vivek Lakshmanan — Fri, 04 Feb 2022 09:00:00 -0800

Back in September 2021 we announced public preview for the Helm Release resource in Pulumi’s Kubernetes provider. Over the last few months, we have had a very encouraging uptake in usage and several meaningful discussions with users in the community that have helped shape improvements to this resource. Thanks to this collaboration, we are now pleased to announce that the Helm Release resource is now GA (generally available) starting in v3.15.0 of the Pulumi Kubernetes Provider and SDK in all Pulumi supported languages. We are excited to offer yet another tool to Pulumi users to effectively manage their Kubernetes footprint.

The existing Helm Chart component resource will continue to be supported.

What is the Helm Release resource?

The Kubernetes provider has supported a means to deploy Helm Charts since 2018 through the Chart resource. This resource simulates Helm installation by resolving the templates for underlying resources from the chart and installing them on the target Kubernetes environment directly.

In contrast, the new Helm Release resource uses the Helm SDK natively to perform all Helm operations. This comes with some inherent advantages:

Support for the full spectrum of Helm features
Ability to import existing Helm releases into Pulumi state
Interact with the installed Helm Release through the Helm CLI

Which Helm Resource Should I Choose?

Both the Chart and Helm Release resource are complimentary with their own pros and cons. Depending on your use case, one might be a better fit than the other. We now have a guide to help you choose the right resource for your use case.

How Do I Get Started?

If you aren’t already familiar with using Pulumi with Kubernetes, head on over to Pulumi’s getting started guide for Kubernetes first.

In your chosen Kubernetes Pulumi project, make sure the referenced version of the Pulumi-Kubernetes SDK is at least v3.15.0.
Like other Kubernetes resources managed by Pulumi, the target cluster for Helm Release is configured with a Provider. The default options work for typical cases, and we also provide advanced configuration options for private registries, etc.
Refer to the instructions for your chosen language in the resource documentation to create a Helm Release resource.

Let’s look at a concrete example of Helm Release in action. In the following snippet we install Redis using the new Helm Release resource:

import * as random from "@pulumi/random";
import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

// Create a password
const pass = new random.RandomPassword("pass", {length:10});
const redisPassword = pass.result;

const namespace = new k8s.core.v1.Namespace("redis-ns");

const release = new k8s.helm.v3.Release("redis-helm", {
 chart: "redis",
 repositoryOpts: {
 repo: "https://charts.bitnami.com/bitnami",
 },
 version: "13.0.0",
 namespace: namespace.metadata.name,
 // Values from Chart's parameters specified hierarchically,
 // see https://artifacthub.io/packages/helm/bitnami/redis/13.0.0#parameters for reference.
 values: {
 cluster: {
 enabled: true,
 slaveCount: 3,
 },
 metrics: {
 enabled: true,
 service: {
 annotations: {
 "prometheus.io/port": "9127",
 }
 },
 },
 global: {
 redis: {
 password: redisPassword,
 }
 },
 rbac: {
 create: true,
 }
 },
 // By default Release resource will wait until all created resources
 // are available. Set this to true to skip waiting on resources being
 // available.
 skipAwait: false,
});


// We can look up resources once the release is installed. The release's
// status field is set once the installation completes, so this, combined
// with `skipAwait: false` above, will wait to retrieve the Redis master
// ClusterIP until all resources in the Chart are available.
const srv = k8s.core.v1.Service.get("redis-master-svc",
 pulumi.interpolate`${release.status.namespace}/${release.status.name}-master`);
export const redisMasterClusterIP = srv.spec.clusterIP;

package main

import (
 "fmt"

 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v3/go/kubernetes/core/v1"
 "github.com/pulumi/pulumi-kubernetes/sdk/v3/go/kubernetes/helm/v3"
 "github.com/pulumi/pulumi-random/sdk/v4/go/random"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {

 namespace, err := corev1.NewNamespace(ctx, "redis-ns", nil)
 if err != nil {
 return err
 }

 redisPassword, err := random.NewRandomPassword(ctx, "pass", &random.RandomPasswordArgs{
 Length: pulumi.Int(10),
 })
 if err != nil {
 return err
 }

 rel, err := helm.NewRelease(ctx, "redis-helm", &helm.ReleaseArgs{
 Chart: pulumi.String("redis"),
 RepositoryOpts: helm.RepositoryOptsArgs{
 Repo: pulumi.String("https://charts.bitnami.com/bitnami"),
 },
 Version: pulumi.String("13.0.0"),
 Namespace: namespace.Metadata.Name(),

 // Values from Chart's parameters specified hierarchically,
 // see https://artifacthub.io/packages/helm/bitnami/redis/13.0.0#parameters for reference.
 Values: pulumi.Map{
 "cluster": pulumi.Map{
 "enabled": pulumi.Bool(true),
 },
 "metrics": pulumi.Map{
 "enabled": pulumi.Bool(true),
 "service": pulumi.Map{
 "annotations": pulumi.StringMap{
 "prometheus.io/port": pulumi.String("9127"),
 },
 "type": pulumi.String("ClusterIP"),
 },
 },
 "global": pulumi.Map{
 "redis": pulumi.Map{
 "password": redisPassword.Result,
 },
 },
 "rbac": pulumi.BoolMap{
 "create": pulumi.Bool(true),
 },
 },

 // By default Release resource will wait until all created resources
 // are available. Set this to true to skip waiting on resources being
 // available.
 SkipAwait: pulumi.BoolPtr(false),
 })
 if err != nil {
 return err
 }

 // We can look up resources once the release is installed. The release's
 // status field is set once the installation completes, so this, combined
 // with `skipAwait: pulumi.BoolPtr(false)` above, will wait to retrieve
 // the Redis master ClusterIP until all resources in the Chart are available.
 svc := pulumi.All(rel.Status.Namespace(), rel.Status.Name()).
 ApplyT(func(r interface{}) (interface{}, error) {
 arr := r.([]interface{})
 namespace := arr[0].(*string)
 name := arr[1].(*string)
 svc, err := corev1.GetService(ctx,
 "redis-master-svc",
 pulumi.ID(fmt.Sprintf("%s/%s-master", *namespace, *name)),
 nil,
 )
 if err != nil {
 return "", nil
 }
 return svc.Spec.ClusterIP(), nil
 })
 ctx.Export("redisMasterClusterIP", svc)

 return nil
 })
}

import pulumi
from pulumi import Output
from pulumi_random.random_password import RandomPassword
from pulumi_kubernetes.core.v1 import Namespace, Service
from pulumi_kubernetes.helm.v3 import Release, ReleaseArgs, RepositoryOptsArgs

namespace = Namespace("redis-ns")

redis_password = RandomPassword("pass", length=10)

release_args = ReleaseArgs(
 chart="redis",
 repository_opts=RepositoryOptsArgs(
 repo="https://charts.bitnami.com/bitnami"
 ),
 version="13.0.0",
 namespace=namespace.metadata["name"],

 # Values from Chart's parameters specified hierarchically,
 # see https://artifacthub.io/packages/helm/bitnami/redis/13.0.0#parameters
 # for reference.
 values={
 "cluster": {
 "enabled": True,
 "slaveCount": 3,
 },
 "metrics": {
 "enabled": True,
 "service": {
 "annotations": {
 "prometheus.io/port": "9127",
 }
 },
 },
 "global": {
 "redis": {
 "password": redis_password.result,
 }
 },
 "rbac": {
 "create": True,
 },
 },
 # By default Release resource will wait until all created resources
 # are available. Set this to true to skip waiting on resources being
 # available.
 skip_await=False)

release = Release("redis-helm", args=release_args)

# We can look up resources once the release is installed. The release's
# status field is set once the installation completes, so this, combined
# with `skip_await=False` above, will wait to retrieve the Redis master
# ClusterIP until all resources in the Chart are available.
status = release.status
srv = Service.get("redis-master-svc",
 Output.concat(status.namespace, "/", status.name, "-master"))
pulumi.export("redisMasterClusterIP", srv.spec.cluster_ip)

using Pulumi;
using System.Collections.Generic;
using Pulumi.Random;
using Pulumi.Kubernetes.Core.V1;
using Pulumi.Kubernetes.Types.Inputs.Helm.V3;
using Pulumi.Kubernetes.Helm.V3;

class MyStack : Stack
{
 public MyStack()
 {
 var ns = new Pulumi.Kubernetes.Core.V1.Namespace("redis-ns");
 var redisPassword = new RandomPassword("pass", new RandomPasswordArgs
 {
 Length = 10,
 });

 var releaseArgs = new ReleaseArgs
 {
 Chart = "redis",
 RepositoryOpts = new RepositoryOptsArgs
 {
 Repo = "https://charts.bitnami.com/bitnami",
 },
 Version = "13.0.0",
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 // Values from Chart's parameters specified hierarchically,
 // see https://artifacthub.io/packages/helm/bitnami/redis/13.0.0#parameters
 // for reference.
 Values = new InputMap<object>
 {
 ["cluster"] = new Dictionary<string,object>
 {
 ["enabled"] = true,
 ["slaveCount"] = 3,
 },
 ["metrics"] = new Dictionary<string,object>
 {
 ["enabled"] = true,
 ["service"] = new Dictionary<string, object>
 {
 ["annotations"] = new Dictionary<string, string>
 {
 ["prometheus.io/port"] = "9127",
 },
 }
 },
 ["global"] = new Dictionary<string,object>
 {
 ["redis"] = new Dictionary<string, object>
 {
 ["password"] = redisPassword.Result,
 },
 },
 ["rbac"] = new Dictionary<string,object>
 {
 ["create"] = true,
 }
 },
 // By default Release resource will wait until all created resources
 // are available. Set this to true to skip waiting on resources being
 // available.
 SkipAwait = false,
 };

 var release = new Release("redis-helm", releaseArgs);

 // We can look up resources once the release is installed. The release's
 // status field is set once the installation completes, so this, combined
 // with `skip_await=False` above, will wait to retrieve the Redis master
 // ClusterIP until all resources in the Chart are available.
 var status = release.Status;
 var service = Service.Get("redist-master-svc", Output.All(status).Apply(
 s => $"{s[0].Namespace}/{s[0].Name}-master"));
 this.RedisMasterClusterIP = service.Spec.Apply(spec => spec.ClusterIP);
 }

 [Output]
 public Output<string> RedisMasterClusterIP { get; set; }
}

Users of the existing Helm Chart resource will notice that we retained the hierarchical values specification experience and a similar means to configure the repository. The above example also highlights the await and resource lookup experience.

Rollout of the above program is shown in the slightly sped up gif below:

What’s New Since the Preview Announcement?

Thanks to our community of early adopters, we received excellent feedback on the new resource which helped us make significant improvements since our initial preview. Apart from several robustness improvements, some major callouts include:

Honor namespace specification for all resources installed by Helm Release [#1747]
Make values optional [#1761]
Make RepositoryOpts optional [#1806]
Support local charts [#1809]
Add import support [#1818]
Support loading values from yaml files [#1828]

Where Can I Learn More?

You can learn more about Helm Release from the following sources:

The Helm Release API Reference docs are an excellent starting point to learn more about the API in each of the supported languages along with code samples.
Our guide on choosing between the Chart and Release resource for your use case
One of our sample projects where you can see the Helm Release resource in action in any one of Pulumi’s supported languages:
- TypeScript
- Python
- C#
- Go

If you have questions or run into issues, please don’t hesitate to reach out by filing issues on GitHub or asking questions on Community Slack. Happy charting!

Full Access to Helm Features for Kubernetes

Vivek Lakshmanan — Tue, 14 Sep 2021 00:00:00 +0000

Kubernetes has been a significant focus of Pulumi since its very beginnings. Pulumi added support for installing Helm charts way back in 2018 and it has seen significant adoption by users since. However, Pulumi’s current Chart integration lacks support for some increasingly common advanced features in Helm charts, e.g.:

Support for Helm lifecycle hooks
Handling sub-charts and Helm chart dependencies

As Helm and its usage evolved over the years, Pulumi users using the Chart resource have often had to get very creative in order to get the desired functionality in their deployments.

Today we are excited to announce the public preview of a new Helm Release resource starting with v3.7.0 of the Pulumi Kubernetes Provider and SDK in all Pulumi supported languages. This new resource provides Pulumi users more options to choose the right tool for their use-case. The rest of this blog post will highlight how this resource differs from the existing Helm Chart component resource and describe how and when to use the new resource.

The existing Helm Chart component resource, which extracts and converts Kubernetes resources into individual resources in the Pulumi program, will continue to be supported.

How is Helm Release resource different?

Pulumi’s Helm Chart integration was implemented as a Component Resource which effectively extracts the corresponding Kubernetes resources’ manifests from the chart and installs them as if they were individually specified in the Pulumi user program. This allowed users to install community developed Helm charts, while providing fine-grained control over the respective Kubernetes resources installed in the process. For instance, the installed resources are subject to all policies specified through Policy-as-Code policy packs registered with Pulumi. Other users have used Pulumi’s powerful transformation support to manipulate Helm charts to fit their desired needs before installing them in their environment.

In contrast, the new Helm Release resource uses the Helm SDK, bringing the full feature set of Helm directly to Pulumi users. This provides the following advantages:

Helm Controls Lifecycle

Unlike the existing Chart component resource, Helm Release directly offloads all responsibility of orchestration and lifecycle management to Helm itself. This means Helm can invoke all necessary lifecycle hooks for the release as it would through the CLI. It also updates chart dependencies and correctly handles sub-chart values.

Configurable Await Support

By default, Pulumi’s Helm Release resource waits for all resources installed by the release to be available, up to a configurable timeout that defaults to 5 minutes. With a sufficient timeout, the resource can be reliably passed to dependsOn clauses in subsequent resources to encode ordering requirements. In addition, users can also require waiting on all Helm hooks to complete by setting jobAwait to true. However, if desired, all await behavior can be disabled by setting skipAwait option to true. See the SDK documentation for details for each of the above options.

Helm State Management

The release’s state is serialized and stored by Helm on the cluster. As a result, users can interact with the installed resource outside of Pulumi through the Helm CLI.

While there are many positives, there are some significant limitations to highlight with this approach as well:

Limited Transforms Support

Helm Release doesn’t support the ability to write transformations for Helm-installed manifests inline in the Pulumi program as supported in Helm Charts. Instead, users can leverage Helm’s post-renderer specified through the postRender option for the Release resource to manipulate manifests.

Limited Visibility

Secondly, only the Release resource and the relevant Kubernetes resources read back in the user program are part of Pulumi state. Other items installed by the chart are only controlled by Helm and are not visible and thus cannot be managed through Pulumi. However, the underlying rendered manifests and resource names will be accessible to users of the resource through the manifest and resourceNames fields as part of the deployment preview.

How do I use it?

If you aren’t already familiar with using Pulumi with Kubernetes, head on over to Pulumi’s getting started guide for Kubernetes first.

In your chosen Kubernetes Pulumi project, make sure the referenced version of the Pulumi-Kubernetes SDK is at least v3.7.0.
Helm Release support is configured to target the relevant cluster pointed to by the Kubernetes provider associated with it and no additional configuration is required by default. However, more advanced configuration options are supported, see the provider configuration documentation for more details.
Refer to the instructions for your chosen language in the resource documentation to create a Helm Release resource.

Lets look at a concrete example of Helm Release in action. In the following snippet we install Redis through a Helm Chart in each of the supported Pulumi languages using the new Helm Release resource:

import * as random from "@pulumi/random";
import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

// Create a password
const pass = new random.RandomPassword("pass", {length:10});
const redisPassword = pass.result;

const namespace = new k8s.core.v1.Namespace("redis-ns");

const release = new k8s.helm.v3.Release("redis-helm", {
 chart: "redis",
 repositoryOpts: {
 repo: "https://charts.bitnami.com/bitnami",
 },
 version: "13.0.0",
 namespace: namespace.metadata.name,
 // Values from Chart's parameters specified hierarchically,
 // see https://artifacthub.io/packages/helm/bitnami/redis/13.0.0#parameters for reference.
 values: {
 cluster: {
 enabled: true,
 slaveCount: 3,
 },
 metrics: {
 enabled: true,
 service: {
 annotations: {
 "prometheus.io/port": "9127",
 }
 },
 },
 global: {
 redis: {
 password: redisPassword,
 }
 },
 rbac: {
 create: true,
 }
 },
 // By default Release resource will wait till all created resources
 // are available. Set this to true to skip waiting on resources being
 // available.
 skipAwait: false,
});


// We can lookup resources once the release is installed. The release's
// status field is set once the installation completes, so this, combined
// with `skipAwait: false` above, will wait to retrieve the Redis master
// ClusterIP till all resources in the Chart are available.
const srv = k8s.core.v1.Service.get("redis-master-svc",
 pulumi.interpolate`${release.status.namespace}/${release.status.name}-master`);
export const redisMasterClusterIP = srv.spec.clusterIP;

package main

import (
 "fmt"

 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v3/go/kubernetes/core/v1"
 "github.com/pulumi/pulumi-kubernetes/sdk/v3/go/kubernetes/helm/v3"
 "github.com/pulumi/pulumi-random/sdk/v4/go/random"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {

 namespace, err := corev1.NewNamespace(ctx, "redis-ns", nil)
 if err != nil {
 return err
 }

 redisPassword, err := random.NewRandomPassword(ctx, "pass", &random.RandomPasswordArgs{
 Length: pulumi.Int(10),
 })
 if err != nil {
 return err
 }

 rel, err := helm.NewRelease(ctx, "redis-helm", &helm.ReleaseArgs{
 Chart: pulumi.String("redis"),
 RepositoryOpts: helm.RepositoryOptsArgs{
 Repo: pulumi.String("https://charts.bitnami.com/bitnami"),
 },
 Version: pulumi.String("13.0.0"),
 Namespace: namespace.Metadata.Name(),

 // Values from Chart's parameters specified hierarchically,
 // see https://artifacthub.io/packages/helm/bitnami/redis/13.0.0#parameters for reference.
 Values: pulumi.Map{
 "cluster": pulumi.Map{
 "enabled": pulumi.Bool(true),
 },
 "metrics": pulumi.Map{
 "enabled": pulumi.Bool(true),
 "service": pulumi.Map{
 "annotations": pulumi.StringMap{
 "prometheus.io/port": pulumi.String("9127"),
 },
 "type": pulumi.String("ClusterIP"),
 },
 },
 "global": pulumi.Map{
 "redis": pulumi.Map{
 "password": redisPassword.Result,
 },
 },
 "rbac": pulumi.BoolMap{
 "create": pulumi.Bool(true),
 },
 },

 // By default Release resource will wait till all created resources
 // are available. Set this to true to skip waiting on resources being
 // available.
 SkipAwait: pulumi.BoolPtr(false),
 })
 if err != nil {
 return err
 }

 // We can lookup resources once the release is installed. The release's
 // status field is set once the installation completes, so this, combined
 // with `skipAwait: pulumi.BoolPtr(false)` above, will wait to retrieve
 // the Redis master ClusterIP till all resources in the Chart are available.
 svc := pulumi.All(rel.Status.Namespace(), rel.Status.Name()).
 ApplyT(func(r interface{}) (interface{}, error) {
 arr := r.([]interface{})
 namespace := arr[0].(*string)
 name := arr[1].(*string)
 svc, err := corev1.GetService(ctx,
 "redis-master-svc",
 pulumi.ID(fmt.Sprintf("%s/%s-master", *namespace, *name)),
 nil,
 )
 if err != nil {
 return "", nil
 }
 return svc.Spec.ClusterIP(), nil
 })
 ctx.Export("redisMasterClusterIP", svc)

 return nil
 })
}

import pulumi
from pulumi import Output
from pulumi_random.random_password import RandomPassword
from pulumi_kubernetes.core.v1 import Namespace, Service
from pulumi_kubernetes.helm.v3 import Release, ReleaseArgs, RepositoryOptsArgs

namespace = Namespace("redis-ns")

redis_password = RandomPassword("pass", length=10)

release_args = ReleaseArgs(
 chart="redis",
 repository_opts=RepositoryOptsArgs(
 repo="https://charts.bitnami.com/bitnami"
 ),
 version="13.0.0",
 namespace=namespace.metadata["name"],

 # Values from Chart's parameters specified hierarchically,
 # see https://artifacthub.io/packages/helm/bitnami/redis/13.0.0#parameters
 # for reference.
 values={
 "cluster": {
 "enabled": True,
 "slaveCount": 3,
 },
 "metrics": {
 "enabled": True,
 "service": {
 "annotations": {
 "prometheus.io/port": "9127",
 }
 },
 },
 "global": {
 "redis": {
 "password": redis_password.result,
 }
 },
 "rbac": {
 "create": True,
 },
 },
 # By default Release resource will wait till all created resources
 # are available. Set this to true to skip waiting on resources being
 # available.
 skip_await=False)

release = Release("redis-helm", args=release_args)

# We can lookup resources once the release is installed. The release's
# status field is set once the installation completes, so this, combined
# with `skip_await=False` above, will wait to retrieve the Redis master
# ClusterIP till all resources in the Chart are available.
status = release.status
srv = Service.get("redis-master-svc",
 Output.concat(status.namespace, "/", status.name, "-master"))
pulumi.export("redisMasterClusterIP", srv.spec.cluster_ip)

using Pulumi;
using System.Collections.Generic;
using Pulumi.Random;
using Pulumi.Kubernetes.Core.V1;
using Pulumi.Kubernetes.Types.Inputs.Helm.V3;
using Pulumi.Kubernetes.Helm.V3;

class MyStack : Stack
{
 public MyStack()
 {
 var ns = new Pulumi.Kubernetes.Core.V1.Namespace("redis-ns");
 var redisPassword = new RandomPassword("pass", new RandomPasswordArgs
 {
 Length = 10,
 });

 var releaseArgs = new ReleaseArgs
 {
 Chart = "redis",
 RepositoryOpts = new RepositoryOptsArgs
 {
 Repo = "https://charts.bitnami.com/bitnami",
 },
 Version = "13.0.0",
 Namespace = ns.Metadata.Apply(metadata => metadata.Name),
 // Values from Chart's parameters specified hierarchically,
 // see https://artifacthub.io/packages/helm/bitnami/redis/13.0.0#parameters
 // for reference.
 Values = new InputMap<object>
 {
 ["cluster"] = new Dictionary<string,object>
 {
 ["enabled"] = true,
 ["slaveCount"] = 3,
 },
 ["metrics"] = new Dictionary<string,object>
 {
 ["enabled"] = true,
 ["service"] = new Dictionary<string, object>
 {
 ["annotations"] = new Dictionary<string, string>
 {
 ["prometheus.io/port"] = "9127",
 },
 }
 },
 ["global"] = new Dictionary<string,object>
 {
 ["redis"] = new Dictionary<string, object>
 {
 ["password"] = redisPassword.Result,
 },
 },
 ["rbac"] = new Dictionary<string,object>
 {
 ["create"] = true,
 }
 },
 // By default Release resource will wait till all created resources
 // are available. Set this to true to skip waiting on resources being
 // available.
 SkipAwait = false,
 };

 var release = new Release("redis-helm", releaseArgs);

 // We can lookup resources once the release is installed. The release's
 // status field is set once the installation completes, so this, combined
 // with `skip_await=False` above, will wait to retrieve the Redis master
 // ClusterIP till all resources in the Chart are available.
 var status = release.Status;
 var service = Service.Get("redist-master-svc", Output.All(status).Apply(
 s => $"{s[0].Namespace}/{s[0].Name}-master"));
 this.RedisMasterClusterIP = service.Spec.Apply(spec => spec.ClusterIP);
 }

 [Output]
 public Output<string> RedisMasterClusterIP { get; set; }
}

Rollout of the above program is shown in the slightly sped up gif below:

Which Helm resource to use and when?

In summary, if your primary goal is to install Helm charts with configuration options natively supported by the chart, then Helm Release should be a good fit. If on the other hand you intend to manipulate Helm Charts through transformations (due to bugs in the charts or restrictions in your environment), then the experience is much richer with the Helm Chart resource.

If you intend to use Policy-as-Code to enforce resource policies at a Kubernetes resource granularity, then the Helm Chart component resource would be the preferred model to use.

Note that both resources can co-exist safely in the same program.

What’s next?

The new Helm Release resource is available in preview now. During the preview the resource may evolve rapidly and users are requested to not use it for critical infrastructure. We have already received valuable feedback to shape this resource and we look forward to hearing more from our users. Please reach out by filing or commenting on issues on GitHub or asking questions on Community Slack. Happy charting!

Top 5 Things an Azure Developer Needs to Know: Kubernetes Applications

Sophia Parafina — Mon, 16 Aug 2021 00:00:00 +0000

All modern software is cloud software, and it’s more than likely that it runs on Kubernetes. Developers are faced with the challenge of deploy applications composed of many microservices. And each microservice adds to the complexity of the deployment.

This article reviews the different methods for deploying applications on Azure Kubernetes Service (AKS).

A quick review on application deployment

If you’re not familiar with deploying applications on Kubernetes, these articles cover the basic and advance cases:

Setting up the AKS sandbox

We’ll set up an AKS sandbox composed of an AKS cluster and Azure Container Registry (ACR) for this article. Make sure you have the Azure CLI installed, and you’re logged into Azure, e.g., az login.

Create a container registry

We’ll create a resource group first, followed by the registry.

$ az group create --resource-group aks-group --location westus2
$ az acr create --resource-group aks-group --name pulumipusContainerRegistry --sku Basic

When the registry is created, the Azure CLI returns JSON with metadata about the registry. Under the id key, you’ll find the URL to the login server.

...
 "id": "/subscriptions/fceb55a3-db15-4f3a-bffa-81271e7b7676/resourceGroups/myResourceGroup/providers/Microsoft.ContainerRegistry/registries/azContainerRegistryPulumi",
 "identity": null,
 "location": "eastus",
 "loginServer": "pulumipuscontainerregistry.azurecr.io",
 "name": "azContainerRegistryPulumi",
...

At this point, you can push and pull container images from your registry. If you want to try out the registry and push an image, you’ll need Docker installed. This example pulls an image from Docker Hub and tags it with the URL to the Azure container registry. We then log into the registry and push the image. To verify the image, we use the Azure CLI to list the images in the registry.

$ docker pull hello-world
$ docker tag hello-world pulumipuscontainerregistry.azurecr.io/hello-world:v1
$ az acr login --name pulumipusContainerRegistry
$ docker push pulumipuscontainerregistry.azurecr.io/hello-world:v1
$ az acr repository list --name pulumipusContainerRegistry
[
 "hello-world"
]

Create an AKS cluster

The previous article this series demonstrated how to create an AKS cluster with the Azure portal and code to illustrate creating an AKS cluster. For this example, we’ll use the CLI. In the example below, the first line creates a basic cluster, and the second line retrieves the cluster credentials that let us connect to it.

$ az aks create --resource-group aks-group --name aks-cluster --node-count 3 --generate-ssh-keys -s Standard_B2ms --disable-rbac
$ az aks get-credentials -g aks-group -n aks-cluster

Deploy an NGINX container on AKS with Helm

Helm is an application package manager for Kubernetes. Follow the directions on helm.sh to install Helm. Helm can install packages from repositories such as ArtifactHUB, Bitnami, and the Azure Marketplace.

First, we’ll add the Azure Marketplace repository to helm and update it. Next, we install the NGINX helm chart to our AKS cluster.

$ helm repo add azure-marketplace https://marketplace.azurecr.io/helm/v1/repo
$ helm repo update
$ helm install my-release azure-marketplace/nginx
NAME: my-release
LAST DEPLOYED: Mon Aug 16 13:58:26 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The NGINX Ingress Controller has been installed.

To find the URL for our NGINX deployment, use kubectl to get the external IP address.

$ kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 3h21m
my-release-nginx LoadBalancer 10.0.212.197 20.72.78.59 80:31251/TCP 175m

Deploy a custom container

In the previous section, we used a Helm chart to deploy an application from a repository. But what if we want to deploy our custom application. In this example, we’ll build our NGINX container, push it to our registry, and deploy it on the AKS cluster.

Create our application container

For this example, you will need Docker. We will replace the default NGINX page with our welcome page. Copy the sample below and save it as index.html.

<!doctype html>
<html lang="en">
<head>
 <meta charset="utf-8">
 <title>AKS NGINX</title>
</head>
<body>
 <h2>Hello from Azure AKS</h2>
</body>
</html>

Next, we’ll make a Dockerfile to build our custom NGINX container. Copy and save the following to a file called Dockerfile.

FROM nginx:latest
COPY ./index.html /usr/share/nginx/html/index.html

Build and tag the image with this command and push it to the repository we created earlier.

$ docker build -t pulumipuscontainerregistry.azurecr.io/my-nginx:v1
$ az acr login --name pulumipusContainerRegistry
$ docker push pulumipuscontainerregistry.azurecr.io/my-nginx:v1

Create a deployment manifest

In the previous example, we deployed NGINX with Helm. Manifests are another way to deploy applications. A manifest contains a deployment that defines pods that run a containerized application. It also contains services that route traffic to pods.

Copy and save the example below to a file named manifest.yml.

apiVersion: apps/v1
kind: Deployment
metadata:
 creationTimestamp: null
 labels:
 app: my-nginx
 name: my-nginx
spec:
 replicas: 3
 selector:
 matchLabels:
 app: my-nginx
 strategy: {}
 template:
 metadata:
 creationTimestamp: null
 labels:
 app: my-nginx
 spec:
 containers:
 - image: pulumipuscontainerregistry.azurecr.io/my-nginx:v1
 name: my-nginx
 imagePullPolicy: Always
 resources: {}
 ports:
 - containerPort: 80
status: {}

---

apiVersion: v1
kind: Service
metadata:
 name: my-nginx
 labels:
 run: my-nginx
spec:
 ports:
 - port: 80
 protocol: TCP
 selector:
 app: my-nginx
 type: LoadBalancer

Deploy the application to AKS

We’ll use kubectl to deploy the application with the manifest. But before we can do this, we need to attach the ACR pulumipuscontainerregistry to the AKS cluster, aks-cluster. Once the registry is added to the cluster configuration, we can deploy our application with kubectl.

$ az aks update --resource-group aks-group --name aks-cluster --attach-acr pulumipuscontainerregistry
$ kubectl create -f manifest.yml

Use kubectl get service to get the load balancer’s external IP as we did with the Helm example.

Deploy with Pulumi

The previous article in this series showed how to deploy AKS with code. Starting with the Python example from GitHub and add to it to create a complete solution

Create a container registry and custom container

First, we create the image registry to hold our custom NGINX container. Put the index.html and the Dockerfile files in a directory called app. The app directory is the context Docker uses to build an image. Note that the Docker provider builds and pushes the image to our repository.

# Create a private ACR registry.
rg = azure.core.ResourceGroup('azure-native-py-aks')
registry = azure.containerservice.Registry('pulumipuscontainerrepository',
 resource_group_name=rg.name,
 admin_enabled=True,
 sku='Basic'

# Get registry info (creds and endpoint).
image_name = "pulumipuscontainerrepository/my-nginx"
registry_info = docker.ImageRegistry(
 server=registry.login_server,
 username=registry.admin_username,
 password=registry.admin_password
)

import pulumi_docker as docker

# Build and publish the container image.
image = docker.Image('my-image',
 build='app',
 image_name=image_name,
 registry=registry_info,
)

Deploy the container

The code below creates a Deployment and a LoadBalancer Service similar to the manifest we used earlier. The script exports the IP address of the LoadBalancer at the end of the script.


# Create a NGINX Deployment
app_labels = { 'app': 'my-nginx' }
app_dep = k8s.apps.v1.Deployment('my-aks-deployment',
 spec={
 'selector': { 'matchLabels': app_labels },
 'replicas': 3,
 'template': {
 'metadata': { 'labels': app_labels },
 'spec': {
 'containers': [{
 'name': 'pulumipuscontainerrepository/my-nginx',
 'image': 'my-nginx;,
 'ports': [
 {
 'name': 'http',
 'containerPort': 80
 }
 ]
 }],
 },
 },
 },
)

# Create a Service
app_svc = k8s.core.v1.Service('my-aks-service',
 metadata={ 'labels': app_labels },
 spec={
 'type': 'LoadBalancer',
 'ports': [
 {
 'port': 80,
 'targetPort': 80,
 'protocol': 'TCP'
 }
 ],
 'selector': app_labels,
 }
)
pulumi.export('application IP', app_svc.status.apply(lambda status: status.loadbalancer.ingress[0].ip))

This example provides a sketch of the process to create an AKS cluster and deploy a custom container. For a fully worked example, check out the Azure Kubernetes Service (AKS) - Hello World! tutorial.

Summary

There are many ways to deploy applications on AKS, including ConfigMaps and Kustomize which were not discussed. We’ve covered deploying with Helm, manifests, and infrastructure as code. This article covers the basics of application deployment using Deployments and Services. To review, Deployments are declarative updates to a Pod such as specifying a container to run, and Services route traffic to Pods either within the cluster or permitting ingress with a LoadBalancer. We should note that templating methods such as Helm or manifests require file management, whereas using cloud engineering practices enables managing both infrastructure and application deployment in a versioned and repeatable way.

The following article will wrap up the series. We’ll examine how to implement DevOps practices with Azure.